the revival of an oral tradition

Around 400 BCE, Socrates stated he didn't trust the written word for the following reasons:

• When the audience was confused, no one could respond to questions
• When we no longer train our memory to recall facts, we become more forgetful
• The written word cannot tailor the message to the audience, nor does it afford the art of performance

We in the Agile Community can appreciate this perspective; in fact we have embraced face-to-face communication in many ways, from encouraging teams to Sit Together, to promoting peer learning through user groups, conferences, and the Gordon Pask Award (or official Pask Award site), as well as the Agile Manifesto's explicit valuing of "working software over comprehensive documentation", and the popularity of story cards/backlogs that are so brief that they offer no more than a "promise for a conversation".

Yet why is the oral tradition so important? I think it is the most direct way to capitalize on the Wisdom of Crowds, to tap the collective intelligence of the people on our teams who are doing the work. When system requirements are written down, organizations tend to be more hierarchical, individuals more specialized, and more ceremony is involved in software processes. On the contrary, when we rely on oral communication, the organization tends to be more flat, relational, and responsibility is shared. This means that people who have to live with a design/process decision also have the ability to fix any mistakes... and so mistakes are fixed more readily.

There is more. Oral tradition fosters an environment where we begin to use words in a particular way--leading to inside jokes, even dialects--and gelled teams. Often the words used by team members helps them communicate faster and more precisely, at the same time as giving them a sense of belonging and identity. If corporate documentation attempts to standardize on certain word choices, this can damage the fabric of a team. On the other hand, if we allow our teams to specialize their language, grow close and communicate face-to-face, they'll make smarter decisions together, and work together to achieve goals better than if we rely on armchair architects.

Agile Besançon -- beautiful code

Last night at Agile Besançon we talked about beautiful code (based on the book Beau Code). We started with about a 25-minute implementation of a binary search algorithm, followed by compare/contrast of code, and an analysis from the book of common mistakes. Apparently this is much harder to implement than one would first expect, and only 10% of programmers can get it right in 2 hours. We all found vulnerabilities in our code (overflows, invalid assumptions about boundary or repeating cases), and it was a great opportunity to reflect on the assumptions we make as we code.

estimate-free still needs estimation

Some agile teams have moved to a kanban-style process in which there is no explicit planning game or estimation cycle--but I think it is an oversimplification to say there are no estimates in the process. How could a team have dozens of story cards approximately the same size if no one ever tried to predict the size in advance?

I posted the following to the XP list today, in response to the Task Underestimation and Overestimation thread:

To [estimate], or not to [estimate]; that is the question,
Whether 'tis nobler in the mind to suffer the slings and arrows of outrageous [deadlines],
or to [code bravely] against a sea of troubles, and by [overtime, to deliver]...

When I think about it, the experiments I've done to remove estimation from our stories were more about reducing the waste, than entirely removing estimates. Consider this--there must be *some* implicit estimation if we're going to slice our stories until they're all about the same size.

My current team has rejected estimation-free cards. We tend to like the size value on the card to remind us of the scope, to keep an urgency about the work, etc.

Still, we've also been unable to add estimates to all cards, using our current planning game; maybe 1/3 of the cards in an iteration show up without estimates at all, and it's too much BDUF to split the stories more. Instead, we treat an un-estimated story as a timeboxed research or a spike--the only expected result of these kinds of cards is a stack of new story cards with estimates. Sometimes the product owners pick how much time to invest in this research, in which case we'll tag the story cost as SPIKE:2, for example, to mean 2 points' worth of spike activity.

flow and concentration

For 15 of the last 16 months I've been working with my dev team to help them get to the next level with agile--something the Scrum folk call hyper-productivity, but I think they use the term too loosely, based on the number of teams I've compared notes with at agile conferences and user group meetings. Anyway, my team hit that 3 weeks ago--something clicked, and everyone is on the same page. The team is self-organizing, committed to iteration goals, and constantly challenging the scope of a story. They're delivering under-budget on major release milestones, and doing so without increasing any debt I can perceive.

What does this have to do with flow? Since I'm currently playing a role of product owner, this means my workload has just doubled or quadrupled. I'm getting interrupted all the time to answer questions about story cards--and I'm constantly trying to dig into current, past, and future cards to find out exactly what the users need. I'm also hyper-aware of flow right now because I'm reading Peopleware, and so I posted the following to the Extreme Programming yahoo groups list:

short version:
Is there new thinking or studies that rationalize away DeMarco's & Lister's advice against open floor plans? Are there new studies on the impact of background noise on creativity?


background:

I know these are old books, but I decided to read some of the classics that were published before I was done with middle school ;), like Peopleware and the Mythical Man Month.

In Peopleware they talk a lot about flow (mental focus). They cite studies on the negative impact of interrupted concentration, the negative effect of background music on creativity, and about the performance dip faced by teams who they sit in open-plan workspaces. I don't know how to assimilate this with contradictory advice and experience, with, for example, Whole Team, the Customer is Always Available, Pair Programming, the Pomodoro technique, etc.

Personally, when I develop I feel like my pair can keep me in the flow if I get interrupted... but then again, sometimes that interruption distracts both of us. On the other hand, I feel like flow is dangerous--if I don't use some external "wake-up" call like a Pomodoro, I might go down the wrong path and deliver nothing of value. I've also observed my team learn how to get back into flow--I'd say they can often do so in one minute (and they learned this after only weeks of using Pomodoro).

My doubts come mostly from the conviction in DeMarco's and Lister's words, combined with what I'm seeing in the next generation of technology users--people that don't realize the impact of multi-tasking (homework+sms+tv+phone+???). Maybe I'm just not aware of the negative effects of being in a team room because that's basically all I've ever worked in...

Is there something missing in the way I do XP that provides guidance on when it's OK and when it's not to interrupt a pair? (we've actually experimented with it, but basically we believe that a developer should try to answer questions alone for 5 minutes, then ask for help--and the team/individual can respond with help or inform that now's not a good time--come back at the next pomodoro pause. I use the term pomodoro loosely, for more info, see http://dhondtsayitsagile.blogspot.com/2010/01/no-more-pomodoros-synch-point.html ).

No More Pomodoros--Synch Point

Over the summer our team experimented with the Pomodoro technique, using one pomodoro to rule them all, and now over 6 months later we still have something that was at least inspired by the technique. We call it a point synchro, in French, and the literal translation works well to call it a "Synch Point", but unfortunately that just doesn't have the same feel for me. I think that this is just a problem of being part of a team--we develop our own dialect, our own vocabulary, and it just loses the inside-joke-intimacy when we use more general terms. Regardless, we have speakers set up on a machine to play a random .mp3 for 30 seconds at 10am, noon, 3pm, and 5pm--four synch points a day--and the music plays a software-induced crescendo. When the music stops, everyone is supposed to be at the front of the room, standing up fashion. People share what they need to, the Product Owners ask what it's going to take to move the cards to the done column, and people discuss in as much detail as long as necessary... anyone who is not interested moves on, or interested parties collect around a pairing station to work out some details. Most sync points are done in under 5-10 minutes, for 9 people.

Wisdom of Crowds Release Planning

At our last corporate meeting, I was invited to facilitate the discussions for the road map / upcoming releases. The slides I used were intended to keep the sentiment light and playful:

Participants were asked to come to the meeting with a list of ideas to work on for the next two releases. (Though we deploy internally every week, and customers can theoretically use these releases, our marketing efforts are geared towards the official releases that come out 3-4 times a year.) Each planning session would run for 90 minutes, and was designed to get everyone talking about what functionality is important.
Each planning segment included a slide that illustrated what each person was to focus on... often for developers it was confirming scope or estimation; for consultants/sales, it was either defining minimally marketable features or re-prioritizing their stories or taking stories away until the sum was 12 weeks. It was frustrating for people when they were asked to move on to another group, but newcomers often had great questions and new ideas. Overall the exercise was a success, from the PO perspective, because it identified some features we hadn't considered, and confirmed other things about our product vision. We ran through the whole process twice, to plan each release separately. We had a quick feedback session/retrospective in between to decide on what to change for the next time around, but mostly we kept the same process. The second cycle went a bit better because people knew what to expect, and they were more comfortable with the aggressive schedule. At the end, everyone was frustrated with the compromises they had to make, but they felt like the draft road map they'd produced reflected their collective priorities.

Setting the Stage
I opened the meeting by setting the stage--saying that officially this gathering was intended to provide input to the Product Owner (PO) team in planning the upcoming releases, but that unofficially I hoped the group could come up with something that would be good enough to be the final road map. I felt that since the whole PO team was present, their input would be considered and therefore the output could be final. I pointed out that each working session would be short--about 10 minutes, and that someone from each group would move on to another group at each pause. I also asked that people aim for equal participation from each member of the group.

Collecting Data/Making decisions
Brainstorming (5 min)--individuals were asked to review and prioritize their lists of what we should work on for the next release (or make one if they didn't do their homework).
Estimating (20 min)--we broke out into two groups of 5-7 people each; each group had at least one consultant, one sales person, a developer, and a PO. Non-devs were expected to take turns describing one idea at a time, in just barely enough detail for a developer to give a coarse-grained estimate in weeks of dev time. Unfortunately people got too caught up in accuracy and only got one turn each... and these story cards were coming in with quotations between 2-8 weeks of development. For the next round we clarified that we wouldn't be keeping these story cards--they're throw-away--and that we don't need really accurate estimates, since we're just trying to get a feel for what the scope might be. In pilot runs of this exercise, when people didn't take it too seriously, they were able to do estimates at about 1 per minute. This time around it was taking 5 minutes per card.
Feasibility (15 min)--Now it was time to see if everything we wanted would fit. The new developer was to confirm story estimates by asking questions about scope. Side conversations in the group were common, and expected, since this increased bandwidth of communication.
Prioritize (10 min)--The group needed to prioritize in order to pick the most two critical stories... these stories were going to be sent to another group. A new PO in the group had veto power over cards, but as far as I know this veto power wasn't used. A few people complained about having to send story cards away, but I thought it was an important element for consensus-building across groups.
Fill in gaps/Find themes (15 min)--A consultant or sales person arrives in a new group with two story cards, and it's time to talk about these new cards, to re-add what was just sent away if necessary, and to re-adjust the set so it sums to 12 weeks.
Prioritize Themes (15 min)--The actual story cards are no longer important--group them into named categories (themes), copy the estimates onto the cards, and be ready to present to the whole room. Everyone's set of stories will have converged, so hopefully a final draft will be easy to make.

Feedback
The time frame was very aggressive. My estimates were off, but ultimately we used the following--brainstorming, 5 minutes; estimation, 20; feasibility, 15; prioritization, 15; themes, 15; prioritizing themes, 10; leaving 10 for shuffle time. I had hoped to have a step or two after what's pictured, where the whole group meets to discuss the set of features for the proposed release, but we ran out of time. People were frustrated with not having prepared enough, with having seen their good ideas get bumped to a later release, or get dropped from the schedule--but the PO team was very happy. We felt the priorities corresponded well with our vision, and some items that weren't on our radar scope showed up.

The Mythical Man-Month by Frederick Brooks

In the 20th anniversary edition of The Mythical Man-Month, Frederick Brooks adds four chapters to re-evaluate some of the positions he made in 1975. This book is mentioned over and over in things I read, and so I decided to go see what makes it a classic for myself. I was quite impressed with the notion that many of Brooks' ideas and projections are dead-on, even today, well over 30 years after he wrote about them. Then again, he says himself that many of our problems are people problems, not technology problems, and I guess human history is condemned to repeat itself if we don't know what came before.

One projection is that there will be no "silver bullet" that offers orders-of-magnitude improvements in productivity. Another is his emphasis that for large systems, we need conceptual integrity, and to organize it, he proposes recursion of architects (layered levels of decreasing abstraction). Many readers of this book were surprised that it devotes more time to people issues than technical--but he says that is and remains the biggest challenge in software development.

Though I found the book a bit dry and out of date, it was worth a quick read for the simple reason that it holds so many ideas that are so strongly advocated by Agile practices, and others that are not. Consistent with agile--we must use a process that assures us that "one always has, at every stage in the process, a working system. I find that teams can grow much more complex entities in four months than they can build"; "conceptual integrity is the most important consideration in system design". Communication is critical--and exponentially more difficult with larger and larger teams. He advocates giving developers room to be creative, saying they need room for "invention and craftsmanship". I guess the craftsmanship/codesmith movement goes way back! Another great line is "whenever talents permit, senior people must... delight in building programs with their own hands". No arm-chair architects or managers in Brooks' world! He also believes in "done-done", in other words--"milestones musbe be concrete, specific, measureable...coding, for counterexample, is '90 percent finished', for half the total coding time. Debugging is '99 percent complete' most of the time...". Though stand-up meetings weren't called it at the time, he knew the wastes of "status-review" meetings, and started calling things problem-action meetings instead. He also states, in more archaic terms, that getting the Acceptance Tests [design] right is the hardest part of development.

Not part of the agile world, he advocates both small iterations or large deployments, depending on context; he's the one who states "the bearing of a child takes nine months, no matter how many women are assigned", he posits that a project gets to be one year late, one day at a time; he (quoting Harlan Mills) suggests the ideal team structure be modeled after a surgical team. He names the "second system effect" as tempting designers to add extra features and frills that they didn't have a chance to add in the first system--and as a result this becomes bloated and fails. I think I saw that effect when I was on the job in New York. Brooks also talks a lot about documentation... I think this has been obsoleted by the use of intent-revealing test-first coding. Another obsolete point is his statement to "plan to throw one away".

Brooks obviously read a lot in preparation for this book--accumulating benchmarks, such as when he proposes that system coding should only be around 1/6th of the schedule; that developer estimates are often 1/2 of the actual time because of meetings and other interruptions from coding the active project; that higher-level languages (and implicitly higher levels of abstraction) are the only thing that's come along that can offer orders-of-magnitude improvements in productivity.