flow and concentration

For 15 of the last 16 months I've been working with my dev team to help them get to the next level with agile--something the Scrum folk call hyper-productivity, but I think they use the term too loosely, based on the number of teams I've compared notes with at agile conferences and user group meetings. Anyway, my team hit that 3 weeks ago--something clicked, and everyone is on the same page. The team is self-organizing, committed to iteration goals, and constantly challenging the scope of a story. They're delivering under-budget on major release milestones, and doing so without increasing any debt I can perceive.

What does this have to do with flow? Since I'm currently playing a role of product owner, this means my workload has just doubled or quadrupled. I'm getting interrupted all the time to answer questions about story cards--and I'm constantly trying to dig into current, past, and future cards to find out exactly what the users need. I'm also hyper-aware of flow right now because I'm reading Peopleware, and so I posted the following to the Extreme Programming yahoo groups list:

short version:
Is there new thinking or studies that rationalize away DeMarco's & Lister's advice against open floor plans? Are there new studies on the impact of background noise on creativity?


background:

I know these are old books, but I decided to read some of the classics that were published before I was done with middle school ;), like Peopleware and the Mythical Man Month.

In Peopleware they talk a lot about flow (mental focus). They cite studies on the negative impact of interrupted concentration, the negative effect of background music on creativity, and about the performance dip faced by teams who they sit in open-plan workspaces. I don't know how to assimilate this with contradictory advice and experience, with, for example, Whole Team, the Customer is Always Available, Pair Programming, the Pomodoro technique, etc.

Personally, when I develop I feel like my pair can keep me in the flow if I get interrupted... but then again, sometimes that interruption distracts both of us. On the other hand, I feel like flow is dangerous--if I don't use some external "wake-up" call like a Pomodoro, I might go down the wrong path and deliver nothing of value. I've also observed my team learn how to get back into flow--I'd say they can often do so in one minute (and they learned this after only weeks of using Pomodoro).

My doubts come mostly from the conviction in DeMarco's and Lister's words, combined with what I'm seeing in the next generation of technology users--people that don't realize the impact of multi-tasking (homework+sms+tv+phone+???). Maybe I'm just not aware of the negative effects of being in a team room because that's basically all I've ever worked in...

Is there something missing in the way I do XP that provides guidance on when it's OK and when it's not to interrupt a pair? (we've actually experimented with it, but basically we believe that a developer should try to answer questions alone for 5 minutes, then ask for help--and the team/individual can respond with help or inform that now's not a good time--come back at the next pomodoro pause. I use the term pomodoro loosely, for more info, see http://dhondtsayitsagile.blogspot.com/2010/01/no-more-pomodoros-synch-point.html ).

No More Pomodoros--Synch Point

Over the summer our team experimented with the Pomodoro technique, using one pomodoro to rule them all, and now over 6 months later we still have something that was at least inspired by the technique. We call it a point synchro, in French, and the literal translation works well to call it a "Synch Point", but unfortunately that just doesn't have the same feel for me. I think that this is just a problem of being part of a team--we develop our own dialect, our own vocabulary, and it just loses the inside-joke-intimacy when we use more general terms. Regardless, we have speakers set up on a machine to play a random .mp3 for 30 seconds at 10am, noon, 3pm, and 5pm--four synch points a day--and the music plays a software-induced crescendo. When the music stops, everyone is supposed to be at the front of the room, standing up fashion. People share what they need to, the Product Owners ask what it's going to take to move the cards to the done column, and people discuss in as much detail as long as necessary... anyone who is not interested moves on, or interested parties collect around a pairing station to work out some details. Most sync points are done in under 5-10 minutes, for 9 people.

Wisdom of Crowds Release Planning

At our last corporate meeting, I was invited to facilitate the discussions for the road map / upcoming releases. The slides I used were intended to keep the sentiment light and playful:

Participants were asked to come to the meeting with a list of ideas to work on for the next two releases. (Though we deploy internally every week, and customers can theoretically use these releases, our marketing efforts are geared towards the official releases that come out 3-4 times a year.) Each planning session would run for 90 minutes, and was designed to get everyone talking about what functionality is important.
Each planning segment included a slide that illustrated what each person was to focus on... often for developers it was confirming scope or estimation; for consultants/sales, it was either defining minimally marketable features or re-prioritizing their stories or taking stories away until the sum was 12 weeks. It was frustrating for people when they were asked to move on to another group, but newcomers often had great questions and new ideas. Overall the exercise was a success, from the PO perspective, because it identified some features we hadn't considered, and confirmed other things about our product vision. We ran through the whole process twice, to plan each release separately. We had a quick feedback session/retrospective in between to decide on what to change for the next time around, but mostly we kept the same process. The second cycle went a bit better because people knew what to expect, and they were more comfortable with the aggressive schedule. At the end, everyone was frustrated with the compromises they had to make, but they felt like the draft road map they'd produced reflected their collective priorities.

Setting the Stage
I opened the meeting by setting the stage--saying that officially this gathering was intended to provide input to the Product Owner (PO) team in planning the upcoming releases, but that unofficially I hoped the group could come up with something that would be good enough to be the final road map. I felt that since the whole PO team was present, their input would be considered and therefore the output could be final. I pointed out that each working session would be short--about 10 minutes, and that someone from each group would move on to another group at each pause. I also asked that people aim for equal participation from each member of the group.

Collecting Data/Making decisions
Brainstorming (5 min)--individuals were asked to review and prioritize their lists of what we should work on for the next release (or make one if they didn't do their homework).
Estimating (20 min)--we broke out into two groups of 5-7 people each; each group had at least one consultant, one sales person, a developer, and a PO. Non-devs were expected to take turns describing one idea at a time, in just barely enough detail for a developer to give a coarse-grained estimate in weeks of dev time. Unfortunately people got too caught up in accuracy and only got one turn each... and these story cards were coming in with quotations between 2-8 weeks of development. For the next round we clarified that we wouldn't be keeping these story cards--they're throw-away--and that we don't need really accurate estimates, since we're just trying to get a feel for what the scope might be. In pilot runs of this exercise, when people didn't take it too seriously, they were able to do estimates at about 1 per minute. This time around it was taking 5 minutes per card.
Feasibility (15 min)--Now it was time to see if everything we wanted would fit. The new developer was to confirm story estimates by asking questions about scope. Side conversations in the group were common, and expected, since this increased bandwidth of communication.
Prioritize (10 min)--The group needed to prioritize in order to pick the most two critical stories... these stories were going to be sent to another group. A new PO in the group had veto power over cards, but as far as I know this veto power wasn't used. A few people complained about having to send story cards away, but I thought it was an important element for consensus-building across groups.
Fill in gaps/Find themes (15 min)--A consultant or sales person arrives in a new group with two story cards, and it's time to talk about these new cards, to re-add what was just sent away if necessary, and to re-adjust the set so it sums to 12 weeks.
Prioritize Themes (15 min)--The actual story cards are no longer important--group them into named categories (themes), copy the estimates onto the cards, and be ready to present to the whole room. Everyone's set of stories will have converged, so hopefully a final draft will be easy to make.

Feedback
The time frame was very aggressive. My estimates were off, but ultimately we used the following--brainstorming, 5 minutes; estimation, 20; feasibility, 15; prioritization, 15; themes, 15; prioritizing themes, 10; leaving 10 for shuffle time. I had hoped to have a step or two after what's pictured, where the whole group meets to discuss the set of features for the proposed release, but we ran out of time. People were frustrated with not having prepared enough, with having seen their good ideas get bumped to a later release, or get dropped from the schedule--but the PO team was very happy. We felt the priorities corresponded well with our vision, and some items that weren't on our radar scope showed up.

The Mythical Man-Month by Frederick Brooks

In the 20th anniversary edition of The Mythical Man-Month, Frederick Brooks adds four chapters to re-evaluate some of the positions he made in 1975. This book is mentioned over and over in things I read, and so I decided to go see what makes it a classic for myself. I was quite impressed with the notion that many of Brooks' ideas and projections are dead-on, even today, well over 30 years after he wrote about them. Then again, he says himself that many of our problems are people problems, not technology problems, and I guess human history is condemned to repeat itself if we don't know what came before.

One projection is that there will be no "silver bullet" that offers orders-of-magnitude improvements in productivity. Another is his emphasis that for large systems, we need conceptual integrity, and to organize it, he proposes recursion of architects (layered levels of decreasing abstraction). Many readers of this book were surprised that it devotes more time to people issues than technical--but he says that is and remains the biggest challenge in software development.

Though I found the book a bit dry and out of date, it was worth a quick read for the simple reason that it holds so many ideas that are so strongly advocated by Agile practices, and others that are not. Consistent with agile--we must use a process that assures us that "one always has, at every stage in the process, a working system. I find that teams can grow much more complex entities in four months than they can build"; "conceptual integrity is the most important consideration in system design". Communication is critical--and exponentially more difficult with larger and larger teams. He advocates giving developers room to be creative, saying they need room for "invention and craftsmanship". I guess the craftsmanship/codesmith movement goes way back! Another great line is "whenever talents permit, senior people must... delight in building programs with their own hands". No arm-chair architects or managers in Brooks' world! He also believes in "done-done", in other words--"milestones musbe be concrete, specific, measureable...coding, for counterexample, is '90 percent finished', for half the total coding time. Debugging is '99 percent complete' most of the time...". Though stand-up meetings weren't called it at the time, he knew the wastes of "status-review" meetings, and started calling things problem-action meetings instead. He also states, in more archaic terms, that getting the Acceptance Tests [design] right is the hardest part of development.

Not part of the agile world, he advocates both small iterations or large deployments, depending on context; he's the one who states "the bearing of a child takes nine months, no matter how many women are assigned", he posits that a project gets to be one year late, one day at a time; he (quoting Harlan Mills) suggests the ideal team structure be modeled after a surgical team. He names the "second system effect" as tempting designers to add extra features and frills that they didn't have a chance to add in the first system--and as a result this becomes bloated and fails. I think I saw that effect when I was on the job in New York. Brooks also talks a lot about documentation... I think this has been obsoleted by the use of intent-revealing test-first coding. Another obsolete point is his statement to "plan to throw one away".

Brooks obviously read a lot in preparation for this book--accumulating benchmarks, such as when he proposes that system coding should only be around 1/6th of the schedule; that developer estimates are often 1/2 of the actual time because of meetings and other interruptions from coding the active project; that higher-level languages (and implicitly higher levels of abstraction) are the only thing that's come along that can offer orders-of-magnitude improvements in productivity.

Agile Besançon Informal Discussion Night

We had scheduled a discussion on lean for the 7th of this month, but due to low turn-out, we ended up talking about one member's frustration about software development--he stared by saying he was sick and tired of it... that in other professions, a person could reach a level of quality that was demonstrable and durable... take a look at quality stereo equipment, or a hand-made wrist watch. The stereo he has at home was built with such pride that they sent out the schematics for it with the packaging--they felt like if anything ever broke, end-users could get it back running with minimal effort the design was so simple. On the other hand, just taking a look at code he wrote a year ago makes this developer sick with code smells. Why????

We talked for about 2 hours on this and other subjects... we hope to see you at the next Agile Besançon event :

lun 1 fev, 2010, 19h – 21h -- Coding Dojo -- "beau code"
mer 3 mar, 2010, 19h – 21h

mar 6 avr, 2010, 19h – 21h

favorite lines from Peter Block's Community -- the Structure of Belonging

This book seems to apply to so many aspects of my life--family, political, work, volunteer efforts in the Agile community--Block begins the book by stating that we need to:

Transform the isolation and self-interest within our communities into connectedness and caring for the whole... by shifting our attention from the problems of community to the possibility of community

We suffer from too much individual self-improvement, and not enough connecting. We need to assume bounty, to be generous, to welcome the gifts of those around us, and find what we might build together... find what we want to create, together.

Organized professionalized systems are capable of delivering services, but only associational life is capable of delivering care.

It is care that we need... individual attention to giving what is needed, when it is needed, because we understand and value one another. Block says that just the very declaration of our desire makes it possible, for before anyone imagined it, it clearly wasn't possible that we would work on it, but now that we've stated our intention, there is a chance, that we could work on realizing this potential. Block goes on to cite some studies of Italian towns by Robert Putnam--apparently the towns that were more democratic, more economically successful, had better health, and higher levels of education, shared a common characteristic--what he called social capital. The citizens of these towns were more connected to each other. Putnam goes on to characterize different types of social capital--bonding (inward-looking) and bridging (cross-pollination), and demonstrates that the most important type of social capital for a community is bridging.

Peter Koestenbaum says that choosing to act upon our freedom makes us accountable, that freedom and accountability are intrinsically connected. In addition, he says that for those that hold power over others, the ultimate act of love is to grant their freedom.

How does Block build community? One meeting at a time--by forming small circle discussions, around a broad question, with a diverse cross-section of people. He asks us to focus on the future--not the past, not our differences, but our potential. He seeks small-scale and slow growth. We depend on people that choose to volunteer, rather than those who feel an obligation to show up (as in paid staff). Word choice is critical, because it is one of our only tools of relating--Block notes the difference between the "problems of community" vs. the "breakdown of community"--the second suggests that there is a loss worth reconciling. Personally I think this is a weak distinction, but may still be needed to complete the other tools he presents--focussing on possibility, looking forward, etc.

Block says that all acounts of the past are "Made up. Fiction." When we consider the narrator has necessarily left out some details, things that are important to other parties in the story, it can be liberating to agree. Though these stories are heartfelt, they can be both divisive and enriching... yet stories are one of the most effective tools for sharing. He invites us to find inspiring, hopeful, impressive stories about the future, that will help us transform those around us.

Block contrasts the corporate-driven marketing of fear, fault, and self-interest with that of community/associational relations. On the one hand, there is an idea that with more control, more laws, more police, a better economy, better press coverage, better leaders, that things would go better. On the other hand, Block advocates using our own power to be the change we wish to see in the world. When we look at what is in our own power, we can have a much greater effect than any paid service. When we choose to integrate with those around us, rather than to be insular and worry about our own survival, we increase social capital and restore community. When we restore the community, people are willing to make promises to one another, to help each other, and to be generous.

Why is it easier to raise money for earthquake survivors thousands of miles away than for the people living six blocks away that are having a hard time paying rent or keeping food on the table? Block says this is partially a result of projections--taking attributes that we deny of ourselves, and placing them on other people, e.g., laziness. What happens is we divide our local community, and lose out on the possibilities that come when we care for the well-being of the whole.

When we hear complaints of "why so few people are involved in the community", or hear people talking about "entitlement", it is time to invert the signs of despair. First, ask if there's something we're doing that is excluding some of the community. Then remember that true commitment is a gift, and can never be enforced from the outside (so no one is entitled to anything). When we have a true restorative community, people are not asking "what's in it for me"--they're giving freely. People have been summoned over and over to be part of focus groups, and then wait for the "professionals" to execute the plan. Instead we need to use these meetings as a way to engage citizens to carry out the plan themselves. In addition, the discussion needs to move away from problem solving, as that is often too mired in what exists now; for true innovation we need open-ended creativity, inspired by the right questions.

Block redefines leadership--it is convening, asking the right questions, and listening. What is interesting to note is this relational leadership cannot be measured in the ways that retributional/hierarchical leadership is used to. Action may simply be discussion and connection, not necessarily a change in physical state or status of "the problem". In fact, people may find a possibility they hadn't considered before, rendering the problem irrelevant, while commiting themselves to a new way of being that supports the whole community.

How do we build community? We find out what creates energy. We provide a forum for small group discussion, which values relatedness, is launched by invitation, is focused on possibility, represents ownership by the participants, supports diversity and dissent, expects no bartering for freely given commitments, and these gifts are received with thanks. These meetings revolve around the leader's questions... a powerful question incites action by the mere fact of answering it, like, for example:

• "what is the price you pay for being here today?"
• "what are the gifts you have to offer that you've not yet brought into this world?"
• "what is the story that you keep telling about the problems of this community?"

Before asking the question, however, one must name what distinguishes this from a retributive context, give permission for dissent, and replace advice with curiosity.

Here are a few more great lines/ideas that I don't think really need comment:

• "Advice is a conversation stopper!" "Don't be helpful!"
• "Real transformation comes only through choice." A leader becomes vulnerable when using the invitation to start out these discussions; "even though there is no cost for refusal, there is a price for coming. Everything that has value has a price."
• "The enemy of commitment is lip service, not opposition"
• "Without doubt, our faith has no meaning"... "there is no way to be awake in this world without serious doubts and reservations"
• "Don't ever underestimate the determination of others to hold on to their stories"
• Block doesn't like protest--"every time we act in reaction, even to evil, we are giving power to what we are in reaction to".
• "our life work is to bring our gifts into the world"
• For snacks at meetings, Block asks us to provide locally grown food that is healthy.

second Agile Besançon event

This is a long-delayed post, seeing that we're about to have our next Agile Besançon event this Thursday--but for our last event, Monday, Dec 14, we joined with Bar Camp for its kickoff event in Besançon. The organizers there took photos and documented a bit more, so I won't repeat what they said. I presented a topic on test-driven development with a colleague (Fréd).