He Came to Bury Agile: Long Live Agile

It's Time for Agile 200x to Cede to the Smaller Conferences

Last year's keynote speech by Alistair Cockburn at the annual Agile Alliance conference was called "I Come to Bury Agile, Not to Praise It". I too am here to bury the Agile so many of us know, the Agile paid for by hollow certifications and expensive conferences and phony marketing. I'd argue that the term "Agile" itself was a phrase coined for marketing purposes--but that's another story(*). This story is about how we need to reach out in support of the whole community, rather than spending our attention/resources on a select few. What do I mean by a select few? Well, there are over 50k CSMs, and only around 2k attendees at the Agile Alliance conference every year. That's 4% of CSMs (and less if you count the rest of the Agile community). Less than 4% is a select few. Why don't more people go? I'm not sure--but for most of the colleagues I've worked with, it's simple--they can't afford to "pay to play". Me neither--if I pay early-bird rates, get super-cheap airline tickets, pay for my hotel and food, this sums to a month's-worth of my take-home pay (I'm working for European wages). A full month. What about corporate expense accounts? Not for this--all my colleagues and I have worked for small companies that can't afford it either. It's time to find an affordable way for the typical developer to participate in Agile conferences.... and that's through small, free/low-cost, local conferences, like the XP Day series, the Simple Design and Test Conference, Bar Camp, Agile Tour, etc.

Where I'm Coming From

Throughout my 10 years of agile software development, I've been getting more and more into the Agile community. I love this community because it is so open-minded, so willing to apply ideas from other arenas, and so ready to share knowledge without claiming intellectual property rights. We've built strong local user groups, e-mail lists, open space conferences, code camps, and have given legitimacy to the idea of sustainable pace--all without spending much money out-of-pocket. This is the Agile I know.

Yet, there's a part of the community I don't know, a part maybe I don't even want to know: the big, expensive conferences and training classes. It seems like all my role models are a part of this, so I figure it's worth trying, at least. Yet, how do I pay for it? This year I was happy to discover that the Agile Alliance offers speaker compensation packages that cover registration fees, hotel, and a stipend big enough to cover food and taxi from the airport, leaving only the international flight for me to cover myself. I thought this was a comparably affordable way for me to go to a conference. So I decide to prepare a submission. A couple years ago one of my role models, Yves Hanoulle, said he only goes to sessions that are prepared by a pair of presenters. OK, so I find a co-presenter. It's bound to make my submission better. So I begin the submission process, and find out right away that the speaker compensation policies for Agile 2010 and XP 2010 only reward the 'first speaker'. What? Aren't we, in Agile, supposed to believe in rewarding the team, not individuals? Well, I promised my co-presenter that I'd split the booty equally. Hmmm... now I'm down a couple hotel nights, stipend money, and worst of all, half of the conference registration fee. Well, I can fix that--I'll do two talks. I find another topic, another co-presenter, and away we go. Time goes on, a few people make good recommendations, we improve our proposals. I start reviewing other people's sessions, and see a lot of good ones. I'm not feeling all that confident, having never been to a big conference before, so I do one more submission. It's so much work to write a submission, I decide I may as well post it for several conferences... and soon get accepted for XP 2010 and Agile France. Woo-hoo! Maybe people do think I have something interesting to say....

With all my submissions made, I start appreciating the peer-review functionality of the Agile 2010 system. I start reading and commenting on other people's submissions--and find that this system is the best I've ever participated in. Here's the real Agile community I was looking for--people collaborating, seeking the best in others, and supporting one another. I ended up leaving comments on 130 proposals--hoping to give back what I had gotten so far from the process. I know stage producers had to do a lot more, but I think I can appreciate what they have done during the selection process. Ultimately, none of my talks were picked for this year at Agile 2010. Still, I noticed that the submissions from people that had established reputations in the field got 5 times as many comments as newcomers. I really wondered how people that hadn't participated much in the community would get mentored if they weren't getting equal support through the comment system. As a result, I think the review process needs to go double-blind.

The Revolution is Afoot

Maybe the rejection from Agile 2010 is clouding my judgement right now. Or maybe it's making it so I see more clearly. I did get in to other conferences, and all along I've been frustrated with the costs of doing a presentation. In any case, this big up-front selection process, in which a few volunteers meet to judge which talks are best suited for the conference, is simply not Agile. It's too cold, it's too Taylorist, it's too micro-optimized. It's not consistent with the relational vision of the Agile Community. There are already several disruptive changes in our midst that will outreach this top-down model of a conference. One, for example, is the Agile Tour, which had more conference attendees last year than the Agile 2009 event, and which cost an order of magnitude less. It gave a stage to practically anyone who wanted to talk, and attendees voted with their feet, during the day, to show what was most valuable to them. Another sea change is the growth of groups like the Agile Skills Project, which has reached over 800 members in less than a year, and the Diversity In Agile Project, which shows that we are recognizing the problems that come from undervaluing the full community.

The Agile 200x series of conferences has struggled for years with their selection process. Why not give up, and go with the Agile Tour model? Keep the peer reviews, keep collaborative improvement of session proposals, and get rid of the high-cost of flying everyone in the world to one city. Send invited keynote speakers on a short tour, and let everyone else in the door for free. Corporations can sponsor the venue and food, and the events could happen twice or more per year.

(*) Note the company names of the best-known consultants in the field--do they have 'agile' in the title? No! Because the word Agile is a marketing gimmick.

When To Slack + What it Gives Us

This is the second in a series of posts, beginning with Slack--we're Not Slacking Off!

We need slack in order to be creative, to maintain a sustainable pace, and to improve. So when should we slack? To answer this, we need to understand the costs and benefits of slack.

What is its cost?

Zero. Apparently, on Ilja Preuss' team, they schedule a full day of slack once per month, and they see no drop in velocity for that week. Our team aims for a full day per week of slack, and I'd say we get an increase in velocity for that. Based on this statistically insignificant sample size, I have to say that slack is free or value-producing. So last week in retrospective we decided to call slack our default activity and label card work a side-job. We said this was to emphasize the "research" part of our R&D group, to make sure everyone knows they're supposed to be coming up with innovative, even disruptive, ideas. I admit the idea of full-time slack is a bit extreme, so let's keep holding on to the one-day-per-week goal, 50% of the time.

What are the benefits of slack?

• Beck, in XP Explained 2nd ed, talks about slack as a way of making sure we meet our commitments--it is about transparency and building trust.
• Slack gives the Product Owners perspective on the health of the code and of innovation. If all developers do during slack time is refactor, we can ask if they're taking on too much work (and creating technical debt). If they're tweaking functionality they worked in the past, we may need to look at functional debt. If they're creating entirely new prototypes, then we only need to monitor the feedback cycle on these ideas to be sure the value is qualified before much is invested. Near the end of the iteration when I see people hovering around the story wall looking for something useful to do, I know they're not overworked--and when I see them starting prototypes, I know the iteration buffer is big enough.
• When the team reaches slack before the end of an iteration, they can share in success together--maybe they were lucky, or maybe they worked together well, but in either case, it's a success. The value is two-fold: the iteration work is Done-Done (and hopefully deployed and generating business value), and the team bonds are stronger.
• DeMarco says slack is a way to give employees control. Dan Pink talks about why control is important--motivation comes from autonomy, purpose, and mastery. So giving employees control is a way to keep them happy, motivated, and productive.
• Retaining our employees, as the Toyoda family and DeMarco have noted, is the most significant means of avoiding huge drops in productivity.
• DeMarco talks about being too busy, the myth of total efficiency, the impact that no slack has on change... we are not cogs in a machine, we can't be interchanged and we can't task switch without penalty. If we don't invest in slack, we will end up being overly bureaucratic, rigid, and conservative.

So when should we slack?

• Whenever there are no more cards to work on
• Towards the middle of the week (because people are tired Friday and Monday)
• When the whole team agrees there is no useful/effective/efficient way to divide up the remaining work any further. If some people are in slack while the rest of the team is pulling in the iteration, it can deteriorate team spirit.

One team I worked with ended iterations Tuesday night--it gave a beautiful cadence to an iteration... we finished Tuesday, slacked Wednesday + had a planning game Wed afternoon, started the iteration Thursday & Friday; by Monday we came in and asked "are we at least half way through? ", then hustled our way to the end of Tuesday again. It was nice to have a weekend break in the midst of an iteration--the distance from the work gave us new insights.

Slack: we’re not slacking off!

In a technical climbing technique known as lead climbing, a rock climber, the lead, takes a calculated risk by climbing above any anchors, in order to climb higher.  This kind of climbing requires precise coordination between the lead and the second, for without slack, the lead cannot advance at all; with too much slack, a fall is more dangerous; and with too little slack the lead can be thrown off balance and fall.  (Photo Credit)

Note that it is physically impossible to climb without slack. When it comes to software, however, I’ve too often become obsessed with productivity, moving cards, increasing velocity, process improvement, and delivering business value, to the point I optimized out all slack.  It’s not that I’m the only one to get lured by the “myth of total efficiency”—it is a common problem in agile and non-agile environments.  Slack is covered authoritatively in Tom DeMarco’s Slack, included in the core XP Practices in Beck’s Extreme Programming Explained, 2nd edition, institutionalized by the Pomodoro Technique, and championed by Deming and the lean school of thought using pull scheduling.  Yet on a recent discussion on the Extreme Programming yahoo list, I got the impression that my understanding of slack is a bit unique.

 

Buffer Slack and Creative Slack

Back to rock climbing—a good second will always give the lead at least some slack until the lead starts descending—triggered either by a fall or the lead calling “belay!”.  There are two kinds of slack on the cliff—the constant foot or so of buffer slack that allows the lead to move freely in the same spot.  The second kind of slack is the excess rope that the second liberates only when the lead is climbing higher.  This creative slack is given when the lead requests it explicitly, by yelling “slack!”.  This call signals that the second must stay vigilant, because it means the lead’s life now depends on correct belaying.

Just as in rock climbing, I see two kinds of slack for a software team.  The first is buffer slack—it is the down time between official work, be that work related to story cards, administration, or meetings.  It is work that happens when we’re waiting on someone else or we’re not yet ready to take on new work.  Buffer slack is what gives us time to reflect and to consider the larger context of what we’re working on.  We might fix a bug, or refactor some code that was out of scope of our last story, but that we just happened to notice.  It doesn’t require communication with the team because it won’t impact them or the goal of the iteration.

Creative slack is analogous to climbing higher on the rock face—it requires a bit of forethought, a consideration of the feasibility and impact of the next few moves.  Creative slack requires a significant investment and focus before one can start advancing. It allows us to improve our current technology, to build prototypes of potential new features, and to consider strategic moves for the product. I’m not giving license here for YAGNI or  BDUF.  It’s just that we can’t make significant gains without some forethought. Creative slack happens when the team explicitly schedules it; on my current team it’s the time between iterations, and it is typically 15% of our work week.

It’s funny that I didn’t discover the second kind of slack in the software realm until I came to France.  Buffer slack happens naturally, unless we over-optimize our process.  So when I intentionally left some room for it on my team in Philly, I thought I had slack—and I did—buffer slack.  Yet our retrospectives in Philly identified a problem—a lack of innovation—that we weren’t sure how to fix.  Our product owner insisted that any time he heard a good idea, he wrote it up on a card and scheduled work on it when he saw fit, so he denied that innovation was suffering.  What I didn’t realize is that we needed to schedule creative slack for the whole team.  When I joined my team in France, with the standard 7-weeks vacation, 35-hour work week, 2-hour long lunch breaks, there was a lot more natural buffer slack.  So when we started nailing iterations early, all the buffer activities were already done.  I noticed that the end-of-iteration slack was creative in ways that simply was not possible in buffer slack.  It was a new kind of slack: creative slack. I’m not the only one to have learned about Slack from the French culture of working to live instead of living to work.  Though he is a native Pennsylvanian, DeMarco studied here in France, at the University of Paris.  Maybe that’s why he knows so much about slack.

Note: Though important for recharging, non-work activity (returning a personal phone call, lunch break, evenings and weekends), is not slack in my opinion.  Slack is not the same as slacking off.

 

The Published Discourse on Slack

One of the main reasons I was attracted to Extreme Programming, when I first read about it in 1999, was the idea that we can do good work without overtime.  In Beck’s Extreme Programming Explained, 2nd ed., he explains that slack is about building trust—about keeping our iteration commitments.  This is obviously buffer slack.  James Shore builds upon the buffer concept by saying that it's the variability in problems that determine how much slack we need.  In the Pomodoro technique, we learn to pause every 25 minutes.  Why?  Partly to make sure we're doing what's important.  In my opinion, this is buffer slack too—it is still focused on the tasks at hand, rather than uncharted territory.

Though not specifically labeled creative slack, DeMarco’s work touches upon the idea, by identifying that if you don't have slack, you can't take risk, because there's no room for recovery from failure.  He also says that the more optimized you are, the less slack you have, and the less you're able to change.  Both risk-taking and change are likely to lead us into new territory, and is therefore, what I’d call, creative slack. 

 

Unanswered Questions

I have a lot of questions about what to do with creative slack.  Apparently Google requires that employees demonstrate a proof of concept before they are allowed to invest significant amounts of slack time in a project.  My team is not doing that—all our creative slack is purely free and unmanaged.  This doesn’t seem quite right to me because some of those projects have no value to me as a Product Owner—but I do not interfere because I think the energy these projects generate gives enough of a velocity boost that it’s justifiable.  I also don’t want to kill good ideas that I just don’t understand yet.

What do you think about the following?

• Can you manage slack (like Google does, without diminishing the energy generation of slack)?
• Can you suggest items for exploration (without impacting the restorative power of autonomy)?
• Should slack be unconditional?
• Should it be a reward?
• Does discussion forum/tech news/book reading/technical exploration count as buffer slack or creative slack?  Or is self-improvement slack a category that should be explained here?
• How do we balance individual needs for slack with team needs?
• Is creative slack part of your job?

Agile Besançon--March

This month at our Agile Besançon meeting (meeting announcements at http://groups.yahoo.com/group/software_dev_in_free-county/), we did a Cowboy Coding Contest.   Six people came,  which is definitelyenough to have fun, and we set up one  laptop with a projector so everyone could see what Ludovic and I were up to.   Initially we thought we’d do multiple attempts at the same  algorithm (implementing a Decimal to Roman Numerals converter), but  we wanted something new for the second try, so we did a binary search instead.  We spent just over 30 minutes for each programming task, followed by a review of our approach.  Anyone who wanted could share their code, their algorithm, or speak more generally if preferred.  Since people were allowed to code in any language, we ended up seeing implementations in Java, Python, and C++!  I would have gone faster if I’d picked Java, but I don’t want to lose everything I know about Python, so I practice it when I get a chance…

As usual, we ended our meeting with a quick feedback cycle… people enjoyed coming out to do code, but thought the algorithms could have been laid out more clearly (bduf?).  They also thought algorithm implementation isn’t good practice for object-oriented programming.

Point Synchro with a timebox

Some of our synch points have been too long recently, so we decided to timebox them. When the music plays, we meet around the story board and start a 5-minute timer. We look at the story board, focused on what it will take to move the stories along--and whoever is closest to the board asks people questions, using the stories as the focal point, instead of going around the circle to give everyone a chance to speak. At noon the meeting is slightly longer, because we make sure everyone has a chance to speak--but the rest of the time the objective is just to find out who should be talking to who.

favorite lines from Peopleware

Peopleware, Productive Projects and Teams, by Tom DeMarco and Timothy Lister, is a classic software project management book for good reason. They discuss problems that have existed for a long, long time, and are not going away any time soon--the High-Tech illusion: "the major problems of our work are not so much technological as sociological in nature".

Counter-productive Managers
Much of the management culture that has been exercised for millennia is simply counter-productive:

• Rather than scolding employees for making mistakes, pro-actively ask them "what dead-end roads they've been down" recently. If none, then they're not being creative enough, not taking enough risks, and not going to provide the value we expect of them.
• On average, they claim that salaried workers spend only 5% of their time on "planning, investigating new methods, training, reading books, estimating, budgeting, scheduling, and allocating personnel". Without a curious/reflective attitude, how will the workers improve?
• There's no such thing as overtime. "The best workers have been through it all before... they take their compensatory undertime when they can, and end up putting in forty real hours of work each week". Instead, managers need to identify workaholics, and encourage them to pay attention to their personal lives, because otherwise, ultimately, they'll burn out.
• We cannot increase productivity in knowledge work by turning up the pressure. Any short-term gains in productivity have to be weighed against future loss of employees.
• Cutting corners on quality is no way to increase productivity. Just as the lean folk say today, DeMarco and Lister said 25 years ago that "the trade-off between price and quality does not exist in Japan. Rather, the idea that high quality brings on cost reduction is widely accepted". They add "Quality... is a means to higher productivity", yet "Quality is free... to those who are willing to pay heavily for it". That is, we must pay heavily for quality but the returns justify the investment.
  
• Parkinson's Law: "work expands to fill the time allocated for it". The authors say this is a destructive myth.
• Coding Wars: the authors collected data from various environments and compared time-to-completion for correct implementations of sample programming tasks. They found a 10:1 ratio between best-to-worst, and better half being 2:1 as fast as the wore-performing half of programmers. What was more interesting, though, is there was a 10:1 ratio between best-to-worst organizations. If you work in a noisy, disruptive office, well, everyone is affected. If no one can can get any work done "between 9 and 5", work on it.
• Disrupting the flow of concentration may require "fifteen minutes or more" to get productive again. Many environments are so disruptive, no one gets into flow enough to get anything done. When applied to an agile context, I think that flow is different. The authors concede that people doing similar work in the same room aren't often disrupted by the hum of their coworkers, and the general consensus on the XP list is that we're trying to optimize the flow of the team, not of individuals in a team environment. That means interruptions related to the current activities on the team are OK, a change in direction and distractions are not (hence the Scrum rule prohibiting changes to the sprint's cards?) Answering the phone--is obviously disruptive. The authors advise against electronic interruptions of any sort.
• Cornell University's findings on the effect of music--workers were less creative!
• "Professional means unsurprising". An organization is constantly becoming more uniform, more rigid, more standardized. This reduces "the potential to generate energy or do work".
• Aptitude testing doesn't work for making hiring decisions--but it does help employees do self-assessments, to motivate them to improve and to work hard. What we need is autonomy, purpose, and mastery, according to Dan Pink.
  

Supporting Teamwork
They say that "someone who can help make a project jell is worth two people who just do work".

• In Alexander's A Pattern Language, he writes "without communal eating, no human group can hold together...we found this worked most beautifully when we took it in turns to cook the lunch"
• "The business we're in is more sociological than technological, more dependent on workers' abilities to communicate with each other than their abilities to communicate with machines"--and so recruiting should include an evaluation of a person's communication skills and sociological aptitude.
• Change the culture of turnover. Encourage employees to sign-on permanently.
• "Voluminous documentation is part of the problem, not part of the solution". And so the Agile manifesto wasn't first to publish this idea. They even touch upon the idea of complex rules-->simple behavior; simple rules-->complex behavior explained in Complex Adaptive System theory, and talk about "malicious compliance" where employees can follow the letter of the law while going entirely contrary to the spirit of it.
• This is the book that coined the term "jell" to refer to a cohesive, interdependent group of individuals whose production of the team is greater than the sum of its parts.
• On a jelled team, managers don't need to provide motivation--it's already there. The people find the work more enjoyable than it should be--just because of the team. The goal could be arbitrary--but everyone has agreed to help achieve it.
• Much of our work is still independent--the team serves to make sure everyone's pulling in the same direction.
• "You can't make teams gel"
• Autonomy only exists when managers give their employees room to make mistakes.
• Managers who support teams aren't doing the work of the team--they're focused on healthy relationships
  
• Promote excellent quality
• Make (even fabricate) milestones, so people can see progress and feel done
• "Promote eliteness"
• "Encourage heterogeneity"
• "Preserve and protect the team"
• "Provide strategic but not tactical direction"
  

What is a jelled team?

• they have a strong sense of identity (shared jokes, shared lingo)
  
• they've got a sense of eliteness
• joint sense of ownership
• individuals seek peer review
• interactions are fun, easy, healthy, warm, confident
• loyalties within the team are stronger than to the company

One example, Black Team (at IBM?), was particularly impressive--they formed a culture that stuck even when individuals eventually moved on; their effectiveness improved dramatically over time, and the company considered them a great success.

we need stories from more lean companies

From what I understand of the mass recalls from Toyota, the company lost its culture of quality first. Apparently some managers chose to prioritize growing the company, which caused a conflict of interest any time a problem was discovered--and managers didn't invest the resources to dig to the root cause of the problem. I think this story doesn't kill the credibility of the Toyota Production System, but I do think that it shows that even a 50-year-old culture can be compromised if a company tries to grow too fast. I think that Toyota still proves that average people can have a collective intelligence greater than their own individual strengths, but that this is very fragile.
So much of the lean literature and discussion is about Toyota that this has been rough news... we need to hear more stories about Dell, Zara, Amazon, and other lean companies. Where do you get this kind of news?