Wisdom of Crowds Release Planning

At our last corporate meeting, I was invited to facilitate the discussions for the road map / upcoming releases. The slides I used were intended to keep the sentiment light and playful:

Participants were asked to come to the meeting with a list of ideas to work on for the next two releases. (Though we deploy internally every week, and customers can theoretically use these releases, our marketing efforts are geared towards the official releases that come out 3-4 times a year.) Each planning session would run for 90 minutes, and was designed to get everyone talking about what functionality is important.
Each planning segment included a slide that illustrated what each person was to focus on... often for developers it was confirming scope or estimation; for consultants/sales, it was either defining minimally marketable features or re-prioritizing their stories or taking stories away until the sum was 12 weeks. It was frustrating for people when they were asked to move on to another group, but newcomers often had great questions and new ideas. Overall the exercise was a success, from the PO perspective, because it identified some features we hadn't considered, and confirmed other things about our product vision. We ran through the whole process twice, to plan each release separately. We had a quick feedback session/retrospective in between to decide on what to change for the next time around, but mostly we kept the same process. The second cycle went a bit better because people knew what to expect, and they were more comfortable with the aggressive schedule. At the end, everyone was frustrated with the compromises they had to make, but they felt like the draft road map they'd produced reflected their collective priorities.

Setting the Stage
I opened the meeting by setting the stage--saying that officially this gathering was intended to provide input to the Product Owner (PO) team in planning the upcoming releases, but that unofficially I hoped the group could come up with something that would be good enough to be the final road map. I felt that since the whole PO team was present, their input would be considered and therefore the output could be final. I pointed out that each working session would be short--about 10 minutes, and that someone from each group would move on to another group at each pause. I also asked that people aim for equal participation from each member of the group.

Collecting Data/Making decisions
Brainstorming (5 min)--individuals were asked to review and prioritize their lists of what we should work on for the next release (or make one if they didn't do their homework).
Estimating (20 min)--we broke out into two groups of 5-7 people each; each group had at least one consultant, one sales person, a developer, and a PO. Non-devs were expected to take turns describing one idea at a time, in just barely enough detail for a developer to give a coarse-grained estimate in weeks of dev time. Unfortunately people got too caught up in accuracy and only got one turn each... and these story cards were coming in with quotations between 2-8 weeks of development. For the next round we clarified that we wouldn't be keeping these story cards--they're throw-away--and that we don't need really accurate estimates, since we're just trying to get a feel for what the scope might be. In pilot runs of this exercise, when people didn't take it too seriously, they were able to do estimates at about 1 per minute. This time around it was taking 5 minutes per card.
Feasibility (15 min)--Now it was time to see if everything we wanted would fit. The new developer was to confirm story estimates by asking questions about scope. Side conversations in the group were common, and expected, since this increased bandwidth of communication.
Prioritize (10 min)--The group needed to prioritize in order to pick the most two critical stories... these stories were going to be sent to another group. A new PO in the group had veto power over cards, but as far as I know this veto power wasn't used. A few people complained about having to send story cards away, but I thought it was an important element for consensus-building across groups.
Fill in gaps/Find themes (15 min)--A consultant or sales person arrives in a new group with two story cards, and it's time to talk about these new cards, to re-add what was just sent away if necessary, and to re-adjust the set so it sums to 12 weeks.
Prioritize Themes (15 min)--The actual story cards are no longer important--group them into named categories (themes), copy the estimates onto the cards, and be ready to present to the whole room. Everyone's set of stories will have converged, so hopefully a final draft will be easy to make.

Feedback
The time frame was very aggressive. My estimates were off, but ultimately we used the following--brainstorming, 5 minutes; estimation, 20; feasibility, 15; prioritization, 15; themes, 15; prioritizing themes, 10; leaving 10 for shuffle time. I had hoped to have a step or two after what's pictured, where the whole group meets to discuss the set of features for the proposed release, but we ran out of time. People were frustrated with not having prepared enough, with having seen their good ideas get bumped to a later release, or get dropped from the schedule--but the PO team was very happy. We felt the priorities corresponded well with our vision, and some items that weren't on our radar scope showed up.