Agile planning and the law of diminishing returns

The post that follows is the translation of a post I originally wrote in Portuguese back in June 2010. I found that these issues are still very much current, even in light of (or perhaps even more so because of) current trends such as the #noestimates movement. I also found that in a rare case of intellectual consistency I still agree with what I said back then.

How long should you take planning a sprint? How detailed should this plan be? Ken Schwaber's Scrum Guide recommends an eight-hour timebox for a one-month sprint, and four of these to be used for estimates. Shorter sprints should use proportionately less time. That is, planning a two-week sprint should take four hours or half a day. The same guide also states that typically only 60 to 70% of the sprint's backlog is known during planning - the remaining will be discussed and defined during the sprint's execution.

Problems start showing up when teams try to unravel the 30 - 40% that are "missing" during the planning meeting. The goal is that these definitions spring up from the execution of the work and the communication between the team and the product owner during the sprint.

If that is not happening, we must look to eliminate the root problem. Identify if the product owner is missing or unavailable during the sprint, or if the team has issues with either quantity or quality of internal communication - but it is dangerous to fall into the temptation to try to anticipate every discussion that should take place during the sprint. An excessively long sprint planning meeting creates an unnecessary overhead to each sprint and does not guarantee more precision in estimates for the stories.

Here we see a concept borrowed from economics come into play, called the law of diminishing returns. This law states that in a production system, there is a point beyond which the increase in an input variable does not produce a proportional increase in the output. That is, if you spend twice as much planning the duration of an activity, your estimate will not be twice as precise. This concept becomes evident in the chart below, based on Mike Cohn'sexample (Agile Estimating and Planning, 2006).

Law of diminishing returns applied to software planning

As with everything in life, balance is fundamental. The team needs to find at which point in this chart's curve they want to be. The point at which everyone is comfortable about the work to be done in the sprint, without adding too much overhead at each sprint with overly long planning meetings. I believe, in fact, that planning sessions that are too long reduce the precision of estimates due to people naturally feeling tired and dispersed in a meeting that becomes very taxing.

There are those who say that breaking stories into tasks is not necessary for experienced teams, but I still prefer the middle path...

You do not need a great project effort here, just enough to get a good task list. ... Remember that the goal is not to design everything that is required for the iteration, but to help identify what is the best task split. You will have other design sessions as part of the execution of the tasks.

Kent Beck and Martin Fowler, Planning Extreme Programming