Estimating.dev

Estimate Confidently

estimation

Agile Estimation Theatre

by

We’ve all experienced what I like to call Agile Estimation Theatre. Imagine it’s the beginning of the sprint and your team is holding a sprint planning meeting. The project manager has built up a set of stories a few of which have subtasks. Now you will spend the next two hours trying to create estimates for each of those stories. 

The project manager starts by reading the first story. Someone asks a question about whether ECS or EC2 are a better solution for the problem which devolves into a 20 minute conversation about the merits of various AWS services. The project manager is forced to stop discussion to leave any possibility of estimating all the stories slated for the sprint. You open your planning poker tab and evaluate your options 1,3,5,7,13,21,34, 55. After a moment you decide on 13 since you are sure the task will take at least a day or two. One of your coworkers then asks whether the numbers are days or hours. That question leads to a 10 minute discussion of what exactly story points are and how they are definitely not a unit of time. The teams votes range from 5 to 34. And then project manager overrides the score to be 21 since that’s what the senior engineer picked. 

The two hours continue to drag on. By the blessed end of the meeting your team has ‘estimated’ 4 out of 20 single sentence stories. Since the sprint starts tomorrow the project manager holds a quick meeting with the senior engineer to estimate the rest of the stories before 5pm. 

The above tale represents what I like to call agile estimation theatre. We have a sprint planning meeting where we’re supposed to involve the entire team in estimation, so we schedule a two hour meeting and hope for the best. Sometimes it actually works, but often we end up with questionable estimates for single sentence tasks. 

The fundamental problem with Agile Estimation Theatre is not investing enough time. To get good estimates you need to have complete stories. You need to break down subtasks and specify them more thoroughly than a single sentence can possibly handle. Then you need to have your team spend time reviewing the stories individually and coming up with estimates. The problem is that it doesn’t make sense to spend several days out of a two week sprint on estimation. To do it right you need to have well specified stories. Building out the stories will take at least a few engineer days. Next you need to have the entire team spend 2 hours each estimating stories individually. Finally you add together the individual estimates and do the sprint planning meeting. Now repeat that process again every two weeks. 

The breakdown is that the expected value of the estimates is typically less than the cost of spending a few days to produce good estimates. The organization intuitively knows this which is why we spend two hours in sprint planning instead of two days. But because we want to do scrum or agile ‘properly’ we need estimates. The end results is that we denature the estimates to comply with the process. 

Estimating Unbounded Agile tasks

by

I have been reading Software Estimation (https://www.amazon.com/Software-Estimation-Demystifying-Developer-Practices/dp/0735605351) after a recent conversation with one of the product owners on my team. The Senior engineer on my team has pushed us into a ‘pointless’ estimating process where the estimate is just the count of the subtasks in the story. 

The Product owner wanted us to provide time or t-shirt size estimates earlier in the process before we did the work to break up the story into subtasks. This discussion broke down into people arguing over agile… 

Personally, I am quite disillusioned with how we create estimates a part of the Software Development lifecycle. The standard  estimation technique in my experience is to compare a new project with a past project based solely on personal memory and gut feel. Then to double your estimate from what ever you came up with. The computer science and engineering classes I took did not cover anything related to estimating projects. Which is a little strange, since other engineering disciplines like Mechanical and Electrical engineering cover estimates. 

The business needs estimates so that we can coordinate work with plans, make deals and hopefully profit. If you promise a client something and the schedule slips 6 months that is a problem. If you sign a 10 million dollar advertising deal and the delivery schedule slips 3 months it is a problem. So the business needs estimates that are accurate to plan things. 

But if we are just guessing as the engineering team what is the point? If our estimates are constantly wrong and the levels above us ignore them why do we need to waste our time ‘voting’ and playing ‘planning poker’? 

Then there is the issue that in agile we are very loose with stories. Subtasks are added and removed with little oversight. After a story is estimated things can change and suddenly the story goes from 3 subtasks to 10.  The next day 8 of those subtasks are moved into another story. What were we estimating again? 

I will be writing a bunch more on estimation, but some of the things I have gained so far are that expert judgement is considered the weakest means of getting an estimate. The best way is to count something whether stories, square feet of drywall or some other work component. The second means is to use heuristics to compute your way into an estimate. The last resort is expert judgement or gut feel. 

Count > Compute > Judgement

Common Software Estimation Pitfalls

by

Relying solely on expert judgement 

It has been a common experience in my career to sit in a room with a team of engineers and project managers to estimate stories. The typical process is to read through a story discuss it and then ask each engineer how long they think it will take. The engineers guess some number of days, hours or story points. Then the number goes into JIRA somewhere. 

The engineers are experts here, they have done this a lot of times and are much better at it than the average person on the street. But if all you are doing is asking for one number from an expert who isn’t trained in estimation, you will not get good estimates. The most effective estimates come from Counting and Computing with Expert Judgement as a last resort. 

Using estimates in a group setting

Most backlog grooming and other estimation meetings are done in a group setting. Engineers make their estimates one at a time, then the team averages them together. This setup has two issues, first each engineer knows what the previous one estimated and will subconsciously adjust his estimate to match. Less experienced or less confident engineers will completely abandon their judgement and defer to more senior and louder spoken team members. Planning poker is a popular solution to this problem, but it still falls victim to the next problem below. 

The second problem is time. You cannot make a good estimate in 5 minutes. Most estimation meetings run over time because the stories are incomplete and the team ends up discussing them. Each discussion has a risk of going off on a 10 minute tangent. Then with 10 minutes left on the clock the project manager wants to finish estimating 8 stories. 

Not having historical data

I have worked with over 10 different teams in my career. None of them have had consistent historical data going back more than 3 months. A lot of agile estimation assumes teams have historical data they can use to calibrate their estimates. The reality is that teams do not have that data. Either the team does not track both the time to completion and the estimate for a story. Or commonly their is a new project manager every 6 months who implements a new agile process based on story points instead of time estimates. 

To get good historical data you need to have consistent systems for estimation and data collection across teams. Teams are not a good level of granularity for historical data because they change all the time. Teams are dynamic managers change, engineers move to new jobs, ownership of a service moves to a new team. You won’t get consistent historical data if the process changes completely every time your project manager job hops.