Estimates are Guesses

Wishful Thinking

Everyone knows why having accurate project estimates would be valuable to a business; knowing when something will be done is like a crystal ball that would let the rest of the company run more smoothly in many different ways. Promises could be made to customers, teams could be perfectly balanced, performance could be objectively measured, and marketing could be scaled up just in time. Everything would be easier, faster, safer, and more efficient.

Unfortunately, this is all built on a fantasy that estimates are at least moderately accurate. Real work is messy in countless ways and even good-faith, well-informed, carefully-considered estimates tend to go wrong and when they do they can be wrong by incredible margins, often even by orders of magnitude.

Moreover, the act of giving estimates can often have an extremely negative effect on teams, especially dysfunctional ones that can’t fall back on a history of respect and trust.

Why are estimates so inaccurate?

Regularly being able to provide accurate estimates requires a huge amount of effort and discipline across an entire company plus the right kind of product at the right stage of its life. Almost no business is actually willing to shoulder this cost and the ones that do tend to work on things like satellites and fighter jets where the cost of long development cycles almost doesn’t matter compared to the cost of mistakes (and these also tend to overrun their estimates and budgets).

The reality of day-to-day work for the average software business, however, often makes an environment where accurate estimates aren’t even realistic.

• Creative work is extremely difficult to reduce to simple numbers, especially across multiple different creative people over time. It has very little in common with assembly-line style factory work where estimates can actually work.
• The reality in many disciplines is that you’ve never done the exact thing you’re estimating before, meaning there is always a level of uncertainty. This is especially true in software development since you’d simply use the existing code if you had written it already.
• There is zero upside for being accurate when giving an estimate but there is often a downside for being wrong.
• It is incredibly rare to get everything right on the first try - this includes the idea, the validation, the spec, the project plan, the tests, the implementation, the rollout, and everything else. Every little misstep has a compounding effect.
• There are too many complicating/time-wasting factors (like meetings that could have been emails) but we tend to estimate using an idealized view of how much time we will be able to commit (eg, 1 week if I was working 8 hours a day uninterrupted).
• Honest or optimistic estimates tend to get abused - there are too many horror stories of estimates being turned into deadlines and people getting punished for missing what was originally 'just a ballpark estimate'.
• Parkinson's law (work expands to fill the time allotted) and its real-life cousin "there is an unending stream of 'little' requests for maker time and makers rarely say 'no' until it starts to impact a deadline".

Why do we keep doing it?

Even after watching estimates be inaccurate over and over again most companies still insist on it (and don’t even try to address the issues above). Having something concrete, even if it might be wildly wrong, makes people feel like they understand more of what is happening, especially for work they don’t know how to do themselves.

If you assume good intentions from everyone involved then estimates, even rough and dangerously inaccurate ones, can at least provide something to use for making tradeoff decisions and a bit of structure that others can plan around.

On the darker side, it is common to hear stories about managers asking for / insisting on / demanding estimates, often with assurances that they are ‘just rough estimates’, then throwing them back on the estimators like deadlines when things end up inevitably taking longer. Sometimes, especially when times are tough, it can feel like your coworkers and bosses only demand estimates from you so that you can be on the hook if things go wrong instead of them.

The sad truth is that many companies simply continue this process because it seems like the safe route since ‘everybody does it’. Nobody gets fired for doing the normal thing, even if it doesn’t work out well, while taking an overt risk like not requiring your teams to estimate projects can put your job at risk if it doesn’t deliver spectacular results.

The problems with "No Estimates"

The "No Estimates" crowd have described the problems with estimation at great length. Allen Holub has a great keynote about why effort put into detailed estimation is almost entirely wasted, both because it doesn’t increase accuracy and because all of the time spent on it could go into working on the actual product instead.

Eschewing estimates completely often seems like a dream for builders but requires an incredible culture of trust and communication in order to not be a huge thorn in the side of the rest of the business. Even then, in order for things to run smoothly, a team with no estimation process requires a very stable project to work on plus consistent, experienced makers who have a good understanding of everything that needs to change or be added. Brand new projects or brand new teams are often too volatile to generate a steady flow of the right work, which is what makes it possible for the rest of the people involved to deal with not having any estimates.

Giving project estimates can have a silver lining

Even though the quest for an accurate estimate is a futile one, there is still a lot of value that can be gained from the estimation process while avoiding the worst of the downsides. “No Estimates” can be very cool in the right environment with the right project, but it loses out on the very valuable data we actually can derive from the estimation process itself.

• Having to think about when a project will be completely done requires thinking about the entire project, which can sometimes help prevent tunnel vision.
• Tracking estimates over time can give an approximation of progress.
• Estimating as a team can provide opportunities for the estimators to learn from each other, especially when discussing system design and dealing with large unknowns.
• Estimating on a regular schedule can lead to breaking tasks down into more detailed subtasks, which makes it easier to work.

If you are forced to provide estimates for a project, make sure you’re at least getting some of the above benefits along the way.

Best of both worlds?

There are a plethora of project management techniques out there that try to make estimating more accurate or less painful, but the one I’ve found to be the most helpful in my own work is a bastardized version of "3 point estimation" specifically designed to avoid the list of bad things discussed above while maximizing the secondary benefits. By embracing the fact that estimates cannot and will not be accurate we can get rid of the toxic parts and expose the good bits underneath.

The goal is to get quick, casual guesses from each team member for an absurdly optimistic date and an unreasonably pessimistic date. Making them explicitly unreasonable and calling them guesses helps take the pressure off of giving them and reduces the worry that someone will treat them like actual estimates. The team should be able to give them within a minute or two; if they can’t then your project plan isn’t clear enough and you should go work on that with the team until it is.

Having both the too-early and too-late guesses also makes it easier to get useful insights than just having a single date. The guesses over time are useful on their own but looking at the difference across the team for each type of guess can highlight places that the team needs to better understand (or agree upon). Most importantly, measuring the range between the optimistic and pessimistic dates is probably the best single indicator for general project health that you can get. If the gap is shrinking faster than time is passing you’re on the right track. If it stays level, you’re failing to make progress. If it expands, you don’t have a viable project plan and need to go make serious changes immediately.

Implementing this is intentionally simple:

First of all, explain this entire plan to the team. Explain that these are guesses, that they should be made quickly and with no stress, and that they are explicitly designed so they can’t be turned into deadlines. Reassure the entire team that you know it is impossible to accurately guess when the project will ship but that by tracking these guesses over time you can get some useful insight into how the work is going and, more importantly, early warning signs if something is going wrong.

Every week (or whatever cadence is right for your group) do the following:

1. 1. Quickly go over your project and the current plan for getting it done with the people who will be actually building it (no PMs, no sales, no marketing, no middle management; builders only).
2. 2. Have every builder involved (and no one else) make 2 guesses (this should only take about a minute):
   1. A. An unreasonably optimistic date for when the project could theoretically ship if everything went perfectly – no mistakes, no surprises, no changes of plan, no meetings, no on call.
   2. B. An unreasonably pessimistic date for when the project would ship if everything went terribly along the way – specs changed, a major system was missed, a team member leaves, production constantly goes down. The only requirement here is being really confident that the real ship date, whatever that is, will be sometime before this pessimistic guess.
3. 3. Track each of these guesses somewhere (I started with a spreadsheet then made devestimates.com to spit out better graphs and to make it easier to track multiple projects over time).
4. 4. Look at the estimates as a whole and use your brain; consider what each metric means and if it should be going up or down over time. When you see something off, spend the effort to figure out the cause. Think about things like the following:
   1. - The average difference between the optimistic and the pessimistic guess is a rough gauge for the team’s uncertainty about the plan. This should go down quickly over time.
   2. - The difference between each of the optimistic guesses or the pessimistic guesses can illuminate when the team doesn’t agree on the plan. Explore this and you might find a shorter path forward but more likely you’ll find a landmine you weren’t expecting (but hopefully before it explodes).
   3. - Sudden spikes in someone’s pessimistic guesses are a waving red flag that something changed - it might be in your project or completely unrelated but you should find out what you can as quickly as possible.
   4. - If you make guesses every week and the pessimistic guesses always push back about a week then you’re making literally zero progress. You should consider the project severely at risk until something changes.
   5. - Estimators are reluctant to move back their pessimistic guesses by much each time, even if they should. This is one of the same reasons that normal estimates are so inaccurate. There probably isn’t a way to fix this but consider changes in the pessimistic guesses with more weight than changes to the optimistic ones.

Don't ruin it!

This is not a process you can blindly follow; it requires diligence, communication, and taking the time to really think about what you’re seeing in the numbers (plus the effort of digging in to find and address the issues it reveals).

• Don't succumb to the temptation of turning the guesses into real estimates you plan with. Doing this will be wrong but will also erode the trust of your team. If your boss demands an estimate then make one up that is after the pessimistic dates but don’t plan on being right.
• Don't suggest people change their guesses, even if you think the guesser missed something.
• Don't let the estimators know the other guesses before submitting their own.
• Don't assume the actual ship date will be before any of the pessimistic estimates; it probably won’t be.
• Don't push back if folks feel like they can’t even give these guesses - if these aren’t easy to make then the project needs better definition and probably a reduction in scope.

Save your hatemail

Like pretty much everything in life, this works better for some teams than others and your mileage may vary. If this resonates with you and your team and you think you can get a lot of value from the steps laid out above without too much downside from not having estimates (even bad ones) then try it out! If not, figure out what problems your team really has and come up with a coherent solution for your situation instead.