Overconfidence: Why You Suck At Making Development Time Estimates

Dan Milstein from Hut 8 Labs has written a lengthy post about why software developers often struggle to estimate the time required to implement their projects. Drawing on lessons from a book called Thinking Fast and Slow by Dan Kahneman, he explains how overconfidence frequently leads to underestimations of a project's complexity. Unfortunately, the nature of overconfidence makes it tough to compensate. Quoting: "Specifically, in many, many situations, the following three things hold true: 1- 'Expert' predictions about some future event are so completely unreliable as to be basically meaningless 2- Nonetheless, the experts in question are extremely confident about the accuracy of their predictions. 3- And, best of all: absolutely nothing seems to be able to diminish the confidence that experts feel. The last one is truly remarkable: even if experts try to honestly face evidence of their own past failures, even if they deeply understand this flaw in human cognition they will still feel a deep sense of confidence in the accuracy of their predictions. As Kahneman explains it, after telling an amazing story about his own failing on this front: 'The confidence you will experience in your future judgments will not be diminished by what you just read, even if you believe every word.'"

Comments

I sucked because I was pressureed to being sucky

[Anonymous Coward]

'nuff said.

We've all been under pressure to give our "best" estimates and then some.

Give a realistic estimate? Off to India!

I guess I'm not an expert then....

[mark-t]

Points 2 and 3 don't seem to apply to me. I know I suck at doing development estimates. When asked for one, I've never been particularly confident about any estimate I give having a good chance of being accurate. I want to estimate conservatively, but project schedules don't allow for that.

Re:I guess I'm not an expert then....

[nine-times]

I wish people would understand that project schedules should *only* be considered guesses and estimates. Take the time you think it will take, and then take a step back and ask yourself, "No really, how long will it take?" When you get a number, take another step back and ask yourself, "No, *really*, when a bunch of things go wrong and it takes longer than I expect, how long will it take?" And then treat that time frame as a best-case scenario.

Part of the problem is that many projects can not be set to a specific schedule. The real answer is usually "it depends". How long will it take to build a new website? Well it depends on what unexpected hurdles we run into. It depends on how many features you want to add after we begin. It depends on how many revisions we go through.

When people ask me to set a firm deadline, I'm always tempted to ask them, vaguely, "When we don't meet that deadline, what do you want me to sacrifice?" Any deadline can be met if you sacrifice enough of the project requirements. So if we're coming up on a deadline, would you rather I miss the deadline or that I sacrifice some of the requirements? That is, let's say you want a website running with features X, Y, and Z, and we have a deadline of June 1st. The question isn't whether I can meet the deadline of June 1st. The question is, on May 31st, when feature Z isn't ready (there will be some feature set "Z" that isn't ready), do you want to go ahead and launch the site anyway? Or is Z worth holding up the launch?

In other words: project managers should should focus on priorities rather than schedule. "Being on schedule" and "being within budget" are just two more features that need to be prioritized within the set of features that a project is trying to meet.

Re:I guess I'm not an expert then....

[Sponge Bath]

I know I suck at doing development estimates.

A struggle is getting people to even agree on what a development estimate is:

Me: "That will take 2 months of development work."
[two months of interruptions, putting out fires and "prioritization" later]
Other: "Why is this not done? You suck at development estimates."

Re:I guess I'm not an expert then....

[mark-t]

The first thing you did wrong is that you estimated 2 months, without taking any time to break down how you were going to spend each and every day of that two months. If you had done that, you would have realized you were falling behind schedule within the first week.

In my experience, any estimate that's longer than 1 day, and often even as little as half a day, generally should require breaking down, so that it is clear exactly what needs to be complete. You break the programming tasks down almost to an atomic level, so that every discrete function of the software is described, along with how long it will take to implement each one. Sometimes you don't know how long something will really take, but that's okay... the time it takes to estimate a project should be factored into the time it will take to complete it. Breaking things down at this level also gives you a clearer idea of the technical requirements to complete the job in the first place, which helps you design technical solutions as you make headway in the project. Further, it gives you a metric once you are partway through a project to determine based on how much of the project you've actually completed within a given time, whether you are even going to complete the project within budget, and if not, institute measures to minimize losses. In practice, you're not going to be right every time, or even necessarily close to being right, but when broken down to this level, the overestimates and underestimates should balance out reasonably well, with perhaps a tolerance of up to about 10 or 20%. If they don't, then there's something else fundamentally wrong with the project, and as a first guess, I'd suggest that it may be on account of unclear program requirements,

Re:I guess I'm not an expert then....

[Blue23]

I know I suck at doing development estimates.

A struggle is getting people to even agree on what a development estimate is:

Me: "That will take 2 months of development work."

[two months of interruptions, putting out fires and "prioritization" later]

Other: "Why is this not done? You suck at development estimates."

Then make sure you're not surprising them at the end of 2 months. If at the end of week 1 you go to them with "I go two days against the project this calendar week, we still have 38 more to go", they are in the groove for project time and calendar time isn't the same. And if they want them to be, they need to stop you from getting interrupted.

Communication. Verrrrrrry important.

Is that really the problem?

[Anonymous Coward]

I often find another problem is management's refusal to accept the estimate of the developer. I am usually pretty good at estimation. Here is what usually transpires for me:

Manager: "How long will it take you?"
Developer: "2 months."
Manager: "You don't have 2 months. You only have 1 month. Redo your estimate. How long now?"
Developer: "2 months."
Manager: "You don't have 2 months. You only have 1 month. Redo your estimate. How long now?"

At this point I feel like saying:
Developer: "Why are you asking for my input? Just write down 1 month. And do you want me to tell you I will be 1 month late right now or in 1 month from now?"

Re:Is that really the problem?

[AK Marc]

I've run into that. "If you want the lies told to the customers to be consistent, you must identify which lies you've already told them." Management didn't like my snark, but I received a briefing before future customer meetings.

Re:Scotty Principal...

[Takatata]

Than a competitor will say it will take two hours and get the job. Ok, finally it will take four hours, but still, he got the job.

Actual article summary

[Anonymous Coward]

Blah, blah, blah. Bad estimates.

Blah, blah, blah. Oh noes! Waterfall!

Blah, blah, blah. Fixed by Agile!

So true

[Dixie_Flatline]

I hate making estimates. I'm always, ALWAYS wrong. I always know I'm GOING to be wrong.

I've been trying to fix this for 12 years. I thought it was just inexperience talking, but I'm a grown-up programmer now. 'Senior', by some estimates. And yet I still have a hard time estimating the time of getting things up and running. I write one thing, and four things that I couldn't have anticipated crop up. This is particularly true in my industry (video games) where you're often working with an engine that's a few years old, and other people are in the middle of working on it, and specs are changing under everyone all the time. Things that look straightforward end up taking bad detours through networking components that nobody else understands because that part was written years ago and those programmers aren't around anymore.

Man, this story makes me feel a lot better about myself.

Hofstadter's Law

[cant_get_a_good_nick]

Hofstadter's Law:

It always takes longer than you expect, even when you take into account Hofstadter's Law.

My reasons why development estimates are hard

[ADRA]

People estimate based on how much time they think it -should- take, but you almost never estimate:
1. External factors which grow time
2. Feature/function clarification takes time
3. Outside resource turnaround takes time
4. QA may never be satisfied
5. We're moral and WE make a lot of mistakes along the way
6. Most likely, you don't know all the caveats of developing the piece of work until AFTER the development is over
7. General personal time spent elsewhere (meetings, consulting with co-workers)

Sadly, the best estimate for completion ends up being 1.5-2x longer than my original gut check, so as long as you pad out your estimates, you should be fine.

Uh no...

[Charliemopps]

My estimates suck because:

Project leader: Ok, so we need to know how long it will take for you to do X
Me: I'm not sure, that's an entirely new API, proprietary to the vendor, there's almost no documentation and their website has a support forum filled with questions and basically no replys to any of them.
Project leader: Well, we need a number.
Me: Why?
Project leader: I have to fill in this box here... see?
Me: Ok fine, 800 hrs
Project leader: Now hold on a minute, this wont take 800hrs
Me: It could, I have no idea. It's already taken the majority of at least one hour and I don't even know what language it's in.
Project leader: Fine, I'll put down 800hrs, but you're the one that's going to look silly.

POST PROJECT REVIEW
Project leader: I can see here your original estimate was 800hrs, and your actual billed time was 1265hrs. What causes led to you missing your estimate, and how can we avoid those in the future.
Me: Don't make estimates.
Project leader: Come on now, I need a real answer.
Me: Why?
Project leader: I have to fill in this box here... see?
Me: ....

Re:Too many factors.

[plover]

Actually, most of those things can be predicted. What is harder to predict is the creative aspect of development. Predicting a civil engineering project, like a bridge, is easy because engineers can compute the number of beams, the volume of concrete, the depths of the footings, etc, and they already have a good idea how construction people will behave, so they can add 5% for vacations, 20% for staffing difficulties, 10% for late trucks, etc. Predicting the creation of a new piece of software is less certain, because so many of those pieces are unknown. You make an initial survey of the requirements, and take an educated guess at what the solution might be, but you know that's never the final picture of the real solution.

If you're on a mature Agile team, you trot out your velocity, map your epics to some t-shirt sizes, and do some simplistic multiplication. But you also know to announce the estimates in terms of your team's delivery dates, and you don't overpromise. At this point, either management trusts your team's reputation and you boldly go forward, or you give them estimates with confidence levels around 20%, because you simply can't be more accurate than that.

If you're on a waterfall team, any software development estimate with an accuracy figure of higher than about 50% should be viewed with suspicion, and anyone claiming 90% or higher should be flagged as a true bullshit artist.

Re:Experiment

[TheNastyInThePasty]

You wind up with Valve. Go look for their leaked employee manual. It's quite interesting.

Re:I sucked because I was pressureed to being suck

[sycodon]

Manager: "We need an application that does X,Y, and Z. When can you have it done?"
Developer: "Well, can you tell me more?"
Manager: "No, time I have a manager's meeting in 5 minutes. Just give a pall park."
Developer:" Ok, umm 3 weeks."
Manager: "THAT LONG?"
Developer: "OK, 2 weeks? Maybe less?"
Manager: "OK"

Later, in the manager's meeting.

Manager:"My developer says he can get it done in less than 5 days."