Slashdot Mirror
Overconfidence: Why You Suck At Making Development Time Estimates
Dan Milstein from Hut 8 Labs has written a lengthy post about why software developers often struggle to estimate the time required to implement their projects. Drawing on lessons from a book called Thinking Fast and Slow by Dan Kahneman, he explains how overconfidence frequently leads to underestimations of a project's complexity. Unfortunately, the nature of overconfidence makes it tough to compensate. Quoting:
"Specifically, in many, many situations, the following three things hold true: 1- 'Expert' predictions about some future event are so completely unreliable as to be basically meaningless 2- Nonetheless, the experts in question are extremely confident about the accuracy of their predictions. 3- And, best of all: absolutely nothing seems to be able to diminish the confidence that experts feel. The last one is truly remarkable: even if experts try to honestly face evidence of their own past failures, even if they deeply understand this flaw in human cognition they will still feel a deep sense of confidence in the accuracy of their predictions. As Kahneman explains it, after telling an amazing story about his own failing on this front: 'The confidence you will experience in your future judgments will not be diminished by what you just read, even if you believe every word.'"
I was once invited to a meeting with the customer because my manager was sick. When people started talking schedule I casually mentioned the 18 months it would take to complete the software. The customer went ballistic. Apparently the schedule I gave my manager never made it to the customer.
I was never invited to a meeting with the customer again.
The Moore-Murphy Law: The number of things that will go wrong will double every 2 years.
I guess part of being an 'expert' is being dumb enough to buy your own crap. That's why they always seem so sure of everything. Meanwhile, folks like you and me hedge our bets, and people attribute that to not knowing enough, rather than knowing all too well what the real deal is.
I suspect that prior to being an 'expert', that person makes one wild guess that they nail bang on. After that, they just point back to the ONE TIME they were right, and that carries them for the next few years.
I've run into that. "If you want the lies told to the customers to be consistent, you must identify which lies you've already told them." Management didn't like my snark, but I received a briefing before future customer meetings.
Heh. I actually worked out a system with one of the sales guys I worked with. When he rubbed his eyebrow in a certain way it meant "I know I'm lying; please don't say anything that might undermine my lie."
In his case I was okay going along with it because he always had some (generally quite reasonable) backup plan that meant my team would never have to actually deliver on his lie. I was still uncomfortable, but he never burned us, and the customer always ended up happy.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
I would love to see an experiment. Take two groups and give them the same job. Group one would be based on a typical American corporate structure with a Boss, Scheduler, budget person, middle management, supervisor, and finally people doing the work.
The other group would have the same number of people but only those that work. No schedule or budget just work until it's done. I wonder what the results would be?
I love Jesus, except for his foreign policy.
If you ask any experienced software developer about estimating when the project will be finally completed you will get a blank stare --- for the simple reason that there are always higher mountain to climb, more features to add, more bugs to be squashed, more optimizations to be made, and so on ...
I do not do time estimation --- I do the reverse
I set out a limit on time before I even begin a project
Within that time span I partitioned it into "exploration", "research", "coding", "debugging", "finishing touch" --- and I can terminate the entire project when any part of the partition takes too long, or produce too few result, or both
That's the way I've been using since the late 1970's --- it might not be the best way, but that's my way of accomplishing my projects --- or abandon it before it dragged out way too long
Muchas Gracias, Señor Edward Snowden !