Slashdot Mirror
The Programmers Who Want To Get Rid of Software Estimates
An anonymous reader writes: This article has a look inside the #NoEstimates movement, which wants to rid the software world of time estimates for projects. Programmers argue that estimates are wrong too often and a waste of time. Other stakeholders believe they need those estimates to plan and to keep programmers accountable. Is there a middle ground? Quoting: "Software project estimates are too often wrong, and the more time we throw at making them, the more we steal from the real work of building software. Also: Managers have a habit of treating developers' back-of-the-envelope estimates as contractual deadlines, then freaking out when they're missed. And wait, there's more: Developers, terrified by that prospect, put more and more energy into obsessive trips down estimation rabbit-holes. Estimation becomes a form of "yak-shaving" — a ritual enacted to put off actual work."
Good managers get my best guess.
Bad managers get my worst case.
Horrible managers get my resignation.
Tag: WORKS4ME
Live today, because you never know what tomorrow brings
so, estimating time is a waste of time, but complaining about estimating time is not?
GET BACK TO WORK, MONKEY.
Business can't plan or talk to customers or have any strategy whatsoever without at least some estimate...that's just the real world. If devs don't give estimates, managers have to make estimates. If managers don't make estimates, business makes estimates. You want devs to do the estimating.
Estimating is a crucial skill for developers. If you don't have it, you'll make bad decisions. For example, answer the question, "should I use framework A, or should I write some code myself?" If you can't estimate how long it will take to use the framework and compare it to how long it will take to write the code yourself, then it is impossible to make a realistic decision. Similarly, "should we write the code using method A, or write the code using method B?" If you don't know how long they take then you can't make good decisions.
This is a trick I learned from agile (although I'm sure it's been around longer than that). Each time you make an estimate, pay attention to how accurate it was, then next time adjust and make a better estimate based on that. Estimating shouldn't take much of your time.....if it does, then you're doing it wrong.
These guys seem to be complaining that their managers aren't doing anything with the estimates. Which probably means they don't understand what the managers do (but it could also mean their managers are bad).
"First they came for the slanderers and i said nothing."
I've worked on a lot of software projects that delivered the original specified product on time. Sometimes the target changes, and the stakeholders need to be willing to give developers the extra time they request to meet the new objectives. Too often I hear, usually from upper managers, "We are still shipping on schedule. Tell the developers to work harder." Of course, that's not realistic, and the result is a predictable "failure to deliver." Alas, the developers get blamed, when it's really management's fault.
On the flip side we have so-called "agile" development teams who simply define a deliverable as whatever they've completed on delivery day. These developers rarely tell management until the 11th hour that what they're going to deliver is below spec. Agile development has its strengths, but this aspect is a giant weakness. The solution is not to eliminate schedules. It's to adapt them to changing conditions and be proactive about slippages.
People do not predict well and it is not smart to use it against them.
You can improve. Most programming we do is boring, and similar to something we've done before. For things like that, you should be able to make your estimates reasonably accurate.
Sometimes projects have large unknowns, and it's impossible to know how long it will take. In that case you need to propagate that information up to the people who need it, either managers or salespeople or whatever.
"First they came for the slanderers and i said nothing."
...Software project estimates are too often wrong, and the more time we throw at making them, the more we steal from the real work of building software...
...developers' back-of-the-envelope estimates...
From the above two quotes, it looks as if the author does not even know if too much or too little time is spent on estimates.
.
No wonder his managers are frustrating with the time estimates provided. The guy has not a clue.
I've seen that sort of thing used fairly effectively. If you keep track of how long past projects took, it's fairly easy to estimate a new project based on an estimated ratio of how complex the new project is compared to the most similar past project(s). This takes practice and a bit of a knack to do well, but it doesn't take much time, and it's certainly better than nothing.
What doesn't seem to work well, in my experience, is breaking down a project into microscopic detail and individually estimating each detail, e.g, the "Microsoft Project" approach. I can understand the appeal of that sort of thing, but it's almost impossible to use a data-driven approach at the microscopic detail unless you use some sort of "Personal Software Process" tracking tool - which very few people want to actually do because it feels like you're instrumenting yourself as a lab rat in someone's twisted experiment.
In any event, having a plan and a schedule, though inevitably imperfect ones, helps motivate everybody and helps keep things on track. The more enlightened managers of the world will allow these things to be revised along the way as needed, as their imperfections get revealed.
On the one hand, it's pretty much impossible to do any project planning with no estimates.
On the other hand, some things are impossible to estimate until you do them.
Years ago I worked with a manager who kept repeating that bad estimates were a project risk and we should give good estimates. We kept telling him that an estimate is, by definition, based on incomplete knowledge and before you have done the work and that if he had a time machine we could give him better estimates.
If I knew exactly how long it would take, and what unforseen things I'd be running into ... it wouldn't be a frickin' estimate, now would it?
People treat estimates like you're expected to have perfect knowledge of the future, and then build their world around those estimates.
I have seen a tremendous amount of bullshit and stupidity from people who do not understand what an estimate is, and how it's done.
I don't think you can get rid of estimates entirely ... but management needs to stop being so stupid about how they interpret them.
If we could tell you for a fact exactly how long it would take, it wouldn't be a fucking estimate.
I rank how people do estimates right up there with how some PMs want you to track your time ... once had a PM say he wanted me to account for my time in 5 minute increments. And I told him in no uncertain terms that would mean 2 out of every 5 minutes would be spent documenting what I'd done the last two minutes, and there would be an additional 1 minute of lost time in each 5 minute increment doing to context switch back to what I was working, and that effectively 60% of my time would be wasted on his stupidity.
And then I told him to piss off.
Lost at C:>. Found at C.
I work for Joel Spolsky and we have a single estimate: 6-8 weeks.
Estimation is basically useless because it has the unspoken assumption of "perfect design", which is an invalid one: software development is not a repeated exercise, it does not have consistency.
Having a single estimate does this to your projects: much smaller things, just do them; much bigger things, they are too big so don't do them, things vaguely in that scale, do them iteratively but define what you are trying to achieve in writing so you know when to stop.
No project managers, product managers work at a way higher scale (decide which are the projects worth working on).
My Stack Overflow user
I used to tilt at this very windmill myself. It took me a long time to realize that people really, really need to hear an answer to "how long?". When we were in startup mode a long project was a matter of a few weeks. Everything had to go into production immediately. So we got used to banging stuff out in small chunks and doing it as quickly as possible. Entire project timelines were completed in less time than it takes to draft a proper requirements document.
But as we grew in size, the new people were not happy with our development team. Even though they would ask for a new report at 10am and have it in production before they returned from lunch, they still felt like we were not responsive. It was one of those reports that began to help me understand the psychology.
There was a problem with a very complex Crystal Reports document one morning. The Director of the department called to let us know about it. I told him we were already working on it and would have it fixed as soon as possible. "When will it be fixed?" he asked. Well, I had no clue. We had only learned about the problem 10 minutes before and hadn't figured out what was broken. So I explained this to him and told him that we had this as our top priority and it would be fixed as quickly as possible. I certainly thought that should make him happy. Well, it didn't.
He was rather pissed that I wouldn't give him a timeline. The day before we had made some changes for him that took about 2 hours, but he was upset that we didn't let him know how long it would take. Being an engineering type, when I hear "how long will this take?", I hear a request for a certain degree of precision. The problem with short projects like this is that by the time you have enough information to give an honest estimate, you are pretty much done. Maybe you have 15 minutes or an hour to go, but nothing worth reporting to anyone. Well, after explaining my position a few times and just making him more angry I finally gave up and just gave him a made-up deadline of Thursday afternoon. He was perfectly happy. I had just spent 15 minutes trying to explain to him that we were working feverishly and would be done as soon as we could (which would likely be a couple of hours, but who knows) and he hated that answer. "We'll have it for you in 3 days!" made him very happy. Even though his production was impeded by the lack of this particular report. From a human psychology standpoint he would rather know that it will be done in 3 days, barring delays, than not know when it will be done and have it in two hours. I personally think that is a dumb way of doing things, but I am the outlier, not the director.
After that learning experience we began implementing a more comprehensive SDLC process and providing timelines for projects. Everyone was much happier with the development team after this. Even though their projects went into production much more slowly. They loved the perceived control that having a timeline gave them. We developers know that these things are basically fictional documents - just educated guesses really - but it provides real customer satisfaction, so we keep with it. In fact, we kept evolving this idea into more and more involvement from the business unit as we moved into Agile and SCRUM methodologies.
I would say that unless you are working in an organization of less than 25 people, providing timelines is an absolute requirement from a purely human standpoint. This comes from hard experience - even though I think that everything about a timeline is probably B.S. and all of the effort that goes into preparing one would have been better spent solving problems and building something useful. In the real world you can't ignore the psychological needs of the group.
No, it's a very common problem in engineering in general, and not unique to software. But the reaction "let's eliminate estimates" appears to be.
As an engineering manager, I learned the hard way many times how estimates turn into deadlines. Your estimate is reported to the manager's manager and so on up the line, and someone uses it in planning their shit.
Your estimate, in which you did not build any schedule margin, then becomes an item in the critical path of someone else's plan, someone who didn't build in any margin either, or —worse— who was pressured to make a completely fictional "plan" which is really just a backwards-calculated paper justification to "prove" that a job could be completed in an impossibly short period of time by assuming nine women can make a baby in one month and things like shipping, reproduction, and quality assurance take place in zero time. This "plan" makes upper management happy. Temporarily.
You, leader of a small team that is working merrily away, accomplishing real work and solving the occasional unexpected problem (OEM pinouts were wrong, widget zeta delayed in shipping, amplifier stage behaving like oscillator, etc.), are asked for a status update. Because of your unexpected problems, your estimated completion date is now two weeks later than your previous estimate.
Now the middle manager, who knew he wasn't going to meet the "plan" he was forced to develop, now has someone to place the blame on. He knows he's going to be in the path of a metric fuckton of shit, but he's placed himself uphill of you.
It's clear even in TFS that the real problem isn't estimation, it's poor program management, lack of requirements management, and often also marketing-driven decision-making.
In other words, the same old shit.
I can see the fnords!
Peter Molyneux is that you?
Lets see... What would they say? This is the one-sided conversation, since it doesn't matter what you say anyways.
"Ok, we can accept that estimate."
"Ya, ya, ya, whatever."
"We'll have that information to you by the start of the project."
"The information isn't ready yet, we'll have that by the time you need it."
"I thought we had that to you already. We'll have to check with the information source."
"The PMs have some changes."
"Here's the information, but there are some small changes."
"No, those are small changes, they won't impact the timeline."
"No, you can't have more time, we already made commitments."
"The PMs have some changes."
"What do you mean you won't have it in on schedule? You agreed with the initial estimate."
"You're going to stay here until it's done, I don't care how long it takes."
"I don't care that you've been in the office 30 hours straight, this is your fault."
"We're hiring an off-shore company to help you with the project. Get them up to speed."
"The PMs have some changes."
"Since we have the off-shore team, we need to cut your department back."
"I read an article saying Java is the future. Redo it in Java."
"What do you mean we're waiting on the off-shore company?"
"We fired the off-shore company. You're good, you can get it done in time."
"Ok, hire more people into your department, but we're only offering half the salary, and no more bodies."
"Why is this project so far behind? Don't you know what you're doing?"
"The PMs have these changes."
"Why aren't you done? We're weeks from the deadline!"
"You didn't meet the deadline. Don't you know deadlines are firm. We have commitments."
"I don't want excuses, I want results."
"You and your idiot team are fired. Get out of my building."
[2 months later]
"We need you to come back and finish the project. We need it by next Monday, that should be plenty of time."
"Here's all the new specs. They should be easy to do."
"What do you mean total rewrite, it's only a few chances. You are an idiot. Get out."
[1 month later]
"We need you to come back and finish the project. We need it by" {click}
"We need you to come back and finish the project. We need it by" {click}
"We need you to come back and finish the project. We need it by" {click}
"Why do you keep hanging up on me?" {click}
Serious? Seriousness is well above my pay grade.