The Programmers Who Want To Get Rid of Software Estimates

An anonymous reader writes: This article has a look inside the #NoEstimates movement, which wants to rid the software world of time estimates for projects. Programmers argue that estimates are wrong too often and a waste of time. Other stakeholders believe they need those estimates to plan and to keep programmers accountable. Is there a middle ground? Quoting: "Software project estimates are too often wrong, and the more time we throw at making them, the more we steal from the real work of building software. Also: Managers have a habit of treating developers' back-of-the-envelope estimates as contractual deadlines, then freaking out when they're missed. And wait, there's more: Developers, terrified by that prospect, put more and more energy into obsessive trips down estimation rabbit-holes. Estimation becomes a form of "yak-shaving" — a ritual enacted to put off actual work."

Comments

ignorant hypocrites

[Anonymous Coward]

so, estimating time is a waste of time, but complaining about estimating time is not?

GET BACK TO WORK, MONKEY.

Well someone has to do it

[Sowelu]

Business can't plan or talk to customers or have any strategy whatsoever without at least some estimate...that's just the real world. If devs don't give estimates, managers have to make estimates. If managers don't make estimates, business makes estimates. You want devs to do the estimating.

Good method for improving

[phantomfive]

Estimating is a crucial skill for developers. If you don't have it, you'll make bad decisions. For example, answer the question, "should I use framework A, or should I write some code myself?" If you can't estimate how long it will take to use the framework and compare it to how long it will take to write the code yourself, then it is impossible to make a realistic decision. Similarly, "should we write the code using method A, or write the code using method B?" If you don't know how long they take then you can't make good decisions.

This is a trick I learned from agile (although I'm sure it's been around longer than that). Each time you make an estimate, pay attention to how accurate it was, then next time adjust and make a better estimate based on that. Estimating shouldn't take much of your time.....if it does, then you're doing it wrong.

These guys seem to be complaining that their managers aren't doing anything with the estimates. Which probably means they don't understand what the managers do (but it could also mean their managers are bad).

Tilting at Windmills

[Cytotoxic]

I used to tilt at this very windmill myself. It took me a long time to realize that people really, really need to hear an answer to "how long?". When we were in startup mode a long project was a matter of a few weeks. Everything had to go into production immediately. So we got used to banging stuff out in small chunks and doing it as quickly as possible. Entire project timelines were completed in less time than it takes to draft a proper requirements document.

But as we grew in size, the new people were not happy with our development team. Even though they would ask for a new report at 10am and have it in production before they returned from lunch, they still felt like we were not responsive. It was one of those reports that began to help me understand the psychology.

There was a problem with a very complex Crystal Reports document one morning. The Director of the department called to let us know about it. I told him we were already working on it and would have it fixed as soon as possible. "When will it be fixed?" he asked. Well, I had no clue. We had only learned about the problem 10 minutes before and hadn't figured out what was broken. So I explained this to him and told him that we had this as our top priority and it would be fixed as quickly as possible. I certainly thought that should make him happy. Well, it didn't.

He was rather pissed that I wouldn't give him a timeline. The day before we had made some changes for him that took about 2 hours, but he was upset that we didn't let him know how long it would take. Being an engineering type, when I hear "how long will this take?", I hear a request for a certain degree of precision. The problem with short projects like this is that by the time you have enough information to give an honest estimate, you are pretty much done. Maybe you have 15 minutes or an hour to go, but nothing worth reporting to anyone. Well, after explaining my position a few times and just making him more angry I finally gave up and just gave him a made-up deadline of Thursday afternoon. He was perfectly happy. I had just spent 15 minutes trying to explain to him that we were working feverishly and would be done as soon as we could (which would likely be a couple of hours, but who knows) and he hated that answer. "We'll have it for you in 3 days!" made him very happy. Even though his production was impeded by the lack of this particular report. From a human psychology standpoint he would rather know that it will be done in 3 days, barring delays, than not know when it will be done and have it in two hours. I personally think that is a dumb way of doing things, but I am the outlier, not the director.

After that learning experience we began implementing a more comprehensive SDLC process and providing timelines for projects. Everyone was much happier with the development team after this. Even though their projects went into production much more slowly. They loved the perceived control that having a timeline gave them. We developers know that these things are basically fictional documents - just educated guesses really - but it provides real customer satisfaction, so we keep with it. In fact, we kept evolving this idea into more and more involvement from the business unit as we moved into Agile and SCRUM methodologies.

I would say that unless you are working in an organization of less than 25 people, providing timelines is an absolute requirement from a purely human standpoint. This comes from hard experience - even though I think that everything about a timeline is probably B.S. and all of the effort that goes into preparing one would have been better spent solving problems and building something useful. In the real world you can't ignore the psychological needs of the group.

Re:Is this really a problem unique to devs??

[bughunter]

No, it's a very common problem in engineering in general, and not unique to software. But the reaction "let's eliminate estimates" appears to be.

As an engineering manager, I learned the hard way many times how estimates turn into deadlines. Your estimate is reported to the manager's manager and so on up the line, and someone uses it in planning their shit.

Your estimate, in which you did not build any schedule margin, then becomes an item in the critical path of someone else's plan, someone who didn't build in any margin either, or —worse— who was pressured to make a completely fictional "plan" which is really just a backwards-calculated paper justification to "prove" that a job could be completed in an impossibly short period of time by assuming nine women can make a baby in one month and things like shipping, reproduction, and quality assurance take place in zero time. This "plan" makes upper management happy. Temporarily.

You, leader of a small team that is working merrily away, accomplishing real work and solving the occasional unexpected problem (OEM pinouts were wrong, widget zeta delayed in shipping, amplifier stage behaving like oscillator, etc.), are asked for a status update. Because of your unexpected problems, your estimated completion date is now two weeks later than your previous estimate.

Now the middle manager, who knew he wasn't going to meet the "plan" he was forced to develop, now has someone to place the blame on. He knows he's going to be in the path of a metric fuckton of shit, but he's placed himself uphill of you.

It's clear even in TFS that the real problem isn't estimation, it's poor program management, lack of requirements management, and often also marketing-driven decision-making.

In other words, the same old shit.

Re:Without estimates you can't budget...

[JWSmythe]

Lets see... What would they say? This is the one-sided conversation, since it doesn't matter what you say anyways.

"Ok, we can accept that estimate."

"Ya, ya, ya, whatever."

"We'll have that information to you by the start of the project."

"The information isn't ready yet, we'll have that by the time you need it."

"I thought we had that to you already. We'll have to check with the information source."

"The PMs have some changes."

"Here's the information, but there are some small changes."

"No, those are small changes, they won't impact the timeline."

"No, you can't have more time, we already made commitments."

"The PMs have some changes."

"What do you mean you won't have it in on schedule? You agreed with the initial estimate."

"You're going to stay here until it's done, I don't care how long it takes."

"I don't care that you've been in the office 30 hours straight, this is your fault."

"We're hiring an off-shore company to help you with the project. Get them up to speed."

"The PMs have some changes."

"Since we have the off-shore team, we need to cut your department back."

"I read an article saying Java is the future. Redo it in Java."

"What do you mean we're waiting on the off-shore company?"

"We fired the off-shore company. You're good, you can get it done in time."

"Ok, hire more people into your department, but we're only offering half the salary, and no more bodies."

"Why is this project so far behind? Don't you know what you're doing?"

"The PMs have these changes."

"Why aren't you done? We're weeks from the deadline!"

"You didn't meet the deadline. Don't you know deadlines are firm. We have commitments."

"I don't want excuses, I want results."

"You and your idiot team are fired. Get out of my building."

[2 months later]

"We need you to come back and finish the project. We need it by next Monday, that should be plenty of time."

"Here's all the new specs. They should be easy to do."

"What do you mean total rewrite, it's only a few chances. You are an idiot. Get out."

[1 month later]

"We need you to come back and finish the project. We need it by" {click}

"We need you to come back and finish the project. We need it by" {click}

"We need you to come back and finish the project. We need it by" {click}

"Why do you keep hanging up on me?" {click}

Re:Simple methodology

[BronsCon]

Depending on how far along in the project you are, it's reasonable to expect changing a single word in the spec to have a major impact on the project. Consider:

Paint my car black.

So you do all the prep work, car's primed with a dark primer, black paint is mixed and ready to go, then this change request comes in:

Paint my car white.

Well, now you've wasted the paint you just tinted black, and you can't paint white on top of dark primer (well, you can, but you need many more coats), so you've got to redo the prep work. That means waiting while the primer fully cures, so you can sand it off properly; otherwise, it'll gum your sandpaper. then re-prime, then you can paint. Assuming you don't see

Paint my car forest green.

in the meantime.

That was one word. Yes, one line matters.

Re:Simple methodology

[Maxo-Texas]

I had no particular problem with estimates.

At a minimum, you could break out easy "construction/recognized pattern" work from risky new stuff.

As far as managing programmers... it was humorous.

Few liked giving estimates. So they would say it couldn't be estimated.

So I would ask, will this project take 2 years... and they would say, "oh no- of course not" and after a bit, we'd get down to 6 months or 6 weeks or 6 hours...

So then we'd time box it to what could be done in a month and move any risky items up to the front so we could establish if a new technology wasn't going to work before we put a lot of work into the project.

Then, I recorded over/under for every project and found (over about 24 programmer data set) that programmers consistently overshot or undershot their estimates. So after a few projects, I had a pretty good idea of their deliverables.

Finally status reports and status meetings with function points and overall percentage delivered kept things on schedule or let us know well ahead of time there was a problem with the estimate/schedule.

Programmers were not my problem- executives were.

They...
a) pushed us to violate standards.
b) ordered overtime without ordering it. As in assign 80 hours work and then insist it be completed when everyone knew it couldn't be completed. Made worse by the fact the indian contractors said "I'll do my best" for "no- you are batshit crazy" and then things fell apart when the indians were unable to deliver. Of course, the indians were very good at delivering to the (crazy/incomplete) specifications on time. At least enough to be testable. I'm not sure if it is because they were contractor types or that they were indian- perhaps a bit of both. I learned in a contracting shop, you always say you can meet the estimate (to get assigned the work) instead of giving a realistic estimate. Then renegotiated it later when it wasn't going to make the schedule. If you didn't, then the three other people bidding on the work would get the work. Executives seemed to have zero memory for the fact that you delivered on time on estimate while the other people were usually late.
c) made everything priority 1a. they had no ability to prioritize as far as I could see. Which really just pushed prioritization down to me or the programmers.
d) cancelled projects without warning.