Slashdot Mirror
The #NoEstimates Debate: An Unbiased Look At Origins, Arguments, and Leaders
New submitter MikeTechDude writes: Estimates have always been an integral part of the software development process. In recent years, however, developers, including Woody Zuill and Vasco Duarte, have begun to question the efficacy, and even the purpose, of using estimates to predict a project's cost and time line. A fierce debate has sprung up on Twitter, between those calling for an end to estimates and those who continue to champion their use in a professional setting. On the surface, it would appear that the debate is black and white. Proponents of the #NoEstimates Twitter hashtag are promoting a hard stop to all estimates industry-wide, and critics of the movement are insisting on a conservative approach that leaves little room for innovation. However, the reality of the debate has unfolded in far more complex, nuanced shades of gray. HP's Malcolm Isaacs digs deep and pinpoints where the debate started, where it now stands, and what its implications are for the future of software development. Meanwhile, Martin Heller offers his less unbiased approach with his post, #NoEstimates? Not so fast.
Correct:
Incorrect:
One of these is merely difficult. The other is doomed to failure from the time The Boss opened his mouth.
No, hashtags are not movements. If you think so, you either have no clue what a movement is or no clue what a hashtag is, and I'm guessing it's the former. Talking about things on Twitter is talking and that's it. It's not some kind of political action.
Book writing duration is random. This is known. Why should book authors be special in their inability to estimate task times using well known probability distributions? They've been writing books for thousands of years.
Finish your draft late? Publisher won't pay you.
Better yet, authors should be properly licensed.
I agree that having enough time taken to make a good product is a good thing.
However, there does need to be some accountability to business goals and the ability to actually get features to customers in a reasonable time frame.
Prioritization is important. There are times where something has to get done by a certain date, or you'd have been better off working on something else because you lost the customer who wanted the feature to the company who did produce it.
In some ways, I blame customer expectations. If they're always hunting to get the most features they can, for the cheapest possible price, they're pretty much accepting the bugs and design flaws that come from software houses that are fulfilling their criteria. And that's basically what happens. If customers did something like insist that bugs in the software would require significant financial penalties or credits as part of their contract, they'd get a better product, but they'd also pay through the nose for it, and probably also have to wait a substantial amount of time for it.
So, I think we're at an equilibrium. Customers hate shitty software, but as long as they pay for it, that's what they're going to get because they don't choose the quality software over the quick and cheap.
And until that changes, businesses need to be able to predict when their software is going to be more or less complete.
We have this already; it's called agile. Forget about the scrum and the planning meetings, they're get emphasized by shitty teams and PHBs because they're something people are used to, namely meetings, and they're easy to do. They're not required and arguably the least important aspects of agile.
The point of agile has always been:
1) Figure out what the most important feature you need to do next.
2) Build it and make sure it's rock solid.
3) Keep repeating #1 and #2 until the client is satisfied.
You add on top of that the minimum number of things (which can include standups and planning meetings and even, (gasp) documentation, if it's helpful to the implementation team, not the PHBs) to make the process work. A key point is there are no hard ship dates; you keep building until you're done.
is giving estimates without a detailed design.
Imagine this interaction:
Customer: I want you to build me a house.
Contractor: Ok, How many square feet?
Customer: I don't know. When can you start?
Contractor: We can't start until we have plans drawn up.
Customer: I don't have time for that. How much will it cost?
Contractor: I can give you a rough idea once we've nailed down the square footage, number of stories, type of foundation, and some other details.
Customer: You are wasting my time with all these questions.
Contractor: Go Away.
Yet software developers agree to this situation, or are forced to agree to it, all the time.
more cowbell
Estimates should be used to prioritize features (cost / benefit) as opposed to being used to set hard deadlines.
Estimates should be one of "hours", "days", "weeks", or "months". It is fairly easy for most people to differentiate between features that take hours to implement vs weeks. In my experience, exact durations with multiples for padding have proven to be less useful / accurate than the former method.
Finish your draft late? Publisher won't pay you.
That's more common than you think. Especially if you're not already an established name, contracts usually have terms stating that if you don't meet the deadline, the publisher has the right to cancel the contract, and demand return of the advance (if any). Whether they actually exercise this right or not varies.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
I'll admit that I'm not a developer, so I may not have a firm grasp on the relevance of everything being talked about. However, I've managed my share of projects, and I definitely think there's reason to doubt the value of estimates for all kinds of projects. Software development projects are not unique in that regard.
Now, I want to start by pointing out the obvious that you often have to make some kind of estimate. Even if the estimate isn't very precise, you have to make some kind of guess-- is this going to take 1 day, 6 months, or 5 years? Being practical, people have to have some idea of what they're getting into, or they can't make decisions.
On the other hand, estimate can only be accurate insofar as the scope of the project is well defined and well understood. For tasks that you do frequently and know exactly what needs to happen, accurate estimates are not very difficult. Even if you are bad at estimating, you can measure how much time and money is spent on those repetitive tasks, and then use that data to figure out how much time and money it typically takes. However, as the scope of work is less well defined, or the nature of the work is less well known, the accuracy of the estimate will be worse.
Imagine building a house. If you're building 100 houses in a development, and you do that work often, and you've already build 30 houses and know how much the labor and materials cost, then you can probably make a good guess of how much time and money it will take to build the remaining 70. However, if someone asks you to "fix all the problems with their house," and you don't know what shape their house is in, what it means to "fix" it, what the laws are regarding construction in the area, or what the local materials/labor cost, then your estimate won't be worth much.
And this brings me back to the issue of "precision" rather than "accuracy". I think part of the issue is to provide an appropriate expectation of precision when providing estimates. I frequently have to provide estimates, and some of them are wildly wrong, but when I'm not confident in the estimate I'm providing, I'll also provide some kind of disclaimer. I admit that I don't know all the details about the situation I'm getting into, and that my estimate could be off. The thing that I'm saying will take 6 weeks might take 2 months. Maybe 2.5 months. Maybe more. Not 6 months-- at least not unless there's something really unexpected or some kind of mission-creep.
But this is really part of a larger problem: the ineffectual nature of "plans". There's a famous quote, something along the lines of, "no battle plan survives the first contact with the enemy". Again, there are projects where the scope is defined and the work to be done is well understood, and in those cases, you can do conventional project planning. You can set milestone dates and make gantt charts, and develop a whole timeline and budget. But on the other hand, Donald Rumsfeld was on to something when he talked about "unknown unknowns". Often when you are embarking on a project, you're not even aware of the obstacles you're going to face, and so you can't account for them. This doesn't mean there's no point in developing a plan or a schedule. It means that your schedule has to be flexible in proportion to the likelihood of "unknown unknowns" and other contingencies outside of your control.
I think if you want to improve the situation, the answer isn't to stop making estimates or project plans, but to develop better methods for quickly estimating the precision of your estimates, providing a margin of error, or the level of flexibility needed in a project. However, I think the #NoEstimates people are correct to point out that there is often diminishing returns on trying to set deadlines and budgets. It doesn't make sense to spend a week refining your plans for a two-week project.
eh,
:)
Agile Manifesto - February 11-13, 2001
Extreme Programming Explained - published October 1999
That's good enough to say "it came first"
As long as you understand, I'm not going to argue about the definition of "first."
And yes, I learned this tip about estimation from agile:
When you make an estimate, afterwards check to see how accurate the estimate was, and use it to improve the next time.
Using this method, over time I've gotten fairly good at not missing estimates. The only times I miss are when there are a lot of unknowns (and even then, I can usually give a reasonable outer-bound).
That seems like an obvious tip, but it's amazing how many people don't use it.
"First they came for the slanderers and i said nothing."
Incidentally, This Manifesto against the agile manifesto entertains me.
"First they came for the slanderers and i said nothing."
There are *several* formulas to do an estimate and several more to tell you if your estimates are on track.
If you understand why estimates are required, you are a business person, if you understand why they are so difficult you are a developer. Managing estimates is a 'Project Management' task and a good PM will keep the pressure of the team by also managing the stakeholder expectations, which is what we are really talking about here.
Complex estimates are closer to the contract and simple task estimates are closer to the metal. If anyone asks for an 'accurate estimate', run - they are an oxymoron who won't de-scope so that deliverables are met. To me it is an immediate sign of project failure.
Estimates are just a tool that are a balancing act for getting the budget required to do something. Good estimates are achievable by iterating three simple questions pessimistic, realistic and optimistic estimation for a smaller task of a large project. After that there are several other formula to determine if you are ahead, behind or on schedule. Ahead or on schedule - great, behind - de-scope. What the final product looks like is a function of the contract that determines the critical path and managing the expectations to get there. Estimations on a small project however are usually a waste of time.
The last thing you want to do is go back to an accounting department or client for more budget because the estimates are way off anymore than having no estimate at all and asking for a big bucket of money that won't get approved and no developers will ever get employed to do that project.
Using 120 characters to discuss such a complex subject, that can't possibly hope to encapsulate the arguments required to understand it, is pointless.
My ism, it's full of beliefs.
If someone makes an estimate for you, or forces you to make a lower estimate, then you have no obligation to meet that estimate.
Yes, you do. It's called keeping your job.
If your job depends on you meeting impossible estimates, then you're not going to be in it very long, no matter how hard you try.
"First they came for the slanderers and i said nothing."
There are many products which have a very limited life span as well. Products who serve a purpose six months from now, are retired in two years time, and never touched again. Products where failing to deliver on time is as bad as failing to deliver at all.
Then there are the projects and products which are only part of a greater whole, where delivering late means holding up that entire larger project, and which has a financial impact which may well exceed the total budget of your entire project.
Despite that, I've never encountered an estimate that was any more than the gut feel of someone giving their best guess based on their experience and what they know so far about the requirements. Customers really need to wake up to the fact that changing requirements and vague/unfinished requirements mean that the actual delivery could be +/- 50% of the estimate, or even more.
The worst overruns I've ever seen were always on projects where the tools to be used were selected before the developers were ever consulted about what made sense to use for developing the project. The inane buzzword projects where someone decides they're going to use NoSQL, or SQL, or flat files, or whatever storage system because that's "company policy" or because some "architect" was in love with that particular technology or product.
Woe betide anyone who lets the buzzword mafia decide the course of their project, for they are doomed to expensive failure in the vast majority of cases.
I do not fail; I succeed at finding out what does not work.
Estimates are useless as a measure of how well an engineer is performing. How far he is ahead or behind schedule only indicates the extent to which he was able to get away with padding his estimate in the first place.
That said, estimates ARE very valuable when you have a complex set of interlocking projects and resources that can be tasked in different places. This is especially true if external pressure require that a project be done on an exact date.
To take an extreme example, if the launch window for Europa is at a known date, the spacecraft firmware must be fully tested and installed by that date. Working backwards that says when the first version must be ready. The estimate helps decide what resources should be applied, and later it lets you know if you are so far behind that you need to change the launch date to the next window (over a year away). That affects budget etc.
At SLAC we have complex projects that require the work of lots of people to all come together. This results in very rigid schedules - There is typically a 2 month window for major upgrades, if you miss it, you wait a year. If someone working for me doesn't like doing estimates, I basically say "we need a guess. I can guess or you can, but since you are doing the work, your guess will be better than mine".
I have worked in the industry for 20+ years and here are my observations - granted, I have worked in smaller companies (i.e 25-4000) and startups.
Estimates work as a means to determine the work effort for a given set of features. They are not to be used for setting schedules and deadlines by themselves. They are to be used for budgeting and cost planning. And, they are not to be done without a detailed design meeting the agreed upon requirements.
Unless your client is very rich and/or stupid or you have a large surplus of venture capital in your startup, you better be concerned with the work effort and time to have your product in a usable state. When you can tell a client that a project is going to take time X and cost Y and meet those values, you gain credibility and trust. In the digital advertising world, those with credibility and trust become the agency of record (AOR). And, the client will stick with you as their AOR until you royally screw up and fail to deliver what was promised, when promised and for the agreed upon price.
You DON'T ask a developer how long something will take - they invariably will underestimate the work effort. Instead, at least until you have measured delivery rates for your team members, you use industry standards. You can ask the developer and then compare their estimates with the actual time and effort. When their estimates start matching up, you can ask them estimate their own assigned work. It can be a good learning experience for them.
Some projects don't require estimates. We had projects that fit a template model based earlier work. We knew how long it took, on average, to fill the various fields of the template. Throw in the project management, QA and deployment components and its pretty easy to do.
When people claim Agile isn't compatible with estimates, it's probably because the team isn't concerned about documentation or planning. They tell you that the system has too many variables and they don't have time or resources to keep the design up to date. I call BS. If you can't do it, then add someone to the team who can. There are great tools for doing design work and capturing requirements at all phases of a project - use them.
Even with Agile, you should layout out a basic design or framework in the early stages of the project. Then, you can determine how long other features will take and what their dependencies are before you attempt to implement them and emptying the client's wallet. Then, you base the number of sprints based on that information. Since you are supposed to have a working product at the end of each sprint, you should be able to tell the client what features will be in each deliverable and cost. If they want to change the requirements or add new features or change functionality, you still have to plan how long those features will take and in which sprint you will deliver a product that meets those changes.
Is the F-35 a real world object? How about those road tunnels in Boston?
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
As they saying goes: Fast, cheap, good - pick any two.
You can usually solve one problem by another. If you see you can't hold the deadline, you can throw more manpower at it (not cheap anymore) or compromise on quality. If you see the budget runs out, you can put the project into idle times (not fast anymore) or compromise on quality.
Sadly, quality is the part that management, customers, clients and developers understand the least. Everyone understands deadlines - either you are done on that day or you are not. Everyone understands money and to convert developer man-hours into money is not so difficult. But quality is tricky. If it runs, ship - because management, customers, etc. they see if it is running, but not what's going on under the hood. And developers too often don't understand that quality is subject to combinatorial explosion - shortcuts don't add up, they multiply.
But because it's the least-understood part of the equation, and compromises matter so much but are not easily visible as long as the core operation functions, software in generally is so absolutely shoddy.
Assorted stuff I do sometimes: Lemuria.org
Not a useful saying for budgeting.
Aphorisms are never good for hard numbers. They are good for making a point in a concise way.
Assorted stuff I do sometimes: Lemuria.org