Slashdot Mirror
The #NoEstimates Debate: An Unbiased Look At Origins, Arguments, and Leaders
New submitter MikeTechDude writes: Estimates have always been an integral part of the software development process. In recent years, however, developers, including Woody Zuill and Vasco Duarte, have begun to question the efficacy, and even the purpose, of using estimates to predict a project's cost and time line. A fierce debate has sprung up on Twitter, between those calling for an end to estimates and those who continue to champion their use in a professional setting. On the surface, it would appear that the debate is black and white. Proponents of the #NoEstimates Twitter hashtag are promoting a hard stop to all estimates industry-wide, and critics of the movement are insisting on a conservative approach that leaves little room for innovation. However, the reality of the debate has unfolded in far more complex, nuanced shades of gray. HP's Malcolm Isaacs digs deep and pinpoints where the debate started, where it now stands, and what its implications are for the future of software development. Meanwhile, Martin Heller offers his less unbiased approach with his post, #NoEstimates? Not so fast.
Correct:
Incorrect:
One of these is merely difficult. The other is doomed to failure from the time The Boss opened his mouth.
Task duration is random. This is known. Project duration is random as a result. Why should software developers be special in their inability to estimate task times using well known probability distributions?
Finish early and too specifications? Get a bonus. Finish late? Get docked or don't get paid at all.
Better yet, let's push for proper licensing of software engineers. Testing, ethics, and all that.
We have known that estimates are largely wild-ass guesses (or wild ass-guesses, if you prefer) for decades.
We have known the economic cost in post release bug fixes and security holes for decades.
We have known the enormous human cost of working more than 40 hours a week and death marches for decades.
Software isn't about cranking out identical widgets on an assembly line day in and day out. It is always a creative process of designing, iterating, and learning, at least for anything bigger than "Hello, world!". It is about goddamned time that we admit that doing a good, reliable, and secure product takes as long as it takes and slamming corporations with painful penalties for the issues caused when they rush things to market.
No, hashtags are not movements. If you think so, you either have no clue what a movement is or no clue what a hashtag is, and I'm guessing it's the former. Talking about things on Twitter is talking and that's it. It's not some kind of political action.
3-5x the initial estimate. If the team is good.
My cellphone ringtone is a ring tone.
It's my initial estimate times three. Always right.
We have this already; it's called agile. Forget about the scrum and the planning meetings, they're get emphasized by shitty teams and PHBs because they're something people are used to, namely meetings, and they're easy to do. They're not required and arguably the least important aspects of agile.
The point of agile has always been:
1) Figure out what the most important feature you need to do next.
2) Build it and make sure it's rock solid.
3) Keep repeating #1 and #2 until the client is satisfied.
You add on top of that the minimum number of things (which can include standups and planning meetings and even, (gasp) documentation, if it's helpful to the implementation team, not the PHBs) to make the process work. A key point is there are no hard ship dates; you keep building until you're done.
is giving estimates without a detailed design.
Imagine this interaction:
Customer: I want you to build me a house.
Contractor: Ok, How many square feet?
Customer: I don't know. When can you start?
Contractor: We can't start until we have plans drawn up.
Customer: I don't have time for that. How much will it cost?
Contractor: I can give you a rough idea once we've nailed down the square footage, number of stories, type of foundation, and some other details.
Customer: You are wasting my time with all these questions.
Contractor: Go Away.
Yet software developers agree to this situation, or are forced to agree to it, all the time.
more cowbell
One of the major complaints about estimates is that it "takes too long." Seriously, if estimations take any appreciable amount of time, you're doing them wrong.(unless you need to estimate some massive project, in which case you understand why estimates are needed).
If you want to remove the "time sucks," get off slashdot, get off twitter. When I turn off the internet while working on difficult code, I literally get twice as much done. I've measured this several times (and you can tell from this comment how much I will get done today).
"First they came for the slanderers and i said nothing."
Estimates should be used to prioritize features (cost / benefit) as opposed to being used to set hard deadlines.
Estimates should be one of "hours", "days", "weeks", or "months". It is fairly easy for most people to differentiate between features that take hours to implement vs weeks. In my experience, exact durations with multiples for padding have proven to be less useful / accurate than the former method.
In the 70s, we got Mythical Man Month
in the 90s we got Extreme Programming, followed by Agile.
Now we get #NoEstimates. I think we can do better.
"First they came for the slanderers and i said nothing."
Customer: build this
Developer: OK, it should take X
Customer: great
Customer: why is it late?
Developer: oh, I used a bunch of new technologies and techniques. Now it does all this crap you never asked for, but I was able to pad my resume.
Customer: but it doesn't even do what I asked for
Developer: you idiot, you didn't spec it correctly.
I approach this in the following way:
I need X done. I think it involves doing A, B, C and if no yaks are shaved can be done by Real Deadline - 2 weeks. Do you agree with this plan? Can you deliver it? If no, I listen why and adjust. If yes, I will hold your feet to fire.
Is this not reasonable? If not reasonable, how can I plan and allocate resources?
When I was working on a videogame a number of years ago, I was asked by the publisher to come up with an initial schedule. We had a more or less fixed deadline, and while we knew roughly what we were going to be working on, the design phase of the game hadn't even begun. I started working on a very rough outline of the phases of the game development based on my previous experience with such projects. Naturally, it was somewhat vague, because we didn't have a design yet.
The publisher rejected the timeline proposal, complaining that it didn't enumerate specific tasks in enough detail. For a year long project, they wanted each task broken down into 2-3 day increments optimally, and no more than one-week tasks. The only thing I could do was to break each task into more techno-babble bullshit, but it really just muddied things. Still not happy with my work, they found someone else to create a nonsensical schedule with lots of detailed tasks that were pulled out of the air. That was then our fixed schedule for the game which, again, hadn't yet been designed. It was entered into a clunky custom project manager we used, and whenever a task came due, we dutifully marked it off as "checked" whatever it was - it certainly had no relation to reality. But we were right on schedule!
What we actually ended up using was a simple Excel spreadsheet with all the *real* tasks listed, and assigned to each person. The project manager kept it organized for us, and as the project evolved, we drilled down and refined the tasks from larger goals to specific items. We had a list of features from core to "nice to have", and we kept focus on what we needed to do first, and only added the superfluous features later in the project. It was just common sense to us, and not surprisingly, it worked fairly well.
I always chuckle a bit at how people/organizations keep looking for some magical "methodology", which needs a fancy name to identify it, with method-specific terms and rules. In my opinion, the more you formalize stuff like this, the more likely you are to value process over projection, and you'll just end up getting mired in that process instead of focusing on the product. I like a lot of the concepts of "agile", but I just cringe whenever I hear about "sprints", "user stories", "stand-up meetings", and so on.
And creating a hashtag-based term doesn't provide any new insight.
Irony: Agile development has too much intertia to be abandoned now.
I'm going to need an estimate for how long it's going to take you to say Mooooooooo........and can you come in and work on Saturday? Yeah.
"First they came for the slanderers and i said nothing."
I'll admit that I'm not a developer, so I may not have a firm grasp on the relevance of everything being talked about. However, I've managed my share of projects, and I definitely think there's reason to doubt the value of estimates for all kinds of projects. Software development projects are not unique in that regard.
Now, I want to start by pointing out the obvious that you often have to make some kind of estimate. Even if the estimate isn't very precise, you have to make some kind of guess-- is this going to take 1 day, 6 months, or 5 years? Being practical, people have to have some idea of what they're getting into, or they can't make decisions.
On the other hand, estimate can only be accurate insofar as the scope of the project is well defined and well understood. For tasks that you do frequently and know exactly what needs to happen, accurate estimates are not very difficult. Even if you are bad at estimating, you can measure how much time and money is spent on those repetitive tasks, and then use that data to figure out how much time and money it typically takes. However, as the scope of work is less well defined, or the nature of the work is less well known, the accuracy of the estimate will be worse.
Imagine building a house. If you're building 100 houses in a development, and you do that work often, and you've already build 30 houses and know how much the labor and materials cost, then you can probably make a good guess of how much time and money it will take to build the remaining 70. However, if someone asks you to "fix all the problems with their house," and you don't know what shape their house is in, what it means to "fix" it, what the laws are regarding construction in the area, or what the local materials/labor cost, then your estimate won't be worth much.
And this brings me back to the issue of "precision" rather than "accuracy". I think part of the issue is to provide an appropriate expectation of precision when providing estimates. I frequently have to provide estimates, and some of them are wildly wrong, but when I'm not confident in the estimate I'm providing, I'll also provide some kind of disclaimer. I admit that I don't know all the details about the situation I'm getting into, and that my estimate could be off. The thing that I'm saying will take 6 weeks might take 2 months. Maybe 2.5 months. Maybe more. Not 6 months-- at least not unless there's something really unexpected or some kind of mission-creep.
But this is really part of a larger problem: the ineffectual nature of "plans". There's a famous quote, something along the lines of, "no battle plan survives the first contact with the enemy". Again, there are projects where the scope is defined and the work to be done is well understood, and in those cases, you can do conventional project planning. You can set milestone dates and make gantt charts, and develop a whole timeline and budget. But on the other hand, Donald Rumsfeld was on to something when he talked about "unknown unknowns". Often when you are embarking on a project, you're not even aware of the obstacles you're going to face, and so you can't account for them. This doesn't mean there's no point in developing a plan or a schedule. It means that your schedule has to be flexible in proportion to the likelihood of "unknown unknowns" and other contingencies outside of your control.
I think if you want to improve the situation, the answer isn't to stop making estimates or project plans, but to develop better methods for quickly estimating the precision of your estimates, providing a margin of error, or the level of flexibility needed in a project. However, I think the #NoEstimates people are correct to point out that there is often diminishing returns on trying to set deadlines and budgets. It doesn't make sense to spend a week refining your plans for a two-week project.
The situation described by the no-estimates people is that you have a stable team (and budget) working on a large list of projects of varying value, any of which can be discarded at will when convenient. This is a dream scenario the VAST majority of people will never see. Most of us working in this kind of field don't have the luxury of telling potential customers to simply wait until their project is our highest priority or until they don't need it anymore.
I work in a scientific field. The results of my work depend on discovery, research and performing unknown tasks. These details of these things are completely unknown at the time projects are started.
Yet it is standard, acceptable, and expected that every project has an estimated budget and timeline. If we can estimate how long it's going to take us to invent a new protein, you can estimate how long it will take to write a piece of software.
Do we miss targeted timelines and budgets? Yes. Does that result in a lot of difficult conversations? Yes. However, being able to provide estimates allows us to get more stable funding. Understand quickly when and why we're seeing schedule and budget issues results in both more funding for difficult projects and a better reputation with our clients.
Giving estimates without a detailed design is bad.
But the thing is, often giving estimates WITH a detailed design is no better.
So much of software development is utterly variable - perhaps you get a guy who doesn't know some aspect of the system they are building in and needs an extra week, while another person does not. That alone is a whole week lost or gained depending on perspective.
And there are thousands of things like that in any given bit of software. The approach you take to design, the overhead of process, even just unexpected interpersonal contention between team members who may not have even been allocated to a project before you are supposed to give an estimate.
I had never even heard of the "#NoEstimates" movement but I am right on board, because from over twenty years of professional software development for companies large and small I have seen that all estimates are horrific lies, that sometimes you get lucky and the estimates match the actual work performed - but far more often people are pulling long hours or other tricks to make the work match the estimate. Sometimes they don't, and then you have what is called a "late" or "failed" project.
To take it down to a macro level, look at a scrum iteration. You are supposed to give time estimates for the work in a sprint, about two weeks long - often even THAT short a timeframe an estimate can be drastically off. You are supposed to provide estimates to get a velocity but what is the point when at any moment you can totally blow out an estimate because of some unexpected factor? You could get the same effect without estimates by having a team agree what to take on in two weeks simply by understanding the work to be done, and measuring progress on tasks moved around during the sprint... saving all the time put into coming up with fake numbers. All of the estimates and thumbs/up down are dancing around the real goal of the whole process which is simply to have a team agree what it can get done in two weeks, something any decent software team can do pretty well without estimates.
There are people who claim not giving estimates is not "professional". But since all estimates are really lies, why is it not considered *un-ethical* to lie to a client?
Far better to my mind, would be some kind of presentation of a probability curve of when work would be done, and even that only from a team that had been working together in the domain the software is in. Otherwise all bets are off and you just need to wait for the team to tell you when they are approaching a point where they understand how much longer something might take to complete by a definition they provide.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
From the article:
That gave our VP enough confidence that he "only" doubled our estimates and offered the customer a firm fixed price. This contract was a success for all concerned: we came in under our estimated time and cost
So two highly experienced developers came up with an "estimate" that was off by nearly a factor of two (in came in "under time" but surely not by half).
How is that really an estimate? From the standpoint of raising that as a banner in support of estimates, how is that success? Remember that was essentially a port of a program already working in another language, and even THEN the estimate was absurdly wrong. How is that viable in any way as a continuing process? They got lucky that round because doubling the incorrect assumptions was enough, but that may (read "will") not always work.
If a client is "allergic" to time & materials contract, too bad for them - in that case they would have expended more on building the physical plant.
There is a big disconnect in the world between people who imagine that software can become bricklaying, and those that know the harsh reality of what really happens in software development and how unrealistic time estimates are.
If bricklayers often got wrong by 3-4x the number of bricks needed for a project something would be changed. But that happens every day in software and we just all carry on as if it doesn't matter.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
air jordan pas cher More and far more folks are showing curiosity in getting to be much more active. There are numerous things they want to do in buy to get the training they want, but running is amid the most favorite that we see. To get commenced with this it is significant to make investments in respectable operating shoes nike air jordan pas cher 2015 Femme . This is what we will appear at a bit even more beneath.A retail outlet that specializes in sports activities like this is the ideal put to get started the hunt for running footwear. The purpose for this is that the workers in these suppliers are trained about what points should be appeared at when persons are seeking for this variety of footwear.Salespeople that work in this region should just take a genuinely near seem at the foot of the man or woman purchasing for operating footwear. They can identify a good deal of issues by seeking at your foot. For illustration the arch is a single portion of the foot that they have to have to know about in purchase to be able to enable you come across the most cozy sneakers that they potentially can.You really should also make guaranteed that the human being supporting you decide on new operating shoes normally requires the measurement of your foot.
There is no way in hades clients are going to write you a blank check on the software they want you to write for them. If you can't tell them what it will cost, they will go find someone else who will.
You cannot talk them out of this. It is impossible. They want to know the price before they buy, and that's that.
I use an internal schedule. Some days I want to work on nasty low-level code, other days I want to do more maintenance and infrastructure work like producing build scripts or automated testing... Some days I'm ready to type code from the instant I walk in to the minute I leave and the creativity is flowing like a river. Some days I have no creativity and really just want a straight-line task with no deep thought process.
I come in and properly gauge my state of mind and decide accordingly. This is how I've done it and I'm a rather respected developer with people asking how I don't get burned out and how I'm able to also work on my own projects at home....
It all boils down to the fact that I don't do anything I don't want to do. Thus I'm always happy, passionate, and never burned out. The thing is I rarely have a day where I want to do nothing, I always want to do *something*, but what that is changes and cannot be predicted or scheduled without forcing me to debug on a day when I'm ready to write 5,000 lines of well thought out code, or forcing me to write lots of code on a day when I had a sweet idea for automating half the QA tests....
That is how software development works. I keep myself running at 9/10ths by never letting someone else direct my work for the day. As soon as you mis-match my state of mind I'm more like 6/10ths and frankly producing "average" work at that point. The above-average genius stuff comes when I'm just left alone. That's how you get your money worth out of me....
There are *several* formulas to do an estimate and several more to tell you if your estimates are on track.
If you understand why estimates are required, you are a business person, if you understand why they are so difficult you are a developer. Managing estimates is a 'Project Management' task and a good PM will keep the pressure of the team by also managing the stakeholder expectations, which is what we are really talking about here.
Complex estimates are closer to the contract and simple task estimates are closer to the metal. If anyone asks for an 'accurate estimate', run - they are an oxymoron who won't de-scope so that deliverables are met. To me it is an immediate sign of project failure.
Estimates are just a tool that are a balancing act for getting the budget required to do something. Good estimates are achievable by iterating three simple questions pessimistic, realistic and optimistic estimation for a smaller task of a large project. After that there are several other formula to determine if you are ahead, behind or on schedule. Ahead or on schedule - great, behind - de-scope. What the final product looks like is a function of the contract that determines the critical path and managing the expectations to get there. Estimations on a small project however are usually a waste of time.
The last thing you want to do is go back to an accounting department or client for more budget because the estimates are way off anymore than having no estimate at all and asking for a big bucket of money that won't get approved and no developers will ever get employed to do that project.
Using 120 characters to discuss such a complex subject, that can't possibly hope to encapsulate the arguments required to understand it, is pointless.
My ism, it's full of beliefs.
That's about the most biased summary I've ever read.
I know Woody, I've seen the environment NoEstimates was born from, it is awesome. That said most corp cultures have zero chance of ever getting to that level of trust and most dev teams are way too dysfunctional to be able to build that level of trust. Which is why when you put pressure on NoEstimate proponents they will state that it is about "starting a conversation" about estimates.
In reading these comments the phrase "Giving estimates without a detailed design is bad" pops up in one form or another. Basically that is asking for a waterfall. I am somewhat uneasy how often it has appeared even in this day and age. Here are my observations on the topic:
1) Forget good requirements, no one has them. No one knows what exactly the software will do at the beginning of the process. One thing I am certain of is that in about 18 months, if you are competent and lucky, you will start to have a good piece of software.
2) Instead think of "Initial Requirements". That the best you can do. Customer has a problem to solve, start solving the problem.
3) As you solve the problem new requirements appear. This is due to two factors, the customer does not know what they do not know until an analysis is done, analysis is often the biggest pay off of a software project in my opinion[1], and the customer starts to see possibilities. Once you give customers new capabilities they often use those ideas to springboard more features. Often too software requires a review of business process which shed light on how a business is *actually* run, not how management *thinks* it is run. This will cause requirements shift.
4) Work more on your customers rhythm, not yours. Don't roll out major changes in December if your client is a retailer for example.
5) Keep the customer happy by giving them continual improvements in functionality. Lots of little releases to throw them a bone and to get feedback since requirements always will shift. Stay close to the end user and listen to them.
6) Good software is a living breathing thing. Expect to live with it for decades and continually improve it. It is never really done.
The core of Agile gets some of this right. Though I am worried about a natural impedance when projects become very globally distributed and in terms of Agile scaling.
[1]Anecdote. I was working on a database development project for a mid-sized company with 2 main offices in two cities 300km apart. I began by learning the business processes and by diagramming them out. The principals of the company were flabbergasted when I showed them my diagram. The two offices were doing things very differently. We spent the next 2 months realigning core business processes before writing any code. It was actually fun.
putting the 'B' in LGBTQ+
Those who are paying the bills want estimates. Those who are getting the money want a blank check. It's that simple!
There are ways to create good estimates, but it does require discipline and training. Steve McConnell wrote an excellent book on the subject. Unfortunately, many software developers aren't well trained in this art. This is a serious failing in many of our computer science degree programs.
Estimates are useless as a measure of how well an engineer is performing. How far he is ahead or behind schedule only indicates the extent to which he was able to get away with padding his estimate in the first place.
That said, estimates ARE very valuable when you have a complex set of interlocking projects and resources that can be tasked in different places. This is especially true if external pressure require that a project be done on an exact date.
To take an extreme example, if the launch window for Europa is at a known date, the spacecraft firmware must be fully tested and installed by that date. Working backwards that says when the first version must be ready. The estimate helps decide what resources should be applied, and later it lets you know if you are so far behind that you need to change the launch date to the next window (over a year away). That affects budget etc.
At SLAC we have complex projects that require the work of lots of people to all come together. This results in very rigid schedules - There is typically a 2 month window for major upgrades, if you miss it, you wait a year. If someone working for me doesn't like doing estimates, I basically say "we need a guess. I can guess or you can, but since you are doing the work, your guess will be better than mine".
#Moo
I have worked in the industry for 20+ years and here are my observations - granted, I have worked in smaller companies (i.e 25-4000) and startups.
Estimates work as a means to determine the work effort for a given set of features. They are not to be used for setting schedules and deadlines by themselves. They are to be used for budgeting and cost planning. And, they are not to be done without a detailed design meeting the agreed upon requirements.
Unless your client is very rich and/or stupid or you have a large surplus of venture capital in your startup, you better be concerned with the work effort and time to have your product in a usable state. When you can tell a client that a project is going to take time X and cost Y and meet those values, you gain credibility and trust. In the digital advertising world, those with credibility and trust become the agency of record (AOR). And, the client will stick with you as their AOR until you royally screw up and fail to deliver what was promised, when promised and for the agreed upon price.
You DON'T ask a developer how long something will take - they invariably will underestimate the work effort. Instead, at least until you have measured delivery rates for your team members, you use industry standards. You can ask the developer and then compare their estimates with the actual time and effort. When their estimates start matching up, you can ask them estimate their own assigned work. It can be a good learning experience for them.
Some projects don't require estimates. We had projects that fit a template model based earlier work. We knew how long it took, on average, to fill the various fields of the template. Throw in the project management, QA and deployment components and its pretty easy to do.
When people claim Agile isn't compatible with estimates, it's probably because the team isn't concerned about documentation or planning. They tell you that the system has too many variables and they don't have time or resources to keep the design up to date. I call BS. If you can't do it, then add someone to the team who can. There are great tools for doing design work and capturing requirements at all phases of a project - use them.
Even with Agile, you should layout out a basic design or framework in the early stages of the project. Then, you can determine how long other features will take and what their dependencies are before you attempt to implement them and emptying the client's wallet. Then, you base the number of sprints based on that information. Since you are supposed to have a working product at the end of each sprint, you should be able to tell the client what features will be in each deliverable and cost. If they want to change the requirements or add new features or change functionality, you still have to plan how long those features will take and in which sprint you will deliver a product that meets those changes.
All I need to do is to look to see where a project lay in the realm of my experience and knowledge. I can tell you that if you want me to make hello world in a language I know well on a platform that I know well that it should be under 5 minutes from beginning to end. But if you want me to program an image recongintion system that can tell if a sparrow is content or agitated on a VAX/VMS. Then I will give you an estimate of maybe 5 years before I can even give you a proper estimate having gathered up the experts on sparrows, VAX, and image recognition.
All other projects generally lay on a line between infinite and 5 minutes.
But much more importantly my real estimates on real projects will largely not depend upon the projects, the code, the specs, or pretty much anything technological but will have a multiplier based upon the company and its culture. If the company is clearly prepared to play games with the contract then the project will slow way down to make sure that every damn checkbox is checked and arguable in court. If the company has a civil war going on and the project is going to be a pawn then the multiplier goes way up. If their is no clear manager of the project and the bosses wife becomes involved with turning a project to make a point of sale system into a dietary recommendation system for her stupid diet beliefs then the project may, again, have hit infinite time.
Where most projects that I have seen blew up was when there was a failure to properly investigate the real project. Maybe someone asked to build a medical system that would do 100 features but failed to identify 1000 other critical features that made the 100 feature system completely useless. Or even worse 1000 features were ladled on after it began and the people running the project were too wimpy to resist.
Also projects can become bogged down in stupid minutia. One project I was working on was a massive invoice system where one marketing guy kept changing the look of the invoice by pixels. Day after day after day he would move things tiny amounts. The project couldn't proceed without his approval and it turned into a demoralizing nightmare. Then to put icing on the cake the problem was that the PDF samples he was printing were coming out differently on the various printers he was using. Thus the changes requested were only possible to achieve if he consistently used the same printer.
Thus this last one was a failure to create some contractual mechanism to prevent this stupidity not a failure in the original estimates which were generous in their margins.
Then you get projects that have entirely outside forces. The project was never meant to succeed. This could be on the part of political desire of the client. Or it could be fraudulent hearts in the consultants. All together many projects are doomed before the contract is drawn up so the only viable estimate would be one written in blood using magical symbols.
But to wholesale say, estimating is bad is wrong. Bad estimating is bad. But too many people get caught up in functionality points and whatnot and miss the fact that they are about to throw their entire project into a war zone. But that with a careful analysis the war zone can be calculated into a multiplier and the risks mitigated.
"You want an estimate? I've got your estimate right here: I estimate 8 hours/day, 40 hours/week, until the Union rep tells me I'm going back on strike."
This is what really happens is most enterprises: http://www.halfarsedagilemanifesto.org/
As they saying goes: Fast, cheap, good - pick any two.
You can usually solve one problem by another. If you see you can't hold the deadline, you can throw more manpower at it (not cheap anymore) or compromise on quality. If you see the budget runs out, you can put the project into idle times (not fast anymore) or compromise on quality.
Sadly, quality is the part that management, customers, clients and developers understand the least. Everyone understands deadlines - either you are done on that day or you are not. Everyone understands money and to convert developer man-hours into money is not so difficult. But quality is tricky. If it runs, ship - because management, customers, etc. they see if it is running, but not what's going on under the hood. And developers too often don't understand that quality is subject to combinatorial explosion - shortcuts don't add up, they multiply.
But because it's the least-understood part of the equation, and compromises matter so much but are not easily visible as long as the core operation functions, software in generally is so absolutely shoddy.
Assorted stuff I do sometimes: Lemuria.org
I've been looking into building a house recently, and it taught me something about software development:
To understand what I would get for how much money, I went to a "park" of houses. 20 or so houses of different styles and sizes, with price tags. So by spending a few hours walking through them all, I could get a pretty good understanding of what I would get for which price. From that point, I can start an informed discussion with a company asking for the house I want, based on what I saw, with an idea of what my expectations will mean in cost.
I have never seen something like that in software. If I want to have a software written or even just a website made, where can I go to check out examples with price tags? It seems to me that everybody keeps the final, real prices a secret. They are not showing you that this website with these functions and this design cost $x to create, and that software tool with these features cost $y.
The industry is based on estimated costs, not real costs.
Assorted stuff I do sometimes: Lemuria.org
Of course, this conversation would never happen if the requirements were on some fucking paper and the customer wasn't a gaslighting psychopathic cocaine-snorting asshole.
Has anyone ever told these people that someone somewhere up the line has to pay for them to f*** about? I'd like to build you a house. I don't know yet how big it's going to be, because I haven't designed it yet. I don't know how long it's going to take to build it. I want you to pay me an indeterminate sum of money for it... And I want you to pay for it before you get it. In fact I want you to start paying me now, and keep paying every month until it's done. I'll tell you when it's done. Who wants to buy my house?
Never trust a man in a blue trench coat, Never drive a car when you're dead
You want some money? Huh? Huh? You want some?
Well do what I say, now.
Who's a good boy? You're a good boy.
Here you go. Have a little money.
====
Whether you like it or not, this is your life. Now the question is, which side of this converstation are you on? Do you have the money? No? Then be a good boy.
Industrial engineer here. The problem with most schedules and by extension most estimates isn't the estimate itself. It is the failure to take into account the variability of the estimate. If you ask me how long a task will take for something that is a non-trivial task and you give me a point answer (like "14 days") then you are almost certainly wrong. The only question is how badly are you wrong. But if you tell me 14 days with a standard deviation of 4 days then the answer might have some credibility. Any answer without error bars is probably useless.
My first job out of college was doing statistical production simulations. Everybody in management wants a single answer for how long something will take. Problem is that if I give them a point answer I'm doing them a disservice because my answer will be wrong. Fortunately finance people actually do know how to deal with a standard deviation if you press them to be bothered because they have to do it for expected return calculations.
When I'm scheduling anything I care a LOT more about the variation than I do the averages, particularly if the averages are certified wild-ass-guesses.
As they saying goes: Fast, cheap, good - pick any two.
Not a useful saying for budgeting. HOW fast are we talking about? HOW good? HOW cheap? Fast/good/cheap are all relative values and can mean hugely different things depending on the scope and difficulty of the project.
If bricklayers often got wrong by 3-4x the number of bricks needed for a project something would be changed. But that happens every day in software and we just all carry on as if it doesn't matter.
Then software developers need to get better at estimating. I fully understand that it is considerably more complex than counting bricks but at the end of the day it doesn't really matter much. The only proper solution is to learn how to provide accurate assessments of resources required to do a project. It's no different in manufacturing, construction or any other complex field. I spend much of my time quoting work for complicated manufacturing assemblies, some of which I have very little experience with. I have to estimate the amount of time it will take our staff and allow for variation in productivity. If I overestimate the amount of work we risk not getting the job. If I underestimate the amount of work, we lose money and can be out of business very quickly. It's not really so different than with software.
Now if you are being asked to estimate the cost of a project where the scope of work is undetermined and/or not fixed to within reasonable bounds then you are probably screwed. When we get asked to do this we turn the work down unless they are willing to sign an open ended time-and-materials contract. Never had one of them do that yet...
some estimate on cost and on return. Also without some idea of how long something should take it could take forever. And then there is budgeting. You have X amount of money and N things to do and you only do the things you can afford to do so you need estimates for that. The major reason why things take longer and take more money is that people don't want to believe the truth or people want to get the project they are sponsoring to get done so it is in everyone interest to understate the cost and the time.
Cowposter is hands down the best commenter on Slashdot.
+
youthfully idealistic "life would be so much cooler without deadlines, schedules, goals, budgets, money, time, calendars, clocks, prices etc... but let's keep the paychecks please"
=
#NoEstimates
There are shitty managers, but only when you've a) communicated what you (think you) know to them, and b) appreciated everything that goes into their decisions, can you say for sure whether you have a shitty manager. Managing is a thing, just like programming is a thing. But if you're pretty sure you can manage yourself better, start a startup.
Development will take however long it takes and the QA cycle will pay the price.
What you do not realize s that IT IS NOT POSSIBLE to get better, because of the variability of EVERYTHING.
Very little in this world exists without variability. You might end up with a pretty wide standard deviation at the end of the day but I don't buy for a second the argument that there is no room for improvement provided the project demands are reasonable ones.
The Proper solution is to accept reality and work around the FACT that you cannot get reasonable estimates for building software.
There are circumstances when providing an estimate for building software is impossible but it is demonstrably not true that it is universally impossible to estimate time required to build software. If software development were truly random as you claim then it would be impossible to manage and it clearly isn't. Pretending that you can never estimate development time for software is preposterous. As long as you are willing to put error bars (possibly large ones) on your estimate then many projects can be estimated. Estimates are rarely exact (that's why they're called estimates) but they absolutely can be useful. If you have adequate and firm specifications early on then you should be able to provide a reasonable estimate of time to completion in many cases. If you don't have adequate and firm specifications then you probably are correct that any estimates that follow will be nothing more than guesses with a high probability of being wildly wrong.
I'm a big proponent of Agile (mostly XP, mostly anti-Scrum) and have contributed some to the #noestimates "movement".
I don't really mean that nobody should ever estimate anything. I mean that I've never seen useful (fine-grained) estimates anywhere. Here are some of the problems with estimates that I've seen frequently:
I'll note that most of these pit management against the team, instead of working together toward a common cause. Most of the practices also lead to seriously demoralizing the team. And most of the time, the estimates aren't really even taken into account very much.
My advice is to first understand the value of a project before you consider estimating the costs. The estimation at this point will be very rough, so make sure that you have a very wide margin between the expected value and the rough estimate of the cost. If you're pretty certain of the expected value, I'd probably want to make sure I could still be profitable even if it took 3 or 4 times as long to complete a
Software sucks. Open Source sucks less.
Over my 25+ years of programming, being able to estimate my time was the last and hardest thing for me to learn how to do.
No matter how much you've mastered computer science and how many clever encryption algorithms you're capable of writing, estimating how much time your work will take is a completely separate ability having nothing to do with your actual programming and/or mathematical skills.
It is possible for every programmer to learn how to do. It's not something you'll figure out in a week or a year or ten years. I promise you that being able to deliver your software on time, every time, will make you the most beloved programmer at your company.
The key is to, instead of jumping right into the coding, spend several days understanding exactly what work you need to do. Learn to be realistic about your abilities. Learn how to communicate with non-programmers so they understand exactly what they're getting. Keep explaining until it's clear that they understand what you're writing for them, and that's exactly what you're writing for them.
When people throw changes at you, warn them that you'll have to start from scratch with understanding exactly the new work that needs to be done, think about the time those changes will take knowing that you may need to discard work you've already done, and continue to be realistic about your abilities. Make sure you get approval for the revised completion time before starting any work. Do not jump right into coding the changes.
If your employer doesn't allow you do to this, quit and go work somewhere else. There is an oversupply of programming work.
Time estimates are something that all professionals do. When you finish your work on time, you are acting professionally. When you reject estimates you look like a rank amateur and I'd never hire you.