Slashdot Mirror
Ask Slashdot: Are Accurate Software Development Time Predictions a Myth? (medium.com)
New submitter DuroSoft writes: For myself and the vast majority of people I have talked to, this is the case. Any attempts we make to estimate the amount of time software development tasks will take inevitably end in folly. Do you find you can make accurate estimates, or is it really the case, as the author, DuroSoft Technologies' CTO/Co-CEO Sam Johnson, suggests via Hacker Noon, that "writing and maintaining code can be seen as a fundamentally chaotic activity, subject to sudden, unpredictable gotchas that take up an inordinate amount of time" and that therefore attempting to make predictions in the first place is itself a waste of our valuable time?
Try getting a contract awarded with "It's too chaotic to tell"
Take your developer's estimate
Increase that by 40%
Double that
It seems incredibly arbitrary, but I have learned that the 40% covers testing and implementation, while doubling the entire amount allows for the customer seeing the first result and _then_ telling you what they really want.
Try it, you'll like it
Mr Scott, have you always multiplied your engineering estimates by a factor of four?
Oh, laddy, you got a lot to learn!
Even with known and well understood languages/technologies/frameworks, you can and will run into glitches that can take days to complete something that was supposed to take hours - or even longer if the developers are not skilled in debugging and isolating problems!
StackOverflow has helped the industry in this regard, because now a lot of times you can reduce some mysterious problem to a fifteen second StackOverflow search which instantly answers the issue. But not always, and there are always issues when actually programming any design that you can uncover hidden flaws and need to correct them.
What I would love to see is some kind of approach that instead of a time estimate, gave a time along with a percentage of confidence. Two different tasks may seem to take about five hours, with one you are 90% sure it can be done in five hours, with another (like brand new code) it can be more of a 50% five hours. Then you could use this percentage to determine the actual areas of coding likely to cause schedule issues and monitor them more closely. The other nice benefit of this approach is that it factors in the actual developer understanding and abilities more than just a straight hour estimate. Maybe you even put a cap on how high a confidence level a developer is allowed to give until they have met given estimates a number of times already...
Coding is a chaotic system, yes, but it's not like it's fully chaotic, and there are patterns within the chaos I think you could determine over time.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
...estimates of resource requirements (including time and budget) for software projects.
However, making estimates of Resource Requirements that will be acceptable to Management...that's impossible. It's amazing how so many people who've never tried can make such ignorant demands...and think they're being "fair."
I've done many project plans for clients, and when I give them the results, they always bitch. But, when the project is actually delivered, they finally agree that I was right in the first place. After that, it gets easier...which THOSE clients.
Fortunately, I'm now retired, and no longer have to suffer these dolts. And, MY projects are quickies that work within a few days, and I love to tenderly "evolve them" over multiple successive iterations, adding new features as I go. My latest one is about 15 generations into development, and I may bring it to market if I can find a suitable partner. It's a novel approach to easily-recovered 100% daily computer backups. it's a much more practical methodology that doesn't require prescience before the first build is done to know every detail about the final product...it naturally evolves.
Like any "process" you can do better but a couple of things have to be true...
1. You had better KNOW what your development process IS (not what you document, or what you want but what it actually IS). Until you know how you develop, test and deliver software from start to end, there is no way you can estimate how long it will take accurately.
2. You need to develop a method of coming up with your educated guesses that is repeatable. The process must allow you to estimate every step of your development process and how much each step is estimated to cost both in time, materials and labor.
3. You MUST collect metrics that allow you to track actuals to estimate for each individual step in your development process though how ever far you get (hopefully to completion).
4. Next time you get a contract proposal and you start though this process again, when you get to step 2, do your estimates, but then compare past projects data from step 3 and see if you need to add or subtract some fudge factors based on past history. Rinse Lather and repeat.
Eventually, you will get better at the estimation and have more confidence in your ability to bid contracts....
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
Quality of prediction comes down to quality of the information fed into making the prediction. Given all information up front with good BA work up front and requirements and scope that don't change I can generally provide good predictions that are close or that even come in under budget for our projects (good predictions always have some buffer built in for staff and technical problems).
Until an even somewhat accurate time is reflected in with the estimated time remaining progress bars there can be no hope for predictions of software development times.
That's similar to what I've experienced and seen reported in studies. When I say "10 hours", that really means "10 times X hours", but that X factor is relatively consistent. Each of my teammates are similar - they are always wrong, but normally by the same multiplier each time.
Developers tend to be reasonably good at estimating the RELATIVE amount of work, they can say "job A will probably take twice as long as job B". This assumes the work is broken down into pieces small enough to estimate. What they tend to NOT be good at is saying how many hours, days, or weeks.
That's where Scrum "story points" come in. We assign each task a number of points. Historical data shows that we can complete 65 points in a two-week sprint. That's relatively consistent.
Some tasks will take longer than expected, some less, but that tends to average out over two weeks of a four-man team. The four of us complete 65 points per sprint.
I'm able to estimate fairly well how long something will take for me, but with a bunch of proviso.
Sadly most projects fail one or more of the points above--which make most estimates absolutely worthless.
I've done many project plans for clients, and when I give them the results, they always bitch.
Oh man this 1000 times, but in my case it usually tends to be management that does this. They ask for an estimate for something, you tell them 100 hours, they say that's crazy and change it to 40. Then when you end up getting it done in 80 (which is better than the original estimate), they complain about it taking twice as long as it what they budgeted. This appears to be pretty common, it's not just a single organization I've been at that is guilty of this (of course someone on here will say maybe the problem is with me, which hell it may be true now that I think about it :-)
My gripe though is that if you already have a number in mind, why bother asking me to provide an estimate?
If you post as Anonymous Coward, don't expect a reply.
Performance predictions have an optimistic bias.
It's known as the planning fallacy
It amazes me how people who should really know better fall for this.
For example, if the last time you did it, it took 3 weeks, a good prediction is that this time it's going to take 3 weeks.
Yet most people will predict less than 3 weeks - even if you point out the planning fallacy to them before hand.
I can almost here the rationalizing; "It's not going to take 3 weeks again, because we aren't going to make the same mistakes again."
But it's far, far, worse than just an inability to predict accurately.
Frequently schedules are determined by need rather than reality. As in, we need this done by Tuesday - make the schedule accordingly.
This:
I always provide my managers with confidence interval estimated times
50% 10 days (assumes *nothing* goes wrong, no interruptions, and high code reuse)
90% 15 days (assumes no catastrophic issues, no interruptions > 2 hours and only 5 of them, moderate code reuse)
100% 55 days (the wheels fell off, severe schedule impacts of interruptions non stop, no code reuse)
My boss laughed at my 100% estimate until it actually happened.
A lead dev (who could be counted on for sound advice delivered in a belittling way) was struck down by lung cancer and ceased to exist. Another lead dev who was even brighter, and much nicer to work with was poached by a competitor, both within days of each other. The code was cutting all new territory in the system, so maybe 15% reuse? *and* some panicky manager started having $deity damned _daily_ meetings about it.
We almost missed my 100% date, made it by about 16 hours.
whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
Finish in 3 weeks: "it was an easy project after all! Your estimate cheated us"
Finish in 5 weeks: "you're a crappy engineer! you cheated us by pretending you could do the job"
Finish in 4 weeks: "that's suspicious! you obviously finished early and then goofed off. You cheated us"
Moral: you can say how many or how long but never both! Or: it'll take between 2 weeks and 2 months, depending.
What he wrote:
What's he's hoping people read:
Unfortunately, version 1.0 typically falls under the thick veil of he said, she said.
Here's the exact point where he wanders off into the weeds:
It doesn't take a 4-Sigmund review to spot the out-of-school litigation here. No one in a state of conflict appreciates the lateral spread of subtext.
I know estimation is often used as a management bully club, and I've had some pretty dark thoughts about some indivisuals who have chosen to behave that way, but sorry, I'm just not feeling the sympathy in this instance.
https://www.joelonsoftware.com...
This is one of his blog posts that is almost an infomercial for his product, but he does describe the concept well enough that you could roll your own if you wanted to.
See that "Preview" button?
Yes, this. Good management knows that you know your trade and accepts your estimates. Bad management thinks they know better and try to negotiate on estimates.
Over the years I've found that feature development, particularly adding to existing systems, will get better estimates if we're allowed a spike to review the affected areas up front - this is when you discover unexpected dependencies or just sheer awful spaghetti. When you're not allowed to do this up front is when you come across the unexpected gotchas that blow out the best estimate you could provide at the time.
Yip, generally it's easy to make an estimate for a clear specification. But, customers rarely know what they REALLY want until they see something in production. This is a very common problem. I don't know any easy solution to that: mind-reading machines don't exist.
One partial solution is for the technical analyst to become a domain expert first. But obviously this is often not practical. Further, sometimes the main customer/manager wants something rather odd that is a quirk of their personality. You may build something that fits the domain, but they want to see their domain in peculiar quirky way.
Another partial solution is "RAD": Rapid application development tools. Someone who knows the tool well can usually spit out something pretty quick and change it fairly quick.
However, RAD tools are not known to be very flexible in the longer run, such as when UI styles and expectations change. They achieve RAD in part by marrying business logic to the UI. This marriage makes less "marshaling" code between the database, biz logic, and the UI; BUT also hard-wires it to UI assumptions. Keeping up with the UI Kardashians can be a major PITA. Just when GUI's were maturing, web came along. Just when web was maturing, multi-device-handling needs came along. The current "in" thing is going to be klunky because it's not mature yet.
For now, it looks like we are stuck with some degree of organic meandering to get something the customer is actually happy with; but organic meandering takes more time and money and is hard to estimate up front.
Table-ized A.I.
The solution is not to *impose* a schedule, but let the developers who are going to do the work make the estimate, and don't argue their estimate down. Others who have suggested multiplying by some factor might be a good idea as well-- but unless the developer has his own credibility on the line for the estimate, he won't put in the extra work to try to make up for a schedule overrun.
Yes when clients and managers choose often it speed and low price in the preparation then demand quality at which point the speed disappears. to maintain the price which rises anyway due to the extra time cost but this isn't passed on leading to failure.
Your'e all thinking it, I just said it for you
(I am the OP)
TLDR: lost my job for writing this article
you can donate here: https://www.gofundme.com/lost-...
While I list my title online as the CTO of DuroSoft Technologies, I am a part-time graduate student, and the vast majority of my income comes from a contracting position I (until just now) held at a popular SaaS company. I am not going to leak the name of this company, and I would politely ask people who reply to this thread not to speculate about about which company this might be, or otherwise go on a witch-hunt at this time.
Effective immediately, I have been terminated from my contracting position at X explicitly because of the views I express in this article, and the article's subsequent popularity on HN, Reddit, Slashdot, Hacker Noon, Medium, and other sites. Earlier today the article was posted and discussed in my employer's private engineering Slack room. A highly positive discussion ensued in which a number of our senior developers identified some key points and takeaways we could use to improve our effectiveness as a team. The CTO intervened and put an end to the discussion with little explanation, even though it had been a very constructive conversation with zero negativity. Despite this, I was floored by the words of praise and encouragement I received privately from other developers, and fully expected my article to become the topic of our upcoming team-wide engineering lunch. Several hours later I was called into my supervisor's office and it was made clear that "your medium article goes against some of the core engineering values of our organization", and that my decision to publish this article was the deciding factor behind his decision to terminate my position. Within 5 minutes, I was kicked out of GitHub and slack, and escorted out of the office of a company where I had worked for the last three years.
I am scrambling right now to figure out what my options are, my main concern being that I am already in debt, supporting my fiance, etc., and need to get my financials figured out ASAP. I just sent out about 20 applications to various senior software engineer jobs, but I likely will go negative well before any of them gets back to me. If you would like to support me while I figure out my options, and potentially help me fight for my right to intellectual freedom online, please consider visiting the gofundme link at the top of this comment.
Sure, coding this up should be pretty quick, since I just need to plug A into B using X. .NET does W, Y and Z, so it should do X.
Me: Wait, where is X? .NET won't allow you to do that with X for security purposes
Microsoft: X has been deprecated in 64-bit.
Me: Wait, why? W, Y, and Z all work, why not X?
Microsoft: Nobody uses it.
Me: It's listed as a feature in your docs! It's the recommended method! If people are doing W, Y and Z, they are definitely doing X!
Microsoft: Whoops, wait a sec... There now they say NOT to use X as it's deprecated.
Me: Fine I'll write a 32-bit shim
Microsoft:
Me: Holy cow I need to write a friggin' service and pipe crap to it just to get X to work?
Microsoft: Or write your own implementation
Me: The whole point of X is it's a pain in the neck to implement properly and I'd rather be spending time on user-facing stuff, rather than become an expert on X, which I'll only be using for this one particular feature. Fine I'm re-implementing, my original estimate has now increased 10x.
Six Months Later:
Microsoft: Good news, X has now been implemented in 64-bit! Download it here...
My Other Computer Is A Data General Nova III.
> I don't know any easy solution to that: mind-reading machines don't exist.
I came across a solution that works really well, whenever you can possibly do it. First, let's be clear about the most common method, which does NOT work. Most commonly, the users' boss talks to someone, maybe a product manager, about what they think the users need. Then the product manager or whoever talks to the developers about what they think the users' manager said. That doesn't work very well most of the time.
Most of the time, new software is needed to handle a process that is currently being done by hand, perhaps on paper or in Excel. Maybe you're replacing legacy software. Normally, the job is getting done *somehow*. So go *watch* the job being done. If it's being done on paper, watch thr person do it on paper. Follow the piece of paper as it is filed with another department and they type the information into some computer system. While watching the person do the job manually or via the legacy software, ask questions and take notes. Then if possible try to do it once with them watching you and correcting your mistakes. Now you know pretty much exactly what's required to get that task done, because you've just done it by hand. Ask what kind of exceptional conditions come up - what kinds of weird things happen that cause a change in the process? Obviously you'll code for those specific exceptional conditions, but also that lets you know what general *types* of variation there might be, meaning where you should try to build some flexibility into your system. When you discover there are three different types of X, you'll build X modularly, knowing that another type of X may come up.
If it's not possible to actually watch the line people doing the job, at least try very hard to get them on the phone. Talking to the actual users, asking them what's frustrating about their current process, will tell you a lot about the requirements that you won't get from listening to your boss talk about what their boss said.
....there are no accurate requirements or no requirements at all. If you don't tell devs exactly what to build they have a tough time estimating. Even when there are requirements, if there is nothing comparable done recently it is difficult. Besides that an estimate implies that it is NOT accurate! Anyone who takes estimates as fact is a fool.
If you're doing it right, you should never be doing anything twice. Anything you do should become a packaged and re-usable element that doesn't need to be coded again. I don't care whether you call it a subroutine, an object, or what have you.
Software development, done right, should grow exponentially--with a highly fluctuating exponent. No task should be predictable because no task should closely resemble anything you've done before. If it does, you shouldn't need to develop new code, you should just be able to re-use old code.
Well, OK, this is a gross oversimplification, but it does capture something fundamental about software development.
In the past I've found that managers almost prefer to do thing repetitively, over and over, the same stupid way. They love what is conceptually close to a duplication of the essentials of the last job, because although it's highly inefficient, it's also highly predictable. They would much rather have a near-linear curve of accomplishment versus time, then a much faster, but much less predictable exponential-with-fluctuating-exponent curve.
The typical manager would probably order you to recode the same thing ten times rather than "waste time" writing a subroutine.
(To be fair--it's hard to write a truly re-usable piece of code and easy to waste time in the name of re-usability and write code that isn't actually re-usable).
"How to Do Nothing," kids activities, back in print!