Slashdot Mirror
The Economics of Perfect Software
An anonymous reader writes "This article takes the interesting perspective that leaving bugs in software is good — little ones, at least. This quote is particularly insightful: 'How do you know whether a bug is big or little? Think about who's going to hit it, and how mad they'll be when they do. If a user who goes through three levels of menus, opens an advanced configuration window, checks three checkboxes, and hits the 'A' key gets a weird error message for his trouble, that's a little bug. It's buried deep, and when the user hits it, he says 'huh,' clicks a button, and then goes on his merry way. If your program crashes on launch for a common setup, though, that's a big bug. Lots of people will hit it, and they will all be pissed. ... The cost of fixing all the bugs in your program and then being sure you fixed them all is way too high compared to the cost of having a few users hit some bugs they won't care about."
Diminishing returns applies to programming too... big surprise...
You cant afford perfect software.
The reason that every piece of software ships with bugs is because:
... and someday we will.
* It's created by people
* The programmers, testers, etc can never find all the bugs
* It's gotta get out the door so you can pay your programmers, testers, etc
* All of the above
There is a matter of pride with individual programmers, small groups, and most open source programming projects. We'd all love to be able to crow about shipping a bug free project/product. We'd do it if we could
Well, sure, except that this assumes that you are PERFECT in your ability to predict the effect of a bug. And if you're not, that bug that you think will only happen in some situation that's vastly improbably will, in fact, actually hit at exactly the WORST possible time, because maybe that key sequence gets used in some extremely important operation that you hadn't realized your software was going to be used for. Or maybe that bug is ALSO triggered by some different sequence that you weren't quite prescient enough to realize would be common.
http://www.geoffreylandis.com
When I'm trying very hard to make a program do what I want it to, the more hoops I have to jump through for every iteration of trying to make it work, the madder I get. So, the fact that the software has dark corners that you can get to like this is already a major strike against it...
This sounds a lot like the "Good Enough Software" policies that certain companies used to have. It's true that having a few minor bugs is acceptable; in fact, it's pretty much inevitable. But you should expect to have bugs that you haven't even noticed at the time you ship your software. If you start ignoring the bugs you do know about, you increase the total number considerably. From there, it's a slippery slope. Pretty soon, you'll be answering bug reports with a chorus of "Minor, leave it," and your code will be riddled with tons of minor bugs. Your users will go from "huh" to "what?" to "dammit" to "f*ck this!" Your software will look sloppy and unprofessional, and even if none of the bugs are showstoppers, your customers will be looking for alternatives.
I think it also depends on how much it costs to remove the bugs. Often times, the small bugs that people are more likely to accept or tolerate are also very easy to fix. When the cost of fixing the bug is very low, it should be a no-brainer to do it. If on the other hand the bug is expensive to fix -- requiring a complete redesign and re-coding, then if the bug is not severe it may well be better to leave it in.
You see? You see? Your stupid minds! Stupid! Stupid!
Software is not so different from other engineering disciplines that we cannot learn from the best practices of adjacent fields. Although no human endeavor -- other than being Pope -- is infallible, we do see disciplined attempts to measure, manage and improve quality levels. The point is not to reduce the quality level to zero, but to know what your quality level is and be able to hit your mark. What level of quality is needed will depend on the market, your customers, the competition, the frequency of releases, whether the software is easily updated or burned into firmware and devices, etc.
In the end, the best way to create higher quality software is not to find and fix more bugs. It is to use a disciplined approach to programming that introduces fewer bugs in the first place. If you focus on defect detection and removal, then you are fighting the wrong battle. You can't win doing it that way. Quality software comes from a disciplined process at all stages, from requirements to testing, and not just as an endgame activity to sway as many bugs as you can.
int main()
{
printf("Hello, world");
return 0;
}
I demand your first born.
Palm trees and 8
Is going to be one extremely unhappy user because they had a reason for going that deep into the sub-cellars of the software to get at that obscure feature ... and it is likely that they are the sport of user who is a bit more tech-savvy and ready to bitch than Joe Average user.
Sometimes it's better to ship a product and deliver it to 95% of the user base instead of having everyone wait because 5% wants feature X which is broken.
But there is the option to disable feature X, so rather than knowingly ship with bugs you can often have the program show a notice that this feature is disabled or something like that.
Generally when building software you'll have "Quickly", "Lots of features" and "Bug free". Depending on the software and project stage you'll want to strike a different balance. Releasing an alpha which has lots of features quickly with loads of bugs is OK. Releasing a final release of a program handling financial transactions with loads of bugs is never OK.
But yes, it's a question of economics, and what makes sense for a given project at a given time.
.: Max Romantschuk
It isn't always easy to judge the severity of bugs. Exploits often grow from a black hat figuring out how to crash a program. Some of the greatest exploits started with the smallest of footholds.
You didn't include stdio.h and you probably wanted a \n at the end of your output string.
First born denied.
There are no karma whores, only moderation johns
In C you can implicitly declare functions! I demand at least half of your firstborn.
Palm trees and 8
"Once genius is submerged by bureaucracy, a nation is doomed to mediocrity. " -Richard Nixon
s/nation/project/
Even if it's bug free, that doesn't mean the software is designed to be easy or intuitive to use. There is often more money to be made from deliberate obfuscation. Every piece of Enterprise software I've used (particularly SAP) seems to follow this principle. And the huge aftermarket of expert consultants charging $150/hr for training and troubleshooting, seems to affirm it. Keep in mind those consultants are usually teaching the corporate experts, I'm not even touching the surface of end user training, which I'm experienced at being responsible for.
The real money comes from support packages and direct-line support levels (ie: silver, gold, platinum and such to maintain a 24h call centre). Let's ask a simple hypothetical: what if Windows was bulletproof and bug free? How would Microsoft make any money? If nobody had reason to fix a problem within 30 minutes like their job depends on it? Or even not upgrade to the next version because the last one works better? With XP and Vista we already know the answer to that.
The post talks about minor bugs being the ones that are hard to reach, but that's not necessarily the case. There was a piece of radiation-beam hardware used for cancer treatment (the Therac-25), that became a case study in engineering because of a literally fatal flaw. It was possible to mis-configure the machine so that it struck the patient with a much more powerful radiation beam than was normally allowed. The relevant point was, this situation only happened if the operator did some obscure, seemingly unlikely combination of actions that would result in a lead plate getting misaligned or something. Yet it happened at least six times, killing at least two people. The fact that the bug was caused by this particular sequence of actions made it that much harder to identify -- and it was not a minor bug worth ignoring.
Revive the Constitution.
Programs run just fine, or so some hope, with compiler warnings. Despite that, parent says his shop considers them to be bugs.
Oh to work in such a shop, where I don't have to sift through 74 warnings every build because someone decides that catch (Exception e) is fine even if you never take so much as a passing glance at what got put into variable e. Hey, dev manager, how about you just turn the fsckin' things off if no one cares?
The economic angle is a dangerous meme, because it lets incompetent management off the hook by allowing them to triage incompetence, and in particular, to ignore process failures, and the failure of team culture, both of which point back to management itself. We all know how well that worked out for NASA. This wasn't fixed until lives were lost.
If you're writing code for NASA, or say the medical industry, I'd agree. But 99.9% of the developers out their are writing code that nobody is ever going to die because of a bug. If you tried to apply the same standards to business software that you do to Nasa or medical software, nothing would ever get done.
As far as a "culture of incompetence" is concerned, and triaging bugs, I don't see the two as terribly connected. There's always going to be bugs, and some of them are going to be minor. This will happen even if you have the best developers in the world. Accepting bugs doesn't mean you give up on quality, or don't look to improve your process. It can actually be PART of that process. You assume all bugs come from "incompetence" or "process failure", or something wrong with the system. That's just silly. People aren't perfect. Hell, machines aren't perfect. The perfect is the enemy of the good, as the saying goes.
AccountKiller