The Economics of Perfect Software

An anonymous reader writes "This article takes the interesting perspective that leaving bugs in software is good — little ones, at least. This quote is particularly insightful: 'How do you know whether a bug is big or little? Think about who's going to hit it, and how mad they'll be when they do. If a user who goes through three levels of menus, opens an advanced configuration window, checks three checkboxes, and hits the 'A' key gets a weird error message for his trouble, that's a little bug. It's buried deep, and when the user hits it, he says 'huh,' clicks a button, and then goes on his merry way. If your program crashes on launch for a common setup, though, that's a big bug. Lots of people will hit it, and they will all be pissed. ... The cost of fixing all the bugs in your program and then being sure you fixed them all is way too high compared to the cost of having a few users hit some bugs they won't care about."

Comments

How is this news?

[Dragoniz3r]

Diminishing returns applies to programming too... big surprise...

Re:How is this news?

[siride]

Then I would say that's a big bug, so it should be fixed.

Re:How is this news?

[obarel]

I hope you're not serious.

The first problem is that it doesn't guarantee anything. There are very few software companies that close down or get rid of people because of small quirky bugs. You're not going to lose your job simply because you write perfect code either...

The second problem is that if you do keep your job, it's only to maintain the same piece of code. Aren't there more exciting things to do than to fix the same bugs you left in some spaghetti code that only you can understand? Isn't it better to write good solid readable code and move on to more interesting problems than off-by-one bugs you left on purpose?

And the third problem is that you can never justify that to your boss: "You've got to keep me because the code is terrible and only I can maintain it". The easiest thing would be to cut the losses and send you free, only to bring someone who can write some code that isn't an expensive liability.

Sadly, you're not the only person that writes bad code for "job security". So I hope your post gets a "Score: 5 Funny" and not a "Score: 2 Insightful"...

Re:How is this news?

[Vellmont]

What if that little UI glitch gives remote root?

Then you're an idiot who didn't separate your software properly into layers and should learn, or get out.

I understand your example, but the ultimate fear you're expressing about "the unknown" isn't warranted. Software is complex and can lead to unexpected problems. But worrying about every single minor bug possibly being a major one is just silly if you understand how the system interacts with itself. That's why good software is developed in layers. If you've really got a system where a UI glitch leads to a root exploit, time to throw away the whole application and start over.

Re:How is this news?

[spongman]

yeah, there's a also a mistaken suggestion that there's a correlation between the severity of the bug and the cost of fixing it.

in general there isn't.

you might have a crashing, data-destroying bug that's trivial to fix. or you might have a minor UI annoyance that would require a complete re-write in order to fix, possibly introducing a whole slew of new bugs.

unless you have a deep understanding of the architecture of the application, there's absolutely no way to judge which class a given bug is in.

Oh Please

[WrongSizeGlass]

The reason that every piece of software ships with bugs is because:
* It's created by people
* The programmers, testers, etc can never find all the bugs
* It's gotta get out the door so you can pay your programmers, testers, etc
* All of the above

There is a matter of pride with individual programmers, small groups, and most open source programming projects. We'd all love to be able to crow about shipping a bug free project/product. We'd do it if we could ... and someday we will.

Re:Oh Please

[zappepcs]

Exactly. Most coders have to use the code they write. Leaving bugs is not something you do if you know they are there. Sure some people do that, but not the ones that want to get it right - at least for their own use. This is why some open source software is not infinitely usable - it did what the original author needed, and was not intended particularly to solve everyone's silly problems. The example of the 'A' key is probably something that was going to work soon, but the need for it never materialized and someone forgot to disable the code that checks for the letter 'A' - and since it was not implemented, nobody checked it.

The first time I had code in production, running 24/7 I was kind of upset that I didn't need to look at it everyday, then amazed that it was working so well. Then, months later when I did have to go look at why it seemed to be not working right, I learned how important documentation and comments are. At this point I stopped being amazed that it worked well, but that it worked at all. Bugs are not left on purpose, only when there is not time or resources to fix them.

Re:Oh Please

[WrongSizeGlass]

even as individuals with all of our time at our discretion we would not waste the rest of lives trying to get the bugs out of something as complex as modern software, so NO we won't do it SOMEDAY either.

We may not ever accomplish it, but that doesn't mean we won't stop trying. Bugs slip through no matter how much we test. One reason is because no matter how we may try to break it we can never imagine the utterly creative stupidity that a user will cast upon our software.

And as far as Skynet is concerned, it will be written by humans or by a system written by humans, so it's gonna have bugs, too (unless it was written by Deep Thought, but that's another story entirely).

Re:Oh Please

[Jurily]

Leaving bugs is not something you do if you know they are there.

Depends on how much of the code you have to refactor to get rid of it, how many users are annoyed at that particular bug in the previous release, and how many new users you'd get by implementing the most asked-for feature instead.

You probably won't fix the bug where your text editor crashes if the user tries to open a file larger than 32 Gb first, if they've been also nagging you for global replace functionality.

Well, sure, if you have precognition

[Geoffrey.landis]

Well, sure, except that this assumes that you are PERFECT in your ability to predict the effect of a bug. And if you're not, that bug that you think will only happen in some situation that's vastly improbably will, in fact, actually hit at exactly the WORST possible time, because maybe that key sequence gets used in some extremely important operation that you hadn't realized your software was going to be used for. Or maybe that bug is ALSO triggered by some different sequence that you weren't quite prescient enough to realize would be common.

Dangerous approach

[mmmmbeer]

This sounds a lot like the "Good Enough Software" policies that certain companies used to have. It's true that having a few minor bugs is acceptable; in fact, it's pretty much inevitable. But you should expect to have bugs that you haven't even noticed at the time you ship your software. If you start ignoring the bugs you do know about, you increase the total number considerably. From there, it's a slippery slope. Pretty soon, you'll be answering bug reports with a chorus of "Minor, leave it," and your code will be riddled with tons of minor bugs. Your users will go from "huh" to "what?" to "dammit" to "f*ck this!" Your software will look sloppy and unprofessional, and even if none of the bugs are showstoppers, your customers will be looking for alternatives.

Re:Lets look at it another way

[sopssa]

Quote from the site:

Consider these stats : the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors.

So yes, they have errors too and their software isn't perfect nor bug-free. But their acceptable cost versus bug-free threshold is just a lot more higher than usual software. Exactly what the summary and article is about.

sometimes crash == possible exploit

[Eponymous Coward]

It isn't always easy to judge the severity of bugs. Exploits often grow from a black hat figuring out how to crash a program. Some of the greatest exploits started with the smallest of footholds.

Re:Wrong name

[The Clockwork Troll]

You didn't include stdio.h and you probably wanted a \n at the end of your output string.

First born denied.

Re:probably good idea; definitely bad example

[WrongSizeGlass]

The only way of eliminating those dark corners is by eliminating them.

And today's Yogi Berra award goes to hedwards. We'd like to thank everyone who participated for participating and the winner for winning.

Re:Wrong name

[betterunixthanunix]

In C you can implicitly declare functions! I demand at least half of your firstborn.

mediocrity

[electrogeist]

"Once genius is submerged by bureaucracy, a nation is doomed to mediocrity. " -Richard Nixon

s/nation/project/

Re:probably good idea; definitely bad example

[pem]

When I wrote:

When I'm trying very hard to make a program do what I want it to, the more hoops I have to jump through for every iteration of trying to make it work, the madder I get.

then you replied:

You must be a Windows user or haven't updated to an Apple OS past the old world Macs.

I have been racking my brain for the last half hour, trying to figure out what I wrote that bothered you so much that you felt compelled to resort to this sort of name-calling, but I can't figure it out, so expect a communication from my lawyer demanding compensation for this terrible libel you have committed about my computing practices in front of the entire slashdot community.

No bugs isn't a perfect product.

[ArundelCastle]

Even if it's bug free, that doesn't mean the software is designed to be easy or intuitive to use. There is often more money to be made from deliberate obfuscation. Every piece of Enterprise software I've used (particularly SAP) seems to follow this principle. And the huge aftermarket of expert consultants charging $150/hr for training and troubleshooting, seems to affirm it. Keep in mind those consultants are usually teaching the corporate experts, I'm not even touching the surface of end user training, which I'm experienced at being responsible for.

The real money comes from support packages and direct-line support levels (ie: silver, gold, platinum and such to maintain a 24h call centre). Let's ask a simple hypothetical: what if Windows was bulletproof and bug free? How would Microsoft make any money? If nobody had reason to fix a problem within 30 minutes like their job depends on it? Or even not upgrade to the next version because the last one works better? With XP and Vista we already know the answer to that.

Obscure bugs can be fatal ones.

[Garrett Fox]

The post talks about minor bugs being the ones that are hard to reach, but that's not necessarily the case. There was a piece of radiation-beam hardware used for cancer treatment (the Therac-25), that became a case study in engineering because of a literally fatal flaw. It was possible to mis-configure the machine so that it struck the patient with a much more powerful radiation beam than was normally allowed. The relevant point was, this situation only happened if the operator did some obscure, seemingly unlikely combination of actions that would result in a lead plate getting misaligned or something. Yet it happened at least six times, killing at least two people. The fact that the bug was caused by this particular sequence of actions made it that much harder to identify -- and it was not a minor bug worth ignoring.