Slashdot Mirror
Why Buggy Software Gets Shipped
astonishedelf writes to mention an article in the Guardian about the hard reality of why buggy code is sold on retail shelves. From the article: "The world's six billion people can be divided into two groups: group one, who know why every good software company ships products with known bugs; and group two, who don't. Those in group 1 tend to forget what life was like before our youthful optimism was spoiled by reality. Sometimes we encounter a person in group two, a new hire on the team or a customer, who is shocked that any software company would ship a product before every last bug is fixed. Every time Microsoft releases a version of Windows, stories are written about how the open bug count is a five-digit number. People in group two find that interesting. But if you are a software developer, you need to get into group one, where I am."
Anyway, I do agree with the author for the most part (its all pretty 101 risk assessment stuff). It is inevitable that software will have bugs in it (especially commercial software shipped to a schedule). This is not really news tho'.
What I would like to see is some vendor honesty. How about making a list of known bugs available to your customer prior to purchase? (I know, I know, fairly warning a customer is madness, etc etc).
There are shills on slashdot. Apparently, I'm one of them.
Theoretically, there is no language that is more or less prone to bugs than any other language as understood in Turing Completeness. Without delving too much into this, it simply states that all languages emulate a Turing machine to some degree and therefore should be capable of everything a Turing machine is capable of (although I don't think this says anything about time/space efficiency). One language may be better supported than another or have more libraries but we are going to assume these issues to be irrelevent to our discussion on applications--let us look as all applications being written in the same low level language that your computer directly understands. I don't want to compare architectures either, let us assume they are equivalently prone to bugs and are basic Turing machines.
If we think about a binary executable program (machine language consisting of ones & zeros) then we must recognize that the program's memory space has many many states. Open up your favorite word editor. Type in a sentence you're thinking about. Highlight part of it and bold it. Highlight a different part and hit escape seven times. Do you think that this scenario was tested?
My point is that it is an impossible herculean task for the developers to test any application in every state. They can come close and smart software design can leave an easier job for the testers but it will never be completely tested.
I would define the term bug as "undesired behavior" and I suggest they be thought of in this manner. I will also make the statement that no piece of software can be garunteed to be free of undesired behavior due primarily to the above analysis of them being amazingly complex machines with a large state space. To take a stab at it mathematically, this browser (Firefox) is operating with a 48,604 Kb memory footprint (I have many tabs open). That's 49770496 bytes or 398163968 bits. Each bit can be on or off meaning that for the amount of memory my browser occupies now, there are 2^398163968 different possible states for any similar sized application running. Now, to add even more complexity, that state space adjusts according to what the application requires for memory.
As our applications become more bloated, the situation only worsens. That is why development phases are either getting longer or requiring larger teams from the beginning of the project (mythical man month noted). At what point does the testing phase end? Hopefully never. Hopefully your software that you acquire is supported until the end of time
If you're wondering how companies can ship software with bugs, you should be wondering how is it possible not to ship software with bugs. You should also understand that there are rigorous standards and practices implemented along the way to prevent the most devestating bugs from escaping. If the company making the software has a history of failing in extinguishing the most glaring of errors, simply stop purchasing their software or demand the support you need.
My work here is dung.
And do we really need that much whitespace on a news page? I know about that whole '10 words per line' usability mantra, but it looks fucking ridiculous. Why can't all the other website owners just think exactly like me?
Wow, look at all that rebuking. Do I win Slashdot?
(IAJAFSS (I Am Just Another Fucking Smartass Student))
The argument about the enormous bug count in Windows isn't really about every last bug being fixed. The article fails to address a separate question: whether you're allowing the public to do your beta testing for you.
The idea of QC/testing/beta/whatever the heck you want to call it is that you get as many bugs as you can fix while accepting the ones that will remain behind. That's absolutely correct. However, there are companies - like Microsoft - that are notorious for either being sloppy and not getting bugs they should have, or just straight up not caring at all and rushing a product to market that legitimately shouldn't be there.
The argument can even be extended to good coding practices, like worrying about security fron start to finish rather than after you've entered beta (another well known Microsoft flaw, though they're getting better at it). That reduces the number of bugs to begin with, which in turn gives a better product.
ACs are modded -6. I don't read you, I don't mod you, I don't see you. Don't like it? Don't be a coward.
In many cases, the customer *needs* the software now, bugs be damned. If you don't, your company goes under.
Help kill corporate productivity!
Managers.
Specifically, managers who don't know enough about the project (whatever it may be) and make unreasonable promises to their superiors about shipping dates. It seems that the way most businesses are set up reward managers who complete projects on time or early, rather than the quality of the product. As a result, managers tend to rush development teams through their tasks and the end result is a buggy release. And a manager with a bonus check.
If software shops could change their focus to creating a better release product but with a flexible time schedule...say for instance, rewarding managers for fewer bug reports and service calls rather than completion dates, you'd have an entirely different picture.
Weaselmancer
rediculous.
Regardless of the nature of the software development team, the nature of competition remains the same.
Stagnation is costly - delaying a product launch drives people to the alternatives (both due to the alternatives updating faster, and due to the lack of progress seen by the outside world).
Of course, buggy software is costly in terms of reputation, as well, so the end question becomes at what point will the delaying of the release cost us more market share then the remaining bugs will.
Unfortunate from a purists eyes, but it's just the way things go.
We buy buggy cars, houses, and anything else you can think. Nothing with the aim of perfection *ever* gets done.
So what's the big deal?
I understand shipping something like bad tires that will eventually kill people should not be done, but anything that does not cause harm or major finantial distress should just be dealt with during the normal lifetime of a product.
I am an embedded systems developer. We take care of the functionality problems before shipping and work on the corner cases as we move along.
There is no way a group of people, doesn't matter how large, can think on every possible problem that can occur.
Show me one thing that's man made and that's perfect and I will eat my shoes.
-later!
Because, by and large, no one gets killed when commercial software crashes.
In those cases where it does; e.g. medical/aviation software, usually embedded people take the time. If aviation software designers cut the same corners (w.r.t. bugs vs. features) that office software designers do, planes would fall out of the air and people would die. So they write well engineered software, in well engineered, fault tolerant languages (lika Ada). (Yes, yes, Ariane, but thats the exception that proves the rule)
The real reason buggy software is shipped, is because buggy software is accepted by the market, and people will keep buying it, and continue to roll their eyes when it crashes, because they're completely inured to it, and many of them have reached the conclusion that its literally impossible to write robust, stable software.
It's not, but the profit margins are narrow, and no-one seems to mind (or rather they mind, but keep forking over their money anyway). So no-one bothers to.
Face if folks, we're enablers.
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
I agree that for a non-mission critical type of software, bugs are acceptable. As long as there are workarounds (e.g. avoid doing things that cause the bug to occur), it would be ok.
However, for mission critical software such as medical devices that deals with raditions output or heck even slot machines, bugs are unacceptable.
A good example is the Therac 25 where due to bugs, it actually injured people. http://www.ganssle.com/articles/disaster.htm
I hope the poster of this article or anyone who is in group 1 will never work on any mission critical software.
Look at Vista. Everyone is complaining about it not shipping on time. I have yet to hear anyone say. It is a good thing that Microsoft is fixing all those bugs.
Product ships late because of bug fixes. Why is it taking so long.
Product ships on time with bugs. Why didn't you fix the bugs before shipping.
You just can't win
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
Now, as to why bugs don't get quashed quickly:
I see each of these every day!
Logic is the beginning of reason, not the end of it.
Read: we got embraced and extended all to hell. What do to? Blame SQL! That's right, the language itself! It "isn't portable". Also blame users! "People who refuse to use SQL Server can't use Vault."
And here's some typical MS morality for you: "I'd probably even patent this algorithm even though, in principle, I believe software patents are fundamentally evil."
I don't expect bug-free software of any real complexity to be shipped often. But the examples are both interoperability problems, and not actual bugs. Looks like an excuse to marginalize the non-windows crowd. "...only affects users on non-Windows platforms, a rather small percentage of our user base."
My turnips listen for the soft cry of your love
It took a leaked Microsoft memo to find out Windows 2000 shipped with 65,000 bugs. Even the author of the memo wrote, ""How many of you would spend $500 on a piece of software with over 63,000 potential known defects?"
The problem is with a number that large, no matter how small the proportion is to code size, the backlash would be huge. No potential customer could hear that number and then actually want to buy a copy. I believe they should disclose as much information as possible. But from their perspective no amount of marketing could make up for the negative impact of disclosure.
Developers: We can use your help.
That's foolish. There are bugs in every project of every size. Including bridges. And skyscrapers. Remember the Tacoma Narrows Bridge?
Normally, those bugs have low Severity or Frequency (or both). Sometimes they have catastrophic severity.
Did you know that the twin towers were built to withstand a direct impact from a 707?
Bugs are a fact of life. They follow from the mantra 'nothing is perfect.'
I currently have no clever signature witicism to add here.
No dude, you're wrong. I suppose you can believe that with sufficient abstraction, you're right, but you're not. All that formal systems theory and Turing business is great talking about abstract systems running abstract algorithms, but such discussions have zero to say about anything having to do with HUMAN error, which is what we're talking about here.
I've probably spent about equal time writing C and writing in higher level languages, and I can promise that I make fewer errors in higher level languages, doing equal tasks. I think anyone with a lot of code under their belts can make similar statements. The closer to the machine a language makes you work, the harder it is to keep higher level details in the back of your head. In a high-level language, you're much less likely to make a low-level error (and any you make will almost certainly be caught by a warnings mode on the compiler, and this leaves you to keep more of your neurons working on, for instance, keeping your database and its wrapper classes working together correctly -- a task that is, perhaps, a simple afternoon's work in Python, Perl, Ruby etc. two days in C# or Java, and a week of hair-pulling in C... and well... I doubt such a thing has ever been done in assembler.
Anyway, drop the semantic B.S. this is a debate about practicalities.
In Capitalist America, bank robs you!
Why do you dismiss a complaint which speaks to the very heart of the problem? A large class of bugs simply would not exist were a different language used. This is not pie in the sky stuff; it's a real phenomenon.
If one language is less error-prone than another, then an application written in that language will have less bugs.
If an error-prone language is being used to write software, then this surely has to be a reason why buggy software gets shipped. Why are you dismissing people who complain about error-prone languages?
Be true and faithful like your dog; but don't eat vomit like your dog
1) All software has bugs. ALL SOFTWARE!
If you think your software is bug free then either:
a) You are clueless, hopelessly naive and inexperienced.
You may not even know what a bug is. Bugs range from cosmetic bugs such as mispellings in text labels, or the wrong color for a panel, to a bug that causes data corruption or system failure - but they are all bugs - including not meeting the user's expectation of how something should work.
Moreover, almost all software uses third party libs or application modules - ranging from the operating system (and all of its modules), to system libs to language libs, to special purpose libs (such as XML parsers, etc.) that you have little to no control over and certainly don't have the time to test or examine. You just take it on faith that these do not have bugs - but they do. If they are part of your application or you have not found and worked around these bugs, then your app has a bug - and if you think every OS and third party lib is bug free then I have to wonder what planet, nay what universe you are from.
or
b) You (or a designated party) have not tested your software properly.
Most developers are clueless when it comes to testing, and even if they had a clue (like me - I am a developer, but I spent almost 8 years professionally testing mission critical apps, including medical software) the developer is not the best person to test their own software - that job is best done by a professional and experienced tester who has not written the software - preferably someone who has not tested your software before.
2) There is no way to completely test any software application - no matter how simple - you can't even approach it, or even *begin* to approach complete testing of software. Even automation can not test every possible data input with every possible logic path. There just isn't enough time:
Testing a very simple program which just adds two 32 bit integers can take years - assuming you covered just the possible data inputs, and assuming you limited the input to integer numbers, not to mention alpha chars and floats - do the math (at 1 billion inputs per second [an extremely optimistic assumption], it would take over 500 years to test all possible inputs of adding two 32 bit integers).
That is just the possible integer inputs, not every possible logic path for every possible data input - and that is about as trivial a program as you could possibly make.
Most testing concentrates around previously known conditions which caused previous bugs (to make sure they don't resurface) and around other inputs/conditions/scenarios which are most likely to find a bug. The art of Software QA/testing is not to prove that there are no bugs, but to find the bugs that we know are there in all software - with priority given to "serious" bugs (what is "serious" depends on the context).
3) The severity or risk of any given bug is highly subjective and contextual. In some contexts a program that crashes routinely is preferable to a program that gives the wrong answer. In other contexts the reverse is true. In some programs (minesweeper?), either crashing or wrong answers doesn't mean a whole lot except that the user may go on to some other program.
Saying that Windows has thousands of bugs doesn't mean a whole lot until you start categorizing the bugs and analyzing the risk vs. benefit of the bug vs. the OS - and that depends a lot on the intended use.
If you don't want to use software that has bugs, better get out the pencil and paper.
The job of developers, QA staff and project managers is to weigh all of these factors and to do the best job we can given the human resources, time and money we have. No software is ever perfect or ever will be. Deal with it.
I share your vision of hope for the future. But I would first like to digress for a moment on your statement before fleshing out how I DO agree with you, too. (BTW it is currently 4 AM so I apologize if this rambles a bit. I've tried to go back and edit it to make it clearer, but I seem to keep making mistakes. That in itself seems apropos to this discussion. <grin>)
In my experience, it's more like: the customers get what they asked for and then find out they did NOT get what they WANTED. The problem is that the customers do not understand software, the environment the software runs in (hardware, political, and legal), what is possible, and what has never been done before. It's more often the case that "they will know it when they see it, but they can't really tell you what THAT is, exactly, before hand." Further, because they do not know what is and is not possible, they don't understand the ramifications of their choices. Lastly, there is a HUGE difference between research and development. It's one thing to code a one-off of something you've done many times before; it's quite another to do something completely new, and get it right the first time. As more and more people become computer literate, and gain first-hand experience on using software, I am cautiously optimistic that this disconnect will diminish over time.
Helmets - As an example, I remember watching some old footage and was amazed to see that professional hockey players did not wear helmets. Now it sure likes ALL players wear them. Why? We learned the added inconveniece and expense was worth it. I've worked on development projects where there was no allowance (i.e. time and money) set aside for anything but the barest amount of testing. Now I am increasingly seeing that built into development schedules, as a matter of course. Granted, in some cases this testing would be analagous to a hockey player wearing only a LEATHER helmet (instead of today's high-impact plastic) but it shows progress and gives me hope for the future. I welcome the day when testing and quality assurance are an integral part of EVERY development effort instead of a rare luxury. The benefit is that libraries of well-documented and thoroughly-tested code will become increasingly available. AND the methodology to USE them PROPERLY, (i.e. SAFELY!)
It's thanks to developers' incessant optimism, I believe, that we have our current problems, and also the seeds for the solution. Please, don't get me wrong on this one. IIRC, it was Tesla who said that "If I had known it was impossible, I wouldn't have done it." We create doftware to do things which have NEVER been done before, ever. Even in the face of statements like: "That'll NEVER work!". The response? "Oh yeah? Hmmmmm. Wait a minute! If I, hmmm, and then... AHA! I think that just might work!" And on nothing more than a hunch, a hope, and a blind ignorance of just how many hours of debugging it will entail, we regularly go off and do something absolutely incredible. I have gained much inspiration from a quote by Mark Twain:
In tandem with this one by Thomas A. Edison:
It's that thrill which drives so much inspiration and pers