Slashdot Mirror
Why (Most) Software is so Bad
Rivard was one of several to point out that
MSNBC
says software sucks.
My opinion is that in software fields where the monetary gap between market-leader and second-place is large, we should expect bad software. Good design, good execution, good debugging all take time, but users can't see under the hood -- and wherever information is scarce or not readily traded among consumers, the free market bogs down. (Note what the article says about McAfee VirusScan.) So companies that don't plan on releasing a crummy 1.0 and fixing it later go under. That's just the way some markets work; if you're a coder or engineer who doesn't like that, find yourself a job in a niche without that monetary gap. Anyway, the really stunning thing is that, of all the media outlets, MSNBC points out that just one of Microsoft's poor design decisions has cost consumers $8.75 billion, and wonders why nobody has
sued.
Update: 06/18 14:10 GMT by J : Readers point out the story is a reprint
from Technology Review
(one of the few good magazines I get -- but this issue hasn't arrived yet :).
Rivard continued his writeup with an interesting point of view, saying that while we all know software sucks, we just accept it:
"Even though 'plenty of reviewers, pundits, hackers and other outsiders' will point out problems, often intentionally left in the product, no one has brought a liability suit against the makers of the known-to-be-vitiated product -- because the software gestapo (the End User License Agreement) has been 'able to avoid product liability litigation partly because software licenses force customers into arbitration' of poorly designed pith."There is a light at the end of the tunnel, believe it or not, and it's Bill Gates. Microsoft suspended coding for two months to seminar on bugs and how to fix them. Gates told his employees he wanted to make 'reliable and secure' software Microsoft's 'highest priority.' If you don't buy Gates' ad-hocking promises of redemption there are other solutions, like creating a programming language that forces good code; going back to the days of intense peer-review, instead of relying on compilers; and intense planning, past the bungling paradigm of the bar napkin."
If you look through the Slashdot archives, MSNBC has historically been one of the loudest mainstream (read: not theregister.co.uk) MSFT-criticisers. This is typical of them.
If you don't buy Gates' ad-hocking promises of redemption there are other solutions, like creating a programming language that forces good code; going back to the days of intense peer-review, instead of relying on compilers; and intense planning, past the bungling paradigm of the bar napkin."
"There's no silver bullet" -Fred Brooks, a long time ago.
I disagree; take a look at other industries. Some of the highest-quality products are produced by the tiny, niche-market manufacturers. The best cigars in the world are not from Phillip-Morris. The finest cuisine on your block is not the mega-corporation with the giant yellow 'M'. The most accurate watches don't come from time-giant Timex. The finest literature on the bookshelf isn't necessarily from the biggest publisher.
Software is hard to compare to other products. It's intangible. It's complex. There are a million different ways to get "Hello World" onto your screen. Maybe that's the problem?
Software is also relatively young. Not just in the sense that it has only been around for 40/50 years, but in the sense that the tools evolve so quickly, that it almost seems like every few years, you're working in a completely different manner than before. And it takes time to become familiar and comfortable with the paradigms associated with your environment. Just when you've got a system worked out, everything changes again.
Before, the customer wanted a library of CGI scripts to run their e-commerce website. Now that've got a handle on scripting design patterns, the customer wants EJBs. By the time you learn what to expect to go wrong with EJBs, the customer will want a cell-phone interface to their inventory.
Sometimes, I'm amazed software works at all.
Like woodworking? Build your own picture frames.
Software's so bad because it's still handcrafted, and the interchangable parts don't. Cars sucked too when when they were done the same way. OSS isn't the solution. The solution is for Computer Engineering to someday become as rigorous as other areas of Engineering.
Maybe the state's highest function is to grind out insoluble problems. (Zelazny, Hall of Mirrors)
"like creating a programming language that forces good code"? I doubt it. It's my impression that even where languages do enforce various checks - perl in taint mode, for example - there are always ways around it or issues that the code can't check. After all, a significant proportion of "bugs" aren't faulty code per se, but correct code that is based upon incomplete or incorect assumptions and design.
The fact is, there are well-researched, well documented means of achieving quality control in development. They're simply not practiced by many because the implementation overheads are just too great. Many coders don't plan their code adequately, don't document their code adequately, and don't test their code adequately. However, this isn't necessarily a 'bad thing' -- frankly, in many circumstances it's just not that important if a few bugs sneak through which can be patched at a later date. As is always the case in such matters, security/stability are sacrificed for convenience and speed of development. And that's not necessarily a bad thing, especially in an industry where a product can be superseded or otherwise rendered obsolete even before its bugs become too much of a problem. (Although I'd admit that there are dangers in taking this for granted, as seen with the y2k issues..)
We should expect bad code because we expect code that rolls out quickly and at a low budget. We should expect bad code because most coders don't want to spend their time testing and documenting, and because most companies don't want to spend money on dedicated testers or on implementing rigid development processes. And we should expect bad code because even bad code can work 'well enough' to keep most people happy most of the time.
This article is out of the July/August MIT Technology Review. My copy of the magazine proves their point in an ironic fashion.
./ probably could--isolate the bug in ten minutes given the source. Likely it assumes that either the first city is valid or that the likelihood of two cities beginning with the same two letters in the same zip code is too small to consider.
The zip code I live in covers two cities, let's call them Appleville (tiny village) and Apricotland (large, sprawling concrete wasteland.) I live in Apricotland which is asciibetically second (based on the third letter.) Note that the first two letters are the same. MIT TR's mailing system lists me as living in Appleville. Why would it assume that zip code 12345 is the smaller village instead of the sprawling metropolis?
Yup. Buggy software. I could--as anyone reading
The joke's on you, MIT.
To me the largest factor in determining whether the 1.0 release of something will be acceptable is time.
Given enough time for proper design and testing, a 1.0 release could be acceptable, but companies hiring consultants do not want to pay for the time, and companies that produce software for the general public have to rush products to market to beat company X, whose competing product is due for release, and (more importantly?) they need to please their stockholders.
Once in a while you get a Mozilla-type thing that takes forever, but puts out somthing worthwhile with 1.0.
The revolution will be televised. Blackout restrictions apply.
I believe that we are seeing the increasing development of issues that were first coming to light in the late 90's, when the Y2K problem was starting to be recognized as a problem for society at large. As the economy comes to rely more heavily on properly functioning software based systems, the push to make the designers and constructers of these systems responsible for them. Too often in the community the various proposed approaches to design and implementation (now being expressed as the battle between methodologists and agile methods proponents) obscure the basic need to build the right systems, in the best known fashion. This is, at heart, being willing to accept responsibility for what has been promised. If the community will not do this, then legal sanctions will be imposed which force the issue.
1. Smart (or dumb) guys form startup around good idea. Version 1.0 gets written in a frenzy of caffeine and beer, riddled with bugs because it has to be delivered before the money runs out.
2. Mistakes made in version 1 are sworn off as version 2 is designed. Version 2 is built by the swell of 2nd-generation coders, hired as fast as possible and sent to work unsupervised by the overworked 1st-generation engineers.
3. Version 2 is delivered with all the good ideas on the surface, but implemented by less-than-excellent coders.
4. Widespread adoption funds much additional hiring. Anything vaguely mammalian is hired to fix bugs and work on new features and new products. Most 1st-generation engineers leave with their money. Product design and development is run by people who don't know what they're doing.
Just like cars, refrigerators and houses, the quality of what you purchase is not perfect. And you should not expect software quality to be perfect any more than you expect your car to be perfect. The more money you pay, the more quality you get. It's an asymptotic approach where increasing quality costs a lot more money.
Just because of a long-standing close relationship with mathematics, people buy into the idea that software should be as ironclad as a theorem.
It just ain't so.
Real software becomes complicated, much in the same way that PDE's governing real physical phenomena become complicated. Small pieces of software can be verified for correctness pretty easily, but complicated interacting pieces of software rapidly will exceed your resources to check for behavior under all circumstances.
There are so many ways for it to behave and misbehave that closure is a process of endurance, enumerating and testing as many options as possible under as many likely and important conditions that you can think of.
At some point, you decide that you've reached an acceptable level of quality (does our regression tests and stays up and running for 99.99% of the time for the sample of typical users) and release the product.
[Bill Gates is another matter altogether. I think he's responsible for some distortion of the software marketplace and, thereby, responsible for software not needing to be up to higher standards. That said, however, even without his influence, software will never be up to perfect standards.]
"Provided by the management for your protection."
I'm against this notion that people want good software. As a software consultant, who has been involved one way or another in a varienty of fields, I can say with authority that companies want software *cheap* not *reliable*. When companies want reliable and secure they pay for reliable and secure. Companies have an absolutely unrealistic expectations of what can be done in a short and cheap development cycle, yet that's what they demand from their consultants. I have an client (a rather large health care provider) that still hasn't changed the default password to the publically available administration section of their system... which is more common than one would care to think. This is a testiment to how high companies consider security important.
Check out my podcast: DreamStation.cc Video Game Show
Insightful, huh?
How about some basic economics:
Value = Benefit - Cost.
If, indeed, Code Red cost $8.75billion (and I'd like to see the analysis that arrived at that figure), that cost was incurred in the process of using Outlook. Presumably, the consumer derived some benefit from using Outlook, at least in their judgement they did.
In any product liability lawsuit that does not result in human death or injury, one would have to account for all aspects of the economic equation.
If humans are mostly water, and beer is mostly water, then humans must be mostly beer.
So that's all fine and dandy, but it's not like you can just take one from each column and have something that makes sense. For example, were bugs in an operating system due to inefficient code that would be fixed by component-based design with an eye towards cost effectiveness? Well, uhhh, maybe, I think.
It didn't help that so many of the people quoted had no idea what they were talking about, and the ones who did had their quotes taken so far out of context that they made no sense. It seems a lot of people who never worked at Microsoft know how Microsoft develops software. Oh well.
It would make more sense to talk about a particular class of software and bug and then discuss why it is there. E.g. why do Microsoft systems products have buffer overflows. Even then you would get a bunch of different answers.
- adam
P.S. Comment first posted by me on Techdirt.
Knowing a bit of how mass mailings work, specifically how you figure out who is where through zip codes, the actual city that gets printed on things mailed to you in that fasion is determined by checking the Post office database, usually through a program such as AccuZip.
Lots of times, the city the post office has you in isn't the city you actually live in, but it will get to you all the same, because the Post Office can't assign multiple municipalities to a single zip code. They probably picked the small town because it didn't have any other zip code, or whatever criteria they have. Don't blame the software for something that isn't it's fault. It's just doing a query based on the official database.
Those reasons probably are (a) first cost of purchase (b) competitive advantage to be gained by using something "good enough" sooner rather than something perfect later.
Which doesn't excuse the behaviour of software suppliers, but the question is more complex than it appears.
sPh
In other engineering disciplines, there is little room for error and your mistakes are readily apparent -- you screw up, and you'll probably wreck a few million dollars worth of equipment during testing or kill someone when the product ships. Software engineering, on the other hand, allows the mediocre on the team to hide behind truly talented individuals.
Code doesn't work? No big deal, just fiddle around and recompile. Still doesn't work? The other guy will probably fix it. Product's shipping and the problem still isn't fixed? Eh, who cares? It's not going to kill anyone, and nobody will notice.
It is precisely because no one can "see" the shoddy workmanship of programs that such behavior exists (and I'm not saying open source programs are immune to this -- patching a crappy open source program is akin to patching up a crappy car engine with duct tape and staples). All those slackers who couldn't code a binary tree to save their life and were generally mediocre college students are writing the software you'll be using. Their mediocrity is hidden by the anonymity and zero-accountability inherent in large software projects. Essentially, they're doing the same thing in the real world as they did in college -- slack off and hope nobody notices.
This, folks, is why colleges need to implement tougher CS curriculums. This is why you need to be able to write code on paper. This is why colleges insist that you don't collaborate on projects in lower-level CS classes. This is why there shouldn't be curves that allow mediocre students to graduate with a CS degree when they should have been weeded out during their freshmen or sophomore year.
I'm almost finished with my CS degree, and it's depressing, even at this level, how few students there are who really have an understanding of and interest in computer science. Most of the students enrolled in CS at my school have never coded before college. The most popular reason for choosing CS? "I used to make webpages, and I figured it would be easy." No joke. Of course, this sort of behavior isn't strictly limited to CS -- I've met more than a few EE students (freshmen, admittedly) who couldn't identify basic electrical components, didn't know Ohm's Law, and chose EE because "I used to take apart telephones and put them back together, so I think electronics is fun." Inevitably though, these clueless students who enroll in other engineering disciplines tend to migrate over to CS since it's not as "balls-to-the-wall" difficult as say, EE or ME.
In short, software "sucks" because our colleges are allowing mediocre students to slip through the cracks into the professional world. Personally, I feel that the CS curriculums should tighten up a bit and put more emphasis on solo projects (and moderately challenging ones, at that) and writing code on paper at the lower levels. A message has to be sent to those students who want to be engineers but don't really have what it takes that CS is NOT an "easy major". Perhaps then it might not be the haven for slackers that it has become.
As a developer, I've seen this happen first hand, but in most cases to blame the engineer is incorrect. In my case, pointy-haired boss types pull a deadline out of their asses and expect the Engineering team to hit the deadline. What happens then? Well here's one quick anecdote.
The company I used to work for decided in February of last year that we would ship version X.2 (where X is greater then 4) of our software package at the end of June. Only problem was, there were still approximately 10,000 known bugs, and the only reason to ship in June was to announce the release at the company sponsored trade show. That, and there wasn't a single engineer asked for an esitmate, unless you count the moron owner who thought he was a developer and was notorious for being way too aggressive in estimates (among other bad habits, like not even building his code before checking it in to the source control system).
Clearly, there was no chance in hell we were going to hit the deadline. Management's solution? Mandatory unpaid overtime. 15 hours a week. All requests for vacation were denied until the product shipped.
To make a long story short, 2 developers quit (myself being one of them), 2 were laid off because they didn't have a new version to sell because X.0 was such a piece of garbage for a lot of the same reasons.. and the company finally trotted out version X.2 to very little fanfare with 3000 known serious defects still not addressed.
And, like X.0 before it, it's a POS - a piece of shit. Engineers get burned out when they are overworked just like anyone else, and are forced under threat of termination to make poor decisions to keep their jobs. Management then oversees the creation of a poor product and then are completely oblivious that they are the root cause of the problem. Instead, the development department gets blamed and in this case the director got canned. (When really all he did was take his marching orders from the owner. No authority to make decisions, and yet they held him accountable.)
The truly sad thing is, all this can be avoided if management would just back off and let developers do their job without micromanaging and shoving their nose to the grindstone, spending the investment capital to pay your employees overtime when appropriate, staffing up your Quality Assurance department (5 Analysts testing code put out by 10 engineers is not enough by a long shot) and not releasing it until it's ready.
One excellent software release can make your company very rich. One poor software release can kill your company. (And in the case of the company I used to work for, I must admit I hope that happens. The world would be a better place without that third rate company run by a fourth rate pompous jack-ass developer who thinks he's above you just because he went to an Ivy League school, and you just went to the local University.)
And don't even ask me to name the company or the product, I won't, I still have close friends that work there. Even if I do wish they'd quit drinking the Kool-Aid..
6. Fixing even the shallow defects would break backwards compatibility and the customers all swear they will go to your competitors.
In the case of IIS exploits, for instance, it has nothing to do with it whatsoever.
free the mallocs!
Kombat's Law:
Before, the customer wanted a library of CGI scripts to run their e-commerce website. Now that've got a handle on scripting design patterns, the customer wants EJBs. By the time you learn what to expect to go wrong with EJBs, the customer will want a cell-phone interface to their inventory.
Observation on Kombat's Law:
Technologies become fashionable soon after they are introduced, usually when the libraries are still version 1.0 or one-point-epsilon.
Related observation on the software industry:
It's standard practise to push a buggy pile of kludges still in need of major debugging out the door to meet overly optimistic deadlines and call it version 1.0.
Sum conclusion of it all:
We're all fucked.
Those who like to use exciting new gizmos are even more fucked. Those with bosses who are attracted by shiny objects are the most fucked.
Software companies could build a rock solid word processor or PIM , even MS, but no one will pay for it. We all scream that the retail price of MS Office is far too much. The corporate world looks at the per seat cost of MS and say, "I wish we could switch to something less expensive." Every product has a price point and software has a very low price point.
The odd part of the problem is the marginal cost of production. As we all know, once the development is done it cost next to nothing to produce. Once a producer has a product that the market will pay for there is little incentive to keep droping millions into making it much better. We would all love to use perfect" software. If "perfect" software can only be done using "traditional" engineering approachs such as is used in aircraft design that means putting massive teams on a word processor. Each team does one function including the code, hand checking the machine output from the compilier, checking it's performance againsta massive design document. We all know the story of how this kind of effort almost sunk IBM. Computer science has progressed to the point where massive effort should be able to be implimented, but no one would be willing to pay for it.
In fact, I remember Gates several years ago bragging about how he prefers not to hire CS grads because they come out of school with too many limitations programmed into their brains.
Yeah, limitations, like how to write good code, how one should avoid side effects in functions, write black box functions, learn how to develop testing functions to push a full range of possible inputs to functions to test them, how to document properly, etc, etc.. You know, all that stuff that cuts down on the number of lines of code per day a programmer pumps out...
How does one test against every possible configuration of every possible computer that could conceivably run one's OS?
You design around it. 3rd party drivers are a constraint for MS. Drivers do not have to run in ring-0. Microsoft chose for drivers to run in ring-0 and we pay for it with crashes.
Joe
Joe Batt Solid Design
We were taught a ton of quality control, testing, and design in school. In fact, design and testing were taught at the major portion of software development. We spent most of our time designing and testing our code than we did writing it. We also did studies on the effects of poor programming and the responsibilities a programmer has in relation to his/her code. I can't speak for every CS program, but I know that these were the core aspects of mine.
You are right though, once I got a job in the real world I found that these virtues were lost in most companies, and the importiance of the training we received in CS classes were dismissed as academic jargon and a waste of time. I saw a lot of people developing software without even an engineering degree of any kind! And the notion that as long as your code isn't in a position to harm a person's life, so what if it has bugs, is rampent...
So I think the problem is not in the teaching, but that a lot (surely not all) of the employers dismiss the talents that their employees learned in school simply because they think the "real world" works differently and that their experience is more importiant that formal teaching.
Popular software isn't reliable, because reliability isn't the highest value. Compatability with a legacy is (e.g. you want to run this application under MS Windows?). Or cheapness (e.g. Do you want to be billed 2 hours of my time (very little testing) or 6 hours (more testing)?). Or having lots of features (Would you like a flight simulator with that spreadsheet?). Or something else.
When reliability is important, you can have it. But it will cost you. Most people consider the cost to be too high. That's why more people run bleeding-edge Linux, Windows, or MacOS, than OpenBSD.
And there's nothing wrong with that. You just have to accept/enjoy the power/responsible that goes with the choice that you made, instead of whining that someone else should have chosen for you.
Irresponsible XP users whining about XP, ispartcularly pathetic. Yeah right, you didn't know what you were getting into. You never heard of this "Microsoft" company before, so you just assumed that they valued reliability over other considerations. Discovering that it was crap primarily intended to play video games and keep MCSEs' jobs secure, was like a cold knife in the back -- such unexpected treachery!
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
"This is one of the better comments on this thread."
To me, these comments seem utterly out of touch with reality. I find bugs and insufficiencies in open source software. But generally open source software impresses me as an attempt to do a good job.
In contrast, Microsoft software seems just sloppy. For example, Microsoft's Internet Explorer has 18 unpatched security bugs (when this was written). These active security risks are different from the recent 15 that have already been fixed. This is sloppiness, not mistakes, and I don't find anything like it in the open source world.
When I have a problem with open source software, I find that I can get help. When I call Microsoft, I find that, usually, no one with whom I am allowed to talk knows any answers. Right now, for example, no one seems to know how to repair a new, Intel Motherboard, Windows XP installation that won't create a virtual memory paging file. It's buggy, and nothing can be done other than re-install the OS and all the applications.
If you find a big problem in open source software, chances are that you will communicate directly with the main authors. With Microsoft, I have not been able to get answers. This article says that the Psychic Friends Network is equally as good as Microsoft technical support: Microsoft Technical Support vs. The Psychic Friends Network The conclusion of the article seems reasonable considering my experience with Microsoft. Neither organization has useful answers, but The Psychic friends Network is more friendly and less expensive.
Quality is important, but how we use software is more important in our buying decision. They made a good point that given the plus side of the equation, most software is worth what you pay for it. Otherwise, people would not buy it.
Quality however is not free and the consumer must, in almost all business equations pay for it. In the case of dropping a Space Shuttle on Atlanta, our willingness to pay for additional quality controls is going to be much higher than testing a text editor.
Using the automotive analogy, I want my car to go 250,000 miles without any upkeep (oil changes, filters, etc.) The diffence in the engineering is that no one considers the auto example reasonable but they do with software for which they are paying 1/20th the price.
Indeed, it should be the market that decides, and not the courts : The market is the reason that North American automakers were forced to dramatically improve their quality (because the Japanese automakers were far ahead of them, and the market started voting for quality with their pocketbooks), and the markets are the reason why people pay a huge premium for IBM laptops. These capitalistic forces are amazingly powerful in getting what people really want, and the only time they fail to work is when someone with a pet issue tries to circumvent market-forces to get their own personal beliefs imposed on everyone through the legal system. It isn't surprizing to see the "sue em!" claims from many of those who already have forsook Microsoft software: These are the type of authoritarian people who believe that because it's not right for them, therefore it shouldn't be right for anyone else.
The irony is that most people, the average Joe and Jane out there, have shown that quality of software is very low on their list of purchasing criteria (which is how software like "ICQ" has never been non-beta...why bother ever calling it production code when millions of people will download it anyways). Given the choice, the market has stated time and time again that features will always trump quality, and that time to market often beats quality.
Sidenote: My own philosophy is that I spend about 90% of QA time code auditing and refactoring, and about 10% of the time doing runtime testing. I find that this is exactly the opposite of many organizations which push runtime testing as the method of evaluating code. To me runtime testing is no different than doing a compiler syntax check: It's an incredibly weak, time intensive, limited case way to assure the quality of your code, and should at most be used to evaluate tha the compiler and dependant third party code is of a good quality.