Slashdot Mirror
Why (Most) Software is so Bad
Rivard was one of several to point out that
MSNBC
says software sucks.
My opinion is that in software fields where the monetary gap between market-leader and second-place is large, we should expect bad software. Good design, good execution, good debugging all take time, but users can't see under the hood -- and wherever information is scarce or not readily traded among consumers, the free market bogs down. (Note what the article says about McAfee VirusScan.) So companies that don't plan on releasing a crummy 1.0 and fixing it later go under. That's just the way some markets work; if you're a coder or engineer who doesn't like that, find yourself a job in a niche without that monetary gap. Anyway, the really stunning thing is that, of all the media outlets, MSNBC points out that just one of Microsoft's poor design decisions has cost consumers $8.75 billion, and wonders why nobody has
sued.
Update: 06/18 14:10 GMT by J : Readers point out the story is a reprint
from Technology Review
(one of the few good magazines I get -- but this issue hasn't arrived yet :).
Rivard continued his writeup with an interesting point of view, saying that while we all know software sucks, we just accept it:
"Even though 'plenty of reviewers, pundits, hackers and other outsiders' will point out problems, often intentionally left in the product, no one has brought a liability suit against the makers of the known-to-be-vitiated product -- because the software gestapo (the End User License Agreement) has been 'able to avoid product liability litigation partly because software licenses force customers into arbitration' of poorly designed pith."There is a light at the end of the tunnel, believe it or not, and it's Bill Gates. Microsoft suspended coding for two months to seminar on bugs and how to fix them. Gates told his employees he wanted to make 'reliable and secure' software Microsoft's 'highest priority.' If you don't buy Gates' ad-hocking promises of redemption there are other solutions, like creating a programming language that forces good code; going back to the days of intense peer-review, instead of relying on compilers; and intense planning, past the bungling paradigm of the bar napkin."
Software's so bad because it's still handcrafted, and the interchangable parts don't. Cars sucked too when when they were done the same way. OSS isn't the solution. The solution is for Computer Engineering to someday become as rigorous as other areas of Engineering.
Maybe the state's highest function is to grind out insoluble problems. (Zelazny, Hall of Mirrors)
"like creating a programming language that forces good code"? I doubt it. It's my impression that even where languages do enforce various checks - perl in taint mode, for example - there are always ways around it or issues that the code can't check. After all, a significant proportion of "bugs" aren't faulty code per se, but correct code that is based upon incomplete or incorect assumptions and design.
The fact is, there are well-researched, well documented means of achieving quality control in development. They're simply not practiced by many because the implementation overheads are just too great. Many coders don't plan their code adequately, don't document their code adequately, and don't test their code adequately. However, this isn't necessarily a 'bad thing' -- frankly, in many circumstances it's just not that important if a few bugs sneak through which can be patched at a later date. As is always the case in such matters, security/stability are sacrificed for convenience and speed of development. And that's not necessarily a bad thing, especially in an industry where a product can be superseded or otherwise rendered obsolete even before its bugs become too much of a problem. (Although I'd admit that there are dangers in taking this for granted, as seen with the y2k issues..)
We should expect bad code because we expect code that rolls out quickly and at a low budget. We should expect bad code because most coders don't want to spend their time testing and documenting, and because most companies don't want to spend money on dedicated testers or on implementing rigid development processes. And we should expect bad code because even bad code can work 'well enough' to keep most people happy most of the time.
This article is out of the July/August MIT Technology Review. My copy of the magazine proves their point in an ironic fashion.
./ probably could--isolate the bug in ten minutes given the source. Likely it assumes that either the first city is valid or that the likelihood of two cities beginning with the same two letters in the same zip code is too small to consider.
The zip code I live in covers two cities, let's call them Appleville (tiny village) and Apricotland (large, sprawling concrete wasteland.) I live in Apricotland which is asciibetically second (based on the third letter.) Note that the first two letters are the same. MIT TR's mailing system lists me as living in Appleville. Why would it assume that zip code 12345 is the smaller village instead of the sprawling metropolis?
Yup. Buggy software. I could--as anyone reading
The joke's on you, MIT.
1. Smart (or dumb) guys form startup around good idea. Version 1.0 gets written in a frenzy of caffeine and beer, riddled with bugs because it has to be delivered before the money runs out.
2. Mistakes made in version 1 are sworn off as version 2 is designed. Version 2 is built by the swell of 2nd-generation coders, hired as fast as possible and sent to work unsupervised by the overworked 1st-generation engineers.
3. Version 2 is delivered with all the good ideas on the surface, but implemented by less-than-excellent coders.
4. Widespread adoption funds much additional hiring. Anything vaguely mammalian is hired to fix bugs and work on new features and new products. Most 1st-generation engineers leave with their money. Product design and development is run by people who don't know what they're doing.
Insightful, huh?
How about some basic economics:
Value = Benefit - Cost.
If, indeed, Code Red cost $8.75billion (and I'd like to see the analysis that arrived at that figure), that cost was incurred in the process of using Outlook. Presumably, the consumer derived some benefit from using Outlook, at least in their judgement they did.
In any product liability lawsuit that does not result in human death or injury, one would have to account for all aspects of the economic equation.
If humans are mostly water, and beer is mostly water, then humans must be mostly beer.
The key to the article is the last section, which talks about remedying the bad software situation, describing massive class-action lawsuits as "a bad idea whose time has come." MS knows that it could live through a class-action suit of this type. Would your favorite open-source project survive being sued back into the stone age? I think this article is an attempt to get public opinion stirred up to the point that UCITA laws - which include things like mandated warranties on software products -seem like a reasonable solution, and thus make life more difficult for MS's competition.
Knowing a bit of how mass mailings work, specifically how you figure out who is where through zip codes, the actual city that gets printed on things mailed to you in that fasion is determined by checking the Post office database, usually through a program such as AccuZip.
Lots of times, the city the post office has you in isn't the city you actually live in, but it will get to you all the same, because the Post Office can't assign multiple municipalities to a single zip code. They probably picked the small town because it didn't have any other zip code, or whatever criteria they have. Don't blame the software for something that isn't it's fault. It's just doing a query based on the official database.
But the refrigerator does exactly one thing, and the interfaces are perfectly standardized. It's not programmable, and it's nearly impossible to use incorrectly. The perfect refrigerator is not a moving target. And the money you put into design and quality control in 1970 is still paying off now.
Hell, if software just had to do what it was doing in 1970, it would be more perfect than refrigerators.
In other engineering disciplines, there is little room for error and your mistakes are readily apparent -- you screw up, and you'll probably wreck a few million dollars worth of equipment during testing or kill someone when the product ships. Software engineering, on the other hand, allows the mediocre on the team to hide behind truly talented individuals.
Code doesn't work? No big deal, just fiddle around and recompile. Still doesn't work? The other guy will probably fix it. Product's shipping and the problem still isn't fixed? Eh, who cares? It's not going to kill anyone, and nobody will notice.
It is precisely because no one can "see" the shoddy workmanship of programs that such behavior exists (and I'm not saying open source programs are immune to this -- patching a crappy open source program is akin to patching up a crappy car engine with duct tape and staples). All those slackers who couldn't code a binary tree to save their life and were generally mediocre college students are writing the software you'll be using. Their mediocrity is hidden by the anonymity and zero-accountability inherent in large software projects. Essentially, they're doing the same thing in the real world as they did in college -- slack off and hope nobody notices.
This, folks, is why colleges need to implement tougher CS curriculums. This is why you need to be able to write code on paper. This is why colleges insist that you don't collaborate on projects in lower-level CS classes. This is why there shouldn't be curves that allow mediocre students to graduate with a CS degree when they should have been weeded out during their freshmen or sophomore year.
I'm almost finished with my CS degree, and it's depressing, even at this level, how few students there are who really have an understanding of and interest in computer science. Most of the students enrolled in CS at my school have never coded before college. The most popular reason for choosing CS? "I used to make webpages, and I figured it would be easy." No joke. Of course, this sort of behavior isn't strictly limited to CS -- I've met more than a few EE students (freshmen, admittedly) who couldn't identify basic electrical components, didn't know Ohm's Law, and chose EE because "I used to take apart telephones and put them back together, so I think electronics is fun." Inevitably though, these clueless students who enroll in other engineering disciplines tend to migrate over to CS since it's not as "balls-to-the-wall" difficult as say, EE or ME.
In short, software "sucks" because our colleges are allowing mediocre students to slip through the cracks into the professional world. Personally, I feel that the CS curriculums should tighten up a bit and put more emphasis on solo projects (and moderately challenging ones, at that) and writing code on paper at the lower levels. A message has to be sent to those students who want to be engineers but don't really have what it takes that CS is NOT an "easy major". Perhaps then it might not be the haven for slackers that it has become.
As a developer, I've seen this happen first hand, but in most cases to blame the engineer is incorrect. In my case, pointy-haired boss types pull a deadline out of their asses and expect the Engineering team to hit the deadline. What happens then? Well here's one quick anecdote.
The company I used to work for decided in February of last year that we would ship version X.2 (where X is greater then 4) of our software package at the end of June. Only problem was, there were still approximately 10,000 known bugs, and the only reason to ship in June was to announce the release at the company sponsored trade show. That, and there wasn't a single engineer asked for an esitmate, unless you count the moron owner who thought he was a developer and was notorious for being way too aggressive in estimates (among other bad habits, like not even building his code before checking it in to the source control system).
Clearly, there was no chance in hell we were going to hit the deadline. Management's solution? Mandatory unpaid overtime. 15 hours a week. All requests for vacation were denied until the product shipped.
To make a long story short, 2 developers quit (myself being one of them), 2 were laid off because they didn't have a new version to sell because X.0 was such a piece of garbage for a lot of the same reasons.. and the company finally trotted out version X.2 to very little fanfare with 3000 known serious defects still not addressed.
And, like X.0 before it, it's a POS - a piece of shit. Engineers get burned out when they are overworked just like anyone else, and are forced under threat of termination to make poor decisions to keep their jobs. Management then oversees the creation of a poor product and then are completely oblivious that they are the root cause of the problem. Instead, the development department gets blamed and in this case the director got canned. (When really all he did was take his marching orders from the owner. No authority to make decisions, and yet they held him accountable.)
The truly sad thing is, all this can be avoided if management would just back off and let developers do their job without micromanaging and shoving their nose to the grindstone, spending the investment capital to pay your employees overtime when appropriate, staffing up your Quality Assurance department (5 Analysts testing code put out by 10 engineers is not enough by a long shot) and not releasing it until it's ready.
One excellent software release can make your company very rich. One poor software release can kill your company. (And in the case of the company I used to work for, I must admit I hope that happens. The world would be a better place without that third rate company run by a fourth rate pompous jack-ass developer who thinks he's above you just because he went to an Ivy League school, and you just went to the local University.)
And don't even ask me to name the company or the product, I won't, I still have close friends that work there. Even if I do wish they'd quit drinking the Kool-Aid..
6. Fixing even the shallow defects would break backwards compatibility and the customers all swear they will go to your competitors.
Kombat's Law:
Before, the customer wanted a library of CGI scripts to run their e-commerce website. Now that've got a handle on scripting design patterns, the customer wants EJBs. By the time you learn what to expect to go wrong with EJBs, the customer will want a cell-phone interface to their inventory.
Observation on Kombat's Law:
Technologies become fashionable soon after they are introduced, usually when the libraries are still version 1.0 or one-point-epsilon.
Related observation on the software industry:
It's standard practise to push a buggy pile of kludges still in need of major debugging out the door to meet overly optimistic deadlines and call it version 1.0.
Sum conclusion of it all:
We're all fucked.
Those who like to use exciting new gizmos are even more fucked. Those with bosses who are attracted by shiny objects are the most fucked.
Software companies could build a rock solid word processor or PIM , even MS, but no one will pay for it. We all scream that the retail price of MS Office is far too much. The corporate world looks at the per seat cost of MS and say, "I wish we could switch to something less expensive." Every product has a price point and software has a very low price point.
The odd part of the problem is the marginal cost of production. As we all know, once the development is done it cost next to nothing to produce. Once a producer has a product that the market will pay for there is little incentive to keep droping millions into making it much better. We would all love to use perfect" software. If "perfect" software can only be done using "traditional" engineering approachs such as is used in aircraft design that means putting massive teams on a word processor. Each team does one function including the code, hand checking the machine output from the compilier, checking it's performance againsta massive design document. We all know the story of how this kind of effort almost sunk IBM. Computer science has progressed to the point where massive effort should be able to be implimented, but no one would be willing to pay for it.
How does one test against every possible configuration of every possible computer that could conceivably run one's OS?
You design around it. 3rd party drivers are a constraint for MS. Drivers do not have to run in ring-0. Microsoft chose for drivers to run in ring-0 and we pay for it with crashes.
Joe
Joe Batt Solid Design
Indeed, it should be the market that decides, and not the courts : The market is the reason that North American automakers were forced to dramatically improve their quality (because the Japanese automakers were far ahead of them, and the market started voting for quality with their pocketbooks), and the markets are the reason why people pay a huge premium for IBM laptops. These capitalistic forces are amazingly powerful in getting what people really want, and the only time they fail to work is when someone with a pet issue tries to circumvent market-forces to get their own personal beliefs imposed on everyone through the legal system. It isn't surprizing to see the "sue em!" claims from many of those who already have forsook Microsoft software: These are the type of authoritarian people who believe that because it's not right for them, therefore it shouldn't be right for anyone else.
The irony is that most people, the average Joe and Jane out there, have shown that quality of software is very low on their list of purchasing criteria (which is how software like "ICQ" has never been non-beta...why bother ever calling it production code when millions of people will download it anyways). Given the choice, the market has stated time and time again that features will always trump quality, and that time to market often beats quality.
Sidenote: My own philosophy is that I spend about 90% of QA time code auditing and refactoring, and about 10% of the time doing runtime testing. I find that this is exactly the opposite of many organizations which push runtime testing as the method of evaluating code. To me runtime testing is no different than doing a compiler syntax check: It's an incredibly weak, time intensive, limited case way to assure the quality of your code, and should at most be used to evaluate tha the compiler and dependant third party code is of a good quality.