Why Most Software Sucks

gregbaker writes "Shift Magazine has an interesting article on bugs. They look at why commercial software has so many bugs and attempt to figure out why the software industry doesn't seem to have the same quality standards as other industries." Every Slashdot reader who manages commercial software projects should print this 8-page article out and glue it to his or her bathroom mirror and reread it every morning. But that's just my opinion - which is probably shared by another 100 million+ disgusted computer users worldwide who the commercial software industry seems to think should happily eat whatever garbage they want to throw at us.

Comments

Bugs

[Entity42]

You'll have to admit that no piece of software is ever free from bugs, they'll always be there because it's impossible to predict conflicts between certain pieces of hardware etc.

The real problem is that many major manufacturers aim for a shipping date rather than a quality product.

I'll prolly get flaim-baited for this but 'That's the way things work these days' I just hope OSS makes a big enough impact

One line says it all...

[weloytty]

That crappy software ships is, of course, no suprise. What I never can figure out is that in every article I see about this phenomenon, you see a line like this one:

"Management hadn't given the programmers nearly enough time to do a decent job, and it showed."

EVERY project I have worked on that turned into a disaster had this happen. 3 different companys, too. And every project that worked and was on time--happend because we did that old boring junk that no one likes to do:

1) Write Specs
2) Follow the spec.
3) When the marketing department trys to add stuff, you say "Is it in the spec?"--"Sure we can add it, but it is going to take X additional weeks".
4) Test.

Writing good software is not brain surgery. But you cant take shortcuts and expect everything to work fine.

And while it is fun to slam managment/marketing, programmers have to take blame too: lots of time we say "Yeah, it *WOULD* take a year for someone else to do it, but I am a programming genuis. I can have it done in a month!".

Re:One line says it all...

[JordanH]

And every project that worked and was on time--happend because we did that old boring junk that no one likes to do:

1) Write Specs
2) Follow the spec.
3) When the marketing department trys to add stuff, you say "Is it in the spec?"--"Sure we can add it, but it is going to take X additional weeks".
4) Test

I don't want to take anything away from your experiences. I'm sure you've had success with this process.

I've been in projects like this where success was declared at the end. But, I knew then, or perhaps just a little bit later that things were not so rosey.

Building something to spec is wonderful. Especially when it's a bridge, or a tower, a road. I've not seen as much real success with this when it's software we're building.

There are many many problems to talk about in the design and development process that I could go over. But one that isn't often talked about that dooms software to "suck" is what I like to call the two customers problem.

The people who write specs generally are marketing people, or managers, or maybe even, if you're lucky, analysts who think they really know the problem domain and have been around it for years and who are sure they know what goes into a good system.

Software is not designed for or by users. It's designed by people who sit around and try to dream up solutions without ultimately taking any responsibility for the useability of the system. Even when the designers make an honest effort to study the problem, talk to real users and do useability studies, too much of the ego of the spec writers comes through. Often, the grand dreams of the spec writers are in opposition with the stability that the real users crave. By stability, I mean both reliability (it doesn't crash) and that a new piece of software should be familiar, should have similarity to the software presently used for this function.

So, a project is initially judged a success or failure based on how you satisfy the management, the analysts and the marketing types. Testing pretty much proves that it works in the ways that the spec writers envisioned it being used. Unfortunately, the software will ultimately be judged by those who actually have to use it, and tested in the real world in ways the spec writers never dreamed. These two groups, the spec writers and the users, the two customers, have very different goals.

There is some hope. Rather than the spec, build, test, release model, a spiral development or RAD prototyping can ultimately get you a lot closer to a satisfying solution.

Even here, I've actually seen cases where management will seem to prefer that the system be hard to use or lack important functionality. You sometimes get the impression that management feels that if a piece of software satisfies the lowly user, then the organization is spending too much on software development.

It's a sad state of affairs. It's ironic that study after study shows that the # 1 customer satisfaction factor is a pleasant experience with the bank teller, the store clerk, the phone order taker, etc. Management consistently shows an almost studied disregard for the tools that these people are forced to use.

And while it is fun to slam managment/marketing, programmers have to take blame too: lots of time we say "Yeah, it *WOULD* take a year for someone else to do it, but I am a programming genuis. I can have it done in a month!".

Very true. One problem is that management often shops for a team or programmer who will tell them the estimate they want to hear. And, when you actually have to "name that tune", corners are cut. The corners that typically get cut are in places that are not visible externally, like bad coding practices, lack of concern for modularity and reuse, memory leaks (As long as memory leaks stay beneath the level that testing finds, it's thought that it's OK to leak some. Enlightened testing checks for memory leaks in the stress tests, but enlightened testing is not that common). Ultimately, this cutting of corners is what leads to the software sucking so badly. There's rarely political will to rewrite large systems, even when they are implemented in sand, so each new release suffers with the accretion of sins of the past.

Bringing this back to the subject of Open Source software, always on topic, never out of style on Slashdot, the problems I've outlined above don't really apply to Open Source development, at least not today. Typically, Open Source projects have not suffered the layers of management, analysts and thinkers that typical software development groans under. Most often, the user is the developer so the result is most likely to be satisfying (or if it's not, you know who to blame and usually know how to go about getting to satisfaction). Even when Open Source projects are written for someone else, the real testing is done on early releases by real users in close communication with the developers. So much organizational simplicity, so little need for endless meetings, project reviews, marketing "input", cost justifications, etc.

I like to think that free software is really about freeing developers to serve people more directly. With GPL software, if you aren't serving the customer, then someone else can take what you've produced and serve that customer better. With proprietary software, the trick is to develop just enough functionality and value into your offering that anyone else who tried to clone your software would incur too much expense and lead time for your customers to bear.

Re:One line says it all...

[Black Parrot]

> This is probably the single largest cause of failure in the producion of new systems.

Imagine what would happen if your company was building a new office tower and some PHManager suggested adding another 20 stories after it was framed up, dried in, and interior finishing was underway. But didn't want the price or completion date to slip.

That's the scale of cluelessness that we're up against.


--
It's October 6th. Where's W2K? Over the horizon again, eh?

Print out the Article?

[The Cunctator]

I really believe some of the problem with both software development and all tech-work in general is our need for paper. At my office, productivity, efficiency, and reliabilty is harmed by the fact that everyone has their own method of dealing with stuff, a lot of it being the constant print-out of web-pages and other computer docs, because there's no one set-up that's good enough. I know it makes QA a nightmare, because you can't find the specs, or problems aren't reported in a standard way, etc. etc.

Let's face it: the promise of HTML won't be realized until the whole Web is Slashdotized (not Slashdotted!). By that I mean that every page can be personalized--for that, effectively, is what these comment forums allow us to do. By the time this thread is fully played out and moderated, the this thread will be more useful than the original article, because it will allow access to the article, and have insightful and useful comments and links highlighted. Can't really do that with paper.

If you think about it, Slashdot is analagous to a QA system. Speaking about that, it might be cool to make a Slashdot-style interface to Bugzilla. Why shouldn't the whole Web, and by extension, the whole world, have a QA system? (I suspect some might argue that's the idea behind Everything.)

The Web has a ways to go before I can really feel it's cool. Which is why Mozilla could be the most important app ever.

It's really quite obvious

[TheBeginner]

There is one and really only one reason that commercial software is buggy: it's more profitable that way. Let's face it, no matter how much we whine and complain about the software being buggy, if it is even reasonably workable we buy it. And this is slashdot readers. Think about the general public. Most of them just assume that is the only way software can be written.

I was talking to my sister recently (not a master computer user by any means) and she asked me why Windows crashes so often. I tried to explain about the bugs and conflicts between the various pieces of software she has, but she could not grasp the idea that a flawed product would be released intentionally.

Why? Because it what other industry is it done? None. The fact is that consumers at large have just accepted the necessity of software bugs. Because of this, the software companies have little or no incentive to release a clean product.

It would take much more money and man hours to realease a clean product and that is time that it is simply not worth it to spend. This is capitalism at its high point. Often it works to bring a higher standard about, but in the case where the buggier product makes more money, it can do the reverse as well.

A solution? I don't know if there really is one... perhaps make it so that a bug causes a computer explosion. Then, just the chips in our airplanes, companies would have to release quality software. Just a thought.

Mythical Man Month

[barlowg]

In 1975, Fred Brooks showed how many of the practices described in this article would actually produce worse software and extend the time necessary to complete it in hia classic, The Mythical Man Month. However, management still does not seem to understand these basic concepts. Any software project, open or closed source, should heed Brooks' words wisely. If you are a programmer or manage programmers, read this book!

It seems a shame that most people in the industry have not read it and that most managers have little or no idea of how managing a software project differs from general management.
--
Gregory J. Barlow
fight bloat. use blackbox.

Print it out? This isn't all very obvious?

[Kitsune Sushi]

I'm going to kill whoever thought it was a good idea to have the ads reload every 10-15 seconds on the site that article is on. Grr.

Talk to any programmer, tester or honest manager and they'll tell you a very different story about software. It is the unspoken scandal of digital culture, and it goes like this:

Software is badly made. More than that, it is often horribly made. It is developed with the sort of irresponsible abandon that would be unconscionable if it were applied to bridge-building, car-making and possibly even plumbing. And the internet has only made matters worse by encouraging dot-com companies to rush products out ever faster, despite the fact that software is now more complex than ever. Desperate to ride stock-market hysteria and the sea of investment dollars for dubious projects and websites, software companies cram their wares through on shorter and shorter timelines, with no latitude for serious planning, testing or concern for quality. "It's ship first and ask questions later," says one weary programmer, a survivor of a database company.

Hmm. Well, most glaringly, this kind of assumes that all programmers are found in the commercial sector. In simpler times, I would not have found this incorrect assumption to be all that surprising. In light of the recent thrust of GNU/Linux into the spotlight, it appears to be more of a gross oversight.

On the other hand, does anyone here really think that every proprietary software product is a horrible piece of.. whatever? And no, Microsoft does not make all known proprietary software products, contrary to the belief of many conspiracy theorists who have spent too much time on board alien space ships.. ;) I would imagine that a greal deal of commercial stuff is actually good and relatively bug free. Not all of it, but just because all of it doesn't kick ass doesn't mean that all of it sucks, either.

In this context, perhaps it's no wonder we face the stinging paradox of the computer age-that even though we have ever more way-kewl digital tools, our productivity has not budged an inch.

Oh wait.. No wonder. The writer has obviously spent too much time around script kiddies and other lower network life. ;)

"The average American would never buy an electric razor... as buggy and unreliable as a PC," says Bruce Brown, founder of BugNet, a Sumas, Washington-based firm that reports on bugs and provides fixes.

Actually, I know a lot of idiots with electric razors that don't shave a person's face (or elsewhere) very well. I consider that to be "unreliable".

Perhaps the most astonishing fact is that we, the customers, let software companies get away with products of such atrocious quality. We take every frozen screen as part of the package. We don't complain. And we sit back while, unregulated by government and cheered on by the stock market, software makers embark on a race to the bottom. They're almost all the way down.

"We, the suckers who don't know any better.." But seriously, who doesn't complain? I'd like to meet the person who isn't at least mildly ticked upon being visited by a BSOD.

Once a program goes over a few million lines of code, no one person can hold the structure neatly in their head.

Encapsulation and modularization are your friends ..

This is particularly true with operating systems, which have skyrocketed in size and developed thousands of byzantine byways. Windows 95 had eleven million lines of code; Windows 2000 is slated to have a mind-blowing twenty-nine million.

..too bad the people from Redmond don't make nice to these two potential buddies..

On top of the software issues, there's the challenge of hardware. The explosion of cheap PCs has created thousands of different machines built in completely different ways. This one has a nVidia graphics chip; that one has a Turtle Beach soundcard; yet another has some memory sold out of the back of a van in Mexico. Such variability makes it hard for software to run smoothly on each box.

GPL your drivers!!

I don't know.. After reading this entire article, I was first surprised that the writer even knows the correct term for a BSOD, and also I just have to ask: Can anyone come up with any reason why free software would not be considered a Very Good Thing to inject into the software industry, as far as the average end-user is concerned? Obviously it's not quite so helpful to big business. ;)

Um, no..

[Kitsune Sushi]

The grand majority of your project time should be spent in the design phases. The less time you spend on design, the more likely you're going to fuck up during implementation, and thus the more time you're going to spend doing testing. Design should consume about 50% to 70% of a project's time (or at least closer to those figures than 33%). After you have a full, working design down on paper, implementation shouldn't be all that hard to nail down quick unless your programmers really are quite clueless. The reason you should spend so much time on design is so that before you even write a single line of code, you know everything that the program is supposed to do. You figure out the best way to implement each feature, and whoosh! You're off. A lot of bugs are solved this way before you even go to your favorite text editor. Moral of this story? If you don't implement something incorrectly in the first place, you won't have to fix it later. It's a Good Thing. And most of your bugs will be typos and other assorted weirdness rather than critical design flaws. A change in design during implementation is much harder, and quite time consuming. You'll be much better off if you have an extremely clear view of the design beforehand. How much testing you do shouldn't really be set down as any predesignated percentage, AFAIC.. You test it until it's done being tested. Besides, how much time that will require depends entirely on your licensing and how you plan to test it. ;)

Customer Choice

[NickHolland]

In the 1950s through the 1970s, people bought cars based on features and looks, not based on reliability and quality. There were a few cars that were well built and lasted, but for the most part, they rotted on the lots, while their flashy but unreliable competition sold.

People used to line up to see the new models. People with money lined up to BUY the new models. Knowing full well it probably didn't start any better than the old one.

It wasn't that Detroit was incapable of BUYING a quality product. It was that the consumer was unwilling to buy the boring but solid product. It wasn't until the late 70s and early 80s that consumers suddenly demanded QUALITY over LOOKS from Detroit, and Detroit responded (in the auto industry, the problem was the speed at which the change in consumer attitude took place. You can't change the manufacturing criteria from flash to quality overnight..and no one was sure if this was a passing fad or a real trend)

The facts are, if the consumer demands something with their money, they will get it. Complaints don't mean a thing.

Another example: Airlines. People complain about late flights, they complain about lousy service, but they book the cheapest flight. Duh! Leaving on time costs money! Great service costs money! If the consumer buys the cheapest product, they can complain until they are blue in the face, nothing will change, at least not for the better.

It is simple economics. There is great demand for flashy software. There is little demand for quality software. While that is the case, that's what the consumer gets. The software industry better hope consumers don't change their mind on flash vs. quality overnight, like happened with the auto industry.

I've been supporting bad software for many years now. I'm starting to detect a change in attitudes towards business people (although, I am probably guilty of contaminating the sample!). I think this is good.

Consumer complaints don't enter into economic decisions. Dollars do.

Nick.

Re:software is not like other industries.

[iso]

perhaps what many are saying is true: software isn't like most other industries, but it's really not that diffent from the semiconductor industry. yet for some reason, we don't just have wild "crashing" and "bugs" in semiconductors (short of the occassional pentium slip-up :).

i think you're cutting software developers too much slack. sure software is difficult and complex, but have you really tried to understand the layout of cutting edge semiconductors?

i used to work for an FPGA manufacturer (who shall remain nameless). our FPGAs were/are cutting edge, and despite my degree in Electrical Engineering (with an emphasis on semiconductor design), i couldn't even come close to truely understanding half of what goes on inside those things.

our chips rarely had a hardware problem when things went to ship. but we were constantly having quality issues in Product Engineering (where i worked). why? because the software was, quite simply, lousy.

now don't get me wrong -- place and route routines can get pretty hairy, but i've seen the source code to the programming software, and i can assure you that's not 1/10th as complex as the chip their trying to program. but when i would confront the Software group about their buggy software, they gave me the typical arrogant "you don't understand computers" response, and more or less stated that software is just plain buggy by nature.

bullshit.

why is it that software programmers don't have the same idea of quality that hardware designers have? why do they automatically assume that software can't be (relatively) bug-free like complex hardware can be? i noticed it more than ever working at this FPGA manufacturer -- many software programmers simply can't think of software being any other way.

and this isn't a case of capitalism-driven bugs. this company isn't making any profit off of selling software updates. in fact, the majority of time-to-market goals we missed were due to the multitudes of software problems we had to overcome.

so to all you software developers who can't seem to comprehend the importance of design and testing: try taking a hint from your hardware-designing associates and smarten up!

let's start dispelling the myth: software doesn't need to suck!

- j
--
"The only guys who might buy [the Apple iBook] are the kind who wear those ludicrous baggy pants with the built-in rope that's used for a belt."
- John C Dvorak - PC Magazine

Free software isn't affected?

[VAXman]

Anybody who read The Cathedral and Bazaar (most people here, I'm assuming), know that the entire PREMISE of the free software industry is "release early, release often" -- which means that free software uses the attitude described in this article, only on steroids.

I find it scary that people think this is limited to commercial software. The article mentions that there are 5 patches for NT 4.0 -- there are 12 for the Linux 2.2 kernel, and that is just for the kernel (in many cases, the packages distributed on top of that have patches also, there are probably thousands of patches and updates collectively to every packages distributed in Red Hat 6.0, for example), AND Linux 2.2 has been around for less than 1/4 as long as Windows NT 4.0.

Feee software usually doesn't have formal testing either. Instead of a dedicated testing team like most commercial software has, the testing philosophy is to release it and for users to test it. Not good preventive treatment obviously. Nobody is going to test if the new SCSI driver is going to wipe off your hard drive - it's left for the beta testers to find this.

I can think of five or six showtopper bugs off the top of my head in Red Hat 6.0 that would have prevented the release from cooming close to shipping out the door had it been a serious commercial system, but it was released anyways.

I have also looked at the source code for many free projects such as GCC and GIMP and noticed that the code quality was quite low. For example, malloc() calls were usually unchecked (especially in GIMP). I have worked on commercial projects before, and checking malloc() is rule #1 -- if you happen to run out of memory while using GIMP, it'll blow up, where as commercial systems will simply fail to complete the current operations. If such a high profile package is of such low code quality, I expect the lesser profile packages are considerably more buggy.

The Road to Software Hell

[rlp]

Want to produce really bad software.
It's easy if you follow just a few
simple rules:

1) Produce no documents - avoid creating
requirements documents, design specs,
etc. Just jump right into coding.

2) If it's a large project, divide the
work into several different development
groups, and make sure they don't
communicate. If they can be
geographically separated, so much the
better.

3) Don't hire any experienced
programmers, or if you make the mistake
of hiring them, don't listen to them.

4) Make sure that managers create
impossible schedules. Nothing produces
bugs like highly caffeinated over-worked
sleep deprived programmers.

5) Change requirements (unwritten of
course) frequently. Be sure and add
plenty of new features at the last
minute.

6) Be absolutely certain, that you don't
learn any lessons from industry history.
Don't read Brooks, Deming, Humphrey, or
any other Software Engineering or
Quality literature. And for God sake's,
DON'T look at 'http://www.sei.cmu.edu/'!

7) Avoid any and all code inspections.

8) Avoid creating any sort of
development processes, or if you do,
make them so pointless and burdensome,
that they are self defeating.

9) Do believe that you can test quality
into a product. But be sure to compress
the testing schedule just in case.

10) Three words - "Ship it anyway".

It's all our fault, isn't it?

[wilkinsm]

*Flame suit on*

Yes.

Question: When was the last time you returned an high tech item because it was flakey?

For the average Joe, I'd venture that the occurance of these events are far and few between. One reason is with complex items it's harder to determined exactly what the problem is, or who is at fault. Consumers are becoming lazy in their shopping skills. If I buy a knife set with a broken blade, it's easier for me to put blame on the manufacter than when I buy a piece of software. I know a broken blade when I see it, but do I recognize a broke program? How can I be sure it's not the hardware's fault or the operating systems fault?

Another Question: When was the last time you bought something new that had a users manual with a wiring scematic in the back? Manufactuers don't even bother anymore. They know that most of us are too clueless these days to figure them out anyway.

So the software companies can get lax behind their lawyers and their propertery magic while the world around them falls apart.

Re:Ok, there are problems, but...

[Anonymous Coward]

You need to understand how the industry really works.

The Vice-president doesn't care if the software works or not. If SOMETHING isn't installed on the target date that the President gave him, he is out the door.

The Director who reports to the VP doesn't care if the software works or not. Actually, he hopes it won't, since when the VP gets canned, the Director hopes to be promoted. Meanwhile, the Director is going to do everything he can to help. Like scheduling seven hours a day of mandatory "specification review" meetings for the developers and their supervisors. And "opening dialogs" with temp programmer agencies to help the managers with their resource management problems. And encouraging the Business Analysts to learn SQL so that they can provide better direction to the programmers in their functional specifications.

The Managers who work for the Director don't have time to care if the code works because they are too busy interviewing the hordes of fresh immigrant (temporary) programmers who have professional experience in every language you ever heard of. Except practical English.

The Supervisors who work for the Managers don't care about the code because they are too busy filling out the project status reports and the time sheets for the contract workers, attending the "specification review" meetings, and sitting on the "issue resolution" committee.

The Business Analysts actually care about the code and are sure that it would work if the programmers would just use the EXACT SQLs that were written in the functional specs. And don't bother them with any techie nonsense about "syntax errors"; the "Learn SQL during Lunch" book said it worked exactly like that. And "We need to have a meeting to discuss YOUR refusal to follow our design. We intend to escalate this as an issue."

The Project Leaders don't care about the code since they are on contract from the consulting arm of one of the "big 42" consulting/accounting firms. They care about three things: keeping their billable hours at maximum, forcing everybody to submit reports formatted according to their company's standard, and seeing that something is installed on target "go-live" date. Since their contract expires the day after "go-live", they are free to piss off everybody in sight. They won't be around when the bomb explodes.

The programming team leaders would like to care about the code. After all, they used to be programmers. And after "go-live", they are going to be stuck supporting the project. But with the sudden influx of temp/contract programmers, the new team leaders are spending all of their time trying to explain how the version control software works and why code is written on the development box, not the QC box, and trying to actually get logins for the temps in the first place. If anybody had asked, the Senior TL could have knocked out half of the project with a handful of Korn shell scripts, but he is busy setting up card tables in the hallway for six new temp programmers whose names he can't pronounce or spell and one whom he is already about to kill. At least setting up card tables serves as an excuse for avoiding the mandatory specification review meetings.

The new temp/contract programmers would also like to care about the code. And as soon as someone comes to their senses and replaces this horrible [AIX | BSD | HP-UX | Linux | NT | Sun ] box with a [decent | larger] [AIX | BSD | HP-UX | Linux | NT | Sun] system and installs a C++ compiler, the code that they have written will work fine. There's not any real difference between MQSeries and DCE. Obviously there was a mistake in the specification so we coded for the one we used last time.

The Tech Writers, meanwhile, not only don't care about the new programs, they don't even know that there is new software coming. Nobody has talked to them about documenting it. Three days before "go-live", one of them will overhear a conversation in the lunchroom. But conversations about the "latest fiasco" are too common and this one will be forgotten for another four days..

The QC/QA group cares about the code. They are already receiving threats from the Operations group about "another delivery of bug-infested code". Consequently, seven of the first ten bug reports will be for misspelled screen prompts. The other three will read "Doesn't work". (It will subsequently be discovered that "it didn't work" because the sysadmins had not installed the test code on the correct box.) Testing might a little faster is someone could answer them just one simple question: "What is it supposed to do?"

The system admins are completely unconcerned about new code. Until it is installed somewhere, they are free to ignore the upcoming need for disk space, printer queues, bandwidth. Just as well, since they are going to have to take the network down for the next week to install new routes in the bridges or bridges in the routers (they seem vague on what they hope to accomplish). But "we should have your workstation IP addresses changed out by the middle of next week, for sure, d00d".

Oh, and the Marketing department just pointed out to the President that there is no certificate of Y2K compliance for the project.

And all vacations and time off has been cancelled.

And the company firewall is now blocking http requests to monster.com.

Shovelling crap

[Anonymous Coward]

I do have an account, I just don't think I want it to be trivial to work out who I work for. You'll see why.

Pieces of this are true, the pressure builds up and management applies pressure, frequently in the wrong places.

I work for a company that makes specialized software for select clients. We have a sales team which goes out and beats the bushes looking for customers. We rarely have more than about thirty active customers at any one time -- we do a large project, deliver it, and get out.

My part of the project was babysitting the deliverable builds (the code builds that actually got shipped to the customer) and constructing the master images of the software the customer would receive. It would get handed off to the QA team, usually with the admonition "Important -- We must ship this today!"

Usually with this sort of admonition, you knew in advance that the disc you'd just sent to QA would go to the customer no matter what (since the inertia in fixing any problem was a minimum of three hours, plus the minimum three hours it takes to install and set up our product and run the basic basic basic smoke tests). And frequently, the contents of the disc were crap.

At one point, we listed (in earshot of the manager in charge of the project) the criteria for QA approval to ship. The candidate master must be round, gold, and have a hole in the middle. Someone observed that a maple dip doughnut could satisfy these requirements, and be just as functional in the hands of the customers. (More so, since they'd have something to eat while calling Customer Support.)

The root cause of our problem was that we were too customer driven. There are direct competitors in our space for the same customers, and the sales team is under incredible pressure to make the sale and bring home the deal. If that means saying Yeah, we can do that when they don't have the first goddamn clue about how hard it might be to do that. The contracts team then rolls in and agrees to many things with regards to functional spec and deadlines, and they are under lots of pressure to close the deal. The technical people who try to estimate the complexity of the requirements are under a lot of pressure to make the estimates low -- still competing against the other companies in our space, you know!

So these deals are impossible before the CEO (or whomever) finishes signing his name on the deal!

And then QA gets the heat to sign off on this candidate which would better be used as a drinks coaster!

These deals we write usually have performance clauses for delivery -- we agree that we will deliver the finished product no later than this date, and will affix penalties for each day (frequently in the area of $10K per day) that the product is late. There have been times we shipped blank tapes or CDs with a product label on it simply because we were contractually obligated to ship something. Now is that insane or what?

The big problem in our space is that no one is willing to say "no" to the customer. If you don't say "yes", then our competitors will, and we will lose the deal. It means that our competitors get the customer's money instead of us while the customer figures out the awful truth!

Our company even went so far as to regiment an ISO9001-like procedure that says every release candidate must to A, B, C, D... There's a form that each candidate has and it goes through each section from being ordered, built, tested and approved by QA, then duplicated and shipped. In practise, we ship a fair number of 'one timers' and dummy up the paperwork afterwards.

I don't know what the solution is. I know in our space the problem isn't likely to go away -- one hears stories of the competitors making huge deals that fall on their faces with the customers paying the tab. Sometimes we get to go in and try our luck -- sometimes not.

So what do we do? Listen to the technical people. You pay them big bucks for a reason! Walk away from sales that require the impossible. Avoid deals with financial penalties for late delivery. And stop trying to lay the failure or success of a product on the heads of QA! They don't make the product crap, it's already crap when it lands on their desk.

Maybe liability for software would help. I'm sure the company would jack prices through the roof to cover the added risk, but it would sure as hell focus the attention of the managers and developers and make sure the company wrote deals that made it better to deliver late software that worked and not the other way around like it is today.

Late and right beats on time and crap every time -- or at least, it should.

That article...

[tzanger]

... sucks.

I am a software developper. Even if I can't spell it right. :-)

While I haven't read a single comment on this article yet, I am willing to bet that most are "great article" comments from people thinking that it is the boss' fault. Let me ask you this: Who told the manager that they could do it in three months? Who didn't tell the boss to fuck himself when he moved the ship date up six months??

Jesus that article riled me up! It makes my profession out to be a gang of unprofessinal 31337 dudes who like to do nothing but play Quake all day, scarfing pizza and mountain dew and whine that they didn't get enough time when the ship date draws near.

If you see the development taking much longer than you originally estimated, you get off your ass and tell your boss. You tell them as soon as you see the problem, not 1 week before the ship date when nothing can be done. You don't sit there and pull all-nighters trying to get it working at the last minute. I know; I've been there. If your boss won't give you more time you tell him that the product will NOT be done and will be full of bugs. Walk if you have to. How many articles on /. of late have talked about all the boundless opportunities for people skilled in software?!!

What else... oh yes... the "big code can't be tested" arguement. Whatever happened to programming modularly, testing it thoroughly and eliminating the bugs through good software design? Oh yeah, I guess that's too logical. The shuttle/MRI/microwave/car/elevator/aircraft computer programmers must have it all wrong, after all. Thank God someone in the games field has it right!

"PCs are all so different!!" Gimme a break! If it doesn't work on PC 'X', and you suspect hardware after you've elimiated all reasonable doubt that it's your software, you buy PC 'Y' and try it. It sounds like these guys have zero problem solving skills! Also sounds like they're programming too close to the hardware; that's what APIs are for and if the APIs are bad... well then at least you've done what you could and can try to contact the vendor for a workaround.

No software can be perfect; another fallacy. Sounds like laziness or lack of time. Or both. Either way, most people know what they're getting into when they sign on, or they find out shortly thereafter. Either way it's ultimately the programmer's responsibility to make the code good. Bugs will crop up after ship date, but if you've programmed correctly and used proper coding procedures you will have a detailed map of what goes where and can test inputs and outputs to find out exactly where the bug lies to correct it and bring you that much closer to a perfect program.

GIGO -- it's as simple as that. If you hack it together, you'll be hacking it forever. Stand up for your rights and stand behind your work. Whatever happened to being proud?

The solution. (Ask any hardware engineer)

[Morgaine]

Question: For similar levels of complexity, why do software systems typically exhibit many more bugs than (digital) hardware systems?

Answer: Because the component parts of hardware systems are almost entirely isolated from each other internally, ie. almost all interaction is through the component interfaces. When one part fails, the others continue working: in computing terms this is very much as if each part had its own processor. The failure of one part can of course stop the hardware system as a whole from functioning productively, but it is far more common that only part of the overall functionality is affected.

Solution: Use the MMU to isolate software components from each other and to make their internal structure entirely hidden, leaving only their interfaces visible (an OO approach is implied). Then multitask their methods using a dedicated event-driven component scheduler.

Needless to say, this would require a dramatic change in almost all our software tools as well. Backward compatibility would be minimal.

In academia I used to do research in hardware/software for parallel OO systems, and this is one of the ideas that popped up. I've had a design for a Unix Object Infrastructure based on this approach on my whiteboard for some years now, but the spare week of kernel hacking I had planned has never materialized. Perhaps this should be turned into a Linux or BSD community project instead.

It would be rather nice for the free/open operating systems to take a quantum leap beyond the traditional Unix feature set, and possibly solve or at least reduce the woes of the software engineering industry at the same time. :-)

Re:The solution. (Ask any hardware engineer)

[wocky]

Question: For similar levels of complexity, why do software systems typically exhibit many more bugs than (digital) hardware systems?

In my opinion, they don't.

I've heard the "hardware engineers know how to build reliable complex systems" argument numerous times, usually from hardware engineers. But the software they complain about is far more complex than the hardware by several measures.

• It requires much more control state to describe the software.
  
• The environment that that the software has to contend with is much less well-defined.
  
• The functionality required of hardware is much less than that required of software.
  

For a typical digital chip, there is in fact some software which is exactly as complex: the hardware description language code for the chip, and perhaps a cycle-accurate simulator written in something like C. How many bugs are there in these pieces of software? As many as there are in the chip.

A modern CPU may have hundreds of errata, but most go unnoticed. You just don't get many level 3 interrupts while a floating point instruction is stalled waiting for the translation lookaside buffer. Besides, if the problem causes the system to crash, the user will just blame Windows anyway :-P.

The hardware engineer complains "but those are very exceptional conditions..." Exactly, and they're similar to the type of situations that tend unmask software bugs.

People are inherently bad at envisioning corner cases. The digital hardware engineers are lucky in that they at least have a well-defined finite-state abstraction to work with. You can buy commercial tools to help with test generation and state-space exploration. The corresponding tools don't exist for software.

It's true that hardware engineers have to deal with issues such as signal integrity and noise, metal migration, power supply droop, and so on. They generally do a fairly good job on these, but these are all phyiscal issues related to implementing the digital abstraction. As such, they are amenable to the standard engineering practice of giving yourself a little extra margin. If we knew how to easily provide extra margin for discrete behavior, software would be a lot more robust.

Re:Capitalism ruins everything

[scrytch]

Why yes, just look at the thriving Chinese, North Korean, and Cuban software outfits. Oh and you can spare me the claptrap about how those aren't "true" communism/socialism/whatever, because central control in the name of "the people" has corrupted and failed every last time.

Shove it.

software tougher to make than soap . . .

[werdna]

Even when there exist objective criteria defining what software is intended or supposed to accomplish, and even when there exist objective and consistent criteria concerning aesthetics of UI design, art and related subject matter, software is just plain hard to make.

Software is much tougher to make than soap.

Why? Because even a relatively trivial program involves express specification of changes to a massively large state space. In the analog world, an engineer infinitely narrows the scope of vastly larger state spaces to be considered by making assumptions that things behave continuously -- not so with code, which grows combinatorially and discontinuously complex with each additional variable, object or control statement.

Nowhere does this become clearer than when one is asked to engage in true quality control: proving a program correct. Methodologies that work beautifully and elegantly in the small to demonstrate the accuracy of a code segment grow unmanageably out of control when facing a 100,000 to 1,000,000 line program. And in turn, we then have the bugginess of the proof --viz. was the spec specified adequately-- to consider as a new "quality control" issue).

Even the very process of quality assurance in code is harder than Q/A for soap. A scientist may often presume safely, and can often prove, that if the soap behaves properly at two ends of a temperature range, that the soap will behave properly between them. This is almost never the case with code, it being in the nature of digital things to exhibit discontinuous behavior.

The bottom line is that excellent code is enormously expensive -- requiring only the brightest, best and most sophisticated management, quality assurance, design engineers and technicians. Noone wants to pay for what they claim they are entitled to expect. However, this is like most things in life. You can get:

Good. Fast. Cheap.

but you only get to pick two.

Regrettably, management is evaluated more closely for fast and cheap, so noone really worries about good, just so long as its "passable" (and its often easier to pass blame to the coders for "good" than it is to blame them for "fast" or "cheap").