Why Most Software Sucks

gregbaker writes "Shift Magazine has an interesting article on bugs. They look at why commercial software has so many bugs and attempt to figure out why the software industry doesn't seem to have the same quality standards as other industries." Every Slashdot reader who manages commercial software projects should print this 8-page article out and glue it to his or her bathroom mirror and reread it every morning. But that's just my opinion - which is probably shared by another 100 million+ disgusted computer users worldwide who the commercial software industry seems to think should happily eat whatever garbage they want to throw at us.

Comments

One line says it all...

[weloytty]

That crappy software ships is, of course, no suprise. What I never can figure out is that in every article I see about this phenomenon, you see a line like this one:

"Management hadn't given the programmers nearly enough time to do a decent job, and it showed."

EVERY project I have worked on that turned into a disaster had this happen. 3 different companys, too. And every project that worked and was on time--happend because we did that old boring junk that no one likes to do:

1) Write Specs
2) Follow the spec.
3) When the marketing department trys to add stuff, you say "Is it in the spec?"--"Sure we can add it, but it is going to take X additional weeks".
4) Test.

Writing good software is not brain surgery. But you cant take shortcuts and expect everything to work fine.

And while it is fun to slam managment/marketing, programmers have to take blame too: lots of time we say "Yeah, it *WOULD* take a year for someone else to do it, but I am a programming genuis. I can have it done in a month!".

Re:One line says it all...

[JordanH]

And every project that worked and was on time--happend because we did that old boring junk that no one likes to do:

1) Write Specs
2) Follow the spec.
3) When the marketing department trys to add stuff, you say "Is it in the spec?"--"Sure we can add it, but it is going to take X additional weeks".
4) Test

I don't want to take anything away from your experiences. I'm sure you've had success with this process.

I've been in projects like this where success was declared at the end. But, I knew then, or perhaps just a little bit later that things were not so rosey.

Building something to spec is wonderful. Especially when it's a bridge, or a tower, a road. I've not seen as much real success with this when it's software we're building.

There are many many problems to talk about in the design and development process that I could go over. But one that isn't often talked about that dooms software to "suck" is what I like to call the two customers problem.

The people who write specs generally are marketing people, or managers, or maybe even, if you're lucky, analysts who think they really know the problem domain and have been around it for years and who are sure they know what goes into a good system.

Software is not designed for or by users. It's designed by people who sit around and try to dream up solutions without ultimately taking any responsibility for the useability of the system. Even when the designers make an honest effort to study the problem, talk to real users and do useability studies, too much of the ego of the spec writers comes through. Often, the grand dreams of the spec writers are in opposition with the stability that the real users crave. By stability, I mean both reliability (it doesn't crash) and that a new piece of software should be familiar, should have similarity to the software presently used for this function.

So, a project is initially judged a success or failure based on how you satisfy the management, the analysts and the marketing types. Testing pretty much proves that it works in the ways that the spec writers envisioned it being used. Unfortunately, the software will ultimately be judged by those who actually have to use it, and tested in the real world in ways the spec writers never dreamed. These two groups, the spec writers and the users, the two customers, have very different goals.

There is some hope. Rather than the spec, build, test, release model, a spiral development or RAD prototyping can ultimately get you a lot closer to a satisfying solution.

Even here, I've actually seen cases where management will seem to prefer that the system be hard to use or lack important functionality. You sometimes get the impression that management feels that if a piece of software satisfies the lowly user, then the organization is spending too much on software development.

It's a sad state of affairs. It's ironic that study after study shows that the # 1 customer satisfaction factor is a pleasant experience with the bank teller, the store clerk, the phone order taker, etc. Management consistently shows an almost studied disregard for the tools that these people are forced to use.

And while it is fun to slam managment/marketing, programmers have to take blame too: lots of time we say "Yeah, it *WOULD* take a year for someone else to do it, but I am a programming genuis. I can have it done in a month!".

Very true. One problem is that management often shops for a team or programmer who will tell them the estimate they want to hear. And, when you actually have to "name that tune", corners are cut. The corners that typically get cut are in places that are not visible externally, like bad coding practices, lack of concern for modularity and reuse, memory leaks (As long as memory leaks stay beneath the level that testing finds, it's thought that it's OK to leak some. Enlightened testing checks for memory leaks in the stress tests, but enlightened testing is not that common). Ultimately, this cutting of corners is what leads to the software sucking so badly. There's rarely political will to rewrite large systems, even when they are implemented in sand, so each new release suffers with the accretion of sins of the past.

Bringing this back to the subject of Open Source software, always on topic, never out of style on Slashdot, the problems I've outlined above don't really apply to Open Source development, at least not today. Typically, Open Source projects have not suffered the layers of management, analysts and thinkers that typical software development groans under. Most often, the user is the developer so the result is most likely to be satisfying (or if it's not, you know who to blame and usually know how to go about getting to satisfaction). Even when Open Source projects are written for someone else, the real testing is done on early releases by real users in close communication with the developers. So much organizational simplicity, so little need for endless meetings, project reviews, marketing "input", cost justifications, etc.

I like to think that free software is really about freeing developers to serve people more directly. With GPL software, if you aren't serving the customer, then someone else can take what you've produced and serve that customer better. With proprietary software, the trick is to develop just enough functionality and value into your offering that anyone else who tried to clone your software would incur too much expense and lead time for your customers to bear.

The Road to Software Hell

[rlp]

Want to produce really bad software.
It's easy if you follow just a few
simple rules:

1) Produce no documents - avoid creating
requirements documents, design specs,
etc. Just jump right into coding.

2) If it's a large project, divide the
work into several different development
groups, and make sure they don't
communicate. If they can be
geographically separated, so much the
better.

3) Don't hire any experienced
programmers, or if you make the mistake
of hiring them, don't listen to them.

4) Make sure that managers create
impossible schedules. Nothing produces
bugs like highly caffeinated over-worked
sleep deprived programmers.

5) Change requirements (unwritten of
course) frequently. Be sure and add
plenty of new features at the last
minute.

6) Be absolutely certain, that you don't
learn any lessons from industry history.
Don't read Brooks, Deming, Humphrey, or
any other Software Engineering or
Quality literature. And for God sake's,
DON'T look at 'http://www.sei.cmu.edu/'!

7) Avoid any and all code inspections.

8) Avoid creating any sort of
development processes, or if you do,
make them so pointless and burdensome,
that they are self defeating.

9) Do believe that you can test quality
into a product. But be sure to compress
the testing schedule just in case.

10) Three words - "Ship it anyway".

Re:Ok, there are problems, but...

[Anonymous Coward]

You need to understand how the industry really works.

The Vice-president doesn't care if the software works or not. If SOMETHING isn't installed on the target date that the President gave him, he is out the door.

The Director who reports to the VP doesn't care if the software works or not. Actually, he hopes it won't, since when the VP gets canned, the Director hopes to be promoted. Meanwhile, the Director is going to do everything he can to help. Like scheduling seven hours a day of mandatory "specification review" meetings for the developers and their supervisors. And "opening dialogs" with temp programmer agencies to help the managers with their resource management problems. And encouraging the Business Analysts to learn SQL so that they can provide better direction to the programmers in their functional specifications.

The Managers who work for the Director don't have time to care if the code works because they are too busy interviewing the hordes of fresh immigrant (temporary) programmers who have professional experience in every language you ever heard of. Except practical English.

The Supervisors who work for the Managers don't care about the code because they are too busy filling out the project status reports and the time sheets for the contract workers, attending the "specification review" meetings, and sitting on the "issue resolution" committee.

The Business Analysts actually care about the code and are sure that it would work if the programmers would just use the EXACT SQLs that were written in the functional specs. And don't bother them with any techie nonsense about "syntax errors"; the "Learn SQL during Lunch" book said it worked exactly like that. And "We need to have a meeting to discuss YOUR refusal to follow our design. We intend to escalate this as an issue."

The Project Leaders don't care about the code since they are on contract from the consulting arm of one of the "big 42" consulting/accounting firms. They care about three things: keeping their billable hours at maximum, forcing everybody to submit reports formatted according to their company's standard, and seeing that something is installed on target "go-live" date. Since their contract expires the day after "go-live", they are free to piss off everybody in sight. They won't be around when the bomb explodes.

The programming team leaders would like to care about the code. After all, they used to be programmers. And after "go-live", they are going to be stuck supporting the project. But with the sudden influx of temp/contract programmers, the new team leaders are spending all of their time trying to explain how the version control software works and why code is written on the development box, not the QC box, and trying to actually get logins for the temps in the first place. If anybody had asked, the Senior TL could have knocked out half of the project with a handful of Korn shell scripts, but he is busy setting up card tables in the hallway for six new temp programmers whose names he can't pronounce or spell and one whom he is already about to kill. At least setting up card tables serves as an excuse for avoiding the mandatory specification review meetings.

The new temp/contract programmers would also like to care about the code. And as soon as someone comes to their senses and replaces this horrible [AIX | BSD | HP-UX | Linux | NT | Sun ] box with a [decent | larger] [AIX | BSD | HP-UX | Linux | NT | Sun] system and installs a C++ compiler, the code that they have written will work fine. There's not any real difference between MQSeries and DCE. Obviously there was a mistake in the specification so we coded for the one we used last time.

The Tech Writers, meanwhile, not only don't care about the new programs, they don't even know that there is new software coming. Nobody has talked to them about documenting it. Three days before "go-live", one of them will overhear a conversation in the lunchroom. But conversations about the "latest fiasco" are too common and this one will be forgotten for another four days..

The QC/QA group cares about the code. They are already receiving threats from the Operations group about "another delivery of bug-infested code". Consequently, seven of the first ten bug reports will be for misspelled screen prompts. The other three will read "Doesn't work". (It will subsequently be discovered that "it didn't work" because the sysadmins had not installed the test code on the correct box.) Testing might a little faster is someone could answer them just one simple question: "What is it supposed to do?"

The system admins are completely unconcerned about new code. Until it is installed somewhere, they are free to ignore the upcoming need for disk space, printer queues, bandwidth. Just as well, since they are going to have to take the network down for the next week to install new routes in the bridges or bridges in the routers (they seem vague on what they hope to accomplish). But "we should have your workstation IP addresses changed out by the middle of next week, for sure, d00d".

Oh, and the Marketing department just pointed out to the President that there is no certificate of Y2K compliance for the project.

And all vacations and time off has been cancelled.

And the company firewall is now blocking http requests to monster.com.

Shovelling crap

[Anonymous Coward]

I do have an account, I just don't think I want it to be trivial to work out who I work for. You'll see why.

Pieces of this are true, the pressure builds up and management applies pressure, frequently in the wrong places.

I work for a company that makes specialized software for select clients. We have a sales team which goes out and beats the bushes looking for customers. We rarely have more than about thirty active customers at any one time -- we do a large project, deliver it, and get out.

My part of the project was babysitting the deliverable builds (the code builds that actually got shipped to the customer) and constructing the master images of the software the customer would receive. It would get handed off to the QA team, usually with the admonition "Important -- We must ship this today!"

Usually with this sort of admonition, you knew in advance that the disc you'd just sent to QA would go to the customer no matter what (since the inertia in fixing any problem was a minimum of three hours, plus the minimum three hours it takes to install and set up our product and run the basic basic basic smoke tests). And frequently, the contents of the disc were crap.

At one point, we listed (in earshot of the manager in charge of the project) the criteria for QA approval to ship. The candidate master must be round, gold, and have a hole in the middle. Someone observed that a maple dip doughnut could satisfy these requirements, and be just as functional in the hands of the customers. (More so, since they'd have something to eat while calling Customer Support.)

The root cause of our problem was that we were too customer driven. There are direct competitors in our space for the same customers, and the sales team is under incredible pressure to make the sale and bring home the deal. If that means saying Yeah, we can do that when they don't have the first goddamn clue about how hard it might be to do that. The contracts team then rolls in and agrees to many things with regards to functional spec and deadlines, and they are under lots of pressure to close the deal. The technical people who try to estimate the complexity of the requirements are under a lot of pressure to make the estimates low -- still competing against the other companies in our space, you know!

So these deals are impossible before the CEO (or whomever) finishes signing his name on the deal!

And then QA gets the heat to sign off on this candidate which would better be used as a drinks coaster!

These deals we write usually have performance clauses for delivery -- we agree that we will deliver the finished product no later than this date, and will affix penalties for each day (frequently in the area of $10K per day) that the product is late. There have been times we shipped blank tapes or CDs with a product label on it simply because we were contractually obligated to ship something. Now is that insane or what?

The big problem in our space is that no one is willing to say "no" to the customer. If you don't say "yes", then our competitors will, and we will lose the deal. It means that our competitors get the customer's money instead of us while the customer figures out the awful truth!

Our company even went so far as to regiment an ISO9001-like procedure that says every release candidate must to A, B, C, D... There's a form that each candidate has and it goes through each section from being ordered, built, tested and approved by QA, then duplicated and shipped. In practise, we ship a fair number of 'one timers' and dummy up the paperwork afterwards.

I don't know what the solution is. I know in our space the problem isn't likely to go away -- one hears stories of the competitors making huge deals that fall on their faces with the customers paying the tab. Sometimes we get to go in and try our luck -- sometimes not.

So what do we do? Listen to the technical people. You pay them big bucks for a reason! Walk away from sales that require the impossible. Avoid deals with financial penalties for late delivery. And stop trying to lay the failure or success of a product on the heads of QA! They don't make the product crap, it's already crap when it lands on their desk.

Maybe liability for software would help. I'm sure the company would jack prices through the roof to cover the added risk, but it would sure as hell focus the attention of the managers and developers and make sure the company wrote deals that made it better to deliver late software that worked and not the other way around like it is today.

Late and right beats on time and crap every time -- or at least, it should.

The solution. (Ask any hardware engineer)

[Morgaine]

Question: For similar levels of complexity, why do software systems typically exhibit many more bugs than (digital) hardware systems?

Answer: Because the component parts of hardware systems are almost entirely isolated from each other internally, ie. almost all interaction is through the component interfaces. When one part fails, the others continue working: in computing terms this is very much as if each part had its own processor. The failure of one part can of course stop the hardware system as a whole from functioning productively, but it is far more common that only part of the overall functionality is affected.

Solution: Use the MMU to isolate software components from each other and to make their internal structure entirely hidden, leaving only their interfaces visible (an OO approach is implied). Then multitask their methods using a dedicated event-driven component scheduler.

Needless to say, this would require a dramatic change in almost all our software tools as well. Backward compatibility would be minimal.

In academia I used to do research in hardware/software for parallel OO systems, and this is one of the ideas that popped up. I've had a design for a Unix Object Infrastructure based on this approach on my whiteboard for some years now, but the spare week of kernel hacking I had planned has never materialized. Perhaps this should be turned into a Linux or BSD community project instead.

It would be rather nice for the free/open operating systems to take a quantum leap beyond the traditional Unix feature set, and possibly solve or at least reduce the woes of the software engineering industry at the same time. :-)