Slashdot Mirror
Why (Most) Software is so Bad
Rivard was one of several to point out that
MSNBC
says software sucks.
My opinion is that in software fields where the monetary gap between market-leader and second-place is large, we should expect bad software. Good design, good execution, good debugging all take time, but users can't see under the hood -- and wherever information is scarce or not readily traded among consumers, the free market bogs down. (Note what the article says about McAfee VirusScan.) So companies that don't plan on releasing a crummy 1.0 and fixing it later go under. That's just the way some markets work; if you're a coder or engineer who doesn't like that, find yourself a job in a niche without that monetary gap. Anyway, the really stunning thing is that, of all the media outlets, MSNBC points out that just one of Microsoft's poor design decisions has cost consumers $8.75 billion, and wonders why nobody has
sued.
Update: 06/18 14:10 GMT by J : Readers point out the story is a reprint
from Technology Review
(one of the few good magazines I get -- but this issue hasn't arrived yet :).
Rivard continued his writeup with an interesting point of view, saying that while we all know software sucks, we just accept it:
"Even though 'plenty of reviewers, pundits, hackers and other outsiders' will point out problems, often intentionally left in the product, no one has brought a liability suit against the makers of the known-to-be-vitiated product -- because the software gestapo (the End User License Agreement) has been 'able to avoid product liability litigation partly because software licenses force customers into arbitration' of poorly designed pith."There is a light at the end of the tunnel, believe it or not, and it's Bill Gates. Microsoft suspended coding for two months to seminar on bugs and how to fix them. Gates told his employees he wanted to make 'reliable and secure' software Microsoft's 'highest priority.' If you don't buy Gates' ad-hocking promises of redemption there are other solutions, like creating a programming language that forces good code; going back to the days of intense peer-review, instead of relying on compilers; and intense planning, past the bungling paradigm of the bar napkin."
It's not real uncommon for this sort of thing to happen. Our local newspaper was bought out by Gannette(sp?) a little bit ago, and the editors let them have it for a long time by publishing articles about how terrible they were and how they were going to ruin the newspaper. (IMHO they have ruined it. Story/Advertisement ratio on each page is about 1/6).
What?
Im working with 2 other developers and one source of fundage.... the client.
I has been an enormus task to get the software: A) bug free B) functional C) better than the compitition in a year.
There are problems, and there are alot of thing that I would like to do different.... but its just not possiable...
I can only hope that it does well after release. but it makes me wonder what I could have done with more fundage...
The Code Ninja is swift with his tool, precise in his delivery, and deadly accurate in his execution.
This article is out of the July/August MIT Technology Review. My copy of the magazine proves their point in an ironic fashion.
./ probably could--isolate the bug in ten minutes given the source. Likely it assumes that either the first city is valid or that the likelihood of two cities beginning with the same two letters in the same zip code is too small to consider.
The zip code I live in covers two cities, let's call them Appleville (tiny village) and Apricotland (large, sprawling concrete wasteland.) I live in Apricotland which is asciibetically second (based on the third letter.) Note that the first two letters are the same. MIT TR's mailing system lists me as living in Appleville. Why would it assume that zip code 12345 is the smaller village instead of the sprawling metropolis?
Yup. Buggy software. I could--as anyone reading
The joke's on you, MIT.
"If it doesn't spit out an error message, it must be done correctly, right?"
Well, that IS how they teach people to do it in college...
I'm disappointed that this article didn't mention open source even once. The process of writing an open source project is much different from that of a proprietary piece of software, and (as far as I'm concerned) the open source method is much better equipped to deal with things like sloppy code. But to the average computer idiot who reads MSNBC, this article makes it seem like all software systems are going to kill U.S. Marines and crash ambulances into each other.
1. Smart (or dumb) guys form startup around good idea. Version 1.0 gets written in a frenzy of caffeine and beer, riddled with bugs because it has to be delivered before the money runs out.
2. Mistakes made in version 1 are sworn off as version 2 is designed. Version 2 is built by the swell of 2nd-generation coders, hired as fast as possible and sent to work unsupervised by the overworked 1st-generation engineers.
3. Version 2 is delivered with all the good ideas on the surface, but implemented by less-than-excellent coders.
4. Widespread adoption funds much additional hiring. Anything vaguely mammalian is hired to fix bugs and work on new features and new products. Most 1st-generation engineers leave with their money. Product design and development is run by people who don't know what they're doing.
MSNBC is in one of the best positions to criticize Micro$oft - since the US theoretically has a free press, M$ really can't censor their own (or perhaps I should say their 0wned) media outlet. If they did so, one of two things would happen: 1) A public outcry of monopolistic muscle, or 2) people would just slowly start recognizing other channels to have more unbiased news, and NBC, with or with the M$ attached, would fall from significance.
The net result is, M$ doesn't want to gamble that. It makes them look good if M$NBC criticizes them because it shows they're not using their money and power to censor the press. So their journalists get to express their views openly.
I did not design this game/I did not name the stakes/I just happen to like apples/And I am not afraid of snakes-AniD
Unbelievable that we should take seriously the complaints of TV, which drowns us in commercials, covers us with misinformation and biased reporting and is soft on corporate abuses until they reach the Enron-like level that can't be ignored.
While all software contains bugs, and it may be impossible to root out all bugs in adequately complex programs, most software does it job almost all of the time. My word processor processes words, my spread sheet spreads sheets of numbers and my browser allows me to browse. These are MUCH more reliable than TV!!
So that's all fine and dandy, but it's not like you can just take one from each column and have something that makes sense. For example, were bugs in an operating system due to inefficient code that would be fixed by component-based design with an eye towards cost effectiveness? Well, uhhh, maybe, I think.
It didn't help that so many of the people quoted had no idea what they were talking about, and the ones who did had their quotes taken so far out of context that they made no sense. It seems a lot of people who never worked at Microsoft know how Microsoft develops software. Oh well.
It would make more sense to talk about a particular class of software and bug and then discuss why it is there. E.g. why do Microsoft systems products have buffer overflows. Even then you would get a bunch of different answers.
- adam
P.S. Comment first posted by me on Techdirt.
In the spirit of "the glass is half full", Steinberg recently released a new version of their music software "Cubase" and, on the very day it was available, released the "x.1" patch. Yes, some of you "the glass is half empty" people will say they should have waited on the major release, but I say this is a step in the right direction.
And until we're all running the exact same hardware I think we're going to have to settle for "less than perfect" from the software industry. When there's a bug with every sound card on the planet I fault the software company's lack of testing/research. When it doesn't work with a sound card only 116 people on the planet own, it's still listed as a "bug" and, lumped together with the other statistically insignificant "bugs" looks awful when reported together.
Finally, to all these estimates of how much $$$ "bad" software is costing....How much would it cost WITHOUT the software? Do you really want to jump in the WayBack machine and return to the glory days of a totally manual world where every problem was described as, "It's somewhere in the secreatarial pool?"
+-+-+-+-+-+-+ "I don't know what's wrong with you, but I'm quite sure it's hard to pronounce."
How does one test against every possible configuration of every possible computer that could conceivably run one's OS?
How is it possible that on the same PC I have a 70 day uptime with linux?
The key to the article is the last section, which talks about remedying the bad software situation, describing massive class-action lawsuits as "a bad idea whose time has come." MS knows that it could live through a class-action suit of this type. Would your favorite open-source project survive being sued back into the stone age? I think this article is an attempt to get public opinion stirred up to the point that UCITA laws - which include things like mandated warranties on software products -seem like a reasonable solution, and thus make life more difficult for MS's competition.
Knowing a bit of how mass mailings work, specifically how you figure out who is where through zip codes, the actual city that gets printed on things mailed to you in that fasion is determined by checking the Post office database, usually through a program such as AccuZip.
Lots of times, the city the post office has you in isn't the city you actually live in, but it will get to you all the same, because the Post Office can't assign multiple municipalities to a single zip code. They probably picked the small town because it didn't have any other zip code, or whatever criteria they have. Don't blame the software for something that isn't it's fault. It's just doing a query based on the official database.
6. Fixing even the shallow defects would break backwards compatibility and the customers all swear they will go to your competitors.
The second company didn't do any of that. When a decision needed to be made, they spent about 10 minutes looking at it, then made a choice. Rather than doing detailed project plans, they would constantly reassess and make direction changes. Their belief was that it was far better to do something and start getting feedback than spend too much time in analysis. This company also does pretty well, with a higer profit margin than the first.
Which approach is better? I don't know, but it was quite a culture shock going from one to the other! However, one thing is for sure and that is that people who are wedded to the "analyze-project manage" way of doing things (think PMI) are firmly convinced that the "fire - ready - fire - reaim" method is nothing but disaster waiting to happen. I am no longer so sure.
sPh
True. Someone once said software engineers are like craftsmen, but they should be more like production line workers (nasty as that sounds...)
A problem I see is a lack of common work practices and coding standards. Sure, there's quite a few standards out there, but there are perhaps too many, and new ones are devised almost weekly it seems. When a programmer joins a development team, how often is that programmer already familiar with the team's coding standards and work practices? Not very often, even when the people involved all work for the same software company.
Programming still needs to mature as a profession. A thing that stands in the way of that is the lack of good senior programmers, who are allowed time to coach and structure the way the juniors work. Yes, even in the current crappy economic situation, there's still a shortage of such people, both on the job market and within companies. Reasons:
- "Senior programmer" is not a viable career in many companies. Such people become designers or managers. It certainly isn't a sexy profession, many people look down on it even in technically oriented companies.
- The shortage of good programmers is such that those few are often too busy coding to concern themselves with coaching and work practices. Also, companies are often unwilling to invest in this and make sure that the senior programmers have time to spare for these things.
Until programmers are taught and coached properly on the job, they are left to discover things for themselves and take a wild stab themselves at developing something resembling good working practices. This way, the profession of software development will not mature.
If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
But it's hard to convince someone trying to make money that their programmers, regardless of previous experience, will not be productive from the first day at work because they need to be coached into the company's own standards.
They would prefer to have them producing code that works, as soon as possible, even if it means bringin into the code repository their own habits, good and bad, picked from their previous jobs, school or their intestinal tract.
If a manager at a construction company is building a bridge, it is likely he will not be satisfied JUST because the bridge seems solid. If the bridge looks unlike anything he has ever seen, its structure defies comprehension, and he's not entirely sure the thing stands by anything more than chance or will survive the touch of a maintenance crew, there will be hell to pay.
On the other hand, if the software works, the manager is probably satisfied.
This may not have as much to do with the programmers' or managers' discipline, but with two other simple facts:
a) Everyone knows how a bridge should look, and if it looks any different (new design), they will be extra-paranoid about it. Soon the new design will join the ranks of ways a bridge should look, and there are not that many.
b) Most of the bridge's features can be seen at first glance, without a peek at the designs.
On the other hand, concerning software:
a) No one knows how software should look. There are a million gurus who think they do, with a million different versions. Given a piece of code and an expert, he will be unlike to tell you if it looks solid or not at first glance, so a non-expert will judge by whether it executes or not.
b) Most of the software's structure is hidden from view in the final product because software is not physical. It's all design. Judging the quality of software, even if you knew what to look for, will depend on a through review of the designs (code) which is as likely to happen as a voluntary visit to the dentist.
Freedom is the freedom to say 2+2=4, everything else follows...
Popular software isn't reliable, because reliability isn't the highest value. Compatability with a legacy is (e.g. you want to run this application under MS Windows?). Or cheapness (e.g. Do you want to be billed 2 hours of my time (very little testing) or 6 hours (more testing)?). Or having lots of features (Would you like a flight simulator with that spreadsheet?). Or something else.
When reliability is important, you can have it. But it will cost you. Most people consider the cost to be too high. That's why more people run bleeding-edge Linux, Windows, or MacOS, than OpenBSD.
And there's nothing wrong with that. You just have to accept/enjoy the power/responsible that goes with the choice that you made, instead of whining that someone else should have chosen for you.
Irresponsible XP users whining about XP, ispartcularly pathetic. Yeah right, you didn't know what you were getting into. You never heard of this "Microsoft" company before, so you just assumed that they valued reliability over other considerations. Discovering that it was crap primarily intended to play video games and keep MCSEs' jobs secure, was like a cold knife in the back -- such unexpected treachery!
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
I think the article has it right with respect to lawsuits, and the current pass companies seem to have wrt accountability and quality in their software performance. I can envision a future in which companies buy each other's product, beat the crap out of it, and if it fails to perform as advertised, sick a team of lawyers on the offending producer. Think of it as greenmail for the software industry.
I hit the Windows Update page fairly regularly, and unfortunately Microsoft doesn't see fit to include such things as the IE6 security patches there. I have to know they exist, which I only know from reading /., and go searching for them. I thought IE was part of the OS? I can download IE6 Punjabi menu support from Windows Update, why can't I get a fscking security patch there? Perhaps I could just reinstall IE? No dice, Windows has detected that I already have IE6 installed.
I'll believe that Microsoft is actually fixing stuff when they start including IE security patches on the main Windows Update download page. I'll believe that they have a commitment to trustworthy computing when they give me an update tool as simple to use as SuSE's YOU. That's right; when a company with $40 billion in the bank can keep up with a company that's had to lay off 90% of their US work force, I might start giving them some respect. Until then "trustworthy computing" is a bunch of marketing hot air.
Under capitalism man exploits man. Under communism it's the other way around.
"This is one of the better comments on this thread."
To me, these comments seem utterly out of touch with reality. I find bugs and insufficiencies in open source software. But generally open source software impresses me as an attempt to do a good job.
In contrast, Microsoft software seems just sloppy. For example, Microsoft's Internet Explorer has 18 unpatched security bugs (when this was written). These active security risks are different from the recent 15 that have already been fixed. This is sloppiness, not mistakes, and I don't find anything like it in the open source world.
When I have a problem with open source software, I find that I can get help. When I call Microsoft, I find that, usually, no one with whom I am allowed to talk knows any answers. Right now, for example, no one seems to know how to repair a new, Intel Motherboard, Windows XP installation that won't create a virtual memory paging file. It's buggy, and nothing can be done other than re-install the OS and all the applications.
If you find a big problem in open source software, chances are that you will communicate directly with the main authors. With Microsoft, I have not been able to get answers. This article says that the Psychic Friends Network is equally as good as Microsoft technical support: Microsoft Technical Support vs. The Psychic Friends Network The conclusion of the article seems reasonable considering my experience with Microsoft. Neither organization has useful answers, but The Psychic friends Network is more friendly and less expensive.
Developers are to blame for creating a "boss jock" culture in the workplace that stresses quick results and sheer brawn over correctness, literacy, and maintainability. Developers are to blame for wallowing in and encouraging over complexity and churn of APIs, platforms, and programming languages. Many developers seem to get their rocks off on "being superior" to each other rather than creating something that is truly usable. Most developers have no conception of the key tradeoff between "fragile" and "simple but robust". Many developers are also to blame for "resume engineering", putting needless complexity into products because they want to justify new buzzwords on their resumes.
Companies are to blame for having absolutely no quality standards beyond profitability, and for creating sheltered sandbox playgrounds of idiot savant techies (the programmers competing with each other for kewlness) that encourages churn of tools for its own sake. Often, there are no grown adults with an ounce of clue paying attention. Also blame dual track career systems that encourage ghettoization of developers and which grooms the incompetent to lead and manage activities that they are demonstrably not good at (the Peter Principle, or the PHB plague.)
Consumers are to blame for demanding feature bulk in new products without being discerning enough to notice that the useful, core features are not well implemented. Consumers are to blame for buying and accepting crap with no comment or protest.
Everyone involved deserves each other. Consumers wouldn't pay the bucks to buy conservatively engineered and reliable software (beyond tiny product niches.) Companies would definitely not fund development of products that didn't have some incredible short term payback potential. Developers won't willingly work on stuff that isn't bleeding edge cool.
In this equation, everyone involved is pushing on each other HARD to make and ship crappy, fragile, bloated products.
I'm seeing lots of comments about how software isn't "engineering" or a "science." Well, I hate to break it to you guys but engineering is just as much hackery as anything else. Yes, there is some science to figuring out how much stress a material can take, but it's not like you can mathematically prove that an airplane is safe. Products are recalled all the time because of oversights and design errors. And planes do crash because of mechanical failures. Aerospace engineers are more afraid of flying than the general populace.
I see two big problems with modern software. The first is that computer systems are much, much, much too complicated, to where people accept lines like "no sofware can be bug free." Sure it can! How depressing it is that we let that be spood fed to us! But not when you have a huge operating system and huge language libraries and huge OS libraries and huge hardware drivers and so on. Heck, there's more code in one of Nvidia's drivers than in OSes from the 1970s. You see many fewer bugs on less complicated platforms, like game consoles and PDAs. PCs are pretty much a lost cause in that respect, and that's not going to change. One day I hope that we can return to more reliable systems, even at the expense of expandability (who cares about expandability any more?).
The second problem is simply that most developers don't give any importance to producing reliable software. In the telecom business, there's no way you could getting away without writing comprensive test suites and a huge amount of work from a dedicated testing group. But you hardly see automated testing for any software these days, outside of embedded systems. And you'll see silly newbie developers argue that they need to use C++ to write some silly app because "it is faster," when they actually should be using Python or Lisp instead.
From the article
"In the last 15 years alone, software defects have...induced a U.S. Navy ship to destroy a civilian airliner"
Will someone inform me as to what the author is implying? KAL Flight 007? TWA Flight 800?
I'll admit there is plenty of bad software out there, but what about all the benefits software has brought us? And technology in general?
You're half right. OSS isn't THE solution, but is part of a larger class of solutions. The reason mechanical and other forms of engineering could evolve into reliable disciplines is the ability to freely and openly communicate between the practitioners. With an industry wide peer review, everyone can analyze someone else's work, share their insights with everyone and everyone can benefit because that new technique or design can be incorporated by others into their work.
Shortly after the WTC attack, the American Civil Engineers society put together a panel of engineers to analyze the failure, and provide a report to the entire civil engineering community. When was the last time any proprietary software company did that? In fact, we've seen how these companies use lawsuits to squelch any such activity.
Openess, peer review and the ability to freely share information, lessons and strategies is what sets the other engineering disciplines apart from software engineering.
The bitter lessons of a veteran coder: http://bitterprogrammer.blogspot.com
Tremendous amounts of time are spent verifying your design works correctly
There you go.
If we were to spend that amount of time verifiying the software design, software would be better.
Somehow, I think that anyone involved in software development or design or testing already knew that.
But to quote from the review: So companies that don't plan on releasing a crummy 1.0 and fixing it later go under.
Given a choice between using a program now that almost works, or waiting another 2 years for the competetion to produce one that really works, most people will pick the one that's available. Then in 2 years, if the competetion hasn't gone out of business they produce a program that is reliable. But meanwhile the first program has fixed most of it's reliability issues and introduced new features. Yes, those new features are buggy, so the overall stability of the product really hasn't improved. But you can still do more with it than with the old, slow, feature-poor but reliable competetion.
Guess which company ends up bankrupt?
Welcome to market-driven development.
On the other hand, if you want to produce completely reliable non-market driven open source software, go ahead. There's nothing to stop you.
1. Make sure you are in a market that you can actually compete in. 2. Set realistic release cycles. 3. Don't go crazy with features on your first release. 4. Test the shit out of your code, better yet have an actual tester test the shit out of your code. I rarely release code with bugs in it - really. 5. Don't be a cocky, jackass developer with an attitude. 6. Take pride in doing this correctly and professionally. 7. Don't code for 16 hours straight and expect your code to not suck. 8. Take breaks. 9. Relax 10. In a professional manner, inform your client or boss that what he/she is asking is not realistic. Offer alternatives. 11. If you can't do the above, find another gig, that that project/company die so we don't have to deal with their crap software.
Some of the worst software written comes in periods of insanely-paced competition. WordPerfect vs. Word, Netscape vs. IE (we'll ignore the question that's begged about Mozilla here). Anyone who remembers the "office suite" wars will remember that Microsoft, Corel/Novell, Borland, and YoMamaSuite were each pumping out verions left and right. And when there's that much competition, quality suffers. (Yes, on a relative scale, the best code wins because consumers have choice, blah blah blah, but on an absolute scale, software produced during heavy competition is more buggy).
This flies against most of the prevailing thought in this forum, but think about the decision from a management point of view. You're about to ship a product. Two pretty bad bugs are found at the last minute. You have to make a go-nogo decision. There are two potential worlds:
Clearly, the manager in the second case chooses to fix the bugs, since stability is a selling point (albeit arguably a small one compared to features, for the average consumer). But a manager in the first case understands that unless his bugs are far worse than the competitions', shipping doesn't hurt (and in fact may be the most viable option). It's a classic case of Prisoners' Dilemma, and ironically the consumer gets screwed.
I do agree with you about developers underestimating; however, I find that developers underestimating is a direct result of management not giving the developers enough time to plan a project. When a project hasn't been planned, it is nearly impossible to forsee all the obstacles ahead for baseing an estimate. On the other hand, if the project was fully designed on paper and documented before anything was implementated, the implementation would go much quicker and there would be a base document for developers to look back on.
Think about building a house. It is nearly as complex as building a software application; however, would you ever let a construction team start building your house without having a set of blueprints? This is my point. How can management tell their team to start buildling without taking the time to draw up the plans. I can understand managing a project without having the development experience but in that case, that manager must know enough to let his/her team take the time to do the job the right way, including design and implementation.
And to further my point about requirements gathering. The manager usually gives an extremely vague spec to the engineers. In terms of building a house, this spec is usually along the lines of--
Requirements--
1. Kitchen
A. Sink
B. Refrigerator
2. Living room
A. Windows
B. Fireplace
3. Bathroom
A. Sink
B. Toilet
C. Shower
4. Front door
A. Door knob
The developer would look at a plan like this and ask--
What about?
A. Bedroom
B. Lights
C. Plumbing
D. Electricity
E. Roof
F. Walls
ETC
When you ask these questions, the manager is like, yeah of course we need all those and looks at you like you're stupid and says the foundation better be done by next week (which isn't in the spec mind you).
Now, why can't we spend a little more time thinking about this project and actually design it before jumping in? And why can't management see that they are the root cause of problems on a software project? The main reason is because software development isn't something you can see. When you build a house, you can go on-site and look at everything that is going on. With software, all you can do is scroll around a text window trying to decypher code. Maybe, it shouldn't be the job of the manager to evaluate progress on a project. Maybe managers need to utilize their lead developers better and actually listen to them for a change.
http://www.askthevoid.com
We all see software that sucks, but most software actually works great.
... every time I call my wife, it goes right through ... and my car regulates its own emissions (even though I haven't changed the oil in 6000 miles). Well I could go on.
....
My microwave works flawlessly
So when Watts S. Humphrey says, "Software's simply terrible today," I just laugh. I spend all day on a computer, but 80% of the software I use I never see, and it performs flawlessly.
I am doing all I can not to slip into a ranting rage at this lame article
"Anyway, the really stunning thing is that, of all the media outlets, MSNBC points out that just one of Microsoft's poor design decisions has cost consumers $8.75 billion, and wonders why nobody has sued."
If you look through the Slashdot archives, MSNBC has historically been one of the loudest mainstream (read: not theregister.co.uk) MSFT-criticisers. This is typical of them.
MSNBC didn't really criticize them. They reprinted an article that in turns quotes a consultant talking about MS's poor judgement in making it easy to run applications via email (and thus paving the road for the I Love Your virus and it's ilk.) That's quite a few degrees of seperation from them running an editorial stating MS should be sued from their 8.75 billion screwup. (Which they should be. It'll be interesting to see if a class-action law suit is ever filed, as it's probably not too late.)
It's an easy mistake to make, especially when Jamie, Timothy, or Michael post a story. (IMHO, they tend to use slashdot as a ranting board with ill-concieved ideas as they would otherwise find themselves rightly ignored or dismissed.[1] Witness that ~69.5% of the story Jamie posted was just drivel from someone who is neither a qualified economist, entrepreneur, or software engineer, yet important enough to justify *NOT* being a simple follow-up comment, but up there with the story.)
Anyway, It's well worth clicking through the story though, to get the straight scoop. Sure, you may not be the first post, but any comments you do make will be better informed, and you'll make slash a better community for it.
-Bill
[1] I know, I know, anything negative about an editor is a troll of flamebait. Mod me down. *sigh*
SlashSig Karma: Excellent (mostly affected by moderatio
There is a simple solution that we consumers can implement, and it doens't involve suing anyone.
Don't buy crappy software. I use all free software at home, not because I'm poor or a communist, but because I rarely see software that's worth paying money for, and when I do, there's also free software that does the same thing, if you're willing to live without bells and whistles. For example, I've discovered that almost everything I want to use MS Word for, I can use "vi" for instead. And VI is much more reliable.
I hate it when I make a joke and I get modded "+5 insightful". Mod the stupid comments "funny", not "insightful", pleas
One reason that I've very often seen lead to failed projects is assigning them to people who simply can't do them. Not necessarily because they don't know anything about programming, but because they don't know the tools they have to use.
Years ago I was assigned a UNIX - MS Access integration project where I had to translate EDI messages to Access tables. I had never worked with Access before. I told my boss that I can't do it in reasonable time, but there wasn't anyone else free at the moment, so I was forced to do it. The result was that after three months of coding, $100/hour from the customer, I had learned to program with MS Access - by learning how not to do it. I burned out and quit. The horrible software was probably rewritten from scratch.
I have seen many really horrible examples of similar situations with other programmers, often with people dabbling with C++, lately with people who simply don't know how to program with C. Oh, what incredible examples I could give you! The result has been indescribable garbage. The garbage often works in the planned test cases, barely, but bleeds memory like a pierced pig and crashes from a slightest variation.
Many of these catastrophies occur because the programmers do not honestly admit: "I can't do this." And even if the programmers would admit, their managers won't when dealing with the customers.
All tools require learning. Some have a soft learning curve, some steep as a mountain. It doesn't matter much how professional one is, when it comes to learning new, conceptionally very different tools. Managers don't often understand this.
Just as often, programmers are assigned to projects of which they do not have a clear vision, usually because of too superficial specifications. Without proper vision, they can't find the right attitude to write beautiful, perfectionist code. They need inspiration, and finding it often takes time.
Every corporate project I've worked on in the last 10 years has been marketing driven. Not only that but the requirements change from day to day. Which in my estimation means that the marketing guys aren't doing their jobs. They will tell you that the market changes fast and we have to adapt, but the truth is that its their job to predict the market ahead of time. That's their job right? Granted it's not an easy task, but the point of marketing is to gauge the market and determine what we need to build now in order to meet market demands later. Again, I've never seen it work that way. Instead, marketing delivers what the market demands are today and expects solutions squeezed into a schedule that is already bursting at the seams.
But wait, it gets worse. When the crappy product goes to market then engineering gets blamed for delivering a bad design. And, on the rare occasion when engineering says "No" and that answer is accepted (rarely happens), then engineering is blamed for losing the sale. Its a lose/lose situation and at no time is marketing *ever* held accountable for *anything*.
Its all very discouraging.
The main problem as I see it is a lack of well defined requirements. If engineering knows what to build and is given the resources then they can get the job done. But, when the requirements are constantly changing then it means we really don't know what we're building and nobody should be surprised if the final product isn't solid, given that environment.
You want a good product here's what you do. Write a very good Marketing Requirments document. Make sure it covers all of the *important* details. Have engineering analyze the requirments and come up with a first order schedule. Then you start writing software/hardware/system requirements documents, followed by design documents. All of the important documents need to be peer reviewed. Then you can make another schedule estimate and start building it. CMM level 2 compliance helps keep the checks and balances in place but of course you still need good designers.
I'm working on a project now that is taking this approach and it is definitely working. The only question is whether or not the Marketroids are going to drag the requirements around all over the map when we are midstream.
Something that software is unique in is that the primary tool to build software is software. You don't use a 2001 car to build the 2002 model. You don't use the Empire State Building to build the World Trade Center. But you use Visual Studio to build JojoApp v3.5, and Microsoft uses Visual Studio 6.0 to build Visual Studio.NET.
So all of our software is very literally tied to it's past, because we very rarely start from scratch and build up from the ISA to an application that works. And lo and behold, when we do (like with cell phone OSes), we achieve pretty stable results that do what they do well, even when what they're doing *is* revolutionary.
- The Amazina Llama