Slashdot Mirror
Examples of Programming Gone Wrong?
LightForce3 asks: "I'm a beginning CS student, and in my studies I've come across examples of programmer error causing very large problems, such as the Ariane 5 failure and the Therac-25 accidents, often as tales of caution to beginner programmers such as myself. My (morbid?) curiosity has been piqued, and I'm looking for other examples of programmer error leading to serious problems. After all, it is better to learn from the mistakes of others than from your own, right? ;) What programming-related accidents, incidents, and failures, both well-known and obscure, do Slashdot readers know about, and are there any good resources for researching these?"
When Programming goes Wrong 2! Thrill to our latest reality TV series where we show REAL LIFE footage of poorly thought out database schemas, unchecked buffers and even explicit shots of forbidden goto statements.
This book is devoted to just that. It's what you're looking for...go get it and read it.
http://slashdot.org/articles/99/09/30/1437217.shtm l
-- Kircle
1.) Patriot missile failure
2.) Intel f*cking up floating-point calculations in one of their chips
3.) High-tech toilet glitch (no, really!)
4.) Windows ME
If you celebrate Xmas, befriend me (538
What happened to Challenger wasn't a programming mistake, but rather a case of not following policy. The solid rocket boosters were never designed to operate in cold temperatures. The result of working outside of design specs was catastrophic failure, yes, but that wasn't the result of a programming error.
In the 80's, Robert T Morris accidentally released a worm that exploited problems in sendmail and other common internet daemons that took down most of what was the internet at that time. This was expecially bad since about half of it was military.
...wherein a technique to save memory on older computers resulted in a massive media panic twenty years later. Oh, and it caused a couple glitches
Co-founder of GerbilMechs
A Central Office (CO) switch is basically a mainframe-class computer programed in assembler. A few years back, a newly-installed switch failed due to a bug in the code, causing a cascading failure of the phone system for a few hours.
Incorrect: This was not a programming issue. Nor was it a software issue at all. The problem was the O-ring seals in the SRBs (Solid Rocket Boosters). The manufacturer stated that they should not be operated under 53 degrees, and NASA overrode the recomendation and launched anyway. The expected happened.
NASA hasn't ever had a hardware problem. Or a software problem. Ever. Every problem can be directly tied to one specific person being a fscking moron. The closest you could come is that Mars probe that crashed because of mismatched units. And that was just poor communication among the software guys.
Professor Falkin was always saying, "Leave a backdoor in any program you write, just in case your code becomes self-aware."
The RISKS Digest is a mailing list and usenet newsgroup that describes all kinds of situations where technology has gone wrong. Many of the stories involve programming errors.
Google's RISKs Archive
Outlook!
Built with the idea that code in attachments should be executable, often automatically. Also full of exploitable bugs, to get even more stuff running automatically, regardless of who who sent it. Responsible for a huge amount of damage by all sorts of worms, trojans, etc.
Someone, somewhere got the idea that email would look better with html; and if it got html, it should get scripting too, that's consistent with web pages! And it's cool if attachments (like pictures) can be opened in their appropriate program automatically - let's run any executables then, that's consistent!
This is oversimplified, but I really feel that this is a case of stupid consistency that caused multi-billion dollar damage. Email should never be executed by the mail client.
I believe posters are recognized by their sig. So I made one.
Why not provide a link instead of saying "Oh yeah, I saw it way back when."
You people who say "use google to find it" or "this was already asked" are worse than the people who actualy ask the question.
Their only problem (if it could be said to be a problem) is ignorance, your kind however are a much better example of the problem of self-rightous lazyness.
Oh wait... -1 Redundant
Here's a good site though with tons of examples.
My favorite would be the infamous time when NASA did half its calculation in metric and the rest in SI. ;)
F-bacher
James Tiberius Kirk: "Spock, the women on your planet are logical. No other planet in the galaxy can make that claim."
MIT runs a class called 6.033: Computer Systems Engineering. These lecture notes contain a list of projects that had great sums of money spent on them only to be abandoned. Also the reading list has a bunch of papers that discuss the "big splash" failures like Therac 25.
that was told to my class about the altitude of fighter jets.
A company was hired to rewrite the code that was used on one of the models of fighter jets, and they offered to fix an unusual bug.
The details are: apparently they had two altimeters - one was barometric, and the other I don't remember.
Anyway, the programmer was coding along, and was writing code to determine what would happen if the altimeters stopped functioning.
He came to the case where they both weren't working, and couldn't figure out what to do, so called one of the pilots that was acting as an information source for the developers, and asked him what altitude they normally flew at, and he answered, "12,000 feet" or something similar.
So the programmer wrote,
if altimeter1 not working
{
if altimeter2 not working
{
set height = 12000;
}
}
Stupid, but this code could not be changed. The pilots had the following rule deeply ingrained: if the altitude stays at 12,000 for more than a few seconds, pull up, as your altimeters aren't working.
couldn't find it either, huh ? ;)
Wouldn't setting it to something like 0 be better? I mean, I could miss it sticking at 12,000 for a while, but if I notice that my altitude is suddenly 0, I think my first instinct will be to pull up as fast as possible.
F-bacher
James Tiberius Kirk: "Spock, the women on your planet are logical. No other planet in the galaxy can make that claim."
I used to work for a 1999/2000 'golden child' dot-bomb which dealt in file trading... a proposed legal form of napster. It was a fucked company from the start, but it still had a lot of traffic in the early days.
:)
:)
We always had problems with downloading files from the site.... the files kept getting corrupted, and occasionaly, a member would complain that they tried to download a powerpoint presentation and ended up getting 4 way anal porn.
This perplexed the developers, and it was not until 9 months after going online with the site, did they realise that the java class that dealt with the downloads was a single process shared by all users!
So, your download would go ok IF nobody else tried to download at the same time. If two people clicked download at about the same time, you would download the file that the second person wished to download.
No wonder they went bankrupt
-- 7 string electric violin + live loop samplers
When they played Heidi over the end of the greatest come-back in football history. Oh wait, you didn't mean that kind of programming, did you?
Do what you can, with what you have, where you are.
Don't be so narrow in your approach. Is it a programming error if a stadium roof collapses because the engineers couldn't understand what the output of their computer model was saying?
What about when the construction crew quietly substituted what they thought was an equivalent design to what the computer program came up with for a skywalk over a hotel lobby?
After almost 20 years in this field, I think that at least 80% of the serious "errors" I see are because the user didn't understand the results of the program, and only 20% of them are due to classic development errors.
The lesson to learn from this: the user interface matters. Give some thought to presenting the information in a meaningful manner (e.g., the infamous pre-Challenger graphs showing O-ring erosion vs. the post-Challenger graph that mapped damage by temperature at the time of launch), and allow users to see the information in the way that makes the most sense to them.
For every complex problem there is an answer that is clear, simple, and wrong. -- H L Mencken
A company I once worked for (as an intern) was in the business of what's called "train control" software. Briefly, it's the software that dispatchers use to monitor the status of the switches, the position of all the trains being tracked by the system, etc. One of the features of the system is to provide early-warning of potential collisions. Well, the system is quite reliable (having been in service, in one form or another, since the 70's). However, there have been some accidents.
Once such accident, in Mexico, was caused by an unexpected combination of several simultaneous failures. One day, for some reason, one of the servers needed to be reset. At the same time, two freight trains were stopped at a switch, in the process of what's called a "pass," where one train turns off onto a side track to let the other train pass by on the main track. Long story short, the status bits of the switch got lost during the server reset (there is a provision for restoring track states when the backup servers take over, but it didn't work for some reason). After asking if the track was clear, the driver for train1 recieved a green light from the dispatch office. The dispatcher, not knowing that train2 hadn't cleared the switch yet, figured everything was ok. The trains collided at very low speed, and not head-on, but nonetheless the collision cost the rail line several million in equipment and downtime. No one was hurt.
The lesson: When writing bullet-proof software, check every possible condition! More extensive field testing would have caught the failover bug.
I'm an AC for a reason...
Let's just say that two years ago a very large international shipping company suffered two days of worldwide failure in the package routings printed on labels. The bug was caused by an incorrectly placed paren in an index offset calculation, leading to truncation of an intermediate result (to a 16 bit unsigned int, when it should have been 32). The bug sat dormant for five years because the result matrix it was indexing into was smaller than 64kbytes. As soon as it grew over that size - boom! What a way to wake up at 2am when the Asian-Pacific region starts calling...
I didn't make it, but I was definitely involved with the fix. After that we did some very thorough auditing on all of the routing code - and fortunately didn't find any other surprises lurking.
This isn't really a programming error, but a user training error.
In the Airbus if the pilot tries to correct (use the flight controls) while the computer is engaged the computer will correct the pilot's correction. Unlike in a car with cruise control where if you hit the breaks it just cuts the cruise control. Many China Airlines planes have crashed due to poor pilot training in this regard. They weren't trained well enough to shut off the computer control before taking control of the plane.
I'm also sure someone can be a little more detailed than this, but it is, IMO, at least a design error that has caused hundreds of deaths.
As a side note, my Software Engineer professor refused to ever fly on a fly by wire plane, and was opposed to SDI simply because he didn't beleive that either had been or ever would be debugged properly. (if there is one error in every 10,000 lines of code, and it has 3 or 4 million Lines of Code, how many errors is that? His answer: too many to trust)
Come on now, that's the lazy way!
How about citing an actual example of windows code bugs causing big problems? I'll go first. The USS Yorktown had to be towed back to harbor when the NT system that was automating most of the ship crashed.
a different toilet story from about 10 years ago:
This appeared in today's (2/17) Seattle Post-Intelligencer:
It was a flush with a rush.
Toilets and urinals in the King County Courthouse exploded yesterday after a worker in Metro's downtown bus tunnel mistakenly connected an air compressor to the building's water line. As soon as hapless individuals flushed the pressurized privies, the plumbing started popping in restrooms throughout the 72-year-old building, said building services manager Bill Kemp. "They started blowing at about 11:30 (a.m.) and it took us awhile to figure it out," he recounted."We knew it had to be air in the system but the Water Department said that was impossible." It wasn't. The source of the problem was finally tracked to the tunnel under Third Avenue, and the errant air compressor was shut down. But not before employees on every floor in the 10-story courthouse had stories to tell about gushing geysers in the john. "We think we've lost about 20 to 25 toilets," said Kemp. "The porcelain is actually cracked." Kemp said no one has admitted being hurt by the unusual blast, although several people were badly drenched. Or very surprised. Explained Kemp, "The urinals acted more like bidets." We had other reports that people were not necessarily on the toilet but close." "This has not exactly been a good day for Metro," he noted. by Mary Rothschild --P-I Reporter
link(story is near bottom, pun intended.
Creationists are a lot like zombies. Slow, but powerful and numerous. And they all want to eat our brains.
Yeah it is probably in the archives. I've read it before.
:-)
Problem is, the slashdot search engine sucks. I haven't yet been able to query the archives and actually find what I'm looking for without needing to dig through hundreds of irrelevent discussions. Sometimes I think it might be faster to just scroll back through the "Older Stuff" section.
Or we could just have another discussion about it.
Ascalante: Your bride is over 3,000 years old.
Kull: She told me she was 19!
US shooting down Airbus 320
You're referring to the destruction of Iran Air flight 655 by the USS Vincennes near the Strait of Hormuz, on July 4, 1988. For one thing, it was an Airbus A300 (bigger and older than an A320). The failure there was mostly in human decision making, not in the AEGIS radar system, which faithfully reported that the airliner was travelling at 450 knots on a steady bearing towards Vincennes, roughly four miles outside the commercial air corridor, and not broadcasting IFF information (which of course they wouldn't, as a foreign civilian airliner). It was the officers of Vincennes who interpreted this information as a threat, misidentified the target as an Iranian F14, and destroyed it.
Toronto-area transit rider? Rate your ride.
http://slashdot.org/articles/02/05/02/1525210.shtm l?tid=128 - Debug your code, or else!
:-)
Using google's serach engine provides better results for slashdot.org that slashdot's own search engine
the database they were using faulted on a divide by zero. nothing to do with NT.
Much as I dislike NT, especially in critical environments, this problem had nothing to do with NT. It had everything to do with bad coding.
As we all know, information systems are only as smart as people make them. In the case of the USS Yorktown, an admin/operator entered data which caused a divide by zero condition in the application. Because the application did not have any exception handling built into it for a divide by zero condition, it died.
You can't blame the OS for this. The application should have had exception handling built into it in a couple of places. It probably should have checked any new entries before comitting them to ensure the new data would not introduce such a condition, and the app itself should have had appropriate error handling to prevent a panic/dump when a divide by zero condition was encountered.
If the app was coded by the same people on another platform, the end result would have been the same.
Idiot, n. A member of a large and powerful tribe whose influence in human affairs has always been dominant
Actually, the switching code was in C and the crash was due to a programmer's apparent misunderstanding of the 'break' statement. See full details at: http://www.csc.calpoly.edu/~jdalbey/SWE/Papers/att _collapse.html
http://wwwzenger.informatik.tu-muenchen.de/perso ns/huckle/bugse.html
Some of the tips, which may appear obvious to some of us, include:
--- Fox
No, I meant read it until you understand it. I don't want anyone working for me that doesn't think understanding documentation is a good thing or doing something the correct way rather than "it works so I might be doing it right."
And there's a difference between not being able to code and understanding a particular function. I may read a function's man page 2 or 3 times to make sure I understand correctly what is going on. Not nessesarly because I'm incompetent, but because the wording my be confusing (wow, confusing wording in a manpage? Who would have thought..). That doesn't mean every single function for a particular language requires you to read the documentation for it multiple times. I assume nothing. Assuming something leads to bugs and insecurity. I've been programming in C for many, many, many years. When I do a little PHP programming to create some web interfaces I don't assume that just because both C and PHP have a function called strlen, and the general documentation says it returns the length of a string, that they work identically. So I read the entire strlen documentation for PHP to understand exactly whats happening. It only took less than a minute, but now I'm not assuming. I know. This goes for lots of things. The more complex functions you use, the more important it is to fully understand them.
The point is coding correctly is the most important skill to learn. I have friends that hack together scripts and programs from examples and snipits of other code and a little bit of their own code to glue it together, with little to no understanding of what they are actually doing. Then months later something breaks they can't fix and they act as if it was the author who wrote the example code's problem.
No, it's there fault. Not because they hacked together examples, but because they didn't take to the time to make sure they knew what the examples were doing, that the examples were implemented correctly, and that they understood exactly how the code in the examples worked.
Take a look at OpenBSD's philosphy.. You can learn a lot from it.
..There's a-dooin's a-transpirin'
From the site:
"In 1969, as part of its global empire, Union Carbide Corporation set up its pesticide formulation unit in the northern end of the city of Bhopal in central India. Initially it mixed and packaged pesticides imported from the US but was gradually expanded. In December 1979 its Methyl Iso Cyanate (MC) plant with an imtalled capacity of 5000 tonnes went into production.
On the night of December 2, 1984, during routine maintenance operations in the Methyl Iso Cyanate (MC) plant, at about 9.30 p.m., a large quantity of water entered storage tank no. 610 containing over 60 tonnes of AEC.
This triggered off a runaway reaction resulting in a tremendous increase of temperature and pressure in the tank and 40 tonnes of MIC along with Hydrogen Cyanide and other reaction products burst past the ruptured disc and into the night air of Bhopal at around 12.30 a.m. Safety systems were grossly under-designed and inoperative. Senior factory officials knew of the lethal build-up in the tank at least one hour before the leakage, yet the siren to warn neighbourhood communities was sounded more than one hour after the leak started.
By then, the poisons had enveloped an area of 40 sq.kms. killing thousands of people in its immediate wake. Over 500 thousand suffered from acute breathlessness, pain in the eyes and vomiting as they ran in panic to get away from the poison clouds that hung close to the ground for more than four hours."
Nothing to do with programming errors here that I can see. Sounds more like gross negligence and incompetence to me.
-A.
student of animation and the fine arts
Well, except for Mars Polar Lander, where the failure review board determined that the lander crashed because a flag indocating contact with the ground was not intialized to zero prior to the start of the retro-thruster loop. So the flag got set by the shock of deploying the landing legs, never got reset, and caused the thrusters to switch off as soon as they were on.
I guess maybe you forgot about Apollo 13 as well (hardware)? Or the Galileo High Gain Antenna that failed to deploy (hardware)? Or the serious telemetry system problems they had with one of the Voyagers (hardware)? Or the faulty landing bag on one of the Mercury flights (hardware)? (was it Glenn's? I don't remember) Or that funky glitch in the landing computer during Apollo 11 (software)? You know, there's a reason that most space mission tend to be heavy on redundant hardware, and invest a lot of time and effort in fault protection software.
Every problem can be directly tied to one specific person being a fscking moron.
Well yeah, but that's the case with a lot of bugs, isn't it? Mistakes tend to be people issues.
The closest you could come is that Mars probe that crashed because of mismatched units. And that was just poor communication among the software guys.
You are at least correct about that - the problem was not a software issue. Lockheed Martin Astronautics was on contract to supply everything to NASA in SI units (which is what NASA uses for everything). LMA - or at least the part the caused this problem - uses English (Imperial) units internally, and neglected to perform the appropriate conversion before they sent the data on to NASA.
Perhaps I should point out the bug: the comment "//check for \" ends with a pre-processor line-entesion character (\), which effective appends the next line onto the current line, thus the code "slashfound=1" is effectively commented out and the next statment (++index) only executes if c=='\'
Bottom line: that stuff about the floating point error in the PAC-2 system looks neat on paper but it's not at all clear that the faulty calculation was responsible for the loss of life.
GMD
watch this
Here are some of the best examples of windows crashing on high visibility systems that are relied upon:
in the street
At the airport
at the atm
on CNN
At disneyland
On your phone
In an airplane
At the bus stop
Thats not entirely true. Adding power will inccrease your altitude, but pulling up will too. When you pull up, you trade altitude for speed. In other words, youll go higher but your plane will be goins slower. Eventually you arent going fast enough to maintain level flight characteristics, so you have to add power or stop trying to go higher. In some cases youre right though, if you already are only going fast enough to maintain level flight, pulling back on the stick will slow you down and decrease your altitude, but this isnt always true. As for the person who didnt understand how adding power increased altitude, when you go faster, you increase the lift coming from your wings (since lift is a function of speed and angle of attack) so there is a net upward force on the aircraft, causing it to go upwards.
Slashdot Math!
cause we all know 50 + 1 - 1 = 49!
Ok, that was lame, go ahead and mod me down...
| - | - |
of the Tacoma Narrows bridge falling. The *fault* was with the design, and hence, the designers.
An extended bolt puncturing the gas tank during a rear end collision was the *cause* of Ford Pintos exploding. The *fault* was with the design, and hence, the designers.
Both of these items could have been claimed to be perfectly free of design flaws while being used as "intended."
This argument did not help the designers in not being found liable for their design flaws.
The divide by zero error was the *cause* of the operating system's failure. The *fault* was with the operating system. The *operating system* crashed. An operating system failure is *always* the fault of the operating system, and hence, its designers.
Read any textbook on the design of operating systems and in the first page or two you find some sort of statement along the line of, " A faulty app should never cause the operating system to fail." This is correct design.
Let me repeat. If an app fails, it is the fault of the app. If the operating system fails, no matter what an app has done, it is the fault of the operating system. An operating system must *assume* apps badly written by complete incompetents.
It doesn't matter what operating system. Windows, Linux, Mac or just the beads on your abacus.
* It is the responsibiltiy of the operating system not to fail.*
The fact that such failures can be explained away as the fault of the app by people who should know better makes me grieve for the state of engineering these days. It can only result in products being produced with greater and greater "craposity" factors eventually resulting in a culture of complete "crapitude."
KFG
You're all sorta right.. here is one of my favorite aviation pages It'll tell you more than you ever wanted to know about airplane physics (from a pilot's point of view). Chapter 1 covers these altitude/speed/power concepts...
"Flight instruments don't lie"
... it has an electronic AOA.
... no matter what the real AOA was.
... Fortunately, it was expensive and not lethal.
First, BEFORE YOU LEAVE THE GROUND, pilots are taught that instruments don't lie. Specifically, when the human inner ear is placed in flight, things go wrong (the inner ear canals are static, not dynamic, devices; the fluid has no dampening or rate sensors). When there is no external reference, the inner ear canals adjust to the eye's visual presentation. It's called the 'leans.' Bad joo-joo. Many a perfectly good aircraft has been flown into the ground because the pilot believed his ears and eyes and not his instruments.
Second, IN FLIGHT, angle-of-attack (AOA) is a spectacular indicator of where your airfoil exists within (or outside) the flight envelope for your aircraft. Inside the flight envelope, you can seek best range (mpg) or best endurance (loiter) or best climb.
In most aircraft, the angle-of-attack indicator is a manual instrument (on the skin is a sensor which looks like a big euro-style handle and it runs to an indicator in the cockpit).
Many pilots are correctly taught to 'fly' the angle-of-attack.
Third, ON THE GROUND, when you land, you use the aircraft shape as an airbrake. You hold the aircraft nose off the ground as long as possible to create drag.
Fourth, ON THE GROUND, when you land, you do not want to hold the aircraft nose too far off the ground or the tail will scrape the runway and your fitness report will reflect and you'll be the butt of bad jokes at Snopes for eternity.
The AOA is used to assist in the performance of aerodynmic braking. The aircraft performance manual publishes the tried and true range of AOAs for aerodynamic braking. [It also indicates when too much AOA will ding the aircraft.]
Aerodynamic braking is part art and part science and requires accurate instruments.
Enter the F-16
F-16 pilots were taught to fly the flight direction indicators to land.
However, many old and new pilots fell back on the old AOA once the wheels touched the ground to do aerodynamic braking.
Suddenly, F-16 tails were scraping along the runway at an alarming (and expensive) rate.
[As an aside, the problem was probably ignored until a senior officer ground off a few inches of aluminum THEN there was a problem.]
The programmers who wrote the AOA routines were rightly told that the AOA is used in flight. So, when the AOA detected that the aircraft had placed weight on the wheels (weight-on-wheels - WOW), it was programmed to quit working. Unfortunately, it kept the last AOA reading
Pilot flies, pilot lands, pilot believes instruments, pilot scrapes multi-million dollar aircraft's tail along runway.
The programming solution was simple: when there was WOW, fade the AOA.
This was another case when contracts pit spec wording against spec intent against functional application and understanding of how it's supposed to work
"Why did they call you 'sparky' and why are you driving school buses in North Topeka?"
A "large quantity of water" entered the storage tank because an employee who had just been fired dropped a hose into it out of spite (he didnt know what would happen, he just wanted to ruin something). Yes the safety precautions were under-par, but when someone with legitimate access wants to destroy something its pretty hard to prevent.
:).
And yes, this has nothing to do with programming error
I think I've recommended this book serveral times on Slashdot. Simply put, THE collection of computing related horror stories.
2 01 55805X/qid=1035769692/sr=8-13/ref=sr_8_13/104-4078 673-1863905?v=glance&n=507846
http://www.amazon.com/exec/obidos/tg/detail/-/0
I swear by MacOS X. Although I use to swear *at* MacOS 9...
If the Y2k bugs hadn't been fixed, things would have broken left and right, and we would have been blamed for not fixing them ahead of time.
Since the Y2k bugs were fixed, very few things broke, and we got blamed for wasting tons of money to no effect.
C'est la vie, I guess.