Slashdot Mirror
Software Problem Linked to Osprey Crash
An Anonymous Coward sent in: "While not the only problem facing the ill-fated V-22 Osprey, a bug in the software controlling the pitch of the Osprey's rotors was listed as a contributing factor to the crash of a Marine Opsrey last December. It appears that a hydraulic leak initiated a sequence of events that included the pilot pressing a computer reset button. Rather than resetting the computer, the software changed the pitch of the rotors. Not so good... One more reason to fear too much technology. Has anybody ever seen a bug-free piece of software of any complexity greater than "Hello World"?"
anyone who has ever written anything, 'nuff said, right?
.
Now, what's described in this Osprey crash, that's definately a bug. The expected behavior was a reset - is the expected behavior of a reset to change the pitch of the rotor? Then the bug was in the TRAINING procedure that recommended hitting that button.
In the real world, bugs range from, mispelled words in dialog boxes, to crashes, to having an OK button two pixels too small. It's all a matter of the opinion of any single user of a peice of software, if a given feature "worked". Worked how? Fulfilled the expected requirements? In 99.999% of the cases out there, the requirements were not well enough defined. For that matter, when you THINK you've got them defined well enough, you start running into semantics issues that would make a lawyer drool. Marketing guy writes requirements, engineer interprets requirements. It's a beautiful world eh? You thought you were writing "Hello World", but now you've got to spit it out in 50 different languages on 10 different OS/Hardware platform combinations, and it's got to be able to notify SNMP and email the administrator if it was unable to do so. And you've got to be able to get it to print out Hello World in the correct language from a single mouse click, and for some languages it has to determine the time of day, and present the greeting as a "good morning, world" or "good evening, world".
Pretty soon you're talking about 200 pages of specs. .
These are my friends, See how they glisten. See this one shine, how he smiles in the light.
Go download the core (e.g. written in CWEB) part of TeX, which was written by probably the greatest computer scientist of our day, Donald Knuth. I don't think a bug has been found in TeX in at least a decade. It's gotten to the point where Knuth will cut you a check for several hundred "hexadecimal" dollars (256 cents ** years since he made the offer) if you find one, which you would never cash anyways but rather mount it on your wall. TeX is definitely a bit more complex than "Hello World". Many people in the publishing industry will tell you that the features it provides could be sold for many thousands of dollars or even tens of thousands as a closed-source software package. It's highly complex.
--
I think there is a world market for maybe five personal web logs.
I think the real problem is the change in the meaning of the word art in the past ~100 years. What most people think of these days when they hear the word "Art" is what is traditionally thought of as "Fine Arts". The traditional meaning of the word Art was more akin to "Technique". The phrase "State of the art" is refering to the most advanced techniques available. This is also why many universities still call their science departments "The college of arts and letters".
It used to be that professions that required a lot of talent and/or practice in order to master a technique, such as painting or dancing were considered the "fine arts". Then, in the latter part of the 19th century or early part of the 20th century some revolutionary fine artists decided to discard these old-school techniques. In the process they succeeded in changing the traditional notion of art from something that was more technique to something that was more creative.
Unfortunately, the traditional use of the word art wasn't obilterated in the process, so many early computer scientists (particularly Knuth) started to talk about "The Art of Computer Programming." What they meant wasn't that it's a creative process, but rather that it's a technique that must requires talent and must be practiced. Knuth preaches a lot of things about computer programming: programs should be simple and comprehendable, they should implement algorithms that are mathematically proven. It might not be anything as formal as methodologies like ISO-9000, but there's no implication of creativity like, "Let's see if we can fit 30 function points on one line because that would be so cool!"
Anyway, I think the only reasonable solution at this point is to abandon the antique word "art" and start talking about software development as a "technique" or other similar term.
My $.02,
-"Zow"
NASA's software development methodology is remarkable. Their work should be the standard by which every programmer measures himself against.
Unfortunately, most programmers are underinformed, and haven't the foggiest idea that there's are methodologies that will reduce their error rates, increase their productivity, and meet their customers' needs fully.
It's a shame that programming still has this bullshit mystique of "art" to it. "Art" is just a lame excuse for laziness: instead of approaching the problems methodically and scientifically, it's just ever-so-much easier to take a half-assed hack-and-patch approach.
I think I'd better stop here, before I really kick into a rant...
--
--
Don't like it? Respond with words, not karma.
I can't believe nobody's posted about Knuth yet. Donald E. Knuth is famous for writing high-quality software, and even proving some of it (all of it?) correct. He offers rewards to people who find bugs in his code. The reward for TeX and METAFONT is described here: http://www-cs-faculty.stanford.edu/~knuth/abcde.ht ml, under the heading "Rewards".
--Jim
I was going to cite the Shuttle as well. Here is a great article about it.
The important question to ask is "Why doesn't good software exist?". The reason is that the consumer wants it yesterday, they will pay for an unfinished and incomplete project (nee almost all windows software). This leads to cut & paste programing, re-use of lots of pre-done code supplied by various manufacturers and the like. If the customer demanded well sorted out code, and refused to "upgrade" to the next best thing then you will see better coding.
Last and not least lets not forget the customer, the US govt. The Osprey has always been considered a boondogle from the get go. The pressure from congress and media to get the progect completed has been extremely intense. I'm sure a lot of pressure was put on the coding team to "get it done now". It's too bad that they didn't learn from the Space Shuttle, over budget, and over designed, it works. It has passed all expectations and is doing things now that it was never designed to do.
There is an engineering credo there, the famous "factor of saftey". A number usually applied to equations to take into affect the "unknown". Looks like the Osprey designers didn't make their "factor of saftey" high enough.
"Science is about ego as much as it is about discovery and truth " - I said it, so sue me.
A group of technical leaders were at a seminar called "Making Reliable Software". The teacher at the seminar, in order to make the participants appreciate the problems of reliable software, posed this question: "How many of you would be willing to fly in an airplane that was controlled by software that your company created?" The technical leaders thought about it, and looked around at each other. Nobody raised their hands, except for one lone guy in the back. The teacher was suprised. "You, in the back," he asked. "Why are you so sure that it would be safe?" "Simple," replied the programmer. "Knowing how my guys code, the plane wouldn't even be able to pull away from the gate."
Back in early 90's I worked for a defense contractor that was involved with the testing of the V22's engines. I remember sitting in a meeting and looking at some of the mechanical drawings and saying to myself, "glad I'm not the test pilot". It is a very complicated design. Most military aircraft like the V22 have quadruple redundant electronic systems and even the engine ECUs will talk to each other and monitor the one that is currently in charge and if the others determine that the main ECU is malfunctioning the other three will vote to remove it. Very complicated algorithms. Also to maintain that both rotors have power with one engine failing a high speed drive shaft is used to transfer power from the good engine to the transmission on the side that lost power. The drive shaft has a very small diameter but would spin at something like 10,000 rpms. Low torque but high power. The designers are thinking about safety and I think this bird can work its just not being funded with the enthusiasm of the old cold-war projects.
Actually, a decent embedded system (RTOS or not) will re-boot itself when it needs to. That is, you got a "watchdog timer" which the main program loop keeps resetting; if it doesn't get reset, it resets the computer. But apparently they gave the pilot a pushbutton just in case the watchdog timer crapped out too. But he pushed it when the real trouble was a chafed-through hydraulic line. And when it didn't help, he pushed it nine more times...
As I see it (and I'm not claiming expertise), the problems were, in descending order of importance:
1. The designers failed to route and fasten hydraulic lines so they wouldn't chafe.
2. The Marine Corps and aircraft mfg both ignored warnings about chafing lines.
3. The system for some reason told the pilot he needed to reset the computers although the computer had nothing to do with the hydraulics failure.
4. The software changed propellor settings erratically while booting up.
5. They didn't give the pilot an effective manual override for any controls the computer might be messing up.
6. The pilot panicked a little. Understandable, but pilots and Marines are supposed to be panic-proof...