Can Software Kill?

← Back to Stories (view on slashdot.org)

Posted by ryuzaki0 on Tuesday March 9, 2004 @08:52AM from the of-course dept.

mykepredko writes "Eweek has an interesting, if somewhat long article titled Can Software Kill? The article focuses on a programming error that resulted in 28 Panamanian cancer patients receiving many times an expected lethal dose of radiation. The article briefly mentions, but doesn't go into detail, the 1991 Patriot Missile Failure that resulted in the deaths of 28 American service men and women."

21 of 562 comments (clear)

Sure it can kill. by grub · 2004-03-09 08:53 · Score: 5, Funny

Can Software Kill?

Certainly. A complete set of Novell manuals dropped from 40 stories up packs the same kinetic energy as a 10 car freight train moving at 80 km/h.

--
Trolling is a art,
1. Re:Sure it can kill. by micromoog · 2004-03-09 08:56 · Score: 5, Funny
  
  Given the choice between that and actually reading them, I'll take the 40 stories. At least then I have an outside chance of survival.
2. Re:Sure it can kill. by Charlton+Heston · 2004-03-09 09:03 · Score: 5, Funny
  
  Any tree that would voluntarily take part in Novel documentation deserved to die.
  
  --
  Get your stinking paws off me you damn dirty ape
3. Re:Sure it can kill. by robslimo · 2004-03-09 09:06 · Score: 5, Informative
  
  Ha, ha.
  
  It's a serious topic, even more so since the over-radiation shit in Panama happened so recently.
  
  The infamous Therac-25 incidents happened between 1985 and 1987 and should be required reading... too bad the three Panamanian medical physicists cited in the article hadn't paid attention to it.
4. Re:Sure it can kill. by maxwell+demon · 2004-03-09 09:21 · Score: 5, Funny
  
  He was talking about throwing the manual, not the terminal. Although being hit by a terminal thrown from a few stories high might actually be terminal as well.
  
  --
  The Tao of math: The numbers you can count are not the real numbers.
Software that kills... by bmorton · 2004-03-09 08:53 · Score: 5, Funny

Apparently it can only kill people in groups of only 28.
1. Re:Software that kills... by fizban · 2004-03-09 08:55 · Score: 5, Funny
  
  Yeah, 7-bit operating systems kill in groups of 28. 8-bit systems kill in groups of 32.
  
  --
  +1 Insightful, -1 Troll. What can I say, I'm an Insightful Troll.
2. Re:Software that kills... by Adriax · 2004-03-09 08:55 · Score: 5, Funny
  
  That's a hardware limitation they hope to have fixed before too long.
  
  --
  I don't suffer from insanity, I enjoy every minute of it!
Of course! by zuikaku · 2004-03-09 08:54 · Score: 5, Funny

One must be very careful when you kill -9!
EULA's by onyxruby · 2004-03-09 08:54 · Score: 5, Interesting

If a software maker is found negligible and convicted of manslaughter (unintentionaly causing death) due to buggy software, would that void out the whole EULA business since they all claim they can't be held responsible? Or would the burden pass on the poor chap that used it for being irresponsible enough to use something where the maker couldn't be held accountable? Lets's face it, why are only software companies able to make themselves free from accountability when every other industry has to design for it?
1. Re:EULA's by Unknown+Relic · 2004-03-09 09:05 · Score: 5, Insightful
  
  I'm not positive, but aren't most of these type of disclaimers saying something along the lines of "We do not give permission for this software to be used in environments where failure could result in loss of life. In the event of such unauthorized use, we will not warranty the product, nor be held accountable for any damages it may cause"? If this is the case, than I have no problem with this, as they are saying the software isn't good enough to use in such a situation, if you do so, you're on your own. Anything that's mission critical to a degree where lives depend on it, should be licensed with that in mind (which I imagine software for nuclear power plants, etc. is).
  
  If the organization that's being entrusted with people's lives cheaps out and uses software in environments it's not rated for, there's no way the manufacturer should be held liable. It's not different than tires on cars. If you're ripping around at 150mph on non Z-rated tired, and one blows, it's your own damned fault, not that of the manufacturer.
2. Re:EULA's by stratjakt · 2004-03-09 09:13 · Score: 5, Interesting
  
  What other manufacturer would be held accountable?
  
  My TV comes with a warrantee, but that says they wont be liable for any damage or caused by the use of the tv.
  
  I bought a bucked of concrete paint a week ago. It's guaranteed not to fail, but that guarantee doesnt cover the cost to remove/strip/repair the damage caused by bad paint (thousands), just 20 bucks for a new can of paint.
  
  In court you'd have to prove negligence or deliberate behavior. You'd have to prove Sony designed the TV to electrocute you, etc.. The fact they get it UL listed is enough to get past that.
  
  For software you'd have to show that they deliberately put the flaws in, or knew about the flaws and didnt care (depraved indifference)..
  
  But I'm no lawyer so who knows.. Everyone can go fucking sue everyone else.
  
  All I know is if Dr Pib puts a family member on an untested, unproven life support system, and it fails, I'm suing the Doctor.
  
  --
  I don't need no instructions to know how to rock!!!!
Yes by paranode · 2004-03-09 08:54 · Score: 5, Insightful

Software can kill, just like any other stupid mistakes if left unchecked.

insert open source plug here
Tonight on Fox... by The+I+Shing · 2004-03-09 08:55 · Score: 5, Funny

Tonight on Fox...

WHEN SOFTWARE ATTACKS!
with host Mitch Pileggi

--
You are in error. No-one is screaming. Thank you for your cooperation.
Software cannot kill ... by maxwell+demon · 2004-03-09 08:57 · Score: 5, Insightful

... but it can make the hardware controlled by it kill.

--
The Tao of math: The numbers you can count are not the real numbers.
A dumb question, yet a good one by phorm · 2004-03-09 08:58 · Score: 5, Interesting

Can negligence in any area kill? Yes.
Software is no different from hardware in this aspect. If it is handling mission-critical or potentially-lethal equipment... great care should be taken to ensure its integrity.

Trusting those that make your irraditation software is no different from trusting the those that made your life-support hardware.

Human error, or mechanical, can mean death in both cases. If the error is glaring, it becomes a case of negligence.

Unfortunately in cases of software or even computer hardware operating environment becomes an often overlooked factor. Stress tests are needed... data collisions checked for, line noise, redundancy, etc. When we're talking about people's lives, that extra parity bit can be just as important as a backup-parachute...
Yes. It can. by Mr.+Slippery · 2004-03-09 09:01 · Score: 5, Informative

Sadly, this is nothing new.
Every software developer needs to read Peter Neuman's book Computer-Related Risks , and keep up with the Risks digest (comp.risks).
Learning from other's mistakes is much less painful.

--
Tom Swiss | the infamous tms | my blog
You cannot wash away blood with blood
Re:of course it will by Bombcar · 2004-03-09 09:04 · Score: 5, Insightful

You see, if I'm a doctor, and I screw up and overdose you, it isn't a news item. I'll get reprimanded, maybe sued. No one will even notice if it happens many times, because each time it is a different doctor in a different circumstance.

But if I'm a computer software engineer and have a bug in a program that gets 3 people an overdose, then it will be noticed and much howling will be done over it. Even if the total number of errors have gone down, the type of error is new and there is a common factor between all the cases. And so we will complain.

And, I think, rightly. Computers are a tool, not to be trusted, always to be checked. I fear many people believe the computer can never be wrong (because it is so complex as to be indistringuishable from magic, and magic is never wrong) - perhaps this is why there isn't much howling about Diebold voting machines: It's digital, so it must be better!

--
Fellowship 9/11
Sure it can by aduzik · 2004-03-09 09:06 · Score: 5, Insightful

Software is an engineered thing, just like any other tool upon which we rely. Think about airplanes, which occasionally have mechanical failures in flight. Think about Columbia, which burned up because of engineering defects. So, if the software is flawed, it will certainly cause eventual damage. Sometimes it's benign -- restarting Word isn't so big a deal -- but sometimes it's catastrophic.
This is why I've always thought it's vitally important to have good, precise specifications in place and excellent quality assurance for any life-critical application. It's even better with many eyes overseeing every step of the process -- wait... that smacks of open source, doesn't it?
If you ask me -- and you haven't, but I'll tell you anyway -- what would be the best way to prevent catastrophe, it would be to PREVENT CHANGES TO THE SPEC. In college, our software engineering prof. gave us an assignment, then halfway through, she changed the spec on us. Well, not surprisingly, there wasn't a single project that worked faultlessly, and many of us were doing really well before that.
Software itself doesn't kill people. Bad software written by overworked developers writing to a constantly-changing specification with not nearly enough QA does. That is, people inadvertantly -- we hope -- kill people with software. Yeah yeah, it's cliche, but it works.

--
If it's not one thing it's your mother.
Is this what you meant? by Anonymous Coward · 2004-03-09 09:09 · Score: 5, Funny

Microsoft Windows: A thirty-two bit extension and graphical shell to a sixteen-bit patch to an eight-bit operating system originally coded for a four-bit microprocessor which was written by a two-bit company that can't stand one bit of competition

-- author unknown
Re:You clueless cretin. by canajin56 · 2004-03-09 09:32 · Score: 5, Informative

The Davis-Besse nuclear reactor in Ohio was running its safty monitoring systems on an NT server. And it got infected by Slammer and crashed. Fortunatly, the system had an analog backup, and the reactor had already been offline all year, after inspectors discovered a 6" hole through the cement in the reactor head, which left the core exposed.

--
ASCII stupid question, get a stupid ANSI