Can Software Kill?

mykepredko writes "Eweek has an interesting, if somewhat long article titled Can Software Kill? The article focuses on a programming error that resulted in 28 Panamanian cancer patients receiving many times an expected lethal dose of radiation. The article briefly mentions, but doesn't go into detail, the 1991 Patriot Missile Failure that resulted in the deaths of 28 American service men and women."

Comments

Yes

[paranode]

Software can kill, just like any other stupid mistakes if left unchecked.

insert open source plug here

Software cannot kill ...

[maxwell demon]

... but it can make the hardware controlled by it kill.

Re:of course it will

[Bombcar]

You see, if I'm a doctor, and I screw up and overdose you, it isn't a news item. I'll get reprimanded, maybe sued. No one will even notice if it happens many times, because each time it is a different doctor in a different circumstance.

But if I'm a computer software engineer and have a bug in a program that gets 3 people an overdose, then it will be noticed and much howling will be done over it. Even if the total number of errors have gone down, the type of error is new and there is a common factor between all the cases. And so we will complain.

And, I think, rightly. Computers are a tool, not to be trusted, always to be checked. I fear many people believe the computer can never be wrong (because it is so complex as to be indistringuishable from magic, and magic is never wrong) - perhaps this is why there isn't much howling about Diebold voting machines: It's digital, so it must be better!

Re:EULA's

[Unknown Relic]

I'm not positive, but aren't most of these type of disclaimers saying something along the lines of "We do not give permission for this software to be used in environments where failure could result in loss of life. In the event of such unauthorized use, we will not warranty the product, nor be held accountable for any damages it may cause"? If this is the case, than I have no problem with this, as they are saying the software isn't good enough to use in such a situation, if you do so, you're on your own. Anything that's mission critical to a degree where lives depend on it, should be licensed with that in mind (which I imagine software for nuclear power plants, etc. is).

If the organization that's being entrusted with people's lives cheaps out and uses software in environments it's not rated for, there's no way the manufacturer should be held liable. It's not different than tires on cars. If you're ripping around at 150mph on non Z-rated tired, and one blows, it's your own damned fault, not that of the manufacturer.

Sure it can

[aduzik]

Software is an engineered thing, just like any other tool upon which we rely. Think about airplanes, which occasionally have mechanical failures in flight. Think about Columbia, which burned up because of engineering defects. So, if the software is flawed, it will certainly cause eventual damage. Sometimes it's benign -- restarting Word isn't so big a deal -- but sometimes it's catastrophic.

This is why I've always thought it's vitally important to have good, precise specifications in place and excellent quality assurance for any life-critical application. It's even better with many eyes overseeing every step of the process -- wait... that smacks of open source, doesn't it?

If you ask me -- and you haven't, but I'll tell you anyway -- what would be the best way to prevent catastrophe, it would be to PREVENT CHANGES TO THE SPEC. In college, our software engineering prof. gave us an assignment, then halfway through, she changed the spec on us. Well, not surprisingly, there wasn't a single project that worked faultlessly, and many of us were doing really well before that.

Software itself doesn't kill people. Bad software written by overworked developers writing to a constantly-changing specification with not nearly enough QA does. That is, people inadvertantly -- we hope -- kill people with software. Yeah yeah, it's cliche, but it works.