Can Software Kill?

mykepredko writes "Eweek has an interesting, if somewhat long article titled Can Software Kill? The article focuses on a programming error that resulted in 28 Panamanian cancer patients receiving many times an expected lethal dose of radiation. The article briefly mentions, but doesn't go into detail, the 1991 Patriot Missile Failure that resulted in the deaths of 28 American service men and women."

Comments

EULA's

[onyxruby]

If a software maker is found negligible and convicted of manslaughter (unintentionaly causing death) due to buggy software, would that void out the whole EULA business since they all claim they can't be held responsible? Or would the burden pass on the poor chap that used it for being irresponsible enough to use something where the maker couldn't be held accountable? Lets's face it, why are only software companies able to make themselves free from accountability when every other industry has to design for it?

Re:EULA's

[stratjakt]

What other manufacturer would be held accountable?

My TV comes with a warrantee, but that says they wont be liable for any damage or caused by the use of the tv.

I bought a bucked of concrete paint a week ago. It's guaranteed not to fail, but that guarantee doesnt cover the cost to remove/strip/repair the damage caused by bad paint (thousands), just 20 bucks for a new can of paint.

In court you'd have to prove negligence or deliberate behavior. You'd have to prove Sony designed the TV to electrocute you, etc.. The fact they get it UL listed is enough to get past that.

For software you'd have to show that they deliberately put the flaws in, or knew about the flaws and didnt care (depraved indifference)..

But I'm no lawyer so who knows.. Everyone can go fucking sue everyone else.

All I know is if Dr Pib puts a family member on an untested, unproven life support system, and it fails, I'm suing the Doctor.

A dumb question, yet a good one

[phorm]

Can negligence in any area kill? Yes.
Software is no different from hardware in this aspect. If it is handling mission-critical or potentially-lethal equipment... great care should be taken to ensure its integrity.

Trusting those that make your irraditation software is no different from trusting the those that made your life-support hardware.

Human error, or mechanical, can mean death in both cases. If the error is glaring, it becomes a case of negligence.

Unfortunately in cases of software or even computer hardware operating environment becomes an often overlooked factor. Stress tests are needed... data collisions checked for, line noise, redundancy, etc. When we're talking about people's lives, that extra parity bit can be just as important as a backup-parachute...

ethics & liability

[v_1_r_u_5]

There must be a point where software makers can no longer say "DISCLAIMER: IF WE BREAK YOUR MACHINE, IT'S NOT OUR FAULT." If you look at every piece of software's license, you'll see a clause like that. Imagine if every industry took that approach:

DISCLAIMER: IF YOUR CAR'S BRAKES FAIL, IT'S NOT OUR FAULT. TOUGH LUCK!

DISCALIMER: IF THIS MEDICINE KILLS YOU, OH WELL.. NOT OUR FAULT!

etc.

Some laws must be passed and software makers must be held accountable- they should no longer be able to hide under the big umbrella of the disclaimer.

We Need Software *Engineers*

[Vagary]

The problem is that in every other development environment, the legal liability ultimately rests on the engineer who signed off on the quality assurance. But because software developers are not professionals and have no professional code of conduct, their signatures are meaningless. The only way software can become as reliable as other engineered products is to create the profession of software engineering*. And I'm not just talking about giving CompSci students a ring: many CompSci curriculums don't require any engineering techniques at all, and those that do usually devote less time to engineering than they do to sorting algorithms. The software industry requires fundamental changes, and legal liability is at most the catalyst.

* Yes, I know there are a couple of schools out there that offer SoftEng degrees, but until industry distinguishes them from CompScists and requires the engineering designation for key positions they are meaningless.

Worry About This Every Day

[Chokai]

The next time you visit the doctor watch the workflow of the office staff. Increasingly chances are they will probably be entering your medical information, and I mean the clinical stuff, not your address into some type of computer system.

I currently work for a small Electronic Medical Records company. At some level I worry about potentially killing someone every day. In fact our bug tracking tool has a special category in it called "Patient Safety" which is the highest priority bug. We deal with things most of you probably wouldn't think of such as a tool for writing Prescriptions, which given the fact that many drugs interact ( potentially fatally) has to catch and alert the physician to such cases. I also deal with lab results which if reported incorrectly could lead to a potentially fatal decision by the doctor and so forth.

Consultants and pundits like to say that computer control reduces the chances of human error and failure, this is said IMO to comfort the masses. To state the obvious I suspect EVERYONE on Slashdot knows that in reality that statement is not true, the human error has just been moved to a different point in the chain. A tired programmer is just as likely to make a mistake as a tired machinery operator. The difference is that that software might be used by 5,000 machines, whearas that operator runs 1.

SCADA software certainly can...

[blueZ3]

In a former life ( :-> ) I was employed by a large multi-national that worked with utilities. Some of our software used SCADA protocols to remotely switch breakers - not household breakers, these switches control significant segments of the US power grid. All the software and documentation contained numerous warnings, because if a utility employee manually switched of a segment to make repairs, and switch was remotely turned on, someone could be killed. There are numerous other software applications that control (potentially) deadly devices - robots, industrial equipment, etc. Failure of the software, or problems with operator headspace, create a potential for death when working with almost any software that controls physical entities.