Halting Problem Proves That Lethal Robots Cannot Correctly Decide To Kill Humans

KentuckyFC writes: The halting problem is to determine whether an arbitrary computer program, once started, will ever finish running or whether it will continue forever. In 1936, Alan Turing famously showed that there is no general algorithm that can solve this problem. Now a group of computer scientists and ethicists have used the halting problem to tackle the question of how a weaponized robot could decide to kill a human. Their trick is to reformulate the problem in algorithmic terms by considering an evil computer programmer who writes a piece of software on which human lives depend.

The question is whether the software is entirely benign or whether it can ever operate in a way that ends up killing people. In general, a robot could never decide the answer to this question. As a result, autonomous robots should never be designed to kill or harm humans, say the authors, even though various lethal autonomous robots are already available. One curious corollary is that if the human brain is a Turing machine, then humans can never decide this issue either, a point that the authors deliberately steer well clear of.

Comments

I think

[fyngyrz]

I'm just going to reformulate the problem by considering idiots who use unrealistic, not-supported-by-evidence premises to make general statements as one that calls for sending killer robots after said idiots.

Re:I think

[ShanghaiBill]

The premise of TFA is that killer robots need to be perfect. They don't. They just need to be better than humans.

Which is more likely to shoot a civilian:
1. A carefully programmed and thoroughly tested robot.
2. A scared, tired, and jumpy 18 year old soldier, who hasn't slept in two days, and saw his best friend get his legs blown off by a booby trap an hour ago.

Re:I think

[Anonymous Coward]

1, when ordered to shoot civilians.

Re:I think

[bmo]

Product liability never results in anyone being actually responsible for the death going to jail or huge penalties.

A multinational might> pay out a couple of million in product liability, but then it will just be chalked up to the cost of doing business.

If the multinational is a defense contractor (BAE, Raytheon, Lockheed, General Dynamics, etc), it will all be swept under the rug and more money will be thrown at the contractor to "fix" it.

That's the reality.

--
BMO

Re:I think

[AthanasiusKircher]

So we're cutting down the criteria to not just people carrying guns, but people carrying guns actively shooting at you?

Actually, the definition of civilian is well-defined in the Laws of War, commonly codified today in international laws by Protocol I of the Geneva Conventions.

In sum, a "civilian" is anyone who is not a "privileged combatant," i.e., basically someone (1) carrying arms, (2) taking orders in an organized military structure, and (3) following the laws and customs of warfare. (Also, usually privileged combatants are required to wear insignia.)

Someone who carries arms but does not satisfy those criteria is still a "civilian," though if those arms are actively used in support of an organized military force, he/she may be a civilian who is also an "unprivileged combatant," i.e., he/she not eligible for protection under the normal rules for prisoners of war.

So, actually the criteria are much more specific than you describe. "Civilians" can fight in wars, in which case they become "combatants," but they do not cease to be "civilians," as the term is commonly understood in contrast to organized military personnel.

As for the farmer in GGP's example, he's clearly a civilian unless he's a member of a military force. If he carries a gun but only for his own protection and does not engage in direct action against an enemy, he is probably assumed to be a "non-combatant" as well, under international legal definitions.

By the same logic

[Anonymous Coward]

By the same logic, computers should not be allowed in any life-critical situation. That includes hospital equipment, airplanes, traffic control, etc. etc.

Fortunately, we don't judge the reliability of computers based on the ability to mathematically prove that nobody has put evil code in on purpose.

Re:By the same logic

[Dutch Gun]

Agreed. The authors set up a nearly impossibly complex ethical dilemma that would freeze even a human brain into probable inaction, let alone a computer one, and then claims "See? Because a computer can't guarantee the correct outcome, we can therefore never let a computer make that decision." It seems to be almost the very definition of a straw man to me.

The entire exercise seems to be a deliberate attempt to reach this conclusion, which they helpfully spell out in case anyone missed the not-so-subtle lead: "Robots should not be designed solely or primarily to kill or harm humans."

I'm in no hurry to turn loose an army of armed robots either, but saying that you can "prove" that an algorithm can't make a fuzzy decision 100% of the time? Well, yeah, no shit. A human sure as hell can't either. But what if the computer can do it far more accurately in 99% of the cases, because it doesn't have all those pesky "I'm fearing for my life and hopped up on adrenaline so I'm going to shoot now and think later" reflexes of a human?

The human brain is not a Turing machine

[Anonymous Coward]

Exhibit A, the human skull: Not enough room for an infinite tape.

Impossible to build purely evil robots?

[Galaga88]

Presuming that this proof reached via impressively tortured logic does have merit: Does it mean that it is also impossible to build a purely evil robot that would always kill maliciously?

Bad Headline as Usual

[Jaime2]

What the paper said is that computers can't provably always make the right choice. Neither can we. I'll bet computers are capable of doing a lot better than humans, especially given the rate of the increase in the number of things a computer can do compared to the rate that humans are (aren't) gaining new abilities.

National Robots Association

[c]

"Robots don't kill people. Robot programmers kill people."

Re:only incorrectly device to kill humans?

[Jason Levine]

If you have a problem with your Killbot's operation, please call 1-800-KILL-HMNS and we'll send a customer service Killbot to execute your trouble ticket right away. We won't rest until there are no bug reports submitted by humans.