Slashdot Mirror
Halting Problem Proves That Lethal Robots Cannot Correctly Decide To Kill Humans
KentuckyFC writes: The halting problem is to determine whether an arbitrary computer program, once started, will ever finish running or whether it will continue forever. In 1936, Alan Turing famously showed that there is no general algorithm that can solve this problem. Now a group of computer scientists and ethicists have used the halting problem to tackle the question of how a weaponized robot could decide to kill a human. Their trick is to reformulate the problem in algorithmic terms by considering an evil computer programmer who writes a piece of software on which human lives depend.
The question is whether the software is entirely benign or whether it can ever operate in a way that ends up killing people. In general, a robot could never decide the answer to this question. As a result, autonomous robots should never be designed to kill or harm humans, say the authors, even though various lethal autonomous robots are already available. One curious corollary is that if the human brain is a Turing machine, then humans can never decide this issue either, a point that the authors deliberately steer well clear of.
The question is whether the software is entirely benign or whether it can ever operate in a way that ends up killing people. In general, a robot could never decide the answer to this question. As a result, autonomous robots should never be designed to kill or harm humans, say the authors, even though various lethal autonomous robots are already available. One curious corollary is that if the human brain is a Turing machine, then humans can never decide this issue either, a point that the authors deliberately steer well clear of.
By the same logic, computers should not be allowed in any life-critical situation. That includes hospital equipment, airplanes, traffic control, etc. etc.
Fortunately, we don't judge the reliability of computers based on the ability to mathematically prove that nobody has put evil code in on purpose.
The premise of TFA is that killer robots need to be perfect. They don't. They just need to be better than humans.
Which is more likely to shoot a civilian:
1. A carefully programmed and thoroughly tested robot.
2. A scared, tired, and jumpy 18 year old soldier, who hasn't slept in two days, and saw his best friend get his legs blown off by a booby trap an hour ago.
1, when ordered to shoot civilians.
Product liability never results in anyone being actually responsible for the death going to jail or huge penalties.
A multinational might> pay out a couple of million in product liability, but then it will just be chalked up to the cost of doing business.
If the multinational is a defense contractor (BAE, Raytheon, Lockheed, General Dynamics, etc), it will all be swept under the rug and more money will be thrown at the contractor to "fix" it.
That's the reality.
--
BMO
So we're cutting down the criteria to not just people carrying guns, but people carrying guns actively shooting at you?
Actually, the definition of civilian is well-defined in the Laws of War, commonly codified today in international laws by Protocol I of the Geneva Conventions.
In sum, a "civilian" is anyone who is not a "privileged combatant," i.e., basically someone (1) carrying arms, (2) taking orders in an organized military structure, and (3) following the laws and customs of warfare. (Also, usually privileged combatants are required to wear insignia.)
Someone who carries arms but does not satisfy those criteria is still a "civilian," though if those arms are actively used in support of an organized military force, he/she may be a civilian who is also an "unprivileged combatant," i.e., he/she not eligible for protection under the normal rules for prisoners of war.
So, actually the criteria are much more specific than you describe. "Civilians" can fight in wars, in which case they become "combatants," but they do not cease to be "civilians," as the term is commonly understood in contrast to organized military personnel.
As for the farmer in GGP's example, he's clearly a civilian unless he's a member of a military force. If he carries a gun but only for his own protection and does not engage in direct action against an enemy, he is probably assumed to be a "non-combatant" as well, under international legal definitions.