Slashdot Mirror
Halting Problem Proves That Lethal Robots Cannot Correctly Decide To Kill Humans
KentuckyFC writes: The halting problem is to determine whether an arbitrary computer program, once started, will ever finish running or whether it will continue forever. In 1936, Alan Turing famously showed that there is no general algorithm that can solve this problem. Now a group of computer scientists and ethicists have used the halting problem to tackle the question of how a weaponized robot could decide to kill a human. Their trick is to reformulate the problem in algorithmic terms by considering an evil computer programmer who writes a piece of software on which human lives depend.
The question is whether the software is entirely benign or whether it can ever operate in a way that ends up killing people. In general, a robot could never decide the answer to this question. As a result, autonomous robots should never be designed to kill or harm humans, say the authors, even though various lethal autonomous robots are already available. One curious corollary is that if the human brain is a Turing machine, then humans can never decide this issue either, a point that the authors deliberately steer well clear of.
The question is whether the software is entirely benign or whether it can ever operate in a way that ends up killing people. In general, a robot could never decide the answer to this question. As a result, autonomous robots should never be designed to kill or harm humans, say the authors, even though various lethal autonomous robots are already available. One curious corollary is that if the human brain is a Turing machine, then humans can never decide this issue either, a point that the authors deliberately steer well clear of.
And let's not forget the "Better at it than humans" heuristic. As long as "Jaywalking under the influence of melanin" is sometimes a capital crime, that's not a hard target to hit.
In your examples, there are humans in the loop.
In this case, you have a robot trying to autonomously decide "kill" or "don't kill" when it encounters a human.
Hospital equipment - it's generally observed by personnel who after failures can decide to not use the equipment further (see Therac 25), or that changes need to be made in order to use the equipment. The equipment never hooks itself up to a patient automatically and provides treatment without a human involved. Sure there are errors that kill people unintentionally, but then there's a human choice to simply take the equipment out of service. E.g., an AED is mostly autonomous, but if a model of AED consistently fails in its diagnosis, humans can easily replace said AED with a different model. (You can't trust said AED to take itself out of service).
Airplanes - you still have humans "in the loop" and there have been many a time when said humans have to be told that some equipment can't be used in the way it was used. Again, the airplane doesn't takeoff, fly, and land without human intervention. In bad cases, the FAA can issue a mandatory airworthiness directive that says said plane cannot leave the ground without changes being made. In which case human pilots check for those changes before they decide to fly it. The airplane won't take off on its own.
Traffic control - again, humans in the loop. You'll get accidents and gridlock when lights fail, but the traffic light doesn't force you to hit the gas - you can decide that because of the mess, to simply stay put and not get involved.
Remember, in an autonomous system, you need a mechanism to determine if the system is functioning normally. Of course, said system cannot be a part of the autonomous system, because anomalous behavior may be missed (it's anomalous, so you can't even trust the system that's supposed to detect the behavior).
In all those cases, the monitoring system is external and can be made to halt a anomalous system - equipment can be put aside and not used, avoiding hazardous situations by disobeying, etc.
Sure, humans are very prone to failure, that's why we have computers which are far less prone to failure, But the fact that a computer is far less prone to making an error doesn't mean we have to trust it implicitly because we're more prone to making a mistake. it's why we don't trust computers to do everything for us - we expect things to work but when indications are that it doesn't, we have measures to try to prevent a situation from getting worse.
That's how you read it? I read it as if you create a robot that tries to evaluate weather or not it should kill someone based on ethics the program will never complete. You can certainly make one that can always kill what you tell it to, but not one that can choose whether or not a given human is a rebel to be shot on site, or a human that is apart of the new world order. However, I'm more likely to have it kill all humans not holding an IFF tag.