Slashdot Mirror
Halting Problem Proves That Lethal Robots Cannot Correctly Decide To Kill Humans
KentuckyFC writes: The halting problem is to determine whether an arbitrary computer program, once started, will ever finish running or whether it will continue forever. In 1936, Alan Turing famously showed that there is no general algorithm that can solve this problem. Now a group of computer scientists and ethicists have used the halting problem to tackle the question of how a weaponized robot could decide to kill a human. Their trick is to reformulate the problem in algorithmic terms by considering an evil computer programmer who writes a piece of software on which human lives depend.
The question is whether the software is entirely benign or whether it can ever operate in a way that ends up killing people. In general, a robot could never decide the answer to this question. As a result, autonomous robots should never be designed to kill or harm humans, say the authors, even though various lethal autonomous robots are already available. One curious corollary is that if the human brain is a Turing machine, then humans can never decide this issue either, a point that the authors deliberately steer well clear of.
The question is whether the software is entirely benign or whether it can ever operate in a way that ends up killing people. In general, a robot could never decide the answer to this question. As a result, autonomous robots should never be designed to kill or harm humans, say the authors, even though various lethal autonomous robots are already available. One curious corollary is that if the human brain is a Turing machine, then humans can never decide this issue either, a point that the authors deliberately steer well clear of.
I'm just going to reformulate the problem by considering idiots who use unrealistic, not-supported-by-evidence premises to make general statements as one that calls for sending killer robots after said idiots.
I've fallen off your lawn, and I can't get up.
By the same logic, computers should not be allowed in any life-critical situation. That includes hospital equipment, airplanes, traffic control, etc. etc.
Fortunately, we don't judge the reliability of computers based on the ability to mathematically prove that nobody has put evil code in on purpose.
Exhibit A, the human skull: Not enough room for an infinite tape.
Presuming that this proof reached via impressively tortured logic does have merit: Does it mean that it is also impossible to build a purely evil robot that would always kill maliciously?
Englert and co say a robot can never solve this conundrum because of the halting problem. This is the problem of determining whether an arbitrary computer program, once started, will ever finish running or whether it will continue forever.
This is simply incorrect. The conundrum (RTFA for details) doesn't involve an arbitrary computer program. It involves a computer program that performs a specific known function. It is perfectly possible for an automated system to verify any reasonable implementation of the known function against the specification. If such a system fails it is because byzantine coding practices have been used - in which case, guilt can be assumed. The Halting problem doesn't apply unless you HAVE to get a correct answer for ALL programs. In this case you just have to get a correct answer for reasonable programs.
No, they can't and it shows.. Furthermore, humans aren't qualified to rule over other humans either. *Might makes right* will always come out on top. That is how nature works.
“He’s not deformed, he’s just drunk!”
What the paper said is that computers can't provably always make the right choice. Neither can we. I'll bet computers are capable of doing a lot better than humans, especially given the rate of the increase in the number of things a computer can do compared to the rate that humans are (aren't) gaining new abilities.
What a silly article, and a waste of three minutes to read it. What they actually showed is that it's possible to construct a scenario in which it's impossible to know for certain what the best decision is, due to lack of information.
That fact, and their argument, is true whether it's AI making the decision or a human. Sometimes you can't know the outcome of your decisions. So what, decisions still must be made, and can be made.
Their logic also falls down completely because the logic is basically:
a) It's possible to imagine one scenario involving life and death scenario in which you can't be sure of the outcome.
b) Therefore, no life-and-death decisions can be made.
(wrong, a) just means that _some_ decisions are hard to make, not that _all_ decisions are impossible to make).
Note the exact same logic is true without the "life-and-death" qualifier:
a) In some situations, you don't know what the outcome of the decision will be.
b) Therefore, no decisions can be made (/correctly).
Again, a) applies to some, not to all. Secondly, just because you can't prove ahead of time which decision will have the best outcome doesn't mean you make make a decision, and even know that that is the correct decision. An example:
I offer to make a bet with you regarding the winner of this weekend's football game.
I say you have to give me a 100 point spread, meaning your team has to win by at least 100 points or else you have to pay me.
It's an even-money bet.
The right decision is to not make the bet, because you'd almost surely lose. Sure, it's _possible_ that your team might win by 150 points, so it's _possible_ that taking the bet would have the best outcome. That's a very unlikely outcome, though, so the correct decision _right_now_ is to decline the bet. What happens later, when the game is played, has no effect on what the correct decision was today.
Fry: "I heard one time you single-handedly defeated a horde of rampaging somethings in the something something system"
Brannigan: "Killbots? A trifle. It was simply a matter of outsmarting them."
Fry: "Wow, I never would've thought of that."
Brannigan: "You see, killbots have a preset kill limit. Knowing their weakness, I sent wave after wave of my own men at them until they reached their limit and shut down."