Slashdot Mirror
Halting Problem Proves That Lethal Robots Cannot Correctly Decide To Kill Humans
KentuckyFC writes: The halting problem is to determine whether an arbitrary computer program, once started, will ever finish running or whether it will continue forever. In 1936, Alan Turing famously showed that there is no general algorithm that can solve this problem. Now a group of computer scientists and ethicists have used the halting problem to tackle the question of how a weaponized robot could decide to kill a human. Their trick is to reformulate the problem in algorithmic terms by considering an evil computer programmer who writes a piece of software on which human lives depend.
The question is whether the software is entirely benign or whether it can ever operate in a way that ends up killing people. In general, a robot could never decide the answer to this question. As a result, autonomous robots should never be designed to kill or harm humans, say the authors, even though various lethal autonomous robots are already available. One curious corollary is that if the human brain is a Turing machine, then humans can never decide this issue either, a point that the authors deliberately steer well clear of.
The question is whether the software is entirely benign or whether it can ever operate in a way that ends up killing people. In general, a robot could never decide the answer to this question. As a result, autonomous robots should never be designed to kill or harm humans, say the authors, even though various lethal autonomous robots are already available. One curious corollary is that if the human brain is a Turing machine, then humans can never decide this issue either, a point that the authors deliberately steer well clear of.
I'm just going to reformulate the problem by considering idiots who use unrealistic, not-supported-by-evidence premises to make general statements as one that calls for sending killer robots after said idiots.
I've fallen off your lawn, and I can't get up.
By the same logic, computers should not be allowed in any life-critical situation. That includes hospital equipment, airplanes, traffic control, etc. etc.
Fortunately, we don't judge the reliability of computers based on the ability to mathematically prove that nobody has put evil code in on purpose.
Exhibit A, the human skull: Not enough room for an infinite tape.
Does that mean we have to file a bug report if they decide to kill a human?
Presuming that this proof reached via impressively tortured logic does have merit: Does it mean that it is also impossible to build a purely evil robot that would always kill maliciously?
Englert and co say a robot can never solve this conundrum because of the halting problem. This is the problem of determining whether an arbitrary computer program, once started, will ever finish running or whether it will continue forever.
This is simply incorrect. The conundrum (RTFA for details) doesn't involve an arbitrary computer program. It involves a computer program that performs a specific known function. It is perfectly possible for an automated system to verify any reasonable implementation of the known function against the specification. If such a system fails it is because byzantine coding practices have been used - in which case, guilt can be assumed. The Halting problem doesn't apply unless you HAVE to get a correct answer for ALL programs. In this case you just have to get a correct answer for reasonable programs.
No, they can't and it shows.. Furthermore, humans aren't qualified to rule over other humans either. *Might makes right* will always come out on top. That is how nature works.
“He’s not deformed, he’s just drunk!”
What the paper said is that computers can't provably always make the right choice. Neither can we. I'll bet computers are capable of doing a lot better than humans, especially given the rate of the increase in the number of things a computer can do compared to the rate that humans are (aren't) gaining new abilities.
that a general algorithm to solve the halting problem for all possible program-input pairs cannot exist
The article misunderstands the halting problem. You could replace robots with humans and murder with any descision involing other people and come to the same conclusion. AI does not try to create perfect solutions. Instead you try to create solutions that work most of the time. Approaches that can evolve with trail and error. Ethically you weigh the positive benifits of success against the negative consequences of your failures.
John: Just put up your hand and say, 'I swear I won't kill anyone.'
Terminator: [Raises hand] I swear I will not kill anyone.
[stands up and shoots the guard on both knees]
He'll live.
What a silly article, and a waste of three minutes to read it. What they actually showed is that it's possible to construct a scenario in which it's impossible to know for certain what the best decision is, due to lack of information.
That fact, and their argument, is true whether it's AI making the decision or a human. Sometimes you can't know the outcome of your decisions. So what, decisions still must be made, and can be made.
Their logic also falls down completely because the logic is basically:
a) It's possible to imagine one scenario involving life and death scenario in which you can't be sure of the outcome.
b) Therefore, no life-and-death decisions can be made.
(wrong, a) just means that _some_ decisions are hard to make, not that _all_ decisions are impossible to make).
Note the exact same logic is true without the "life-and-death" qualifier:
a) In some situations, you don't know what the outcome of the decision will be.
b) Therefore, no decisions can be made (/correctly).
Again, a) applies to some, not to all. Secondly, just because you can't prove ahead of time which decision will have the best outcome doesn't mean you make make a decision, and even know that that is the correct decision. An example:
I offer to make a bet with you regarding the winner of this weekend's football game.
I say you have to give me a 100 point spread, meaning your team has to win by at least 100 points or else you have to pay me.
It's an even-money bet.
The right decision is to not make the bet, because you'd almost surely lose. Sure, it's _possible_ that your team might win by 150 points, so it's _possible_ that taking the bet would have the best outcome. That's a very unlikely outcome, though, so the correct decision _right_now_ is to decline the bet. What happens later, when the game is played, has no effect on what the correct decision was today.
"Robots don't kill people. Robot programmers kill people."
Log in or piss off.
Fry: "I heard one time you single-handedly defeated a horde of rampaging somethings in the something something system"
Brannigan: "Killbots? A trifle. It was simply a matter of outsmarting them."
Fry: "Wow, I never would've thought of that."
Brannigan: "You see, killbots have a preset kill limit. Knowing their weakness, I sent wave after wave of my own men at them until they reached their limit and shut down."
branch to the HCF operand on any error.
(newbies, that is the Halt, Catch Fire command)
if this is supposed to be a new economy, how come they still want my old fashioned money?
The flaw in their logic is this, we don't really care I if works every time, just most of the time. So if the robot can do the right thing more often than not, rather like people, to such a degree that we view it as being a net benefit, we are willing to accept the occasional mistake or failure for a net overall viewed good. So they would have to prove the program would fail more often than succeed, which they probably can't. That said, I DID wish it were possible enforce Asimov's laws of robotics. Maybe some day..
When theory conflicts with observation, You have two choices. You can modify your theory to fit the observation, or your observations to fit your theory. The first choice is what we generally regard as science. The second choice occurs in a number of circumstances including, but by no means limited to: religion, politics, mental illness, and general stupidity.
Note, checking to make sure that your observations are accurate is not the same thing as modifying them. "Did I fail to see the gorilla?" is valid when theory indicates gorillas should be present. "I saw a gorilla because my guru said I should" isn't.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
Or use the current method ... "Kill them all and let $DIETY sort it out."
"Transparent" is a shit show that trades on every stereotype going. A man in drag is NOT a transsexual.
My original post was simply pointing out that the human brain is NOT and can never be a Turing machine due to the fundamental randomness of the universe. This means that no study of Turing Machines can make any claim on human judgment calls.
I am not sure the random nature of the universe is sufficient to allow for true 'judgement' but it MIGHT.
The past doesn't exist. The future doesn't exist. You are standing on the pinnacle of now. Watch your step.
“He’s not deformed, he’s just drunk!”
The real problem is that the actions of people, in some circumstances, are considered beyond good and evil, and all the silly hypothetical situations in the world doesn't begin to capture this. In the heat of the moment, with only seconds to decide, people can't be relied on to make a choice that conforms to some explicit moral code. On account of that, when faced with passing judgement on the actions of people in emergency situations, we don't pass judgement; rather, we forgive them.
Robots, however, are programmed, and "split seconds" don't mean the same thing to robots that they do to us. Thus, there is no way around what they're going to do. They will be programmed to do one thing or another, and someone is going to have make the bad decision—since, in many cases, there are no good decisions to be made. And that poor bastard may have to program the machine anonymously, because what he will get is not forgiveness but, "What were you thinking!"
quiquid id est, timeo puellas et oscula dantes.
I just finished my ISIS killing robot and it's doing just fine. It hasn't killed any ISIS members, yet, but it does seem to be doing a fine job killing hipsters. I might not fix that for awhile...
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
(I've totally got an ISIS beard.. Please don't kill me, robot.)
If you were me, you'd be good lookin'. - six string samurai
Sorry, but that is not what the halting problem say.
The halting problems state that "For any interesting property(In this example: "Is this robot code safe to run") there exists programs with this property, but where you can not prove that the program has the property.
That is: There exists robot programs which are safe to run, but where we can newer prove that they are safe.
And the general solution is to only run programs where we can prove that they are safe. This mean that we do reject safe programs because we can't prove that they are safe*, but it does not in any way change the programs which we can express. That is: For any program which is safe, but where safety cant' be proved, there exists a program which behave in exactly the same way for all input, but which is safe.**
*If we can't prove that a program is safe, then it is either because no such prof exists, or it is because we are not good enough to prove it.
**No this does not contradict the halting problem, due to the assumption that the program is safe. If the program is not safe, then the transformation will convert the program to a safe program which obviously will not do the same
Which is more likely to shoot a civilian...
That's not entirely the right question. You need to account for which is more predictable for another human. If you are in the middle of a war zone with soldiers getting blown up by booby traps then you might expect a human soldier to be rather nervous and so you would approach them with extreme caution or get out of the way. However if you have a robot wandering down a street in a peaceful area and the right set of circumstances just happen to cause it to misidentify a random, innocent person as a target that person has no possible way to predict that they need to be extremely cautious.
The result is a complex combination of both a human's ability to know when they are in danger and the predictability of the gun owner. While a human may be more likely to make wrong decisions under pressure fellow humans are also going to be aware of this and take extra precautions. With a robot the decision will be entirely based on how good the robot makes a decision since the human has no way to know whether the robot is likely to be hostile or not.
My original post was simply pointing out that the human brain is NOT and can never be a Turing machine
This is true but it has exactly nothing to do with quantum mechanics or randomness. To see this, understand that we can't tell if QM is "truly" (metaphysically) random or just mocking it up really cleverly. Or rather, we can tell, but using inferences so indirect that they make no difference to the operation of the human brain, which is an extremely strongly coupled environment that is completely unlike the areas where "true" quantum randomness exhibits itself. No process in the brain depends in any way on metaphysical randomness: we could write a Monte Carlo simulation of the brain using entirely pseudo-random number generators and it would be accurate.
Brains and robots and computers have a number of properties that Turing Machines do not, however. In particular, I/O and realtime interrupts. Turing's model is strictly limited to what is on the tape. There is no way to hook up a sensor to a Turing machine and still have any of Turing's proofs still apply. The moment you allow even one bit to come in from the outside world, you no longer have a Turing machine.
So what Turing machines cannot do is not all that interesting to the design of robots. Turing's most important proof is that of universal computation: that any machine that can do at least what a Turing machine can do can compute anything that any Turing machine can compute. But this tells us nothing about what a machine that contains a Turing machine but is not itself a Turing machine can do. Robots (and humans) exhibit emergent properties from their interaction with the world, and that interaction is simply not part of Turing's model.
Blasphemy is a human right. Blasphemophobia kills.
http://worldnews.nbcnews.com/_...
This method for counting civilian casualties was probably not Obama's idea to begin with, but he has adopted it. So now, as Davester666 said, "gov't defines anybody they kill as "the bad guy". You can quibble about women and children, but the point largely remains. Davester666 was referencing a govenment definition, not what any individual (including Mr. Obama) actually thinks. There is a difference. Treating the government as though it were just a very big individual leads to serious miscalculations. The government does not behave like a person.
Join the IParty!