Slashdot Mirror

← Back to Stories (view on slashdot.org)

Halting Problem Proves That Lethal Robots Cannot Correctly Decide To Kill Humans

Posted by Soulskill on from the guess-we're-back-to-coinflips dept.

KentuckyFC writes: The halting problem is to determine whether an arbitrary computer program, once started, will ever finish running or whether it will continue forever. In 1936, Alan Turing famously showed that there is no general algorithm that can solve this problem. Now a group of computer scientists and ethicists have used the halting problem to tackle the question of how a weaponized robot could decide to kill a human. Their trick is to reformulate the problem in algorithmic terms by considering an evil computer programmer who writes a piece of software on which human lives depend.

The question is whether the software is entirely benign or whether it can ever operate in a way that ends up killing people. In general, a robot could never decide the answer to this question. As a result, autonomous robots should never be designed to kill or harm humans, say the authors, even though various lethal autonomous robots are already available. One curious corollary is that if the human brain is a Turing machine, then humans can never decide this issue either, a point that the authors deliberately steer well clear of.

9 of 335 comments (clear)

  1. I think by fyngyrz · · Score: 4, Insightful

    I'm just going to reformulate the problem by considering idiots who use unrealistic, not-supported-by-evidence premises to make general statements as one that calls for sending killer robots after said idiots.

    --
    I've fallen off your lawn, and I can't get up.
    1. Re:I think by ShanghaiBill · · Score: 5, Insightful

      The premise of TFA is that killer robots need to be perfect. They don't. They just need to be better than humans.

      Which is more likely to shoot a civilian:
      1. A carefully programmed and thoroughly tested robot.
      2. A scared, tired, and jumpy 18 year old soldier, who hasn't slept in two days, and saw his best friend get his legs blown off by a booby trap an hour ago.

    2. Re:I think by Anonymous Coward · · Score: 5, Insightful

      1, when ordered to shoot civilians.

    3. Re:I think by bmo · · Score: 5, Insightful

      Product liability never results in anyone being actually responsible for the death going to jail or huge penalties.

      A multinational might> pay out a couple of million in product liability, but then it will just be chalked up to the cost of doing business.

      If the multinational is a defense contractor (BAE, Raytheon, Lockheed, General Dynamics, etc), it will all be swept under the rug and more money will be thrown at the contractor to "fix" it.

      That's the reality.

      --
      BMO

  2. By the same logic by Anonymous Coward · · Score: 5, Insightful

    By the same logic, computers should not be allowed in any life-critical situation. That includes hospital equipment, airplanes, traffic control, etc. etc.

    Fortunately, we don't judge the reliability of computers based on the ability to mathematically prove that nobody has put evil code in on purpose.

    1. Re:By the same logic by Dutch+Gun · · Score: 4, Insightful

      Agreed. The authors set up a nearly impossibly complex ethical dilemma that would freeze even a human brain into probable inaction, let alone a computer one, and then claims "See? Because a computer can't guarantee the correct outcome, we can therefore never let a computer make that decision." It seems to be almost the very definition of a straw man to me.

      The entire exercise seems to be a deliberate attempt to reach this conclusion, which they helpfully spell out in case anyone missed the not-so-subtle lead: "Robots should not be designed solely or primarily to kill or harm humans."

      I'm in no hurry to turn loose an army of armed robots either, but saying that you can "prove" that an algorithm can't make a fuzzy decision 100% of the time? Well, yeah, no shit. A human sure as hell can't either. But what if the computer can do it far more accurately in 99% of the cases, because it doesn't have all those pesky "I'm fearing for my life and hopped up on adrenaline so I'm going to shoot now and think later" reflexes of a human?

      --
      Irony: Agile development has too much intertia to be abandoned now.
  3. The human brain is not a Turing machine by Anonymous Coward · · Score: 4, Insightful

    Exhibit A, the human skull: Not enough room for an infinite tape.

  4. Impossible to build purely evil robots? by Galaga88 · · Score: 4, Insightful

    Presuming that this proof reached via impressively tortured logic does have merit: Does it mean that it is also impossible to build a purely evil robot that would always kill maliciously?

  5. Bad Headline as Usual by Jaime2 · · Score: 4, Insightful

    What the paper said is that computers can't provably always make the right choice. Neither can we. I'll bet computers are capable of doing a lot better than humans, especially given the rate of the increase in the number of things a computer can do compared to the rate that humans are (aren't) gaining new abilities.