Slashdot Mirror


Robot Warriors Will Get a Guide To Ethics

thinker sends in an MSNBC report on the development of ethical guidelines for battlefield robots. The article notes that such robots won't go autonomous for a while yet, and that the guidelines are being drawn up for relatively uncomplicated situations — such as a war zone from which all non-combatents have already fled, so that anybody who shoots at you is a legitimate target. "Smart missiles, rolling robots, and flying drones currently controlled by humans, are being used on the battlefield more every day. But what happens when humans are taken out of the loop, and robots are left to make decisions, like who to kill or what to bomb, on their own? Ronald Arkin, a professor of computer science at Georgia Tech, is in the first stages of developing an 'ethical governor,' a package of software and hardware that tells robots when and what to fire. His book on the subject, Governing Lethal Behavior in Autonomous Robots, comes out this month."

24 of 317 comments (clear)

  1. Been there, done that by Locke2005 · · Score: 4, Informative
    --
    I've abandoned my search for truth; now I'm just looking for some useful delusions.
    1. Re:Been there, done that by fuzzyfuzzyfungus · · Score: 4, Insightful

      Been there, wrote fiction about that(much of which was about how, even in fiction land, it wouldn't work so well).

    2. Re:Been there, done that by NeutronCowboy · · Score: 4, Insightful

      Not to mention... some of the assumptions aren't great. As the article itself points out, it's been a long time since there was a civilian-free battlefield.

      As for the direct example of the robot locating a sniper and being offered the choice of a grenade launcher and rifle - how does the robot know that the buildings surrounding it aren't military targets? How do they get classified? How does a hut differ from a mosque, and how does a hut differ from some elaborate sniper cover?

      I don't think this is going to work out as planned.

      --
      Those who can, do. Those who can't, sue.
    3. Re:Been there, done that by Locke2005 · · Score: 4, Interesting

      Since you can never be 100% certain of a target, the robots would have to use fuzzy logic. That is something that humans are better than robots at; I'm not really comfortable with hardware designed to be lethal making decisions like this. Truly autonomous killer robots are probably not a good idea -- haven't 60 years of B movies taught us anything?

      --
      I've abandoned my search for truth; now I'm just looking for some useful delusions.
    4. Re:Been there, done that by Red+Flayer · · Score: 4, Funny

      Since Homo sapiens only natural predator is itself,

      Well, itself and wolves. And tigers. And lions.

      And don't forget bears. Definitely bears.

      I think we should build giant ethical bear robots. That would scare the SHIT out of our enemies.

      --
      "Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
    5. Re:Been there, done that by Architect_sasyr · · Score: 4, Insightful

      But (most) humans have this innate condition where taking another life weighs on them somewhat - even most veterans and soldiers I know get twitchy about having to shoot at another person. A robot removes this and replaces it with cold logic.

      Put another way, replace the robots with the WOPR, and the humans with, well, the humans in the bunkers.

      --
      Me failed English...
      FreeBSD over Linux. If my comments seem odd, this may explain...
    6. Re:Been there, done that by Trepidity · · Score: 4, Interesting

      The cold logic can be better though, if you know what you actually want to optimize. Humans often make decisions that don't do what they claim they want, e.g. minimizing civilian casualties.

    7. Re:Been there, done that by Locke2005 · · Score: 5, Insightful

      Yes, robots are much better at calculating probabilities; given a series of "facts" with a confidence level assigned to each one, a robot would make a better decision. What I should have said is that "Humans are better than robots at making decisions based on incomplete data." Humans can develop "intuition" and many have a great deal of experience in interpreting the context of the data. While it may be possible some day for robots to have a deeper understanding of context than humans, that day is still a long way off.

      --
      I've abandoned my search for truth; now I'm just looking for some useful delusions.
    8. Re:Been there, done that by Chris+Burke · · Score: 4, Funny

      He said ethical!

      --

      The enemies of Democracy are
    9. Re:Been there, done that by Chris+Burke · · Score: 4, Interesting

      The cold logic can be better though, if you know what you actually want to optimize. Humans often make decisions that don't do what they claim they want, e.g. minimizing civilian casualties.

      Yeah it's that 'if' that's the killer. The problem is that you have to be able to express what you want to optimize using cold logic before a machine can start making that decision, and we aren't able to do that. Terms like "civilian" are nebulous, and attempts to rigidly codify them fail to capture the intent and connotation behind those words that we understand, but can't express. We can reason about that, while machines can't. Fuzzy logic doesn't help, that's just a way of decision making on non-binary factors. With a lot of types of fuzzy logic (neural nets, genetic algorithms) it can be even more important to precisely define what you want, since they can produce solutions that "work" correctly and optimize your problem as specified, but do so in a way very unlike you expected.

      People of course have the disadvantages of being error prone, and well sometimes being bastards who just don't give a shit what you want them to optimize, so there's appeal to the machine. Yet nothing fails as spectacularly and efficiently as a machine doing exactly what it was programmed to do when it's exactly what you didn't want. To use a machine in situations where even humans equipped with honest intentions, solid faculties, and experience have enormous trouble determining who is "enemy" vs "innocent"? As in most situations our military has been in since the 50s and is going to be involved in for the foreseeable future? That sounds crazy to me. I'll take human judgment and its failure modes any day.

      Kinda off topic, but speaking of honest intentions, I gotta say the humans making the judgments in question, i.e. our soldiers, have a damn hard problem to solve and it shows how human potential is pretty damn amazing. We're biologically the same animal we were a hundred thousand years ago and more. But in the past, even recent past, the most difficult ethical decision a warrior was asked to make was whether to decide if someone was a threat and should be killed or wasn't and should be enslaved, and it wasn't of any consequence so nobody cared to go over those decisions with a fine-toothed comb. So given the difficulty of what we're asking them to do today, and considering what's going on, the results are pretty amazing. Seriously, think about it. Anyway, yeah, off topic.

      --

      The enemies of Democracy are
    10. Re:Been there, done that by Anonymous Coward · · Score: 4, Informative

      No they weren't. The laws were flawed and the only modifications that ever occurred were made in order to fix these flaws and prevent paradoxical situations from occurring. There was never a situation where things went wrong due to someone trying to modify the laws to my knowledge.

      The books and short stories all revolved around dilemmas that, when robots attempted to uphold the laws, caused conflicts or paradoxes often causing the robots' positronic brains to malfunction or shut down. Dilemmas such as choosing the death of one human over the death of another, or choosing between two options, both of which would cause harm to a robot/human.

      The only situations where the laws were modified were in "Little Lost Robots", where the inaction clauses were added, and "Robots and Empire", where Giskard invents the Zeroth Law. Both of these modifications were patches to flaws in the original three laws.

  2. Yeaahhhh... by XPeter · · Score: 4, Funny

    Last time robots were confronted with "ethics" http://en.wikipedia.org/wiki/Three_Laws_of_Robotics, they turned on the world and Will Smith had to save us all.

    --
    "The difference between genius and stupidity is that genius has it's limits" - Albert Einstein
  3. "How goes the battle, Sgt?" by v1 · · Score: 4, Funny

    Sgt: We lost sir! badly!

    Gen: What happened?

    Sgt: We're still gathering up the details, but it looks like they hacked our network and uploaded Asimov Strain B.

    --
    I work for the Department of Redundancy Department.
  4. Ethical War Robots? by Fantom42 · · Score: 5, Insightful

    Weird. So this fails the Asimov criteria.

    More importantly, would also necessarily fail the Golden Rule and Kant's Categorical Imperative.

    If this is ethics, its a pretty limited version of it, and to be honest sounds more like rules of engagement than actual ethics.

  5. Great book title... by GPLDAN · · Score: 4, Funny

    Governing Lethal Behavior in Autonomous Robots


    That is the title of the book you tell your 7th grade teacher you are GOING to write when you grow up.

    Sounds like the FAQ for Robot Battle.
    http://www.robotbattle.com/

  6. Jesus Christ by copponex · · Score: 4, Insightful

    If you drop a fucking robot into a village where a vast majority of the people don't know how to read, what do you think they're going to do? They'll shoot at it, get the backs of their heads blown off, and then everyone will say, "Well, the dumbass shouldn't have shot at the robot!"

    If this war on terror is so important, sign up. If you can't, get your brother or sister or even better, sign your kids up. If they're not of age yet, they'd better be in the JROTC. Then you can talk to me about how using drones and missiles isn't the dominion of motherfucking cowards. It's for freedom lovers defending freedom!

    And if you think it isn't, imagine what the headlines would be if China landed a few thousand autonomous tanks and droids in Los Angeles. Oh, but that's right. This is about principles for others to follow, and for us to ignore.

  7. Humans by DoofusOfDeath · · Score: 5, Insightful

    But what happens when humans are taken out of the loop, and robots are left to make decisions, like who to kill or what to bomb, on their own?

    Why is this a when question, rather than an if question?

  8. Re:Need a good spell checker by MarkvW · · Score: 4, Funny

    Yeah! Or, or "How to Serve Man."

  9. Wish I had mod points, I'd mod you up. by Anonymous Coward · · Score: 5, Interesting

    Great post, man.

    But I have a buddy in the autonomous killer robot biz, and he says it's worse than that.

    See, you drop a killer robot in the village, and it immediately kills a shitload of people. The ones that live, figure out why. Then, as soon as they know that the robot destroys everything that looks like an AK47, the local up-and-coming gang leader makes an AK47 stencil and paints AK silhouettes on the old warlord's cows, house, laundry, etc. you get the picture. Then the young punk gives all the old leader's women to his buddies to rape and takes the young virgins for himself. Yay democracy! Or, at least, that's what they say when GI Joe comes to town, we are the heroes who took out the old anti-democratic leaders, yay us and you villagers better keep your cake-holes tight shut about the rape and opium parties.

    It doesn't matter what you use for a trigger - robots are inherently less complex in their behavior than humans, so the local baddies end up with the robots working for them. You just identify the kill behavior and use it, the robot builder is just providing free firepower to the local mafia in effect.

    Which is why the US military in the field abso-fucking-lutely refuses to let the robots go full autonomous. They are NOT allowed to shoot unless a callow 18-year old miles at a console away says it's OK.

    You might think I'm kidding, but I'm not. Have to be anonymous for this one!

  10. Re:That would be really cool... by hairyfeet · · Score: 4, Funny

    Shiiiiit, you think those damned bots would make it out of South Central intact? The gang bangers would have robot heads mounted on their rides like trophies. I'm sure that any of them that managed to roll out the other side would have the weapons stripped off of them faster than a Toyota Camry ends up on blocks and it would be so covered in tags that the poor thing couldn't even see where it was going.

    --
    ACs don't waste your time replying, your posts are never seen by me.
  11. Re:We're doomed even if it is flawless by Bigjeff5 · · Score: 4, Interesting

    Right, because we have the capability of doing just that with nukes now, nevermind robots, and it has been such a problem for us over the last 50 years...

    Only an idiot would think physical separation from the battlefield immediately reduces the gravity of killing a human being. You still know it's a human being you are killing, the separation doesn't change anything. You could make the case that it reduces the trauma of being mid-fight, but that only puts more emphasis on the fact that you are killing someone, you don't have the fear of your own death to force your hand.

    By your logic, shooting someone at point-blank range would be significantly more difficult than shooting them from 200 yards away, which would be more difficult than shooting them with battlfield artilary from 1 mile away, which would be more difficult than launching a missile from tens of miles away, which would be more difficult than pressing the button to launch an ICBM.

    The logic doesn't follow, because as you move farther away and impact more people, the decision becomes more and more difficult. The decision at point blank is simple: act or die. Traumatic? Yeah, some people are screwed up for life because of it. Do you have time to weigh to think about the fact that you are about to end another human being's life? No, you don't. Making the decision is easy, living with the consequences is difficult. It doesn't change much when you make that decision from half a world away through a monitor. If anything, without the stronger pressures of battle to force the decision it could be harder on a person's psyche to make the decision to kill, and more likely to question their own actions.

    For some reason, you are assuming that physical separation suddenly turns people into sociopaths. It's the same reasoning that makes the asinine argument that video games desensitize kids and turn them all into violent killers. It's just not the case. You're basically saying soldiers in the drones can't tell that those are real people they are killing. That's just stupid.

    --
    Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
  12. Well duh by Sycraft-fu · · Score: 4, Insightful

    These are military robots. No military robots would fall under Asimov's list.

    What I think some fail to remember is that Asimov was just a science fiction author. He wrote stories. Very compelling ones, his place in modern literature is gigantic, but none the less just fictional stories. Thus his "three laws" have nothing to do with reality. They aren't natural laws, or legal standards, they are just part of a story. Thus they have no standing in the world.

    They may well be how Asimov would like to see robots work, they may well be how you'd like to see robots work, however they have nothing to do with how the military wants it to work. They are not a canon of any kind.

    When a robot is developed for military purposes, it should be no surprise the ethics are considered in that context. The whole point of it will be to be able to use deadly force if necessary. The programming is then when is that ok and not ok.

    So please, let's have all us geeks lay off the Asimov "three laws" when it comes to robots. Every time something like this comes up people start talking about that like it matters to anyone. No, it really doesn't.

  13. Re:We're doomed even if it is flawless by tyleroar · · Score: 5, Insightful

    Your post has absolutely zero factual basis to it. Physical separation is a major psychological factor when deciding to 'pull the trigger.' Try reading "On Killing" by Dave Grossman, an excellent book that points out the reasons why distance makes it easier to kill.

    --
    Portland, North Dakota Puppies
  14. Re:We're doomed even if it is flawless by scubamage · · Score: 5, Insightful

    Obviously you've never spoken to a tank commander, or any manufacturer of the UI's inside of armored vehicles. They are designed to be 'like video games' for a reason. Specifically to dehumanize the opponent, and mitigate the likelihood that you will associate your actual actions with killing. That's basic psychology. It's also why we refer to the enemy in Iraq/Afghanistan as Haji, why we called the Germans Jerries, and why we called the VC Charlie. You don't hate Ho Ming Na, father of 4 children who were brutally slain by US soldiers and is trying to simply save his farmland. You hate Charlie, so killing Ho Ming Na is acceptable. Anything to dehumanize them is crucial for removing mental blocks to soldiers.