Slashdot Mirror


The UN Will Consider Banning Killer Robots (hrw.org)

Friday the United Nations agreed to discuss a ban on "killer robots" in 2017. The 123 signatories to a long-standing conventional weapons pact "agreed to formalize their efforts next year to deal with the challenges raised by weapons systems that would select and attack targets without meaningful human control," according to Human Rights Watch. "The governments meeting in Geneva took an important step toward stemming the development of killer robots, but there is no time to lose," said Steve Goose, arms director of Human Rights Watch, a co-founder of the Campaign to Stop Killer Robots. "Once these weapons exist, there will be no stopping them. The time to act on a pre-emptive ban is now."
schwit1 reminded us that IEEE Spectrum ran a guest post Thursday by AI professor Toby Walsh, who addressed the U.N. again this week. "If we don't get a ban in place, there will be an arms race. And the end point of this race will look much like the dystopian future painted by Hollywood movies like The Terminator."

28 of 210 comments (clear)

  1. Meh. by Frosty+Piss · · Score: 4, Insightful

    Like many of the proclamations from the UN, such a ban will have little influence over the development and use of "killer robots".

    --
    If you want news from today, you have to come back tomorrow.
    1. Re:Meh. by LWATCDR · · Score: 5, Informative

      It is also about 60+ years too late. G7es, Mk24 Mine, V1, JB-1 Loon.... All killer robots as is every modern torpedo and missile.

      --
      See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
    2. Re:Meh. by fuzzyfuzzyfungus · · Score: 3, Interesting

      Like many of the proclamations from the UN, such a ban will have little influence over the development and use of "killer robots".

      Some arms control treaties have been relatively successful; but I wouldn't be optimistic here:

      Somethings are just too convenient to ban; and those are usually a lost cause; but among stuff you can get support for; there seems to be a difference between hardware where you can say 'no legitimate use, period' and hardware where certain uses are forbidden; but there are enough legitimate use cases that the relevant hardware remains in stock, widely available to people relatively low on the food chain, and easily amenable to quiet 'off label' use.

      Military small arms ammunition, say, tends to be pretty reliably jacketed, even disreputable outfits don't tend to produce their own dum-dums and hollowpoints; though irregular forces and police-derived paramilitaries are obviously more likely to be using weapons not concerned with Geneva convention compliance in the first place.

      Stuff with both 'legitimate' and 'illegitimate' uses has been harder to keep a lid on. Phosphorus is a lovely illuminating agent; and produces great smoke; but it's considered poor taste to use it as an incendiary. Hard to make that stick when a large number of people, relatively low on the food chain, have access to it because of its legitimate applications, though.

      In the case of 'killer robots'; the obvious problem is that all the hardware; and much of the software for a 'killer robot' will be identical to that of a 'human directed' robot with some automation of routine navigation stuff; machine-vision-assisted targeting and IFF, and so on. So long as you pinkie-swear that a human will have to press the 'make it so' button; you can develop all the elements, navigation, targeting, etc. that an autonomous killbot would need; but make sure that the firmware running on anything pesky journalists get to see has a human pressing the red button to approve what the autonomous killbot comes up with.

      "Good faith" adherents to a 'no killbots' rule will likely find themselves easing their way toward autonomy with some (admittedly plausible) special cases: "We can't keep a human in the loop for our CIWS/RAM system; human reflexes aren't fast enough for contemporary missile intercept; but don't worry, the CIWS turrets are bolted onto the ship and aren't going to go wandering off." Not false; but an autonomous killbot. Then we'll need an 'emergency defensive protocol' for human-oversight robots that lose their link to HQ, whether to technical failure or hostile jamming; which will be OK; because it's strictly for the robots to engage in defensive actions in their existing location until communication is restored!

      People who don't give a damn, of course, will just stub out the 'ask human for confirmation' part and carry on with their day.

      Aside from this, some lucky logic-chopper is going to have the unenviable task of explaining why existing, more or less universally accepted, 'fire-and-forget' missiles and other similar hardware that gets its activation command from a human; but thereafter guides itself to target without external intervention, isn't a killbot; but the more drone-shaped hardware that gets its initial activation command from a human; but thereafter guides itself to target without external intervention, is a killbot.

      People claiming that diplomatic pressure and arms control conventions are totally useless are seriously exaggerating their case(land mines, chemical, and biological weapons certainly are way down; people have been jumpy about blinding laser weapons, etc.); but there is a lot less room for optimism when you can't draw a bright line around whatever you are trying to ban and declare it and everything similar Off Limits.

      With Killbots, I'm not optimistic: too much of the tech is shared with 100% legit 'human approved, machine assisted' systems; and the excuses to get a foot in the door(even if you care enough about perception of legality to not simply quietly switch off human approval) are too plausible.

  2. Yes, a law! by SeaFox · · Score: 3, Funny

    That will stop those killer robots from killing us.
    If there's one thing Skynet recognizes, it's the authority of the written word.

    1. Re:Yes, a law! by TheRaven64 · · Score: 3, Informative

      Unfortunately, a lot of that R&D is dual use. Self-driving cars will require the same navigation systems as autonomous robots. Remote-controlled drones will require the same chassis and control hardware as autonomous versions. Even the targeting systems can be developed under the ban: the difference between the system on the Apache that can target and prioritise a few hundred targets for the human gunner and one that then fires at the highest priority targets itself is a couple of lines of code. There isn't much that you need to build an autonomous robot soldier that you don't need for something else that such a treaty would permit. It's then just the final assembly step that's needed, and doing that in secret doesn't need too many people to know about it.

      --
      I am TheRaven on Soylent News
  3. Re:What about the ones we have now? by Anonymous+Cow+Ward · · Score: 2

    From the summary: "weapons systems that would select and attack targets without meaningful human control"

    The Reaper does not fall under this category.

    --
    Examine even your most deeply held beliefs. Nobody is always right.
  4. Automation of the military by unixisc · · Score: 2, Interesting

    Not just that - this is a macrocosm of 2nd amendment arguments in the US. Just like people who advocate banning assault rifles would only affect law abiding citizens, similarly, such a ban would affect law abiding nations, but do nothing about rogue nations who sooner or later would have that capability

    Besides, I disagree w/ this proposal for a simple reason. We should avoid intervening in other countries, such as Syria. But if we have to go in, I'd rather send in killer robots after ISIS rather than American (or any other) humans who'll get killed or maimed for life. We should have killer robots substitute soldiers: it would also solve the issue of a depleted military as well as the idea for a draft. If we transform our soldiers into simulation controllers where each soldier controls up to say, 10-16 robots, we then have an army that can theoretically take on every one of our adversaries should we need to fight them - China, North Korea, Iran, Syria, Pakistan, Saudi Arabia, the list goes on...

    The other thing about killer robots: it's automation in the military. In the civilian world, we've seen a whole lot of harmless but grunt jobs get automated - from store clerks to robocalls. In the military world, this is a function ideal for automation: it not only saves the lives and limbs of our soldiers, but also extends their efficiency. They get to learn more deeply about robotics and computing principles, and come out of it body intact, no PTSD and merge seamlessly into the civilian work force. In fact, the military is where we should have as much automation as possible, not less.

    1. Re:Automation of the military by K.+S.+Kyosuke · · Score: 3, Insightful

      Generals and terrorist leaders have already been using robots for quite some time. Biological ones, that is. They haven't risked their own lives, either.

      --
      Ezekiel 23:20
    2. Re:Automation of the military by Applehu+Akbar · · Score: 3, Insightful

      "If you're not man enough to look the enemy in the eyes while killing him, you're... American. The term is becoming synonymous with coward, and I'm ashamed."

      The objective of war is not to look "man enough" but to kill sufficiently large numbers of enemy that he will no longer be inclined to attack you.

    3. Re:Automation of the military by arth1 · · Score: 2

      Generals and terrorist leaders have already been using robots for quite some time. Biological ones, that is. They haven't risked their own lives, either.

      Yes, it has been a long time since the days of chivalry, and leaders being called leaders because they were leading.

    4. Re:Automation of the military by amiga3D · · Score: 3, Insightful

      The idea is to kill the enemy without getting killed. There are no points for style. We try to limit death of non combatants but if they're in the kill zone that's just the way the cookie crumbles. You fight wars to win because the alternative is something no one want to experience. American soldiers put themselves in harms way if needed but they are trained to survive while accomplishing the mission.

    5. Re:Automation of the military by AJWM · · Score: 2

      Oh, not looking the enemy in the eyes started long, long before Dresden, or even The Blitz. Hell, it was old when the English at Crecy and Agincourt rained death on the French with longbow and cannon. It was probably old when Xerxes attempted to do the same (well, sans cannon) to the Greeks at Thermopylae.

      War isn't a contact sport, it's a continuation of politics by other means.

      --
      -- Alastair
    6. Re:Automation of the military by SeattleLawGuy · · Score: 3

      The idea is to kill the enemy without getting killed. There are no points for style. We try to limit death of non combatants but if they're in the kill zone that's just the way the cookie crumbles. You fight wars to win because the alternative is something no one want to experience. American soldiers put themselves in harms way if needed but they are trained to survive while accomplishing the mission.

      Wars serve a political objective, usually badly. There are points for style, primarily because style has propaganda value, and propaganda changes political support for wars, leaders, and causes--whether that means someone voting to bring troops home or someone telling a private where the enemy ambush is waiting for his unit. Even when you lose a battle or part of one, there can be points for style--Dunkirk, Pearl Harbor, Bastogne, all involved at least temporary defeats that were in some sense spun into political victories that served the war effort. On the offensive side, style matters too--the drone attack with collateral damage that injures or kills non-terrorists, for example, can create more terrorists in the next generation.

      --
      Real lawyers write in C++
    7. Re:Automation of the military by CrimsonAvenger · · Score: 4, Insightful

      "Your purpose is not to die for your country. Your purpose is to make the other poor dumb bastard die for his country."

      If you think that making war is all about being "fair", you're doing it wrong....

      --

      "I do not agree with what you say, but I will defend to the death your right to say it"
    8. Re: Automation of the military by TheRaven64 · · Score: 3, Insightful

      US gear only falls into the hands of our enemies in 2 scenarios: we sell it to them, or it gets seized from our allies

      Actually, there's another third common scenario: You sell it to your 'allies' (you know, nice friendly countries like Saudi Arabia) and they sell it on, leaving you with no control over its final destination.

      --
      I am TheRaven on Soylent News
  5. In related news... by Black+Parrot · · Score: 2

    Killer Robots Will Consider Banning UN

    --
    Sheesh, evil *and* a jerk. -- Jade
  6. What about cruise missiles? by Streetlight · · Score: 2

    It's my understanding that the way cruise missiles work is the operator inputs the GPS coordinates of the target and off it goes to the target. The only human input is the target. What's difference between this and and autonomous robot that has various built in systems that do human soldier recognition as the target? Once released each weapon has no human intervention or control.

    --
    In a time of universal deceit, telling the truth is a revolutionary act. George Orwell
    1. Re:What about cruise missiles? by Anonymous Coward · · Score: 2, Insightful

      The same can be said about any bullet, when fired they can not be stopped. This proposal says picking the tearget must involve humans.

  7. "Select and attack targets without human control" by raymorris · · Score: 2, Insightful

    As the summary says, the proposed ban is on devices which "select and attack targets without meaningful human control". So basically none of what you wrote applies.

    In fact, it's the exact opposite of "macrocosm of 2nd amendment arguments in the US" - supporters of the second amendment point out that "guns don't kill people, people kill people"; their argument is that the device is controlled by a person, who can do good or bad with a steel pipe too.

  8. Re:Only if they aren't aimed by K.+S.+Kyosuke · · Score: 2

    Anti-ship missiles often pick their target on their own, for example. It wasn't always possible to "select" one.

    --
    Ezekiel 23:20
  9. Re:We've had this before. by arth1 · · Score: 5, Insightful

    Whoever develops them anyway wins world war 3.

    No one wins world war 3.

  10. United Nations by PPH · · Score: 3, Funny

    Why not just ban war?

    --
    Have gnu, will travel.
  11. Re:Only if they aren't aimed by lxs · · Score: 5, Informative

    As for banning landmines.
    There is a landmine treaty which isn't signed by a handful of nations that don't take human rights too seriously. Not surprisingly, the US is on that list, rubbing shoulders with the likes of North Korea, Uzbekistan and Syria.

    After all, maiming civilians is what it's all about for these brave warrior nations.

  12. Re:Only if they aren't aimed by ShanghaiBill · · Score: 2

    After all, maiming civilians is what it's all about for these brave warrior nations.

    The US uses landmines only along the Korean DMZ, where there are no civilians.

  13. Re:"Select and attack targets without human contro by Sarten-X · · Score: 2

    Nations aren't American people, though.

    Americans tend to care about "freedom". Not any particular freedom, mind you, but they cling to the fantasy story that they are somehow "free" in an abstract sense, and any limit on that freedom is a grave assault on their very essence. However, the more recent evolution of this philosophy has extended the concern to others' freedoms as well. The privacy advocates don't have anything to hide themselves, but they're sure that someone out there has horrible secrets they're trying to keep from the Big Bad Government. The civil rights advocates aren't being persecuted, but they're supporting those who are being persecuted. The gun lovers don't own bazookas or urban-assault weaponry, but they want to be absolutely sure someone else can get them.

    International politics, on the other hand, looks a lot more like the stereotypical spaghetti westerns. Every nation is concerned primarily with their own interests, and everybody's just one sufficient excuse away from attacking someone else. Their cattle came too close to your territory? That must mean they're your cattle now. They cut down a tree you liked to look at? That must mean they owe you reparations. Their drunk belligerent son insulted your father? This calls for a duel to the death.

    There are also the outlaws. There are certain countries who have caused more trouble than their existence is worth, and it'd probably be best if they were relieved of their sovereignty, but, they still have a few powerful friends. You can't just kill that guy who annexed your land, because you'd be dead yourself in short order.

    Now, the UN is suggesting new rules. Continuing the metaphor, this is effectively the equivalent of the townsfolk coming together and agreeing to not breed coyotes. No, there's no real way to stop someone from breeding coyotes if they want to, but everyone else has said they don't want more coyotes around.

    What this means is that there is a new excuse in town. If someone accuses you of coyote-breeding, they might use that as justification to steal your horse. If they can prove you're breeding coyotes, they might be able to get a posse to kick you off of your land and take it for their own. Your friends won't have much ability to protect you, because they don't want to be associated with a low-down dirty rotten coyote-breeder. With your allies gone and a bounty on your head, everyone with an opportunity will try to prove their commitment to the law by coming after you.

    Of course, it's very difficult to claim that breeding coyotes is bad when one is already doing it themselves. This is exactly what happened with nuclear weapons. The United States had them, so we couldn't effectively ask everyone else to ban them. Right now, though, there's very little interest in fully-automatic lethal robots. Even drone strikes are commanded by a human, and there is significant political pressure to keep them that way. The closest we have to automated killing machines is the common land mine, whose use is finally banned or regulated by most countries. There are also IEDs, whose use is being publicized as an indiscriminate assault on civilians... and in response, the good guys of the world continue to hunt those dastardly villains to protect the innocent!

    In the public perception, there is a huge difference between the freedom of people and the freedom of nations. Yes, a country bound by the UN rules could still create AI killers, but there are effective human-led counterattacks to them. There is no benefit to pursuing the prohibited weaponry.

    --
    You do not have a moral or legal right to do absolutely anything you want.
  14. Mod Parent Up by SeattleLawGuy · · Score: 3, Insightful

    "Death, destruction, disease, horror. That's what war is all about, Anan. That's what makes it a thing to be avoided."
    Star Trek The Original Series: "A Taste of Armageddon"

    Exactly. One of the consistent and reasonable critiques of modern American warfare is that because there is no draft, the influential wealthy and policy-making classes have no personal incentive to avoid war. Many people know few or even no service members. The further you push human beings away from the horrors of war, the more those people will be willing to engage in war.

    I've met people who've been personally tortured by foreign heads of state. I've seen people fighting politically to pull their countries together in the face of what seems like neverending war and oppression by warlords. And I've read the stories of people who have seen their countries fall apart in the face of characteristic propaganda and strong men taking power. The less real all of this is and the less human it is, the more people will be willing to stay unengaged in matters of life and death.

    --
    Real lawyers write in C++
  15. Sentience is not the biggest danger by djinn6 · · Score: 2

    We should worry about the concentration of power rather than robots becoming sentient. Sentience is a long ways off, whereas even with the technology today, a single human can command a large number of autonomous drones and missiles. The more autonomy the weapon has, the less the human is required in the loop, which means eventually, a single general or hacker can command the equivalent of the entire armed forces. At that point, who can stop them from starting unnecessary wars or becoming dictators?

  16. Re:Only if they aren't aimed by Ambassador+Kosh · · Score: 3, Informative

    The USA also uses limited lifetime landmines so they don't stick around after a conflict to keep killing people. I may not like landmines but I understand why they are used and having ones that self destruct is much better than ones that stay around.

    --
    Computer modeling for biotech drug manufacturing is HARD! :)