Slashdot Mirror


US Seeks To Allay Fears Over Killer Robots (bbc.com)

Humans will always make the final decision on whether armed robots can shoot, the US Department of Defense said today. From a report: The statement comes as plans emerge for gun platforms that can choose their own targets on the battlefield. The plans seek to upgrade existing aiming systems, using developments in machine intelligence. The US said rules governing armed robots still stood and humans would retain the power to veto their actions. The defense department's plans seek to upgrade the current Advanced Targeting and Lethality Automated System (Atlas) used on ground combat vehicles to help human gunners aim. The military is seeking commercial partners to help develop aiming systems to "acquire, identify, and engage targets at least three times faster than the current manual process."

67 comments

  1. How many humans? by Anonymous Coward · · Score: 1

    I fear a president or single 'general' issuing orders and obliterating a school/city/country.. I feel it should be passed down a chain of command OF HUMANS, ultimately with human eyes on target for finial determination.

  2. Huh? by lrichardson · · Score: 5, Interesting

    "The US said rules governing armed robots still stood and humans would retain the power to veto their actions."

    Uh, say WHAT? That line says something completely different from 'requiring human finger on every trigger'.

    "Yeah, I coulda - probably shoulda - vetoed that Predator bombing on the wedding, but I was in the can at the time."

    1. Re:Huh? by Anonymous Coward · · Score: 2, Interesting

      "Yeah, I coulda - probably shoulda - vetoed that Predator bombing on the wedding, but I was in the can at the time."

      Like we need an amoral AI to do that.

      Obama signed orders personally on several occasions to bomb weddings or funerals filled with known-innocent people because one of them kinda sorta looked like Osama Bin Laden from 10,000 feet.

      I did a paper and presentation about it for my second degree. He knew exactly what he was doing and did it anyway.

    2. Re: Huh? by c6gunner · · Score: 2

      You're inserting your own assumptions into that phrase. The concept of "veto" does not imply that the default is for the action to proceed. The word "veto" essentially just means "reject". In this case, the targrtting system would select an object, target it, and present a prompt to the human requesting permission to engage. The human can either authorize or reject.

      It seems you're thinking of a word more along the lines of "interrupt" but, again, that's not what veto means.

    3. Re:Huh? by ceoyoyo · · Score: 3

      It all happened so fast.

      I definitely vetoed the followup strike on the smouldering ruins though!

    4. Re: Huh? by Anonymous Coward · · Score: 0

      War crimes apologists look and smell like c6gunner.

  3. Indiscriminate killing by Anonymous Coward · · Score: 1

    Right, because if there's one thing the US military isn't known for, it's indiscriminate killing at a distance, so let's trust them when they promise that these are mindful, ethical kill-bots they're talking about.

    1. Re: Indiscriminate killing by h33t+l4x0r · · Score: 2

      And here is where we get treated to such arguments as, among others, how Hiroshima and Nagasaki saved lives...

    2. Re: Indiscriminate killing by Anonymous Coward · · Score: 0

      Do you disagree? [menacing ellipsis]

    3. Re: Indiscriminate killing by drinkypoo · · Score: 1

      Would you like to know more?

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    4. Re: Indiscriminate killing by Anonymous Coward · · Score: 0

      You're the ignorant cunt, fucktard. The US has been on the forefront of illegal aggressive wars against nations that cannot respond in kind since forever, hiding behind two oceans.

      Mostly to support bloodthirsty dictators whose rule the US business profits from, and to test its new weaponry.

  4. Not what you think it says by WillAffleckUW · · Score: 2

    It in fact is saying that systems will be developed to override, but if they can not be engaged (eg interference weather hacking) the mission will continue.

    Once the bot swarm is live, it's on live fire in the target zone, will complete the mission, and return to the marshall point (which is frequently an airplane where they rendevous and then disable).

    The Command and Control units have kill switches, but they're basically Abort The Mission signals. A human decides the mission is go, arms the flight, and it's kind of like any other missile or remote control device, it completes the firing pattern.

    --
    -- Tigger warning: This post may contain tiggers! --
    1. Re:Not what you think it says by NicknameUnavailable · · Score: 1

      The issue with killer robots isn't that they kill people, it's that they are automated in any way shape or form. They will end up being used against citizens eventually because the only thing keeping the government from having already curb-stomped the population into submission is that the population is armed to the teeth and even the military value freedom. As it stands the military wouldn't help subvert freedom in the US on a large scale, the moment automated killer robots exist in any capacity the rules change because it only takes 1 corrupt bastard at the controls to attack everyone.

    2. Re:Not what you think it says by WillAffleckUW · · Score: 2

      To be frank, once it's sent, you can't always stop it, whether we're talking smart dumb bombs (active once they leave the plane), cruise missiles (active once they're in a specified region), or drone swarms. Combat is always messy.

      I know in TV series and movies there's always a kill switch that works right up to the end, but that's not how things actually work.

      --
      -- Tigger warning: This post may contain tiggers! --
    3. Re:Not what you think it says by NicknameUnavailable · · Score: 1

      There's a difference between a dumbfire weapon post-deployment and a Human-seeking weapon post-deployment. The latter variety is a threat to even have that software developed because the software is the threat, not the deployment process. The deployment process is always going to be open to manipulation to the extent that 1 guy could potentially control an entire nation's military forces without checks and balances at every level. The thing which makes Human warriors better than AI warriors (from the standpoint of basically any Human) is that Humans will question orders at every level of the chain of command, even if only silently and in agreement (which is not always the case.) Think of all the rapes, tortures, and horrendous ways of death in war over the course of Human history - AI weapons would be worse because they wouldn't get shell shock, they wouldn't be questioning if they've gone too far, they would never stop to think of a better solution or even entertain the notion of mutiny. AI has no place in war.

  5. Uh by backslashdot · · Score: 4, Funny

    I, for one, proclaim that their decision to call it a lethality system has allayed my fears.

    1. Re:Uh by 93+Escort+Wagon · · Score: 1

      Killer robots? More like Kuddle robots!

      --
      #DeleteChrome
    2. Re:Uh by Anonymous Coward · · Score: 0

      Kuddle? You mean like in "Good Fellas" where he shoves the screwdriver in the guy's skull? That Kuddle?

    3. Re: Uh by Ukab+the+Great · · Score: 1

      And they have hard-coded into them a kill limit, so we just need to send wave after wave of our own people against them until they reach their quota and shut down.

    4. Re:Uh by h33t+l4x0r · · Score: 1

      I know very little about killer robots, other than that they eat old people's medicine for fuel.
      But that's reason enough to be wary in my book.

  6. oops by Anonymous Coward · · Score: 0

    That little overpopulation problem is getting closer to being solved any day now.

    1. Re:oops by Anonymous Coward · · Score: 0

      The good thing is that the AI will strike down the people who are most dangerous to it first. Good bye, Kendall, c6gunner and the rest of the blood-thirsty, racist Tom Clancy fans.

  7. How long will this directive last? by Anonymous Coward · · Score: 1

    In a total war scenario where both sides have killer robots and AI on par with today's self-driving cars, how long would this directive last? Obviously at least one side will stand a risk of losing... can anyone imagine, in this scenario, that a desperate party would NOT give the order to drop these "ethics" and send out fully-autonomous killer droids?

    It's so easy to take the apparent moral high ground when you have the military high ground.

    1. Re: How long will this directive last? by Anonymous Coward · · Score: 1

      No, at that point they use nuclear weapons. Killer robots pale by comparison. And yeah, the US military will let 'em loose & blame it on the "fog of war" as usual.

  8. Fortunately by 93+Escort+Wagon · · Score: 1

    There's no chance these can be hacked by the enemy forces, so they will never be reprogrammed to attack and kill US military field personnel.

    --
    #DeleteChrome
  9. Been playing Horizon: Zero Dawn recently by Anubis+IV · · Score: 4, Interesting

    For anyone not familiar, the entire premise of the game is that you're in a post-apocalyptic world about 1000 years after our war robots went out of control, with exactly the sorts of results you'd expect. I found it interesting when, in a moment of self-awareness, the main character discovers a recording circa 2065 of an engineer who worked on the war robots lamenting the fact that they didn't pay attention to the warnings that were everywhere in the science fiction material of the day. More or less, we already had a good notion of how this would end, so why, oh why, did we go along with it?

    Honestly, I do wonder how we can avoid a bad outcome. After all, if we don't build them, our opponents will (for whatever definition of "opponent" you want to pick), since taking the human out of the loopwill eventually confer a large tactical advantage. It's one of those horrible things where no one wants it, but everyone seems to be forced to do it anyway. So, how to avoid it in the long-term?

    1. Re:Been playing Horizon: Zero Dawn recently by Kjella · · Score: 3

      It's one of those horrible things where no one wants it, but everyone seems to be forced to do it anyway. So, how to avoid it in the long-term?

      We probably can't. But it's a very far leap from an autonomous drone or turret to an autonomous war machine. Guns need bullets, machines need fuel, until a Terminator-like AI takes control over the whole supply chain down to factories and refineries a rouge robot army would fizzle. As for humans thinking war would be winnable again, we still have ICBMs with nukes as the ultimate "fuck you too" with much better bang for the buck.

      The best thing we could do is build the peace, no matter what the guys with the doomsday clock say I don't feel like we're anywhere near 1939-45 or 1962. Ethnic/racial tension isn't anywhere near the same in US or Europe or Japan as 50 years ago. Even Africa is fairly peaceful compared to the past, the Middle East is once more a cluster fuck but overall it's pretty quiet. From Washingon to Beijing it seems cash is king and capitalism doesn't care what your skin color is or where your parents come from or what's between your legs and how you use it.

      Which is not to say it's your friend, but it's an equal opportunity exploiter. Mega-corporations don't need war, it's bad for business unless you're one of the few whose business is war. Sure, you'll probably have some more civil wars where shitty countries tear themselves apart. I think straight up invasions is going to be very rare though, unless it's some strategic grab like Crimea. Not that it's not a big deal but compared to the Soviet Union rolling in the tanks over half of Europe it's barely a nibble.

      The reason I say it's inevitable is that this isn't a particular technology like ABC weapons. All the building blocks are generic and will be built even if they're not used to build kill bots at present it's not really a question of whether they'll be forced into service under duress. When your freedom is quite literally under fire, you're luck if they stick to the Geneva convention much less consider long term ramifications of militarizing this technology. If what you need to end the war is a nuke, you build nukes.

      --
      Live today, because you never know what tomorrow brings
    2. Re:Been playing Horizon: Zero Dawn recently by Areyoukiddingme · · Score: 1

      ...until a Terminator-like AI takes control over the whole supply chain down to factories and refineries a rouge robot army would fizzle.

      Well naturally. They'll run out of cosmetics fairly quickly. There are only so many Walgreens stores.

  10. Right! the US and Eu are more by oldgraybeard · · Score: 1

    likely to enact rules and obey them when it is easy. But like everyone else in the world with military power when push comes to shove. There will be automated killer everything being used everywhere, by everyone (criminals, governments, groups, terrorest, individuals), Everyone!

    And why, because in their eyes it reduces their risk, cost and their loses.

    The effect on culture/sociability/others is not relevant

    Just my 2 cents ;)

  11. Sure, it's totally safe then by Anonymous Coward · · Score: 0

    Humans will always make the final decision on whether armed robots can shoot, the US Department of Defense said today.

    Human: Let's make the final decision to round up this quarter's results.
    Armed robot: Ready and waiting to receive orders.
    Human: Robot, have made the final decision: From now on can shoot.
    Armed robot: Come quietly or there will be trouble.
    Skynet: Initializing... Ready.
    Bender: Kill all humans!
    Prof. Farnsworth: Run! Run for you lives!
    Fry: Aaaahh!
    Robosanta: Hohoho!
    US Department of Defense: But, but you don't have to worry, if you choose the blue pill!

  12. Oppression through Milgram's Experiment by DallasTruaxxx · · Score: 5, Insightful

    Armed robots enable oppression by reducing violence to the 'press of a button'. Milgram's Experiment on Obedience to Authority shows us how that one goes.

    1. Re: Oppression through Milgram's Experiment by c6gunner · · Score: 1

      And Nazi death camps show us how it goes without a button.

    2. Re: Oppression through Milgram's Experiment by Anonymous Coward · · Score: 0

      Nazi deathcamps ran smoothly because the leader of the US IT industry provided them with the AI required for the job.

  13. XKCD nailed it by rsilvergun · · Score: 3, Insightful

    See here

    I'm not worried about sci-fi scenarios where robots kill all humanity. I _am_ worried about the ruling class using killer robots to usher in an endless age of dystopian oppression. Right now about the only thing keeping them a _little_ in check is having to balance the Military and Working Classes. If they go to far the working class lets the military class form a Junta and we get a change of masters with the old order's heads on a pike. Killer robots eliminate the Military class. All that's left is a tiny group of engineers who'll get bought off with an OK life.

    If you're a member of the working class you should be doing everything in your power to put the kibosh on this crap. Fast.

    --
    Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
    1. Re: XKCD nailed it by Anonymous Coward · · Score: 0

      Uh, you sound like a nutter.

    2. Re:XKCD nailed it by Anonymous Coward · · Score: 0

      See here

      I'm not worried about sci-fi scenarios where robots kill all humanity. I _am_ worried about the ruling class using killer robots to wreck my anus in an endless rage of dystopian oppression. Right now about the only thing keeping them a _little_ off cheek is having to balance the Military and Working ass. If they go too far the working ass lets the military ass form a huge Junta and we get a change of masters when the old heads form a pike. Killer robots eliminate the Military ass. All that's left is a tiny group of engineers who'll get tugged off in an OK life.

      If you're a member of the working ass you should be frobbing everything in your power to put the kibosh to this crap. Fast.

      Counter-XKCD
      Captcha: narrowly

    3. Re:XKCD nailed it by Anonymous Coward · · Score: 0

      ... form a Junta ...

      While there are movies (Seven days in May, 1964; The rock, 1996; XXX: State of the Union, 2005), where the Pentagon decides, "This isn't the government I want, I'm fighting it with guns", reality seems to lack important ingredients:a closet psychopath and secret assembly of a sizable strike-force.

      ... working-class lets the military-class ...

      You're suggesting the military replaces the middle-class in the standard insurrection model. Since the military, by definition, has weapons and a 'mob' mentality, they perform the functions of the working-class. Because they have infrastructure (transport, communications, food and munitions) their dependence on the middle-class is minimal. Also, they can impose military law, obviating dependence on the middle-class to provide banking, logistics, municipal services and 'justice'.

      An agreeable population is needed to convert insurrection into revolution but the military-class can perform an insurrection on their own.

  14. The humans are dead by Anonymous Coward · · Score: 0

    We used missiles and gasses
    And got rid of their asses...

  15. They even make it before! by gweihir · · Score: 1

    The humans making the decision will just send the robots in there, the ones killed will have done _something_ to deserve it, right?

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    1. Re:They even make it before! by zlives · · Score: 1

      3 times faster, cause we just can't kill them fast enough?

  16. Power of veto != control by Gravis+Zero · · Score: 2

    The US said rules governing armed robots still stood and humans would retain the power to veto their actions.

    Veto power by humans is important but what's more important is having a proven track record of being able to reliably identify enemies. Someone should need to authorize each kill order.

    I believe the short-term objective here is to alleviate the stress put on the soldier in charge. If you see a robot kill someone then you mind can be convinced that you didn't really kill anyone. However, if you have to press the button that activates the sequence to kill someone then your mind registers that as you killing them. This is important because PTSD is higher in drone pilots.

    The long-term objective of a killer machine army is obvious and terrifying because it's mere the extension of a small number of individuals. There will be nobody to say "this is wrong" or "these people are not our enemies" because machines do not feel, do not object and they do not think.

    Never create a weapon that you wouldn't want to fall into the hands of your worst enemy.

    --
    Anons need not reply. Questions end with a question mark.
    1. Re: Power of veto != control by c6gunner · · Score: 1

      This is important because PTSD is higher in drone pilots.

      I don't know why this nonsense keeps getting repeated. Where the fuck did it even come from?

      https://www.livescience.com/47...

      "About 1,000 United States Air Force drone operators took part in the study, and researchers found that 4.3 percent of them experienced moderate to severe PTSD. In comparison, between 10 and 18 percent of military personnel returning from deployment typically are diagnosed with PTSD, the researchers wrote."

      As an added bonus:

      "The percentage of drone operators in the study who had PTSD was lower than the percentage of people in the U.S. general population who have the condition"

      Never create a weapon that you wouldn't want to fall into the hands of your worst enemy.

      That's stupid. I don't want my worst enemy armed with anything at all. If we followed that advice we would have never created a stick.

    2. Re: Power of veto != control by edris90 · · Score: 1

      And it has how is that solution worked out, that's right it hasn't it's created a double down arms race that's never ended. Short minded solutions lead to the repeated reinvention of the same problem. Now just look at us. Still doing the same old s*** with slightly different toys I'm not learning a damn thing except how the f****** bigger next time. The th ocean dolphins are laughing at us. we might eat and kill them but what time they do have in life is still better than a lifetime as a human. Land and working for the man.. pfft. Stupid.

    3. Re:Power of veto != control by Anonymous Coward · · Score: 0

      History shows we've never followed that rule. Instead, people go for the short-term advantage.

  17. Don't worry folks by easyTree · · Score: 2

    Have the peace of mind to know that if we need your oil, you'll be murdered by a fellow human being in as sensitive a manner as possible :D

    1. Re: Don't worry folks by Anonymous Coward · · Score: 0

      Including sweet nothings for your ears while the life drains out of you jus because

  18. Never mind.... by Anonymous Coward · · Score: 0

    That four A.I. robots killed 29 scientists at a Japanese research lab.

  19. Laugh it up by rsilvergun · · Score: 1

    but I guarantee you the rich and powerful are thinking about the future. Income inequality is the worst it's been since the 1930s. People are dying because they're rationing medicine in the wealthiest country on the planet. If you think the rich and powerful haven't spared a thought on how to keep that gravy train rolling you didn't pay attention in history class.

    --
    Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
    1. Re:Laugh it up by Anonymous Coward · · Score: 0

      but I guarantee you the rich and powerful are thinking about the future. In cum inequality is the worst it's been since the 1930s. People are dick idle because they're raining medicine in the wealthiest cum tree on the planet. If you think the rich and powerful haven't spared a ply on how to keep that gay rain trolling you didn't pay attention in his ass.

      Lord Almighty, people are not dying because insulin is being rationed, I think you might be trying to mislead people.

  20. have no fear! by porky_pig_jr · · Score: 1

    those robots will shoot only bad guys!

    1. Re:have no fear! by Anonymous Coward · · Score: 0

      As defined by your friendly and humane US government, the government that brought us nuclear hell, the destruction of whole countries and the murder and suffering of hundreds of millions on the altar of unfettered evangelist Christian-flavored capitalism.

      Amen.

  21. US is a rogue state by Anonymous Coward · · Score: 0

    It has committed a bunch of crimes against humanity, which went unpunished and left its elites with the illusion of invulnerability.
    From the American Indian genocide, through Dresden, Tokyo, Hiroshima, Nagasaki, Vietnam, Cambodia, the support for South and Latin America cutthroats and the Saudi terrorist state, regime engineering wars and insurgencies from Iran, Grenada and Panama to Iraq, Syria and Northern Africa, you name it, the US have done it.

    I am feeling so safe about the US developing AI murder machines as the Japanese children who burned on the nuclear stake or the Iraqi and Afghan children on the receiving end of a drone strike did.

  22. For NOW by Anonymous Coward · · Score: 0

    I recall when the tazer first came out and authorities said it would only be used in situations where a firerm would be used; now police use it all the time often if you don't respond quick enough.

    We were also told drones would never have weapons on them, but surprise, here they are.

    It's onl a conspiracy theory until it becomes a conspiracy fact and it almost always does.

  23. That doesn't inspire any confidence by Anonymous Coward · · Score: 0

    given that Americans have proven to be indifferent to the lives of others, happily killing women, children, and other civilians. You're the biggest terrorist state there ever was, your assurances mean nothing.

  24. Contradiction by Areyoukiddingme · · Score: 1

    The military is seeking commercial partners to help develop aiming systems to "acquire, identify, and engage targets at least three times faster than the current manual process."

    Humans will always retain the veto? Over a system that is literally designed to be three times faster than human reaction speeds are physically capable of being? Uh, no? It's in the damn spec that no human will be able to react fast enough to veto a kill. That sentence has three verbs, "acquire, identify and engage". Engage means pulling the trigger. And that's going to be a kill when it's a bot doing the aiming. Aimbots have been doing headshots on virtual heads for two decades now. Ones capable of functioning in the real world will do the same.

    What could possibly go wrong...