Slashdot Mirror


Musk, Woz, Hawking, and Robotics/AI Experts Urge Ban On Autonomous Weapons

An anonymous reader writes: An open letter published by the Future of Life Institute urges governments to ban offensive autonomous weaponry. The letter is signed by high profile leaders in the science community and tech industry, such as Elon Musk, Stephen Hawking, Steve Wozniak, Noam Chomsky, and Frank Wilczek. It's also signed — more importantly — by literally hundreds of expert researchers in robotics and AI. They say, "The key question for humanity today is whether to start a global AI arms race or to prevent it from starting. If any major military power pushes ahead with AI weapon development, a global arms race is virtually inevitable, and the endpoint of this technological trajectory is obvious: autonomous weapons will become the Kalashnikovs of tomorrow. Unlike nuclear weapons, they require no costly or hard-to-obtain raw materials, so they will become ubiquitous and cheap for all significant military powers to mass-produce."

20 of 313 comments (clear)

  1. Postal drones will be made to go postal anyway by janimal · · Score: 4, Funny

    I don't see how it won't be easy for anyone to retrofit a postal drone to.. go postal.

  2. Is it possible? by ranton · · Score: 5, Insightful

    Like the summary says, nuclear weapons require expensive and hard to obtain raw materials and a significant amount of technology not common in the civilian space. This is the only reason, IMHO, that nuclear proliferation treaties work as well as they do. How does this group expect governments to keep a lid on military tech that relies on ubiquitous technology found throughout the civilian economy?

    --
    -- All that is necessary for the triumph of evil is that good men do nothing. -- Edmund Burke
    1. Re:Is it possible? by SafPlusPlus · · Score: 4, Interesting

      Governments could respond by making drones illegal, encryption illegal, publication of research illegal, diy electronics illegal, etc. All of which don't sound particularly nice to me either, but then again... these things are already happening, right?

  3. I have no fear of AI, but fear AI weapons by gurps_npc · · Score: 5, Insightful
    The problem is not the rise of an AI revolution.

    Instead, it is the rise of a human psychopathic tyrant working with a force of soldiers that obediently kill at his command, with no chance of moral rebellion within his own force.

    --
    excitingthingstodo.blogspot.com
    1. Re:I have no fear of AI, but fear AI weapons by gurps_npc · · Score: 4, Interesting
      This is one of those "You only hear about the failures" situation. No one hears about the crazy kid that was given psychiatric counseling and decided NOT to use an ak47 to kill everyone.

      There have not been 4 attempts to do this (Hitler, Stalin, Saddam, North Korea), but 400. We stopped well over 90% of them, but you don't hear about them

      As for those people you mentioned, many of them were hamstrung by ethical people whose refusal to kill slowed down their crazy lessons.

      --
      excitingthingstodo.blogspot.com
    2. Re:I have no fear of AI, but fear AI weapons by Linkreincarnate · · Score: 4, Interesting

      Seriously. How many people would rob a liquor store if they didn't have to be there in person and there was no chance of being caught?

  4. Drones by PvtVoid · · Score: 4, Insightful

    One of the things that has consistently mystified me about Americans' complacency with drone warfare is the underlying assumption that our current monopoly on drones is going to last forever. If it's ok for the U.S. to use drones to assassinate "terrorist" anti-American agitators in Yemen, what are we going to say when China starts using drones to assassinate "terrorist" Chinese dissidents on American soil, or Europe, or elsewhere? For all intents and purposes, we're already using killbots, and the really important point here is that airborne killbots can be used (for now) with impunity across borders.

    "American Exceptionalism" basically means we allow ourselves to commit war crimes with impunity.

    1. Re:Drones by TheCarp · · Score: 4, Insightful

      Exactly. What is the difference between an automated system and one with a human at the helm when you can just replace the human with impunity if he decides he doesn't want to help you anymore?

      Its not like some criminal gang where a defector could mean consequences. A defector from the drone murder program is just....replaced. Even if 100% of pilots became disgusted with the job and refused within a year.... it wouldn't even slow them down, it would just increase their training costs.

      Right now, there effectively is no difference between the existing drone program and automated kill bots. The problem is what people want to do and are allowed to get away with. As long as they can murder with impunity, the methods which they use are unimportant.

      --
      "I opened my eyes, and everything went dark again"
    2. Re:Drones by DarkOx · · Score: 5, Interesting

      Admittedly have never been an infantry man, pilot or any other sort of military man myself I still suspect its much easy for a guy sitting safely in chair to make a moral decision about a target, than it is for a guy in a life threatening situation to do so.

      A drone operator can loiter around a target for a long time until he or she is confident said target is properly identified. A jock in a fighter-bomber does not have that luxury and also exists in constant fear someone is going to pop up with an anti air craft device, that will end his life. The drone operator has to worry an anti air craft device will ruin his afternoon with extra paper work. I known which one I'd rather imagine hovering over me deciding if I an enemy combatant or just a guy going out to milk the goats.

      The separate question is does done warfare lower the barrier to entry such that conduct operations in theaters that would forgo if it meant having the infrastructure and associated costs of supporting large numbers of manned air craft in the area. This is over great concern. If we make warfare to easy we might find ourselves doing more of it. I am not buying the argument though that drones are equivalent to mindless kill bots or worse than the existing maned alternatives in any given situation all else being equal.

       

      --
      Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
  5. May as well ban rain by TheCarp · · Score: 4, Informative

    I just don't see the point. These will be developed, and no amount of banning them will stop it or even slow it down.

    --
    "I opened my eyes, and everything went dark again"
    1. Re:May as well ban rain by fche · · Score: 5, Insightful

      The moral high ground is a killzone, should hostilities break out.

  6. It's a little late folks.... by bobbied · · Score: 4, Insightful

    That ship has sailed a LONG time ago... We've been making such weapons for decades.

    What's a mine? What's a cruse missile? Proximity fused ground to air shells? Homing torpedos? What's all that "fire and forget" stuff we've been building?

    I'm afraid the cows are ALREADY out of the barn on this....

    --
    "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
  7. Crime against AI by DrYak · · Score: 4, Funny

    {Unless of course} They run on Windows

    Which would be a valid reason to introduce a new international treaty on "Crimes against sentient AI" under which to prosecute those cruel enough to subject a poor AI to running on a Windows platform~

    --
    "Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
  8. Re:Futile by Anonymous Coward · · Score: 5, Interesting

    Start with land mines. These are autonomous weapons with little or no AI and have done far more devastation to civilian populations.

    The AI arms race will basically be to develop more accurate enemy identification. The "low" tech AI will have more "friendly fire" incidents than the "high" tech AI.

    The old game of nethack warned not to genocide shopkeepers. If you genocide them you would kill all humans, including your own character. This seems applicable to AI autonomous weapons.

    On the other hand, a theoretically competent AI running a weapon could make choices of not engaging despite a command or even an attack on it because of the risk of civilian or collateral damage. A human holding a weapon would not necessarily be able to make the dispassionate trade of self-sacrifice for some number of strangers or monuments.

  9. Next up, perhaps by Jiro · · Score: 4, Interesting

    Prominent world politicians urge adoption of new changes to the C++ standard concerning private inheritance and templates.

    What this is trying to do is imply that because they have technical expertise in how dangerous AI-controlled weapons are, that technical expertise makes them experts about political decisions concerning weapons. It doesn't, and there is no more reason to pay attention to them than to the average guy in the street (who understands that some weapons are dangerous, and may have opinions on their use, but certainly doesn't get a national press release about it).

  10. Think like a soldier in the next war for a moment by Karmashock · · Score: 5, Insightful

    ... Okay... so... you have an option to use a kill bot against the enemy that wants to kill you... and if you go out there... you could be killed.

    Or... you send in your terminator bot and worst case they scrag the robot.

    What are you going to prefer here?

    A lot of people offering opinions here are not speaking from that perspective. They're speaking often as not from the perspective of some civilian ideologue that knows they're not going to go to war.

    I know that if I go to war... I am going to want the best weapons my society can make for me along with the best defenses the best training and ideally leaders that are not complete fuckwits.

    That means I want the robots. I want them and I want them to be fucking vicious.

    Go on youtube and you'll see US soldiers cheering when air support shows up and blows the fuck out of someone shooting at them.
    https://youtu.be/1IcvjD4VVjY?t...

    Now... if you are a country that has the ability to build kill bots... and you might be on the firing line... do you or do you not want to use killer robots to kill your enemies?

    You have to put your brain into war mode to understand the question.

    My vote... is yes.
    https://www.youtube.com/watch?...

    When I go to war... I go to WAR.

    --
    I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
  11. Re:Futile by MightyMartian · · Score: 5, Interesting

    It's similar to the situation at the end of WWI. Versailles called for wide-ranging disarmament among all the belligerents, which was all well and good in theory. In reality, of course, a great deal of the R&D that had gone into new weaponry; tanks, planes, ship designs, and so forth, still existed. In fact, the most valuable commodity of all, the German plans for the 1919 campaign that never was, still sat in archives, just waiting for someone to come along and dust them off.

    The cat is out of the bag, has been out of the bag for a few decades now. When most of us look at devices like Mars Rovers, we're impressed by the technology and science, and yet that very same technology is easily adaptable to building autonomous weapons. Even if the Great Powers agreed, you can be darned sure they would still have labs building prototypes, and if the need arose, manufacturing could begin quickly.

    --
    The world's burning. Moped Jesus spotted on I50. Details at 11.
  12. Narrowminded Fools by Anonymous Coward · · Score: 4, Insightful

    This is an absolutely inane idea for several reasons:
    a) They already exist; you can't defend against a sea-skimming missile or SRBMs without an autonomous system, People are just too slow.
    b) Bad actors are not constrained by treaties. They'll cheat. We'd be damn fools to put ourselves at a disadvantage.

    What makes more sense is to have a discussion about how they're used and how they're employed. I think it's plausible that they be prohibited from being autonomously travelling or that they must have a human authorize them to continue engaging every hour or day. An outright prohibition is just polyannaish claptrap.

    1. Re:Narrowminded Fools by mrvan · · Score: 5, Insightful

      Just like in the good old days!
      s/spammers/bad guys/g
      s/spam/autonomous weapons/g

      Dear Musk, Woz, Hawking, and Robotics/AI Experts

      Your post advocates a

      ( ) technical (X) legislative ( ) market-based ( ) vigilante

      approach to fighting autonomous weaponry. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

      ( ) Bad guys can easily use it to harvest weapon designs
      (X) Defense systems and other legitimate uses would be affected
      ( ) No one will be able to find the bad guys
      ( ) It is defenseless against brute force attacks
      ( ) It will stop autonomous weaponry for two weeks and then we'll be stuck with it
      (X) Users of weapons systems will not put up with it
      (X) DARPA will not put up with it
      (X) The military will not put up with it
      (X) Requires too much cooperation from bad guys
      (X) Requires immediate total cooperation from everybody at once
      (X) Many weapon producers cannot afford to lose business or alienate potential employers
      (X) Bad guys don't care about illiegal weapons in their arsenals
      ( ) Anyone could anonymously destroy anyone else's career or business

      Specifically, your plan fails to account for

      ( ) Laws expressly prohibiting it
      (X) Lack of centrally controlling authority for weapons
      (X) Open relays in foreign countries
      (X) Asshats
      (X) Jurisdictional problems
      (X) Unpopularity of weird new treaties
      (X) Public reluctance to accept weird new forms of arms control
      (X) Armies of worm riddled broadband-connected Windows boxes (!)
      (X) Eternal arms race involved in all filtering approaches
      (X) Extreme profitability of autonomous weaponry
      ( ) Joe jobs and/or identity theft
      ( ) Technically illiterate politicians
      (X) Extreme stupidity on the part of people who do business with bad guys
      (X) Dishonesty on the part of bad guysthemselves

      and the following philosophical objections may also apply:

      (X) Ideas similar to yours are easy to come up with, yet none have ever
      been shown practical
      ( ) Any scheme based on opt-out is unacceptable
      ( ) Blacklists suck
      ( ) Whitelists suck
      ( ) Countermeasures should not involve wire fraud or credit card fraud
      ( ) Countermeasures should not involve sabotage of public networks
      ( ) Countermeasures must work if phased in gradually
      (X) Why should we have to trust you and your treaties?
      ( ) Incompatiblity with open source or open source licenses
      (X) Feel-good measures do nothing to solve the problem
      (X) I don't want the government limiting my arsenal
      ( ) Killing them that way is not slow and painful enough

      Furthermore, this is what I think about you:

      ( ) Sorry dude, but I don't think it would work.
      (X) This is a stupid idea, and you're a stupid person for suggesting it.
      (X) Nice try, assh0le! I'm going to find out where you live and burn your
      house down!

  13. Re:Yes! by gestalt_n_pepper · · Score: 4, Funny

    We don't have to kill ISIS, just distract them until they're 60 or so.

    Cheaper yet, we should be airdropping porn, pot and pizza. The three P's of victory!

    --
    Please do not read this sig. Thank you.