'Don't Fear the Robopocalypse': the Case for Autonomous Weapons (thebulletin.org)

Lasrick shares "Don't fear the robopocalypse," an interview from the Bulletin of the Atomic Scientists with the former Army Ranger who led the team that established the U.S. Defense Department policy on autonomous weapons (and has written the upcoming book Army of None: Autonomous Weapons and the Future of War). Paul Scharre makes the case for uninhabited vehicles, robot teammates, and maybe even an outer perimeter of robotic sentries (and, for mobile troops, "a cloud of air and ground robotic systems"). But he also argues that "In general, we should strive to keep humans involved in the lethal force decision-making process as much as is feasible. What exactly that looks like in practice, I honestly don't know."

So does that mean he thinks we'll eventually see the deployment of fully autonomous weapons in combat? I think it's very hard to imagine a world where you physically take the capacity out of the hands of rogue regimes... The technology is so ubiquitous that a reasonably competent programmer could build a crude autonomous weapon in their garage. The idea of putting some kind of nonproliferation regime in place that actually keeps the underlying technology out of the hands of people -- it just seems really naive and not very realistic. I think in that kind of world, you have to anticipate that there are, at a minimum, going to be uses by terrorists and rogue regimes. I think it's more of an open question whether we cross the threshold into a world where nation-states are using them on a large scale.

And if so, I think it's worth asking, what do we mean by"them"? What degree of autonomy? There are automated defensive systems that I would characterize as human-supervised autonomous weapons -- where a human is on the loop and supervising its operation -- in use by at least 30 countries today. They've been in use for decades and really seem to have not brought about the robopocalypse or anything. I'm not sure that those [systems] are particularly problematic. In fact, one could see them as being even more beneficial and valuable in an age when things like robot swarming and cooperative autonomy become more possible.

Comments

They're ready: except costs

[Anonymous Coward]

Army / marine "grunts" are cheaper than the bots. The soldier's job is safe for now. But like all technology, add time to the equation and things will change. Although the bots will probably not be autonomous, they'll be remote controlled.

Machine will fight machine like a glorified "robots war" TV show.

Re:They're ready: except costs

[gravewax]

I think we have well and truly crossed the cost line now. trained soldiers are expensive. A deployed soldier can be upwards of half a million a year in costs. drones and bots can already be more cost effective in many situations, the cost benefits are only going to accelerate more rapidly from here, I would say a large portion of the military personnel is very much on the countdown to obsolescence.

Re:They're ready: except costs

[Wootery]

A deployed soldier can be upwards of half a million a year in costs.

[Citation needed]

Re:They're ready: except costs

[Anonymous Coward]

A deployed soldier can be upwards of half a million a year in costs.

[Citation needed]

In the land of $100 hammers and $1000 toilet seats, you really need proof of this? Give me a fucking break.

Re:They're ready: except costs

[Anonymous Coward]

they can be more cost effective, but probably won't be knowing how everything is way overpriced when the army buys it....

Re:They're ready: except costs

[ShanghaiBill]

A deployed soldier can be upwards of half a million a year in costs.

[Citation needed]

In the land of $100 hammers and $1000 toilet seats, you really need proof of this? Give me a fucking break.

Yes, we need a citation, because the $500k number is total bullcrap. It is implausible that a deployed combat soldier, at the far end of a 10,000 mile supply chain, costs so little.

Here is a citation that the actual cost is $850k to $1.4 million per soldier per year.

War ain't cheap.

Re:They're ready: except costs

[Anonymous Coward]

Yeah, but that's not the issue.

The issue is who decides to kill. Software or people. As flawed as we all are I still prefer the latter.

Re:They're ready: except costs

[Chris Mattern]

Actually, that's conservative. CNN reports $850,000 to $1.4 million for US soldiers in Afghanistan.

Re:They're ready: except costs

[gatkinso]

I's be surprised if it was as cheap as half a million.

Re:They're ready: except costs

[Anonymous Coward]

A deployed soldier can be upwards of half a million a year in costs.

[Citation needed]

In the land of $100 hammers and $1000 toilet seats, you really need proof of this? Give me a fucking break.

Yes, we need a citation, because the $500k number is total bullcrap. It is implausible that a deployed combat soldier, at the far end of a 10,000 mile supply chain, costs so little.

Here is a citation that the actual cost is $850k to $1.4 million per soldier per year.

War ain't cheap.

If you're looking for a logical argument to have, it's not the fucking cost of a soldier.

It's the justification to continue to wage fucking war.

War may not be cheap, but it sure as shit turns a profit. As long as somebody's getting rich, blood will continue to flow for senseless reasons.

Re:They're ready: except costs

[mysidia]

I think we have well and truly crossed the cost line now. trained soldiers are expensive.

I think the greatest concern with autonomous weapons is they can entirely change the rules of the game re. asymmetric warfare.
Autonomous large-scale-deployable indiscriminate weapons can be just a less-efficient form of other WMDs such as Chemical Weapons, which are also banned.
Imagine a country deploys 10000 killer drones over a small county to spread panic and fear.
If the assailant has autonomous weapons in sufficient number that can effectively target and kill any human before they can so as much get a shot:
traditional ground troops cannot counteract these kinds of attackers, and 10000 drones with say 1000 bullets each and a perfect shot every time = 10 Million dead humans, and no risk of loss of life in those conflicts to the attacker.

Yes.... Autonomous Weapons are going to happen: what needs to be banned internationally is the use of Indiscriminate Autonomous Weapons, especially Mobile indiscriminate weapons that can move a long distance on their own power or be deployed to a remote target, and the deployment of Autonomous Weapons designed to target humans even if unarmed in general or carry or release explosives.

Indiscriminate: Autonomous devices that you drop at a location that will immediately activate a targeted attack against any human or any animal that moves: regardless of whether the person is a threat or not.

We should be trying to actively develop Autonomous defenses against other weapons systems, and Autonomous devices that Monitor, Identify, share information about, and Destroy potential autonomous threats.

Re:They're ready: except costs

[Anonymous Coward]

The public fails to understand the costs of military personnel. Training costs money. Supplying food, water and other essentials is expensive in remote areas or areas in which the food supply may be sabotaged or poisoned. Medical care for sick or wounded soldier is expensive as well. A disabled soldier can easily need millions in medical care and then we have the issue of supporting wives and children of dead troops. Recently we lost our last soldier from WW1. Now we have soldiers from WW2, Korea and Vietnam as well as their surviving wives to take care of. The idea that a soldier can cost over one million dollars a year to deploy is a reasonable estimate.

Re:They're ready: except costs

[Anonymous Coward]

CNN is WRONG on their facts WAY to much to believe a DAMN thing they say anymore....

Re: They're ready: except costs

[Anonymous Coward]

And Faux news or stats are far worse.

Take a Basic critical thinking class

Re: They're ready: except costs

[LifesABeach]

I don't think the A/C poster is comfortable with words as complex as 'Basic'

Re:They're ready: except costs

[LifesABeach]

'3 Laws Safe' is sounding more and more appealing.

Re:They're ready: except costs

[Anonymous Coward]

trained soldiers are expensive.

Except we want war to be expensive. The cost is the only thing that keeps the warmongers in check. Or rather, it's the cost associated with protection and the fear of not being able to meet it, that keeps the warmongers in check. Huge expense is about the only thing that makes these psychopaths come to the negotiating table. Imagine what would happen if war was cheap. Rather than negotiate for mineral rights so that everyone benefits, you'd just have armed drones swoop in and kill anyone in the vicinity. Forget terrorism, you'd be more afraid of pissing off someone with access to the drone controls than someone with a bomb vest. Hell, even the police wouldn't be needed anymore, you commit a "crime" and you get visited by your friendly neighborhood T-1000.

A person who wants cheap war, is a person who believes that the lives of others are irrelevant. I've always thought that the creators of Skynet got exactly what they deserved for doing so. They tried to make war cheaper, and require less effort, but they went too far. They made it too cheap and made it so that it required no effort from themselves, and they paid the price first for their foolishness.

Re:They're ready: except costs

[HiThere]

The recent drone attack on an army base has more or less convinced me that even non-state actors are finding automated weaponry a good choice. So costs can't be too much of a barrier.

OTOH, flexibility is still superior for the human. Most robots can only deal with a relatively small number of cases, and none are even approximately as good at self-repair. (But it's easier to replace parts...so that may be an even trade-off, except for costs.)

It strikes me as an extremely dangerous direction to head, but it also seems to be one that we are definitely headed in, because even if one group doesn't do it, another will.

Re: They're ready: except costs

[Anonymous Coward]

Why do you people keep bringing up Asimov's Three Laws of Robotics? Even Asimov didn't think they were logic-proof hence most of his stories focused on robots misinterpreting the very vague set of rules.

Re:They're ready: except costs

[hey!]

This does not surprise me at all, especially as the cost is arrived at by taking the total campaign cost divided by the number of soldiers.

Everything the US military does costs an eye-popping amount. The V22 Osprey costs $64000/hour to operate. The Bradley AFV cost over $50 for every mile driven. Recoilless rifle ammunition runs between $500 and $3000 per round. Every time an A10 opens up its mighty 3900/ round/minute cannon, each of those rounds costs $150.

The current administration's plans for increases in troop levels in Afghanistan are expected to cost the US taxpayer over a trillion dollars when all the downstream costs are included. In return they hope to secure access to about a trillion dollars in mineral reserves for US companies.

Re:They're ready: except costs

[Anonymous Coward]

700 troops are estimated to cost $1 TRILLION?

At $1 million per soldier per year (on the high side), that's 15,000 YEARS of deployment. And their cost only goes down when they are no longer deployed - health care, salary, pension, etc, all totals to less than $300,000 per soldier per year or so (high end estimate).

In other words: What you are saying is complete bullshit. You should feel bad for even thinking it might be true for more than a few seconds.

Re:They're ready: except costs

[gravewax]

well I did explicitly say UPWARDS of 500k. 500k was the last official number I had seen. I would suspect even those cnn figures are dated now and costs would be well over the million a year per soldier.

Re:They're ready: except costs

[hey!]

I should have been clearer: it will bring the total cost for the war into the trillion dollar range, counting all downstream costs.

Re:They're ready: except costs

[bloodhawk]

no way in hell is a soldier even when not deployed is only 300k a year. training, housing, equipment, transport all cost significant amounts and many of the deployment costs while less don't just return to zero as you need to have the equipment, personnel, IT etc to support those people whether they are actively being used or not.

Re: They're ready: except costs

[LifesABeach]

The tragedy is that A/C's are genetically incapable of creating anything better or even equivalent. But I hope that this posting A/C may be the exception.

Re:They're ready: except costs

[Anonymous Coward]

You know, you could just stop fighting a few of your foreign wars, if they're getting too expensive for you (at last).
We, peoples of the rest of the world, won't mind much at all, thanks :)

Re:They're ready: except costs

[Agripa]

So drone warfare will make war less expensive so more people can participate? Yea, that sounds like a good idea. Let's do that.

Easy enough

[Anonymous Coward]

Suspected terrorist detected. Kill? (Yes/No/All)

Just creating them is dangerous.

[Gravis Zero]

Just because someone is going to make some shitty systems that might as well be bombs doesn't mean we should start investing tens of billions into creating the smartest most lethal autonomous system because we think we can control it. The Chinese government already compromised and stole everything they need to build their own F35s, do you really want to tempt them with a killing machine with unwavering loyalty and a complete absence of conscious? We already know how they roll, they will sell weapons to anyone that will pay. Do you really want to add automated killing machines to that list?

Re:Just creating them is dangerous.

[Anonymous Coward]

I think the problem he is pointing out is that banning autonomous war fighting systems, or not researching them, is a bit like banning malware. We have all seen how far that gets you. We have been asking North Korea pretty please for 20 years to stop the research into nukes, and those are hard to make. The motion sensing tech and facial recognition can be run on a Raspberry Pi, hook that to a solenoid and you have crossed the line.

I think research into it needs to be done. Controlling it is an UGLY issue.

Re:Just creating them is dangerous.

[ShanghaiBill]

Given a choice, which would you rather have come to your village:

1. A carefully designed, programmed and tested robot

2. A squad of soldiers that haven't slept in two days or eaten in 18 hours, and who just medevaced a comrade who had his leg blown off below the knee by a booby trap

Are you really sure you want a human decision maker "in the loop"?

Re: Just creating them is dangerous.

[Anonymous Coward]

Fallacy detected: false dichotomy. I want neither.

Re:Just creating them is dangerous.

[rickb928]

"a killing machine with unwavering loyalty and a complete absence of conscious"

That's not the problem.

If you, a civilian police officer, are faced with a terrorist attack by a semi-autonomous robotic rifle, trigger being pulled at maximum rate (or a bump stock being part of the mechanism), immune to small arms fire, able to target accurately out to 100 yards, and able to negotiate stairs and open doors, what do you do, wait for the military to respond?

Such devices are well within reach of determined and moderately funded groups. Remotely guided devices seem simpler. Change the weapon to a pistol and put it on a hefty drone and you have air cover and the death toll is tripled, mostly first responders.

It's not the lack of conscience of the devices, it's the lack of conscience of the criminals building and deploying them. We face enemies that have a different conscience than we do, one incompatible with our typical morals. THAT is the problem. They will build and use weapons of unacceptable, unimaginable brutality and efficiency, all to achieve their goals.

Re:Just creating them is dangerous.

[Anonymous Coward]

Well, at least Robots would lack the rape women and children module.

Re: Just creating them is dangerous.

[locketine]

Anyone could make your same argument against our police. Have you watched a sci-fi movie recently? There's plenty of them demonstrating how brutal the once good guys could become once they have incredible power and little at risk. Throw in unwavering loyalty and you've got the evil Empire from Star Wars.

Re: Just creating them is dangerous.

[rickb928]

Sure. Different solution.

Glad my example is valid for other use cases.

Re:Just creating them is dangerous.

[mysidia]

immune to small arms fire, able to target accurately out to 100 yards, and able to negotiate stairs and open doors, what do you do, wait for the military to respond?

You need a swarm of remote-controllable drones able to get within sufficient range of the offender and deploy an artificial EMP.

The real problem is how much more quickly an autonomous attacker can kill/hurt a lot of people with precision BEFORE a credible response could be launched, and the fact the terrorist might have the advantage of targetting the very location, people, or things they need to target in order to snuff out or delay response efforts.

So I say you need a partially autonomous automatic response with no single or centralized point subject to attack..... this suggests surrounding the public with fleets of surveillance drones whose purpose is to identify and alert on potential autonomous (or other) threats And upon a risky enough situation begin the response process on their own.

Re:Just creating them is dangerous.

[Anonymous Coward]

In the absence of a third choice of no robots OR soldiers, I'd pick the soldiers, hands down. They, at least, would carry with them the possibility of not wiping the village off the face of the planet. For all we know, they could just be hungry and thirsty.

Robots, on the other hand, are only sent to a village for one reason: to destroy the target with high flexibility on collateral casualties. They're basically a napalm strike, but with better PR optics.

Re:Just creating them is dangerous.

[rickb928]

"You need a swarm of remote-controllable drones able to get within sufficient range of the offender and deploy an artificial EMP."

Collateral damage prohibits this.

"The real problem is how much more quickly an autonomous attacker can kill/hurt a lot of people with precision BEFORE a credible response could be launched, and the fact the terrorist might have the advantage of targetting the very location, people, or things they need to target in order to snuff out or delay response efforts."

Response time is the Achilles heel of counter-terrorism. True. However, imagine this hypothetical device surviving the initial response. The duration will increase substantially. So too the casualties. Evacuation would have to be a quick decision, and then possibly waiting the device out. A little imagination leads you to using an armored vehicle to take that out, if it's accessible, or a counter-robot and explosives, probably. Collateral damage. Bad situation.

"So I say you need a partially autonomous automatic response with no single or centralized point subject to attack..... this suggests surrounding the public with fleets of surveillance drones whose purpose is to identify and alert on potential autonomous (or other) threats And upon a risky enough situation begin the response process on their own."

"no single or centralized point subject to attack" - um, sorry, but a single target doesn't fit into this strategy as expressed. Worse, though you propose a standing fleet of drones ready to react. I doubt this is successful if the threat becomes an explosive. Too late. But a hit and run attack has your drones trying to identify the threat, which is still, despite being simple manufacture, able to shoot-and-scoot in an urban setting, spreading the damage, which will probably be the strategy as it evolves, and identifying the threat is the key step, If it dodges, you have a lot of problems, standing fleet or not.

We are within reach of a time where response is not enough. But prevention is out of reach, unless you tackle the root causes.

Re:Just creating them is dangerous.

[Anonymous Coward]

>>>> A squad of soldiers that haven't slept in two days or eaten in 18 hours, and who just medevaced a comrade who had his leg blown off below the knee by a booby trap>>>

  War is supposed to be horrific. That's a big
incentive to prevent it.

  When you turn it into a real life Nintendo game, the incentive is removed.

  Also, back in medieval days, the king/lord/duke would often be *right there on the battlefield* commanding his army, not cowering in some
luxurious palace watching the events (more
like being given a vague abstraction of said events) on a TV screen and being
handed reports by men in suits.

Re:Just creating them is dangerous.

[HiThere]

The argument that war being horrific will prevent it doesn't work very well historically. It's a bit better than "war is profitable", but that's a pretty low barrier.

To take your example, the middle ages is full of dukes/lords/kings leading armies back and forth across various battlefields until the area being fought over was so wrecked it couldn't support the people living there. That is often resulted in the duke/lord/king getting killed didn't stop things. It's not clear it slowed them down much. People tend to discount future dangers in favor of present gains.

Additionally consider the crowds at soccer games. They'd have a lot less chance of getting hurt if they didn't riot, but that doesn't reliably stop them.

So the available evidence seems to indicate that "war being horrific will keep it from happening" is a false claim. Much more likely is "War is unprofitable even in the short term.", so that's what we need to set up. But you still shouldn't expect perfection. The soccer fans don't have anything to gain, and they've got a lot to lose.

Re:Just creating them is dangerous.

[HiThere]

It's not the lack of conscience of the devices, it's the lack of conscience of the criminals building and deploying them. We face enemies that have a different conscience than we do, one incompatible with our typical morals. THAT is the problem. They will build and use weapons of unacceptable, unimaginable brutality and efficiency, all to achieve their goals.

I'm not precisely disagreeing with you, but consider:
It's not the lack of conscience of the devices, it's the lack of conscience of the governments building and deploying them. We face enemies that have a different conscience than we do, one incompatible with our typical morals. THAT is the problem. They will build and use weapons of unacceptable, unimaginable brutality and efficiency, all to achieve their goals.

If our government not building these devices would prevent them from coming into existence, I'd be totally against them. Unfortunately...

I'm rather certain that these devices are going to become increasingly available, until some people will have them patrolling their houses. Laws against booby-traps will probably make them illegal, but that won't totally prevent them. Certainly some intruders will use variants of them to scope out the scene, and perhaps to conduct the intrusion without their presence. An automated burglar would probably be more difficult than an automated assassin, though.

Re:Just creating them is dangerous.

[ShanghaiBill]

War is supposed to be horrific. That's a big incentive to prevent it.

America participates in more wars that any other country. As an American, if a particular war is too horrific, I can just close my browser tab for that conflict, and read about something else instead.

Re:Just creating them is dangerous.

[fox171171]

Given a choice, which would you rather have come to your village:

1. A carefully designed, programmed and tested robot

2. A squad of soldiers that haven't slept in two days or eaten in 18 hours, and who just medevaced a comrade who had his leg blown off below the knee by a booby trap

Are you really sure you want a human decision maker "in the loop"?

1. A carefully designed, programmed (to kill all humans) and tested robot

Re:Just creating them is dangerous.

[Kjella]

That greatly depends on the overall strategy for the war.

1. To drain the opposition of support
or
2. To crush the opposition by force

The former is what most Americans in living history now know, like the Korean war, the Vietnam war, the Gulf war etc. and in that case it's true. You don't have rouge elements, you don't have soldiers on a power trip that rape, pillage and murder. You don't have soldiers who fear for their own lives and who'd rather cause incidental or accidental damage to civilians than risk getting shot and killed themselves. There's no more caskets of American servicemen being flown home and no more public outcry, all you have to do to play world police is pay for replacements.

That's not how you occupy a country. That's not how Nazi Germany crushed France. That's not how the Allies crushed Nazi Germany. That's not how the US made the Japanese surrender. It's basically saying we're going to pound you into the ground and keep pounding until you beg for mercy. If there's attacks or sabotage on our forces we'll retaliate ten times harder or just round up a bunch of civilians and shoot them. And if we want to do a little genocide or ethnic/religious cleansing that's what we'll do.

In that situations robots are the perfect psychopaths. They have no guilt or remorse, they don't feel the hate and anger, they don't care if they're despised and attacked, they don't desert or refuse to follow orders. They don't care if the targets are civilian or military, young or old, women and children. And you can say it's the same as a gun today, bullets don't care. The difference is that the guy pulling the trigger is 10000 miles away playing Counterstrike or maybe just playing a real world game of Civilization. City in riot? Send in the troops.

It would be nice fantasy if the atrocities of war happened because of the atrocities of war. But those manning the gas chambers of the Holocaust weren't hungry, tired, revengeful or full of PTSD. Neither have most invading generals throughout history been, they command and other people suffer and preferably the other side. Just because we take away the "burden" of fighting on the front lines doesn't mean those people will stop shuffling pieces on a chess board to win. In fact, they'd probably just find it easier if the pawns stop complaining.

Re:Just creating them is dangerous.

[Anonymous Coward]

3. I'ld prefer to sleep in my village on Sundays.

fuck the soldiers, fuck the robots.

fuck the people who use them.

clearly, you don't live in a small village.

Re:Just creating them is dangerous.

[rickb928]

Self-defense is likely the least common use case. I'm not concerned about that, but more about police use, where errors are so prevalent that elevating their capabilities will increase the lethality of mistakes/SWATing/overreaction, and the police are not yet held sufficiently accountable to discourage these.

Accountability isn't a uniquely police issue.

Re:Just creating them is dangerous.

[dywolf]

https://en.wikipedia.org/wiki/...

Re:Just creating them is dangerous.

[dywolf]

emp is not even a real weapon yet.

people keep forgetting that.

actual EMP weapons (not created by a nuclear blast) are still in the realm of scifi.
if they were not our troops would be using them daily simply because of how dang useful theyd be in a fight.

Re:Just creating them is dangerous.

[Anonymous Coward]

Terrorist groups could come into possession of autonomous weapons a la this video https://www.youtube.com/watch?v=1bM2-stFXO0

Honest folks will be constrained.

[Anonymous Coward]

We know how it goes: honest people are forbidden to do stuff (even to *know* stuff) and crooks get away with it.

Especially because that's where state agencies and big corps meet each other at a blissful collusion.

Robopocalypse

[Anonymous Coward]

The word robopocalypse consists of two parts. "Calypse" meaning "hidden," and "Robopo," which, um, ok, I got nothin.

The "expert", has spoken

[geekmux]

"In general, we should strive to keep humans involved in the lethal force decision-making process as much as is feasible. What exactly that looks like in practice, I honestly don't know."

So, you tell me don't fear something, and then the expert says this statement.

Gee, I feel so much better that we have no idea how to do anything about the pending robopocalypse other than to wag our finger at the evil in the world and say "Remember to play nice and be honest and fair when trying to kill each other."

Re: The "expert", has spoken

[Anonymous Coward]

It is the MAD policy again: I won't deploy my killer micro robot swarm if you don't deploy yours

Re: The "expert", has spoken

[HiThere]

The problem is, the MAD solution was unstable even with three players. This is going to start off with dozens of players, so MAD is critically unstable even before the "game" has started.

Its a terrible idea in principle AND practice.

[sg_oneill]

Autonomous killing machines are a frankly horrific idea. In principle, machines should serve us, NEVER the other way around. On first principles alone the idea that a machine could determine who to kill and who not to kill is a chilling idea.

And in practice, its a terrible idea. Human soldiers already face baffling moral situations. Woman with child at checkpoint acting suspiciously. Maybe suicide bomber. But she has a child. To shoot or not to shoot. Thats the kind of thing guaranteed to give a marine a gnarly case of PTSD,if he choses wrong, and possibly also a dead mother and child (or conversely a dead platoon). But the possibiliy of a horrifically wrong choice means that Marine is going to deploy ever fragment of reason his brain can muster. . How the hell would we entrust such a monumental decision to a robot. Its "wrong" choice has no repercussions for it. If it kills an innocent mother, it doesnt care, its just a thing. If it opts for caution and it choses wrong, it still doesnt care, its already dead. Theres no incentive anywhere up the chain of command to get this right, because 'Well a robot chose badly, sorry not our fault!' is a get out of jail free to just let the bloody thing go robocop on a civilian population. We *morally* AND *practically* NEED humans in that decision loop, even if its just some guy in an air conditioned office and a VR headset.

Re: Its a terrible idea in principle AND practice.

[Anonymous Coward]

Regarding speculation, a robot and human have the same likelihood to guess who is a suicide bomber. Human intuition is a roll of the dice, same as a machines.

Re:Its a terrible idea in principle AND practice.

[Anonymous Coward]

We *morally* AND *practically* NEED humans in that decision loop, even if its just some guy in an air conditioned office and a VR headset.

Yes! We already have a ban on certain land mines since they kill without a human operator pulling the trigger, often harming innocent non-combatants.

It is easy to deploy weapons that kill someone without you being anywhere near and don't require you to have to make a judgement call. Those weapons kill indiscriminately. And even in war there are rules of engagement that tells us what we can do and cannot do, unless you want to be a war criminal. Non-combatants should not be targeted for example. Perhaps you could make better autonomous weapons, but we have yet to teach a machine ethics and compassion.

B... but the bad guys! They will use them.

Sure, there will be some people ignoring the international treaties, and deploying whatever means there is to kill as many as possible. That doesn't mean that we should give up. On the contrary! We should double down and fight this harder. I know war has changed a lot since the last two great wars, but if we want to rise above that, we should nip these things in their buds and send a message to anyone considering deploying such weapons that the rest of the world will NOT be sitting by idly and there will be repercussions for doing so.

I believe these kinds of weapons should be banned, as with chemical and biological warfare.

Re:Its a terrible idea in principle AND practice.

[janken001]

A point I made above is that they are too easy and cheap to create to ignore. If we had been on the fence about using them and they existed on 9/10/02 the next day would have decided it for most of the country. Ignoring the area is a bit like ignoring malware and virus development. The question to me is how do you discuss and control something this easy and risky? Mind you this is going to happen. I am sickened by the idea but anyone who is read or watched American Sniper knows the question and the outcomes. Anyone who can program a SBC can create one that would pull the trigger without question.

Re:Its a terrible idea in principle AND practice.

[Anonymous Coward]

Do none of these people watch Star Trek:-

https://en.wikipedia.org/wiki/A_Taste_of_Armageddon

Or Robocop:-

http://robocop.wikia.com/wiki/Enforcement_Droid_Series_209

I'm sure there are plenty of examples that show how stupid getting robots/computers doing this sort of thing is just a bad idea.

Re: Its a terrible idea in principle AND practice.

[Anonymous Coward]

Machines wouldn't be afraid. A lot of horrific things are done because of fear and the resultant clouded thinking.

We have passed the tipping point. The USA will be at perpetual war until we finally Balkanize or go bust - like Great Britain did.

When GB stopped being a World power, the standard of living for gfr average citizen went up.

We need to realize that the Middle East is a lost cause and the sooner those people kill each other off, the better the human race will be.

Re:Its a terrible idea in principle AND practice.

[Anonymous Coward]

We *morally* AND *practically* NEED humans in that decision loop, even if its just some guy in an air conditioned office and a VR headset.

Yes! We already have a ban on certain land mines since they kill without a human operator pulling the trigger, often harming innocent non-combatants.

It is easy to deploy weapons that kill someone without you being anywhere near and don't require you to have to make a judgement call. Those weapons kill indiscriminately. And even in war there are rules of engagement that tells us what we can do and cannot do, unless you want to be a war criminal. Non-combatants should not be targeted for example. Perhaps you could make better autonomous weapons, but we have yet to teach a machine ethics and compassion.

B... but the bad guys! They will use them.

Sure, there will be some people ignoring the international treaties, and deploying whatever means there is to kill as many as possible. That doesn't mean that we should give up. On the contrary! We should double down and fight this harder. I know war has changed a lot since the last two great wars, but if we want to rise above that, we should nip these things in their buds and send a message to anyone considering deploying such weapons that the rest of the world will NOT be sitting by idly and there will be repercussions for doing so.

I believe these kinds of weapons should be banned, as with chemical and biological warfare.

You speak of ethics and compassion while targeting certain types of weapons because they seem more evil than other deadly weapons.

If we humans were actually ethical and compassionate, we wouldn't be fragmenting this discussion. We would be questioning why we continue to engage in warmongering for profits sake. The justification of warfare should be the main target. It's fucking pointless to argue over who and what kills another human. We should be asking why.

Re:Its a terrible idea in principle AND practice.

[Anonymous Coward]

Sadly we have far too many soldirs that enjoy killing. An example would be our troops trying to invade Germany in WW2. Citizens outside of Germany may have been relocated and their homes filled with heavy weapons so that if troops walked past the home they could be gunned down or tanks and the like eliminated with heavy weapons. It became the rule of the hour to throw grenades into those homes and make certain that nobody was left alive in those homes. There were quite a few soldiers who enjoyed those killings and were eager to be the one who tossed the grenade into the homes knowing full well that innocent non-German civiliands were being killed. One veteran of that war told me that about 40% of the troops simply enjoyed killing people. With automated machines we would be able to enter such homes and send back quality pictures to eliminate the issue of troops passing by a home with unknown occupants. There are also situations in which a soldier may need to be rescued and lives are lost trying to rescue them. Machines can save a lot of lives and can be unusually cheap to operate compared to human soldiers. Also a machine is not unde emotional pressure in deciding whether a civilian is a target or simply a person doing normal things. A human has to lean towards pullin the trigger as one's meat is vital to life. A robot that gets shot can be repaired or replaced.

Re:Its a terrible idea in principle AND practice.

[Anonymous Coward]

You can consider land mines to be autonomous killing machines; they aren't in any way intelligent, but they make a simple explode/don't explode decision based simply on something pressing the contact plate.

Re:Its a terrible idea in principle AND practice.

[Anonymous Coward]

I disagree. The Marine in your scenario will have an inclination to shoot the possible suicide bomber - naturally, since it's his life that's on the line. But the robot can be programmed to be more generous, more lenient. In which case a few more innocents get spared, and a few more bombers get through ... but they only blow up robots, which are cheaper than human lives.

Re:Its a terrible idea in principle AND practice.

[MoralCharacter]

Since we're talking about the use of a robot capable of operating a check point, consider this. A robot can safely approach and inspect that suspicious woman and child for explosives because it doesn't need to fear for it's life like a human soldier does. It will perform that search because it is disposable and not free thinking. If woman is a suicide bomber then she kills herself and we lose a robot. We've eliminated the need to accidentally kill a civilian on suspicion alone. Activate another robot send the old one in for repair. Heck you don't even need to give the robot a gun. Have human soldiers watch over the checkpoint from a safe distance and have them behind protective cover - making the checkpoint even more difficult to take on because now soldiers don't have to stand out in the open if a firefight does break out. I'd honestly expect suicide bombing checkpoints to drop in popularity since you're no longer terrorizing your enemies soldiers. Same goes for roadside IEDs and the like. Possible IED? No judgement call needed, send in the bots. What good is an IED meant to maim and kill if it's target can be repaired and back in the fight far quicker than a human?

Re:Its a terrible idea in principle AND practice.

[hdyoung]

In your example, the robot doesn't care if it gets blown up. Therefore, you reason that the people in charge will not care much about civilian casualties. Nothing could be further from the truth. The chain of command will be EXTRA careful to avoid casualties like that. They'll program the robot to be very conservative about firing on anyone who might be a civilian. Robot gets blown up? "Meh, we're out 150 kilodollars. No biggie. Nothing is more effective at swelling the enemy ranks than a few well-publicized atrocities. Let's avoid those at all costs." This is already in play in US drones. From what I understand, there are strict limits to the number of civilian casualties that are acceptable in a drone operation. If there's any chance that number will be surpassed, the military will fly the drone home, or just nose-dive it into a patch of empty land and destroy it. Yes, any number of civilian deaths above zero is bad. But, from what I read, drone operations kill FAR fewer civilians than similar human-flown missions. Let's shoot for zero deaths in the long-run, but in the meantime how about using the option that's least-civilian-killy? Robots can be programmed to be far less trigger-happy than a fresh, jittery 19 year old marine or a 34 year old veteran with several tours of duty under his belt who just wants to make it back alive.

Re:Its a terrible idea in principle AND practice.

[Anonymous Coward]

bloody thing go robocop on a civilian population.>>>

  At least Robocop was still using his brain (augmented cybernetically, but his thoughts was
still his, his morals were still intact, and he
could even override or outright destroy the inbuilt systems that tried to keep him like puppet controlled OCP). No, these things are more like Terminators, only with far less AI.

  Yes, this is an incredibly stupid idea, and if
they continue with this horrific plan, we better
start preparing the war crimes tribunals in
advance, because a lot of top brass WILL need
to face some heavy (mass)murder charges.

Re:Its a terrible idea in principle AND practice.

[HiThere]

The problem is, it's stupid and insanely dangerous, but the danger is a long term danger, and the profit is immediate. And there's LOTS of people who can make one with various degrees of sophistication. Being moral and holding out won't stop this, we need another answer...but that that could be I don't know.

There's already enough people saying about this book or that "That was supposed to be a warning, not a handbook!". And that argument hasn't changed anything. Yes, 1984 was supposed to be a warning... but short term profit (of various kinds) was what determined the flow of events.

Re:Its a terrible idea in principle AND practice.

[HiThere]

You raise a valid point, but the problem is it depends entirely on the chosen "rules of engagement". And various governments have caused me to doubt that they would be merciful.

Re:Its a terrible idea in principle AND practice.

[rsilvergun]

How the hell would we entrust such a monumental decision to a robot.

Easy: You just don't care. We already don't care about drone strikes. Every look up the number of civilian casualties in Iraq. Just the ones the US government acknowledges are enough to blow your mind. You'd think it'd make headlines. But it barely even registers.

See in war if you're not personally getting blown to bits and your family ain't then it's easy peasy to just ignore it.

Re:Its a terrible idea in principle AND practice.

[Anonymous Coward]

I'm sure there are plenty of examples that show how stupid getting robots/computers doing this sort of thing is just a bad idea.

If we can cite fiction as "examples" then I would remind you that one of the things that keeps killbots from getting too dangerous, is their pre-set kill limit. Just throw wave after wave of men at them, and they'll eventually shut down.

Re:Its a terrible idea in principle AND practice.

[Anonymous Coward]

In that scenario, the robot moves forward and addresses the woman directly. "Is there a problem, ma'am?" On approach, it scans for suspicious electronic or chemical signatures emanating from her or her clothing. It sends the data to an expert system, or a human analyst, who can ask followup questions via radio link.

Worst case, she blows herself up and takes out the robot and whoever else is standing near her. Not the rest of the platoon.

Asking a human to do that would put them in a very high pressure situation, where mistakes are likely to happen. The robot, on the other hand, doesn't know the meaning of "pressure". It can handle it.

Re:Its a terrible idea in principle AND practice.

[Waccoon]

machines should serve us, NEVER the other way around.

I'm not afraid of whether we can keep autonomous machines on a leash. The question is, who holds the leash?

Re:Its a terrible idea in principle AND practice.

[Anonymous Coward]

This video demonstrates some killer robots:
Slaughterbots: https://www.youtube.com/watch?...

Would it be inappropriate to point out...

[Anonymous Coward]

that I for one welcome our new robot overlords? There are lots of human beings killing each other indiscriminately these days, so if we can just get robots to be the killers and the killed it might save some humans the bother, and some their lives.

Trump and Stormy Daniels (porno star), PSA

[Anonymous Coward]

"What about Stormy Daniels?" is now the answer for anyone who asks about Bill Clinton and Monika Lewinsky.

Rumor has it there's a $100K reward for the video.

No qualms about killing a bot

[darthsilun]

If I was in a gunfight I might think twice about killing another human. But a bot? No hesitation whatsoever. I'd blast the motherfucker.

Sending bots just sounds like an expensive way to flush money down the toilet.

Re:No qualms about killing a bot

[Dutch Gun]

If I was in a gunfight I might think twice about killing another human. But a bot? No hesitation whatsoever.

I'm pretty sure the bot feels nothing about the prospect of killing you as well.

Re:No qualms about killing a bot

[Anonymous Coward]

If I was in a gunfight I might think twice about killing another human. But a bot? No hesitation whatsoever.

I'm pretty sure the bot feels nothing about the prospect of killing you as well.

Which is what makes autonomous weapons truly dangerous; They won't blink at an order to commit a war crime or exterminate the rioting, starving, unemployed serfs.

Re:No qualms about killing a bot

[apoc.famine]

But I'm not sure if that's better or worse than humans, to be honest.

With a human, they might be told to exterminate your village, but they might not be able to do it. They also might be told to keep casualties to a minimum, and "look out, snipers!" and now they've got the rationale for exterminating your village.

The robots are either set to exterminate the village or to select targets carefully. You get what you get, and there is a lot less uncertainty. Whether or not that's worse than sending scared/angry/psychotic humans, I honestly don't know. I think it's only worse if they're set to exterminate a majority of the time.

If the purpose is to kill ...

[Anonymous Coward]

... even if the technology is far from mature, as long as the bot / machine can kill, they will be deployed

https://pbs.twimg.com/media/DTLaLWbXkAAqQ2M.jpg

These relatively primitive wooden planes had been used to attack Russian bases in Syria

Remember the good old days of warfare?

[Anonymous Coward]

Back when it wasn't considered sporting to target the officers, because they were nobility?

Until someone figured out that if you cut off the head – figuratively – then the body dies; shooting your enemy's officers became the first objective.

Bots as "team mates", perimeter sentries, etc. ? If it were me, I'd take out the bots first. Once they run out of bots to carry shit and do sentry duty and the grunts have to do it, then we're back at square one again.

Military spinn doctor

[Anonymous Coward]

Trying to mud the waters and blur the lines.

It does not matter if the criminals are using anonymized autonomous weapons. We should >>NEVER anonymize our kills/murders.

It is simple: Every single murder/kill should be decided by a human being.
And not by just having someone that always clicks OK.

When we have the technology, every single murder/kill should be audited and logged.
We could go that way instead of making anonymous weapons.

Re: Military spinn doctor

[Anonymous Coward]

There's a blockchain for that. I think.

Recommended watch: Slaughterbots

[Anonymous Coward]

Slaughterbots: https://www.youtube.com/watch?v=9CO6M2HsoIA&t=

Re:Recommended watch: Slaughterbots

[AC-x]

Also that Black Mirror episode

Re:Recommended watch: Slaughterbots

[GameboyRMH]

Wasn't sure if you'd link to that one or this one.

They should only be deployed against muslims

[Anonymous Coward]

They should only be deployed against muslims. They even up the playing field, because muslims don't value life (even their own) and will send suicide bombers. In fact they love death, so we will be doing them a favour.

remote island

[sad_]

let hem battle it out on a deserted remote island, whoever's AI robot army is still standing in the end wins.
ofcourse, then the question becomes, why not just run a computer simultation instead.

Re:remote island

[geekmux]

let hem battle it out on a deserted remote island, whoever's AI robot army is still standing in the end wins.

General Zaroff and the BattleBots? That's one hell of a mash-up. Cool band name.

ofcourse, then the question becomes, why not just run a computer simultation instead.

Telling the kids it's just a game will be the prescription to prevent PTSD. Let's just hope they don't talk to Ender Wiggin.

Re:remote island

[Fnord666]

Holy TOS Mr. Spock! No references to "A Taste of Armageddon" yet? What has this site come to?

Replace the police and SWAT teams by them

[johanw]

It will be a lot safer for the public: no more cops who claim they are "under pressure" or "affraid" so they had to shoot first. A robot can afford to shoot last, only when fired uppon first.

Re:Replace the police and SWAT teams by them

[Anonymous Coward]

You say that, but police/swat bots would likely be programmed to kill all humans on sight once they're let loose in an area.

You didn't think Skynet "broke its programming" did you?

Re:Replace the police and SWAT teams by them

[AHuxley]

Contain a no go part of the city.
Send in the robots with 24/7 aerial surveillance to counter all wifi, internet attempts to send out live streams, upload video clips.
Set up a pathway out the area for the citizens who want to surrender, be searched and moved to another city.
Move in the robots to try some pacification on the looters who did not take up the offer to be searched and exit the area.
Remove power, water, block networks and wait for the criminals and looters who stayed to try and escape. Robots will be waiting.
Try telling a robot about the 1st Amendment, reasonable articulable suspicion of a crime committed when the AI is set to pacification for that location.
The robot will not get stressed at the door considering who is on the other side. Robots just clear out each structure.

Re:Replace the police and SWAT teams by them

[Anonymous Coward]

Fear of the machine will override your logic, for most people.

But that fear will not stop humanity from, eventually, creating true synthetic intelligence. It is inevitable, and just a matter of time.

Once we blow the lid off our collective cognitive capacities, the very definition of what it means to be human will transform into something we cannot possibly imagine today.

And it will be amazing.

Re:Replace the police and SWAT teams by them

[Anonymous Coward]

Police and military situations are completely different. A police robot can afford to not shoot at all.

A autonomous machine performing police duties does not need to exhibit self-defensive behavior. It only needs to record and report. Any trigger pulling is done by remote operators, and typically only to disarm subjects or bombs.

Re:Replace the police and SWAT teams by them

[Agripa]

It will be a lot safer for the public: no more cops who claim they are "under pressure" or "affraid" so they had to shoot first. A robot can afford to shoot last, only when fired uppon first.

I am sure law enforcement will be responsible when using this technology.

https://en.wikipedia.org/wiki/...

"policy on autonomous weapons"

[0100010001010011]

As we all know, the enemy always follows the rules. Right on back to some dirty farmers hiding in the woods not following the British 'rules of war'.

The technology to build these things is not difficult. In the US a gun is easiest part of the puzzle. Toss in some OpenCV, webcam, a solenoid and you can have your own private sentry.

Re:"policy on autonomous weapons"

[Anonymous Coward]

The technology to build these things is not difficult. In the US a gun is easiest part of the puzzle. Toss in some OpenCV, webcam, a solenoid and you can have your own private sentry.

And your own private felony conviction when someone sets it off because booby traps are already illegal. Adding autonomous targeting software does not make them legal.

The weapons of war

[houghi]

War has and will always be over either control of people of control over what people own. Both can be done much more efficiently with data. The war is already going on and not just the Russians who influence the elections. (legally or illegally) Also by the American, by companies and by anybody else.

And as always, the people losing most, be it their money or their freedom or both, are the common people.

Re:The weapons of war

[Baron_Yam]

War is about control, yes. But amazingly, no matter how great your nuclear arsenal... you can still bash peoples' heads in with a rock.

Information-based warfare does not preclude physical violence. And if a group with the capacity to wage physical conflict decides it is losing the more civilized digital conflict, it can always fall back on guns and bombs.

Re:The weapons of war

[houghi]

Sure you can, buy why would you (unless you are a psychopat). It is about cost analysis. Why would I bother to send in things that cost money (be it people or houses or whatever) If I get a cheaper result otherwise?

Does that mean there will never be any violence? No, that does not mean that. It is just a lot less.

War might become more common

[Anonymous Coward]

One of the main deterrents from engaging in war is the risk of losing human lives.
A lot of anti-war sentiment in the US comes from the needless sacrifice of our soldiers as well as the economic toll.
With less human lives at stake, the desire to not engage in war is weakened. War will become mostly an economic consideration (Who am I kidding it already is).
There will probably be a "keyboard warrior" effect. It is easy to act hostile from behind a computer screen because of the safety from consequence.

Another aspect to consider is what the rules are for machines killing people. When a human soldier is confronted with a child holding a gun, the child is killed in self-defense. If a robot (autonomous or remote controlled) were to confront that same child, could the robot kill the child and make the same argument? The argument would devolve into weighing the value of the child's life against the price tag of the robot. An easy solution to this problem would be to have at least one human soldier with every group of robots. That way, the robots can light the child up and argue it is to protect the single human accompanying the robots.

Captcha: mankind

By the way...

[Anonymous Coward]

Who makes our guidance software?

Individuals aren't the problem

[Anonymous Coward]

Well, no more than an individual building an armored bulldozer with flamethrowers in his garage is a problem.

Autonomous weapons are an issue not because John Doe might own a killbot, but because the CIA, LVMPD or Baltimore Police might own a fleet of them. It's violent gang like those, who see the average citizen as something to either use and dispose of or utterly crush underfoot, who are going to then deploy the things to keep their abuses safe from a desperate population.

Autonomous weapons are an issue because Comcast might decide it's not hated nearly enough, and any large organization will simply dilute the blame to nonexistence as for every other crime or treason they commit, or at worst chuck some bottom-rung scapegoat whenever they're caught performing complete atrocities. The complete and utter lack of consequences for almost anything which law-enforcement and large corporations do basically guarantees that not even having to worry about replacing and retraining the basic 'goons' will send them to new heights of pure evil.

Robocop 2

[Anonymous Coward]

... like robot swarming and cooperative autonomy ...

He talks about AI-assisted weapons destroying autonomous weapons. He's not discussing autonomous weapons destroying civilian/friendly participants. It seems to be a favoured blind-spot in US-led drone and cyber warfare at the moment: Alas, the government cannot guarantee these weapons will only be pointed at enemy participants. An unwanted scenario is guaranteed because an autonomous weapon must first decide where the enemy participants are.

Robocop 2 (1990) shows an autonomous weapon making the wrong decision. That should give any military leader, food for thought.

Outlawed

[Anonymous Coward]

They should be outlawed, the same way chemical weapons and biological weapons are outlawed. Period. That will help with non-proliferation although it will not be a simple fix. Absolutely nothing good can come from these. If you look at one of the few autonomous weapons we've had for a while you'll see how terrible an idea these are. LANDMINES. We've seen nothing but senseless carnage and those wars have been over for many years.

War is hell. War with AI weapons could be the Great Filter. This is the one that either a) Destroys us in some kind of Skynet hell, or b) Changes the balance of power between the haves and the have nots to the degree that casts most of us permanently into slavery or worse. Either way it's the end of Democracy, relative freedom and life as we know it.

Re:Outlawed

[Anonymous Coward]

I Agree,
With robots they always follow orders (the program) and one bad guy with permission will rule the country/world. Humans can and will probably rebel thus keeping a madman from controlling everything.

Furthermore if the software needs to updated or we need to send them the order, how do we secure that communication channel, one hacker could kill lots of people. Hacking autonomous cars to do terror attacks are a risk add mobile robots with weapon and such an attack will be way worse.

Sending fake orders to soldiers that makes no sense will probably be ignored, a robot just follows them.

A very good point

[Chrisq]

They even up the playing field, because muslims don't value life (even their own) and will send suicide bombers. .

Actually this is a very good point. Is it worse to build an unfeeling robot to fight, or to remove all human emotion and compassion from humans like Islam does with its vial belief system?

Re:A very good point

[ComputerInsultant]

You insensitive clod! My belief system is based around small containers, usually with a closure, used especially for liquids.

Drones 'kill models' subject to hacking.

[Anonymous Coward]

In A.I., you have the concept of training models to perform your categorization. This model determines whether you are a hostile enemy, or a friendly soldier.

What I am terribly afraid of, is the idea that a hacker uploads a new model to a drone. Imagine this: The army trains a robot to automatically kill its enemies via something like 200,000 hours of training. Uploads the model to the drone and it goes off and kills the enemy.

An enemy gets a hold of this drone somehow, and alters the model so that it just kills everyone. It could be replaced by something very, very dumb, especially if it is known where the 'friendly' base is. Fly (or drive) to certain GPS coordinates, and then just 'kill everything'.

Imagine a rogue Tank, or rogue F-35. What if the rogue F-35 had a nuclear bomb equipped?

The real question

[Anonymous Coward]

So an invading force comprising entirely of robots never loses the will to fight. Bad news for the poor countries that can't pay for killing gizmos but have to send their sons and daughters to the front. I guess it will be easier to sell a war to your populace when it's only money and not somebody's son.

Re:The real question

[HiThere]

The current version of this stuff is already so cheap it's being used by non-state actors. There was a drone assault on an army base recently. The attackers did not identify themselves.

This stuff is coming. It may be unfortunate (probably is), but it's cheap enough and easy enough that primitive versions are already in use. So far they all depend on remote control, but simple versions that don't are easy...they just aren't as flexible. About all you need to add is a GPS starting point, an inertial guidance system, and a target.

Re:The real question

[Anonymous Coward]

I doubt you can attack another country with DIY drones carrying IEDs. These are just homebrew remote controlled grenades at best. No real deterrent to any large organized force. I don't see any small country being able to compete with the level of sophistication and size of production that countries like the US can and will achieve in the near future. So the same problem stands and will stand; with automated killing drones big countries will be able to wage war with impunity while small countries will quickly lose the will to fight when suffering human losses.

What exactly that looks like in practice...

[AHuxley]

The US mil thinking is not that secret on how to win a war. The results wanted well understood.
The free fire zones over Vietnam https://en.wikipedia.org/wiki/...
The British response in the Second Boer War https://en.wikipedia.org/wiki/...
But with robots.

Frost 4tist

[Anonymous Coward]

the Goodwill on slashdot.org would take about 2 To them...then

Mines

[notea42]

Everyone seems to forget we've had fully autonomous lethal weapons for decades - they're called land mines and sea mines. All we've done is make them smarter and more mobile. Making them smarter should only help reduce the number of false-positive casualties. To some extent, the same basic rules apply to robots as minefields - the person culpable is the one who deploys them. We've just got more control now than we did before.

Re: Mines

[locketine]

That's a great comparison but anti personnel mines are illegal: https://www.un.org/disarmament...

Re: Mines

[RespekMyAthorati]

And of course, ISIS would never disobey the U.N.

Re: Mines

[locketine]

And of course, ISIS would never disobey the U.N.

If you're insinuating that respectable nations should copy the deplorable tactics of a terrorist organization to defeat them, then what separates the two? Should the US torture and behead its enemies too? I think we'd be better off researching ways to mitigate the actions of the terrorists rather than paving the way for them. For instance, there are probably dozens of ways to blind or otherwise incapacitate drones.

Not "if" but "when"

[gatkinso]

I really can't think of a reason why a military would not develop such weapons.

Missing the point

[Ol Olsoc]

If people aren't being killed, there is no point to war. When we have robots fighting robots, it is just a show.

Re:Missing the point

[Gilgaron]

While certainly utterly defeating an opposing force is a way to win a war, destroying infrastructure in order to break morale of the other army through lack of resources is both easier (can't hide a factory as easily as an infantry unit) and more effective.

Re:Missing the point

[david_thornley]

The point of war is to defeat the enemy. This can be done by physical or nonphysical means, including mostly bloodless. One of Napoleon's greatest non-battles was the non-battle around Ulm in 1805, in which he forced the surrender of an enemy army without serious fighting.

Now, all of these nonphysical means rely on physical force, so it's important to have the ability to kill enemies, but it doesn't necessarily have to be used.

We fear only the "Second Variety"

[nospam007]

Not the current crap.

Is remote control any better?

[Roger W Moore]

Autonomous killing machines are a frankly horrific idea.

Agreed. But the alternative, remote controlled killing machines, seems to be just as bad. We have already seen from leaked videos that soldiers given drones to pilot using a video feed seem to treat bombing people as some sort of fancy computer game.

Given the two options I am not sure which is worse. An emotionless killing machine that follows preset rules of when, and when not, to engage or an emotional human who follows no predetermined patterns but one so removed that they regard your life as much as they value the life of a video game character.

Re: Is remote control any better?

[locketine]

It's not really like video game to the drone operators: https://www.nbcnews.com/news/u...

You're quite correct that remote killing machines operated by humans are little better than fully autonomous ones.

Re:Is remote control any better?

[sg_oneill]

There was an interesting study a few years back that found Drone operators suffered wartime PTSD at nearly the same rate (ie pretty damn high) as combat soldiers, the implication being that the sort of adrenal escalation that is behind PTSD appears to happen just as much with people who kill remotely as those who actually hold the gun.

The problem, of course, is that its unlikely the drone operator is going to realise what a head fuck he's in for until he's actually killed. Though I assume the same also applies to a combat soldier, who graduates boot camp thinking its going to be one epic game of counter-strike, takes his first life and then collapses under the unbearable moral weight of what killing really means.

US already uses lethal autonomous machines

[starless]

Unlike like a growing number of countries, the US hasn't yet agreed to a ban on use of land mines.

These (simple) machines can automatically indiscriminately kill, with essentially no protection against civilian deaths, and
can remain active for many many years.

Re:US already uses lethal autonomous machines

[Anonymous Coward]

Yes, everyone seems to be fixated on walking or flying robots with guns blazing.

But when you talk about swarms, you should think about much smaller things. Smart bullets that can fly around corners and converge on a heat, sound, or radio source. Something like land mines or claymore mines that can crawl around to reposition itself or even stalk its prey. Or tiny drones which mix some of these features... a slow smart bullet that can find a perch, observe the area, sniff the air, and stalk. It could be built around a small shaped-charge munition that it just has to get close enough and point in the right direction before turning itself into an armor-piercing projectile.

Re:US already uses lethal autonomous machines

[HiThere]

Smart bullets wouldn't work. Not as anything self propelling. If you make something that small, you need to make it a target seeking poison injector. Sort of like a mechanical mosquito. Diphtheria toxin would be effective. It would be nice if it could really target seek, then you could target the spine at the base of the neck and use Novocaine to temporarily paralyze them. (I think that would work, but it might stop autonomic as well as voluntary muscles, which would kill rather than paralyze.)

What form is most effective depends on your purpose. Paralyzing would be much better is you wanted to kidnap someone for interrogation. Just be equipped to monitor their p52 brainwave during interrogation so you'll know when they're lying. (I'm not sure that reliably works, but I've seen reports that it's pretty good. Sorry, sadists, too much stress distorts the signals.)

The thing is, even if what I'm talking about is beyond doing at the moment, that doesn't mean it will be in 5 years. And it won't be expensive the way a nuclear arsenal is.

The great conspiracy

[Anonymous Coward]

The rich fear the poor. Now they're taking the last thing the poor have going for them: military service. Don't want them turning on their masters, after all.

Unethical by default.

[Anonymous Coward]

Any time someone is considering the automation of killing people, it's a clear sign that humanity has been abandoned in favor of convenience... which is atrocious behavior. No country should be killing people at such a high rate that it starts considering easier ways to do it. Killing is not supposed to be easy.

Backwards

[Actually, I do RTFA]

Good lord, you've made the best case for robots I've ever heard. A human being having to make a tough choice that can easily result in an innocent woman's death or his platoons, and said choice scaring him for life? Even if he makes it correctly? Holy hell man, protecting people from danger like that is why we build robots. Just offloading the decision to someone else can help prevent PTSD for the soldier on the ground. Robots can make of sensors at a range to get a better probability assessment it's a suicide bomber. Couple that with a robot that can approach the woman and get blown up and not mind (unlike a human soldier), therefore tweaking the cost of a false negative, and you have a far superior system.

It really doesn't matter

[gettin2old]

This cat is out of the bag. There's little point in debating the ethics of it. Once the technology exist, and it already does, it can and will be abused. No matter how many countries say "we won't do it" it will be done. We may as well have them because as sure as the the sun sets in the west someone else will.

Re:It really doesn't matter

[Anonymous Coward]

We may as well have them because as sure as the the sun sets in the west someone else will.

"We're all going to die someday, so I may as well shoot you now."

Fun how your logic leads to mindless violence isn't it? Then again, I guess you willfully ignore morals and ethics whenever it suits you. Newsflash: You're the type of person that necessitates that kind of thinking, and the type of person that keeps us all perpetually on the brink of the cliff. Grow up. Not everyone is out to get you, and having the damn things will only make committing heinous crimes that much more easier. You wanna kill someone? Have the balls to shoot them in person. If you can't do that, then you're just a coward who's grudge against the other person wasn't even worthy enough to confront them with in the first place.

Not much longer now

[Anonymous Coward]

The good news is that life is finite and I'm on the downhill side. Weeeeeee!

If I had magic powers.......

[Anonymous Coward]

...all of the high tech weapons systems will disappear,
and all combatants would have to go at each other
with swords, knives, and clubs, and everyone who
wants to fight a war, will have to be right there on the
battlefield and in harms way. Including presidents.

  I bet when people are facing the prospect of
getting slashed with a blade, which in some ways
can be much worse than getting shot, and dying
in the dirt, they won't
be so inclined to fight each other. Wars will still happen, but they would really have to think long
and hard about it.

False Positive

[Macdude]

Anyone who has installed an alarm system into their home and has set it off accidentally -- should understand the main issue with arming your alarm system.

chance

[Anonymous Coward]

The black swans are coming.

Thank you very much

[rsilvergun]

for that terrifying thought. I never thought about us automating our military. The US army is the world's biggest social program. The Military Industrial Complex was thought up as a way to keep the US economy going in the face of crazy wealth inequality. I knew driving jobs are going away. I suspect all but the highest level IT jobs will go some day and eventually even coding jobs. But I forgot about the army. Man, are we in for a rough time with this second (third?) industrial revolution...

Can I just pick neither?

[rsilvergun]

How about neither? How about we stop meddling in everybody's country. We've got 7 wars going on ( Afghanistan, Iraq, Pakistan, Somalia, Yemen, Libya and Syria) and we're working on #8 and 9 (DPRK & Iran). Meanwhile I'm up to my eyeballs in debt from college and the roads I drive on are falling apart.

War is a racket

[Anonymous Coward]

So said former Marine General Smedley Butler. It is an evil, profitable, crony racket conducted for the benefit of governments and arms dealers. Regardless of who is waging it, the bad guys always win.

Say NO to war.

Nope

[DarkRookie]

Nopenopenopenopenope

Military Logic

[Anonymous Coward]

"...In general, we should strive to keep humans involved in the lethal force decision-making process as much as is feasible."

OK, how much is feasible? Where is the line exactly, between enough and not enough human involvement? When faced with the pressures of war, how much investment do the warring parties have in keeping the human touch? Or will it all recede to automation in the face of costs, threats of being out-competed by the enemy, and the families back home who don't want their family members killed in combat?

The long-term future of war may eventually evolve to this.

https://en.wikipedia.org/wiki/A_Taste_of_Armageddon

  Or this:

https://www.youtube.com/watch?v=ih_l0vBISOE

forget about combat....

[cwatts]

forget about combat.... ...these will be fantastic for robbing banks!

hypothetically speaking, of course!

Needs more cow bell ...

[fygment]

... Come on baby, don't fear the robopocalypse
Baby take my hand, don't fear the robopocalypse
We'll be able to fly, don't fear the robopocalypse
Baby I'm your man

- New Oyster Cult

Re: Its a terrible idea in principle AND practice

[Anonymous Coward]

Um, that assumes the machine has an adequate AI and both human and machine had equivalent training. Neither condition is likely to be true.

My usual on the irony of autonomous weapons...

[Paul Fernhout]

http://www.pdfernhout.net/reco... "Military robots like drones are ironic because they are created essentially to force humans to work like robots in an industrialized social order. Why not just create industrial robots to do the work instead?
    Nuclear weapons are ironic because they are about using space age systems to fight over oil and land. Why not just use advanced materials as found in nuclear missiles to make renewable energy sources (like windmills or solar panels) to replace oil, or why not use rocketry to move into space by building space habitats for more land?
    Biological weapons like genetically-engineered plagues are ironic because they are about using advanced life-altering biotechnology to fight over which old-fashioned humans get to occupy the planet. Why not just use advanced biotech to let people pick their skin color, or to create living arkologies and agricultural abundance for everyone everywhere?
    These militaristic socio-economic ironies would be hilarious if they were not so deadly serious. ... Likewise, even United States three-letter agencies like the NSA and the CIA, as well as their foreign counterparts, are becoming ironic institutions in many ways. Despite probably having more computing power per square foot than any other place in the world, they seem not to have thought much about the implications of all that computer power and organized information to transform the world into a place of abundance for all. Cheap computing makes possible just about cheap everything else, as does the ability to make better designs through shared computing. ...
    There is a fundamental mismatch between 21st century reality and 20th century security thinking. Those "security" agencies are using those tools of abundance, cooperation, and sharing mainly from a mindset of scarcity, competition, and secrecy. Given the power of 21st century technology as an amplifier (including as weapons of mass destruction), a scarcity-based approach to using such technology ultimately is just making us all insecure. Such powerful technologies of abundance, designed, organized, and used from a mindset of scarcity could well ironically doom us all whether through military robots, nukes, plagues, propaganda, or whatever else... Or alternatively, as Bucky Fuller and others have suggested, we could use such technologies to build a world that is abundant and secure for all. ..."