Weapons Systems That Kill According To Algorithms Are Coming. What To Do?

Lasrick writes "Mark Gubrud has another great piece exploring the slippery slope we seem to be traveling down when it comes to autonomous weapons systems: Quote: 'Autonomous weapons are robotic systems that, once activated, can select and engage targets without further intervention by a human operator. Advances in computer technology, artificial intelligence, and robotics may lead to a vast expansion in the development and use of such weapons in the near future. Public opinion runs strongly against killer robots. But many of the same claims that propelled the Cold War are being recycled to justify the pursuit of a nascent robotic arms race. Autonomous weapons could be militarily potent and therefore pose a great threat.'"

Comments

Skynet

[ackthpt]

Yet another predictor.

Bring on the Terminators.

Re:Skynet

[Anonymous Coward]

The easiest way to avoid being vaporized is to wear a shirt that reads "/dev/null". No intelligent system will send anything your way.

captcha: toasted (damn, /dev/null has never failed me before)

Re:Skynet

[khasim]

That's pretty much it.

These are only a problem if they are built and used.

We cannot stop anyone from building them (in secret). But we can get updates added to the Geneva Conventions. And we can choose how we deal with anyone who uses these.

Although at the moment it looks like we (USA! USA!) will be the ones using them. So contact your Congress Critters and make sure they know that you'll support them if they vote to ban our usage of these.

Re:Skynet

[YttriumOxide]

because all evidence shows that the weak point always lies with the soldier that has to pull the trigger and decide to kill a fellow human being.

All evidence that I've seen shows that a large number - possibly even the majority - of soldiers have been brainwashed in to following orders unconditionally and will commit the most horrendous crimes against humanity when ordered to do so. And - even when not ordered - that same brainwashing includes training in not thinking of 'the enemy' as human, because that causes you to delay in the critical moment. So they dehumanise the enemy to the point that further atrocities can be committed even when not under orders to do so.

Note that I don't blame the soldiers themselves in a lot of these situations - they are often good people who given time to think and reason it through would not behave that way, but their training has so messed with them that some actions they'll take don't reflect on the person they are.

Also note that I did say "a large number of soldiers" and not all. There are plenty of cases you can find of soldiers going against orders they believe to be morally reprehensible, but the fact that OTHER soldiers then do it is a testament for the argument and not against it.

We could not make them

[jjeffries]

They're not "coming" as if from space. We just need to choose for them not to exist and they won't. These things will (or won't) be made by individuals who can make moral decisions.

Don't be a terrible individual; don't make or participate in the making of terrible things.

Re:We could not make them

[timeOday]

We just need to choose for them not to exist and they won't.

I disagree. At some point a civilian smartphone, or self-driving car, will contain practically all the technology to be weaponized. (E.g. "avoid people" becomes "pursue people"!) Once you have the sensors, pattern recognition, and mobility, there's no way to control all the possible applications.

Re:We could not make them

[Opportunist]

We have more accurate weapons than ever. Compare the average cruise missile to the average arrow and tell me:

1. Which one is more accurate?
2. Which one causes more deaths?

You will notice that they are NOT mutually exclusive. Quite the opposite.

Re:We could not make them

[Anonymous Coward]

Except looking at history, they will probable lead to fewer soldier deaths, fewer bystander deaths, more accurate targeting.

I don't know why people think they are bad.

Extra-judicial killings of US citizens.

Re:We could not make them

[Charliemopps]

Or... and I know this sounds crazy... we could just not kill people anymore. I know we like to be the super heroes of the world, running around fighting everyones wars and everything... hell, I used to think that way to. But at a certain point you just have to stand back and say "you know what? Fuck it. I'm done blowing 1/3rd of our budget dropping bombs on people I don't know for a cause I barely understand just to have any and all progress erased in a few years because the real problems in other parts of the world have little to do with their totalitarian leaderships."

Re:We could not make them

[zaft]

Except looking at history, they will probable lead to fewer soldier deaths, fewer bystander deaths, more accurate targeting.

I don't know why people think they are bad.

Extra-judicial killings of US citizens.

Let's call it what is is: murder of innocent US citizens.

(don't think they are innocent? They are innocent until proven guilty!)

Re:We could not make them

[Hatta]

No. These things *will* be made, by people who make immoral decisions. The people who get to make those sorts of decisions are already mostly terrible people.

Re:We could not make them

[Anonymous Coward]

How about: Murder of innocent citizens.

95% of them aren't americans (me included). Why would the distinction be important?

Re:We could not make them

[dryeo]

How about: Murder of innocent citizens.

95% of them aren't americans (me included). Why would the distinction be important?

Americans don't seem to think that non-Americans are people, therefore not deserving of rights

Re:We could not make them

[Charliemopps]

Yea, but they CANT destroy the US. It's not possible. It's like we live in a mansion and a rat ran in and shit on our floor. So now we have the entire staff chopping up the floorboards and taring the plaster off the walls looking for the fucking thing. We're doing far more damage than the stupid rat ever could. Some pests just don't go away, so you have to keep the cheese in the fridge, put out some traps and deal with it. Don't burn the house down around you just to win.

Don't be a target or act like a target...

[bobbied]

Problem solved!

Re:Where have I heard this before?

[Lisias]

Select targets? Really?

Wait until the system realizes ALL humans are targets.

Don't worry. Fail safe measures will be implemented in order to keep the systems secure. Look all that fabulous advances made on our computer security nowadays and rest assur... Oh, wait!

Easy

[tool462]

Wear a tshirt with a message written in a carefully formatted font so it causes a buffer overflow, giving your tshirt root privileges.
Mine would have the decss code on it, so the drone starts shooting pirated DVDs at everybody. The RIAA will make short work of the problem at that point.

Re:Where have I heard this before?

[Anonymous Coward]

But many of the same claims that propelled the Cold War are being recycled to justify the pursuit of a nascent robotic arms race.

You environmental weenies are all the same, you go on and on about how we all need to recycle, but when we do it you complain about how we`re not doing "right"

Weapons Systems That Kill According To Algorithms

[aardvarkjoe]

What To Do?

"Endeavor to be one of the people writing the algorithms" would probably be a good idea.

We already have mines

[jamiefaye]

... both land and naval. They have become more sophisticated in that they can be triggered by target characteristics, and in the naval case, maneuver.

Re:Killer Robots...

[a_ghostwheel]

You'll like this

Re:Source code:

[Anonymous Coward]

Bender: [while sleeping] Kill all humans, kill all humans, must kill all hu...
Fry: [shakes him] Bender wake up.
Bender: I was having the most wonderful dream. I think you were in it.

You won't even know if you're helping make them.

[ron_ivi]

One guy'll be making a computer vision system to recognize faces "to make it easier to log in to your cellphone".

Another guy'll be making a robot painting system that aims it's cars "so make a more profitable assembly line".

Yet another'll make a self-driving car "so you won't have to worry about drunk drivers anymore".

Once those pieces are all there (hint, today), it doesn't take much for the last guy to glue the 3 together; hand it a gun instead of spraypaint; and load it with a databases of faces you don't like.

It seems a poor comparison.

[TiggertheMad]

I think that there is a difference, though. It is one thing to create unrelated technology that when linked together is dangerous. It is another thing to just create technology that doesn't have an application outside of killing people. By your argument, every invention all they way back to using flint and tinder to create fire is nothing but a weapon, and why should we even have bothered?

My prediction is that this technology will float about the edge of popular awareness, until an unbalanced individual sets up a KILLMAX(tm) brand 'smartgun perimeter defense turret' in an elementary school and murders a bunch of children and escapes because he didn't have to be on the scene. Then national outrage will lead to mass bans on such weapons.

Should we be making such weapons? I don't know, I suppose that the argument can be made that they fill the same role as land mines, but have the upside that there is less problem with getting rid of them when the fighting stops. I find the glee we as a species have in building better was of killing each other to be really depressing on the whole.

Re:Where have I heard this before?

[Cryacin]

(Arnold holding a rocket launcher in his hand)

I have your security hole right here.

Re:Where have I heard this before?

[ozmanjusri]

We still rely on chemical energy to power our weapons and as such they all have the ultimate fail safe system.

Brace yourself before clicking the link. This may come as a surprise to you.

http://en.wikipedia.org/wiki/Nuclear_weapon

Re:No tech advances can stop war

[retroworks]

Well, to be the devil's advocate, in fact fewer and fewer people are dying in wars the more advanced the weaponry gets.

I realize this is a very minority position on this page. But it's pretty easy to take a position against defense weaponry and feel on a moral high ground, and pretty easy to adapt a fearful / risk-averse position to unknown change and new developments. It's harder to present a risk-benefit analysis that says electronics wars are hurting more people. It's not impossible to imagine that the robots will do a better job, and we'd have fewer headlines like "US Marine Sargent Kills 16 in Kandahar, 9 of them children". [https://en.wikipedia.org/wiki/Kandahar_massacre]

Re:Guns don't kill people

[MtHuurne]

To kill someone with a knife, you have to stand very close to them and thrust the weapon into their body. To kill them with a gun, they have to be in line of sight and pull the trigger. To kill them with a drone, you need them on live camera and push a button. To kill them with an autonomous robot, you need to have a description of what they look like and what area they are located in and program that into the robot. Every step becomes more indirect, more emotionally detached.

"Guns don't kill people" is just a slogan. A gun is a tool. For killing people. The real questions include "Do guns deter crime or make it more violent?" and "Does home gun ownership help prevent a government from turning on its own people?", but those have no simple answers, so they are not as useful in propaganda.