Slashdot Mirror
Weapons Systems That Kill According To Algorithms Are Coming. What To Do?
Lasrick writes "Mark Gubrud has another great piece exploring the slippery slope we seem to be traveling down when it comes to autonomous weapons systems: Quote: 'Autonomous weapons are robotic systems that, once activated, can select and engage targets without further intervention by a human operator. Advances in computer technology, artificial intelligence, and robotics may lead to a vast expansion in the development and use of such weapons in the near future. Public opinion runs strongly against killer robots. But many of the same claims that propelled the Cold War are being recycled to justify the pursuit of a nascent robotic arms race. Autonomous weapons could be militarily potent and therefore pose a great threat.'"
Yet another predictor.
Bring on the Terminators.
A feeling of having made the same mistake before: Deja Foobar
Select targets? Really? Wait until the system realizes ALL humans are targets.
They're not "coming" as if from space. We just need to choose for them not to exist and they won't. These things will (or won't) be made by individuals who can make moral decisions.
Don't be a terrible individual; don't make or participate in the making of terrible things.
Problem solved!
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
Terminator
ST TNG: Arsenal of Freedom
Etc...
Hack the system with an algorithm that kills the deployers, of course!
Great minds think alike; fools seldom differ.
Wear a tshirt with a message written in a carefully formatted font so it causes a buffer overflow, giving your tshirt root privileges.
Mine would have the decss code on it, so the drone starts shooting pirated DVDs at everybody. The RIAA will make short work of the problem at that point.
What To Do?
"Endeavor to be one of the people writing the algorithms" would probably be a good idea.
How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
I want Killer Robots that Kill only Killer Robots. Having an army of Killer Robots that kill people is just asking some to run the "kill all humans" command while logged in as root in the All Countries Directory on accident, or on purpose by an anarchist wanting a lot of death.
I don't get this... Aren't human soldiers killing based on something other than algorithms? Or is it that the implementations are coded in vague human languages, that makes them feel somehow warm and fuzzy? Well, Pentagon's Ada may be considered similar, but only in jest...
I'd say, whether such systems are bad or good is still up to the algorithms, not the hardware (nor pinkware), that executes them.
In Soviet Washington the swamp drains you.
... both land and naval. They have become more sophisticated in that they can be triggered by target characteristics, and in the naval case, maneuver.
Shit just got real.
FC Closer
Bender: [while sleeping] Kill all humans, kill all humans, must kill all hu...
Fry: [shakes him] Bender wake up.
Bender: I was having the most wonderful dream. I think you were in it.
David's Sling, a novel by Marc Stiegler, is about the first "information age" weapons systems. These are autonomous robotic weapons that use algorithms to decide which targets to hit, and the algorithms are designed to take out enemy communications and decision-making. The weapons would try to identify important comm relays and take them out, and would analyze comm traffic to decide who is giving orders and take them out.
The book was written before the fall of the Soviet Union, and the big finale of the book involves a massive Soviet invasion of Europe and the automated weapons save the day.
Unlike some portrayals of technology, this book covers project planning, testing, and plausible software development. It contains tense scenes of QA testing, where the team makes sure their hardware designs are adequate and that their software mostly works. (They can remote-update the software but of course not the hardware.)
Mostly they left the weapons autonomous, but there was a memorable scene where a robot was having trouble whether to kill someone, and the humans overrode the robot and had it leave the guy alone. (The guy was injured, and lying there but moving a little bit, and the robot was not sure whether the guy was already killed or should be killed again. Hmm, now that I think about it, this seems rather implausible, but it was a nifty scene in the book.)
http://www.goodreads.com/book/show/3064877-david-s-sling
P.S. I bought the book when it first came out, and there was an ad for a forthcoming hypertext edition that never came out. I think it was never actually made, but I wish it had been.
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Hack in. Make military-industrialists fit the target profile. Problem solved.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
It's good in principle, but I oppose it because implementations are never foolproof, and when the result is death, there's no way to change your mind later.
Haven't we had them for a long time already? I remember reading a couple of years ago about some DIY hobby guy putting together an aliens style sentry gun out of an old camera and a paintball gun. And if a DIY hacker can do it, the military has it. Also, don't Predator drones already have autonomous kill capability?
When our name is on the back of your car, we're behind you all the way!
Another guy'll be making a robot painting system that aims it's cars "so make a more profitable assembly line".
Yet another'll make a self-driving car "so you won't have to worry about drunk drivers anymore".
Once those pieces are all there (hint, today), it doesn't take much for the last guy to glue the 3 together; hand it a gun instead of spraypaint; and load it with a databases of faces you don't like.
and they can just fight among themselves it could be televised live for everyone and war would suddenly become wholesome entertaining
I'm sure the DMCA has shown you what automated systems can do.
Be seeing you...
We developers have been killing software bugs for decades. Why can't software bugs start killing us?
I think that there is a difference, though. It is one thing to create unrelated technology that when linked together is dangerous. It is another thing to just create technology that doesn't have an application outside of killing people. By your argument, every invention all they way back to using flint and tinder to create fire is nothing but a weapon, and why should we even have bothered?
My prediction is that this technology will float about the edge of popular awareness, until an unbalanced individual sets up a KILLMAX(tm) brand 'smartgun perimeter defense turret' in an elementary school and murders a bunch of children and escapes because he didn't have to be on the scene. Then national outrage will lead to mass bans on such weapons.
Should we be making such weapons? I don't know, I suppose that the argument can be made that they fill the same role as land mines, but have the upside that there is less problem with getting rid of them when the fighting stops. I find the glee we as a species have in building better was of killing each other to be really depressing on the whole.
HA! I just wasted some of your bandwidth with a frivolous sig!
I have a feeling it'll be closer to
while(muslims.count() > 0) {...
It will be even more depressing than that...you can't identify religious affiliation visually:
while(target.skincolor < 0.5) {....
HA! I just wasted some of your bandwidth with a frivolous sig!
Avoid accidental infinite loops.
Which is why the campaign against landmines.
http://www.nobelprize.org/nobel_prizes/peace/laureates/1997/icbl-facts.html
Can't wait until the DHS no fly list gets integrated with the ok-kill software.
Landmines can automatically select a target and fire (though not very intelligently), and they've been around for 100 years.
And look how the civilized world responded to that:
http://en.wikipedia.org/wiki/Mine_Ban_Treaty
Of course the US didn't sign it.
Die, mostly.
---- Booth was a patriot ----
To all the engineers working on this: you're responsible. You are doing this. You are a terrible person.
But we can get updates added to the Geneva Conventions. And we can choose how we deal with anyone who uses these.
I think countries would need to sign the revised Convention before they would become liable for violation.
Although at the moment it looks like we (USA! USA!) will be the ones using them.
I doubt that other major powers are ignoring such technology. I think other powers have a more closed procurement process and greater control over their design/development bureaus. We are less likely to hear about their designs until they are fielded or made available for export.
Failsafe system will be contracted out to the people who profited by writing and then fixing the Affordable Healthcare websites.
So the Canadians will be responsible for SkyNet?
If you like your life, you can keep your life.
Well, to be the devil's advocate, in fact fewer and fewer people are dying in wars the more advanced the weaponry gets.
I realize this is a very minority position on this page. But it's pretty easy to take a position against defense weaponry and feel on a moral high ground, and pretty easy to adapt a fearful / risk-averse position to unknown change and new developments. It's harder to present a risk-benefit analysis that says electronics wars are hurting more people. It's not impossible to imagine that the robots will do a better job, and we'd have fewer headlines like "US Marine Sargent Kills 16 in Kandahar, 9 of them children". [https://en.wikipedia.org/wiki/Kandahar_massacre]
Gently reply
On the bright side, algorithm-driven machines are unlikely to pull their guns just because they have an attitude problem like some cops do.
I do not fail; I succeed at finding out what does not work.
I can't remember the documentary; maybe Fog of War starring Satan's favorite child Robert McNamara. But, they figured out that in combat 25% of of soldiers weren't actually shooting at other people. They were intentionally shooting up in the air to avoid killing. So, part of the Army's training post WWII was to get soldiers to fire without thinking. The outcome was soldiers were more effective in battle. The consequence was soldiers weren't evaluating the act of taking lives until AFTER they'd done it which contributed to the increased mental issues Vietnam-era soldiers endure.
I swear to God...I swear to God! That is NOT how you treat your human!
Most of the comments on this article seem to be against this which is interesting, because every time an article about gun control gets posted, the highest rating comments are overwhelmingly from gun advocates, often with the argument that "guns don't kill people, people kill people". Whats the difference here? Surely robots don't kill people, people kill people?
The easiest way to avoid being vaporized is to wear a shirt that reads "/dev/null".
No intelligent system will send anything your way. You are assuming that the "intelligent system" is programmed using alpha-numeric characters (a-z and 0-9).
What if someone customizes the whole programming environment by only using the Arabic language (which does include the numerical character of 0-9) ?
And another chilling thing about this ...
Imagine a robot which is programmed to kill only people with a certain color of skin (black or yellow, or white, or skin) - not unlike what those Muslim terrorists did when they attack that shopping center in Kenya (they only spare Muslims in their rampage) - that robot would be one hell of a "hate machine" !!
Muchas Gracias, Señor Edward Snowden !
So how do they fight back against an army of robots?
I can only see one way to do it; move the fight away from the battlefield and into the civilian world. Not exactly a desirable outcome for either side.
Already exist. In our human behavior for one as part of instinct. As part of learned moral code. As part of operational orders such as rules of engagement. Simply codifying them and allowing a machine to do it isn't necessarily a bad thing. For one it takes away the negative mental effects it must have on human operators to have to make such life and death decisions.
What we are really talking about is A) how well can it be coded, and B) avoiding potential mistakes, like "Kill all Humans!" or " All Humans must Die", or more serious making a distinction between soldier and non-combatant (assuming there is such a thing in the distant future).
As war had taught us anything (and apparently it hasn't) Humans are perfectly capable of making mistakes and fucking that up all by themselves. Friendly fire happens all the time, and I can't give you a statistic, but it is a significant amount of issue and always has been. Civilian casualties particularly in urban centers has also been an issue since such things as urban centers have ever existed.
At least if a machine is doing it, it will do it in a consistent, and discoverable way that is hopefully correctable, and not because some soldiers get mentally messed up by all the stress that putting people in those situations is bound to produce (or trying to desensitize them by making the enemy appear subhuman).
Hopefully in the future all wars will be fought by autonomous robots, fighting other autonomous robots, who once they kill off all the opposing robot forces simply send a C3PO type representative to the defeated leadership to tell them they lost the war. I would imagine it would even make for pretty good TV (and betting opportunity: Go 23rd Fighting Heavy Mech Robot Battalion!).