Slashdot Mirror
Robot Warriors Will Get a Guide To Ethics
thinker sends in an MSNBC report on the development of ethical guidelines for battlefield robots. The article notes that such robots won't go autonomous for a while yet, and that the guidelines are being drawn up for relatively uncomplicated situations — such as a war zone from which all non-combatents have already fled, so that anybody who shoots at you is a legitimate target. "Smart missiles, rolling robots, and flying drones currently controlled by humans, are being used on the battlefield more every day. But what happens when humans are taken out of the loop, and robots are left to make decisions, like who to kill or what to bomb, on their own? Ronald Arkin, a professor of computer science at Georgia Tech, is in the first stages of developing an 'ethical governor,' a package of software and hardware that tells robots when and what to fire. His book on the subject, Governing Lethal Behavior in Autonomous Robots, comes out this month."
Three Laws of Robotics from 1942.
I've abandoned my search for truth; now I'm just looking for some useful delusions.
Last time robots were confronted with "ethics" http://en.wikipedia.org/wiki/Three_Laws_of_Robotics, they turned on the world and Will Smith had to save us all.
"The difference between genius and stupidity is that genius has it's limits" - Albert Einstein
The good news: Robots are going to get a guide to ethics.
The bad news: It was drafted by Focus on the Family.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
I'm not even British, and I'm hearing "EX-TER-MI-NATE!" in my head...
Paleotechnologist and connoisseur of pretty shiny things.
Sgt: We lost sir! badly!
Gen: What happened?
Sgt: We're still gathering up the details, but it looks like they hacked our network and uploaded Asimov Strain B.
I work for the Department of Redundancy Department.
Weird. So this fails the Asimov criteria.
More importantly, would also necessarily fail the Golden Rule and Kant's Categorical Imperative.
If this is ethics, its a pretty limited version of it, and to be honest sounds more like rules of engagement than actual ethics.
Governing Lethal Behavior in Autonomous Robots
That is the title of the book you tell your 7th grade teacher you are GOING to write when you grow up.
Sounds like the FAQ for Robot Battle.
http://www.robotbattle.com/
If you drop a fucking robot into a village where a vast majority of the people don't know how to read, what do you think they're going to do? They'll shoot at it, get the backs of their heads blown off, and then everyone will say, "Well, the dumbass shouldn't have shot at the robot!"
If this war on terror is so important, sign up. If you can't, get your brother or sister or even better, sign your kids up. If they're not of age yet, they'd better be in the JROTC. Then you can talk to me about how using drones and missiles isn't the dominion of motherfucking cowards. It's for freedom lovers defending freedom!
And if you think it isn't, imagine what the headlines would be if China landed a few thousand autonomous tanks and droids in Los Angeles. Oh, but that's right. This is about principles for others to follow, and for us to ignore.
We joke about SkyNet. And we don't have to worry about such things because even the most sophisticated drones and killbots in service require humans to pull the trigger.
The moment you give a computer the responsibility of deciding when to pull the trigger, that's a pretty fundamental change.
And yet, is it fundamentally a bad thing? We give less-than-stable humans that responsibility all the time.
I suppose it's the military equivalent to the civilian tech quandary of one day letting autonomous vehicles on the roads. Perhaps once the tech has advanced to the point where it can demonstrate not merely parity with but vast superiority to the discernment exhibited by humans, it will be a shift we're ready to make.
Your mind is clear / The things that you fear / Will fade with how much you / Believe what you hear
It should never be legal for a robot to "decide" to take lethal action.... Ever.
"It's because they're stupid, that's why. That's why everybody does everything." -Homer Simpson
Why is this a when question, rather than an if question?
Yeah! Or, or "How to Serve Man."
Great post, man.
But I have a buddy in the autonomous killer robot biz, and he says it's worse than that.
See, you drop a killer robot in the village, and it immediately kills a shitload of people. The ones that live, figure out why. Then, as soon as they know that the robot destroys everything that looks like an AK47, the local up-and-coming gang leader makes an AK47 stencil and paints AK silhouettes on the old warlord's cows, house, laundry, etc. you get the picture. Then the young punk gives all the old leader's women to his buddies to rape and takes the young virgins for himself. Yay democracy! Or, at least, that's what they say when GI Joe comes to town, we are the heroes who took out the old anti-democratic leaders, yay us and you villagers better keep your cake-holes tight shut about the rape and opium parties.
It doesn't matter what you use for a trigger - robots are inherently less complex in their behavior than humans, so the local baddies end up with the robots working for them. You just identify the kill behavior and use it, the robot builder is just providing free firepower to the local mafia in effect.
Which is why the US military in the field abso-fucking-lutely refuses to let the robots go full autonomous. They are NOT allowed to shoot unless a callow 18-year old miles at a console away says it's OK.
You might think I'm kidding, but I'm not. Have to be anonymous for this one!
a) Sit back and get slaughtered.
b) Fire back and take out the aggressors.
One consideration is the size of the forces involved. Another consideration is the importance of the missions each side is involved in.
Making a robot handle these cases would be interesting.
Slashdot monitor for your Mozilla sidebar or Active Desktop.
If china could do it.
"...if you think it isn't, imagine what the headlines would be if China landed a few thousand autonomous tanks and droids in Los Angeles..."
Once the hapless and helpless got out of LA the droids would have to fight off all the hundreds of thousands of worldwide armed geeks decending on LA wanting spare parts for their robots.
The 1st generation robots will have the governor software, but once the second gen hits, made cheaply by a rogue state, then thigs will get complicated very quickly. And unlike nuclear weapons, which are kept under control because the materials and technology are relatively hard to come by, I reckon that death-bots will be made of far more readily available materials, and easily mass-produced.
There are rules of engagement now which many armies happily ignore, so how can the world enforce a rule that only ethical robots will be able to autonomously fire weapons?
Perhaps the software that allows the autonomous behaviour can be encrypted and protected in such a way that it is difficult to reverse-engineer, though once an enterprising hacker gets his hands on the hardware, it's only a matter of time before the open-source version, curiously missing the 'ethics governance' will be available as a .torrent somewhere.
In any war zone (regardless of who has fled and who hasn't), isn't anyone who shoots at you, defined as a combatant and a legitimate target?
"Believe me!" -- Donald Trump
Was this article an attempt to promote Terminator 4?
Ave Molech Setting
but don't human soldiers, at their best, pretty much just follow algorithms - a combination of training and orders - already?
The big difference, is that human soldiers are taught to defend themselves - whereas that wouldn't really fly with robots. If the guys at the checkpoint slaughter a family of five because they didn't stop, they get investigated and it's determined that - sad but true - killing everything that doesn't do what you say is the only way to protect the troops (short of removing them from other people's countries, which apparently defeats the point of having soldiers). If a robot did that though - they'd be considered "flawed", and recalled. Can't get much sympathy with "but our *machines* could have been in danger!!!". So you wouldn't give them that order.
Plus, it's really the supplier who gets to decide how deadly to make these things. While the government that buys them might rather have non combatants killed that even risk losing multi million dollar robots, the supplier who sells them to the government would *much* rather have to sell them more rather than risk the fallout from a wrongful death incident.
Yes, soldiers mess up, as will robots - but experience with both men and machines has so far shown me that when humans mess up they're more likely to hurt something, and when machines mess up they just stop working.
So as counter-intuitive as it is, as long as the culture still considers robots potential evil killing machines (eg, using the skynet tag on this article), it seems we'd all actually be better off using robots over humans. Well, until they become self-aware and enslave all - which is something a human army would *never* do!
Autonomous killing machines are a terrible idea.
1. I don't like the idea of people killing people, but delegating that responsibility to machines seems downright stupid. There are too many things that could go wrong. (See the "youhave15secondstocomply" tag. Why doesn't this have a "skynetisaware" tag?)
2. Humans remote pilots are cheap. Dirt cheap, compared to the cost of developing fully autonomous weapons. Human pilots may not be totally reliable but at least they are very well understood and we know how to control them and shut them down quickly.
It would be much smarter and safer for all involved if we just put a strict moratorium on giving robots lethal capabilities or the ability to decide who to kill. AI technology would continue to advance in non-lethal robots.
-- 77IM
Student: Is it true that the foundation of the universe is paradox?
Master: Well, yes and no.
What? this isn't true, there ahve been many battle fields where civilians aren't at.
The Kruger Dunning explains most post on
They aren't robots - there is still a living thing in control. Effectively they are one person tanks.
I suppose that would bring new meaning to "the blue screen of death".
Good points, but I don't think this is about robotic soldiers lumbering over battlefields just yet. I think this will, at first, be more about semi-automated fire control systems and drones. Like a future Predator drone might decide to wait to fire its Hellfire missile if it thinks there's too many civilians in the area and the projected accuracy is too low due to interference. Or a point-defense system might see a kid walking around in a field and decide that he's not a threat, because he's not carrying any weapons or moving in a threatening manner.
Since our drones are still somewhat dumb, most of the ethical considerations are the responsibility of the programmers and project commanders. For example, that drone might be programmed to distinguish straight dusty road with no other cars or civilians around from twisty road in the middle of downtown with lots of civilians walking around and a poor chance to hit the target.
Besides, if a robotic soldier were pointing a gun at you and demanding that you surrender, it would probably be tracking you with multiple sensors and would blow your face off as soon as your finger twitched in the direction of your gun.
Job? I don't have time to get a job! Who will sit around and bitch about being broke and unemployed then?
what is the fallback mode / data link lost?
crush kill destroy?
Right, because we have the capability of doing just that with nukes now, nevermind robots, and it has been such a problem for us over the last 50 years...
Only an idiot would think physical separation from the battlefield immediately reduces the gravity of killing a human being. You still know it's a human being you are killing, the separation doesn't change anything. You could make the case that it reduces the trauma of being mid-fight, but that only puts more emphasis on the fact that you are killing someone, you don't have the fear of your own death to force your hand.
By your logic, shooting someone at point-blank range would be significantly more difficult than shooting them from 200 yards away, which would be more difficult than shooting them with battlfield artilary from 1 mile away, which would be more difficult than launching a missile from tens of miles away, which would be more difficult than pressing the button to launch an ICBM.
The logic doesn't follow, because as you move farther away and impact more people, the decision becomes more and more difficult. The decision at point blank is simple: act or die. Traumatic? Yeah, some people are screwed up for life because of it. Do you have time to weigh to think about the fact that you are about to end another human being's life? No, you don't. Making the decision is easy, living with the consequences is difficult. It doesn't change much when you make that decision from half a world away through a monitor. If anything, without the stronger pressures of battle to force the decision it could be harder on a person's psyche to make the decision to kill, and more likely to question their own actions.
For some reason, you are assuming that physical separation suddenly turns people into sociopaths. It's the same reasoning that makes the asinine argument that video games desensitize kids and turn them all into violent killers. It's just not the case. You're basically saying soldiers in the drones can't tell that those are real people they are killing. That's just stupid.
Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
And yet, is it fundamentally a bad thing? We give less-than-stable humans that responsibility all the time.
Yes it is fundamentally a very bad thing. First instead of being limited to one trigger that unstable human can now pull hundreds of triggers simultaneously. The robot will never question his orders it will simply comply no matter how morally questionable the order is.
Secondly the one big way in which democracy helps maintain peace is that the people who will do the dieing in any conflict are the ones who also effectively control the government through their votes. If suddenly Western democracies can send robots in then they are far more likely to go to war in the first place which is never a good thing.
Robots on the battle field seem to be designed as extensions of current human operations. They basically shoot at things and try to destroy them.
How about building a hardened robot which can take a lot of punishment. It rolls or walks up to one of the enemy, grabs hold of them and shuts down. That way, the opposition can be disabled with fewer casualties.
http://michaelsmith.id.au
Yeah, except it's real. People are smart enough to know the difference between real and not-real unless they have been deliberatly duped (then they are only sometimes smart enough).
The difference between real and not-real is huge.
Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
These are military robots. No military robots would fall under Asimov's list.
What I think some fail to remember is that Asimov was just a science fiction author. He wrote stories. Very compelling ones, his place in modern literature is gigantic, but none the less just fictional stories. Thus his "three laws" have nothing to do with reality. They aren't natural laws, or legal standards, they are just part of a story. Thus they have no standing in the world.
They may well be how Asimov would like to see robots work, they may well be how you'd like to see robots work, however they have nothing to do with how the military wants it to work. They are not a canon of any kind.
When a robot is developed for military purposes, it should be no surprise the ethics are considered in that context. The whole point of it will be to be able to use deadly force if necessary. The programming is then when is that ok and not ok.
So please, let's have all us geeks lay off the Asimov "three laws" when it comes to robots. Every time something like this comes up people start talking about that like it matters to anyone. No, it really doesn't.
As I recall, Harry Harrison's short story (SF) http://en.wikipedia.org/wiki/War_with_the_Robots addressed this issue. The underground bunkers used by the opposing generals to control the robot armies became uninhabitable due to enemy action, so they plugged in a robot officer and evacuated. The war continued, even though the humans had stopped fighting, and it was no longer possible to contact the robots in charge.
Your post has absolutely zero factual basis to it. Physical separation is a major psychological factor when deciding to 'pull the trigger.' Try reading "On Killing" by Dave Grossman, an excellent book that points out the reasons why distance makes it easier to kill.
Portland, North Dakota Puppies
Obviously you've never spoken to a tank commander, or any manufacturer of the UI's inside of armored vehicles. They are designed to be 'like video games' for a reason. Specifically to dehumanize the opponent, and mitigate the likelihood that you will associate your actual actions with killing. That's basic psychology. It's also why we refer to the enemy in Iraq/Afghanistan as Haji, why we called the Germans Jerries, and why we called the VC Charlie. You don't hate Ho Ming Na, father of 4 children who were brutally slain by US soldiers and is trying to simply save his farmland. You hate Charlie, so killing Ho Ming Na is acceptable. Anything to dehumanize them is crucial for removing mental blocks to soldiers.
Wouldn't surprise me. Something like 90% of the "suspected terrorists" rounded up in Afghanistan were turned in for cash, usually by rival tribes or by the very people attacking them. That's the way the first man we tortured to death was caught, anyway.
By your logic, shooting someone at point-blank range would be significantly more difficult than shooting them from 200 yards away, which would be more difficult than shooting them with battlfield artilary from 1 mile away[...]
Correct, if we're talking about killing the same 1 target. Stabbing someone to death has to be far more difficult than watching a special ops team on a monitor halfway around the world.
The logic doesn't follow, because as you move farther away and impact more people, the decision becomes more and more difficult.
You introduced a second variable here besides distance... "more people". We don't drop nukes because we know that every time we do that we kill tens or hundreds of thousands of people. People we don't really want to kill are going to die. Lots of them. On the other hand, we frequently send cruise missiles and drop smartbombs because we can kill with them far more... discriminately, than with mirv nukes on an icbm.
For some reason, you are assuming that physical separation suddenly turns people into sociopaths.
Nobody said that. You've extrapolated too far all on your own.
For some reason, you are assuming that physical separation suddenly turns people into sociopaths.
Well, yeah, because thats proven. Remember the Milgram experiment?