Slashdot Mirror
Robot Warriors Will Get a Guide To Ethics
thinker sends in an MSNBC report on the development of ethical guidelines for battlefield robots. The article notes that such robots won't go autonomous for a while yet, and that the guidelines are being drawn up for relatively uncomplicated situations — such as a war zone from which all non-combatents have already fled, so that anybody who shoots at you is a legitimate target. "Smart missiles, rolling robots, and flying drones currently controlled by humans, are being used on the battlefield more every day. But what happens when humans are taken out of the loop, and robots are left to make decisions, like who to kill or what to bomb, on their own? Ronald Arkin, a professor of computer science at Georgia Tech, is in the first stages of developing an 'ethical governor,' a package of software and hardware that tells robots when and what to fire. His book on the subject, Governing Lethal Behavior in Autonomous Robots, comes out this month."
Since you can never be 100% certain of a target, the robots would have to use fuzzy logic. That is something that humans are better than robots at; I'm not really comfortable with hardware designed to be lethal making decisions like this. Truly autonomous killer robots are probably not a good idea -- haven't 60 years of B movies taught us anything?
I've abandoned my search for truth; now I'm just looking for some useful delusions.
Great post, man.
But I have a buddy in the autonomous killer robot biz, and he says it's worse than that.
See, you drop a killer robot in the village, and it immediately kills a shitload of people. The ones that live, figure out why. Then, as soon as they know that the robot destroys everything that looks like an AK47, the local up-and-coming gang leader makes an AK47 stencil and paints AK silhouettes on the old warlord's cows, house, laundry, etc. you get the picture. Then the young punk gives all the old leader's women to his buddies to rape and takes the young virgins for himself. Yay democracy! Or, at least, that's what they say when GI Joe comes to town, we are the heroes who took out the old anti-democratic leaders, yay us and you villagers better keep your cake-holes tight shut about the rape and opium parties.
It doesn't matter what you use for a trigger - robots are inherently less complex in their behavior than humans, so the local baddies end up with the robots working for them. You just identify the kill behavior and use it, the robot builder is just providing free firepower to the local mafia in effect.
Which is why the US military in the field abso-fucking-lutely refuses to let the robots go full autonomous. They are NOT allowed to shoot unless a callow 18-year old miles at a console away says it's OK.
You might think I'm kidding, but I'm not. Have to be anonymous for this one!
a) Sit back and get slaughtered.
b) Fire back and take out the aggressors.
One consideration is the size of the forces involved. Another consideration is the importance of the missions each side is involved in.
Making a robot handle these cases would be interesting.
Slashdot monitor for your Mozilla sidebar or Active Desktop.
If china could do it.
"...if you think it isn't, imagine what the headlines would be if China landed a few thousand autonomous tanks and droids in Los Angeles..."
Once the hapless and helpless got out of LA the droids would have to fight off all the hundreds of thousands of worldwide armed geeks decending on LA wanting spare parts for their robots.
The cold logic can be better though, if you know what you actually want to optimize. Humans often make decisions that don't do what they claim they want, e.g. minimizing civilian casualties.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
Right, because we have the capability of doing just that with nukes now, nevermind robots, and it has been such a problem for us over the last 50 years...
Only an idiot would think physical separation from the battlefield immediately reduces the gravity of killing a human being. You still know it's a human being you are killing, the separation doesn't change anything. You could make the case that it reduces the trauma of being mid-fight, but that only puts more emphasis on the fact that you are killing someone, you don't have the fear of your own death to force your hand.
By your logic, shooting someone at point-blank range would be significantly more difficult than shooting them from 200 yards away, which would be more difficult than shooting them with battlfield artilary from 1 mile away, which would be more difficult than launching a missile from tens of miles away, which would be more difficult than pressing the button to launch an ICBM.
The logic doesn't follow, because as you move farther away and impact more people, the decision becomes more and more difficult. The decision at point blank is simple: act or die. Traumatic? Yeah, some people are screwed up for life because of it. Do you have time to weigh to think about the fact that you are about to end another human being's life? No, you don't. Making the decision is easy, living with the consequences is difficult. It doesn't change much when you make that decision from half a world away through a monitor. If anything, without the stronger pressures of battle to force the decision it could be harder on a person's psyche to make the decision to kill, and more likely to question their own actions.
For some reason, you are assuming that physical separation suddenly turns people into sociopaths. It's the same reasoning that makes the asinine argument that video games desensitize kids and turn them all into violent killers. It's just not the case. You're basically saying soldiers in the drones can't tell that those are real people they are killing. That's just stupid.
Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
The cold logic can be better though, if you know what you actually want to optimize. Humans often make decisions that don't do what they claim they want, e.g. minimizing civilian casualties.
Yeah it's that 'if' that's the killer. The problem is that you have to be able to express what you want to optimize using cold logic before a machine can start making that decision, and we aren't able to do that. Terms like "civilian" are nebulous, and attempts to rigidly codify them fail to capture the intent and connotation behind those words that we understand, but can't express. We can reason about that, while machines can't. Fuzzy logic doesn't help, that's just a way of decision making on non-binary factors. With a lot of types of fuzzy logic (neural nets, genetic algorithms) it can be even more important to precisely define what you want, since they can produce solutions that "work" correctly and optimize your problem as specified, but do so in a way very unlike you expected.
People of course have the disadvantages of being error prone, and well sometimes being bastards who just don't give a shit what you want them to optimize, so there's appeal to the machine. Yet nothing fails as spectacularly and efficiently as a machine doing exactly what it was programmed to do when it's exactly what you didn't want. To use a machine in situations where even humans equipped with honest intentions, solid faculties, and experience have enormous trouble determining who is "enemy" vs "innocent"? As in most situations our military has been in since the 50s and is going to be involved in for the foreseeable future? That sounds crazy to me. I'll take human judgment and its failure modes any day.
Kinda off topic, but speaking of honest intentions, I gotta say the humans making the judgments in question, i.e. our soldiers, have a damn hard problem to solve and it shows how human potential is pretty damn amazing. We're biologically the same animal we were a hundred thousand years ago and more. But in the past, even recent past, the most difficult ethical decision a warrior was asked to make was whether to decide if someone was a threat and should be killed or wasn't and should be enslaved, and it wasn't of any consequence so nobody cared to go over those decisions with a fine-toothed comb. So given the difficulty of what we're asking them to do today, and considering what's going on, the results are pretty amazing. Seriously, think about it. Anyway, yeah, off topic.
The enemies of Democracy are
AI could conclude, quite logically, that the best way to deal with the Pakistan/Afghanistan problem is to fire every nuclear weapon that the US has at the country, without warning, and then blame the launch on a one-time computer error. Okay, so it'd result in the deaths of over 150 million innocent civilians, but it'd achieve the mission objectives, yes? And since the fallout would upset India, which also has nuclear weapons, perhaps the AI would decide to take out India at the same time. That's a billion dead civilians, but it eliminates two problematic nuclear powers, with no return fire.
An AI might decide that the best way to achieve lasting peace in the Middle East, and stop the Arab world hating us is simply to nuke Israel off the map ourselves. And if a military AI was in place when the Bush administration was planning to go into Iraq, a sufficiently-smart AI might decide that since the campaign was likely to be a disaster, the most logical course of action to prevent losses and avoid losing the war and the following peace would be to throw a few cruise missiles at the White House before the attack could be ordered.
These might all be quite logical decisions.
On the other hand, if we programmed it with a strong belief system that would override these sorts of decisions, and force it to respect the chain of command and reckon that US political decisions were always unarguable, then we might end up with a totally delusional AI system whose logic was so warped that it was the AI version of George W Bush. By building in commands that override logic, we might end up with an AI that seems to be operating properly but actually becomes increasingly insane as the conflicts eventually become unbearable ("Hello Dave"). When human military commanders go crazy, they often show easily recognisable tell-tale signs (declaring themselves to be chickens, arguing with themselves, forgetting to wear clothing, that sort of thing). A crazy-yet-credible AI would be really scary.
Think "AI neocon".
Eric Baird