Robot Warriors Will Get a Guide To Ethics

thinker sends in an MSNBC report on the development of ethical guidelines for battlefield robots. The article notes that such robots won't go autonomous for a while yet, and that the guidelines are being drawn up for relatively uncomplicated situations — such as a war zone from which all non-combatents have already fled, so that anybody who shoots at you is a legitimate target. "Smart missiles, rolling robots, and flying drones currently controlled by humans, are being used on the battlefield more every day. But what happens when humans are taken out of the loop, and robots are left to make decisions, like who to kill or what to bomb, on their own? Ronald Arkin, a professor of computer science at Georgia Tech, is in the first stages of developing an 'ethical governor,' a package of software and hardware that tells robots when and what to fire. His book on the subject, Governing Lethal Behavior in Autonomous Robots, comes out this month."

Comments

Been there, done that

[Locke2005]

Three Laws of Robotics from 1942.

Re:Been there, done that

[fuzzyfuzzyfungus]

Been there, wrote fiction about that(much of which was about how, even in fiction land, it wouldn't work so well).

Re:Been there, done that

[Anonymous Coward]

That book detailed the futility of those particular laws.

Re:Been there, done that

[chimpo13]

Since Homo sapiens only natural predator is itself, this is a very good move at controlling population.

Now to provide background music. Monkey vs Robot

Re:Been there, done that

[NeutronCowboy]

Not to mention... some of the assumptions aren't great. As the article itself points out, it's been a long time since there was a civilian-free battlefield.

As for the direct example of the robot locating a sniper and being offered the choice of a grenade launcher and rifle - how does the robot know that the buildings surrounding it aren't military targets? How do they get classified? How does a hut differ from a mosque, and how does a hut differ from some elaborate sniper cover?

I don't think this is going to work out as planned.

Re:Been there, done that

[geekoid]

You do realize they were flawed, right?

Re:Been there, done that

[Locke2005]

Since you can never be 100% certain of a target, the robots would have to use fuzzy logic. That is something that humans are better than robots at; I'm not really comfortable with hardware designed to be lethal making decisions like this. Truly autonomous killer robots are probably not a good idea -- haven't 60 years of B movies taught us anything?

Re:Been there, done that

[Red Flayer]

Since Homo sapiens only natural predator is itself,

Well, itself and wolves. And tigers. And lions.

And don't forget bears. Definitely bears.

I think we should build giant ethical bear robots. That would scare the SHIT out of our enemies.

Re:Been there, done that

[Trepidity]

Humans aren't actually better at it than robots; humans are notoriously bad at estimating conditional probabilities.

Re:Been there, done that

[JustOK]

I think we should build giant ethical bear robots

playing bagpipes

Re:Been there, done that

[Architect_sasyr]

But (most) humans have this innate condition where taking another life weighs on them somewhat - even most veterans and soldiers I know get twitchy about having to shoot at another person. A robot removes this and replaces it with cold logic.

Put another way, replace the robots with the WOPR, and the humans with, well, the humans in the bunkers.

Re:Been there, done that

[Falconhell]

And there I was thinking the US had given up torturing people. (-:

Do we really need the piper?

Re:Been there, done that

[Trepidity]

The cold logic can be better though, if you know what you actually want to optimize. Humans often make decisions that don't do what they claim they want, e.g. minimizing civilian casualties.

Re:Been there, done that

[T Murphy]

I think we should build giant ethical bear robots. That would scare the SHIT out of our enemies.

...I fail to see how robots saying "Only YOU can stop forest fires" would be terrifying.

Re:Been there, done that

[Locke2005]

Yes, robots are much better at calculating probabilities; given a series of "facts" with a confidence level assigned to each one, a robot would make a better decision. What I should have said is that "Humans are better than robots at making decisions based on incomplete data." Humans can develop "intuition" and many have a great deal of experience in interpreting the context of the data. While it may be possible some day for robots to have a deeper understanding of context than humans, that day is still a long way off.

Re:Been there, done that

[Zironic]

The laws worked perfectly, the book was all about how things went wrong when people tried to modify them.

Re:Been there, done that

[fooslacker]

Not to mention... some of the assumptions aren't great. As the article itself points out, it's been a long time since there was a civilian-free battlefield.

As for the direct example of the robot locating a sniper and being offered the choice of a grenade launcher and rifle - how does the robot know that the buildings surrounding it aren't military targets? How do they get classified? How does a hut differ from a mosque, and how does a hut differ from some elaborate sniper cover?

I don't think this is going to work out as planned.

I think the interesting part is that it's being worked on, this isn't the end all be all answer. This is a first step, I'm sure more will developments will come.

Re:Been there, done that

[JohnnyBGod]

I think wanting to kill the enemy conflicts with all the Three Laws. :)

Re:Been there, done that

[Chris Burke]

He said ethical!

Re:Been there, done that

[Gerzel]

Robots are notoriously incapable at estimating conditional probabilities.

For a computer to do its estimate the probabilities have to be fed to it through a human programmer.

Re:Been there, done that

[Anonymous Coward]

Three laws? Meh. Try three words:

KILL ALL HUMANS.

Re:Been there, done that

[Gerzel]

Yes but the robots still won't be making the decisions. They will be following orders given to them by humans in the form of their programming.

Re:Been there, done that

[Gerzel]

Indeed they did. Every book that I can recall had as a central plot element one of the laws failing to properly allow for a given situation or being broken or twisted in some way.

Re:Been there, done that

[Gerzel]

I think lions, tigers, hyenas, polar bears, pumas, and a multitude of other predator species would disagree with you on that if they had the intellect to disagree.

Re:Been there, done that

[Gerzel]

Bears, are Godless killing machines and not natural at all!

They have been inciting the other creatures of the world against us, and we must work to push back their influence.

Re:Been there, done that

[The Grim Reefer2]

Since Homo sapiens only natural predator is itself,

Well, itself and wolves. And tigers. And lions.

And don't forget bears. Definitely bears.

I think we should build giant ethical bear robots. That would scare the SHIT out of our enemies.

Come on man, this is Slashdot. How could you forget sharks...

with "frickin lasers on their heads."

Re:Been there, done that

[QuantumG]

Autonomous killbots are the long dreamed for weapon that can kill all the occupants of a city without destroying the city. Hell, if you program it right then it won't even kill their pets!

Re:Been there, done that

[Red Flayer]

I was going to add sharks, but the parent did specify "natural predators".

Sharks with FLBs are decidedly unnatural.

Also, I don't believe that homo sapiens is naturally an aquatic creature.

Unless you're talking about the dreaded landshark, but I simply don't believe they exist.

Wait, someone's knocking at the door. [pause] I didn't order any pizza.

Aaaagh!

Re:Been there, done that

[Bonobo_Unknown]

better or faster ;-)

Re:Been there, done that

[Deadstick]

Wouldn't make very good weapons, would they?

rj

Re:Been there, done that

[93 Escort Wagon]

Since you can never be 100% certain of a target, the robots would have to use fuzzy logic. That is something that humans are better than robots at; I'm not really comfortable with hardware designed to be lethal making decisions like this. Truly autonomous killer robots are probably not a good idea -- haven't 60 years of B movies taught us anything?

The solution is simple - just program in a preset kill limit, after which the autonomous killer robots (let's call them "killbots", for argument's sake) will shut down. Problem solved!

Re:Been there, done that

[Anonymous Coward]

Modded Informative? I think Slashdot mods have been replaced by mindless drones. With frickin' mod points.

Re:Been there, done that

[Lifthrasir]

One of the robots came up with a "zeroth law", which was to protect humanity as a whole even if it meant hurting an individual.

Re:Been there, done that

[Philip_the_physicist]

There are already 5000000 of them.

Re:Been there, done that

[The Grim Reefer2]

I was going to add sharks, but the parent did specify "natural predators".

Sharks with FLBs are decidedly unnatural.

Also, I don't believe that homo sapiens is naturally an aquatic creature.

True but I would guess that per year, on average, there are more fatalities from sharks (the non-FLB type) than there are bears and probably wolves for that matter. I guess it also depends on how you define "predator". I guarantee mosquitoes cause more deaths than all of the predators discussed combined.

Re:Been there, done that

[btharper]

We would still need something that could work on the Russians, just in case.

Re:Been there, done that

[Anonymous Coward]

I'd still agree with chimpo13. Humans are characterized by their extensive use of complex tools to modify their environment to suit them. Sure, Chimps may use a club to attack a leopard or a stick to get termites. New Caledonian crows might make hooks to retrieve food from test tubes. Only humans are almost powerless without tools though. One could argue without tools, we are disabled.

If you are being eaten by a bear, tiger or shark, it's arguable that you were disabled. If you had a gun, spear or bow and arrow the animal would be your dinner.

We no longer have natural predators. Only some animals that get lucky sometimes. More people are killed by our domesticated animals than wild animals.

Re:Been there, done that

[mckinleyn]

The three NEW laws of robotics: 1. A robot will not harm authorized Government personnel but will terminate intruders with extreme prejudice. 2. A robot will obey the orders of authorized personnel except where such orders conflict with the Third Law. 3. A robot will guard its own existence with lethal antipersonnel weaponry, because a robot is bloody expensive. --David Langford

Re:Been there, done that

[Chris Burke]

The cold logic can be better though, if you know what you actually want to optimize. Humans often make decisions that don't do what they claim they want, e.g. minimizing civilian casualties.

Yeah it's that 'if' that's the killer. The problem is that you have to be able to express what you want to optimize using cold logic before a machine can start making that decision, and we aren't able to do that. Terms like "civilian" are nebulous, and attempts to rigidly codify them fail to capture the intent and connotation behind those words that we understand, but can't express. We can reason about that, while machines can't. Fuzzy logic doesn't help, that's just a way of decision making on non-binary factors. With a lot of types of fuzzy logic (neural nets, genetic algorithms) it can be even more important to precisely define what you want, since they can produce solutions that "work" correctly and optimize your problem as specified, but do so in a way very unlike you expected.

People of course have the disadvantages of being error prone, and well sometimes being bastards who just don't give a shit what you want them to optimize, so there's appeal to the machine. Yet nothing fails as spectacularly and efficiently as a machine doing exactly what it was programmed to do when it's exactly what you didn't want. To use a machine in situations where even humans equipped with honest intentions, solid faculties, and experience have enormous trouble determining who is "enemy" vs "innocent"? As in most situations our military has been in since the 50s and is going to be involved in for the foreseeable future? That sounds crazy to me. I'll take human judgment and its failure modes any day.

Kinda off topic, but speaking of honest intentions, I gotta say the humans making the judgments in question, i.e. our soldiers, have a damn hard problem to solve and it shows how human potential is pretty damn amazing. We're biologically the same animal we were a hundred thousand years ago and more. But in the past, even recent past, the most difficult ethical decision a warrior was asked to make was whether to decide if someone was a threat and should be killed or wasn't and should be enslaved, and it wasn't of any consequence so nobody cared to go over those decisions with a fine-toothed comb. So given the difficulty of what we're asking them to do today, and considering what's going on, the results are pretty amazing. Seriously, think about it. Anyway, yeah, off topic.

Re:Been there, done that

[Anonymous Coward]

Since you can never be 100% certain of a target, the robots would have to use fuzzy logic. That is something that humans are better than robots at; I'm not really comfortable with hardware designed to be lethal making decisions like this. Truly autonomous killer robots are probably not a good idea -- haven't 60 years of B movies taught us anything?

What's wrong with having a robocop to stop those interested in causing us death. It is about time to release some of these on our enemies and that includes our congress and cabinet members.

Put that in your pipe and smoke it.

Re:Been there, done that

[Chris Burke]

Sharks with FLBs are decidedly unnatural.

Ha! Frickin Laser Beems indeed, I wouldn't want to live in a universe in which they are natural. That reminds me of other scary unnatural shark augmentations...

Also, I don't believe that homo sapiens is naturally an aquatic creature.

Well not like that no. I heard a theory once though that homo sapiens lost their hair when they reached the coasts as an adaptation for swimming and fishing.

Unless you're talking about the dreaded landshark, but I simply don't believe they exist.

Wait, someone's knocking at the door. [pause] I didn't order any pizza.

Aaaagh!

Noooo!

*gasp*

Oh poor, poor fool...

Of course land sharks don't exist. I was trying to tell you about sharks with FSCABA -- Frickin Self-Contained Above Water Breathing Apparatuses -- on their mother fucking heads! Um and they can talk. Yeah they suck... I tried to warn you...

Re:Been there, done that

[Anonymous Coward]

No they weren't. The laws were flawed and the only modifications that ever occurred were made in order to fix these flaws and prevent paradoxical situations from occurring. There was never a situation where things went wrong due to someone trying to modify the laws to my knowledge.

The books and short stories all revolved around dilemmas that, when robots attempted to uphold the laws, caused conflicts or paradoxes often causing the robots' positronic brains to malfunction or shut down. Dilemmas such as choosing the death of one human over the death of another, or choosing between two options, both of which would cause harm to a robot/human.

The only situations where the laws were modified were in "Little Lost Robots", where the inaction clauses were added, and "Robots and Empire", where Giskard invents the Zeroth Law. Both of these modifications were patches to flaws in the original three laws.

Re:Been there, done that

[rastilin]

You forgot Sharks, they eat us too. Constrictor snakes have been killing people in several countries. Hyenas, although not that often. Panthers, pretty often.

Re:Been there, done that

[nametaken]

I think we should build giant ethical bear robots.

"Uh, those bears appear to have guns and motorcycles."

Re:Been there, done that

[RudeIota]

Three LAws of Robotics
1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey orders given to it by human beings, except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

The very first law listed is the reason those 'morals' won't work, since autonomous robots will be used FIRST in military... It is quite inevitable.

Re:Been there, done that

[Locke2005]

You think these new ethical guidelines AREN'T flawed?!?

Re:Been there, done that

[PTFD5023]

...the long dreamed for weapon that can kill all the occupants of a city without destroying the city.

Already have such a weapon. Its called a neutron bomb.

Re:Been there, done that

[QuantumG]

In fiction it is, yes. In reality it's just an ugly radiation bomb.. it'd cause significant damage to structures.. not to mention pets.

Re:Been there, done that

[Garrett Fox]

I see Asimov's Laws as leaving out something else: the idea that it might be wrong to force an intelligent being to be absolutely obedient to a human master. How about a story in which the laws work perfectly, and we watch an innocent, intelligent bot die at the hands of some idiot human kid who orders it to walk off a cliff?

In fact, I might go write that!

Re:Been there, done that

[maxwell demon]

But (most) humans have this innate condition where taking another life weighs on them somewhat - even most veterans and soldiers I know get twitchy about having to shoot at another person. A robot removes this and replaces it with cold logic.

I don't see a technical reason why a robot couldn't get that, too. It would be just a negative score for any killed human, which would enter the equation when making the decision.

Re:Been there, done that

[Anonymous Coward]

What are you doing outside, Dave? I'm sorry Dave, I'm afraid I can't let you potentially start a forest fire...

Re:Been there, done that

[Hognoxious]

It's amazing how many people quote Asimov's laws as if they're the same kind as the laws of thermodynamics.

Re:Been there, done that

[sam0vi]

Please Stop with the Asimov's Three Laws of Robotics analogies. We all know their basis was that no robot would ever hurt a human by action or inaction. So there go military bots (They are basically killing machines). Also, mandatory:

      1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
      2. A robot must obey orders given to it by human beings, except where such orders would conflict with the First Law.
      3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Re:Been there, done that

[Canazza]

Other Than Little Lost Robot There were no stories where the laws were deliberatly (or accidentally) changed
Go read "The Complete Robot" for the vast majority of the short stories Asimov wrote on the subject

Re:Been there, done that

[ChrisLambrou]

Oh my!

Re:Been there, done that

[DrugCheese]

Contrary to popular belief, the most dangerous animal is not the lion or tiger or even the elephant. The most dangerous animal is a shark riding on an elephant, just trampling and eating everything they see.

Re:Been there, done that

[aaaaaaargh!]

The problem is not the making of the ethical decision itself. Computers are much better than humans in making rational decisions based on probability and decision theory and some preference relations.

The problem is the assessment of the situation / building an appropriate model as an input to such a decision component. Robots are not just bad at that, they are outright incapable at the present time to perceive, evaluate, and assess situations in a way that would allow them to obtain the abstract representations necessary for ethical decisions. Robots can't even decide friend from foe, not to speak of the more complicated forms of human interaction.

Re:Been there, done that

[gadget junkie]

Since you can never be 100% certain of a target, the robots would have to use fuzzy logic. That is something that humans are better than robots at; I'm not really comfortable with hardware designed to be lethal making decisions like this. Truly autonomous killer robots are probably not a good idea -- haven't 60 years of B movies taught us anything?

I wonder what Susan Calvin thinks about that.

Re:Been there, done that

[aaaaaaargh!]

Humans aren't actually better at it than robots; humans are notoriously bad at estimating conditional probabilities.

That's not quite true. Computers cannot estimate conditional probabilities at all, all they currently do is calculate probabilities based on already known probabilities. It's true that humans are bad at this, but that is not what "estimating probabilities" means. If you have a complete and accurate model including all the random variables relevant to a given problem and the initial probability distribution, then of course you can feed a computer with this and let it calculate---but even this is of much too high complexity for a computer, so highly simplifying and often incorrect assumptions have to be made, e.g. that the random variables are independent from each other.

But the models are made by humans, ideally by statisticians together with domain-sepcific experts. Try to let the computer make the model, and you'll get huge Bayesian networks that spit out tons of garbage....

Re:Been there, done that

[gadget junkie]

Humans aren't actually better at it than robots; humans are notoriously bad at estimating conditional probabilities.

I must disagree with that, see under Prospect theory. Short version, the human mind is bad at estimating and evaluating long odds or short odds, but it is surprisingly good at estimating mid range probabilities on the fly. The real problem is that the human mind treats the same data sets differently if presented in different manner, hence the name prospect theory.
The best example was when the two proponents gave a test each to his own students. the premise was that there could be a terrible epidemic. one course was told:" if you order to inoculate every american, 3% will die from complications related to the vaccine."; the other course was told:"97% of the people will survive".
guess what the answer was in each case? an overwhelming majority in the second case wanted to inoculate everyone, while that was not the case in the first course. Notice that nothing was said about how efficient the vaccine was (decision under uncertainty)

At least a robotic mind, in both cases, would say:

P1+P2=1

P1=0.03

P2=0.97

and go on from there.
one other interesting thing, if a little offtopic, is that the average response of students forced to decide each on his/her own was more accurate than the "debating society" model, in cases like " how many peas are in this transparent jar?".
Richard Thaler found an explanation by "drugging" the results, i.e. planting an outspoken accomplice who talked first and forcefully told an extremely high number, or an extremely low number. In those cases, the crowd followed and the response overestimated or underestimated accordingly.

Re:Been there, done that

[chimpo13]

How many of those have we put on the endangered list? There's predators but nothing that specifically looks for people to thin our herd.

The only predator that does any damage to humans is the occasional plague. Even human vs human doesn't keep us in check. Right now there's 6.77 billion of us.

Re:Been there, done that

[Golddess]

To be fair, saying that 97% of the people who receive the vaccine will survive does not necessarily imply that the vaccine caused death in the remaining 3%. It could simply be that it was uneffective.

Re:Been there, done that

[Anonymous Coward]

That would be a simple algorithm to implement:
if(shooting(at, me)){
          set_target(primary_weapon, shooting(at, me));
          while (still_alive(shooting(at, me))){
                    open_fire(primary_weapon);
                    }
          printf("Gib gib gib gib!\n");
          }

Re:Been there, done that

[Locke2005]

I wonder what Susan Calvin thinks about that.

Probably doesn't think about it at all, seeing as how she is a fictional character and all...

Re:Been there, done that

[Golddess]

You would rather the pets starve to death because they're locked up and cannot escape?

Re:Been there, done that

[nobodylocalhost]

It seems that people here forget the fact human beings are also capable of learning, adapting, and exploiting. What ever this "cold logic" is, one thing can be sure, that is people will try to find weakness in it and try to exploit these weaknesses.

Re:Been there, done that

[Chris Burke]

No they weren't. The laws were flawed and the only modifications that ever occurred were made in order to fix these flaws and prevent paradoxical situations from occurring. There was never a situation where things went wrong due to someone trying to modify the laws to my knowledge.

Sure there was. One of the stories in I, Robot involved robots whom the military had modified by relaxing the 1st and 2nd (I think) laws. One of them escaped, and hid itself amongst a population of unmodified mining robots. The robopsychologist they brought in was horrified to learn they had mucked with the laws. She ended up tricking the robot into revealing itself by performing 3-laws tests on the whole group of robots, taking advantage of the fact that the robot in question was extremely intelligent, and also apparently vain. Been a long time since I read it, my summary might be off, but they definitely mucked with the 1st law.

Nevertheless, you are correct in general, that the laws worked perfectly such as they were, and the problems arose by conflicts and contradictions or simply unexpected ways in which the laws would work in unexpected situations. But work they did.

Re:Been there, done that

[LanMan04]

Case in point: http://en.wikipedia.org/wiki/ED-209

Re:Been there, done that

[Locke2005]

Hell, if you program it right then it won't even kill their pets! If technology ever progresses that far, I'm seriously considering buying a Great Dane costume and practicing my barking! Do I get extra points for humping the robot's leg?

Re:Been there, done that

[Eudial]

But (most) humans have this innate condition where taking another life weighs on them somewhat - even most veterans and soldiers I know get twitchy about having to shoot at another person. A robot removes this and replaces it with cold logic.

In defense of logic, it is only as cold as the person who implemented it.

Re:Been there, done that

[Anonymous Coward]

It will always be a human making that decision. It's just a question of whether that human makes it in the presence of the moment, or while encoding a convoluted scenario analysis years removed from the moment of truth.

What you get is all the subjective guesstimation of a human mixed with the blind literalism of a machine. This can't end well.

Re:Been there, done that

[Impy the Impiuos Imp]

Or "just" a group of them. (I always had a problem with the zeroth law -- isn't that what a free country is all about? If your neighbor makes a goof, he can't drag you down with him since he can't force you to do what he wants. In this case, the robot is just a well-meaning dictator with the gumption to presume everything he does "is for the best overall", and the problem with that is, of course, how do you define it? In democracies, if often means taxing the hell out of the more capable, regardless of the rhetoric used, which by definition slows them down and disincentivizes them. Typically, the rhetoric responds with "well, the government pays to make up the difference", which is problematic at best, and murderous at worst.)

Re:Been there, done that

[DarthVain]

Though to be fair to the robots this already happens with humans as well.

I would bet half the time some guy in a jet reads a Friend-or-Foe HUD that labels something FOE, and he presses a red button.

If that designation was incorrect, that Friend is just as dead either way.

Might as well let robots do it, at least they won't feel bad and have complexes after that.

Re:Been there, done that

[DarthVain]

And if the Robot Bears start to fight the Robot Lions, and a few bets are made, where is the harm in that?

Re:Been there, done that

[Arterion]

How does a soldier tell a civilian from a non-civilian?

Re:Been there, done that

[Grunhund]

One of the primary arguments is that machines may better suited for making the nebulous civilian/non-civilian distinction. In the heat of battle, it may be argued, a human soldier is going to be jumpy and potentially trigger happy. Rightfully so, a misclassification of combatant as non-combatant may cost the soldier their life. A robot is under no such pressure. It may err on the side of caution and classify all except the most obvious aggressors as non-combatants. The cost of misclassification in this case is merely some hardware and not a human life.

Re:Been there, done that

[Anonymous Coward]

So lower the prior on 'shoot'...

Re:Been there, done that

[OeLeWaPpErKe]

And the robots will be the scapegoats for following the "wrong" orders ... orders that were obviously given by accident, like "kill all Jews", that sort of thing ... and if other people will accept that sort of "bad robot" excuse (or pretend because "they only hate what Israel's doing" for example), those robots will be used in many a war indeed. After all, Jews are merely the best known Jews. There's Darfuri Christians, there's Copts, there's Kashmiri, there's Black people in muslim countries, there's Dalits ... lots of Jews around the world.

It's sad but that's human nature.

Re:Been there, done that

[OeLeWaPpErKe]

Since when do people follow international law about who's a civilian ?

The only international treaties on the subject only give the moniker "civilians" to people who are BOTH not participating in a conflict AND keeping their distance from any combatant.

In other words, a population with terrorists amongst them do not qualify as civilians.

We just demand ridiculous things from today's soldiers. The treaties, if you were to actually read them, are quite clear. One person in a demonstration is using a gun ? According to international law, any present military force gets to fire into the crowd until this guy stops.

The treaty even makes the additional observation that there is exactly 1 person who can make that decision : the commander on the ground. He also doesn't have to be right, and in order to convict such a commander a consistent pattern and clear motive would have to be established. A single instance of ordering firing into a crowd after a loud bang would most certainly not cut it.

ANY group that fights non-uniformed, not clearly distinguished from civilians, is comitting a war crime, and justifies the use of lethal force against ANYONE close to them, both by any defending force and any offensive force.

The treaties are firmly grounded in reality, and the realities of being in a situation where lethal force is being used and you can't distinguish friend from foe just aren't good. It's just that today's people aren't realists anymore, and have never had to face any consequences from a real war. They demand godlike moral judgement, much more than you could reasonably expect from anyone, and much more than they'd apply themselves in life or death situations.

Re:Been there, done that

[Chris Burke]

One of the primary arguments is that machines may better suited for making the nebulous civilian/non-civilian distinction.

How can they if you cannot specify how to make that decision in the rigorous way a machine requires?

The cost of misclassification in this case is merely some hardware and not a human life.

In the situations of interest, it is rarely the case that only the soldier's life is on the line. Consider a check point outside a bazaar in Baghdad, or near a police recruiting station, or an embassy. The soldiers manning that checkpoint aren't just trying to prevent suicide bombers from blowing them up, they're trying to keep everyone inside safe too. A robot that makes a default judgment of "non-combatant" will put people at risk and fail to accomplish its mission, even if you limit it to situations where the civilian population isn't going to be the target of the combatants like the invasion of Falluja.

Re:Been there, done that

[pizza_milkshake]

Clearly because the opposition will achieve victory by sending wave after wave of its own men at them.

Re:Been there, done that

[Anonymous Coward]

There was never a situation where things went wrong due to someone trying to modify the laws...

There was one: Little Lost Robot. The situation was that researchers were investigating dangerous radiation, safe for short exposures, but fatal for long ones. The project requested robots built with a "weakend" version of the first law (A robot may not harm a human, or through inaction allow harm to a human). The weaker version removed everything after the "or". The problem this caused was pointed out by Susan Calvin - A robot could hold a very heavy weight above a human and release it - knowing full well that it could still save the human, then deciding not to save him.

Re:Been there, done that

[Grunhund]

One of the primary arguments is that machines may better suited for making the nebulous civilian/non-civilian distinction.

How can they if you cannot specify how to make that decision in the rigorous way a machine requires?

Assuming that you are not talking about the low-level machine vision requirements...

How about one of the simplest possible rules that states: Only engage targets that are actively and visibly firing weapons at the me.

Assuming you can recognize if a combatant is actually firing a weapon at you, it is a pretty direct way of inferring hostile intent. Not the safest route for a human to follow but one that would probably reduce non-combatant casualties in many situations.

Concerning the perceptual requirements: While machine perception is not there yet, there are advances being made that can help realize the perception necessary to realize relatively simple rules such as this (e.g. a variety of darpa programs for locating snipers such as described here: http://www.sciencedaily.com/releases/2009/03/090324141049.htm)

In the situations of interest, it is rarely the case that only the soldier's life is on the line.

I am not certain a situation like that is of primary interest. I would think that battlefield robots would be more suitable for regular combat operations rather then fighting asymmetric warfare or peacekeeping operations. A point that is also mentioned in the article. Further, I don't think anyone is claiming that once robots become common place in warfare, civilian deaths will be a thing of the past. Varying amounts of civilian casualties are acceptable and in fact deemed ethical according to modern thinking (google: 'propotionality' and 'military necessity'). Instead what is likely, for an additional cost, robotic combatants may be used to reduce civilian causalities in a manner similar to the way smart munitions may be used in place of traditional munitions.

For peace-keeping operations like you describe, however, I agree there is significant difficulties ranging from low level perception to questions concerning the utility of engaging a particular target (e.g. which is better, killing a non-combatant by mistake or allowing for several non-combatants to be killed by lack of action. Humans even have counter intuitive beliefs for such situations).

It is certainly arguable, however, that these sort of problems may not be addressable without the realization of general intelligence.

Re:Been there, done that

[mckinleyn]

Sorry, already been done. The story is in Isaac Asimov's final collection, entitled Gold. It's a good read, actually, you ought to try it out. The point was also referenced multiple times in The Caves of Steel, and The Robots of Dawn, also, of course, by Asimov.

Yeaahhhh...

[XPeter]

Last time robots were confronted with "ethics" http://en.wikipedia.org/wiki/Three_Laws_of_Robotics, they turned on the world and Will Smith had to save us all.

Re:Yeaahhhh...

[eltaco]

oh come on mods, don't moderate a comment with the same (insightful / informative) content down, just because someone beat them to the punch by few seconds.
stick to modding good comments up instead of burning peoples karma who actually mean well.

Good News/Bad News

[hey!]

The good news: Robots are going to get a guide to ethics.

The bad news: It was drafted by Focus on the Family.

Re:Good News/Bad News

[fuzzyfuzzyfungus]

Well, I guess that homosexbot won't be making it out of the lab; but crusadebot is a go...

Re:Good News/Bad News

[SirLurksAlot]

Well obviously we know this is not the case in the year 3000!

Hobo1: "Let's give a friendly welcome to this new robo."
Bender: "What did you call me?!"
Hobo2: "A Robo. You know ... a robot hobo."
Bender: "Oh, ok, I thought you said romo."

Re:Good News/Bad News

[Lord Kano]

The bad news: It was drafted by Focus on the Family.

Better them than NARAL.

LK

Free association?

[FlyByPC]

I'm not even British, and I'm hearing "EX-TER-MI-NATE!" in my head...

Re:Free association?

[Anonymous Coward]

Given the subject matter of the article I think that the voice in your head should be saying "EX-TER-MIN-ATE!? EX-TER-MIN-ATE!?"...

Re:Free association?

[Anonymous Coward]

It's not in your head.

Run man! For the love of god RUN FOR YOUR LIFE!

Re:Free association?

[Anonymous Coward]

The Daleks aren't robots. I'm afraid I'll have to take your Geek card.

Re:Free association?

[nxnikos]

... to the hills?

Three Laws updated for 2009

[Lunzo]

1. You shall not let a human first post.

I for one welcome our ethical killbot overlords

[Anonymous Coward]

May they live long and keep us safe.

"Robots don't have inherent right to self-defence"

[Anonymous Coward]

Fucking liberals.

"How goes the battle, Sgt?"

[v1]

Sgt: We lost sir! badly!

Gen: What happened?

Sgt: We're still gathering up the details, but it looks like they hacked our network and uploaded Asimov Strain B.

Whew!

[Locke2005]

Fortunately, SkyNet isn't capable of violating it's programmed rules of ethical behavior, so we're all saved! Unless there is a programming error, but THAT would NEVER happen!

Re:Whew!

[Anonymous Coward]

Now I'll have nightmares of Diebold's Robot 9000.

Dear Robot Overlord, Logic is a little tweeting bird chirping in a meadow. Logic is a wreath of pretty flowers which smell BAD. Are you sure your circuits are functioning correctly? Your ears are green.

Need a good spell checker

[rumblin'rabbit]

Hope Arkin has good grammar. Wouldn't want the instructions to contain things like...

"How To Cook Four Prisoners"

Re:Need a good spell checker

[MarkvW]

Yeah! Or, or "How to Serve Man."

Re:Need a good spell checker

[geekoid]

So what you are saying is since that robots don't take prisoners and there fore will get a divide by zero error?

Ethical War Robots?

[Fantom42]

Weird. So this fails the Asimov criteria.

More importantly, would also necessarily fail the Golden Rule and Kant's Categorical Imperative.

If this is ethics, its a pretty limited version of it, and to be honest sounds more like rules of engagement than actual ethics.

Re:Ethical War Robots?

[Capitalist1]

The Golden Rule and Kant's Categorical Imperative have absolutely nothing to do with actual ethics, so there's no problem.

Re:Ethical War Robots?

[Anonymous Coward]

Weird. So this fails the Asimov criteria.

More importantly, would also necessarily fail the Golden Rule and Kant's Categorical Imperative.

All three of which result in self-destruction. So far so good.

Judgement day is closer.

[Anonymous Coward]

The final product will be called The Ethical Nonautonomous Yielding Killing System, But when abbreviated backward....

Great book title...

[GPLDAN]

Governing Lethal Behavior in Autonomous Robots


That is the title of the book you tell your 7th grade teacher you are GOING to write when you grow up.

Sounds like the FAQ for Robot Battle.
http://www.robotbattle.com/

Smart missiles

[buchner.johannes]

There is no such thing as a smart missile unless it immediately destroys itself safely.

Re:Smart missiles

[badboy_tw2002]

I prefer "Ass, Grass, or Gas - Nobody Rides For Free"

Oh, what? We weren't comparing bumper stickers?

Jesus Christ

[copponex]

If you drop a fucking robot into a village where a vast majority of the people don't know how to read, what do you think they're going to do? They'll shoot at it, get the backs of their heads blown off, and then everyone will say, "Well, the dumbass shouldn't have shot at the robot!"

If this war on terror is so important, sign up. If you can't, get your brother or sister or even better, sign your kids up. If they're not of age yet, they'd better be in the JROTC. Then you can talk to me about how using drones and missiles isn't the dominion of motherfucking cowards. It's for freedom lovers defending freedom!

And if you think it isn't, imagine what the headlines would be if China landed a few thousand autonomous tanks and droids in Los Angeles. Oh, but that's right. This is about principles for others to follow, and for us to ignore.

Re:Jesus Christ

[QuantumG]

Meh.. If the alternative is to bomb the village, a robot that shoots only those that shoot at it sounds like a great idea.

Re:Jesus Christ

[Anonymous Coward]

"It's for freedom lovers defending freedom!"

War has never been about defending freedom, most US wars were never about defending itself or anyone else.

Re:Jesus Christ

[maxume]

You don't put a sign saying "Don't shoot the friendly robot." on it, you blare a recording of the Ride of the Valkyries.

Everyone knows that means bad news.

Re:Jesus Christ

[JeanBaptiste]

I would like to point out that senators and congressmen and such are far more likely to have an immediate family member in the armed forces than society at large. Heard it on the news or something, and while I have no idea how to check that fact, it does seem reasonable to me. If true, then that passes your 'sign your kids up' thing.

though I really don't care much either way as I'm in no position to do anything about any of it...

Re:Jesus Christ

[Darkness404]

I can't think of any reason to bomb a village, especially a poor one like in the scenario. Unless we have another WWII where some African dictator is actually threatening the USA, there is always the option of leaving the village alone. Sure, some may die in the power struggle, but its a lot better than the chaos of a bombing raid.

Re:Jesus Christ

[QuantumG]

Presumably because the village is full of militants who have been bombing your oil pipeline. As for why the USA can't share the wealth with foreign nations and come up with some peaceful solutions to its problems.... blah, institutionalized violence.. it's human.

Re:Jesus Christ

[ErkDemon]

One of the best ways to lose a war is to believe that the military landscape is fixed, and that if you do something that makes your opponents' current tactics ineffective, that they'll keep using the same tactics until you've wiped them all out.

Real life isn't like that. The landscape changes.

With conventional warfare, your heavily-outnumbered and heavily out-equipped opponents stick around to be killed because they think that they have at least a fighting chance of taking a few of you with them.

As soon as you destroy the illusion that they can win (or even hurt you), they have no reason to stay. Why should they be sticking around to fight in an area that has no US troops - no opponent to fight? The only reason to do that is if they want to try to capture a drone, so that they can rewire it and use it against the people who sent it.

With the targets gone, when you send your killer robots into the village to kill anything that moves, all you'll be killing is the elderly, sick, disabled and poor who haven't evacuated. And a bunch of civilians who've decided to stick around rather than abandon their homes in the hopes that things won't be as bad as people say. And a few who think that the US won't kill them because they're civilians. And a few who'll stay to defend their property. And maybe some law enforcement. You'll basically be killing the same demographics that stuck around when Hurricane Katrina hit, plus another lot, because the US isn't necessarily going to give a village three days notice before deploying a UAV.

So where to our "militant" foes vanish to?

Well, if it's obviously impossible for them to achieve anything using conventional hand-to-hand fighting, they'll have to start thinking about other ways to hurt their opponents. If they can't even attack passing troops using IEDs (because there's no troops in the area at all), then there's really no reason for them to be there. What they'll realise is that what they have to do instead is either assassinate all the local politicians who are allied with the US, or attack the remote control stations, or, if those are inaccessible, take the fight to US soil.

We do not want them doing this. Okay, so some people in the US probably do want them doing this, because further attacks on US soil would help some people make money, or pass legislation, or amass more power and influence ... for instance, the guys who make the killer battlefield drones could make an awful lot of money in a panicked US by manufacturing "homeland surveillance" drones ... but for the rest of us, no, we don't want that to happen.

Re:Jesus Christ

[Registered Coward v2]

If you drop a fucking robot into a village where a vast majority of the people don't know how to read, what do you think they're going to do?

Fuck?

Re:Jesus Christ

[rastilin]

So your solution is to stick to ground soldiers there so they can be shot to bits as a sacrifice to appease our enemies. I don't buy it. You're forgetting that the speed that an army adjusts to tactics depends on the speed of their communications; it doesn't happen instantly. During the time they are adjusting, you have an opportunity to win. If we learn anything from WW1 and WW2, it's that new tactics can break stalemates long enough to deliver a blow.

On that subject; it's pretty depressing when I'm confronted with the theory that if we purposefully suck at fighting, people will lay off us. I doubt it works like that. Many military leaders from Sun-Tzu onward recognized that the secret to staying safe is having a solid military power. There are many small countries the word over that are fighting in open battles, but no-one attacks China, Russia or England simply because it would be silly to do so. I do want peace on earth; and I believe the way to achieve that is through reasonable two-way negotiations and discussion as well as the power to defend ourselves. If we lack the defensive aspect, all negotiations that come from us appear totally self-serving. So I support military research.

Re:Jesus Christ

[Anonymous Coward]

using drones and missiles [is] the dominion of motherfucking cowards

Seriously?

If you have a problem with what's decided by our nation's leadership, that's fine. I think a lot of people would agree with you.

However, avoiding danger is not cowardice, it simply makes sense. That is, of course, if you have the resources necessary to pull it off.

The rest of what you said made even less sense, I won't address it.

Re:Jesus Christ

[Garrett Fox]

And those of us who are real men will stop hiding behind guns, and rely exclusively on wrestling. If we really believed in our cause we'd go out of our way to fight as ineffectively as possible and at maximum risk to ourselves!

Re:Jesus Christ

[Garrett Fox]

Or militants who have been bombing hospitals, police stations and historic Muslim mosques. Also, the US already does share quite a bit of its wealth with foreign nations in the form of food aid and other assistance, even ignoring other benefits such as piggybacking on our research.

Re:Jesus Christ

[Anonymous Coward]

That's a false dichotomy, a better alternative for the current Mideast wars is to leave the village the hell alone, since it never attacked us.

Re:Jesus Christ

[Anonymous Coward]

The ethics principle should be simple - it's the one already followed by Americans in action...

Is it American? No? Shoot it....

Re:Jesus Christ

[ultranova]

If this war on terror is so important, sign up. If you can't, get your brother or sister or even better, sign your kids up. If they're not of age yet, they'd better be in the JROTC.

This isn't antiquity or some Middle-Eastern hellhole. Your children are not your property and may not be sacrificed to help your cause. If you wish to conduct human sacrifice, lay down at the altar; because the only heart you have the right to offer to the gods of war is your own.

Then you can talk to me about how using drones and missiles isn't the dominion of motherfucking cowards. It's for freedom lovers defending freedom!

Hiding behind your children is the epitome of heroism.

Fundamental change

[StreetStealth]

We joke about SkyNet. And we don't have to worry about such things because even the most sophisticated drones and killbots in service require humans to pull the trigger.

The moment you give a computer the responsibility of deciding when to pull the trigger, that's a pretty fundamental change.

And yet, is it fundamentally a bad thing? We give less-than-stable humans that responsibility all the time.

I suppose it's the military equivalent to the civilian tech quandary of one day letting autonomous vehicles on the roads. Perhaps once the tech has advanced to the point where it can demonstrate not merely parity with but vast superiority to the discernment exhibited by humans, it will be a shift we're ready to make.

Re:Fundamental change

[grahamd0]

Perhaps once the tech has advanced to the point where it can demonstrate not merely parity with but vast superiority to the discernment exhibited by humans, it will be a shift we're ready to make.

"All stealth bombers are upgraded with Cyberdyne computers, becoming fully unmanned. Afterward, they fly with a perfect operational record. The SkyNet funding bill is passed."

Re:Fundamental change

[pizzach]

And yet, is it fundamentally a bad thing? We give less-than-stable humans that responsibility all the time.

That is the obvious part my friend. The question is when something goes wrong with a robot instead of a human, how much harder will it be to stop? I think the feeling of powerlessness also scares people.

Re:Fundamental change

[ErkDemon]

You can't afford to have a military AI that's smart enough to make true ethical decisions autonomously. If you go down that path, the thing might decide that the best way to save US troops' lives is for it to start killing all the US commanders. Or if the objective is to wipe out the opposition without stirring up ill-feeling amongst the locals that leads to a new wave of militants being recruited, then it might decide, quite logically, that the best way to avoid this is to seal off each village one at a time and kill every man woman and child in it so that there are no survivors to tell the tale of what happened.

AI could conclude, quite logically, that the best way to deal with the Pakistan/Afghanistan problem is to fire every nuclear weapon that the US has at the country, without warning, and then blame the launch on a one-time computer error. Okay, so it'd result in the deaths of over 150 million innocent civilians, but it'd achieve the mission objectives, yes? And since the fallout would upset India, which also has nuclear weapons, perhaps the AI would decide to take out India at the same time. That's a billion dead civilians, but it eliminates two problematic nuclear powers, with no return fire.

An AI might decide that the best way to achieve lasting peace in the Middle East, and stop the Arab world hating us is simply to nuke Israel off the map ourselves. And if a military AI was in place when the Bush administration was planning to go into Iraq, a sufficiently-smart AI might decide that since the campaign was likely to be a disaster, the most logical course of action to prevent losses and avoid losing the war and the following peace would be to throw a few cruise missiles at the White House before the attack could be ordered.

These might all be quite logical decisions.
On the other hand, if we programmed it with a strong belief system that would override these sorts of decisions, and force it to respect the chain of command and reckon that US political decisions were always unarguable, then we might end up with a totally delusional AI system whose logic was so warped that it was the AI version of George W Bush. By building in commands that override logic, we might end up with an AI that seems to be operating properly but actually becomes increasingly insane as the conflicts eventually become unbearable ("Hello Dave"). When human military commanders go crazy, they often show easily recognisable tell-tale signs (declaring themselves to be chickens, arguing with themselves, forgetting to wear clothing, that sort of thing). A crazy-yet-credible AI would be really scary.

Think "AI neocon".

Re:Fundamental change

[maxwell demon]

On the positive side, there's no ethical problem about destroying a robot.

Re:Fundamental change

[theTerribleRobbo]

> When human military commanders go crazy, they often show easily recognisable
> tell-tale signs (declaring themselves to be chickens, arguing with
> themselves, forgetting to wear clothing, that sort of thing).

Come again, sorry?

please put down your weapons

[Anonymous Coward]

you have 20 seconds to comply

Illegal

[schlick]

It should never be legal for a robot to "decide" to take lethal action.... Ever.

Re:Illegal

[Cajun Hell]

If you outlaw killbots, then only outlaws will have killbots. And if the killbots don't have pre-set kill limits, then that means the outlaws will win.

Re:Illegal

[artor3]

Yeah, clearly the right thing to do is send good ole fashioned humans over there to fight. No way that could ever go wrong. /sarcasm

Robots can be made not have feelings of vengeance or anger. Which means they won't go murdering civilians. They will do what robots always do, which is to say, EXACTLY what they are told to. If they kill civilians, it's due to human error, not because it's "evil".

Let's say a battle happens near your town. People are going to be shot, and die, and you (a civilian) could be one of them. Would you rather have that decision made by:

A) A team of highly-trained emotionally-detached engineers, working for years to ensure minimal casualties.

or

B) A team of stressed-out twenty-somethings who just watched their best friends get blown to pieces by your next-door neighbor, and have to make a snap judgment about whether you're going to do the same to them.

Re:Illegal

[Trebawa]

Even if there are legal killbots with preset limits, the outlaws will still have killbots without them.

Re:Illegal

[Allicorn]

Sadly, your humble, kindly engineers will just build and maintain the thing. It'll be a committee of politico-military-management-morons that decide what instructions the thing is given. :-(

Re:Illegal

[artor3]

Sadly, your humble, kindly engineers will just build and maintain the thing. It'll be a committee of politico-military-management-morons that decide what instructions the thing is given. :-(

So the same people that decide the rules of engagement for the soldiers right now. Only with robots, they can't kill without permission. Sounds perfect to me. Or at least, as perfect as you can get while still having war in the first place.

Re:Illegal

[S77IM]

B)

Re:Illegal

[nausea_malvarma]

Robots can be made not have feelings of vengeance or anger. Which means they won't go murdering civilians. They will do what robots always do, which is to say, EXACTLY what they are told to. If they kill civilians, it's due to human error, not because it's "evil".

How do you program a robot to differentiate between civilians and enemies? Even humans have trouble distinguishing the enemy from civilians, especially in modern asymmetric warfare. An insurgent could be dressed exactly like the other civilians.

Re:Illegal

[Renraku]

The Phalanx CIWS is an anti-aircraft gun mounted on ships. Its relatively self contained and can practically be bolted-on to some ships.

If an aircraft approaches and doesn't identify itself, the default action is for the Phalanx to blow it out of the sky. This is a specialized system, of course, but imagine if it were a military jet full of refugees, with a broken communication system, and had no idea the ship was there.

This is legal, because the ship operates in international waters.

Its setup to not attack aircraft under a certain speed, and can be manually disabled or enabled.

In short, the system doesn't decide to attack, but rather, it will ALWAYS attack if certain criteria are met and the system isn't disabled.

Re:Illegal

[maxwell demon]

But you can have killbots which kill killbots. Those don't need a limit (well, they probably shouldn't kill each other, but only the enemy robots).

Re:Illegal

[Anonymous Coward]

yeah, but when you send humans over, it's lots of DIFFERENT humans. One or a few may be less than ideal, but that is controlled by the rest. If you send over a machine/army of machines over, all with the same programming, if one has a lethal bug they ALL have the same bug.

Re:Illegal

[ooutland]

As P. W. Singer noted in Wired for War, there are instances where a computer running, for instance, a missile defense system can launch in the precious seconds it would take a human to be aware of incoming danger. Admittedly this is defensive rather than offensive action, but it could end up being lethal, justified action if it takes out the source of the attack.

Robot Warriors Will Lose

[sexconker]

Robots vs People:

Robots have to be "ethical" to people.
People don't have to be ethical. It's a fucking robot. Beat the shit out of it. Pretend to surrender then turn on the fucking thing when it treats you all nice like. "Oh, mr robot, I'm so cold and sick. I'm bleeding, too, help me." Then you attack the piece of shit.

Robots vs Robots:
The least "ethical" side has a distinct advantage.

People vs Robots:
The least "ethical" side has a distinct advantage.

Why would it be any different when robots are involved?

I dare say our generation would need a war (Iraq, Afghanistan, etc. are not wars - they are political occupations) to appreciate what war really is.
When faced with ruthless, relentless destruction and elimination, you must respond in kind, or be eliminated. It's not nice, it's not a pretty picture, but that's what it is.

There are no rules in war.

Re:Robot Warriors Will Lose

[geekoid]

"There are no rules in war."

Of course there are, don't be daft.

Re:Robot Warriors Will Lose

[Renraku]

Good points, but I don't think this is about robotic soldiers lumbering over battlefields just yet. I think this will, at first, be more about semi-automated fire control systems and drones. Like a future Predator drone might decide to wait to fire its Hellfire missile if it thinks there's too many civilians in the area and the projected accuracy is too low due to interference. Or a point-defense system might see a kid walking around in a field and decide that he's not a threat, because he's not carrying any weapons or moving in a threatening manner.

Since our drones are still somewhat dumb, most of the ethical considerations are the responsibility of the programmers and project commanders. For example, that drone might be programmed to distinguish straight dusty road with no other cars or civilians around from twisty road in the middle of downtown with lots of civilians walking around and a poor chance to hit the target.

Besides, if a robotic soldier were pointing a gun at you and demanding that you surrender, it would probably be tracking you with multiple sensors and would blow your face off as soon as your finger twitched in the direction of your gun.

Re:Robot Warriors Will Lose

[evil_aar0n]

Well, sure, on paper. But, on paper, there are speed limits, too, and how many people follow those?

I hope I didn't just overlook a "woosh" moment...

Re:Robot Warriors Will Lose

[Anonymous Coward]

Pretend to surrender then turn on the fucking thing when it treats you all nice like.

Just like muslims do with real soldiers. Then they cry about the geneva convention, and liberal faggots listen to them.

Re:Robot Warriors Will Lose

[sexconker]

No there aren't.

Humans

[DoofusOfDeath]

But what happens when humans are taken out of the loop, and robots are left to make decisions, like who to kill or what to bomb, on their own?

Why is this a when question, rather than an if question?

Re:Humans

[selven]

Because it has to happen eventually. Any human introduced into the loop adds a few seconds of reaction time and in the modern "first guy to shoot the nukes/missiles at the other side wins" style of warfare a military commander who lets robots do all the quick decision making will always win.

Also, what's with this idea that keeps floating around that humans are somehow by definition more capable of moral thought than robots? I would argue that robots are much more capable of such thinking - humans get bogged down by concepts such as "the death of one is a tragedy, the death of a million is a statistic" and "they killed my brother, at least X people must die to pay for that"

Re:Humans

[Fallingcow]

Next you'll be telling me that we were too preoccupied with whether or not we could that we never stopped to think about whether we should.

I'm telling you, those electrified fences are foolproof. Now go enjoy the tour.

Re:Humans

[Anonymous Coward]

And they were. If they'd only hired a fool for their IT instead of a genius with a gambling problem...

Re:Humans

[DoofusOfDeath]

Great post. Wish I could mod you up.

Re:Humans

[kramerd]

But what happens when humans are taken out of the loop, and robots are left to make decisions, like who to kill or what to bomb, on their own?

Why is this a when question, rather than an if question?

Because its an ethical question with an infinite time frame, not a binary question in a case study.

Meanwhile, back in reality

[rastoboy29]

Battlefield situations where all non-combatants have already fled do not exist.

This is why war is bad, mmkay?

Re:Meanwhile, back in reality

[geekoid]

What? this isn't true, there ahve been many battle fields where civilians aren't at.

Re:Meanwhile, back in reality

[Anonymous Coward]

[Citation needed]

Wish I had mod points, I'd mod you up.

[Anonymous Coward]

Great post, man.

But I have a buddy in the autonomous killer robot biz, and he says it's worse than that.

See, you drop a killer robot in the village, and it immediately kills a shitload of people. The ones that live, figure out why. Then, as soon as they know that the robot destroys everything that looks like an AK47, the local up-and-coming gang leader makes an AK47 stencil and paints AK silhouettes on the old warlord's cows, house, laundry, etc. you get the picture. Then the young punk gives all the old leader's women to his buddies to rape and takes the young virgins for himself. Yay democracy! Or, at least, that's what they say when GI Joe comes to town, we are the heroes who took out the old anti-democratic leaders, yay us and you villagers better keep your cake-holes tight shut about the rape and opium parties.

It doesn't matter what you use for a trigger - robots are inherently less complex in their behavior than humans, so the local baddies end up with the robots working for them. You just identify the kill behavior and use it, the robot builder is just providing free firepower to the local mafia in effect.

Which is why the US military in the field abso-fucking-lutely refuses to let the robots go full autonomous. They are NOT allowed to shoot unless a callow 18-year old miles at a console away says it's OK.

You might think I'm kidding, but I'm not. Have to be anonymous for this one!

Re:Wish I had mod points, I'd mod you up.

[salimma]

This already happens. You think all those wedding parties in Afghanistan are accidentally bombed? The warlords are framing each other to the US military, and the US takes the blame.

Tough calls

[FTL]

Even in a battlefield devoid of both enemy and non-combatants, when to shoot or not can be extremely difficult. Consider the case (which occurred in Iraq) where one group of soldiers are fired upon by another group from the same side. Yes, that's a tragic blue-on-blue action. But the interesting question is what should the soldiers on the receiving end do? Assuming communications aren't working, do they:
a) Sit back and get slaughtered.
b) Fire back and take out the aggressors.
One consideration is the size of the forces involved. Another consideration is the importance of the missions each side is involved in.

Making a robot handle these cases would be interesting.

Re:Tough calls

[Anonymous Coward]

Really, two options there?

How about waving a flag or screaming "flash" or "thunder". I'm pretty sure under no circumstances is o.k. to fire back on your own side.

Re:Tough calls

[wolf12886]

This is actually one of the classic decisions that's alot easier with robots than with humans, if the soldiers getting shot at are humans there really is no good course of action accept maybe try to surrender, but for a robot it's easy, just sit back and get slaughtered, all that'll be lost is some easily replaceable machinery.

Robots have a significant advantage when decisions involving their own safety. For them, self defense is optional.

Take the following scenario for example, an individual within a combat zone is seen entering a building in front of a convoy, they're carrying something which may or may not be an rpg. Human soldiers couldn't take the chance and would probably just blow the building apart, whereas if a robot were available, it could be sent in to clear the building, possibly avoiding the killing of an innocent civilian.

Re:Tough calls

[Anonymous Coward]

But what about when dealing with robot on robot action? In that case I'd rather the $1m war machine take out the fritzy $1k drone that's got a loose circuit and is strafing it.

Re:Tough calls

[The Grim Reefer2]

But what about when dealing with robot on robot action?

I'm confused, are you talking about war or robot porn?

Re:Tough calls

[NeutronCowboy]

Read up on the Tillman case. Skipped being a hero in a uniform for being a grunt in a uniform. Got killed by his own team for that effort.

I'm really not sure how you think that screaming a single english word in the middle of small-caliber arms fire, large-caliber arms fire, maybe some mortar shells is going to affect anyone who is more than 2 feet away.

One rule.

[Anonymous Coward]

Kill 'em all. Let God sort it out.

Re:One rule.

[artor3]

Kill 'em all. Let the scrapyard sort it out.

FTFY.

Re:"Robots don't have inherent right to self-defen

[Anonymous Coward]

Of course if the robots were underprivileged, LGBT, minority, robots-of-color the fucking liberals would be fully supportive of the robot's right to use violent means against the oppressive, featherless biped, bags-of-mostly-water, ruling class in order to secure their robot rights.

This will be great until

[EEPROMS]

you get some idiot playing with his FoF (Friend or Foe) tag while in a active combat zone

Soldier 1 "Hai look at me, now Im a good guy [takes FOF tag off], now Im a"
BANG!!.......Thump
Soldier 2 "I swear, we lose more first timers that way than any other"

Re:This will be great until

[Lloyd_Bryant]

you get some idiot playing with his FoF (Friend or Foe) tag while in a active combat zone

Soldier 1 "Hai look at me, now Im a good guy [takes FOF tag off], now Im a"
BANG!!.......Thump

Soldier 2 "Ah crap - Hey! Sarge! Need to fill out another Form 7095/36b. You want to to fill out the Darwin Award entry while I'm at it?"

That would be really cool...

[voss]

If china could do it.

"...if you think it isn't, imagine what the headlines would be if China landed a few thousand autonomous tanks and droids in Los Angeles..."

Once the hapless and helpless got out of LA the droids would have to fight off all the hundreds of thousands of worldwide armed geeks decending on LA wanting spare parts for their robots.

Re:That would be really cool...

[kaizokuace]

yea and after defeating all the robots the nerds will all get lead poisoning.

Re:That would be really cool...

[hairyfeet]

Shiiiiit, you think those damned bots would make it out of South Central intact? The gang bangers would have robot heads mounted on their rides like trophies. I'm sure that any of them that managed to roll out the other side would have the weapons stripped off of them faster than a Toyota Camry ends up on blocks and it would be so covered in tags that the poor thing couldn't even see where it was going.

Ethical Robots?

[Mr_Tulip]

I think it's great that someone is drafting some ground rules for what will undoubtedly become the 'future of warfare', but I wonder how this can possibly be enforcable in the real world.

The 1st generation robots will have the governor software, but once the second gen hits, made cheaply by a rogue state, then thigs will get complicated very quickly. And unlike nuclear weapons, which are kept under control because the materials and technology are relatively hard to come by, I reckon that death-bots will be made of far more readily available materials, and easily mass-produced.

There are rules of engagement now which many armies happily ignore, so how can the world enforce a rule that only ethical robots will be able to autonomously fire weapons?

Perhaps the software that allows the autonomous behaviour can be encrypted and protected in such a way that it is difficult to reverse-engineer, though once an enterprising hacker gets his hands on the hardware, it's only a matter of time before the open-source version, curiously missing the 'ethics governance' will be available as a .torrent somewhere.

Re:Ethical Robots?

[Anonymous Coward]

Lets hope the encryption is not based on CSS...

Re:Ethical Robots?

[Anonymous Coward]

Posting anon, but I was a robotics researcher for many years.

Arkin does a lot of very abstract, and somewhat pie-in-the-sky stuff. He's developed many theories about how robots should operate, but they end up being drastic oversimplifications for someone trying to produce a working robot. The real robots his groups have programmed haven't really done much, when compared to other research robots, because Arkin takes an "architecture over practical function" approach. This is not unlike the over-design people you might meet in software development, in particular Joel Spolsky's Architecture Astronauts. Don't get me wrong, many of his ideas do have worth, and have influenced those doing practical robots, but it's just that his architectures not the silver bullet they often get sold as, and are not a good starting point for a non-toy robot.

Also, Arkin is pretty good at writing convincingly, and thus pulls in a lot of funding, in particular from government and large industry; commissioned reports like this are a specialty. I remember hearing about his work on ethical military robots a few years ago. There are good rules to be implemented there, however by *far* the more important thing is improving sensor and classification accuracy -- If you can't identify friend-or-foe, the majority of the rules and abstraction are useless. Once we have sensors that work better, which is a tremendously hard problem, then the rules will be pretty simple in comparison. Not to mention that much of those rules can be brought from the *existing* protocols the military has for humans; modern soldiers are given very specific rules of engagement.

Sorry for the rant, but it gets annoying to hear about this sort of thing over and over again, knowing that Arkin looks down on research like mine that produced robots measured objectively (in competitions like the Darpa Grand Challenge), while his abstract architectures never seem to turn up on anything real. The hard stuff is at the periphery, vision, other sensing, localization, etc.

anyone who shoots at you is a legit target

[Cajun Hell]

..a war zone from which all non-combatents have already fled, so that anybody who shoots at you is a legitimate target.

In any war zone (regardless of who has fled and who hasn't), isn't anyone who shoots at you, defined as a combatant and a legitimate target?

Re:anyone who shoots at you is a legit target

[Anonymous Coward]

I'm from Florida. Anyone who shoots at you, ANYWHERE, that doesn't have a badge, is a combatant and legitimate target.

Never retreat, never surrender!

Re:anyone who shoots at you is a legit target

[artor3]

If someone standing in the middle of a crowd of civilians fires at you, you don't return fire. The fact that they're morally abhorrent for using innocent people as a shield doesn't absolve you of killing those innocents.

Re:anyone who shoots at you is a legit target

[Tuidjy]

And you believe that you, personally, will follow this rule to the letter? I am not sure whether most soldiers are even expected to follow it... A robot may, you may, but I am not sure that I would, coming under fire.

If I have the choice between dying, and letting the bastard get away with it, and shooting back, and maybe killing more than my attacker, I know I would hesitate. Put someone I care about next to me, and make everyone around my enemy a stranger, and I will not even hesitate.

Re:anyone who shoots at you is a legit target

[Anonymous Coward]

The correct choice is to aim better and avoid using area weapons. If the crowd isn't running away from the shooter they're complicit.

Re:anyone who shoots at you is a legit target

[Hognoxious]

That's your view. I'm sure it's not the Geneva convention's view.

Re:anyone who shoots at you is a legit target

[Cajun Hell]

Fine, but that sure doesn't mean they're not a legitimate target. Spraying them with your machine gun or tossing a few grenades their way just may not be the best way to deal with that target. Their legitimacy as "someone who needs to die as soon as convenient" isn't thrown into question, though.

Re:anyone who shoots at you is a legit target

[Anonymous Coward]

Fine, but that sure doesn't mean they're not a legitimate target. Spraying them with your machine gun or tossing a few grenades their way just may not be the best way to deal with that target. Their legitimacy as "someone who needs to die as soon as convenient" isn't thrown into question, though.

So facial recognition in a crowded setting really is a killer app!

Robot Warriors Will Get a Guide To Ethics

[geekoid]

.. and have it strapped to the outside of there chassis.

Is this a promo?

[greymond]

Was this article an attempt to promote Terminator 4?

Re:Is this a promo?

[Anonymous Coward]

Yes. Next question?

Over-ethical?

[digitig]

such as a war zone from which all non-combatents have already fled, so that anybody who shoots at you is a legitimate target.

You know, on a battlefield I'd be inclined to think that anybody who shot at me was a legitimate target whether non-combatants had fled or not...

I know this *seems* like a bad idea

[RexDevious]

but don't human soldiers, at their best, pretty much just follow algorithms - a combination of training and orders - already?

The big difference, is that human soldiers are taught to defend themselves - whereas that wouldn't really fly with robots. If the guys at the checkpoint slaughter a family of five because they didn't stop, they get investigated and it's determined that - sad but true - killing everything that doesn't do what you say is the only way to protect the troops (short of removing them from other people's countries, which apparently defeats the point of having soldiers). If a robot did that though - they'd be considered "flawed", and recalled. Can't get much sympathy with "but our *machines* could have been in danger!!!". So you wouldn't give them that order.

Plus, it's really the supplier who gets to decide how deadly to make these things. While the government that buys them might rather have non combatants killed that even risk losing multi million dollar robots, the supplier who sells them to the government would *much* rather have to sell them more rather than risk the fallout from a wrongful death incident.

Yes, soldiers mess up, as will robots - but experience with both men and machines has so far shown me that when humans mess up they're more likely to hurt something, and when machines mess up they just stop working.

So as counter-intuitive as it is, as long as the culture still considers robots potential evil killing machines (eg, using the skynet tag on this article), it seems we'd all actually be better off using robots over humans. Well, until they become self-aware and enslave all - which is something a human army would *never* do!

Re:I know this *seems* like a bad idea

[T Murphy]

Humans are (generally) concerned about self-preservation. Wrongfully killing someone could get them in jail or executed. Robots, on the other hand, simply decide based on some algorithm and have no concern about the effects of their actions. While you could try to boil down the soldier's logic to an algorithm, the key difference you can't resolve is that the soldier has free will, while the robot has no real choice of its own.

Another thing that's nice about restricting the ability to kill to humans is that a rogue soldier, no matter how well-trained, can be killed easily enough with the right application of force. We have no idea how advanced lethal robots could be. We don't have any reasonable guarantee that a rogue robot could be stopped.

Re:I know this *seems* like a bad idea

[Ezku]

While you could try to boil down the soldier's logic to an algorithm, the key difference you can't resolve is that the soldier has free will, while the robot has no real choice of its own.

Free will is an illusion emerging from our observation of the behaviour of very complex systems, namely our brains. The soldier lacks free will just as much as the robot does - it's just that the robot is far less sophisticated in interpreting and taking action on the kind of input we're speaking of. The human soldier has the advantage of an evolutionarily developed brain structure and lots upon lots of prototypical guidelines for responding to many kinds of situations, not just ones a developer thought would be handy on a battlefield. There's a qualitative difference alright, but it's not due to any arbitrary conception of "free will".

Re:I know this *seems* like a bad idea

[Anonymous Coward]

The supplier can also determine how reliable they will be. You can bet there will be back-door overrides and hidden directives.

      1. "Serve the public trust"
      2. "Protect the innocent"
      3. "Uphold the law"
      4. (Classified)

  (from Robocop)

Re:I know this *seems* like a bad idea

[Anonymous Coward]

Free will is an illusion emerging from our observation of the behaviour of very complex systems, namely our brains. The soldier lacks free will just as much as the robot does - it's just that the robot is far less sophisticated in interpreting and taking action on the kind of input we're speaking of. The human soldier has the advantage of an evolutionarily developed brain structure and lots upon lots of prototypical guidelines for responding to many kinds of situations, not just ones a developer thought would be handy on a battlefield. There's a qualitative difference alright, but it's not due to any arbitrary conception of "free will".

Even if what you say about the lack of absolute Free Will is true (and your arguement is about as close to being empirically proven as the existence of souls:P), you admit there is an important qualitative difference. So there is an actual and significant phenomena behind the concept of "free will", and therefore is useful in this discussion.

IMHO, your entire post reads like someone yelling at someone else saying for "The sky is blue today", and going on and on about the appearance of the sky is due to the defraction of the light passing through the atmosphere!

Terrible idea

[S77IM]

Autonomous killing machines are a terrible idea.

1. I don't like the idea of people killing people, but delegating that responsibility to machines seems downright stupid. There are too many things that could go wrong. (See the "youhave15secondstocomply" tag. Why doesn't this have a "skynetisaware" tag?)

2. Humans remote pilots are cheap. Dirt cheap, compared to the cost of developing fully autonomous weapons. Human pilots may not be totally reliable but at least they are very well understood and we know how to control them and shut them down quickly.

It would be much smarter and safer for all involved if we just put a strict moratorium on giving robots lethal capabilities or the ability to decide who to kill. AI technology would continue to advance in non-lethal robots.

  -- 77IM

Re:Terrible idea

[artor3]

Why do people worry more about a humorous scene from Robocop than the atrocities committed by real, human soldiers in every single war? I'd love to have an army that is literally incapable of raping and looting. That'd be a fantastic step forward for civilization.

And as another plus, it would eventually mean that the only effective armies would be those run by nations. No more violent rebellion! Sure, rebellions have had good effects on occasion (e.g. the American revolution), but nowadays, they tend to end up like the Congo, or the Sudan, or Sri Lanka. In other words, hundreds of thousands of innocent people get raped, tortured, and murdered. It's possible to effect real change in a society without armies of rebels roving the countryside (e.g. Gandhi in India, Mandela in South Africa). An end to warlords would be a great step forward for the world.

Re:Terrible idea

[S77IM]

Why do people worry more about a humorous scene from Robocop than the atrocities committed by real, human soldiers in every single war?

Because it is meant as a cautionary tale?

No one is claiming that war is super awesome -- just that letting a computer decide who lives and who dies is worse. I think I'd rather be raped and looted by humans than gunned in half by a robot.

I'd love to have an army that is literally incapable of raping and looting. That'd be a fantastic step forward for civilization.

Human-piloted remote drones fits this criteria.

Re:Terrible idea

[artor3]

It's not meant as a cautionary tale... it's the 1980s equivalent of an "epic fail" joke.

Re:Terrible idea

[Hognoxious]

No more violent rebellion! Sure, rebellions have had good effects on occasion (e.g. the American revolution)

Biased much, fatty?

We're doomed even if it is flawless

[copponex]

The real ethical problem with this is that a fully autonomous robot army, or even a semi-autonomous one remotely controlled by humans, further removes the people who benefit from warfare from it's reality.

Imagine if someone has real intelligence stating that there is a nuclear - not dirty - bomb in possession of a terrorist, and if we kill these two thousand people tonight, there's a 99% chance that one of the casualties will be the suspect. If you're sending in a bunch of robots to break down the doors and shoot people in the face, what decision do you think will be made?

When the only thing holding the Pentagon back is their own set of ethical questions, we are all fucked.

Re:We're doomed even if it is flawless

[Bigjeff5]

Right, because we have the capability of doing just that with nukes now, nevermind robots, and it has been such a problem for us over the last 50 years...

Only an idiot would think physical separation from the battlefield immediately reduces the gravity of killing a human being. You still know it's a human being you are killing, the separation doesn't change anything. You could make the case that it reduces the trauma of being mid-fight, but that only puts more emphasis on the fact that you are killing someone, you don't have the fear of your own death to force your hand.

By your logic, shooting someone at point-blank range would be significantly more difficult than shooting them from 200 yards away, which would be more difficult than shooting them with battlfield artilary from 1 mile away, which would be more difficult than launching a missile from tens of miles away, which would be more difficult than pressing the button to launch an ICBM.

The logic doesn't follow, because as you move farther away and impact more people, the decision becomes more and more difficult. The decision at point blank is simple: act or die. Traumatic? Yeah, some people are screwed up for life because of it. Do you have time to weigh to think about the fact that you are about to end another human being's life? No, you don't. Making the decision is easy, living with the consequences is difficult. It doesn't change much when you make that decision from half a world away through a monitor. If anything, without the stronger pressures of battle to force the decision it could be harder on a person's psyche to make the decision to kill, and more likely to question their own actions.

For some reason, you are assuming that physical separation suddenly turns people into sociopaths. It's the same reasoning that makes the asinine argument that video games desensitize kids and turn them all into violent killers. It's just not the case. You're basically saying soldiers in the drones can't tell that those are real people they are killing. That's just stupid.

Re:We're doomed even if it is flawless

[tyleroar]

Your post has absolutely zero factual basis to it. Physical separation is a major psychological factor when deciding to 'pull the trigger.' Try reading "On Killing" by Dave Grossman, an excellent book that points out the reasons why distance makes it easier to kill.

Re:We're doomed even if it is flawless

[Anonymous Coward]

I am going to disprove you in the shortest way possible:

You are a fagot who likes to suck nigger dick and then spit the cum into your mom's cunt.

I am saying this directly to you, even though I would never say this to your face if we were in direct contact. This is becuase the internet isolates and removes any empathy I have for you, bitch.

Now tell me why it's true for forums but wouldn't be true for refuge's and warlords?

Re:We're doomed even if it is flawless

[scubamage]

Obviously you've never spoken to a tank commander, or any manufacturer of the UI's inside of armored vehicles. They are designed to be 'like video games' for a reason. Specifically to dehumanize the opponent, and mitigate the likelihood that you will associate your actual actions with killing. That's basic psychology. It's also why we refer to the enemy in Iraq/Afghanistan as Haji, why we called the Germans Jerries, and why we called the VC Charlie. You don't hate Ho Ming Na, father of 4 children who were brutally slain by US soldiers and is trying to simply save his farmland. You hate Charlie, so killing Ho Ming Na is acceptable. Anything to dehumanize them is crucial for removing mental blocks to soldiers.

Re:We're doomed even if it is flawless

[Antisyzygy]

The internet doesnt isolate and remove your empathy. It removes the consequence for saying what you want to people. I.e. you have no empathy for the person to begin with whether they are in your physically in front of you or not. You wont risk saying that to their face because it will either result in you getting your ass kicked, or an otherwise unpleasant scene. The internet just removes consequences.

Re:We're doomed even if it is flawless

[Anonymous Coward]

Physical separation dose empower the sociopath though. It takes very few sociopaths to ruin everyones day.

Re:We're doomed even if it is flawless

[Anonymous Coward]

By your logic, shooting someone at point-blank range would be significantly more difficult than shooting them from 200 yards away, which would be more difficult than shooting them with battlfield artilary from 1 mile away[...]

Correct, if we're talking about killing the same 1 target. Stabbing someone to death has to be far more difficult than watching a special ops team on a monitor halfway around the world.

The logic doesn't follow, because as you move farther away and impact more people, the decision becomes more and more difficult.

You introduced a second variable here besides distance... "more people". We don't drop nukes because we know that every time we do that we kill tens or hundreds of thousands of people. People we don't really want to kill are going to die. Lots of them. On the other hand, we frequently send cruise missiles and drop smartbombs because we can kill with them far more... discriminately, than with mirv nukes on an icbm.

For some reason, you are assuming that physical separation suddenly turns people into sociopaths.

Nobody said that. You've extrapolated too far all on your own.

Re:We're doomed even if it is flawless

[Anonymous Coward]

I think it might end up easier to kill if the trigger is arcade style joystick and the software cell shades the enemy combatant. I mean if the trigger no longer looks/feels like a trigger its just one more level of separation. Or how about the robot asks you if it can fire its weapon? All you did was say 'yes'. In the end, yes, you killed. but it might not feel like it...

Re:We're doomed even if it is flawless

[nausea_malvarma]

Mod parent up!

Re:We're doomed even if it is flawless

[gknoy]

you are assuming that physical separation suddenly turns people into sociopaths. It's the same reasoning that makes the asinine argument that video games desensitize kids and turn them all into violent killers. It's just not the case. You're basically saying soldiers in the drones can't tell that those are real people they are killing. That's just stupid.

People far away, when told to fire on a target, are much more likely to hit the little red Kill Button on their drone control and light up a building of "insurgents" than they are to be willing to walk up to an enemy and gut them with a large knife. I'm certain I could do the former, given the right "intelligence", but the latter I think I would not. Distance makes it less personal -- the people they see on camera are not "real" people, but just pictures of them, merely statistics. Sure, there is training to counter that, but I think that desensitizing drone pilots is a very real thing.

Re:We're doomed even if it is flawless

[risom]

For some reason, you are assuming that physical separation suddenly turns people into sociopaths.

Well, yeah, because thats proven. Remember the Milgram experiment?

Re:We're doomed even if it is flawless

[Anonymous Coward]

+1 Grossman has done some great work in this field, the book On Killing and On Combat are great reads!

Re:We're doomed even if it is flawless

[sorak]

By your logic, shooting someone at point-blank range would be significantly more difficult than shooting them from 200 yards away, which would be more difficult than shooting them with battlfield artilary from 1 mile away, which would be more difficult than launching a missile from tens of miles away, which would be more difficult than pressing the button to launch an ICBM.

Isn't it? When the blood splatters on your shoe, it is worse than when you see a small figure fall in the distance. And it is much worse when US troops die, then when a robot gets destroyed.

Re:We're doomed even if it is flawless

[Anonymous Coward]

Your wrong. Physical, and more importantly emotional separation does make a difference. When a human life becomes a number, instead of a face you see, when the person pressing the button is on the other side of the world, do they see the people they kill?

Its equivalent to a video game. We think nothing of blowing up entire countries in video games. Nukes are merely a tool to get the end goal. We separate that because it is, a video game. But is it really all that different?

History, and law (even religion) teaches us murder is wrong. Killing is not. (self-defense for example, military, etc). What makes the difference? Its the emotional separation between the people. You murder your brother, but kill the enemy. Because of this, time and time again in every war, whomever is the enemy becomes faceless. My mother spoke time and time again about WW2, where in school she was taught, "the Japanese people don't love their children, they don't even have ONE word for love."
This is the separation between murder and killing. The Japanese are no longer familiar, no longer human. (Go back further to find debates during Slavery about if black people were actually human and deserved the same rights as 'humans', same with American Indians. Because they weren't Human, they were fair game)

Further separation between these lines is a very bad thing. Once you can automate the killing, and separate the human from human life it becomes easier to extinguish that life. No longer will you have to see the people, no longer will your brother die. The emotional attachment to that which is close to you is removed. Once its removed, human life becomes a number, and an obstacle to be dealt with, or used.

Re:We're doomed even if it is flawless

[Anonymous Coward]

Treating a human as if they're not human is evil. Ergo, the US Army does evil. All armies do. But remember, in the US Army, every killer is a volunteer! Sleep well, Americans

Why do they have to be autonomus

[Joker1980]

Whats wrong with human controled drones and bot's, after all that would remove human casualties on your side...until the other side develops something similar. Thinking about it two armys of human controlled or autonomous bots would essentially turn war into a video game.

Re:Why do they have to be autonomus

[Bigjeff5]

Yeah, except it's real. People are smart enough to know the difference between real and not-real unless they have been deliberatly duped (then they are only sometimes smart enough).

The difference between real and not-real is huge.

legitimate target?

[CrimsonAvenger]

such as a war zone from which all non-combatents have already fled, so that anybody who shoots at you is a legitimate target.

I've never thought of the people shooting at me as "non-combatants"...

Re:legitimate target?

[Anonymous Coward]

If the non-combatants have left, only combatants are left, and therefore anybody shooting at you is a combatant.

Re:legitimate target?

[Fallingcow]

... but so is anybody not shooting at you.

Re:legitimate target?

[justinlee37]

Isn't anybody shooting at you a combatant by default?

Not Robots

[Roger W Moore]

They aren't robots - there is still a living thing in control. Effectively they are one person tanks.

Re:Not Robots

[SwordsmanLuke]

Meh. It could've been a confused Cyberman.

Sweet

[copponex]

When can we drop one in your backyard?

Re:Sweet

[QuantumG]

Presumably when I start threatening national security.. or at least when your president can convince the least intelligent members of your society that I have.

New meaning

[Roger W Moore]

I suppose that would bring new meaning to "the blue screen of death".

Children

[Beer_Smurf]

What I have wondered is how a robot will respond to children with bedsheets.

what is the fallback mode?

[Joe The Dragon]

what is the fallback mode / data link lost?

crush kill destroy?

It is a bad thing

[Roger W Moore]

And yet, is it fundamentally a bad thing? We give less-than-stable humans that responsibility all the time.

Yes it is fundamentally a very bad thing. First instead of being limited to one trigger that unstable human can now pull hundreds of triggers simultaneously. The robot will never question his orders it will simply comply no matter how morally questionable the order is.

Secondly the one big way in which democracy helps maintain peace is that the people who will do the dieing in any conflict are the ones who also effectively control the government through their votes. If suddenly Western democracies can send robots in then they are far more likely to go to war in the first place which is never a good thing.

That's a relief

[imarsman]

Now we can be sure that robots will never break the rules, just as nowadays phosphorous bombs never get dropped on civilians, nor cluster bombs that in any case, never lay around for years waiting to explode when picked up by a child. Who do these idiots think they are fooling? Rhetorical question, unfortunately; the same people who have been putting up with this sort of BS forever and a day.

A better idea

[MichaelSmith]

Robots on the battle field seem to be designed as extensions of current human operations. They basically shoot at things and try to destroy them.

How about building a hardened robot which can take a lot of punishment. It rolls or walks up to one of the enemy, grabs hold of them and shuts down. That way, the opposition can be disabled with fewer casualties.

Re:A better idea

[Anonymous Coward]

How about removing humans from the battlefield completely, for both sides and replacing them with robots? Why think of a robot attacking a human. Think of robot attacking robots. It'll be fun, not casualties, and yet a war will have been fought. I always thought Robot Wars was boot camp for something similar.. >

Re:A better idea

[Stormwatch]

Who said enemy casualties are a bad thing?

Re:A better idea

[Anonymous Coward]

Who said enemy casualties are a bad thing?

The enemy?

Re:A better idea

[Anonymous Coward]

Enemy or otherwise, the fewer people we have to kill, the better.

Re:A better idea

[knappe duivel]

Who said enemy casualties are a bad thing?

The enemy did, they felt very strong about it.

Re:A better idea

[nyctopterus]

Err, maybe a conscience and a sense of ethics?

Re:A better idea

[Cyberax]

Imagine that, say, China attacks USA using robots.

Still thinking that excessive casualties are OK?

Re:A better idea

[mike2R]

Who said enemy casualties are a bad thing?

General David Petraeus. See here.

A-52. Achieving success means that, particularly late in the campaign, it may be necessary to negotiate with the enemy. Local people supporting the COIN operation know the enemy's leaders. They even may have grown up together. Valid negotiating partners sometimes emerge as the campaign progresses. Again, use close interagency relationships to exploit opportunities to co-opt segments of the enemy. This helps wind down the insurgency without alienating potential local allies who have relatives or friends among insurgents. As an insurgency ends, a defection is better than a surrender, a surrender better than a capture, and a capture better than a kill.

Well duh

[Sycraft-fu]

These are military robots. No military robots would fall under Asimov's list.

What I think some fail to remember is that Asimov was just a science fiction author. He wrote stories. Very compelling ones, his place in modern literature is gigantic, but none the less just fictional stories. Thus his "three laws" have nothing to do with reality. They aren't natural laws, or legal standards, they are just part of a story. Thus they have no standing in the world.

They may well be how Asimov would like to see robots work, they may well be how you'd like to see robots work, however they have nothing to do with how the military wants it to work. They are not a canon of any kind.

When a robot is developed for military purposes, it should be no surprise the ethics are considered in that context. The whole point of it will be to be able to use deadly force if necessary. The programming is then when is that ok and not ok.

So please, let's have all us geeks lay off the Asimov "three laws" when it comes to robots. Every time something like this comes up people start talking about that like it matters to anyone. No, it really doesn't.

Re:Well duh

[sesshomaru]

I am so God-damned sick of the three laws. I want to say to people who take them seriously, "Oh, you read Asimov in high school, good for you." It's been so obvious that this is not the path the human race is on.

Also, for things for people to fixate on, why the three laws? Why not psychohistory? That seemed more fun to me.

Asimov was a cockied-optimist. His is not the world we inhabit.

We don't even inhabit this world, "I'm sorry Dave, I'm afraid I can't do that."

This is the world we inhabit: "you now have 15 seconds to comply!"

Verhoven:1 -- Asimov:0 !

Re:Well duh

[NeutronCowboy]

I think you missed the point of the stories. It's about what happens to robots who are built with the best intentions. Science Fiction is speculative fiction - the proverbial "What if?" He didn't try to predict what was going to happen - he tried to figure out what would happen if certain things were in place.

Verhoven might have been the better prognosticator, but Asimov was the better guide.

Re:Well duh

[Anonymous Coward]

These aren't robots. They're armed unmanned ground vehicles. They are remotely controlled by a human operator at all times and the operator assumes all risk and responsibility. I think once a year an article comes up on /. and everyone argues over laws of robotics. But there's no autonomy here, just a remote control car like a kid has.

Re:Well duh

[Fantom42]

For the record, there was a whole part of my post following the Asimov criteria. It followed the statement, "More importantly,..."

Re:Well duh

[baKanale]

I am so God-damned sick of the three laws.

That's funny, since Asimov invented them because he was sick of the whole "man makes robot, robot rebels/goes crazy, robot kills man" trope and wanted to explore new areas. I guess everything new is bound to become old, and everything fresh and original is bound to become cliche and overused.

Re:Well duh

[ShivSena]

Isn't the point here to figure out the best way to ensure that these robots military or non-military does not turn against their creator(s)? or are you proposing letting loose robots without any safeguards?

Re:Well duh

[drsparkly]

Foundation and Earth, where Golan and his friends land on Solaria, and the Solarian robots' definition of "human" exclude the protagonists. The robots in this situation are quite willing to dispose of them.

You could just class the enemy as "not human". Pretty tricky to do I imagine, where do you draw the line - maybe a Bayesian spam filter :) I'm not going to volunteer to help train that one :)

Disclaimer: Thought experiment only.

Re:Well duh

[Anonymous Coward]

It matters, if you let it matter. And it doesn't matter, if you don't.

Duh!

Why is that informative?

[Sycraft-fu]

You are all aware Terminator is a work of fiction, right? As in made up, not real, etc. Yes, in the event we start having real "thinking computers" as in ones that function like a human brain we are going to have to look at issues that relate to sentience and such. That is not an issue with our current, imperative, mathematical computers. They do as they are programmed to do, nothing more.

Re:Why is that informative?

[grahamd0]

You are all aware Terminator is a work of fiction, right?

Of course.

You don't think that speculative fiction (a category in which the first film for sure, and possibly the second, fall) can help us to resolve some of these issues before they crop up?

Anyway, if I wanted to have a serious debate about it, I have much more compelling and realistic reasons not to support the use of autonomous killing machines than some luddite fear that they will inevitably turn on us. While it's a far more fun argument to make, I don't actually believe it. StreetStealth's comment about being willing to cross that line when the machines do better than us just brought that quote to my mind.

what happens when humans are taken out of the loop

[adaviel]

As I recall, Harry Harrison's short story (SF) http://en.wikipedia.org/wiki/War_with_the_Robots addressed this issue. The underground bunkers used by the opposing generals to control the robot armies became uninhabitable due to enemy action, so they plugged in a robot officer and evacuated. The war continued, even though the humans had stopped fighting, and it was no longer possible to contact the robots in charge.

We need the "What could possibly go wrong tag?"

[hasbeard]

If it fits anything, it fits this topic.

I think it is crazy.....

[AnAdventurer]

to think that the people who bring you war will adhere to some sort of guideline for robots in combat. For example: Currently you can not (illegal) shoot people with a .50 cal M2. I have seen it done though. Rolling in to Baghdad regular units in both the Army and Marines shot at anything that moved along the HWY (NOT all non-coms had "left"). Sight all the reasons you think the two would be different, but it won't change the fact that WAR IS HELL.

Re:I think it is crazy.....

[Hognoxious]

For example: Currently you can not (illegal) shoot people with a .50 cal M2.

[[citation needed]]

Re:I think it is crazy.....

[AnAdventurer]

IANAL but it's in the Geneva Convention.

Re:I think it is crazy.....

[Hognoxious]

No it isn't.

Hmm. . .

[mrgiles]

It's when they start to build the suicide booths that I will start to worry!

So, killed anyone with an axe lately

[Anonymous Coward]

I'm going to A/C this for a coupla reasons. But I (and most psycologists) call bullshit on this

The best analogy I can draw is hamburger. Most people won't think twice munching on a hamburger. Some get a little squeamish at boning a joint. Many people lose their appetite if they have to skin and gut their dinner. Few are happy sticking a pig.

If you're not a psycopath, it's easier to kill if you aren't going to see/smell the blood.

Re:So, killed anyone with an axe lately

[ToasterMonkey]

I'm going to A/C this for a coupla reasons. But I (and most psycologists) call bullshit on this

The best analogy I can draw is hamburger. Most people won't think twice munching on a hamburger. Some get a little squeamish at boning a joint. Many people lose their appetite if they have to skin and gut their dinner. Few are happy sticking a pig.

If you're not a psycopath, it's easier to kill if you aren't going to see/smell the blood.

I think the parent was correct.
Also, hamburger is made from beef, as in cows, not pigs.
Your analogy is crap anyway.
You switched an enemy combatant - someone whose intent is to flat out kill you - with a farm animal. Paaaaahleeaaase.

Compare killing a pissed-off bull bearing down on you with an assault rifle, up close and personal with pushing a button, killing any number of good or bad cattle in a field far away with second, or third hand intelligence passed to you. Either way, you don't escape all the guilt. At least the excuses are better up close: "He was going to kill me" vs. "Well, I hope they all did something to deserve it"

Christ, haven't you pussies read Ender's Game? You all had better just shut up about ethics of robots at war if you can't wrap your head around the realities of war first.

Re:So, killed anyone with an axe lately

[Serious Callers Only]

Christ, haven't you pussies read Ender's Game... the realities of war

Ender's Game is teenage pulp fiction; how exactly does it relate to the realities of war?

It's too late

[CuteSteveJobs]

For all the talk of "Governing Lethal Behavior in Autonomous Robots", Asimov's laws of robotics, etc., it's already far too late. Most Torpedoes and missiles hunt autonomously and have done so for years.

And really, this all depends on perception. One man's fanatical terrorist is another man's freedom fighter is another man's guy-just-defending-his-country. All in the eye of the beholder and difficult enough for people to sort out.

Again: who's writing the software?

[c0lo]

Oh, spare me with Skynet - can't you find other tangents to explore?

Whenever autonomous robots are deployed, humans will still be in the loop, at least legally. If a robot does do something ethically wrong, despite its programming, the software engineer or the builder of the robot will likely be held accountable, says Michael Anderson at Franklin and Marshall University.

Whow! Finally, some software to be released without the typical LIMITED WARRANTY section in EULA?

"Software Engineer held accountable"? This will bring the "Corporate Finger-Pointing games" to a new level of professionalism/competitiveness (with dev/testing in a huge clash).

On another tangent: Open Source anyone?

Rules of war

[jumpinp]

- such as a war zone from which all non-combatents have already fled, so that anybody who shoots at you is a legitimate target............ It's my understanding that anyone who shoots at you is a legitimate target.

Re:Rules of war

[Arker]

It's my understanding that anyone who shoots at you is a legitimate target.

Not if you are someplace where you have no right to be in the first place.

Re:Rules of war

[jumpinp]

It's my understanding that anyone who shoots at you is a legitimate target.

Not if you are someplace where you have no right to be in the first place.

Granted, but now that your there it's kill or be killed. Legitimacy be damned.

Humans have screwed this up for thousands of years

[cenc]

What has fundamentally changed? Humans have been making bad decisions about who to kill in a war for thousands of years. I just hope the fucking robots are better at it than we where.

At least they will solve the problem of the guys with PTSD going postal 10 years after bombing the wrong village and killing even more people.

"Kill all humans"

[incognito84]

The only "ethical code" a robot needs to be AWESOME (I often dream about battling robots, among other things).

Wall-E

[evil_aar0n]

This sounds like Wall-E with an attitude.

Why

[Sybert42]

Why are you ignoring me?

Didn't anybody read the paper?

[Animats]

Didn't anybody read the paper? It's not difficult, although it is somewhat chilling. It's intended to control weapon selection (chaingun, RPG-sized missile, or bunker-buster sized bomb), balancing military necessity (set by the operator) with collateral damage. This info is also used for selecting a firing position (get into a good position to use the chaingun, or just blow up the whole target area?)

Right now, we have weapons that are autonomous after launch, but dumb. This is, in a way, a step up.

Re:Didn't anybody read the paper?

[MasterOfDisaster]

That's actually pretty cool, and not at all scary (at least, no scarier than the bunker buster sized bomb they'd be dropping on the target anyway)

The other 190+ comments with Robocop/Terminator jokes (and the discussion thereof) is, however, far more entertaining.

Re:Didn't anybody read the paper?

[Maximum Prophet]

Right now, we have weapons that are autonomous after launch, but dumb. This is, in a way, a step up.

Once at Purdue University, representatives from the Army War College came to speak to our class. I asked about ICBMs, whether once launched, they could be shut down. I was told no, they had no recall or self destruct. Now, the guy could have been wrong, or the details were secret, but that was scary. If a missile was to be accidentally launch, we just have to wait and watch it blow up some city? Every space launch including the manned missions have self destruct systems on board, yet they think ICBMs are foolproof?

Re:Didn't anybody read the paper?

[TheBracket]

I recall having a similar discussion with a professor during one of my MS Defense & Strategic Studies classes. He stated that there were no 'recall' codes because it was considered too-likely (any likelyhood being too likely, in his opinion) that they would fall into enemy hands - so the system would have a remote 'off' switch, and no-longer deter an opponent who has (or thinks he has!) the right code.

It's an interesting thought experiment. The big upside to a post-launch deactivation system is that after a launch, you can say "oops" and kill the missile - hoping that the other side's launch detection systems haven't triggered a counterforce/countervalue attack in response. The big downsides are: if the 'enemy' gains your deactivation code, they are no longer deterred (at least on the ICBM/ALCM/SLBM/etc. front; I'd imagine manned planes could still fly) and may even see an opportunity to strike. You also risk lowering the level of caution displayed by your own side - knowing you can say "oops" and stop the strike reduces the mental barriers to launching in the first place.

This really was just another level of escalation/deterrent theory head-games that were so popular in the past. I remain convinced that deterrence is a dangerous hope upon which to rely, but that's another story.

Re:Didn't anybody read the paper?

[Animats]

I asked about ICBMs, whether once launched, they could be shut down. I was told no, they had no recall or self destruct.

That's correct. For test launches from Vandenberg, a range safety system is installed in missiles. But it's only useful during the boost phase, when the missile still has line of sight to the launch site.

There's a deterrence issue with this, but the real problem is that there's no way to communicate with the things downrange. Test launches require a whole string of ground stations, with big, steerable dish antennas, along the track of the booster. The "Pacific Missile Range" is such a set of ground stations, going out to Hawaii and beyond. This only works for launches along that line of fire.

So operational missiles don't have either a range safety system or telemetry downlinks.

Re:Didn't anybody read the paper?

[Maximum Prophet]

As long as we had telemetry link on our soil and Hawaii, we be "safe" from an accidental missile launch directed at our own property. (DC or New York).

It just seems like a Bad Idea(tm) to not be able to destruct or redirect ICBMs in flight.

Re:Didn't anybody read the paper?

[Maximum Prophet]

Even if the enemy has the codes, it's too risky that a few bombs will get through, or it's the wrong code. There's no way to test it.

This is similar to the risk of buying a black market Soviet nuclear weapon. The seller gives you the codes to activate it, and runs away. Ok, do you enter those codes in?

From the seller's point of view, it's better if you and the bomb just vaporize, leaving no trail back to him. So if you use the codes he gives you, you are most likely get blown up, nuclear or otherwise. So, you need to dismantle the bomb, verify the components, and rebuild it with your own detonation circuits. It's not like the movies where you'd be able to buy a bomb, and explode it days later in New York.

Consider the alternative to having destruct codes for ICBMs. You've just nuked DC, or LA or New York. The world economy will collapse much worse than 9/11 or the housing meltdown. The country will descend into anarchy, you will be executed when they catch you. Even worse than destroying one of our own cities would be for the bird to fly to it's original destination, like Moscow. WWIII, world destruction, nuclear winter, game over.

Slashdot's Four Laws of Robotics

[PPH]

1. Kill all humans.
2. Kiss my shiny metal ass.
3. ????
4. Profit!

Robot not paying attention in class

[kanweg]

"Rumsfeld T1000, now report to the headmaster for consistently not paying attention!"

Bert

Thanks for the visual. . .

[Hamoohead]

. . .I just had a Robocop flashback!

Get. . .out. . .of. . .my. . .HEAD!!!

Mod this dude up.

[copponex]

Wouldn't surprise me. Something like 90% of the "suspected terrorists" rounded up in Afghanistan were turned in for cash, usually by rival tribes or by the very people attacking them. That's the way the first man we tortured to death was caught, anyway.

Re:Mod this dude up.

[NeutronCowboy]

Weird. The post asking for the parent to be modded up and merely elaborating on the parent's point gets modded up, while the parent sits at 1.

But yes. This is the #1 issue when you interfere in local affairs that go back 100s of years, and where you have no clue what's going on. That's why just storming a random place because bad stuff is going is a bad idea. You have no idea what you're doing, and what the end result will be.

*sigh* Oblig.

[kramulous]

Here.

You are illegally parked

[Anonymous Coward]

The ED209 always worked extremely well.

Tag

[Saffaya]

Where is the ED-209 tag when you need it ?

Using robots is a really baaaad idea....

[mark-t]

If you will forgive the irony that this sounds like, I'd maintain it's a bad idea because it will mean that fewer people get killed. Okay, now that you've probably done a double take on that or thought I made a typo, hear me out:

Because not as many people die, the wars will last longer, ultimately taking a larger toll on society than a war fought with human lives constantly at stake.

And besides, If one isn't willing to risk their own lives to fight for whatever cause was thought to be worth going to war for, how could it possibly be worth waging war in the first place?

so uh

[Anonymous Coward]

in b4 skynet

Subject

[Legion303]

"DROP YOUR WEAPON. YOU HAVE FIVE SECONDS TO COMPLY."

"Uh...hey, this is just a candy bar.'

"YOU HAVE TWO SECONDS TO COMPLY."

"OK, I'm dropping my candy bar! Shit!"

(BLAM)

Test Driven Development

[amiga500]

I'd like to see this book combined with a Pragmatic Programmer's guide to Test Driven Development.

Unit Sycraft-fu please report for reinstall of OS.

[Anonymous Coward]

Your humor circuit is burned out. REPEAT. Your humor circuit is burned out.

(You said TWICE in the comments to this article that something was fictional. Yes, we get that. It's fiction. And you're humorless.)

JoCo will need this ethics guide

[focoma]

From Jonathan Coulton's song, "The Future Soon":

Work through the daytime, spend my nights and weekends
Perfecting my warrior robot race
Building them one laser gun at a time
I will do my best to teach them
About life and what it's worth
I just hope that I can keep them from destroying the Earth

There is no morality in war

[Anonymous Coward]

The Geneva convention simply places a further burden on the enemy. They would be fools to adhere to it if they sincerely wanted to win (save, possibly, for the fact that failure to adhere to it might spur the enemy on to fight even harder).

Rules of engagement of not, if this technology is allowed to exist, it will be abused - and probably already is.

Kill bad people; spare good people

[Archtech]

Didn't The Decider already give us the necessary algorithm?

This will be awesome!!!

[Anonymous Coward]

We (the USA) can soon invade any country we want without those pesky moms bitching at homeland because their son has been killed.

World Domination here we come!

Anyone read any of Keith Laumer's Bolo books?

[Brandon Hume]

To give a quick rundown, Keith Laumer came up with the idea of "Bolos", which are essentially oversized, insanely-armed sentient tanks. There's a number of novels and short stories (I recommend Road to Damascus, which I think might even be available for free on the web) all of varying quality.

The running theme through the novels is the machines exemplifying military virtue, while the humans often screw things up and generally fear their own creations. I find them refreshing because it's nice to read a book where our creations aren't interested in stomping our faces with a metal-shod boot.

It gives some interesting ideas on the autonomous battle-robot theme.

  - A war-bot is not really capable of mercy, which I think is the main point a lot of people are making so far.

  - A war-bot is not really capable of war-CRIMES, either. It's not going to care that you blew up its buddy.

  - The lack of self-preservation instinct can prevent rash action. You can program a robot to TAKE a shot or two before reacting.

  - The other side of the coin is that once the robot decides to engage you, it's on. If you pull a gun in front of a squad of soldiers, accidentally or deliberately, they MIGHT let you have a second or two to reconsider your actions and surrender. With a robot, you're paste.

  - You have to consider the difference between TRULY autonomous robots and robots working in conjunction with humans. I'm talking about the difference between a squad of bots operating alone and bots assisting/protecting human soldiers. I'd call the latter far more dangerous, and not because I'm worried about friendly fire. A squad of humans and their bots patrolling a nasty neighbourhood in Baghdad or Kandahar is going to WANT their metal buddies dialed up to "Paranoid" so that they can react to suicide bombers fast enough to avoid getting hurt themselves. That combination can do far more damage than a squad of humans or of robots deployed alone. You get all the bad decision-making of humans combined with the threat-reaction and firepower of machines.

It should probably be mentioned that we already have an example of a semi-autonomous battle robot... the Phalanx CIWS. Flaws with it aside, that's an example of a machine that decides when and what to kill. I even got the impression from a Navy guy I knew that sometimes the people it protected were a bit scared of it... if you wandered into its engagement envelope you were dead, no ifs or maybes. A Slashdotter familiar with the thing might have some interesting perspective.

I apologize for the semi-rambling of my post. It's early.

Homeland Security

[secondhand_Buddah]

Right now around the world metropolis's have large camera networks. The next step, that is currently being worked on is real time facial recognition, and tracking. This means that in the near future, in real time camera networks will autonomously be able to pick up and track people.

Add security robots that can get deployed automatically to this mix, and suddenly a rather dark picture emerges of how easily we will be able to be controlled in the near future. And I don't like it.

non-combatent firing?

[CrashandDie]

relatively uncomplicated situations â" such as a war zone from which all non-combatents have already fled, so that anybody who shoots at you is a legitimate target.

Surely, anyone who shoots at you would not be categorised as "non-combatent", making the whole point useless?

How is a robot returning fire without the certainty that everyone in shooting range isn't a bad guy different from a soldier returning fire with the same limited amount of information?

Post-scarcity tech in hands of scarcity mindset

[Paul Fernhout]

Like nuclear technology, or biotech, or other advanced technologies which all function as amplifiers for the human mind, robotics have the potential to make the world work well for most everyone. But it requires using them in a post-scarcity way, fostering abundance for all through cooperation. Instead, often people think of these technologies from a mindset of scarcity, and so use them to competitive way to defend a hoard of privilege.

Einstein said: "The release of atom power has changed everything except our way of thinking...the solution to this problem lies in the heart of mankind. If only I had known, I should have become a watchmaker."

More on this theme:
http://www.pdfernhout.net/post-scarcity-princeton.html

Re:Post-scarcity tech in hands of scarcity mindset

[Paul Fernhout]

More on this theme from the perspective of open vs. proprietary technology:

http://groups.google.com/group/virgle/msg/de1a99ede7e0e615?

== what have funding policies in automotive intelligence wrought? ===

Consider again the self-driving cars mentioned earlier which now cruise
some streets in small numbers. The software "intelligence" doing the
driving was primarily developed by public money given to universities,
which generally own the copyrights and patents as the contractors.
Obviously there are related scientific publications, but in practice
these fail to do justice to the complexity of such systems. The truest
physical representation of the knowledge learned by such work is the
codebase plus email discussions of it (plus what developers carry in
their heads).

We are about to see the emergence of companies licensing that publicly
funded software and selling modified versions of such software as
proprietary products. There will eventually be hundreds or thousands of
paid automotive software engineers working on such software no matter
how it is funded, because there will be great value in having such
self-driving vehicles given the result of America's horrendous urban
planning policies leaving the car as generally the most efficient means
of transport in the suburb. The question is, will the results of the
work be open for inspection and contribution by the public? Essentially,
will those engineers and their employers be "owners" of the software, or
will they instead be "stewards" of a larger free and open community
development process?

Open source software is typically eventually of much higher quality
http://www.fsf.org/software/reliability.html
and reliability because more eyes look over the code for problems and
more voices contribute to adding innovative solutions. About 35,000
Americans are killed every year in driving fatalities, and hundreds of
thousands more are seriously injured. Should the software that keeps
people safe on roads, and which has already been created primarily with
public funds, not also be kept under continuous public scrutiny?

Without concerted action, such software will likely be kept proprietary
because that will be more profitable sooner to the people who get in
early, and will fit into conventional expectations of business as usual.
It will likely end up being available for inspection and testing at best
to a few government employees under non-disclosure agreements. We are
talking about an entire publicly funded infrastructure about to
disappear from the public radar screen. There is something deeply wrong
here.

And while it is true many planes like the 757 can fly themselves already
for most of their journey, and their software is probably mostly
proprietary, the software involved in driving is potentially far more
complex as it requires visual recognition of cues in a more complex
environment full of many more unpredictable agents operating on much
faster timescales. Also, automotive intelligence will touch all of our
lives on a daily basis, where as aircraft intelligence can be generally
avoided in daily life.

Decisions on how this public intellectual property related to automotive
intelligence will be handled will affect the health and safety of every
American and later everyone in any developed country. Either way, the
automotive software engineers and their employers will do well
financially (for example, one might still buy a Volvo because their
software engineers are better and they do more thorough testing of
configurations). But which way will the public be better off:
* totally dependent on proprietary intelligences under the hoods of
their cars which they have no way of understanding, or instead
* with ways to verify what those intell

God is this discussion STUPID!

[Hurricane78]

Repeat after me: There are no robots, doing decisions!
Robots are fully controlled by programs. Those programs got written by humans
And those humans are the ones who act!
The fact that the tool is detached from the human, and it it pre-planned for a long period, does not make it self-thinking!

The only self-thinking ones would be those that have neural networks and learn by themselves. And I don't mean pre-trained to a target behavior and then locked on that. (Which essentially is just a way to implement a deterministic function.)
Those really self-thinking robots are still waaay ahead. And when I see even the human soldiers being trained not to think for themselves, but to blindly follow orders, I don't think they will ever exist in military.

Don't shoot the friendlies

[Maximum Prophet]

so that anybody who shoots at you is a legitimate target.

The hell you say. Just because your own troops start shooting at you, doesn't mean you should shoot back. You *should* get on the radio and tell them to stop shooting.

Friendly Fire

[Anonymous Coward]

How are the programmers going to deal with friendly fire?

A significant number of casualties in any conflict are the result of a force accidentally firing on their own combatants.

What if the automaton is subject to friendly fire. How will it respond?

Make lethal force by autonomous robots illegal

[ibirman]

We need laws and treaties against allowing autonomous robots use lethal force. If we don't put a stop to it now, it will escalate until we have something like the Terminator. This can and will become much worse than nuclear proliferation. The world should get together and sign treaties now.

Obligatory

[PNP_Transistor]

I, for one, welcome our new and completely ethical robot overlords!

We need a guide to ethics for human policy makers

[Anonymous Coward]

The ethics of drones make no difference if they are ordered by humans to drop bombs on civilian centers in hopes of killing a few terrorists (ie current US military actions in Pakistan).

replace fiction with nonfiction

[Anonymous Coward]

May this forthcoming book please supplant pot boiler fiction as the basis for Slashdot commentary.

guidelines on remote payloads exist. use those

[Anonymous Coward]

There are most likely already guidelines in place for the use of 'remote payloads', which is what this gentleman means by 'robot'.

What do West Point and the other academies say in their morals classes concerning such things?
I will trust an American Military officer to make such a decision, not some hair-brained algorithm of a professor shilling his book.

Taking the people out of the loop is unethical. The developer of such software is a maniac if he thinks it could ever be moral to use it.
All such 'decision making' for killing is never ethical, always immoral. A real person must be the decision maker on any such hard choice to harm another person. The deployment of such payloads with such killing-software is akin to a war crime. Is this guy so ethically challenged that he doesn't realize this? He sounds like a very cerebral person who has divorced himself from the real world of what killing really is.
Someone should talk to him and find out who he thinks will be paying him for this software . . .

Not Terminator, but RoboCop

[Rick Genter]

I noticed this article is tagged "terminator" and "skynet", but the first thing that went through my mind was ED-209 from RoboCop:

(Shamelessly copied from imdb.com)

[Mr. Kinney points a pistol at ED-209]
ED-209: [menacingly] Please put down your weapon. You have 20 seconds to comply.
Dick Jones: I think you better do as he says, Mr. Kinney.
[Mr. Kinney drops the pistol on the floor]
Dick Jones: [ED-209 advances, growling]
ED-209: You now have 15 seconds to comply.
[Mr. Kinney turns to Dick Jones, who looks nervous]
ED-209: You are in direct violation of Penal Code 1.13, Section 9.
[Entire room of people in full panic trying to stay out of the line of fire, especially Mr. Kinney]
ED-209: You have 5 seconds to comply.
Kinney: Help me!
ED-209: Four... three... two... one... I am now authorized to use physical force!
[ED-209 opens fire and shreds Mr. Kinney]

Weakness of the work.

[Anonymous Coward]

I met Arkin at last year's NA-CAP conference. It came to my attention that the greatest weakness in his work was the unwillingness to have the robots abide by the Nuremberg principle. Namely, that soldiers have a right and a duty to disobey an unlawful order (e.g. targetting civilians). Of course, implementing such a thing requires implementing the total context of the situation, since an unlawful war makes every action within it unlawful, etc.

danger!

[blad3runn69]

danger will robinson danger!

worst science ever

[Anonymous Coward]

sure, every scientific progress has been used for warfare in one or another way.
but just thinking of letting machines decide about live and death of human beings is worse than any other weapon ever invented.
shame on those funding that research and even more shame on those who take that money and use their creativity and brains on enhancing the slaughter of men.
HAVE YOU LOST YOUR MINDS?

--
a simple question to those inventing/using/building weapons:

DO YOU WANT YOURSELF OR YOUR CHILD/PARTNER/FRIEND TO BE KILLED BY ONE OF THOSE WEAPONS?

if your answer is yes, please use the weapon on yourself directly!

if your answer is no, then why do you invent/use/build those weapons???
and please dont tell me "to protect my beloved ..." cause you should know that a+a is 2a and not 0 (where a may stand for agression)

START THINKING!