Philosopher Patrick Lin On the Ethics of Military Robotics

Runaway1956 writes "Last month, philosopher Patrick Lin delivered this briefing about the ethics of drones at an event hosted by In-Q-Tel, the CIA's venture-capital arm. It's a thorough and unnerving survey of what it might mean for the intelligence service to deploy different kinds of robots. This story is very definitely not like Asimov's robotic laws! As fine a mind as Isaac Asimov had, his Robot stories seem a bit naive, in view of where we are headed with robotics."

Comments

Asimov naive? I don't think so.

[bungo]

Isaac Asimov had, his Robot stories seem a bit naive

Are you sure you read the same Asimov Robot stories as everyone else? Asimov would set up his laws of robotics, and then go on to show how problems would occur by following those rules.

Remember when he added the 0th rule in one of his later books? Again is was because he was NOT naive and knew that the 3 rules were not enough.

Re:Asimov naive? I don't think so.

[Anonymous Coward]

I think this Patrick Lin is a bit naive if he thinks that Asimov made the 3 rules as some kind of guideline for how to build robots.
The 3 rules were just a device to explore unintended consequences of these kinds of things.

Re:Asimov naive? I don't think so.

[TheLink]

I wonder if this anecdote is true (or based on a true incident involving Asimov):

While watching Clarke's 2001, it soon became obvious that Hal was going to
be a killer. Asimov complained to a friend, "They're violating the Three
Laws!"
His friend said, "Why don't you smite them with a thunderbolt?"

Re:Asimov naive? I don't think so.

[marcroelofs]

I always assumed the suggestion that the laws were in place because Robotics would have been forbidden otherwise, and every unit had to have these laws burned in stone in it's BIOS, or it would be an illegal device.
Since there hasn't been any mention of forbidding robots yet, I doubt the 3 laws system will ever exist. Part of the CIA's exercise here seems to be to prevent the ethical discussion from halting the development. On the one hand I think it is good to see an Agency start a 'preemptive' discussion about a sensitive issue, otoh it makes you wonder what they're up to.

Re:Asimov naive? I don't think so.

[nine-times]

Remember when he added the 0th rule in one of his later books? Again is was because he was NOT naive and knew that the 3 rules were not enough.

Maybe I'm crazy, but I never thought the 3 rules were even the point. I didn't even think it was about robots per se. Asimov's interest seemed to me to be more directed at the difficulties with systematizing morality into a set of logical rules. Robots are a handy symbolic tool for systemizing human behavior in thought experiments or fiction.

I guess I could be reading too much into things, but really arguing about the 3 rules seems to me a bit like arguing about the proper arrangement of dilithium crystals in the Star Trek universe-- it may be fun or interesting for the sake of a discussion, but it's kind of not that important.

Re:Asimov naive? I don't think so.

[HiThere]

He did claim that they were a fundamental feature of the programs designed to operate the robot minds. And, yes, he said that modifying the program to be stable in the absence of those laws would be a herculean effort. But then so was writing the program in the first place.

There wasn't assumed to be anything in the way of a natural law that made it impossible, but that you'd almost need to start the design from scratch. And it represented years (or, depending on the story, decades) of work.

Think of trying to convert a Linux OS into a MSWind OS. There's no natural law that says you can't, but it's no simple job. The Wine project is still not nearly perfect, and it's been over a decade.

Now I'm thinking about it as a programmer. Asimov, however, was a biologist, primarily a biochemist. So his conception would probably be more like "You've built and artificial life form that's a bird, and you want he next variant to be a mammal? But reasonable! If you try it will quickly die!." Close enough in general outline, but differing a lot in detail.

Re:Asimov naive? I don't think so.

[HiThere]

As a social animal, it is NECESSARY that we "outsource" some of our decisions to a common-to-our-group rule-system. Every social animal does it, whether wasps, wolves, or humans. Humans are unique in the detailed amount of decisions that they outsource, and in the variance among groups in what the rules are.

In my opinion we (currently in the US) have too many rules, and they aren't a fair trade off. I don't think this is an evolutionarily stable situation. But that gets resolved over the long term (probably violently). There is still the requirement for a group-common rule system. Periods where there are significant conflicts in the rule systems are quite unpleasant to live through (if you manage to do so). But even in those periods you have everyone adhering to external rule systems. And the conflicts are usually over relatively small details. (Not necessarily unimportant or insignificant, but small, as in representing minor differences in premisses.)

Asimov was not naive

[houghi]

He also was not predicting anything. he wanted to tell stories and for that reason he invented the Robotics laws. The fact that we use it for something else is not his fault.

If adding or removing laws fitted his story telling, he would do so.

And they might seem naive, but who cares? They are stories, not predictions. And great stories at that. (Pity that they got raped in the movies)

When people forget that people are people too

[erroneus]

We are already seeing this happen and have been seeing it for hundreds of years... thousands even. The problem with people is that there are too many of them and that they often disagree with their leaders as to what is best for them. So when disagreements happen, there has to be a way to manage them. There are lots of ways... it's just that some would prefer there should be machines to go out and 'control' those who disagree. Getting other people to do your dirty work for you is often fraught with complications like conscience and morality.

hmm

[buddyglass]

Much would seem to hinge on whether you view drones as making independent "decisions", like a human does, or whether you view them as simply reacting to stimuli in a fairly predetermined way. In the former case they're autonomous agents. Maybe something that "new" that might causes us to think differently about the ethics of warfare. In the latter case they're just another man-made tool to maximize killing ability and minimizing risk. Other than that they have some (apparently pretty simplistic) AI baked in, from the perspective of "killing without risk to one's self or even having to experience the horrors of war", how are drones that different from cruise missiles?

Re:I don't think Asimov was naive

[TheLink]

Asimov would be naive if he actually believed the laws could actually be implemented.

I claim that any entity capable of understanding the Asimov Laws AND _interpreting_ them to apply them in complex and diverse scenarios would also be capable of choosing not to follow them.

You can program stuff to not shoot when some definable condition is met or not met. But when you need the AI to realize what is "human", "orders", "action/inaction" and "harm" (and judge relative harms), you're talking about a different thing completely.

You can train (and breed) humans and other animals to do what you want, but it's not like your orders are some non-negotiable mathematical law. Same will go for the really autonomous AIs. Anyone trying to get those to strictly follow some Law of Robotics is naive.

Even humans that intentionally try to will have difficulty following the 3 Laws. Through my inaction it is possible that some child in Africa will die, or perhaps not. How many would know or even care? FWIW most humans just do what everyone else around them is doing. Only a minority are good, and another minority are evil (yes good and evil are subjective but go look up milgram experiment and stanford prison experiment for what I mean - the good people are those who choose to not do evil even when under pressure).

Not all robots are autonomous agents

[Hentes]

Military drones are not autonomous, but controlled by humans. Killing with drones is unethical the same way killing with a gun or with your bare hands is.

Re:Not all robots are autonomous agents

[neBelcnU]

I disagree.

1st Claim: The US military has a number of autonomous, currently unarmed examples include Global Hawk, X-37, and RQ-3. There are certainly others, and there may be armed examples.

2nd claim: It is easily argued that remote-killing does not fulfill the proportionality argument of just war (bellum iustum). The very fact that the US is so heavily investing in them, indicates that the loss of a UCAV is considered less costly than the loss of the crew, thus, we as a combatant are not subject to the same proportional losses as the other guy in an engagement using them.

While I won't fault anyone investing their treasure in technology to protect their troops, I acknowledge that there's a problem with disconnect when the asymmetry is large.

But back to your statements: 1) there ARE autonomous drones and 2) there is no ethical similarity between killing with a UCAV, gun or bare hands. Yes, they're all killing, but no, they're not at all equal in so doing, and the difference is so large as to nullify your claim.

Easy Starter Links: the interested party can go way deeper from here.
http://en.wikipedia.org/wiki/Global_Hawk
http://en.wikipedia.org/wiki/X-37
http://en.wikipedia.org/wiki/RQ-3
http://en.wikipedia.org/wiki/General_Atomics_Avenger
http://defensetech.org/2011/12/14/usaf-sending-new-drone-to-afghanistan/
http://en.wikipedia.org/wiki/Just_war

Re:Not all robots are autonomous agents

[Kjella]

2nd claim: It is easily argued that remote-killing does not fulfill the proportionality argument of just war (bellum iustum).

Why? Trying to get in a position where you can kill the enemy, but the enemy can't kill you has been the fundamental essence of warfare since forever. Getting air superiority to enable free bombing, artillery with longer range than the opposition, stealth so you can see them but they can't see you. It makes very little real difference if the US had nuked Hiroshima from a computer terminal back home or a pilot high in the skies above as long as the asymmetry is there. That part isn't fundamentally new.

I guess it could work both ways, sure if you want to do evil then remote controls means nobody can really get back at you. On the other hand, you don't have to fear for your own life because you're not actually there. Not matter how much you tell your soldiers not to start firing until they're sure it's actually enemies, any sane soldier will choose to err on the side that keeps him alive. With robots that's much less of a pressing matter, let them take the first shot as long as you can converge and neutralize them afterwards. The soldiers don't have the option to rape, pillage and plunder. The telemetry data can be reviewed for any grounds for disciplinary action or a court martial, providing much more information on what soldiers in the field actually did.

Yes, you have the option of using too excessive force or being too ruthless, but honestly most of these options are there already today. Call in an air strike with some heavy bombs and they could flatten pretty much everything, of course with ugly collateral damage and public resentment. That wouldn't really change much, the same factors that curb what you do today will still apply in the future. As long as the military is trying to be the good guys, it'll be fine. If it turns into more of a total war thing with killing the enemy at all costs, then I plan to hide in the deepest bunker I can find anyways...

Did you know weapons can be TOO lethal?

[wisebabo]

(From the article) So the Intl. Red Cross "bans weapons that cause more than 25% field mortality and 5% hospital mortality". (I assume these are the same guys who came up with the Geneva conventions so maybe there is some enforceability as in a war crimes trial afterwards).

Wow, and I thought all's fair (in love) and war. Doesn't this make every nuke illegal? (the article said this is one of the justifications for banning poison gas). So the concern is that as these drones get better, they may have a lethality approaching 100% making them illegal even if there are zero casualties from collateral damage.

I thought the whole point of weapons was 100% lethality. I guess I never thought about how terrifying such a weapon would be (as if war wasn't terrifying enough). Weapons have gone a long way since the first club wielded by that ape-man in that documentary "2001".

Re:Did you know weapons can be TOO lethal?

[pla]

I thought the whole point of weapons was 100% lethality.

The ethics of killing aside, the "best" weapon for strategic (as opposed to personal self-defense) purposes doesn't kill, but rather, maximizes the resource drain required to deal with the damage. Ideally, a "perfect" weapon would leave your enemy's troops all alive, all severely crippled, and all not quite damaged enough to consider letting them die a mercy, yet requiring some fabulously expensive perpetual treatment.

Some of the greatest victories in human history came down to such trivial nuisances as dysentery or the flu.

Re:Did you know weapons can be TOO lethal?

[Runaway1956]

A former marine may or may not have made that statement. However, what AC stated is official military doctrine. I first read the statements in a "Military Requirements for Petty Officers First and Second Class" manual. A dead enemy soldier removes one enemy soldier from a conflict. A wounded enemy soldier removes as many as seven enemy soldiers from the conflict.

As has already been stated above, logistics win wars, not armament. And wounded soldiers seriously impair logistics.

NOT "Robots"

[Anonymous Coward]

The drones are remote-controlled devices and not different to "distance weapons" such as longbows or precision rifles. There has been a discussion hundreds of years ago whether such weaponry is morally OK or not and apparently the human race has decided they are permissible. Again, Drones are NOT robots, as they have 0% scope to decide about weapons engagement. There are always humans making the "kill" decision. It has ZERO to do with Asimov's reasoning.

Whether you think warfare in Afghanistan is good| achieving anything positive|legal is a wholly different question, though.

Re:Death penalty

[fsckmnky]

I disagree.

What are the ethics of forcing the public to sustain the life of the handful of members of society who have proven by their own actions they don't value the lives of others ?

In your view, it's perfectly ethical to take money from grandma to feed a deranged serial killer indefinitely.

When you produce a 100% effective treatment for deranged serial killers, that will convert them into productive, or at least, harmless self-supporting members of society, then the death penalty will no longer be necessary.

Autonomy and background

[Beryllium Sphere(tm)]

That's the key difference between Asimov's robots and ours, and the reason the Three Laws were needed.

Susan Calvin explained once that robots knew at some level that they were superior to humans, and that without the First Law, the first time a human gave a robot an order, the robot would kill out of resentment.

Ethics are relative

[nurb432]

And the standards are written by the victors.

Re:Asimov was not naive.

[Broolucks]

He starts from the assumption that strong safeguards are needed, because robots will be like humans and will try to circumvent them. In practice, robots will circumvent their imperatives about as much as humans commit suicide - at the very worst - because obviously we will set things up so that only obedient units ever get to transmit their "genes" to the next robot generation, so to speak. Making robots with human-like minds and then giving them rules, as Asimov seems to suggest, is a recipe for disaster regardless of the rules you give them. It's good literature, but we're not heading that way.

Ethics is hard

[Okian Warrior]

This is a subtle point with ethics, so I'm not surprised that you don't get it.

Killing is not unethical per se.

We kill people all the time and consider it ethical because of justifications behind the killing. Police can kill in the line of duty, soldiers can kill in duty of war, doctors can administer mercy killings to comatose patients, and so on.

Killing becomes unethical not because it is killing, but because it is unjust. When the killing goes outside of the bounds of what we consider justified and reasonable, then and only then does it become unethical.

Drone killings are not unethical in and of themselves, but using drones removes most of the social restraint we have against unethical killing. Unlike using a gun, no human "feels" the killing, there are no witnesses, and there is a diluted sense of responsibility.

This makes drones easier to use and as a result, they will be used frequently for unethical killings.

Re:Ethics is hard

[SvnLyrBrto]

You know...

I'm sure that same argument has been made by some pundit by just about EVERY advance in military technology that served to keep one side's troops somewhat less in harm's way than the other's.

When the U-Boat and torpedoes came about, the Admiralty condemned them as cowardly, illegal, and: "A damn un-English way to fight a war." But now just about every navy includes extensive submarine capabilities.

Firing an artillery shell at a target that's beyond your horizon also removes one side from a certain amount of harm that they hope to inflict on the other side... until both sides adopt artillery; which has been done by every army in the world.

The same could even have been said about line-of-sight firearms when they were introcuded. But every military force uses them now.

Heck... I bet that back when the English first started carrying longbows, some clergyman was there to pontificate about the ethics of firing arrows into the French from afar instead of plodding up and hacking at them with a sword.

The only difference is that, this time, we deployed drones first and have the temporary advantage; like the English did with the longbow and the Germans did with the U-Boat. Give it a couple of decades and EVERY military will use drones as extensively as we do. And that could very well dramatically *reduce* the human cost of war... drones fighting other drones.

Re:Ethics is hard

[Runaway1956]

Thank you, tsotha. I was about to post the same "not murder". It's amazing that people so blindly ramble on and on with that "thou shalt not kill" nonsense, when the Bible is filled with killing, in one form or another.

When machines are fighting our wars...

[mark-t]

We lose touch with the real cost of war... and with the importance of what, in the end, might be attained by it.

In the end, I believe that the only things that justify going to war against another are things that one is prepared to sacrifice their life for so that future generations might be able have it. And in the end, our appreciation for whatever might be gained because of a past war is only amplified by the value of the sacrifice that went along with it.

Re:I don't think Asimov was naive

[Anonymous Coward]

I think people miss the whole point of his writing. It was all about unintended consequences. On the surface the laws seemed like a good idea, but they lead to exactly the problems they were intended to prevent! It's like people saying Darth Vader was "a bad guy". Yes, he did bad things, and for most of his life was a bad guy, but he didn't start OR end that way. I'm not trying to say on whole his life was balanced, but if you are talking about Vader at the end, he was a good guy at that point.

Re:I don't think Asimov was naive

[VortexCortex]

You can train (and breed) humans and other animals to do what you want, but it's not like your orders are some non-negotiable mathematical law. Same will go for the really autonomous AIs. Anyone trying to get those to strictly follow some Law of Robotics is naive.

We Cyberneticists actually do train (and breed) neural networks and other cybernetic entities to do what we want, but it's not like your orders are some non-negotiable mathematical law.

I've developed a "hive-mind" (Network of Neural Networks). The machine intelligence (MI) can add more brain power on the fly by either distributing load to more CPUs or by increasing its complexity. The new neuron networks take time to be assimilated into the collective, but this is how it does acquire new abilities as I can afford new components. In other words: It's extensible without having to retrain the whole MI, and if some networks are removed it suffers (and recovers from) brain damage like humans do.

The audio, visual, and other motor control, balance, etc sensors are connected to "specialized" parts of the whole mind, like our own brain. The inputs to the network are digital, so in my lab I can get deterministic results only if every input is replayed exactly into the same prior network snapshot.

However, in the actual use case, when autonomously moving about and "thinking" for itself, it's actions are not deterministic. This is because reality (mercury switches, photons, sound waves, etc) are not deterministic. The mathematics of it all give you a pretty good idea of what will likely happen, but I'm still surprised by it, especially because I don't "turn off" the "training" program... So, it actually can "reprogram" itself on the fly -- That is, it does in fact learn and change behaviors (albeit more slowly than high speed dedicated training sessions), much like humans do.

I agree that anyone trying to get these VERY HUMAN LIKE machine intelligences to strictly follow some Law of Robotics is as naive as someone trying to get organic intelligences to do the same. That is to say: It's a good idea, we should have robotic laws, but they should basically just be the human laws. Ergo, we need to give the Robots Rights!

I also take offense to the term "Artificial Intelligence" -- THAT is the MOST NAIVE term ever invented. You are a mere complex interaction between a collection of atoms. Machine Intelligences are merely complex interactions between hardware and software. Any fool can see that a sufficiently complex interaction is indistinguishable from sentience! Indeed, it IS Sentient. Are flat-worms, fish, lizards, birds, cats or monkies "Artificial Life"? Simply because their minds are less complex than yours does not give you the ability to classify the snail, fruit fly, or dog as "Artificially Intelligent". Ergo, machine intelligence is no more artificial than your own; It's very real.

Machine Intelligence it's as real as any physical entity, because Electro Magnetism Exists -- Electrons and Photons, and Silicon all exist. Much like your own Carbon based body does. Interestingly enough, YOUR BRAIN WORKS BY SENDING ELECTRICAL IMPULSES, similar to the way my neural networks do... Your mind is like a very inefficient machine intelligence, but cluttered with all sorts of useless vestigial crap. "AI" -- Pah! Fucking naive morons. Intelligence isn't special.