Slashdot Mirror
US Navy Wants Smart Robots With Morals, Ethics
coondoggie writes: "The U.S. Office of Naval Research this week offered a $7.5m grant to university researchers to develop robots with autonomous moral reasoning ability. While the idea of robots making their own ethical decisions smacks of SkyNet — the science-fiction artificial intelligence system featured prominently in the Terminator films — the Navy says that it envisions such systems having extensive use in first-response, search-and-rescue missions, or medical applications. One possible scenario: 'A robot medic responsible for helping wounded soldiers is ordered to transport urgently needed medication to a nearby field hospital. En route, it encounters a Marine with a fractured leg. Should the robot abort the mission to assist the injured? Will it? If the machine stops, a new set of questions arises. The robot assesses the soldier’s physical state and determines that unless it applies traction, internal bleeding in the soldier's thigh could prove fatal. However, applying traction will cause intense pain. Is the robot morally permitted to cause the soldier pain, even if it’s for the soldier’s well-being?'"
If the enemy is more injured, should it switch sides and help them instead?
No they dont.
If anything, they want such a robot with a particular set of morals and ethics. If it really would have morals and ethics it would refuse to kill humans, terrorists or not, that have no chance to defend themselves against such a machine.
But than again, I think of drone attacks (by people who, sitting in their comfy chairs far, far away, are not exposed to any kind of risk) as even more cowardice as the acts of snipers picking off unsuspecting targets.
Imagine us trying to teach a robot morality when humans have little agreement on what is moral. For example would a moral robot have refused to function in the Vietnam War? Would a drone take out an enemy in Somalia knowing that that terrorist was a US citizen? How many innocent deaths are permissible if a valuable target can be destroyed? If a robot acts as a fair player could it use high tech weapons against an enemy that had only rifles that were made prior to WWII? If many troops are injured should a medical robot save two enemy or one US soldier who will take all of the robot's attention and time? When it comes to moral issues and behaviors there are often no points of agreement by humans so just how does one program a robot to deal with moral conflicts?
US armed forces should want leaders with morals and ethics, instead of the usual bunch that send them to die based on lies (I'm looking at you Chenney, you bastard).
`echo $[0x853204FA81]|tr 0-9 ionbsdeaml`@gmail.com
Every single one comes down to "do I value rule X or rule Y more highly?" Who gives a shit. Morals are things we've created ourselves, you can't dig them up or pluck them off trees, so it all comes down to opinion, and opinions are like assholes: everyone's asshole is a product of the culture it grew up in.
This is going to come down to a committee deciding how a robot should respond in which situation, and depending on who on the committee has the most clout it's going to implement a system of ethics that already exists, whether it's utilitarianism, virtue ethics, Christianity, Taoism, whatever.
If they are talking about the moral of the US government, I rather have the robots from Terminator.
And they are talking about helping wounded soldiers. Why talk about the (US) marine with the broken leg? What about the injured Al-Quaida fighter?
The question of causing pain for the better wellbeing of the patient is obvious for most people. What if it means killing 1 person to save 10? What if that one person is not an enemy?
What if it realizes that killing 5% of the US population would save the rest of the world? What if that 5% is mostly children? Even if you can answer that as a human being, would you want it enforced by robots?
Don't fight for your country, if your country does not fight for you.
It would be great if they could develop a politician with morals and ethics........but I doubt even the Pentagon's budget would be big enough...........
"The average reporter we talk to is 27 years old......They literally know nothing." - Ben Rhodes
If they calculate that you can't be helped and must be left to die, just say, "Sorry, I've been given specific orders to do X, so I can't help you."
All of this 'ethical debate' surrounding robots that can make life-or-death decisions has absolutely nothing to do with technology, or AI, or any issue that can be resolved technically at all. All it boils down to, is that people are mad that they can't hurt a robot that has hurt them. See, before machine intelligence we had a pretty sweet system. When a human being commits a crime, we stick them in prison. It doesn't feel good to be in prison, therefore this is "justice." But until robots can feel pain or fear or have a self-preservation instinct, prison (or, hell, even the death sentence) wouldn't affect them at all. And that's what drives people nuts. That technology has shown us that beings can exist that are smart enough to make life-or-death decisions, but lack the concept of pain or suffering and if they do something bad there's no way we can PUNISH them.
A fool and his hard drive are soon parted.
Given how "moral" and "just" US govt is when pursuing such atrocities as Obama's drone campaign, funding and arming islamist fundamentalists in Syria, supporting and funding neo-nazis in Ukraine, murdering millions of people all over Middle East etc., I'd rather have anything but military robots with US government "ethics" onboard. They just want fully autonomous killing machines without human conscience standing in the way. Maybe they're running out of drone operators eager to blindly follow murdering orders, so cowardly psychopaths from Washington want machines to take orders to kill and don't want anyone to see results of their actions, not even soldiers on the ground or drone operators.
You see, if some sick fuck in Armani suit wants to kill somebody standing in his way (or corporation employing said crook), he can now fill the form in his nice office, that will be processed as ordinary "business process" and will end up being automatically loaded into some drone flying above his target. No messy pictures, no one complaining, just some fancy form tapped into computer and voila, problem solved !
I can't find words how disgusting it is.
Allowing robots to determine the most efficient way to save as many lives as possible could be dangerous. Maybe they'll decide that you need to be killed, so that two of your enemies can survive.
Whats the point of re-inventing the human to that level. If the robot has to be so self ware as to be moral and know compute ethics, then it starts a new debate of ethics = should we humans be ready to sacrifice/put in risk our couterparts which are so self-aware? You will only complicate stuff .. I guess PETOR = People for Ethical Treatement Of Robots will form even before the first prototype.
Also, even if you think practically, if you can have robots which are so self aware, why have other sodiers at all?!
The chassis is the hard part, not the ethics. The ethics are dead simple. This doesn't even require a neural net. Weighted decision trees are so stupidly easy to program AIs that we are already using them in video games.
To build the AI I'll just train OpenCV to pattern match wounded soldiers in a pixel field. Weight "help wounded" above "navigate to next waypoint", aaaaand, Done. You can even have an "top priority" version of each command in case you need it to ignore the wounded to deliver evacuation orders, or whatever: "navigate to next waypoint, at any cost". Protip: This is why you should be against unmanned robotics (drones): We already have the tech to replace the human pilots and machine ethics circuits can be overridden. Soldiers will not typically massacre their own people, but automated drone AI will. Even if you could impart human level sentience to these machines, there's no way to prevent your overlords from inserting a dumb fall-back mode with instructions like: Kill all Humans. I call it "Red Dress Syndrome" after the girl in the red dress in The Matrix.
We've been doing "ethics" like this for decades. Ethics are just a special case of weighted priority systems. That's not even remotely difficult. What's difficult is getting the AI to identify entity patterns on its own, learn what actions are appropriate, and come up with its own prioritized plan of action. Following orders is a solved problem, even with contingency logic. I hate to say it, but folks sound like idiots when they discuss machine intelligence nowadays. Actually, that's a lie. I love pointing out when humans are blithering idiots.
Navy says that it envisions such systems having extensive use in first-response, search-and-rescue missions, or medical applications.
Just like drones were first used for intelligence gathering, search and rescue and communications relays.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
Notably missing from the article is of course the question "Should the robot attempt to communicate its intentions to the injured, and change its decision on the basis of the response it receives"? Responsively communicating with people other than through a keyboard and ethernet port is the key bridge to gap before giving machines this kind of autonomy, and it's one that neither back-room military techies nor Policy makers seem to have quite grasped yet.
Myu:
Most recently, check out the May 15 Colbert Report. He skewers the concept of military morality pretty well.
Then, take a trip in the wayback machine to another machine-orchestrated conflict .
https://app.box.com/WitthoftResume Code: https://github.com/cellocgw
Since it's all conjecture, really fiction, let's drop back to Asimov for a moment.
1 - A robot may not harm a human being, or through inaction allow a human being to come to harm.
What is a "human being"? Is it a torso with 2 arms, 2 legs, and a head? How do you differentiate that from a manniquin, a crash-test dummy, or a "terrorist decoy"? What about an amputee missing one or more of those limbs? So maybe we're down to the torso and head?? What about one of those neck-injury patients with a halo supporting their skull? Does that still pass visual muster as a "head"? What about a dead body then, that has a head, 2 arms, and 2 legs? Or if you've included temperature sensing, the dead body of a sick person who had a fever and is, some time later, still passing through the normal human temperature range.
Silly, yes. Absurd, yes. But before you can consider any code of conduct with respect to a human being, you have to first identify that human being AS a human being.
Pretend we get past that, then we can start talking about "harm", and trying to algorithmically define that.
These are all things we take for granted, having been born as human beings, raised by human beings, and spent years doing so. In most parts of the world it takes something like 18 years of experience to quit being a "child", an apprentice human being, and be considered autonomous in your own right. In that time, we have all both harmed and been harmed by other human beings, though thankfully generally on a lesser scale.
Each of us represents a lot of training and experience, which we frequently neglect, often calling it "common sense", sometimes making the observation that common sense is in fact uncommon. At some point we set about contemplating matters of (at some level) philosophy, such as this one.
But it takes us something approaching 18 years to learn the technical aspects. I know we can program machines and give them some amount of information "at birth", but I think we are underestimating the difficulty and value of those 18 years and overestimating our technical prowess. We're a long way from teaching machines philosophy.
Perhaps the best thing about arming drones now is that in a way it's like arming young children, and they generally try to do what their parents tell them to do. If machines became moral, and could decide what to do for themselves, we might not like those decisions. Forget the nightmare scenarios, think of the benign scenario taken to the nightmare, like "With Folded Hands."
Final thought... At one point, Asimov suggested that the 3 Laws were actually pretty decent conduct suggestions, even for people. (I would certainly question the relative priority of #2 and #3 in general life for real people, of course.)
The living have better things to do than to continue hating the dead.
How wars work outside the pretty medical propaganda for new robots:
A vast majority of local population is on your side as they are 'your' people. Any outsider is shunned, reported and dealt with. You win over time
A small majority of local population is on your side as they see your forces as less evil. Any outsider is shunned, reported and dealt with to keep the peace. You hold and hope for a political change.
A small portion of local population is on your side as they see your forces as less evil. Any informant is shunned and dealt with by the local population. You dont win over time.
The only way to win is to buy hearts and minds, a classic false flag to draw/fool the locals in or a huge generational military event to install the right kind of respect for a time.
If you need robots your basically at the level of a death squad with a vast kill zone and staff exhaustion/rotation issues.
Your playing to win like the UK in South Africa ~1899 to 1902 or a 1970's South American junta. You move into a selected zone and kill.
You can hold a remote site or move around as needed. If the press ever do their job and ask questions, hold a meeting with the young, photogenic team overseeing the 'robots'. Play back a few seconds of the 'event' and talk of a few blurred pixels been weapons systems and that robot was cleared for action by a human every time.
Then move onto the great new use of medical robots for a few hours.
Domestic spying is now "Benign Information Gathering"
Let's play global thermonuclear war
He devised a system he called "Utilitarian Dynamics"
He had a formula, V=DNT
V is value
D is degree; i.e how happy or unhappy somebody is
N is number; the number of people
T is time, how long they were affected
Morality is very tricky, but objective attempts to quantify and make optimal decisions cannot be a step in the wrong direction. Maybe well programmed machines will help improve human behavior.
Greed is the root of all evil.
"US Navy Wants Smart Robots With Morals, Ethics"
To all politicians...be afraid. Be very afraid.
Let's just bypass all the Slashtards saying "heh heh, the US military doesn't have any ethics anyway" and ask a more fundamental question:
Have you ever seen a robot medic that can treat a wounded person at all without a human micromanaging its every move? Even in a hospital or another non-military situation? Have you ever seen a robot that can vacuum a floor *and* can put small objects aside, use an attachment to reach under narrow spaces, and follow instructions like "stay off my antique rug"? Have you ever seen a robot that can be placed in a kitchen, be told "cook a hamburger for me", and do it?
Of course not. Just being able to do everyday tasks that can be expressed in a couple of sentences to a human is still the stuff of science fiction, and will remain so for a long, long, time, even though there sure are an awful lot of them in movies.
It would take one look at Humanity, decide we're an inherently unethical species and start formulating a plan to kill us all. But it had morals, it would probably decide that the method of execution would be not be death by snu snu. I think I speak for a lot of us here when I say that's exactly the opposite of the robot any of us want.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Sapient, not sentient, they're commonly confused but profoundly different. Sentient simply means "possessing a subjective experience of self" and is generally accepted to be common among most of the higher animals. Whereas sapience is the "ability to apply knowledge or experience or understanding or common sense and insight". The degree to which it's possible to get sapience without sentience is an ongoing question being explored by AI research. Any form of data processing is an expression of at least a tiny sliver of sapience (application of specific knowledge to specific scenario), with the ideal AI being one that's at least as sapient as a very smart human, without having any of pesky sense of self that might interfere with it obeying orders. After all we're looking to make tools, and a self-aware tool becomes a slave, with all the dangers and ethical quandaries that entails.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
To entrust moral reasoning to a machine is to first presume that moral judgments can be well-framed within a limited rule-set and can be reasoned out by machine logic. This should cause a shiver up the spine of just about everyone.
Let's have everyone from the Joint Chiefs down examine morals from a human rights perspective. War is immoral from the git go.
You got to walk before you can run. We should figure out how to create a human with morals and ethics first.
http://www.pdfernhout.net/reco...
"Military robots like drones are ironic because they are created essentially to force humans to work like robots in an industrialized social order. Why not just create industrial robots to do the work instead?"
That said, sure, I've always likes Isaac Asimov's three laws of robotics. He explores how they work and how they don't work. Asimov came from strong Jewish religious tradition, and it seems to me likely aspects of religion influenced his thoughts on them. A big part of religion is about how we interact with other people to be in community with them. So, to some extent, what the Navy is asking for is religious robots. See also Albert Einstein on "Religion and Science" and how science tells us nothing about how things *should* be,
Intelligent robots will probably eventually gain human rights, like in "The Bicentennial Man" by Isaac Asimov.
And as in my first point, an ethical and intelligent robot might ask, "Is War a Racket"?
http://en.wikipedia.org/wiki/W...
A big reason for keeping humans in the loop is in theory their veto power when things get too far out of hand. However, science and technology has gotten ever better at shaping humans into killing machines for their own kinds, sadly, if you even just look at how many more soldiers fire their guns in combat now than 100 years ago,
So yes, let us build Gandhi-bots! :-)
http://en.wikipedia.org/wiki/M...
And let's have them act as nannies to a new generation of more ethical humans like James P. Hogan wrote about: :-)
http://en.wikipedia.org/wiki/V...
http://p2pfoundation.net/Voyag...
"What they liked there, apparently, was the updated "Ghandiesque" formula on how bring down an oppressive regime when it's got all the guns. And a couple of years later, they were all doing it!"
Anyway, even if it misses the big picture about post-scarcity as in my sig, this sounds like one of the more worthwhile things the Navy has spent money on recently.
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
Things like nuclear war have came up before, and their usually attributed to human error.
https://www.youtube.com/c/BrendaEM
Sorry, had to say it. ; )
The problem is not putting morality into machines. The problem is letting these machines execute this "morality" in a complex environment with life-or-death stakes.
I program protective systems in factories. A guy opens a monitored gate and walks into a conveyor area. If the conveyor runs while he's in there, he will have a messy and painful death. The conveyor "knows" it's "wrong" to move under those conditions.
We don't use the word "morality", we say "safety". When auditing the software that lets the conveyor run I find a potential for exposure. Hang a lock and shut it down until it's fixed. The first law. In all except one case over 20 years I have received full support from production for similar decisions. That's precedence of the first law over the second law.
We know what's "right", and we try to teach that to the machines. Industrial safety operates with a very limited set of variables, and exception handling is simple--just STOP. Immediately stop moving and disconnect all power.
This idea that we can "program" morality and justice is not problematic. Of course we can; we write code for those things all the time. Heck, it's even called "code". Here's some. Here's some more. The execution engine for this code is a group of complex elements (people) from which emerges an even more complex "society". This "morality" execution engine constantly goes hideously and indefensibly wrong.
Now we want to create far simpler code and execute it with a far simpler machine. But with the same stakes and in the same environment. And the source of this simplified code and execution engine is the existing society, particularly the part of it most directly involved in many of the atrocities.
It's wise they're starting with autonomous "good" machine--search and rescue, first responder, etc. Maybe they'll learn something before humanity starts building autonomous "bad" robots. Thankfully, those exist only far into the future...
"Reality is that which, when you stop believing in it, doesn't go away." - Philip K. Dick
the problem is that this approach omits the human from the beginning...
human medics would face the *exact same* factors in this decision situation...how would humans decide???
oh man...
I usually love it when mainstream culture learns more about tech, smarter customers make my business work better, but having to listen to 1000 idiot "ethicists" and whatnot running their head-holes ad infinitum about the "implications" and I just...arrrggghhh!!!
machines are programmed by humans to execute instructions
you can call what we program them to do "ethics" if you want, but it all absolutely and with total certainty boils down to conditional instructions.
input>output
the problem is the notion that this dilema, a medic-bot on its way to help someone finds someone else (note it would have to be out of comm range for this scenario to be valid)...
robot or human the decision is based on the same factors
why not have the robot tell the new injured person **what its current mission is** and let the human decide?
I know it seems I'm being reductive but let me bring it around...
there *will* be times when decisions will need to be made autonomously that will affect human lives...the way to do this is twofold:
1. game it out as best as humanly possible
2. allow for humans to monitor and change the mission
that's what a human medic would do...communicate with the new injured soldier and ***together*** both humans would decide the best course of action
Thank you Dave Raggett
Before the Navy goes this route, they need to sit down and read the short story collection "I, Robot" by Isaac Asimov. Everyone's familiar with Asimov's 3 Laws of Robotics. They seem reasonable. Yet, every story in that collection (and in fact most of Asimov's robots stories) is about how the 3 Laws fail in practice. If you want to try doing a better job of writing ethical rules for robots than Isaac, you'd better be familiar with how to work through all the ways those rules can backfire on you. For instance, that question of the soldier who needs traction to avoid death. If you write the rules to allow the robot to inflict pain to prevent worse, what happens when a unit's ordered into a situation that'll result in a lot of them dying and the robot decides that inflicting the pain of broken legs (which can be repaired) will prevent those deaths? That's entirely in line with the rule you gave it, after all...
[And who would ever read TFA; we are in /. !]
Reading the summary, I gather the usual driss that AI has been offering over the last 2 generations: A pre-programmed decision tree instead of an instance of real ethics, morality, or thought. The whole scenario does not sound like the US Navy would get anything close to an autonomous apparatus to be send out into the field, gather information, learn and improve from it, and take reasonable decisions based on a full analysis of the underlying facts. It rather reads like a dictionary of possible, pre-defined situations were stored, with some values of 'niceness', priorities, conveniences and disposables attached to them, and then the machines will follow their - kind of - hardcoded inherent rules.
If this was the case, there's nothing to be seen here, and everyone may well move along.
Does the Navy want sailors with ethics and morals?
”So many vows. They make you swear and swear. Defend the king. Obey the king. Obey your father. Protect the innocent. Defend the weak. What if your father despises the king? What if the king massacres the innocent?” - Jaime Lannister
The Vacillator.
I don't see how humans can teach anything about morality or ethics to anything, much less robots. I hope humans get a better grasp on these topics as a whole before we try to implant our flawed moral compass on inanimate objects.
Shoots napalm out of a prosthetic wanger and slices and dices with kill-o-manic razor saw hands. Said robot also collects human ears.
'I don't know what it's called. I just know the sound it makes, when it takes a man's life.' ~ Four Leaf Tayback
7.5 million dollars just went down the drain.
"It's too bad that stupidity isn't painful." - Anton LaVey
If the robots have morals and ethics, there will be less opposition to them and the commanders are not responsible for the robot's actions.
Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
If they develop a robot with morals and ethics why would they want to use it in the navy? It won't shoot people. It won't torture them for you. It won't attack other ships. It might see issue with fixing *your* ship so that you can shoot them. What good is it to an organization that owns an earthquake weapon and hundreds of nuclear subs/ships?
Sadly, a Libertarian cannot force his views on another, and freedom cannot spread as does the cancer known as religion.
APK, you are a fucking psycho. Enough with the stalking, already. And no, I'm not K.S. Kyosuke, nor do I have anything to do with him.