Developing the First Law of Robotics

wabrandsma sends this article from New Scientist: In an experiment, Alan Winfield and his colleagues programmed a robot to prevent other automatons – acting as proxies for humans – from falling into a hole. This is a simplified version of Isaac Asimov's fictional First Law of Robotics – a robot must not allow a human being to come to harm. At first, the robot was successful in its task. As a human proxy moved towards the hole, the robot rushed in to push it out of the path of danger. But when the team added a second human proxy rolling toward the hole at the same time, the robot was forced to choose. Sometimes, it managed to save one human while letting the other perish; a few times it even managed to save both. But in 14 out of 33 trials, the robot wasted so much time fretting over its decision that both humans fell into the hole. Winfield describes his robot as an "ethical zombie" that has no choice but to behave as it does. Though it may save others according to a programmed code of conduct, it doesn't understand the reasoning behind its actions.

Comments

Same as humans ...

[BarbaraHudson]

Though it may save others according to a programmed code of conduct, it doesn't understand the reasoning behind its actions.

Someone sacrificing their lives by throwing themselves on a grenade to save others doesn't have time to think, never mind understand the reasoning behind their actions. And that's a good thing, because many times we do the right thing because we want to, and then rationalize it later. Altruism is a survival trait for the species.

Re:Same as humans ...

[gl4ss]

sure, but this is a fucking gimmick "experiment".

the algo could be really simple too.

and for developing said algorithm, no actual robots are necessary at all - except for showing to journos, no actual AI researchers would find that part necessary, the testing can happen entirely in simulation - and no actual ethics need to enter the picture even, the robot doesn't need to understand what a human is on the level a robot that would need to in order to act by asimovs laws.

a spinning blade cutting tool that has an automatic emergency brake isn't sentient- it's not acting on asimovs laws, but you could claim so to some journalists anyways.. the thing to take home is that they built into the algorithm the ability to fret over the situation. if it just projected and saved what can be saved, it wouldn't fret or hesitate - and hesitate is really the wrong word.

Re:Same as humans ...

[gweihir]

That is the other thing. Some physicist did an estimation of the most efficient way to do massively parallel computations, including node speed, communications peed, interconnect length, etc. Turns out the human brain is pretty much optimal in this universe, everything larger or with faster nodes or the like will be performing worse. So it is entirely possible, that human intelligence (such as it is in the average case) is really the best possible.

Similar to "Runaround" in I, Robot...

[VitrosChemistryAnaly]

A story in which a robot is stuck between two equal potentials and therefore cannot complete its task.

http://en.wikipedia.org/wiki/Runaround_(story)

Re:Similar to "Runaround" in I, Robot...

[amakawa.yuuto]

The idea is much, much older. Google "Buridan's Donkey". They just replaced the donkey with a robot and hunger/hay with programmed orders.

Re:Similar to "Runaround" in I, Robot...

[radtea]

Yup, and the solution available to any rational being is the same: since by hypothesis the two choices are indistinguishable, flip a coin to create a new situation in which one of them has a trivial weight on its side.

Starving to death (or letting everyone die) is obviously inferior to this to any rational being (which the donkey and the robot are both presumed to be) and adding randomness is a perfectly general solution to the problem.

Buridan's donkey is not in fact an example of a rational being, but rather a passive, uncreative being, who must for some unspecified reason decide without acting on the situation, as if it was living in some bizarrely unrealistic world like Plato's Cave, where it could only know the world via shadows on the wall which it cannot act on in any way.

Why anyone thinks thought-experiments about such limited beings, which are completely unlike humans in their inability to act on the world to change their situation, is beyond me.

That's interesting data but....

[Trailer Trash]

The real question is "how well do normal humans perform the same task?" My guess is "no better than the robot". Making those decisions is difficult enough when you're not under time pressure. It can be very complex, too. Normally I'd want to save the younger of the two if I had to make the choice, but what if the "old guy" is "really important"? Or something like that.

Re:That's interesting data but....

[neoritter]

But a middle aged woman can produce more offspring. Yes the productivity may be reduced for a few years, but will double the productivity.

It's a very dangerous slippery slope when you start trying to value human lives. I think the correct calculation is, which subject has the lowest chance of survival if the robot executes no action against that subject.

So, he's a crappy programmer...

[msauve]

and couldn't program it to prioritize based on which one was seen first, was closest, was apt to fall first based on speed/distance, or any one of many other possibilities. You could even place weights on them, and throw a die at the end as a tiebreaker. The rule should be interpreted as "allow the least harm," not "allow no harm."

Simplification into irrelevance

[kruach aum]

Leaving aside that Asimov's laws of robotics are not sufficiently robust to deal with non-fictional situations, everything about this is way too simplified to draw conclusions from that could ever be relevant to other contexts. Robots are not human beings, nor are they harmed by falling into a hole. What happened here is a guy programmed a robot to stop other moving objects from completing a certain trajectory. Then, when a second moving object entered the picture, in 14 out of 33 trials his code was not up to the task of dealing with the situation. If he'd just been a little more flexible as a programmer (or not an academic trying to make a "point") there would have been no "hesitation" on the part of the robot. It would just do what it had been programmed to do.

I, Robot from a programmers perspective

[wisnoskij]

Don't get me started on Asimov's work. He tried to write allot about how robots would function with these laws that he invented, but really just ended up writing about a bunch of horrendously programmed robots who underwent 0 testing and predictably and catastrophically failed at every single edge case. I do not think there is a single robot in any of his stories that would not not self destruct within 5 minutes of entering the real world.

Re:I, Robot from a programmers perspective

[thewolfkin]

to be fair I thought the whole point of the book was a series of edge cases which would be hard to think of that cause all the "malfunction". The whole point of the book wasn't that the three laws were perfect but that they SEEMED perfect until we put them in the real world and suddenly they would appear to "malfunction"

Re:I, Robot from a programmers perspective

[wisnoskij]

Yes, which is great. Except that it was not just some edge cases, it was not just hard to think of plausible edge cases. It was every single edge case, so much so that, like I said, none of his robots would last 5 minutes in the real world.

Re:I, Robot from a programmers perspective

[houghi]

The three laws were a plot device for his stories, not a programming guide.

Re:I, Robot from a programmers perspective

[DutchUncle]

I think you missed the point of many of Asimov's stories. Edge cases are the normal situation - human beings are always on an edge case in some dimension. Any simplistic set of rules, including all the great slogans and sound bites of capitalism and marxism and socialism and every other political system, are just too simple because the real world is complex.

So, a design failure then.

[RoverDaddy]

In both Asimov's story and this experiment, the real moral seems to be that somebody failed to specify the proper requirements, or run a reasonable design review. "If you can't save everybody, save who you can" seems like a reasonable addition to the program.

Re:So, a design failure then.

[BaronAaron]

Unlike the robots in this experiment, most Asimov robots are not programmed in the traditional sense. Their positronic brains are advanced pattern recognition and difference engines much like our own brains. The Three Laws are encoded at a deep level, almost like an instinct.

In the story Runaround, Speedy is much like a deer in headlights, stuck between the instinct to run away and remain concealed. Doing neither very well. The design mistake was putting more emphasis on the third law versus the second. The PHBs knew better though and felt the robot was too expensive to leave to the command whims of the human mining workers.

I like that story because it illustrates what happens when managers make engineering decisions. ;-)

Re: So, a design failure then.

[jd2112]

Something simple like "with all other factors processed within x milliseconds being equal, save the closest one on the right."

Re:So, a design failure then.

[shadowrat]

Not only that, but the stories in I Robot and asimov's use of the 3 laws were not about laying an actual groundwork for how robots should function, but to illustrate that there are always unintended consequences to the laws. While the stories are really about the unpredictable outcomes of the interplay of those 3 constraints, it is kind of fitting that someone going down the road of trying to realize just one law would not quite get what they were hoping for.

the real genius of the stories of course, isn't that robots should have these laws. it's that he was able to so accurately describe the process of debugging software right up to that a-ha moment where you realize that it's actually doing exactly what you told it to do all along. the guy wrote that stuff in the 30's and 40's and i'm still having those irobot moments every day.

Re:So, a design failure then.

[plover]

I would grant that "fretting" was poetic license. Consider that the life-saving robot must continually evaluate all factors.

Let's say I was closer to a lava flow than you, but your path was on a slightly more direct course into it than mine, and the robot is located at the lava's edge midway between both of us. I will hit the lava in 30 seconds, but you will hit it in 20. The robot needs two seconds to have a high probability of saving someone, but one second is enough for a moderate chance. Factoring in the motion required, the chances of saving us both is high. As you are in more immediate peril than I, it should intercede on your behalf first, so the robot starts to move in your direction. Now, I change my course slightly so I will hit it in 15 seconds. The robot still has time to save us both, but the chances are slightly lower. It moves on a path to intercept me first. You then change your path so you will hit it in 10 seconds. The chances of saving us both is now only moderate, but still possible. So the robot alters its path again to save you first. Now, we both steer directly toward the lava, with only one second to intercept for either of us. The robot's continual path changing introduced so much delay it was no longer in a position to save either of us. We both die.

To the outside observer, it fretted, but the algorithm made continually logical decisions.

50/50

[visionsofmcskill]

why would it waste any time fretting? i presume its decision is by the very nature of computing and evaluation a function of math... therefor the only decision to cause delay would be the one wherein the odds of success are 50/50... but it needs not be delayed there either... just roll a random and pick one to save first.

Sounds like a case of a unnecessary recursive loop to me (where the even odds of save/fail cause the robotic savior to keep reevaluating the same inevitable math in hopes of some sort of change). Maybe the halfway solution is the first tiome you hit a 50/50 you flip a coin and start acting on saving one party while continuing to re-evaluate the odds as you are in motion... this could cause a similar loop - but is more likely to have the odds begin to cascade further in the direction of your intended action.

Seems silly to me.

Priority

[shuz]

An interesting experiment would be to include actions that affect other actions. Such that when one specific proxy falls into a hole, multiple others fall into a hole. Would the robot learn? Would the robot assign priority over time? For any given decision there is yes, no, and maybe with maybe requiring a priority check to figure out what the end result is. In programming we tend towards binary logic, but the world is not black and white. Likely if the robot was programmed to learn, the robot would eventually come to the conclusion of save proxy A = yes, save proxy B = yes.Followed by Save A first = maybe, save B first = maybe. Followed by likely hood of success A > B = Yes/No and B>A Yes/No. Followed by action. The next question would be what happens if A=B? What you would likely find is that the robot would either randomly choose or go with the first or last choice, but would likely not fail to take some action. I would find it interesting if the robot didn't take action and then try to explain that.

Buridan's Principle

[rlseaman]

For those who think the only problem is bad programming, see Leslie Lamport's analysis: http://research.microsoft.com/... Some race conditions are built into the real world.

Re:Buridan's Principle

[neoritter]

Do you really think a donkey will starve to death because you place two bales of hay equidistant from the donkey?

obvious error

[rickb928]

The article misstated First Law. Get that right first.