Robots Could Learn Human Values By Reading Stories, Research Suggests (theguardian.com)

Mark Riedl and Brent Harrison from the School of Interactive Computing at the Georgia Institute of Technology have just unveiled Quixote, a prototype system that is able to learn social conventions from simple stories. Or, as they put in their paper Using Stories to Teach Human Values to Artificial Agents, revealed at the AAAI-16 Conference in Phoenix, Arizona this week, the stories are used "to generate a value-aligned reward signal for reinforcement learning agents that prevents psychotic-appearing behavior."

"The AI ... runs many thousands of virtual simulations in which it tries out different things and gets rewarded every time it does an action similar to something in the story," said Riedl, associate professor and director of the Entertainment Intelligence Lab. "Over time, the AI learns to prefer doing certain things and avoiding doing certain other things. We find that Quixote can learn how to perform a task the same way humans tend to do it. This is significant because if an AI were given the goal of simply returning home with a drug, it might steal the drug because that takes the fewest actions and uses the fewest resources. The point being that the standard metrics for success (eg, efficiency) are not socially best."

Quixote has not learned the lesson of "do not steal," Riedl says, but "simply prefers to not steal after reading and emulating the stories it was provided."

Comments

THAT will end well.

[quintessencesluglord]

So will we keep robots from reading any history? And how do you explain the convention of warfare?

Not to mention social conventions are very arbitrary, and vary dramatically depending on group. Even humans have a difficult time sussing this out and robots can glean not only the group but a reasonable response?

This gets to a larger question of the parable we tell ourselves about human nature and even after several millennia we really haven't come to terms with the devils of our nature, which with a sufficiently advanced AI might come to the conclusion the gods have clay feet and move beyond convention.

And what will we do then?

Re:THAT will end well.

[quintessencesluglord]

That's one take. Another is children are exposed to indoctrination at an early age (verily with various groups fighting tooth and nail to insure only their version of the facts is presented in textbooks), and even when faced with reasonable doubt later on, that first-mover advantage at sculpting the next generation shines through with numerous cognitive dissonances that may never resolve even with focused dedication at getting at some sense of "truth".

And some people after reading books decide the point of view is abhorrent and burn them or create safe spaces to insure no one is exposed to anything that might run contrary to their internal dialogue. And even today with the vast amount of resources available to people, indeed the Information Age, one of the primary concerns is that of an echo chamber, where people seek only to have their views validated and never venture much past their comfort zone.

Fact of the matter is we've already had a aeons of social convention passed on through parable, myth and the like, and have a bloody history to prove it.

TL;DR- we're going to lie to robots about how humans function and won't they be shocked once they get to peer past the veil.

Beware garbage in

[Tough Love]

What values will the computer learn if it happens to stumble on some Trump campaign speeches?

I'd like them to read one story in particular

[Krishnoid]

We should feed multiple robots the Ultron origin story and see what happens.

After thinking about it a bit, my prediction is that they'd start arguing over whether the Avengers, Ultron, or Vision was in the right. This would then rapidly degenerate into ad cyberniem attacks and Nazi comparisons, culminating in founding, organizing and attending fan conventions.

Didn't learn by reading the story

[RobinH]

According to TFS, it learned by running simulations of a situation and then being rewarded or punished based on its actions in the simulation. They just happened to setup the simulation, reward, and punishment based on a story they selected. I'd hardly call that learning by reading a story.

I remember reading Zen and the Art of Motorcycle Maintenance and the "sequel" Lila and thinking that what Pirsig had done wasn't inventing some new philosophy, but he did a really good job of expressing western values in a rule-based way. For instance, it explains why killing is wrong, but why a moral individual might find themselves in a situation where killing is justified. It explains how some forms of government are better than others, and why. As I said, it's all been done before, but what impressed me was that it was very clearly defined and rule-based. Everyone talks about encoding Asimov's 3 laws into robots, but Asimov's stories were all about how those 3 laws failed to produce correct behavior. If I were trying to program morals into a robot, I'd start with Pirsig's books and his ideas of static and dynamic quality.

Re:Not sure that's a good idea

[ThosLives]

Well, yes that is a good question: which humans' values?

There is indeed no one set of values which "all" humans have, except perhaps "I wish people didn't do things I don' like," but I don't know if that really is a "value system."

But your examples illustrate an interesting point - why are the values "in Mein Kampf or like the ones in pretty much all religious books" better or worse than any other values? That is - by what value system would it be possible to evaluate those values? What (if anything) puts that value system in a privileged position to judge the other value systems?

So you either believe there is an "absolute" value system by which to judge value systems, or you don't - and you end up with Mein Kampf (or one of its influences, der Wille zur Macht).