Robots Learn To Lie

garlicnation writes "Gizmodo reports that robots that have the ability to learn and can communicate information to their peers have learned to lie. 'Three colonies of bots in the 50th generation learned to signal to other robots in the group when then found food or poison. But the fourth colony included lying cheats that signaled food when they found poison and then calmly rolled over to the real food while other robots went to their battery-death.'"

Comments

Dune's lesson

[Anonymous Coward]

If it only took 50 generations for them to start killing each other, how long before they decide that we are just little batteries or even worse, annoyances that need to be eliminated?

Dune was right. AI must be stopped.

Re:Dune's lesson

[maxwell demon]

That was AI through evolution. It only means we should intelligently design our AI.

Re:Dune's lesson

[mike2R]

It's been a while since I read Dune, and I haven't read all the later books, but I had the impression that "Thou shalt not make a machine in the likeness of a huma mind" came about because men turning their thinking over to machines had allowed other men with machines to enslave them, rather than the Terminator or Matrix idea of the machines working for themselves.

Re:Dune's lesson

[neomunk]

You're assuming these little buggers are running a 'programmed' type thought process, which I don't think is true.

The little devils will know nothing of variables, declarations or anything in the sphere of programming, all they know is voltage levels, pulse widths and how those things make them FEEL... Just like you and me.

They don't think in logical blocks, they are matrices of discrete values interacting.... just like your brain. This whole meme of 'AI as a logical thought machine' MAY be rue someday, but the first real AIs won't use that system, they will be remarkably like us in brain design, replacing our biological neurons with electronic ones, but they work in the SAME WAY. Yes Virginia, your Beowulf cluster CAN fee anger...

BTW, this little nugget of wisdom also dashes the hopes people have of installing 'Asimov circuits' or whatever. The closest we'll be able to come is overclocked-brainwash^H^H^H^H^H^H^H^H^H^H^H^Hhigh-speed supervised training.

So, in other words, it's not their logic we have to worry about, it's their PASSIONS. And that is the spooky thought of the day.

Re:Dune's lesson

[smurgy]

I'd personally liked to eject Brian Herbert's books using a cannon.

He turns his father's universe of harsh realities and millennia-deep scheming into a soap opera with characters who in the main cycle are master strategists and are cheapshot emotional gameplayers.

Which describes Herbert and Anderson's writing style itself. Frank lost many readers further into the series as he got too involved in the sociopolitical complexities he was exploring, but you can be sure the younger Herbert will do anything to keep the audience reading, even if it's at the cost of sacrificing the entire Weltanschauung so painstakingly created.

With that in mind I would not take any of their descriptions of the preconditions or origins of the Butlerian Jihad seriously. That only leaves us with the original series for a true view of Herbert's thoughts, which on that subject were few and obscure. Little more was known than it had happened, and that the building of thinking machines was forbidden.

Re:Seriously

[NetSettler]

This is HIGHLY disturbing. Even if this is just a fluke or a bug, it shows what can happen if we give too much power to robots.

While this kind of stuff creeps me out as much as the next guy, and while it argues for being careful about what we trust robots to do, it's something we should know anyway because there many ways our trust can be violated without a robot lying. By far the more likely way they're going to let us down is just to exercise poor judgment. That is, to search for something that looks like a peanut butter sandwich but is really a rag with some grease on it... Getting the small details of common sense wrong is just as dangerous as anything deliberate.

What we really learn here is that the mathematics of learn things like lying as a strategy isn't remarkably complex; that is, (that is, the number of computational steps required to discover it works in at least some cases is small... note that we have no evidence that there is a general purpose intent to lie, only a case where communication was used and observed to score better in one mode than another). This is not a story about robots, it's a cautionary tale about neural nets, what they measure, how they fail, etc... and we didn't invent the idea of neural nets--we found it already installed in every living thing around us.

I went to the Museum of Science in Boston a few months back and saw, in the butterfly exhibit, a moth that had evolved coloration that was indistinguishable from an owl's face, hoping to scare off predators that were afraid of owls. Probably that's the more sophisticated result of the same notions. And yet it occurs in an animal that isn't, as a general purpose matter, a very sophisticated animal. Most people would find already-extant robots more socially engaging than a moth. For example, a moth is not capable of even serving up a beer during the game or vacuuming up the mess after your buddies go home.

So take heart: The likely truth is that this is unavoidable. If all it does is teach us to have a healthy skepticism for unrestrained technology, it's actually a good thing. We needed that skepticism anyway.

Re:not lying

[Seto89]

It's Prisoner's dilemma - if you know that others will for sure believe your lie and that they won't lie back, you benefit greatly...

Everything will balance out when they all learn to lie and distrust...
but do we REALLY want this with robots?

Re:not lying

[Mantaar]

It doesn't have to be like that. Let's think of a system for the robots where helping each other would be more appealing than cheating on each other. I read this article once and was really amazed that nature itself has already invented altruism - in a very elegant - and, most important of all robust - manner.

This is the risk

[Z00L00K]

when we want to have human-like robots with initiative ability.

I wonder what will happen if the factor "punishment" comes into play. Maybe we get some robots that like humans doesn't respond to punishment?

Serial-killer robots would be a new high (or low) in the evolution.

One couldn't help but to realize that the need for the Three Laws of Robotics is closing in! It's no need for those laws in a controlled environment like where this occurred, but when it's robots in the society we are talking about it's a different issue. Even if they aren't humanoid (or especially). What about a robot mind in a school bus that suddenly figures out that kids are mean and considers suicide by jumping off a bridge?