Do Neural Nets Dream of Electric Sheep? (aiweirdness.com)

An anonymous reader shares a post: If you've been on the internet today, you've probably interacted with a neural network. They're a type of machine learning algorithm that's used for everything from language translation to finance modeling. One of their specialties is image recognition. Several companies -- including Google, Microsoft, IBM, and Facebook -- have their own algorithms for labeling photos. But image recognition algorithms can make really bizarre mistakes. Microsoft Azure's computer vision API added the above caption and tags. But there are no sheep in the image. None. I zoomed all the way in and inspected every speck. It also tagged sheep in this image. I happen to know there were sheep nearby. But none actually present. Here's one more example. In fact, the neural network hallucinated sheep every time it saw a landscape of this type. What's going on here?

Are neural networks just hyper-vigilant, finding sheep everywhere? No, as it turns out. They only see sheep where they expect to see them. They can find sheep easily in fields and mountainsides, but as soon as sheep start showing up in weird places, it becomes obvious how much the algorithms rely on guessing and probabilities. Bring sheep indoors, and they're labeled as cats. Pick up a sheep (or a goat) in your arms, and they're labeled as dogs.

Comments

This is Known...

[dcw3]

They can find sheep easily in fields and mountainsides, but as soon as sheep start showing up in weird places, it becomes obvious how much the algorithms rely on guessing and probabilities

This is known as "profiling". The sheep will protest, especially the black ones.

They don't form proper models

[iTrawl]

Now what is that story where an AI is trained to turn the air on in an alien(?) train station when the train enters the platform? I can't find it on Google.

The way I remember it the AI is trained, and then left alone and does a great job until one day when it kills all the passengers because it didn't turn the air on. The reason was that the station clock was broken. The AI didn't learn the train-at-platform correlation, but rather the wall clock schedule (I guess those trains were never early or late).

Re:This is where prejudices come from

[SirGarlon]

I'm with you except for the part about the general rules underlying prejudices being usually correct. I don't believe that is a requirement for human beings to accept the rule. So I would say the "pre" in "prejudice" really means the rule doesn't get tested for accuracy or revised.

Fundamentally, thinking of deep learning as machine-generated prejudice changes one's enthusiasm for the technology.