Machine Learning Confronts the Elephant in the Room (quantamagazine.org)

A visual prank exposes an Achilles' heel of computer vision systems: Unlike humans, they can't do a double take. From a report: In a new study [PDF], computer scientists found that artificial intelligence systems fail a vision test a child could accomplish with ease. "It's a clever and important study that reminds us that 'deep learning' isn't really that deep," said Gary Marcus, a neuroscientist at New York University who was not affiliated with the work. The result takes place in the field of computer vision, where artificial intelligence systems attempt to detect and categorize objects. They might try to find all the pedestrians in a street scene, or just distinguish a bird from a bicycle (which is a notoriously difficult task). The stakes are high: As computers take over critical tasks like automated surveillance and autonomous driving, we'll want their visual processing to be at least as good as the human eyes they're replacing.

It won't be easy. The new work accentuates the sophistication of human vision -- and the challenge of building systems that mimic it. In the study, the researchers presented a computer vision system with a living room scene. The system processed it well. It correctly identified a chair, a person, books on a shelf. Then the researchers introduced an anomalous object into the scene -- an image of an elephant. The elephant's mere presence caused the system to forget itself: Suddenly it started calling a chair a couch and the elephant a chair, while turning completely blind to other objects it had previously seen.

"There are all sorts of weird things happening that show how brittle current object detection systems are," said Amir Rosenfeld, a researcher at York University in Toronto and co-author of the study along with his York colleague John Tsotsos and Richard Zemel of the University of Toronto. Researchers are still trying to understand exactly why computer vision systems get tripped up so easily, but they have a good guess. It has to do with an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance.

Comments

limited concepts

[DrYak]

it has probably seen an elephant, but probably not in a living room.

and the net has probably a limited concept of the context.
(the big gray blob with a leathery texture in the middle of aiving room is usy a sofa)

cue in the recently published research about machine vision and sheeps
(whenever the system sees white dot spread on a green scenery backfround, it says "sheep". even if it is white rocks sprinkled around the grass.
this prompted the researcher to crowd-mine pictures of goats and sheeps doing unusual stuff. and whenever the CV net saw a fluffy texture, it assumed the most frequent word in that context, calling "dog" any fluffy texture carried by a human in their arms, and "cat" any fluffy texture on a kitchen table, even in case of a shpeherdess carrying a lamb, or a mischievous goat invading a kitchen)

the thing is: CV Net are basically only at what they were trained for. if you give them something completely weird an unusual, they might reacg weirdly.