Slashdot Mirror
Machine Learning Confronts the Elephant in the Room (quantamagazine.org)
A visual prank exposes an Achilles' heel of computer vision systems: Unlike humans, they can't do a double take. From a report: In a new study [PDF], computer scientists found that artificial intelligence systems fail a vision test a child could accomplish with ease. "It's a clever and important study that reminds us that 'deep learning' isn't really that deep," said Gary Marcus, a neuroscientist at New York University who was not affiliated with the work. The result takes place in the field of computer vision, where artificial intelligence systems attempt to detect and categorize objects. They might try to find all the pedestrians in a street scene, or just distinguish a bird from a bicycle (which is a notoriously difficult task). The stakes are high: As computers take over critical tasks like automated surveillance and autonomous driving, we'll want their visual processing to be at least as good as the human eyes they're replacing.
It won't be easy. The new work accentuates the sophistication of human vision -- and the challenge of building systems that mimic it. In the study, the researchers presented a computer vision system with a living room scene. The system processed it well. It correctly identified a chair, a person, books on a shelf. Then the researchers introduced an anomalous object into the scene -- an image of an elephant. The elephant's mere presence caused the system to forget itself: Suddenly it started calling a chair a couch and the elephant a chair, while turning completely blind to other objects it had previously seen.
"There are all sorts of weird things happening that show how brittle current object detection systems are," said Amir Rosenfeld, a researcher at York University in Toronto and co-author of the study along with his York colleague John Tsotsos and Richard Zemel of the University of Toronto. Researchers are still trying to understand exactly why computer vision systems get tripped up so easily, but they have a good guess. It has to do with an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance.
It won't be easy. The new work accentuates the sophistication of human vision -- and the challenge of building systems that mimic it. In the study, the researchers presented a computer vision system with a living room scene. The system processed it well. It correctly identified a chair, a person, books on a shelf. Then the researchers introduced an anomalous object into the scene -- an image of an elephant. The elephant's mere presence caused the system to forget itself: Suddenly it started calling a chair a couch and the elephant a chair, while turning completely blind to other objects it had previously seen.
"There are all sorts of weird things happening that show how brittle current object detection systems are," said Amir Rosenfeld, a researcher at York University in Toronto and co-author of the study along with his York colleague John Tsotsos and Richard Zemel of the University of Toronto. Researchers are still trying to understand exactly why computer vision systems get tripped up so easily, but they have a good guess. It has to do with an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance.
If an elephant suddenly appeared in my room I'd lose my shit to.
Sig. Sig. Sputnik
These problems have been well known in AI circles for decades. The crappy tech media are finally catching on that marketing departments selling AI solutions maybe exaggerate the capabilities of their tech a twinge.
My Other Computer Is A Data General Nova III.
So you're angry because they're trying to get funding for their work? You want them to research for free, and then only once they have something that can catch up to moving goalposts, THEN you'll have no problem funding them?
Those who do not learn from commit history are doomed to regress it.
AI vision can do some things that no human can do. Quickly and accurately identify handwritten postcodes on envelopes was an early win. Matching colours happens at every paint shop.
It is certainly not human capable, yet. But it has improved dramatically over the last decade, and is likely to do so. And tricks such as stereo vision, wider colour sense, and possibly Lidar help a lot.
The one elephant example seems to be a shitty AI. There is a modern tendency to leave everything to a simplistic Artificial Neural Network, and then wonder why weird things can happen. Some symbolic reasoning is also required, ultimately.
When AI approaches human capability, it will not lose its other abilities. So it will be far better than human vision, eventually.
Ask yourself, when the computers can eventually program themselves, why would they want us around?
it has probably seen an elephant, but probably not in a living room.
and the net has probably a limited concept of the context.
(the big gray blob with a leathery texture in the middle of aiving room is usy a sofa)
cue in the recently published research about machine vision and sheeps
(whenever the system sees white dot spread on a green scenery backfround, it says "sheep". even if it is white rocks sprinkled around the grass.
this prompted the researcher to crowd-mine pictures of goats and sheeps doing unusual stuff. and whenever the CV net saw a fluffy texture, it assumed the most frequent word in that context, calling "dog" any fluffy texture carried by a human in their arms, and "cat" any fluffy texture on a kitchen table, even in case of a shpeherdess carrying a lamb, or a mischievous goat invading a kitchen)
the thing is: CV Net are basically only at what they were trained for. if you give them something completely weird an unusual, they might reacg weirdly.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]