Tiny Changes Can Cause An AI To Fail (bbc.com)

Luthair writes: According to the BBC there is growing concern in the machine learning community that as their algorithms are deployed in the real world they can be easily confused by knowledgeable attackers. These algorithms don't process information in the same way humans do, a small sticker placed strategically on a sign could render it invisible to a self driving car.
The article points out that a sticker on a stop sign "is enough for the car to 'see' the stop sign as something completely different from a stop sign," while researchers have created an online collection of images which currently fool AI systems. "In one project, published in October, researchers at Carnegie Mellon University built a pair of glasses that can subtly mislead a facial recognition system -- making the computer confuse actress Reese Witherspoon for Russell Crowe."

One computer academic says that unlike a spam-blocker, "if you're relying on the vision system in a self-driving car to know where to go and not crash into anything, then the stakes are much higher," adding ominously that "The only way to completely avoid this is to have a perfect model that is right all the time." Although on the plus side, "If you're some political dissident inside a repressive regime and you want to be able to conduct activities without being targeted, being able to avoid automated surveillance techniques based on machine learning would be a positive use."

Comments

Re:Mistakes

[gweihir]

"Weak" AI (and that is what we are talking about here) cannot "learn from mistakes". That skill is reserved for actual intelligence and "strong" AI. Strong AI has the little problem that it does not exist as it is currently completely unknown how it could be created, despite about half a century of intense research.

Re:"AI"

[mesterha]

How do you program for even every physical condition a stop sign may find itself in?

This assume the AI even needs to see the stop sign. A driverless car has many advantages over a human. It can have a database of the locations of all stop signs. It have telemetry information from other nearby cars. It can have 360 degree sensors that include cameras and lidar. It doesn't get tired or drunk. It can receive updates based on "mistakes" made by other driverless cars.

Even if there are problems with some of the information, the system can still perform an action based on the total information that is safe for the people in the situation. For example, even if doesn't see a new stop sign, it might still have enough information to see that there is another car entering the intersection.

Of course, it will make mistakes, but it just has to make significantly fewer mistakes than humans. Honestly, given the pace of progress, that doesn't seem too hard.

Re:Speed Bump

[gweihir]

I have heard about it, but unlike you I actually understand what it means. It only surpasses humans in its "Big Data" aspects, not in the actual AI parts. These are so bad that the expert "beaten" thought he would have no trouble finding a strategy to beat it, and that after he had seen it play only a few times. AlphaGo had the full history of the expert's playing style, the expert had nothing the other way round before.

In short, this was a stunt. It does not show what most people think it shows. No AI expert got really excited about this either.