Tiny Changes Can Cause An AI To Fail (bbc.com)

Luthair writes: According to the BBC there is growing concern in the machine learning community that as their algorithms are deployed in the real world they can be easily confused by knowledgeable attackers. These algorithms don't process information in the same way humans do, a small sticker placed strategically on a sign could render it invisible to a self driving car.
The article points out that a sticker on a stop sign "is enough for the car to 'see' the stop sign as something completely different from a stop sign," while researchers have created an online collection of images which currently fool AI systems. "In one project, published in October, researchers at Carnegie Mellon University built a pair of glasses that can subtly mislead a facial recognition system -- making the computer confuse actress Reese Witherspoon for Russell Crowe."

One computer academic says that unlike a spam-blocker, "if you're relying on the vision system in a self-driving car to know where to go and not crash into anything, then the stakes are much higher," adding ominously that "The only way to completely avoid this is to have a perfect model that is right all the time." Although on the plus side, "If you're some political dissident inside a repressive regime and you want to be able to conduct activities without being targeted, being able to avoid automated surveillance techniques based on machine learning would be a positive use."

Comments

Re:Speed Bump

[SlaveToTheGrind]

Fine, but you only need a great model that's right more often than humans.

I don't know that I've ever heard of a human driver who ran a stop sign thinking it was a banana.

Remember, this is "weak" AI

[gweihir]

Weak AI is characterized by not being intelligent. It is merely statistical classification, algorithmic planning and things like that. It has the advantage that (unlike "strong" AI) it is actually available. But it has the disadvantage that is has zero understanding of what it is doing. As strong AI is not even on the distant horizon, in fact it is unclear whether it is possible to create it at all (despite what a lot of morons that have never understood current research in the field or have not even looked at it like to claim), weak AI is all we will have for the foreseeable future. This means that we have to fake a lot of things that even the tiniest bit of actual intelligence could easily do by itself.

Of course, weak AI is still massively useful, but confusing it with actual intelligence is dangerous. It is however noting any actual expert will ever do. They know. It is just the stupid public that does not get it at all. As usual.

Current AI isn't...

[hughbar]

We've been going through this since the 1980's when we started to make ruled-based expert systems and put them into production. We called that AI too. Now we're doing the same with statistical machine 'intelligence' (optimisation, often), various configurations of trainable neural networks and some hybrids.

These are trainable appliances, not intelligences. They don't have the adaptability and recovery from mistakes of human or (in the case of statistical, sub-symbolic etc.) any explanatory power. To some extent, that's why I liked the ancient expert systems with a why? function, but they were also very brittle. So I think the current hype curve has inflected and this is a good thing, since, apart from this, there are some quite weighty ethical problems as well.

This is not the view of a neo-Luddite, but there's stuff to think about here.

Re:Speed Bump

[gweihir]

That is nonsense. AIs have never surpassed human performance (of course, you always need to compare to a human expert) and there is no rational reason to expect that they ever will. Incidentally, said "great" model is currently completely out of reach, even for relatively simple things like driving a car (which almost all humans can learn to do, i.e. it does not require much). The best we will get is a model that solves a lot of standard situations from a catalog and appeals to human help in the rest. That is pretty useful and will make things like self-driving cars a reality, but some things that smart human beings can do will likely remain out of reach for a long time and quite possibly forever.

Re:Mistakes

[gweihir]

"Weak" AI (and that is what we are talking about here) cannot "learn from mistakes". That skill is reserved for actual intelligence and "strong" AI. Strong AI has the little problem that it does not exist as it is currently completely unknown how it could be created, despite about half a century of intense research.

Re:SIGH

[wonkey_monkey]

I've been saying it before and I'll say it again.

And you'll presumably keep saying it until it suddenly isn't true, when you'll have to stop.

It doesn't matter much if auto-cars do get in accidents as long as they get in fewer accidents than humans do, as a result of the scenarios you've outlined and more. One day they will be smart enough to consider that a child might appear when a ball does, but for now they can just stop or slow down when they see the ball (which is an obstruction in the road).

They used to think computers would never beat humans at chess. Then it was Jeopardy. Then it was Go. One of the few certainties in life is that the "it can't be done!" crowd are invariably proven wrong, sooner or later.

Re:"AI"

[fluffernutter]

I think most people putting a lot of money into AI don't really understand the difference between programming a response to a stop sign and understanding what a stop sign is. If a stop sign is bent over from a previous accident and covered in snow, you will still stop if you truly understand what that object is. If you have programmed a stop sign, the vehicle is lucky to sail on through because stop signs aren't white objects on a pole close to the ground. How do you program for even every physical condition a stop sign may find itself in?

This has been known for over 30 years

[Solandri]

AI researchers first ran across it when developing neural nets. The longer you allowed a neural net to learn, the more rigid its definition of boundary conditions became. Sometimes so rigid that the net became useless for its intended task. e.g. You could develop a neural net which would stop a train in the correct position at the platform 80% of the time. Further training would increase this to 90%, then 95%, then 99% of the time, but resulted in the net completely flipping out the remaining 1% of the time when it calculated it was going to overshoot by 1 mm outside the trained parameters. The first solution was to stop the learning process and freeze the neural net before it reached this stage, then simply use it in production with the learning capability (ability to modify itself) disabled. The next solution was to use simulated annealing to occasionally reset the specific things the neural net had learned, while retaining the general things it had learned.

You also see this in biological neural nets. As people get older, they tend to get set in their ways, less likely to change their opinions even in the face of contradictory evidence. (As opposed to younger people who are too eager to form an opinion despite weak or the lack of evidence.) I suspect this is also where the aphorism "you can't teach an old dog new tricks" comes from. IMHO this is why trying to lengthen the human lifespan in the pursuit of immortality is a bad idea. Death is nature's way of clearing out neural nets which have become too rigid to respond properly to common variability in situations they encounter. My grandmother hated the Japanese to her dying day (they raped and killed her sister and niece during WWII). If people were immortal, we'd be completely dysfunctional as a society because everyone would be holding grudges and experience-based prejudice for hundreds of years, to the detriment of immediate benefit.

Re:"AI"

[mesterha]

How do you program for even every physical condition a stop sign may find itself in?

This assume the AI even needs to see the stop sign. A driverless car has many advantages over a human. It can have a database of the locations of all stop signs. It have telemetry information from other nearby cars. It can have 360 degree sensors that include cameras and lidar. It doesn't get tired or drunk. It can receive updates based on "mistakes" made by other driverless cars.

Even if there are problems with some of the information, the system can still perform an action based on the total information that is safe for the people in the situation. For example, even if doesn't see a new stop sign, it might still have enough information to see that there is another car entering the intersection.

Of course, it will make mistakes, but it just has to make significantly fewer mistakes than humans. Honestly, given the pace of progress, that doesn't seem too hard.

Re:Speed Bump

[gweihir]

I have heard about it, but unlike you I actually understand what it means. It only surpasses humans in its "Big Data" aspects, not in the actual AI parts. These are so bad that the expert "beaten" thought he would have no trouble finding a strategy to beat it, and that after he had seen it play only a few times. AlphaGo had the full history of the expert's playing style, the expert had nothing the other way round before.

In short, this was a stunt. It does not show what most people think it shows. No AI expert got really excited about this either.