Slashdot Mirror
Tiny Changes Can Cause An AI To Fail (bbc.com)
Luthair writes:
According to the BBC there is growing concern in the machine learning community that as their algorithms are deployed in the real world they can be easily confused by knowledgeable attackers. These algorithms don't process information in the same way humans do, a small sticker placed strategically on a sign could render it invisible to a self driving car.
The article points out that a sticker on a stop sign "is enough for the car to 'see' the stop sign as something completely different from a stop sign," while researchers have created an online collection of images which currently fool AI systems. "In one project, published in October, researchers at Carnegie Mellon University built a pair of glasses that can subtly mislead a facial recognition system -- making the computer confuse actress Reese Witherspoon for Russell Crowe."
One computer academic says that unlike a spam-blocker, "if you're relying on the vision system in a self-driving car to know where to go and not crash into anything, then the stakes are much higher," adding ominously that "The only way to completely avoid this is to have a perfect model that is right all the time." Although on the plus side, "If you're some political dissident inside a repressive regime and you want to be able to conduct activities without being targeted, being able to avoid automated surveillance techniques based on machine learning would be a positive use."
The article points out that a sticker on a stop sign "is enough for the car to 'see' the stop sign as something completely different from a stop sign," while researchers have created an online collection of images which currently fool AI systems. "In one project, published in October, researchers at Carnegie Mellon University built a pair of glasses that can subtly mislead a facial recognition system -- making the computer confuse actress Reese Witherspoon for Russell Crowe."
One computer academic says that unlike a spam-blocker, "if you're relying on the vision system in a self-driving car to know where to go and not crash into anything, then the stakes are much higher," adding ominously that "The only way to completely avoid this is to have a perfect model that is right all the time." Although on the plus side, "If you're some political dissident inside a repressive regime and you want to be able to conduct activities without being targeted, being able to avoid automated surveillance techniques based on machine learning would be a positive use."
Does he really think there won't be 100,000 First World jackasses defacing stop signs for the lulz and religious terrorists hoping that defaced stop signs will cause school buses to crash into synagogues and girls' schools for every 1 political dissident fighting the good fight against repressive regimes?
"I don't know, therefore Aliens" Wafflebox1
Weak AI is characterized by not being intelligent. It is merely statistical classification, algorithmic planning and things like that. It has the advantage that (unlike "strong" AI) it is actually available. But it has the disadvantage that is has zero understanding of what it is doing. As strong AI is not even on the distant horizon, in fact it is unclear whether it is possible to create it at all (despite what a lot of morons that have never understood current research in the field or have not even looked at it like to claim), weak AI is all we will have for the foreseeable future. This means that we have to fake a lot of things that even the tiniest bit of actual intelligence could easily do by itself.
Of course, weak AI is still massively useful, but confusing it with actual intelligence is dangerous. It is however noting any actual expert will ever do. They know. It is just the stupid public that does not get it at all. As usual.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
That is nonsense. AIs have never surpassed human performance (of course, you always need to compare to a human expert) and there is no rational reason to expect that they ever will. Incidentally, said "great" model is currently completely out of reach, even for relatively simple things like driving a car (which almost all humans can learn to do, i.e. it does not require much). The best we will get is a model that solves a lot of standard situations from a catalog and appeals to human help in the rest. That is pretty useful and will make things like self-driving cars a reality, but some things that smart human beings can do will likely remain out of reach for a long time and quite possibly forever.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
The problem with this kind of "AI" (it's not, but let's not go there) is that there's no understanding of what it's actually doing. We're creating tools, "training" them, and then we have no idea what it's basing decisions on past that point.
As such, outside of toys, they aren't that useful and SHOULDN'T BE used for things like self-driving cars. You can never imagine them passing, say, aviation verification because you have literally no idea what it will do.
And it's because of that very problem that they are also unfixable, and unguaranteeable. You can't just say "Oh, we'll train it some more" because the logical consequence of that is that you have to train it on virtually everything you ever want it to do, which kind of spoils the point. And even then, there's no way you can guarantee that it will work next time.
Interesting for beating humans at board games, recognising what you're saying for ordering online, or spotting porn images in image search. Maybe. Some day. But in terms of reliance, we can't rely on them which kills them for all the useful purposes.
It's actually one of the first steps of humans creating systems to do jobs, that the humans do not and cannot understand. Not just one individual could not understand, but nobody, not even the creator can understand or predict what it will do. That's dangerous ground, even if we aren't talking about AI-taking-over-the-world scenarios.
I've been saying it before and I'll say it again.
And you'll presumably keep saying it until it suddenly isn't true, when you'll have to stop.
It doesn't matter much if auto-cars do get in accidents as long as they get in fewer accidents than humans do, as a result of the scenarios you've outlined and more. One day they will be smart enough to consider that a child might appear when a ball does, but for now they can just stop or slow down when they see the ball (which is an obstruction in the road).
They used to think computers would never beat humans at chess. Then it was Jeopardy. Then it was Go. One of the few certainties in life is that the "it can't be done!" crowd are invariably proven wrong, sooner or later.
systemd is Roko's Basilisk.
AI researchers first ran across it when developing neural nets. The longer you allowed a neural net to learn, the more rigid its definition of boundary conditions became. Sometimes so rigid that the net became useless for its intended task. e.g. You could develop a neural net which would stop a train in the correct position at the platform 80% of the time. Further training would increase this to 90%, then 95%, then 99% of the time, but resulted in the net completely flipping out the remaining 1% of the time when it calculated it was going to overshoot by 1 mm outside the trained parameters. The first solution was to stop the learning process and freeze the neural net before it reached this stage, then simply use it in production with the learning capability (ability to modify itself) disabled. The next solution was to use simulated annealing to occasionally reset the specific things the neural net had learned, while retaining the general things it had learned.
You also see this in biological neural nets. As people get older, they tend to get set in their ways, less likely to change their opinions even in the face of contradictory evidence. (As opposed to younger people who are too eager to form an opinion despite weak or the lack of evidence.) I suspect this is also where the aphorism "you can't teach an old dog new tricks" comes from. IMHO this is why trying to lengthen the human lifespan in the pursuit of immortality is a bad idea. Death is nature's way of clearing out neural nets which have become too rigid to respond properly to common variability in situations they encounter. My grandmother hated the Japanese to her dying day (they raped and killed her sister and niece during WWII). If people were immortal, we'd be completely dysfunctional as a society because everyone would be holding grudges and experience-based prejudice for hundreds of years, to the detriment of immediate benefit.
If tiny changes cause these "weak AI" algorithms to fail, then they've been trained badly, or else aren't sophisticated enough algorithms at their core. That, or they don't have enough context. For instance, a stop sign should be recognizable almost purely based on the fact that it's a uniquely shaped sign (octagonal) in the US, at least, along with its proximity and relative position to an intersection. An AI looking at a photo has none of this contextual information, and so has a severe disadvantage.
More importantly, no car manufacturer will be relying solely on vision systems to make navigation systems, which is a huge advantage they'll have over human drivers. Suggesting otherwise seems disingenuous at best on the part of these "computer academics" quoted in the summary. These vehicles will be relying on a *range* of sensors to detect what's going on around them, and I'd argue that vision may well be the least important among them.
Irony: Agile development has too much intertia to be abandoned now.