Researchers Create 'Psychedelic' Stickers That Confuse AI Image Recognition (techcrunch.com)

"Researchers at Google were able to create little stickers with 'psychedelic'-looking patterns on them that could trick computer AI image-classifying algorithms into mis-classifying images of objects that it would normally be able to recognize," writes amxcoder: The patterned stickers work by tricking the image recognition algorithm into focusing on, and studying, the little pattern on the small sticker -- and ignoring the rest of the image, including the actual object in the picture... The images on the stickers were created by the researchers using knowledge of features and shapes, patterns, and colors that the image recognition algorithms look for and focus on.

These stickers were created so that the algorithm finds them 'more interesting' than the rest of the image and will focus most of it's attention on analyzing the pattern, while giving the rest of the image content a lower importance, thus ignoring it or confusing it.
The technique "works in the real world, and can be disguised as an innocuous sticker," note the researchers -- describing them as "targeted adversarial image patches."

Comments

Detail vs shape

[QuietLagoon]

It looks as if the AI is concentrating on the area with the most detail, even though it is not really relevant. I've seen similar, ummmm, distractions confuse AI. For example, disguising a stop sign so that a self-driving car is confused.

let's get that on clothing

[iggymanz]

Remember the "worlds ugliest t-shirt" in one of William Gibson's novels? All cameras in that book's world were compelled by their firmware to fill image of the wearer of that suit with background. One could laugh at such a notion except ....scanners won't do banknotes

I thought what I'd do was I'd pretend...

[AtomicSymphonic]

"I thought what I'd do was I'd pretend I was one of those deaf-mutes"

Reminds me of Ghost in the Shell's Laughing Man calling card... His sticker would appear over people's faces in VR if they were infected.

ALPR?

[Ralgha]

Would one of these stickers on the bumper of my car defeat the automated license plate readers?

Computer Chess

[bussdriver]

With a similar enough network or access to the targeted network, simply create a network that learns to fool the other one. Loosely like two computers playing chess but more like a spam generator to defeat filters.

Adversarial network learning... just not an official use of it... The solution is to add this kind of learning to the network... except it won't be fool proof until the network is quite good; since the adversary could have as many variations of attack as the classifier has in recognition.

If you created the adversarial network used to train it, you could leave INTENTIONAL holes for future exploitation. Even going so far as to purposely train in holes if you had that kind of access. It's not like anybody is going to spot your code in the AI -- only the training setup... which could be long gone after years of training... In the future, I would expect to have VALUE in AI training whereby the cost of "reboot" would be quite significant... finding bad training data over millions of samples and years of experience could be difficult and who's to say all that would be retained? You take the resulting network from last week and retrain from that point-- you'd not go back years ago and restart. I'm talking way out... because AI is so simple now you can just archive all input data... maybe by that point we can still archive it all and learning hardware will be faster... anyhow, it makes for interesting Sci-Fi possibilities even if it may never become an issue (even if it doesn't, there would still be a cost involved in retraining from scratch.)

Actual Intelligence

[DCFusor]

Is not as easily fooled as this pattern matching NN grossly incorrectly hyped as Artificial intelligence. Just saying - hype is hype no matter how much you want to believe you've got the next big thing and innovation (and in this case, NN research and pattern matching work go WAY back).