You Can Trick Self-Driving Cars By Defacing Street Signs (bleepingcomputer.com)

An anonymous reader quotes a report from Bleeping Computer: A team of eight researchers has discovered that by altering street signs, an adversary could confuse self-driving cars and cause their machine-learning systems to misclassify signs and take wrong decisions, potentially putting the lives of passengers in danger. The idea behind this research is that an attacker could (1) print an entirely new poster and overlay it over an existing sign, or (2) attach smaller stickers on a legitimate sign in order to fool the self-driving car into thinking it's looking at another type of street sign. While scenario (1) will trick even human observers and there's little chance of stopping it, scenario (2) looks like an ordinary street sign defacement and will likely affect only self-driving vehicles. Experiments showed that simple stickers posted on top of a Stop sign fooled a self-driving car's machine learning system into misclassifying it as a Speed Limit 45 sign from 67% to 100% of all cases. Similarly, gray graffiti stickers on a Right Turn sign tricked the self-driving car into thinking it was looking at a Stop sign. Researchers say that authorities can fight such potential threats to self-driving car passengers by using an anti-stick material for street signs. In addition, car vendors should also take into account contextual information for their machine learning systems. For example, there's no reason to have a certain sign on certain roads (Stop sign on an interstate highway).

Comments

Better solution

[cunina]

Why not just have a geospatial database of signs that self-driving cars access? Then it won't matter what's on the sign, or if the sign even physically exists. Why is anti-stick coating the solution that "researchers" suggest?

Octagon?

[im_thatoneguy]

What horrifically terrible machine learning algorithm sees a red octagon and thinks it's a black and white rectangular speed limit sign? How is the visual machine learning matrix so bad that a triangular yellow sign would be registered as a stop sign?

Do they not train the machine learning algorithms with color images? Considering you can rely on 1-2 seconds of latency for a sign there is no reason to use the same sort of low latency machine learning algorithms used for pedestrian identification or road lines.

Re:dumb machines

[swillden]

This 'technology' is being rushed way too quickly to market.

I'd like to agree with you, particularly with respect to the semi-autonomous systems presently deployed. I argued for years that having a system that worked most of the time but expected the user to take over when necessary was extremely dangerous. But the thing is that human drivers are extremely dangerous. Tesla has very compelling data showing that, as half-baked as their system is, it's actually better than the human drivers that it's replacing. The same will be even more true of the first fully-autonomous vehicles.

The systems don't have to be perfect, they just have to be better, and the bar is not very high.