Slashdot Mirror
Researchers Trick Tesla Autopilot Into Steering Into Oncoming Traffic (arstechnica.com)
An anonymous reader quotes a report from Ars Technica: Researchers have devised a simple attack that might cause a Tesla to automatically steer into oncoming traffic under certain conditions. The proof-of-concept exploit works not by hacking into the car's onboard computing system. Instead, it works by using small, inconspicuous stickers that trick the Enhanced Autopilot of a Model S 75 into detecting and then following a change in the current lane. Researchers from Tencent's Keen Security Lab recently reverse-engineered several of Tesla's automated processes to see how they reacted when environmental variables changed. One of the most striking discoveries was a way to cause Autopilot to steer into oncoming traffic. The attack worked by carefully affixing three stickers to the road. The stickers were nearly invisible to drivers, but machine-learning algorithms used by by the Autopilot detected them as a line that indicated the lane was shifting to the left. As a result, Autopilot steered in that direction.
The researchers noted that Autopilot uses a variety of measures to prevent incorrect detections. The measures include the position of road shoulders, lane histories, and the size and distance of various object. [A section of the researchers' 37-page report] showed how researchers could tamper with a Tesla's autowiper system to activate wipers on when rain wasn't falling. Unlike traditional autowiper systems -- which use optical sensors to detect moisture -- Tesla's system uses a suite of cameras that feeds data into an artificial intelligence network to determine when wipers should be turned on. The researchers found that -- in much the way it's easy for small changes in an image to throw off artificial intelligence-based image recognition (for instance, changes that cause an AI system to mistake a panda for a gibbon) -- it wasn't hard to trick Tesla's autowiper feature into thinking rain was falling even when it was not. So far, the researchers have only been able to fool autowiper when they feed images directly into the system. Eventually, they said, it may be possible for attackers to display an "adversarial image" that's displayed on road signs or other cars that do the same thing. In a statement, Tesla officials said that the vulnerabilities addressed in the report have been fixed via security update in 2017, "followed by another comprehensive security update in 2018, both of which we released before this group reported this research to us." They added: "The rest of the findings are all based on scenarios in which the physical environment around the vehicle is artificially altered to make the automatic windshield wipers or Autopilot system behave differently, which is not a realistic concern given that a driver can easily override Autopilot at any time by using the steering wheel or brakes and should always be prepared to do so and can manually operate the windshield wiper settings at all times."
The researchers noted that Autopilot uses a variety of measures to prevent incorrect detections. The measures include the position of road shoulders, lane histories, and the size and distance of various object. [A section of the researchers' 37-page report] showed how researchers could tamper with a Tesla's autowiper system to activate wipers on when rain wasn't falling. Unlike traditional autowiper systems -- which use optical sensors to detect moisture -- Tesla's system uses a suite of cameras that feeds data into an artificial intelligence network to determine when wipers should be turned on. The researchers found that -- in much the way it's easy for small changes in an image to throw off artificial intelligence-based image recognition (for instance, changes that cause an AI system to mistake a panda for a gibbon) -- it wasn't hard to trick Tesla's autowiper feature into thinking rain was falling even when it was not. So far, the researchers have only been able to fool autowiper when they feed images directly into the system. Eventually, they said, it may be possible for attackers to display an "adversarial image" that's displayed on road signs or other cars that do the same thing. In a statement, Tesla officials said that the vulnerabilities addressed in the report have been fixed via security update in 2017, "followed by another comprehensive security update in 2018, both of which we released before this group reported this research to us." They added: "The rest of the findings are all based on scenarios in which the physical environment around the vehicle is artificially altered to make the automatic windshield wipers or Autopilot system behave differently, which is not a realistic concern given that a driver can easily override Autopilot at any time by using the steering wheel or brakes and should always be prepared to do so and can manually operate the windshield wiper settings at all times."
So, optical illusions fool a driver. They just fund a kind that fools a digital driver. Film at 11
Because machines "think" very differently from people, the optical illusions will be very different. No surprise there,
Next we'll get a headline that if you put a number sticker over speed limit signs, human drivers can be tricked into driving at the wrong speed - even though very clearly the stickers have the wrong UV patterns and react to LIDAR clearly in an altered way.
They, in fact, did not "steer a Tesla into oncoming traffic", but instead made the software "think" there was a lane line where there was none. The car did go the wrong way (or would have if they'd let it), but there was no traffic. They even said, if there had been cars there, the Tesla likely would have noticed them and not blithely crashed head on.
Everything you know is wrong, Just forget the words and sing along.
I found that it's super easy to make human drivers crash with a simple $5 laser.
It's amazing how many of our systems only work with the underlying assumption that we're not actively trying to murder each other at any given moment.
---- El diablo esta en mis pantalones! Mire, mire!
Real airline pilots have tons of training, understand the limits of the systems, and are literally PAID over six figures to do the terribly boring job of monitoring the system. Tesla owners could have zero training, are certainly not privy to the actual system limitations, and are shown tons of marketing indicating the main benefit of autopilot is the ability to NOT pay attention.
But I can see how they are "basically the same thing".
http://www.humoar.com/fake-roa...
You're not in control, but you have to be constantly ready to take control. You don't have insight into its mental processes so you never know what it's about to do, but you have to be constantly ready to react to what it just did.
And people find driving with Autopilot to be less stressful than driving without it? I guess I'm different from most people.
The difference being a human that sees lane markers leading into active oncoming traffic will decide there are shenigans and not follow.
I guarantee you I can find examples of humans would would be fooled. There are a LOT of humans that are quite easy to mislead and all humans can be mislead sometimes. The only difference is that the tactics that fool a human will usually be different than those that fool a machine but make no mistake that both can be fooled. There are plenty of examples of people very dutifully following the instructions from their GPS into trouble despite it being painfully obvious that the GPS instructions were faulty in some way.
Your notion that people are harder to fool is not entirely supported by the facts.
This is not 'a machine can be fooled like a human', it's a reminder that the machine is still a *lot* dumber than a human.
That depends very much on the human in question. I will be happy to introduce you to some humans I know who should not be permitted to drive on public roads. I'm pretty sure you know some like this as well. Not all humans are "smarter" than machines for driving purposes even today. Your average human almost certainly is a better driver than the current state of the art machine but some machines have already surpassed some humans and they are getting better all the time while human drivers aren't.
You'd have to be able to predict the future to know exactly where to place the stickers where the assassination victim would head-on the truck coming the other way
After observing someone for a little while, you can start to predict their movements. Most people operate on a general routine and schedule. You can know their route to work, and approximately when they will get there. Your confederate could be driving the truck. I'm sure there are less elaborate ways to do someone in, however.
"What the American public doesn't know is what makes them the American public." -Ray Zalinsky (Tommy Boy)
The military finally admitted to the UFOs. Do you get any news at all?
LOL.. Technically maybe, as in "We don't (at this point) know what that was." but not in the sense that aliens have landed. Could have been swamp gas, weather balloons, optical illusions or even a bad acid trip, we don't know, but nobody has any evidence that aliens landed.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
Just don't place the stickers on both sides or they will still miss each other.