Slashdot Mirror
Researchers Discover How To Fool Tesla's Autopilot System (cnet.com)
An anonymous reader writes from a report via CNET: Researchers from the University of South Carolina, Zhejiang University and Qihoo 360 have discovered how to fool Tesla's Autopilot sensors, according to a report from Wired. The researchers were able to trick the system into thinking an object didn't exist when it did, and that an object existed when in fact it did not. Therefore, possible security concerns arise as Autopilot could drive incorrectly, potentially putting passengers and others in danger. CNET reports: "Two pieces of radio equipment were used to convince Tesla's radar sensor that a cart was not placed directly in front of it. One of those pieces, a signal generator from Keysight Technologies, costs about $90,000. The group also tricked the car's short-range parking sensors into malfunctioning using about $40 worth of equipment. Wired points out that this was, thankfully, a rather difficult feat. Most of the technological tomfoolery was done on a stationary car. Some of the required equipment was expensive, and it didn't always work. But it brings up an important point -- even though Autopilot is quite capable, there's still no substitute for an attentive human driver, ready to take control at a moment's notice."
Because it's so hard to make humans see or not see things.
That's not a reason to worry about it happening.
If you spent the same resources to fool a human driver, how hard would that be?
...researchers place a camouflaged boulder on a highway, film at 11.
It seems to me that the best choice is if both the human driver is paying attention and the autopilot is on. Second best is the autopilot by itself (at least it can pay attention all of the time), and last is the human (who can both be fooled and be inattentive).
Of course you can trick any sensor invented by man some way or other. That's nothing new. We even know tons of ways to trick the sensors made by god/nature aka our eyes as well. Shine a bright light into them for $10 or maybe $100 and the driver will be forced to drive blind. Or you can have a $0 natural snow storm and the driver will also be on literally very dangerous ground: zero visibility and icy roads.
The point is not that either can be fooled, the point is, is the mechanical sensor better or at least equal to mark I eyeball? Is the program doing the automatic driving at least as good as an average driver? as good as the best driver possible?
Researchers have discovered a way to disable the human "autopilot" system using just $10 worth of equipment. By shining a flashlight in their eyes, they are able to totally disable the primary optical sensors, so preventing the navigation system from avoiding objects in the car's path. While this was only tested in a lab environment, it is feared that Russian agents and ISIS terrorists could use a similar technique to cause mass casualties.
tell us how great this tech is, then you can walk in front of Teslas to prove it
Don't drive behind a truck full of signal generators on a bumpy road.
So for $90,000 you can set up a trap for a Tesla car running with autopilot, and possibly cause a crash and kill the driver. Lots of people can do that a lot cheaper using a gun. So what's the problem?
Using enough equipment they were able to fool Tesla's sensors, including in some cases needing hardware more valuable than a car. You know, a bucket of paint can obscure a windshield for a few bucks. Not sure this is really news-worthy, even with the "good point" made that sometimes sensors aren't perfect...
Seriously, who autoenables autonomous driving but a fool.
Now, if you'll excuse me, I'll go back to using my 60 mpg stick vehicle that doesn't bleed security info as it was made in 1989.
-- Tigger warning: This post may contain tiggers! --
Researchers managed to get autopilot drunk or texting, otherwise, i'm not impressed and Autopilot proves to be way better than us
There is another way to beat auto driving - reprogram traffic lights.
BTW it's illegal. Anyone doing it is not a "researcher", they are a felon.
Yes, someone going through great effort can cause a crash. I've know cases where people stoodn on overpasses and threw down bricks to cause crashes. Nobody published papers on the "brick loophole" in car security. In most of the examples, it'd have been easier to just cut the brake lines. But we have to target the sensors to get media attention, for a non-story.
Learn to love Alaska
There are plenty of optical illusions for people, too.
Unless this was done on purpose/malevolently (and that could be prosecuted regardless), this seems to me far different from things like hacking into the car's computer itself.
I wanna know how to fool youtube to keep them from taking down the vids I'm trying to watch
So it's easy to do an DOS attack to auto drive cars just wait for the days when you can drop an $40 box and shut down a major road
For that much money (and even with an order of magnitude less), there are probably a ton of ways to fool humans. What exactly are they accomplishing here?
Researchers discover that for $100 they can dig a hole, cover it with a thin layer of asphault and potentially kill a driver.
In other news, researchers also discover that $3 hedge clippers can cut a brake line endangering drivers.
Researchers discover that $10 high powered flash light carefully timed at a blind corner can confuse a human driver's imaging sensors.
The last sentence sums up what Elon Musk has been saying about AutoPilot:
"even though Autopilot is quite capable, there's still no substitute for an attentive human driver, ready to take control at a moment's notice."
The technology is not called "self driving" - it is called autopilot. Similar to plane where course and speed are maintained. Tesla reminds users to keep hands on the wheel and remain attentive.
No news here. Couple that with the cost of the hack, and there is not much to report. I could fool a real driver with mirrors and some Acme landscape canvas.
So not something you would ever encounter.
We aren't worrying about people using high end equipment to make a car crash.
We are worrying about a passive object that is shiny and reflective in just the right way to make it invisible.
Or an active object the size of a small house that because of the light colored paint appears invisible on the horizon.
Those are legitimate concerns at this point.
ZOMG system crashes when exposed to light at 572nm flashing at 13.37HZ while the left rear window control is pressed in the down direction while the car is driving upside down in reverse on july 22 2017 at 2:32.021PM in the rain while being struck by lightning.
Is not however anything anyone is or should be concerned about at this point.
Minimum threshold fixed. Thanks!
That it takes $90,000 worth of equipment and then always doesn't work right is pretty darn impressive to me. Where I live a good 30 percent of drivers are too old to be behind the wheel and another 10 percent are functional alcoholics. Share the road with south Florida drivers long enough and you'll be begging for autopilot.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
Obviously, you want a self-driving car to have the best possible auto-pilot technology you can put into it. But purposeful attempts to trick it into not detecting objects, or into thinking objects are there that aren't really there? That means nothing, IMO. What matters is that it does a reliable job of these things in real-world situations where nobody is TRYING to fool the system.
Human drivers see things all the time and misinterpret them. (There's that popular photo going around social media where someone painted a tunnel on the side of a concrete wall and a car tried to drive off the road, into it, for example. And certainly, people have reported mirages ahead of them on roads for decades.)
Did they paint a tunnel in a wall?
Warning! Keep Out of Eyes! Wash Out with Water! Don't Drink Soap! Dilute! Dilute!
Mirages and heat haze can cause the exact same false positive and false negative responses in human drivers.
And that is the reference here. Automatic drive systems do not need to be perfect to be a good replacement. They just need to be better than the average driver. They will start to safe lives when they are better than bad drivers though.
Of course, everybody believes themselves to be good drivers, but the simple statistical reality is that most are in the range from somewhat above average to really bad.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
http://www.mandatory.com/2016/03/16/a-graffiti-artist-painted-a-road-runner-tunnel-on-a-wall-and-som/
Basically.
I was thinking more of painting the road lines into a rock wall, then painting a picture of a tunnel on said wall.
(Then waiting for a roadrunner to come by)
I would guess that the flawed auto-pilot is already, overall, safer/better than an attentive human driver. True, it makes mistakes, but I suspect it makes fewer, less dangerous ones, and is much more consistent than any human. Momentary lapses of attention? Looking the wrong way? Being impatient? The systems suffer from none of these.
The butthurt here is so strong. God I hate you self driving car faggots. Little bitches every one of you.
Cheaper still, 3 kids, 1 brick, 1 bridge, bang!
Happens all the time, just now it costs a lot more
HAHA I am coming for you! You will never get me! You can't prove shit! HAHAHAHAHA. Anybody who owns a Tesla MUST DIE!
And this accomplishes what exactly? What are the chances that some idiot is going to set up a $90k signal generator in the middle of a damn freeway, to try and trick the sensors on a vehicle, and without any guarantee of success. And what would be the point?
Show me a real world condition that could cause a problem with the car and then you can crow about something.
0.o "Autopilot" literally means "self piloting".
Seriously Tesla/Musk apologists words mean things - and autopilot does not mean what you keep claiming it does. It doesn't mean "an assistant which still requires constant human monitoring and supervision", it means "an automatic system that replaces human operators". From the first line of the Wikipedia entry on autopilots entry - An autopilot is a system used to control the trajectory of a vehicle without constant 'hands-on' control by a human operator being required . (Emphasis mine.) And if you go searching dictionary definitions, you'll find endless repitions of the same themes. Etc.. etc... *That* is what the general public (correctly) believes an autopilot does - automatically operates the aircraft.
Tesla is using the word in a misleading fashion. And if this were Toyota or GM, the tech hipsters would be breaking out the torches and pitchforks.
*Dons Nomex suit* Flame away at the truth, I've got karma to to burn.
If the goal is to create an illusion to fool the driver - we already know how to do that with human drivers. But why would we want to fool any driver?
Couldn't you just use black tape to cover the road markings and white tape to make new ones, to veer the car off the road and over a cliff?
Quote: "... there's still no substitute for an attentive human driver, ready to take control at a moment's notice."
Sorry, but it's easier to simply drive a car yourself than to seat there, trying to pay attention so you can be "ready to take control at a moment's notice."
And notice that in this case, "moment's notice" means that split second before the car smashes into a telephone pole.
"there's still no substitute for an attentive human driver,"
Oh yes there is. It's this complete denial of the bleedin' obvious that will keep killing thousands and injuring 100s of thousands every day. The sooner we ban humans from driving vehicles, the better.
Don't need bricks; get a scantly-clad attractive female* to walk down the side of a busy road. I can think of several times where such distractions almost got me into a tangle. And it's legal, unlike bricks.
I'd be happy to assist in such research by inspecting the applicants for free.
* Or scantly clad males might work also, who knows. Didn't mean to be discriminatory. (Insert obligatory Commander Taco joke here.)
Table-ized A.I.
To be fair. that $90k worth of Keysight signal generator can be purpose-built with about $20 in modern parts. Fucking kids these days want to push a button and have magic happen. And fuck the major manufacturers gouging us to pay for their crappy firmware devs. One day a smart person will realize they can give their software talent to a Chinese manufacturer and sweep the field clean (talking about test equipment like oscilloscopes, power supplies, and signal generators). And no, not you Mr Engineer that thinks he knows how to write software. I'm talking about an actual software developer that probably doesn't even know how to work an oscilloscope.
Autopilot may "literally mean self piloting", but at no point in any industry has it *ever* been used to mean that. And every time you start the car and enable the feature, a little disclaimer pops up that reminds you of this. It's pretty clear you've never been inside a Tesla. In which case, why are you even talking?
"Come, Watson, come. The game is afoot."
"But, Sherlock, if they were in a car, then they weren't on foot."
For that kinda scratch, I'm pretty sure we could rig something up that fools a human driver too.
This signature is false.
Basically.
I was thinking more of painting the road lines into a rock wall, then painting a picture of a tunnel on said wall.
(Then waiting for a roadrunner to come by)
After all, Coyote v. Acme was this country's longest running product liability suit. Though the plaintiff, a partially disabled male Canis latrans, meticulously documented his problems with Acme's mail order line of bird-trapping hardware in a long series of filmstrips and videos, his evidence was leaked to the public, exposing Coyote to generations of ridicule. Most recently the Ninth Circuit sent the case back to lower courts, ruling that Coyote had no standing to invoke the Americans With Disabilities Act, since the ADA applies only to humans.
Wow. Talk about reaching. I'm no tesla fan boy, but saying you can trick a radar system with RF equipment is like saying you can talk down a building with dynamite. Of course you can! In fact there exists RF test sets specifically for simulating radar targets for easier system validation.
I'm musing this 'attack' is silly.
Per the other posts, let's start firing chaff from a vehicle driving in front of a Google car on the 5 doing the speed limit. Hijinks ensue.
I guess 'why just Tesla' is cuz I can't lay my hands on a Google car?
People are susceptible to optical illusions. So are machines when you understand the assumptions made. People and machines don't have to make the same assumptions but each is fallible in it's own way. As they say nothing is perfect. No one and no thing is perfect. But is it good enough? Or which is better?
... by a painted tunnel.
That that is is that that that that is not is not.
You can also blind a human driver by shining a sufficiently bright light in the eyes. This fact has not caused anyone to say, "even though human drivers are quite capable, there's still no substitute for a good horse."
That that is is that that that that is not is not.
>"there's still no substitute for an attentive human driver, ready to take control at a moment's notice."
That would be utterly useless. Try "at a fraction of a second's notice". And that simply won't happen once one is in any type of autopilot mode.
Researchers learn how to "fool" a human driver into crashing by aiming a big laser/floodlight into their faces when they're about to enter a segment of road with a sharp turn.
> After all, Coyote v. Acme was this country's longest running product liability suit.
I don't see where he'd have standing to sue under the ADA in any case, since Wile. E. Coyote _won_ his lawsuit for manufacturing defects in 1990.
http://www.newyorker.com/magaz...
Yay, signal sensing equipment can be fooled by sending it fake signals!
Oh, and rain is wet.
I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
Actually researchers discovered that beta version of a system is not ready.
You can always DoS a human driver too. Laser pointers, or even perfectly natural feminine breasts could distract a human driver.
Patents Drive Free Software as Hurricanes Drive Construction Industry
You keep using that word, capable. I don't think it means what you think it means.
Human drivers can be jammed or spoofed, too. Laser pointers, mirrors.... All possible, and all highly illegal. I don't see news reports highlighting deaths caused by hoodlums sat by the freeway with mirrors reflecting the sun into drivers' eyes. Why would this be any different with autonomous vehicles?
The more interesting point is that given this can happen, whether by artificial means or by some natural accident, how to make the cars notice and adopt an alternative driving strategy - use another sensor, slow down, whatever - until the jamming / spoofing conditions cease, in just the same way that a human driver blinded by sunlight would slow down, stop, filter the sun with sunglasses or a visor etc. Given that humans just have eyes whereas cars have sensors working at really diverse parts of the EM spectrum - LIDAR, optical cameras, mmW radar - it should be a lot harder to completely jam (or spoof) a car than a human, because you would need to jam (or spoof) all of the sensors at once. The car should certainly be noticing discrepancies between what its various sensors are telling it.
There are plenty of companies who do anti-jamming / anti-spoofing tech for the military, I would imagine they are now in a good position to support the auto industry in making their products as resilient to adverse EM conditions as possible. Just another example of trickle-down from high end military R&D.
News at eleven. Specialists using radio equipment is able to simulate objects on a radar, a technology also based on radio waves. Also, the world ends in about one hour.
I'm sure there are much better ways to "fool" it with much more low-tech items.
How does it distinguish the density of an object it detects? Does a paper bag blowing across the road get marked as something it can drive over or will it take evasive action? Where does it draw the line between running over the paper bag and steering into the car in the next lane to have what it might think is a "softer" crash? And how does it tell the difference between a bag and a child, or even a pigeon?
Isn't most of its autopilot visual rather than actual sensing? So how does it determine whether or not the bag is solid (a paper-coloured rock in the road), or not?
There are just too many scenarios for a blanket rule, and - as in the blasphemous I-Robot movie - the wrong decisions will be made.
Once you start including deliberate interference, these things just get dangerous.
I recall a popular science magazine from the early 90s, detailing automated, self-driving carts. They were guided using radio beacons in the road surface, and AFAIK are still operational carting shipping containers around in the harbour of Rotterdam. They could (and generally do) assume "no untrained humans around", but still have some sensors to avoid running people over.
Anyway, the rest of the issue was about the problems of using various sensors to detect objects, with early 90s tech. Seeing how much tech has improved in the meantime might be interesting.
Of course, that's not how it gets presented. It has to be bandied around in a breathless attention-seeking manner like everything else vaguely related to "computer security". To the point you have to be glad they didn't say "it was hackers with hacks that did it!" That makes it just as useless as the entire computer security cottage industry.
But as an informal benchmark of how far we've come it mightn't be bad.
As a driver would never notice their foot going straight to the floorboard when they had to apply the brakes to shift the car out of park.
one could argue that you could easily blind a human pilot with a floodlight, he will also not know if there was an object in front of him or not ..
So, would it be anywhere near as difficult to distract or confuse a human driver and accomplish the same thing?
I mean a loud sound from a foghorn at the wrong moment, that is too close. A ringing cell phone little lone intentionally camouflaging a barrier with say paint.
It sounds like it is MUCH harder to endanger people when the system is driving than when a human is driving, make the computer a much safer driver , at least for the type of thing being tested.
Actually to play devil's advocate here...a smaller cut would give you similar feeling on the brake pedal for light braking and the fail catastrophically (or at least not provide adequate breaking) under emergency braking...or when you finally emptied all the brake fluid.
Point being, there are malicious failure methods available that work just fine on plain old humans. Simple, cheap, easy ones. To say autopilot is less safe than a human drive because other, complicated and technical, things can fool it is misleading and disingenuous. Autopilot, especially in it's more advanced iterations that are coming, is highly likely to be a much, much safer driver than people in the vast majority of situations.
Will some circumstances fool autopilot and cause an accident? Yep. Could some of those have been avoided by a human driver? Yep. However, when you compare the number of times a computerized driver doesn't something wrong vs. a human I've got my money on the computer. They don't play pokemon go while driving, txt, get drunk, etc.
You can get rich if you own a politician, but you have to be rich to buy one in the first place.
A crack that allows you to control a large number of vehicles IS important and needs to be solved
OTOH, a hack in which you modify a single car to interfere with another single car's ability to drive is absolutely WORTHLESS. Why?
Assume that somebody is going to go after another person because they are rich or something. Then said person likely is driving themselves OR has a chauffer.
Let assume that this is used to go after a kid or a neighbor, etc. IOW, you just want to punish somebody. Now, you have to spend 100K to do so.
How many of you will do so? None.
This is your typical CHinese BS in which these ppl are trying to make themselves look good by making wild ass claims.
With that said, the real culprit on these was NOT Telsa, but Mobileye. And in this case, Tesla is RUNNING away from them. IOW, they know that they are too expensive for what they deliver.
I prefer the "u" in honour as it seems to be missing these days.
Seem to be aimed at the current Tesla autopilot is the end all evolution as far as autonomous driving is concerned? I personally would believe that this is just a starting point . It is not the end. This stuff will get better and better as time goes on . That is why as a human driver you have to be ready to take over . If someone has a better idea please tell me where my thinking is wrong
Paywalled, but whatever pittance Coyote may have won in the lower courts after being rejected by the Ninth Circuit was Pyrrhic. When Sanyo Portable Hole K. K. of Akihabara, Tokyo, became Roadrunner's exclusive supplier of these highly effective countermeasures to Coyote's use of Acme's flawed devices, Coyote's medical bills from running into unexpected canyon walls made it impossible to continue work. He now resides in an assisted care home in Tusayan, AZ, and is permanently barred by the Parks Service from ever revisiting the adjacent Grand Canyon.
Driving people have blind spots, so illusion is not required.
Faux road runner tunnel causes accident.
http://www.inquisitr.com/2638705/road-runner-faux-tunnel-painted-on-wall-causes-car-crash-gets-2-3-million-views-in-6-hours/