Slashdot Mirror
Driver Killed In a Tesla Crash Using Autopilot Ignored At Least 7 Safety Warnings (usatoday.com)
An anonymous reader quotes a report from USA Today: U.S. investigators said a driver who was killed while using Tesla's partially self-driving car ignored repeated warnings to put his hands on the wheel. In a 538-page report providing new details of the May 2016 crash that killed Ohio resident Joshua Brown in a highway crash in Florida, the National Transportation Safety Board described the scene of the grisly incident and the minutes leading up to it. The agency, which opened an investigation to explore the possibility that Tesla's Autopilot system was faulty, said it had drawn "no conclusions about how or why the crash occurred." The NTSB report appears to deliver no conflicting information. The agency said the driver was traveling at 74 miles per hour, above the 65 mph limit on the road, when he collided with the truck. The driver used the vehicle's self-driving system for 37.5 minutes of the 41 minutes of his trip, according to NTSB. During the time the self-driving system was activated, he had his hands on the wheel for a total of only about half a minute, investigators concluded. NTSB said the driver received seven visual warnings on the instrument panel, which blared "Hold Steering Wheel," followed by six audible warnings.
Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.
Maybe he was already dead.
What we know from this incident;
1) The driver was responsible for the accident because he didn't maintain control
2) Tesla Autopilot was not good enough on its own to prevent the car from driving into the truck.
but what could have been done to prevent such a tragedy?
Of course he was going to ignore a warning that said, "Hold steering wheel."
Instead, the car should have said:
"What the hell are you doing with your hands off the wheel, you idiot???! Are you trying to crash? Do you want to die? Do you want to make your kids orphans?"
The warnings could get increasingly forceful as the car complains that its own safety is being jeopardized.
"I don't want to go to a body shop. They use hammers! Kill yourself if you want, but leave me out of it."
The accident was therefore Tesla's fault.
After a couple of warnings why didn't the Tesla just pull over and stop? Possibly issue a 911/medical call as the driver could have had a medical issue etc.
but that they got enough data out of the car's computers to write a five-hundred-plus page report on the incident.
You will never know, because you can't ask the guy the car killed.
In theory that's how they're going to make auto-pilot good enough that it doesn't need human intervention......collecting data over time to avoid more and more problems until they approach the limit of zero problems.
"First they came for the slanderers and i said nothing."
This reminds me of the Toyota failures, where the car's blackbox confirmed that the car was indeed accelerating because the driver accelerated.
Without seeing the code, how do can we know that the car doesn't just record such things even when it shouldn't record them. In the case of the "unintentional acceleration" claims against Toyota, a lot of it probably was user error and many more of those cases were probably because the floor mat got the accelerator stuck (that happened to me in a 1999 Volkswagon Passat, which was scary as hell until I figured out what was happening after I physically turned off the car so that it would stop accelerating "on its own").
I am not trying to say that this guy died because of Tesla, nor that Tesla did anything wrong in this scenario, but how do we know that the car didn't record that it flashed the warnings or audibly signaled the driver? Certainly anyone that has written code has done something stupid like this in their career and it slipped through QA:
try {
flashWarning();
} catch (Exception e) {
cleanUpAndIgnoreException();
} finally {
recordWarningWasShown();
}
This is my fear with these black boxes in our cars!
I have a 2016 F150 that has lane sensing, and it continually alerts me to put my hands on the wheel while driving. The issue with that is my hands have been on the wheel the whole time, at least one of them. So what happens when I get into an accident, the black box is pulled, and it says I wasn't holding onto the wheel?! Who is going to believe the human vs the computer?
I realize this isn't a fancy Tesla, but I think it is worth considering!
Retard driver doesn't pay attention and kills himself. More at 11
With current technology, people are trained to ignore meaningless errors.
Proper error handling for anything important require you to take action, especially if you repeat the error.
That is, if they want people to pay attention to a "keep hands on wheels" warning, the speed should drop significantly. Not as if the brake was applied, but instead as if the foot was taken off the gas (even if they tried to floor it.). Oh, and the brake light should flash to let people behind know you are slowing, even though no brake is applied.
excitingthingstodo.blogspot.com
Tesla's partially self-driving car
A partial self driving car is like sorta being pregnant, the car is either self driving or not, any grey area = not.
This seems like Tesla getting the public to do QA for them untill they have a fully self driving car, it's clear the public does not know what "partially" means...
"Vertically oriented 7” laceration extending from superior right orbit into posterior right parietal scalp, empty cranial cavity visible"
The driver had an empty cranial cavity
Republicans like Musk be.
Even the best engineers in the industry can't fix stupid.
"The driver used the vehicle's self-driving system for 37.5 minutes of the 41 minutes of his trip"
"Driver Killed In a Tesla Crash Using Autopilot Ignored At Least 7 Safety Warnings"
If your autopilot misses a truck, how can you say for sure the driver didn't die 30 minutes before the crash?
https://en.wikipedia.org/wiki/...
Good then. Glad he died if he's that stupid.
Since it is common practice to ignore that warning and it is expected to ignore it, the warning might be not legally significant.
that the prompts were just there to make Tesla blameless and they implemented the policy after realizing that you can't be blameless when you kill people who can afford an $80k sportscar.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
If that's a requirement, why didn't the car just pull over and shut off?
The main reason is that in this exact type of car (Tesla Model S) pulling over - specifically, the "change lane" part of it - can't be automated 100% yet.
The car sense cars in neighboring lanes (BLIS - Blind Sport Information System) by using its ultra-sound sonars. Basically a type of souped up parking assistance with a little bit more range.
This is good at detecting whether the space right next to the Tesla is free, this isn't that good at detecting incoming cars in the lane.
(i.e.: if the Tesla only trusts its sonars as currently done, there's a risk that it cuts of someone)
That why currently the car will never initiate a lane change on its own, the driver is required to action the blinker to initiate a lane change.
(Compare with camera-based BLIS, like the past couple of year in Volvos. The camera has much longer range and can actually spot a car incoming from far behind. These Volvos don't change lanes by themeselves, but at least are able to bring to attention to the driver cars not only in the blind spot but even further back in the lane)
Also, I suspect that the machine learning / deep net in Tesla doesn't have a notion/concept that the shoulder lane is something different and special.
(i.e.: I mean it can't even *target* the shoulder lane as a destination to change lane into, because it won't necessarily recognize it as such).
Note that none of the above is a hard obstacle :
- as mentioned, camera based BLIS - that can look much further back and realise if there's incoming traffic - do exist and have been in circulation for a couple of year. And I know that at some point in time, Tesla was toying with the idea of camera at side mirrors (except that they wanted to completely drop the side mirror and only use cameras/virtual mirrors - This would have been much more difficult to get past regulations).
- training the system to recognize shoulder lanes shouldn't be that much complicated.
- it's probably an emergency situation (if the driver hasn't been able to touch the wheel for the past 30 minutes, he might not even be conscious). It would be acceptable for the car to blink and honk to notify that something is wrong so other driver can see and take evasive manoeuvre.
- Bonus point if the car can automatically drop a triangle 100m before stopping on the shoulder lane for safety (or whatever is the required distance in your jurisdition). And Elon Musk is the kind of geek who'll pay special attention to these kind of small details (see the uncanny automatic charging snake).
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
That Humans do NOT, have as many lives as a cat... only 7 :-P
I wonder how many of these message that the drive had heard and learned to consider them as "false alerts"?
If the car is not smart enough to see a truck by itself,
This may also not have been a problem of smart, but a problem of sensors not registering the truck in this peculiar situation.
(e.g.: unlike other cars, Teslas only have forward facing radars and 2D video camera, no LIDARs, nor 3D-pair of stereo cameras.)
it may not be smart enough to find a safe space to pull over and shut itself off.
It's definitely not as much a problem of smart as it is of sensors :
whereas some other cars feature read-facing video cameras under the side mirrors (that can see if there's a car in the next lane much further back),
Teslas are among the car that exclusively rely on their ultra-sound sonars (basically souped up parking assistance with a bit improved range) which are much shorter-range. They can reliably tell you when a car is in the next lane or if there's space right next to the Tesla, they can't see if there's a car coming from behind in the lane.
With its current set of sensors, Tesla definitely can't change lane without risking to cut someone of.
That's part of the reasons why the driver needs to signal his request by using the blinker (driver checks blind sport and behind on the lane, and only then pushes the blinker lever).
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Indeed he seems to have been negligent, but clearly Musk's warning system is also inadequate. There needs to be no question a person needs to take the wheel, a flashing cartoon isn't going to cut it. Musk is culpable too, and if I were an attourney I'd go for his throat based on that.
* Why didn't the car slow down if you have your hands off the wheel for more then 5+ minutes?
slowing down brings a slight risk of getting rear-ended, specially in the middle of other wise fluid traffic.
that's why I suspect that the engineers at Tesla erred on the side of not stopping.
* Why didn't the car's sensor detect the impending crash?
Tesla in particular lack LIDARs and/or 3D-pair of stereo video camera.
They only have a forward facing 2D video camera and a radar.
These might get slightly confused by distance and size and might confuse the truck with a street sign much further.
(Highway signs are rather reflective and might seem closer on the radar that they actually are.
A truck - i.e.: a big close signal - could thus be confused with artifacts caused by a highway sign - the car might think that it could be looking closer than it is, even if actually the radar was right and the truck is indeed closer)
(A 2D camera would be poor at judging distances).
(On the other hand, because they have accurate Z coordinate, stereo cameras and LIDARs could more reliabily confirm that this is indeed a closer object (a truck right in front of the car) than expected).
* Why didn't the car pull over the side of the road after 15 minutes of hands free driving?
Sensors/safety again.
Tesla only uses ultra-sound sonars (souped up equivalent of parking assistance) to check nearby lanes for free space to change lane into.
Backward facing camera - as used in some other cars - would be better at spotting a car coming from far behind and avoid cutting it of.
With its current set of sensors, a Tesla can reliably change lane safely.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Buying a car with autopilot seems like buying a computer that can only work with three digit numbers.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
- add a forward facing LIDAR and or 3d-pair of stereo camera (instead of 2d), to supplement the current radar and 2d camera.
This would help the car disambiguate between a truck just in front, and a highway sign much further in the same direction - both could look similar to the current set of sensors)
- add backward facing cameras under the rear-view mirror :
this would help the car change lane safely.
Means that it could cover the scenario "user doesn't touch the wheel for the past 30min despite alarms" => "user has probably passed out / is unconscious / etc." => "telsa should automatically pull to the side" => "means must be able to change lanes until side is reached"
(The car should also automatically call emergency services, and should blink and honk to attract attention)
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
...give people the chance to have a self-driving car, and 99% of them will use so they can fall asleep in the back seat while driving on the highway. Fucking idiots.
I think this AI assisted driving should be taken out of the cars. This technology is not ready for the real world. It's just too tempting to turn this gadget on and doze off or some other totally stupid behavior, thinking the car can deal with driving on it's own.
Having this incomplete technology in service could hamper efforts to convince government entities that self-driving cars can and will be safe, when an immature technology is turning out to be not so safe. And trust me, regulators are looking at this and saying to themselves, "If this can happen, this technology is not safe."
Trust me, I want a self-driving car like yesterday, but the technology needs to mature more, more testing, in more situations needs to be done before this is ready for the end-user who's going to take a nap while his/her car drives itself.
That's the thing about the Tesla drivers don't realize. There is no more "He said, she said" with a Tesla. That car will tattletale every single little detail and if the driver is at fault then it'll be known for the world to see.
Given the history of how to handle inattentive drivers on machines that require very infrequent action, they should have designed the auto pilot with random reaction testing alerts and challenges.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
So do these cars have a "black box" in them like airplanes do? Inquiring minds want to know.
Since 'self-driving systems' aren't really self driving yet, perhaps the best thing is to stick to selling manual drive cars until it is actually possible to build cars that can safely navigate unattended. We already have too much of a problem with distracted driving, so it's better if everyone just 'knows' they are actually expected to pay attention when getting behind the wheel.
If the DC Metro couldn't get autonomous driving down (for a train that's on tracks (much fewer variables)), why on Earth would we think that autonomous cars are a good idea? Or, in general, that the whole highway system was the right choice to move people around the country. A distributed system like this simply doesn't work at scale.
That's dangerous to others on the road, but then so is a sudden accident. I wonder what's best.
Twinstiq, game news
It seems to me that people are going to have to get trained to know what the warnings are and what to do with them. Maybe Tesla does this already?
Maybe it will have to start becoming part of the Driving School tests.
You may hear the warnings, but that doesn't mean you know what to do.
Bad interface and interaction designs do not help people understand to complexities of the information.
It's quite annoying.
The other annoying aspect of Autopilot is when you're in the #1 lane next to the median barrier (and very close to it) - and there's a slightly wider section of median - the car wants to swerve into the adjacent #2 lane.
Poor design killed this man. If you don't think so, consider ... the car knew the driver didn't have his hands on the wheel for 37 minutes. Then why is it still doing the driving for him?. It had about 37 minutes to come to a slow, safe stop with blinkers and headlights flashing and horns blaring. Instead, Autopilot kept driving and eventually made the wrong decision that killed this man!
Before people start mumbling 'Darwin Awards', consider this: your daughter may be the next passenger in a Tesla. Or a Tesla may T-bone you, not a semi-trailer, next time.
This (and throw in hazard lights for good measure).
The mind boggles at all the people fellating Musk here claiming that slowing down only makes things worse somehow.
A "partially" self-driving car only has a benefit if i can take my mind of the road. Needing to be aware of the situation at all times is what makes driving exhausting, nudging the steering wheel every once in a while isn't.
So if there is an accident about to happen it doesn't matter much if my hands are on the wheel when I'm daydreaming.
If I'm not aware of the situation I can't react, and anyway I first need time to asses the situation before i can react. If my assessment is wrong (e.g. I'm not aware of another car in the next lane) my intervention might even be the cause of an accident.
In principle, to handle this "partially self driving car" safely, not only do I need to be aware of the traffic situation as if I were driving, I'd also need to know the systems assessment of the situation.
"By the way if anyone here is in advertising or marketing... kill yourself." -- Bill Hicks
He deserved to die, because he is that much of an idiot. He was doing this purely so he could get injured and sue the company. He didn't pay with his life, because his life is clearly worthless as he's the kind of people that actively, willfully, compulsively looks for things without warnings on them and deliberately injures himself for money, pity, and attention.
This is a net gain for society.
Slow down how much? And if nothing changes, then what? Come to a stop in the middle of the road?
Of course, yes.
If you ever find yourself in a situation where you are no longer able to safely drive your vehicle for any reason your first course of action is to stop it from moving, and of course activate your hazard lights.
If the AI was not able to pull over then the correct action would be for it to just stop.
In the middle of the road or freeway? Oh no you don't. You always pull to the side of the road. Sitting in the middle of it with hazards or not and you will end up causing an accident and you WILL be held to blame no matter what.
Last time I drove one of those, it was more annoying than it was worth... The damned little light kept coming on as I was driving a winding mountain road, telling me there was something in my blind spot. No shit; it's called a mountain.
That's typical with ultra-sound sonar based BLIS.
The sonar in cars was developed initially for park-assistance (or later autonomous parking), so it was chosen because it can detect *any* object in the way (anything that exists will register on the sonar. Not only cars, but even mountains.
Which makes sense for parking, because you don't want the car to hit the wall next to your parking place.
But is completely unadapted for BLIS)
Camera BLIS are a bit better.
The camera detect vehicle-shaped things.
It will not blink in case of a mountain or a wall, it's not an incoming vehicle on the side lane (and detecting lane where you can drive into is already the job of the front-facing camera and lanes-departure-alerts. If you're driving into the mountain instead of another lane, that alert should be already sounding).
It's not perfect though, specially most manufacturers use the same off-the-shelf part from Mobileye which are based on a bit older and less performant machine learning algorithm than the current edge (Deep neural nets) and thus might miss identify object :
- the oldest Mobileye platform couldn't recognize bicycles.
- on my parent's older Volvo, under very weird lighint conditions, it sometime mis identifies its own reflection (or even its own shadow, according to the book), and has occasionnally blinked while I was driving next to a plexiglas "sound barrier".
Some cars (I've seen it with french manufacturer, but it exists elsewhere too) go the other way around :
they supplement their parking assistance with a simulated "top view" around the car obtained by distorting the video feed of the cameras around the car).
During the day it was no big deal, but at night, that light blinking on and off in the peripheral vision was quite distracting.
For me, it looks mostly a UX / UI problem :
- BLIS should be something that you notice when looking sideways before changing lanes,
- BLIS should NOT be something that attracts too much attention when you look forward.
(And in my parent's Volvo, indeed, it's not distracting. At least that car got it right).
They should make the light dimmer (follow the setting of the dash-board light dimmer).
Eventually only lighting up when really needed (when the turn signal/blinkers are on. And/or when the lane-derparture alert notice that the user is going to change lane without a turn signal)
Anyhow, I know how to set my mirrors properly to minimize blind spots, and I always shouldercheck.
Still better safe than sorry. You can garantee that the human driver will always be perfect forever in absolutely 100% of time. Mistakes could happen one day.
Alerts in modern cars (forward collision avoidance, etc.) are a good way to supplement the human and an extra step to avoid accident.
(Saddly, some humans treat them as "excuse to completely shut of the brain and try dozing as mush as possible").
It would be nice if things like that could be turned off when not needed.
Again a UX / UI problem : shutting it off should be an easy setting, at least in the menu on the infotainment center.
(Or a small button on the dash board as most advanced feature).
But beware, I'm sure that some US insurance companies are going to jack up price if they detect shutting down security features)
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
It's the same as if the driver failed to obey traffic signs.
Tesla doesn't know what partially means then either? Or are their employees not public?
Nonsense. A normal adult with an inkling of intelligence knows what 'autopilot' means in the context of driving a Tesla, just like a pilot knows what it means for his vehicle.
It's described in the manual, it's clearly mentioned when they buy it, and it freakin warns about it every time one uses the system.
THERE IS NO WAY a Tesla driver doesn't know full well what to expect of the autopilot system thus. A 'reasonable person' thus, should expect what is clearly said and indicated what 'autopilot' means for a tesla car (just as pilots know what it means for their airplanes).
Expecting that it does something else as is repeatedly said what it does, is unreasonable. Just because you ignore all of it and prefer to think it means more and something else, based on the semantics of the word, even though you actually know it doesn't, is the epitome of NOT being rational and reasonable.
--- "To pee or not to pee, that is the question." ---
"Steer for yourself or die!" probably would have been a better line for the audible warning.
An autopilot handles routine operations. Every aircraft autopilot requires a qualified human pilot supervising it in case the operation ceases to be routine. The Tesla autopilot works in precisely the same way.
There's nothing in TFA about the status of the driver while this was happening. If the driver were unconscious (e.g.: heart attack, etc.) during the warnings (which were apparently being shouted at him), of course he wouldn't be able to respond. (Seems like a good way to assassinate somebody without leaving a trace. Give them a poison that's difficult to detect after the trauma of a car accident, stick them in a Tesla, and hit the autopilot. Guess they died of their own malfeasance.)
Last month when I test drove the Model 3, AP was apparently only available for speeds upto 35mph. Even at that speed, the car went over double-yellow 3 times, and for the 3rd time, which was the last straw, when I got honked by the oncoming car I nearly rammed into, the dealer says 'hey look, they're honking coz they like your tesla !
Automated self driving cars are years away from being tweaked and fine tuned. I do not see why people want any automated driving features in a car. Hell half the people cant drive non automated wise. Keep both hands on the wheel kids..
The AI could send a notice to the local traffic police informing them of the driver's dangerous behaviour and have them
booked for dangerous driving...
Tesla-1
All other cars-37,897*
*approximately.
I was thinking that something different should happen if the warning was ignored - although I was thinking more along the lines of volume increases, maybe eventually a siren, dashboard lights flashing momentarily, anything that would make the idiot in the seat pay attention and not get themself dead. Maybe that combined with slowing down and maybe eventually coming to a stop if the guy doesn't respond would be a good way to go.
Also what happens if someone has a heart attack while driving? Presumably that's going to result in them taking their hands off the wheel. They want some good press to balance out the bad, start saving lives by making the car coast to a stop (and ideally cause passers by to stop and find out wtf, then call 911/999/000) where in a normal car the person would have just plowed into a tree/embankment/oncoming traffic.