Slashdot Mirror
Driver Killed In a Tesla Crash Using Autopilot Ignored At Least 7 Safety Warnings (usatoday.com)
An anonymous reader quotes a report from USA Today: U.S. investigators said a driver who was killed while using Tesla's partially self-driving car ignored repeated warnings to put his hands on the wheel. In a 538-page report providing new details of the May 2016 crash that killed Ohio resident Joshua Brown in a highway crash in Florida, the National Transportation Safety Board described the scene of the grisly incident and the minutes leading up to it. The agency, which opened an investigation to explore the possibility that Tesla's Autopilot system was faulty, said it had drawn "no conclusions about how or why the crash occurred." The NTSB report appears to deliver no conflicting information. The agency said the driver was traveling at 74 miles per hour, above the 65 mph limit on the road, when he collided with the truck. The driver used the vehicle's self-driving system for 37.5 minutes of the 41 minutes of his trip, according to NTSB. During the time the self-driving system was activated, he had his hands on the wheel for a total of only about half a minute, investigators concluded. NTSB said the driver received seven visual warnings on the instrument panel, which blared "Hold Steering Wheel," followed by six audible warnings.
What we know from this incident;
1) The driver was responsible for the accident because he didn't maintain control
2) Tesla Autopilot was not good enough on its own to prevent the car from driving into the truck.
Of course he was going to ignore a warning that said, "Hold steering wheel."
Instead, the car should have said:
"What the hell are you doing with your hands off the wheel, you idiot???! Are you trying to crash? Do you want to die? Do you want to make your kids orphans?"
The warnings could get increasingly forceful as the car complains that its own safety is being jeopardized.
"I don't want to go to a body shop. They use hammers! Kill yourself if you want, but leave me out of it."
The accident was therefore Tesla's fault.
Nothing. Not a damned thing. Someone this stupid was going to take themselves out of the gene pool, sooner or later. I'm sometime baffled that we've managed to keep ourselves from going extinct.
"So long and thanks for all the fish."
Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.
Because they trusted that the owner of an $80,000 car had at least some minimal intelligence and even if the driver had blind trust in the car, that when the car says "put your hands on the wheel and pay attention", that the driver would listen.
Yet this driver has demonstrated that people are about as dumb as you think they can be, so now they've implemented a 3 strikes policy that disabled autopilot after 3 reminders.
With current technology, people are trained to ignore meaningless errors.
Proper error handling for anything important require you to take action, especially if you repeat the error.
That is, if they want people to pay attention to a "keep hands on wheels" warning, the speed should drop significantly. Not as if the brake was applied, but instead as if the foot was taken off the gas (even if they tried to floor it.). Oh, and the brake light should flash to let people behind know you are slowing, even though no brake is applied.
excitingthingstodo.blogspot.com
Tesla's partially self-driving car
A partial self driving car is like sorta being pregnant, the car is either self driving or not, any grey area = not.
This seems like Tesla getting the public to do QA for them untill they have a fully self driving car, it's clear the public does not know what "partially" means...
If that's a requirement, why didn't the car just pull over and shut off?
Because the car isn't smart enough to do that. It can keep you between the lines on the road; it can't take you out of the lanes and park you up. That's actually a harder thing to do.
systemd is Roko's Basilisk.
Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.
Because they trusted that the owner of an $80,000 car had at least some minimal intelligence and even if the driver had blind trust in the car, that when the car says "put your hands on the wheel and pay attention", that the driver would listen.
Yet this driver has demonstrated that people are about as dumb as you think they can be, so now they've implemented a 3 strikes policy that disabled autopilot after 3 reminders.
For the first few days people will be extremely cautious letting the autopilot do anything.
For the first few weeks they'll give it more leeway, but be very attuned to any warnings it gives.
After a few months, if they haven't had any real scares, they'll assume the auto-pilot knows what it's doing and generally ignore warnings.
Some people will be more cautious, but as a software developer this is exactly what I expect to happen with a significant portion of people. Everyone knows the right thing to do, we should backup our data rigorously, always use good unique password, follow the proper procedures, etc. But that's not how people work. If it's not part of a routine, and it's not given an immediate payoff, then people won't do it.
Give people a car that can self-drive in some situations and they will inevitably let it self-drive in every situation they can.
I stole this Sig
Because they trusted that the owner of an $80,000 car had at least some minimal intelligence and even if the driver had blind trust in the car, that when the car says "put your hands on the wheel and pay attention", that the driver would listen.
What if I go unconscious in my $80,000 car? Perhaps I have a heart attack, stroke, or simply had an unexplained fainting spell. So the car just beeps at me uselessly until it drives into something? Might as well use a traditional car, set it to cruise at 75 mph and take a nap because the end result will be the same.
“Common sense is not so common.” — Voltaire
Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.
The real problem is that Tesla's "Autopilot" is the worst possible solution to the problem. One of the realities of the human brain is that shifting your attention is hard... To put it in computational terms, the context switch is expensive. Even when the car is doing the driving, it theoretically requires the driver to be auditing it, and be paying attention to their surroundings in case things go sideways. Concentrating on something that you're not actively participating in can be quite difficult, as we already know. If the driver is actually, well, driving, they are already hopefully paying attention to the road and their surroundings.
Until we get self-driving cars that can run without even having a steering wheel, it is a bad idea to have the computer half in control. The better solution is to work the other way. The driver does most of the driving, and the computer only takes over in emergency/collision type situations (avoiding that box on the highway, the idiot merging into your blind spot, etc...)
...si hoc legere nimium eruditionis habes...
If it's "Far from autonomous", they shouldn't be calling it autopilot.
Seems like an appropriate name, actually. A plane on autopilot also just keeps course and altitude, but doesn't fly around storms, or start an automatic landing procedure when fuel is running low.