Driver Killed In a Tesla Crash Using Autopilot Ignored At Least 7 Safety Warnings (usatoday.com)

An anonymous reader quotes a report from USA Today: U.S. investigators said a driver who was killed while using Tesla's partially self-driving car ignored repeated warnings to put his hands on the wheel. In a 538-page report providing new details of the May 2016 crash that killed Ohio resident Joshua Brown in a highway crash in Florida, the National Transportation Safety Board described the scene of the grisly incident and the minutes leading up to it. The agency, which opened an investigation to explore the possibility that Tesla's Autopilot system was faulty, said it had drawn "no conclusions about how or why the crash occurred." The NTSB report appears to deliver no conflicting information. The agency said the driver was traveling at 74 miles per hour, above the 65 mph limit on the road, when he collided with the truck. The driver used the vehicle's self-driving system for 37.5 minutes of the 41 minutes of his trip, according to NTSB. During the time the self-driving system was activated, he had his hands on the wheel for a total of only about half a minute, investigators concluded. NTSB said the driver received seven visual warnings on the instrument panel, which blared "Hold Steering Wheel," followed by six audible warnings.

Comments

Simple question

[Anonymous Coward]

Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.

Re:Simple question

[hawguy]

Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.

Because they trusted that the owner of an $80,000 car had at least some minimal intelligence and even if the driver had blind trust in the car, that when the car says "put your hands on the wheel and pay attention", that the driver would listen.

Yet this driver has demonstrated that people are about as dumb as you think they can be, so now they've implemented a 3 strikes policy that disabled autopilot after 3 reminders.

Re:Simple question

[wonkey_monkey]

If that's a requirement, why didn't the car just pull over and shut off?

Because the car isn't smart enough to do that. It can keep you between the lines on the road; it can't take you out of the lanes and park you up. That's actually a harder thing to do.

Re:Simple question

[quantaman]

Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.

Because they trusted that the owner of an $80,000 car had at least some minimal intelligence and even if the driver had blind trust in the car, that when the car says "put your hands on the wheel and pay attention", that the driver would listen.

Yet this driver has demonstrated that people are about as dumb as you think they can be, so now they've implemented a 3 strikes policy that disabled autopilot after 3 reminders.

For the first few days people will be extremely cautious letting the autopilot do anything.

For the first few weeks they'll give it more leeway, but be very attuned to any warnings it gives.

After a few months, if they haven't had any real scares, they'll assume the auto-pilot knows what it's doing and generally ignore warnings.

Some people will be more cautious, but as a software developer this is exactly what I expect to happen with a significant portion of people. Everyone knows the right thing to do, we should backup our data rigorously, always use good unique password, follow the proper procedures, etc. But that's not how people work. If it's not part of a routine, and it's not given an immediate payoff, then people won't do it.

Give people a car that can self-drive in some situations and they will inevitably let it self-drive in every situation they can.

Re:Simple question

[OrangeTide]

Because they trusted that the owner of an $80,000 car had at least some minimal intelligence and even if the driver had blind trust in the car, that when the car says "put your hands on the wheel and pay attention", that the driver would listen.

What if I go unconscious in my $80,000 car? Perhaps I have a heart attack, stroke, or simply had an unexplained fainting spell. So the car just beeps at me uselessly until it drives into something? Might as well use a traditional car, set it to cruise at 75 mph and take a nap because the end result will be the same.

Re:Simple question

[The Evil Atheist]

Driving while tired is also a decision. If you make that decision, you are also dumb.

Re:Simple question

[Mr D from 63]

The driver was watching a Harry Potter movie when he crashed, according to earlier published reports.

Turns out that was just rumor. The investigation concluded there was no movie being watched.

Re:Simple question

[Mr D from 63]

For the first few days people will be extremely cautious letting the autopilot do anything.

For the first few weeks they'll give it more leeway, but be very attuned to any warnings it gives.

After a few months, if they haven't had any real scares, they'll assume the auto-pilot knows what it's doing and generally ignore warnings.

Some people will be more cautious, but as a software developer this is exactly what I expect to happen with a significant portion of people. Everyone knows the right thing to do, we should backup our data rigorously, always use good unique password, follow the proper procedures, etc. But that's not how people work. If it's not part of a routine, and it's not given an immediate payoff, then people won't do it.

Give people a car that can self-drive in some situations and they will inevitably let it self-drive in every situation they can.

Its human nature. Bad driving habits reinforced over time. A certain percentage of people will grow unsafely confident and increase their risk. Just like texting while driving .

Re:Simple question

[HiThere]

Well, one doesn't need to have anticipated this precise incident to have anticipated it. Preventing it, however, is a different matter. The classic way to say this it "You can't make something idiot-proof, because idiots are so ingenious.".

Re:Simple question

[Strider-]

Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.

The real problem is that Tesla's "Autopilot" is the worst possible solution to the problem. One of the realities of the human brain is that shifting your attention is hard... To put it in computational terms, the context switch is expensive. Even when the car is doing the driving, it theoretically requires the driver to be auditing it, and be paying attention to their surroundings in case things go sideways. Concentrating on something that you're not actively participating in can be quite difficult, as we already know. If the driver is actually, well, driving, they are already hopefully paying attention to the road and their surroundings.

Until we get self-driving cars that can run without even having a steering wheel, it is a bad idea to have the computer half in control. The better solution is to work the other way. The driver does most of the driving, and the computer only takes over in emergency/collision type situations (avoiding that box on the highway, the idiot merging into your blind spot, etc...)

Re:Simple question

[BronsCon]

And when you engage autopilot in the left lane of a 5 lane freeway? What, it's supposed to signal and get over 4 lanes before pulling to the side of the road (where the shoulder may be closed or nonexistent alongside a concrete wall or rock embankment and stopping may be forbidden by law) and stopping?

Re:Simple question

[BronsCon]

It's not autonomous driving, it doesn't navigate, it's glorified lane assist with the ability to maintain speed with the flow of traffic and a bit of accident avoidance logic mixed in. Far from autonomous.

Other than that, we're on the same page.

Re:Simple question

[religionofpeas]

If it's "Far from autonomous", they shouldn't be calling it autopilot.

Seems like an appropriate name, actually. A plane on autopilot also just keeps course and altitude, but doesn't fly around storms, or start an automatic landing procedure when fuel is running low.

Two Things

[Mr D from 63]

What we know from this incident;

1) The driver was responsible for the accident because he didn't maintain control
2) Tesla Autopilot was not good enough on its own to prevent the car from driving into the truck.

Re: Two Things

[guruevi]

There are various assumptions to make before pulling over the side of the road which may or may not be safer than simply having the car continue. You have to make sure there is an unobstructed emergency strip and you're not just careening the vehicle down a cliff. If your car makes the decision to go on a shoulder and something happens (or the shoulder doesn't exist), at that point the liability shifts because you've gone from passive "cruise control with intelligent lane following" to active intervention in a situation.

Re:Two Things

[wonkey_monkey]

* Why didn't the car slow down if you have your hands off the wheel for more then 5+ minutes?

Slow down how much? And if nothing changes, then what? Come to a stop in the middle of the road?

* Why didn't the car pull over the side of the road after 15 minutes of hands free driving?

Because that's a much harder thing to do than simply keeping a car between standardised lane markings and away from relatively slow-moving traffic (not that it fully succeeded in the latter in this case).

Insufficient warnings

[dlleigh]

Of course he was going to ignore a warning that said, "Hold steering wheel."

Instead, the car should have said:

"What the hell are you doing with your hands off the wheel, you idiot???! Are you trying to crash? Do you want to die? Do you want to make your kids orphans?"

The warnings could get increasingly forceful as the car complains that its own safety is being jeopardized.

"I don't want to go to a body shop. They use hammers! Kill yourself if you want, but leave me out of it."

The accident was therefore Tesla's fault.

Re:but... /s

[KGIII]

Nothing. Not a damned thing. Someone this stupid was going to take themselves out of the gene pool, sooner or later. I'm sometime baffled that we've managed to keep ourselves from going extinct.

Ignored 1 warning repeated, not 7 different ones

[gurps_npc]

With current technology, people are trained to ignore meaningless errors.

Proper error handling for anything important require you to take action, especially if you repeat the error.

That is, if they want people to pay attention to a "keep hands on wheels" warning, the speed should drop significantly. Not as if the brake was applied, but instead as if the foot was taken off the gas (even if they tried to floor it.). Oh, and the brake light should flash to let people behind know you are slowing, even though no brake is applied.

partially self-driving cars shouldn't exist.

[s1d3track3D]

Tesla's partially self-driving car

A partial self driving car is like sorta being pregnant, the car is either self driving or not, any grey area = not.

This seems like Tesla getting the public to do QA for them untill they have a fully self driving car, it's clear the public does not know what "partially" means...

Re:partially self-driving cars shouldn't exist.

[chrisautrey]

the car is either self driving or not

Tesla did not do themselves any favors with the name. 'Enhanced cruise control' or something similar as a name would have gone a long way.