Driver Killed In a Tesla Crash Using Autopilot Ignored At Least 7 Safety Warnings (usatoday.com)

An anonymous reader quotes a report from USA Today: U.S. investigators said a driver who was killed while using Tesla's partially self-driving car ignored repeated warnings to put his hands on the wheel. In a 538-page report providing new details of the May 2016 crash that killed Ohio resident Joshua Brown in a highway crash in Florida, the National Transportation Safety Board described the scene of the grisly incident and the minutes leading up to it. The agency, which opened an investigation to explore the possibility that Tesla's Autopilot system was faulty, said it had drawn "no conclusions about how or why the crash occurred." The NTSB report appears to deliver no conflicting information. The agency said the driver was traveling at 74 miles per hour, above the 65 mph limit on the road, when he collided with the truck. The driver used the vehicle's self-driving system for 37.5 minutes of the 41 minutes of his trip, according to NTSB. During the time the self-driving system was activated, he had his hands on the wheel for a total of only about half a minute, investigators concluded. NTSB said the driver received seven visual warnings on the instrument panel, which blared "Hold Steering Wheel," followed by six audible warnings.

Comments

Simple question

[Anonymous Coward]

Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.

Re:Simple question

[Anonymous Coward]

Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.

Because they trusted that the owner of an $80,000 car had at least some minimal intelligence and even if the driver had blind trust in the car, that when the car says "put your hands on the wheel and pay attention", that the driver would listen.

So rich people don't have medical issues while driving that might prevent them from responding to verbal cues? A car capable of driving itself (mostly) isn't capable of pulling over safely if it can not confirm that the driver is doing what they are supposed to do?

If zillionaire Elon Musk can't anticipate such an obvious contingency, expecting $80K to be a guarantee of intelligence is seriously foolish.

Re:Simple question

[wonkey_monkey]

If that's a requirement, why didn't the car just pull over and shut off?

Because the car isn't smart enough to do that. It can keep you between the lines on the road; it can't take you out of the lanes and park you up. That's actually a harder thing to do.

Re:Simple question

[110010001000]

No, the point is should the car have stopped, or kept driving after seven warnings and 37 minutes? It is supposed to be a smart car.

Re:Simple question

[quantaman]

Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.

Because they trusted that the owner of an $80,000 car had at least some minimal intelligence and even if the driver had blind trust in the car, that when the car says "put your hands on the wheel and pay attention", that the driver would listen.

Yet this driver has demonstrated that people are about as dumb as you think they can be, so now they've implemented a 3 strikes policy that disabled autopilot after 3 reminders.

For the first few days people will be extremely cautious letting the autopilot do anything.

For the first few weeks they'll give it more leeway, but be very attuned to any warnings it gives.

After a few months, if they haven't had any real scares, they'll assume the auto-pilot knows what it's doing and generally ignore warnings.

Some people will be more cautious, but as a software developer this is exactly what I expect to happen with a significant portion of people. Everyone knows the right thing to do, we should backup our data rigorously, always use good unique password, follow the proper procedures, etc. But that's not how people work. If it's not part of a routine, and it's not given an immediate payoff, then people won't do it.

Give people a car that can self-drive in some situations and they will inevitably let it self-drive in every situation they can.

Re:Simple question

[The Evil Atheist]

Driving while tired is also a decision. If you make that decision, you are also dumb.

Re:Simple question

[HiThere]

Well, one doesn't need to have anticipated this precise incident to have anticipated it. Preventing it, however, is a different matter. The classic way to say this it "You can't make something idiot-proof, because idiots are so ingenious.".

Re:Simple question

[Strider-]

Why would the car continue to operate for 37.5 minutes of the trip if the driver didn't have his hands on the steering wheel? If that's a requirement, why didn't the car just pull over and shut off? It seems like Tesla failed to implement some common sense safety protocols here.

The real problem is that Tesla's "Autopilot" is the worst possible solution to the problem. One of the realities of the human brain is that shifting your attention is hard... To put it in computational terms, the context switch is expensive. Even when the car is doing the driving, it theoretically requires the driver to be auditing it, and be paying attention to their surroundings in case things go sideways. Concentrating on something that you're not actively participating in can be quite difficult, as we already know. If the driver is actually, well, driving, they are already hopefully paying attention to the road and their surroundings.

Until we get self-driving cars that can run without even having a steering wheel, it is a bad idea to have the computer half in control. The better solution is to work the other way. The driver does most of the driving, and the computer only takes over in emergency/collision type situations (avoiding that box on the highway, the idiot merging into your blind spot, etc...)

Re:Simple question

[BronsCon]

And when you engage autopilot in the left lane of a 5 lane freeway? What, it's supposed to signal and get over 4 lanes before pulling to the side of the road (where the shoulder may be closed or nonexistent alongside a concrete wall or rock embankment and stopping may be forbidden by law) and stopping?

Re:Simple question

[BronsCon]

It's not autonomous driving, it doesn't navigate, it's glorified lane assist with the ability to maintain speed with the flow of traffic and a bit of accident avoidance logic mixed in. Far from autonomous.

Other than that, we're on the same page.

Re:Simple question

[speedplane]

It's not autonomous driving, it doesn't navigate, it's glorified lane assist with the ability to maintain speed with the flow of traffic and a bit of accident avoidance logic mixed in. Far from autonomous.

If it's "Far from autonomous", they shouldn't be calling it autopilot.

Re:Simple question

[religionofpeas]

If it's "Far from autonomous", they shouldn't be calling it autopilot.

Seems like an appropriate name, actually. A plane on autopilot also just keeps course and altitude, but doesn't fly around storms, or start an automatic landing procedure when fuel is running low.

Two Things

[Mr D from 63]

What we know from this incident;

1) The driver was responsible for the accident because he didn't maintain control
2) Tesla Autopilot was not good enough on its own to prevent the car from driving into the truck.

Re:Two Things

[UnknownSoldier]

> 2) Tesla Autopilot was not good enough on its own to prevent the car from driving into the truck.

^ THIS.

That is the more troubling question that needs to be asked and answered. Why weren't there more fail safes such as ... ?

* Why didn't the car slow down if you have your hands off the wheel for more then 5+ minutes?
* Why didn't the car's sensor detect the impending crash?
* Why didn't the car pull over the side of the road after 15 minutes of hands free driving?

Re:Two Things

[wonkey_monkey]

* Why didn't the car slow down if you have your hands off the wheel for more then 5+ minutes?

Slow down how much? And if nothing changes, then what? Come to a stop in the middle of the road?

* Why didn't the car pull over the side of the road after 15 minutes of hands free driving?

Because that's a much harder thing to do than simply keeping a car between standardised lane markings and away from relatively slow-moving traffic (not that it fully succeeded in the latter in this case).

Re:Two Things

[Trogre]

Slow down how much? And if nothing changes, then what? Come to a stop in the middle of the road?

Of course, yes.

If you ever find yourself in a situation where you are no longer able to safely drive your vehicle for any reason your first course of action is to stop it from moving, and of course activate your hazard lights.

If the AI was not able to pull over then the correct action would be for it to just stop.

Re:but... /s

[KGIII]

Nothing. Not a damned thing. Someone this stupid was going to take themselves out of the gene pool, sooner or later. I'm sometime baffled that we've managed to keep ourselves from going extinct.

Ignored 1 warning repeated, not 7 different ones

[gurps_npc]

With current technology, people are trained to ignore meaningless errors.

Proper error handling for anything important require you to take action, especially if you repeat the error.

That is, if they want people to pay attention to a "keep hands on wheels" warning, the speed should drop significantly. Not as if the brake was applied, but instead as if the foot was taken off the gas (even if they tried to floor it.). Oh, and the brake light should flash to let people behind know you are slowing, even though no brake is applied.

partially self-driving cars shouldn't exist.

[s1d3track3D]

Tesla's partially self-driving car

A partial self driving car is like sorta being pregnant, the car is either self driving or not, any grey area = not.

This seems like Tesla getting the public to do QA for them untill they have a fully self driving car, it's clear the public does not know what "partially" means...

Re:partially self-driving cars shouldn't exist.

[chrisautrey]

the car is either self driving or not

Tesla did not do themselves any favors with the name. 'Enhanced cruise control' or something similar as a name would have gone a long way.