Slashdot Mirror


'Operational Limitations' In Tesla Model S Played a 'Major Role' In Autopilot Crash, Says NTSB (reuters.com)

Mr D from 63 writes from a report via Reuters: The chairman of the U.S. National Transportation Safety Board (NTSB) said on Tuesday "operational limitations" in the Tesla Model S played a "major role" in a May 2016 crash that killed a driver using the vehicle's semi-autonomous "Autopilot" system. Reuters reported on Monday that the NTSB is expected to find that the system was a contributing factor because it allows drivers to avoid steering or watching the road for lengthy periods of time. The NTSB is also expected to find that Tesla Inc could have taken additional steps to prevent the system's misuse and will fault the driver for not paying attention. "Today's automation systems augment, rather than replace human drivers. Drivers must always be prepared to take the wheel or apply the brakes," NTSB Chairman Robert Sumalt said. The system could not reliably detect cross traffic and "did little to constrain the use of autopilot to roadways for which it was designed," the board said. Monitoring driver attention by measuring the driver's touching of the steering wheel "was a poor surrogate for monitored driving engagement." At a public hearing Tuesday on the crash involving Brown, NTSB said the truck driver and the Tesla driver "had at least 10 seconds to observe and respond to each other."

42 of 210 comments (clear)

  1. Anybody know what this means? by Marc_Hawke · · Score: 3, Insightful

    """Monitoring driver attention by measuring the driver's touching of the steering wheel "was a poor surrogate for monitored driving engagement." """

    How would you monitor their engagement? Eye tracking? Manual corrections to the car's path/speed?

    What happens when people ignore the "please grab the wheel?" Does the car pull over and park? Is that what it should do?

    --
    --Welcome to the Realm of the Hawke--
    1. Re:Anybody know what this means? by PolygamousRanchKid+ · · Score: 3, Interesting

      How would you monitor their engagement? Eye tracking? Manual corrections to the car's path/speed?

      Well, with any automobile . . . the biggest mechanical danger is . . . "The Loose Nut Behind the Wheel" . . .

      "Autopilots" are probably something that most "normal" drivers should not be using anyway. Hey, driving is a privilege, and not a right.

      Hey, take someone who can't read the traffic signs and is abysmally clueless as to traffic laws . . . no wonder that stuff like this will happen more often. We'll just have to wait and see how the American lawyers will deal with this. They could kill self-driving cars . . . but then again . . . it would be more lucrative for them to milk the industry.

      More and the weather at eleven . . .

      --
      Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
    2. Re:Anybody know what this means? by Anonymous Coward · · Score: 2, Insightful

      Auto Pilot for cars is a stupid idea altogether. For airplanes with very little traffic interaction is OK, but for automobiles it reeks.
      There a millions of decisions made during driving in urban traffic, judgment calls are constantly being made. For example, if meeting traffic would suddenly breach center and you would have to decide what object to crash, in less than a second, the auto pilot would fail. However many drivers are so uneducated and mis-educated, that a shitty autopilot would be better.
      The US population has _extremely_ weak drivers education, I do not think 0.0001% would pass a west European driver test.
      Even if the software and and hardware involved would be sufficient and bug-free, the legal liability would shift to the auto maker.
      An all auto pilot system could work, if the expectations are lowered. Use a 5 mph speed limit.

    3. Re:Anybody know what this means? by JaredOfEuropa · · Score: 2

      It means you can behave like an idiot and ignore all safety warnings and common sense, have an accident, and you (or your next of kin) still get to sue the car manufacturer.

      If anything is going to stop self driving cars, it's moronic "victims" looking for a payday.

      --
      If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
    4. Re:Anybody know what this means? by Ichijo · · Score: 3, Insightful

      How would you monitor their engagement?

      There may not yet be an effective way to monitor driver engagement. This doesn't invalidate the NTSB's conclusion.

      --
      Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
    5. Re:Anybody know what this means? by Mr+D+from+63 · · Score: 3, Insightful

      No, it means that human factors play a role in safety with these systems. The study has nothing to do with legal culpability.

    6. Re:Anybody know what this means? by quantaman · · Score: 5, Insightful

      """Monitoring driver attention by measuring the driver's touching of the steering wheel "was a poor surrogate for monitored driving engagement." """

      How would you monitor their engagement? Eye tracking? Manual corrections to the car's path/speed?

      What happens when people ignore the "please grab the wheel?" Does the car pull over and park? Is that what it should do?

      It means that it's really hard to make a partially self-driving car that is safe.

      People have two mode, driving and not-driving. If the car isn't safe while you're not-driving then the car isn't safe.

      --
      I stole this Sig
    7. Re:Anybody know what this means? by BasilBrush · · Score: 3, Interesting

      Yes, eye tracking is the obvious way. And the Tesla Model 3 has a camera in the rear view mirror area that faces back towards the inside of the car. AFAIK it's not used yet, but it's obvious use case is monitoring driver attention. They could deliver that in a future software update.

    8. Re:Anybody know what this means? by Anonymous Coward · · Score: 5, Insightful

      ...unless the chainsaw company names it the Jugglesaw 9000 and markets it with commercials starring chainsaw jugglers.

    9. Re: Anybody know what this means? by Anonymous Coward · · Score: 2, Insightful

      People still don't know that autopilot will happily fly into the side of a crossing airplane...
      The professional environment that autopilot was designed for still has 2 pilots!

      Also, cars are driven, not piloted.

    10. Re:Anybody know what this means? by MightyYar · · Score: 4, Informative

      Cadillac uses eye tracking to monitor the driver's attentiveness.

      --
      W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
    11. Re:Anybody know what this means? by quantaman · · Score: 2

      Or you could not be a tard and assume you should still pay "some" attention to the world flying by at 70 MPH outside that could, you know, kill you. But nah, keep facebooking and slowly making the world a better place by removing yourself from the gene pool.

      You could assume that, you could also assume people will floss every day, eat healthy food, and get 8 hours of sleep every night. But then you'd be a poor judge of human nature.

      Sure you'll watch the road like a hawk for the first few weeks, but then you'll realize it's been going without incident all that time and you just want to reply to a text message from your girlfriend, or respond to an important email from work, or maybe just zone out and start at the scenery.

      Give people a car that drives without their constant interaction and a lot of those people, probably most, will become very complacent very quickly.

      --
      I stole this Sig
    12. Re:Anybody know what this means? by aberglas · · Score: 2

      Wont work for me. I generally drive with my eyes closed.

    13. Re: Anybody know what this means? by gravewax · · Score: 3, Informative

      cruise control doesn't remove the requirement for you to be paying 100% attention on the road at all times, it only removes the requirement to constantly glance down to verify you are going the correct speed.

    14. Re:Anybody know what this means? by Pentium100 · · Score: 2

      The problem is that the current "autopilot" is pretty much like being a driving instructor.

      The student (or in this case, autopilot) drives, but you have to be always looking out for his mistakes and prepared to take control in an instant.

      Also, in this case the "student" is high or something because, while he usually does not do the mistakes other students do (forgetting a turn signal, missing a red traffic signal) he ma, once in a while, not notice a huge 18-wheeler just in front of him.

    15. Re:Anybody know what this means? by Pentium100 · · Score: 2

      They find it a huge load off to not have to be constantly monitoring every bit about you drive

      But now you have to constantly monitor every bit about how the AI drives. Especially since you may have to take control in less than a second, because you will always recognize the problem later than normal, since you expect the AI to take care of it and only notice when the AI doesn't do that in a normal fashion ("Hmm, there is a huge hole in the road, the AI is going to stop... Oh shit!! The AI did not start braking at the normal distance now I have to brake or swerve into a less deep ditch.").

      Part of the irony of EAP is that as they improve it, it may actually lead to more accidents.

      I completely agree. If anyone doubts it, do a simple experiment: there are websites where you can measure your reaction time. Usually you have to wait for a signal and then click a button as quickly as possible. Try to do that with the wait time before the signal set to about 10 seconds, and then set it to an hour. I am sure that after staring at the screen for an hour you will react slower than waiting for 10 seconds.
      OTOH, manual driving is more like waiting for a second or even playing a FPS game - you are engaged all the time, so the reaction time is the fastest.

  2. This type of accident will increase by hawguy · · Score: 4, Insightful

    As autonomous cars get better and better, we'll see more and more accidents attributed to driver inattention -- the better the car is at driving, the less the human is going to pay attention to the car or the road, and by the time the car tells the driver "Oh hey, I don't know how to handle this situation, you take over!", the driver won't have enough situational awareness to get out of the situation.

    Though the flip side is that as the cars get better at driving, the overall accident rate will decrease.

    The same problem already exists with airplane pilots , and it can be even worse where the autopilot compensates for some building condition (like icing), and by the time it gives up control to the pilot, the plane may already be in a bad state and the pilot has little time to figure out why.

    1. Re:This type of accident will increase by Xylantiel · · Score: 2

      You have no evidence to back up your claim. It is entirely possible that the overall accident rate will increase because people will start to rely on autonomous driving systems in situations where they are not safe or use them in ways that are unsafe. Many people already drive in surprisingly unsafe ways.

  3. Pay-to-debug by Roger+W+Moore · · Score: 3, Interesting

    As autonomous cars get better and better, we'll see more and more accidents attributed to driver inattention

    Not if we can have a system which is better at driving than a human. In fact, other than the "cool factor" I'm not sure I see the point of a semi-autonomous system which requires me to watch the road all the time since it is no different from driving myself and potentially a lot more annoying. Frankly, it sounds more like paying to debug a final system which will drive itself.

    1. Re:Pay-to-debug by slack_justyb · · Score: 3, Interesting

      Friend's car on a trip to a conference I got to actually try one of these out, these systems are a godsend in stop and go traffic. Did not test the thing on the open road.

    2. Re:Pay-to-debug by Moof123 · · Score: 4, Insightful

      Once we are on the other side of the autonomous creepy valley, sure things will be great and perfect and all that jazz. So far that is a mythical future you can't buy yet.

      In the meantime the current crop of systems, and the ones planned for the next several years, all augment the driver rather than replace them. As such driving will be inherently BORING as hell while you are a quasi-passenger in your own car. Bored humans check phones, read email, nod off, watch movies, and other dumb stuff when they are still legally required to be in command and control of the vehicle. Google even admitted their employees were sleeping and working on laptops on the way home in their very beta test fleet (not that they fired them as they should have).

      So the upshot is that while these autonomous systems will create this whole new class of collision by zombified drivers, even while potentially lowering the overall rate with their mostly good collision avoidance systems. Drivers will be out of practice and situationally unaware when HAL throws up its hands and gives back control, or when it fails to respond to trucks in the roadway, construction, black ice, pot holes, etc, etc.

      So until your car is fully licensed to drive itself, these semi-autonomous systems need to be designed to keep the licensed driver on the ball and paying attention. Of course that takes away 90% of the sex appeal, so I expect the legal/ethical envelope to keep being pushed and more of these new types of crashes to keep occurring.

    3. Re:Pay-to-debug by BasilBrush · · Score: 4, Insightful

      They'll learn less from them by firing them than by devising methods to stop them doing these things. After all, they can't fire the eventual customers that will also do these things.

    4. Re:Pay-to-debug by Dutch+Gun · · Score: 2

      Driving long distances has always been pretty damned boring, and today's drivers are already distracting themselves to death using their smartphones with no assistance from semi-autonomous cars. And I'd posit that future drivers can't really do any worse than current drivers do with black ice - which is to say, typically losing complete control of the vehicle. So, I think that ship has long sailed, and I'm not really sure there's any turning it around.

      It's going to be a rough few years of transition, most likely, but the eventual move to fully autonomous systems and their subsequent refinement over the next decade or two are inevitable, after which this will be much less of issue.

      --
      Irony: Agile development has too much intertia to be abandoned now.
  4. This is dumb by ArchieBunker · · Score: 2

    Didn't Tesla say this guy had to acknowledge like seven warnings before the crash?

    --
    Only the State obtains its revenue by coercion. - Murray Rothbard
    1. Re:This is dumb by Pinky's+Brain · · Score: 3, Funny

      They allowed the system to be active with seven warnings and over the speed limit.

    2. Re:This is dumb by Anonymous Coward · · Score: 2, Funny

      They allowed the system to be active with seven warnings and over the speed limit.

      So it's somehow Tesla's fault that the driver was an idiot? The software is nowhere near a state where it can refuse to defer to a meatbag. The driver has to be assumed to be in control. The software can notify the driver of sub-optimal conditions; it shouldn't be blamed for the idiot behind the wheel ignoring repeated warnings.

    3. Re:This is dumb by Moof123 · · Score: 2

      You read all the TOS before clicking "Accept", right?

    4. Re:This is dumb by Richard_at_work · · Score: 3, Informative

      The issue is the duration in which the Tesla system gives warnings - the industry standard recommended timespan between inattentive-driver prompts is 15 seconds, while in this case, there were no inattentive-driver prompts from Autopilot for the two minutes leading up to the accident.

      This is the issue when you call something "Autopilot" and give it to a consumer base that is used to being spoonfed fictitious understandings of such systems from superficial TV shows - they are led to believe it does something that it most certainly does not.

      Yes, Tesla put all sorts of warnings in their manuals about this, but theres absolutely no requirement to read those manuals before jumping into the car, hitting the highway and engaging the system. Thats where the disconnect from reality and theory occurs - in theory, everyone reads the manual and understands the intimate details of the vehicle before setting off, while in practice people jump into new cars all the time and try things out.

      Who here has been the person sat in their rental car for 30 minutes reading the manual before driving off for the first time? I bet the number of people who response affirmatively to that question is .... low.

      Thats the issue Tesla need to solve.

  5. Well duh. by Gravis+Zero · · Score: 4, Interesting

    The NTSB is also expected to find that Tesla Inc could have taken additional steps to prevent the system's misuse

    Of course they could have taken additional steps to prevent the system's misuse before the crash because that's exactly what they did right after the crash.

    --
    Anons need not reply. Questions end with a question mark.
    1. Re:Well duh. by srw · · Score: 4, Interesting

      But they want it misused.

      This is the absolute best R&D that money can't buy. Idiots let the autopilot drive for them, even when they're not supposed to, and Tesla gets amazingly good real-life data and very little of the blame when things go wrong. (none, if they play it just right)

    2. Re:Well duh. by thegarbz · · Score: 2

      Once an always take an "additional" step. The question was what was in place before the crash, and it wasn't nothing. As part of the investigation Tesla showed that the driver actively ignored the warnings that were already in place.

      If you try to fix stupid with technology it will be a race to the bottom.

    3. Re:Well duh. by AmiMoJo · · Score: 2

      The report is pretty damning. Over a period of 37 minutes he had his hands on the wheel for 25 seconds, and the car didn't stop or force him to pay attention.

      If all you need to defeat their attention test is to touch the wheel for 25 seconds out of every 37 minutes, that's not going to stop anyone playing with their phone or taking a nap.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
  6. Queue lawsuit in 3... 2....1 by bobbied · · Score: 2, Insightful

    This is BAD for Tesla. The NTSB basically found fault in the "auto pilot" system's user interface AND it's technical capability. I am NOT surprised by this.

    Automatic driving of cars and trunks needs to be thoroughly thought through. Not just the technology required to keep the car on the road, sensing what's going on around it and dealing appropriately with this dynamic environment, but also the complex human factors considerations. Tesla may have the first part working fairly well within the given limits of their sensors, but the second part of this problem hasn't been designed very well.

    Human Factors engineering has only recently been a consideration for *real* auto pilots (those in aircraft) and flight automation systems. And it has become clear that all the automation in aircraft has given us great efficiency and smooth operation a the cost of inexperienced pilots with poor flying skills who don't recognize when something is gravely wrong until it is too late. They trust the automation, because it just works, at least until it doesn't, and something really bad happens that was easily preventable. The folks over at the NTSB are very familiar with this issue because there have been a number of notable commercial aircraft crashes where this was a contributing factor, where the automation failed to do what the pilot expected and a crash happened in a perfectly flyable aircraft.

    Tesla has a serious level of risk with this feature. It may be wiz bang cool and Musk may love calling it an "autopilot" but the legal liability is huge unless they can keep people from crashing while it's on. The NTSB's statements here are NOT going to bode well for Tesla's legal liability and all the EULA's in the world won't stop the lawsuits when crashes happen.

    --
    "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
  7. You can't have it both ways by markdavis · · Score: 5, Interesting

    You can't have it both ways. Really, you can't. Either the car is driving or the person is driving. Expecting that a person will let the car drive AND ALSO stay 100% ready to take over is just not reality. If you are not the one in control, then your mind will not focus on it. Driving is boring enough as it is, expecting someone to babysit a semi-autonomous car is way beyond what we can expect people to do.

    Just as an example, 9 years ago when I got my fully loaded Infiniti G37S with technology package, it was one of the first vehicles to have laser-controlled intelligent cruise control. It can match speeds of the cars in front and actively adjust, even brake if necessary. And just that ONE feature of driving assistance sounded like it would be very useful. OMG no. I tried many times to use it and found that just fully automated speed control was enough to disengage me from being an active driver. I could not adapt to it and ultimately decided I would never use it again. It was simply unsafe! Regular cruise control- no problem, I have to pay attention and I bump the speed up and down manually with the thumb control and take other action when necessary. But as soon as that was taken away from me, it became nearly impossible to stay attentive, even though I still had to steer!

    Now, maybe different brains work differently and some people can handle semi-automation, but I know I can't. So don't even TRY to give me a car that can sorta drive itself and expect ME to be the ultimate failsafe... that just isn't going to happen. And I expect I am far, FAR from alone in this.

    1. Re:You can't have it both ways by jezwel · · Score: 2

      Now, maybe different brains work differently and some people can handle semi-automation, but I know I can't. So don't even TRY to give me a car that can sorta drive itself and expect ME to be the ultimate failsafe... that just isn't going to happen. And I expect I am far, FAR from alone in this.

      My partners' new car has this. She uses it all the time, and I do as well when I'm driving.
      Once I figured out how to adjust the distance between the car and the one ahead, and added the max cruise speed, it is super simple to use.

      Neither of us have had difficulty in being overly bored whilst using it.

      There are obviously groups of people where this does work, and quite well.

    2. Re:You can't have it both ways by Cyberax · · Score: 2

      I drive Tesla and I love the self-driving. It definitely makes me a safer driver - I have enough attention to actually monitor surroundings while the car takes care of staying in the lane. It probably saved me from a couple of deer strikes already. I have no problem staying alert and I can actually stay alert for much longer with self-driving during road trips.

  8. Re:Since When...? by BasilBrush · · Score: 2

    I wish I had that for podcasts or TV. That way if I fall asleep when listening/watching I can resume later from the point I fell asleep.

  9. attention is strange thing by 140Mandak262Jamuna · · Score: 2
    When I code, when I am typing function names or variable names, I am super careful about spelling and watch every letter. Somehow, as soon as I start a literal string or a comment block, the brain instantly switches off attention and I make typos.

    I could easily see how taking away speed control is enough to let your mind wander.

    --
    sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
  10. Re:Autopilots and planes by K.+S.+Kyosuke · · Score: 2

    Not quite. Heavies all have TCAS, which is transponder based and will make a loud sound if anything gets to near.

    Also known as "Sergey, what is that weird sound?"

    --
    Ezekiel 23:20
  11. Re:Autopilots and planes by michelcolman · · Score: 5, Interesting

    It's not quite so easy to spot a flock of birds taking off as you reach the critical speed from which you can no longer abort a take-off. The pilot flying the airplane (captain or F/O) does have his or her eyes outside during the take-off roll until lift-off, as this is a completely manual and visual phase. The other pilot is looking at engine parameters, airspeed and a few other instruments and just taking a few occasional glaces outside.

    But once you're above the abort speed (v1), there's little you can do about birds. You can't just yank the airplane left or right to avoid them when you're doing 150 knots. We are actually advised not to change our flight path because the birds act in unpredictable ways and violent maneuvers may actually increase the chance of a bird strike with birds dashing off precisely in the same direction you decided to turn towards. Also, it's easier to react to an engine failure if the plane's path is nice and stable rather than while you're violently banking or pitching.

  12. Re:Autopilots and planes by michelcolman · · Score: 3

    What makes you think that makes it any easier? They were probably doing well over 200 knots by then so it's even harder to make a tight turn. For information, a 2G turn at 200 knots (passengers screaming in the back) has a radius of 630 meters.

    Trust me, I fly the A320. You don't just yank such a plane around to avoid some birds. When you see them, it's usually already too late. Too much inertia, and you don't know which direction the birds are going to take anyway. During approach, at low speeds, you might consider going around (pulling the nose up and adding full power) but it's a risky decision because it might be preferable to continue the approach with a limp airplane rather than losing engines during a go-around if you didn't manage to miss the birds.

    With most bird encounters, we end up missing them. Sometimes we hit one or more, but it has no effect on the airplane. Very rarely, an engine gets seriously damaged. Loss of all engines in an airliner happens once every few decades in the whole world. Now imagine we would "yank the plane around" every time we saw some birds. You would not want to fly on those planes.

  13. Re:Autopilots and planes by michelcolman · · Score: 2

    You are obviously not a pilot.

    After take-off in a busy area like New York, the flying pilot's eyes are supposed to be on the instruments most of the time to fly the trajectory as accurately as possible. We do try to look outside as much as we can to spot traffic, but mostly our eyes are inside. That's not because we're lazy, but because there's just too much stuff to monitor. Meanwhile the other pilot may be busy with the flaps, radio frequency changes, monitoring the flight path (it takes two to make sure), etcetera. It's not a Cessna 150 where you're navigating by sight and feeling. It's just not possible to be looking outside continuously in a large airliner.

    Also, with the nose high up, visibility is not all that great. Anything flying up from below is pretty much invisible. And birds are not very big when seen from a kilometer away. I've seen some pretty big birds pass by very closely only a few seconds after I spotted them, while flying a visual approach with both pilots looking outside. They really are hard to spot, especially if they're coming relatively straight towards you because they're not moving against the background.

    And even if you did spot the flock, where are they going to go? Birds tend to move in unpredictable ways.

    O, and you really think we can safely yank the airplane in any direction in a busy area like New York, with lots of other planes taking off from different runways?

    Seriously, you have no idea how complex airline flying is. Oh, why am I even bothering, you just go on believing that we're useless meat bags reading magazines while the autopilot does all the flying.