Slashdot Mirror
Sorry Elon Musk, There's No Clear Evidence Autopilot Saves Lives (arstechnica.com)
Timothy B. Lee writes for Ars Technica: A few days after the Mountain View crash, Tesla published a blog post acknowledging that Autopilot was active at the time of the crash. But the company argued that the technology improved safety overall, pointing to a 2017 report by the National Highway Traffic Safety Administration (NHTSA). "Over a year ago, our first iteration of Autopilot was found by the U.S. government to reduce crash rates by as much as 40 percent," the company wrote. It was the second time Tesla had cited that study in the context of the Mountain View crash -- another blog post three days earlier had made the same point. Unfortunately, there are some big problems with that finding. Indeed, the flaws are so significant that NHTSA put out a remarkable statement this week distancing itself from its own finding.
"NHTSA's safety defect investigation of MY2014-2016 Tesla Model S and Model X did not assess the effectiveness of this technology," the agency said in an email to Ars on Wednesday afternoon. "NHTSA performed this cursory comparison of the rates before and after installation of the feature to determine whether models equipped with Autosteer were associated with higher crash rates, which could have indicated that further investigation was necessary." Tesla has also claimed that its cars have a crash rate 3.7 times lower than average, but as we'll see there's little reason to think that has anything to do with Autopilot. This week, we've talked to several automotive safety experts, and none has been able to point us to clear evidence that Autopilot's semi-autonomous features improve safety. And that's why news sites like ours haven't written stories "about how autonomous cars are really safe." Maybe that will prove true in the future, but right now the data just isn't there. Musk has promised to publish regular safety reports in the future -- perhaps those will give us the data needed to establish whether Autopilot actually improves safety.
UPDATE (2/16/19): The study's underlying data reveals serious flaws in the methodology that undermine its credibility, according to new analysis from a research and consulting firm.
"NHTSA's safety defect investigation of MY2014-2016 Tesla Model S and Model X did not assess the effectiveness of this technology," the agency said in an email to Ars on Wednesday afternoon. "NHTSA performed this cursory comparison of the rates before and after installation of the feature to determine whether models equipped with Autosteer were associated with higher crash rates, which could have indicated that further investigation was necessary." Tesla has also claimed that its cars have a crash rate 3.7 times lower than average, but as we'll see there's little reason to think that has anything to do with Autopilot. This week, we've talked to several automotive safety experts, and none has been able to point us to clear evidence that Autopilot's semi-autonomous features improve safety. And that's why news sites like ours haven't written stories "about how autonomous cars are really safe." Maybe that will prove true in the future, but right now the data just isn't there. Musk has promised to publish regular safety reports in the future -- perhaps those will give us the data needed to establish whether Autopilot actually improves safety.
UPDATE (2/16/19): The study's underlying data reveals serious flaws in the methodology that undermine its credibility, according to new analysis from a research and consulting firm.
Unless the autopilot feature is actively instigating accidents, it's impossible for it not to be safer. Anything above and beyond relying solely on driver's response is an improvement, even if only minimally.
The system is flawed because it has to rely on a lazy, distracted driver who really doesnâ(TM)t want to drive his own car to begin with.
What the hell are they trying to convey here?
If value A is "2 times less" than value B, does that mean A is 1/2 B? So "3.7 times lower" means a factor of 1/3.7 ?
While I'm on the subject, there is no unit for "coldness" or "slowness", so please stop saying nonsense like "twice as cold" or "ten times slower" and stick to "half the temperature" and "one-tenth the speed". FFS