Uber's Self-Driving Car Saw Pedestrian 6 Seconds Before Fatal Strike, Says Report (tucson.com)

An anonymous reader quotes a report from Arizona Daily Star: The autonomous Uber SUV that struck and killed an Arizona pedestrian in March spotted the woman about six seconds before hitting her, but did not stop because the system used to automatically apply brakes in potentially dangerous situations had been disabled, according to federal investigators. In a preliminary report on the crash, the National Transportation Safety Board said Thursday that emergency braking is not enabled while Uber's cars are under computer control, "to reduce the potential for erratic vehicle behavior." Instead, Uber relies on a human backup driver to intervene. The system, however, is not designed to alert the driver. The report comes a day after Uber announced it will be ending it's self-driving vehicle testing in Arizona. The full NTSB report is available here.

Comments

Wait, what now?

[Narcocide]

This can't be right. They are saying that Uber's self-driving car rig is neither designed to stop for nor alert the driver about pedestrians obstructing the path of the vehicle. It's just designed to... log them?!

What part about this is considered "self-driving" then, exactly?

Re:Wait, what now?

[Anonymous Coward]

Except the driver they hired to do the stopping thing was texting instead of watching the road, from what the video looked like.

If you RTFA (I know, I know), you'll see that the backup driver said she was monitoring the self-driving interface, not fiddling with a phone.

Re:Wait, what now?

[MDMurphy]

A very poorly written article which resulted in a poorly written summary.

Buried in the article: "Uber also disabled the Volvo's factory-equipped automatic emergency braking system when the vehicle is in autonomous mode, the report said."

The Volvo XC90 comes with a feature they call "City Safety". https://www.media.volvocars.co...

This is an auto-braking system with sensors. Uber's autonomous system has its own braking and sensors. It's understandable from a system perspective that they don't have two separate, independent, systems deciding when to apply the brakes operational at the same time.

The poorly written article makes it sound like Uber's system either didn't have a feature for braking for obstacles or that it was disabled. This is not accurate. It does appear that Uber's system failed to either detect the pedestrian or to brake when detected.

It's probably also true that when testing they don't use the factory cruise control to maintain speed on the highway. There are likely other standard functions not used when the autonomous equipment is under test.

Re:Wait, what now?

[Kjella]

The poorly written article makes it sound like Uber's system either didn't have a feature for braking for obstacles or that it was disabled. This is not accurate.

No, you're the one spreading misinformation. To quote the NTSB report directly:

According to Uber emergency braking maneuvers are not enabled while the vehicle is under computer control to reduce the potential for erratic vehicle behavior. The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator.

Basically they had a system that calculated it would crash, but did nothing and warned no one. In other words, the same as having no system at all.

Re:Wait, what now?

[Rei]

What did you expect from a company like Uber, exactly? Their entire business model was built around the concept of "laws don't apply to us, because, internet." This is the same company that beyond running an unregistered taxi service (cheaply disguised as "ride sharing", as if people just happened to be going in the same direction), was tracking critical journalists with "God Mode" and hiring private investigators against them to find things to blackmail them with, ran an active campaign to track law enforcement officers in order to evade them, ran a campaign against competitors like Lyft involving the massive use of fake ride requests, and literally dozens and dozens more types of general scumbaggery.

Anyone shocked that this same company shut off all pedestrian safety controls in order to get their "self driving cars" on the market sooner? Bueller?

Re:Wait, what now?

[vtcodger]

"They are saying that Uber's self-driving car rig is neither designed to stop for nor alert the driver about pedestrians obstructing the path of the vehicle."

What they seem to have done is taken a vehicle with substantial safety equipment, left the sensors operating, but turned off the accident avoidance features. Then they added a distracting task (monitor the system) for the driver. Then, SURPRISE!!!, something went lethally wrong.

We'll have to wait until folks with time and full information perform an analysis. But it sure looks at first glance like this was/is a questionably well designed test program.

Re:Wait, what now?

[slew]

Apparently the way they had it was that the computer would drive and the driver would stop it from driving, if needed. That doesn't seem like an obviously ridiculous arrangement, even if having the computer ping the driver would have been better.

I suspect that the alerting the driver wasn't gonna be any better. This is all speculation, but if Uber turned off the AEB (automatic emergency braking) system in self driving mode because it would have actuated the brakes too often making the driving erratic, simply notifing the driver about the same potential collisions would eventually result in alert fatigue in the driver.

Then the driver would start ignoring the alerts for basically the same reason that they turned off the AEB system originally.

If you've ever taught someone to drive who was overly cautious and braked all the time you know how that goes. The flaw is that simply seeing the pedestrian isn't enough to make the braking decision and the driver has to learn to anticipate the actions in the environment to drive successfully.

The driving instruction not only has to anticipate the environment, but also the driver to know when to intervene (grab steering wheel or apply the brake). Most self driving systems don't give these driving observers (not instructors) enough information and training to anticipate what the system is going to do to make them effective at intervening.

This is the problem of allowing people to walk before they can crawl... Sometimes you can't bypass stages of learning. If the system's AEB system activates too much, you kind of have to let it do that until it can learn not to do that.

Re:Wait, what now?

[uncqual]

http://dilbert.com/strip/2018-01-30

Time for Regulatory Control

[djbckr]

It's time for regulatory control - like how the FAA regulates the skies, the autonomous car industry needs to have sensible regulations. Right now it just seems like a bunch of cowboys in the wild west are trying to one-up each other.
Get this: The car "saw" the person 6 seconds before striking the person. The emergency system was disabled. The emergency system was not set up to alert the driver. So many things wrong here that would have been avoided had there been sensible regulations written by sensible engineers.

No, you are wrong.

[thesupraman]

And I suspect intentionally so.

It has been clearly stated, BY UBER that they both disabled the Volvo emergency braking AND their own emergency stop/avoidance code.
To imply it was just the Volvo system is trying to avoid the fact that Uber had intentionally removed their own code that would make emergency stops.

They had intentionally made a system that had NO automated method of making an emergency stop, and no method of warning the secondary driver that one was needed (which would be stupid anyway,. they are supposed to be testing a system for future use without such a driver).

They should be hammered for this - it would be the equivalent of removing the emergency brakes from a buildings lifts, or the airbags from a car, without telling anyone.

None

[rsilvergun]

they originally had 2 people driving the car, one to watch the road and one to watch the numbers and make notes about the car's performance. As a cost cutting measure they dropped it down to 1 person doing both jobs. The woman behind the wheel was busy making notes on the screen when she should have been watching the road.

Uber wants data and they don't care how they get it. You can't get data on risky events if you're too cautious. kinda like how they used to vivisect criminals.

Not According to the FA

[rsilvergun]

According to the FA she wasn't texting, she was minding the screen in the car, per Uber's instructions, to take notes on interesting events for later research. Uber used to have two people, one to watch the road and one to take the notes and they did away with the second one to cut costs.