Slashdot Mirror
Uber Ordered To Take Its Self-Driving Cars Off Arizona Roads (nytimes.com)
After failing to meet an expectation that it would prioritize public safety as it tested its self-driving technology, Uber has been ordered to take its self-driving cars off Arizona roads (Warning: source may be paywalled; alternative source). "The incident that took place on March 18 is an unquestionable failure to comply with this expectation," Gov. Doug Ducey of Arizona wrote in a letter sent to Dara Khosrowshahi, Uber's chief executive. "Arizona must take action now." The New York Times reports: Uber had already suspended all testing of its cars in Arizona, San Francisco, Pittsburgh and Toronto. "We proactively suspended self-driving operations in all cities immediately following the tragic incident last week. We continue to help investigators in any way we can, and we'll keep a dialogue open with the governor's office to address any concerns they have," said Matt Kallman, an Uber spokesman. The rebuke from the governor is a reversal from what has been an open-arms policy by the state, heralding its lack of regulation as an asset to lure autonomous vehicle testing -- and tech jobs. Waymo, the self-driving car company spun out from Google, and General Motors-owned Cruise are also testing cars in the state. Mr. Ducey said he was troubled by a video released from the Tempe Police Department that seemed to show that neither the Uber safety driver nor the autonomous vehicle detected the presence of a pedestrian in the road in the moments before the crash.
Testing is done for finding problems. They found one. Don't stop testing now!
Testing is also done (or should be done) in controlled environments until you get way past the alfa and beta stages. Putting the autonomous car on the road can be justified when the car doesn't need human supervision and it can deal with normal traffic conditions in day and night with the same performance as that of a human driver.
I seriously have no idea why autonomous cars in pre-alfa stage are on the roads.
Unfortunately most people are treating autonomous cars as software. And we know how software engineers think. Throw the alfa software to the public and fix mistakes afterwards. Oh and we're not responsabile for anything the software might do that brings down your house, empties your bank account etc....
I didn't see any system to keep the driver's eyes on the road and hands and feet on the controls, you can't rely on the human to remain vigilant. You have to force him to be vigilant. Just like in trains.
Because the driver in this instance is a bogus legal device to pretend that self driving cars running self driving tests are somehow not doing the driving.
He can only judge the cars choices by the outcome of those choices and that would be too late to intervene. He is not making the driving decisions, he is not driving the car.
Instead of testing self driving cars until they are safe to unlease on public roads, they let car makers get away with this bogus legal device.
More troubling in this case is the doctored video they released. A normal car recorder video in the dark, would have the gain turned up, and would be bright but grainy. That video was dark, suggesting it had been darkened. I ask again, did the police obtain the video from the car recorder themselves, or did they ask Uber's technical assistence, because the video shouldn't be dark. Especially darker around the edges which suggests intentional vignetting.
I guess you're not aware that the governor of Arizona is a Republican? I don't know much about him, but I'd presume that means he leans conservative, not liberal. Self-driving cars are being tested in California as well, and now perhaps you see why you can't always trust a corporation to self-regulate in the absence of government oversight. This is, sadly, how government regulations tend to come about.
The pedestrian was technically in the wrong, but we've heard a lot of rumors recently that Uber's self-driving software was being pushed forward recklessly. And given the wonderful people at Uber and their stellar track record, this isn't exactly hard to believe. A competently programmed car with a properly functioning lider probably should have seen that woman on the bike and reacted to it by braking far earlier than it did.
Yes, deaths will inevitably occur, but let's at least try to make sure there are as few as possibly going forward. This is a good reminder that machines can be just as fallible as the humans who create them.
Irony: Agile development has too much intertia to be abandoned now.
Since the driver was unable to detect this incident too, they better remove all drivers as well!
Change is certain; progress is not obligatory.
which presumably he was there for, and trained to do,
. . . or he was some poor guy, so desperate for money, that he was willing to work for Uber, despite their stellar record of how they treat their employees.
With Uber, don't presume anything that will cost them money.
why is is this not simply a case of dangerous driving/driving without due care and attention?
Because Uber had a status in Arizona of being "more equal than others." The cops originally wanted to dust this under the rug . . . after all . . . it was just a homeless bag lady who died, right . . . ?
Someone at the police needs to call up Facebook and take a look at the driver's call data. Was he texting at the time of the accident . . . ?
we all know they are not good enough 'yet', so surely it's down to human failure, as any ordinary accident involving a motor vehicle.
"Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error."
Among the many safety concerns of autonomous vehicles . . . the biggest danger is mechanical:
The loose nut behind the wheel.
Autonomous cars safety mechanisms need to be human driver proof!
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
SAE Level 3 automation should be illegal. Period. The backup driver simply cannot be expected to go from "no interaction with the vehicle for 1500 miles on average" to "rescuing the vehicle from an emergency".
Automation should be locked at Level 2 (ProPilot, Autopilot, Supercruise, etc, etc) until vehicles are at least ready for Level 4, if not Level 5. Level 2 = hands on the wheel, ideally with eye tracking, ideally making the user drive for themselves for at least a couple minutes every hour to stay alert.
And it should not be up to companies when their vehicles are ready to put them on the roads, as they're in a race to be seen as first movers. Governments should have their own review and testing processes, which involve both code audits (in the case of neural nets, audits of the net core and how the nets are trained) and real-world testing of simulated hazard scenarios.
"99 dead duelists of Dios on the wall. 99 dead duelists of Dios! Take one's ring, pass it around..."
According to the Arizona Department of Transportation, at least as of 2016 [azdot.gov], there were 952 fatalities in car accidents in Arizona, or approximately 2.61 deaths per day.
So far so good - now look up the number of cars in Arizona (about 2.4 million) and the number of Uber self-driving cars (200 across 4 cities). Now apply "appropriate precision" in Uber's favour - 200 out of 2 million = 1/10,000 of AZ cars are uber self-drivers. So, with 1000 fatalities/year, Uber get to kill someone every 10 years - they've used that up in one. (Of course, that's an unspeakably crude and dubious calculation, but its better than yours).
Then, of course, of those 1000 regular fatalities, many will be attributed to drunk-driving, speeding, texting (or other forms of reckless driving), non-roadworthy vehicles etc. all of which carry potential criminal penalties - including possible driving bans - so its not the case that nothing is being done about them.
Uber were allowed to test experimental vehicles on the condition that they'd have a safety driver ready to take over - and one thing that the video clearly shows was that the safety driver was not paying attention (to the surprise of absolutely nobody except, apparently, Uber). The video also shows that the pedestrian was crossing the road in clear line-of-sight, in a street-lit area, from left-to-right yet the car made no attempt to brake or swerve. If you believe that the video truly represents what the Mk 1 eyeball and/or the car's sensors could "see" then all that proves is that the car was going too fast for the conditions - outdriving its headlights - and the driver should have taken action to slow it down.
In a survey of 100 programmers, 111111 thought that duck-typing was a good idea.
Level 3 is fine if there is plenty of warning to take over, say a minimum of 30 seconds. Time to put away your phone, pack away the sandwich and slip your shoes back on.
Unfortunately what we have is crap like this from Audi: https://youtu.be/WsiUwq_M8lE
Note the way it disengages suddenly and the guy has to instantly take over. That's dangerous and unacceptable.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Level 3 is fine if there is plenty of warning to take over, say a minimum of 30 seconds.
In other words level 3 is never fine.
But that more or less gets you to Level 4, which IIRC is just Level 3 + pull over safely if something goes wrong.
Socialism: a lie told by totalitarians and believed by fools.
It gets better, hereis what the street really looks like at night. About 1 second before the accident there is a yield to bikes sign due to all the bicycle traffic in the area.
Having a human that can get bored babysitting automation is fundamentally the wrong way to do it anyway. Until the automation is reliable to not need human supervision, it should be the other way around: the automation monitoring the human to provide an extra safety net, so that the human has to be actively driving and alert rather than getting bored and not monitoring the automation properly.
Oolite: Elite-like game. For Mac, Linux and Windows
What's your proposed model for testing an autonomous car driving amidst normal traffic conditions that does not include actually having it drive among normal road traffic?
Secondly measured in terms of accidents and fatalities, autonomous cars have already caused less accidents per miles driven than your average human driver, so if that's your metric, the argument can be made that said bar has already been passed, although that obviously does not mean that the safety cannot and should not be further improved until the fatalities drop to zero.
We should also know all know that alpha or not, there's no bug-free software. You can do all the simulations and all the testing you want, bugs and accidents will still happen. However, once they do and are fixed, the vehicles will not do the same mistakes again, which is not true for most human drivers. Human drivers also do not learn from the mistakes of other human drivers that they've never met. Autonomous cars do.
I fail to see what the problem is here. We all knew this was going to happen because it always happens with new transportation technologies that are in the process of being perfected. Plane safety has gone up dramatically as a result of tens and hundreds of accidents in both software and hardware, and the same's true with 'regular' cars. Your standard seems to me to be an illogical 'unless it's completely failsafe it should not be used at all', and if we followed that principle we'd still be using horses.
"It is the business of the future to be dangerous" -Alfred North Whitehead
Flash a dim light on the windscreen at random intervals, and ensure the human responds. No mobile phone usage then. This sort of thing has been done for trains for ages (I'm not talking about dead man, but attention monitors.)
Uber did not care. And that video they released was dubious, someone else took a dash cam of the area and it was reasonably well lit, even if there was no Lidar.
Uber were totally negligent. And I suspect they did not pay that driver very much. Monitoring a test car should not be a minimum wage job.
proposed model for testing an autonomous car driving amidst normal traffic conditions that does not include actually having it drive among normal road traffic?
Hire people operate vehicles on closed tracks to simulate traffic and specific scenarios.
Or, better yet, orchesate dozens of other self driving cars to move in fixed choreographed patterns that the autonomous vehicle must recognize and react appropriately to.
Might be pointless but one recommendation I would have to the automated car companies is any warning sign, like yield to bikes, should have a visual and audible alert by from the car to "wake up" the human monitoring the car. Though given how often nothing would happen after that alert, it would probably get ignored after a while.
What's your proposed model for testing an autonomous car driving amidst normal traffic conditions that does not include actually having it drive among normal road traffic?
Perhaps they should focus first on not killing pedestrians on an otherwise empty road, and worry about "normal traffic conditions" later.
autonomous cars have already caused less accidents per miles driven than your average human driver
Statistics are like bikinis. What they show is not anything special. What they conceal is vital.
Self driving cars only operate in a very subset of extremely limited conditions. When you compare like for like, limit the stats on human drivers to the same limited conditions that self driving cars, do you sell that have killed (3 people killed so far) and been the cause of far more accidents than human drivers.
Perhaps they should focus first on not killing pedestrians on an otherwise empty road, and worry about "normal traffic conditions" later.
And will you apply the same standard to human drivers, who kill 270000 pedestrians per year on average?
Shadow operations. Don’t control anything, just observe. If software action deviates from human action, review and analyize. It appears that this is the stage in product development that Uber needs to be in.
Once the shadow driving has advanced to a point where there are no negative deviations in a control environment, define that envelope and test rigorously within the envelope. Continue to shadow outside the envelope and slowly expand the envelope. Within test envelope, thoroughly validate performance with external telemetry— were there any cases where nothing bad happened, but the action taken should have been different.
..unless AI can read the future.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
In most resorts, the class action suite is designed to be a rugged and easily cleanable venue for homecomings and spring break: vomit proof furniture, a punch bowl full of condoms, quick-change sheets, and the usual stack of brochures for 'safety' schools to apply to after your kid gets expelled.
Back to the fact that Uber doesn't legally get to kill people just because they are stupid. Just when I think the conversation is rising above, someone brings it back down; usually Anonymous Coward.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
Yes they would have seen her and so would you. The video released is overly dark. If you headlights only saw two stripes ahead you would be hitting a lot more things. But also check out other examples of the exact stretch of road.
On top of that why didn't the car even attempt to break? I don't care about human reaction time, I would just expect my self driving software to be able to pick up an obstacle that big in sub 1.5 seconds. And that is just off crappy video, what the hell was the lidar doing?
Uber obviously has shitty software. Force them to make it better before trying again.
"We all knew this was going to happen because it always happens with new transportation technologies that are in the process of being perfected."
So people dying during testing is considered normal??? What kind of a fucking douche bag are you....
For the brain damaged out there, no people dying during testing is not considered normal. As others have pointed out, testing is done in controlled conditions before they are allowed in live environments. THIS WAS NOT DONE.
This kind of scenario is a rather fucking simple one to test!!
So fuck off and die.
This is not a person who suddenly jumped out onto the road here.... while she was jaywalking, she was also *WALKING*... I've seen an overhead view of the section of road where the incident occurred, and there's no significant occlusions there; ordinarily, vision seems that it would be pretty good there in daylight conditions. It's my understanding that self-driving cars use lidar sensors, and should even be able to detect a person in an absence of any visible light at all, so the fact that it was night should be immaterial. Reasonably, the car should have seen that she was on the sidewalk long before she stepped out into the road, and the very *instant* that she started to go off of the sidewalk should have been detected by the car, and the car should immediately begin to slow down.
Yet, by all reports I've heard, the car did not even see this pedestrian at all, and had not even tried to slow down until after the collision. Why? What the fuck happened?
File under 'M' for 'Manic ranting'
What's your proposed model for testing an autonomous car driving amidst normal traffic conditions that does not include actually having it drive among normal road traffic?
I thought Uber had an entire town set up for this sort of thing. On a related note, someone should develop mechanized crashtest dummies that can be programmed to walk into the path of an autonomous car; when the cars are capable of avoiding people who are actively trying to get hit, then (and only then) have they achieved enough superiority over humans that we can justify (maybe and only maybe) thinking about sharing our roads with them
In the Uber video, I thought it was a lonely desert road. In the 'real' video, OH! There's an entire city there!
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
What's your proposed model for testing an autonomous car driving amidst normal traffic conditions that does not include actually having it drive among normal road traffic?
As far as I've read, Google (Waymo) created an entire fake town to test different traffic situations out before letting their autonomous cars onto the actual roads in actual towns...I have not heard of Uber doing such a thing.
Secondly measured in terms of accidents and fatalities, autonomous cars have already caused less accidents per miles driven than your average human driver, so if that's your metric, the argument can be made that said bar has already been passed, although that obviously does not mean that the safety cannot and should not be further improved until the fatalities drop to zero.
This is a completely irrelevant statistic at this point. You're comparing accidents per miles driven of regular vs. experimental self-driving cars (all of which, btw, have human drivers which are supposed to - and often do - intervene to prevent accidents)...the two "sample sizes" so to speak are so vastly different that no valid comparison is possible. How many miles have self-driven cars driven? Several orders of magnitudes less than regular cars. How many self-driving cars are on the roads at any point? Tens? Hundreds? On the other hand there are hundreds of millions of cars in the US alone, which means tens of millions of cars are on the road every day, i.e. at least a couple of million driving at any one moment. Those regular cars drive in all conditions, on all roads, during all times of day, in every imaginable traffic situation, and, apart from beginner drivers riding around with instructors (who have a second brake to use at the passenger seat), none of those drivers are being constantly supervised by someone who can take control of the vehicle if they make a mistake. So comparing their accident record to self-driving cars which are being tested in a handful of cities and which, by admition of the manufacturers of said self-driving cars, cannot yet handle all weather conditions for example, is just...nonsense.
Bottom line: put 5 million self-driving cars on the road (using current technology), let them drive randomly around (i.e. no careful selection of driving days based on weather, routes based on street suitability for current technology, etc.), remove the human driver supervising them, and I'm pretty sure that accident rate will be higher than with normal cars.
Finally, the link you provided talks only about Google cars. No data about Uber or the other players in this game.
Plane safety has gone up dramatically as a result of tens and hundreds of accidents in both software and hardware, and the same's true with 'regular' cars. Your standard seems to me to be an illogical 'unless it's completely failsafe it should not be used at all', and if we followed that principle we'd still be using horses.
You're missing a crucial point here. When cars first came about, many cities would not allow them...many others allowed them if a person would walk (yes, walk) in front with a red flag, a horn, or shouting "danger!" or something like that (yes, this is true...). Early cars were, in modern terms, painfully slow - not just due to lack of engine power but also due to lack of infrastructure (no paved roads) and regulation ("jaywalking" had to be invented as a crime to allow car traffic to move more smoothly and more safely, on the back of a lot of lobbying by the car companies). There was a gradual ramp up of cars and the necessary infrastructure to support them. Similar things happen with airplanes - airports were placed away from built up areas, flight routes restricted, and this meant that potential harm from a failure was limited to the passengers - whom no one had forced to fly. Testing of self-driving cars on infrastructure not built for them (but built for huma
I can imagine some lawmaker somewhere declaring a halt to driverless cars after this accident.
I have already several articles suggesting that this should not be done because only more and more refinement of such a complex product will cause it to become viable. Also even with a few bugs, driverless cars are possibly already less accident-prone than humans.
As a software developer, I naturally side with continuing development.
Looking at the FAA gives a good model on how to proceed.
When an airplane crashes, the FAA sometimes grounds all models of that plane until the cause of the crash is determined and, if it was a technology error, will not allow the planes to fly again until the problem is satisfactorily resolved.
That would appear to be a measured response to this type of problem.
Don't halt all development. Don't proceed, ignoring the death(s).
Prohibit the specific driverless system from using the public roads until the problem is determined and an acceptable fix is made.
Just as cars have model years that receive approval, so should specific versions of driverless systems.
Then we can have official patches deployed on an as-needed basis, not just when a software engineer declares a bug has been fixed.
Very strict controls need to be in place to allow/deny a software/hardware update to a driverless system.
I don't want my car to be hacked and used as a killer weapon.
Let's not pretend they are even close to as safe as human driven vehicles. If you look at the real metrics, cars killed 1.18 people per 100 million miles. Autonomous vehicles are on track to do much worse, but we don't have clear statistics yet.
I'm also not confident that software engineers will have the same level of respect for life that other engineers have shown.
Cheap storage VM.
responsibility is on the driver. Read the traffic laws.
There was no driver: my arguments
1. No eyes on road.
2. No hands on wheel
3. No feet on pedals.
LITERALLY no driver in the car.
Imagine if they used Plan C: Vehicle-to-Vehicle communication to share sensor data. Crack open a VTV module, take the private key, spoof people everywhere suddenly to get cars to slam brakes and cause collisions all over the place.
VTV is an excuse for municipal governments to not invest in infrastructure (sensors on posts and poles), but rather to make people pay for it with not-tax-money.
I'm surprised Google's gotten to that phase already, in any case.
Support my political activism on Patreon.