Uber 'Neglected' Simulation Testing For Its Autonomous Vehicles, Says Report (engadget.com)

According to a report from The Information, Uber allegedly "neglected" simulation testing for its autonomous vehicles. "The publication's sources claim that there was a dearth of investment in the simulation software, and lots of incompatible code between the autonomous vehicle software and simulation software Uber is developing internally," reports Engadget. "However, the sources said there isn't a direct link between the lack of investment and the fatal accident involving one of Uber's autonomous taxis and a pedestrian." From the report: It's worth noting that the Unreal Engine-powered simulation software is still relatively new. The Information writes that the suite wasn't developed until after self-driving project lead Anthony Levandowski was fired mid-2017. To add insult to injury, initially, there were also differences in pay between simulation engineers and other engineers in the department. The end goal was to release a self-driving car in Arizona this year, codenamed "Roadrunner," to compete with Waymo's offering just outside of Phoenix.

Comments

Dearth (derp) vader

[nnet]

Uber non.

Re: Dearth (derp) vader

[Anonymous Coward]

Only company with a worse failure rate is Shitla and that POS Musk.

Hi, Rei.

Re:Dearth (derp) vader

[Anonymous Coward]

"How young Donald Trump was slapped and punched until he made his bed"

http://www.nydailynews.com/opinion/donald-trump-fellow-cadet-article-1.3401110

My last conversation with Donald Trump was at the New York Military Academy, where we were both cadets. It was 1964, the year he graduated. We were walking together near the baseball field where, he reminded me, he'd played exceptionally well. He demanded that I tell him the story of one of his greatest games.

"The bases were loaded," I told him. "We were losing by three. You hit the ball just over the third baseman's head. Neither the third baseman nor the left fielder could get to the ball in time. All four of our runs came in; we won the game."

"No," he said. "That's not the way it happened. I want you to remember this: I hit the ball out of the ballpark! Remember that. I hit it out of the ballpark!"

Ballpark? I thought. We were talking about a high school practice field. There was no park to hit a ball out of. And anyway, his hit was a blooper the fielders misplayed.

But I wasn't going to argue with Donald. What was the harm in a little embellishment, if it helped him survive New York Military Academy?

NYMA, the private boarding school where Trump's parents sent him and where mine sent me, could be a brutal place where grown men who were veterans of the real military ruled with threats and force.

Trump's first year, under the command of Major Theodore Dobias, was hellish. Dobias slapped and punched him until he learned to make his bed and polish his shoes — things that Donald, an aggressive little wiseguy, had at first refused to do.

At some point Dobias assumed that he had broken Trump and eased up. More probably, Trump had figured out Dobias' weak points and had begun to exploit them. He flattered the major and became one of his "winners" who was favored with privileges and praised.

As the Academy's unofficial PR man, Dobias even contributed to the Trump myth, eventually telling Rolling Stone that pro scouts vied to sign Trump. As with many things Dobias asserted about Trump, this story may or may not be true.

Besides sports, most of Donald's years at the Academy were unremarkable. In his junior year, he was a supply sergeant in charge of the World War II M1s rifles we all lugged around at parade. But even in this laid-back position he was brash and assertive.

A member of the school band recallsTrump throwing shoes at him and yelling at him to shut up when this young man stood too close to the barracks trumpeting Reveille. Rumor had it that he got away with stuff like this because his father donated large sums to the school.

In his senior year, Donald was promoted to captain of A Company. Unlike other cadet captains who took an interest in the lives of the adolescents in their charge, Trump commanded at a remove. Aside from a determination that cadets in his care would always polish their brass belt buckles and keep the spit-shine on their boots, come evening he'd retreat to his room.

My friend Peter Ticktin, who was an A Company platoon sergeant, emailed me recently to say he saw Trump as someone who kept his thoughts to himself and delegated his responsibilities. "DT put his trust in me," Ticktin wrote . "(Although trust) may be too strong (a word), as I was not a confidant as to his personal thoughts. No one was. He was much to himself. A good guy, but no one's real buddy."

Trump couldn't remain aloof after one of his minions allegedly hazed a younger cadet. Ignoring the unwritten barracks rule that no report to the adult authorities be made, this cadet finked to his parents, who demanded a meeting with the superintendent. It resulted in Donald's removal as captain.

Any other cadet caught in such a scandal would have been busted to a lower rank and exiled to a different barracks. But Donald was transferred, with no loss of rank, to what was probably intended as a desk job. (He called it a promotion.)

While Donald had not succeeded as a manager of young men

Re:Dearth (derp) vader

[sexconker]

Imagine being so salty about the election that you shit out this tripe on a daily basis.

Re:Dearth (derp) vader

[Anonymous Coward]

Yeah I type it every time Conker-wannabe, you're not only such a genius that you think that but your life is so good that you comment so! :D But at least you (probably?) won't die a traitor in Federal Prison like Dunny.

But it worked fine in Kerbal Space Program

[Jeremi]

It's worth noting that the Unreal Engine-powered simulation software was relatively new

Hmm, turns out the real world is different than the Unreal world. :/

Re:But it worked fine in Kerbal Space Program

[philmarcracken]

KSP engine is unity.

Re:But it worked fine in Kerbal Space Program

[Anonymous Coward]

You don't very often launch portable nuclear weapons upon your fellow road users in the real world, even though some clearly would require it for the Redemption.

Backseat Engineering

[Anonymous Coward]

Who is to say that they "neglected" simulation? I'm no Uber fanboy. Engineers working on the main show make more than simulation and test engineers at a great many companies. In addition, in my experience, simulation, especially of something complex and cutting edge like this (rather than say, FEA), is often of little value, because it is so hard to really model the world. Right or wrong, paying less to simulation engineers was not negligence. These wild conclusions seem to stem not from the facts, but from "The Information's" desire to create a sensational headline. Maybe Uber WAS negligent. This article does not convincingly make this case.

Re:Backseat Engineering

[Luthair]

You don't need to model the real world. You drive the car around with sensors on and record the input, then in simulation you feed the input from all the weird scenarios you recorded into your system.

The behaviour of Uber, Tesla and others is quite frankly reprehensible. The testing of these systems need should be at a higher bar than anything NASA or aeronautical firms given the widespread deployment, reduced control and much wider variety of possibility of harm.

Re: Backseat Engineering

[Anonymous Coward]

No, no, musk is our hero and you must worship him. He can do no wrong. Deliberately designing life safety critical code is not the muskian way, and is wrong.

I say that not knowing Teslas design processes, but having seen SpaceX's pencil whipped space rating artifacts.

Re:Backseat Engineering

[Anonymous Coward]

I'm GP, and I work in the auto industry. Admittedly, I don't know that much about simulation, but I know a bit about how rigorously the software that runs the vital control systems is tested. The answer is not very. Much less than a jumbo jet or a Nasa spacecraft. MUCH less. And, cars, for the most part, don't have many catastrophic software bugs. There is an opportunity cost of delaying automated vehicles until they are close to perfect (a large one). I am in favor of the wild, wild west style with which we're approaching automated vehicles. I think Uber is taking it too far, but Tesla is pretty close to being reasonable. You can't expect them to test cars like a boondoggle government spacecraft or you'll end up with a $100,000 Model 3, or, worse, we'll set automated driving back a decade or two, costing countless lives in the process.

Re:Backseat Engineering

[Pinky's Brain]

In the case of Tesla I also lay blame with the regulators. At least for Uber and Google the regulators thought they had professional test drivers (which in Uber's case wasn't true, they had a professional diagnostic screen watcher ... which did her job, which was not paying attention to the road as the car killed someone).

In Tesla's case we know the system as designed insures there will often be no one paying attention behind the wheel, yet the system as designed requires the driver to pay attention. Tesla's system is unsafe by design. Regulators should have never allowed this nor keep allowing it. It will continue to kill people and it's only by accident it keeps being the occupants of the Tesla instead of innocents.

Re: Backseat Engineering

[fluffernutter]

Also, how is the guy supposed to sell stuff? Selling stuff for profit is an American right!

Re:Backseat Engineering

[fluffernutter]

There is no assurance that they will ever get to a point where they will save a significant number of lives. Until we have hard evidence of that, I would prefer they stop creating their own deaths. In the case of Autopilot, people are lulled into a false sense of safety until they make a mistake that kills or injures themselves or others.

Re:Backseat Engineering

[HornWumpus]

Weird scenarios are very uncommon.

Unless you captured a _buttload_ of sensor data, you'd overtrain for the few you caught.

A major car manufacturer will need to capture years worth of 'entire production line' worth of autonomous car grade data. Then curate the accidents and near accidents into a trainable set.

Then your still left with an AI that can't infer intent, at best, immediate course.

Re:Backseat Engineering

[murdocj]

I don't know about this. I see weird stuff every day... people opening car doors into traffic, people wandering out into traffic, cars wandering into my lane. I'd have to think if you recorded a month's worth of data you'd have seen a LOT of stuff. Not to mention you could even generate your own weird scenarios. The fact that the Uber car didn't respond to a major obstacle in front of it seems like there wasn't a lot of testing of any form done.

Re:Backseat Engineering

[fluffernutter]

They have likely had an opportunity to record the types of occurrences you are talking about already, and it hasn't gotten them very far. I'm not pretending I know what percentage of possible world scenarios they need in order to be 'safe enough'. I just know that they need a lot more than they are getting, and that it is a problem that they don't seem to know what that level is.

Re:Backseat Engineering

[Anonymous Coward]

'Weird' scenarios like the car immediately in front of you leaving your lane to overtake a stationary vehicle he'd spotted?
'Weird' scenarios like a pedestrian pushing a bicycle crossing the road in front of you?
'Weird' scenarios like there being a police car or firetruck stopped up ahead of you?

Sounds to me like all the accidents with autonomous cars so far have been utterly predictable, commonplace, and avoidable.

That the companies involved get to just shrug, claim "teething problems" and nobody else gives a fuck or demands they be held to account, or trots out "but on average autonomous cars are safer!" as though that excuses gross failures to avoid obvious hazards, is worrying.

Re:Backseat Engineering

[MobyDisk]

That approach works but is limited in several ways: You can only playback scenarios that have actually happened, and it doesn't work if the hardware & sensor arrangement is still changing. It is also slow since it is more of an integration test than a unit test. I've worked with this approach before and it is nice once everything is locked-down, but it is not great during the R&D phase.

Re:Backseat Engineering

[Anonymous Coward]

None of those examples are weird. Google's example of a person in a motorised wheel chair chasing ducks in circles on the road, now that's weird.

Re:Backseat Engineering

[Luthair]

If you change your sensor arrangement you start from scratch.

Re:Backseat Engineering

[Luthair]

There is an opportunity cost of delaying automated vehicles until they are close to perfect (a large one).

What exactly? Your short term stock options?

we'll set automated driving back a decade or two, costing countless lives in the process.

Ah, this specious argument again. Self-driving cars have yet to show this to be accurate, Waymo is apparently the best of the bunch yet requires human intervention on an average of 5000 miles - given that the average American drives 13k miles a year the best automation would crash a little under 3 times a year. While Tesla touts its autopilot notice how it never discloses how frequently it disengages, how often the driver must pro-actively prevent an accident, nor how often it does something dangerous without resulting in an accident.

Re:Backseat Engineering

[Dutch Gun]

Maybe Uber WAS negligent. This article does not convincingly make this case.

Perhaps. But we do have the evidence that their car mowed down a pedestrian it should have easily detected.

"A person walking in front of a moving car" isn't exactly some incredibly rare edge case that no one could have thought to test for.

Re:Backseat Engineering

[Dare nMc]

> The fact that the Uber car didn't respond to a major obstacle in front of it seems like there wasn't a lot of testing of any form done.

Just shows a lack of regression testing. I guarantee it stopped for people walking in front of it before placing the car into street testing. They changed object identification, almost certainly without repeating those tests. This is why you want simulated testing, when you change any software, you can re-run hundreds of hours of tests in minutes, without wasting any fuel or having tons of hardware. You then run a few important scenarios on real hardware before pushing out to the real world.

Uber clearly missed regression testing that with the death. Tesla clearly missed the real hardware regression testing when their software changes added 20 feet to their stopping distance.

Re:Backseat Engineering

[Anonymous Coward]

You drive the car around with sensors on and record the input, then in simulation you feed the input from all the weird scenarios you recorded into your system.

How would that work? Obviously the input is not going to include outcomes of various decisions, only the "before" data. Then what?

Re: Backseat Engineering

[Anonymous Coward]

a person in a motorised wheel chair chasing ducks in circles on the road, now that's weird.

Hey, I don't tell you how to live your life!

Re:Backseat Engineering

[Paradise Pete]

Weird scenarios are very uncommon.

Here's a good start...

Re: Backseat Engineering

[houghi]

Why should it be higher? I could understand as high, but why would ot be higher and that raises the question why it could be lower for NASA than for cars.

Re:Backseat Engineering

[thegarbz]

The testing of these systems need should be at a higher bar than anything NASA or aeronautical firms given the widespread deployment

Why? NASA considers loss of life unacceptable, yet we're quite happy to accept over 5 million accidents causing over 30000 fatalities every year from cars. Your demands are completely unreasonable in the face of the risk that has already been accepted by the population as well as the policy makers.

Re:Backseat Engineering

[Anonymous Coward]

Valley internet companies are the wrongest places on earth to be developing life-safety critical systems.

They are shielded from any consequences of bad design by the EULA you click when you sign up for their services, and their corporate culture values rapid deployment over everything, with users acting as alpha testers for all changes that make it through the regression test suite.

"Move fast and break things" is not the right culture for developing a guidance system for 1 ton killing machines.

Tesla is probably the only company with the right culture, since Musk makes cars and rockets which both have to work right, but he seems to overpromise and push too hard on the car side, plus he's got those saboteurs running around. Google doesn't seem to be that bad on quality despite being an internet software company, and they have a gaggle of top level AI people, so maybe they're the best ones for the job.

But Uber? I wouldn't trust them to build a bicycle. They should just wait for waymo and license the tech like everybody else.

Re: Backseat Engineering

[Anonymous Coward]

I work in the self driving car industry and safety or regulation is considered a priority if it does not interfere with short term deliverables. The goal is to make the demos look impressive enough so that VCs will want to invest hundreds of Millions. A lot of the AI is âenhancedâ(TM) by invisible remote human drivers, so that braking and turning looks smooth and smart. I definitely would be very worried if we started having our cars drive near playgrounds or schools, they just would not understand that a small kid is actually a human. The greed is strong in this business, and I donâ(TM)t think Iâ(TM)ll be part of it much longer. Iâ(TM)m kind of pissed off at CA DMV for not making the permits more expensive so that they can fund regular safety practices inspections to ensure that the companies are abiding by the letter and spirit of the regulation.
Iâ(TM)m really hoping that charges of involuntary manslaughter will be filed against the business decision makers at Uber but also any and all engineers that disabled the safety features without any form of protest.

Re: Backseat Engineering

[Anonymous Coward]

GP again. I invest primarily in index funds. I own no Tesla stock (unless through a fund).

Re: Backseat Engineering

[Anonymous Coward]

Higher, due to the potential to kill more people.

Re:Backseat Engineering

[Anonymous Coward]

> in simulation you feed the input from all the weird scenarios you recorded

And in simulation, you can run lots of variations of these weird scenarios, so that the system learns to handle the weirdness in a broader way. Or at least, that's how Waymo have been doing it.

Re:Backseat Engineering

[Anonymous Coward]

Beyond reprehensible. The company should be given the "death" penalty for this and even the engineers should serve some time. The responsible managers should serve more time, and the CEO should get 20-life. That would send a "chill" down the tech community that new and shiny when it deals with human life should be extensively tested, you know like the medical industry does. I know as a EE I never got my PE, but CE's tend to have to sign off on stuff that can cause death. Maybe if someone had to "sign off" and was culpable criminally when it failed it might light a fire to you know TEST.

Re:Backseat Engineering

[MobyDisk]

3/4 of the way through the project: "If we move the front-left imager 6 inches to the left and adjust the angle 5% up we should fix scenario thusandsuch..." That means you need all new test data, which in this case might require running over several old ladies a few times to get the test data you need. ;-)

Re:Backseat Engineering

[HornWumpus]

Uber _started_ setting up virtual world testing of its AI _after_ this accident. They are clowns.

Tesla appears to be a little better. But how do you debug an neural net 'AI'? All you can do it change the training dataset and test the results to death. Your test/training (hopefully different/built independently) set will be incomplete and you will kill more people, you're just hoping it will be less than would have died on their own.

So, the Microsoft ...

[CaptainDork]

... model.

Re:So, the Microsoft ...

[Anonymous Coward]

Microsoft models are little soft at all the right places.

beta testing doesn't belong on the road

[Anonymous Coward]

all these self-driving companies are literally beta testing this with people's lives

it's one thing to rush your software out, try to patch later and make up for hardware flaws with workarounds

it's another thing when painted lines and stationary objects are completely missed or purposely ignored by the software

I know everyone wants their fully automatic cars right now but they needs YEARS of independent testing and debugging before this code is allowed on the road

The word you're looking for

[rsilvergun]

is 'skipped', not 'neglected'.

So how did they train their AI?

[Anonymous Coward]

Huh. You would have imagined that modelling some roads, street furniture, other cars and pedestrians, then running massive training and validation datasets based on synthesised sensor readings, would have been how they initially trained their autonomous AI's, before moving out into the real world...

Did they do something different, somehow? Why?

What about the safety driver/face detection?

[Anonymous Coward]

I am glad Uber's involuntary manslaughter is getting fully investigated and all the complexities involved discussed. However, this is one much more simplistic question that I can never find the answer to:

Was Uber performing automated monitoring of the safety driver?

There has been discussion on the AI ignoring what the sensors detected to avoid over-reaction to false-positive events. It makes sense the AI would be sometimes tuned incorrectly provided there was a safety driver *paying attention* to address that situation.

This article points out the degree to which simulation software available today wasn't available before. Again, it makes sense to only use the resources currently available at the time if it is augmented by a safety driver that is *paying attention* to address gaps in the pre-road testing.

Even if unreal engine didn't have a simulation application available until recently, there has been face detection software available for a long time. I have a camera from a decade ago that does auto-focus based on face detection. The ability to monitor the face of the safety driver and automate triggering a response to the driver not performing the role they need to be has been available for a long time. What is Uber's excuse for the car never reacting to the safety driver looking down for extended periods of time while the Uber car performs termination of a person's life?

At the end of the day, if Uber is going to refuse to answer the what/how/why regarding the degree to which they had automated monitoring of the safety driver, then we need to respond accordingly. CEO Dara Khosrowshahi's response that it is "some incredibly sad news" is not good enough. So far after waiting nearly three months the information regarding Uber's method of making sure the safety driver was actually performing as a safety driver has been zilch. This is not "sad news" as much as it is *HORRIFYING* news. I personally do not trust Uber CEO Dara Khosrowshahi at all or the company he runs.

Uber has shown itself to be the Russian roulette of self driving car programs. We shouldn't be forced to ask ourselves how long will it be before we have another tweet from @uber_comms about their hearts going out to another Uber victim's family. Instead of extending their hearts they should get a CEO that makes holding safety drivers accountable via an automated system a top priority of field testing with a zero tolerance policy in performing field testing while the safety driver is not doing their job.

No XIL testing?

[0100010001010011]

How in the world did this make it to production without SIL/MIL/HIL testing?

dSpace sells HIL benches specifically to test ADAS

Re: No XIL testing?

[Anonymous Coward]

Came here to say exactly the same thing. How the hell was uberâ(TM)s autopilot software allowed on the road without SIL/HIL testing?

catch-22

[fluffernutter]

I sense there is a catch-22 here. If they were able to write a simulator capable of using some sort of chaos theory to produce any possible situation that may occur in he real world, wouldn't they know how to write an AI engine that could anticipate any possible situation in a self driving car?

The new way

[Anonymous Coward]

It's the new way of software development.
Fail fast, fail often.
and
I don't want it perfect, I want it Wednesday.

Uber is dying

[Anonymous Coward]

Once upon a time when you called for an Uber, you'd have a nice courteous, clean driver. Usually a college student or someone between jobs. Invariably white, respectful, knowledgeable and conscientious. That was then.

Now you are more then likely to get some greasy dirty dune coon or ay-rab muslim who doesn't even know the area. The car is dirty and the sand n1gger spends all his time fiddling with his GPS and talking oogabooga language on his cell phone even though it's illegal. To hell with Uber. Did I mention the ripe aroma that only a person of that ethnicity seem to possess? It's as though they have simian musk glands.

I can call a legitimate taxi company and at least I will be assured of a brick and mortar organization standing by their product. Unfortunately I may still get a dune coon but they will have been fingerprinted, and photographed, and screened by the police. Uber was OK when it was strictly amateur, but then middle eastern hustlers and rag heads took over, and it ruined a good thing.

Sequeezing toothpaste back into the tube.

[aberglas]

The hardest part is to take input data and understand what it means. Games engines actually do a pretty good job of creating that data. It is not substitute for real testing, but it is an essential part. Because you can test much more. And because you can test crash scenarios that would be too dangerous otherwise.

If generating an image from a model is like squeezing toothpaste out of a tube, understanding the generated image is like squeezing it back in.

And a good simulator will introduce some noise etc.

Just like Facebook

[cleavet]

I think Uber has adopted the "Move fast and break things" development model...