Slashdot Mirror


Uber Vehicle Saw But Ignored Woman It Struck, Report Says (engadget.com)

gollum123 writes: Uber has reportedly discovered that the fatal crash involving one of its prototype self-driving cars was probably caused by software faultily set up to ignore objects in the road, sources told The Information. Specifically, it was that the system was set up to ignore objects that it should have attended to; Herzberg seems to have been detected but considered a false positive.

39 of 323 comments (clear)

  1. Oops! We left it in murder mode. by Anonymous Coward · · Score: 5, Funny

    So sorry for any inconvenience.

    1. Re: Oops! We left it in murder mode. by Anonymous Coward · · Score: 2, Insightful

      Not a bug, a feature! Cleaning up the streets one (homeless) cyclist at a time.

      Functions as designed.

  2. So who is to blame? by LynnwoodRooster · · Score: 3, Interesting

    Who is guilty of vehicular manslaughter, here?

    --
    Browsing at +1 - no ACs, I ignore their posts. So refreshing!
    1. Re:So who is to blame? by mea_culpa · · Score: 5, Interesting

      Probably the person playing on their phone as it was their job to override decisions made by buggy software.

      Also the video Uber released is highly altered. I drive on that street frequently and it is very well lit.

    2. Re:So who is to blame? by Narcocide · · Score: 2

      They outsourced that part to India.

    3. Re:So who is to blame? by Xylantiel · · Score: 3, Insightful

      What you appear to want is called "ghosting", where the driver drives and the autonomous software pretends to drive and the differences are evaluated. That is not the role of the safety operator. You appear to agree that a safety operator is a stupid concept because they have no way to know if the car is correctly evaluating any given obstacle until it is likely too late.

    4. Re:So who is to blame? by bondsbw · · Score: 2

      Any "ghosting" should be done by the automation system, with the driver in primary control.

      --
      All my liberal friends think I'm a conservative, all my conservative friends think I'm a liberal.
    5. Re:So who is to blame? by HornWumpus · · Score: 2

      If the purpose is to test, you can't disconnect it before it makes a mistake.

      But video shows the 'safety operator' was not watching the road. Which is understandable. Try watching a CNC machine with you hand hovering over the e-stop button, see how long you last. I guarantee you, it won't be an 8 hour shift of alert watching. Then again, he was in the car. You'd likely watch a lot better if your skull was in reach of the tool.

      Testing autodrive cars on the road is hubris. Build an artificial test environment first, let it run for virtual giga car-years. _Constantly_ collect new data for it with manually driven cars and traffic cams. Don't bulshit yourself, you won't be 'done and ready' in a year. Test it against Mumbai traffic...check that, maybe not, that's how you get Stephen King movies realized.

      --
      John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
    6. Re:So who is to blame? by Xylantiel · · Score: 3, Insightful

      My argument is that is the obstacle was too obvious. The safety operator had to also overcome their expectation that the car would do the right thing. The engineers doing post-crash analysis appear to only have a vague idea why the car didn't appropriately evaluate this blatantly obvious obstacle. But the safety operator was supposed to figure this all out in less than 3 car lengths.

    7. Re:So who is to blame? by msauve · · Score: 4, Insightful

      Does the safety driver get any feedback on what the autopilot is planning? I'd think that even a simple green/yellow/red indication to show what it's perceiving (everything's OK/I see something and am prepared to take action/I am taking action) would be useful.

      I could see the car recognizing a potential hazard well in advance of a need to take action - that info should be given to the safety driver. If they in turn take action before the autopilot would have, perhaps an algorithm needs tweaking. And, if the driver sees a potential hazard first, they should be able to provide feedback on that, too, so they can figure out why the human is doing a better job.

      --
      "National Security is the chief cause of national insecurity." - Celine's First Law
    8. Re:So who is to blame? by I4ko · · Score: 2

      Exactly. The bridge is 16 lane widths, and crossed almost at 45 degree angle at that. It is very very wide. Heck.. at that angle at 45mph which is the allowed speed I need 3 lane widths (effectively 4.24 at that angle) to bring the car to complete stop.

    9. Re:So who is to blame? by barc0001 · · Score: 3, Interesting

      > Probably the person playing on their phone as it was their job to override decisions made by buggy software.

      I dunno, looking at the video of the crash, the victim crossed the road outside of a crosswalk and wasn't even LOOKING in the direction of potential traffic. I'd assign the lion's share of the blame to the person who literally walked into the path of a brightly lit car without noticing.

    10. Re: So who is to blame? by Anonymous Coward · · Score: 2, Funny

      That sounds an awful lot like something someone guilty of vehicular manslaughter would say.

    11. Re:So who is to blame? by amicusNYCL · · Score: 2

      The road there is about as well lit as you would expect a big road to be, it's not as dark as the video implies. Granted, most of the time I'm down there I'm there for a concert and they might have additional lights on, but the video was obviously darker than what a person would see. At the place where she was crossing, the driver should have been able to see her crossing the road for several hundred feet at least.

      --
      "Our two-party system is like a bowl of shit looking at itself in a mirror." - Lewis Black
    12. Re: So who is to blame? by amorsen · · Score: 5, Insightful

      Of course you jail the engineers or architects, when they are criminally negligent.

      There is little doubt that the Uber CEO, Dara Khosrowshahi, was criminally negligent and aware that this kind of accident was not only possible but likely.

      --
      Finally! A year of moderation! Ready for 2019?
    13. Re:So who is to blame? by sexconker · · Score: 3, Interesting

      Try watching a CNC machine with you hand hovering over the e-stop button, see how long you last. I guarantee you, it won't be an 8 hour shift of alert watching.

      I'd probably go 12 hours or more. I love watching those things work.

    14. Re:So who is to blame? by sexconker · · Score: 2, Insightful

      I've seen plenty of videos. The street is very well lit in all of them except Uber's video.

      Interestingly, the camera facing the human "driver" is crisp and clear, using your standard "night vision" mode.
      The Uber video is either doctored or doctored.

    15. Re:So who is to blame? by vivian · · Score: 4, Interesting

      When I drive and see a possible danger (kid playing with a ball by the side of the road, dog wandering around near the edge of the road without a leash, motorcycle rider looking the wrong way on a side street as I approach, etc.) I always take my foot off the accelerator and cover the break - ready to instantly respond if something stupid happens. It's called defensive driving, and is how everyone should drive.
      I never really picked up this habit though until I had been riding motorbikes for a while, when you absolutely have to drive defensively id you want to survive commuting in London or Tokyo traffic. (I used to ride in both).

      The safety driver should have been doing this, and it would not be any impediment to testing the autonomy of the car - it can still do the driving, but the safety driver would have had time to react appropriately.
      The safety driver completely failed in her duty - possibly due to lack of training - but if your getting paid to be a safety driver then you should do your job instead of buggering around with your phone.

    16. Re: So who is to blame? by LynnwoodRooster · · Score: 5, Insightful

      No one is guilty of vehicular manslaughter. This is an accident due to bad design. You don't jail the engineers or architects who design a building that fails in an earthquake.

      You do if it wasn't designed to code.

      You don't arrest the airline execs when a plan component fails.

      Maybe not the exec, but certainly the maintenance engineer who committed fraud that resulted in the death of people.

      So here - who's to blame? Who decided to live-test an experimental system that can operate with the safety disengaged?

      --
      Browsing at +1 - no ACs, I ignore their posts. So refreshing!
  3. Oh good. by Ichijo · · Score: 3, Interesting

    The autonomous programming detects items around the vehicle and operators fine-tune its sensitivity to make sure it only reacts to true threats (solid objects instead of bags, for example).

    Then it's an easy fix. Just move the "sensitivity" slider a little to the left.

    Actually, it's kind of terrifying that all that stands between life and death is a sensitivity setting.

    --
    Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
    1. Re:Oh good. by cascadingstylesheet · · Score: 2

      The autonomous programming detects items around the vehicle and operators fine-tune its sensitivity to make sure it only reacts to true threats (solid objects instead of bags, for example).

      Then it's an easy fix. Just move the "sensitivity" slider a little to the left.

      Actually, it's kind of terrifying that all that stands between life and death is a sensitivity setting.

      Absolutely.

      I hope they use material design so the settings are all hard to see.

      And I really hope it's totally ambiguous whether you have to click Save, or if the changes to the Sensitivity slider will just save automatically, just because you touched them or something.

    2. Re:Oh good. by jellomizer · · Score: 2

      That is general true about the world.
      If you are a driver and you are distracted or not fully focused on the world around you, your sensitivity setting is just off too. The biggest reason why Motocycles get in accidents is because automobile drivers fail to see them, just because they may not be expecting a Motocycle, so their eyes are on the look out for fast moving objects that fill up at least 2/3 of the lane. This fact that we fail to comprehend things that we don't expect is how magicians trick us to see these magic tricks. Our brain takes a lot of shortcuts in processing the world the same way that the Uber Car decided not to deal with processing the women.

      When growing up as a kid, We needed to learn what is acceptable and what isn't. Growing up we have said and done things that had hurt people and ourselves. We learned from it so we changed our settings, if we went too far, then we found we couldn't get things done, so we more or less settled around some happy medium that works for our lifestyle.

      --
      If something is so important that you feel the need to post it on the internet... It probably isn't that important.
  4. Too large! by HornWumpus · · Score: 4, Interesting

    I understand that programmatically telling a blowing plastic bag from a child's toy is difficult.

    But she (and her bike) were clearly large enough to damage the vehicle. Even if the code saw her as debris, the car should have avoided it.

    I think the code had to have dismissed her as lens flair or something similar.

    --
    John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
  5. Re:The War Between Man and Machine by SeaFox · · Score: 4, Funny

    Yeah, really. How do they know the vehicle "ignored" the woman? Maybe it just acted like it didn't recognize it needed to take action when it really was targeting her.

  6. Uber and people who authorized this experiment by goombah99 · · Score: 5, Insightful

    This is a clinical trial. The FDA has long long long long long experience in conducting clinical trials. Now one can argue if FDAs caution is too much but even in the worst case everyone would agree they have a well established process for assuring something is safe and effective before you release it onto the public.

    Uber is conducting experiments on the public.

    If this were a new drug or treatment or medical procedure they would be shut down.

    This is actually far worse than that because most new drugs or treatments have clear lineages from prior ones that give us high expectations of what the outcome will be.

    The argument that something has to be allowed prematurely because in the long run it will save lives is a failed argument for medicine.

    In this case there is nothing to support the claim that this will save lives in the long run. Sure one could imagine that it would. But I don't think thats very well established. And if this were a drug study people would have spent the time and money to establish that.

    The claim that they have conducted 5 million miles (or whatever of testing) is rubbish. Those are not statistically valid tests. We execs dashing in front of the cars going 50 miles per hours in any of those tests? I assure you that did not happen.

    Moreover we already have evidence from those tests that driver re-aqusitions do happen frequently, and there is a substatnial lag in the hand over dues to human inattention. THe fact that they only had one driver in it says Uber is negligent.

    --
    Some drink at the fountain of knowledge. Others just gargle.
    1. Re:Uber and people who authorized this experiment by Locke2005 · · Score: 2

      It's not a clinical trial unless all the "patients" give informed consent. This is testing on people that have not be forewarned and have given zero consent. That is a very different animal.

      --
      I've abandoned my search for truth; now I'm just looking for some useful delusions.
    2. Re:Uber and people who authorized this experiment by stephanruby · · Score: 4, Informative

      If this were a new drug or treatment or medical procedure they would be shut down.

      Uber self-driving tests have been (mostly) shut down.

      Uber makes it sound like they suspended their testing operations voluntarily, but the fact is they lost their testing permits in Arizona, California, and one other state.

      And if there is any testing going now with Uber, it's only happening now in computer simulations, or in mocked up urban environments with fake pedestrians and bicyclists.

  7. Uber cuts corners by DogDude · · Score: 5, Insightful

    Uber's entire business model is based on cutting corners (not paying employees as employees, not following local taxi laws/regulations, etc.). I wasn't at all surprised to hear that one of their self-driving test cars killed somebody. I immediately assumed that it was the result of yet another corner that they cut.

    --
    I don't respond to AC's.
    1. Re:Uber cuts corners by HornWumpus · · Score: 4, Insightful

      Roads are dirty places, the better your sensor array, the more signals it will have to see and decide, hopefully correctly, to ignore.

      --
      John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
    2. Re:Uber cuts corners by quantaman · · Score: 4, Insightful

      Maybe that's a fair immediate assumption, but did you view the video and, if so, did it cause you to reevaluate your initial assessment?

      I started by assuming that the technology was still its infancy and hence crap[1], then I saw the video and realize that no only is the technology crap, but it's literally a person jumping out from a shadow at the last possible moment on a large thoroughfare nowhere near a crosswalk.

      [1] Not even a judgment on Uber TBQH, could have been Tesla or GM or Toyota. I've seen enough technologies come up to realize that the cutting edge is riddled with snakes. By the time it's thoroughly ironed out, it's also super boring.

      I did view the video.

      And like most people I came to the conclusion that Uber was either using ridiculously bad cameras or the video was altered. This impression was only compounded when 3rd party videos came out that showed the road in question was actually quite well lit.

      Either way Uber was still fully to blame for the collision, the tech was obviously not ready for testing on live roads, especially not with a single driver who was prone to being distracted. Authorizing that test is damn well close to negligent homicide.

      --
      I stole this Sig
    3. Re:Uber cuts corners by Green+Mountain+Bot · · Score: 3, Insightful

      ... it's literally a person jumping out from a shadow at the last possible moment on a large thoroughfare nowhere near a crosswalk.

      If by "jumping out from a shadow" you mean "slowly crossing the street", and by "at the last possible moment" you mean "and had nearly crossed all three lanes", ignoring that there's plenty of evidence that the released video did not even vaguely show the actual level of light in the location.

  8. Re:Oops! We left it in murder mode. by tattood · · Score: 5, Funny

    I think you mean kill mode

    --
    WTB [sig], PST!!!
  9. Re:Not si simple... by I4ko · · Score: 2

    If people follow at appropriate distance for the speed, no crash will happen.

  10. Re:Oops! We left it in murder mode. by Mister+Transistor · · Score: 2

    That will be her epitaph:: "False Positive".

    --
    -- You are in a maze of little, twisty passages, all different... --
  11. Re:Should you name your self driving car? by goose-incarnated · · Score: 3, Informative

    Should you name your self driving car? If so, are Christine and Kitt off limits?

    ITYM Karr (Knight Automated Roaming Robot) - that was the evil one. Kitt (Knight Industries Two Thousand) was the good one.

    (I feel very old now)

    --
    I'm a minority race. Save your vitriol for white people.
  12. I'm still more worried about the driver by MobyDisk · · Score: 2

    I still want to know why nobody seems to care that the driver wasn't looking at the road. The software bug is secondary.

  13. Re: Oops! We left it in murder mode. by Anonymous Coward · · Score: 5, Insightful

    Actually, there are a number of videos that people took from their cars on the following nights that show the area as well lit and it seems unlikely a moderately attentive human would have hit her. The video released from the car camera does not appear to be representative of what a human would have perceived. Notice that the "safety" driver obviously could see her easily when he glanced up from whatever she was distracted by. I am, of course, assuming that the street lighting hadn't suffered a massive failure that night and been restored the next few nights.

  14. Skin in the game by iMadeGhostzilla · · Score: 4, Informative

    Excellent point re execs. I read that in England sometime in the middle ages bridge engineers were required after the construction to sleep for two weeks under the bridge -- with their families.

  15. Re: Oops! We left it in murder mode. by houghi · · Score: 5, Insightful

    Even if it was as dark as they edited it, it would mean the car was driving too fast for the human in the car and the lights on the car wbhere not bright enough for the human in the car.
    You can not blame the human if you close his eyes and then ask him what he sees and he is wrong.

    --
    Don't fight for your country, if your country does not fight for you.