Slashdot Mirror

← Back to Stories (view on slashdot.org)

Experts Say Video of Uber's Self-Driving Car Killing a Pedestrian Suggests Its Technology May Have Failed (4brad.com)

Posted by BeauHD on from the system-failure dept.

Ever since the Tempe police released a video of Uber's self-driving car hitting and killing a pedestrian, experts have been racing to analyze the footage and determine what exactly went wrong. (If you haven't watched the video, you can do so here. Warning: it's disturbing, though the actual impact is removed.) In a blog post, software architect and entrepreneur Brad Templeton highlights some of the big issues with the video:
1. On this empty road, the LIDAR is very capable of detecting her. If it was operating, there is no way that it did not detect her 3 to 4 seconds before the impact, if not earlier. She would have come into range just over 5 seconds before impact.
2.On the dash-cam style video, we only see her 1.5 seconds before impact. However, the human eye and quality cameras have a much better dynamic range than this video, and should have also been able to see her even before 5 seconds. From just the dash-cam video, no human could brake in time with just 1.5 seconds warning. The best humans react in just under a second, many take 1.5 to 2.5 seconds.
3. The human safety driver did not see her because she was not looking at the road. She seems to spend most of the time before the accident looking down to her right, in a style that suggests looking at a phone.
4.While a basic radar which filters out objects which are not moving towards the car would not necessarily see her, a more advanced radar also should have detected her and her bicycle (though triggered no braking) as soon as she entered the lane to the left, probably 4 seconds before impact at least. Braking could trigger 2 seconds before, in theory enough time.)

To be clear, while the car had the right-of-way and the victim was clearly unwise to cross there, especially without checking regularly in the direction of traffic, this is a situation where any properly operating robocar following "good practices," let alone "best practices," should have avoided the accident regardless of pedestrian error. That would not be true if the pedestrian were crossing the other way, moving immediately into the right lane from the right sidewalk. In that case no technique could have avoided the event. The overall consensus among experts is that one or several pieces of the driverless system may have failed, from the LIDAR system to the logic system that's supposed to identify road objects, to the communications channels that are supposed to apply the brakes, or the car's automatic braking system itself. According to Los Angeles Times, "Driverless car experts from law and academia called on Uber to release technical details of the accident so objective researchers can help figure out what went wrong and relay their findings to other driverless system makers and to the public."

19 of 325 comments (clear)

  1. Doesn't matter by Snotnose · · Score: 5, Insightful

    Forget the Lidar, or lack of (they were testing cameras?). Forget the dude (heh, the first 12 hours thought he was a she. That's gotta hurt).

    Had I been driving that car, full alert, I would have killed that chick. I'd have felt bad, even knowing it was her fault. But the fact is, this dumbass walked in front of a fast moving car, at night, when she had no illumination, and the car had headlights. Her best hope of survival was a 100% functioning self driving car, anything less and she's dead.

    1. Re:Doesn't matter by Anonymous Coward · · Score: 5, Insightful

      Your missing the point. The point of the 'dynamic range' bits of the summary are to say "just because the video didn't show enough light doesn't mean there actually wasn't enough light".

      When driving a car with normal headlights, can you genuinely not see what's the next lane over 150 feet ahead? If not, you need new headlights. Or new eyes.

    2. Re:Doesn't matter by haruchai · · Score: 5, Informative

      Forget the Lidar, or lack of (they were testing cameras?). Forget the dude (heh, the first 12 hours thought he was a she. That's gotta hurt).

      Had I been driving that car, full alert, I would have killed that chick. I'd have felt bad, even knowing it was her fault. But the fact is, this dumbass walked in front of a fast moving car, at night, when she had no illumination, and the car had headlights. Her best hope of survival was a 100% functioning self driving car, anything less and she's dead.

      The released video is very misleading. While the deceased made a stupid decision to cross at that point, it was quite well lit.
      No one with even average vision or reflexes would have hit her.

      https://www.youtube.com/watch?...

      --
      Pain is merely failure leaving the body
    3. Re:Doesn't matter by bigwheel · · Score: 4, Insightful

      The fact that she had a bicycle leads me to believe that she was not blind. And based on her walking speed, I doubt that she was being chased. However, IF it was an electric car, she might have misjudged the vehicle's distance and speed.

      What I saw on the video was an inattentive "driver", looking down for a full 5 seconds just before impact, and not hitting the brakes or making any attempt to avoid the pedestrian. I suspect that the "driver" was lulled into believing that the car was better than it really was, and therefore, behaving like a passenger rather than a driver.

      Very sad.

    4. Re:Doesn't matter by PixetaledPikachu · · Score: 4, Insightful

      Experimental technology doesn't work as promised. Total shocker there.

      Yes, experimental technology obviously fails every time, but that's not the point. The point is Uber is deploying fail-prone experimental technology in public and it took someone's life.

    5. Re:Doesn't matter by harperska · · Score: 3, Insightful

      Exactly. The car was a prototype, which by its very nature could be expected to fail in unexpected ways. Thus, you put a human in the car as a backup whose sole job is to remain attentive to react to any sudden failure of the car's self driving capabilities. Clearly, the emergency backup driver was treating the car as a complete functional autonomous vehicle rather than a prototype, so this is kind of on her.

    6. Re:Doesn't matter by Anonymous Coward · · Score: 5, Insightful

      Exactly. The car was a prototype, which by its very nature could be expected to fail in unexpected ways. Thus, you put a human in the car as a backup whose sole job is to remain attentive to react to any sudden failure of the car's self driving capabilities. Clearly, the emergency backup driver was treating the car as a complete functional autonomous vehicle rather than a prototype, so this is kind of on her.

      No, no... this has been said so many times, that when people keep ignoring it, it's no longer funny.

      Not even if you paid people a huge salary to remain fully attentive, would you get an attentive driver. The brain is remarkably apt at learning not to care about the car when it drives itself. One learns that "something else will do this, I don't need to" most of the time, so that robs you of countless *seconds* of reaction time in an event like this. By the time you realize you really do need to act, the accident already happened.

      This is what everybody has been saying all along. No, human backups are unreliable and unworkable. You have to perfect the technology first in controlled conditions, because in the field the risk *is* high. This is NOT a task for agile methodologies.

    7. Re:Doesn't matter by amicusNYCL · · Score: 5, Insightful

      However, IF it was an electric car, she might have misjudged the vehicle's distance and speed.

      That doesn't matter. She was crossing a road with 4 lanes coming at her, and when she stepped off the curb she should have seen the car on the Mill bridge. That bridge is covered in lights, too, you can see the bridge lights on the internal camera behind her. There's no reason the woman crossing the completely dark section of road wouldn't have been able to look towards Tempe and see the car coming. And who crosses 4 lines, in the dark, with a car approaching, without looking at the car to see if they need to pick up the pace? She never looked at the car. The Uber car obviously experienced some sort of catastrophic failure, but in the same way that this is a case study for computer science or engineering students, it's also a case study on how not to cross a street. The woman easily could have avoided being hit. I expect her to avoid getting hit the same way I expect the autonomous car to not hit her.

      --
      "Our two-party system is like a bowl of shit looking at itself in a mirror." - Lewis Black
    8. Re: Doesn't matter by FF-Loucks · · Score: 3, Insightful

      Who cares what the victim was classified as. Seems like the car should know when it is about to hit something and preserve itself. What if it had been a lawn cart rolling across the road? Wouldn't you want to not hit something that might cause damage to your car? Reminds me of the story of the guy that saw a refrigerator box in the road and decided to hit it for fun - too bad it still contained the refrigerator. These cars need to recognize that they shouldn't hit anything in the road no matter if it is a person, pothole, or box

    9. Re:Doesn't matter by amicusNYCL · · Score: 3, Informative

      That video definitely shows better lighting. For reference, the impact happens around the 0:33 mark of the above video, there's a sign on the right side of the road for reference.

      But, that video is a little bit over-saturated. The light strings on the bridge look like a solid light, they aren't that bright. You can clearly make out the individual lights when you're actually there. Like usual, the reality is somewhere between these videos. I've been to the theater on that corner many times, and I remember it as being a poorly-lit street.

      --
      "Our two-party system is like a bowl of shit looking at itself in a mirror." - Lewis Black
    10. Re:Doesn't matter by michelcolman · · Score: 3

      Maybe they should have called the system "autopilot" instead of "self-driving"...

    11. Re:Doesn't matter by thegarbz · · Score: 3, Interesting

      The point is Uber is deploying fail-prone experimental technology in public and it took someone's life.

      What you just said describes a huge majority of industries. Be thankful you didn't apply that thinking to oil refining or you wouldn't have cars at all.

  2. An unfortunate coincidence of failures by NewtonsLaw · · Score: 4, Interesting

    There were multiple failures all around which caused this death. If any one of those failures had not happened then the pedestrian would likely still be alive today.

    I've summed it up here in a column which was written almost 24 hours ago so it's nice to see that others have come to similar conclusions.

    1. Re: An unfortunate coincidence of failures by c6gunner · · Score: 3, Funny

      To prove that we're not heartless we have posthumously given her a Darwin Award. That ought to set your mind at ease.

  3. What you don't see - when did movement start by SuperKendall · · Score: 5, Interesting

    I agree the LIDAR should have been able to see her before she entered the light.

    However, what we still do not know is - when did she start moving?

    If she was just standing in the left lane waiting to cross, the LIDAR may have seen her and just thought "well that lane is blocked, stick to this one". With the bike she might have looked like a barricade of some kind.

    It could still be she started moving around the time we see her in the video, which means she essentially jumped in front of the car...

    There could be a reason for her to do that - what if the car saw her, and slightly slowed out of caution? We know the car was going well under the limit when it hit, that could be a sign the car slowed down a bit prior.

    The human, seeing a car slow light that might have assumed it saw her and was going to stop to let her cross. So it could easily be a case of mixed signals, with the cars cautious actions in the end being a bad thing, when driving an over-abundance of caution can often have bad consequences.

    I'm still not sure the human safety driver would have seen her though, even though humans do have better dynamic range than cameras there still are times when you really can't see outside the headlights, and the woman crossing was all in dark clothing.

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
  4. You can't in shadow, at night by SuperKendall · · Score: 5, Insightful

    When driving a car with normal headlights, can you genuinely not see what's the next lane over 150 feet ahead?

    Lots of cars fudge the headlights a bit to the right to keep from blinding oncoming cars. Combine that with her stepping out of a very dark shadow of the tree, as well as her dark clothing (jeans and black jacket) and it's very likely that because of the dynamic range of the scene (bright headlights making darker anything street lights shone on, then her being in the shadow from even the street lights) a driver would have been too blinded by available light to see the pedestrian.

    She didn't even have reflectors on the wheel of the bike, much less herself - even one might have saved her.

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
    1. Re:You can't in shadow, at night by sh00z · · Score: 3, Informative

      Aren't bicycle reflectors a legal requirement in the US?

      It's a legal (FTC) requirement that bicycles are sold with reflectors. There is no Federal obligation on the buyer to keep them. the State of Arizona or City of Tempe may have statutes, but there's no applicable US Code for riding a bicycle with or without reflectors.

  5. Re: Yeah, no by c6gunner · · Score: 3, Insightful

    The lesson for the rest of us is that Uber's self driving technology is not ready for prime time, for whatever reason(s).

    I dunno ... I mean even if we assume that a human driver could have avoided this particular accident, that doesn't mean the technology isn't still an improvement over human drivers in other cases. You'd need a lot more data to reach that conclusion. It could very well be that lives saved in other, more common types of preventable accidents massively outweigh the lives lost in these types of abnormal occurrences.

  6. I call bullshit by Anonymous Coward · · Score: 3, Informative

    I've got a forward facing car camera, and its very clear that eyes see things far bigger and brighter at the horizon than cameras. You would see her, even in that light.

    This is why the moon seems bigger and brighter when its near the horizon. When actually its dimmer due to atmosphere and the same size as normal.

    *But*, more than that.

    I've also seen the other video of this stretch of road filmed with a normal smartphone camera and its clear the camera in the car had terrible dynamic range. There is absolutely no way a person looking at the road would not have seen her.

    https://www.youtube.com/watch?v=CRW0q8i3u6E

    Level 1 self drive is where the car acts as a safety feature for the driver.
    Level 2 is where the driver acts as a safety feature for the car!

    Which is it!! Is the car safer or the driver?? You cannot have Level 2 because its simply dumping liability on the driver, when he's not party to the decisions the car is making and does not know what it will do, till after its done it.

    The car drives, the man observes, then deduces what decision its made from the changes to the car. He cannot be the safety backup for the car. It's not possible, its a legalese get out for self driving car makers.