Slashdot Mirror
Experts Say Video of Uber's Self-Driving Car Killing a Pedestrian Suggests Its Technology May Have Failed (4brad.com)
Ever since the Tempe police released a video of Uber's self-driving car hitting and killing a pedestrian, experts have been racing to analyze the footage and determine what exactly went wrong. (If you haven't watched the video, you can do so here. Warning: it's disturbing, though the actual impact is removed.) In a blog post, software architect and entrepreneur Brad Templeton highlights some of the big issues with the video:
1. On this empty road, the LIDAR is very capable of detecting her. If it was operating, there is no way that it did not detect her 3 to 4 seconds before the impact, if not earlier. She would have come into range just over 5 seconds before impact.
2.On the dash-cam style video, we only see her 1.5 seconds before impact. However, the human eye and quality cameras have a much better dynamic range than this video, and should have also been able to see her even before 5 seconds. From just the dash-cam video, no human could brake in time with just 1.5 seconds warning. The best humans react in just under a second, many take 1.5 to 2.5 seconds.
3. The human safety driver did not see her because she was not looking at the road. She seems to spend most of the time before the accident looking down to her right, in a style that suggests looking at a phone.
4.While a basic radar which filters out objects which are not moving towards the car would not necessarily see her, a more advanced radar also should have detected her and her bicycle (though triggered no braking) as soon as she entered the lane to the left, probably 4 seconds before impact at least. Braking could trigger 2 seconds before, in theory enough time.)
To be clear, while the car had the right-of-way and the victim was clearly unwise to cross there, especially without checking regularly in the direction of traffic, this is a situation where any properly operating robocar following "good practices," let alone "best practices," should have avoided the accident regardless of pedestrian error. That would not be true if the pedestrian were crossing the other way, moving immediately into the right lane from the right sidewalk. In that case no technique could have avoided the event. The overall consensus among experts is that one or several pieces of the driverless system may have failed, from the LIDAR system to the logic system that's supposed to identify road objects, to the communications channels that are supposed to apply the brakes, or the car's automatic braking system itself. According to Los Angeles Times, "Driverless car experts from law and academia called on Uber to release technical details of the accident so objective researchers can help figure out what went wrong and relay their findings to other driverless system makers and to the public."
1. On this empty road, the LIDAR is very capable of detecting her. If it was operating, there is no way that it did not detect her 3 to 4 seconds before the impact, if not earlier. She would have come into range just over 5 seconds before impact.
2.On the dash-cam style video, we only see her 1.5 seconds before impact. However, the human eye and quality cameras have a much better dynamic range than this video, and should have also been able to see her even before 5 seconds. From just the dash-cam video, no human could brake in time with just 1.5 seconds warning. The best humans react in just under a second, many take 1.5 to 2.5 seconds.
3. The human safety driver did not see her because she was not looking at the road. She seems to spend most of the time before the accident looking down to her right, in a style that suggests looking at a phone.
4.While a basic radar which filters out objects which are not moving towards the car would not necessarily see her, a more advanced radar also should have detected her and her bicycle (though triggered no braking) as soon as she entered the lane to the left, probably 4 seconds before impact at least. Braking could trigger 2 seconds before, in theory enough time.)
To be clear, while the car had the right-of-way and the victim was clearly unwise to cross there, especially without checking regularly in the direction of traffic, this is a situation where any properly operating robocar following "good practices," let alone "best practices," should have avoided the accident regardless of pedestrian error. That would not be true if the pedestrian were crossing the other way, moving immediately into the right lane from the right sidewalk. In that case no technique could have avoided the event. The overall consensus among experts is that one or several pieces of the driverless system may have failed, from the LIDAR system to the logic system that's supposed to identify road objects, to the communications channels that are supposed to apply the brakes, or the car's automatic braking system itself. According to Los Angeles Times, "Driverless car experts from law and academia called on Uber to release technical details of the accident so objective researchers can help figure out what went wrong and relay their findings to other driverless system makers and to the public."
Forget the Lidar, or lack of (they were testing cameras?). Forget the dude (heh, the first 12 hours thought he was a she. That's gotta hurt).
Had I been driving that car, full alert, I would have killed that chick. I'd have felt bad, even knowing it was her fault. But the fact is, this dumbass walked in front of a fast moving car, at night, when she had no illumination, and the car had headlights. Her best hope of survival was a 100% functioning self driving car, anything less and she's dead.
Duh!! yes! along with the convept of a safety driver.
;)
Just my 2 cents
Uber must be using some bottom-of-the-barrel piece of shit from Wally World. Here are pictures of the scene. It's plenty well lit, there's no excuse for not having seen the biker well ahead of time.
Why is this still a thing people are talking about everywhere? The car had the right of way. Doesn't that mean anything to pedestrians? Get the fuck out of the way if you don't want to be killed.
"She seems to spend most of the time before the accident looking down to her right, in a style that suggests looking at a phone"
I knew it ! Another proof that looking at your phone while driving is so dangerous that even a self-driving car wont save your ass !
There were multiple failures all around which caused this death. If any one of those failures had not happened then the pedestrian would likely still be alive today.
I've summed it up here in a column which was written almost 24 hours ago so it's nice to see that others have come to similar conclusions.
China is not doing a great job on these. Sad.
I prefer the "u" in honour as it seems to be missing these days.
Because this experiment will lead to millions of cars you fucking idiot
Continuing to develop and test self-driving cars will result in more pedestrian deaths. That much of any driving will result in pedestrian deaths. It is going to be interesting to see how this develops over time.
"When the going gets weird, the weird turn pro" -- HST
...or due to changes made in wake of the Google-Uber lawsuit settlement. But the questions probably need to be asked, in light of statements made by both companies: 1. A note on our lawsuit against Otto and Uber: "Recently, we uncovered evidence that Otto and Uber have taken and are using key parts of Waymo's self-driving technology." 2. Uber and Waymo Reach Settlement: "We are taking steps with Waymo to ensure our Lidar and software represents just our good work."
I agree the LIDAR should have been able to see her before she entered the light.
However, what we still do not know is - when did she start moving?
If she was just standing in the left lane waiting to cross, the LIDAR may have seen her and just thought "well that lane is blocked, stick to this one". With the bike she might have looked like a barricade of some kind.
It could still be she started moving around the time we see her in the video, which means she essentially jumped in front of the car...
There could be a reason for her to do that - what if the car saw her, and slightly slowed out of caution? We know the car was going well under the limit when it hit, that could be a sign the car slowed down a bit prior.
The human, seeing a car slow light that might have assumed it saw her and was going to stop to let her cross. So it could easily be a case of mixed signals, with the cars cautious actions in the end being a bad thing, when driving an over-abundance of caution can often have bad consequences.
I'm still not sure the human safety driver would have seen her though, even though humans do have better dynamic range than cameras there still are times when you really can't see outside the headlights, and the woman crossing was all in dark clothing.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
You know how UBER has tried all the dirty tricks in the book, be it ignoring local laws, making fake bookings on competitors etc etc :)
How about if they though, hey, all those LIDAR and radar systems are expensive, how about if we try to do it with a cheap offshore dev team and just a chinese dashcam? That would save a lot of money, no?
And, the released video is what the system was actually running with...
Well there you go. Clearly these cars should be kept on the road so Uber has every opportunity to make their technology better.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
The lesson for the dead person would have been to look both ways. The lesson for the rest of us is that Uber's self driving technology is not ready for prime time, for whatever reason(s).
This is in no way meant to fault that “safety driver”, but she should have been filtered out by the Uber personnel selection process. What you MUST have with an experimental vehicle such as this is someone with a Test Pilot temperament: a tiger, who shows quick, fearless, and decisive actions in a stress situation. Look at the video of the driver: she reacts in panic and throws up her hands when she sees the pedestrian/cyclist. This is testable in a simulator; what you want to have instead is someone who shouts “NO!”, decisively stomps on the brakes, and grabs the wheel to swerve, even at the risk of their own life. These autonomous vehicles are not play-toys, that can be watched over but just anyone. For Uber to have apparently had such lackadaisical hiring practices is IMHO criminally negligent. BTW, this is NOT a gender issue, as I’ve seen some female test pilots who had as much of a Tiger in them as any man.
Where pedestrians do this stunt *all the fucking time at crosswalks* you best have your foot on the brake waiting for the douche who blissfully steps out in front of you when passing.
In the middle of the highway when there is slush and heavy snow. Does it just stop in the middle of the highway and you get crushed by the Semi truck behind you? With no steering wheel you don't have a chance. In my 2016 Civic, many times when I was driving in the bad conditions, the car warned me that the safety features were turned off because sensors temporarily are down. But I can still drive the car myself. So I still don't get how the self driving cars will handle this without getting destroyed on the highway
"To be clear, while the car had the right-of-way and the victim was clearly unwise to cross there..."
Back in the day when I learned to drive the rule was that pedestrians ALWAYS have the right of way -- even if they're crossing in the middle of the road.
So if it was daylight where the SNR of the LIDAR is worse with ambient light hitting the sensor and the car hit her where would you place blame? Because that is exactly what would have happened. The LIDAR system would still have missed her. The system failed period full stop. It is as bad as when Intel released the processor that did not always add right. I know the tech world is spending billions to make this work and it seems to be every techies wet dream for it to work, but it does not work yet. And deploying it in the wild is murder. It is not ready.
Personally, I'm expecting to find out that Uber was secretly testing their car with the LIDAR intentionally off, to compare how well the system worked with and without LIDAR (i.e. to see if they could save a couple of bucks by not installing LIDAR).
Community: Uber please release possibly incriminating technical details of the accident to help us make our systems safer ; Uber:... Crickets...
dead people don't learn lessons
The main failure will be holding uber accountable.
When driving a car with normal headlights, can you genuinely not see what's the next lane over 150 feet ahead?
Lots of cars fudge the headlights a bit to the right to keep from blinding oncoming cars. Combine that with her stepping out of a very dark shadow of the tree, as well as her dark clothing (jeans and black jacket) and it's very likely that because of the dynamic range of the scene (bright headlights making darker anything street lights shone on, then her being in the shadow from even the street lights) a driver would have been too blinded by available light to see the pedestrian.
She didn't even have reflectors on the wheel of the bike, much less herself - even one might have saved her.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Man. Convicted felon.
http://abc7.com/driver-in-uber...
The street was well lit, the camera should have seen her.
"First they came for the slanderers and i said nothing."
The car had a single occupant who was not focused on the road. In a test vehicle, regardless of how autonomous the vehicle is if one person is required to monitor computer systems whilst the car is travelling, they need a second occupant to monitor the road.
The Lidar system should work with no light at all, its an infrared laser system, it emits its own light! In fact if anything it would probably work better in complete darkness!
This was obviously a major technology failure, however it needn't have resulted in someones death.
The greatest failure here is not in the technology but in Ubers testing procedures.
That video is somewhat brighter than what a real driver would see.
But even so, look at your video at 33 seconds - that's the point where the much wider camera in the uber catches sight of her shoes in his lane (at about 7 seconds in this video). The pedestrian is in shadow and if she wasn't moving before it would be very easy not to see her until she started moving. If you go back a bit further in the video she would be standing right between some bright background lights, making it even more difficult to see her if not moving.
Now look at the video of the safety driver. It's not like she was never looking up, she was looking back up every few seconds or so. If the woman were really as visible as you are thinking, the safety driver would have seen her way ahead of time. The last time in the video of the driver she was looking ahead was about 17 seconds into the driver view video, which equates to about 30 seconds in your video (look at the poles passing by the window), if the whole scene were really so bright shouldn't the safety driver have seen the pedestrian at that point? Again, if not moving the pedestrian (in all dark clothing and no reflectors) would have been far harder to see. Pedestrians are killed in this kind of situation by drivers who are watching the road continuously... or as continuously as a human can.
What would be really informative to get, is the LIDAR data from this incident. That would really say a lot about what the car itself did or did not see, then we could go on to speculate how it processed what it "saw".
"There is more worth loving than we have strength to love." - Brian Jay Stanley
The lesson for the rest of us is that Uber's self driving technology is not ready for prime time, for whatever reason(s).
I dunno ... I mean even if we assume that a human driver could have avoided this particular accident, that doesn't mean the technology isn't still an improvement over human drivers in other cases. You'd need a lot more data to reach that conclusion. It could very well be that lives saved in other, more common types of preventable accidents massively outweigh the lives lost in these types of abnormal occurrences.
Why exactly is it not ready for prime time? Do deaths like this not happen with human drivers? Do we all stop driving because 1 person got into an accident?
A good driver would not be going that fast. Yes, maybe the limit was X... but clearly there wasnâ(TM)t enough time to stop once an object entered the detection area so the car should have slowed down. Period. No good driver would go 55 in a snow blizzard if you canâ(TM)t see 5 feet in front of the car.
Thatâ(TM)s why rear end collisions are always the fault of the car hitting from behind. The back car was too close at that speed to stop in time... or failed to stop. Doesnâ(TM)t matter why the front car slammed on the brakes. The back car should be far enough back at a slow enough speed to stop after detecting the stop.
The idea of "safety driver" might be inherently flawed. When you drive a normal car, your attention is fixed on the traffic and surroundings. When the car does the driving, how are you supposed to sustain your attention for more than a few minutes? Boredom and attention lapses may be inevitable. If the safety driver is told ahead of time that attention lapses have severe penalties, they might struggle to remain alert for a while longer, but it might be a fact of human nature, that avoiding distractions is an uphill battle.
update quickly = new car each 2-4 years no regulation saying free updates for at least 9-12 years
... another shitstorm for uber to face. Eventually a company that collects shameful cases, isn't profitable and very much looks like a sinkhole for investment, will make investors think they may have to cut their losses and move on.
Look at the video again. As soon as the driver looked up, she saw the pedestrian.
When she looked up and saw the pedestrian, she was well past the bridge, and it was way too late to do anything. She basically looked up as the car hit the person.
I was talking about the time she looked up BEFORE that time (in the driver video she looked up from her phone several times), which you can clearly tell by the poles passing by outside her window is when she is passing under the bridge, and match that with the other brighter video when it passes under the bridge and the poles are too the side (in the driver video remember you are looking at poles that are well behind the car as you are looking backwards).
The timeline is thus: She looks up as she is going under the bridge, looks down again, then a few seconds later she looks up basically as the car is hitting the woman.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Not that this means anything, but having lived in the greater Phoenix area for many years, I can say that the pedestrian's expectations may have been different than you might expect in your area.
My experience in the area was that if a person were to step off a curb to cross a road, or intersection, legally or illegally, traffic just flat stopped until the pedestrian was clear, even on a four lane divided road. Not what you would expect anywhere else. It was odd, yet welcome, behavior on the part of drivers in the area. Probably due to the age of the average resident; snowbirds, and all.
I did seem to recall less and less of that behavior as time went on, but it still happened often by the time I left the area in 2009.
Well you've been in a dark corner of the internet, haven't you?
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
I've got a forward facing car camera, and its very clear that eyes see things far bigger and brighter at the horizon than cameras. You would see her, even in that light.
This is why the moon seems bigger and brighter when its near the horizon. When actually its dimmer due to atmosphere and the same size as normal.
*But*, more than that.
I've also seen the other video of this stretch of road filmed with a normal smartphone camera and its clear the camera in the car had terrible dynamic range. There is absolutely no way a person looking at the road would not have seen her.
https://www.youtube.com/watch?v=CRW0q8i3u6E
Level 1 self drive is where the car acts as a safety feature for the driver.
Level 2 is where the driver acts as a safety feature for the car!
Which is it!! Is the car safer or the driver?? You cannot have Level 2 because its simply dumping liability on the driver, when he's not party to the decisions the car is making and does not know what it will do, till after its done it.
The car drives, the man observes, then deduces what decision its made from the changes to the car. He cannot be the safety backup for the car. It's not possible, its a legalese get out for self driving car makers.
The first time I saw the video, I went, "Man, somebody's going to win a multimillion dollar judgement against Uber!" True be know, due to my night blindness, I probably would have hit the pedestrian too under similar circumstances. I constantly see pedestrians or bicyclists in dark clothing on the side of the road only after it would be too late to avoid them if they moved in front of my car.
I've abandoned my search for truth; now I'm just looking for some useful delusions.
In perfect dark condition, no new moon possibly cloud coverage no good street lamp, I can barely see 30-40 meter away object which have no reflective properties. Try it. With a new moon or better reflective object I can naturally as you say see muuuuch further away. The question is : what are the condition here. People says "human eye see further away" yes and no. Was there other source of lights ? Because if yes it ruins your night vision, and if there is only shadow and nor reflective surface, forget your 150 feet. You'll be lucky with 3/4 of that.
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
Fuck off.
If she had crossed at the crosswalk this self-driving car would still have run her down.
Didn't you notice that the car didn't even brake? Or attempt to slow down?
The overall consensus among experts is that one or several pieces of the driverless system may have failed, from the LIDAR system to the logic system that's supposed to identify road objects, to the communications channels that are supposed to apply the brakes, or the car's automatic braking system itself.
But the most important thing that failed was the human driver taking over control in an emergency when all systems fail to work properly, destroying a long held "get out of jail free card" creators of self driving cars use whenever anyone challenges them on the tech.
I only please one person per day. Today is not your day. Tomorrow isn't looking good either. - Scott Adams
Seriously, having quality infrared sensors would be a huge boon to detection.
But, ITAR regulates IR sensors due to their use for nightvision systems (thus dual use technology), which is why you usually have sucky consumer nightvision cameras. Anything better requires attestation that devices will only be sold in the US to US entities, sometimes also only sold to US citizens.
If we didn't have ITAR regulating nightvision, every autonomous platform would be augmenting their sensor arrays with IR sensors at high resolution. Assuming a human doesn't pop out a freezer truck, or from behind an obstacle, they will stick out like a sore thumb on optical systems.
That said, every sensor system has weaknesses. You need to do proper sensor data fusion to get the complete picture. Solid state LIDAR, high rez millimeter wave radar, high rez visual optical, high rez IR optical, high rez ultrasound are some of the major sensor families you should be using.
If you look at the dash cam more closely, you can track the earlier movements of the pedestrian.
First she appears to possible occlude the head lights of a turning car up ahead.
Secondly, she occludes small bright sign in the back ground.
The timing all seems to fit with her later movement.
Clearly this was a "Trolley Car Problem" situation; and we now know that self-driving cars - will choose to hit a pedestrian to minimize risk to passenger. Amateur philosophers have been debating this problem in the context of SD cars for some time now. Now, at last, we can put an end to any further discussion of Trolley Cars by telling them: "Problem Solved".
I'm no expert, but I've seen a lot of news about self-driving cars over the last ten years or so.
At first it was DARPA, then Google, and they seemed to be testing for years . . .
Then Tesla, and now Uber ?
I guess Uber suddenly realized that self-driving cars might be a threat to their business in the long term.
So they are notoriously getting caught with stolen code and poaching engineers.
Seems like they are playing catch-up.
That usually means skipping testing.
Maybe the states should require that self-driving equipment and algorithms get a serious "drivers test" ?
Even a semi-automated software test suite and an independent engineering review might be good enough.
Eventually, we WILL need this.
The technology is now easy enough for teenagers to start doing it on their own cars at home.
I don't want MY life to depend on THEIR coding skills.
Absolutely agree here. I drove along city streets last night. I could easily see people wearing dark clothes, beside the road, at a distance of 75 feet. The camera doesn't give the full story.
A human safety driver will not work without some means of keeping the driver engaged. These cars had driven millions of miles between them with no accidents until this one. Obviously the passenger's attention will wander. The human was there to mollify critics, not to act as actual safety driver.
The pedestrian doesn't seem to be in much of a rush. This is something I simply don't understand. I'd have thought she'd be moving a lot faster.
Honestly, the only thing I see anyone talking about is braking. There are other defensive moves, like swerving. I've used that one myself many times to hit pigeons, they are hard to hit too. And don't go whining about killing them. They're an invasive species that is killing the local ecosystem and the corpse is picked up within minutes by a homeless person for a meal. win-win.
Look both ways before crossing the street..
Twice or thrice at night..
If it's a driverless car that requires driver to be attentive of the road and what's goign on - it's never gonna work right.
Either driver actively drives the car, or it doesn't... there's no in between. We're humans, not robots...
This person was looking at his facebook all the time, next one will nod off.. etc.
This is not only a failure in Uber's testing procedure. This is the failure of the whole concept of autonomous "driver assist only" systems. There are a lot of people out there who would do the exact same thing in e.g. their Tesla - let their attention slip for a while and let the system do its job, because humans are lazy, plus we are bad at drawing conclusions - nothing bad happened the first two weeks we drove the car, so that means nothing bad is going to happen after that either - right?
This is also the reason that with only a "regular" cruise control in my own traditional car, I always keep the foot loosely on the gas pedal while cruise control is engaged. Why? Because if something happens and I need to react fast to apply the brakes, then I don't have time to figure out where I have my foot. I need to let brain automation of moving the foot from the gas pedal to the brake - something for which I have muscle memory - happen very, very quickly.
Now consider "reacting" in a half-autonomous car. You are half using your cell phone or whatever. A situation arises. You need to (a) shift focus from your distraction, (b) fully process what is the situation and what do you need to do to react, (c) figure out how to do it. And way before you get to (c), the accident has already happened.
The whole concept of "driver assist" - for drivers who are not consciously aware of the dangers mentioned above, and actively engaging their focus, body posture and pre-frontal lobe in order to override said danger - which let's be honest is actually most people out there - is quite dangerous, for any dangerous situation the car cannot handle by itself.
So this is not really about Uber. It is about human nature, and why this technology can never fully take off unless car AI gets near perfect (which I don't see happening, which I will write about in a separate comment), or drivers get as disciplined as North Koreans.
Pull up google maps.
https://www.google.com/maps/@3...
Observe the sign that says not to cross there (one on the other side of the median too btw).
Observe the road widens from 2 to 4 lanes right before where the accident occurred.
Observe that trees and bushes on the edge of the median to the left would have blocked Lidar until the car was within 50' of where the accident occurred.
Confirm the scene against the video with the sign that reads
"
Begin right turn lane
-
Yield to Bikes.
"
Understand that it refers to bikes in the bike lane on the right.
Pull up a top down view and get a ruler and note that the street lights are about 120' apart here. They are much closer in my area ( 50-100' apart).
(Also, if you look up the light pollution map, the area to the right of the road is very dark).
Go 600' back down Mills road and observe the 45mph speed limit on this road.
It's a terrible accident. The road widening like that with bushes and small trees blocking lidar, the pedestrian walking across the road without even looking for oncoming traffic (because it was late at night on a sunday in what looks like a kinda empty area so there probably isn't much traffic).
Wait for people to really dig into this. Don't make a snap judgement.
I'm sorry someone died, but there has been a lot of bad information and a lot of people rushing to judgement before they have the facts.
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
SkyNet doesn't like jaywalkers.
IMNSHO, "autonomous driving" is a huge fad - the idea is good, but it is _way_ ahead of its time.
The problem is all those situations in traffic that require understanding a situation, processing what is going on, predicting potential risks, and responding appropriately. One example - you are driving and up ahead on the sidewalk there are two children. Do you slow down? What are the children doing? Do they seem to be behaving erratic, or are they engaged in playful behaviour like pushing each other around a bit, and is there a risk they could get into the road without paying attention? How old do they seem to be? Is there an obstacle in front of them on the sidewalk which is going to force them to move onto the road about the time when you pass them? Is there a puddle of water in the road which will create a huge splash unless you drive around it or slow down a lot? Are they on their way around a bend where you cannot see what is on the other side, e.g. whether the sidewalk ends or some bicycle might approach and force them out in the road?
There is a _huge_ number of such little judgement calls we make every single time we drive. And AI is nowhere _near_ of being able to process anything similar.
Forget the whole "this could be fixed with better LIDAR" etc. The reason why this whole auto-drive concept is a bust the way people are being way too excited talking about it, is not those things we could fix with current technology - it is the type of examples I am describing that we do not have a solution for.
The fix is either to force a "driver assist only" type of regime, which obviously doesn't work because of human nature, as we saw an example of with the Uber incident. Alternatively you compensate by making the auto-pilot system super risk averse and slow down for just about anything, which would be ok - except that really brings down efficiency, I am not willing to spend twice as long getting to work because of some drive that is insanely cautious. Or third, move the vehicles off regular roads into protected areas where they can drive without the dangers and unpredictable conditions of regular roads - which is an option, but requires building all the infrastructure to support it - hugely costly, and not something I see happening.
TL;DR; IMNSHO automonous cars will only happen when AI reaches human levels
And at which point we *tinfoil hat on* should think twice before getting in a car that may decide it likes us better when we are dead.*tinfoil hat off*.
When the car is deemed ready and put into production ans so on, yes absolutely.
But this was a *test driving* situation, during which unexpected and possibly dangerous situations may occur. It is irresponsible of Uber to not have two people in the car at all times, one to monitor the road and take evasive action if necessary, and one to monitor the systems. One person cannot do both at the same time.
Eat the rich.
It's disturbing how many comments in the media have tried blaming the pedestrian for crossing at the wrong spot. There seems to a be a bit of an astroturfing campaign â" Reddit is full of such comments.
In much of the world (where auto makers have not managed to buy a change the law) this is not illegal and in fact standard practice, such as in the UK for example. Any self driving technology must clearly be able to deal with this, not least for handling unpredictable young children in residential areas.
You have been run over by a robot, clean your mess.
0.5 seconds breaking time would dropped the speed to about 30mph
"for making false statements when obtaining unemployment benefits and attempted armed robbery" Not really relevant to driving ability
I don't know how anyone can say that it's clear the car could have seen her 4 seconds before impact and know that she was going to dash across the street. It appears to me that the woman waited until the last moment and then tried to run across the street. The video does not have enough information.
IMELPHO*, AI already has reached and even exceeded human levels in some areas. It remains laughably far away in others. But I'll echo others' observation that it does not have to be perfect in order to be a safer alternative to human drivers. It just has to be better than they are. And that's a very low bar. I'm not sure we're there yet, but I think we're close, closer than I'd have thought possible 10 years ago, and I think we'll get there. Sadly, but just as in other human endeavors, there will be mistakes and accidents and innocent people will suffer and die in the short term. But, hopefully, fewer than if we did not even attempt to progress toward something we know in the end will be, if not perfectly safe, at least much safer than what we have now.
* In my even less plausibly humble opinion.
Nonaggression works!
Likely pedestrian deaths will spike then decline to levels lower than they are now. Probably because people will start remembering that the two ton death missiles are controlled by computers now and will not give one single fuck when they make a red puddle out of you.
I for one welcome our stupid pedestrian murdering overlords.
And that's a good thing. Save a man from walking in front of traffic and he will live another day. Run over a man and you teach everyone else to not walk out in traffic and save them all.
But... but.... They can see in the dark!... ok never mind that one. But they have amazing reaction time! Ok never mind that one too. Ok I guess you're right.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
This fatal, a complete catastrophic robotics design fail, is not an accident. Accidents happen by circumstance, calamity or conscious error. Here what you see is a First Order deliberate design point failure. Its immoral, unpredictable and unsafe at any speed.
The death last year of a factory worker cleaning inside an assembly robo cage was no accident neither. That one also bothered. The robot was not operating, supposedly non-operational during off-hours though obviously not powered down. The cleaning lady was attacked by its arms, thrashed and shredded to death. Not even the motion of the machine to effect such horror mimics its design function, programmed range of duty nor safety protocol. That was the first most disturbing, unpredictable and unsafe robo-kill.
Now we have a 2nd RoboKill. An unexplained kill by robot. A phenomenon.
A person with 1+ second reaction time should have his license immediately terminated.
Thanks for the expert opinion to clarify the situation. A lot of us thought killing a pedestrian was normal.
One person cannot be expected to keep their attention on the road either, so the whole premise fails.
They absolutely can, if that is their job as a professional test driver, and they have someone else monitoring the systems.
Eat the rich.
This is one thing that I don't understand at all. Why in the world are self-driving cars allowed on the road without *two* human backups. One should be monitoring the systems to see if they're working. The other should be the safety driver. What should have happened here is that the engineer (for lack of better word) should have been saying to the driver. "Hey, the Lidar doesn't seem to be detecting that pedestrian as a stationary object" and writing a bug report. The driver would have been at heightened alert.
While I pretty much agree with the OP's points; at least that they are possible issues, I want to point out that the deconstruction and analysis of this is going to take months, just like it does for an airplane crash or a ship collision (although except for the victim, the technology is intact and so readily available for analysis (not that I'd view Uber as being a trustworthy steward, should they have gotten their hands on it). The enormously premature statements by the police dump a ton of doubt on whether we will ever know what really happened. I wonder what his motives were for such an obviously biased and false (in the sense that he couldn't possibly have enough facts in front of him to make the conclusions he apparently DID make, AND he should have known that he didn't) statements?
This technology learns how to behave from past experience, just like human drivers. The car probably didn't expect the woman to walk right in front of it outside of an intersection or crosswalk. People do lurch into the roadway at random places but they usually ease up to let the car pass if there's no reasonable way the car could stop. When they don't let the car pass, this is what happens.
Totally agree with CptJeanLuc. Only I tend to have my foot above the brake pedal when the cruise-control is on :)
But what also caught my attention is this ....
"To be clear, while the car had the right-of-way and the victim was clearly unwise to cross there, especially without checking regularly in the direction of traffic, this is a situation where any properly operating robocar following "good practices," let alone "best practices," should have avoided the accident regardless of pedestrian error. That would not be true if the pedestrian were crossing the other way, moving immediately into the right lane from the right sidewalk. In that case no technique could have avoided the event."
This is a clear indication of the difference between how people behave behind a steer and how autonomous vehicles seem to be designed/programmed. An, no matter how well designed, autonomous vehicle system, it will always only follow the programme. When we humans evaluate a certain situation as potentially dangerous or when we expect a mistake from someone else (and without this crucial ability to expect someone else's faults, driving would not be possible) our brain focuses on the threat/point with increased risk while still being able to evaluate what's happening around. Thus the line from the above quote "moving into the right lane from the right sidewalk ... no technique could have voided the event" says it all. A person would or at least a decent human being should expect that if someone is near your vehicle's trajectory, things can go wrong - so you automatically take distance, slow down, make sure you are noticed ... whatever. Something, I don't believe, any autonomous vehicle won't be able to do for many many years. Because that is truly intelligent behaviour, something noone can programme today.
programming intelligence. Truth is we don't have AI good enough or portable enough to drive autonomously. The whole thing is premature and just another "monorail in every city" type scheme to cheat the middle and lower class.
Multinational corporations are not your friend, they exist to exploit.
They absolutely can, if that is their job as a professional test driver, and they have someone else monitoring the systems.
I don't think this is true. Being a "professional" doesn't change human nature, especially if the human is being asked to remain attentive but inactive for long stretches of time. But there are solutions.
A common practice early in Google's testing was to have three people in the car. A safety driver behind the wheel, an engineer in back to monitor the system and a random Google employee in the front passenger seat who was getting a "fun ride", to experience the coolness of a self-driving car. But... that random employee wasn't just cargo. The engineers realized that the safety drivede a tendency to get complacent, even with adequate breaks. So the "extra" employee provided an additional set of eyes for whom the experience was novel and interesting and who could be expected not only to remain focused on the driving environment but also to stimulate the attention of the safety driver, who acted as "tour guide", pointing out the things the vehicle was reacting to.
I got such a "fun ride" a few years ago. It was rather impressive. What convinced me of the promise of the self-driving tech was when the car noticed and reacted to a cyclist that no human could have seen. I had already realized, intellectually, that the perfect attentiveness of a computer plus the superhuman abilities of LIDAR + RADAR promised dramatically safer "drivers", but seeing it in action drove the point home.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Another $5 says this truth will be buried because there is so much money at stake, and why should they care about the life of one random woman?
Progress!
Seriously. Get a life, grow some compassion.
When I saw the video my first reaction was, how did the car not detect her?? She was right there, in front of the car! It would have been trivial to see her crossing lanes towards the vehicle, and you'd be able to see that hundreds (or even thousands) of feet away! This is the most important, most monitored zone when driving, you know, the place you are driving towards.
Yeah the pedestrian shouldn't have crossed there, but people do stupid stuff all the time. Same thing for the monitoring driver. My guess is that it is difficult to maintain situational awareness in the driver's seat, if the autonomous features seem to be working well. Yes, sure, for 10-15 minutes it's no problem, but more than that? I know my attention would wander after a while. More to the point, could you grab the steering wheel and take over fast enough to have saved the pedestrian?
The point is, autonomous vehicles need to be able to handle these kinds of things. And if they can't, get them off the road and back on to test tracks.
How many times have you gotten burned by a defective piece of debug code or hardware setup? Because it's a one-shot thing, you just hack it together. No serious design. Then you end up wasting days because that throwaway is buggy and giving you bad info. Or blowing out the piece of hardware you're debugging.
Here you're saying is that the developers are just dropping in bodies as safety drivers without engineering that temporary job. The difference being that cars driving in public are a loaded gun. It's not a controlled lab at all.
Training? Judging by the driver's panicked inaction in the video, they don't even train them in simulators for surprises.
Inattention is indeed well-studied. Has been for over 50 years. There are experts--often industrial psychologists--you can hire to design tasks like this. Same goes for training people to not freeze when they're in a pinch.
Safety drivers smack of "I know my system won't fail, but I'll quickly throw in something for show just to shut up the critics."
However, what we still do not know is - when did she start moving?
The tight question to ask is "When should the car start slowing down?" Recognize that a pedestrian may move onto the road unpredictably. Consider also the behavior of children at play, pets, deer, and so on. Expressway driving is easy for a machine in part because generally almost nothing is in motion but other vehicles and they are - most of the time, anyway -- essentially running on tracks. Their movements clearly defined and limited.
There will be a whitewash soon, saying everything was the woman's fault, and Uber is as blameless as a newborn babe.
In addition to what has been noted above, the impact occurred on the right headlight area. Looking even at the officially supplied video, I'm sure that if I were driving, I would have had time to at least swerve. In addition, the braking time stated is BS. My actual braking time has been measured at 0.4 seconds. The reaction time itself is typically about 0.25 seconds for visual stimuli. If I had been driving, I would have had time to both brake and swerve.
The lidar system, including code, obviously failed to detect that an object was moving across the road close enough to get hit. If the lidar was operational, it surely got back reflections from the woman. So, for some reason, the code failed to interpret that as an object that was moving into the vehicle path. It may be that something occurred, like once the lidar-related code classified the lidar returns as say landscaping with 99% confidence, it did not believe the landscaping could be moving and ignored that data.
The person they had as backup driver was obviously just a warm body hired to keep regulators happy. Whaddaya bet it gets minimum wage.
Lets put it this way: The accident happened in what appears to be ideal conditions for a self driving car. A wide open road, clear weather, no movement outside the car itself and a single bedestrian with their bicycle. If it had been rainy weather, thick fog or even just a busy street with the victim jumping out from somewhere it would have been a reason to claim that it may have worked in different circumstances. Instead we have an extremely basic failure with what has to be near perfect sensor data. If that had been a boulder dropped from some construction vehicle we may have a dead Uber driver instead.
This is the point we all knew was coming. Somewhere in the development cycle of the self-driving car it was inevitable that a self-driving car would cause a human death. There was really no way around it. The fact that it's likely a human driver might have caused the same death will be completely overlooked. So now comes the decision point. Will autonomous car development be buried for a decade or two or will people admit that no matter what precautions are taken there are just some circumstances in which, when a pedestrian chooses to jump in front of a car on a dark while dressed all in black that they'll get hit, irregardless if the driver is a human or computer.
At this stage of testing the warm bodies are there to deal with situations in which someone needs to explain what's happening or unexpected breakdowns. They're not supposed to be hovering over the controls and ready to take over a split second after the cars misses something.
I can imagine some lawmaker somewhere declaring a halt to driverless cars after this accident.
I have already several articles suggesting that this should not be done because only more and more refinement of such a complex product will cause it to become viable. Also even with a few bugs, driverless cars are possibly already less accident-prone than humans.
As a software developer, I naturally side with continuing development.
Looking at the FAA gives a good model on how to proceed.
When an airplane crashes, the FAA sometimes grounds all models of that plane until the cause of the crash is determined and, if it was a technology error, will not allow the planes to fly again until the problem is satisfactorily resolved.
That would appear to be a measured response to this type of problem.
Don't halt all development. Don't proceed, ignoring the death(s).
Prohibit the specific driverless system from using the public roads until the problem is determined and an acceptable fix is made.
Just as cars have model years that receive approval, so should specific versions of driverless systems.
Then we can have official patches deployed on an as-needed basis, not just when a software engineer declares a bug has been fixed.
Very strict controls need to be in place to allow/deny a software/hardware update to a driverless system.
I don't want my car to be hacked and used as a killer weapon.
Yeah, except that the consensus is quite clear that given today's level of technology this is *not* a reasonable failure and lidar etc. at its current state of development *should* have enabled the vehicle to avoid the collision or at the very least start to slow down before impact.
The real lesson appears to be that given today's technology level, Uber's implementation is seriously defective and should not be allowed on public roads until proved otherwise.
https://www.google.com/maps/@33.4349572,-111.9414301,3a,75y,355.76h,69.39t/data=!3m6!1e1!3m4!1s_i-hwzem35hckWpcSqL_DA!2e0!7i13312!8i6656
Street View of Mill Avenue northbound just past the river bridge. https://www.google.com/maps/@33.4349572,-111.9414301,3a,75y,355.76h,69.39t/data=!3m6!1e1!3m4!1s_i-hwzem35hckWpcSqL_DA!2e0!7i13312!8i6656
Poe's Law? I can't ever tell.