Slashdot Mirror
Experts Say Video of Uber's Self-Driving Car Killing a Pedestrian Suggests Its Technology May Have Failed (4brad.com)
Ever since the Tempe police released a video of Uber's self-driving car hitting and killing a pedestrian, experts have been racing to analyze the footage and determine what exactly went wrong. (If you haven't watched the video, you can do so here. Warning: it's disturbing, though the actual impact is removed.) In a blog post, software architect and entrepreneur Brad Templeton highlights some of the big issues with the video:
1. On this empty road, the LIDAR is very capable of detecting her. If it was operating, there is no way that it did not detect her 3 to 4 seconds before the impact, if not earlier. She would have come into range just over 5 seconds before impact.
2.On the dash-cam style video, we only see her 1.5 seconds before impact. However, the human eye and quality cameras have a much better dynamic range than this video, and should have also been able to see her even before 5 seconds. From just the dash-cam video, no human could brake in time with just 1.5 seconds warning. The best humans react in just under a second, many take 1.5 to 2.5 seconds.
3. The human safety driver did not see her because she was not looking at the road. She seems to spend most of the time before the accident looking down to her right, in a style that suggests looking at a phone.
4.While a basic radar which filters out objects which are not moving towards the car would not necessarily see her, a more advanced radar also should have detected her and her bicycle (though triggered no braking) as soon as she entered the lane to the left, probably 4 seconds before impact at least. Braking could trigger 2 seconds before, in theory enough time.)
To be clear, while the car had the right-of-way and the victim was clearly unwise to cross there, especially without checking regularly in the direction of traffic, this is a situation where any properly operating robocar following "good practices," let alone "best practices," should have avoided the accident regardless of pedestrian error. That would not be true if the pedestrian were crossing the other way, moving immediately into the right lane from the right sidewalk. In that case no technique could have avoided the event. The overall consensus among experts is that one or several pieces of the driverless system may have failed, from the LIDAR system to the logic system that's supposed to identify road objects, to the communications channels that are supposed to apply the brakes, or the car's automatic braking system itself. According to Los Angeles Times, "Driverless car experts from law and academia called on Uber to release technical details of the accident so objective researchers can help figure out what went wrong and relay their findings to other driverless system makers and to the public."
1. On this empty road, the LIDAR is very capable of detecting her. If it was operating, there is no way that it did not detect her 3 to 4 seconds before the impact, if not earlier. She would have come into range just over 5 seconds before impact.
2.On the dash-cam style video, we only see her 1.5 seconds before impact. However, the human eye and quality cameras have a much better dynamic range than this video, and should have also been able to see her even before 5 seconds. From just the dash-cam video, no human could brake in time with just 1.5 seconds warning. The best humans react in just under a second, many take 1.5 to 2.5 seconds.
3. The human safety driver did not see her because she was not looking at the road. She seems to spend most of the time before the accident looking down to her right, in a style that suggests looking at a phone.
4.While a basic radar which filters out objects which are not moving towards the car would not necessarily see her, a more advanced radar also should have detected her and her bicycle (though triggered no braking) as soon as she entered the lane to the left, probably 4 seconds before impact at least. Braking could trigger 2 seconds before, in theory enough time.)
To be clear, while the car had the right-of-way and the victim was clearly unwise to cross there, especially without checking regularly in the direction of traffic, this is a situation where any properly operating robocar following "good practices," let alone "best practices," should have avoided the accident regardless of pedestrian error. That would not be true if the pedestrian were crossing the other way, moving immediately into the right lane from the right sidewalk. In that case no technique could have avoided the event. The overall consensus among experts is that one or several pieces of the driverless system may have failed, from the LIDAR system to the logic system that's supposed to identify road objects, to the communications channels that are supposed to apply the brakes, or the car's automatic braking system itself. According to Los Angeles Times, "Driverless car experts from law and academia called on Uber to release technical details of the accident so objective researchers can help figure out what went wrong and relay their findings to other driverless system makers and to the public."
Forget the Lidar, or lack of (they were testing cameras?). Forget the dude (heh, the first 12 hours thought he was a she. That's gotta hurt).
Had I been driving that car, full alert, I would have killed that chick. I'd have felt bad, even knowing it was her fault. But the fact is, this dumbass walked in front of a fast moving car, at night, when she had no illumination, and the car had headlights. Her best hope of survival was a 100% functioning self driving car, anything less and she's dead.
Duh!! yes! along with the convept of a safety driver.
;)
Just my 2 cents
Uber must be using some bottom-of-the-barrel piece of shit from Wally World. Here are pictures of the scene. It's plenty well lit, there's no excuse for not having seen the biker well ahead of time.
There were multiple failures all around which caused this death. If any one of those failures had not happened then the pedestrian would likely still be alive today.
I've summed it up here in a column which was written almost 24 hours ago so it's nice to see that others have come to similar conclusions.
It'll save your ass. God help the pedestrians, though.
China is not doing a great job on these. Sad.
I prefer the "u" in honour as it seems to be missing these days.
Continuing to develop and test self-driving cars will result in more pedestrian deaths. That much of any driving will result in pedestrian deaths. It is going to be interesting to see how this develops over time.
"When the going gets weird, the weird turn pro" -- HST
...or due to changes made in wake of the Google-Uber lawsuit settlement. But the questions probably need to be asked, in light of statements made by both companies: 1. A note on our lawsuit against Otto and Uber: "Recently, we uncovered evidence that Otto and Uber have taken and are using key parts of Waymo's self-driving technology." 2. Uber and Waymo Reach Settlement: "We are taking steps with Waymo to ensure our Lidar and software represents just our good work."
I agree the LIDAR should have been able to see her before she entered the light.
However, what we still do not know is - when did she start moving?
If she was just standing in the left lane waiting to cross, the LIDAR may have seen her and just thought "well that lane is blocked, stick to this one". With the bike she might have looked like a barricade of some kind.
It could still be she started moving around the time we see her in the video, which means she essentially jumped in front of the car...
There could be a reason for her to do that - what if the car saw her, and slightly slowed out of caution? We know the car was going well under the limit when it hit, that could be a sign the car slowed down a bit prior.
The human, seeing a car slow light that might have assumed it saw her and was going to stop to let her cross. So it could easily be a case of mixed signals, with the cars cautious actions in the end being a bad thing, when driving an over-abundance of caution can often have bad consequences.
I'm still not sure the human safety driver would have seen her though, even though humans do have better dynamic range than cameras there still are times when you really can't see outside the headlights, and the woman crossing was all in dark clothing.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
You know how UBER has tried all the dirty tricks in the book, be it ignoring local laws, making fake bookings on competitors etc etc :)
How about if they though, hey, all those LIDAR and radar systems are expensive, how about if we try to do it with a cheap offshore dev team and just a chinese dashcam? That would save a lot of money, no?
And, the released video is what the system was actually running with...
Well there you go. Clearly these cars should be kept on the road so Uber has every opportunity to make their technology better.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
The lesson for the dead person would have been to look both ways. The lesson for the rest of us is that Uber's self driving technology is not ready for prime time, for whatever reason(s).
In the middle of the highway when there is slush and heavy snow. Does it just stop in the middle of the highway and you get crushed by the Semi truck behind you? With no steering wheel you don't have a chance. In my 2016 Civic, many times when I was driving in the bad conditions, the car warned me that the safety features were turned off because sensors temporarily are down. But I can still drive the car myself. So I still don't get how the self driving cars will handle this without getting destroyed on the highway
"To be clear, while the car had the right-of-way and the victim was clearly unwise to cross there..."
Back in the day when I learned to drive the rule was that pedestrians ALWAYS have the right of way -- even if they're crossing in the middle of the road.
So if it was daylight where the SNR of the LIDAR is worse with ambient light hitting the sensor and the car hit her where would you place blame? Because that is exactly what would have happened. The LIDAR system would still have missed her. The system failed period full stop. It is as bad as when Intel released the processor that did not always add right. I know the tech world is spending billions to make this work and it seems to be every techies wet dream for it to work, but it does not work yet. And deploying it in the wild is murder. It is not ready.
Personally, I'm expecting to find out that Uber was secretly testing their car with the LIDAR intentionally off, to compare how well the system worked with and without LIDAR (i.e. to see if they could save a couple of bucks by not installing LIDAR).
Community: Uber please release possibly incriminating technical details of the accident to help us make our systems safer ; Uber:... Crickets...
The main failure will be holding uber accountable.
When driving a car with normal headlights, can you genuinely not see what's the next lane over 150 feet ahead?
Lots of cars fudge the headlights a bit to the right to keep from blinding oncoming cars. Combine that with her stepping out of a very dark shadow of the tree, as well as her dark clothing (jeans and black jacket) and it's very likely that because of the dynamic range of the scene (bright headlights making darker anything street lights shone on, then her being in the shadow from even the street lights) a driver would have been too blinded by available light to see the pedestrian.
She didn't even have reflectors on the wheel of the bike, much less herself - even one might have saved her.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Probably looking at system information for the car itself.
If you're blamed for killing a pedestrian, you may have your ass handed to you.
He's getting rather old, but he's a good mouse.
The street was well lit, the camera should have seen her.
"First they came for the slanderers and i said nothing."
The car had a single occupant who was not focused on the road. In a test vehicle, regardless of how autonomous the vehicle is if one person is required to monitor computer systems whilst the car is travelling, they need a second occupant to monitor the road.
The Lidar system should work with no light at all, its an infrared laser system, it emits its own light! In fact if anything it would probably work better in complete darkness!
This was obviously a major technology failure, however it needn't have resulted in someones death.
The greatest failure here is not in the technology but in Ubers testing procedures.
That video is somewhat brighter than what a real driver would see.
But even so, look at your video at 33 seconds - that's the point where the much wider camera in the uber catches sight of her shoes in his lane (at about 7 seconds in this video). The pedestrian is in shadow and if she wasn't moving before it would be very easy not to see her until she started moving. If you go back a bit further in the video she would be standing right between some bright background lights, making it even more difficult to see her if not moving.
Now look at the video of the safety driver. It's not like she was never looking up, she was looking back up every few seconds or so. If the woman were really as visible as you are thinking, the safety driver would have seen her way ahead of time. The last time in the video of the driver she was looking ahead was about 17 seconds into the driver view video, which equates to about 30 seconds in your video (look at the poles passing by the window), if the whole scene were really so bright shouldn't the safety driver have seen the pedestrian at that point? Again, if not moving the pedestrian (in all dark clothing and no reflectors) would have been far harder to see. Pedestrians are killed in this kind of situation by drivers who are watching the road continuously... or as continuously as a human can.
What would be really informative to get, is the LIDAR data from this incident. That would really say a lot about what the car itself did or did not see, then we could go on to speculate how it processed what it "saw".
"There is more worth loving than we have strength to love." - Brian Jay Stanley
The lesson for the rest of us is that Uber's self driving technology is not ready for prime time, for whatever reason(s).
I dunno ... I mean even if we assume that a human driver could have avoided this particular accident, that doesn't mean the technology isn't still an improvement over human drivers in other cases. You'd need a lot more data to reach that conclusion. It could very well be that lives saved in other, more common types of preventable accidents massively outweigh the lives lost in these types of abnormal occurrences.
The idea of "safety driver" might be inherently flawed. When you drive a normal car, your attention is fixed on the traffic and surroundings. When the car does the driving, how are you supposed to sustain your attention for more than a few minutes? Boredom and attention lapses may be inevitable. If the safety driver is told ahead of time that attention lapses have severe penalties, they might struggle to remain alert for a while longer, but it might be a fact of human nature, that avoiding distractions is an uphill battle.
update quickly = new car each 2-4 years no regulation saying free updates for at least 9-12 years
Look at the video again. As soon as the driver looked up, she saw the pedestrian.
When she looked up and saw the pedestrian, she was well past the bridge, and it was way too late to do anything. She basically looked up as the car hit the person.
I was talking about the time she looked up BEFORE that time (in the driver video she looked up from her phone several times), which you can clearly tell by the poles passing by outside her window is when she is passing under the bridge, and match that with the other brighter video when it passes under the bridge and the poles are too the side (in the driver video remember you are looking at poles that are well behind the car as you are looking backwards).
The timeline is thus: She looks up as she is going under the bridge, looks down again, then a few seconds later she looks up basically as the car is hitting the woman.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Well you've been in a dark corner of the internet, haven't you?
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
I've got a forward facing car camera, and its very clear that eyes see things far bigger and brighter at the horizon than cameras. You would see her, even in that light.
This is why the moon seems bigger and brighter when its near the horizon. When actually its dimmer due to atmosphere and the same size as normal.
*But*, more than that.
I've also seen the other video of this stretch of road filmed with a normal smartphone camera and its clear the camera in the car had terrible dynamic range. There is absolutely no way a person looking at the road would not have seen her.
https://www.youtube.com/watch?v=CRW0q8i3u6E
Level 1 self drive is where the car acts as a safety feature for the driver.
Level 2 is where the driver acts as a safety feature for the car!
Which is it!! Is the car safer or the driver?? You cannot have Level 2 because its simply dumping liability on the driver, when he's not party to the decisions the car is making and does not know what it will do, till after its done it.
The car drives, the man observes, then deduces what decision its made from the changes to the car. He cannot be the safety backup for the car. It's not possible, its a legalese get out for self driving car makers.
The first time I saw the video, I went, "Man, somebody's going to win a multimillion dollar judgement against Uber!" True be know, due to my night blindness, I probably would have hit the pedestrian too under similar circumstances. I constantly see pedestrians or bicyclists in dark clothing on the side of the road only after it would be too late to avoid them if they moved in front of my car.
I've abandoned my search for truth; now I'm just looking for some useful delusions.
"you best have your foot on the brake waiting"
Actually, dickhead, you should have your foot on the brake waiting, it's called "defensive driving", and you have obviously never received any decent driver education in your life.
You should be anticipating what can go wrong around you every step of the way. Does that mean you slow down a bit at times? Damn straight it does. Does that mean you hover on the brake while a situation develops in front of you? Damn straight? Does that mean that at times, when it's not safe to proceed, you simply stop your car and wait for the situation to resolve? Damn straight it does.
I find these car/driving posts on Slashdot amusing, but it becomes tedious after a while.
The USA: An entire country afflicted with Dunning-Kruger...
In perfect dark condition, no new moon possibly cloud coverage no good street lamp, I can barely see 30-40 meter away object which have no reflective properties. Try it. With a new moon or better reflective object I can naturally as you say see muuuuch further away. The question is : what are the condition here. People says "human eye see further away" yes and no. Was there other source of lights ? Because if yes it ruins your night vision, and if there is only shadow and nor reflective surface, forget your 150 feet. You'll be lucky with 3/4 of that.
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
The overall consensus among experts is that one or several pieces of the driverless system may have failed, from the LIDAR system to the logic system that's supposed to identify road objects, to the communications channels that are supposed to apply the brakes, or the car's automatic braking system itself.
But the most important thing that failed was the human driver taking over control in an emergency when all systems fail to work properly, destroying a long held "get out of jail free card" creators of self driving cars use whenever anyone challenges them on the tech.
I only please one person per day. Today is not your day. Tomorrow isn't looking good either. - Scott Adams
Clearly this was a "Trolley Car Problem" situation; and we now know that self-driving cars - will choose to hit a pedestrian to minimize risk to passenger. Amateur philosophers have been debating this problem in the context of SD cars for some time now. Now, at last, we can put an end to any further discussion of Trolley Cars by telling them: "Problem Solved".
I'm no expert, but I've seen a lot of news about self-driving cars over the last ten years or so.
At first it was DARPA, then Google, and they seemed to be testing for years . . .
Then Tesla, and now Uber ?
I guess Uber suddenly realized that self-driving cars might be a threat to their business in the long term.
So they are notoriously getting caught with stolen code and poaching engineers.
Seems like they are playing catch-up.
That usually means skipping testing.
Maybe the states should require that self-driving equipment and algorithms get a serious "drivers test" ?
Even a semi-automated software test suite and an independent engineering review might be good enough.
Eventually, we WILL need this.
The technology is now easy enough for teenagers to start doing it on their own cars at home.
I don't want MY life to depend on THEIR coding skills.
Absolutely agree here. I drove along city streets last night. I could easily see people wearing dark clothes, beside the road, at a distance of 75 feet. The camera doesn't give the full story.
A human safety driver will not work without some means of keeping the driver engaged. These cars had driven millions of miles between them with no accidents until this one. Obviously the passenger's attention will wander. The human was there to mollify critics, not to act as actual safety driver.
The pedestrian doesn't seem to be in much of a rush. This is something I simply don't understand. I'd have thought she'd be moving a lot faster.
If it's a driverless car that requires driver to be attentive of the road and what's goign on - it's never gonna work right.
Either driver actively drives the car, or it doesn't... there's no in between. We're humans, not robots...
This person was looking at his facebook all the time, next one will nod off.. etc.
This is not only a failure in Uber's testing procedure. This is the failure of the whole concept of autonomous "driver assist only" systems. There are a lot of people out there who would do the exact same thing in e.g. their Tesla - let their attention slip for a while and let the system do its job, because humans are lazy, plus we are bad at drawing conclusions - nothing bad happened the first two weeks we drove the car, so that means nothing bad is going to happen after that either - right?
This is also the reason that with only a "regular" cruise control in my own traditional car, I always keep the foot loosely on the gas pedal while cruise control is engaged. Why? Because if something happens and I need to react fast to apply the brakes, then I don't have time to figure out where I have my foot. I need to let brain automation of moving the foot from the gas pedal to the brake - something for which I have muscle memory - happen very, very quickly.
Now consider "reacting" in a half-autonomous car. You are half using your cell phone or whatever. A situation arises. You need to (a) shift focus from your distraction, (b) fully process what is the situation and what do you need to do to react, (c) figure out how to do it. And way before you get to (c), the accident has already happened.
The whole concept of "driver assist" - for drivers who are not consciously aware of the dangers mentioned above, and actively engaging their focus, body posture and pre-frontal lobe in order to override said danger - which let's be honest is actually most people out there - is quite dangerous, for any dangerous situation the car cannot handle by itself.
So this is not really about Uber. It is about human nature, and why this technology can never fully take off unless car AI gets near perfect (which I don't see happening, which I will write about in a separate comment), or drivers get as disciplined as North Koreans.
Pull up google maps.
https://www.google.com/maps/@3...
Observe the sign that says not to cross there (one on the other side of the median too btw).
Observe the road widens from 2 to 4 lanes right before where the accident occurred.
Observe that trees and bushes on the edge of the median to the left would have blocked Lidar until the car was within 50' of where the accident occurred.
Confirm the scene against the video with the sign that reads
"
Begin right turn lane
-
Yield to Bikes.
"
Understand that it refers to bikes in the bike lane on the right.
Pull up a top down view and get a ruler and note that the street lights are about 120' apart here. They are much closer in my area ( 50-100' apart).
(Also, if you look up the light pollution map, the area to the right of the road is very dark).
Go 600' back down Mills road and observe the 45mph speed limit on this road.
It's a terrible accident. The road widening like that with bushes and small trees blocking lidar, the pedestrian walking across the road without even looking for oncoming traffic (because it was late at night on a sunday in what looks like a kinda empty area so there probably isn't much traffic).
Wait for people to really dig into this. Don't make a snap judgement.
I'm sorry someone died, but there has been a lot of bad information and a lot of people rushing to judgement before they have the facts.
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
IMNSHO, "autonomous driving" is a huge fad - the idea is good, but it is _way_ ahead of its time.
The problem is all those situations in traffic that require understanding a situation, processing what is going on, predicting potential risks, and responding appropriately. One example - you are driving and up ahead on the sidewalk there are two children. Do you slow down? What are the children doing? Do they seem to be behaving erratic, or are they engaged in playful behaviour like pushing each other around a bit, and is there a risk they could get into the road without paying attention? How old do they seem to be? Is there an obstacle in front of them on the sidewalk which is going to force them to move onto the road about the time when you pass them? Is there a puddle of water in the road which will create a huge splash unless you drive around it or slow down a lot? Are they on their way around a bend where you cannot see what is on the other side, e.g. whether the sidewalk ends or some bicycle might approach and force them out in the road?
There is a _huge_ number of such little judgement calls we make every single time we drive. And AI is nowhere _near_ of being able to process anything similar.
Forget the whole "this could be fixed with better LIDAR" etc. The reason why this whole auto-drive concept is a bust the way people are being way too excited talking about it, is not those things we could fix with current technology - it is the type of examples I am describing that we do not have a solution for.
The fix is either to force a "driver assist only" type of regime, which obviously doesn't work because of human nature, as we saw an example of with the Uber incident. Alternatively you compensate by making the auto-pilot system super risk averse and slow down for just about anything, which would be ok - except that really brings down efficiency, I am not willing to spend twice as long getting to work because of some drive that is insanely cautious. Or third, move the vehicles off regular roads into protected areas where they can drive without the dangers and unpredictable conditions of regular roads - which is an option, but requires building all the infrastructure to support it - hugely costly, and not something I see happening.
TL;DR; IMNSHO automonous cars will only happen when AI reaches human levels
And at which point we *tinfoil hat on* should think twice before getting in a car that may decide it likes us better when we are dead.*tinfoil hat off*.
When the car is deemed ready and put into production ans so on, yes absolutely.
But this was a *test driving* situation, during which unexpected and possibly dangerous situations may occur. It is irresponsible of Uber to not have two people in the car at all times, one to monitor the road and take evasive action if necessary, and one to monitor the systems. One person cannot do both at the same time.
Eat the rich.
It's disturbing how many comments in the media have tried blaming the pedestrian for crossing at the wrong spot. There seems to a be a bit of an astroturfing campaign â" Reddit is full of such comments.
In much of the world (where auto makers have not managed to buy a change the law) this is not illegal and in fact standard practice, such as in the UK for example. Any self driving technology must clearly be able to deal with this, not least for handling unpredictable young children in residential areas.
0.5 seconds breaking time would dropped the speed to about 30mph
"for making false statements when obtaining unemployment benefits and attempted armed robbery" Not really relevant to driving ability
IMELPHO*, AI already has reached and even exceeded human levels in some areas. It remains laughably far away in others. But I'll echo others' observation that it does not have to be perfect in order to be a safer alternative to human drivers. It just has to be better than they are. And that's a very low bar. I'm not sure we're there yet, but I think we're close, closer than I'd have thought possible 10 years ago, and I think we'll get there. Sadly, but just as in other human endeavors, there will be mistakes and accidents and innocent people will suffer and die in the short term. But, hopefully, fewer than if we did not even attempt to progress toward something we know in the end will be, if not perfectly safe, at least much safer than what we have now.
* In my even less plausibly humble opinion.
Nonaggression works!
She was standing on the side of the road for the full approach. Headlights work for over 100 feet, the car should definitely not been speeding past the capability of the headlights.. You can't trust a video to show human eye accurate details in darkness. Automated cars are supposed to have sensors that are not defeated by darkness.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
This fatal, a complete catastrophic robotics design fail, is not an accident. Accidents happen by circumstance, calamity or conscious error. Here what you see is a First Order deliberate design point failure. Its immoral, unpredictable and unsafe at any speed.
The death last year of a factory worker cleaning inside an assembly robo cage was no accident neither. That one also bothered. The robot was not operating, supposedly non-operational during off-hours though obviously not powered down. The cleaning lady was attacked by its arms, thrashed and shredded to death. Not even the motion of the machine to effect such horror mimics its design function, programmed range of duty nor safety protocol. That was the first most disturbing, unpredictable and unsafe robo-kill.
Now we have a 2nd RoboKill. An unexplained kill by robot. A phenomenon.
Thanks for the expert opinion to clarify the situation. A lot of us thought killing a pedestrian was normal.
We still don't kill inattentive pedestrians in the Phoenix area because we think they should know better. Even on McDowell or Grand traffic stops for those who seem to be oblivious to traffic. And even those who mock the traffic and cross, knowing they are too expensive to strike.
I wouldn't try cross the 128 in the Boston area, but 1) there are no crosswalks, it's freeway that makes the 101 look a little tame and 2) they don't even stop to change tires. I sure wouldn't try crossing Power Road at night without having a good look, even north of Brown.
deleting the extra space after periods so i can stay relevant, yeah.
They absolutely can, if that is their job as a professional test driver, and they have someone else monitoring the systems.
Eat the rich.
This is one thing that I don't understand at all. Why in the world are self-driving cars allowed on the road without *two* human backups. One should be monitoring the systems to see if they're working. The other should be the safety driver. What should have happened here is that the engineer (for lack of better word) should have been saying to the driver. "Hey, the Lidar doesn't seem to be detecting that pedestrian as a stationary object" and writing a bug report. The driver would have been at heightened alert.
This technology learns how to behave from past experience, just like human drivers. The car probably didn't expect the woman to walk right in front of it outside of an intersection or crosswalk. People do lurch into the roadway at random places but they usually ease up to let the car pass if there's no reasonable way the car could stop. When they don't let the car pass, this is what happens.
They absolutely can, if that is their job as a professional test driver, and they have someone else monitoring the systems.
I don't think this is true. Being a "professional" doesn't change human nature, especially if the human is being asked to remain attentive but inactive for long stretches of time. But there are solutions.
A common practice early in Google's testing was to have three people in the car. A safety driver behind the wheel, an engineer in back to monitor the system and a random Google employee in the front passenger seat who was getting a "fun ride", to experience the coolness of a self-driving car. But... that random employee wasn't just cargo. The engineers realized that the safety drivede a tendency to get complacent, even with adequate breaks. So the "extra" employee provided an additional set of eyes for whom the experience was novel and interesting and who could be expected not only to remain focused on the driving environment but also to stimulate the attention of the safety driver, who acted as "tour guide", pointing out the things the vehicle was reacting to.
I got such a "fun ride" a few years ago. It was rather impressive. What convinced me of the promise of the self-driving tech was when the car noticed and reacted to a cyclist that no human could have seen. I had already realized, intellectually, that the perfect attentiveness of a computer plus the superhuman abilities of LIDAR + RADAR promised dramatically safer "drivers", but seeing it in action drove the point home.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Another $5 says this truth will be buried because there is so much money at stake, and why should they care about the life of one random woman?
Progress!
How many times have you gotten burned by a defective piece of debug code or hardware setup? Because it's a one-shot thing, you just hack it together. No serious design. Then you end up wasting days because that throwaway is buggy and giving you bad info. Or blowing out the piece of hardware you're debugging.
Here you're saying is that the developers are just dropping in bodies as safety drivers without engineering that temporary job. The difference being that cars driving in public are a loaded gun. It's not a controlled lab at all.
Training? Judging by the driver's panicked inaction in the video, they don't even train them in simulators for surprises.
Inattention is indeed well-studied. Has been for over 50 years. There are experts--often industrial psychologists--you can hire to design tasks like this. Same goes for training people to not freeze when they're in a pinch.
Safety drivers smack of "I know my system won't fail, but I'll quickly throw in something for show just to shut up the critics."
However, what we still do not know is - when did she start moving?
The tight question to ask is "When should the car start slowing down?" Recognize that a pedestrian may move onto the road unpredictably. Consider also the behavior of children at play, pets, deer, and so on. Expressway driving is easy for a machine in part because generally almost nothing is in motion but other vehicles and they are - most of the time, anyway -- essentially running on tracks. Their movements clearly defined and limited.
I can imagine some lawmaker somewhere declaring a halt to driverless cars after this accident.
I have already several articles suggesting that this should not be done because only more and more refinement of such a complex product will cause it to become viable. Also even with a few bugs, driverless cars are possibly already less accident-prone than humans.
As a software developer, I naturally side with continuing development.
Looking at the FAA gives a good model on how to proceed.
When an airplane crashes, the FAA sometimes grounds all models of that plane until the cause of the crash is determined and, if it was a technology error, will not allow the planes to fly again until the problem is satisfactorily resolved.
That would appear to be a measured response to this type of problem.
Don't halt all development. Don't proceed, ignoring the death(s).
Prohibit the specific driverless system from using the public roads until the problem is determined and an acceptable fix is made.
Just as cars have model years that receive approval, so should specific versions of driverless systems.
Then we can have official patches deployed on an as-needed basis, not just when a software engineer declares a bug has been fixed.
Very strict controls need to be in place to allow/deny a software/hardware update to a driverless system.
I don't want my car to be hacked and used as a killer weapon.
So you're turning your license in?
Don't be so quick to say this is a long time. Could have been an animal that would have been clear by the time the car was there. Instead it was just a stupid woman that should have stayed where she was until the car passed.
Poe's Law? I can't ever tell.