Slashdot Mirror
When Mercedes-Benz Starts Selling Self-Driving Cars, It Will Prioritize Driver's Safety Over Pedestrian's (inverse.com)
From a report on Inverse: When Mercedes-Benz starts selling self-driving cars, it will choose to prioritize driver safety over pedestrians', a company manager has confirmed. The ethical conundrum of how A.I.-powered machines should act in life-or-death situations has received more scrutiny as driverless cars become a reality, but the car manufacturer believes that it's safer to save the life you have greater control over. "You could sacrifice the car. You could, but then the people you've saved initially, you don't know what happens to them after that in situations that are often very complex, so you save the ones you know you can save," said Christoph von Hugo, Mercedes' manager of driver assistance systems. "If you know you can save at least one person, at least save that one. Save the one in the car. This moral question of whom to save: 99 percent of our engineering work is to prevent these situations from happening at all. We are working so our cars don't drive into situations where that could happen and [will] drive away from potential situations where those decisions have to be made."As long as they are better at driving and safety than humans, it is a progress, in my opinion.
I'm working on self-walking pedestrian Gatling guns. Guess what *it* prioritizes?
99% of time, the correct action is to stop. If a crash is unavoidable though, if you are solely concerned about the safety of the passenger, then it is safer for the passenger to hit a soft target like a crowd of people than something hard like a telephone pole. The passenger is much more likely to survive hitting a person than a brick wall but a human will usually choose the wall.
S-class & AMG Models: Maximum driver and driver property prioritization.
E-class models: Minor driver prioritization, slightly better than 50/50 odds
C-class models: Pedestrian prioritization
Sounds logical to me. Otherwise, why would I pay Mercedes-Benz to save other people? I am not an altruist and don't inspire to be one in life&death situations.
Saving the occupants of the car is the only choice that makes sense in the context of potentially malicious input. For instance, if Mercedes stated that their car would swerve into a tree instead of hitting a crowd of 5 pedestrians, what's to stop me and 4 friends from jumping out in front of the cars just to laugh as it crashes itself to "save" us.
We have got to start embedding deep into the mind of every software engineer that any information from outside your system can be manipulated to cause maximum damage or disruption. It is your system's responsibility to safely handle malformed and malicious inputs. Until this becomes a common mode of thought, expect more IoT botnets, SQL injections, buffer overflows, DOS amplifiers and the entire realm of "oh crap someone somewhere could be evil, I only engineered for the happy case".
It means if they are much safer than human drivers it doesn't really matter who they are prioritizing since everyone will be benefitting still.
We hope your rules and wisdom choke you / Now we are one in everlasting peace
There really is no other logical way to approach this. If they went the other way and prioritized the pedestrian, a psychopath could sprint back and forth across a busy freeway, causing accident after accident and injuring or killing lots of innocent passengers.
Self driving cars don't need to be safer than the most safe human driver, they only need to be safer than the average human driver. And those are pretty disregarding of safety. So yes, maybe for a safe driver stepping into a self driving car will the risk will be increased, but for most people it will be lower.
This is declared intent to cause injury
If it crashes into the tree, it injures the driver. It will cause injury one way or another.
AI vehicles currently have about 4.5 times the fatality rate of human drivers per mile traveled.
I would say this is a bogus averaging process that's come up with 4.5x fatality rate, Because there are too few accidents involving AI vehicles
for that calculation to have a theoretical foundation.
There have been what, less than 10 accidents involving AI? Not even enough to discount random chance and successfully calculate a meaningful average accident frequency.
Driving is about anticipation of events way more than reaction time.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
You should read non-fiction "The Robot's Rebellion." Ignore the preface of author rambling... the 8 core chapters are some of the most heavily footnoted text I've ever waded through detaining just how many guess-and-check shortcuts the human brain takes. The book lays out study after study, including some things you can try on your own, that pretty much denies any argument about humans having a serious advantage over computers in high-speed decision making. With deliberation, our brains are awesome. In the heat of the moment, we aren't even consistent in our heuristics.
Clearly you've never been to Paris. I'm not even sure if brakes exist there.
If you're lucky, they might blow the horn.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
That's a big "if", is currently false and will be false for hundreds of years still. This is declared intent to cause injury, making it a bit past borderline illegal. It is poorly thought through immoral marketing buzz. There is no positive angle to this "story" or even much to say except Mercedez-Benz has decided to let the interns do PR.
Actually, it's not that big an if.
Earlier this year, at a roundtable on connected car security headed up by the NHTSA, the chairman of the NHTSA stood up and cited some interesting numbers. A bit more than 32,000 people had died in vehicle-related accidents the prior year, and about 97% of those were the direct result of, and I quote, "driver error or driver choice." He went on to point out that autonomous vehicles would, if done correctly, eliminate most of those deaths. A car that will refuse to drive in certain conditions if, for example, the tire pressure is too low on one or more tires, or the brakes require more than a certain amount of force to slow the car to a certain standards...these are the less-obvious ways in which such cars are safer. Obviously, they can't drive drunk, don't commit road rage, and don't have any sense of ego about saying that they are having trouble with their eyesight. The car can be objective about its limits, its skills, and any impairment it suffers due to weather, maintenance issues, or any other potential problems. Just the degree of data logging alone that is inherent to autonomous vehicles is already producing useful information about how to prevent crashes, and that's before there are any such vehicles for sale. (And I hear it now..."Tesla sells autonomous vehicles!...but Tesla's system doesn't count, as evidenced by the fact that the maker of that system has cut ties with Tesla, basically saying "It's not supposed to be used that way!") Cars have reached the point where humans are the main source of the risk, and while the technology isn't quite ready-for-market, it's not "hundreds of years" away and it's very, very promising.
And no, what Mercedes is saying is not intent to cause injury. It's a statement about which injury to try and prevent in situations where...as this has been discussed for quite some time now...an injury is deemed inevitable. They have not said, "our cars will drive through schools for no particular reason, just to annoy Jzanu,." They have said, "our car's logic knows what's in the car, what's going on with the car, and can directly control the car. It does not know that much about the rest of the world, so we believe the odds of the best possible outcome in a situation with no good outcomes lies with letting the car preserve its own passengers."
And there is absolutely nothing illegal about that whatsoever. It's the same logic behind why paramedics don't run, ambulances slow down through intersections where they can't see past a certain distance, and a whole bunch of other situations where you have to weigh risk of one bad outcome against risk of another one.
For your security, this post has been encrypted with ROT-13, twice.
Your last one is the # reason I make driving mistakes. Luckily I haven't had one translate to an accident, but my rule is I pull over and wait for the mouth to shut. I have pulled over. Waited 10 minutes. Pulled out and immediately pulled back in because it started up again around 5mph. Then I pulled over and waited, then repeated this 2 more times. I then pulled over, got out and called a cab.
Who's the asshole here?
PS The mouth does not drive.
Anyone who swerves is an idiot and did not pay attention during driver's education. You can steer around obstacles, but swerving is one of the worst decisions you can make.
"As long as they are better at driving and safety than humans, it is a progress, in my opinion."
I'm not convinced. Right now, when people die in car crashes, and I can blame a human driver for something, then it's totally understandable. When humans die by the hands of other humans, and especially through the errors of other humans, that's just a reality that I can comprehend and accept.
But when a self-driving car is ultimately responsible for killing a human, that's a different thing entirely. That's a lot closer to just humans-get-killed-at-random scenario. That's not something that I can accept.
It's actually even worse than that. It's like a neighbourhood pet dog kills a neighbour. If your typically-well-behaved-and-friendly boxer suddenly kills your neighbour's teenager one day, what happens? Look, your dog killed one neighbour over the course of thirty years of you owning dogs. Most wild animals are far more dangerous than that. But I think we all know what happens. I think your dog is dead pretty quickly -- even if that teenager provoked your dog; even if it was a lot; even if your dog was defending its own life.
I accept, today, that millions of humans driving millions of cars on millions of roads, kills thousands of people every year. I'm not happy about it, but I accept it as a part of humans being free to not be perfect. But I don't think that I'd be accepting of millions of self-driving cars on millions of roads, killing dozens of people every year.