When Mercedes-Benz Starts Selling Self-Driving Cars, It Will Prioritize Driver's Safety Over Pedestrian's (inverse.com)

From a report on Inverse: When Mercedes-Benz starts selling self-driving cars, it will choose to prioritize driver safety over pedestrians', a company manager has confirmed. The ethical conundrum of how A.I.-powered machines should act in life-or-death situations has received more scrutiny as driverless cars become a reality, but the car manufacturer believes that it's safer to save the life you have greater control over. "You could sacrifice the car. You could, but then the people you've saved initially, you don't know what happens to them after that in situations that are often very complex, so you save the ones you know you can save," said Christoph von Hugo, Mercedes' manager of driver assistance systems. "If you know you can save at least one person, at least save that one. Save the one in the car. This moral question of whom to save: 99 percent of our engineering work is to prevent these situations from happening at all. We are working so our cars don't drive into situations where that could happen and [will] drive away from potential situations where those decisions have to be made."As long as they are better at driving and safety than humans, it is a progress, in my opinion.

Comments

What a coincidence

[Anonymous Coward]

I'm working on self-walking pedestrian Gatling guns. Guess what *it* prioritizes?

The fringe cases are still going to be hard

[Wycliffe]

99% of time, the correct action is to stop. If a crash is unavoidable though, if you are solely concerned about the safety of the passenger, then it is safer for the passenger to hit a soft target like a crowd of people than something hard like a telephone pole. The passenger is much more likely to survive hitting a person than a brick wall but a human will usually choose the wall.

With Mercedes, I expect it by model

[swb]

S-class & AMG Models: Maximum driver and driver property prioritization.

E-class models: Minor driver prioritization, slightly better than 50/50 odds

C-class models: Pedestrian prioritization

Re:With Mercedes, I expect it by model

[Overzeetop]

If I'm driving an S class, I expect a little more granularity than that. I need to know that, given the possibility of hitting one or more people in a crowd, the avoidance decision process will go Wealthy > White People > Males > Everyone else.

Resiliency in the face of malicious inputs

[Wrath0fb0b]

Saving the occupants of the car is the only choice that makes sense in the context of potentially malicious input. For instance, if Mercedes stated that their car would swerve into a tree instead of hitting a crowd of 5 pedestrians, what's to stop me and 4 friends from jumping out in front of the cars just to laugh as it crashes itself to "save" us.

We have got to start embedding deep into the mind of every software engineer that any information from outside your system can be manipulated to cause maximum damage or disruption. It is your system's responsibility to safely handle malformed and malicious inputs. Until this becomes a common mode of thought, expect more IoT botnets, SQL injections, buffer overflows, DOS amplifiers and the entire realm of "oh crap someone somewhere could be evil, I only engineered for the happy case".

Re:What does this even mean ?

[amRadioHed]

It means if they are much safer than human drivers it doesn't really matter who they are prioritizing since everyone will be benefitting still.

This has to be the way it works.

[olsmeister]

There really is no other logical way to approach this. If they went the other way and prioritized the pedestrian, a psychopath could sprint back and forth across a busy freeway, causing accident after accident and injuring or killing lots of innocent passengers.

Re:What does this even mean ?

[fluffernutter]

Driving is about anticipation of events way more than reaction time.

Re:What does this even mean ?

[Shoten]

That's a big "if", is currently false and will be false for hundreds of years still. This is declared intent to cause injury, making it a bit past borderline illegal. It is poorly thought through immoral marketing buzz. There is no positive angle to this "story" or even much to say except Mercedez-Benz has decided to let the interns do PR.

Actually, it's not that big an if.

Earlier this year, at a roundtable on connected car security headed up by the NHTSA, the chairman of the NHTSA stood up and cited some interesting numbers. A bit more than 32,000 people had died in vehicle-related accidents the prior year, and about 97% of those were the direct result of, and I quote, "driver error or driver choice." He went on to point out that autonomous vehicles would, if done correctly, eliminate most of those deaths. A car that will refuse to drive in certain conditions if, for example, the tire pressure is too low on one or more tires, or the brakes require more than a certain amount of force to slow the car to a certain standards...these are the less-obvious ways in which such cars are safer. Obviously, they can't drive drunk, don't commit road rage, and don't have any sense of ego about saying that they are having trouble with their eyesight. The car can be objective about its limits, its skills, and any impairment it suffers due to weather, maintenance issues, or any other potential problems. Just the degree of data logging alone that is inherent to autonomous vehicles is already producing useful information about how to prevent crashes, and that's before there are any such vehicles for sale. (And I hear it now..."Tesla sells autonomous vehicles!...but Tesla's system doesn't count, as evidenced by the fact that the maker of that system has cut ties with Tesla, basically saying "It's not supposed to be used that way!") Cars have reached the point where humans are the main source of the risk, and while the technology isn't quite ready-for-market, it's not "hundreds of years" away and it's very, very promising.

And no, what Mercedes is saying is not intent to cause injury. It's a statement about which injury to try and prevent in situations where...as this has been discussed for quite some time now...an injury is deemed inevitable. They have not said, "our cars will drive through schools for no particular reason, just to annoy Jzanu,." They have said, "our car's logic knows what's in the car, what's going on with the car, and can directly control the car. It does not know that much about the rest of the world, so we believe the odds of the best possible outcome in a situation with no good outcomes lies with letting the car preserve its own passengers."

And there is absolutely nothing illegal about that whatsoever. It's the same logic behind why paramedics don't run, ambulances slow down through intersections where they can't see past a certain distance, and a whole bunch of other situations where you have to weigh risk of one bad outcome against risk of another one.