When Mercedes-Benz Starts Selling Self-Driving Cars, It Will Prioritize Driver's Safety Over Pedestrian's (inverse.com)

From a report on Inverse: When Mercedes-Benz starts selling self-driving cars, it will choose to prioritize driver safety over pedestrians', a company manager has confirmed. The ethical conundrum of how A.I.-powered machines should act in life-or-death situations has received more scrutiny as driverless cars become a reality, but the car manufacturer believes that it's safer to save the life you have greater control over. "You could sacrifice the car. You could, but then the people you've saved initially, you don't know what happens to them after that in situations that are often very complex, so you save the ones you know you can save," said Christoph von Hugo, Mercedes' manager of driver assistance systems. "If you know you can save at least one person, at least save that one. Save the one in the car. This moral question of whom to save: 99 percent of our engineering work is to prevent these situations from happening at all. We are working so our cars don't drive into situations where that could happen and [will] drive away from potential situations where those decisions have to be made."As long as they are better at driving and safety than humans, it is a progress, in my opinion.

Comments

Resiliency in the face of malicious inputs

[Wrath0fb0b]

Saving the occupants of the car is the only choice that makes sense in the context of potentially malicious input. For instance, if Mercedes stated that their car would swerve into a tree instead of hitting a crowd of 5 pedestrians, what's to stop me and 4 friends from jumping out in front of the cars just to laugh as it crashes itself to "save" us.

We have got to start embedding deep into the mind of every software engineer that any information from outside your system can be manipulated to cause maximum damage or disruption. It is your system's responsibility to safely handle malformed and malicious inputs. Until this becomes a common mode of thought, expect more IoT botnets, SQL injections, buffer overflows, DOS amplifiers and the entire realm of "oh crap someone somewhere could be evil, I only engineered for the happy case".

This has to be the way it works.

[olsmeister]

There really is no other logical way to approach this. If they went the other way and prioritized the pedestrian, a psychopath could sprint back and forth across a busy freeway, causing accident after accident and injuring or killing lots of innocent passengers.

Re:The fringe cases are still going to be hard

[Anonymous Coward]

Are you so sure? SUV's are very popular in the US, and they are designed in a way where when they hit a normal car, they hit it further above, where the car is "softer" than below, where there is a crunch zone. Unfortunately the soft part is partly made of the inhabitants of the normal car.

So people already have decided that they like the "crowd" variant and not the "brick wall" one.

Re:The fringe cases are still going to be hard

[AmiMoJo]

What they are really saying here is that the car is designed with passenger safety in mind, and the AI won't even try to consider pedestrians and other drivers. It will just stop as quickly as possible and avoid things that might hurt the occupant, like most humans given a fraction of a second to act on mostly instinct would.

The trolley problem relies on there being sufficient time to make a decision, but not enough to take any other action. It's unrealistic and was only ever intended as a thought experiment.