Slashdot Mirror
When Mercedes-Benz Starts Selling Self-Driving Cars, It Will Prioritize Driver's Safety Over Pedestrian's (inverse.com)
From a report on Inverse: When Mercedes-Benz starts selling self-driving cars, it will choose to prioritize driver safety over pedestrians', a company manager has confirmed. The ethical conundrum of how A.I.-powered machines should act in life-or-death situations has received more scrutiny as driverless cars become a reality, but the car manufacturer believes that it's safer to save the life you have greater control over. "You could sacrifice the car. You could, but then the people you've saved initially, you don't know what happens to them after that in situations that are often very complex, so you save the ones you know you can save," said Christoph von Hugo, Mercedes' manager of driver assistance systems. "If you know you can save at least one person, at least save that one. Save the one in the car. This moral question of whom to save: 99 percent of our engineering work is to prevent these situations from happening at all. We are working so our cars don't drive into situations where that could happen and [will] drive away from potential situations where those decisions have to be made."As long as they are better at driving and safety than humans, it is a progress, in my opinion.
I'm working on self-walking pedestrian Gatling guns. Guess what *it* prioritizes?
"As long as they are better at driving and safety than humans, it is a progress, in my opinion." ?
This is common sense. All the self driving car moral bullshit have simply been some philosophy student trying to prove themself not obsolete by injecting their retarded trolley experiment into reality.
They should make the car use facial recognition on the driver, if you're found to be a philosopher it instantly self destructs to save some victim from having to listen to your ethics moral bullshit memes.
99% of time, the correct action is to stop. If a crash is unavoidable though, if you are solely concerned about the safety of the passenger, then it is safer for the passenger to hit a soft target like a crowd of people than something hard like a telephone pole. The passenger is much more likely to survive hitting a person than a brick wall but a human will usually choose the wall.
S-class & AMG Models: Maximum driver and driver property prioritization.
E-class models: Minor driver prioritization, slightly better than 50/50 odds
C-class models: Pedestrian prioritization
Sounds logical to me. Otherwise, why would I pay Mercedes-Benz to save other people? I am not an altruist and don't inspire to be one in life&death situations.
Why not just make it a configuration setting?
No automatic driving car is better than a human, and that will always be true regardless of transportation method. Auto-anything is a poor substitute for the billions of years of evolution that it takes to make a mouse gazelle run from a lion. The same reason you sci-fi geeks don't see reality here is the same reason the oil tycoons don't see global warming. You both have fantasies you prefer more, and some of you profit substantially at the cost of others whom you don't care about.
as long as long as the plutocrats and their champagne caviar cocktails arrive safely. Somehow I think Marketing and Legal will want to rework that presentation. And BTW, I'm pretty sure Pedestrians (in the US) have right of way in just about every situation. So deliberately programming the car to violate that is saying you don't care what harm you do. Legal may not be happy with that either.
Saving the occupants of the car is the only choice that makes sense in the context of potentially malicious input. For instance, if Mercedes stated that their car would swerve into a tree instead of hitting a crowd of 5 pedestrians, what's to stop me and 4 friends from jumping out in front of the cars just to laugh as it crashes itself to "save" us.
We have got to start embedding deep into the mind of every software engineer that any information from outside your system can be manipulated to cause maximum damage or disruption. It is your system's responsibility to safely handle malformed and malicious inputs. Until this becomes a common mode of thought, expect more IoT botnets, SQL injections, buffer overflows, DOS amplifiers and the entire realm of "oh crap someone somewhere could be evil, I only engineered for the happy case".
There really is no other logical way to approach this. If they went the other way and prioritized the pedestrian, a psychopath could sprint back and forth across a busy freeway, causing accident after accident and injuring or killing lots of innocent passengers.
It has sensors for the car, its pretty much knows how the car will behave. It DOES NOT KNOW HOW THE OUTSIDE WORLD WILL BEHAVE.
So you're suggesting it has some perfect knowledge and will make the decision to kill a pedestrian rather than run the car into concrete and kill the driver. Or similar situation. It could never know if the pedestrian will survive or not, it does not know if they will roll, or jump or otherwise.
It's simple probabilities and simple common sense.
And the best chance for a person to avoid an oncoming car is if the car behaves in a predictable way.
Seriously, makes sense as Pedestrians and Cyclists should be looking out for themselves as part of the activity of walking.
Good to see some thought going into this.
Mimetics Inc. Twitter
Self-Driving Car Ethics
Automakers need to standardize their behaviors or there are going to be a lot of cases of two or more A.I.s with differing goals ending up making a bigger mess than no A.I. involvement at all. A good example is right of way in international waterways. There are rules for how to handle passing (you always pass on the right), crossing another ships path, etc. It makes things a lot safer when you know the other ship is going to behave in a particular manner (I'm talking about professional sailors on container ships, oil tankers, etc... not drunk Uncle Bob tearing up Lake Havasu).
Psycho
If cars prioritized pedestrian safety over that of the driver, I can see a "challenge" developing where the same kind of morons who get burned in those "how much cinnamon can you swallow" games step in front of self-driving cars at the last second to see how close they can come to getting killed and/or how much damage they can inflict on a vehicle forced to avoid them.
They'd probably call it "Bullfighting", or something similar.
I've calculated my velocity with such exquisite precision that I have no idea where I am.
Smart. Case Closed.
If there's malicious input, then the driver is the problem. Stop and lock the car. If the car has been compromised, then any focus on safety must be assumed to also be compromised. Do everything possible to stop and lock the car.
Talk is easy, I'd like to see some example how the self driving car would actually perform in those freak accident situations, especially in cases where it could avoid them by going outside the traffic rules (e.g. dodge a truck by driving into the grass or reversing).
They're trying to sell these things. "This car will kill you instead of running over an idiot who crosses the street without looking if those are the only two choices." Doesn't sound like it would sell well, does it?
This is common sense. All the self driving car moral bullshit...
I agree but the problem here is the law which can rarely be accused of following common sense. A reasonable person would look at the number of lives saved by the car and decide that this was, on average, a very good thing. The law will look at one instance where a life was lost and, unlike a "gut reaction" of human will show that this was a calculated decision (I expect pre-meditated might even be used) to kill a pedestrian and will then sue the manufacturer who has far deeper pockets than the driver.
While the law can be changed I expect most politicians will be very wary about passing a law which might appear to declare open season on pedestrians...or at least I expect that is how the other problem, the media, will present it.
Human drivers prioritize their own lives above the lives of others. So, this is just equivalent on that front. Once the software is better at driving than humans, it will be a net win all-around anyway.
For more than a hundred years, millions of cars have shared the roads, driven by people who prioritize their own safety in an emergency, because self-preservation is part of human nature. Around that, codes and conventions have been built. That assumption is baked in every piece of existing infrastructure and equipment, and it's baked in the way human drivers that will soon share the roads with AIs, react to circumstances and the environment. It would actually be unsafe to turn around that assumption for part of the vehicles on the road.
There really is no other logical way to approach this. If they went the other way and prioritized the pedestrian, a psychopath could sprint back and forth across a busy freeway, causing accident after accident and injuring or killing lots of innocent passengers.
I agree with the first statement but your argument does not hold water because with this priority setting that same psychopath can now just drive back and forth setting up situations in which the car mow down pedestrians. The problem here is that you have a psychopath, it has nothing to do with the decisions made by the car.
For instance, if Mercedes stated that their car would swerve into a tree instead of hitting a crowd of 5 pedestrians, what's to stop me and 4 friends from jumping out in front of the cars just to laugh as it crashes itself to "save" us.
How about the same thing that stops you dropping rocks on cars from a bridge over a road? You know, basic ethics and the consequences of breaking the law. In your example the problem is you and your psychopathic fiends, not the decision made by the car. The best arguments for the self preserving algorithm is that this is what a human driver will instinctually do so it is no worse in causing deaths than a human (and given the far faster reaction time almost certainly far better) and that nobody will ever buy a car that ranks their own lives below that of everyone else around.
What do you expect from the very germans who gassed 7,9 million jews, gipsy and homosexuals?
Anyhow, this is not a vendor choice. These decisions will be put into law by the nation state or the European Union (Mercedes is german and Germany is part of the EU). It will be put into a rule just like speed limits or the min and max blinking pace of the turn signal light. If Mercedes doesn't like what a nation state orders, they are free to go away.
Furthermore, even in US states or other backwards places which allow the Mercedes self-driving logic, the natural right to self-defence entitles pedestrians to carry an RPG-7 for use against ramming wehrmacht panzers, since pistols and other small arms are not effective at stopping a charging limousine.
I'm buying the car that prioritizes my life.
I'm hoping that you'll buy the car that prioritizes my life, too!
This is why ethnocentrism wins.
The side effect of your Mercedes choosing to impact the young mother with her baby stroller instead of the nearby telephone pole (ouch! that could hurt!) is that the customer's testicles fall off, and his dick never rises for the rest of his miserable, injury-free life (female customers sensibly snipped the wires on this pathetic contraction long ago).
The Mercedes survivor can always tell his disappointed women, "not MY fault, the Mercedes made me do it". Mercedes! Modestly dressed women cross themselves. Everyone spits.
All this spit makes the sidewalks dangerous to navigate for the common folk, but we can all rest safe knowing that the privileged remain comfy and cozy inside their steel cocoons.
>> As long as they are better at driving and safety than humans, it is a progress, in my opinion.
Well, in my opinion, everyone seems to be too quick to presume all automated cars are necessarily safer than all drivers.
Its probably actually true for some people in the US at least but not everyone. On my commute I frequently see people (especially women) texting and driving at the same time, even on the freeway. For example on Friday evening in rush hour I saw a lone female Lexus driver (illegally) in HOV lane (illegally) doing about 80mph while (illegally) texting with both thumbs, no hands on the wheel, hardly even glancing forward.
I would suggest that most human drivers' instinct would be to avoid collisions (swerve instinct) and to protect themselves if possible.
Mercedes should have framed this like "we worked with various DoTs and insurance companies and did an analysis of many common human-driver car crash scenarios and analyzed what human drivers typically do, and what the outcomes were. We then engineered our car to try to have similar priorities (and overall outcomes that are at least as good) w/r/t trying to avoid damage to persons and property."
Walking is the most basic form of human transportation and the risks associated with it should be lowest.
When someone choses to get into a vehicle they should understand that they are assuming the risks associated with such a machine. They are getting into something that travels so fast that should it crash, they risk serious injury or death. If self-driving cars prioritize the safety of the occupants over others, then it is transferring what should be that persons assumed risk onto others.
There are comments about malicious intent here too. They say a pedestrian could run into the road to cause a self-driving car to crash. What if someone were to cause an unexpected obstruction in the road so that self-preserving AI cars would swerve to miss it and run over the pedestrian? Malicious intent can work against the pedestrians too. There is no fail safe option to the choice of occupant prioritization or pedestrian prioritization. Again, it is the person in the car that should assume the risks of high speed travel, not the pedestrian.
Ironically, the AI system in the car will likely apply some kind of point system for hitting pedestrians, much like what people have been joking about for some time now.
Who would you prioritize, and why should the others not hate your guts and call you names as a result?
In Soviet Washington the swamp drains you.
If I paid for the car you can be damn sure I want it to prioritize my safety over some outsider's. Particularly since they may have caused the problem themselves (assuming that the car AI has very high safety). Imagine if it was the opposite: anybody could jump in front of the car and laugh as it swerves and crashes into a tree to avoid you...
Non-Linux Penguins ?
Since the occupants of the vehicle will have no input (except possibly as witnesses, but probably worse witnesses that the vehicles instruments and recorders) there will be nobody in the frame for liability except those who were killed or injured by the collision and the organisation who defined the vehicle's behaviour in that situation.
If that court finds there was any way that the vehicle makers could have avoided the "accident", they will assign liability and costs. So we can expect that on the one hand will be the technical, legal andfinancial might of an international, multi-billion-$$$ company - and on the other the grieving (and possibly penniless) family of the injured party. It doesn't take a genius to see which way that "justice" will go (in the USA, at least - other countries will find differently).
However, once a vehicle's occupant is the one making a claim, exactly the same power dynamic will come into play. But this time in reverse: with the company claiming that the occupant suffered because the vehicle took a decision in favour of the safety of others, I guess the case law and the whole future of the self-driving car's legal position will be decided by who wins, what claim, first?
It will come down to a flip of the coin. Sounds like it will be a good time to become a lawyer - just make sure you pick the right side.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
People won't buy a car that will drive itself off a Cliff if someone tosses a human shaped meatbag on the road.
Website Just Down For Me? Find out
A car that will protect the driver over pedestrians is a car that is a weapon designed to deal damage to others in preference to keeping those inside safe.
I foresee lawsuits.
One thing good that this might eventually do; and I'm talking decades here, is allow the US to demand real drivers training and tests rather than the "can you parallel park?" test for which we now give out licenses. People who are actually trained to drive, with skill, are often safe. And by drive with skill I mean skid-pad tests, threshold braking, understand chassis balance, proving you can control a car in a skid, on ice... Etc etc. If you want to be the typical driver in the USA, then all you get to drive is a self-driving car. If you want to actually drive a car then you have to prove you have actual driving skills before you are given a real drivers license. That provides for a good two-tiered system; one for casual "drivers" who are, 99% of the time no more than passengers with a wheel and pedals in front of them, and a second class of people who actually drive.
In my mind, there's probably no right answer, so if we want self-driving cars, we're going to have to live with a reasonably wrong one.
I mean, just consider the following scenario:
Our self-driving car is driving down a straight road.
Suddenly, it detects a tree falling across the road just ahead!
It's impossible to stop in time, so let's fire up the accident minimisation AI.
Hitting the tree has a projected 8% risk of killing the driver, so the car considers swerving off the road.
To the right side is a forest. The car calculates that, at best, swerving into the forest has a 95% risk
of killing the driver.
To the left side is a pile of orphans (long story). The car calculates that swerving into the orphans has
a mere 1% risk of killing the driver, but that doing so is certain to kill at least five orphans, possibly as
many as twenty.
There's not enough time to ask the passenger for input.
How should the vehicle proceed?
Do the specific calculated risks and values provided above change this choice?
(Say, for example, that hitting the tree has a 20% risk of driver death, or that hitting the pile of orphans has a 0%, 7%, or 19% risk of driver death?
Or perhaps the pile of orphans is dense enough to guarantee killing fifty or sixty of them?)
Drivers should assume the risks of their motorized couches whether or not a computer is in control of it. Had they been walking, the issue would not exist. I don't hear many stories of people walking into each other and dying.
The first company to implicitly value a car over a pedestrian. Sick shit here.
Well, no.
Pedestrian makes a mistake : pedestrian is dead.
Driver makes a mistake : pedestrian is dead.
Notice something?
The way I see it is simple, you can't sell a car saying:"allright, if the vehicle that you are paying for ever gets in a situation where it has to choose between hitting someone or crashing itself and killing you, it will kill you..."
Well at least I wouldn't buy that car... I don't know about everyone else...
Presuming the car is only driving where cars should be driving, any situation where the car would need to "sacrafice" the driver for a pedestrian is where the pedestrian has done something stupid, such as stepping out onto a highway.
While I certainly hope the car would avoid even the stupidest pedestrians, quite simply if you step out onto a highway, you should expect to become bug-splatter.
I would be enraged if some one ran out onto the road and my car drove me into a pole avoiding them. I certainly wouldn't buy a car that planned on this. and I would modify (even if illegal) my car if it were mandatory. I would certainly vote any politician out of office who pushed for this.
Some might argue, "What about kids chasing balls?" again, I hope the car would do something reasonable to avoid said child, but given the choice between me and the improperly parented child... well darwin will handle this one.
My theory on this is very simple. At first there will still be accidents caused by driverless cars. But the data gathered by the driverless cars will easily be enough to reproduce those accidents. With each accident, a team of brilliant engineers will pour over the data and figure out how to deal with that accident. It wont only be solved for that accident but the changes will be tested against a zillion hours of difficult data to make sure that the new code doesn't cause new accidents. All cars will then get an upgrade and will now be safer.
After a while the driverless cars will have hit a point where solving for the tiny remaining number of edge cases will actually reduce overall safety. But these edge cases will be so few and far between that they will make national news when they happen. Thus, the stats behind driverless cars will be so extraordinarily safe that for a single human driver to "prove" that they are safer would take lifetimes of flawless driving to prove. This number will only spread as time grows.
I would not be surprised if the experimental driverless cars are in the top 1% of the top 1% of drivers in the world.
The key being that a single driver gets to accumulate only their own stats, while 1,000,000 driverless cars will do more than a taxi driver's lifetime driving nearly every day or so.
Then to make it worse, we meatbags are variable. Even the best driver in the world with a flawless record, might be forced to drive in non-optimal conditions. They haven't slept in a few days, they are sick, and a sudden emergency forces them to drive someone to the hospital. None of that applies to a driverless car.
For instance. I have a 20+ year flawless driving record. Part of that is that I know not to drive when I have not slept well the night before, or any time between 1am and 7am.
One other bit is that driverless cars will soon have some interesting abilities. Things such as gathering data from other driverless cars. Thus if something has dropped onto the highway, one driverless car can alert the rest about it. Or if one car hits a surprising icy slick, it will not only notify other cars, it will start to build a pattern of when that road is icy. These would be the few initial areas where people would be better; where you know that a certain intersection seems to have ice on it in these conditions. Or observing the behavior of other cars to possibly be alerted to problem such as trash on the road.
They will be able to easily discern true Aryans from untermensch and follow the rightful Nazi laws.
Everyone seems to be of the opinion that all automated cars will be safer than most human drivers eventually. I don't think anyone really thinks that they're safer than even a somewhat-below-average human driver yet
These are the comments? Blame pedestrians, imagine pseudo-logical scenarios blinded by lack of self-awareness, and the apparently in-vogue 'fuck everybody else'? May God save your souls before the devil welcomes you all into hell.
"As long as they are better at driving and safety than humans, it is a progress, in my opinion."
I'm not convinced. Right now, when people die in car crashes, and I can blame a human driver for something, then it's totally understandable. When humans die by the hands of other humans, and especially through the errors of other humans, that's just a reality that I can comprehend and accept.
But when a self-driving car is ultimately responsible for killing a human, that's a different thing entirely. That's a lot closer to just humans-get-killed-at-random scenario. That's not something that I can accept.
It's actually even worse than that. It's like a neighbourhood pet dog kills a neighbour. If your typically-well-behaved-and-friendly boxer suddenly kills your neighbour's teenager one day, what happens? Look, your dog killed one neighbour over the course of thirty years of you owning dogs. Most wild animals are far more dangerous than that. But I think we all know what happens. I think your dog is dead pretty quickly -- even if that teenager provoked your dog; even if it was a lot; even if your dog was defending its own life.
I accept, today, that millions of humans driving millions of cars on millions of roads, kills thousands of people every year. I'm not happy about it, but I accept it as a part of humans being free to not be perfect. But I don't think that I'd be accepting of millions of self-driving cars on millions of roads, killing dozens of people every year.
What happens when someone uploads the death race 2000 virus into the OS?
Most instances where you have kids "jumping out in front of cars", you're talking about parking lots or side roads where the kids didn't look both ways before trying to cross the intersection. It's not something that should really ever happen on a highway or interstate where you've usually got fences, a thick tree line, or other things blocking easy access to get onto the highway by way of walking up to it.
The car will probably try to slow down as much as possible, if not come to a complete stop. At most - that would bump the kid(s) but not seriously injure them. Perhaps it would even turn to avoid them while slowing down, since it would know it wasn't going to flip the vehicle at that relatively slow rate of speed.
MB Builds cars, which they sell to passengers. If MB announced that pedestrian safety was paramount over passenger safety, how many people would borrow tens of thousands of dollars to buy a MB vehicle only to be put at greater risk than someone that didn't buy a car.
I can just imagine the advertising campaign: "Everyone is more important to us than our customers."
Ken
That software is better at driving and safety than humans. The kool-aid around this everyone is drinking seems to be made of crack.
This is a bad decision. The pedestrian has no protection from an impact with a car. But, the driver has protection from an impact like seat belts and air bags.
Have gnu, will travel.
Because if they do, they will find that they cannot get their cars legally on the roads or insured. But what they will to is lie to their customers to make them feel safe. As the incidents where this will be a concern are rare enough, nobody will notice for a long time. After that, a balanced solution that minimizes overall damage (the only really defensible approach in a society with equality of its citizens) will be generally accepted and they can admit that they did that all along.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Funnily enough, drivers tend to buy more cars than do pedestrians.
Who is going to buy a car that puts the life of somebody else above your own? How would that conversation go at the dealership?
Salesguy: Yeah, the autopilot is great. Especially for pedestrians.
Buyer: The autopilot sounds gr... wait, what? What was that about the pedestrians?
Salesguy: Oh, if it thinks it is about to run over a pedestrian, it will swerve off of the road. Possibly off a bridge or into a solid object like a tree or something, possibly killing you in the process. Makes thing safer for pedestrians.
Buyer: Well, what about me? Why am I being sacrificed? Shouldn't it just come to a stop as fast as it can and try to save us both? After all, I'm the one that paid for the car.
Salesguy: But it might not stop in time and kill the pedestrian. We can't have that.
Buyer: I'm, uh, going to go check out the next dealership. Bye!
Algorithms don't deal in philosophy or value judgments. They deal with things like angles, speed, and the car's capabilities.
The scenarios are so contrived it doesn't even make sense to argue them. Mostly, they are based on reckless driving scenarios, which self-driving cars won't engage in in the first place. By human standards, they will be overly cautious. These so-called moral dilemmas won't actually happen in such a way that the car can make such a decision. Any real-life decisions of this type are likely to be beyond the ability of the car itself to make a rational choice, it will usually just have to hit the brakes.
Say a self-driving car passes in front of me in a crosswalk, and I kick its side. Is it going to do anything funny? I might just make it routine if something funny happens.
The driver and passengers are protected by chassis and safety devices, the pedestrian is fully exposed.
Wrong decision Mercedes,
crash the car, you can buy another,
the occupants will probably be safe,
save the pedestrian
Does this make the programmer responsible for the pedestrians death?
Go well
A brutal brake to avoid a pedestrian may become a medium threat to the passenger. Will you reduce the strength of the brake at the expense of hitting pedestrians to avoid hurting passengers ? We are handling probabilities, no absolute certitudes.
Meanwhile, AI at competetior BMW is being trained using GTA ...
Pedestrians have to be watchful of mistakes by others, because their lives are at stake.
...I was thinking that the attitude displayed by Mercedes was more like a BMW driver's attitude.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
How do you know this? You're making the same large generalization.
>> I would not be surprised if the experimental driverless cars are in the top 1% of the top 1% of drivers in the world.
Keep drinking the koolaid. Self driving cars have had plenty of accidents, such as these:
http://money.cnn.com/2016/02/2...
And they can't drive at faster than 25mph.
I drive consistently faster than the speed limit and haven't had a car accident in the last 30 years or so. Does that make me better than the 1% of the 1% (presumably of that 1%)?
So the car manufacturer decided to prioritize the guy giving them money instead of the guy that doesn't even have a car.
Shocker!
If you think about this, there really isn't another option. Would you buy a car that wasn't going to prioritize your life in the event of a crash?
Save the lives of the people that can afford to buy our product.
Makes sense.
MB's brand had always been safest car on the road for its passengers. That brand is HUGE for MB.
Hence like any corporation, who cares about its customers . To protect that brand, remove legal liability from the owner suing MB, and keep their products simple but not worrying about pedestrians, and all more cars.... Having logic of AI driver over anyone else is... logical... for them.
Is it right for transportation and the public is another question.
...can we back up and talk about the almost innumerable variables that would have to go into the decision here? One passenger - one pedestrian = relatively short lost of things to consider. One passenger and ten pedestrians = not so much. How is the priority weighed? Is kinetic energy transfer factored into the decision? I will wager that it is not, nor are most of the variables that would affect outcomes. So just what is Mercedes talking about?
come on, at least make a half assed effort.
They do this for liability. Kill the driver, and the estate can not sue the software company, because the software was sold WITHOUT WARRANTY, and WITHOUT FITNESS FOR A PARTICULAR PURPOSE.
The max the driver's estate could get out of the software company, is a refund for the software, creating a huge burden on the insurance company.
Now, kill the pedestrian, and the driver is at fault, even for a driverless car. The software company can't be sued, and even if they could be, the max they could get is a refund for the software.
The pedestrian's estate could sue the drivers insurance company, and the driver, and could bleed them both dry.
What am I supposed to do with this insightful information?
they just say "buy our cars and we will kill others not you..."