Slashdot Mirror
Autonomous Car Ethics: If a Crash Is Unavoidable, What Does It Hit?
An anonymous reader writes "Patrick Lin of California Polytechnic State University explores one of the ethical problems autonomous car developers are going to have to solve: crash prioritization. He posits this scenario: suppose an autonomous car determines a crash is unavoidable, but has the option of swerving right into a small car with few safety features or swerving left into a heavier car that's more structurally sound. Do the people programming the car have it intentionally crash into the vehicle less likely to crumple? It might make more sense, and lead to fewer fatalities — but it sure wouldn't feel that way to the people in the car that got hit. He says, '[W]hile human drivers may be forgiven for making a poor split-second reaction – for instance, crashing into a Pinto that's prone to explode, instead of a more stable object – robot cars won't enjoy that freedom. Programmers have all the time in the world to get it right. It's the difference between premeditated murder and involuntary manslaughter.' We could somewhat randomize outcomes, but that would lead to generate just as much trouble. Lin adds, 'The larger challenge, though, isn't thinking through ethical dilemmas. It's also about setting accurate expectations with users and the general public who might find themselves surprised in bad ways by autonomous cars. Whatever answer to an ethical dilemma the car industry might lean towards will not be satisfying to everyone.'"
I'm reminded of Michael Sandel's televised series on ethics.
If you could stop a runaway train from going over a ravene, by pulling a lever, thus saving 300 people, but the lever sent the train down a different track on which 3 children were playing, what do you do?
Somehow, involving innocents seems to change the ethical choices. You're no longer just saving the most lives, but actively choosing to kill innocent bystanders.
Just run the car into the nearest programmer.
Why is Snark Required?
Slam the brakes on and don't swerve either way. It's by no means optimal, but as far as lawsuits are concerned, it's much easier to defend "the car simply tried to stop as soon as possible" than "the car chose to hit you because it didn't want to hit someone else".
Let's be honest. The job of YOUR car is to keep YOU safe, so the smaller car is probably the better bet as it will have less inertia and cause you less harm. Sure, the most important law of robotics is to protect human life... but if it's going to prioritize, it should probably start with its owner.
In Soviet Russia, dot slashes YOU!
Why do poeple always give such easy examples when asking this question?
Of course you save the 300 people! There's probably a lot more innocent people than 3 in that group of 300... You'd have to be very stupid to save 3 over 300 or too lazy to think about it and you make a random decision.
The question should be more like this:
On one track there's 10 escaped criminals and the other is your wife with son and another child in the belly.
That's a decision you might have to think about, but most people would easily save their own wife.
In my opion this shows most people are not ethical at all. So when someone asks you this question they pose it extremely in favor of sacraficing the innocent to make certain people will make the 'ethical' decision.
This notion kind of cropped up in last weekend's episode of "Continuum" where a next of kin was informed of a crash by an actuary in terms of write downs, compensation, loss adjustments and so on. Given the way insurers tend to operate and how in bed they are with the legal profession I can see that's exactly how this would go in the long run; an evaluation designed to produce the lowest price tag for those that ultimately get to pay the financial/legal bill. Looking at the problem another way, that means the structural integrity of the two cars in the example is probably moot; if the more structurally sound car is an expensive vehicle with a lone occupant owning a huge life insurance policy and the other is a decrepit bus full of uninsured kids, then it's probably not a good day to be one of the kids... or the driver of the car that crashes into them.
UNIX? They're not even circumcised! Savages!
What if one car has two guys with multiple convictions for armed robbery and the other has a working dad with a family and three kids at home? OK, the algorithm would have to be pretty sophisticated to detemine that, but who knows...
Or something slightly more realistic, a car with an couple of 80 year olds versus a 25 year old mom of three? Should the car kill the mom rather than the couple that will be dead in less than 10 years? One death is worse than two, no matter what?
Or yet another one, what if two people cross the street without looking, and the car swerves off the road to avoid them and rather kill one person who was walking on te pavement, not doing anything wrong? One casualty is better than two, right?
Those are just questions, mind you. Only shows how "minimize casualties" is not always so clear cut.
Congratulations, you've given me a great go-to example of a non-answer.
Just leave that kind of behavior undefined.
Programs are generally deterministic beasts, by nature. What are you trying to say?
There are very few "accidents" just people taking stupid risks. Maintain a safe distance, ie enough manouvering room so you don't join an accident, don't overtake when you can't see the end of the manouvere e.g going up hill or on a bend. Stop when necessary. Procede with caution sometimes you might want to turn off the radio open a window and listen. Use your indicators. Drive within your lights or as conditions allow. Don't be an asshole.
Sometimes you will come across assholes on the road it is best to give them a wide birth even stop and pull over in order to get them out of your way, but don't dawdle if you want or need to drive slow make opportunities for people to overtake.
Bad planning and poor judgement are the most common causes of accidents which is why schools have low speed limits around them as kids can be stupid around roads.
Be helpful, I remember one time I was filtering down the centre line on a motorbike (dispatch rider) past stationary traffic and a taxi driver stuck his hand out. I braked and a pushchair popped out from between the stationary traffic. Without that warning I could have killed a toddler as it was no harm was done and I don't think the mother was ever aware of the danger.
One thing about london traffic professional drivers work the streets most of the day and they are very road aware. The most dangerous times are when schools start and when schools let out, followed by the rush hours when the non professionals are on the road.
Blarney Quality Restaurant, Plants
"Programmers have all the time in the world to get it right". HAHAHAHAHAHA.
No, we have deadlines like everyone else. And even then we only have all the time in the CPU. Yeah, we can add more CPUs to the system, but that makes it more complex, and that makes it harder to hit that deadline. What kind of idiot made that statement?
Why not? The simplest method is to choose the collision with the lowest speed differential. In fact, this whole post is pointless. The self-driving car doesn't need to choose based on abstract concepts--choose the collision with the lowest speed differential. Lower speed differential means less energy transferred in the impact means less damage and less injuries. Moreover this is trivial for the cars to determine at this stage already. They can already calculate relative speeds between themselves and other objects, so if not all of the objects can be avoided, the choice is obvious.
While a complex guidance system may be designed from the top down with such sorts of questions raised, a crashing vehicle is always a deadly weapon. Effort in reducing the risk of the accident, itself, by improving brakes, sensors, headlight effectiveness, and crash resistance of the vehicle itself is likely to be far more efficient and reliable than complex advance modeling or moral quandaries. The sophistication needed to evaluate the secondary effects of a crash is far, far beyond the capabilities of what must be a very reliable, extremely robust guidance system. Expanding its sophistication is likely to introduce far more _bugs_ into the system.
This is a case where "Keep It Simple, Stupid" is vital. Reduce speed in a controlled fashion: Avoid pedestrians, if they can be detected, because they have no armor. Get off the road in a controlled fashion.
definitely, a cat, I hate them.
So you would have it choose to mow down the stationary infant in its stroller as opposed to tapping a parking pickup truck backing up at at 10 MPH?
The problem with his original question is that he assumes the self-driving car has knowledge of the type, mass, and vulnerability of things around it. This might be the test case for the three laws of robotics - do not ever choose to hit an unprotected human (probably includes motorcyclists, bicyclists, and pedestrians). If you know (by a beacon or whatever) that a vehicle is completely autonomous and does not contain humans and has comparable delta-V, give that preference. If hitting a vehicle likely containing a human is inevitable, choose the lowest speed impact.
Tiller's Rule: Never use a word in written form that you've only heard and never read. You will end up looking foolish.
The car would of course make an online crosscheck to the economic value of the potential targets. And check their medical records in case somebody is terminally ill, you yourself included if a wall is an option too.
I, for one, would start car pooling with lots of small children inside. With a big enough critical mass of children I would even qualify for green lights, just for me!
That said, you can calculate how fast the politicians would add "features" (like with ISPs and mandatory website filtering) which would automatically upload a secret white lists and black lists into your car.
I am guessing here:
White list: Nobel prize winners, The Pope, politicians and multinational CEOs.
Black list: The no-fly list from the US.
I wonder if we would be allowed to make a personal priority list for your own car. For example, to take out mimes and lawyers first.
That is not possible. I can see it in the court case. You had tge capacity to choose, yet you chose not to choose and my daughter is dead.
But, the simple reality it, that will happen anyway, no matter what decision is made. ("You chose to minimize the probability of X, and now my daughter is dead.")
I don't, by the way, buy the "Programmers have all the time in the world to get it right" bit. Programmers will not be able to anticipate everything, and their software will not always be able to calculate everything in the few milliseconds or so you might have to make such decisions.
This is a weird segue, but which car does it hit? The more expensive car with better insurance, or the cheaper car that explodes?
Will you be able to buy "don't choose me" premiums?
How will this affect emergency vehicles?
"Helping to keep you two steps ahead of the Thought Police!"
There's no such thing as an intentional accidents. An autonomous program that is paying attention will not have such a situation and therefore the manufacturers will always be responsible for failure.
If a car shoots out from a blind junction at speed and you can't stop in time, that's an unavoidable accident - the car could not be seen in advance, so the autonomous program couldn't have avoided the accident even if its paying attention the whole time. You could argue that you should be going slow enough that your stopping distance is short enough to avoid the collision, but on a lot of roads this would seriously hinder traffic flow - at some point you just have to trust that other drivers are following the rules of the road and accept that the risk can't be completely eliminated.
Similarly, mechanical failures can't always be predicted - you're overtaking someone and their wheel comes off causing them to swerve into you. Impossible to predict so now you're left trying to reduce the seriousness of the inevitable accident. Hell, your own car may have a mechanical failure that the computer couldn't detect.
http://blog.nexusuk.org
Since when do trees move faster than children?
Usually save the coredump and reboot the machine if necessary. Some clueless windows developers insist on powering off, power off the router, unplug the router and wait for the capacitors to discharge before rebooting them all.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
One thing I believe was not mentioned in the article (though I only quickly scanned it) is that if such cars start behaving too predictively, they can be gamed. Once we know that a car will do whatever it can to avoid a collision with a pedestrian, it will be extensively gamed; cars will be tricked into doing stupid things.
So when the decision who to hit comes up, the only way to be reasonably safe is to determine who's not following the rules and to hit that one. Any other rules will be gamed extensively. This will become a major hassle to adoption of autonomous vehicles; they will probably need to drive much slower than actual humans to avoid getting into such situations continuously, especially in built-up areas where any parked car could hide an annoying car-bully trying to trick your car into acting like an idiot.
0x or or snor perron?!
Assuming a collision is unavoidable, and the choice are Car A or B, it's not just a matter of choosing one or the other car to hit.
The logic should be actively working to avoid collision until the last second. The car cannot anticipate what actions the other vehicles may take. Until the actual collision occurs, maintain efforts to minimize the velocity and/or angel of collision. Better to hit the little electric car at 15 MPH after continuing to brake than to have hit the sturdy Escalade at 40 MPH.
Additionally, are there not some foundation rules that apply? We're taught that when in doubt, try and stay in your own lane, because hitting a car that suddenly pulled out in front of you is "less bad" than swerving into another lane and hitting a car that was obeying all of the rules. The basic scenarios need to be worked out and applied as much as possible. (not to mention the whole "oncoming car will be a much worse accident than a car traveling in the same direction as you are but at a different speed" scenario)
I think the scenario being postulated is a bit simplistic and meant to drive an ethics debate for attention. In reality this should be about improving the programs to the point of making the right choices based on more common sense rules than those proposed.
Warning: Teh poster of this messaeg is lysdexic
Speed *differential*. So the answer is when the child is moving the same direction as you are. This isn't rocket science.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
"Patrick Lin of California Polytechnic State University explores one of the ethical problems autonomous car developers are going to have to solve: crash prioritization.
http://en.wikipedia.org/wiki/T...
It is not a new notion, and the ethics of it have been more or less resolved and understood for quite some time. So I fail to see why this is new.
It's important to keep in mind that when such crashes happen, the programmers/manufacturers/insurance companies won't have to defend them to a committee of ivory-tower utilitarian philosophers. They're going to have to defend them to a jury made up of ordinary citizens, most of whom believe that strict utilitarian ethics is monstrous sociopathy (and probably an affront to their deontological religious beliefs as well). And of course, these jury members won't even realize that they are thinking in such terms.
Thus, whatever the programming decisions are, they have to be explicable and defensible to ordinary citizens with no philosophical training. That's why I agree with several other commenters here that "slam on the brakes" is the most obvious out. It's a lot easier to defend the fact that the car physically couldn't stop in time than to defend a deliberate choice to cause one collision in order to avert a hypothetical worse crash. This is especially true since a well-designed autonomous car drives conservatively, and would only be faced with such a situation if someone else is doing something wrong, such as dashing out into traffic right in front of the vehicle at a high rate of speed without looking. In any other situation, the car would just stop before any crash with anything took place. If you absolutely can't avoid hitting something, slamming on the brakes makes it more likely that at least you hit the person who did something to bring it on themselves, rather than one who's completely innocent.