Slashdot Mirror
The Problem With Self Driving Cars: Who Controls the Code? (theguardian.com)
schwit1 writes with Cory Doctorow's story at the Guardian diving into the questions of applied ethics that autonomous cars raise, especially in a world where avoiding accidents or mitigating their dangers may mean breaking traffic laws. From the article: The issue is with the 'Trolley Problem' as applied to autonomous vehicles, which asks, if your car has to choose between a maneuver that kills you and one that kills other people, which one should it be programmed to do? The problem with this formulation of the problem is that it misses the big question that underpins it: if your car was programmed to kill you under normal circumstances, how would the manufacturer stop you from changing its programming so that your car never killed you?
Not only that, another big problem with the "trolley problem" is that it doesn't pass the "could a human driver do better" test.
It assumes that you have lost control of the vehicle to the extent where you can only select between two choices. While every driver will claim that they are superior it is all just bullshit and they will be unable tho make a choice at all in those situations.
The main point of automatic drivers are to not get into or cause situations where you don't have control of the vehicle and that includes keeping the speed limit and being more aware of the surroundings.
When the automatic driver comes into a situation where a "hard choice" would have to be made then it is past the point where the human driver would already have flattened that teleporting, wheelchair bound, ball playing child that suddenly appeared in front of the car.
Who controls the code? Maybe you, maybe them. If you tamper with it, you're responsible. Otherwise, they will probably be responsible for its behavior. But the computer is not going to be "programmed to kill you", that is bollocks. The computer is going to be programmed to follow the law. That means that it's going to be less likely to be at fault in an accident to begin with, that it's going to be more likely to successfully mitigate the accidents it does get into, and it means that rather than being programmed to kill you, it's going to be programmed to stay in the lane and hit the truck rather than swerve and hit the pedestrian because to do otherwise would be illegal — not just because of the pedestrian, but because of the lane marking. That is not remotely the same thing.
The car will be programmed to do its best not to kill you, and that's going to take the form of yielding gracefully to fuckheads rather than getting in an accident to begin with.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Exactly: there is never a such clear-cut decision between "my owner or that bus full of nuns". I can imagine that cars will be programmed to make decisions based on certain basic principles, but those principles will be nothing like the 3 laws of robotics. Taking drastic action to avoid an accident may lead to a worse one, and it'll be a long while before our machines will be anywhere near able to make such complex decisions.
To begin with: if everyone sticks to the rules of the road and drives normally, there is very little chance of an accident occurring. If an exceptional situation occurs, the fault of an ensuing accident primarily lies with whomever caused that exceptional situation (even if it's unintentional). If someone's tyre blows and they swerve into your lane as a result, if a child chases a ball into the road, or if a cyclist runs a red light in front of your car, you'd (probably) do everything to avoid a crash and so should a self-driving car, but you are not under any moral obligation to drive yourself into the side of a building in order to avoid the other car, child or cyclist. Self driving cars should operate under the same premise: it should never be considered necessary to sacrifice the driver.
If something unexpected happens, cars might follow a protocol similar to this one:
1) Stay in your lane and come to a controlled stop.
2) If a controlled stop will not prevent a collision (and this is something that self-driving cars should be able to assess fairly accurately), change to a different lane if there is an unobstructed one.
3) If there are no unobstructed lanes but the road ahead is clear and the local speed limit is below x, change into oncoming traffic.
4) If all else fails, reduce speed as much as possible and allow the collision to happen.
These are not meant to be complete and valid for all situations, it's just to give an idea of how such "laws" could be formulated, in the form of a decision tree that self-driving cars would be able to follow, without having to make complex judgment calls or difficult moral decisions. And I can well imagine that a basic set of such rules will be set into law so that all self-driving cars will follow the same basic protocol. As a driver you'd have little incentive to change the programming in your favour, and if you did, it would become immediately apparent as soon as you're involved in an accident.
If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
if your car has to choose between a maneuver that kills you and one that kills other people, which one should it be programmed to do?
How about you tell us what should a HUMAN driver choose in a similar situation first, before you ask what should a computer do?
These kind of stupid questions are well, stupid. And they come up often simply because there is no real valid worry about autonomous cars. Humans make lots of mistakes and having a computer drive would remove a whole range of avoidable accidents. Worrying about a few boundary cases is as stupid as all the "what if my car is burning and I need to get out quickly?!" objections to wearing seat belts. It is unfounded fear that is not based on facts.
Oliver.
How much sleep have you lost over the engineering decision to make trains so large and heavy that the simply CAN'T stop for pedestrians and other vehicles. Yeah, I thought so. People will kvetch about how self driving cars are programmed right up until they become every-day objects an after that they'll be just as accepted (benefits AND dangers) as trains are today.
I'd also have it sound the horn and flash the lights.
And this is, IMO, exactly what will happen.
Remember, these vehicles have complete records of everything that is happening around them at all times. Everything that can be recorded, that is. So the insurance companies will have exact records of how the robot was 100% within the law AND had taken every REASONABLE response to mitigate the collision.
The robots do not have to be 100% at determining whether your life is worth more/less than someone else's. They just have to be 100% at showing that they were following the law and attempting to avoid the collision.
The legal system and the insurance companies will sort out the rest. And the insurance companies will pay to have the legal system write the new laws to reflect that.
If a child can run into the street from a blind spot faster than you can break, you're driving too fast. The autonomous car will not drive too fast to break in such a situation.
If the child is deliberately hiding in a place it shouldn't be, near a higher speed road and manages to quickly cross what should usually be a wide clear area around such roads (or in an unpopulated area), then there won't be time for a human to react at all. An autonomous car could probably cut down the speed a bit, but avoiding people who deliberately try to throw themselves in front of traffic simply isn't doable or even something to care that much about. You're not going to be able to avoid jumpers, or the human cannonball either, nor is an autonomous car. As the other reply pointed out, whether it's wildlife or people, usually the best option is to simply do a predictable controlled break unless it's basically a slow-motion situation playing out over many seconds (such as road conditions making braking very ineffective, while speed isn't particularly high).
What you are taught when you learn how to drive is "always brake, never swerve". Which also happens to be exactly what a self-driving car would and should do in an emergency situation.
Reformulated as a trolley problem, this is: do not pull the lever.
Why? Because swerving will almost always put the car in more danger for itself and others, since it may cause the car to spin, or approach cars in incoming lanes or pedestrians on the footpath, or any number of bad things.
Since, by law, other cars must maintain a safe distance behind you, slamming on your brakes is always a safe response to an obstruction. Any other response, such as changing lanes, should be considered if it is safe or not. If it is unsafe, then you shouldn't do it, things like "busses of schoolchildren" is completely outside the parameters of the problem.
This whole self driving car issue took a wrong turn the day that someone had decided that they had the moral imperative to break the law in order to prevent crashes. You do not have that imperative yourself, you have the imperative to follow the law in order to prevent crashes. The reason that two drivers can pass each other at enormous speeds without prior communication is that the law forces them into their lanes. The law is the protocol that all drivers follow to interact, since they are unable to talk to each other and as soon as you break it, you expose yourself and others to untold danger. If the law says "don't leave your lane" and best driving practices says "don't swerve", then you don't swerve.
When Argumentum ad Hominem falls short, try Argumentum ad Matrem