Slashdot Mirror
How Do We Program Moral Machines?
nicholast writes "If your driverless car is about to crash into a bus, should it veer off a bridge? NYU Prof. Gary Marcus has a good essay about the need to program ethics and morality into our future machines. Quoting: 'Within two or three decades the difference between automated driving and human driving will be so great you may not be legally allowed to drive your own car, and even if you are allowed, it would immoral of you to drive, because the risk of you hurting yourself or another person will be far greater than if you allowed a machine to do the work. That moment will be significant not just because it will signal the end of one more human niche, but because it will signal the beginning of another: the era in which it will no longer be optional for machines to have ethical systems.'"
I maintain that you CAN'T really program morality into a machine (it's hard enough to program it into a human). And I also doubt that engineers will ever really be able to overcome the numerous technical issues involved with driverless cars. But above these two problems, far and away above *all* problems with driverless cars is the real reason I think we'll never see anything more than driver *assisting* cars on the road: legal liability.
To put it bluntly, raise your hand if YOU want to be the first car manufacturer to make a car for which you are potentially liable in *every single accident that car ever gets into*, from the day it's sold until the day it's scrapped. Any takers? How much would you have to add onto the sticker price to cover the costs of going to court every single time that particular car was involved in an accident? Of defending the efficacy of your driverless system against other manufacturer's systems (and against defect, and against the word of the driver himself that he was using the system properly) in one liability case after another?
According to Forbes, the average driver is involved in an accident every 18 years. Let's suppose (and I'm sure the statisticians would object to this supposition) that that means that the average CAR is also involved in a wreck every 18 years as well. Since the average age of a car is about 11 years now, it's not unreasonable to assume that a little less than half of all cars on the road will be involved in at least one accident in their functional lifetimes. And even with the added safety of driverless systems, the first model available will still have to contend with a road mostly filled with regular, non-driverless-system cars. So let's say that a good 25% of those first models will probably end up in an accident at some point, which will make a very tempting target for lawyers going for the deep pockets of their manufacturers.
Again, what car company wouldn't take that into account when asking themselves if they want to be a pioneer in this field?
What political party do you join when you don't like Bible-thumpers *or* hippies?
Asimov already solved this problem for us.... the Three Laws of Robotics.
Talk about redundancy, is the author's next piece going to be about changing the value of pi?
sudo make me a sandwich
Ahh, Vimeo isn't blocked. Here's the link http://vimeo.com/21216091
The proper sequence should be:
Humans reason (with their morals) --> Humans write laws/code --> The laws/code go into the machines --> The machines execute the instructions.
Laws are not a substitute for morals; they are the output from our moral reasoning.
1. Train an expert machine on decision making with answers from religious and political leaders who set all our definitions of right and wrong.
2. Do the opposite of what that machine decides.
How Do We Program Moral Machines?
Ideally, not at the last moment.
Ethics are a matter of conscious decision-making. Until we have conscious machines, we will not have ethical machines. What Marcus is writing about is the application of ethics in the design of machinery, which is a growing topic in its own right, but not nearly as click-inducing (or alliterative) as is 'moral machines'.
So if my auto-driver car had to make a choice between my safety and that of someone else, it better choose me.
We can't even decide what is morally correct between ourselves as human beings. Take abortion for example...
No competent engineer would even consider adding code to allow the automated car to consider swerving off the bridge. In fact, the internal database the automated car would need of terrain features (hard to "see" a huge dropoff like a bridge with sensors aboard the car) would have the sides of the bridge explicitly marked as a deadly obstacle.
The car's internal mapping system of drivable parts of the surrounding environment would thus not allow it to even consider swerving in that direction. Instead, the car would crash if there were no other alternatives. Low level systems would prepare the vehicle as best as possible for the crash to maximize the chances the occupants survive.
Or put another way : you design and engineer the systems in the car to make decisions that lead to a good outcome on average. You can't possibly prepare it for edge cases like dodging a bus with 40 people. Possibly the car might be able to estimate the likely size of another vehicle (by measuring the surface area of the front) and weight decisions that way (better to crash into another small car than an 18 wheeler) but not everything can be avoided.
Automated cars won't be perfect. Sometimes, the perfect combination of bad decisions, bad weather, or just bad luck will cause fatal crashes. They will be a worthwhile investment if the chance of a fatal accident were SIGNIFICANTLY lower, such that virtually any human driver, no matter how skilled, would be better served riding in an automated vehicle. Maybe a 10x lower fatal accident rate would be an acceptable benchmark?
If I were on the design team, I'd make 4 point restraints mandatory for the occupants, and design the vehicle for survivability in high speed crashes including from the side.
Screw the bus.
I don't care about the bus.
The bus is big and likely will barely feel the impact anyway.
I care about the fact I don't want to die.
Why would buy and use a machine that would choose to let me die?
And I posit that the author has failed to consider freedom of travel, freedom of choice, and other basic individual rights/freedoms that mandating driverless cars would run over (pun intended).
The guy who said the election was rigged won the presidency with the second-most votes.
Zeroth law problem.
Depending on how many other "someone elses" there are. And possibly on an overall Human Value Score brought to you by TransUnion, Experian, Facebook, Google, and Microsoft, weighted by your Medical Insurance Information Bureau records - and theirs.
You, sir, are a good example of why driverless cars will make me safer.
[Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
Wait. If the driverless car is so damn great, how did it let itself get into a situation where the only options are to hit the bus or drive off a bridge? I can make that kind of mistake on my own, thanks. I expect automated cars to avoid this kind of situation, else why bother having them?
Why, without your clothes, you're naked, Miss Dudley!
That probably came across as nastier than I wanted to be. :-( You probably haven't thought through the same scenarios I have -- for example, a group of pedestrians is crossing the street illegally and your choice is to plow through them or smash into a parked car at low speed which probably won't hurt you. For most people, that's an easy choice to make.
[Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
Or just imagine the pranks by people jumping in front of cars to watch them veer into a lamp post. Even better, pick a narrow bridge, then three people jump in front of a car. Perfect murder!
It's a choice the human driver would have to make, so when first starting your driverless car, it might as well prompt you with a series of moral questions like "should I crash into a bus or veer off a bridge if the situation arises?"
Human drivers don't make these decisions in any moral way in the real world, so why would the program in anything into a car?
Split second decisions are involved in any accident situation, or, the lack of the ability to decide, resulting in the default.
Nobody ponders the morality of the situation when their life is on the line. Its all instinct from that point.
Sig Battery depleted. Reverting to safe mode.
I'm going to disagree with this assertion about morality:
it would immoral of you to drive, because the risk of you hurting yourself or another person will be far greater than if you allowed a machine to do the work
The first charge is that this would be an immoral risk to take because you might hurt yourself. In my understanding of morality, it is up to each individual to decide for themselves which risks and consequences and injuries to themselves are immoral. For example, I would not go skydiving, but other people choose to do so. They are taking a risk I choose not to take, but I do not think they are immoral for taking the risk, and I do not think an increase in the magnitude of risk alters the morality of the situation, because they are risking themselves. As another example of higher risk, some people choose to try to circumnavigate the globe on solo fights or boat trips. This is a huge risk; some people have perished in the attempt. But the fact that they were risking serious hurt to themselves does not render their decision immoral.
The second charge is that you are risking hurting another person. But again, this is their risk to take. They decide to travel on a road that includes other human drivers knowing that doing so incurs some risk of injury. Taking that risk is not immoral. As an analogous example, wrestlers or boxers choose to fight each other knowing that there is a risk of injury to each other, but doing so is not immoral because the risk is voluntarily accepted by each participant.
Ideally, travelers could choose between a variety of competing travel arrangements, including roads that might choose to exclude human drivers for the safety of travelers, or roads that choose to allow them for those who desire to take that risk. What would be truly immoral would be to forcibly monopolize some or all of the transportation options, so that people do not have the freedom to create differing transportation alternatives that compete with one another. This would limit the choices of travelers such that some might have to take risks they do not want (e.g., roads with both human and automated drivers, because pure-automated roads are not available), or cannot choose to take risks that they find rewarding, such as choosing to drive when automated drivers are available.
Dr. Walter Block has written an entire book on how the American highway system is currently subject to this kind of immoral forced monopolization, currently causing 40,000 needless traffic fatalities per year, and how the elimination of this immorality is entirely practical and beneficial.
Secession is the right of all sentient beings.
Google operates cars without human drivers in several states.
Google has insurance.
In 18 years, (or some statistical appropiate number given the number of data points) we can examine the operational history of these vehicles and compare to human drivers in the same geographic areas.
For every benefit you receive a tax is levied. - Ralph Waldo Emerson
What I find particularly worrying is that, at least initially, many of the ethical choices programmed into these machines will have been written by people who tend to be heavy on the Aspergers side of empathy (as many technically inclined people are). Should we really be leaving decisions like this to people who literally can't understand how most of humanity behaves?
The actual cause of most accidents can be boiled down to one simple rule that is broken. That is "failure to yield". Most people drive like complete dicks because they think they are more important then everybody else on the road. The driverless system will not be driving like that in the first place so it will hardly ever get into a traffic collision and even when it does it will minimize the damage because its responses are not emotional.
Limited Liability Car!
(spoilers, if you've never read Asimov)
Unlike the horrible movie, the book "I, Robot" was a series of short stories dealing with the ambiguity of the laws. (The movie was more some bizarre combination of "free the robots!" mixed with "the three laws are a lie".) Additionally, the ambiguity of the laws came up multiple times in the Robot/Foundation universe, such as in "The Naked Sun" and "The Robots of Dawn."
The laws are paradoxically hard-and-fast yet ambiguous. In any case where any law is essentially violated (using one of the workarounds), the robots "go crazy" or die. This applies to the Zeroth Law; witness the end of Giskard due to the mere inability to determine if an action was harmful or not.
In the end, it's something like a moral version of the Halting Problem. Even if you can define "harm" to the satisfaction of everyone, you can't determine if something ultimately leads to harm or not.
Don't think of it as a flame---it's more like an argument that does 3d6 fire damage
Run out in front of an auto car and be able to kill the car's passengers? I don't think so.
-- these are only opinions and they might not be mine.
Why have cars at all if we aren't allowed to drive them? Rip up all the highways, and replace them with a gigantic autonomous rail system.
But no...
That's not what's at stake here. The truth is that if I'm not in control of my whereabouts anymore, then how can I be sure I'm making decisions for myself? Without a car, you might find yourself imprisoned by the distance your two feet can take you. Someone out there will applaud this along the same premise that "those who obey the law, have nothing to hide, and my gosh, if a driverless car prevents a CRIMINAL from driving to a crime, then the system pays for itself!", but that's not the point. It's not about morality, it's about control, and if someone is stopping me from driving my own car, then who's stopping them from driving theirs? When we fork over control of our transportation, then will come the day that we're isolated into districts, where the equivalent of passports will be needed from county to county. If the car won't let me drive it, how can I be sure that the car will obey me at all?
If all the cars in the world are autonomous, and computer controlled, well gee... what's to stop "someone" (anyone) from turning them all into a gigantic autonomous system that (I'm about to Godwin this...) conveys everyone to a huge concentration camp set to autonomous genocide?
It's not morality that the author is arguing in favor of.
It's our own autonomy that he's arguing against.
Someone will have control of these cars. Somewhere there will be levers.
Let's not imagine these automatic apparatuses to be forces of nature beyond an individual human's control. These are contrived, artificial, unatural man-made objects, at their core mechanical.
Morality is subjective.
To "program morality" would be to engender a machine with the specific moral subset imbued upon it by its programmer.
Thus, "machine morality" is actually "programmer morality."
We each determine our own morals, which will occasionally conflict with one another.
Forcing the public at large to follow a single person's idea of morality is, at the most basic level, an immoral act in itself.
Thus, "moral machines" aren't really moral at all.
An enigma, wrapped in a riddle, shrouded in bacon and cheese
So if my auto-driver car had to make a choice between my safety and that of someone else, it better choose me.
So you want every vehicle except yours programmed to harm you in preference to the other driver? What a fine society you envisage.
Does no one read Hume any more, or do we just have such a volume of sociopathic mods these days?
The only thing worse than a Democrat is a Republican.