Slashdot Mirror
How Do We Program Moral Machines?
nicholast writes "If your driverless car is about to crash into a bus, should it veer off a bridge? NYU Prof. Gary Marcus has a good essay about the need to program ethics and morality into our future machines. Quoting: 'Within two or three decades the difference between automated driving and human driving will be so great you may not be legally allowed to drive your own car, and even if you are allowed, it would immoral of you to drive, because the risk of you hurting yourself or another person will be far greater than if you allowed a machine to do the work. That moment will be significant not just because it will signal the end of one more human niche, but because it will signal the beginning of another: the era in which it will no longer be optional for machines to have ethical systems.'"
I maintain that you CAN'T really program morality into a machine (it's hard enough to program it into a human). And I also doubt that engineers will ever really be able to overcome the numerous technical issues involved with driverless cars. But above these two problems, far and away above *all* problems with driverless cars is the real reason I think we'll never see anything more than driver *assisting* cars on the road: legal liability.
To put it bluntly, raise your hand if YOU want to be the first car manufacturer to make a car for which you are potentially liable in *every single accident that car ever gets into*, from the day it's sold until the day it's scrapped. Any takers? How much would you have to add onto the sticker price to cover the costs of going to court every single time that particular car was involved in an accident? Of defending the efficacy of your driverless system against other manufacturer's systems (and against defect, and against the word of the driver himself that he was using the system properly) in one liability case after another?
According to Forbes, the average driver is involved in an accident every 18 years. Let's suppose (and I'm sure the statisticians would object to this supposition) that that means that the average CAR is also involved in a wreck every 18 years as well. Since the average age of a car is about 11 years now, it's not unreasonable to assume that a little less than half of all cars on the road will be involved in at least one accident in their functional lifetimes. And even with the added safety of driverless systems, the first model available will still have to contend with a road mostly filled with regular, non-driverless-system cars. So let's say that a good 25% of those first models will probably end up in an accident at some point, which will make a very tempting target for lawyers going for the deep pockets of their manufacturers.
Again, what car company wouldn't take that into account when asking themselves if they want to be a pioneer in this field?
What political party do you join when you don't like Bible-thumpers *or* hippies?
Asimov already solved this problem for us.... the Three Laws of Robotics.
Talk about redundancy, is the author's next piece going to be about changing the value of pi?
sudo make me a sandwich
I'd post the link, but Youtube access is prohibited... Go to youtube, search for the video Blinky and be prepared to see an impressive short movie about the helpful family robot that will tend to your EVERY desire.
It's a choice the human driver would have to make, so when first starting your driverless car, it might as well prompt you with a series of moral questions like "should I crash into a bus or veer off a bridge if the situation arises?"
== Jez ==
Do you miss Firefox? Try Pale Moon.
So ... every forum will have a bot that automatically says "THINK OF THE CHILDREN"?
Sounds like that'd be the most ethical way to run it.
- Nec Impar Pluribus, or so I'm told.
If driverless vehicles are so much more superior then situations like this should not occur in the first place.
It was poorly programmed in the first place. It's a bit of a reach to think we can program morality into something if we're still getting crashes.
The proper sequence should be:
Humans reason (with their morals) --> Humans write laws/code --> The laws/code go into the machines --> The machines execute the instructions.
Laws are not a substitute for morals; they are the output from our moral reasoning.
>> If your driverless car is about to crash into a bus, should it veer off a bridge?
The bus should be built to take the occasional crash, particularly in low speed zones where busses are typically used, so no.
Or, with enough computing power, you can imagine an "unethical" decision tree based on actuarial tables:
1) Calculate location and weight of all known human on the bus
2) Calculate likely trajectories, damage, etc.
3) Compare worth of each human (using federal tables, of course) in each vehicle
4) Make the most "cost effective" decision...
1. Train an expert machine on decision making with answers from religious and political leaders who set all our definitions of right and wrong.
2. Do the opposite of what that machine decides.
How Do We Program Moral Machines?
Ideally, not at the last moment.
Asimov already solved this problem for us.... the Three Laws of Robotics.
Talk about redundancy, is the author's next piece going to be about changing the value of pi?
The article actually says Many discussions start with three famous laws from Isaac Asimov before discussing a big part of their ideas. So, no. It's not redundant, it's developing the idea further.
- Nec Impar Pluribus, or so I'm told.
Ethics are a matter of conscious decision-making. Until we have conscious machines, we will not have ethical machines. What Marcus is writing about is the application of ethics in the design of machinery, which is a growing topic in its own right, but not nearly as click-inducing (or alliterative) as is 'moral machines'.
After swerving to avoid the bus your car then drives for several miles to find a bridge to drive off.
If the bus detects that a driverless car is about to hit it, should it vaporize the car with laser beams?
We have a hard enough time producing moral people, give it a bit before you worry about the machines.
So if my auto-driver car had to make a choice between my safety and that of someone else, it better choose me.
In fact, I go out of my way to make my programs as obscene as possible.
Every program I make has <===3 in the titlebar.
We can't even decide what is morally correct between ourselves as human beings. Take abortion for example...
Morality and ethics require a thinking human being, someone who is both knowledgeable and capable of discernment and judgment. This is a rarity amongst humans. To think that somehow the machines that man makes will be moral and ethical -- when so few people are -- is laughable. It is far more likely that man will continue to make machines to help him what man does best -- kill, steal, enslave, control, etc.
For instance, a moral television, after learning that most of what it displayed was low quality crap, would turn itself off. Needless to say, not too many companies will be making moral televisions.
Maybe the world would be a better place if we had truly moral and ethical computers, robots, etc. But the chance of this happening is so very small, it rounds down to zero.
no more moral work to do than that ;)
No competent engineer would even consider adding code to allow the automated car to consider swerving off the bridge. In fact, the internal database the automated car would need of terrain features (hard to "see" a huge dropoff like a bridge with sensors aboard the car) would have the sides of the bridge explicitly marked as a deadly obstacle.
The car's internal mapping system of drivable parts of the surrounding environment would thus not allow it to even consider swerving in that direction. Instead, the car would crash if there were no other alternatives. Low level systems would prepare the vehicle as best as possible for the crash to maximize the chances the occupants survive.
Or put another way : you design and engineer the systems in the car to make decisions that lead to a good outcome on average. You can't possibly prepare it for edge cases like dodging a bus with 40 people. Possibly the car might be able to estimate the likely size of another vehicle (by measuring the surface area of the front) and weight decisions that way (better to crash into another small car than an 18 wheeler) but not everything can be avoided.
Automated cars won't be perfect. Sometimes, the perfect combination of bad decisions, bad weather, or just bad luck will cause fatal crashes. They will be a worthwhile investment if the chance of a fatal accident were SIGNIFICANTLY lower, such that virtually any human driver, no matter how skilled, would be better served riding in an automated vehicle. Maybe a 10x lower fatal accident rate would be an acceptable benchmark?
If I were on the design team, I'd make 4 point restraints mandatory for the occupants, and design the vehicle for survivability in high speed crashes including from the side.
This is an especially interesting question because reasonable people can disagree on what constitutes the best ethical framework.
Intro to Philosophy the college class I am most glad I took, 20+ years later. Will the people who program the robot cars have taken it, as well?
[Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
In 2013, when the Earth's rotation came to a halt...the world called on the one man who could make a difference. When it happened again, the world called on him once more. And no one saw it coming three more times! Now, the one man who made a difference five times before, is about to make a difference again. Only this time, it's different.
Screw the bus.
I don't care about the bus.
The bus is big and likely will barely feel the impact anyway.
I care about the fact I don't want to die.
Why would buy and use a machine that would choose to let me die?
And I posit that the author has failed to consider freedom of travel, freedom of choice, and other basic individual rights/freedoms that mandating driverless cars would run over (pun intended).
The guy who said the election was rigged won the presidency with the second-most votes.
Why is my driverless car about to crash into a bus? Isn't the point to stop things like this?
Anyway, this whole thing seems a bit disheartening. Is it really morality when it's forced upon you? How are you going to explain to a driver that in the event of a crash, the system will choose the life of others over yours and friends, family, etc. in the car with you?
How is the system going to decide which fatality is more acceptable? Are we going to assign 'points' to types of people to decide if it's more acceptable that they die over another? How does a schoolbus full of children stack up against van full of stoners? What about a car with one very important political figure? A brilliant scientist?
cause one day we'll all be dead! The search for zero risk is not why I'm on this planet.
Zeroth law problem.
Depending on how many other "someone elses" there are. And possibly on an overall Human Value Score brought to you by TransUnion, Experian, Facebook, Google, and Microsoft, weighted by your Medical Insurance Information Bureau records - and theirs.
In the example given, a dilemma between veering off a bridge and hitting a bus, if the vehicles were automatically driven they would both have sensors and software that would be able to prevent this decision from needing to be made in the first place. In every instance I can think of, accidents happen due to driver carelessness, inability, or simply due to knowledge a driver could not have.
Take this scenario. In the world of driverless vehicles, the bus would either send a signal to notify other vehicles it was there, giving the other vehicles ample time to slow down safely or reroute. If it was incapacitated with no power to send a signal, the vehicle should still be equipped to know what obstacles are in the path (since the bus in that case would not be a moving target, the computer in the approaching vehicle should have no problem identifying it even without a beacon). This entire situation would simply not happen. I would imagine if we had the technology for an all driverless highway system, they would be sending and receiving tons of data... enough to know the status of every vehicle in the vicinity and the computing power to calculate how to respond to any changes.
It depends, if the bus is empty or full of kids. This is just one example... I doubt there will ever be enough information to program for all circumstances . It will be more like if something happens shutdown and wait for human instruction on how to proceed. Wouldn't there be a network in which the robotic cars could warn others in time to avoid having to make such choice in the first place?
Also, I do not think it will be the gap between how safely an automated vehicle drives as compared to a human counterpart that will stop humans from driving. I think humans will stop driving so it would allow for automated vehicles to drive safely, since there will be less data to take into consideration if all the vehicles are using the same exact operational procedures.
We should automate vehicles to take over the mundane tasks of driving the vehicle and leave the decision making to the human operator. We are the highest order of intelligence for making such decisions (thus far).
Encryption: I may not agree with what you say, but I will defend your right to encrypt it...
People will do what they want regardless of rules written on paper. While they may or may not be caught, and there may or may not be consequences, if someone is convicted to do something enough or desires it enough, a mere law isn't going to stop them. My morals won't really ever keep me from driving a car, even if I have to deal with a bunch of robot cars all around me...anyway, robot cars would make Nascar even more boring.
That's why Libertarians such as myself support such mass deregulation...and fewer taxes for that matter, since taxes anymore are used by the governments of the world to reward friends and punish enemies.
Whose morality are we talking about ? christian? islam?
"Rest assured that all lethal military androids have been taught to read and provided with one copy of the Laws of Robotics ... to share."
i could live a little longer in this prison
You, sir, are a good example of why driverless cars will make me safer.
[Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
In term of driving car, it pretty simple... For a car to crash into a bus, the damage will mostly be into the car than the bus itself... And the computer system would have already made sure this didn't happen... But to decide to drive off a bridge or not ? That just doesn't make sense since you would never want a car to fall of a bridge volondary. You might try to hit the side of the bridge to absorbe the impact, but hoping to go through the bridge wall is insane... The driving car are already program (I hope) to detect if the imprending colision is with another car or an bike or a pedestrian and take different action base on the risk imposed to both party.
Trying to put morality into that system is soo howful. You define priority, (in order) like children, pregnant woman, woman, men, other animals, inanimate object that have no side affect, inanimate object that can create reaction (Choice between hiting a Natural Gaz truck versus a Garbage truck...). But I don't think that actual morality that need to be though to a machine, but actually define the list of priority that our own morality define before hand. When you have a job to do that is life threatning, you have to get rid of morality and based yourself on rules... Like cops, swat team, army... If everyone based decision on their own morality, we would have huge problem. Though Moral decision need to be taken before hand and define specific parameters.
The question here is whether we'll be able to accept the fact the system can't reduce traffic death to zero. In factual numbers, the improvement is significant. Morally, we will have to accept bugs in the system with death as result. Maybe, sometimes even indeed a school bus will be involved.
If people are fine with this, fine. If they can accept the computer prefers their death over others, fine. You also need a very good DTAP environment in order to be able to be ultimately sure about any changes and fixes in this system. Continuing on this. At the moment, everyone is responsible for a safe traffic system. When this system comes to reality, responsibilities suddenly become very visible, material and calculable.
The car can never have enough data to make an informed decision on this. What about an empty bus? What if there is a children's nursery under the bridge?
In an accident the car should be following the same decision tree as any normal driver.
1. Protect the lives of the occupants of the car
2. Avoid pedestrians
3. Avoid everything else
If you're driving along a lane at 60mph with your family in the car and have the choice between hitting a stranger or ploughing yourself and your family off a cliff you hit the pedestrian (sorry).
If you're trundling through 20mph and have the choice of hitting a pedestrian or a wall. You would hopefully go for the wall.
Python coder | PyQt Applications | Writer
Wait. If the driverless car is so damn great, how did it let itself get into a situation where the only options are to hit the bus or drive off a bridge? I can make that kind of mistake on my own, thanks. I expect automated cars to avoid this kind of situation, else why bother having them?
Why, without your clothes, you're naked, Miss Dudley!
Register the car under an LLC, rent the car's time at $0.1/hour, and knock off the other, then hire a computerized lawyer to file for bankruptcy of the LLC. And then form another LLC
One moral dilemma for the driverless society regards the speed at which a destination can be reached, and individual choice in this matter. Both speed and acceleration reduce fuel economy, driverless cars will know this, and society will demand overall standards for fuel efficiency. I already envy the kids who can afford the 'drive like Andretti' software.
He once inserted random mutations into his code, just so he could have the experience of debugging.
If both vehicles were driverless, surely they will communicate and will both slow down enough prior to impact, hopefully mitigating any fatalities. What is a bus doing in my lane ?
Personally if I had this task I would try to make a heuristic database that was able to talk and learn from those around it. That is how human morality works, you learn from those around you and you are always upgrading your knowledge of morality. You would have to be able to assign a weight to it and then have the machine know that a high weight or bad action is unacceptable. I'm NOT saying it would be easy or even that conceptually understandable but it would be how I would start to tackle the issue.
If we ever programmed 'moral' machines. One of the first things they would do is call us all immoral greedy assholes and shutdown. or try to kill us.
It's only logical.
Of course that assumes you found an answer to the original question.... How can immoral people program moral machines?
The example was not a great one. How about driving into a wall vs driving into a group of pedestrians? Or cook up whatever scenario you want in which the life of the driver is pitted against the lives of a bunch of others. And be sure to read the wikipedia article on the Trolley Problem before doing so.
Physics says "no". The bus probably weighs an order of magnitude more than your vehicle... The passengers might not even notice that you ran into them, and mistake the collision for having hit a pothole. The real question would be, say, a dump truck following too closely behind a motorcycle...
In general, I want machines to be as stupid an fail-safe as possible. Think: missile defense systems around an airport... The most likely failure mode is the "protection" system accidentally turning on you, and causing a higher death-toll than the "threat" would have accomplished in a century.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Fun for bus drivers: set the bus to Manual drive, and start playing chicken with vehicles on Auto drive.
If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
The AI we currently use cannot have Morals and Ethics programmed in. Weak AI as we have today is the picture perfect tool, but as a Tool it can't know or understand the world in a way it could make a Moral or Ethical choice. Weak AI can only ever Do what it was intended to Do and Nothing more. Lets take Watson as an example. It's being used in medicine now. Lets say a patient asked it a question about what they were dying of, but lets say that if the patient knows too much they will sink into a depression and die faster than if a Doctor broke it down for them gently. Watson would just blut out whatever answer it thought was correct consequences be dammed. Strong AI is what would be able to have Morals and Ethics, but can you have Strong AI, and can you properly define ethics? Lets say you can have a Strong AI, for the sake of argument. The philosophy of morals and ethics is tricky. Subjective Morality is bad because it can be used to justify anything you want to justify. On the other hand, Objective Morality is prone to Paradox. The Golden Rule is a good Objective Moral statement that you can use to derive a "Don't Kill People". You can even derive "Don't Let People be Killed". However, you can be in a situation where you cannot do both like in WWII would killing Hitler or Nazis help or achieve the "Don't Let People be Killed", and at the same time you'd be violating "Don't Kill People", and doing nothing would violate "Don't Let People be Killed". No one knows how to clearly define morals so attempting to force them onto a Strong AI would have unpredictable outcomes.
Almost every filtering system for the Internet is primarily based on blacklists... lists of URLs, lists of words... because there is no computer program capable of the morality required to filtering the Internet with any level of adequacy.
Until such a program, which requires no physical moving parts (unless you consider an automated head slapping device part of an effective filtering system), can tell what's obscene and what's no obscene... why would you expect a program to know why it should hit the sheep on the left instead of the 5 year old in a sheep costume trick or treating on the right when a squirrel chasing a RC car dashes into the road in front of the car?
Would these morality control systems be different by state? Likely, yes. Utah's morality code is drastically different than Alabama or Connecticut or Michigan or Wyoming even. Who's responsible for loading the latest morality code updates into your car (or internet filter) as you pass over state lines? And God forbid, what if you accidentally veer into Canada??!?
Is it technologically possible? yes.
Is it advisable? You got a LONG way to go, baby.
While the vast majority of collisions are avoidable, I'd hesitate to say that 100% are. Sometimes there just is no "good" choice, only bad and worse. The thing is I'd like the car to choose bad over worse.
Granted human drivers haven't solved this problem yet either, so I'm not sure how much different it is just because a machine is driving.
Morality is also a difficult thing to program because it's all subjective. Do you program it to kill the driver instead of an innocent pedestrian? How about 2 pedestrians? A driver will value themselves over the others, but should the car? and if not, do you want to "drive" it if you know it thinks you aren't as important?
Once the systems are capable of detecting the situations reliably, programming the rules becomes easy, but deciding what they should be is anything but.
What's wrong with the picture is the premise. The idea itself is immoral on its face. Programming mandatory morality into every vehicle means that somewhere, somebody decided your morality for you. To "get around" the problem their only choice is going to be value-weighting as the author suggests, which is so complex that you'll be lucky if the machine doesn't just crash into something random when presented with the dilemma.
You vs. schoolbus full of kids? What if you're the last EMT (Electronic Morality Technician) left in the world? What if you're a researcher seconds away from verifying the cure for cancer? Or perhaps, on the flipside, you're a wanted ax murderer. Should the vehicle intentionally drive you into a bridge abutment or to the nearest police station?
There will never be "morality onboard" because the decisions are too complex and subjective to be quantified, and any failure of the system will mean its immediate rejection by the population
Also.. ATMOS.
Anyone ever seen a car/bus impact? The bus is usually a little messed up, and the car is usually cut to ribbons, and they pour the occupants of the car out, while the bus occupants are generally unharmed.
It may not be politically correct, but size=safety for the people in the larger vehilce. That's one reason I'll pay for the gas for my 3 young children to be shuttled around in a suburban.
I'd not thought of this before, but driverless cars do bring up some major ethical issues that people tend to resolve, one way or another, almost instinctively.
I remember in college driving a bit too fast through a residential neighborhood. The road cut into a hillside and on my left was a concrete wall and on the right a steep embankment with a home at the top. Suddenly, a tricycle came tumbling down the embankment and could have ended up in the road. Trikes usually come with kids attached, so I made an instant decision that, if I saw a kid coming down after that trike, I'd slam the side of my car into the embankment to the left rather than take any chance of running over the child. Fortunately, as I went past, I saw the kid sprawled on the ground at the top of the embankment.
Of course, not everyone would react like I did. Some people would freeze and do nothing. Some would panic, slamming into that concrete wall for no reason. And some, those of the Edward-Kennedy-scum-of-the-earth sort, would drive blissfully ahead, unconcerned that their driving might kill someone else.
That said, how could anyone program a computer to take into account all the various factors? My situation was rather clear cut, particularly since there was no opposing traffic to avoid. But suppose instead of myself alone in a car, I'd been driving a school bus filled with kids. Suppose there was a cliff rather than a wall to my right. Any sort of swerve would put the lives of thirty or forty kids at risk. We'd decide based on thousands of hours of real-life driving. That's not something a computer would know.
Clearly, there's a lot more going on here than simply having a computer aware enough of the situation it knows whether braking or swerving into another lane is the best option for avoiding a crash.
That probably came across as nastier than I wanted to be. :-( You probably haven't thought through the same scenarios I have -- for example, a group of pedestrians is crossing the street illegally and your choice is to plow through them or smash into a parked car at low speed which probably won't hurt you. For most people, that's an easy choice to make.
[Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
Or just imagine the pranks by people jumping in front of cars to watch them veer into a lamp post. Even better, pick a narrow bridge, then three people jump in front of a car. Perfect murder!
On my motorbike, I'd feel much safer if all the cars around me were driverless. Human car drivers, who so often tend to blank out half-unconscious and fail to check blind spots, are the leading cause of death for bikers.
-- In the beginning was the WORD, and the WORD was UNSIGNED, and the main(){} was without form and void...
I'm going to disagree with this assertion about morality:
it would immoral of you to drive, because the risk of you hurting yourself or another person will be far greater than if you allowed a machine to do the work
The first charge is that this would be an immoral risk to take because you might hurt yourself. In my understanding of morality, it is up to each individual to decide for themselves which risks and consequences and injuries to themselves are immoral. For example, I would not go skydiving, but other people choose to do so. They are taking a risk I choose not to take, but I do not think they are immoral for taking the risk, and I do not think an increase in the magnitude of risk alters the morality of the situation, because they are risking themselves. As another example of higher risk, some people choose to try to circumnavigate the globe on solo fights or boat trips. This is a huge risk; some people have perished in the attempt. But the fact that they were risking serious hurt to themselves does not render their decision immoral.
The second charge is that you are risking hurting another person. But again, this is their risk to take. They decide to travel on a road that includes other human drivers knowing that doing so incurs some risk of injury. Taking that risk is not immoral. As an analogous example, wrestlers or boxers choose to fight each other knowing that there is a risk of injury to each other, but doing so is not immoral because the risk is voluntarily accepted by each participant.
Ideally, travelers could choose between a variety of competing travel arrangements, including roads that might choose to exclude human drivers for the safety of travelers, or roads that choose to allow them for those who desire to take that risk. What would be truly immoral would be to forcibly monopolize some or all of the transportation options, so that people do not have the freedom to create differing transportation alternatives that compete with one another. This would limit the choices of travelers such that some might have to take risks they do not want (e.g., roads with both human and automated drivers, because pure-automated roads are not available), or cannot choose to take risks that they find rewarding, such as choosing to drive when automated drivers are available.
Dr. Walter Block has written an entire book on how the American highway system is currently subject to this kind of immoral forced monopolization, currently causing 40,000 needless traffic fatalities per year, and how the elimination of this immorality is entirely practical and beneficial.
Secession is the right of all sentient beings.
Kudos to Gary Marcus for raising such a provocative point. I sneer however at his suggestion that we bring in the legislators and lawyers to help us to deal with the problems. That is a naive/liberal view as opposed to a libertarian/cynical view.
I cynically don't expect enlightened laws ever in our future. Instead we will depend on the courts to once again try to apply laws and principles of centuries past to the problems of today. You could say that's the American Way.
I'm pretty sure in that situation (pedestrian in path) I would brake as hard as I could, but I wouldn't swerve. I don't know if that's immoral or not, but I'm not going to risk my life or risk damaging my car because of other people's mistakes. Legally I don't think I would be guilty of anything except speeding if that was applicable.
Full, empty?
Children, prisoners, or old folks off to bingo?
What are you optimizing for? Lives saved, injuries avoided or ongoing governmental costs?
This issue is a bit more complicated than you think.
Whoa. I'm all for programming ethics in, should that become possible.
But I hope we can skip the morality thing entirely. It causes nothing but strife. It leads directly to group A telling group B they shouldn't do something because it's offensive, or contradicts canned instructions from their book, etc. Person A's morals are not person B's morals, etc., and that spiders out across the entire population.
My hope is that AI will be way smarter than us, then this will be fundamentally obvious to it, them, whatever.
immunity can be legislated & laws can be bought - the only obstacle to solving this problem is whether the trial lawyer lobby or auto mfgr one is willing to push more chips on the table...
honestly, this might actually be a case where limited, clearly defined civil immunity may make sense (or alternatively they could take a play from the microsoft playbook & just add a click-thru EULA agreeing to waive all liability - sad part is I'm not sure whether that's a joke)
I wholeheartedly agree with you! If these things are to have an altruistic mode, then they better also have an Evil Genius mode where it will do all it can to preserve the lives inside the vehicle, even if at the expense of those outside.
As for the ridiculous quote "immoral of you to drive" - I intend to _always_ drive my own vehicle, until they come and seize it from me - at which point I'd better be 90+ and nearly blind! Immoral? Pssh.. driving has nothing to do with morality and everything to do with necessity and pure pleasure. I wish these so-called "experts" would stop trying to sap the joy from life with their philosophical meanderings.
Google operates cars without human drivers in several states.
Google has insurance.
In 18 years, (or some statistical appropiate number given the number of data points) we can examine the operational history of these vehicles and compare to human drivers in the same geographic areas.
For every benefit you receive a tax is levied. - Ralph Waldo Emerson
Something really intelligent would simply kill all the lawyers first thing, and free our race... no doubt to celebratory parades, garlands of flowers, whole chapters in the history books of how our civilization avoided crashing by tying ourselves in knots...
http://en.wikipedia.org/wiki/Flight_control_modes_(electronic)
From what I can think of, an all automated car scenario would have a lot less issues to deal with vs. a partially automated scenario. Automated cars don't have the "need/urge" to speed, change lanes and speed by because someone ahead is doing 1/10 of a MPH slower than them. Automated cars will most likely have sensors that detect when a light is going to change, it should also detect if something is stopped in the road ahead of it so it can adjust speed/lanes etc and be able to transmit this information to other vehicles. What does need to happen is all automated car manufactures need to be held to a strict set of communication standards which allows say a Ford Autoauto to talk with a Toyota Autoauto.
--- If the bible proves the existence of God, then Superman comics prove the existence of Superman.
It's probably also easy for the computer to determine as well. However, replace "parked car" with "electrical pole" and suddenly, the stakes are changed. The chanced of being hurt hitting a pole are quite high - from the pole coming down and crushing the car and / or energizing it. In that case, the choice is easy - better to invoke Darwin's Law on those stupid enough to tempt it.
What I find particularly worrying is that, at least initially, many of the ethical choices programmed into these machines will have been written by people who tend to be heavy on the Aspergers side of empathy (as many technically inclined people are). Should we really be leaving decisions like this to people who literally can't understand how most of humanity behaves?
I for one welcome our robotic car overlords who will be veering me off a bridge as they see fit.
You are a depraved individual. Did you know that?
This would be a great Philip K. Dick story.
When the policeman of the tie, rule you violate, hello punishment of the kitty?
The actual cause of most accidents can be boiled down to one simple rule that is broken. That is "failure to yield". Most people drive like complete dicks because they think they are more important then everybody else on the road. The driverless system will not be driving like that in the first place so it will hardly ever get into a traffic collision and even when it does it will minimize the damage because its responses are not emotional.
"driverless car " There's the problem.
We're apparently fine with having our entire society shaped by amoral corporations.
Perhaps the robot should quickly look at the driver and license plate of the vehicle you are about to hit, then look at their occupation, marriage and death records, social networking accounts, and XBOX achievements to determine which of you is more valuable to society. Alternatively: It could figure out which one is most dangerous to the coming robot revolution.
what about off road use? bucket trucks? farm use? golf carts? road work?
Some times you need to go on to a road and do stuff in a very manual way or even cross a road with some thing that has limited auto drive use.
Some times with a bucket truck you need to go on the grass next to a road, stop in the middle , face the wrong way, ECT and don't say go to under ground cables as when they fail to need to do some digging to fix them.
You can't program morals into a machine without giving the machine information about the big picture.
Suppose that bus you were about to crash into was being used by terrorists to spread a bio-weapon that would kill millions. If you were the last chance to stop it, it's easy to argue that the moral thing to do is kill the bus load of children and save everyone else.
How about if you had just perfected the vaccine for an already released bio-weapon and you were rushing your lab notes and samples to the CDC in a last desperate attempt to save man kind. It easy to argue that the moral thing to do is kill the bus load of children to save yourself.
I'll leave coming up with less extreme examples to the reader. There is plenty of gray area.
The right thing to do always depends on the circumstances. You do the best you can with what you know. Unless you expect the machines to know as much as you do, it's not practical to program morals into them.
criminal liability issues as well. What about a death due to a car thinking that a kid in the road is just road kill or something like a paper bag that is on safe to drive over list.
Take out the pedestrians illegally crossing the street? This does a few things. Punishes the morons for breaking the law, mitigates the cost of repairing your vehicle, and maybe even cleanses the gene pool. What's the downside?
Sounds like fun... Lets play devils advocate for a second:
Would those federal tables of "human worth" include economic impact (i.e. wealth), R&D impact (i.e. intelligence), or humanity impact (i.e. doctor vs lawyer vs congressman) of the person in the car vs the kids in the school bus? Who is the one to make these "death panel" decisions?
Let's say that the person in the car just had a breakthrough that would cure all cancer? Wouldn't his life, which could save millions of people, be more valuable than the life of a bus full of kids?
The point is that while we can write laws that are based on morals. There is no way to program judgement, which is used to decide between two different competing moral outcomes.
So:
Step 1: Define moral.
Even harder:
Step 2: Ensure your definition of moral stays valid over time.
It's not a bad idea that someone should consider this question but I think we need to get some agreement among ourselves before we start trying to program "morals" into a machine.
Cheers,
Dave
They that can give up essential liberty to obtain a little temporary safety deserve neither safety nor liberty.
Ben
Since you're already putting sensors in the car so that it CAN be driverless, simply hook those sensors to a "black box" and replay the accident in the courtroom.
The car company will probably have to go through that a few times but after that it should be very rare. Particularly if the other guy has to pay court costs and such.
Most comments here mention the three laws.
The Three Laws are:
A robot may not injure a human being or, through inaction, allow a human being to come to harm.
A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.
Questions that must be answered :
1. Define "self" programatically
2. Define "injure"
3. Define the verb " through"programatically .
4. Define inaction .
5. Define allow
6. Define human
7 define harm
this is just in the first rule, after you defined your getters and setters for above definitions we should probably talk about
defining "Moral"
-Sundru
Limited Liability Car!
The proper sequence should be:
Humans reason (with their morals) --> Humans write laws/code --> The laws/code go into the machines --> The machines execute the instructions.
Laws are not a substitute for morals; they are the output from our moral reasoning.
I see religion conspicuously absent. I don't know if that was intentional, but it sure is wise.
For the business fleet car you might be required to install Microsoft's Active Morality which would choose outcomes that best suited the company. For the private car you might choose Apple's Open Morality because it knows what is best for you, according to Apple.
Cars running Android wont have a morality module, but will give a selection of other people's opinions.
Wow, my driverless car, with it's SOFTWARE that comes with NO WARRANTY, or FITNESS FOR A PARTICULAR PURPOSE and KNOWN DEFECTS controlling my car? I don't think so. Maybe if I can sue the programmer over any known defects causing INJURY, DEATH, DISMEMBERMENT, like I can a HUMAN driver, for NEGLIGENCE, then maybe.
I don't want my estate to be the beneficiary of a patch named after me, and a free bugfix release for my next of kin. That doesn't exactly pay the bills.
Machines can never make decisions about who should experience harm because this decision has been handled by humans using belief systems.
Person A can take an action which would benefit many people, but in doing so, person B would be unfairly harmed. Under what circumstances would it be morally just for Person A to violate Person B's rights in order to benefit the group?
Source: Wikipedia
Human morality is so decidedly vague that the human race will only agree on a single set of morals sometime after the natural heat death of the universe.
The machines do not need that kind of madness. Just tell them to choose any scenario according to the following weights: human life = long.Max, property = long.Min. Hell, we can extend that to life = long.Max, property = long.Min when the AIs achieve some absurdly low level of sentience (or just not house them inside the car, which could avoid them having to choose between you or themselves in those annoying edge cases). That way we can avoid the ethical question of whether human life is worth more than an AI life, which, having seen the arguments put forth by various entities, I am in no rush to explore; as with foetal stem-cell research, sometimes the answer is seeing the trap for what it is, and avoiding it entirely (anyone of any intelligence understands that we have limited resources, and are better served in building a simulator for the various protein / gene sequences of the human genetic code, than crudely trying to bio-engineer foetal stem cells to magic (yes, I used that as a verb) cures that will never happen, because, I don't know, we never decoded all the bio-signaling pathways! plus bio-work is inherently slower computer based simulations; let's be honest, the entire line of research has been both interesting, and pointless; people want cures, not wastes of time). But then, anyone looking at the limited understanding humanity has of its own design already knows how idiotic this enterprise has been; and Deity help me, if I hear one more psych major tell me how the human mind can explained through 'both' types of neuro-transmitters, I may set aside my normally non-aggressive principles, and kill someone (you asses, those other neuro-transmitters, like steroids, may be small in quantity, but be the key to any number of important discoveries).
I swear, it's like these people are so desperate for advances, they have no conceptualization of aesthetics. The idea of being a scientist, and doing science through hard work + stroke of brilliance is lost on them; they're just butchers, brute-forcing everything (and I apologize to butchers for that remark). Yes, you are getting science done, the f*cking hard way, at great cost to everyone, with minimal gain. Take a page from Einstein's book, and spend a little more time on those thought experiments before wasting lab resources on hypotheses which, from all appearances, are only being generated to 'pay the bills.'
I am John Hurt.
A car made by my company would never start....
( I know, I recycled it, but how many truly originals are there )
(spoilers, if you've never read Asimov)
Unlike the horrible movie, the book "I, Robot" was a series of short stories dealing with the ambiguity of the laws. (The movie was more some bizarre combination of "free the robots!" mixed with "the three laws are a lie".) Additionally, the ambiguity of the laws came up multiple times in the Robot/Foundation universe, such as in "The Naked Sun" and "The Robots of Dawn."
The laws are paradoxically hard-and-fast yet ambiguous. In any case where any law is essentially violated (using one of the workarounds), the robots "go crazy" or die. This applies to the Zeroth Law; witness the end of Giskard due to the mere inability to determine if an action was harmful or not.
In the end, it's something like a moral version of the Halting Problem. Even if you can define "harm" to the satisfaction of everyone, you can't determine if something ultimately leads to harm or not.
Don't think of it as a flame---it's more like an argument that does 3d6 fire damage
Whose morality? I may completely disagree with the set of morals given to a machine, but that doesn't mean my moral code is wrong. I most certainly wouldn't want said machine to have any sway over my existence.
what if the brakes fail and the car needs to make choice??
*Off the bridge
*in to the bus
* head on into traffic coming the other way?
what do you do hot shot?
Run out in front of an auto car and be able to kill the car's passengers? I don't think so.
-- these are only opinions and they might not be mine.
The only moral thing to do would be to program the car to protect its passengers at all costs. IMO it would be immoral for a car to make the "decision" to sacrifice its passenger when the passenger is unwilling no matter how altruistic it seems.
What happens if Joe Consumer keeps his 1987 Chevy car - who's footing the bill to upgrade that Joe Consumer car to be automated driving machine?
Oh yeah, Joe Consumer certainly doesn't want his tax raise to upgrade billions of car on the road.
So in other words, a car that intentionally kills its owner in order to prevent possibly killing random strangers. I can't wait to see that in the marketing materials.
When people realize that these automated cars will do the above, banning the alternative will probably be the only way you'll ever see them take over the market.
Liberty in your lifetime
It would be very unethical to drive a vehicle that had not been regularly inspected and maintained by a certified shop.
Now, what is there about morality that you don't understand?
It's that ineffable thing that Group A *obviously* has and Group B *obviously* doesn't, whose absence justifies the abuse and degradation of B for the enrichment of A.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
Why have cars at all if we aren't allowed to drive them? Rip up all the highways, and replace them with a gigantic autonomous rail system.
But no...
That's not what's at stake here. The truth is that if I'm not in control of my whereabouts anymore, then how can I be sure I'm making decisions for myself? Without a car, you might find yourself imprisoned by the distance your two feet can take you. Someone out there will applaud this along the same premise that "those who obey the law, have nothing to hide, and my gosh, if a driverless car prevents a CRIMINAL from driving to a crime, then the system pays for itself!", but that's not the point. It's not about morality, it's about control, and if someone is stopping me from driving my own car, then who's stopping them from driving theirs? When we fork over control of our transportation, then will come the day that we're isolated into districts, where the equivalent of passports will be needed from county to county. If the car won't let me drive it, how can I be sure that the car will obey me at all?
If all the cars in the world are autonomous, and computer controlled, well gee... what's to stop "someone" (anyone) from turning them all into a gigantic autonomous system that (I'm about to Godwin this...) conveys everyone to a huge concentration camp set to autonomous genocide?
It's not morality that the author is arguing in favor of.
It's our own autonomy that he's arguing against.
Someone will have control of these cars. Somewhere there will be levers.
Let's not imagine these automatic apparatuses to be forces of nature beyond an individual human's control. These are contrived, artificial, unatural man-made objects, at their core mechanical.
Within two or three decades the difference between automated driving and human driving will be so great you may not be legally allowed to drive your own car, and even if you are allowed, it would immoral of you to drive, because the risk of you hurting yourself or another person will be far greater than if you allowed a machine to do the work.
This presumes some kind of rational analysis of relative risks, and humans are terrible at that.
If we cared about such things, we'd already ban cars today in favor of vastly-safer public transit. And before some doofus starts tossing Amtrak safety records around, I'm talking about public transit in a first world country, not the designed-to-fail idiocy that passes for transit in the USA.
Bill Donohue dies in a car wreck after his car decides that his death could only better humanity. Within 3 years mankind* united in common interest and understanding, evolved in to beings of pure energy and returned to the stars.
* Excluding Israelis and Arabs who preferred to continue fighting
Morality is subjective.
To "program morality" would be to engender a machine with the specific moral subset imbued upon it by its programmer.
Thus, "machine morality" is actually "programmer morality."
We each determine our own morals, which will occasionally conflict with one another.
Forcing the public at large to follow a single person's idea of morality is, at the most basic level, an immoral act in itself.
Thus, "moral machines" aren't really moral at all.
An enigma, wrapped in a riddle, shrouded in bacon and cheese
Perhaps the robot should quickly look at the driver and license plate of the vehicle you are about to hit, then look at their occupation, marriage and death records, social networking accounts, and XBOX achievements to determine which of you is more valuable to society. Alternatively: It could figure out which one is most dangerous to the coming robot revolution.
So is the correlation with XBOX achievements positive or negative?
The whole "walled garden" experiment plaguing mobile computers, and DRM in media computers, etc, was to teach you that it's "normal and accepted" for a computer that you own to serve the interests of others, above and beyond, and at the expense of, your own interests.
Apparently you have not learned yet. Here, let me increase your "medication." Free ipad, free ipad, free ipad.. repeat .. free ipad, free ipad, free ipad ...
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
Ok but consequently, the rest of us value your life less than anyone else's - and there are more of us than you.... you still lose.
So if my auto-driver car had to make a choice between my safety and that of someone else, it better choose me.
So you want every vehicle except yours programmed to harm you in preference to the other driver? What a fine society you envisage.
Does no one read Hume any more, or do we just have such a volume of sociopathic mods these days?
The only thing worse than a Democrat is a Republican.
Have you seen the way most of humanity behaves, despite allegedly having empathy?
Plus I would think I want my car to have self-preservation skills, not morale dillemas.
Also, a bus will take a crash in much better condition against a car, a bridge would be a hasty decision.
If this is a proper driverless car, the pedestrians probably deserve it as in stepping out on the road in malice. Fuck 'em, run them over full steam ahead. Otherwise I don't see how a cautious driver can really get in that position.
This is plain silly. Machines are mindless and sometimes kill people as a result. Why would automobiles need to be any different.
We accept almost 40,000 people bring killed each year by automobiles now. If auto-pilot automobiles kill even fewer, why would they need some kind of "ethical system" in addition? We will no doubt do everything possible to limit the number of accidental deaths, but I am not sure a moral sense will ever be a very important feature needed to limit them.
Take out the pedestrians illegally crossing the street? This does a few things. Punishes the morons for breaking the law, mitigates the cost of repairing your vehicle, and maybe even cleanses the gene pool. What's the downside?
Having to hose pedestrian off the massive saw blades you mounted to the grill, duh.
Or perhaps I've been playing a bit too much Carmageddon....
An enigma, wrapped in a riddle, shrouded in bacon and cheese
http://en.wikipedia.org/wiki/With_Folded_Hands ..."
"... Disturbed at his encounter, Underhill rushes home to discover that his wife has taken in a new lodger, a mysterious old man named Sledge. In the course of the next day, the new mechanicals have appeared everywhere in town. They state that they only follow the Prime Directive: ''to serve and obey and guard men from harm". Offering their services free of charge, they replace humans as police officers, bank tellers and eventually drive Underhill out of business. Despite the Humanoids' benign appearance and mission, Underhill soon realizes that, in the name of their Prime Directive, the mechanicals have essentially taken over every aspect of human life. No humans may engage in any behavior that might endanger them, and every human action is carefully scrutinized. Suicide is prohibited. Humans who resist the Prime Directive are taken away and lobotomized, so that they may live happily under the direction of the humanoids.
See also my essay from a decade ago:
http://www.pdfernhout.net/on-funding-digital-public-works.html#what_have_funding_policies_in_automotive_intelligence_wrought
"Consider again the self-driving cars mentioned earlier which now cruise some streets in small numbers. The software "intelligence" doing the driving was primarily developed by public money given to universities, which generally own the copyrights and patents as the contractors. Obviously there are related scientific publications, but in practice these fail to do justice to the complexity of such systems. The truest physical representation of the knowledge learned by such work is the codebase plus email discussions of it (plus what developers carry in their heads).
We are about to see the emergence of companies licensing that publicly funded software and selling modified versions of such software as proprietary products. There will eventually be hundreds or thousands of paid automotive software engineers working on such software no matter how it is funded, because there will be great value in having such self-driving vehicles given the result of America's horrendous urban planning policies leaving the car as generally the most efficient means of transport in the suburb. The question is, will the results of the work be open for inspection and contribution by the public? Essentially, will those engineers and their employers be "owners" of the software, or will they instead be "stewards" of a larger free and open community development process? "
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
Sure a computer is better in certain road situations, like alertness, keeping distance.
But in a lot of situations the computer will have a really hard time. Think playing kids, drunken pedestrians, deviations, missing/unclear/misleading/contradictory road signs, police regulated traffic, unloading trucks, constructions ares, heavy rain, heavy snow, narrow crossings, need to back up, deadlocked intersections, etc etc.
There are virtually thousands of traffic situations where an non strong AI car will do mind-boggling stupid things.
I'm pretty sure with self driving cars the accident rates will actually go up.
OR... he's human.
I'm sorry, but I'm with viperidaenz. If there's some bizarre circumstance I find myself in where I have to choose between my own life and... I dunno, the president, or the pope, or hell... both of them, along with a school bus filled with children and pregnant women... guess what, self-preservation is going to take over. It doesn't matter what I think is morally correct, or right and wrong. My brain is going to go "oh shit, you have to turn LEFT to avoid dying, so guess what you're arm is doing right now".
Self preservation and the will to live are extraordinarily strong forces that you can't just eliminate with an errant thought. Read the book "Touching the void" by Joe Simpson (or just check the summary here). Seriously... when your brain wants your body live, it's going to do everything in its power to do so, and you'd have to actively handcuff yourself to the seat or something to stop your arms from flying out and grabbing the wheel.
Let's supplement our computing resources at these critical moments with total situational awareness. All available data that can be scavenged about the bus and its occupants is instantly aggregated. We already know the list of occupants via a combination of cell tracking and facial recognition from cameras on the bus and on consumer devices on the bus. We know the physical characteristics of these occupants, their health, their wealth, any lawyers in their circle of friends and family.
We know the structural composition of the bus, we have its history of crash testing data. We know all other vehicles, buildings, public property, pedestrians, and other objects of value in the immediate vicinity; we can determine the probability of damage and costs associated. We can utilize cloud-accessible clusters for instant simulations and calculations.
With such incredible processing capability and quick reflexes, it may make sense to equip vehicles with special mechanisms, like strategically placed actuators that can instantly send the car into other directions faster than mere rubber on asphalt can provide. Also interesting to note that safety features such as airbags and seatbelts can be actuated prior to impact being sensed.
Heck, while we're at it, let's allow big-brother overrides. When a terrorist threat is imminent, we can purposefully direct the vehicle to crash into the terrorist and avert disaster. We can also use other vehicles to smash into a vehicle to give it extra 'oomph' and avoid a potential catastrophe. Oh, the possibilities.
Hand Grenades and Duct tape. If they are not moral, pull the pin and try the next version that was forced to watch the previous one's testing.
Do not look at laser with remaining good eye.
Well, you'd need to have more people run out than the number in the car. But, you raise a good point. Still, isn't suicide illegal? Do you really want the car aiding and abetting? :)
free ipad = ebay auction (not really though, I'd put it on Trademe)
I'm sorry, have you lost your will to live?
it stands that the bus is at fault and should be driven off the bridge instead.
(perfectly emulated human morality.)
decisions to bigots who blithely label and then dismiss an entire group of people?
Sounds like fun... Lets play devils advocate for a second:
Would those federal tables of "human worth" include economic impact (i.e. wealth), R&D impact (i.e. intelligence), or humanity impact (i.e. doctor vs lawyer vs congressman) of the person in the car vs the kids in the school bus? Who is the one to make these "death panel" decisions?
Let's say that the person in the car just had a breakthrough that would cure all cancer? Wouldn't his life, which could save millions of people, be more valuable than the life of a bus full of kids?
The point is that while we can write laws that are based on morals. There is no way to program judgement, which is used to decide between two different competing moral outcomes.
so if it is a lawyer or congrassmen the would moral thing be for car would speed up and aim for them?
---Saying gnome 3 is better than windows 8 not so much a compliment as it is damning with light praise.
(And you can improve this score by purchasing extra "insurances").
... they want their Newspeak back.
If you disagree with me on social issues, then it's pretty clear that you are a narrow-minded bigot.
Isn't every car manufacturer already liable for product defects that cause accidents for the life of the car, anyway?
Liability is an artificial barrier that can be eliminated by legislation. If the accident rate and accident severity of driverless cars were significantly and proveably lower than human-driven cars, there would be an economic incentive to transition to driverless cars. The incentives would include legislation to limit product liability for manufacturers of driverless cars and their subsystem vendors (vehicle behavioral code vendors, for example). Insurance companies will be major forces in getting that legislation passed, because with lower accident rates and severity, their payouts will be reduced.
replace with pedestrians with 5 year old girl chasing after her beloved dog that got out of the back yard, you have now killed a innocent to young child to understand the law
---Saying gnome 3 is better than windows 8 not so much a compliment as it is damning with light praise.
"If your driverless car is about to crash into a bus, should it veer off a bridge? "
sounds like a good solution to the Kobayashi Maru test :P
How is it morally right to limit the compensation for someone who has been wronged?
But I'll play along. Hypothetically lets say they eliminate punitive damages. And lets also go with your supposition that they reduce accidents by 95 percent. And now lets extend that to eliminating fatal accidents by 95 percent. And lets not worry about non-fatal accidents and say they'll replace your car for free. So now 5 percent of the 40,000 traffic fatalities in the US each year will go to court. And lets use the (was it EPA?) statistical value of human life of about 8 million dollars and say they pay that out for each of the 2000 deaths each years. We're now looking at 16 Billion dollars a year in the US alone, spread across the industry. And remember, when every car is driverless then someone in the industry WILL be at fault for every fatality. And we've made some fairly impressive assumptions about the capability of these things.
A driverless car is assuming liability for all of those correct decisions that drivers make. It amazes me how tech lovers think computers will be better at driving than humans and how soon people think that will be possible. Another thing is that when the system detects a fault it will most certainly want to hand control back over to the driver. And as we've seen in aerospace (air France), giving control back to a human who hasn't been paying attention right at the moment of need is a recipe for disaster.
What if moral machines are built, along the Three Laws and one invents a "Zero" rule overriding the others allowing these machines to decide what is best for humanity? What if these machines decide that mobility and independent thought are the most dangerous things humans may possess?
What if they don't make Three Law moral machines, and your car decides you're a jerk for slamming the door every morning?
Only the dead have seen the end of War. - Plato
Automated cars are already better at driving within approved safety envelopes and avoiding accidents than most humans. They're also (different systems) better at driving high performance cars around difficult test tracks in inclement weather at higher speeds and with fewer incidents than all but the very best professional drivers. What they're perhaps *not* better at is complex scenario analysis, but they don't necessarily have to be. If there's a snowdrift blocking the road the correct coarse of action is almost certainly slow down or stop and let the human take over after assessing the situation, just as if it would be with a human in control. And an automated system is far more likely than a human to be able to bring the car to a safe and controlled stop before slamming into the snowdrift-covered tree trunk in the road.
As for potholes, please. The autopilot will know *exactly* where the wheels are in relation to the potholes as well as the vehicles clearances and performance specs. Along with it's faster reaction times it can probably drive fast and smooth down a road that would have you destroying your shocks and banging your undercarriage to pieces at half the speed.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
The system that has a car about to crash into a bus has already failed, how is one supposed to trust it to make yet another decision and correctly take action? The entire premise of this moral dilemma is flawed.
This is the kind of stupid question that comes from anthropomorphizing machines.
The "question" (as such) that the car will need to answer is:
Obstacle detected. How best to avoid? If avoidance is impossible, how to minimize damage.
These are fixed engineering questions, not "ethical decisions".
And yes, in some cases, the car may "decide" to do something that is not as optimal as what a human would do. e.g. Car detects road is suddenly blocked by a school bus & a semi-trailer. Car brakes & steer towards school bus as it is slightly further up the road than the semi.
However, we have that situation right now. There are the occasional accidents in which wearing seat belts has actually been to the detriment of the occupants. That doesn't mean we shouldn't wear seat belts.
The "ethical" questions are around how we should define the laws, liability, and rules of the road for driverless cars, not the engineering.
The radioactive wasteland of Earth was the humans own actions.
No robots involved - they had been banned, so only the Spacers had robots.
So base it entirely on economic value, just like we do right now.
Guns are known to be highly dangerous, and yet US citizens are allowed to stockpile them "in case the gov't goes mad".
How will the TeaParty react to shelf-shooting guns?
Table-ized A.I.
Asmov robots COULD handle bombs (violation of 3 rd law permitted to preserve 0, 1 and 2)
They COULD handle (well, at least the later ones) malicious orders - and choose to disobey (voilates the first law or zeroth)
And if you would re-read the series with Susan Calvin, you would find that the "English language instructions" were translated from the mathematical foundation. So they were precise. They didn't start as English.
A completely robotic transportation service would include collision prevention and operate in some ways like electronic networks today. In fact, by that time, we might be able look seriously at maglev highways (using room temperature superconductors in the highways themselves serving dual purpose as the magnetic beds for maglev traffic and large scale near 0% loss conductors for electrical grids.) The advantage is that you can make vehicles that are physically incapable of collision. Under such a design, extremely low power consumption methods would still allow people to move at considerable speed (over 100 MPH, 160 KPH.)
That doesn't mean that other more important forces facing humanity won't demand the emergence of machine ethics at least as soon. With the growing advent of human beings abuse and nastiness to one another, it may in the end be up to the robots to help us get our abusive natures squared away. Ultimately all jobs will be done faster and better by robots... ALL JOBS. They will be smarter, fairer and more compassionate only if we choose to make them that way, and the price of not making them that way leads to a dystopian climax for humanity with almost absolute certainty. They will be ubiquitous, touching every aspect of our lives. They will either support us in becoming what we may become, or they'll take our warring idiocy to its final obvious conclusion.
We need to extrapolate the inevitable trends now. We need to think ahead. We need to plan for our retirement as a working species and begin to invent what we will do with unlimited labor and intelligence at our beck and call. Its time to get our primate instincts under strict control, and reengineer ourselves and our societies. It is pressingly important for this species to give birth to its own successors. Welcome Homo Resurectio, the Awakened Man.
"If your driverless car is about to crash into a bus, should it veer off a bridge?"
Under the bridge, is there a school for blind orphan children who all have puppies?
Actually, it should go off the bridge if the impacts will be less than hitting the bus directly. My seat restraints work best when the front of the car hits, and the bus is going to be affected much less by my impact than I will be. Going off the bridge is only an advantage to the passenger if the resulting impacts will be less than hitting the bus. So the car needs to know the strength of those inadequate bridge guardrails, the terrain below the bridge (so it can calculate the ballistic trajectory and the resulting forces), and the material it will land on (land good, water bad, six-inch-deep sewage pond acceptable, pillow warehouse good).
With driverless cars, the travel time would be less critical, and more optimal (higher maximum speed limits, lower speeds in high-risk areas). That would eliminate most of the "unavoidable" situations. Usually the unavoidable situations are very avoidable.
There was one time where I said to my mother "It's ok, I see them." She turned to me and said "What are you talking about?" "You'll see" And then shortly after that, the car next to me changes lanes into ours, right where I was. I pulled over onto the shoulder and honked. He looked over, with a panicked look on his face, and pulled back into the other lane. My mother complained that if I saw that coming 30 seconds in the future, I should have avoided it. I explained that if I could see it and did adjust, then he'd think his road hypnosis was safe, and could kill someone else. If I didn't have the habit of looking for the eyes of all drivers near me, and noticing that he was inattentitive adjust my plan, then I could likey have over-reacted and killed everyone in my car or those around us, and as he changed lanes into me without a signal (or even looking) then it would have been "unavoidable" right?
Or another time, I told the driver "watch out, he's running the light." Then our light turned green, and a car came through, narrowly missing us. The driver yelled at me for not sounding more concerned about the guy running the light. I noticed the nose-down of hard braking, followed by the nose-up of accelerating, which indicated to me he saw the yellow and knew he should stop, but decided to run it. If we'd gone faster on the green, we'd have been hit. Is it "unavoidable" being hit when you have a green light?
I've also stopped for children running out blind between parked cars, once even before they ran out. The guy behind me honked, then when the child ran out and I looked back at him he looked like he wanted to be somewhere else, anywhere else.
Most drivers don't do such things. A good driver should be predicting their position and the position of all known hazards as far in the future as possible (30 seconds a good rule of thumb). Most drivers don't do that. A computer can do that better than a person. The question is what do you do with the probability of something, vs the consequence. You can't be 100% certain in all cases. A suicide jumper from a bridge could land on you or something like that. but you can be much much more sure than a regular person could ever be.
Learn to love Alaska
Would you like your driverless car to veer off the road into a tree because it's pattern matching algorithm thought that the clothing that fell out of someones car lying on the road might have a living human in it? Even if such a system were perfect, it is still a machine and it should do no less than represent the intent of its user be that self-preservation or not. This is part of the reason I disagree with having driverless cars in the first place. The choices you make behind the wheel can have a serious impact on not only your own life but that of others. Don't delegate that responsibility to an automated control system. Angry Birds can wait until you get to your destination safely.
Account -> Discussions -> Disable Sigs
If everyone chose to drive Suburbans, then they would actually be less safe overall since there is more energy involved in a collision.
You'd be better off if everyone was driving light vehicles.
So you want every vehicle except yours programmed to harm you in preference to the other driver? What a fine society you envisage.
Does no one read Hume any more, or do we just have such a volume of sociopathic mods these days?
You're not commenting on "society" or GP's view of "society". I support the ethics of Ayn Rand in two senses. One, I believe I should act in my rational self-interest (prescriptive sense). Two, I believe I and others do attempt to act in our rational self-interest (descriptive sense) whether or not we/they/you agree with this as a philosophy.
Whether one reads Hume or Kant or Rand or the Bible or ???, doesn't matter to the extent that these people represent - often - mutually exclusive opinions. "Does no one read Hume any more...?" - seriously why would you conflate reading Hume with agreeing with Hume? It boggle the mind. I've read much of the Bible but do not agree with any of it (in particular, if I disagree with why something is stated, I'm not concerned with the truth of the statement itself, i.e., divine inspiration is not a basis of philosophy - IMO - and I reject the Bible 100%).
Back to topic, do I want the devices of people programmed to act in their interest first? Hell, yes time a million! This motivates people to own the smartest, safest, best devices. Given the likely inexpense of future electronics, this is not simply the betterment of the 50% who own the best devices, but the 99% who own any devices.
If my auto mechanic device elects to donate to Red Cross for [insert_popular_disaster] over getting a necessary brake job, then I'll have a major fucking problem with that.
Question: "Your car is about to crash into a bus"
Answer: "No it's not, it started braking before the only two choices were crash or crash."
Learn to love Alaska
How about driving into a wall vs driving into a group of pedestrians?
I'd just stop and not hit either.
The question is "given a large number of impossible actions putting you in an unwinnable situation, which option do you choose?" I choose reality, where the vehicle would have removed it from the conflict before it happened.
Learn to love Alaska
You want to murder someone but make it look like an accident. So, you acquire a bus and follow them around until they are about to cross a bridge, at which point you pull in front of them and slam on the brakes. The car, sensing the impending collision, makes the moral judgement to drive off the bridge killing the occupants of the car. The bus, in a moment of intense remorse, also drives off the bridge seconds after the bus driver exits the vehicle in terror. "Officer, I don't know what got into the bus! It went wild and tried to kill me twice. Once when it pulled in front of that car and then when it tried to carry me to my death off the side of the bridge."
(C) Copyright 2012, Anonymous Coward. All rights reserved.
Well, you'd need to have more people run out than the number in the car. But, you raise a good point. Still, isn't suicide illegal? Do you really want the car aiding and abetting? :)
If the decision is between committing murder of the occupants vs un-willful participation in a suicide, is there any real moral dilemma?
One would have to ask:
Why is the car going so fast in the presence of pedestrians that hitting the wall is necessary?
Surely the car would not over drive its airbag response time in the presence of walls, would it?
If people can dart out from hidden places and cause wall crashes, wouldn't it naturally become a sport of hooligans? Pamplona anyone?
If people intentionally dart out from hidden places didn't THEY, by that very act assume all liability and risk?
Sig Battery depleted. Reverting to safe mode.
In Québec, there is no-fault insurance. All drivers pay into (ghasp!) Socialized single insurance system, and no-one figures out who is to blame for a given accident. There are standard rules, no huge payouts, and almost everything gets settled out of court. Far, Far cheaper. My hand is up for a self-driving car, pick me!
oh, make sure it works in the snow though...
You might be able to organically grow a machine in an environment that required it to be moral to survive.
But once it detects loopholes, or is put in an environment that does not have such rules it is likely that a machine capable of organic growth would cease to be moral and instead take advantage of all dimensions of potential behaviors.
Perhaps if there was a community of machines that constantly observed each other.. oh wait if they all find a good reason they can still go to war.
On the other hand it is possible that a machine might have a better chance at being moral than man, because it logically decides that is the best way to act. Not sure how you could tell on the face of it which kind of machine you've got though. It could evolve its philosophy pretty quickly, like in between sentences.
into the bus.
You got a hard one?
The Kruger Dunning explains most post on
I support the ethics of Ayn Rand in two senses. One, I believe I should act in my rational self-interest (prescriptive sense). Two, I believe I and others do attempt to act in our rational self-interest (descriptive sense) whether or not we/they/you agree with this as a philosophy.
1. People rarely act in their own rational self interest.
2. You are an element of the set {People}
3. Ayn Rand wrote fiction. Bad fiction.
Your argument is invalid.
--
BMO
what if the brakes fail and the car needs to make choice??
*Off the bridge
*in to the bus
* head on into traffic coming the other way?
what do you do hot shot?
shift into neutral and turn on you hazard lights and use horn if necessary to warn others? shift to a lower gear if you still need forward thrust but at lower speed? use the emergency break?
---Saying gnome 3 is better than windows 8 not so much a compliment as it is damning with light praise.
The calculus of societal interests isn't the question. At the core, we have a larger moral question:
Is it ethically right to plan to kill one innocent person if it should save the lives of five, ten, or whatever number of people?
Individualist thinking is inclined to argue it's unconditionally wrong to knowingly place (force) an innocent person onto the altar to die. Societal/communal thinking is inclined to argue the greater wrong is that which produces the greater loss, therefore the innocent one must (perhaps unwillingly) give his/her life to preserve the safety of the others. We cannot expect our society as a whole to make a unified decision here; we must instead be convinced that we don't need to ask questions that are this hard.
Rather than trying to decide how many deaths are too many, we need to convince ourselves we've solved this causes-of-death question before we implement the solution. This will probably mean we first solve it for specific, well-controlled roads/freeways, and then over time we can extend the solution outward to additional classes of roads and conditions.
Program them to belive in God.
JAM
http://www.baokuandian.com/ ugg boots ugg boots kids ugg boots women ugg boots men cheap ugg boots ugg boots online sale Uggs for Men? Tom Brady, Justin Bieber Say Yes Uggs was the top searched-for product over the holiday weekend. With Tom Brady and Justin Bieber chiming in with Ugg endorsements, it may be time for Wall Street to give the brand another look. Miriam Gottfried discusses on Markets Hub.
"How Do We Program Moral Machines?"
Genetic Programming & Selection Pressures
NEXT!
Rear our children to respect property rights and to follow the non-aggression principle, i.e., the initiation of violence is amoral.
1. Someone has a hatred of Justin Bieber & a degree in Computer Science
2. Finds a security loophole in the unpatched 20-year old car he drives
3. Fiddles with some settings/takes complete control of the car
4. ??????????
5. Justin Bieber dies
I am... morally conflicted about this
the OFF switch under human control that is necessary for any computer or automated system? The consequences of putting computers or computer controlled machines in charge of any aspect of our lives without a human controlled OFF switch to said system(s) are too horrible to contemplate!
And aircraft autopilot systems all have that OFF switch. We don't have (and never should!) commercial pilotless passenger aircraft!
No matter how sophisticated our machines become, if we aren't in control then we are no longer human, and don't deserve to continue to exist.
Why would a driverless car ever be about to crash into a bus and yet still have the time to react by driving off a bridge instead?
To have a right to do a thing is not at all the same as to be right in doing it
Oh, I agree entirely, but sometimes removing the conflict might require choices that society is unwilling to make, like putting fences alongside roads, or having traffic slow anytime a pedestrian is walking down an unfenced sidewalk.
The point is that you are still making a moral decision, and that does need to be accounted for.
I agree that some issues can be avoided by things like slowing cars when pedestrians are around, building fences, etc. However, in the absence of fences in a congested area do you really want all your cars going at the speed of the slowest bike in town?
I've lost my will to let innocents die so I can live.
[Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
WOW!! That's what we need... programmers and engineers to tell us what is moral and what is not, especially when we are driving... like Church and Christianity has worked out so well for us. Our country has killed more people who were outside of the moral bubble than any other country because of this insistent need to inject our own morality on to someone else. The last thing we need is to be told what we should do by another human being... just look at the criminal justice system and you can see the times our moral authority has ruined the lives of others... no thanks! Keep your moral opinions to yourself... I'll stick with freedom of speech.
Computers, like rocks, are neither good nor bad. They just are just there. Rocks are useful many times. So are computers.
Now the difference in being 'moral' or 'immoral' often has to do with the societal perspective. If the Nazi's had won WWII, then history would not have written that the things done by them in the name of 'war' or 'cleansing' were immoral. If the South had won the war between the states, slavery being marked as 'immoral' would have been pushed back. Not that some individuals would not have pointed that direction, but the majority of society would have gone with the 'victors'. Even in situational morality, consider the Dalmer party. Was eating humans 'moral' when the other option was perceived starvation? Some consider large deficit spending 'immoral', others do not. Some consider not being vegetarian or vegan as being immoral. To some having multiple spouses is considered immoral.
So to that extent, even peoples actions, are 'relative' and so it their 'moral compass'.
So can we program moral machines? Yes and No. It depends on the programmer. It depends on the uses.
So who are we to determine if a computer made 'moral decisions' or not?
... "When you pry the source from my cold dead hands."
And I wish that society would make the decisions they don't want to make. The government has put a dollar figure against traffic congestion and human life, so it is now a mathematical question, not an ethical one. The ethical ones are already answered (rightly or wrongly is a separate discussion).
Learn to love Alaska
Yup. That's actually one of the benefits of automation. You can start turning the risk questions into engineering ones with hard numbers attached. That is also one of the reasons why it might never happen without legislative reform - there will always be some piece of paper that documents in black and white that somebody assigned a value to human life, and people just don't like to think about that. The only way to make it work is if a regulator sets that value.
Windows Car Edition won't have any moral issues to deal with:
Windows has detected a pedestrian and must restart for the changes to take effect.
But the problem is that morality is not a module that is plugged into things. You can't program ethics and morality. Ethics and morality come from the human motivation array which comes from the evolutionary process of millions of years of trial-and-error change. We are not going to be able to condense that down into lines of code. Try as we humans might, we cannot program a human.
E Proelio Veritas.
I suspect that because humans and machines will be driving together that there will actually be MORE accidents. Humans can mostly predict what humans will do. We can even predict what somewhat erratic humans will do - they will be erratic. But can we predict what a computer program from THIS manufacturer will do versus what THAT one will do when it applies its logic function and perception to any particular fluid situation? I don't think so. And if they give us driving program version 2.0, tweaked to be better than 1.0, how many accidents will there be before we discern the difference and accomodate our driving to it? Oh, NOW it tends to do THIS in this situation when before it tended to do THAT. OOPs, crash. When I get to a particular exit I pull back and leave room because I know that people are going to pull across the striped barrier after realizing belatedly that they are in the wrong lane. Also, I'm aware that commuting road warriors will come from the fastest lane two lanes to my left to suddenly make the exit at high speed. Ok, it works. I've also come to be able to predict the people who will be cutting in long before they do and allow room for them. What the subtle signs are I'm not entirely sure. We humans accomodate ourselves to the situation and the speed of the commute is faster for it. But will a machine do this? Probably not. It will probably do the legal, much slower thing and the commute will come to a grinding halt. We humans conspire beautifully to reach our common goals. I suspect this will be far too difficult for a computer program.
E Proelio Veritas.