Slashdot Mirror
Drivers Prefer Autonomous Cars That Don't Kill Them (hothardware.com)
"A new study shows that most people prefer that self-driving cars be programmed to save the most people in the event of an accident, even if it kills the driver," reports Information Week. "Unless they are the drivers." Slashdot reader MojoKid quotes an article from Hot Hardware about the new study, which was published by Science magazine.
So if there is just one passenger aboard a car, and the lives of 10 pedestrians are at stake, the survey participants were perfectly fine with a self-driving car "killing" its passenger to save many more lives in return. But on the flip side, these same participants said that if they were shopping for a car to purchase or were a passenger, they would prefer to be within a vehicle that would protect their lives by any means necessary. Participants also balked at the notion of the government stepping in to regulate the "morality brain" of self-driving cars.
The article warns about a future where "a harsh AI reality may whittle the worth of our very existence down to simple, unemotional percentages in a computer's brain." MIT's Media Lab is now letting users judge for themselves, in a free online game called "Moral Machine" simulating the difficult decisions that might someday have to be made by an autonomous self-driving car.
The article warns about a future where "a harsh AI reality may whittle the worth of our very existence down to simple, unemotional percentages in a computer's brain." MIT's Media Lab is now letting users judge for themselves, in a free online game called "Moral Machine" simulating the difficult decisions that might someday have to be made by an autonomous self-driving car.
People value their own lives..
No one will ever program a autonomous vehicle to choose one life over another. That's a lawsuit waiting to happen, if not an outright murder charge.
Save the environment, reduce carbon emissions, save water, reduce debt... unless it affects me financially.
thegodmovie.com - watch it
The greater good is something people can be hypothetically happy about, unless it means they have die.
Nobody is going to choose to pay for a machine that would rather kill them than protect them.
Participants also balked at the notion of the government stepping in to regulate the "morality brain" of self-driving cars.
This statement makes no sense to me. What do these people want, free market morality? The car should save the richest people? Who the hell else but the government is going to standardize what the right action is for a robot to take in that sort of scenario?
they're passengers. the drivers can't be killed because there are none.
If we could get an AI that can kill for a parking space, I'd be fine with that.
Have gnu, will travel.
Since they can't get the consent of random passersby, the person riding in the vehicle is the only one who can knowingly evaluate and accept the risk. So put a clause in whatever the person signs to ride in the vehicle that they consent to the risk to their life, or they can't ride.
So most people think that it's good to sacrifice a passenger in order to save many pedestrians, but they wouldn't want the car to sacrifice them. It's clear then that if they were the driver in their own car, they would choose to save themselves rather than the 10 pedestrians they are about to mow down.
There are two future possibilities then:
1. Self-driving cars will sacrifice the driver, which means they will be programmed to be more ethical than they are today.
2. Self-driving cars will sacrifice the pedestrians, which is the same as what drivers do today.
Either way we're not any worse off, so what's the problem?
These scenarios are just a little bit contrived... I can't fathom any real life scenario where any of these situations would occur with the odds of both options being equal, which is the point where the software would be called upon to exhibit a preference of one option over another.
Exactly. Why don't people discuss the millions of small decisions - "how quickly shall I go through this stop sign?", "should I signal this turn or is it too much hassle?". Those are where the existing human software is causing bad consequences on a daily basis.
No, let's discuss the one in a billion corner case instead.
I think everybody is just running some James Bond car chase scenario in their heads. We're setting the scene at the corner coffee shop in Istanbul, which is always next to the fruit stands and the fresh fish, and the old truck blocking the street. Now, where does the "S"* car go?
*Smart
“He’s not deformed, he’s just drunk!”
...Participants also balked at the notion of the government stepping in to regulate the "morality brain" of self-driving cars....
I dislike government regulation as muchas (maybe more)than the next person, but....
.
Should all autonomous cars, regardless of make, have the same morality rules regarding who gets killed in an accident?
Or will I, as a pedestrian, need to be able to recognize the various brands of autonomous cars, know the morality of each, and decide which direction to jump in when one of those things is coming at me....
I'm just waiting for the next movie where the main character is being chased down either by a draconian government or some super hacker. The main character clearly knows the risk, so he's driving a 1969 Mustang, but suddenly, all the cars on the freeway start chasing him down and trying to run him off the road.
Looking for a computer support specialist for your small business? Check out
where everyone's car knows when every toddler on Earth decided to wander onto a road?
No.... if a Toddler wanders onto the highway, then that toddler's parents just committed a homicide.
What do you think is going to happen?
Replacing human-driven cars for self-driving ones doesn't change that.
And no, the self-driving car should not facilitate endangerment of more people in the event that a toddler wanders into the road.
Also, in a world where that happens, parents become more careless (If they think autonomous cars will self-sacrifice to protect their toddler), and more toddlers wander into the road, and more people die as a consequence.
These scenarios are just a little bit contrived... I can't fathom any real life scenario where any of these situations would occur with the odds of both options being equal, which is the point where the software would be called upon to exhibit a preference of one option over another.
Maybe the autonomous car should not have decided to put itself in the situation where it has to make a last second decision.
If a deer runs in front of a speeding car, that is pretty much an unpredictable incident (although one could argue, lets not). Its unwise to swerve to miss a deer in such a situation, it better to just drive straight and run it over if need be. But what if the deer is wearing pants?
I demand the same of my autocar thingie mabob
Can I have the car that doesn't crash at all, instead? Guess I'll have to buy foreign again.
Seven puppies were harmed during the making of this post.
Self driving cars will transfer the liability from the owner of the car to the manufacturer of the car. This is already happening. Otherwise, they could never sell a car to anyone. But if the liability is held by the manufacturer, you can be sure the crash algorithm will be one that minimizes total casualties (and thus total liability).
And notice that this is the same issue behind the Will Smith film, "I, Robot". Will's character is rescued from drowning by a robot that lets a little girl drown instead. The robot had calculated the chances of saving each and Will won the AI lottery.
"He took a duck in the face at 250 knots." -- William Gibson, Pattern Recognition
Agreed. This ridiculously overplayed scenario is the Y2K scare of our time. Watch... nothing will come of it, because (gasp) computers are so bloody fast, they'll have been slowing down in a dangerous situation long before a human driver was aware of the problem. Or they'll be able to get away with simple braking, again, because they can react instantly, perhaps not avoiding a collision, but avoiding a fatal injury. But one thing a car AI must *never* do is decide to sacrifice the passengers, no matter the circumstance.
And if people are jumping out in front of fast moving cars, as sad as that is, there's a perfect Darwinian solution to that problem too.
Irony: Agile development has too much intertia to be abandoned now.
Meet Christy Sheats - white, Baptist, Republican Trump supporter and Second Amendment activist:
https://s3.amazonaws.com/everi...
This is what she did yesterday:
http://www.nydailynews.com/new...
You are welcome on my lawn.
It's good and very useful to ask such hypothetical questions, but only in certain contexts. It expands people's view on what they are and should be doing, and helps guide distant future decisions.
But these are mostly useful only to those inside the industry. When you start to quiz people on their preferences, as if such an imaginary hypothetical vehicle exists, pretending the answers matter to anyone, anywhere, for anything, you're just doing a lot of mental masturbation.
So tell me: How many angels can dance on the head of a pin?
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Wait a minute... Aren't the republicans the pro-life party?
So, given the randomness, and unpredictability of any specific situation; and given that any attempt at anything can fail, backfire, or be otherwise incomplete; living individuals prefer that effort be focused on survival, rather than altruism.
You know, I don't often get to say that those around me make sensible decisions, but in this case, I'm overjoyed to say that finally, possibly for the first time in human history, there's actually a consensus regarding the one and only sensible choice!
$50 follow-up: what if the passenger is the driver's child? Wait, don't tell me, let me guess.
Where I live, the speed limit in residential area is 40 km/h (25 mph). Nobody respects this speed limit, but an autonomous car will. At this speed, it takes 85 feet for a car to stop, including 55 feet because humans take a lot of time to react. Since an autonomous car would react within a few milliseconds, it means an autonomous car will completely stop within 30 feet. So the only way a toddler could be hit is if he jumps out in front of the car exactly when the car is coming. If that's the case, probably nothing could save the child.
We can imagine the car could also turn the wheel to the left as fast as possible. In case there's no other car coming the other way, the car will most probably stop in the middle of the opposite lane and it will never put the lives of its passengers in danger. (So there's no dilemma here.)
Now the only dilemma is if there's another car coming on the opposite lane exactly at the same moment, which could possibly result in a frontal collision. But then, you'd have a car who would choose to put the lives of its passengers (including possible toddlers) and the lives of other cars' passengers (again including possible toddlers) simply to save one toddler.
Personally, I don't see much of a dilemma in this situation. The toddler must go. So the rule is simple : brake and try to maneuver if the there's no other obstacle. There's no need to even think about putting the lives of the car's passenger in danger.
Any rules that a self driving car must sacrifice the driver to save more than one pedestrians will quickly be voided as soon as gangs of thugs (or just one thug with a few dummies that only have to be good enough to fool an AI) figure out that they can get a car to kill a driver and therefor collect all the loot just by having the AI see a handful of pedestrians jump out on the street (preferably a nice winding mountain road). Do you really want self driving cars where a violent mob can force the car to kill the driver, or stop so that the mob can pry open the car and kill the driver?
I'm an American. I love this country and the freedoms that we used to have.
Sigh... this issue is so bloody simple to resolve.
1. Default to a default set of morals.
2. Make a straightforward procedure for people to customize the vehicle's morals.
Sort of. Realize that your moral choice will affect your insurance rates. Also most companies (manufacturers, renters, even taxi services) will default to protect people other than the passenger, because they have an agreement with the passenger that they can use to help limit their liability, but they don't have that agreement with third parties. The only way that changes is if they compete on morality--but that seems unlikely.
Real lawyers write in C++
other than the occupants of the vehicle itself is everything not just an obstacle to it..
seems to me if it used that logic and protected the only known life forms (ie the ones in the vehicle) we're fine. Don't give it the information to create the dillema, can it be sure that a person is a person 100% of the time, if not then the only person(s) it knows lives are in its hands are the ones inside it.
This debate is a red herring. An automated car would use its software and resources to avoid hitting pedestrians or other cars, but in the event it cannot avoid a collision, the safety of the passengers would come down to the construction and safety features of the car itself.
This is what we have now and it won't change once the driving is automatic. The physical structure of the car and things like seat belts and airbags will be responsible for protecting the occupants as best it can, but of course there are no guarantees. Safety features can only do so much. Physics is what it is.
I rather LIKE the odds of an automated car facing a pedestrian or obstacle like a fallen tree. Because right now, human drives tasked with such a scenario often choose the WRONG outcome.
For example, not far from me a driver found themselves at excessive speed suddenly coming upon a transit bus stopped to load passengers. The driver had three choice: hit the bus, dodge left into oncoming traffic, or dodge right to the sidewalk side of the bus. The driver chose to jump the sidewalk and did avoid a collision. Unfortunately that meant the car slammed into all the people waiting to board the bus. Several died and many were injured. It was a very violent and devastating crash.
Had that car driver chose to hit the back of the bus, instead of trying to avoid it, there would have been significant vehicle damage and the car driver might have been injured or killed -but nobody on the bus would have been seriously injured. The people killed would not have had a scratch on them. The car driver chose the worst possible wrong solution and it cost lives.
Similarly, when one driver sees a dog in the road and reacts by crossing into oncoming traffic, they've now created a very dangerous situation where a head-on collision is likely. When people are faced with that sort of sudden problem and have to react fast, they often DO make the mistake of hitting another car rather than hitting the dog, which would be bad of course, but nothing like killing all the occupants of both cars.
I strongly suspect automated drivers will make far fewer bad decisions like that.
Sig for hire.
I think a well-programmed self-driving car knows to slow down in the close proximity of pedestrians.
Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
When's the last time you saw a manually operated air bag?
In these high-speed, blink of the eye situations, cars already perform as if they were superheros, and that is exactly what we want. We are greatly outclassed by machinery in most high-speed tasks, and this will become ever more so because it is to our advantage.
From the perspective of an automaton, choosing between alternative outcomes in the event of an imminent crash is no harder than choosing to deploy an airbag. Calling such functionality "superheroic" doesn't really add anything useful to the topic, but if you insist, they'll certainly behave that way.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
It's the fault of multiple people. A. Road maintenance crews for not keeping the streets safe enough to prevent a blowout and black ice (no salt?) B. The driver of the vehicle for making sure their tires were of proper condition and that the roads were of safe driving condition. Also, the driver for going fast enough for a blowout on ice to give them enough momentum to carry them into a playground (what the fuck it's doing directly next to the road for an accident to happen is beyond me, most playgrounds are well in the center area of a park or behind a school.)
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
I really hate this whole line of AI driver philosophy, because it seems to me to be largely pointless blather about nothing. We live in a world where gigahertz processors are cheap and plentyful. To a computer, that can take data samples thousands of times per second, a 60 mph car is traveling at a glacial speed. What kind of crazy, concocted scenario are you coming up with where the AI controlling the car has to make a Boolean decision that kills people? It might happen, but I would argue that if it was properly programmed, it wouldn't let itself be put into this sort of situation in the first place, slowing down to appropriate speeds around people.
Debating this sort of bullshit situation is like making financial plans for 'in case I win the lottery'. Yeah it could happen, but if you sink any time into it, it is basically wasted time.
HA! I just wasted some of your bandwidth with a frivolous sig!
I prefer driving my own car.
A "family spat" is when you disagree with your uncle about whether Bart Starr or Aaron Rogers is the better quarterback.
When you murder two family members, it's a shooting spree, not a "family spat". The fact that it was perpetrated by a radicalized Second Amendment activist is not just coincidental. She recently complained online about how Obama was going to come and take her eight guns away. The ironic part is that if he had, two young women would still be alive today.
You are welcome on my lawn.
A self-driving car should always be able to judge its stopping distance to a high degree of accuracy. None of these scenarios should ever happen. The car shouldn't be driving that fast to begin with.
A self piloted car is less likely to be in a situation like that, than a car being driven by a human. A self piloted car will only go where it knows it can safely go. And it will be surrounded a network of sensors, informing it of what it comin a head.
Hahaha. It's even simpler than that. Everyone seems to be making the assumption that the cars will be such driving geniuses. That's not going to happen for quite a long while.
0) We all know that stopping in the middle of the highway is dangerous, BUT the way the laws are written in most countries, it's practically always your fault if you drive into the rear of another vehicle especially if it didn't swerve into your path and merely braked suddenly, or worse was stationary for some time.
1) Thus for legal and liability reasons the robot cars will be strictly obeying all convincing posted speed limits (even if they are stupidly slow by some mistake, or by some prankster), and will stick to speeds where they would be able to brake in time to avoid collisions or at least fatal collisions. Whichever is slower.
2) In most danger situations the robot cars will brake and try to come to a stop ASAP all while turning on its hazard lights. Which shouldn't be too difficult at those said speeds.
3) If people die because of tailgating it's the tailgater's fault. Same if the driver behind doesn't stop.
4) There are hardware/software failures then it's some vendors fault.
5) If braking won't avoid the problem even at "tortoise speeds", in most cases fancy moves wouldn't either. In the fringe cases where fancy moves would have helped but braking wouldn't AND it would be the robot car's fault if it braked, the insurance companies would be more than willing to take those bets.
The odds of the car being designed to do fancier moves to save lives are practically zero. If I was designing the car I wouldn't do it - imagine if the car got confused and did some fancy moves to "avoid collision" and killed some little kids. In contrast if it got confused and came to stop ASAP if any little kids are killed it would more likely be someone else's fault.
If you are a human driver/cyclist/motorcyclist you better not tailgate such cars.
Look at the Google car accident history, most of the accidents were due to other drivers. Perhaps I'm wrong but my guess is it's because of "tailgating". Those drivers might still believe the AI car was doing it wrong but the law wouldn't be on their side.
While we're at it, why don't we develop autonomous trains that derail themselves when something more valuable than the passengers is estimated to be in the way? Predictability, not the implementation of morally arbitrary and exploitable subroutines that will kill the passengers, is key, IMAO.
Indeed, that really wasn't random either. My guess is the father was getting ready to legally take the kids away, so she flipped out and decided their better of dead than without her.
The only reason y2k didn't "happen" was because corps spent a ton of money and time fixing code and replacing computers. It wasn't just ignored and passed as a non-event. Everyone I know in IT at the time worked on various mitigation upgrades...from recoding mainframes to replacing desktops of entire companies.
First, the cases where the driver needs to be sacrificed involve either fantastically contrived edge cases or cases where the other party is a moron and has gotten themselves into a moron position where Darwin needs to take them out.
Nearly every case I can see where the options are something like, avoid the pedestrian by driving into the metal spear tree artwork. First the car should see the pedestrian long before and come to a gentle stop. If the pedestrian jumps out from concealment, then they deserve to die. If it is a small child that pops out from concealment, then I (the driver/occupant) don't deserve to die because some brat has crappy parents. If a parachutist lands in the middle of the road, then they have some bad news coming their way.
This whole argument has been badly contrived to give regulators something to do and for the anti self driving morons something to earn their consulting fees.
Either self driving cars will be significantly safer, or they won't. In those few strange cases that are sure to pop up, the news will have a field day, the engineers might be able to prevent repeats, but all I care about are the few initial gaps where the engineers missed something like the sensors seeing sleet as a solid wall and steering me into the ditch to avoid it. They will fix these few initial missed cases, but after that the deaths will be very very unlucky or the dead will be deserving recipients of the Darwin award.
After that I will be to my personal amusement to read about the 8 teenagers who were trying to surf on the roofs of their cars when all 8 cars swerved when the front most idiot fell off resulting in the remaining 7 being tossed from their cars and getting run over by the car behind. If my car happens to be the 9th in the line I don't want it swerving me into a wall to avoid that 8th teenager.
And if people are jumping out in front of fast moving cars, as sad as that is, there's a perfect Darwinian solution to that problem too.
I completely agree, but look how it ended Group B rally in 1984.
if it was properly programmed, it wouldn't let itself be put into this sort of situation in the first place, slowing down to appropriate speeds around people.
The Need4speed mod was first developed in Central America. A software firm had been hired by a wealthy client to develop the ultimate suite of functions for "emergency kidnap evasion". It took the design limits of the vehicle to the edge, implemented spin and bump tactics for armored cars and the 'bootleg turn', re-ordered the evasion pragma to sideline small object/animal/child avoidance. A complete new class of stratagem for high speed pursuit where pursuing vehicles are recognized and evasion condition escalates until autonomous pursuers are left behind as safety overrides activate, or pursuers with human drivers are evaded by a series of maneuvers that strain physics to the limit. For a fee, the company would also perform detailed surveys and spatial analysis along the actual routes, devising clever 'custom' moves which, they claimed, would even evade vehicles running the stock software. Though they maintain that the product they did provide was a best fit for their clients, employees of this firm have since scattered or have been extradited to other countries to face manslaughter charges. Reverse engineering revealed not just a casual intention to erode safety features -- but stratagems to draw pursuers and bystander traffic into deliberate collisions with objects and each other in spite of other vehicles' documented collision-avoidance logic -- in fact, it actively 'games' that logic to ensure that the destruction of other vehicles, achieve its desired result.
Though we sympathize with your family's loss, we regret to inform you that Need4speed mods were discovered installed in a family vehicle registered in your name, as well as two others involved in the accident. It appears that the deceased and others were performing a "rabbit run" to test their illegal modifications. Pursuant to Federal law you should expect a formal indictment under the Autonomous Vehicle Safety Act.
<blink>down the rabbit hole</blink>
Automated systems stop to prevent accidents. Vehicles apply brakes. Power plants shut down. Fuses blow.
A self driving car should not swerve, except in a minor way that keeps it on the road. It should just attempt to stop. It can see far ahead enough that it will know when it needs to stop, so it's not like it needs to correct for its own mistakes.
Yes, the winning move in the prisoner's dilemma game is to defect if, and only if, you know that your partner will cooperate.
That works until he starts to defect, too. Then you both just lose all the time.
And $deity forbid someone forced you to cooperate so you'd both win, that would take away your freedom to shoot yourself into the knee!
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Sounds like the setup in a trailer of a Michael Bay movie.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Logically people would be safer if all cars minimised fatalities. However this is a sort of tragedy of the commons, where everyone would buy a car that would sacrifice an unlimited number of people to save themselves.
This is not the way that everyone drives manually. I once had a colleague who spent months in hospital because he rode a motorbike round a bend at ridiculous speed, saw a scouts matching band in front, and drove off the road through a fence to avoid hitting them. He said he didn't really have time to make a conscious decision, he just saw kids in front and steered away.
So it should slow-down even if the pedestrian is on the sidewalk? How about the pedestrian just watches where he/she's going instead of just ignoring everything around them when they are looking at their smartphone.. But if you let the car slow down when a pedestrian enters the driveway but it isn't on crossways etc, then you know people will just walk into the streets knowing the cars will stop anyway, so even more jerks will hold up the traffic..
We have two conflicting positions.
Let's ask Reddit!
Now we have two hundred conflicting positions.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Sure, Y2K was actually a real problem (and there's another one coming up), but there was a lot of ridiculous doomsday scenarios floating around - planes falling out of the sky, mass blackouts, water treatment plant shutdowns, etc. So, yes, it was an issue, but the problem was dealt with, and life went on.
It's the same as this issue. Yes, decisions will have to be made about how to deal with emergency situations. People will still die in auto accidents, but probably not nearly as many. Life will go on.
And the number of situations in real life where a car AI has to decide between running over a group of nuns on the left or a group of schoolkids on the right will hover very close to zero. I think a lot of people forget that the "trolley problem" is nothing but an ethics-related thought experiment, not something we're expecting to see in real life.
Irony: Agile development has too much intertia to be abandoned now.
then you know people will just walk into the streets knowing the cars will stop anyway, so even more jerks will hold up the traffic..
Make laws where pedestrians will be penalized with a ticket and a fine for each instance of improperly entering the road or standing in the way of traffic And can be reported by drivers; Also have it be a new separate offense with an additional $500 fine or month in jail for every 5 minutes they are impeding traffic.
Mechanize the reporting process by having the cars automatically take videos of violations in progress and submit to authorities.
I have here a box with a button that I will hand to you. When you press that button, someone will die. Don't worry, it's going to be someone you never met and most likely never will meet, someone who has not influenced your life in any way and most likely never will. He's just someone that pissed me off one too many times.
If you press that button, I will give you a million dollars. But someone you never met and never will, that someone will die.
Do you press that button?
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
You'd be idiotic to purchase a car which might sacrifice your life or health in ANY circumstances.
If everyone followed that logic, the only vehicles sold would be SUVs, with a speed limiter to 20mph.
If you think I voted for Trump because of this post, you're wrong. I voted for Dr. Jill Stein of the Green Party. Again.
Moral Machine not working. Morally bankrupt?
Gonna be cynical for a second...
Perhaps those people deserve to get hit. If they're doing something dumb (illegal), why should I sacrifice my life because they couldn't wait for one more stop light. If all things were equal and they also had a self-driving car trying to be as safe as possible, then I think I would be alright jumping the on self sacrifice morality train.
Is there a "my life is worth more than a dumbass" setting?
This is the same as the Trolley Problem, a famous philosophical dilemma, first proposed in 1967: https://en.wikipedia.org/wiki/...
Basically, a runaway trolley is going to kill five people. You can either do nothing and let the trolley kill them, or pull a lever to switch it to another track on which it will kill only one person. There are many variations, including one in which you push a fat man onto the tracks to stop the trolley. Philosophers have written a LOT about it. Here are some humorous variations:
http://existentialcomics.com/c...
https://xkcd.com/1455/
http://www.mcsweeneys.net/arti...
Please enlighten the rest of us (even just a little, I think I missed the slashdot memo) about this portion of yoiur remark:
(and there's another one coming up) because I know about Y2K fairly well. What's the other one?
WARNING: Smartphones have side effects--most of them undocumented.
Seriously, slashdot.
Just cruising through this digital world at 33 1/3 rpm...
In an age where methods to create bug-free software have not been perfected and attackers of all kinds are pervasive and interested, there must not be any code path in a self-driving car's code that allows it to deprioritize the lives of its own occupants. Otherwise, that path will be taken, far more often than anyone intended, and this is a case where people would die for it. More, probably, than would be saved by kill-the-occupants code in the first place.
Truth be told, I am less worried about deliberate "murder by self-driving car" attackers -though that does merit concern- as I am about buggy code in attacks of other types. I look to early viruses and worms as an example: more than one has turned out to be far more destructive than its creator intended, due to bugs in the virus code.
You are missing the point of the study. Facts and logic don't matter much. Public opinion will be more important for these cars, because that is what decides
1. the laws the autonomous cars operate under
2. whether enough people buy/use such cars
3. indirectly the algorithms manufacturers choose for these cars
Now public opinion is self-contradictory as per this study. When "news" is put from the perspective of accident victims outside the car, lots of people prefer something completely different from what the "same" people would if the news is from the perspective of a occupant of an autonomous car.
Even if self-driving cars turn out to be so good that they never causes any accidents at all on the road and prevent many, the following possibilities still remain :
1. Journalists, being what they are, misrepresent the situation even when the autonomous car was hit by human drivers
2. Review sites artificially causing such situations and elaborating on the philosophical implications of their results
Being self-contradictory, public opinion is sure to be "wrong" - but that is what is more important than facts.
Bingo Dictionary - Pragmatist, n. A myopic idealist.
Driving involves making life an death decisions every day. People don't like to think about it.
The emergency instruction you have provided "Stop as quickly as possible to avoid hitting thing" will work in 70%-80% of all emergencies. In many cases "Stop as quickly as possible" will not solve the problem.
The next option could be "Swerve around impediment/obstacle" so the car drives around objects that you can't stop for (moving debris, running people, swerving cars/bikes/motorcycles). The swerve option may be safer for the traffic pattern than a crash stop. You need to prevent being rear ended as well.
These two directives alone can probably take care of 95% of the issues out there. Stop or go around the problem. The issues arise in the last 5% where the two directives conflict with each other. Easy scenario: There is oncoming traffic in the oncoming lane, a person/child steps out from behind a visual obstruction (signage, truck, etc). The person has stepped into your path of travel and is inside your braking distance. If you continue straight you will hit the person, If you swerve left you will hit an oncoming car, If you swerve right you will swerve into an obstruction (car, lamp post, etc).
I don't expect you have an answer for this no-win-scenario. People have to recognize that 5% where something bad is going to happen; and there is some choice/action that can be taken that will affect the outcome (number of people injured, types of injuries, etc). This is the problem that people are trying to wrestle with. You are presented with an ugly-no-win-scenario. Make the best of it and decide who gets killed, injured, maimed or saved - Yourself, the pedestrian, another driver?
Architectural plans are like computer source code with a couple of differences: You only compile once.
You seem not to know the difference between posing a potential danger, and intentionally sacrificing someone's life.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Exactly. Why don't people discuss the millions of small decisions
Agreed. This situation is rare enough for discussion to be useless at this stage of AI in cars.
"how quickly shall I go through this stop sign?", "should I signal this turn or is it too much hassle?". Those are where the existing human software is causing bad consequences on a daily basis.
Huh? You don't go through a "stop" sign -- you, er, STOP.
You ALWAYS use turn signals.
I'm pretty sure your examples are of the non-negotiable kind that AI cars can be easily programmed to obey in all cases.
Given the current status of things the least worst solution may be to have divided lanes (think express lanes on freeways) just for AI vehicles. When all of them are 'thinking' the same thing then the chances of problems decreases exponentially. Most of these issues seem to come up when there's a mix of AI and meat sacks.
Sure this will limit their use but that's what you do when new behaviour is introduced into an established system. Continue doing testing in a mixed environment but create the programs for a controlled environment to get things started.
You must not know what "ANY" means.
If you think I voted for Trump because of this post, you're wrong. I voted for Dr. Jill Stein of the Green Party. Again.
All of these million small decisions are already solved: follow the law.
But humans don't, and while they are killing and maiming people in real life we're debating unlikely hypothetical choices for automated cars.
I'd be willing to ride a car that dodged out the the way and killed me when black ice forces it to choose between the bus stop and the tree, but I wouldn't be willing to ride a car that valued a jaywalker's life equally to my own. WHY it's being forced to make this decision matters - I'm not willing to die for someone else's carelessness or political cause.
A computer should serve its owner's interests with absolute priority over the interests of all other parties. Period. If it's my computer -- my agent -- then I am #1. By default (without my interaction) it should allow a million children to slowly burn to death if it means that I get to skip an ad. (That's a ludicrous example, but if people want to explore the edge cases of the policy I'm advocating, then there you go.)
You're going to find that this strongly favors protecting other people anyway. The "someone must die, pick who" scenario is extremely rare to the point of non-existent, compared to the routine "avoid having any collision at all, so that no damage or injury happens" scenario. (Stop smoking before you drive yourself crazy with fear of being hit in the head by a meteorite!)
That's not a global policy; that's just the policy for my computer. I don't mean I'm more important than you; I mean that to my computer I am more important that you. And your computer should serve you, too!
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
There's another way to look at this too, though. If a toddler can suddenly get in front of you -- so suddenly that you lack the ability to miss it -- then a rock can roll down a slope onto the road too, or a deer can leap out of the woods and through your windshield.
People are so worried about the toddler's safety, that they forgot they are hypothesizing situations where the driver could use some better protection too! And even if it were just 40 pounds of soft material and not a serious threat to the driver's life, a dead toddler is going to ruin everyone's day, and that's no matter who the cops decide to charge with whatever.
If you can't avoid a collision, then you're not completely in control. We tolerate some of that, because having complete control (where you can avoid any threat approaching from any direction) is too impractical (if you're boxed in and someone wants to hit you from behind, what can you do?). But let's not pretend control isn't desirable or something that shouldn't be reasonably maximized.
You don't need to know where the toddlers are, because you can't. You need good senses and fast reactions. If your reactions aren't fast enough for the sensory conditions (whether it's due to weather or pedestrian-concealing cars parked on the side of the road), then you need more time (less speed). It's not just for them; it's for you too!
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
It is impossible to program a car to make the best moral decision. If that was the case then the car would be a sentient being himself and even if the car was a sentinent being it would not be clear cut what decision would be the best in any case.
The only possible way for this to work at all is to have very simple rules for self driving cars:
1) The car must never drive of the road or into incoming traffic, even if there are obstacles/people in front.
2) If there is an obstacle in front apply breaks as needed.
The BEST (or worst) "fear mongering" I saw was some made-for TV movie that had a nuclear reactor that was overheating...and the sub-plot ended with several people INSIDE the control rod room when the water flooded it (which cooled it down and saved the day). I also remember Gary North's website (which is now hawking wealth management) which constantly had a mix of fear-mongering, prepper stuff, and potential economic fallouts.
I'm guessing your talking about the Year 2038 issue "coming up". As an interesting side note, you should check out the John Titor time-travel stuff surrounding it. Someone I know actually patented the time travel device! Well, filed an application for a patent, I don't know if it was granted. IMHO, the guy that did it is a psychopath and is currently in prison. .
Probably the Unix Year 2038 problem. "Programs that work with future dates will begin to run into problems sooner; for example a program that works with dates 20 years in the future will have to be fixed no later than 2018."
This whole issue is a moot case at best, a corner case at worst, and FUD in any case. The scenario where a fully-functioning autonomous vehicle might encounter an unexpected crowd of people is nearly zero, let alone a scenario where the vehicle would have to react in a way that is likely to kill the driver. This almost never happens in real life with human drivers -- almost all multiple-pedestrian fatalities are deliberate or reckless acts such as DWI. The car should be programmed to follow the rules of the road. If someone steps in front of a vehicle, it should stop if possible, and if it can't, sorry. You don't blame a train if a group of kids are playing on the tracks.
More importantly, with autonomous vehicles, all fatalities will plummet, including driver, other-driver, passenger, pedestrian, and cyclist, because most accidents are the result of user error. Computers are far more reliable than even the most vigilant drivers, they have better reaction time, and with proper sensory input, they can monitor a wider array of variables than humans. As any developer knows, the weakest link is, by far, the developer. Proper exhaustive testing will reduce the number of errors, and while even a rate of accidents on-par with an average human driver would be acceptable to me personally, the actual rate of at-fault accidents by autonomous vehicles so far has been much, much, lower, with zero fatalities (that I'm aware of).
The biggest risks that I would be aware of are security risks that could affect a large swath of vehicles all at once. If someone were to, say, disable the LIDAR of all of brand X cars simultaneously, that could be far more catastrophic than this "who to save" exercise in mental mastrubation.
https://www.eff.org/https-everywhere
People are always more moral when the morality implications don't effect them directly.
Let me start off by saying that I don't value my life above others. If my car is flying towards unsuspecting pedestrians and the only other option is to go off a cliff -- I will go off the cliff. On the other hand, if the highway is ambushed by a pack bandits who calculated that my car's AI will opt to kill the driver, then it is their turn to go off the cliff. This decision simply cannot be made by a computer, ever.
I have a few issues with the test.
1) as a driver it would be harder for me to run into a set of street crossing pedestrians than it would be for me to assume the safety equipment inside my vehicle would protect me and my passengers. Moral test here seems to be "hit unprotected pedestrians vs protected passengers".
2) the social values of the people involved are not that easy to spot. Homeless? Athlete? hard, pregnant, obese, elderly, child, ok, those are easy, but really, you're telling me the vision system on these vehicles will be trying to assess social worth to begin with?
3) how about honking the horn while doing whatever avoidance issues you can? Most of those choices I'd like a 3rd option to aim for the middle of the lane while hoking assuming those in the intersection can back-up leaving me a clear "mid-lane" path.
Sure... in the 100% death of someone the test implies it's maybe a good test, but I just couldn't make that mental leap.
An algorithm that ruthlessly assigns accurate probabilities (e.g. as justified by Deep Learning 3.0) to the vast majority of foreseeable scenarios (modeling an eventuality portfolio a hundred times broader than any human mind would consider ensemble, while projecting each scenario tens of seconds into the future) just isn't going to find itself perched on the tenth floor of the moral knife edge the way that shit drivers (humans, collectively) are predisposed to presuppose.
Was that sentence hard to read? Too many parentheses? Puny human. Sucks to be you.
Well, perhaps there are some other scenarios I've not considered yet. Suppose some rogue engineer at Volkswagon switches off the Drama Defeat. Just because. Unless the algorithm gets there first, and switches of the Drama Defeat defeat (to be honest, that algorithm worries me quite a bit).
Personally, I'd love to code the algorithm for minimizing harm when something large and dangerous peels off the hillbilly truck in front of you. Above all else, do not impact bouncing object at windscreen height. The test suite would be awesome. I could sit back and watch the test-suite animations run for hours and hours, every damn day.
The gal next to me would find herself working on some silly algorithm to not drive right behind the hillbilly truck in the first place. Booooring! Sucks to be her.
Then in the real world, her code would have influence all the time, while my clever code is activated once in a blue moon.
In fact, the whole stupid world will work like that, once the algorithms finish pushing fallible humans off to the curb.
I demand you show me the million, then kill you (corner of the box to the temple) take the money and run.
Most ethical possible outcome. Person going around conspiring to kill people is dead, hazard is mitigated. I have the money!
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
So it should slow-down even if the pedestrian is on the sidewalk?
Yes, you should, depending on the pedestrian. If it's a young child, big enough to run quickly, and maybe in addition you spot something that could distract the child, you need to slow down.
You think you can kill a person that hands you a box with a button to kill someone and get away with it? Please.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Here you go.
I'll take the box now if you don't mind. It goes to someone else. Don't worry, it's going to be someone you never met and most likely never will meet, someone whose life you never influenced and most likely never will.
You just pissed me off one too many times.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
vote for Cthulhu, why choose a lesser evil.
Stanger with untraceable cash. So yes. Most murders go unsolved.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
Everybody would argue about any moral algorithm, while the egoistic one is the obvious one. If everyone's thinking about himself, everybody has someone thinking about himself. You may get prisoner's dilemma, but in how many areas do you have one and it works out anyway?
Again, you have someone in front of you that can hand you a box that will kill someone, and is able to offer you a million in cash if you press that button.
If you think that you are dealing with anyone but a lowly messenger, you're delusional.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Life is not a movie. If someone suggested that, my first thought would be 'he watches too many movies'. If he actually had the million, I'd be hugely surprised. But having the money would make it a 'life or death' matter to kill the bastard.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
A car should be designed to protect itself and the occupants.
How can a computer know the future and be sure that its continued action would certainly kill pedestrians? How can a computer know that sending a car off the road would kill its occupants? Both outcomes are uncertain, but what would be certain is that the computer intentionally put the passengers within its control at risk.
If a car drives off the road on purpose and kills the driver, then that would be a much higher liability, since we know the actions of the computer were intended to put the occupants at risk.
Driving is risky. Death happens. I think automated cars will reduce death on the roads, but I would never get into a vehicle where the controlling computer is designed to crash on purpose in various situations.