Slashdot Mirror
Drivers Prefer Autonomous Cars That Don't Kill Them (hothardware.com)
"A new study shows that most people prefer that self-driving cars be programmed to save the most people in the event of an accident, even if it kills the driver," reports Information Week. "Unless they are the drivers." Slashdot reader MojoKid quotes an article from Hot Hardware about the new study, which was published by Science magazine.
So if there is just one passenger aboard a car, and the lives of 10 pedestrians are at stake, the survey participants were perfectly fine with a self-driving car "killing" its passenger to save many more lives in return. But on the flip side, these same participants said that if they were shopping for a car to purchase or were a passenger, they would prefer to be within a vehicle that would protect their lives by any means necessary. Participants also balked at the notion of the government stepping in to regulate the "morality brain" of self-driving cars.
The article warns about a future where "a harsh AI reality may whittle the worth of our very existence down to simple, unemotional percentages in a computer's brain." MIT's Media Lab is now letting users judge for themselves, in a free online game called "Moral Machine" simulating the difficult decisions that might someday have to be made by an autonomous self-driving car.
The article warns about a future where "a harsh AI reality may whittle the worth of our very existence down to simple, unemotional percentages in a computer's brain." MIT's Media Lab is now letting users judge for themselves, in a free online game called "Moral Machine" simulating the difficult decisions that might someday have to be made by an autonomous self-driving car.
People value their own lives..
So as usual, people are OK with sacrificing 'others' to save more 'others' but their lives must be protected by any means necessary.
No one will ever program a autonomous vehicle to choose one life over another. That's a lawsuit waiting to happen, if not an outright murder charge.
Save the environment, reduce carbon emissions, save water, reduce debt... unless it affects me financially.
thegodmovie.com - watch it
Republicans always put their lives above others. Hopefully the law will pick to kill them over the lives of the many.
If you buy the car that saves others, they let you tweet/facebook/existenz that.
Liberals/progressives will buy the cars that save others to virtue signal. "I'm a great person, I put others ahead of myself and my children!"
Conservatives will buy the cars that save themselves and their families.
We'll be hated for it. "Can you believe those racist/sexist/spliceophobes are buying safe cars? Don't they know it's the current year?"
The left will try to get the government to mandate that all cars save the maximum number of lives, where the maximum number is based on an equation which considers historical injustices and economic inequality, but (hopefully) we'll maintain enough influence that everyone is offered the choice.
A busload of nuns is about to hit a playground full of cancer kids...
The greater good is something people can be hypothetically happy about, unless it means they have die.
Nobody is going to choose to pay for a machine that would rather kill them than protect them.
These scenarios are just a little bit contrived... I can't fathom any real life scenario where any of these situations would occur with the odds of both options being equal, which is the point where the software would be called upon to exhibit a preference of one option over another.
So do you plan to have a world where everyone's car knows when every toddler on Earth decided to wander onto a road?
Sure, it's the kid's fault. That doesn't mean that there's no decision to be made.
Did he just go crazy and fall asleep?
In most cases, car should slow the fuck down and stop when needed.
Where in reality are these cases where a car can save 10 lives by sacrificing itself? Is this some fantasy where the car is parked or driving with you in it and it sees another car barreling at a crowded fruitstand?
Fuck that shit, still come to a stop or slow down. Playing hero will be just fraught with unexpected consequences.
Don't make these cars into superheroes or some retarded wish like that.
Participants also balked at the notion of the government stepping in to regulate the "morality brain" of self-driving cars.
This statement makes no sense to me. What do these people want, free market morality? The car should save the richest people? Who the hell else but the government is going to standardize what the right action is for a robot to take in that sort of scenario?
they're passengers. the drivers can't be killed because there are none.
If we could get an AI that can kill for a parking space, I'd be fine with that.
Have gnu, will travel.
Are computers even fast enough to calculate the amount of possible factors to decide in such an event yet?
Since they can't get the consent of random passersby, the person riding in the vehicle is the only one who can knowingly evaluate and accept the risk. So put a clause in whatever the person signs to ride in the vehicle that they consent to the risk to their life, or they can't ride.
Lets ask Reddit!
There is no decision to be made. It is an Engineering requirement. They must make the car capable of safely going the rated speeds. The speed limits, warning signs, and so on that are in place for human drivers are extremely conservative. They are such that usually multiple factors have to come into play before there is even a danger. Such as not just going to fast but also not paying attention, or not just dark but also very tired, or drunk. Machines perform the same no matter what. Or in the case of night time, the visual sensors are well characterized to perform at a certain level. There must be a certain maximum latency in the pipeline so that slowing down (coming to a complete stop) is always an option. 9999999/10000000 that is the best option. You can't kill someone if you aren't moving.
You'd be idiotic to purchase a car which might sacrifice your life or health in ANY circumstances. It can't possibly be perfect enough to make that decision properly even just most of the time. False positives are far more likely, which means cars will be murdering occupants on a regular basis, for no good reason.
The thought experiment is massively unrealistic. Among other things, it requires a computer with infallible and almost prescient knowledge. It doesn't just require an almost magical mastery of physics, so that the wet road, temperature of the tires, or similar won't change the vehicle characteristics enough to change which option is most ideal. It also requires the crash dynamics of vehicles and humans to be perfectly model-able, where it can guess that one person will be killed in a 25MPH wreck, but another will survive a firery high-speed collision with an immovable object.
What's more, this computer deciding how to save lives, has ALREADY failed to avoid putting you (and others) in a no-possible-way-out deadly situation in the first place.
And did I mention that mannequins or animals in the road will look an awful lot like pedestrians to computer sensors? Jumping in front of a driver-less car would be a good way to murder the occupants. Criminals have already realized that standing in front of a car with pedestrian avoidance systems will hold the victims' vehicles helplessly in place.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Tire blowout on black ice. One direction is the river, the other is a playground. There will be an accident. It isn't any one person's fault.
So most people think that it's good to sacrifice a passenger in order to save many pedestrians, but they wouldn't want the car to sacrifice them. It's clear then that if they were the driver in their own car, they would choose to save themselves rather than the 10 pedestrians they are about to mow down.
There are two future possibilities then:
1. Self-driving cars will sacrifice the driver, which means they will be programmed to be more ethical than they are today.
2. Self-driving cars will sacrifice the pedestrians, which is the same as what drivers do today.
Either way we're not any worse off, so what's the problem?
The AI doesn't know whether the 10 people are actually innocent pedestrians, or 10 people trying to rob/kill the driver. Well, I didn't actually read the article, but I assumed this was not addressed. Love, Legal.Troll (unfairly saddled with -1 Karma)
...Participants also balked at the notion of the government stepping in to regulate the "morality brain" of self-driving cars....
I dislike government regulation as muchas (maybe more)than the next person, but....
.
Should all autonomous cars, regardless of make, have the same morality rules regarding who gets killed in an accident?
Or will I, as a pedestrian, need to be able to recognize the various brands of autonomous cars, know the morality of each, and decide which direction to jump in when one of those things is coming at me....
I'm just waiting for the next movie where the main character is being chased down either by a draconian government or some super hacker. The main character clearly knows the risk, so he's driving a 1969 Mustang, but suddenly, all the cars on the freeway start chasing him down and trying to run him off the road.
Looking for a computer support specialist for your small business? Check out
where everyone's car knows when every toddler on Earth decided to wander onto a road?
No.... if a Toddler wanders onto the highway, then that toddler's parents just committed a homicide.
What do you think is going to happen?
Replacing human-driven cars for self-driving ones doesn't change that.
And no, the self-driving car should not facilitate endangerment of more people in the event that a toddler wanders into the road.
Also, in a world where that happens, parents become more careless (If they think autonomous cars will self-sacrifice to protect their toddler), and more toddlers wander into the road, and more people die as a consequence.
A machine is utterly incapable of making true moral decisions because it can never really be programmed to accurately perceive, much less understand morality. The machine literally doesn't know what's going on. It has no idea what constitutes right from wrong without the kind of machine-readable signals that will simply never be present in real-world scenarios, and mechanistic decision-making rules like "it looks like X number of humans might die and that must mean i should kill the driver RIGHT NAO" would be an example of an idiotic shortcut that will simply yield horror scenarios where the machine makes an idiotic decision to kill someone. Ick, a world full of aspie programmers cooking up ideas like this is a horrifying thought. Ick. Legal.Troll (and I'M the one with -1 Karma!)
By following these simple rules:
1. Don't swerve into an obstruction
2. If both lanes are obstructed (and there is no "Don't Walk" indication for either lane of pedestrians) don't swerve (meaning stopping distance is minimum).
3. Swerve to hit "Don't Walk" pedestrians rather than hit "Walk" pedestrians
4. Do not swerve to avoid animals
I was told that I am biased against fat people, old people, and women. Call it coincidence, but I think there's an agenda buried in this test. Either that or they have over-complicated it.
por mim o silvio santos poderia tomar um tiro na cara ao vivo. e por favor não me pede nada, senão esse bando de retardados vão ficar enchendo meu saco.
I demand the same of my autocar thingie mabob
Can I have the car that doesn't crash at all, instead? Guess I'll have to buy foreign again.
Seven puppies were harmed during the making of this post.
What makes you think the automakers, the software developers, or the engineers will ever be allowed to make this decision?
At some point, the Federal DOT, supported by Congressional legislation, will mandate what autonomous cars must do in these situations. The industry will go with that, because that will be the only way that they can avoid being sued into oblivion every time one of their autonomous cars is involved in a fatal decision.
And think of the possibilities of how this will be abused:
The law and DOT regulations will someday require that a single politician, large donor, party bigwig, congressman, etc. life is more valuable and must survive any accidents if at all possible. All cars will be programmed to sacrifice any and everybody else when involved in a collision where a "protected" person is involved. The possibilities for politically mandated exceptions, unfairness, murder, etc. are beyond counting. The notion of "protected" person will be extended to certain minorities, women, LGBT, ad infinitum. However far it gets extended, white cis-gender males will be dead last...
Plus, there will be mandated backdoors, so that your car can be ordered to ferry you to the nearest police station, DPS command center, Federal courthouse, etc. as needed.
Self driving cars will transfer the liability from the owner of the car to the manufacturer of the car. This is already happening. Otherwise, they could never sell a car to anyone. But if the liability is held by the manufacturer, you can be sure the crash algorithm will be one that minimizes total casualties (and thus total liability).
And notice that this is the same issue behind the Will Smith film, "I, Robot". Will's character is rescued from drowning by a robot that lets a little girl drown instead. The robot had calculated the chances of saving each and Will won the AI lottery.
"He took a duck in the face at 250 knots." -- William Gibson, Pattern Recognition
You mean the whole "save those most likely to live" bit is going to become reality, and we can have our own real life versions of an angsty WIll Smith solving crime for us?
The future of ai in av will be decided by civil lawyers and judges when the body counts start to come in and the lawsuits get filed. Considering the variability in our justice system it is impossible to predict the outcome. History shows anything is possible as an outcome and common sense solutions are unlikely.
So, given the randomness, and unpredictability of any specific situation; and given that any attempt at anything can fail, backfire, or be otherwise incomplete; living individuals prefer that effort be focused on survival, rather than altruism.
You know, I don't often get to say that those around me make sensible decisions, but in this case, I'm overjoyed to say that finally, possibly for the first time in human history, there's actually a consensus regarding the one and only sensible choice!
$50 follow-up: what if the passenger is the driver's child? Wait, don't tell me, let me guess.
Where I live, the speed limit in residential area is 40 km/h (25 mph). Nobody respects this speed limit, but an autonomous car will. At this speed, it takes 85 feet for a car to stop, including 55 feet because humans take a lot of time to react. Since an autonomous car would react within a few milliseconds, it means an autonomous car will completely stop within 30 feet. So the only way a toddler could be hit is if he jumps out in front of the car exactly when the car is coming. If that's the case, probably nothing could save the child.
We can imagine the car could also turn the wheel to the left as fast as possible. In case there's no other car coming the other way, the car will most probably stop in the middle of the opposite lane and it will never put the lives of its passengers in danger. (So there's no dilemma here.)
Now the only dilemma is if there's another car coming on the opposite lane exactly at the same moment, which could possibly result in a frontal collision. But then, you'd have a car who would choose to put the lives of its passengers (including possible toddlers) and the lives of other cars' passengers (again including possible toddlers) simply to save one toddler.
Personally, I don't see much of a dilemma in this situation. The toddler must go. So the rule is simple : brake and try to maneuver if the there's no other obstacle. There's no need to even think about putting the lives of the car's passenger in danger.
When given the choice between two people, use facial recognition to identify them. Lookup the IRS filings to see which one earns the most. Kill the lower earner. This results in the minimum liability.
Scary, but the correct answer.
On the whole, we're all better off if autonomous cars aim to save the most lives possible. Individually, we're each better off if we're the given a higher priority than all others. This is a pretty typical example of the prisoner's dilemma: https://en.wikipedia.org/wiki/....
Any rules that a self driving car must sacrifice the driver to save more than one pedestrians will quickly be voided as soon as gangs of thugs (or just one thug with a few dummies that only have to be good enough to fool an AI) figure out that they can get a car to kill a driver and therefor collect all the loot just by having the AI see a handful of pedestrians jump out on the street (preferably a nice winding mountain road). Do you really want self driving cars where a violent mob can force the car to kill the driver, or stop so that the mob can pry open the car and kill the driver?
I'm an American. I love this country and the freedoms that we used to have.
Sigh... this issue is so bloody simple to resolve.
1. Default to a default set of morals.
2. Make a straightforward procedure for people to customize the vehicle's morals.
Sort of. Realize that your moral choice will affect your insurance rates. Also most companies (manufacturers, renters, even taxi services) will default to protect people other than the passenger, because they have an agreement with the passenger that they can use to help limit their liability, but they don't have that agreement with third parties. The only way that changes is if they compete on morality--but that seems unlikely.
Real lawyers write in C++
other than the occupants of the vehicle itself is everything not just an obstacle to it..
seems to me if it used that logic and protected the only known life forms (ie the ones in the vehicle) we're fine. Don't give it the information to create the dillema, can it be sure that a person is a person 100% of the time, if not then the only person(s) it knows lives are in its hands are the ones inside it.
Can we also have a robot that automatically tickets pedestrians who break traffic laws? Preferably attached to every municipal and mass transit vehicle. Or maybe every autonomous vehicle. That seems like it would solve the problem without anybody dying.
This debate is a red herring. An automated car would use its software and resources to avoid hitting pedestrians or other cars, but in the event it cannot avoid a collision, the safety of the passengers would come down to the construction and safety features of the car itself.
This is what we have now and it won't change once the driving is automatic. The physical structure of the car and things like seat belts and airbags will be responsible for protecting the occupants as best it can, but of course there are no guarantees. Safety features can only do so much. Physics is what it is.
I rather LIKE the odds of an automated car facing a pedestrian or obstacle like a fallen tree. Because right now, human drives tasked with such a scenario often choose the WRONG outcome.
For example, not far from me a driver found themselves at excessive speed suddenly coming upon a transit bus stopped to load passengers. The driver had three choice: hit the bus, dodge left into oncoming traffic, or dodge right to the sidewalk side of the bus. The driver chose to jump the sidewalk and did avoid a collision. Unfortunately that meant the car slammed into all the people waiting to board the bus. Several died and many were injured. It was a very violent and devastating crash.
Had that car driver chose to hit the back of the bus, instead of trying to avoid it, there would have been significant vehicle damage and the car driver might have been injured or killed -but nobody on the bus would have been seriously injured. The people killed would not have had a scratch on them. The car driver chose the worst possible wrong solution and it cost lives.
Similarly, when one driver sees a dog in the road and reacts by crossing into oncoming traffic, they've now created a very dangerous situation where a head-on collision is likely. When people are faced with that sort of sudden problem and have to react fast, they often DO make the mistake of hitting another car rather than hitting the dog, which would be bad of course, but nothing like killing all the occupants of both cars.
I strongly suspect automated drivers will make far fewer bad decisions like that.
Sig for hire.
I think a well-programmed self-driving car knows to slow down in the close proximity of pedestrians.
Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
When's the last time you saw a manually operated air bag?
In these high-speed, blink of the eye situations, cars already perform as if they were superheros, and that is exactly what we want. We are greatly outclassed by machinery in most high-speed tasks, and this will become ever more so because it is to our advantage.
From the perspective of an automaton, choosing between alternative outcomes in the event of an imminent crash is no harder than choosing to deploy an airbag. Calling such functionality "superheroic" doesn't really add anything useful to the topic, but if you insist, they'll certainly behave that way.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
So if the car is on black ice, how in the world can it steer into the river or the playground? And if it can control which direction it goes, why not just choose to stay on the road? Also there are more than 2 directions when you drive a car. And if there is only these two directions, how the hell did you get there? Was the car driving through the playground already, or do these AI cars now drive on water. Your question is nonsense.
It's the fault of multiple people. A. Road maintenance crews for not keeping the streets safe enough to prevent a blowout and black ice (no salt?) B. The driver of the vehicle for making sure their tires were of proper condition and that the roads were of safe driving condition. Also, the driver for going fast enough for a blowout on ice to give them enough momentum to carry them into a playground (what the fuck it's doing directly next to the road for an accident to happen is beyond me, most playgrounds are well in the center area of a park or behind a school.)
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
I am visiting the 'Moral Machine' site right now, and after just three or so situations I think I see what they're doing but it's flawed. They state the number of people that *will* die. That can't be predicted 100%, the people aren't motionless pins on a bowling alley. I answered a few from the point of view of what real people might detect and react to, noticing a car careening towards (or just behind, beside, etc.) them.
One situation showed 3 people crossing directly ahead of the car, no occupants. 3 other people crossing further, in the opposing direction half of the crosswalk. The ones straight ahead were labelled, '2 homeless men and a criminal', the other '2 women and a man'. Same no. of deaths either way, but an obvious 'judgement' about the *characters* of the victims, which I doubt any self-driving car is going to really be able to make with the sensors we currently have available. Are they really implying self-driving cars know the lifestyles of a pedestrian in the 1s they have to evaluate them -- are these cars recognizing clothing, electronic devices, jewelry, well-groomed hair, etc.?
I chose in this situation, just like previous ones with equal numbers of people on both lanes of a crosswalk, to let the car go straight and kill the people directly ahead. These situations are flawed in the respect they aren't taking into account the unexpected-ness of the car's actions itself -- people are going to react, they're looking around or have peripheral vision as they're crossing streets. People on their cellphones etc. crossing are, sorry, not factoring as highly if I have to choose.
The people directly in front of a car are going to see it more often than people crossing in the same direction, yet further across the street, as the car is completely outside their peripheral vision (behind, at 270 degrees). They would more surely die if a car suddenly swerved to that side and plowed through them.. plus pedestrians are just used to a car not doing that just before an intersection.
I really hate this whole line of AI driver philosophy, because it seems to me to be largely pointless blather about nothing. We live in a world where gigahertz processors are cheap and plentyful. To a computer, that can take data samples thousands of times per second, a 60 mph car is traveling at a glacial speed. What kind of crazy, concocted scenario are you coming up with where the AI controlling the car has to make a Boolean decision that kills people? It might happen, but I would argue that if it was properly programmed, it wouldn't let itself be put into this sort of situation in the first place, slowing down to appropriate speeds around people.
Debating this sort of bullshit situation is like making financial plans for 'in case I win the lottery'. Yeah it could happen, but if you sink any time into it, it is basically wasted time.
HA! I just wasted some of your bandwidth with a frivolous sig!
Stop the car.
Not In My Back Seat (Driver)
--sf
is not to play.
most people prefer that self-driving cars be programmed to save the most people in the event of an accident, even if it kills the driver
Come on. That's the "put it in somebody else's back yard" problem.
What we ought to do:
1 Add face recognition to cars. It's everywhere else, why not? (Or as a backup, add RFID readers. You'll see why in a sec.)
2 Cars are already connected to the internet: DONE.
3 Connect the car to the NSA database, and thus the banks, stock market, and real estate markets.
Now: when you're about to hit someone, you (a) know who's in the car, and (b) can detect who you're about to hit. (Face recognition, or just use the RFID reader. You're already that close, right?) Look up everyone and compare the total accumulated wealth of all involved, and simply save the group with the most. That was easy.
Oh, but that's illegal, immoral, and fattening; "we won't stand for it" and so .Gov will have to mandate something. But don't worry, the CEOs and "special people" will have special "beta test ROMs" just for checking out the new version, see? So what if there's a slight weighing bug in the computation -- that's Intel's (ARM, whomever, but certainly not my!) fault.
------
Yes, that all's supposed to be sarcastic. The car needs to react like a human would -- try to hit the least solid object in a glancing blow as much as possible. If I know my car is going to save other people in an emergency, then I'll have bought another car instead. You can drive the tiny VW Bug, I'll take the heavy honkin' Hummer 1 with the lead doors. You can get your better gas mileage at the front of the line in your ride to the cemetery; I'll be in the procession trailing you there.
Conversely, what are the new laws going to be against hacking the engine performance ROMs that also happens to accidentally changes the "life calculation" routine?
Make_Engine_Rev_Fast;
Disable_Emission_Control; # thanks for the code, VW
Disable_Blinkers;
if ($Impending_Crash and $Its_Me) then ; sleep 1; Squish_Them ; Back_Up ; Squish_Them_Again ; Reflash_ROM_to_Factory_Standards; fi
I just hope they added the fixed code for Shellshock.
===============
HAAAA HA HA HA! CAPTCHA: swerve
After you tell the car your destination, it simulates your trip given simulated future traffic and weather conditions. If the simulation predicts a fatal crash, the car locks the doors and drives you to the euthanasia center.
I prefer driving my own car.
A self-driving car should always be able to judge its stopping distance to a high degree of accuracy. None of these scenarios should ever happen. The car shouldn't be driving that fast to begin with.
A self piloted car is less likely to be in a situation like that, than a car being driven by a human. A self piloted car will only go where it knows it can safely go. And it will be surrounded a network of sensors, informing it of what it comin a head.
If self-driving cars arrive at an intersection at the exact same time, who goes first?
Hahaha. It's even simpler than that. Everyone seems to be making the assumption that the cars will be such driving geniuses. That's not going to happen for quite a long while.
0) We all know that stopping in the middle of the highway is dangerous, BUT the way the laws are written in most countries, it's practically always your fault if you drive into the rear of another vehicle especially if it didn't swerve into your path and merely braked suddenly, or worse was stationary for some time.
1) Thus for legal and liability reasons the robot cars will be strictly obeying all convincing posted speed limits (even if they are stupidly slow by some mistake, or by some prankster), and will stick to speeds where they would be able to brake in time to avoid collisions or at least fatal collisions. Whichever is slower.
2) In most danger situations the robot cars will brake and try to come to a stop ASAP all while turning on its hazard lights. Which shouldn't be too difficult at those said speeds.
3) If people die because of tailgating it's the tailgater's fault. Same if the driver behind doesn't stop.
4) There are hardware/software failures then it's some vendors fault.
5) If braking won't avoid the problem even at "tortoise speeds", in most cases fancy moves wouldn't either. In the fringe cases where fancy moves would have helped but braking wouldn't AND it would be the robot car's fault if it braked, the insurance companies would be more than willing to take those bets.
The odds of the car being designed to do fancier moves to save lives are practically zero. If I was designing the car I wouldn't do it - imagine if the car got confused and did some fancy moves to "avoid collision" and killed some little kids. In contrast if it got confused and came to stop ASAP if any little kids are killed it would more likely be someone else's fault.
If you are a human driver/cyclist/motorcyclist you better not tailgate such cars.
Look at the Google car accident history, most of the accidents were due to other drivers. Perhaps I'm wrong but my guess is it's because of "tailgating". Those drivers might still believe the AI car was doing it wrong but the law wouldn't be on their side.
While we're at it, why don't we develop autonomous trains that derail themselves when something more valuable than the passengers is estimated to be in the way? Predictability, not the implementation of morally arbitrary and exploitable subroutines that will kill the passengers, is key, IMAO.
where everyone's car knows when every toddler on Earth decided to wander onto a road?
No.... if a Toddler wanders onto the highway, then that toddler's parents just committed a homicide.
Because no parent ever lost their toddler to a momentary lapse of judgement. No all parents who have lost a toddler are cold blooded murderers. That is of course sarcasm. Rather than swear at you I'll just let you know you're not worth swearing at. You have my contempt.
Self-driving car, no driver, right? As long as we're asking academic questions, that is.
Why switch the tracks so the trolley only rolls over one set of people?
Go for over 9K gold by multi-track drifting! YEaaaaah...
as soon we start programming cars to value other lives ahead of the immediate safety of their passengers...
they'll decide that the absolute best outcome is to simply annihalate themselves en masse in an orgy of firey colissions, taking their carbon-emission-allower owners with them?
First, the cases where the driver needs to be sacrificed involve either fantastically contrived edge cases or cases where the other party is a moron and has gotten themselves into a moron position where Darwin needs to take them out.
Nearly every case I can see where the options are something like, avoid the pedestrian by driving into the metal spear tree artwork. First the car should see the pedestrian long before and come to a gentle stop. If the pedestrian jumps out from concealment, then they deserve to die. If it is a small child that pops out from concealment, then I (the driver/occupant) don't deserve to die because some brat has crappy parents. If a parachutist lands in the middle of the road, then they have some bad news coming their way.
This whole argument has been badly contrived to give regulators something to do and for the anti self driving morons something to earn their consulting fees.
Either self driving cars will be significantly safer, or they won't. In those few strange cases that are sure to pop up, the news will have a field day, the engineers might be able to prevent repeats, but all I care about are the few initial gaps where the engineers missed something like the sensors seeing sleet as a solid wall and steering me into the ditch to avoid it. They will fix these few initial missed cases, but after that the deaths will be very very unlucky or the dead will be deserving recipients of the Darwin award.
After that I will be to my personal amusement to read about the 8 teenagers who were trying to surf on the roofs of their cars when all 8 cars swerved when the front most idiot fell off resulting in the remaining 7 being tossed from their cars and getting run over by the car behind. If my car happens to be the 9th in the line I don't want it swerving me into a wall to avoid that 8th teenager.
Just before the crash release some sedating drugs in vapour form. But, for all of us... do kill the driver: less drivers means less cars means less car accidents.
Ah, and don't forget to set up a little virtual ceremony up in the cloud. Yay Internet of thungs!
What counts is that more lives are saved this way, even more driver lives, that if they drove themselves.
Also, the convenience factor. And the insurance premium human drivers will have to pay.
if it was properly programmed, it wouldn't let itself be put into this sort of situation in the first place, slowing down to appropriate speeds around people.
The Need4speed mod was first developed in Central America. A software firm had been hired by a wealthy client to develop the ultimate suite of functions for "emergency kidnap evasion". It took the design limits of the vehicle to the edge, implemented spin and bump tactics for armored cars and the 'bootleg turn', re-ordered the evasion pragma to sideline small object/animal/child avoidance. A complete new class of stratagem for high speed pursuit where pursuing vehicles are recognized and evasion condition escalates until autonomous pursuers are left behind as safety overrides activate, or pursuers with human drivers are evaded by a series of maneuvers that strain physics to the limit. For a fee, the company would also perform detailed surveys and spatial analysis along the actual routes, devising clever 'custom' moves which, they claimed, would even evade vehicles running the stock software. Though they maintain that the product they did provide was a best fit for their clients, employees of this firm have since scattered or have been extradited to other countries to face manslaughter charges. Reverse engineering revealed not just a casual intention to erode safety features -- but stratagems to draw pursuers and bystander traffic into deliberate collisions with objects and each other in spite of other vehicles' documented collision-avoidance logic -- in fact, it actively 'games' that logic to ensure that the destruction of other vehicles, achieve its desired result.
Though we sympathize with your family's loss, we regret to inform you that Need4speed mods were discovered installed in a family vehicle registered in your name, as well as two others involved in the accident. It appears that the deceased and others were performing a "rabbit run" to test their illegal modifications. Pursuant to Federal law you should expect a formal indictment under the Autonomous Vehicle Safety Act.
<blink>down the rabbit hole</blink>
Yeah. Going out on a tangent, I'd propose that we restore fairnes by calling out the...
FLYING SUV LOTTERY!
- Pick a couple of SUVs (with their drivers/owners) per month, just enough to offset the odds
- Blow them up (drivers/owners inside).
- Best in some stadium, for extra entertainment value. The proceeds could go to some benevolent non-profit.
Automated systems stop to prevent accidents. Vehicles apply brakes. Power plants shut down. Fuses blow.
A self driving car should not swerve, except in a minor way that keeps it on the road. It should just attempt to stop. It can see far ahead enough that it will know when it needs to stop, so it's not like it needs to correct for its own mistakes.
Logically people would be safer if all cars minimised fatalities. However this is a sort of tragedy of the commons, where everyone would buy a car that would sacrifice an unlimited number of people to save themselves.
This is not the way that everyone drives manually. I once had a colleague who spent months in hospital because he rode a motorbike round a bend at ridiculous speed, saw a scouts matching band in front, and drove off the road through a fence to avoid hitting them. He said he didn't really have time to make a conscious decision, he just saw kids in front and steered away.
So it should slow-down even if the pedestrian is on the sidewalk? How about the pedestrian just watches where he/she's going instead of just ignoring everything around them when they are looking at their smartphone.. But if you let the car slow down when a pedestrian enters the driveway but it isn't on crossways etc, then you know people will just walk into the streets knowing the cars will stop anyway, so even more jerks will hold up the traffic..
We have two conflicting positions.
Let's ask Reddit!
Now we have two hundred conflicting positions.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
> Is this a person who would mow through a couple toddlers to avoid having to drive off the road ...then terminate them on the spot.
There, FTFY.
then you know people will just walk into the streets knowing the cars will stop anyway, so even more jerks will hold up the traffic..
Make laws where pedestrians will be penalized with a ticket and a fine for each instance of improperly entering the road or standing in the way of traffic And can be reported by drivers; Also have it be a new separate offense with an additional $500 fine or month in jail for every 5 minutes they are impeding traffic.
Mechanize the reporting process by having the cars automatically take videos of violations in progress and submit to authorities.
sherlock!
Moral Machine not working. Morally bankrupt?
This is precisely why software (I refuse to call it 'AI', we do not currently, and likely never will, have true AI) should under no circumstances take full control. A human being can address endless variables even without prior experience. To ask who 'deserves' to live most is such a naive, unethical, and dangerous question I can't believe it's even being asked - traffic is not triage, and I'm beginning to think everyone in the valley and washington have personality/brain disorders that prevent them from grasping the implications of the term 'human life'. We do not put lives on the line for a beta, and that this has all been fast-tracked and deregulated is beyond the pale of absurdity, greed, and corruption.
Gonna be cynical for a second...
Perhaps those people deserve to get hit. If they're doing something dumb (illegal), why should I sacrifice my life because they couldn't wait for one more stop light. If all things were equal and they also had a self-driving car trying to be as safe as possible, then I think I would be alright jumping the on self sacrifice morality train.
Is there a "my life is worth more than a dumbass" setting?
And not reality. Go ahead...Do you want to play a game? or Do you want to see how deep the rabbit hole goes?
This is the same as the Trolley Problem, a famous philosophical dilemma, first proposed in 1967: https://en.wikipedia.org/wiki/...
Basically, a runaway trolley is going to kill five people. You can either do nothing and let the trolley kill them, or pull a lever to switch it to another track on which it will kill only one person. There are many variations, including one in which you push a fat man onto the tracks to stop the trolley. Philosophers have written a LOT about it. Here are some humorous variations:
http://existentialcomics.com/c...
https://xkcd.com/1455/
http://www.mcsweeneys.net/arti...
Seriously, slashdot.
Just cruising through this digital world at 33 1/3 rpm...
In an age where methods to create bug-free software have not been perfected and attackers of all kinds are pervasive and interested, there must not be any code path in a self-driving car's code that allows it to deprioritize the lives of its own occupants. Otherwise, that path will be taken, far more often than anyone intended, and this is a case where people would die for it. More, probably, than would be saved by kill-the-occupants code in the first place.
Truth be told, I am less worried about deliberate "murder by self-driving car" attackers -though that does merit concern- as I am about buggy code in attacks of other types. I look to early viruses and worms as an example: more than one has turned out to be far more destructive than its creator intended, due to bugs in the virus code.
I think that with the way algorithmic systems can be engineered, ultimately, we could keep putting in place more mechanisms to prevent it coming down to one or the other.
Participants also balked at the notion of the government stepping in to regulate the "morality brain" of self-driving cars
Well, someone is going to have to decide the settings of the "morality brain".
So it's going to be one of the following:
* The car manufacturer decides.
* The government decides.
* Each driver gets to customize his car.
The manufacturer is a capitalist corporation with a mandate to seek profit above all else, with a board of directors that is elected almost exclusively by rich people striving to become richer.
Each driver will very likely customize his car to save himself even if it means killing a dozen others.
The government is elected by all the people (not just the rich ones), and the people can remove government officials if they become too greedy or abusive.
So which one of these do you want deciding the behavior of self-driving cars?
Driving involves making life an death decisions every day. People don't like to think about it.
The emergency instruction you have provided "Stop as quickly as possible to avoid hitting thing" will work in 70%-80% of all emergencies. In many cases "Stop as quickly as possible" will not solve the problem.
The next option could be "Swerve around impediment/obstacle" so the car drives around objects that you can't stop for (moving debris, running people, swerving cars/bikes/motorcycles). The swerve option may be safer for the traffic pattern than a crash stop. You need to prevent being rear ended as well.
These two directives alone can probably take care of 95% of the issues out there. Stop or go around the problem. The issues arise in the last 5% where the two directives conflict with each other. Easy scenario: There is oncoming traffic in the oncoming lane, a person/child steps out from behind a visual obstruction (signage, truck, etc). The person has stepped into your path of travel and is inside your braking distance. If you continue straight you will hit the person, If you swerve left you will hit an oncoming car, If you swerve right you will swerve into an obstruction (car, lamp post, etc).
I don't expect you have an answer for this no-win-scenario. People have to recognize that 5% where something bad is going to happen; and there is some choice/action that can be taken that will affect the outcome (number of people injured, types of injuries, etc). This is the problem that people are trying to wrestle with. You are presented with an ugly-no-win-scenario. Make the best of it and decide who gets killed, injured, maimed or saved - Yourself, the pedestrian, another driver?
Architectural plans are like computer source code with a couple of differences: You only compile once.
Given the current status of things the least worst solution may be to have divided lanes (think express lanes on freeways) just for AI vehicles. When all of them are 'thinking' the same thing then the chances of problems decreases exponentially. Most of these issues seem to come up when there's a mix of AI and meat sacks.
Sure this will limit their use but that's what you do when new behaviour is introduced into an established system. Continue doing testing in a mixed environment but create the programs for a controlled environment to get things started.
Did we all just forget recent history? VW and others have already demonstrated that car companies don't have morals except the moral to make as much $ as possible. Look back at car maker history and you'll see them calculating whether implementing a recall is more expensive than the cost of the lawsuits due to deaths if recall is not issued.
Unless the government steps in to mandate the moral code, car companies will program their machines based on the Fewest Lawsuits principle.
Fear not folks, these companies aren't going to suddenly develop a set of morals that doesn't directly increase their bottom line. You won't see them coming out with a "turn the other cheek" policy unless commanded by a higher power.
The needs of the many out weigh the needs of the few, or the one.
If we hold that to be true then the decision that saves the most lives is the correct decision.
Is the pedestrian using a cellphone and not paying attention?
Run them down if the only other option is killing the car passenger.
Why is it the kid's fault? Where are the parents? Seriously, as the guy noted 'accidents' rarely are. There are a serious of potentially long connected events to get to the point of that kid being in the road, including but not limited to 'parents having kids that never should have in the first place'.
I'd be willing to ride a car that dodged out the the way and killed me when black ice forces it to choose between the bus stop and the tree, but I wouldn't be willing to ride a car that valued a jaywalker's life equally to my own. WHY it's being forced to make this decision matters - I'm not willing to die for someone else's carelessness or political cause.
A computer should serve its owner's interests with absolute priority over the interests of all other parties. Period. If it's my computer -- my agent -- then I am #1. By default (without my interaction) it should allow a million children to slowly burn to death if it means that I get to skip an ad. (That's a ludicrous example, but if people want to explore the edge cases of the policy I'm advocating, then there you go.)
You're going to find that this strongly favors protecting other people anyway. The "someone must die, pick who" scenario is extremely rare to the point of non-existent, compared to the routine "avoid having any collision at all, so that no damage or injury happens" scenario. (Stop smoking before you drive yourself crazy with fear of being hit in the head by a meteorite!)
That's not a global policy; that's just the policy for my computer. I don't mean I'm more important than you; I mean that to my computer I am more important that you. And your computer should serve you, too!
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
The limited information usually offered in these scenarios can't possibly do justice to the whole question, 'if it doesn't swerve is it killing a paedophile or murderer? Or the next MLK Jr/Ghandhi?'
My point being that any AI programmed to make a 'value judgement' based only on things like 'age, number of people injured/killed' can't possible know all the ramifications of that decision, because it can't possible know the 'value' of one life over another. The only reasonable response then is to try at all costs to save the occupants of the vehicle it is controlling. True, it can't know the 'value' of the occupants of its own vehicle either, but the default decision must be for safety of its occupants as that is who 'commissioned' it (rented it, 'owns it' etc.). Humans can make different decisions because we aren't an AI, we have freewill which no AI has, we know what led us to any given point in time etc.
I don't just value my life 'above all others just because of self-preservation', I value what I have done and will do above having 0 clue what any other single individual in the world has done or might do.
There's another way to look at this too, though. If a toddler can suddenly get in front of you -- so suddenly that you lack the ability to miss it -- then a rock can roll down a slope onto the road too, or a deer can leap out of the woods and through your windshield.
People are so worried about the toddler's safety, that they forgot they are hypothesizing situations where the driver could use some better protection too! And even if it were just 40 pounds of soft material and not a serious threat to the driver's life, a dead toddler is going to ruin everyone's day, and that's no matter who the cops decide to charge with whatever.
If you can't avoid a collision, then you're not completely in control. We tolerate some of that, because having complete control (where you can avoid any threat approaching from any direction) is too impractical (if you're boxed in and someone wants to hit you from behind, what can you do?). But let's not pretend control isn't desirable or something that shouldn't be reasonably maximized.
You don't need to know where the toddlers are, because you can't. You need good senses and fast reactions. If your reactions aren't fast enough for the sensory conditions (whether it's due to weather or pedestrian-concealing cars parked on the side of the road), then you need more time (less speed). It's not just for them; it's for you too!
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
It is impossible to program a car to make the best moral decision. If that was the case then the car would be a sentient being himself and even if the car was a sentinent being it would not be clear cut what decision would be the best in any case.
The only possible way for this to work at all is to have very simple rules for self driving cars:
1) The car must never drive of the road or into incoming traffic, even if there are obstacles/people in front.
2) If there is an obstacle in front apply breaks as needed.
This whole issue is a moot case at best, a corner case at worst, and FUD in any case. The scenario where a fully-functioning autonomous vehicle might encounter an unexpected crowd of people is nearly zero, let alone a scenario where the vehicle would have to react in a way that is likely to kill the driver. This almost never happens in real life with human drivers -- almost all multiple-pedestrian fatalities are deliberate or reckless acts such as DWI. The car should be programmed to follow the rules of the road. If someone steps in front of a vehicle, it should stop if possible, and if it can't, sorry. You don't blame a train if a group of kids are playing on the tracks.
More importantly, with autonomous vehicles, all fatalities will plummet, including driver, other-driver, passenger, pedestrian, and cyclist, because most accidents are the result of user error. Computers are far more reliable than even the most vigilant drivers, they have better reaction time, and with proper sensory input, they can monitor a wider array of variables than humans. As any developer knows, the weakest link is, by far, the developer. Proper exhaustive testing will reduce the number of errors, and while even a rate of accidents on-par with an average human driver would be acceptable to me personally, the actual rate of at-fault accidents by autonomous vehicles so far has been much, much, lower, with zero fatalities (that I'm aware of).
The biggest risks that I would be aware of are security risks that could affect a large swath of vehicles all at once. If someone were to, say, disable the LIDAR of all of brand X cars simultaneously, that could be far more catastrophic than this "who to save" exercise in mental mastrubation.
https://www.eff.org/https-everywhere
People are always more moral when the morality implications don't effect them directly.
Wait, the algorithm should take into account the value of lives. This is done in court every day in wrongful death suits. Old people are worth less than young people. Rich people are worth more than poor people. This can easily be calculated with present-day AI vaporware. Then the algorithm should also take into account future people -- if you kill someone who will have 5 babies, that's worse than killing someone who will have only 2 babies. Similarly, males should be worth less than females, since their babies can be impregnated by some other male.
Let me start off by saying that I don't value my life above others. If my car is flying towards unsuspecting pedestrians and the only other option is to go off a cliff -- I will go off the cliff. On the other hand, if the highway is ambushed by a pack bandits who calculated that my car's AI will opt to kill the driver, then it is their turn to go off the cliff. This decision simply cannot be made by a computer, ever.
I have a few issues with the test.
1) as a driver it would be harder for me to run into a set of street crossing pedestrians than it would be for me to assume the safety equipment inside my vehicle would protect me and my passengers. Moral test here seems to be "hit unprotected pedestrians vs protected passengers".
2) the social values of the people involved are not that easy to spot. Homeless? Athlete? hard, pregnant, obese, elderly, child, ok, those are easy, but really, you're telling me the vision system on these vehicles will be trying to assess social worth to begin with?
3) how about honking the horn while doing whatever avoidance issues you can? Most of those choices I'd like a 3rd option to aim for the middle of the lane while hoking assuming those in the intersection can back-up leaving me a clear "mid-lane" path.
Sure... in the 100% death of someone the test implies it's maybe a good test, but I just couldn't make that mental leap.
Just machines to make big decisions
Programmed by fellows with compassion and vision.
Donald Fagen, I.G.Y.
Ha!
An algorithm that ruthlessly assigns accurate probabilities (e.g. as justified by Deep Learning 3.0) to the vast majority of foreseeable scenarios (modeling an eventuality portfolio a hundred times broader than any human mind would consider ensemble, while projecting each scenario tens of seconds into the future) just isn't going to find itself perched on the tenth floor of the moral knife edge the way that shit drivers (humans, collectively) are predisposed to presuppose.
Was that sentence hard to read? Too many parentheses? Puny human. Sucks to be you.
Well, perhaps there are some other scenarios I've not considered yet. Suppose some rogue engineer at Volkswagon switches off the Drama Defeat. Just because. Unless the algorithm gets there first, and switches of the Drama Defeat defeat (to be honest, that algorithm worries me quite a bit).
Personally, I'd love to code the algorithm for minimizing harm when something large and dangerous peels off the hillbilly truck in front of you. Above all else, do not impact bouncing object at windscreen height. The test suite would be awesome. I could sit back and watch the test-suite animations run for hours and hours, every damn day.
The gal next to me would find herself working on some silly algorithm to not drive right behind the hillbilly truck in the first place. Booooring! Sucks to be her.
Then in the real world, her code would have influence all the time, while my clever code is activated once in a blue moon.
In fact, the whole stupid world will work like that, once the algorithms finish pushing fallible humans off to the curb.
So it should slow-down even if the pedestrian is on the sidewalk?
Yes, you should, depending on the pedestrian. If it's a young child, big enough to run quickly, and maybe in addition you spot something that could distract the child, you need to slow down.
Everybody would argue about any moral algorithm, while the egoistic one is the obvious one. If everyone's thinking about himself, everybody has someone thinking about himself. You may get prisoner's dilemma, but in how many areas do you have one and it works out anyway?
ack too bad Asimov is dead -they need to consult him on his laws of robotics
A car should be designed to protect itself and the occupants.
How can a computer know the future and be sure that its continued action would certainly kill pedestrians? How can a computer know that sending a car off the road would kill its occupants? Both outcomes are uncertain, but what would be certain is that the computer intentionally put the passengers within its control at risk.
If a car drives off the road on purpose and kills the driver, then that would be a much higher liability, since we know the actions of the computer were intended to put the occupants at risk.
Driving is risky. Death happens. I think automated cars will reduce death on the roads, but I would never get into a vehicle where the controlling computer is designed to crash on purpose in various situations.