Slashdot Mirror


The Moral Dilemma of Driverless Cars: Save The Driver or Save The Crowd?

HughPickens.com writes: What should a driverless car with one rider do if it is faced with the choice of swerving off the road into a tree or hitting a crowd of 10 pedestrians? The answer depends on whether you are the rider in the car or someone else is, writes Peter Dizikes at MIT News. According to recent research most people prefer autonomous vehicles to minimize casualties in situations of extreme danger -- except for the vehicles they would be riding in. "Most people want to live in in a world where cars will minimize casualties," says Iyad Rahwan. "But everybody wants their own car to protect them at all costs." The result is what the researchers call a "social dilemma," in which people could end up making conditions less safe for everyone by acting in their own self-interest. "If everybody does that, then we would end up in a tragedy whereby the cars will not minimize casualties," says Rahwan. Researchers conducted six surveys, using the online Mechanical Turk public-opinion tool, between June 2015 and November 2015. The results consistently showed that people will take a utilitarian approach to the ethics of autonomous vehicles, one emphasizing the sheer number of lives that could be saved. For instance, 76 percent of respondents believe it is more moral for an autonomous vehicle, should such a circumstance arise, to sacrifice one passenger rather than 10 pedestrians. But the surveys also revealed a lack of enthusiasm for buying or using a driverless car programmed to avoid pedestrians at the expense of its own passengers. "This is a challenge that should be on the mind of carmakers and regulators alike," the researchers write. "For the time being, there seems to be no easy way to design algorithms that would reconcile moral values and personal self-interest."

18 of 364 comments (clear)

  1. Seems this topic is stuck in the roundabout. by Hylandr · · Score: 4, Insightful

    Here we go again. We just had this discussion last week too.

    If the new slashdot owners are using the client base as fodder for some think-tank the least you could do is provide compensation after the first few times an article is recycled.

    --
    ~ People that think they are better than anyone else for any reason are the cause of all the strife in the world.
    1. Re:Seems this topic is stuck in the roundabout. by Anonymous Coward · · Score: 5, Interesting

      My point is still valid though. False dichotomy. The car should (and pretty much every driver-less car will) use maximum braking power to reduce speed as much as possible. In almost all cases it will do this long before becomes too late to stop without hitting anyone. This gives pedestrians the most time to get out of the way and if it hits them it does so at the lowest possible speed.
      Further, when swerving you run the risk of a pedestrian diving out of the way, in the SAME direction that the car swerves.
      Typically such "oh no I must choose which object hit" scenarios occur when the car is driving recklessly or the driver is inattentive, neither of which should apply to non-hacked self-driving cars.

    2. Re:Seems this topic is stuck in the roundabout. by Hylandr · · Score: 4, Interesting

      Expound on the morality of the issue all you want. The final decision as to whether the outcome was predetermined or premeditated will belong to the jury.

      The real question I want the answer to is who will be on trial? Even then, until there is a sufficient body of judicial precedent I refuse to own, operate or allow to be carted away to my funeral in one.

      --
      ~ People that think they are better than anyone else for any reason are the cause of all the strife in the world.
    3. Re:Seems this topic is stuck in the roundabout. by Anonymous Coward · · Score: 4, Insightful

      THIS!

      I think this topic is really representative of the media scaremongering today :

      1 - Take a situation which presents a moral dilema, however rare it may arise in real life even now ... How many times a day does this exact situation REALLY happens in the US for example? I wanna know to check it is not an imaginary problem!
      2 - Ask the wrong questions about the part of the situation that is the closer to a catastrophic failure that it can be, in a way that sound as scary or horrific as possible, to get the answer you are after : What if YOU have to die to save 10 strangers (and one may be the next Stalin anyway)?
      3 - Make sure to blow up the importance of this extreme-odds problem : like millions of people will die everyday ...
      4 - Find a culprit that is different from your readership : migrants, err ... sorry AI, robots! They're commin' for ya!
      5- Conveniently forget that the problem can be even rarer as AI won't be texting, as even if a glitch happens, it could be corrected after that and for all cars on the road! So really, what is the actual frequency now and what would it be with driverless cars?
      6 - Make it a priority : After all, we don't even know if it is a common problem now, if it will be in the future, but this make nice click-bait headlines and as I enjoy driving if I appeal to the luddite feeling/loss of control fear/hero-complex of readers and sway them I will avoid people to take my wheel/gun from me!

      Really, asking questions like : do you want people to die? and do you want to die? Of course, both will be answer by no, then proclaming people don't want driverless cars is just sleazy ...

      Meteorites fall on earth all the time, they can kill people too, where is our anti-meteorite patriot missile system? Quick crawl back to the caves and call your congress critter to do something about this life threatening problem! YOUR life is at stakes! /s

      Show us the numbers, and projections based on cause of these accidents right now, with number of people involved and outcome. Then you can convince me driverless cars are more dangerous than the actual situation now in that particular case ...

    4. Re:Seems this topic is stuck in the roundabout. by catchblue22 · · Score: 4, Insightful

      I think something that is usually not emphasized is that in most cases, human drivers will not have time to make such moral decisions. If you had time enough to think about moral implications, you would in most cases have time to avoid the accident in the first place.

      --
      This and no other is the root from which a tyrant springs; when first he appears as a protector - Plato (423 to 327 BC)
    5. Re:Seems this topic is stuck in the roundabout. by localman · · Score: 4, Insightful

      I like how everyone assumes people make carefully considered, rational decisions in a high-speed crisis.

      People probably choose to veer away from hitting people because they don't realize they might kill themselves - they just see what is in front of them and sure to happen, and don't have the time or wherewithall to consider the unknown consequences.

      People will reach out to catch a falling knife, too, but that doesn't mean that they thought about the implications.

    6. Re:Seems this topic is stuck in the roundabout. by Pentium100 · · Score: 3, Interesting

      I did not say that I can carefully consider all the outcomes before deciding whether to hit a tree or a man. The time is usually too short to consider anything, other than trying to stop and maybe turning the car in a direction away from any object (maybe unsuccessfully).

      However, computers do what they are told and AI most likely does have time to consider this. Which means that this is now a problem - I do not want the AI in my car (or a taxi driver) to carefully consider the outcomes and decide to kill me instead of a pedestrian. Since it is most likely impossible for the taxi driver to carefully consider all options, I accept that the outcome is gong to be random (he may be too slow to react and hit the object in front whether it's a tree or a man, he may try to avoid the man in front only to hit a tree he didn't notice or he might try to avoid hitting the tree only to hit the man).

      Not so when such situations are considered well in advance (when programming the AI) - in that case I will not want to ride in a car that is driven by AI that will predictably choose to hit a tree instead of a man.

      For the purposes of the example, assume that the speed is high enough that hitting a tree will kill or permanently disable the people in the car, while hitting the man will kill the man, but leave the passengers better off (without permanent disability).

      In addition to that, when I am driving, I am in control and responsible for my decisions (whether they are thought out or I was just too slow to react). Not so, when the AI is in control.

    7. Re:Seems this topic is stuck in the roundabout. by silentcoder · · Score: 3, Informative

      >People probably choose to veer away from hitting people because they don't realize they might kill themselves - they just see what is in front of them and sure to happen, and don't have the time or wherewithall to consider the unknown consequences.

      Well that fits my personal experience. The worst car accident I ever had happened when I swerved to avoid a hazzard on a highway while travelling at high speed. I ended up on the traffic island where I crashed into a tree.
      This is where modern automotive technology makes a huge difference however. Despite hitting a tree head-on at 120km/h I walked away with nary a scratch. Airbags and crumplezones kept myself and my passengers alive and almost entirely uninjured. Car was utterly destroyed, but that's better than humans being hurt.

      But thinking back - yes, that's exactly how it went. When you see a sudden hazard on the road at high speed there is simply no TIME to think through a chain of consequences or evaluate multiple possible chains of events. You can do this when you have more time - but modern ABS enabled cars can probably achieve a safe dead-stop in the same time - but when it's a sudden hazard like a large animal running onto the road out of bushes where it was hidden (as happened to me)
        there is just no time to do that. You deal with the problem immediately in front of you using the first viable option - you swerve to avoid, trying to regain control and avoid subsequent problems caused by the swerve becomes something you think about *after* you've swerved. You may not have the time to actually process what new problems there are and react to them at all (I sure didn't) but you simply cannot consider them beforehand. Not to mention that the bit of thought you can spare is based on the extremely limited information and judgement calls. Part of why I chose to swerve towards the island was that (1) it meant not crossing other lanes which would potentially cause me to hit other cars and (2) the plants on the island appeared to be small shrubs - unlikely to cause major damage even if I couldn't avoid hitting one. Turns out that despite being pruned low - that thing had a massive trunk capable of turning my engine into something resembling an empty tin can in a vaccuum.

      --
      Unicode killed the ASCII-art *
    8. Re:Seems this topic is stuck in the roundabout. by dave420 · · Score: 4, Insightful

      Your car is filled with airbags and seatbelts and crumple zones and all sorts designed to protect you during a crash. Pedestrians have none of that (at least for the time being). The CAR should protect you (using those safety features), the AI should do what drivers are supposed to do - cause the least amount of carnage on the road.

    9. Re:Seems this topic is stuck in the roundabout. by Wycliffe · · Score: 3, Insightful

      That is extremely narcissistic and the argument is only valid if you always go for the tree.
      You are in control of the vehicle. The pedestrian does not share that responsibility.
      It is also you who put the car in a situation where you have to choose between hitting the pedestrian and the tree.
      There is no moral justification for not going for the tree in that case.

      What about on a limited access highway where there is a reasonable expectation that people aren't suppose to be?
      What about in the situation where the person intentionally jumps out in front of traffic in an attempt to commit suicide?

      The issue I have with this question is that I doubt they are going to be programming the car to count the number of passengers in the car, the number of pedestrians or even distinguishing between a person and a deer. To make an accurate crash prediction you would likely want to even know the diameter of the tree and what is behind the tree. The goal of driverless cars is to avoid most crashes in the first place. The idea that someone would be adding all this moralistic code for rare never really happens events is a bit far fetched. In almost all cases, the goal of the car is to avoid collision and if that is not possible to minimize the speed of impact. After that, it becomes very complex because you have to look at what you're impacting and how much give it has. A deer/person has more give that a cement pillar and would actually be a safer option. Also, in many cases, staying on the road and hitting the deer/person would be safer than swerving and rolling down an embankment even if you technically didn't hit anything. It would be interesting to know though if they are coding for doing different behaviors based on whether the unknown obstacle on the road is a dog, a deer, or a person because many people would make different calculations depending on whether the animal in the road is human or not.

  2. The moral dilemma of posting dupes by Yvan256 · · Score: 4, Informative
  3. Crowds of teens will jump into the road as a joke by e1618978 · · Score: 5, Interesting

    ... and laugh and run off as the driver's car kills the driver.

  4. Intelligent Steering by SmaryJerry · · Score: 5, Funny

    Self-driving cars will have face recognition, evaluate the net worth of the targets compared to the net worth of the driver and choose who lives accordingly.

  5. I hope the car AI decides... by Nova+Express · · Score: 4, Funny

    ...to run over whoever keeps posting this dupe.

    BOOM! Problem solved!

    --
    Lawrence Person (lawrencepersonh@gmailh.com (remove all "h"s to mail)

    http://www.lawrenceperson.com/

  6. I don't think the algorithms work this way by Chuckstar · · Score: 3, Insightful

    As far as I can tell, the autonomous algorithms don't work this way and probably never will work this way. That is, they don't calculate potential fatalities for various scenarios and then pick the minimum one. The car's response in any particular situation will be effectively some combination of simpler heuristics -- simpler than trying to project casualty figures, while still being a rather complex set of rules.

    Take one of these situations, and let's say the car ended up killing pedestrians and saving the occupants. The after-incident report for an accident like that is not going to read "the algorithm chose to save the occupants instead of the pedestrians". It's not going to read that way simply because that's not how the algorithm makes decisions. Instead the report is going to read something like "the algorithm gives extra weight to keeping the car on the road. In this situation, that resulted in putting the pedestrians in greater danger than the car's occupants. However, we still maintain that, on average, this results in a safer driving algorithm, even if it does not optimize the result of every possible accident."

    And regarding the "every possible accident" part of that: it is simply impossible to imagine an algorithm so perfect that, in any situation, it can optimize the result based on some pre-determined moral outcome. So it's not just "well, let's change how the algorithms work, then". Such an algorithm that makes driving decisions in any possible weird decision based on predicting fatalities, rather than relying on heuristics (however complex they are) is simply not realistic.

  7. Not even think-tank shit. by khasim · · Score: 3, Insightful

    1. Any company TRYING to write code with the intention of killing/injuring the user will be sued out of existence.

    2. Whichever executive ordered the techs to write such code would never work again.

    3. Even if you allow a theoretical situation that bypasses #1 & #2, complex software is very difficult to write. The company (and executive and coders) would be sued out of existence when the car killed/injured the passenger to avoid running over a box of toy dolls.

    And yet we keep seeing this bullshit on /. People here are supposed to be more informed on the topics of AI and robotics and programming than the average. But here we are, again.

    1. Re:Not even think-tank shit. by Anonymous Coward · · Score: 3, Interesting

      The situation won't actually happen in real life... take NYC for example. Speed limit is 25mph just about everywhere---self driving cars *will* actually drive 25mph. At that speed, unless the pedestrian jumped right in front out of nowhere, the car can stop on a dot.

      Now imagine the pedestrian really did jump right out of "nowhere"; is that the fault of the car? And yes, 25mph hit would hurt, but with telemetry of the incident, it's gonna pretty easy to prove that the pedestrian was suicidal.

      Now the supposed situation in the story, you're going on a highway at a respectable 65mph, and a group by teenagers hops right out from say behind a tunnel post or something---considering stopping distance, car has to either change lanes or hit them (all at the same time trying to reduce speed). So lets assume they hop out say 20 feet ahead of the car, and lets say there's no place to change lanes safely (edge of a cliff)... I'd say the car *should* hit them (at reduced speed as much as it can) instead of say sending the drive off the cliff at 20mph (reduced from 65 after breaking).

      There's safe behavior expectations on the road... and if you're the kind of asshole that jumps in front of moving cars without regard for safety, then perhaps you should get the Darwin award :-)

  8. Dude, you're messed up. by stomv · · Score: 3, Interesting

    I salute your honesty, but in a situation where the outcomes are known in advance, you'd prefer breaking somebody else's leg to a total loss on the car?

    Even if the leg heals up fully, the pain could be tremendous. The inconvenience massive -- perhaps the victim lives on the 3rd floor? How about work -- lots of people require mobility for their job (think: waitress). Oh, yeah, and the financial cost to repair the leg could easily outpace the cost of replacing the car.

    You'd rather break someone else's bones than total a car where everyone escapes injury free? That's messed up.