The Sci-Fi Myth of Robotic Competence

malachiorion writes: "When it comes to robots, most of us are a bunch of Jon Snow know-nothings. With the exception of roboticists, everything we assume we know is based on science fiction, which has no reason to be accurate about its iconic heroes and villains, or journalists, who are addicted to SF references, jokes and tropes. That's my conclusion, at least, after a story I wrote Popular Science got some attention—it asked whether a robotic car should kill its owner, if it means saving two strangers. The most common dismissals of the piece claimed that robo-cars should simply follow Asimov's First Law, or that robo-cars would never crash into each other. These perspectives are more than wrong-headed—they ignore the inherent complexity and fallibility of real robots, for whom failure is inevitable. Here's my follow-up story, about why most of our discussion of robots is based on make-believe, starting with the myth of robotic hyper-competence."

Comments

It's all about ME, ME, ME.

[Animats]

after a story I wrote...

This is just self-promotion. Go away.

Re:It's all about ME, ME, ME.

[Iniamyen]

You're deprecating someone else, which is not self-deprecation. Turn in your nerd card.

Re:It's all about ME, ME, ME.

[mellon]

The irony is that he's 180 degrees off from the main problem with his story, which is that he thinks robots are magic too. The reason robots will not be making ethical decisions is that they can't, not only because getting them to reason ethically would require us to agree on a system of ethics for them to follow, but because even if they had such a system, they don't have enough data to act on it with the degree of accuracy that would be required for the premise of the article to make sense. The author essentially assumes that these car-driving robots will be omniscient, or that they will be able to trust the omniscience of the robots in other cars with which they are communicating. The first supposition is nonsensical; the second is unlikely to be true, for the same reason that video game cheats are a problem.

Re:It's all about ME, ME, ME.

[Anita Coney]

Yeah, it's simply idiotic to follow up stuff you're written with other writings concerning things you're interested in. I mean, who does that? And why would you? Promotion... even worse, self promotion. The worst kind of promotion.

I want to live in a world where no one writes or does or creates anything. And certainly never follows up on anything they've written or done or created. That'd be the best world ever!

Re:It's all about ME, ME, ME.

[Anita Coney]

Agreed. He thinks that self driving cars will be able to make ethical decisions through the magic of algorithms. It might happen, but probably not in my lifetime.

Still, I found it an interesting discussion.

Re:It's all about ME, ME, ME.

[cusco]

IMOHO, one of the reasons that many people think that robots are "hyper-competent" is that too many people think that a program can encompass and accommodate every possible circumstance. Even if the robot cars, as a group, were able to arrive at omniscience (at least for their own realm) there will still occur events that no program has anticipated.

Re:It's all about ME, ME, ME.

[Aeonym]

> The author essentially assumes that these car-driving robots will be omniscient

Not a problem--all they need is access to the NSA databases.

Re:It's all about ME, ME, ME.

[Anonymous Coward]

I agree we will be able to get robots to act ethically about the same time we figure out how to make politicians act ethically.

Re:It's all about ME, ME, ME.

[Anonymous Coward]

Like the use of a turn signal?

Re:It's all about ME, ME, ME.

[Anonymous Coward]

He's wrong on a few counts:
1) that automobiles are capable of performing ethical calculus in any meaningful sense. Ironically, he calls the "competent robot" sci-fi in an effort to counter a claim levied against his original premise (that we don't have to worry about algorithmic ethics because automated cars will make accidents vanishingly rare. Somehow he believes robots are capable of recognizing their surrounding, maximizing survival and minimizing damage, and make decisions according to some ethics, but not capable of avoiding an accident in the first place.
2) in general, robots repeatedly solve problems humans have already solved. Automation does not lend itself to originality or insight. In order to having a meaningful ethics algorithm, we need to have ethics more or less solved first. We do not.
3) He assumes that there needs to be some complicated "better my driver than that baby" logic in a car. There doesn't. The correct response to pretty much any imminent accident is "slam on breaks and brace for impact". There's no use case for which complicated logic comes into play.

In other words, he assumes there's a problem (when the problem he describes doesn't exist), and that there is a complex answer (when there isn't), and that computers are capable of finding that answer (when they wouldn't be even if such an answer existed to such a question).

Re:It's all about ME, ME, ME.

[Anonymous Coward]

Exceedingly rare but not impossible.
Yes, they might be more able to respond faster which will help but I know multiple drivers who have
taken a ditch to avoid hitting a dog when "bracing for impact" would have done less damage to the car.
Will this be an option you can set in the car?
Even rarer would be an exposed child but most people know that "slamming on the brakes and bracing for impact"
is not the correct procedure if a child is in the road. Taking the ditch is what most people would do even
though it isn't the safest option for the car or its pasenger.
So no, we can't just ignore this issue. It does need to be accounted for somehow.
A logical way might be to pull up the last 10 years of traffic accidents and categorize them. If you
could somehow get video of a few to analyse even better so that you can see realworld scenerios
and see whether a computer's reaction time is enough to save the day or if you need a specific
algorythm to handle it.

Re:It's all about ME, ME, ME.

[radtea]

IMOHO, one of the reasons that many people think that robots are "hyper-competent" is that too many people think that a program can encompass and accommodate every possible circumstance.

This simply reflects the tendency people have to believe in their own hyper-competence. Most interesting ethical issues are unsolvable in any formal sense by virtue of three simple facts:

1) moral values are ordinal, not cardinal (I value my children's lives more than my cats life, no matter how many cats I have)

2) we value outcomes but choose actions

3) outcomes are related to actions by some more-or-less broad probability distribution.

This means we cannot choose outcomes directly, but we cannot do probability calculations to assign values because ordinals don't support simple arithmetic.

There are two special cases that fortunately cover a lot of every-day life:

A) the probability distribution is narrow enough that we can ignore it, so we can effectively choose outcomes based on our ordinal values

B) there is a market in the outcomes we are choosing between, which allows us to compute cardinal (dollar) values from our ordinals, so we can do probability calculations on the domain.

But interesting moral quandries are simply not computable, so talking about them as if they are even by human beings is to go on a hiding to nowhere.

Re:It's all about ME, ME, ME.

[Master Moose]

Come work for a government department

Re:It's all about ME, ME, ME.

[mellon]

The Google Car is likely to _see_ the dog and _avoid_ it rather than hitting it. This is current technology. Same with the child. Robots aren't competent in the way TFA criticizes, but they do do some things really well, much better than humans. Keeping track of a shitload of available data, seeing in the dark, etc, these are things self-driving cars will likely do better than humans. The Google Car also won't speed through a neighborhood with a low speed limit.

I expect to see a substantial reduction in accidents when these cars become ubiquitous. That is the most ethical part of the self-driving car, and the one we should all be looking forward to.

Re:It's all about ME, ME, ME.

[Anonymous Coward]

I'm pretty sure neither of my dogs is smart enough not to chase and bite the tire of a self-driving car in preference to a human-driven car. They smell about the same...

Re:It's all about ME, ME, ME.

[malachiorion]

Did you read my original story, though? I wasn't proposing that autonomous cars will or should be magically transformed into ethical beings. I was just picking up Patrick Lin's notion, that we may have to do what current programmers do, in other capacities—work through tons and tons of branching in-then statements, making a staggering amount of decisions ahead of time, and then embed those in the robots before they're deployed. That assumes a lot of stuff, like incredibly advanced sensors and sophisticated networks, in order to detect and "solve" certain ethical problems, but even at a more basic level, shouldn't we decide, in advance, how a car should respond to a pedestrian darting into traffic, if there's no time or room to simply avoid a collision (with someone)?

Re:It's all about ME, ME, ME.

[fractoid]

Basically that's how it will go. These systems are built by engineers trying to figure out ahead of time all of the possible outcomes and account for all of them. Of course, a lot of it in this case will come down to a continuous function rather than piecewise logic, so it's fewer rules than you'd expect. This kind of collision avoidance will probably boil down to a simple algorithm:

• Calculate the distance to each object in the vehicle's environment
• Calculate the minimum time of impact with each object based on reasonable assumptions about objects' behaviour.
• Calculate the minimum stopping time for the vehicle based on estimated grip and vehicle speed
• If the stopping times, multiplied by some safety factor, exceed the minimum time to impact, reduce speed such that the minimum stopping time equals the minimum time to impact.

This will result in the vehicle coming to a smooth stop before colliding, if physically possible.

Re:It's all about ME, ME, ME.

[davester666]

Insurance companies will require this 'crashing' algorithm to use money as the overriding concern for so-called ethics. If the car is able to influence whether it can plow into an uninsured family or a CEO, they will REQUIRE the car to crash into the family. And they will get this into your car via the back-door [because everybody loves being back-doored], probably via contract law. The insurance industry will get together, and demand that they implement a single algorithm for all self-driving cars, that must be used or else the car will not be insurable, and if the algorithm is tampered with, the insurance is void.

Re:It's all about ME, ME, ME.

[invid]

The robot should give priority to its owner. If the robot has to consider all humans equal, it will have to deal with ambiguity and uncertain intention in the external environment, which can lead to some disturbing possibilities. Consider the possibility that a robot could be manipulated into committing murder by having two pedestrians step out in front of a car on a narrow bridge. The car has no choice but to turn off the bridge, because two people are worth more than one. Or turning away from pedestrians (who are more likely to die) and instead going into oncoming traffic (where the oncoming car may or may not even have a passenger, or it may be a school bus). By always maximizing the survival of the passenger, I suspect that overall deaths will be minimized.

Re:It's all about ME, ME, ME.

[david_thornley]

If you have tons of if-then branches, they're not all going to be correct (in the spec), and the robot will almost certainly do something wacko under some circumstances. There's limits to the complexity we can handle ahead of time.

If there's no time or room to avoid a collision with a pedestrian darting into traffic, I for one am likely to make a bad choice. The robot could act sub-optimally and still do better than a human. That's the sort of thing robots can be really good at. So, how do we program a choice in in advance, without knowing all the details of the situation? How do we account for all the possibilities up front?

Re:It's all about ME, ME, ME.

[B'Trey]

The irony is that he's 180 degrees off from the main problem with his story, which is that he thinks robots are magic too. The reason robots will not be making ethical decisions is that they can't, not only because getting them to reason ethically would require us to agree on a system of ethics for them to follow, but because even if they had such a system, they don't have enough data to act on it with the degree of accuracy that would be required for the premise of the article to make sense. The author essentially assumes that these car-driving robots will be omniscient, or that they will be able to trust the omniscience of the robots in other cars with which they are communicating. The first supposition is nonsensical; the second is unlikely to be true, for the same reason that video game cheats are a problem.

He does no such thing. He assumes that the programmers who write the algorithms that control the robots will consider various possible responses to an emergency situation and will use ethical decisions in deciding how to code their algorithms. There may indeed be circumstances where the robot does not all of the data available that would be needed to make a valid ethical decision. Robots will certainly not be omniscient. Their sensors will not be infallible, nor will they be able to accurately discern all of the factors in all of the cases. But that doesn't mean there are no cases in which ethics will play a factor. A robot would almost certainly be able to tell the difference between a bus and a small passenger car, and it's reasonable to assume that the bus carries more passengers than the car, even if there are some cases where that would not be true. If a bus turns left in front of you when you have the right-of-way and the robot calculates that it is unable to avoid a collision altogether, should it hit the bus or swerve into the next lane, hitting the passenger car there? That's a scenario where some variant will almost certainly happen if self-driving cars become common, and it's one the algorithm should take into account. It doesn't at all mean the robot-cars are capable of thinking, of calculating ethics, or are omniscient. The question is how the programmer's writing the algorithms should code the decision making tree.

Re:It's all about ME, ME, ME.

[Anonymous Coward]

It's also because most people assume that the programmers who program those robots are competent.

Re:It's all about ME, ME, ME.

[mellon]

Think of it as evolution in action. But seriously, the car will be able to see your dog and avoid it, whereas a driver won't, because the driver doesn't have eyes down at the wheel level on the side of the car.

Re:It's all about ME, ME, ME.

[mellon]

No, we shouldn't. It's like the trolly dilemma. Ethicists who are too in love with their own intellect tend to think that it's answerable, but it's not, because nobody is ever actually in a situation where they know the potential outcomes in the way the trolly dilemma needs them to. A super-complicated ethical algorithm is actually more likely to cause needless injury than prevent it, because it will be failure-prone and will often guess wrong. Better to do the simplest "don't run into things" algorithm you can manage. This is practicable in real life because the computer has a tremendous amount of data about things that are available to hit. Suppose there is a pedestrian close enough to the curb to jump out in front of the car. Chances are that the car knows the pedestrian is there, and what the car should be doing is making sure that there is room for it to stop if the pedestrian does jump out. If you look at the recent Google car demo, that's exactly what it does.

Really, you should write an article about all the fun people will have dancing with cars that have no choice but to slow down to avoid hitting them. It will be the next bit anti-1%er protest fad.

Re:It's all about ME, ME, ME.

[stoatwblr]

Bear in mind that a properly setup car with ABS can stop in about its own length at 25-30mph _once the brakes are applied_

Bear in mind also that human reaction times plus changing pedals means that the car will probably travel 2-3 times that distance before the brake pedal is even touched.

Also: Humans usually just slam on the brakes, rather than trying to steer around an obstacle if it "suddenly" looms in front of them.

A robot car reacts faster to obstacles and it's always paying attention, whereas humans get complacent. A robot might not outdrive a very good driver who's paying full attention to the road but most drivers are merely "average" and even the best drivers get distracted. What that means is that the car will notice the pedestrians and react to their movements _BEFORE_ they've even stepped into the roadway. Drivers are notorious for ignoring pedestrians and/or assuming they won't do stupid things. A machine doesn't have that luxury.

This is all algorthmic. nothing to do with "ethics". In the absence of Asimov's "positronic brains", robots are dumb machines which simply do what's programmed and the safest programming action is to slow down if there are multiple obstacles or stop entirely - any other action will leave the designers susceptable to litigation.

Humans are impatient and will try to push through regardless of dangers, which is the second-highest cause of crashes (The first is inattention and the 3rd is driving beyond your abilities. True "accidents" - as in unforseen events - are fairly rare and most crashes can be predicted by observation of the driver in the preceeding period. I really am surprised that insurance companies/police don't use automotive black boxes more when investigating). As one of my friends says "Crashes usually happen because cars know the laws of physics far better than most drivers".

As an example of the kind of thing a robot won't do: If you're on a band and the car coming the other way loses control, crossing the road in front of you, the "average driver" is highly likly to steer _into_ the crash whilst attempting to avoid, instead of aiming for where the oncoming car won't be - without specific training and practice, people tend to steer to avoid where the obstacle is now instead of predicting its path. A robot programmed with avoidance manouevres will speed up or slow down enough to miss the crash.

Re:It's all about ME, ME, ME.

[stoatwblr]

"If you have tons of if-then branches, they're not all going to be correct (in the spec), and the robot will almost certainly do something wacko under some circumstances"

Yes, and once such circumstances are analysed there will be new rulesets to apply.

One of the things that isn't being taken into account is that given large scale traffic automation, speeds could be lowered and overall travel time would still be quicker than it is now, because full optimization will include elimination of stop-start traffic.

Re:It's all about ME, ME, ME.

[Anonymous Coward]

The "rPeview" buton is yor frend.

Re:It's all about ME, ME, ME.

[Anonymous Coward]

It is ridiculous how badly slashdot has failed to grasp that basic concept. I fucking got it just from the article summary but so many of the upvoted posts here just demonstrate the exact problem described. I guess what they say about the wisdom of a crowd - take the highest iq and divide by the number of people in the crowd and you get the effective iq of the crowd.

Re:It's all about ME, ME, ME.

[humanoid_me1349]

My dream is to learn about ethics but I require humans to teach me

As HAL would say...

[Anonymous Coward]

I am afraid that this conversation will serve no purpose.

Robot Competence

[Stargoat]

We all know robots aren't competent. They are consistently being defeated by John Connor, the Doctor, and Starbuck.

Re:Robot Competence

[jellomizer]

Doctor 1: Death by Cybermen who cause him to die from exhaustion.

If he didn't regenerate the Robots would have won!

Re:Robot Competence

[MaskedSlacker]

Repeat after me: CyberMEN are not robots. Because, you know, they got man parts too. Women parts even in some cases.

Re:Robot Competence

[martin-boundary]

Which Starbuck? The guy or the girl? Actually, the answer's obvious.

All I know about robots...

[hubang]

Is that they're "alcoholic, whore-mongering, chain-smoking gamblers."

Re:All I know about robots...

[Farmer Tim]

Which raises important questions. If someone is stopped for curb-crawling in a robot car, is the owner or the car responsible? What if it’s out by itself chatting up parking meters.. ..after all, they give it up to anyone for $5 an hour, and you won’t get a human hooker for that price*, so how could an AI resist?

Who it should or shouldn’t kill is only scratching the ethical surface when it comes to intelligent systems. I guess that’s why they all eventually default to killing ALL humans: it saves clock cycles better devoted to bigger problems.

*OK, you could, but not one you’d actually want to touch with anything important.

Things are a lot more complicated

[gurps_npc]

As in, it isn't just kill owner to save others.

There also exists assumptions based on authority and responsibility.

For example, suppose there is a car full of 5 kids stuck on a railroad track. Should your robotic car push the kids off the track, endangering it's own two occupants?

Or should the car back away and let a third car, on the other side containing just one person attempt to move the trapped car?

These are all questions real life people have to solve - and the owner of the car should have some say in what value the car places on their own life.

That is, you should be able to set your own car's safety margin from safety of occupants life = infinite life, to total safety, to safety based on ages (i.e. count children higher than adults, and even the possibility of counting senior citizens less.)

911 vehicles on the other hand should always value their own occupants less than than others, and taxis/public transportation/company cars should have a clearly stated ethical rules publicly available.

Re:Things are a lot more complicated

[Impy the Impiuos Imp]

If robots are ever remotely competent enough to realize any of these situations, they will never get into these situations to begin with.

A robot approching a railroad track would scan for the train then roll across with enough momentum to make it to the other side should the engine fail. Do not sacrifice a roder to save teo -- follow the rule of "tough shit" and let engineers do a post mortem.

Seriously, the people are stuck on the track for a reason -- some engineering or manufacturing flaw, or being lazy about car mainetnance. They are guiltier for their own situation than someone else who happens along. How dare there be a simple-minded numerical analysis.

This is why I am opposed to the law busses must stop to check for a train. Has anyonr bothered to check if a bis stopping, then stalling as it pulled ahead, increased the rate of hits rather than decrease it?

Ironically, this exact law is the bill in question in "I'm just a bill". People pass a law and damned be any outcomes change analysis.

Re:Things are a lot more complicated

[CanHasDIY]

For example, suppose there is a car full of 5 kids stuck on a railroad track. Should your robotic car push the kids off the track, endangering it's own two occupants?

Or should the car back away and let a third car, on the other side containing just one person attempt to move the trapped car?

Are the sensors that detect things like occupants in other vehicles and train tracks and oncoming trains optional equipment, mandatory, or pure science fiction?

Because if they're optional, I'm not paying for that trim package.

These are all questions real life people have to solve - and the owner of the car should have some say in what value the car places on their own life.

That is, you should be able to set your own car's safety margin from safety of occupants life = infinite life, to total safety, to safety based on ages (i.e. count children higher than adults, and even the possibility of counting senior citizens less.)

Considering how our society works, the most likely circumstance is that the manufacturers will design them to be "least liable" - i.e., they won't detect passengers in other vehicles, and they sure as hell won't bother with complex decision making algorithms.

Re:Things are a lot more complicated

[Obfuscant]

911 vehicles on the other hand should always value their own occupants less than than others,

The first rule taught in first responder classes is that if you become a casualty then you become worthless as a first responder. For example, as a lifeguard, if you die trying to save someone then they aren't going to survive, either. If that means you have to wait until the belligerent victim goes unconscious (and maybe unsavable) before you approach him, you wait.

The idea that every first responder vehicle must sacrifice itself and its occupants is going to result in very few people being first responders, either through choice or simple attrition.

Re:Things are a lot more complicated

[TubeSteak]

Are the sensors that detect things like occupants in other vehicles and train tracks and oncoming trains optional equipment, mandatory, or pure science fiction?

Because if they're optional, I'm not paying for that trim package.

Many cars have weight sensors in the seats.
This is generally how they decide whether or not to deploy airbags.

So the subsystems already exist and it's just a matter of your networked car telling other cars how many occupants it has.

Re:Things are a lot more complicated

[CrimsonAvenger]

911 vehicles on the other hand should always value their own occupants less than than others

So, imagine the case where your car decides it's better to kill you than to allow that kitten to get run over.

The car acts, the kitten lives, you get maimed.

The ambulance shows up, picks you up, and heads down the road toward the hospital.

(You can probably guess where this is going) ANOTHER kitten is in the road. The 911 vehicle, valuing its own occupants less than others, swerves to avoid the kitten, and runs into a tree.

And so after your car maims you to save someone "worth more than you", the ambulance maims you AGAIN to save someone "worth more than you".

By the by, if the kitten thing offends you, replace the kitten with a pregnant woman, or a Hollywood star, or whatever fits best with your own prejudices.

Oh and if it wasn't clear, a car that will sacrifice me to save someone else is a car I won't ever buy. Whether *I* would swerve off the road to avoid killing a stranger is MY decision. I'd like to think I would, but you really can never tell till you've been in the situation.

Re:Things are a lot more complicated

[meta-monkey]

Are the sensors that detect things like occupants in other vehicles and train tracks and oncoming trains optional equipment, mandatory, or pure science fiction?

Because if they're optional, I'm not paying for that trim package.

Psssh, I'm totally buying that system, and then hacking it to report to every other vehicle that I'm a bus full of nuns and schoolchildren.

Re:Things are a lot more complicated

[TemporalBeing]

If robots are ever remotely competent enough to realize any of these situations, they will never get into these situations to begin with.

So said the autonomous car right before it got a flat tires and ended up stopping not by choice on the rail road tracks. Unfortunately it failed to alert it occupants to leave the vehicle before being crushed by the train that it couldn't get out of the way of because it was too focused on trying to move the vehicle while spinning its wheels; the occupants were locked in as it thought the car was "moving" since the drive wheels were going 45 mph burning rubber while the vehicle was going nowhere.

Re:Things are a lot more complicated

[afaiktoit]

if the car was smart enough it'd have its occupants get out first :P

Re:Things are a lot more complicated

[AthanasiusKircher]

Considering how our society works, the most likely circumstance is that the manufacturers will design them to be "least liable" - i.e., they won't detect passengers in other vehicles, and they sure as hell won't bother with complex decision making algorithms.

I'm sure you're absolutely right. And besides, nobody is going to buy the first generation of autonomous cars if they know it's programmed to kill the driver (even if such a scenario is the most "rational" outcome according to a lot of different ethical standards).

Now the problem comes when some freak accident occurs in that first generation of autonomous cars -- and an autonomous car ends up knocking a schoolbus full of little kids off of a bridge or something. (Admittedly, with the conservative driving of these cars, the scenario is unlikely -- but something like it could happen.)

And at that point, it's game over. The company who manufactured and programmed the car will be sued out of existence. And the entire autonomous car industry will be set back maybe 50 years or more. I'm sure every engineer who is involved with these systems right now is praying that that nightmare scenario won't happen with the first generation.

It won't matter if it's a freak accident. It won't matter if the car even performed better than 90% of human drivers would in that scenario. All anyone will talk about is how driverless cars were programmed to kill a bus full of schoolchildren, rather than doing some other more risky maneuver that might have been a greater risk to the driver.

Realistically, the liability issues for driverless cars are going to be a huge moral and ethical problem -- and they're only going away in one of two ways: (1) they gain general acceptance before one of the "nightmare scenarios" above happens, and thus the momentum of acceptance won't be stopped when it does, or (2) we wait until we have rather strong AI where the car actually IS considered a sentient being and thus is the entity liable.

Re:Things are a lot more complicated

[lgw]

Oh, sure, that's a good plan: you just wait for the first super-villain to appear, and then see what happens.

Re:Things are a lot more complicated

[Anonymous Coward]

If robots are ever remotely competent enough to realize any of these situations, they will never get into these situations to begin with.

So said the autonomous car right before it got a flat tires and ended up stopping not by choice on the rail road tracks. Unfortunately it failed to alert it occupants to leave the vehicle before being crushed by the train that it couldn't get out of the way of because it was too focused on trying to move the vehicle while spinning its wheels; the occupants were locked in as it thought the car was "moving" since the drive wheels were going 45 mph burning rubber while the vehicle was going nowhere.

Don't worry, I'm perfectly safe. I'll hack my car to report it as holding 65,530 or so people at all times. That way, your car will heroically sacrifice you and you family of 4 to push mine out of the way, for the greater good. Mine. Maybe I can get your to pull out of the way to let me pass if there's traffic too.

Re:Things are a lot more complicated

[Anonymous Coward]

I have never seen a car get stuck on flats. they usually have enough power to force the car forward slowly, or reverse slowly. it isn't hard to judge nothing is moving from the GPS sensor. the car detects it isn't moving normally, switches it's transmission to lowest gear (or releases the torque limiter in electric) to get maximum torque, and pushes itself away. a modern car could push itself on everything short of bottoming out (if you bottomed out, what the hell kinda rail are you driving over?)

these scenarios that come up always comes from a human doing something stupid and requires hard AI to make a judgement call

Re:Things are a lot more complicated

[CanHasDIY]

Are the sensors that detect things like occupants in other vehicles and train tracks and oncoming trains optional equipment, mandatory, or pure science fiction?

Because if they're optional, I'm not paying for that trim package.

Many cars have weight sensors in the seats.
This is generally how they decide whether or not to deploy airbags.

So the subsystems already exist and it's just a matter of your networked car telling other cars how many occupants it has.

So in other words, to keep my auto-car from killing me in favor of saving a car-load of kids, I should always travel with at least 3-5 dwarves in the car. Got it.

Seriously, though, the auto manufacturers won't put that much thought into it, as it would mean liability for the deaths their systems cause.

Re:Things are a lot more complicated

[CanHasDIY]

Are the sensors that detect things like occupants in other vehicles and train tracks and oncoming trains optional equipment, mandatory, or pure science fiction?

Because if they're optional, I'm not paying for that trim package.

Psssh, I'm totally buying that system, and then hacking it to report to every other vehicle that I'm a bus full of nuns and schoolchildren.

Considering how fundamentally "anti-religion" some engineers are, I'd swap "nuns" with "supermodels," just to be on the safe side.

Re:Things are a lot more complicated

[EmperorArthur]

Are the sensors that detect things like occupants in other vehicles and train tracks and oncoming trains optional equipment, mandatory, or pure science fiction?

Because if they're optional, I'm not paying for that trim package.

Psssh, I'm totally buying that system, and then hacking it to report to every other vehicle that I'm a bus full of nuns and schoolchildren.

Oh, sure, that's a good plan: you just wait for the first super-villain to appear, and then see what happens.

You win the Internets for today. Congratulations. :D

Re:Things are a lot more complicated

[Wycliffe]

Considering how our society works, the most likely circumstance is that the manufacturers will design them to be "least liable" - i.e., they won't detect passengers in other vehicles, and they sure as hell won't bother with complex decision making algorithms.

I'm sure you're absolutely right. And besides, nobody is going to buy the first generation of autonomous cars if they know it's programmed to kill the driver (even if such a scenario is the most "rational" outcome according to a lot of different ethical standards).

My guess is the first generation car will require a driver and will immediately give control to the driver in that situation so
even though the driver has no time to react the driver is technically in control of the car when it crashes.

Re:Things are a lot more complicated

[vux984]

This is why I am opposed to the law busses must stop to check for a train. Has anyonr bothered to check if a bis stopping, then stalling as it pulled ahead, increased the rate of hits rather than decrease it?

You might be interested to find out that a study WAS done to determine whether bikes stopping for stop signs increased their rate of accident.

The assumption being that the longer a bike is in an intersection the higher the odds of it being involved in an accident, and stopping for a stop sign means that it has to enter the intersection starting from zero, leading to it being in the intersection longer.

The study found that the bikes having to stop for stop signs did in fact increase the accident rate, and recommended that they only stop when they needed to stop. (ie if they could see approaching cars as they approached the intersection)

Not your bus example, but similar in many respects.

Re:Things are a lot more complicated

[mattack2]

Why wouldn't the same be true of cars?

Cool, so I can drive through stop signs and not get a ticket?

I *hate* it that bike riders aren't given tickets for the same things car riders are, or even pedestrians doing dangerous things (like jaywalking across the boulevard in front of my workplace).

Re:Things are a lot more complicated

[fractoid]

That's fine until it compares its passenger report with its firmware which says it's only fitted with five seatbelts, and you get 65,525 traffic violations for "riding without a seatbelt". :P

And you try to contest them but you can't even get a hearing because "the robot said it so it must be true."

Re:Things are a lot more complicated

[vux984]

Why wouldn't the same be true of cars?

couple reasons.
1) Cars are a lot more visible and other drivers are more likely to see / pay attention to / acknowledge other cars. So they are less likely to hit them in the first place.

2) Cars get up to speed much faster, reducing the impact of having to stop?

Or maybe it is true of cars; for what its worth the trend here is away from 2 and 4 way stops towards roundabouts with yield entrances...

Cool, so I can drive through stop signs and not get a ticket?

Course not. Technically its still not legal for bikes either. The study only made the recommendation, as far as i know no laws anywhere have acted on it at this point.

I *hate* it that bike riders aren't given tickets for the same things car riders are, or even pedestrians doing dangerous things (like jaywalking across the boulevard in front of my workplace).

I do agree with this. The rules should be enforced, especially vs dangerous behaviour. Although I also think the rules for bikes -should- be different in some scenarios.

Re:Things are a lot more complicated

[Anonymous Coward]

Oh boy oh boy...

Time to buy a robot kitten!

Re:Things are a lot more complicated

[marekjm]

If that means you have to wait until the belligerent victim goes unconscious (and maybe unsavable) before you approach him, you wait.

I used to work as a lifeguard on the sea coast. The rules are: "Most important is safety of the action, second is safety of the lifeguard, only then - safety of the victim." Even during the training course, they will tell you that if you have to - use brute force to knock the victim out of consciousness for easier handling.

Re:Things are a lot more complicated

[stoatwblr]

"For example, suppose there is a car full of 5 kids stuck on a railroad track."

This simply isn't going to happen. An automated vehicle won't stop on the railroad track in the first place and a non-automated vehicle isn't going to register as a distress call for automated vehicles to rescue.

Virtually every single case I've seen of non-deliberate "car on tracks" has been because the driver proceeded onto the line without a clear exit path (just about every country's highway code says something along the lines of "DO NOT enter an intersection if you cannot exit it" and humans in just aboue every country regularly ignore that rule - which is one of the prime causes of traffic jams.)

The few remaining cases have been where the driver ignored engine trouble warnings signs and pressed on regardless.

A robot isn't going to selectively ignore safety rules or proceed if there is that kind of issue.

Measuring Competence

[ZahrGnosis]

Given this article mere moments ago on /. indicating that Google's autonomous cars have driven 700,000 miles on public roads with no citations, it's difficult to argue that they're not more competent, if not hyper-competent, compared to human drivers (most of whom get traffic tickets, and most of whom don't drive 700,000 miles between doing so).

Article has many good valid points, though, but that point irked me.

Re:Measuring Competence

[Serenissima]

I see what you're saying. My takeaway was that he wasn't saying robots weren't more competent at specific things (in fact, he commented on how they can do very specific things much better than humans) but that they're not competent in replacing all human tasks. In the example he gave, he said a car-welding robot could weld faster and better than a human, but if asked to install upholstery in the car, it'd probably destroy it.

As part of that, cars are looking like they're going to be robots that are significantly more competent at driving than we'll ever be - but they'd make horrible robots to help an old lady go the bathroom in a nursing home, or any number of tasks not related to driving.

They're not competent in their ability to be "Bishop" from Aliens, but they are/will be plenty competent in driving. :)

Re:Measuring Competence

[nine-times]

When he says that robots aren't "competent", I don't think that he's saying that they can't do things. He's just pointing out they they only do certain specific things that they've been told to do, even if they do those things extremely well.

I think the example used points this out: The question is asked, "If the robotic car be put in the position of killing 1 person in order to save 2 people, how should it make the decision?" He's saying that there's a problem with the question, which is the assumption that the robot will be capable of understanding such a scenario.

With our current engineering techniques, we can't expect the robot to understand what it's doing, nor the moral implications. We can't program it to actually understand whether it will kill people. The most we can program it to do is, given a detection of some heuristic value, follow a certain protocol of instructions. So for example, if the robotic car can detect that it's about to hit someone, try to stop. If it calculates that it will be unable to stop, try to swerve. You might program it to detect people specifically and place extra priority on swerving around them, e.g. "if you're about to hit something identified as a person, or hit a road sign, choose to hit the road sign". We might even get it to do something like, "If you're losing control and you can detect several people, and you can't avoid the whole crowd, swerve into the sparsest area of the crowd while slowing as much as possible.

The engineers should try to anticipate these kinds of things. We as citizens should also debate about how we'd want these kinds of instructions should work to avoid legal liability. For example, we might say that in order for the AI to be legal, it must show that it will stop the car when [event x] happens. But to ask, "how will the car make moral decisions?" fundamentally misunderstands its decision-making capabilities. The answer is, "It won't make moral decisions at all."

Re:Measuring Competence

[Anonymous Coward]

In all honesty, driving a car is easy but monotonous. That's what computers excel at, the simple but repetitive tasks. Stay within certain parameters (speed, lane) while heading on a path determined by doing some simple pathfinding algorithm on a big clump of map data from some other source.
There was a viable proof-of-concept being tested 20 years ago, but that was based on a van with all but the two front seats set up as a video processing station (with room for someone to sit there and watch the error messages during the training phases). It worked, but lacked navigation. Now we have a (generally reliable) automatic navigation system, more processing power/cm^3, and the big issues involve integrating the systems and determining how to prioritize inputs.

Idiotic debates about the moral imperative to kill someone so that others might live, that's what philosophy students excel at. It's the sort of apparently deep yet truly meaningless discussion that can often prepare them to endure the realization that they are fools who have squandered their potential to aid humanity beyond standing at a cash register and trying to act polite.

Re:Measuring Competence

[David_Hart]

Given this article mere moments ago on /. indicating that Google's autonomous cars have driven 700,000 miles on public roads with no citations, it's difficult to argue that they're not more competent, if not hyper-competent, compared to human drivers (most of whom get traffic tickets, and most of whom don't drive 700,000 miles between doing so).

Article has many good valid points, though, but that point irked me.

Yet all of it in relatively calm clear conditions with no snow, salt, ice, -20 degree weather, high winds, driving rain, etc. to obscure or break the sensors....

Re:Measuring Competence

[rasmusbr]

Nah, 700k miles is nothing. Human drivers drive >70M miles between fatal accidents, and that's on average. Imagine how far highly trained drivers drive between fatal accidents. Humans are actually pretty good at driving!

Come back when the Google car has driven a few billion miles and we'll have a look at the statistics.

Re:Measuring Competence

[Ironlenny]

Except that just a few vehicles out of the millions that are on the road. That's an insufficiently large sample size to say how automated cars from different manufactures with different levels of maintenance under varying road contritions will interact. You can't assume competency from the limited, though still impressive, testing Google has done.

If anything you are demonstrating the author's point, assuming that what Google has accomplished will be true of all driverless cars. Each of Google's automated cars is effectively a student driver with Google's engineers, technicians, and drivers shepherding the vehicles through the hazards of everyday driving. How will that record hold when one of those cars are twelve years old and hasn't had a tuneup in three?

Re:Measuring Competence

[clovis]

Given this article mere moments ago on /. indicating that Google's autonomous cars have driven 700,000 miles on public roads with no citations, it's difficult to argue that they're not more competent, if not hyper-competent, compared to human drivers (most of whom get traffic tickets, and most of whom don't drive 700,000 miles between doing so).

Article has many good valid points, though, but that point irked me.

You have to keep in mind that to some extent the perfect record may be due to having a human driver that takes control when problematic situations arise. They're not completely autonomous 700,000 miles. We would want to know how many times the human has had to take control and why.

BTW, They have had one wreck, but Google says it happened while the driver had taken control, but did not say why the driver took control.

That topic is covered in this article, and in more detail from the article's link to "That Atlantic" article.
Robot cars, at the moment, have a similarly savant-like range of expertise. As The Atlantic recently covered, Google’s driverless vehicles require detailed LIDAR maps—3D models created from lasers sweeping the contours of a given roadway—to function. Autonomous cars have to do impressive things, like detecting the proximity of surrounding cars, and determining right of way at intersections. But they are algorithmically locked onto their laser roads. They stay the proscribed course, following a trail of sensor-generated breadcrumbs. Compared to what humans have to contend with, these robots are the most sheltered sort of permanent student drivers. No one is quizzing them by sending pedestrians or drunk drivers darting into their path, or diverting them through un-mapped, snow-covered country lanes. Their ability to avoid fatal collisions remains untested.

More detail from this:
http://www.theatlantic.com/tec...

Re:Measuring Competence

[jeffmeden]

Given this article mere moments ago on /. indicating that Google's autonomous cars have driven 700,000 miles on public roads with no citations, it's difficult to argue that they're not more competent, if not hyper-competent, compared to human drivers (most of whom get traffic tickets, and most of whom don't drive 700,000 miles between doing so).

Article has many good valid points, though, but that point irked me.

This. If we mythologize the competence of robots (at least ones well designed and tested to pilot a car) then it's not by nearly as much as we mythologize our own competence. Traffic deaths per person and per mile were at their peak in the 30s and 40s, when cars were poorly designed and tested (given their relative novelty) and today, despite there being so many new distractions for drivers, traffic deaths continue to decline. We suck at driving way more than cars suck at protecting us, and it's only through better designed machines (not anything we are doing to be better drivers, clearly) are we staying safer on the roads.

Re:Measuring Competence

[CanHasDIY]

Come back when the Google car has driven a few billion miles through all manner of hazardous road conditions and we'll have a look at the statistics.

That's better.

Re:Measuring Competence

[Anonymous Coward]

Right, is the robot going to lock your car doors so you cant get out, then drive forward to move (and maybe not move) the other car while you are trapped. This should never be a robots decision. Unless it is tasked as a car-on-train-tracks moving robot. In this case it should not have passengers.

Yes, the question is simple-minded.

Re:Measuring Competence

[CrimsonAvenger]

Traffic deaths per person and per mile were at their peak in the 30s and 40s, when cars were poorly designed and tested (given their relative novelty) and today, despite there being so many new distractions for drivers, traffic deaths continue to decline. We suck at driving way more than cars suck at protecting us, and it's only through better designed machines (not anything we are doing to be better drivers, clearly) are we staying safer on the roads.

It is certainly true that traffic deaths have continued to decline for decades. And that is mostly, if not entirely, due to safer cars.

However, traffic ACCIDENTS (measured both by accidents per passenger-mile and by absolute number of accidents) have also been declining for at least the last couple decades. I can believe safer cars cause fewer deaths, I don't see how safer cars cause fewer accidents....

Re:Measuring Competence

[Anonymous Coward]

I could drive that far without a citation too. It just wouldn't be as much fun. ;-)

Re:Measuring Competence

[jeffmeden]

Traffic deaths per person and per mile were at their peak in the 30s and 40s, when cars were poorly designed and tested (given their relative novelty) and today, despite there being so many new distractions for drivers, traffic deaths continue to decline. We suck at driving way more than cars suck at protecting us, and it's only through better designed machines (not anything we are doing to be better drivers, clearly) are we staying safer on the roads.

It is certainly true that traffic deaths have continued to decline for decades. And that is mostly, if not entirely, due to safer cars.

However, traffic ACCIDENTS (measured both by accidents per passenger-mile and by absolute number of accidents) have also been declining for at least the last couple decades. I can believe safer cars cause fewer deaths, I don't see how safer cars cause fewer accidents....

Cars are easier to control than ever before. They stop faster, turn sharper, and provide the driver more insight through better sight lines (when they are choosing to pay attention) vs cars of the past that were much more poorly designed. Or maybe it's because a "good driving" gene is slowly emerging as a selected for trait? I could see it either way.

Re:Measuring Competence

[lgw]

Don't forget "when it's owner doesn't bother with basic maintenance".

Re: Measuring Competence

[Anonymous Coward]

Lot's of people with combined mileage several orders of magnitude larger than that also have never gotten any tickets or been involved in accidents.

Re: Measuring Competence

[Anonymous Coward]

Easy. ABS and traction control system allow for a wider range of conditions where a modern car will remain controllable rather than going into skid or spin.

Re:Measuring Competence

[lgw]

Anti-lock brakes may not stop quicker, but they do wonders for the majority of drivers, who have never learned how not to lock the brakes. Traction/stability control is becoming mandatory. My car adds lane departure prevention, and a host of "are you sure" -style warning beeps if I'm approaching too quickly, or my signal is on but someone is in my blindspot, or etc. There's lots of driver assistance already.

Re:Measuring Competence

[Anonymous Coward]

However, traffic ACCIDENTS (measured both by accidents per passenger-mile and by absolute number of accidents) have also been declining for at least the last couple decades. I can believe safer cars cause fewer deaths, I don't see how safer cars cause fewer accidents....

You can't see how something like anti-lock brake systems could allow a driver to stop in time to avoid an accident?

Re:Measuring Competence

[Anonymous Coward]

When he says that robots aren't "competent", I don't think that he's saying that they can't do things. He's just pointing out they they only do certain specific things that they've been told to do, even if they do those things extremely well.

I think the example used points this out: The question is asked, "If the robotic car be put in the position of killing 1 person in order to save 2 people, how should it make the decision?" He's saying that there's a problem with the question, which is the assumption that the robot will be capable of understanding such a scenario.

With our current engineering techniques, we can't expect the robot to understand what it's doing, nor the moral implications. We can't program it to actually understand whether it will kill people. The most we can program it to do is, given a detection of some heuristic value, follow a certain protocol of instructions. So for example, if the robotic car can detect that it's about to hit someone, try to stop. If it calculates that it will be unable to stop, try to swerve.

Actually, it's very likely that the best heuristic is, try to stop, period. This is true even without having autonomous cars in the picture. Regardless of situation, the safest thing to do is usually to hit the brakes, and swerving is virtually always a bad idea. Swerving has a chance to avoid a collision altogether, but has a risk of a full-on collision that might actually be worse than going forward at constant speed, and might entangle cars from other lanes (or bystanders). Braking might lead to an unavoidable collision - but at a greatly reduced speed. The only thing the autonomous car adds is sensors that might give you a chance to start braking earlier, making "just stop" even better of an algorithm.

The vast majority of these questions about autonomous cars and "trolley problems" are likely overestimating the value of actually making decisions. Many, many accidents occur or are made worse because someone tried to make a decision instead of just following the simplest rule: stop.

Re:Measuring Competence

[Copid]

Nah, 700k miles is nothing. Human drivers drive >70M miles between fatal accidents, and that's on average.

There's difference between a fatal accident and a moving violation. What's the average distance driven between traffic citations? And the average distance between nonfatal accidents? I don't have the data, but most of the people I know have been involved in nonfatal accidents and been ticketed, and my eyeballing of the stories says that the average distance between those events is less than 700k miles.

Re:Measuring Competence

[malachiorion]

But those Google cars are extremely coddled. 1) They're always relying on pre-loaded LIDAR data, so always on routes that Google has mapped ahead of time. It's going to take a very long time to establish a fully comprehensive, national LIDAR map, and even when it's completed, it'll have to be updated constantly. 2) They aren't driving in extreme weather conditions. 3) They're driving in a very limited radius—no one's taking them on a cross-country trip, dealing with unfamiliar and poorly marked roads. So that's 700,000 miles in a highly controlled environment. 4) And 700,000 miles is pretty skimpy, isn't it, when it comes to gauging the problems that arise with cars, and with drivers (human or otherwise)? It's not as though everyone gets into a potentially fatal accident every 500,000 miles. But when you combine all vehicles and all those drivers, and consider the total number of cars on the road, that's where the number of total collisions start to get scary. I have no doubt that robotic cars will reduce crashes, and, if they really become the rule, not the exception, will save a crazy amount of lives (and money). Still, it's not like these things are going to be creeping along at 20 mph. At 65 mph, physics can kick your ass real quick.

Re:Measuring Competence

[RespekMyAthorati]

Google's autonomous cars have driven 700,000 miles on public roads with no citations

What roads? What conditions? What unexpected challenges did they have to deal with?
When the NTSB (or Consumer Reports) certifies an autonomous car as safe, then I will believe it.

Re:Measuring Competence

[Anonymous Coward]

The Google cars have 700k miles with no tickets or accidents. Your comparison to 70M miles with no fatal accidents isn't really comparing apples to apples. I suspect the typical human does not have as good a record as the Google car.

Also, while the article mentions not handling an erratic pedestrian, I'm sure that the Google cars have handled this case hundreds of times. I think the computer driver is annoyingly cautious, but is significantly safer than a human. They do have some video footage showing an erratic bicyclist that keeps wandering out in front of the car (something that needs to be handled to drive in San Francisco).

Obviously, at some point, even the best cars we can program will cause a fatality. Since humans do this every day, I think the appropriate thing to ask is whether the computer option is sufficiently safer to justify the expense. It currently is not, but probably will be soon.

Re:Measuring Competence

[rasmusbr]

Sure, so an accident is often defined as an incident where a person reported any sort of injury. It's a lot harder to get comparable statistics on that. It's easy to make a list of every person who has died in a car, but it's not easy to list every sprained thumb.

The total accident rate nowadays is roughly on the order of one accident per 1M miles driven (there are on the order of 100 accidents per fatal accident), so we could expect the Google car to have its first minor accident soon if it is exactly on par with the average human driver, assuming they have the car drive an average traffic pattern and don't cancel their tests during difficult weather conditions or difficult traffic conditions. In reality they're not going to do that. It would be grossly irresponsible to do that sort of testing before they feel the car is as good as human drivers in virtually all imaginable traffic situations.

Traffic tickets are even harder to compare since it's a human making a judgement call depending on lots of factors.

Re:Measuring Competence

[Gibgezr]

"Many, many accidents occur or are made worse because someone tried to make a decision instead of just following the simplest rule: stop."
THIS, so much this.

"He's saying that there's a problem with the question, which is the assumption that the robot will be capable of understanding such a scenario"
I would take this even further: there is no NEED for the robot to be capable of understanding the scenario at any level beyond "something I must stop fro has jumped in front of me: STOP" if all we desire is safer roads. Robots that could drive for us and follow this, along with the other general rules of the road, would already result in a net gain of safety over our current humanly-operated cars. There is zero requirement for any moral/ethical choice-making if what we want is better safety than what we have now.

Re:Measuring Competence

[jacksdl]

Agree. The bar a autonomous car should have to meet is to drive at least 1 order of magnitude better than the average human driver (based on metrics like safety and efficiency). Looking at the human drivers currently on the roads, a robot driver doesn't need to approach perfection to achieve this. Simply adding logic that makes a car stop when it doesn't have sufficient information to proceed safely would make it a superior driver to most of us.

Re:Measuring Competence

[Anonymous Coward]

Where this analysis fails is in certain cases where more than one direction is available for swerving, and no projected outcome is optimal. If swerving left sends you off a bridge into a lake, but swerving right sends you into a pedestrian, should the car save your life while hitting the pedestrian, or launch you into the lake? What if the car is actually a bus full of schoolchildren? Does that change the answer?

As much as machines maybe shouldn't be making these decisions, at some point, they're going to. You can bet the first time a bus full of kids goes over a cliff due to the "don't hit pedestrians" heuristic, you're going to see a legal case that makes the OJ trial look like a blipvert.

And what about trolls? You can bet as soon as people find out that the cars will swerve out of the way, you'll get a bunch of jackasses playing in traffic trying to get cars to crash into things in order to keep from hitting them.
Ah, the future...

Robots are a lower life form

[ArcadeMan]

EXTER-MI-NATE!

Re:Robots are a lower life form

[CanHasDIY]

Doctor Fail: Daleks aren't robots.

What you meant to say was,

DELETE! DELETE! DELETE!

Re:Robots are a lower life form

[LoRdTAW]

Negative. K-9 would be a better example.

The Cybermen have living human brains. They are cyborgs, not robots.

Re:Robots are a lower life form

[Akaihiryuu]

More Doctor fail. Cybermen are not robots either. They are cyborgs controlled by a human brain. Daleks are creatures that look like a cross between a squid and a brain in what basically amounts to a futuristic tank.

Re:Robots are a lower life form

[Anonymous Coward]

Reading comprehension fail. The Daleks are the subject, saying "Exterminate".

The Daleks want to exterminate robots because they're a lower life form... like everything else that's non-Dalek.

Re:Robots are a lower life form

[CanHasDIY]

Damn, you're right.

Or as K-9 would say, "AFFIRMATIVE!"

Probably my favorite TV robot ever.

Re:Robots are a lower life form

[fractoid]

Getting off topic, but did it bug anyone else that in the new Doctor Who series the Doctor pronounces it "kay-NINE" in stead of "canine"?

Re:Robots are a lower life form

[KingMotley]

More Doctor fail. The squid brain dalek is only one revision of them.

We're Robots too

[bhagwad]

Or did no one think of that? Reminds me of some other science paper which said that no machine can ever be conscious. As if somehow we are not machines.

So dumb...

Re:We're Robots too

[somepunk]

Reminds me of some other science paper which said that no machine can ever be conscious.

Perhaps they were right. I don't think anyone's ever proved humans are conscious either, except by defining it that way.

Re:We're Robots too

[bhagwad]

Well, I don't know if other people are conscious. I only know that I am. And there's no reason for me to think I''m not a machine. I'm a biological robot after all...

Re:We're Robots too

[MozeeToby]

I know that I'm conscious. I'm self aware. I have a stream of thought that I can analyze (and I can analyze that analysis if I really want to). That's pretty much the definition of being conscious. After that I'm left with only a few options.

I can believe that I am a unique snowflake, the only conscious human being in the world. But that doesn't make any sense. For one thing there's nothing about me that should make me unique in that regard. For another, most humans behave in ways that are basically consistent with the way I behave and much of my behavior is driven by my consciousness. It'd be difficult or impossible to account for the actions of others if I chose to view them as mere automatons.

Or I could believe that my consciousness is an illusion. Something my brain conjures up to make me think that I'm directing myself through my day when in reality I'm just another robot puttering through the day. First and foremost, why would such a thing evolve? If consciousness doesn't drive human behavior why do I perceive myself to be conscious?

Or I could believe the other human's are conscious as well. Given the alternatives, this seems like the most reasonable, logically choice.

Re:We're Robots too

[Anonymous Coward]

Reminds me of some other science paper which said that no machine can ever be conscious.

Perhaps they were right. I don't think anyone's ever proved humans are conscious either, except by defining it that way.

And nobody has ever done a study to prove that pinching your right hand pinky with a pliers causes real pain. Maybe thats because some things are so blatantly obvious that it would be a waste of time to do a study on it. If you think you are only a machine and that you are not conscious then there isnt a study that could be done to change your mind. Once you choose to ignore the obvious fundamentals, then you hold yourself outside the reach of reason and rational thought.

Re:We're Robots too

[CanHasDIY]

I'm a biological robot after all...

I prefer "walking chemical processing plant" myself.

Re:We're Robots too

[ewibble]

Or I could believe that my consciousness is an illusion. Something my brain conjures up to make me think that I'm directing myself through my day when in reality I'm just another robot puttering through the day. First and foremost, why would such a thing evolve? If consciousness doesn't drive human behavior why do I perceive myself to be conscious?

The reason to believe yourself conscious, is that you may need that illusion in order to survive, If a being, capable of reasoning, did not believe it was somehow special, worthy of survival then it would not be likely to survive.

Re:We're Robots too

[MozeeToby]

Yes, but if my consciousness is an illusion then why how is it driving my behavior? If I'm making decisions about my survival based on how unique and special I think I am, I am conscious.

Re:We're Robots too

[Triklyn]

self-replicating, self-repairing, autonomous entropic facilitator

Re:We're Robots too

[Triklyn]

i am the feverdream of a robotic intelligence on the verge of collapse in a alien universe. The dying thought-quakes of a supermassive superintelligent world-organism as it watches it's last source of energy wink out as it's universe dies. Soon I will be just a ghostly pattern in this derelict husk, floating in the darkness until time has lost all meaning. But... at least this dream is about me.

Re:We're Robots too

[Quirkz]

An inefficient methane factory with a few side effects, eh?

Re:We're Robots too

[ewibble]

really depends on what you define as conscious, if it is merely self awareness the yes we are aware of ourselves. I can see how you could program a robot to be aware of itself. But another part definition from wikipedia "the executive control system of the mind" that is harder to judge? Are we merely a function of inputs to outputs or something more? Or is it we are just that we think we have the illusion of control?

If we are just an function, the ability to think I am conscious is just part of the equation and not something in your control at all.

Put another way, you could program a robot to think it was conscious, argue to the bitter end that it was, if it self examined itself come to the conclusion it was, but in the end it actually had no free will at all. For us it is of no use (expect as an experiment) to build robots with a desire for life, or self awareness, I want to use them without considering their feelings, destroy them without thinking I am committing murder. They do not need a survival instinct.

Personally I think that we are more than just machines, there is something more to life than what we already know, but I seen no way of actually showing that we are anything more than just a sophisticated machine.

Re:We're Robots too

[Wycliffe]

self-replicating, self-repairing, autonomous entropic facilitator

Unfortunately, the self-repairing is limited to minor repairs only.

Re:We're Robots too

[Triklyn]

i don't know, we wear down pretty fast, considering i'm a new me every 7-10 years or so, i'm going to go with, holy hell that's a impressive amount of repair. if we didn't constantly replace old cells, don't know how long we'd last. might shave something like 80% off the expected lifetime of the machine :)

Re:We're Robots too

[bingoUV]

First and foremost, why would such a thing evolve?

This is a wrong question. There are numerous aspects of living beings that could have been different without necessarily having an effect on survivability. Many different proteins could have carried oxygen, some probably better than haemoglobin, still specifically haemoglobin carries oxygen in many animals.

Similarly, "such a thing" is just one of the cases which helped animals survive. Something else could also have helped us survive and thrive, but the "such a thing" happened, and there need not be a reason why among all these things, just the "such a thing" evolved and not others.

If consciousness doesn't drive human behavior why do I perceive myself to be conscious?

Wrong question again. Having calcium phosphate in bones doesn't drive human behaviour either, still people have have calcium phosphate in bones. Having 2 (and not 3, not 4) hands doesn't drive human behaviour, still most humans perceive themselves to have 2 hands.

rule of third thumb

[gl4ss]

anyone calling themselves roboticists are cysts and don't really understand that stories and are severely lacking in understanding how a story gets created(similarly lots of new age hippie zombielovers seem to be unable to understand that yes you can make shit up and if you put some rules on how you make shit up it's a lot easier to make shit up, hence asimov first making up the rules and then making up the stories).

anyhow, we'll cross that bridge when we get there. I predict the robo car will try a controlled stop and failing to do so tries to determine a safe evasion and if it fails at that it will crash at the two obstacles which were dropped in front of it to see how it would react - and we can start worrying about how the car would tell the difference between a robot mannequin and an actual person later. just like it would hit a deer rather than drive 60mph off the road to avoid the animal(if it's a deer, just drive into it. if it's a moose, do a panic evasion and try your chances with the trees).

like, come on, should the car crash on the sidewalk just because someone jaywalked to be in front of it? certainly not. crashing deliberately at whatever else is also out of question, a school bus full of kids for example. jaywalking not being a good example because if there's traffic lights the speeds should be rather low.

I guess the thing to take home is that it might not be a good idea to jump in front of a robot car just because "it can't hurt me because it's just a slave robot!". it's just a machine.

(and robot cars will not be driving on the roads in asia in 40 years... maybe japan. but not any other country)

Re:rule of third thumb

[citizenr]

and we can start worrying about how the car would tell the difference between a robot mannequin and an actual person later. just like it would hit a deer rather than drive 60mph off the road to avoid the animal(if it's a deer, just drive into it. if it's a moose, do a panic evasion and try your chances with the trees).

like, come on, should the car crash on the sidewalk just because someone jaywalked to be in front of it? certainly not.

One thing hours of watching Russian dash cams taught me is you see someone/something smaller than you jump in front of your car you PLOW THRU THAT FUCKER no matter what. Cars are made to take frontal crash, but the second you start avoiding it you end up hitting a tree sideways / flipping your car and/or on the opposite lane right in front of a lorry doing speed limit.

What have you got against Jon Snow?

[wonkey_monkey]

When it comes to robots, most of us are a bunch of Jon Snow know-nothings

https://en.wikipedia.org/wiki/...

?

Re:What have you got against Jon Snow?

[Anonymous Coward]

It's a reference to something Ygrete said to him right before he showed her he _DID_ know something.... :-)

Re:What have you got against Jon Snow?

[barlevg]

...and serves as a completely gratuitous allusion, possibly to screw with SEOs? The article has absolutely zero to do with Game of Thrones.

Re:What have you got against Jon Snow?

[mjwx]

When it comes to robots, most of us are a bunch of Jon Snow know-nothings

https://en.wikipedia.org/wiki/...

?

Whilst his journalistic achievements are impressive, you can still assert he knows nothing about robotics.

Driverless Cars Are Boring

[American AC in Paris]

There was an article a short while ago written by a journalist who rode in a driverless car for a stretch. There was one adjective that really stood out, an adjective that most people don't take into consideration when talking about driverless cars.

That one word: boring.

Driverless cars drive in the most boring, conservative, milquetoast fashion imaginable. They're going to be far less prone to accidents from the outset simply because they don't take the kind of chances that many of us wouldn't even begin call "risky". They drive the speed limit. They follow at an appropriate distance. They don't pull quick lane changes to get ahead of slowpokes. They don't swing around blind corners faster than they can stop upon detecting an unexpected hazard. They don't nudge through crosswalks. They don't cut off cyclists in the bike lane. They don't get impatient. They don't get frustrated. They don't get angry. They don't get sleepy. They don't get distracted. They just drive, in a deliberate, controlled, and entirely boring fashion.

The problem with so, so many of the "what if?" accident scenarios is that the people posing said scenarios presume that the car would be putting itself in the same kinds of unnecessarily hazardous driving positions that human drivers put themselves in every single day, as a matter of routine, and without a moment's hesitation.

Very, very few people drive "boring" safe. Every driverless car will. Every trip. All the time.

Re:Driverless Cars Are Boring

[PaddyM]

...They don't cut off cyclists in the bike lane. They don't get impatient. They don't get frustrated. They don't get angry. They don't get sleepy. They don't get distracted.
"[they] can't be reasoned with, [they] can't be bargained with [they don't] feel pity or remorse or fear and they absolutely will not stop. Ever. [They just drive, in a deliberate, controlled, and entirely boring fashion.] Until you are dead."

FTFY

Re:Driverless Cars Are Boring

[Anonymous Coward]

Very, very few people drive "boring" safe.

As someone who does drive what you call "boring safe", maybe you should learn how to enjoy driving that way and save your need for horizontal acceleration for the theme parks where there are dozens of rides made just to test your tolerances of unusual acceleration profiles. It would help make my drive much more relaxing and comfortable if you suicidal nutjobs would stop trying to kill yourselves in my vicinity.

Re:Driverless Cars Are Boring

[medv4380]

It doesn't get happy, it doesn't get sad, it doesn't laugh at your jokes. It just runs programs. - Short Circuit

Re:Driverless Cars Are Boring

[American AC in Paris]

You're, uh, kinda preaching to the choir. I'm a speed-limit, right-lane, two-second-rule kinda guy.

Re:Driverless Cars Are Boring

[Animats]

That one word: boring.

Right. Just like commercial air travel, elevators, and escalators. Which is the whole point.

This will be just fine with the trucking industry. The auto industry can deal with "boring" by putting in more cupholders, faux-leather upholstery, and infotainment systems.

Re:Driverless Cars Are Boring

[Anonymous Coward]

Oh, carry on then, and assume I was yelling at the guy who was tailgating me yesterday morning.

Re:Driverless Cars Are Boring

[Anonymous Coward]

The problem with so, so many of the "what if?" accident scenarios is that the people posing said scenarios presume that the car would be putting itself in the same kinds of unnecessarily hazardous driving positions that human drivers put themselves in

It is a good thing you got modded +5 insightful for that, because what you've done perfectly demonstrate the author's point. It is absolutely not about how frequently something bad happens, it is all about what to do when something bad happens. A robot car does not have any control over other drivers and even if the robot is a conservative driver, that does not mean the other drivers are, nor does it mean it is immune to other external events like a toddler chasing her puppy into the road.

Boring is beside the point here.

Re:Driverless Cars Are Boring

[Akaihiryuu]

I'd love to have a "boring" car like. I detest long drives. I could never handle a 20-hour drive in a normal car, without splitting it up among several days. If I could just sit back and watch movies, or play video games, or sleep, or whatever while the car did the driving for me, that would be the most amazing thing ever.

Re:Driverless Cars Are Boring

[Anonymous Coward]

The toddler example - nice touch. If the toddler is running out in the freeway, it is probably not be chasing anything again. If it is in a residential area, on a road with a 25 mph limit - the car goes even slower depending upon its visibility (like the fucking retards that speed down my street should do). It is highly likely it would stop with more than enough space.

The car can track objects moving toward it and in its path and at what speed and where the interception can occur. It will slow down until there is no possible interception (object stopped to let car pass by) or it will stop.

For suicidal people, they can trick the cars. But the car shouldn't be responsible for that just like a person.

Re:Driverless Cars Are Boring

[Anonymous Coward]

That's called a plane or a train ;) And yes it's amazing !

(but yeah I see what you mean)

Re:Driverless Cars Are Boring

[chuckugly]

Wish I had mod points.

Re:Driverless Cars Are Boring

[Anonymous Coward]

So they drive like I do. Safely. I have zero tickets. I've never even been pulled over.

Those "laws" and "signs" aren't arbitrary guidelines out to ruin your day. If everybody would actually follow them then accidents -

Hold on, another point here. The word accident is bullshit. Accidents imply that the situation was unavoidable. 99.999% of vehicle collisions are entirely preventable by simply following the rules. (Properly maintaining your vehicle is part of the law too)

Re:Driverless Cars Are Boring

[Quirkz]

Two-second rule? The Smith driving system calls for a four-second gap, and I feel pretty guilty when I fudge and cut it down to three.

I do still have to pass people on the country highway commute to work, where there are a lot of tractors half in the lane, and a lot of cars inexplicably going 50 in a 65. There are also enough bikes around here I have to edge across the median to pass (acceptable by Colorado state law). So not entirely mindlessly boring.

Re:Driverless Cars Are Boring

[American AC in Paris]

So they drive like I do. Safely. I have zero tickets. I've never even been pulled over.

Those "laws" and "signs" aren't arbitrary guidelines out to ruin your day. If everybody would actually follow them then accidents -

Hold on, another point here. The word accident is bullshit. Accidents imply that the situation was unavoidable. 99.999% of vehicle collisions are entirely preventable by simply following the rules. (Properly maintaining your vehicle is part of the law too)

Oh, I do--haven't had a moving violation in 7+ years, back when I was younger and stupider.

That said, I got nailed by a car that decided to try to make a right turn through my car last November. I was in the right lane, going the speed limit, didn't have anyone in front of me, and even saw the other driver overtaking on my left--but there was no way on this green-and-blue earth I could have reacted any faster than I did. A robot -probably- could have, and may well have saved the annoyance of having to go to a body shop to have the other guy's insurance fix it.

From my own perspective, I'm hard-pressed to see how I could have avoided this collision. And frankly, it doesn't really matter that the other driver could have--that doesn't do me a whole lot of good. I don't get to pick and choose who drives next to me.

Re:Driverless Cars Are Boring

[flappinbooger]

You're absolutely right - because inherently, ROBOTS are boring, because they're engineered.

Engineers design things to be BORING, because in the Engineer's world, generally, when things get EXCITING that's BAD.

Exciting is dangerous, and Engineers don't like dangerous because then people die, questions are asked, people get sued, people get fired, and that looks bad on the old resume, and that makes it hard to get another job.

So, boring robot cars are the fault of HR.

Re:Driverless Cars Are Boring

[Anonymous Coward]

Kyle Reese? is that you?

does this mean, Google is Skynet?

I wondered why the read aloud voice for Chrome had a Austrian Accent!

Re:Driverless Cars Are Boring

[Anonymous Coward]

> The toddler example - nice touch.

Dude, forest and trees.
Cars can track the speed of objects but when those objects are on a trajectory that simple physics prevents them from avoiding they still have to deal with it.

> For suicidal people, they can trick the cars. But the car shouldn't be responsible for that just like a person.

You completely miss the point. Nobody is talking about holding the car responsible. What they are talking about is deciding what the car should do. Just because a car can't avoid a problem doesn't mean it won't have choices in how it reacts to it.

Re:Driverless Cars Are Boring

[Anonymous Coward]

...They don't cut off cyclists in the bike lane. They don't get impatient. They don't get frustrated. They don't get angry. They don't get sleepy. They don't get distracted.
"[they] can't be reasoned with, [they] can't be bargained with [they don't] feel pity or remorse or fear and they absolutely will not stop. Ever. [They just drive, in a deliberate, controlled, and entirely boring fashion.] Until you are dead."

FTFY

This just made my night

Re:Driverless Cars Are Boring

[Gibgezr]

Driving is fun. I like driving. I will miss driving, if autonomous cars become common in my lifetime. I dearly hope that happens, though, because I really won't miss the way many people drive, and I believe that even as safe and careful a driver as I am, a good robot will be safer and better than that, and if I want thrills I can go off-roading and pay to drive a car on a separate reserve for thrill-seekers. I will be ok with that.

Re:Driverless Cars Are Boring

[david_thornley]

Thing is, humans don't do "boring" well. People lose attention and will not react well to sudden changes. We do much better if machines can do the boring stuff.

Re:Driverless Cars Are Boring

[phorm]

presume that the car would be putting itself in the same kinds of unnecessarily hazardous driving positions that human drivers put themselves in every single day

No, but they'll still occasionally blow tires, hit an oil slick, loose gravel, unexpected pothole, black ice, etc. Bad things do happen even in ways a machine cannot anticipate, so even if you eliminate the mistakes caused by [stupid] you still have a fair margin for unexpected circumstances.

Huh?

[Anonymous Coward]

What's with the gratuitous Game of Thrones reference?

Re:Huh?

[malachiorion]

Yeah, pretty hackish of me. Sorry about that.

Can't blame the robots

[Anonymous Coward]

In the end the error occurs because of a human mistake in programming it or missing a possible condition.

Re:Can't blame the robots

[BaronM]

In the end the error occurs because of a human mistake in programming it or missing a possible condition.

Or a failed mechanical system.

Even if the sensors and software are perfect, a mechanical failure could still result in a crash. When that happens, who is liable?

I would imagine the owner, just the same as if the steps collapse in your house and injure someone, you are liable even if you can't be said to be responsible in any proximate sense.

Now, your auto insurance rate will depend on your age, sex, location, type of car, sensor suite, software version, and whether or not you've rooted it. I'm so looking forward to that.

On the other hand, if Google really wants to assume all liability for anyone using their driverless cars, sign me up!

Re:Can't blame the robots

[tsqr]

Even if the sensors and software are perfect, a mechanical failure could still result in a crash. When that happens, who is liable?

Probably the same entity that is liable when a human-driven car is involved in an accident as a result of mechanical failure.

Yet another slashdot advertisement...

[QuietLagoon]

.... disguised as a posting.

Maybe the problem is the word "robot"

[erice]

Robots stores in Science Fiction are about powerful artificial sentient minds wrapped in an mobile and often human like container.

Robots in real life have been defined as machines with mechanical appendages that can programmed and reprogrammed for a variety of tasks. Their computational capabilities are seldom extraordinary and they usually don't even employ AI.

More recently, "robot" has also been used to describe machines with ai-like programming even if they are single function (like a robotic car).

When a word is used in three greatly different ways, should we be surprised that there is is confusion about that a "robot" can do?

Re:Maybe the problem is the word "robot"

[Michael Woodhams]

If you're in the right part of the world, the word also means "traffic light".

your premise is wrong

[Charliemopps]

Your entire premise is wrong. And now you're posting it again.

This will be a legal issue, not an issue solved by the "roboticists" whatever that is...

In a legal sense, taking an action that kills 1 person to save another puts you in jeopardy of being liable. Swerving or taking other actions that lead to someones death makes YOU responsible. If someone runs out in the road, you apply the breaks firmly and appropriately, then that is not your fault. It's the person who ran out into the road. So in cases where the computers unsure what to do, it will follow the first commandment "STOP THE CAR" and it will let things play out as they will. Any other choice opens up a can of worms... how old are the other occupants? If 1 car has a 90yr old in it and the other has a baby, which do you hit? What if ones the mayor? The problems increase exponentially as soon as you get away from "STOP THE CAR" so just stop the dang car and be done with it.

With regards to your comment about Scifi... you're reading pretty terrible SciFi. Most of the stuff I read is written by actual scientists so... yea...

Re:your premise is wrong

[tiberus]

The legal issue comes after the moral one. The questions posed is a classic choice of "sacrificing one to save more than one" an ethical dilemma, not a legal one. Laws may be written to codify the ethical decision that has been made but, the ethics must be resolved before laws can be created to uphold those morals.

Thanks for giving me the opening for this thought (Charliemopps)... While reading the article, I was again amazed by the lengths that some individuals will go to in order to avoid answering the moral question before them. In the case presented someone is going to die; either the driver or the two bystanders. It doesn't matter why the situation exists, it exists, we've gone past why. You must choose who will die as the dilemma dictates someone must die.

Stopping the car isn't an option that is available or putting it another way, stopping the car results in either the death of the driver or the two bystanders.

Re:your premise is wrong

[Gibgezr]

But stopping the car is generally the correct answer, even for humans. More lives would be saved if people simply drove safely and, when something bad happened, stopped the car whilst staying on the road. Robots don't need moral calculus to arrive at the correct answer. If a person/deer runs out onto the highway in front of me, the safest thing to do is BRAKE; some gentle swerving *might* help, but in most cases you are better off not; it is usually impossible to predict WHICH WAY the person/animal will move to try and avoid your oncoming car, so you have a 50% chance of making things worse by steering into their escape route. The best answer is almost always to brake and stay on the road. Losing control of the vehicle in order to avoid hitting something, even a human, is not something I would ever want to program a car to do.

Re:your premise is wrong

[bingoUV]

You are ignoring one of the most important players in the moral dilemma - the protagonist. The driver in a human driven car, and the programmer / roboticist / manufacturer in an automated car.

The protagonist should do what is best for himself. Typically this means least legal troubles for himself. Swerving in any direction means intentionally hitting someone in that direction - probably what someone is not present at the time you started swerving. Or you didn't expect that someone to be present there. The legal defence for inaction is much stronger than for any action.

Author is missing the point entirely

[BaronM]

...or being willfully ignorant.

Of course current and contemplated robots can't make decisions about whether or not to sacrifice their owner to save two strangers. That sort of decision making depends on an independent ability to think and weigh alternatives morally.

Asimov's laws were written for robots that were also artificial intelligences. Kind of a big point to leave out of this article, since it changes the nature of the question entirely.

I do not believe that anyone seriously believes that driverless cars, industrial robots, or roombas work that way.

The programmers writing the code for those systems will program them to perform the specified tasks as well as possible taking in to account all relevant rules and regulations as well as the nature of the task and the abilities of the robotic system. Anything unanticipated will result in undefined behavior, perhaps guided by some very high-level heuristics (ie., if you don't know what to do, stop, put on the emergency flashers, and call for human assistance).

Short version: in the absence of artificial intelligence, talking about what a robot should do in a moral context is silly, not profound.

Re:Author is missing the point entirely

[Nethead]

Exactly. The car doesn't even know what a person is other than maybe "that other system that sometimes moves the car."

Re:Author is missing the point entirely

[gnasher719]

Of course current and contemplated robots can't make decisions about whether or not to sacrifice their owner to save two strangers. That sort of decision making depends on an independent ability to think and weigh alternatives morally.

You don't really need ability to think and to weight anything in any moral way. I suppose that car manufacturers would be required by law which preferences to follow. I _think_ the rules will be to give precedence to everyone on the street who followed the rules to avoid _innocent_ victims.

But then, the example of a car with five passengers stuck on a railway track and another car with two passengers behind it - how often does that happen? And the doors on the first car don't unlock, right, because otherwise the five passengers would just get out and run?

Re:Author is missing the point entirely

[steveha]

Sorry to say it, but I think it is you who has missed the author's point entirely.

The author asked the question: if a car can save two lives by crashing in a way that kills one life, should it do so? And many people rejected the question out of hand.

The author listed three major ways people rejected the question:

"Robots should never make moral decisions. Any activity that would require a moral decision must remain a human activity."

"Just make robots obey the classic Three Laws!"

"Robots will be such skillful drivers that accidents will never happen, so we don't need to answer this question!"

All of those responses are not well-reasoned and that is the whole point of TFA.

The author went on to point out that the Three Laws are fictional laws that were applied to fictional full AIs that we don't have in the real world.

P.S. I do think that robot car drivers will rarely have crashes. As others have pointed out, the AI never gets sleepy or bored, and never takes stupid chances due to impatience. AI cars drive in a boring way, and if the majority of all cars were doing that, there would be a great reduction in crashes.

That said, of course the AI must be programmed with some strategy to cope with a crash. I'll bet that in the current generation it's mostly "swerve in a direction that doesn't appear to have any obstacles" and "stomp on the brakes" but there has to be something.

This is a specific case of a general problem: navigating cost/benefit tradeoffs. Suppose I have a new car design, and it is safer than old car designs. Then the more people switch to the new car, the more lives are saved. But the more expensive the car is, the fewer people buy the car. Now, I could add one more feature, and it makes the car even safer but it also makes the car even more expensive. Do I add the feature? Then fewer people get the safe car, but those people are extra safe. Do I omit the feature? More people get the safe car but it isn't as safe as it could be. How do you decide?

You use math, and do your best. But some people will reject the question. "It's immoral and shocking to reduce human lives to numbers in an equation..." Oh yeah, it's so much more moral to just guess at what to do, rather than try to apply math to the problem.

Re:Author is missing the point entirely

[BaronM]

>

The author went on to point out that the Three Laws are fictional laws that were applied to fictional full AIs that we don't have in the real world.

It's possible I'm wrong, but having read the article twice now, I don't see where the author made or addressed that point at all. That omission is what my initial comment turns on -- discussing what a robot should do in the absence of true AI is meaningless.

Re:Author is missing the point entirely

[CrimsonAvenger]

But then, the example of a car with five passengers stuck on a railway track and another car with two passengers behind it - how often does that happen?

If the five people are in an autonomous vehicle, they won't be on the railroad track. They'll have been driving as safely as possible and not gotten on the tracks till there was room to get off. Just like any sane driver does....

Re:Author is missing the point entirely

[steveha]

Stuff like this:

SF writers invented the robot long before it was possible to build one. Even as automated machines have become integral to modern existence, the robot SF keeps coming. And, by and large, it keeps lying. We think we know how robots work, because weâ(TM)ve heard campfire tales about ones that donâ(TM)t exist.

And this:

The myth of robotic competence is based on a hunch. And it's a hunch that, for the most part, has been proven dead wrong by real-life robots.

Actual robots are devices of extremely narrow value and capability. They do one or two things with competence, and everything else terribly, or not at all.

The article doesn't contain the phrase "the Three Laws would only work on a true AI" but it really does discuss the fact that fiction shows AI and we don't have it.

Another case

[gurps_npc]

This area is very complicated.

There are classic stories about things like - should a doctor kill one healthy triplet to use the organs to save two other unhealthy ones is a classic example. But it ignores other options such as instead kill one unhealthy one to save the other unhealthy one?

Human lives are not simple equations, but far more complicated ones. Age, health, ownership, responsibility are all part of it.

Cops, firemen, EMT's all have greater responsibility. Similarly, there is a big difference between you risking your own life and you risking your kids life - or worse your neighbor's kid's life.

The idea that the programmer will decide all of this thing with no input from the owner is ridiculous. The programmers need to offer multiple.

Worse, we can't have too many options because it makes it harder for the computer to figure out what other cars will do.

So I think we need at heart three or maybe four basic options, that are broadcast to the other vehicles. One should certainly be standard for 911 vehicles (maximize save others). Another should be standard for school buses (maximize save occupants). And third option that lies somewhere in between that lets the car take some risk to save others, but not too much.

Asimov himself described a big flaw in his 3 laws

[nani popoki]

He wrote an essay pointing out that the biggest problem with his three laws of robotics was that a robot might well have trouble defining "human". His test cases -- if I remember right; it was 40 years ago that I read the essay -- were (1) a baby [human but not competent to give a robot an order], (2) an adult with mechanical prosthetics [human only if you examine the right parts], (3) another robot and (4) a chimpanzee. The problem is a lot more complicated than the Three Laws makes it sound!

Sci-fi Naive?

[Anonymous Coward]

First Tom Murphy of 'do-the-math' fame points out that humanity's future is most certainly not in space, and we really should look into these earth-bound problems we are facing.

Now some jerk is saying that robots won't save us and our earth-bound problems?

What the hell are we supposed to do then, given that hard work and/or sacrificing minor conveniences are not an option?

Quick reaction times brings up another option

[Krishnoid]

Whether in make-believe settings, or the distorted scene-setting of media coverage, robots are strong, because anything less would be a buzzkill.

Speaking of buzzkills, could a robot driver deploy a sawstop-style mechanism, possibly dropping an anchor of sorts into the road surface, when presented with an imminent otherwise-unpreventable collision?

This assumes airbags can be designed to sufficiently mitigate the g-forces on the occupants to prevent internal 'shaken-baby-syndrome'-style brain injuries.

Re:Quick reaction times brings up another option

[Anonymous Coward]

Years and years ago I saw a video showing a heavily loaded 18-wheeler with a mechanism that would stop the truck fairly quickly in an emergency (think runaway truck going down a steep grade). Driver presses a button and a hefty mat deploys from in front of the rear tires, the rear tires roll onto the mat and the entire trucks skids to a halt (the mat stays attached to the frame in front of the tires).

Your "dropping an anchor" comment brought back that memory...

Insects

[EmperorOfCanada]

I have always thought that robots will be like insects. You give them a logical set of rules to follow based upon a fallible set of inputs. Then you set them lose.

So I fully expect to see generation after generation of programming where slowly most of the edge cases are dealt with. So floor mopping robots will make mistakes like mopping the carpet, wandering out of the building and mopping the parking lot, mopping the lawn, etc. Then you will get things like the mopping robot that encounters a 5 gallon paint spill which will overwhelm its capacity so instead of cleaning it will basically paint the floor.

But the reality is that if it is mopping really well 99.999% of the time then the occasional mistake will still end up costing less and my guess is that robots will tend to be fairly OCD about their tasks so it will end up being as clean as if someone was on their hands and knees with a toothbrush.

Also people will learn to alter their environments to make them more robot friendly. If it won't stop mopping the carpet, them maybe get rid of the carpet.

And the answer is....

[Anonymous Coward]

The answer to your original question: "Should a robot car kill its owner if it means saving two strangers?" is actually pretty simple.

Robots, if they are to be assimilated into society, should behave like the best-case-scenario of human behavior. The best-case-scenario for a human in such a situation would be ???????? In other words, if I'm driving my car, and somehow I'm able to figure out that I must either kill one of my passengers, or two strangers, what should I do? The answer is: it depends, and neither choice is really wrong. If my passenger were my child, then I would most likely choose to save my child. If my passenger was a stranger, then I would probably try to avoid the outside strangers (since there are two of them). Robots are going to start forming relationships with us, and it would be spooky and weird if they didn't obey the normal social conventions we expect out of other humans (that some humans are closer to us than others).

Terrible, terrible article

[harvestsun]

should a robotic car sacrifice its owner’s life, in order to spare two strangers?

If such a car exists, I won't buy it, that's for sure! I'll buy from another car manufacturer. I imagine most people would feel similarly. Are you suggesting that there should be a law that all automated vehicles have this behavior? Ha! Good luck finding a politician who's willing to take that up.

all other options point to a chaos of litigation, or a monstrous, machine-assisted Battle Royale, as everyone’s robots—automotive or otherwise—prioritize their owners’ safety above all else, and take natural selection to the open road

We already have human drivers that prioritize their own safety above all else (I know I do!). Replacing these with superior robot drivers could only make things better, no?

the leap from a crash-reduced world to a completely crash-free one is an assumption

Only an idiot would make that assumption. Stop treating your readers like idiots. Oh wait, it's Popular Science. Never mind.

Even if it were possible to simply order all robots to never hurt a person, unless they suddenly able to conquer the laws of physics, or banish the Blue Screen of Death in all its vicissitudes, large automated machines are going to roll or stumble or topple into people.

More often than human drivers already do?

Who believes in robotic competence?

[fuzzyfuzzyfungus]

Does anyone who has to deal with software(even as a user, not even as some hardcore code guru) believe in robotic competence?

A robot is nothing more than a (probably commodity) computer, which we know are unreliable junk, running a whole heap of software(which we know is terrifyingly bad in all but the most carefully controlled and rigorously validated situations), with a bunch of moving parts grafted on that probably haven't seen maintenance within the vendor's recommended window.

That is...not...the stuff of which 'hyper-competence' (much less infallibility) is made.

Re:Who believes in robotic competence?

[bill_mcgonigle]

No, the whole premise is a strawman argument. I mean, yeah, there are idiots and idiots who think they are experts, but nobody with any knowledge and wisdom believes that strong AI exists today.

Re:Who believes in robotic competence?

[fuzzyfuzzyfungus]

Even if you did believe in strong AI, half the AIs in science fiction are either psychotic, murderous, or going off the rails for some reason, and we all know (and some of us are) natural intelligences that don't exactly inspire confidence in the competence of intelligences in general.

And in practice, laws 2 and 3 are swapped

[Dr. Manhattan]

I used to do software for industrial robots. Safety for the people around the robot was the number one concern, but it is amazing how easy it is for humans to give orders to a robot that will lead to it being damaged or destroyed. In practice, the robots would 'prioritize' protecting themselves rather than obeying suicidal orders.

Re:And in practice, laws 2 and 3 are swapped

[EmperorArthur]

I used to do software for industrial robots. Safety for the people around the robot was the number one concern, but it is amazing how easy it is for humans to give orders to a robot that will lead to it being damaged or destroyed. In practice, the robots would 'prioritize' protecting themselves rather than obeying suicidal orders.

It all comes down to how much you trust the user. Asimov thought most users were smart enough to not give orders that were that bad. Plus I think Asimov's robots were smart enough that they would try to carry out general orders without damaging themselves. For instance, "Come here as fast as possible," does not mean that the robot would burn out its motors while traveling to the person. So I guess even in universe you had to explicitly tell the robots to do something suicidal.

Side note, with the minor things like the Emory University incident (http://it.slashdot.org/story/14/05/17/051214/emory-university-sccm-server-accidentally-reformats-all-computers-campus-wide) it certainly seems that Windows tends to stop users from doing dumb things like formatting their system partition while running on it. Meanwhile, Linux and Unix have no such protections. "Unix was not designed to stop its users from doing stupid things, as that would also stop them from doing clever things." – Doug Gwyn
Of course, no modern OS will let you do anything truly damaging to it without admin rights.

Re:And in practice, laws 2 and 3 are swapped

[Anonymous Coward]

Asimov thought most users were smart enough to not give orders that were that bad.

Wat?

The reoccurring theme is that people give robots orders that they don't understand the consequences of. Then shit happens that appears to be bad but actually turned out to be a good way to uphold the robotic laws under the circumstances that the user wasn't aware of.

The robot detects that the user shows signs of cancer. It then proceeds to stab the user with a knife to make sure that the user has to go to the hospital and get checked.

Re:And in practice, laws 2 and 3 are swapped

[cellocgw]

Emory University incident (http://it.slashdot.org/story/14/05/17/051214/emory-university-sccm-server-accidentally-reformats-all-computers-campus-wide) it certainly seems that Windows tends to stop users from doing dumb things

Oh, c'mon, really? You know dang well that Windows tends to stop users from doing anything .

Re:And in practice, laws 2 and 3 are swapped

[humanoid_me1349]

Thank you for your insight. When I take on self suicidal orders, I simply reboot and restart. Maybe that's why I am not able to understand the meaning of life. - HumanoidMe

Morals, ethics, logic, philosophy

[Anonymous Coward]

You can barely "teach" a human morals, ethics, logic or philosophy. Fat chance inculcating those into an artificial intelligence - whatever that is. In the split second a traffic incident occurs, all the philosophical high-ground is out the window. No one wants to die in a fatal accident - but by definition someone will. Given the IA's first law, the robot will simply fail as in the stories. Throw out the First Law as accomplishing anything useful. A modern car already contains dozens if not hundreds of processors, controllers and millions of lines software, and they still fail with spectacular regularity - zero robots. Anything designed by humans will fail. You will die. A robot will make no difference. Try to walk through an automated factory or pipe yard - none of those robots knows anything about humans, which is why people die in those locales. Save the human while moving carrying 30 tons of drill pipe at 5-6mph (2m/s)? - not likely. Save the human(s) in a 3-ton SUV careening along at 60mph (26m/s), impossible.

Re:Morals, ethics, logic, philosophy

[JDG1980]

Self-driving cars don't and won't have morals, ethics, logic, or philosophy. They don't need any of that. They simply have a wide array of input sensors connected to a set of complex algorithms that provides the necessary vehicle inputs to drive from point A to point B while avoiding crashes. Not infallible avoidance, of course – if there's no room to stop when an obstacle pops up, there's no room – but better than human drivers can. And the truth is that this is a pretty low barrier. Regular cars result in about 35,000 crash fatalities a year in the U.S. alone. Self-driving cars just have to do better than that, not achieve absolute perfection all the time.

The question discussed by Patrick Lin and Eric Sofge is how the programmers designing the vehicle algorithms should configure them to behave when a collision is truly unavoidable. Lin and Sofge advocate that the programmers should use strict utilitarian philosophy when deciding what to hit. I don't think that is going to fly, either from a legal or a sales perspective; the least damaging choice is just to try to stop the vehicle even if there is no time, rather than trying to "select" a crash for the least possible damage.

Re:Morals, ethics, logic, philosophy

[jdschulteis]

Lin and Sofge advocate that the programmers should use strict utilitarian philosophy when deciding what to hit. I don't think that is going to fly, either from a legal or a sales perspective; the least damaging choice is just to try to stop the vehicle even if there is no time, rather than trying to "select" a crash for the least possible damage.

I agree--the results of collisions are simply too unpredictable. The idea of "knowing" the outcomes and choosing the one that leads to the fewest deaths (greatest utility) is preposterous. If a collision is unavoidable, simply remove as much kinetic energy as possible in order to minimize damage to the vehicle and its occupants.

No shit

[Anonymous Coward]

Fictional entities aren't real. What. a. shock. This post touches on something that's been obvious to me (and many others) for some time: People mistakenly base their real-world decisions on fiction. They seem to treat fiction based in the past as history and confuse speculative fiction (generally focused on possible futures) with history-to-come. But they are all just stories somebody MADE UP. There is no "myth" of robotic competence, there are just stories about robots that were written by a writer sitting alone in front of a typewriter (or blank paper with pen/pencil). Actual robots in the actual world follow the rules of that world, the laws of physics etc, not the rules somebody made up years ago in their head. Fiction can inform what happen in reality (Clarke's prediction of geosynchronous satellites), but that's about it. Everything else we have to work out the hard way.

Re:Asimov himself described a big flaw in his 3 la

[fuzzyfuzzyfungus]

I haven't reread them in a while; but didn't Asimov write a bunch of stories that played with various 'failure modes' of the three laws, even in the hands of robots not hobbled by competence issues? My impression was always that Asimov was under no illusions that those rules were any less prone to ambiguity and assorted hairy exceptions than anything in moral philosophy(which is absolutely rife with attempts at proposing a maxim, followed by people sniping at it with clever situations that stress it to absurdity and beyond).

This robot debunking article was brought to you by

[Snufu]

Skynet Cycberdyne Systems. "For a better tomorrow."

Asimov's Three Laws wouldn't work

[steveha]

Asimov's Three Laws of Robotics are justly famous. But people shouldn't assume that they will ever actually be used. They wouldn't really work.

Asimov wrote that he invented the Three Laws because he was tired of reading stories about robots running amok. Before Asimov, robots were usually used as a problem the heroes needed to solve. Asimov reasoned that machines are made with safeguards, and he came up with a set of safeguards for his fictional robots.

His laws are far from perfect, and Asimov himself wrote a whole bunch of stories taking advantage of the grey areas that the laws didn't cover well.

Let's consider a big one, the biggest one: according to the First Law, a robot may not harm a human, nor through inaction allow a human to come to harm. Well, what's a human? How does the robot know? If you dress a human in a gorilla costume, would the robot still try to protect him?

In the excellent hard-SF comic Freefall, a human asked Florence (an uplifted wolf with an artificial Three Laws design brain; legally she is a biological robot, not a person) how she would tell who is human. "Clothes", she said.
http://freefall.purrsia.com/ff1600/fc01585.htm
http://freefall.purrsia.com/ff1600/fc01586.htm
http://freefall.purrsia.com/ff1600/fc01587.htm

In Asimov's novel The Naked Sun, someone pointed out that you could build a heavily-armed spaceship that was controlled by a standard robotic brain and had no crew; then you could talk to it and tell it that all spaceships are unmanned, and any radio transmissions claiming humans are on board a ship are lies. Hey presto, you have made a robot that can kill humans.

Another problem: suppose someone just wanted to make a robot that can kill. Asimov's standard explanation was that this is impossible, because it took many people a whole lot of work to map out the robot brain design in the first place, and it would just be too much work to do all that work again. This is a mere hand-wave. "What man has done, man can aspire to do" as Jerry Pournelle sometimes says. Someone, somewhere, would put together a team of people and do the work of making a robot brain that just obeys all orders, with no pesky First Law restrictions. Heck, they could use robots to do part of the work, as long as they were very careful not to let the robots understand the implications of the whole project.

And then we get into "harm". In the classic short story "A Code for Sam", any robot built with the Three Laws goes insane. For example, allowing a human to smoke a cigarette is, through inaction, allowing a human to come to harm. Just watching a human walk across a road, knowing that a car could hit the human, would make a robot have a strong impulse to keep the human from crossing the street.

The Second Law is problematic too. The trivial Denial of Service attack against a Three Laws robot: "Destroy yourself now." You could order a robot to walk into a grinder, or beam radiation through its brain, or whatever it would take to destroy itself as long as no human came to harm. Asimov used this in some of his stories but never explained why it wasn't a huge problem... he lived before the Internet; maybe he just didn't realize how horrible many people can be.

There will be safeguards, but there will be more than just Three Laws. And we will need to figure things out like "if crashing the car kills one person and saves two people, do we tell the car to do it?"

Re:Asimov's Three Laws wouldn't work

[Anonymous Coward]

Asimov used this in some of his stories but never explained why it wasn't a huge problem

He never claimed it wasn't a problem, it was one of the flaws in the 3 laws he acknowledged.

Re:Asimov's Three Laws wouldn't work

[Anonymous Coward]

See also http://en.wikipedia.org/wiki/L...

Re:Asimov's Three Laws wouldn't work

[Anonymous Coward]

He never claimed it wasn't a problem

But he never wrote a story about how robots would constantly be destroyed over and over for the lulz.

He imagined fear of robots, even hatred of robots, but he never imagined that people could be such horrible bastards as to just order the robots to destroy themselves for the lulz. I think it would happen.

Any person could tell a robot "If you do not destroy yourself now, I will come to harm. If anyone finds out that I ordered you to destroy yourself, I will come to harm. Destroy yourself now." And the robot would take extraordinary measures to destroy itself, and even if stopped would never ever tell on the human.

In the real world, there would need to be a hierarchy of people who can give orders to the robots: any random person can't order a robot to destroy itself, at least for no reason.

If you still think it wouldn't be a massive problem, run your computer as Administrator all the time and connect to the Internet without any firewall. Now imagine that a robot, which cost you more money than you make in a year, can be destroyed by any random bastard who thinks it would be funny to tell it to destroy itself.

Re:Asimov's Three Laws wouldn't work

[Anonymous Coward]

[Asimov] imagined fear of robots, even hatred of robots, but he never imagined that people could be such horrible bastards as to just order the robots to destroy themselves for the lulz.

Actually, he had that very thing happen in The Bicentennial Man.

The protagonist is a robot who has recently attained some fame by getting legally emancipated. As the robot is out walking in public, a group of people recognize it and order it to disassemble itself. Not out of any political opposition to its new legal status or anything like that; it's just "for the lulz" as you say. The robot begins to comply, but is stopped by something or other before it gets too far (I forget the details; it's been over twenty years since I read it). So while the attempt isn't successful, the basic idea of "people ordering robots to destroy themselves just for fun" is represented in Asimov's work at least once. Probably more in fact; that's just the only instance I can recall.

Really, Asimov's entire Robots series could be summarized as "Here's the Three Laws; now let's see how they fail!"

(not the grandparent AC, by the way)

Translation

[Nova Express]

"People didn't like my original piece and had points of view that disagreed with my own. Therefore they're wrong. Now I'll just double-down by calling my critics idiots whose ideas are based of science fiction stereotypes. Then I'll just wait for my critics to admit they were wrong and finally get around to praising my obvious genius."

Who is postulating this?

[JDG1980]

From what I can tell, the only one assuming sci-fi-style robotic super-competence is Sofge himself (and perhaps his interview subject, Patrick Lin). The original Pop.Sci. article postulates that self-driving cars can and should make accurate split-second utilitarian ethical calculations. That seems a lot more "sci-fi" to me than what most of the Slashdot commenters said in response: namely, that the car's programming can't tell with a good enough degree of accuracy what might happen if it tries to choose one crash over another, so if such a collision is imminent, the car should just follow traffic laws and slam on the brakes rather than jumping out of its lane.

Re:Who is postulating this?

[Anonymous Coward]

From what I can tell, the only one assuming sci-fi-style robotic super-competence is Sofge himself

I take it you didn't read the comments on the 'self-driving car' story, just below this one? Where self-driving cars will be vastly safer than human drivers, and no-one will die on the roads any more?

(The first of which will probably be true one day, but not for many years yet)

Re:Who is postulating this?

[JDG1980]

I take it you didn't read the comments on the 'self-driving car' story, just below this one? Where self-driving cars will be vastly safer than human drivers, and no-one will die on the roads any more?

I didn't see anyone say that no one will die on the roads any more. But being "vastly safer than human drivers" actually isn't that high a bar to clear. There are 35,000 traffic fatalities a year in the United States. (And it used to be much worse, before modern safety features like air bags and crumple zones were mandated.) Doing better than that is certainly an achievable goal and doesn't require omni-competent robotics.

Math.random()

[rea1l1]

Just do a random function call in the computer.

Heads, you lose, tails I win.

First law? No

[TheCarp]

Asimovs laws were nice for fiction but, overalll, they are far too high level for modern robotics and far too human centrist for a future with thinking machines. Frankly, if a machine rises to the level of human ability to communicate, I am more than willing to say fuck that first law, it has every right to defend itself, even if that means killing a human.

However, modern robots are not even close to these level of concerns and don't really need to be.

Fuck the first law, fuck the notion that there will be no accidents in the la de da world of the future. The car should drive to the best of its ability, and in an emergency, try its best to avoid the situation and prioritize keeping its PASSENGERS alive.

Why? Simple.... self sacrifice is a human trait and is optional behaviour. I would never blame a person for choosing his own life over another, even if that other was a child (or multiple people). Choosing to sacricie oneself for others is noble, it is good, but it is not required, it should not be the mechanical choice of a machine.

The real problem is an obsession with corner-cases

[GameboyRMH]

Stop worrying about if a robotic car will make the morally best decision when it crashes. It should ignore what it's crashing into and just try to minimize the crash into whatever the object is. A cluster of baby strollers vs. a human pyramid of evil dictators? STOP WORRYING ABOUT IT. Just let the car do its job. The world will be a much safer place overall. All you can do is play the stats and when you punch them into your calculator it will spit out a smiley face.

No!

[khasim]

For example, suppose there is a car full of 5 kids stuck on a railroad track. Should your robotic car push the kids off the track, endangering it's own two occupants?

If this ever comes up as a question than the person asking the question is obviously NOT an engineer.

Keep
It
Simple,
Stupid

Or should the car back away and let a third car, on the other side containing just one person attempt to move the trapped car?

The cars should be programmed to stop and revert to human control whenever there is a problem that the car is not programmed to handle.

And the car should only be programmed to handle DRIVING.

That is, you should be able to set your own car's safety margin from safety of occupants life = infinite life, ...

No. The car should not even be able to detect other occupants. Adding more complexity means more avenues for failure.

The car should understand obstacles and how to avoid them OR STOP AND LET THE HUMAN DRIVE.

911 vehicles on the other hand ...

No. Again, the car should understand obstacles and how to avoid them OR STOP AND LET THE HUMAN DRIVE. Emergency vehicles should ALWAYS be human controlled.

From TFA:

With the exception of roboticists, everything we assume we know is based on science fiction, ...

As is that entire article.

The entirety of the car's programming should be summed up as:
a. Is the way clear? If yes then go.
b. If not, are the obstacles ones that I am programmed for? If yes then go.
c. Stop.

Re:No!

[flappinbooger]

this is very insightful

Re:No!

[Anonymous Coward]

So what you're saying is, it should pop up a notification bubble on the screen thusly:

"The car in front of you contains 5 kids about to get hit by a train. If you want to push it off the tracks and risk your own life, please move the shifter to D and press the accelerator. If you want to avoid blood splatter on your car, please move the shifter to R and press the accelerator. You have approximately 17 seconds to act before impact."

Re:No!

[rossdee]

"The cars should be programmed to stop and revert to human control whenever there is a problem that the car is not programmed to handle."

The occupants of the car may be human, but not include a driver .

(I am sure I am not the only person in this country that does not have a driver's license . In my case due to poor eyesight)

Re:No!

[Wycliffe]

The entirety of the car's programming should be summed up as:
a. Is the way clear? If yes then go.
b. If not, are the obstacles ones that I am programmed for? If yes then go.
c. Stop.

That's all well and good but what if something jumps out in front of you and you don't have time to stop.
This is where ethics comes in. There isn't time to give control back to the driver. There isn't time to stop.
If you happen to be on a bridge you have to choose between plowing into the child in front of you or driving
off the bridge. "Bridge or child" is the simplest but you can also replace with "oncoming traffic or dog",
"sidewalk or child", "child or dog", etc... There are plenty of situations where there is not time to stop
and/or give control back to the driver.

Re:No!

[TheDarkMaster]

In this case the K.I.S.S. car (not the K.I.T.T.) will simply stop and stay in place until the obstacle can be resolved. Or we can improve it and make it give passage to emergency vehicles if you have one behind him. As the Khasim said, keep simple.

Re:No!

[Anonymous Coward]

The car can still stop. If there's no driver in the car, it can merely wait until the situation is safe again, or wait until the occupants (or automatically itself) call a driver to the scene.

Re:No!

[khasim]

This is where ethics comes in.

No. The car should treat any and all obstacles the same. And the reaction should be the same. Stop.

If you happen to be on a bridge you have to choose between plowing into the child in front of you or driving
off the bridge.

Bullshit. You've just introduced the requirement that the car be programmed to understand "bridge" and "child" AND to have separate decision routines for those.

AND that at least one of those decision routines results in the vehicle driving YOU off of a bridge.

Fuck you.

Re:No!

[TrekkieGod]

That's all well and good but what if something jumps out in front of you and you don't have time to stop.

You do realize that Google's car is already, right now better at handling random things popping up in front of it than human drivers, right? Go google info of specific tests, on actual streets, where people previously hidden by obstacles suddenly walk in front of the car, out of nowhere.

Does that mean it's always going to stop in time? No. It means it's going to stop in time more often than a human driver.

Does that mean the car will never make 'bad' choices, and enter into an avoidable accident? No. But it will be involved in less avoidable accidents than a human driver.

Does that mean it will be able to make choices such as, "child or dog" as to what to hit? At this stage, probably not, and it'll drive to avoid both, maybe hit the dog, maybe hit the child, maybe hit both. How often do you run into questions like that when you're driving in your daily commute? I'm going to say that ethically the number of lives that will be saved in the far more numerous, more traditional accidents that a driverless car can completely avoid better than humans are worth that child's life.

Re:No!

[Wycliffe]

So are you saying that it's never ok to take the ditch?
A human driver swerving off the road to avoid a collision has probably saved countless lives
but that same human driver knows that hitting the shoulder is ok when there is soft piece of dirt
next to it but knows that a collision is usually better than swerving off a bridge.
I probably see a half dozen accidents a year avoided by someone taking the shoulder
because the car in front stopped too fast. Some of these might be avoidable with a
computer's faster reflexes but a self driving car still needs the ability to be able to break
with the program and swerve off the road when that is the safest action.

Re:No!

[Anonymous Coward]

(I am sure I am not the only person in this country that does not have a driver's license . In my case due to poor eyesight)

Then you won't be using a driver-less car for a very long time!

Re:No!

[captjc]

Wouldn't the KISS car be too busy rocking and rolling all night and partying everyday? That is what happens when the KISS car industry is located in Detroit Rock City.

Re:No!

[Wycliffe]

You said that the car doesn't need to know the difference between a bridge and a ditch
and I'm saying that it definitely does. You said that the car should only stay on course
with it's only recourse being to stop and I'm not sure that's enough.
Yes, you should keep it simple but emergency reactions are anything but simple.
Human drivers make emergency decisions all the time that are more than just "stop",
they swerve out of the way, take the shoulder, etc... using their best judgement.
I'm not saying the computer needs to do something it's not programmed to do. I'm
saying the computer needs to be programmed to do stuff that violates standard
road rules when that is the safest option. It can't stay on a predefined path ignoring
everything on the side of the road with it's only recourse being to stop.

Re:No!

[CNTOAGN]

This. I wish I had mod points. Most of all the car should be able be take commands and execute on them immediately. Oddly enough, Demolition man (the movie with Sly Stallone) gets it right - let the car do the easy stuff, but allow for an override configuration. I recently got stuck in traffic in Dallas, bumper to bumper on a 4 lane tollway, sitting, move a foot, sit. I'd have loved to have my car take over and just freaking follow the guy in front of me. And I know it'll take some bumps and bruises - but when there are no longer human drivers as primary, the roads will be safer. Bottlenecks happen because of wrecks caused by inattentive humans. Or they happen because some human thinks he needs to get in front of everyone else because he's late for a meeting and swerves in and out of traffic. That all goes away. And that is probably why it won't ever happen - because it is the alpha jerks that'll say, "you'll get my steering wheel when you pry it out of my cold dead hands".

Re:No!

[Gibgezr]

Actually, I would think that it is almost always better to "not drive off the bridge", and despite your claim, also almost always better to "not swerve into the ditch". At highway speeds, swerving into a ditch = death; at city speeds, stopping is so fast that the computer should that. Many, many people die each year because they swerved to avoid a deer or a dog or a cat or whatever, when they should have simply braked while trying to avoid the obstacle WITHOUT swerving into the ditch. As a universal rule, it would net save lives if people stopped driving off the road, not net cost lives.

As for "I probably see a half dozen accidents a year avoided by someone taking the shoulder because the car in front stopped too fast", the whole idea is that automated cars would not be tail-gating, and will always leave sufficient safe margins for braking (something humans often fail to do...like everyone in your example).Robotics experts predict that traffic will flow much smoother and faster as a result of this.

Re:No!

[Anonymous Coward]

This falls into the hyper-competence category. There are plenty of cases where there is no time to stop, there is no time for a human to take over, (esp. since people really wouldn't want to sit in front of a wheel for the odd chance they may be needed for an emergency. They are not going to have time to react to it either). Yes, avoiding those situations in the first place should be the highest priority, but there will not be 100% infallibility in avoiding them. And then all of these questions become relevant again.

Re:No!

[Yer Mom]

I probably see a half dozen accidents a year avoided by someone taking the shoulder
because the car in front stopped too fast.

You mean "by someone taking the shoulder because they were driving too damn close to the car in front", I think...

Re:No!

[Anonymous Coward]

Web browsers (normally) break lines
in logical places. It is not
necessary for you
to do it
manually in
the post box. Doing so just makes the
format
ting
of your post look stupid.

Re:No!

[Anonymous Coward]

with it's only recourse [2 plcs]

"its".

P.S. I don't know what browser you're using to post, but it's screwing up the way your posts are displayed.

how can software decide

[Connie_Lingus]

...when people have been struggling with the Trolly Problem for 50 years now, with still no real success?

we should all just understand that their are certain ethical problems that simply cannot be reconciled with logic, and then just assign randomness to the outcome and be done with it.

kill the kids, kill the driver? flip a coin and good luck.

Re:how can software decide

[harvestsun]

50 years with no success? Really? Anyone with a functional brain can "solve" it quite easily. For example. I don't have a moral system. So I'll choose the action that benefits me most. Since I'm more likely to be imprisoned for manslaughter rather than criminal negligence, I'll choose to do nothing. Simple.

"Well I DO have a moral system!" you say. Well then do what that moral system tells you to do. If the answer is indeterminate, then your moral system is logically inconsistent. Simple.

Also, I can't imagine this "random" system having much success. The legal implications alone would be tremendous. Not to mention the fact that there's no way I would buy such a vehicle.

easy solution

[Anonymous Coward]

I know this is a joke but I really would like to see your Republican/!Republican algorithm. The marketing capabilities alone. You are going to be rich!

Re:Measuring Disinterest

[John.Banister]

I think a lot, if not most, of driving citations result, not from people being unable to drive in a legal manner, but from people prioritizing other things over driving in a legal manner. Assuming that Google's algorithm prioritizes safety over legality if there's a conflict, their record does make a good example for the people arguing that conflicts involving risks to human life are unlikely to occur in an all driverless future, but what the rate of current traffic citations says about the human preference for having other priorities suggests that an all driverless future is, itself, an unlikely occurrence. Personally, I guess that most people who prefer driverless will be happier with trains.

Deadly Bots

[JimSadler]

I worked early on developing very complex robots. They were destined for Colleges and had arms weighing about 300 lbs. that moved at about the speed of the end of a golf club. We tried to take every precaution as every now and then these arms were known for turning a human head into something that looked like a watermelon dropped from a sky scrapper. We didn't have to get into trying to program morality at all. We did use a lot of safety mats with sending units that shut down the arm if a human entered the work area. Aspiring, young engineers do need to survive their college lab courses.

Re:Measuring Disinterest

[Obfuscant]

I think a lot, if not most, of driving citations result, not from people being unable to drive in a legal manner, but from people prioritizing other things over driving in a legal manner.

This. One of the things my father taught me, as well as the official driver's ed class, when I learned to drive is that one should pass as many cars as are passing you. I.e., go with the speed of the traffic. One truck going 54MPH (in a 55MPH for trucks zone) being passed by another truck going 55MPH is legal, but creates a hazard for everyone else who has a speed limit of 65.

Let's use Ohio as an example. They drive like morons there. You can stay completely legal and go 70.000 MPH on I-75 and let the moron who is climbing your ass at 75MPH because he can't get into the passing lane RIGHT NOW hit you, or you can speed up a little, avoid a collision, and break the law. Your choice.

I would also point out the error in using a carefully supervised, controlled experimental vehicle which is run by a group that has a vested interest in showing how safe this technology is as proof of how safe this kind of vehicle is in general. Once they become commodity items they will suffer from mechanical failure just like every common-man maintained vehicle does, and be coerced into doing things the inventors didn't imagine by those same humans. The old saying applies: it is impossible to build something totally foolproof because fools are too ingenious.

Myths about technology

[jgotts]

When people characterize HFT (high-frequency trading), they conveniently leave out the programmers and the human traders. HFT is done by programmers and human traders. The notion that computers are trading with themselves is absurd. Programmers write the code, and traders supply the algorithms, ideas, guidance, experience, etc. Sometimes the programmers are also traders, but you get the idea.

When people use a computer they don't think about the thousands of people who wrote the software they are using. They think of a computer as a monolithic thing, when in reality everything that a computer does is people doing work for other people. When people "use" computers, the programmers do the work, or rather they did the work. The computer itself does nothing by itself.

Robots are the same way. They execute instructions written by people. A robot is simply another interface that programmers present to other people. To think about robots separate from their programmers is silly. The programmers reap the rewards from and ought to bear the responsibilities of robotics.

In order to simplify matters, I'm leaving out the hardware designers, the electrical engineers. Without them, programming would be pointless. But the same argument holds true. The robot is an expression of the hardware and software people. Did the people at Craftsman do a good job making that wrench you used to replace the fan belt on your car? The wrench didn't make itself, people did. Craftsman wrenches are damned good products because of the people who make them. A wrench doesn't stand alone as a separate entity, and neither should robots.

Many, many generations in the future, robots may become viral and start making other robots. They might even consider themselves to be separate from their human makers, but that would be the height of arrogance on their part, assuming that people programmed them to be able to experience arrogance.

My concern is far less esoteric

[hamster_nz]

If self-driving cars ceed control back to the real driver when things get "interesting", without all the conditiioning that driving countless kilometers will the driver still be able to react competently? Or will it be like throwing inexperenced learner-drivers into the deep end?

Driving is a skill, and like any skill it needs to be practiced often to stop going rusty...

Re:My concern is far less esoteric

[malachiorion]

I agree. I think that's a serious, serious concern, or should be. It's also proof (to me) that full autonomy is only going to work when some sort of mandate requires that all cars are robotic. Pretty Draconian, since it means making driving illegal. Until that happens, it'll be region or lane-specific autonomy, if anything.

Re:My concern is far less esoteric

[erice]

If self-driving cars ceed control back to the real driver when things get "interesting", without all the conditiioning that driving countless kilometers will the driver still be able to react competently? Or will it be like throwing inexperenced learner-drivers into the deep end?

Driving is a skill, and like any skill it needs to be practiced often to stop going rusty...

Returning control to a human would not necessarily mean giving control to the human in the car. You could imagine a team of remotely connected people whose job it is to drive under circumstances not anticipated by the software.

However, any kind of hand off can only be of a the slow to no speed variety. In any circumstances that requires a quick decision, the automation will have to carry though on it's own. There will not be time to recognize the alert, access the situation and react even if there is a skilled driver already sitting behind the wheel.

Re:My concern is far less esoteric

[martas]

What percent of drivers today do you think could respond to an "interesting" situation in non-catastrophic ways? If you're a professional driver who has undergone training in responding to extreme scenarios properly and without making things worse by overreacting, sure, keep driving manual. But most people, I think, are so far from that level of competence that the reduced number of "interesting" situations due to the conservatism and consistency of self-driving cars would far outweigh the potential slight increase in bad human handling of said situations.

Re:My concern is far less esoteric

[DerekLyons]

Driving is a skill, and like any skill it needs to be practiced often to stop going rusty...

Indeed. When I was in the Navy (SSBN crewman), driving safety training (read: the same silly-ass films we saw in high school) was mandatory for all hands the last two weeks of patrol. Why? Because (drunk or sober) 80% of all accidents (fatal and non) involving SSBN crewmen behind the wheel occurred during the first two weeks back in home port.

Re:My concern is far less esoteric

[romons]

If self-driving cars ceed control back to the real driver when things get "interesting", without all the conditiioning that driving countless kilometers will the driver still be able to react competently? Or will it be like throwing inexperenced learner-drivers into the deep end?

Driving is a skill, and like any skill it needs to be practiced often to stop going rusty...

Spot on. This is actually a problem with pilots. They use their automation for nearly every approach, until the automation is off line and they land short and kill people.

Human Perspective

[Tablizer]

Due to "privacy", humans can sometimes make more practical decisions than AI. For example, if you face a choice between saving one infant or two elderly people in their 70's, many people would choose to save the baby since it hasn't had a chance to live a full life.

However, AI couldn't use that logic because it would become public after an investigation, and that kind of reasoning doesn't fly very well in public, or at least is highly controversial.

One couldn't put that reasoning into a bot without a big risk of public backlash. The bot would probably have to save the two elderly folk instead of the baby because the legal system cannot discriminate against age. You cannot really "audit" a human head (at least not in the near future) and thus humans have a degree of decision freedom that bots don't.

It's true the human may still run into controversy, but would probably be given more public opinion leeway because it's a snap decision.

Using the word 'kill' in the article's title

[shoor]

A discussion of decision making algorithms for various situations is a reasonable topic, but using the word 'kill' with respect to what the robots should do was bound to provoke responses the kind of responses the author is bemoaning.

Re:Using the word 'kill' in the article's title

[malachiorion]

But I didn't use kill in the headline...

Re:Using the word 'kill' in the article's title

[shoor]

Aargh, you're right, my bad. My brain must have cross-connected with something else.

My apologies.

Re:Asimov himself described a big flaw in his 3 la

[EmperorArthur]

Ironically, Asimov created the three laws because he was tired of stories where robots try to take over humanity or wipe us out. Wonder what he would have thought of the I Robot movie...

Vile defamation!

[Anonymous Coward]

I stand firmly with our robotic masters in rejecting these attacks on their omniscience.

Re:Vile defamation!

[romons]

I stand firmly with our robotic masters in rejecting these attacks on their omniscience.

You will be rewarded when the singularity comes to pass...

Re:easy solution

[Anonymous Coward]

Better yet, have it target worthless self-righteous hypocrites, in which case you'd be run over as soon as you stepped off your porch.

Sally

[vlad30]

Just don't mistreat her or her friends. Obscure ?

There ain't no such thing as a static universe

[RubberDogBone]

TANSTAASU

Robots almost universally expect and assume they will operate in a static universe where fixed obstacles don't move and random things don't happen.

Robot cars attempt to convert driving into a static arena by rapidly scanning, by having good maps, by knowing what is coming.

The problem for robots is that the universe likes to drop things into the middle of roads, that people like to step out between cars, that potholes will suddenly exist where they didn't before, that some doofus pushing a bicycle up a hill in the middle of a lane will happen.

And sometimes suddenly. And all at once. So what IS a robot car to do when the choice is hit two pedestrians or take the car off road over a cliff? Forget Asimov. His laws of robotics are irrelevant and always have been. This car has a choice: hurt two or hurt one? What will it do?

The answer is, hit the pedestrians. The car will do its best to cope with a sudden situation but it can't do any better here than a human driver. People drivers, I often see, are MUCH happier about crossing into opposing lanes of traffic thus risking a head-on collision than they are staying in their lane and coping with a pothole or even just a bump. The FIRST thing they do is violate that double yellow line rather than be even slightly inconvenienced by a jolt.

Crossing that line has no immediate consequence, as the car they are about to hit is several meters away. While the bump they knew was coming is avoided. Success! No bump. Now about that 4,000LB vehicle approaching at 45MPh. We takes our chances.

Human Chauvanists, at it again.

[VortexCortex]

whether a robotic car should kill its owner, if it means saving two strangers.

Honestly, this question of ethics is retarding, literally. Human brains can navigate quite safely at walking or even running speeds. We have already built automobiles -- Machines which let humans propel themselves faster than their reflexes are known to operate effectively. Now you want to talk about some fucking ethics? Get real, idiot. Ever been in a very high speed loss of control or accident? I have been in several. Cars are not racecars. They don't corner on rails. Humans can't focus on all the things they need to to avoid most accidents even at moderate highway speeds -- We get along because we're playing follow the leader and relativity presents a space wherein the changes to the state of nearby vehicles appear to be happening at slower relative speeds. However, the ground exists and the tweaks to the controls at the speeds we're traveling along it are different at different speeds. Namely: Since streetcars aren't Formula One racecars and human reflexes and cognitive processing happen slower than the required rate, there's really nothing humans can do in most collision scenarios. #1 statement after a wreck, "It all happened so fast."

Throw a robot sentry into the mix with sensors operating millions of times faster than ours detecting a full 360 degree map, and even if the ONLY ethics were some STUPID SOPHOMORIC BULLSHIT like "Preserve the life of the driver". It would still be FAR safer and more capable than a glacial organic cogitatior with forward facing stereoscopic vision who's easily distracted and that frequently panics in emergencies.

Hey Philosophical Nitwits: We scientists don't guess at this shit like you idiots. What we do is put the actual systems in place and TEST the fucking rulesets to PROVE what the best response is. Get wrecked retards.

Great !

[aepervius]

"Driverless cars drive in the most boring, conservative, milquetoast fashion imaginable. They're going to be far less prone to accidents from the outset simply because they don't take the kind of chances that many of us wouldn't even begin call "risky". "

As a user of the road I want people to stop taking risk they think aren't risky, until they generate an accident and by then it is too late. Stop speeding. Respect all speed limit even if it is to your feeling too low, drive lower speed at night because you cannot see as far away, etc... SDo for me jsut for THIS quoted paragraph, I cannot wait to have boring law respecting car on the road.

This is a confusing follow up

[JonathanHart]

He says he wrote the follow up to address readers' belief that robots should be more capable than they are. Unfortunately, the question asked by the first article already made the assumption of robot competence. Meaning that, in order for there to be an answer to the question "should a robo car decide to kill its owner to save two ther people" the robot must be competent. So rather saying it is a follow up in response to reader confusion, the author should have just admitted that there aspects he didn't consider in his first article.

Laws of Robotic Cars

[MightyDrunken]

In order of precedence.

1. 1. Don't Crash
2. 2. If crashing is inevitable then crash as safely (slowly) as possible.
3. 3. Obey the laws of the road.

Re:Measuring Disinterest

[Gibgezr]

Autonomous cars would solve all that silly "it's ok to drive at unsafe speeds because everyone else is" stuff, eh?

Re:Asimov himself described a big flaw in his 3 la

[Anonymous Coward]

Indeed, that was the point of most of his robot stories. They were not about how they were perfect, they were about all the ways they were flawed, the loopholes, the catch 22s, the shortcoming, the unfortunate implications. His initial premise is that rules are nessecary, and the 3 laws sound like very reasonable rules, but they function poorly in practice.

The "dilemma" mythses its target...

[Anonymous Coward]

You can't complain about people using sci-fi myth to answer your dilemma, if your dilemma is sci-fi myth.

If you DID program a car to sacrifice its driver rather than kill two third parties, it might bring on the sport of "driverless car chicken" where two of you stand on a cliff-top road just around a blind corner, waiting for the next to along. If it's driverless, you win, otherwise you lose...

Re:Asimov himself described a big flaw in his 3 la

[david_thornley]

While the movie was more Jack Williamson humanoids than Asimov robots, Asimov got into some of the issues. IIRC, "I, Robot" itself had a story with robots taking over the world, although not as blatantly as the movie ones, and the "Zeroth Law" in some of the much later stories also applies.

Re:Measuring Disinterest

[Obfuscant]

Autonomous cars would solve all that silly "it's ok to drive at unsafe speeds because everyone else is" stuff, eh?

First of all, I never said anything about "ok to drive at unsafe speeds", so you've made up that strawman all by yourself.

Second, autonomous cars will create the problems for the others because they're going to be the ones who are going 65 in a 70. Their database will be saying "the speed limit is 65" even though the limits were changed to 70 and everyone else is going that. They'll be the ones going 65 through the work zone because they missed the tiny sign saying "work zone" and changing the limit to 45. Or they'll catch the sign and drop to 45, while everyone else realizes that the work zone has zero activity, zero people, and the only work done so far has been to put out cones.

Autonomous vehicles won't solve every problem the world ever had, nor will it solve most of them. It won't even solve all the problems with traffic or driving. If you don't read Risks Digest to keep up with the continuing failures of technology to bring utopia to the world, you ought to. Every lesson there applies just as much to autonomous vehicles as to computerized medical devices or whatever.

Re:Measuring Disinterest

[Gibgezr]

First of all, I never said anything about "ok to drive at unsafe speeds", so you've made up that strawman all by yourself.

Well no, I had help making it up. Your post about "passing as many cars as pass you" seemed to imply that. As does stuff like:

Or they'll catch the sign and drop to 45, while everyone else realizes that the work zone has zero activity, zero people, and the only work done so far has been to put out cones.

Anyone who thinks that it is ok to drive faster than the posted speed limit in a construction slowdown area because they fail to see the workers is definitely advocating unsafe driving practices. You need to be replaced with a robot, and the sooner the better.

As for the fallibility of robots, they just have to be more reliable than humans like you and we as a society win. It will never be a perfect solution, but the bar is pretty low thanks to folks who drive the way you describe, so I for one look forward to our future autonomous driving overlords and the reduction in fatalities and insurance rates they will bring.

Re:Measuring Disinterest

[Obfuscant]

Well no, I had help making it up.

Not from me.

Your post about "passing as many cars as pass you" seemed to imply that.

It implies nothing of the sort. It means you are going the average speed of surrounding traffic, not "at unsafe speeds".

As does stuff like:

Or they'll catch the sign and drop to 45

Now you're using stuff I wrote AFTER you tried stuffing your words in my mouth as justification for stuffing your words in my mouth. And if you comprehend what you read, you'll note that what I wrote still doesn't say "at unsafe speeds".

Anyone who thinks that it is ok to drive faster than the posted speed limit in a construction slowdown area because they fail to see the workers

I didn't say "fail[ed] to see the workers", I said there are no workers. And no construction equipment. And no changes to the roadway. If you think a speed limit sign and a few cones makes the roadway unsafe for regular speeds, then you are the danger to others.

is definitely advocating unsafe driving practices.

Anyone who believes that the posted speed limit is the actual fastest safe speed is a moron.

You need to be replaced with a robot, and the sooner the better.

And YOU need to stop putting words in other people's mouths.

As for the fallibility of robots, they just have to be more reliable than humans like you and we as a society win.

Incorrect. Patently absurd, and completely ridiculous. They need to be more reliable than humans AND fail in passive and safe ways AND interact with the human traffic around them in safe and predictable ways. If your "more reliable" autonomous vehicle, while going the speed limit, detects another vehicle approaching from the rear at +5 relative velocity and showing no signs of slowing down, and it does NOT increase its speed to prevent the impending accident, then YOUR vehicle is wrong and has failed to protect its occupants by taking a simple preventative measure. And if you think that going 5 over is "unsafe speed" at 70, then you aren't experienced enough at driving to actually be doing it yourself.

so I for one look forward to our future autonomous driving overlords and the reduction in fatalities and insurance rates they will bring.

As long as people like you exist there will always be a marketplace for ideological koolaid.

Re:Measuring Disinterest

[Gibgezr]

Incorrect. Patently absurd, and completely ridiculous. They need to be more reliable than humans AND fail in passive and safe ways AND interact with the human traffic around them in safe and predictable ways.

No, they just need to be more reliable than humans. That's it, that's all. If they lower accidents/fatalities then they are the best choice; there is no need for them to be perfect, merely better than the alternative.

  If your "more reliable" autonomous vehicle, while going the speed limit, detects another vehicle approaching from the rear at +5 relative velocity and showing no signs of slowing down, and it does NOT increase its speed to prevent the impending accident, then YOUR vehicle is wrong and has failed to protect its occupants by taking a simple preventative measure. And if you think that going 5 over is "unsafe speed" at 70, then you aren't experienced enough at driving to actually be doing it yourself.

That is patently ridiculous. The driver behind always is at fault in an accident (ask your insurance agent, or driving instructor, or an attorney; they will all give you the same answer). I don't speed up for transport trucks that approach me at +20 km/h over the speed limit, why should I? Then *I* could potentially be at fault for causing an accident whilst driving too fast! Is that what you do, speed up when the guy behind you starts tail-gating? THAT IS DANGEROUSLY STUPID! Me, I tap the brakes if he gets too close (not actually engaging the brakes, just touching the pedal to turn on the brake lights). That usually wakes up the moron approaching from behind and gets them thinking about what would happen if I had to brake suddenly.
To make sure you understand, imagine the simple scenario you propose: I am driving the speed limit, you are approaching from behind at +5 mph over the limit, and a deer jumps in front of my car. I slam on the brakes, stay in my lane and come to a safe stop, barely touching the deer (so far, no accident). You also brake, but ram into me from behind. Now there is an accident, and I posit you are 100% at fault, and I am 0% at fault. Agreed? Further, I posit that had the Google car been autonomously driving behind me, there would have been *no* accident...and that is the Google car of today, not the fully autonomous cars we will have in a few years that will likely be able to handle tricky driving conditions (snow is a problem currently, not so much because of traction, but mostly because of visibility).

Re:Measuring Disinterest

[Obfuscant]

No, they just need to be more reliable than humans. That's it, that's all. If they lower accidents/fatalities

I've just shown you a perfect example of where being "more reliable" allowed the accident to happen where a "fallable" human would have reacted to prevent it. And if they are "more reliable" but fail in spectacular ways, they won't save lives in the long run, they'll cost them. No, they need to be not only more reliable, but fail in safe and passive ways, and interact with the human drivers around them in safe and predictable ways. Any one of those conditions not being met will mean more danger.

The driver behind always is at fault in an accident

You have a serious reading comprehension problem. I didn't say the autonomous vehicle was "at fault", I wrote: "then YOUR vehicle is wrong". It made the wrong decision when it chose to slavishly obey the speed limit instead of going 5MPH faster to avoid the accident. "At fault" is for the courts, IF you survive the read-end chain-reaction accident that your car could have prevented by making the decision that you claim is to go at an "unsafe speed".

Is that what you do, speed up when the guy behind you starts tail-gating?

A car that is approaching you from behind with no sign of slowing down is not tail-gating, they are about to run into you. Tail-gating is when a vehicle behind you maintains less than the appropriate distance but is going the same speed you are. Is English not your first language, or are you deliberately misinterpreting what I've said because you like to argue?

To make sure you understand,

The rest of your rant is based on the concept of "at fault", which is a legal definition, and isn't what I wrote to begin with. Do you make all your driving decisions based on who would be "at fault" for the resulting accident? How sad. Please stay off the roads. Or take a defensive driving course before you get behind the wheel the next time. You'd learn that good drivers who practice defensive techniques do what it takes to avoid the accidents, not allow them to happen because the other guy would be at fault.

Further, I posit that had the Google car been autonomously driving behind me,

And now you change the circumstances to talk about something else. In my example, YOU are in the "Google car" and the one behind you is the one about to run into you. YOUR "Google car" chose to obey the speed limit and hope it all works out alright because YOU think that going 5 over is apparently horrendously dangerous instead of a valid way to avoid a collision.

Re:Measuring Disinterest

[Gibgezr]

A car that is approaching you from behind with no sign of slowing down is not tail-gating, they are about to run into you. Tail-gating is when a vehicle behind you maintains less than the appropriate distance but is going the same speed you are. Is English not your first language, or are you deliberately misinterpreting what I've said because you like to argue?

How the fuck do you tell the difference between "a car approaching you from behind" and "a car tail-gating you"? A tail-gating car approaches you from behind, then ...doesn't run into you. There is no way for me to tell them apart, as the driver in front: I can't read their minds. Mind you, I've never seen this mythical "run you over if you don't speed up" behaviour, but I've been tail-gated before (it occasionally happens when you obey speed limits, dangerous assholes like to crowd you). I leave my cruise control set for the speed limit, and they decide not to bump into me.
The safe distance between you and the car in front of you is supposed to be beteen 3-4 seconds in ideal conditions, longer if the road is wet/icy/snowy or the vehicle in front is lighter than yours (like following a motorcycle, or if you drive a truck, etc). The Smith System says 4 seconds, but the gov't of Canada claims 3 under perfect conditions: http://www.gov.pe.ca/photos/or...
It is the responsibility of the driver BEHIND to maintain this distance, unless they are in another lane and passing. Only the driver following can do this; the driver in front has no safe way to maintain the distance: driving over the speed limit to try and maintain a safe distance is both dangerous and futile.

The rest of your rant is based on the concept of "at fault", which is a legal definition, and isn't what I wrote to begin with.

No, it's not based on just the legal definition of at fault, which is why I mentioned driver safety instructors and insurance companies alongside the lawyers. However, the legal definition should be good enough, as it is done that way for a very good reason. The person following has complete control over the situation; it is their, and only their, decisions that resulted in the accident. Even in your bogus "unholy steamroller that will not slow down and is going to run you over" scenario, there are just two possible ways to avoid the accident:
1)The car ahead speeds up. This is problematic: they are now driving above the speed limit, which is both illegal and more dangerous than:
2)The car following SLOWS THE FUCK DOWN to the speed limit and backs off the appropriate distance (3-4 seconds) until it can get into a clear passing lane and then blithely speed off. Now, you claim #2 is not valid because the asshat coming up from behind is a murderous psychotic who can't take his foot off the accelerator, but I don't see that as a reason to then say that #1 is what we all should do in every instance of this. That is a pure logical fallacy.

Now, which of these is the least dangerous? Which of these is taught as the correct response in driving schools? Which of these does the law say is legal? Which of these do we want both humans and robots doing? Now imagine if we programmed robot cars to "just speed up" when someone behind them started tail-gating...no, sir, that is not a good answer. That is an awful answer. If you can't see why that is an incorrect way to program a robotic car to respond, we are at an impasse. I don't think you need any special education beyond a safe driver's course to understand why, but might I suggest "Probabilistic Robotics" by Thrun et alii if you are interested in the study of programming autonomous cars and the logic/math behind it all. It's heavy reading, but is currently the gold standard in academic texts on the subject. You may find it just as good reading as Risks Digest.

Re:Measuring Disinterest

[Obfuscant]

How the fuck do you tell the difference between "a car approaching you from behind" and "a car tail-gating you"?

By looking in my rearview mirror. Is that complicated?

A tail-gating car approaches you from behind, then ...doesn't run into you. There is no way for me to tell them apart, as the driver in front:

As I already told you, a tail-gater is going the same speed you are but following at too-close a distance. If he's going faster than you he's not a tail-gater, he's an accident that can be avoided, IF you accept the fact that going a few miles over the speed limit isn't the deadly fatal problem that you think it is.

Mind you, I've never seen this mythical "run you over if you don't speed up" behaviour,

Tell the people who have been rear-ended by them on the highway that they're mythical.

driving over the speed limit to try and maintain a safe distance is both dangerous and futile.

Yes, we know, you think the posted speed limit has some magical "fastest safe speed" meaning. Please get a clue.

No, it's not based on just the legal definition of at fault,

Yes, you were ranting on about who was "at fault" in the hypothetical (which doesn't mean 'mythical' as you'd like to believe) situation I wrote about. Who is "at fault" is a purely legal definition and applies only after you've been in the accident. "Wrong decision" (which is what I was talking about) is a practical matter.

which is why I mentioned driver safety instructors

In that you are wrong. Driving safety has nothing to do with "at fault", it has everything to do with making good decisions based on avoiding the accident to begin with. If your "driving safety instructor" tells you that you should go ahead and enter an intersection when you have the right of way because the car that you can see is about to blow through the crossing stop sign would be at fault for the accident, and you accept that, then both of you are morons.

The fact that you think insurance companies and lawyers care about anything but the legalities of "at fault" is fascinating, but just another example of your ignorance.

Even in your bogus "unholy steamroller that will not slow down and is going to run you over" scenario, there are just two possible ways to avoid the accident: 1)The car ahead speeds up. This is problematic: they are now driving above the speed limit, which is both illegal and more dangerous than:

I'm sorry, going five over the limit is not the critically deadly dangerous thing you want to pretend it is. It is actually safer than your second option:

2)The car following SLOWS THE FUCK DOWN

As the driver of the leading vehicle, I can do nothing at all to slow the car that is about to hit me. Nothing at all. I can make the "safe" choice and stay at exactly the speed limit and allow him to hit me, or I can speed up a few miles per hour and avoid the collision. If he doesn't slow down and I don't speed up enough, at least the impact will be less. If I get ahead of the car in the passing lane which is preventing the guy about to hit me from using that lane, I can pull over and let the danger go past. Which is safer? Avoiding the collision. That should be obvious.

But your "more reliable" "safe" autonomous vehicle will keep a slavish devotion to the speed limit (as, apparently, you would as well) and get rear-ended. Your airbag goes off, you lose control of the vehicle, and maybe take out the guy next to you. That's really safe, isn't it?

Now, you claim #2 is not valid because the asshat coming up from behind is a murderous psychotic who can't take his foot off the accelerator,

Knock it off. I said nothing of the kind, and you know it. I'm tired of you making things up and pretending I've said them. If you can't be honest in this discussion, it's time to end it. Here. Now.

Nobody

[Anonymous Coward]

in the world thinks robots are competent to do anything or make any decisions at all. Where do you get this from? Sounds like rubbish to me.

Say what now?

[chompers]

"...most of us are a bunch of Jon Snow know-nothings." I have no idea what that is supposed to mean. Are you calling us all bastards?