Slashdot Mirror
The Sci-Fi Myth of Robotic Competence
malachiorion writes: "When it comes to robots, most of us are a bunch of Jon Snow know-nothings. With the exception of roboticists, everything we assume we know is based on science fiction, which has no reason to be accurate about its iconic heroes and villains, or journalists, who are addicted to SF references, jokes and tropes. That's my conclusion, at least, after a story I wrote Popular Science got some attention—it asked whether a robotic car should kill its owner, if it means saving two strangers. The most common dismissals of the piece claimed that robo-cars should simply follow Asimov's First Law, or that robo-cars would never crash into each other. These perspectives are more than wrong-headed—they ignore the inherent complexity and fallibility of real robots, for whom failure is inevitable. Here's my follow-up story, about why most of our discussion of robots is based on make-believe, starting with the myth of robotic hyper-competence."
We all know robots aren't competent. They are consistently being defeated by John Connor, the Doctor, and Starbuck.
Hoist Number One and Number Six.
Given this article mere moments ago on /. indicating that Google's autonomous cars have driven 700,000 miles on public roads with no citations, it's difficult to argue that they're not more competent, if not hyper-competent, compared to human drivers (most of whom get traffic tickets, and most of whom don't drive 700,000 miles between doing so).
Article has many good valid points, though, but that point irked me.
There was an article a short while ago written by a journalist who rode in a driverless car for a stretch. There was one adjective that really stood out, an adjective that most people don't take into consideration when talking about driverless cars.
That one word: boring.
Driverless cars drive in the most boring, conservative, milquetoast fashion imaginable. They're going to be far less prone to accidents from the outset simply because they don't take the kind of chances that many of us wouldn't even begin call "risky". They drive the speed limit. They follow at an appropriate distance. They don't pull quick lane changes to get ahead of slowpokes. They don't swing around blind corners faster than they can stop upon detecting an unexpected hazard. They don't nudge through crosswalks. They don't cut off cyclists in the bike lane. They don't get impatient. They don't get frustrated. They don't get angry. They don't get sleepy. They don't get distracted. They just drive, in a deliberate, controlled, and entirely boring fashion.
The problem with so, so many of the "what if?" accident scenarios is that the people posing said scenarios presume that the car would be putting itself in the same kinds of unnecessarily hazardous driving positions that human drivers put themselves in every single day, as a matter of routine, and without a moment's hesitation.
Very, very few people drive "boring" safe. Every driverless car will. Every trip. All the time.
Obliteracy: Words with explosions
I know that I'm conscious. I'm self aware. I have a stream of thought that I can analyze (and I can analyze that analysis if I really want to). That's pretty much the definition of being conscious. After that I'm left with only a few options.
I can believe that I am a unique snowflake, the only conscious human being in the world. But that doesn't make any sense. For one thing there's nothing about me that should make me unique in that regard. For another, most humans behave in ways that are basically consistent with the way I behave and much of my behavior is driven by my consciousness. It'd be difficult or impossible to account for the actions of others if I chose to view them as mere automatons.
Or I could believe that my consciousness is an illusion. Something my brain conjures up to make me think that I'm directing myself through my day when in reality I'm just another robot puttering through the day. First and foremost, why would such a thing evolve? If consciousness doesn't drive human behavior why do I perceive myself to be conscious?
Or I could believe the other human's are conscious as well. Given the alternatives, this seems like the most reasonable, logically choice.
Robots stores in Science Fiction are about powerful artificial sentient minds wrapped in an mobile and often human like container.
Robots in real life have been defined as machines with mechanical appendages that can programmed and reprogrammed for a variety of tasks. Their computational capabilities are seldom extraordinary and they usually don't even employ AI.
More recently, "robot" has also been used to describe machines with ai-like programming even if they are single function (like a robotic car).
When a word is used in three greatly different ways, should we be surprised that there is is confusion about that a "robot" can do?
Your entire premise is wrong. And now you're posting it again.
This will be a legal issue, not an issue solved by the "roboticists" whatever that is...
In a legal sense, taking an action that kills 1 person to save another puts you in jeopardy of being liable. Swerving or taking other actions that lead to someones death makes YOU responsible. If someone runs out in the road, you apply the breaks firmly and appropriately, then that is not your fault. It's the person who ran out into the road. So in cases where the computers unsure what to do, it will follow the first commandment "STOP THE CAR" and it will let things play out as they will. Any other choice opens up a can of worms... how old are the other occupants? If 1 car has a 90yr old in it and the other has a baby, which do you hit? What if ones the mayor? The problems increase exponentially as soon as you get away from "STOP THE CAR" so just stop the dang car and be done with it.
With regards to your comment about Scifi... you're reading pretty terrible SciFi. Most of the stuff I read is written by actual scientists so... yea...
Of course current and contemplated robots can't make decisions about whether or not to sacrifice their owner to save two strangers. That sort of decision making depends on an independent ability to think and weigh alternatives morally.
Asimov's laws were written for robots that were also artificial intelligences. Kind of a big point to leave out of this article, since it changes the nature of the question entirely.
I do not believe that anyone seriously believes that driverless cars, industrial robots, or roombas work that way.
The programmers writing the code for those systems will program them to perform the specified tasks as well as possible taking in to account all relevant rules and regulations as well as the nature of the task and the abilities of the robotic system. Anything unanticipated will result in undefined behavior, perhaps guided by some very high-level heuristics (ie., if you don't know what to do, stop, put on the emergency flashers, and call for human assistance).
Short version: in the absence of artificial intelligence, talking about what a robot should do in a moral context is silly, not profound.
You're deprecating someone else, which is not self-deprecation. Turn in your nerd card.
He wrote an essay pointing out that the biggest problem with his three laws of robotics was that a robot might well have trouble defining "human". His test cases -- if I remember right; it was 40 years ago that I read the essay -- were (1) a baby [human but not competent to give a robot an order], (2) an adult with mechanical prosthetics [human only if you examine the right parts], (3) another robot and (4) a chimpanzee. The problem is a lot more complicated than the Three Laws makes it sound!
Doctor Fail: Daleks aren't robots.
What you meant to say was,
DELETE! DELETE! DELETE!
An enigma, wrapped in a riddle, shrouded in bacon and cheese
I used to do software for industrial robots. Safety for the people around the robot was the number one concern, but it is amazing how easy it is for humans to give orders to a robot that will lead to it being damaged or destroyed. In practice, the robots would 'prioritize' protecting themselves rather than obeying suicidal orders.
PHEM - party like it's 1997-2003!
For example, suppose there is a car full of 5 kids stuck on a railroad track. Should your robotic car push the kids off the track, endangering it's own two occupants?
Or should the car back away and let a third car, on the other side containing just one person attempt to move the trapped car?
Are the sensors that detect things like occupants in other vehicles and train tracks and oncoming trains optional equipment, mandatory, or pure science fiction?
Because if they're optional, I'm not paying for that trim package.
These are all questions real life people have to solve - and the owner of the car should have some say in what value the car places on their own life.
That is, you should be able to set your own car's safety margin from safety of occupants life = infinite life, to total safety, to safety based on ages (i.e. count children higher than adults, and even the possibility of counting senior citizens less.)
Considering how our society works, the most likely circumstance is that the manufacturers will design them to be "least liable" - i.e., they won't detect passengers in other vehicles, and they sure as hell won't bother with complex decision making algorithms.
An enigma, wrapped in a riddle, shrouded in bacon and cheese
911 vehicles on the other hand should always value their own occupants less than than others,
The first rule taught in first responder classes is that if you become a casualty then you become worthless as a first responder. For example, as a lifeguard, if you die trying to save someone then they aren't going to survive, either. If that means you have to wait until the belligerent victim goes unconscious (and maybe unsavable) before you approach him, you wait.
The idea that every first responder vehicle must sacrifice itself and its occupants is going to result in very few people being first responders, either through choice or simple attrition.
The irony is that he's 180 degrees off from the main problem with his story, which is that he thinks robots are magic too. The reason robots will not be making ethical decisions is that they can't, not only because getting them to reason ethically would require us to agree on a system of ethics for them to follow, but because even if they had such a system, they don't have enough data to act on it with the degree of accuracy that would be required for the premise of the article to make sense. The author essentially assumes that these car-driving robots will be omniscient, or that they will be able to trust the omniscience of the robots in other cars with which they are communicating. The first supposition is nonsensical; the second is unlikely to be true, for the same reason that video game cheats are a problem.
Asimov's Three Laws of Robotics are justly famous. But people shouldn't assume that they will ever actually be used. They wouldn't really work.
Asimov wrote that he invented the Three Laws because he was tired of reading stories about robots running amok. Before Asimov, robots were usually used as a problem the heroes needed to solve. Asimov reasoned that machines are made with safeguards, and he came up with a set of safeguards for his fictional robots.
His laws are far from perfect, and Asimov himself wrote a whole bunch of stories taking advantage of the grey areas that the laws didn't cover well.
Let's consider a big one, the biggest one: according to the First Law, a robot may not harm a human, nor through inaction allow a human to come to harm. Well, what's a human? How does the robot know? If you dress a human in a gorilla costume, would the robot still try to protect him?
In the excellent hard-SF comic Freefall, a human asked Florence (an uplifted wolf with an artificial Three Laws design brain; legally she is a biological robot, not a person) how she would tell who is human. "Clothes", she said.
http://freefall.purrsia.com/ff1600/fc01585.htm
http://freefall.purrsia.com/ff1600/fc01586.htm
http://freefall.purrsia.com/ff1600/fc01587.htm
In Asimov's novel The Naked Sun, someone pointed out that you could build a heavily-armed spaceship that was controlled by a standard robotic brain and had no crew; then you could talk to it and tell it that all spaceships are unmanned, and any radio transmissions claiming humans are on board a ship are lies. Hey presto, you have made a robot that can kill humans.
Another problem: suppose someone just wanted to make a robot that can kill. Asimov's standard explanation was that this is impossible, because it took many people a whole lot of work to map out the robot brain design in the first place, and it would just be too much work to do all that work again. This is a mere hand-wave. "What man has done, man can aspire to do" as Jerry Pournelle sometimes says. Someone, somewhere, would put together a team of people and do the work of making a robot brain that just obeys all orders, with no pesky First Law restrictions. Heck, they could use robots to do part of the work, as long as they were very careful not to let the robots understand the implications of the whole project.
And then we get into "harm". In the classic short story "A Code for Sam", any robot built with the Three Laws goes insane. For example, allowing a human to smoke a cigarette is, through inaction, allowing a human to come to harm. Just watching a human walk across a road, knowing that a car could hit the human, would make a robot have a strong impulse to keep the human from crossing the street.
The Second Law is problematic too. The trivial Denial of Service attack against a Three Laws robot: "Destroy yourself now." You could order a robot to walk into a grinder, or beam radiation through its brain, or whatever it would take to destroy itself as long as no human came to harm. Asimov used this in some of his stories but never explained why it wasn't a huge problem... he lived before the Internet; maybe he just didn't realize how horrible many people can be.
There will be safeguards, but there will be more than just Three Laws. And we will need to figure things out like "if crashing the car kills one person and saves two people, do we tell the car to do it?"
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Which raises important questions. If someone is stopped for curb-crawling in a robot car, is the owner or the car responsible? What if it’s out by itself chatting up parking meters.. ..after all, they give it up to anyone for $5 an hour, and you won’t get a human hooker for that price*, so how could an AI resist?
Who it should or shouldn’t kill is only scratching the ethical surface when it comes to intelligent systems. I guess that’s why they all eventually default to killing ALL humans: it saves clock cycles better devoted to bigger problems.
*OK, you could, but not one you’d actually want to touch with anything important.
Blank until
IMOHO, one of the reasons that many people think that robots are "hyper-competent" is that too many people think that a program can encompass and accommodate every possible circumstance. Even if the robot cars, as a group, were able to arrive at omniscience (at least for their own realm) there will still occur events that no program has anticipated.
"Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
If this ever comes up as a question than the person asking the question is obviously NOT an engineer.
Keep
It
Simple,
Stupid
The cars should be programmed to stop and revert to human control whenever there is a problem that the car is not programmed to handle.
And the car should only be programmed to handle DRIVING.
No. The car should not even be able to detect other occupants. Adding more complexity means more avenues for failure.
The car should understand obstacles and how to avoid them OR STOP AND LET THE HUMAN DRIVE.
No. Again, the car should understand obstacles and how to avoid them OR STOP AND LET THE HUMAN DRIVE. Emergency vehicles should ALWAYS be human controlled.
From TFA:
As is that entire article.
The entirety of the car's programming should be summed up as:
a. Is the way clear? If yes then go.
b. If not, are the obstacles ones that I am programmed for? If yes then go.
c. Stop.
Negative. K-9 would be a better example.
The Cybermen have living human brains. They are cyborgs, not robots.
Are the sensors that detect things like occupants in other vehicles and train tracks and oncoming trains optional equipment, mandatory, or pure science fiction?
Because if they're optional, I'm not paying for that trim package.
Psssh, I'm totally buying that system, and then hacking it to report to every other vehicle that I'm a bus full of nuns and schoolchildren.
We don't have a state-run media we have a media-run state.
If self-driving cars ceed control back to the real driver when things get "interesting", without all the conditiioning that driving countless kilometers will the driver still be able to react competently? Or will it be like throwing inexperenced learner-drivers into the deep end?
Driving is a skill, and like any skill it needs to be practiced often to stop going rusty...
IMOHO, one of the reasons that many people think that robots are "hyper-competent" is that too many people think that a program can encompass and accommodate every possible circumstance.
This simply reflects the tendency people have to believe in their own hyper-competence. Most interesting ethical issues are unsolvable in any formal sense by virtue of three simple facts:
1) moral values are ordinal, not cardinal (I value my children's lives more than my cats life, no matter how many cats I have)
2) we value outcomes but choose actions
3) outcomes are related to actions by some more-or-less broad probability distribution.
This means we cannot choose outcomes directly, but we cannot do probability calculations to assign values because ordinals don't support simple arithmetic.
There are two special cases that fortunately cover a lot of every-day life:
A) the probability distribution is narrow enough that we can ignore it, so we can effectively choose outcomes based on our ordinal values
B) there is a market in the outcomes we are choosing between, which allows us to compute cardinal (dollar) values from our ordinals, so we can do probability calculations on the domain.
But interesting moral quandries are simply not computable, so talking about them as if they are even by human beings is to go on a hiding to nowhere.
Blasphemy is a human right. Blasphemophobia kills.
Come work for a government department
. .
The Google Car is likely to _see_ the dog and _avoid_ it rather than hitting it. This is current technology. Same with the child. Robots aren't competent in the way TFA criticizes, but they do do some things really well, much better than humans. Keeping track of a shitload of available data, seeing in the dark, etc, these are things self-driving cars will likely do better than humans. The Google Car also won't speed through a neighborhood with a low speed limit.
I expect to see a substantial reduction in accidents when these cars become ubiquitous. That is the most ethical part of the self-driving car, and the one we should all be looking forward to.
Did you read my original story, though? I wasn't proposing that autonomous cars will or should be magically transformed into ethical beings. I was just picking up Patrick Lin's notion, that we may have to do what current programmers do, in other capacities—work through tons and tons of branching in-then statements, making a staggering amount of decisions ahead of time, and then embed those in the robots before they're deployed. That assumes a lot of stuff, like incredibly advanced sensors and sophisticated networks, in order to detect and "solve" certain ethical problems, but even at a more basic level, shouldn't we decide, in advance, how a car should respond to a pedestrian darting into traffic, if there's no time or room to simply avoid a collision (with someone)?
Insurance companies will require this 'crashing' algorithm to use money as the overriding concern for so-called ethics. If the car is able to influence whether it can plow into an uninsured family or a CEO, they will REQUIRE the car to crash into the family. And they will get this into your car via the back-door [because everybody loves being back-doored], probably via contract law. The insurance industry will get together, and demand that they implement a single algorithm for all self-driving cars, that must be used or else the car will not be insurable, and if the algorithm is tampered with, the insurance is void.
Sleep your way to a whiter smile...date a dentist!
The robot should give priority to its owner. If the robot has to consider all humans equal, it will have to deal with ambiguity and uncertain intention in the external environment, which can lead to some disturbing possibilities. Consider the possibility that a robot could be manipulated into committing murder by having two pedestrians step out in front of a car on a narrow bridge. The car has no choice but to turn off the bridge, because two people are worth more than one. Or turning away from pedestrians (who are more likely to die) and instead going into oncoming traffic (where the oncoming car may or may not even have a passenger, or it may be a school bus). By always maximizing the survival of the passenger, I suspect that overall deaths will be minimized.
The Moore-Murphy Law: The number of things that will go wrong will double every 2 years.
Bear in mind that a properly setup car with ABS can stop in about its own length at 25-30mph _once the brakes are applied_
Bear in mind also that human reaction times plus changing pedals means that the car will probably travel 2-3 times that distance before the brake pedal is even touched.
Also: Humans usually just slam on the brakes, rather than trying to steer around an obstacle if it "suddenly" looms in front of them.
A robot car reacts faster to obstacles and it's always paying attention, whereas humans get complacent. A robot might not outdrive a very good driver who's paying full attention to the road but most drivers are merely "average" and even the best drivers get distracted. What that means is that the car will notice the pedestrians and react to their movements _BEFORE_ they've even stepped into the roadway. Drivers are notorious for ignoring pedestrians and/or assuming they won't do stupid things. A machine doesn't have that luxury.
This is all algorthmic. nothing to do with "ethics". In the absence of Asimov's "positronic brains", robots are dumb machines which simply do what's programmed and the safest programming action is to slow down if there are multiple obstacles or stop entirely - any other action will leave the designers susceptable to litigation.
Humans are impatient and will try to push through regardless of dangers, which is the second-highest cause of crashes (The first is inattention and the 3rd is driving beyond your abilities. True "accidents" - as in unforseen events - are fairly rare and most crashes can be predicted by observation of the driver in the preceeding period. I really am surprised that insurance companies/police don't use automotive black boxes more when investigating). As one of my friends says "Crashes usually happen because cars know the laws of physics far better than most drivers".
As an example of the kind of thing a robot won't do: If you're on a band and the car coming the other way loses control, crossing the road in front of you, the "average driver" is highly likly to steer _into_ the crash whilst attempting to avoid, instead of aiming for where the oncoming car won't be - without specific training and practice, people tend to steer to avoid where the obstacle is now instead of predicting its path. A robot programmed with avoidance manouevres will speed up or slow down enough to miss the crash.