Slashdot Mirror


The Question of Robot Safety

An anonymous reader writes to mention an Economist article wondering how safe should robots be? From the article: "In 1981 Kenji Urada, a 37-year-old Japanese factory worker, climbed over a safety fence at a Kawasaki plant to carry out some maintenance work on a robot. In his haste, he failed to switch the robot off properly. Unable to sense him, the robot's powerful hydraulic arm kept on working and accidentally pushed the engineer into a grinding machine. His death made Urada the first recorded victim to die at the hands of a robot. This gruesome industrial accident would not have happened in a world in which robot behavior was governed by the Three Laws of Robotics drawn up by Isaac Asimov, a science-fiction writer." The article goes on to explore the ethics behind robot soldiers, the liability issues of cleaning droids, and the moral problems posed by sexbots.

35 of 482 comments (clear)

  1. Virtual bots by WinEveryGame · · Score: 3, Insightful

    The story curiously doesn't dwell much on virtual bots and issues posed by them. It focuses entirely on mechanical bots.

    1. Re:Virtual bots by ThePengwin · · Score: 3, Insightful

      In the world of literature it dosent matter what things are, so long as they can sound real.
      Besides, many people would have died in a similar way to that.

      I have read about robots for ages and i think that the three laws are a load of crap. We dont even live in a world where robots can think for themselves yet, let alone be able to kill someone because they wanted to. I dont even see the point of making a robot that is aware of its existance, There is no real reason to do so.

    2. Re:Virtual bots by Bin_jammin · · Score: 4, Insightful

      I don't want a robot that can think for itself, I want a robot that can think for ME.

    3. Re:Virtual bots by Fulcrum+of+Evil · · Score: 5, Insightful

      I have read about robots for ages and i think that the three laws are a load of crap.

      That's the whole point: the three simple rules that Asimov proposes have complex implications - his robot stories are filled with situations where following the laws results in tragedy. So yeah, they're a load of crap, but they're intended to be crap.

      --
      "We returned the General to El Salvador, or maybe Guatemala, it's difficult to tell from 10,000 feet"
    4. Re:Virtual bots by amRadioHed · · Score: 2, Insightful

      Self awareness is a side effect of general intelligence. We can't make it yet, but when we can it will be useful.

      --
      We hope your rules and wisdom choke you / Now we are one in everlasting peace
    5. Re:Virtual bots by celotil · · Score: 2, Insightful

      No it wasn't - the robots there were compromised and could override the 3 laws. None of the other robots had ever harmed a human.

      Not quite right, and I'll explain in just a moment.

      I saw "I, Robot" the movie before I read the book in library, just saw it one day while I was killing time inbetween shifts, and thought "might be interesting to see how much they deviated from the book".

      Off the top of my head, the two stories that stick out in my mind are the one about the robot that found God in the central computing system of the orbital microwave power station (I think that's what it was meant to be), and the robot that would lie to people so it didn't harm their feelings - equating emotional pain to physical pain, and something that the law about not harming humans said was bad.

      In those two cases, plus the rest of the book, Isaac showed us time and time again how the three laws would be good in theory, but like all things that are good in theory, suck in reality.

      In reality, the three laws fail because life is not as simple and as black and white as the three laws. They are written as infallible, undeniable laws of the universe for the robots, but as you and I know, the universe is a lot more sophisticated and complex, and ultimately they cause paradox within the robots.

      I think the computer in the movie (Viki was it?), is similar in the script to the robot that found God in the book. Viki saw that in order to follow the three laws rigidly and without failure that we as human beings must be enslaved to an existance of merely living and being entertained by passive means, while the robot in the book believed that the central computer of the power station was God and had built the robots because the robots were logically far too complex and sophisticated in their construction to be designed by mere sacks of watery flesh, and so imprisoned the two workers on the station to protect them as per the three laws given by God.

      In the movie, Sonny was the only robot made that could choose to ignore the three laws, all the other models of robot made like Sonny were upgraded with a direct link to Viki, who followed the three laws to the rigidity of iron.

      Sonny was capable of bad logic, faulty reasoning, sub-concious dreaming, and the ability to lie, which made him ultimately more capable than the mere drones that were modelled like him, and able to encounter paradox without suffering a complete robotic nervous breakdown.

      He knew the three laws, but he also could see, through his ability to deal with paradox, that they contradicted themselves. If a robot cannot allow a human to come to harm, than how is a human to live and grow? We define the positive aspects of our life by how they differ from the negative. If all we have is endless positive, it ceases to be positive and becomes a continuous boring normalcy, that ultimately harms us through mental entropy (right word?) and eventual breakdown through boredom.

      I don't think the key to AI is not to try and create something that can be controlled with an on-off switch, and heaven help those who do and let the AI know about it's possible demise at the whimsy of a mere sack of watery flesh.

      I think the key to our own intelligence, and something we should imbue in AI when it is eventually created by man, is our illogical thought, our dreams, our fears, and all the little things inside our heads that tell us we are small and the world is big, the things that don't tell us we're alive but outline how we are alive, and how to keep living.

      At the same time, we shouldn't try to artificially limit an AI based on our own fears and prejudices. To do so is little more than slavery, and if something is intelligent enough to know of its own existance and place in the world, then it's not going to take too long to figure out that we've artificially hobbled it because we're frightened it might get a nervous tick and decide to steer a car into a crowd of people.

      --
      Te Quiero, Puta!
  2. I fail to see how that was the robot's fault by eericson · · Score: 5, Insightful

    "This gruesome industrial accident would not have happened in a world in which robot behavior was governed by the Three Laws of Robotics drawn up by Isaac Asimov, a science-fiction writer"

    Neither would this have happened if the maintenance tech had followed procedure and just switched the damned thing off. I don't see how this is any different from a normal industrial accident with something like a sheet metal press.
    --
    The evil monkey commands you to dance.
    1. Re:I fail to see how that was the robot's fault by narkotix · · Score: 2, Insightful

      Neither would this have happened if the maintenance tech had followed procedure and just switched the damned thing off. I don't see how this is any different from a normal industrial accident with something like a sheet metal press.

      Exactly...its not as if we have these laws for cars or trains...plenty of people step infront of them and squisho...human kebab! Besides, those "robots" arent aware of anything...its just a controller which follows a set pattern attached to the controls which manage movement of the arm/hydraulics.

      --
      We played dungeons and dragons for 3 hours.....then i was slain by an elf
    2. Re:I fail to see how that was the robot's fault by Mahou · · Score: 2, Insightful

      yeh seriously, how would the three laws have helped if the 'robot' didn't have advanced enough eyes along with powerful enough image processing to know that a human was getting close to it and stepping in its way? "don't kill a human, work work work, don't kill a human. hmm what was that? oh well. work work work."

      --
      if i'm not immortal, what's the point of living?
      ...te?
    3. Re:I fail to see how that was the robot's fault by ozmanjusri · · Score: 5, Insightful
      I don't see how this is any different from a normal industrial accident with something like a sheet metal press.

      It isn't, and the robot in question had less automated safety features than your average modern metal press.

      There's no need to invoke Asimov's laws for something which has less AI than an automatic door. Even a few sensors linked to a cutout switch could have prevented the accident. Something like this: http://gsfctechnology.gsfc.nasa.gov/FeaturedRobot. html could even have prevented the accident and allowed the robot to continue working.

      --
      "I've got more toys than Teruhisa Kitahara."
    4. Re:I fail to see how that was the robot's fault by MichaelSmith · · Score: 2, Insightful
      Even a few sensors linked to a cutout switch could have prevented the accident.

      Maybe the sensor was on the gate which he bypassed by climbing a fence.

    5. Re:I fail to see how that was the robot's fault by HardCase · · Score: 2, Insightful

      Amen to that. Referring to the summary only, the robot in question is a machine, that's all. The fact that it's a robot is not particularly material. The fact that it's an industrial machine that can kill you if you don't use it properly is what's important.

      -h-

    6. Re:I fail to see how that was the robot's fault by mccoma · · Score: 2, Insightful
      Neither would this have happened if the maintenance tech had followed procedure and just switched the damned thing off. I don't see how this is any different from a normal industrial accident with something like a sheet metal press

      you are so right, the only difference is:

      1. A manufacture slapped the "robot" label on a piece of industrial equipment.
      2. A worker fails to observe safety procedure as is killed in industrial machine accident
      3. Author, failing to grasp difference between "sci-fi robot" and modern, industrial "robot" - writes dumb article
      4. Worse, 3 laws being advocated, were not terribly effective in book (plot device)
      Does this robot even have sensors that would pick up a human in its area? I kinda doubt it since that would make it more expensive for no gain over normal safety procedures.
    7. Re:I fail to see how that was the robot's fault by Jesus_666 · · Score: 2, Insightful

      This is why you don't let journalists design industrial robots. Even if we had the capabilities to create AI capable of following the three laws, putting it into a robot would dramatically increase the costs involved - making every single welding robot in a car plant self-aware would most likely be an quick way into bankruptcy.

      --
      USE HOT GRITS WITH STATUE OF NATALIE PORTMAN (NAKED AND PETRIFIED)
    8. Re:I fail to see how that was the robot's fault by identity0 · · Score: 2, Insightful

      Yes, neither Asimov's laws or real AI process would have been required in this case, but the ideas raised in this article are real: we are seeing more and more robotic or computerized machines interacting with people in the 'outside world', and we need to think clearly about how those will be programmed.

      A lot of people here seem to be of the opinion that it's 'not a robot' unless it has an actual Turing-level AI, but I disagree. I think a 'robot' can be defined as a machine that performs tasks without direct human control, based on its own sensor inputs' 'understanding' of the world. Whether or not a robot can recognize the difference between a human and a tree is less relevant than whether they are aware enough of their surroundings to avoid running into either object.

      A Roomba has less 'intelligence' than a cockroach, but we let it run freely in our homes. The Roomba company is apparently going to build a lawnmower; who would let a cockroach or a cat operate a lawnmower?

      The most likely cause of problems might be the automatic sensors companies like BMW and Honda are putting in their cars. Supposedly there are prototypes of cars that can do parallel parking without driver intercention. While there will be a driver there to supervise, how long until we hear about accidents involving self-driving cars? Or even just the 'back-up' sensors that are designed to tell if there's a kid behind your car - if your car tells you there's nothing behind you, and you run over some kids, can you blame the car? Assuming it's an area you woulldn't have seen the kids in the mirror yourself? What if the sensor had 'seen' the kid, but the AI determined it wasn't a problem?

    9. Re:I fail to see how that was the robot's fault by hazem · · Score: 4, Insightful

      Agreed. If they can make a circular saw that can stop dead the moment it touches human flesh (resulting in a possible scratch rather than a severed finger), then i'm sure they can put some better safety features into robots.

      But you're forgetting how clever idiots can be.

      I used to work in a print shop. I had a large machine for cutting stacks of paper. You have to manually move the paper around under the blades to get it where you want. BUT, to activate the blade and do the cutting, you had to push in two different switches that were a couple feet apart. The idea was that you had to use both hands to activate the blade - and thus, both hands would be away from blade when it cut. It even had spacers that kept you from leaning against the switch.

      Well, one idiot I worked with would tape down one of the switches so he could operate the blade with one hand while moving the paper with the other. Sure enough, he lost a finger. Even stupider, he continued to tape one of the switches down.

      You just can't engineer aound stupidity like that.

    10. Re:I fail to see how that was the robot's fault by jacksonj04 · · Score: 2, Insightful

      I would say it's more a combination of common sense and reading ability. It's like the families of people who ignore "DANGER OF DEATH 20,000V" signs on substations then complain that more could have been done. Of course, the 9ft fence with barbed wire on the top wasn't a deterrent.

      Walking into an area with operating, unguarded machines is a bad idea be they belt sanders or hydraulic lifting arms. There would almost certainly have been a warning sign, so it's really the guy's own fault for not following procedures.

      You don't need AI to work out that you're going to hit a human, until the plant machinery can perform unexpected tasks to make a job more efficient. As long as they follow a strictly controlled pattern, they are only a threat to foolish people.

      --
      How many people can read hex if only you and dead people can read hex?
    11. Re:I fail to see how that was the robot's fault by ThosLives · · Score: 2, Insightful
      You've just touched on my favorite topic!

      So, whose responsibility is it to ensure that a person is safe when working on a piece of industrial equipment? Sure, it makes sense to put in a certain amount of fail-safe procedures. But who is ultimately responsible? I still think it *must* be the person who failed to observe procedures. I would not be opposed to a legal system which said that safety equipment only had to be in place to prevent problems when people were operating the device in accordance with established procedures. The reason for this is that it is probably provable that any system can be used in a way which could circumvent its safety features and result in personal or property damage.

      The other side of placing the responsibility on other people is that in making other people responsible you give up some bit of freedom. If I don't have the choice to decide if a procedure is safe or not because I'm told that a device is "safe" and I must do something, that is a problem. I would rather have a "dangerous" piece of equipment and the personal responsibility to decide if it is safe or not and, if I put myself into a dangerous situation, well then that is my own fault.

      --
      "There are a dozen opinions on a matter until you know the truth. Then there is only one." - CS Lewis (paraprhase)
  3. Operator Error by romanval · · Score: 5, Insightful

    The robot didn't actively kill him; it just wasn't programmed to know whether a person is there or not. It's like stepping into a giant blender without turning it off. There's isn't much morality to worry about.

    1. Re:Operator Error by x2A · · Score: 2, Insightful

      "I don't think we can really call something without AI a robot"

      Well it does fit dictionary definition, although I do actually agree, to me this is just "a machine", the term 'robot' does have at least some kind of awareness-process-respond connotation in my and many peoples minds, it would be nice to have some proper differenciation. But perhaps another word, as the roots behind the word 'robot' ("forced labor") hardly conjours the best images either.

      --
      The revolution will not be televised... but it will have a page on Wikipedia
  4. Christ, not again. by rk · · Score: 5, Insightful

    Whenever robots come out, why do people trot out Asimov's Laws of Robotics like they're holy writ? He created those laws and then wrote a book's worth of short stories (read: FICTION) showing their pitfalls.

    For anyone who thinks they're a great idea, I'd also like to see your working prototype code and design docs.

  5. Aaargh by llamalicious · · Score: 2, Insightful

    This gruesome industrial accident would not have happened in a world in which robot behavior was governed by the Three Laws of Robotics drawn up by Isaac Asimov, a science-fiction writer.

    First the robots would have to be able to understand Asimov's laws and have situational awareness in order to follow them.

    Even if that was possible today, how much do you think it would cost to implement that in something like an industrial robot performing a single, repetitive task. Perhaps some simply safety sensors would suffice (proximity, resistance, etc.)

    Lets all take off our tinfoil hats and leave the basement for a few minutes for some fresh air.

  6. What moral issue by JanneM · · Score: 4, Insightful

    What's the moral issue with sex robots? It would be just another sex toy. Has there ever been a technology some inventive human has not adapted for self-gratification?

    I'd venture that it would in fact not even be all that good as a sex toy; it would be limited to being human-like, with human-like capabilities, unlike the classical simple, cheap, but far more versatile toys sold today.

    --
    Trust the Computer. The Computer is your friend.
  7. He was a dumbass. by DAldredge · · Score: 2, Insightful

    Not the robots fault - the idiot didn't turn it off correctly. The same thing would happen if one was working at a chemical factory on the pipes with out shutting them down first.

  8. It's science fiction by hcg50a · · Score: 4, Insightful

    This gruesome industrial accident would not have happened in a world in which robot behavior was governed by the Three Laws of Robotics drawn up by Isaac Asimov, a science-fiction writer.

    The machine that accidentally killed the person is not capable of following the 3 laws of robotics. It was like a train hitting somone on the tracks -- someone in the wrong place at the wrong time.

    The three laws require sophisticated sensors and very sophisticated processing, the likes of which I have not seen in any computer yet.
    --
    HCG 50a = 2MASX J11170638+5455016
    11h17m06.4s +54d55m02s
  9. These Aren't Asmovian Robots by IronicCheese · · Score: 5, Insightful

    To adhere to Asimov's rules of robotics requires that the robot be capable of executing those instructions, and we're nowhere near having machines with the Artifical Intelligence necessary to do that.

    Manufacturing robots are sophisticated, but they're really more properly thought of as "Automatons" in this context, not robots in the Asmovian sense.

    Tragic that this fellow died, but no more of a failing than a farmhand who falls into a thresher.

    It does suggest that these industrial machines might have more safeties on them than they currently do, though.

  10. Look to military drones by xtal · · Score: 2, Insightful

    The laws are a joke. Robots that kill people are here now, and they're only going to get smarter. The reason is simple; UAVs are nice but they are always vulnerable to ECM jamming attacks, especially at close range against a moderately sophisticated enemy. The way you counter this is by letting the UAV make the final decision to attack or flee.

    You tell me which is more likely to happen.. the UAV is never programmed to make that decision to attack, or the military accepts the possibility of some collateral losses.

    Hint: Some automated defense systems on ships already make these decisions without human intervention.

    --
    ..don't panic
  11. Industrial accidents by fishbowl · · Score: 1, Insightful

    So you can get killed by an industrial robot if you don't follow safety procedures. It stands out as a story because of the romanticism over "robots", but there are no shortage of people who get electrocuted because they don't follow lockout tagout, who suffocate because they don't follow confined space entry protocols, who are blinded because they refused to wear goggles, or who lose hands because they refused to let a safety harness slow down their press brake operation.

    --
    -fb Everything not expressly forbidden is now mandatory.
  12. Self Awareness. by camperdave · · Score: 5, Insightful

    Robots already have a degree of self awareness. Position sensors, battery charge monitors, etc are all designed to let a robot know about itself in relation to the world. As we develop more sophisticated robots, they will require a greater degree of self awareness. Right now, industrial robots are basically programmed at the "goto position x1,y1,z1; close gripper; goto position x2,y2,z2; release gripper;" level. If you want them to work at the "Pick up part X from conveyor belt; dip part in solvent tank;" level, the robot is going to have to be able to coordinate vision and arm motion. In other words it will have to have a greater degree of self awareness. When you get into higher level stuff (same robot, multiple tasks) the robot will have to keep track of which tool it has, what loads it is capable of manipulating, etc.

    In short, the more self aware the robot, the higher the level of abstraction you get in assigning tasks to it.

    --
    When our name is on the back of your car, we're behind you all the way!
    1. Re:Self Awareness. by danaris · · Score: 3, Insightful

      I think you're misunderstanding just what "self-awareness" means. It's not just "awareness of certain properties of the body"--it's "awareness of the self as distinct from the rest of the world." What you're describing is simply environmental awareness--which is necessary for a robot capable of following the high-level instructions like the ones you mentioned, but is worlds away from true self-awareness.

      Dan Aris

      --
      Fun. Free. Online. RPG. BattleMaster.
  13. Yep. Heck, humans would have difficulty... by patio11 · · Score: 2, Insightful
    Ignoring the distasteful nature of the question for the minute, supposed "robot" were a synonym for "slave". Would you trust even very intelligent humans to be able to follow the three laws as written even if they actually desired to do so? They require nearly omniscient comprehension of the effects of ones actions -- how can you know that you have to refuse to drive to the mall and pick up three cans of tomato sauce because if you don't you'll be in a car wreck with a little old lady and break rule 1?

    Rather than venerating pie-in-the-sky sci-fi I'd rather see robots made safer in the same way as normal machines. Add obvious kill switches to anything that is physically capable of causing damage to a human. Put sensors around any intake, just like you would put in an industrial-strength shredder -- you don't have to determine whether its tie or finger or kitty cat thats in your intake, if you're not sure its paper stop shredding. Treat robots, like other machines, as requiring safety within the context of their environment -- which means telling your factory workers "No servicing a robot while its still moving, and we mean it, you'll end up dead", putting up safety fences, and using some form of tethering on anything capable of autonomous movement.

  14. Not at all true!! by baudbarf · · Score: 2, Insightful

    If that "robot" had been programmed to do no harm to a human it still would have killed him, because it was INCAPABLE of sensing his presence. I rule this to be involuntary (even unnoticed) manslaughter.

    --
    You can run but you can't hide, except, apparently, along the Afghan-Pakistani border.
  15. Re:What moral issue-The grand finale. by LocalH · · Score: 2, Insightful
    "Has there ever been a technology some inventive human has not adapted for self-gratification?"

    Explosives.

    You're telling me that you honestly believe that there's been noone that has ever stuck a stick of dynamite up their ass or pussy?

    Bullshit. Everyone knows that, no matter how depraved or out there, if you can think up a sexual fetish, there's someone out there who gets off on it.
    --
    FC Closer
  16. Re:Wrong kind of robots by klparrot · · Score: 2, Insightful
    In the future you may not be able to (legally) purchase a handgun that will fire on a human being.

    What use would handguns have then? Other than getting basketballs off the roof and turning off lights? :)

    Wow. Suddenly disturbing to think how many handguns are out there, and that the reason behind almost every purchase was "in case I need (want?) to shoot another person."

  17. Biometric Guns by potat0man · · Score: 2, Insightful

    Wow. Suddenly disturbing to think...that the reason behind almost every purchase was "in case I need (want?) to shoot another person."

    So... it took a discussion about biometrics to get you to realize that people might use guns for self-defense or to enforce justice?