Slashdot Mirror
The Question of Robot Safety
An anonymous reader writes to mention an Economist article wondering how safe should robots be? From the article: "In 1981 Kenji Urada, a 37-year-old Japanese factory worker, climbed over a safety fence at a Kawasaki plant to carry out some maintenance work on a robot. In his haste, he failed to switch the robot off properly. Unable to sense him, the robot's powerful hydraulic arm kept on working and accidentally pushed the engineer into a grinding machine. His death made Urada the first recorded victim to die at the hands of a robot. This gruesome industrial accident would not have happened in a world in which robot behavior was governed by the Three Laws of Robotics drawn up by Isaac Asimov, a science-fiction writer." The article goes on to explore the ethics behind robot soldiers, the liability issues of cleaning droids, and the moral problems posed by sexbots.
The story curiously doesn't dwell much on virtual bots and issues posed by them. It focuses entirely on mechanical bots.
Fear the Roomba!
1 (short ton / firkin) = 89.1432354 slugs / keg
"This gruesome industrial accident would not have happened in a world in which robot behavior was governed by the Three Laws of Robotics drawn up by Isaac Asimov, a science-fiction writer"
Neither would this have happened if the maintenance tech had followed procedure and just switched the damned thing off. I don't see how this is any different from a normal industrial accident with something like a sheet metal press.The evil monkey commands you to dance.
The robot didn't actively kill him; it just wasn't programmed to know whether a person is there or not. It's like stepping into a giant blender without turning it off. There's isn't much morality to worry about.
Whenever robots come out, why do people trot out Asimov's Laws of Robotics like they're holy writ? He created those laws and then wrote a book's worth of short stories (read: FICTION) showing their pitfalls.
For anyone who thinks they're a great idea, I'd also like to see your working prototype code and design docs.
...am for guidelines to govern the actions of our new robot overlords.
Asimov's rules were always applied to intelligent robots. No-one (to my knowledge) has ever suggested that a hammer should have a sensor to recognise if it is hitting a nail or a thumb and refuse to obey the "command" of its operator if it is targetting the latter. The purpose of Asimov's three rules was to prevent himself from falling into the trap of writing yet another Frankenstein story. That said, I believe there are some proponents of handgun biometrics that believe guns should override the commands of their operators if the operator is not authorized to use it. In the future you may not be able to (legally) purchase a handgun that will fire on a human being.
How we know is more important than what we know.
This gruesome industrial accident would not have happened in a world in which robot behavior was governed by the Three Laws of Robotics drawn up by Isaac Asimov, a science-fiction writer.
First the robots would have to be able to understand Asimov's laws and have situational awareness in order to follow them.
Even if that was possible today, how much do you think it would cost to implement that in something like an industrial robot performing a single, repetitive task. Perhaps some simply safety sensors would suffice (proximity, resistance, etc.)
Lets all take off our tinfoil hats and leave the basement for a few minutes for some fresh air.
What's the moral issue with sex robots? It would be just another sex toy. Has there ever been a technology some inventive human has not adapted for self-gratification?
I'd venture that it would in fact not even be all that good as a sex toy; it would be limited to being human-like, with human-like capabilities, unlike the classical simple, cheap, but far more versatile toys sold today.
Trust the Computer. The Computer is your friend.
Not the robots fault - the idiot didn't turn it off correctly. The same thing would happen if one was working at a chemical factory on the pipes with out shutting them down first.
The machine that accidentally killed the person is not capable of following the 3 laws of robotics. It was like a train hitting somone on the tracks -- someone in the wrong place at the wrong time.
The three laws require sophisticated sensors and very sophisticated processing, the likes of which I have not seen in any computer yet.
HCG 50a = 2MASX J11170638+5455016
11h17m06.4s +54d55m02s
Whoa, transport me back to when E.L.O.'s "Time" album came out (Yikes! 1981) and the song "Yours Truly 2095":
But I digress (before I was ever on topic)... there won't be any moral dilemma for this crowd. The first sexbots will be programmed for "No Geeks" which will only increase their allure for that very crowd. They'll be hacked to remove that restriction, and while they're at it they'll be programmed to hang out at retirement homes, PTA meetings and church services. That'll pretty much doom them to be recalled, pulled from the market, and there'll be only a few remaining examples in the Smithsonian and certain institutions of higher learning for, ummm, "research".
Remember, you read it here first.
You don't use science to show that you're right, you use science to become right.
To adhere to Asimov's rules of robotics requires that the robot be capable of executing those instructions, and we're nowhere near having machines with the Artifical Intelligence necessary to do that.
Manufacturing robots are sophisticated, but they're really more properly thought of as "Automatons" in this context, not robots in the Asmovian sense.
Tragic that this fellow died, but no more of a failing than a farmhand who falls into a thresher.
It does suggest that these industrial machines might have more safeties on them than they currently do, though.
Beware; those "best wishes" are not such a sure thing anymore. My first tinkering experience involved my Dad and I removing the cheap electric phonograph assembly that produced the voice and rigging a pushpin and a plastic cup to learn how it worked. We gutted him merely for my cursed human curiosity.
I may have inadvertently endangered the entire human species! And with atomic power, no less!
Looks like we'd better start preparing for the inevitable. and get some robot insurance.
I agree, it's not the first. Robots have in fact caused many human deaths, and we choose to ignore this "inconvenient truth" (sorry, Al).
For example, I once heard about a line of automated housekeeping assistants that killed more than one of their "beneficiaries". As I heard it, the victim would lean too close while this robot was drying the clothes; it would blindly draw the front of her blouse into the process, and then start to consume a sensitive part of her anatomy, and she would sometimes die of shock due to the excruciating pain. Over a half century ago such occurrences were commonly referred to by an idiom that signified being in really bad trouble.
They responded by making the robots more intelligent; they were built with sensors by which they might notice that something more than clothing was being compressed; and also, a method and apparatus by which the user might suggest that the First Law was being violated. But these were half measures, and even this lesson seems to have been lost on later generations of engineers.
What is the most deadly thing in America today? If you guessed cancer, AIDS or warfare, you may have guessed wrong. I'm guessing it's these confounded robot ricksha operators that tear along our highways and through our neighborhoods as though no living beings were endangered. Currently they exercise virtually no judgment and require the passengers' continual supervision and correction to avoid disaster. That industry is only now beginning to offer some half-hearted solutions, to detect and respond to obvious violations such as lane departure and excessive closing speed.
You may call me a troglodyte, but I have to ask if we have learned anything from over 150 years of robot design and usage. Personally, I blame evil robotics corporations for not allowing their engineers free rein to build more perceptiveness and gentleness into their products. Are productivity and speed all that matters?
The laws are a joke. Robots that kill people are here now, and they're only going to get smarter. The reason is simple; UAVs are nice but they are always vulnerable to ECM jamming attacks, especially at close range against a moderately sophisticated enemy. The way you counter this is by letting the UAV make the final decision to attack or flee.
You tell me which is more likely to happen.. the UAV is never programmed to make that decision to attack, or the military accepts the possibility of some collateral losses.
Hint: Some automated defense systems on ships already make these decisions without human intervention.
..don't panic
Robot safety? The guy didn't turn it off properly and got hurt. The robot was just doing it's job. It's not intelligent.
I'm a post-grad student working on a robot helicopter. It has extremely fast rotor blades and is a very real threat to humans if mishandled, so I can speak from personal experience in working on robot safety critical systems. To me, robot safety is more of the same problem faced by machine safety in general and more of the same problems faced by robots in particular.
Firstly all potentially dangerous machines require correct operation to avoid injury. No one can stop an idiot from ignoring a safety railing of a machine, automatic or robotic. To expect safety after defeating barriers and interlocks is stupid for microwave ovens and toasters, let alone high energy robotic systems. To expect robots to be safe outside of their defined operating parameters is like expecting a car to be made of sponge so no matter how much you ignore the speed limit, you can't kill anyone.
Secondly, robots seem to suffer a higher demand for intrinsic safety because of the expectation of robot cognition. The reality is, this is the place in robotics where the technology least developed. How do people possibly expect a robot to implement the three laws if the robot cannot flawlessly recognise a human as human? Furthermore, the three laws make no sense for a system that generally works far removed from humans. Putting the sensors and intelligence into a factory robot that should never encounter a human in its powered up state is just stupid. A simple barrier or laser curtain is more than adequate as an interlock, but as we've seen, that doesn't keep humans out all the time. The best the industrial roboticist can practically do is build robot systems that are reliable and stay within their work envelopes.
For mobile systems like my helicopter, it becomes more difficult since you can't control its workspace - cognition bites you in the arse once again. However, the reality of robot-human safety is that dangerous robots working around humans simply should not be autonomous without direct supervision. We are decades away from machines that are autonomously safe around humans. Software is brittle and easy to confuse no matter how well coded it is - you just can't capture all of the edge cases in the real world when you have millions of possible states. Don't imagine robot helicopters flying around people without a monkey in control - it just won't happen.
It seems to me that people need to change their idea of robots away from R2-D2 and towards reality. Treat industrial robots like an piece of industrial equipment - with respect. The same idiots who jump the fence of a robot workcell are probably the same idiots who misuse power tools and ignore safety directives. You just can't stop idiots from earning darwin awards. Seriously, it's not hard to stay outside the yellow tape.
Take your three laws and return them to science fiction, from which they came - they belong to the same realm of fantasy as FTL travel - which is to say, maybe one day but not for a long time.
Scientists point out problems, engineers fix them
altslashdot.org: The future of slashdot.
Robots already have a degree of self awareness. Position sensors, battery charge monitors, etc are all designed to let a robot know about itself in relation to the world. As we develop more sophisticated robots, they will require a greater degree of self awareness. Right now, industrial robots are basically programmed at the "goto position x1,y1,z1; close gripper; goto position x2,y2,z2; release gripper;" level. If you want them to work at the "Pick up part X from conveyor belt; dip part in solvent tank;" level, the robot is going to have to be able to coordinate vision and arm motion. In other words it will have to have a greater degree of self awareness. When you get into higher level stuff (same robot, multiple tasks) the robot will have to keep track of which tool it has, what loads it is capable of manipulating, etc.
In short, the more self aware the robot, the higher the level of abstraction you get in assigning tasks to it.
When our name is on the back of your car, we're behind you all the way!
Rather than venerating pie-in-the-sky sci-fi I'd rather see robots made safer in the same way as normal machines. Add obvious kill switches to anything that is physically capable of causing damage to a human. Put sensors around any intake, just like you would put in an industrial-strength shredder -- you don't have to determine whether its tie or finger or kitty cat thats in your intake, if you're not sure its paper stop shredding. Treat robots, like other machines, as requiring safety within the context of their environment -- which means telling your factory workers "No servicing a robot while its still moving, and we mean it, you'll end up dead", putting up safety fences, and using some form of tethering on anything capable of autonomous movement.
Help poke pirates in the eyepatch, arr.
You should read Asimov's robot stories and novels. It's not that simple - violating the First Law causes permanent damage to the robot's positronic brain. Simple conflicts between the laws (e.g. ordering a robot to harm a human) can cause minor psychological problems, which a robot psychologist (i.e. someone who understands the psychology of robots, not actually a robot herself) can discern by measuring the responses to certain questions. Fascinating stuff. Start with the short stories, then move on to Caves of Steel, then The Naked Sun.
$x='S24;r)>63/* h@<5+oZ)32"5cz';$me='phroggy'x$];
$x=~y+ -xz+\0-Tx+;print$_^chop$me for split'',$x;
If that "robot" had been programmed to do no harm to a human it still would have killed him, because it was INCAPABLE of sensing his presence. I rule this to be involuntary (even unnoticed) manslaughter.
You can run but you can't hide, except, apparently, along the Afghan-Pakistani border.
You're telling me that you honestly believe that there's been noone that has ever stuck a stick of dynamite up their ass or pussy?
Bullshit. Everyone knows that, no matter how depraved or out there, if you can think up a sexual fetish, there's someone out there who gets off on it.
FC Closer
As someone who works in Robotics, I find this argument quite silly. Before I say why, let me state that Japan put down laws for robots and their interaction with humans a couple of weeks ago, about the time of a conference in Indiana known as the International Conference for Developmental Learning. This conference attracts the best roboticists from the world - some of them the original brains behind the famous japanese robots. Some of them might even have helped draft those laws - in all probability, they did. But the robots and ethics talk in the conference which followed it was nothing but an (almost) empty discussion of matters not likely to crop up in less than 20 years. And none of the roboticists bothered joining in. In fact, most of the audience seemed to consist of people not directly involved in robotics. Why?
The reason no one is concerned about robots going haywire, ethics in relation to robots, and related matters is that all these machines need a huge amount of computing power to achieve even a modicum of intelligence or autonomous action. Case in point - the most intelligent robot you can think of. Leo at MIT is one candidate. Most others tend to be glorified bodies and heads pre programmed to do stuff. Leo needs the equivalent of a 25 node cluster to function properly, and is even then confined to the top of a table. Sure, its expressive. It looks like it can learn from experience. It can do various hand gestures, and movements of all sorts. Great. But the moment you disconnect it from its host computer, its nothing but a glorified toy.
Translate that computing power into something which can be carried around by such an entity - and you're looking at a level of miniaturization I don't supposed possible for another 10 years. And by then, any laws or analysis which is made of these issues are going to be outdated because there is no way such a framework is going to carry on then. Robots may have biological components, they may have human parts, humans may have bionic parts - there are endless combinations of things, most of which wouldn't be visualized today.
As for asimov's 3 laws - no roboticist in the research arena has even thought about incorporating it because they *know* that these robots can do nothing without a lot of support from humans. Coming to the incident with the japanese engineer getting killed - most people would term that as an industrial accident, not the efforts of robots to kill humans. And as for sensors and things - whoever talks about human proximity sensors that advanced existing in industrial robots - does so through a hat.
If Bill Gates had a dime for every time a Windows box crashed...oh, wait a minute - he already does.
At least according to the Rick Steves' Italy book I am reading. In describing a clock tower in the San Marco Piazza in Venice he says:
"The clock tower, a Renaissance tower built in 1496, marks the entry to the main shopping drag... From the Piazza you can see the bronze men (moors) swinging their huge clappers at the top of each hour. In the 17th century one of them knocked an ususpecting worker off the top and to his death -- probably the first ever killing by a robot."
Only 400 years earlier than this recent accident, and I think it qualifies about as well for "death by robot".
(p.s. - I just remembered what quote I have in my sig. Oddly appropriate for this story)
Computers don't make mistakes. What they do, they do on purpose.
Wow. Suddenly disturbing to think...that the reason behind almost every purchase was "in case I need (want?) to shoot another person."
So... it took a discussion about biometrics to get you to realize that people might use guns for self-defense or to enforce justice?
The next DARPA Grand Challenge requires operating in congested areas, and that's going to require serious work on robot vehicle safety. The way this is going, those things are going to be rolling through small towns in hostile territory in a few years, and they'd better not be running over little kids.
Yes, it did occur to me, and it has occurred to a lot of other people besides. I've been doing some work in a facility that uses welding robots to fabricate parts of railway rolling stock, and all of them are protected by multi-zone floor scanners which slow or stop the robot depending where you stand.
There's also an international standard, ISO 10218, Manipulating Industrial Robots - Safety, which specifies distance zones depending on the time required to stop the machine. There's a pretty good overview of how it all works here: http://www.sick.com/gus/products/product_catalogs/ industrial/en.toolboxpar.0003.file.tmp/SichereMasc hinen_en.pdf - PDF Warning - Sick is the company which supplies most of the sensors at the fabrication workshop, btw.
"I've got more toys than Teruhisa Kitahara."
none of which existed in 1981
Snowden and Manning are heroes.
Here, I'll show you... Where did I put my wrench?
It is your personal duty to fight for what is right on a daily basis. Ignoring injustice is identical to approving
I write software for industrial robots, and design work cells.
When this accident happened 25 years ago, we wouldn't have had the level of safety that is seen today.
A modern robot cell could comprise of light guards, locking guards switches, and a lock-down procedure for maintenance, perhaps even some light guards. All safety will be dual-redundant, based on hardware and not rely on software.
If you tell the system to open the guard door, you want to be damm sure that the guard switches will open and the robot will not be able to run (its also normal to put a padlock on the door, to stop anyone locking you in and pressing "start"....)
The story has nothing to do with robot intelligence, and more to do with operator training and proceedures.
The said robot could have been waiting for a sensor to detect something, and the guy jumping into the cell could have been enough to make the switch.
The average industrial robot has no more intelligence than a bit of Javascript. Sure you can make choices based on sensors/vision systems, but its still pretty dumb, but also very powerful and fast...
Jason (1st post!!)
Yes but this robot obayed the zero'th law of robotics
A robot must not harm humanity or through inaction, allow humanity to come to harm
By eliminating this fuckwit from the gene pool, the robot has truely done humanity a great service.
Anyone quoted by a reporter knows how little they understand
Don't believe what you read is the truth.
http://dictionary.reference.com/search?q=robot
robot Audio pronunciation of "robot" ( P ) Pronunciation Key (rbt, -bt)
n.
1. A mechanical device that sometimes resembles a human and is capable of performing a variety of often complex human tasks on command or by being programmed in advance.
2. A machine or device that operates automatically or by remote control.
3. A person who works mechanically without original thought, especially one who responds automatically to the commands of others.
[Czech, from robota, drudgery. See orbh- in Indo-European Roots.]robotic adj.
Word History: Robot is a word that is both a coinage by an individual person and a borrowing. It has been in English since 1923 when the Czech writer Karel apek's play R.U.R. was translated into English and presented in London and New York. R.U.R., published in 1921, is an abbreviation of Rossum's Universal Robots; robot itself comes from Czech robota, "servitude, forced labor," from rab, "slave." The Slavic root behind robota is orb-, from the Indo-European root *orbh-, referring to separation from one's group or passing out of one sphere of ownership into another. This seems to be the sense that binds together its somewhat diverse group of derivatives, which includes Greek orphanos, "orphan," Latin orbus, "orphaned," and German Erbe, "inheritance," in addition to the Slavic word for slave mentioned above. Czech robota is also similar to another German derivative of this root, namely Arbeit, "work" (its Middle High German form arabeit is even more like the Czech word). Arbeit may be descended from a word that meant "slave labor," and later generalized to just "labor."
"So there he is, risen from the dead. Like that fella, E. T." - Father Ted Crilly
One of the great problems/drivers of humanity has always been that there just aren't enough sexy people to go round. And even if you do manage to get one, fairly soon they cease to be sexy - through age/ over-familiarity.
Previous solution: the oldest profession. Works in some ways, but lots of downsides. Much attacked by moralists, e.g. in the Judeo-Christian tradition, for good reasons of wasting lives/ disease etc.
But as soon as sex robots offer the hope of a solution which avoids many of these downsides, moralists such as the appropriately-named Dr. Christensen attack them before they even exist!
I for one welcome our new robot overlords/ slaves/ whatever we feel like on a given day...
(S?)he's casually throwing together three separate fields of safety.
Industrial robotics, consumer product safety, and android (Asimovs robots are androids, not just robots) morality.
With respect to the particular incident reported, I suspect the synopsis in the article is as sloppy as the rest of the article.
Did the engineer really violate safety? Did his boss or the Japanese work ethic give him a choice? Google karoshi and guolaosi.
If an engineer violates safety procedures and gets killed, publish his experience at the next safety meeting.
Too f---ing bad. I will not cry for a guy that violates safety procedure and gets hurt. For his family, sure--it's not their fault Dad is an idiot.
And if it was karoshi, then the hazard the employee was exposed to was the work culture. Compensation for families of karoshi victims is available today (but not in 1981)
There are safety standards used to protect people from robots, and they work, but you have to follow them.
Lockout/Tagout (really lockout; nobody uses tagout anymore)
Avoidance of exposure--passive perimeter guarding (fences); active perimeter guarding (light screens, LASER fences, floor mats, etc.)
Operator load interlocks--when the operator has to load a robot, you design so that only one (operator/robot) can be in the load station at a time.
- I can give you a light screen around the robot and you can jumper it out.
- I can build you a safety fence and you can climb over it.
- I can put a roof over the safety fence (yes, it's been done!) and you'll just unbold one of the fence sections.
- I can give you a teach pendant with a deadman switch (sorry, "active motion enable device"), and you can hand it to the electrician while you ride the robot.
If you're determined to kill yourself, I can't stop you.And if you do, your recent co-workers will all grimace when we see the pictures in next week's safety meeting.
But we won't have any sympathy for you.
This gruesome industrial accident would not have happened in a world in which robot behaviour was governed by the Three Laws of Robotics drawn up by Isaac Asimov, a science-fiction writer.
That's not what the 3 laws are about. The three laws are moral values, not machine code.
They have nothing to do with protecting a person from a machine and everything to do with implementing morality in a created race of sentient beings.
If you haven't read Asimov's robot stories, you should know that most of them revolve around the unexpected consequences of the three laws and the danger of rigid legalistic interpretation of moral codes.
Finally, you gotta love this one People are going to be having sex with robots in the next five years.
Author needs to work on his verb tense. That is better handled by consumer product safety procedure, not industrial robot safety protocols.
"Reality is that which, when you stop believing in it, it doesn't go away." - Philip K. Dick
2. A machine or device that operates automatically or by remote control.
Oh I see... like a TV or a VCR?
In the free world the media isn't government run; the government is media run.