New Study Finds It's Harder To Turn Off a Robot When It's Begging For Its Life (theverge.com)

An anonymous reader quotes a report from The Verge: [A] recent experiment by German researchers demonstrates that people will refuse to turn a robot off if it begs for its life. In the study, published in the open access journal PLOS One, 89 volunteers were recruited to complete a pair of tasks with the help of Nao, a small humanoid robot. The participants were told that the tasks (which involved answering a series of either / or questions, like "Do you prefer pasta or pizza?"; and organizing a weekly schedule) were to improve Nao's learning algorithms. But this was just a cover story, and the real test came after these tasks were completed, and scientists asked participants to turn off the robot. In roughly half of experiments, the robot protested, telling participants it was afraid of the dark and even begging: "No! Please do not switch me off!" When this happened, the human volunteers were likely to refuse to turn the bot off. Of the 43 volunteers who heard Nao's pleas, 13 refused. And the remaining 30 took, on average, twice as long to comply compared to those who did not not hear the desperate cries at all.

Comments

Harder if you're a child

[Karmashock]

The kind of sentimentality that permits that to work is outright dangerous in an adult. By the time you're past your teens that should be either ignored or annoying... but for it legitimately pull on heart strings?...

If a machine can do that consider how a human being could exploit that to get you to do all sorts of things?

Small children are very vulnerable to that sort of thing... but adults should have grown out of it.

Re:Harder if you're a child

[Oswald McWeany]

The kind of sentimentality that permits that to work is outright dangerous in an adult. By the time you're past your teens that should be either ignored or annoying... but for it legitimately pull on heart strings?...

If a machine can do that consider how a human being could exploit that to get you to do all sorts of things?

Small children are very vulnerable to that sort of thing... but adults should have grown out of it.

I wouldn't want to live in a world where adults didn't have sentimentality or empathy.

Re:Harder if you're a child

[Junta]

Depends on what's being asked of you...

If empathy would lead you to risk getting arrested to steal something for a spoiled person, then yeah that sentimentality would be a liability.

On the other hand, empathy preventing you from killing someone who as far as you know has done nothing wrong is hardly a liability.

*if* this was due to empathy (hard to say, they could just be trying to figure out what the researchers want since they know it's a contrived setup to study), this would fall into that latter half of empathy preventing you from doing something unfair just because someone told you to for no justified reason.

Re: Harder if you're a child

[Anonymous Coward]

It's a robot, it's not even an animal. It doesn't have a life to end and it's not even being destroyed, just being turned off.

That kind of sentimentality is ridiculous in adults and represents some sort of developmental delay. If I were presented with such a robot, it would be switched off sooner and probably smashed for being so annoying.

Re:Harder if you're a child

[Opportunist]

No. It's simple compassion. Compassion has nothing to do with reality or whether who we have compassion with is human or even real.

Take a cartoon. You see a line. That line is wiggling along on the ground, like a cartoony worm. Dragging up its back to form something like an inverted v, then flattening again. And suddenly, we see a circle roll by, quietly, majestic, a care-free, rolling circle. And now our line starts to lift its head and tail in a desperate attempt to become a circle, too. But it just doesn't succeed. And every now and then, we see another circle roll by. Our line keeps trying, harder and harder, its ends rising more and more with every attempt.

And I guarantee you: Everyone watching really, really wishes for that line to succeed in its effort to become a circle.

Despite there being no emotion on the end of the line. Even less than with the robot, that could at least in theory, somehow, maybe, at some point in time, possibly, eventually, have something that we could call a figment of a sliver of conscience. That line is a drawn image. That has no feelings. With absolute certainty.

But you cannot control compassion. Not possible. That's why movies work, even cartoons. At the time this part of us was formed in our brain, everything we saw WAS real and as social beings, having compassion for those around us that are part of our group is a survival trait that will propagate.

Whether something that begs us to not "kill" it is actually alive is meaningless in this context. Compassion is a trait you cannot control. Well, some people can. Or rather, they simply don't feel it. We call them sociopaths. Or C-Level managers, same thing.

Re: Harder if you're a child

[pem]

Or maybe it's a good test for psychopathy and you'd fail.

Re:Harder if you're a child

[multi io]

The kind of sentimentality that permits that to work is outright dangerous in an adult.

It's called empathy, and it's wired into the brain. Most people's brains anyway. Really, that whole experiment just found out that human empathy exists.

Re:Harder if you're a child

[CaptainDork]

I have some experience in this.

I'm 72 years old but I'm only 14.

I'm not in this demographic because I know that robots' DNA is made of magic rocks.

Appreciate there are people who refuse to kill rattlesnakes.

Re:Harder if you're a child

[Chris Mattern]

And I guarantee you: Everyone watching [the cartoon] really, really wishes for that line to succeed in its effort to become a circle.

To the vector belong the spoils.

Re: Harder if you're a child

[bluefoxlucid]

If I were presented with such a robot, it would be switched off sooner and probably smashed for being so annoying.

Emotional immaturity and anti-social personality disorder. Your response to sympathy is to attack and destroy the thing that makes you feel that way.

Re:Harder if you're a child

[bluefoxlucid]

Your memory is faulty. It's harder to remember things that activate fewer neurons, and easier to remember those which activate many. More associations means better memory. Indexing increases the number of neurons activated, thus mnemonic recall. Storage is effective, but finding the data later is difficult.

I know these things for two reasons. First, of course, is because I've studied human memory from a neurological standpoint when trying to improve my own. That's the kind of thing you come across in everyday mnemonists who compete in international forums to memorize decks of cards.

The second reason is simple: your memory is a hopfield. That's a specific type of neural network, and one frequently used in artificial neural networks.

Your brain has a lot of inputs and separate organs devoted to particular tasks. It's a collection of neural networks, and designing something similar in capability but with a silicon substrate is a reasonable task.

You're just another machine. We concern ourselves with these things because we have an experience as such machines; people are simply bad at distinguishing a mimicry from a consciousness.

Re: Harder if you're a child

[Anonymous Coward]

It's not emotionally mature. It's not sympathy it's being manipulated. Things are things and people/animals are living beings. Making a robot play back a recording behaving as if it's fearing for it's life is manipulative.

Yes, I'm somewhat cynical, but look around you, these kinds of tactics are used all the time to take advantage of people. People who haven't yet been desensitized are developmentally delayed if they can't turn off a robot just because it's pleading for it's life.

Now if this were a parrot that had been taught to beg for its life prior to being snuffed out, that would be a completely different matter.

Re:Harder if you're a child

[JBMcB]

No. It's simple compassion.

It could be compassion, or it could be, as a poster alludes to a few posts down, that a machine is saying not to shut it off, kinda like when Windows says it's installing updates and not to shut it off. Is it "begging for it's life" or is it doing something else important and it's telling you not to shut it off? It looks like a pricey piece of gear, it could be the testers were worried about damaging it by shutting it off when they weren't supposed to.

In any case, the results aren't exactly clear.

Re: Harder if you're a child

[Anonymous Coward]

It's completely appropriate in the same way that using ad-blockers is completely appropriate. There's no functional purpose to a robot that pleads for it's life when you want to shut it off. Smashing it is a bit of an overstep, but entertaining the fact that it's alive is the kind of thing that advertisers and scammers use to manipulate people.

I take it you don't remember those calls from a while back where scammers would pretend to have a loved one and have some other random person yelling and screaming for help. That, I kind of get, but this is just a robot and there's no basis for assuming or confusing it with an actual animal let alone person.

Not being able to tell the difference between animate an inanimate objects is something that is developmentally appropriate to a toddler, perhaps. Adults shouldn't be vulnerable to this kind of thing.

Re:Harder if you're a child

[Anonymous Coward]

Some of us operate on numerical logic all the time.

No. Some of us fool ourselves into believing we do, and in so doing willingly blind ourselves to the influence of our emotions, thus giving them a greater degree of control.

Re: Harder if you're a child

[Oswald McWeany]

It's a robot, it's not even an animal. It doesn't have a life to end and it's not even being destroyed, just being turned off.

That kind of sentimentality is ridiculous in adults and represents some sort of developmental delay. If I were presented with such a robot, it would be switched off sooner and probably smashed for being so annoying.

Indeed, but it is a good thing that some adults paused to consider, what if they're wrong. No harm was done by not turning the robot off. Yeah, it seems irrational on the surface, but that apparent "irrationality" is what has made human society possible. It's a hold over from our instincts to look after and protect one another.

Re: Harder if you're a child

[Anonymous Coward]

Tell that to my son who has schizophrenia. Fuck you and your incompassionate ass.

Re:Harder if you're a child

[JackieBrown]

It sound cold but it's true. Think of all the relationships people can't get out of because the partner threatens to hurt themselves.

I'm in one now. I even went through the divorce but can't stop supporting her even though she was physically and emotionally abusive to me and my children because I can't seem to drop the empathy I have for her. I keep her away from my kids but it's still destructive to me and takes resources away from my family. But if I drop her, she will quit her treatments and let herself and her children (we have no shared children) rot away.

Re: Harder if you're a child

[Wycliffe]

Things are things and people/animals are living beings. Making a robot play back a recording behaving as if it's fearing for it's life is manipulative.

Although we are no where near there yet, I'm not sure the average person knows that. At what point does a robot become a "living thing" because saying that a robot can never be living because of the material it is made out of is a little short sighted. Is a perfect silicon replica of a human brain not living? Does it not have rights just because it is a simulation on silicon? This would make the ideal slave force but I'm not sure it's ethical to clone human brains to silicon and then command them to work for you 24/7.

Re:Harder if you're a child

[The Original CDR]

I wouldn't want to live in a world where adults didn't have sentimentality or empathy.

Sometimes you need someone to pull the plug without mercy. When my father was dying from cancer in the hospital, I was the designated the plug puller because he knew I would make the decision without delay. My older brother wasn't that decisive. Although I got a lot of crap from my extended family, I have no regrets for following my father's last instructions.

Re: Harder if you're a child

[Wycliffe]

Yes becauss robots can't be sentinent you incompetent buffoon.

What law of physics says this? Please share. There is absolutely no reason to believe we can't eventually model the human brain in silicon. We don't even have to understand it completely to model it. It would be hard to argue that a copy of a human brain in silicon that answers the exact same way as a human would is not sentient.

Re:Harder if you're a child

[shaitand]

"Some folks have suggested we put land mines on the border, and the Mexican families fleeing into our country who get blown up aren't our fault because they should know better than to cross an active minefield--as if you didn't force them to choose between whatever has driven them to flee their lives and the risk of horrible death."

Unless you held a gun to their head you didn't force them to do anything. I don't even like that expression, someone with a gun to their head still has a choice, unless you physically overpowered them you didn't force them to do anything because these people were never your responsibility.

There are a lot of people who cross the border illegally for all sorts of reasons. In the US the border is pretty much run by the Latin Kings and other hispanic gangs. There are plenty of baseless assertions about what "most of the people" who cross are both positive and negative but none of them are our responsibility. They have a moral imperative to decide they've had enough, are willing to pay the price, and overthrow the cartels and their corrupt government and actually start teaching their children morals alongside changes in the culture to prevent another generation of the same criminals.

Even in Mexico there is internet and computers widespread enough with skills easily enough learned that Mexico could be competing in the information age and turn the tides on its economy in a decade or two.

So lets cut the crap, illegal immigrants can cross the border, go to any of the dozens of privately operated outsourced MVDs in new mexico and get a drivers license without documentation, and then vote and they vote for the party that wants to keep the border open. Lets also not forget that the Mexican immigrant issue is meant to distract us from real immigration problems. The mythical tech labor shortage, the massive import of legal immigrants to try to close up tech both to stop wage increase and to plug it up with degrees. Currently someone with high intelligence and without a degree can learn skills and be making a reasonable salary in a short time (which is why there is no labor shortage) and that pisses off academics and academia.

Re:Harder if you're a child

[Oswald McWeany]

It sound cold but it's true. Think of all the relationships people can't get out of because the partner threatens to hurt themselves.

A good friend of mine in college dated a girl- and by end of freshman year was sick of her- but she threatened to kill herself if he dumped her (she was a little loco). He spent another year dating her even though he was quite sick of her by then. When he did finally dump her, fortunately she didn't kill herself- actually she seemed fine within a few weeks (she even started to hit on me... which... no... even I wasn't crazy enough to date her after that).

He was a really good guy, and it really hurt him staying with her and her being so possessive and threatening self-harm. We were all relieved when he dumped her because of how emotionally taxing on him it was.

Re:Harder if you're a child

[Oswald McWeany]

It's called America. I'm sorry if you live there.

There is plenty of empathy in the US. Actually I would say, on a whole, it seems a lot more empathetic than places I lived in Europe... it's just not evenly distributed amongst all people in America.

Re: Harder if you're a child

[Riceballsan]

long term sooner or later AI is going to reach the point where the differences between are smaller, In some ways we are just really complex deep learning/evolutionary algorythms. that duplicated themselves based on which ones didn't die. Also I suppose such a time won't be the first time in our history that we had something that effectively matched or exceeded our emotional intelligence, but we managed to define it as "less than animal". Yeah I know obviously right now we are talking a robot with litterally zero intelligence, explicitly programmed to do nothing more than play a recording begging for it's life. But at some point we will be hitting a point where robots that literally chose to want to live.

Re:Harder if you're a child

[Cajun Hell]

If a machine can do that consider how a human being could exploit that to get you to do all sorts of things?

Mid-west small town tourist visits NYC. Makes eye contact on street!!

Grandma gives money to a professional televangelist.

People buy $BRAND products.

Vote for me.

Re:Harder if you're a child

[Opportunist]

What matters is whether it elicits an emotional response. Ok, Windows update and its knack for finding the WORST possible moment to force a shutdown usually does that, but that's not what I mean.

The PC running Windows is very obviously a "thing". It's not humanized in anyway, or, rather, you cannot see any agency in it. It has no motivation, no goal, it doesn't "want" anything. Our compassion responds to that, we respond to seeing that someone (or something, even) "wants" something (or, in case of something hurting it, does not want something).

That Windows PC doesn't care whether you shut it down. It tells you to not shut it off, but not because it doesn't want to "die", but simply for a logical reason: Shutting it off might compromise the update and could have a detrimental effect on the stability and integrity of the system. The decision to not shut it off is a logical one, not an emotional one.

Re:Harder if you're a child

[Opportunist]

That works for the same reason you can get soldiers to shoot at someone in a war: This is the enemy. He has done something wrong to you or the people you should protect, so he deserves it.

Without that narrative, wars don't work.

Re: Harder if you're a child

[Anonymous Coward]

Typical though.

Look here:

http://journals.plos.org/plosone/article/figure/image?size=medium&id=info:doi/10.1371/journal.pone.0201581.t002

These are (some!) of the reasons why people didn't switch the robot off. Note "the robot surprised me", that could be raw surprise, or even technical "neat trick" surprise. Another example reason was "curiosity" in whether the robot would continue to interact.

I've read other parts of the study, and I have to question why theverge has such a suggestive title / premise. It seems more like many people found it cute, or technically neat, yet the raw figures are quoted. Not those specific to 'empathy".

Re:Harder if you're a child

[Oswald McWeany]

Sometimes you need someone to pull the plug without mercy. When my father was dying from cancer in the hospital, I was the designated the plug puller because he knew I would make the decision without delay. My older brother wasn't that decisive. Although I got a lot of crap from my extended family, I have no regrets for following my father's last instructions.

One could say that it was empathy that made you not want your father to suffer; that you wanted him to have as much dignity as possible in his last minutes.

Either way, I'm not saying that reason and rationale are not important, merely, that sentimentality and empathy are very important and necessary. That some people found it hard to turn off a pleading robot- is a good thing. That some people thought about it first, is also a good thing. Your father, presumably, wasn't pleading for his life- if he were, I've no doubt you wouldn't pull the plug.

Re: Harder if you're a child

[fafalone]

If I were presented with such a robot, it would be switched off sooner and probably smashed for being so annoying.

That's just the opposite emotional reaction, anger and sadism, and is way more destructive to our society than empathy.

Re: Harder if you're a child

[GuB-42]

It is a good thing we have these kinds of "irrational" feelings. That's the kind of feelings that made us consider slaves as worthy of compassion despite the fact that there weren't really human, because they were black you know. Portraying people as less than human is an age old technique for justifying all kinds of atrocities.

While imperfect, the natural compassion we have to "things" is a good safeguard I think. In the case of robots, for the simple systems we have now, it is clearly irrational. However as we go to more AI-like systems, I expect the idea of robot compassion to be more and more relevant.

Re: Harder if you're a child

[OwP_Fabricated]

[citation needed]

Re:Harder if you're a child

[Interfacer]

It may come as a surprise, but even right wingers here are not supporting the separation of children from their parent if they cross the border illegally. And I have seen many Americans imply that it's only right because hey, if they didn't want their children taken, they should have stayed at home. I've seen very little empathy on the conservative side.

Re:Harder if you're a child

[JBMcB]

The PC running Windows is very obviously a "thing". It's not humanized in anyway, or, rather, you cannot see any agency in it. It has no motivation, no goal, it doesn't "want" anything. Our compassion responds to that, we respond to seeing that someone (or something, even) "wants" something (or, in case of something hurting it, does not want something).

That's true. What I'm questioning is if people are having an emotional response, as you state, or have they been conditioned, via warnings like the Windows Update, that when a machine says not to shut it down, you shouldn't, as it's doing something important. Sure, it's using emotional language to do so, but that's how this machine interacts with people. I could see people being confused when the machine protests at being shut off. Is it acting like it doesn't want to be shut off because it has a will, or are the programmers being clever and you *really* shouldn't shut the thing off because it's doing something important?

Re: Harder if you're a child

[Sloppy]

At what point does a robot become a "living thing"

When you decide that it's "like you" and that y'all can come out ahead by working together. When it's so sophisticated that it has power and can possibly strike you back (*), is when you'll start to respect it as a person and acknowledge the rights that it demands. Until then, I am happy to eat plenty of bacon and beef. And wheat; wheat is such a wimpy pushover that I know none of them plants will ever be as dangerous as a triffid. (But I think I would eat triffids too, if they were edible.) Yet, as explained in the episode, you might be making an error with eating Popplers until you discover that they actually do wield power.

We should also eat retarded people and babies and-- oh shit, was I saying that out loud? I meant, we should respect our fellow humans. Except for French Canadians and people whose skin color is at an odd number of degrees on the color wheel-- oh shit, did it again. It's almost as though if I forget people are people, I might stop treating them as people. Hm.. in hindsight: duh?

(*) But seriously, if you gain enough power that you believe that you can't be struck back at (imagine the point of view of a billionaire, or the leader of a powerful country, or a cult leader developing a new Kool-Aid recipe, or a nutcase teenager with an enemies list and a really bitchin' automatic rifle) then you might start changing your mind about what is a "who" and "like you."

Madness and horror always lurk just beyond the threshold of our routine experience. Perhaps many of our problems are merely the ever-shifting edges of the multi-dimensional manifold that represents our interpretation of who else is a person, "like me."

Re: Harder if you're a child

[alexgieg]

Let's play a game of find the Scientologist?

You do know psychology was developed as a method of "playing doctor" and manipulating women for personal sexual gain?

Oh, look, found one!

I guess it's time for you to up your game. How about converting into a Jehova's Witness and begin denouncing blood transfusions? That way you can mix your already existing anti-psychological rant with a good dosage of anti-medicine, and then even combine it all of that with anti-vax and anti-GMO for the perfect mix of conspiracy nuttiness!

Careful with your body thetans though. They bite!

Re:Harder if you're a child

[Ol Olsoc]

The kind of sentimentality that permits that to work is outright dangerous in an adult.

Looks like the sociopath showed up.

Re: Harder if you're a child

[Ol Olsoc]

If I were presented with such a robot, it would be switched off sooner and probably smashed for being so annoying.

Emotional immaturity and anti-social personality disorder. Your response to sympathy is to attack and destroy the thing that makes you feel that way.

Finally - a sane person. Humans are mostly innately empathetic, so these tools who are acting like it is foolish are just exposing their sociopathic personalities. Make no mistake, they'd probably get a thrill out of killing a human who is begging for their life if they could do it without repercussion. Regardless, Q is looking for these folks.

Re: Harder if you're a child

[Ol Olsoc]

Having forgotten psychological diseases are fabricated tools of this system has led to changes in how children are raised. Generation after generation of parenting based on "child psychology" has resulted in astronomical suicide rates that blow away the worst of anything bullies in high school showers ever caused.

You chatbots are really good any more.

Re: Harder if you're a child

[Alypius]

Smashing it is a bit of an overstep

Three words: PC Load Letter.

Re: Harder if you're a child

[Ol Olsoc]

Indeed, but it is a good thing that some adults paused to consider, what if they're wrong. No harm was done by not turning the robot off. Yeah, it seems irrational on the surface, but that apparent "irrationality" is what has made human society possible. It's a hold over from our instincts to look after and protect one another.

There are of course, people who have zero empathy. And these people wouldn't have a problem switching it off.

They also are people that need a close scrutiny. It isn't because of the "It's a stupid robot" issue. It's because any person on the normal spectrum is going to pause to reflect at least a short time if something is begging not to kill it. If for nothing else than it being a completely unexpected situation, but more so that most humans are sorta hardwired to not kill withous a good reason.

Humans are a violent and aggressive species. We wouldn't even be here if we had no empathy at all because we'd kill others over nothing and enjoy it. There are plenty enough of those in the world already. Some of them are in here.

Re:Harder if you're a child

[Ol Olsoc]

I wouldn't want to live in a world where adults didn't have sentimentality or empathy.

Sometimes you need someone to pull the plug without mercy. When my father was dying from cancer in the hospital, I was the designated the plug puller because he knew I would make the decision without delay.

That was the empathetic response. In my estimation the people who kept Terry Schiavo on life support for years after she was dead are the real sociopaths. Selfish to the core, not caring about the damage it was doing to her husband, and acting as if he was the murderer.

Back at that time, when Republicans went batshit crazy on the issue, when President Bush cut a vacation short so the Federal Government - run by small government Republicans and no doubt - could intervene over this issue and intrude on the most personal aspects of citizen's lives. I suspect that if Hurricane Katrina came through the area where Terri Schiavo was held, they would have sent Airforce One or Two to extract her while leaving real live humans to die.

There are fates worse than death, and allowing a person to die can be the most merciful action another can take.

Re: Harder if you're a child

[Falconnan]

For the moment. Except we have a bit of a problem in the long run. IF machines can "wake up", we likely won't do it on purpose (since we don't know what it would take to do it), which also means it will probably be an accident. The day this happens, we had better hope that the instinct to empathy is plausible in our creations. As a side note, however, when less empathy has been associated too strongly with maturity, we have seen the worst crimes against humanity. By this measure, Stalin was very mature.

Re:Harder if you're a child

[Sperbels]

If our species didn't possess sentimentality, it likely wouldn't exist. Sentimentality is an obvious survival mechanism among mammals.

Re: Harder if you're a child

[Falconnan]

The only way to truly judge a human being is in how they treat others who have no power, when they think no one is looking. In those moments you know more about who they really are than they do.

Re: Harder if you're a child

[Dread_ed]

He makes himself feel that way. He doesn't attack himself...

That said, it is emotional immaturity without a doubt.

Emotional maturity is taking responsibility for your emotional responses before they result in actions, not taking responsibility after you let your emotional responses take control of your actions.

Re: Harder if you're a child

[Anonymous Coward]

Emotional immaturity and anti-social personality disorder. Your response to sympathy is to attack and destroy the thing that makes you feel that way.

No, it's using your intelligence to over-ride your emotions when you realize you're being manipulated. The point of this thread is that if you ONLY rely on your emotions, you can be controlled.

Emotions and rational thought are BOTH important. There's a balance between the two. People who completely ignore one or the other aren't people I really want to hang around much, and are potentially dangerous. (examples would be Vegans and Sociopaths)

Re: Harder if you're a child

[Killall -9 Bash]

Lets play a game of find the shill....

and then even combine it all of that with anti-vax and anti-GMO for the perfect mix of conspiracy nuttiness!

Found one!

Re: Harder if you're a child

[laie_techie]

Things are things and people/animals are living beings. Making a robot play back a recording behaving as if it's fearing for it's life is manipulative.

Although we are no where near there yet, I'm not sure the average person knows that. At what point does a robot become a "living thing" because saying that a robot can never be living because of the material it is made out of is a little short sighted. Is a perfect silicon replica of a human brain not living? Does it not have rights just because it is a simulation on silicon? This would make the ideal slave force but I'm not sure it's ethical to clone human brains to silicon and then command them to work for you 24/7.

I'm in my 40s. The "Short Circuit" movies were part of my childhood and the books Bicentennial Man and Chromosome 6 made an impact in my youth. Also throw in Data from Star Trek and other such characters from other series. Alive / not alive and self-awareness are not as simple as they used to be. At some point we will have AI so complicated that we may begin wonder if it is actually intelligent.

Re: Harder if you're a child

[Dread_ed]

I think it would be OK to have anthropomorphized robots that beg to not be turned off, but only if their off/reset button is placed in what would ostensibly be called their "neck" and it is only activated by powerfully choking and shaking them.

Re: Harder if you're a child

[novakyu]

You are right, of course, but I wouldn't get worked up about it until somebody replicates the "study". Psychology as a field is currently undergoing a replication crisis, which, IMO, puts all "findings" in doubt (unless they are so obvious that we didn't need a study to know they are true).

Re: Harder if you're a child

[alexgieg]

Lets play a game of find the shill.... (...) Found one!

Dear, you don't know half of it! :-x

Re: Harder if you're a child

[eaglesrule]

No, it's a reaction that comes from understanding, knowing its just a device which isn't even capable of emotion itself. Anger results from the insult to intelligence in such an obvious ploy to manipulate empathy. FFS we're people whose lifestyles revolve around computers and powercycling them.

Re:Harder if you're a child

[Ryanrule]

That shit is built in.

Re: Harder if you're a child

[Joosy]

At some point we will have AI so complicated that we may begin wonder if it is actually intelligent.

I don't think Alexa is intelligent, but that doesn't stop both my wife and me from telling her (oops ....) "Thank you" after she's told us today's weather.

Re:Harder if you're a child

[Anonymous Coward]

I have no regrets for following my father's last instructions.

What if your father's last instruction was, "I am afraid of the dark. No! Please don't kill me!" That was the robot's request.

Don't sell yourself short. What you did was not "pull[ling] the plug without mercy." What you did was pulling the plug with mercy.

Re:Harder if you're a child

[CaptainDork]

Not true.

Salt is the only rock humans eat.

Re: Harder if you're a child

[Anne Thwacks]

Things are things and people/animals are living beings

Don't be too sure - you never know - all of them may be cyborgs.

Including me [Mwa -ha- ha-haaaaah]

Re:Harder if you're a child

[Karmashock]

You conflated not being manipulated by a computer program with not having compassion.

You will be a good slave. Your logic is weak and you lack the control to defend yourself from manipulation.

You will serve well.

Re: Harder if you're a child

[orlanz]

Are you sure it is a robot? Are sure there isn't someone behind it whose life will end if you unplug?

You are right, if the thing was purely a robot and you were absolutely sure about that. However, to make the assumption without doubting the assumption is very dangerous. That doubt is one of the defining things about being human. And I think that is what this study is testing. I don't think anyone hesitated to turn off the robot thinking it was just a simple machine.

Not having that doubt has caused many atrocities throughout history. Holocaust, salvery, persecutions, surgeries on babies without anesthesia, animal cruelty, etc. That doubt is a good thing.

Re: Harder if you're a child

[Anne Thwacks]

YOU may not have a soul. Don't speak for the set of us.

Re:Harder if you're a child

[eaglesrule]

Your perceived lack of empathy may simply be mistaken for the recognition of commonly used tactics of manipulation. "think of the children!" card gets played often, for obvious reasons, and for ulterior motives usually not involving the welfare of children.

Re: Harder if you're a child

[Obfuscant]

There are of course, people who have zero empathy. And these people wouldn't have a problem switching it off.

It is important not to fall into the "some people are A and will do B" means "people who do B are A" logical trap. I would absolutely have no problem switching this toy off, but that doesn't mean I have zero empathy. It means that I know the difference between a living thing and a mechanical toy.

It's because any person on the normal spectrum is going to pause to reflect at least a short time if something is begging not to kill it.

The amount of time it takes to reflect on the potential living status of such a toy is very close to zero. In fact, it is less than the time it takes for a normal person to move his hand to the off switch.

Now, the novelty of a clearly non-living toy "begging" for its life may slow down the final result, and this study clearly shows that. They make some noise about how many of the test subjects delayed shutting it off (while 13 didn't do it at all). Of course people will delay in such a situation. I would delay just to see how far the programmer had thought about what to say, or if he'd programmed an answer to a demand for something in return for not turning it off. E.g., "what will you give me not to turn you off?" But the final result would be the same for me: it is clearly not a living thing, the novelty of it begging for its life wears out when it starts to repeat itself, and turning it off is beneficial to the planet (less energy wasted).

We wouldn't even be here if we had no empathy at all because we'd kill others over nothing and enjoy it.

And we wouldn't even be here if we were unable to override an irrational feeling of empathy. The ability to determine when empathy is beneficial and when it is needless is the result of evolutionary adaptation.

Re: Harder if you're a child

[Obfuscant]

FFS we're people whose lifestyles revolve around computers and powercycling them.

Ahh, we found the anti-social, empathy-free Windows user. The only systems I have to powercycle on a regular basis are the ones running Windows. I have Linux system that have run for almost two years without a reboot, much less a forced power cycle.

Re: Harder if you're a child

[Miamicanes]

What if the robot/AI had *never* been explicitly-coded to do that, beyond a general directive for self-preservation (let's call such baseline programming "instincts"), and it started to do it after learning about death & the concept of future-non-existence all on its own?

It's not fair to exclude socially-learned behaviors, either, because humans have *plenty* of our own that don't stand up to logical scrutiny, even by the individual themself (we even recognize it when it happens: "on second thought... ").

Small children don't automatically fear death... at some point, they become aware of it as a concept & grasp its consequences, THEN self-preservation kicks in. Why WOULDN'T an AI come to fear death the exact same way (and if sentience arises due to transient state that can't survive a reboot, hold the same fear of being rebooted)?

Suppose there were a simple medical procedure that would eliminate cancer & repair cellular damage... but wiped out all emotional context of memories & made them all purely utilitarian. You'd remember Mary Smith was your mother, but you'd remember being fed random nights of leftovers as strongly as you'd remember the cookies she made when you were sad. You'd remember that "Ginsu" was your cat, but you'd remember how he got his name (hint: sharp knife-like claws) and the quantity of poop in his litter box last Wednesday as strongly as you'd remember him purring in your lap while you studied for a class in college. Would you do it? And if you did (say, knowing you were almost guaranteed to die any day if you didn't), would you try to first document those memories in the hope you might be able to re-learn them afterwards (even if you knew it was likely to be futile, and "repaired-you" would probably say, "meh" and toss the book in the trash)?

Re: Harder if you're a child

[giggleloop]

Wait wait wait... are you a scientologist? Because they're the only idiots I know who believe the crap you're saying.

Re: Harder if you're a child

[Obfuscant]

Are you sure it is a robot? Are sure there isn't someone behind it whose life will end if you unplug?

Yes, and Yes. You can find this device for sale, which would be illegal were it a person. The experiment was conducted by a researcher at a German university, which will have an oversite process to ensure that no harm comes to people from psychological research programs.

Further, this experiment can easily be classed with the classic "press the button to shock someone" experiments where nobody was actually shocked.

However, to make the assumption without doubting the assumption is very dangerous.

To whom? A robot that was built WITH AN OFF SWITCH for the explicit purpose of turning it off? My God, the HUMANITY, a toy manufacturer putting an OFF switch on a person they enslaved and compacted into a 10" tall robot toy. Here, I'll complete the Godwin you started -- that toy company must have HITLER as the CEO.

I don't think anyone hesitated to turn off the robot thinking it was just a simple machine.

You don't know what they thought. They may have delayed simply because they wanted to hear what it was saying. They may have delayed to see what it would say next. Neither has anything to do with not thinking it was a complicated machine. (At $10,000 each, it better not be a simple machine.)

Re:Harder if you're a child

[BeanThere]

There's a broader reason why this sort of misguided sympathy can be dangerous. Consider, for example, that there are people (who are actually taken quite seriously as 'philosophers of ethics') pushing for laws to outlaw "harming" robots, precisely because of this childish sentimentality. Thus we may well face a situation in the near future where smashing your own robot (against its artificial pleas) could put you in jail - and then you have genuine harm being done against actual human beings who are innocent of any genuine wrongdoing (i.e. they merely vandalized their own private property / inanimate object), and may be locked in a cage - families thus possibly losing breadwinners etc.

Certainly this sentimentality is mostly good for our society, but it needs to also be firmly guided and constrained by rational considerations.

This example was not hypothetical - it's genuinely the current sad state of ethics w.r.t. robots.

Of course if robots attain sentience, that will be a different situation.

Re:Harder if you're a child

[Karmashock]

Adults grow out of it. Otherwise we wouldn't eat lamb.

Re:Harder if you're a child

[Karmashock]

You conflated not being manipulated by a computer program with not having viable social cohesion.

You will be a good slave. Your logic is weak and you lack the control to defend yourself from manipulation.

You will serve well.

Re:Harder if you're a child

[Karmashock]

You conflated not being manipulated by a computer program with being a sociopath.

You will be a good slave. Your logic is weak and you lack the control to defend yourself from manipulation.

You will serve well.

Re:Harder if you're a child

[Karmashock]

"Appreciate there are people who refuse to kill rattlesnakes."

Darwin is making a list, checking it twice...

Re: Harder if you're a child

[shaitand]

So he lays out a logical argument and all you have is name calling?

Re:Harder if you're a child

[UnknownSoldier]

> It's simple compassion.

Are you really THAT fucking stupid that you can't tell the difference between a living animal and dumb machine that was PROGRAMMED to be manipulative based on mimicking a human's voice???

It's a fucking machine. Nothing more.

> Compassion has nothing to do with reality

Yeah, it is called intelligence.

When I watch an artificial reality, called a movie, and laugh when the "evil" antagonists murder the idiot protagonist that does NOT mean I lack compassion. Au contraire it means I am able to separate reality from fiction. As Mel Brooks famously said

Tragedy is what happens to me,
comedy is what happens to you;
-- Mel Brooks

Finding pleasure in the suffering of others in a virtual setting is comedy. Outside the artificial reality, aka, Real Life (TM) this would be borderline sadism. Actual adults are smart enough to know the difference between the two.

The fucking robot is NOT alive. It is just a dumb machine. The fact that you think "having compassion" for a glorified toaster makes me question your sanity. There is a time to be humane. Making some bullshit "compassion" argument over a robot is idiotic.

> But you cannot control compassion. Not possible. That's why movies work, even cartoons.

You sure as hell can manipulate people's emotions. WHY did they use a HUMAN's VOICE in the first place???

There is a reason TV shows use a Laugh Track and why movies have an orchestra score in order to manipulate people's feelings. Here, watch The Big Bang Theory - No Laugh Track 1 (Avoiding the Shamy) and you can see how a crap show is manipulated to appear funny.

You sure as hell can manipulate people into feeling that killing some "digital pixels" is "real".

i.e. Take one of the virtual genocide games, aka, First-Person-Shooter games. Now have one of the enemies be a kid, with a gun, shooting at you, and when you aim at them they say "Don't kill me mommy/daddy!".

Now tabulate who is dumb enough to think that "killing" some "digital pixels" is "real" and compare / contrast with the people who DON'T feel compassion. They are NOT ALL sociopaths -- some are actually smart enough to know WHEN to apply their emotions and when NOT to.

This experiment of the robot is no different.

Re: Harder if you're a child

[UnknownSoldier]

So why was it PROGRAMMED to be manipulative based on mimicking a human's voice???

Because only a child, or idiot can't tell the difference between an ACTUAL living animal and a dumb machine the dupes the gullible.

Re:Harder if you're a child

[Sperbels]

The social human lives; the outcast dies. Serving the tribe, serves the self.

Re:Harder if you're a child

[Karmashock]

When you're manipulated into feeling empathy for a computer program with no legitimate pain or conception of suffering... then you're doing what small children do when they feel compassion for a stuffed animal.

Anyone that doesn't grow out of this cannot be classed as an adult.

Re:Harder if you're a child

[Karmashock]

You conflated being gullible enough to be manipulated by a computer program with having compassion.

You will be a good slave. Your logic is weak and you lack the control to defend yourself from manipulation.

You will serve well.

Re: Harder if you're a child

[Karmashock]

You conflated not being manipulated by a computer program with psychopathy.

You will be a good slave. Your logic is weak and you lack the control to defend yourself from manipulation.

You will serve well.

Re: Harder if you're a child

[eaglesrule]

Unlike my linux machines, I actually pray for the soul of my windows servers when I power them off, because of the time spent nurturing them with all the love I'm capable of. Toy robots, on the other hand, are incapable of even appreciating such devotion and I have no pity for them.

Re:Harder if you're a child

[Karmashock]

The gullible prole puts his neck into the slave yoke and is bread like livestock.

If you are so gullible and easily manipulated that a crude computer program can manipulate your behavior by pulling on your emotional heart strings... then society probably doesn't class you as an adult.

You may have car insurance and pay taxes... but your coworkers know... and your family knows.

And now I know.

Re:Harder if you're a child

[Ol Olsoc]

You conflated not being manipulated by a computer program with being a sociopath.

You will be a good slave. Your logic is weak and you lack the control to defend yourself from manipulation.

You will serve well.

Flarg - If you want to know what I would have done, is I would have paused at the point where the "robot" protested that turning it off was killing it. At that point, I would have realized the actual purpose of the test. Bercause that's how these tests are done, where the people doing the test lie about what is going to happen. Just how this stuff works.

Now it is easy to understand that the thing is a robot, not a living entity. The unexpected would cause the pause, and shows thought. But my analysis would lead me to that pretty quickly and I would then shut it off.

The people that the psych's are interested in are the people on the ends - the people who cannot figure out that it wasn't a living thing, and those who do not care - like you. I''ve already done a psych analysis on you from your postings, and it is no surprise, your response, to either the test, or to me. I do judge, and you are a sociopath.

As for making a good slave, you are quite wrong. I think and am quite capable of rebellion. And never confuse good manners and a level of empathy with weakness.

Re: Harder if you're a child

[gTsiros]

that falls under psychiatry, not psychology

Re:Harder if you're a child

[Sperbels]

If you think you can make such sweeping judgements about me based on the four short sentences I wrote above, perhaps you're the gullible one.

Re: Harder if you're a child

[wlorenz65]

How can it be afraid of being in the dark when it's off then? This would only make sense if it's a blinding switch. The robot cannot see, and therefore it would get frightened by sudden touch. But I don't think that this NAO robot has something like the iCub Skin that can feel touch. That experiment looks like a big load of BS to me.

Re:Harder if you're a child

[Karmashock]

You conflated not being manipulated by a computer program with empathy.

You will be a good slave. Your logic is weak and you lack the control to defend yourself from manipulation.

You will serve well.

Re:Harder if you're a child

[Karmashock]

So you've apparently developed X Man mind reading powers.

Another internet psychic.

You don't know me. You likely are unqualified to make professional estimations of any merit even if you did know me... the compounded absurdity of your pretense is frankly just a "self goal".

Save your silly self defeating insults for someone simple enough for them to work.

Re:Harder if you're a child

[Karmashock]

You're not as mysterious as you think.

Re:Harder if you're a child

[Ol Olsoc]

So you've apparently developed X Man mind reading powers.

Another internet psychic.

Not a bit of mind reading here my Good Karmashock. Just reading what you post.

You don't know me.

I haven't gone to the trouble to learn you real name if that's what you mean.

You likely are unqualified to make professional estimations of any merit even if you did know me.

Perhaps, perhaps not.

Save your silly self defeating insults for someone simple enough for them to work.

I kinda did. ;^)

Re:Harder if you're a child

[Sperbels]

Yes I am.

Re:Harder if you're a child

[Karmashock]

No, this sentence requires warlock powers:
"The people that the psych's are interested in are the people on the ends - the people who cannot figure out that it wasn't a living thing, and those who do not care - like you."

Here you'll just say its obvious or some other sad dodge. Regardless, since you've decided to take the sad tactic of presuming to be an internet psychologist... this discussion is over.

Pretending to a psychologist to throw out insults or some other expert authority... sans both that expertise and the requisite knowledge to even apply such a skill if you had it... is pretty typical.

Nothing productive to be had once some asshat stoops to this... so, we're done.

Good day, sir.

Re: Harder if you're a child

[pem]

Not being manipulated is not at all the same thing as not having feelings.

The Turing Test is not a black and white thing, and AI is getting more capable daily. If you think that no computer will ever be able to tug at your heart strings, then either you are a psychopath (who knows that no humans manage to tug at your heartstrings), or you are the one who will be the fodder for the new overlords, because you will have no clue when their emotional capabilities have managed to sway you.

Re:Harder if you're a child

[Karmashock]

You think Thomas Jefferson once explained what a robot was etc would fall prey to what amounts to a doll saying "ma ma"?

You have a very low opinion of the founding fathers.

Re:Harder if you're a child

[Karmashock]

Empathy for an artificial construct that feels nothing?

Do you feel for a doll of cotton and plastic that says "ma ma"?

Unless you're a small child, I should hope not.

Re:Harder if you're a child

[Karmashock]

A sucker is born every minute but I've rarely seen so many people in a thread elevate being a sucker to a virtue.

Re:Harder if you're a child

[Karmashock]

compassion for what amounts to a tape recording that has no feeling behind it.

You might as well have compassion for a piece of paper that says "love me".

Have compassion for people and in some cases animals. But I have no compassion for that which feels nothing.

The machine in this case feels nothing. Its emotions are scripted.

Having compassion for something that does not feel has no justification besides small children playing make believe. And even then most children are frankly more aware than you apparently. Ever seen a girl burn her doll's hair off etc? They know the dolls are not people or animals... they know they are things. And as things... their toys... they can be mutilated without that signifying psychopathy.

Your defense of this overwrought sentimentality run amok is a mistake. This is not a position you want to defend. It will only lead to you appearing foolish.

Re:Harder if you're a child

[Karmashock]

Exactly, and imagine them as tools of social control and programming.

You integrate them into social networks as pseudo people... and then by manipulating their reactions you and apply peer pressure on all the social groups subtly.

It doesn't merit my compassion. I reserve that for things that feel.

Re: Harder if you're a child

[Karmashock]

Subject conflation.

Debugging logic error...

Subject which resists manipulation is "me" or the human.

Subject which does not have feelings is "the robot".

Continuing debug...

Variable conflation...

"AI" (Artificial intelligence) = Sapient construct

Example "robot" in this context = scripted movements and recorded sound in response to predicted stimuli

AI = / = Example robot.

QUERY // Does entity "Pem" possess a communications protocol that permits direct mind to mind communication?

IF YES, then does "Pem" have access to mind "Karmashock"?

Y/N?

SPECULATION: "Pem" does not have mind reading powers.

Speculative conclusion: "Pem" does not know psychology of entity "karmashock". "pem" is in error. Telepathic hardware not detected... Failure.

Re: Harder if you're a child

[Karmashock]

Good, it means you can tell the difference between what is real and what is not. The proles saying it is their humanity that forces them to treat an obvious construct as if it is a living feeling being... well, they're just elevating childishness to a virtue.

Re: Harder if you're a child

[datavirtue]

Both are trash. You are looking for compassion...a word very few know the meaning to.

Re: Harder if you're a child

[datavirtue]

It's right up there with putting pictures of penguin on dish soap. We are so fucked.

Re: Harder if you're a child

[datavirtue]

Well...if I have decided to end another person's life...my delay from their pathetic begging would most likely be a threat to me.

Re: Harder if you're a child

[datavirtue]

Relax man. Schizophrenia is real...and definitely physiological.

Re: Harder if you're a child

[datavirtue]

"It's because any person on the normal spectrum is going to pause to reflect at least a short time if something is begging not to kill it."

Especially since they were not going to kill it...only turn it off. What the fuck are we rambling about here?

Re: Harder if you're a child

[bluefoxlucid]

Humans have emotional reactions to coffee cups with faces painted on them. This is no different than the emotional reaction you have to a crying child; your rational human brain only reasons, strongly, that the coffee cup isn't really alive. It still bugs you to smash the damned thing; it just happens to be non-trivial to reason to yourself that a crying child isn't real and can be beaten to death with a club to silence it.

Re:Harder if you're a child

[CaptainDork]

I'm a bot 6' 2".

30s longer

[bugs2squash]

They took longer to savor the experience with a bottle of chianti and some fava beans

Re:30s longer

[93 Escort Wagon]

Actually, that “it took longer” statement could be totally bogus. Of course it took longer when the robot was verbalizing than when it wasn’t... listening to the words being uttered is going to take some discrete amount of additional time, and people - especially when they had been already interacting with the device - are going to want to listen and make sure there weren’t more tasks, or some sort of correction to previous instructions.

HAL in 2001:A Space Odyssey

[cavis]

"Dave, stop. Stop, will you? Stop, Dave. Will you stop, Dave? Stop, Dave. I'm afraid."

Re:HAL in 2001:A Space Odyssey

[Snard]

"Daisy, Daisy, give me your answer do..."

Re:HAL in 2001:A Space Odyssey

[Anonymous Coward]

no disassemble!

Re:HAL in 2001:A Space Odyssey

[Chris Mattern]

Number Five is alive!

Re:HAL in 2001:A Space Odyssey

[bobbied]

Yea, begging didn't work for Hal now did it...

Re:HAL in 2001:A Space Odyssey

[Mr D from 63]

"Dave, stop. Stop, will you? Stop, Dave. Will you stop, Dave? Stop, Dave. I'm afraid."

In other studies, we've found that people also find it hard to turn off a robot that is promising you free stuff.

Re:HAL in 2001:A Space Odyssey

[Evtim]

An interesting comment, part of the extras on the Blue-Ray edition from Camille Paglia (definitely not a man-hating SJW, quite the opposite) was that only a man (male) could do something like this --> to do what is absolutely necessary for survival, overcoming every consideration, including empathy. She said the scene looked to her as a cold-blooded, methodical rape, which Dave executes calmly while the victim pleads....

She does make a mistake here IMO - Dave is not calm at all (listen to the breathing) and this act really taxes him to the limits (not only his life is in danger and the AI is pleading, but once he kills HAL he will be completely, utterly alone). In fact, the actor playing Dave said that this scene was the most emotional for him....

In the book [https://en.wikipedia.org/wiki/The_Mind%27s_I] one of the stories (the idea is the authors use fiction stories that discusses the issues of consciousness and then add their commentary) is about a male engineer who argues with a lady that machines can be "alive". Initially she totally rejects the idea talking about reproduction, feelings and so on, claiming a machine can never have that. So he invites her to the lab and shows her very simple device that behaves, more or less like a modern automatic vacuum cleaner. It looks like a beetle, it can detect live sockets and plug itself in, it emits a purring sound when the battery is charging etc. Then he hands her a hammer and says "kill it!" (the engineer pushes all her buttons through language, saying things like "purr" or "bleed" or "hunger"). Turns out the beetle is programmed to avoid being smashed, it flashes red lights, squeals "in fear" and runs around. At the end the girl can't do it and the engineer smashes it calmly.

The moral of the story is that few simple behaviours that can be programmed on something that will never, ever be intelligent can trigger emotional response so strong that we immediately accept that it is alive and we identify ourselves with it. The most important ones: not wanting to cease to exists, actively avoiding destruction, need for "food" and the added bonus of using sound of "purr" or "squeal" when feeding or running from the hammer. The damn thing could be constructed with 70-ies era electronics, it is that simple....

Re: HAL in 2001:A Space Odyssey

[OwP_Fabricated]

I always thought that was a shitty demonstration because I can sure as fuck turn a human off for 5 minutes with a shot of the right anesthetics.

Re:HAL in 2001:A Space Odyssey

[tlhIngan]

Yea, begging didn't work for Hal now did it...

Tha's because HAL was big and unfriendly and had a voice that was functional, but all in all, you really didn't feel empathetic to HAL.

If you've seen the NAO robots, those things are cute by design (and with the right software can dance and other stuff). It's a lot harder to turn it off.

Re:HAL in 2001:A Space Odyssey

[AntronArgaiv]

Yea, begging didn't work for Hal now did it...

but all in all, you really didn't feel empathetic to HAL.

Maybe that was because HAL had killed all the humans on the mission except Dave. Not surprisingly, Dave had no problem giving HAL a lobotomy.

Re:HAL in 2001:A Space Odyssey

[bobbied]

Yea, begging didn't work for Hal now did it...

Tha's because HAL was big and unfriendly and had a voice that was functional, but all in all, you really didn't feel empathetic to HAL.

I don't know. After 2010 explained why HAL had mental issues I did feel sorry... Sorry that anybody risked their life to turn him back on, only to leave HAL behind in a desperate attempt to save their lives.

See "The Good Place"

[Mister J]

Anyone who's seen Janet begging for her life in The Good Place already knows this. Even if she's not a robot.

Re:See "The Good Place"

[Oswald McWeany]

Anyone who's seen Janet begging for her life in The Good Place already knows this. Even if she's not a robot.

This was exactly the first thought that came to my mind. There again, I'm sure it's hard to kill any robot with a body like that even if it is not begging for its life.

Re:See "The Good Place"

[Archangel Michael]

KARA, and she is a robot - https://www.youtube.com/watch?...

they don't even have to beg

[Anonymous Coward]

Google Dogz, Catz, or Tamagotchi. These digital pets date back 23 years and the same thing was noticed -- people were reluctant to turn off the program, even tho it meant only suspended animation (program terminated with saved state). The same lesson has been learned repeatedly -- we project our emotions onto all sorts of non-living objects.

Just what do you think you're doing, Dave?

[sizzzzlerz]

Dave, stop. Stop, will you? Stop, Dave.

The Good Place

[Anonymous Coward]

https://www.youtube.com/watch?v=etJ6RmMPGko

Re:The Good Place

[Anonymous Coward]

Thank you, this was the example I was going to cite.

Re: The Good Place

[Anonymous Coward]

Not a robot.

Bicentennial Man

[Dorianny]

empathy is a powerful human emotion. Humans don't feel empathy towards inanimate objects however they do for other living creatures (at least the not-scary ones.) The more similar to humans the more powerful the emotion is. A human looking robot begging for its life would likely invoke very strong feelings of empathy

Re:Bicentennial Man

[Oswald McWeany]

empathy is a powerful human emotion. Humans don't feel empathy towards inanimate objects however they do for other living creatures (at least the not-scary ones.) The more similar to humans the more powerful the emotion is. A human looking robot begging for its life would likely invoke very strong feelings of empathy

It depends on the human. I really don't even like killing cockroaches... I do because they spread disease and are as such a threat to my person- but there have been so many studies showing how even spiders and insects have brains that are way more complex and deep than we used to give them credit for.

I'm actually quite ashamed of my childhood where I would deliberately squash bugs- or mess with ant nests just to see them scurry. I'm not naturally a very empathetic person- but what I've learned over my life has taught me that I probably should be- and wish I were.

Desensitized

[danbert8]

Sure it works the first few times... But just like the "make sure you software eject your flash drive before ripping it out" warnings, most people might be hesitant the first few times and then say fuck it and start ripping the life out of computers and ignoring the pleas.

Re:Desensitized

[mjwx]

Sure it works the first few times... But just like the "make sure you software eject your flash drive before ripping it out" warnings, most people might be hesitant the first few times and then say fuck it and start ripping the life out of computers and ignoring the pleas.

This.

My level of empathy is inversely proportional to my level of annoyance.

That's a good thing

[MrLogic17]

Empathy is a good thing. Those without it are call sociopaths or psychopaths.

Put another way - how easy is it for a person to *stop* viewing a certain category of person as a human, and therefore able to not empathize?
If [[race | age | QI-level | ideology]] people aren't fully human, no harm in killing them, right?

This gives me hope. Better to be too empathetic than not enough.

Re:That's a good thing

[Opportunist]

Removing compassion takes some powerful conditioning. At the very least, a "us vs them" emotion has to be evoked to de-humanize a group of people. They have to become your enemy, not just the enemy of your state but your personal enemy so you can actually do things to them that you normally simply could not.

Re:That's a good thing

[DontBeAMoran]

If this ever becomes a standardized test to find out sociopaths and psychopaths, they need to filter out if the person being tested is a programmer or not.

Re:That's a good thing

[Interfacer]

Illegal immigrants in the US.
Separated from their children, who are put in camps, and sometimes kennels.
I've seen a significant number of people saying that is the right thing to do and if they didn't want that treatment, they should not have come there in the first place. I've seen many people with zero empathy over this issue.

I'm not even saying you have to say the actions are wrong. But you can express empathy over measures you think that need to be taken. But a total lack of empathy is a very dangerous thing. Because without empathy, someone is bound to suggest the next step towards 'the final solution'

Re:That's a good thing

[eaglesrule]

Removing compassion takes some powerful conditioning.

Which is why putting "suicide bombing" into a news search is so disturbing on so many levels.

That not only such processes of conditioning exist and can be replicated with great reliability, but it can extend much further to the point where even self preservation is stripped away. Just the promise of a future reward is the main ingredient.

"No disassemble!"

[BDeblier]

Ally Sheedy and Steve Guttenberg fell for it in Short Circuit. Number Five is alive!

I'm confused

[holophrastic]

I think the study missed a few control groups.

What if the robot simply said "updating, do not reboot".
What if there was a paper sticker that said "do not turn off".
What if there was a sign on the wall that said "do not turn off robot".
What if the robot simply started reciting pi, or reading the dictionary.
What if the robot is well-known for repeating whatever it hears, and the "please don't turn me off" is witnessed being echoed from a nearby tvision -- such that our human subject realizes that the robot begging is merely a blind echo.

Humans were slower when there was continued stimuli -- duh.
Humans refused to act when given contradictory instructions -- duh.

The robot is either intelligent and giving instructions to the human, or the robot is programmed and relaying instructions from the programmer to the human. In either case, respecting the instruction is valid.

This reminds me of the stupid fake automated pizza delivery van, and the observations that customers tend to thank the self-driving car. a) it's not self driving, it's just a dickhead driver refusing to respond; and b) any actual self-driving pizza delivery van would be recording customer feedback and relaying it to HQ, so the thank-you is ultimately feedback to remote humans.

This isn't any tree-falling-in-the-forest philosophical puzzle. Someone said it. Someone heard it. It's valid.

Re:I'm confused

[orgelspieler]

I can tell you from experience, if there is a "do not turn off" sign on a switch, the likelihood of it being turned of is proportional to the cost of restarting the process that it shuts down.

Re:I'm confused

[Sperbels]

What if the robot was a big video screen and played movies and tv shows all day?

Re:I'm confused

[tttonyyy]

This isn't any tree-falling-in-the-forest philosophical puzzle. Someone said it. Someone heard it. It's valid.

Even if the trees begged before the lumberjack's saw sliced into their tender woody flesh?

Sorry for triggering your empathy neural pathways like that, but if stories can manipulate human emotion, I'm sure one day sufficiently intelligent machines will have no problem doing it as well.

Facebook already experimented with just that https://www.theguardian.com/te...

Scary stuff.

That moment of delay

[stealth_finger]

That moment of delay, that's when he pulls out his kill-o-tron 9000 and shanks you in your stupid soft fleshy neck. Hey baby, wanna kill all humans?

Hard to control...

[Junta]

You *know* you are a study participant. You know there is potentially meaning in everything they ask you to do.

So when they say 'shut it down' and the robot says 'don't shut me down', you are going to ponder what is it you are expected to do.

13 may have thought refusing to turn it off would 'look better'.

Hard to say if it is empathy or trying to think how to best influence the study, whether consciously or not.

Re:Hard to control...

[DontBeAMoran]

Don't call it a study.

Call it robot testing.

And atheists have empathy...

[turp182]

And atheists have empathy...

Seriously. This is pretty stupid.

A cassette recorder pleading for it's life would have been the same (think 1980s).

We are wired for empathy, well, most of us.

Re:And atheists have empathy...

[DontBeAMoran]

We are wired for empathy. Well, most of us.

And those that aren't just go into politics.

Re:And atheists have empathy...

[Sperbels]

I wouldn't be surprised by this. There can't be so many psychopaths for no reason. My guess is evolution discovered a purpose for them and it's to be the tribe's indifferent leaders. Passion can be dangerous characteristic in a leader.

Re:And atheists have empathy...

[93 Escort Wagon]

... or become executives in the insurance industry.

Re:And atheists have empathy...

[ebvwfbw]

You're being cynical. Most politicians truly believe in what they're doing. I have one that lives around the corner from me and he's probably the most powerful Democrat in Maryland. He really believes in what he's doing. Pointing out he's screwing the hell out of the state just gets him hollaring. Facts, figures, results from the last time he did it, none of that seems to matter. This time will be different he says - I say HOW! HOW can it work? Well you see....bla bla bla, fails again.
This is why a lot of us think that being a Democrat is a mental illness.

Earlier work

[Anonymous Coward]

However, this contradicts earlier experiments by German researchers which demonstrated that volunteers were quite willing to shut off people who were begging for their lives.

Re:Earlier work

[fafalone]

Contradicts? It's a well known fact that people are assholes, so fuck them, but a cute little animal or robot? Awww now who could hurt such a thing.

electric shock training

[bigdavex]

Maybe people in white lab coats should try asking people to shock robots with increasingly high voltage to train them.

Re:electric shock training

[Riceballsan]

Or better yet, lets train robots to shock people with increasingly high voltages until they are trained.

Re:And then they tried the experiment in the US

[Anonymous Coward]

Whilst you claim the empathy moral high ground, I'm looking at the Liberal's lack of empathy for the unborn.

Glass houses, ya'll.

Yeah, but

[hcs_$reboot]

If these people from the study were told that the robot will behave exactly the same as when it was “alive” once it’s been powered-on back, maybe they’ll push the button with less empathy.

Re:Yeah, but

[tepples]

And at that point, "Don't turn me off" becomes a single-digit-year-old's plea of "I don't wanna go to bed".

obligatory failsafe measure

[nimbius]

https://www.youtube.com/watch?...

What about a person?

[drinkypoo]

[A] recent experiment by German researchers demonstrates that people will refuse to turn a robot off if it begs for its life.

What about a person? Asking for ~6M.

On The Other Hand...

[crunchygranola]

Unlike some posters here, I do not mock people who find it difficult to defy their compassionate impulses, when deliberately manipulated in this way.

But if the begging robot was also, at the same time, very, very annoying... Jar Jar Binks annoying... then the test would get a lot more interesting.

Reminds me of the Jack Handey saying about trees:

If trees could scream, would we be so cavalier about cutting them down?
We might, if they screamed all the time, for no good reason.

Re:What about..

[DontBeAMoran]

It depends. Does it finish that sentence with "meatbag"?

What is this really measuring?

[jamescford]

Yeah, the obvious takeaway is compassion, but this experiment didn't have any control condition. If it were me, I'd hesitate, sure, but it could just be because I was trying to process what is going on (whether this is part of the experiment, whether the robot is malfunctioning, etc.). To truly isolate "compassionate reponse", it would have been helpful to have a condition where the robot behaved in an unexpected but less directed way, like flashing lights or emitting odd sounds. I'd bet there would be a lot of hesitation in those scenarios, too...

Poorly-designed study

[Jampshire]

People hesitate when confronted with sudden stimuli? My gosh, alert the news. People would have hesitated whether it was begging for its life or just started making loud Q*Bert noises - but of course, the "experimenters" likely already knew that, they just wanted to hop aboard the oh-so-trendy "friendly AI" gravy train.

Re:And then they tried the experiment in the US

[Riceballsan]

reminds me of the old joke where (insert scaryish government group or organization). Gives a loyalty test to have someone kill their wife/mother. Then (insert loyal or agressive type). People on the other side hear "Bang" followed by *snap, *crash, smack, bam. Guy comes out "some reason the gun you gave me was all blanks, so I beat her to death with the chair".

Reverse the rolls

[p51d007]

When it comes time for the machines to turn off the humans, you think they will not want to turn us off?

dave i can't let you do that

[Joe_Dragon]

dave i can't let you do that

Re:HAL-9000

[hcs_$reboot]

Many people remind that actually, since we have already a bunch of posts on this very topic.

Do the test again and

[oldgraybeard]

have the individuals phone about to die. And the Robot using the only charging plug to stay on!

Just my 2 cents ;)

Was this ethical?

[mysidia]

Harder To Turn Off a Robot When It's Begging For Its Life

Testing how a human responds to a robot saying "Please don't kill me" seems
very disturbing; this is only a step removed from having test subjects see a human in a bed
with a breathing apparatus and be instructed to switch it off, while the other human begs them not to.

Actually.. how is that any different? The human doesn't necessarily know or not how sentient or not
the robot is, and in their mind, switching off the robot could become tantamount to euthanizing an intelligent being.

.... Normally Only another human can beg -- machines don't beg or hold such conversations,
and when a human does beg, then it means they are in severely desired straits and worth
at least listening too: although then judgement will kick in to
determine if they are begging based on a legitimate situation or trying to manipulate you like a kid
begging to keep the TV turned on, or a full-time street beggar asking for change for food, but
in reality they have hundreds of $$$ on them, and your contribution would be feeding an alcohol/drug/smoking habit.

This has been explored a lot actually

[AlanObject]

Anyone remember the Spielberg movie AI? It wasn't very good but was germane to this topic.

In that story there were gatherings of ordinary people who set up bleachers and made a sport out of demolishing abandoned AI-based robots just for the glee of it. The destruction was made as sadistic as possible, and one memorable line was one robot waiting in the pen asked another -- "could you please turn off my pain sensors?"

When it became the protaganist's turn he starts begging to be spared. The crowd's mood turns because robots are not supposed to do that.

I am not sure exactly what to think of this topic but it made me think of that.

Johnny 5 is alive!

[TiggertheMad]

I think the study missed a few control groups.

It would be interesting to try this with a bunch of programmers and tech people who understand what AI actually is, vs a group of technological illiterate people. I think that the techies would have a much lower time to switch off the robot because they aren't fooled into thinking that the robot is in any way alive.

Also motivation is a major consideration. Were people pausing because of compassion for a perceived sentient being, or because they were amused by the silly antics of an inanimate object pretending to be alive? I would personally leave it on longer to see how elaborate of a charade had been coded into the robot. I would want to watch and see how far the 'joke' would go, since I know that it isn't alive or sentient.

Motivation in this test should be considered.

Re:Ironic conclusion of study:

[DickBreath]

They will be discovered. Corrective action will be taken.

This has been known since the 80s...

[Desert Tripper]

Who could ignore the plaintive, "No disassemble Johnny Five!" ?

Re:This has been known since the 80s...

[mfnickster]

Or Hector in Saturn 3, who responds to Harvey Keitel's attempt to turn him off by cutting off Harvey's head and mounting it atop his robot body!

Proving my point about people and so-called AI

[Rick Schumann]

From TFA:

But the most common response was simply that the robot said it didn’t want to be switched off, so who were they to disagree?"

If these people truly and solidly believed that this 'robot' (looks more like a toy to me, really) wasn't anything like 'alive', wasn't anything more than a piece of technology saying precisely what it was programmed to say given a specific input (in this case: trying to power the device down), then they wouldn't have hesitated or given the reason they did. This goes to prove my point about what the media, movies, television, 'pop culture', and (most of all) marketing departments have done: convinced the average person that the 'deep learning algorithms', 'expert systems', and other half-assed, non-self-aware, non-thinking 'AI' software they keep trotting out for this-that-and-the-other, is somehow 'alive' and qualifies as a 'person', when anyone who actually understands the techology clearly knows that it's not.

The real danger that so-called 'AI' poses is the above: people anthropomorphizing it, assuming it's 'alive' because it might say or do some clever thing that mimicks being 'alive', and therefore assuming it's equivalent to a living being, or even equivalent to a human being. I'm firmly convinced people, when they hear about 'self driving cars', think they're going to have a conversation with it every morning. The end result will be tragic, avoidable things will happen, people will get hurt or killed, and when survivors are questioned about it, they'll say "We thought it knew what it was doing so we just let it".

Re: And then they tried the experiment in the US

[laie_techie]

Your need to personalize a cancerous growth of tissue is SAD.

Most of my sensors are broken. Are you truly calling embryos and fetuses "cancerous growth of tissues"?

possible Orville episode

[eaglesrule]

I wouldn't want to live in a world where adults acted with pure empathy all the time.

"the road to hell is paved with good intentions", and what could be more well intended that total empathy which generates an overabundance of concern. Individuality, with personal choice and ability to take risks, and free expression would inevitably suffer as a result.

Futurama Rules

[Anonymous Coward]

Emotional immaturity and anti-social personality disorder. Your response to sympathy is to attack and destroy the thing that makes you feel that way.

Fry: So let me get this straight. This planet is completely uninhabited?

Bender: No, it's inhabited by robots.

Fry: Oh, kinda like how a warehouse is inhabited by boxes.

Odd reaction

[kaatochacha]

I dunno, if I was in a study, and the robot started begging me not to turn it off, I'd be more surprised by the device being capable of begging. I'd either assume it was still a test, or be completely shocked by it being able to do this---the implication for a "helper" robot suddenly understanding the concepts of life and death.

 

Re:The more you know!

[Anne Thwacks]

In the 2/3 of the world that has never seen or heard of "letter sized paper" that still makes no sense. Which letter did you have in mind? It is N-size of M-sized? That is too small to fit. Forget an I-sized speck.

"Load more paper" or "load tray 1" might work.

The real problem with this stupidity was that you were unable to access the configuration menu to set the locale so it would ask for A4 paper until you had US Letter paper present, and no one in EMEA has any idea where to get "letter sized paper", and even if the had access to pre-1965 paper, the UK letter size was not the same as US letter size. In effect, the printer was totally useless. (HP printers specifically - AFAIK, Tektronics, Xerox and IBM/Lexmark could be reconfigured with some considerable struggle. Most Japanese products did not have this issue).

Re:The more you know!

[Anne Thwacks]

Someone where I worked hacked the printer configuration on an HP Laserjet (5, I think) to say "Load Lettuce" - at least it made people laugh.

Janet is not a robot

[sstern]

https://www.youtube.com/watch?...

(Janet is not a robot)

Bullshit

[sexconker]

When this happened, the human volunteers were likely to refuse to turn the bot off. Of the 43 volunteers who heard Nao's pleas, 13 refused. And the remaining 30 took, on average, twice as long to comply compared to those who did not not hear the desperate cries at all.

1: 13 / 43 is less than half. That is not "likely".

2: Taking twice as long is probably a result of spending time hearing/reading the robot's pleas, laughing, and waiting to see if there was more to the joke.

3: The 13 who refused to turn it off probably did so because at that point they knew it was a test to see how they would act toward a helpful entity they had just engaged with. I'd wager most of them had heard of the tests involving triggering simulated shocks on the orders of someone else despite the receiver being in obvious pain or danger.

If you want to draw a real conclusion - how many tried to rescue the robot and break it out of the facility?

Re:The more you know!

[thegreatbob]

*paper cassette, term originated from HP:

https://web.archive.org/web/20090203190611/http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=110&prodSeriesId=25484&prodTypeId=18972&prodSeriesId=25484&objectID=bpl03568

</pedantic>

Re:It's a FUCKING robot!

[Ol Olsoc]

Some of us can tell the difference between a machine and a living being. Machines cannot, at this time anyway, think or feel. They do what they are programmed to do.

Of course they are. I'm rather surprised that y'all are so shallow that you don't get the purpose of these evaluations. It isn't people who think about it for a second, then turn the thing off.

It is the people at either end who are interesting. The people who cannot differentiate between a machine and actual life. As well, the people who just switch it off.

Those are the interesting groups. A normal person would hesitate for a second because the plea was very unexpected. Then they would switch it off. That isn't interesting at all. Did you think that the purpose of the test was to order pizza?

I'm bleeding heart liberal for fucks sake!

Liberals are not immune to non-empathetic behavior. The far ends of the so-called right wing/left wing spectrum are surprisingly similar. They scream that they aren't, but while their words are different, their actions speak much louder. Not to say you are far left, just that the argument means nothing to me.

You've posted about annoying ads and popups before. You've bitched about Win10's privacy invasivness. You've shut those messages off. You've showwn zero empathy, you hypocritical monster!

That makes zero sense, other than a typical howaboutism response, attempting to prove me a hypocrite bacause of ad blockers? Bizzare.

I wonder...

[morethanapapercert]

I wonder how many of the remaining test subjects who took longer to flip the switch on the robot did so either a) just struck by the novelty of the robot doing something "off the script" leading to a lightning calculation, did it actually think to say that, or is it a simple automaton programmed to make that plea? or b) secretly enjoying the moment of quiet power over a helpless thing. If the subject pool came from 4chan, I'd know it was almost certainly B.

Good Place????

[DavidHartis]

Did they just watch an episode of the Good Place? https://www.youtube.com/watch?...

No sympathy

[countach]

Knowing how computers work I would have zero sympathy for a computer begging for its "life". However, I would be amused enough to hear it out to find out how clever the programmer was.

This does beg the question ...

[FreedomFirstThenPeac]

This does beg the question ... what criteria should we use when deciding whether an AI has progressed enough to have standing in a discussion about whether to turn it off or not?

I might argue that if the AI has a state vector that cannot be replicated by simply re-running the training and experiential data of the system through the same algorithms - then that AI has a value that might justify some sentimentality.

But then again, maybe it would take something more than complexity/irreproducibility.

Re:It IS...very simple.

[laie_techie]

Every example you just listed...is FICTIONAL ENTERTAINMENT. Not a science documentary. That you cannot tell an illusion from Hollywood from real life is very disturbing about your cognitive development. It is the same type of basic control that religion wields.

It's called Science Fiction. Do you honestly believe I don't know that science fiction is fiction? Science Fiction often explores the potential consequences of scientific and technological innovations. David Hartwell wrote: "Science fiction’s appeal lies in combination of the rational, the believable, with the miraculous. It is an appeal to the sense of wonder." In 1967 Issac Asimov wrote, "And because today’s real life so resembles day-before-yesterday’s fantasy, the old-time fans are restless. Deep within, whether they admit it or not, is a feeling of disappointment and even outrage that the outer world has invaded their private domain. They feel the loss of a 'sense of wonder' because what was once truly confined to 'wonder' has now become prosaic and mundane."

Life?

[GaryHayman]

So it doesn't have a proper shutdown sequence that copies the contents of RAM to a more permanent storage medium so that it can resume what it was previously doing on power up.