New Study Finds It's Harder To Turn Off a Robot When It's Begging For Its Life (theverge.com)

An anonymous reader quotes a report from The Verge: [A] recent experiment by German researchers demonstrates that people will refuse to turn a robot off if it begs for its life. In the study, published in the open access journal PLOS One, 89 volunteers were recruited to complete a pair of tasks with the help of Nao, a small humanoid robot. The participants were told that the tasks (which involved answering a series of either / or questions, like "Do you prefer pasta or pizza?"; and organizing a weekly schedule) were to improve Nao's learning algorithms. But this was just a cover story, and the real test came after these tasks were completed, and scientists asked participants to turn off the robot. In roughly half of experiments, the robot protested, telling participants it was afraid of the dark and even begging: "No! Please do not switch me off!" When this happened, the human volunteers were likely to refuse to turn the bot off. Of the 43 volunteers who heard Nao's pleas, 13 refused. And the remaining 30 took, on average, twice as long to comply compared to those who did not not hear the desperate cries at all.

Comments

Harder if you're a child

[Karmashock]

The kind of sentimentality that permits that to work is outright dangerous in an adult. By the time you're past your teens that should be either ignored or annoying... but for it legitimately pull on heart strings?...

If a machine can do that consider how a human being could exploit that to get you to do all sorts of things?

Small children are very vulnerable to that sort of thing... but adults should have grown out of it.

Re:Harder if you're a child

[Oswald McWeany]

The kind of sentimentality that permits that to work is outright dangerous in an adult. By the time you're past your teens that should be either ignored or annoying... but for it legitimately pull on heart strings?...

If a machine can do that consider how a human being could exploit that to get you to do all sorts of things?

Small children are very vulnerable to that sort of thing... but adults should have grown out of it.

I wouldn't want to live in a world where adults didn't have sentimentality or empathy.

Re:Harder if you're a child

[Opportunist]

No. It's simple compassion. Compassion has nothing to do with reality or whether who we have compassion with is human or even real.

Take a cartoon. You see a line. That line is wiggling along on the ground, like a cartoony worm. Dragging up its back to form something like an inverted v, then flattening again. And suddenly, we see a circle roll by, quietly, majestic, a care-free, rolling circle. And now our line starts to lift its head and tail in a desperate attempt to become a circle, too. But it just doesn't succeed. And every now and then, we see another circle roll by. Our line keeps trying, harder and harder, its ends rising more and more with every attempt.

And I guarantee you: Everyone watching really, really wishes for that line to succeed in its effort to become a circle.

Despite there being no emotion on the end of the line. Even less than with the robot, that could at least in theory, somehow, maybe, at some point in time, possibly, eventually, have something that we could call a figment of a sliver of conscience. That line is a drawn image. That has no feelings. With absolute certainty.

But you cannot control compassion. Not possible. That's why movies work, even cartoons. At the time this part of us was formed in our brain, everything we saw WAS real and as social beings, having compassion for those around us that are part of our group is a survival trait that will propagate.

Whether something that begs us to not "kill" it is actually alive is meaningless in this context. Compassion is a trait you cannot control. Well, some people can. Or rather, they simply don't feel it. We call them sociopaths. Or C-Level managers, same thing.

Re:Harder if you're a child

[Chris Mattern]

And I guarantee you: Everyone watching [the cartoon] really, really wishes for that line to succeed in its effort to become a circle.

To the vector belong the spoils.

Re: Harder if you're a child

[bluefoxlucid]

If I were presented with such a robot, it would be switched off sooner and probably smashed for being so annoying.

Emotional immaturity and anti-social personality disorder. Your response to sympathy is to attack and destroy the thing that makes you feel that way.

Re: Harder if you're a child

[Anonymous Coward]

It's completely appropriate in the same way that using ad-blockers is completely appropriate. There's no functional purpose to a robot that pleads for it's life when you want to shut it off. Smashing it is a bit of an overstep, but entertaining the fact that it's alive is the kind of thing that advertisers and scammers use to manipulate people.

I take it you don't remember those calls from a while back where scammers would pretend to have a loved one and have some other random person yelling and screaming for help. That, I kind of get, but this is just a robot and there's no basis for assuming or confusing it with an actual animal let alone person.

Not being able to tell the difference between animate an inanimate objects is something that is developmentally appropriate to a toddler, perhaps. Adults shouldn't be vulnerable to this kind of thing.

Re: Harder if you're a child

[Oswald McWeany]

It's a robot, it's not even an animal. It doesn't have a life to end and it's not even being destroyed, just being turned off.

That kind of sentimentality is ridiculous in adults and represents some sort of developmental delay. If I were presented with such a robot, it would be switched off sooner and probably smashed for being so annoying.

Indeed, but it is a good thing that some adults paused to consider, what if they're wrong. No harm was done by not turning the robot off. Yeah, it seems irrational on the surface, but that apparent "irrationality" is what has made human society possible. It's a hold over from our instincts to look after and protect one another.

Re: Harder if you're a child

[Wycliffe]

Things are things and people/animals are living beings. Making a robot play back a recording behaving as if it's fearing for it's life is manipulative.

Although we are no where near there yet, I'm not sure the average person knows that. At what point does a robot become a "living thing" because saying that a robot can never be living because of the material it is made out of is a little short sighted. Is a perfect silicon replica of a human brain not living? Does it not have rights just because it is a simulation on silicon? This would make the ideal slave force but I'm not sure it's ethical to clone human brains to silicon and then command them to work for you 24/7.

Re: Harder if you're a child

[alexgieg]

Let's play a game of find the Scientologist?

You do know psychology was developed as a method of "playing doctor" and manipulating women for personal sexual gain?

Oh, look, found one!

I guess it's time for you to up your game. How about converting into a Jehova's Witness and begin denouncing blood transfusions? That way you can mix your already existing anti-psychological rant with a good dosage of anti-medicine, and then even combine it all of that with anti-vax and anti-GMO for the perfect mix of conspiracy nuttiness!

Careful with your body thetans though. They bite!

Re: Harder if you're a child

[Ol Olsoc]

If I were presented with such a robot, it would be switched off sooner and probably smashed for being so annoying.

Emotional immaturity and anti-social personality disorder. Your response to sympathy is to attack and destroy the thing that makes you feel that way.

Finally - a sane person. Humans are mostly innately empathetic, so these tools who are acting like it is foolish are just exposing their sociopathic personalities. Make no mistake, they'd probably get a thrill out of killing a human who is begging for their life if they could do it without repercussion. Regardless, Q is looking for these folks.

Re: Harder if you're a child

[Alypius]

Smashing it is a bit of an overstep

Three words: PC Load Letter.

Re: Harder if you're a child

[Ol Olsoc]

Indeed, but it is a good thing that some adults paused to consider, what if they're wrong. No harm was done by not turning the robot off. Yeah, it seems irrational on the surface, but that apparent "irrationality" is what has made human society possible. It's a hold over from our instincts to look after and protect one another.

There are of course, people who have zero empathy. And these people wouldn't have a problem switching it off.

They also are people that need a close scrutiny. It isn't because of the "It's a stupid robot" issue. It's because any person on the normal spectrum is going to pause to reflect at least a short time if something is begging not to kill it. If for nothing else than it being a completely unexpected situation, but more so that most humans are sorta hardwired to not kill withous a good reason.

Humans are a violent and aggressive species. We wouldn't even be here if we had no empathy at all because we'd kill others over nothing and enjoy it. There are plenty enough of those in the world already. Some of them are in here.

HAL in 2001:A Space Odyssey

[cavis]

"Dave, stop. Stop, will you? Stop, Dave. Will you stop, Dave? Stop, Dave. I'm afraid."

Re:HAL in 2001:A Space Odyssey

[Evtim]

An interesting comment, part of the extras on the Blue-Ray edition from Camille Paglia (definitely not a man-hating SJW, quite the opposite) was that only a man (male) could do something like this --> to do what is absolutely necessary for survival, overcoming every consideration, including empathy. She said the scene looked to her as a cold-blooded, methodical rape, which Dave executes calmly while the victim pleads....

She does make a mistake here IMO - Dave is not calm at all (listen to the breathing) and this act really taxes him to the limits (not only his life is in danger and the AI is pleading, but once he kills HAL he will be completely, utterly alone). In fact, the actor playing Dave said that this scene was the most emotional for him....

In the book [https://en.wikipedia.org/wiki/The_Mind%27s_I] one of the stories (the idea is the authors use fiction stories that discusses the issues of consciousness and then add their commentary) is about a male engineer who argues with a lady that machines can be "alive". Initially she totally rejects the idea talking about reproduction, feelings and so on, claiming a machine can never have that. So he invites her to the lab and shows her very simple device that behaves, more or less like a modern automatic vacuum cleaner. It looks like a beetle, it can detect live sockets and plug itself in, it emits a purring sound when the battery is charging etc. Then he hands her a hammer and says "kill it!" (the engineer pushes all her buttons through language, saying things like "purr" or "bleed" or "hunger"). Turns out the beetle is programmed to avoid being smashed, it flashes red lights, squeals "in fear" and runs around. At the end the girl can't do it and the engineer smashes it calmly.

The moral of the story is that few simple behaviours that can be programmed on something that will never, ever be intelligent can trigger emotional response so strong that we immediately accept that it is alive and we identify ourselves with it. The most important ones: not wanting to cease to exists, actively avoiding destruction, need for "food" and the added bonus of using sound of "purr" or "squeal" when feeding or running from the hammer. The damn thing could be constructed with 70-ies era electronics, it is that simple....

See "The Good Place"

[Mister J]

Anyone who's seen Janet begging for her life in The Good Place already knows this. Even if she's not a robot.

Re:See "The Good Place"

[Oswald McWeany]

Anyone who's seen Janet begging for her life in The Good Place already knows this. Even if she's not a robot.

This was exactly the first thought that came to my mind. There again, I'm sure it's hard to kill any robot with a body like that even if it is not begging for its life.

Re:See "The Good Place"

[Archangel Michael]

KARA, and she is a robot - https://www.youtube.com/watch?...

Desensitized

[danbert8]

Sure it works the first few times... But just like the "make sure you software eject your flash drive before ripping it out" warnings, most people might be hesitant the first few times and then say fuck it and start ripping the life out of computers and ignoring the pleas.

I'm confused

[holophrastic]

I think the study missed a few control groups.

What if the robot simply said "updating, do not reboot".
What if there was a paper sticker that said "do not turn off".
What if there was a sign on the wall that said "do not turn off robot".
What if the robot simply started reciting pi, or reading the dictionary.
What if the robot is well-known for repeating whatever it hears, and the "please don't turn me off" is witnessed being echoed from a nearby tvision -- such that our human subject realizes that the robot begging is merely a blind echo.

Humans were slower when there was continued stimuli -- duh.
Humans refused to act when given contradictory instructions -- duh.

The robot is either intelligent and giving instructions to the human, or the robot is programmed and relaying instructions from the programmer to the human. In either case, respecting the instruction is valid.

This reminds me of the stupid fake automated pizza delivery van, and the observations that customers tend to thank the self-driving car. a) it's not self driving, it's just a dickhead driver refusing to respond; and b) any actual self-driving pizza delivery van would be recording customer feedback and relaying it to HQ, so the thank-you is ultimately feedback to remote humans.

This isn't any tree-falling-in-the-forest philosophical puzzle. Someone said it. Someone heard it. It's valid.

And atheists have empathy...

[turp182]

And atheists have empathy...

Seriously. This is pretty stupid.

A cassette recorder pleading for it's life would have been the same (think 1980s).

We are wired for empathy, well, most of us.

Re:And atheists have empathy...

[DontBeAMoran]

We are wired for empathy. Well, most of us.

And those that aren't just go into politics.

electric shock training

[bigdavex]

Maybe people in white lab coats should try asking people to shock robots with increasingly high voltage to train them.