Killer Robots Will Only Exist If We Are Stupid Enough To Let Them (theguardian.com)

Heritype quotes the Guardian's science correspondent: The idea of killer robots rising up and destroying humans is a Hollywood fantasy and a distraction from the more pressing dilemmas that intelligent machines present to society, according to one of Britain's most influential computer scientists. Sir Nigel Shadbolt, professor of computer science at the University of Oxford, predicts that AI will bring overwhelming benefits to humanity, revolutionising cancer diagnosis and treatment, and transforming education and the workplace. If problems arise, he said, it will not be because sentient machines have unexpectedly gone rogue in a Terminator-like scenario.

"The danger is clearly not that robots will decide to put us away and have a robot revolution," he said. "If there [are] killer robots, it will be because we've been stupid enough to give it the instructions or software for it to do that without having a human in the loop deciding...."

However, Prof Shadbolt is optimistic about the social and economic impact of emerging technologies such as machine learning, in which computer programmes learn tasks by looking for patterns in huge datasets. "I don't see it destroying jobs grim reaper style," he said. "People are really inventive at creating new things for humans to do for which will pay them a wage. Leisure, travel, social care, cultural heritage, even reality TV shows. People want people around them and interacting with them."

Comments

stupid enough

[PopeRatzo]

It's 2018. We've broken through the "stupid enough" barrier.

Re:stupid enough

[NoNonAlphaCharsHere]

We don't have a "human in the loop, deciding" for the current generation of neural net AIs. We don't have a human deciding over genetic algorithms, either. We create a fitness function, back propagation, whatever, and it's off to the races, unintended consequences be damned. The fact that we build/design along the lines of "improve yourself along criterion X" means we're slowly building an existential threat to ourselves whether we realize it or not. Clearly, a soldier-bot has two prime directives: 1) wipe out the enemy 2) stay functional, so you can wipe out the enemy.

Not much of a comfort.

[religionofpeas]

We clearly are stupid enough.

After trump anything is possible.

[Anonymous Coward]

After trump anything is possible. Think we blew past stupid at mile marker 2016 and are careening towards an uncertain future.

It's not a matter of stupid

[Jeremi]

People won't make or deploy killer robots "by accident". If a robot goes on a killing spree, it will be because somebody deliberately programmed it to go on a killing spree.

Are people perverse enough to make a machine that will deliberately kill other people, either based on specific entry-conditions or even just randomly? The existence and widespread use of land mines and car bombs demonstrates that the answer is yes.

So really we know the answer; we're only arguing about an implementation detail: exactly how sophisticated people will allow their automated killing machines' triggering-mechanisms to be.

Re:It's not a matter of stupid

[gweihir]

Not at all. If machines have code and capabilities ready to go on a killing spree, it will also happen by accident. Remember the world was almost nuked by accident several times.

Only one guy

[lorinc]

It takes only one guy with the right capabilities and stupid enough to do it. History has proven that there are plenty of such people. You can be sure that there are plenty of high level military officers in many countries that are day dreaming of something from Screamers, and will do anything that is in their power to make it a reality...

Computer scientist rather than software engineer..

[fuzzyfuzzyfungus]

The theory that enormously complex software systems specifically designed to be capable of novel behavior definitely won't go off the rails seems like something that you could only embrace if you've never actually interacted with real software as written by real people.

There is also the...minor...problem that "have a human in the loop deciding" will be a feature that will have to be implemented in software; and we definitely don't have a history of either unhelpful program output or unpleasant reaction to malformed inputs; so that will go well.

They already exist

[gweihir]

Every landmine qualifies as a very low capability "killer robot". The insane harm landmines to around the globe is a good indicator that there are by far enough people with power and money and absolutely no qualms about maiming and killing innocent bystanders and civilians in general. Hence we will definitely see killer robots of much higher capabilities, unless we get the fucked-up part of the human race under control that simply cannot stop killing others and using violence to solve disagreements.

TRUMP = EVIDENCE

[Anonymous Coward]

Trump is "in charge" of America, so yes, very fucking stupid people must live here.

Not easy to avoid in an arms race

[Kjella]

Side A builds robots that can't fire without human control. Side B builds jammers. Side A decides robot soldiers need to be able to act in "self-defense". Side B puts civilians in harm's way. Side A decides they need "smart robots" who can tell friend from foe by themselves. Or that we need tighter coordination between light arms, heavy arms, air support, putting down covering fire for advancing troops etc. with so tight margins that it can't be done on manual. If you're being mauled to death by a perfectly coordinated fully automatic enemy you will fight fire with fire. Maybe you're creating the world where we'll lose control of our Terminators. But in the short term if you're not playing the game you're going to lose right now.