Killer Robots Will Only Exist If We Are Stupid Enough To Let Them (theguardian.com)

Heritype quotes the Guardian's science correspondent: The idea of killer robots rising up and destroying humans is a Hollywood fantasy and a distraction from the more pressing dilemmas that intelligent machines present to society, according to one of Britain's most influential computer scientists. Sir Nigel Shadbolt, professor of computer science at the University of Oxford, predicts that AI will bring overwhelming benefits to humanity, revolutionising cancer diagnosis and treatment, and transforming education and the workplace. If problems arise, he said, it will not be because sentient machines have unexpectedly gone rogue in a Terminator-like scenario.

"The danger is clearly not that robots will decide to put us away and have a robot revolution," he said. "If there [are] killer robots, it will be because we've been stupid enough to give it the instructions or software for it to do that without having a human in the loop deciding...."

However, Prof Shadbolt is optimistic about the social and economic impact of emerging technologies such as machine learning, in which computer programmes learn tasks by looking for patterns in huge datasets. "I don't see it destroying jobs grim reaper style," he said. "People are really inventive at creating new things for humans to do for which will pay them a wage. Leisure, travel, social care, cultural heritage, even reality TV shows. People want people around them and interacting with them."

Comments

stupid enough

[PopeRatzo]

It's 2018. We've broken through the "stupid enough" barrier.

Re:stupid enough

[NoNonAlphaCharsHere]

We don't have a "human in the loop, deciding" for the current generation of neural net AIs. We don't have a human deciding over genetic algorithms, either. We create a fitness function, back propagation, whatever, and it's off to the races, unintended consequences be damned. The fact that we build/design along the lines of "improve yourself along criterion X" means we're slowly building an existential threat to ourselves whether we realize it or not. Clearly, a soldier-bot has two prime directives: 1) wipe out the enemy 2) stay functional, so you can wipe out the enemy.

Re:stupid enough

[Ol Olsoc]

It's 2018. We've broken through the "stupid enough" barrier.

We've completely demolished it.

Olsoc's rule if killy stuff : If there is a method of killing people, Governments will rush to it like a dog to bacon. The more barbaric, the more bacon.

Olsoc's second rule of killy stuff: Unless people are being killed, there isn't much point of warfare.

Which brings us to Olsoc's third rule of killy stuff: War robots will be specifically designed to kill people.That is how humans work.

Re:Not much of a comfort.

[Hognoxious]

Speak for yourself, you insensitive clod!

I'm just lazy and careless.

The rich are going to want automated kill bots

[rsilvergun]

because they'll be cheaper than maintaining a huge standing army and you don't have to worry about a general taking over. The engineers who keep the things running will lack the charisma and ambition to overthrow the current ruling class (they're part of the merchant class after all and will be doing well enough).

The way to stop this crap is pretty clear. Declare all human beings deserving of a decent quality of life and then make that happen. Get over the fact that you'll have a few surfer dudes and wellfare queens that don't work very much or at all (shouldn't be too hard, most of us have long since stopped getting mad at the idle rich with inherited wealth). If you want a population smart enough and paying enough attention to see this kind of crap coming and stop it you need to take care of their basic needs first. Otherwise they'll be too busy fighting for survival to do anything about it, which is kind of the point.

It's not a matter of stupid

[Jeremi]

People won't make or deploy killer robots "by accident". If a robot goes on a killing spree, it will be because somebody deliberately programmed it to go on a killing spree.

Are people perverse enough to make a machine that will deliberately kill other people, either based on specific entry-conditions or even just randomly? The existence and widespread use of land mines and car bombs demonstrates that the answer is yes.

So really we know the answer; we're only arguing about an implementation detail: exactly how sophisticated people will allow their automated killing machines' triggering-mechanisms to be.

Re:It's not a matter of stupid

[gweihir]

Not at all. If machines have code and capabilities ready to go on a killing spree, it will also happen by accident. Remember the world was almost nuked by accident several times.

Only one guy

[lorinc]

It takes only one guy with the right capabilities and stupid enough to do it. History has proven that there are plenty of such people. You can be sure that there are plenty of high level military officers in many countries that are day dreaming of something from Screamers, and will do anything that is in their power to make it a reality...

What a sweet talk!

[MerlinTheWizard]

I'm not sure this "professor" has really understood what AI was all about. Thinking that any AI-enabled device will just act as it is "programmed to" is clearly simplistic (although by itself a tautology, since software-based machines are just running 'programs') and a complete misconception of where AI is heading to IMO.

AI without the internal ability of devising new ways of doing things is NOT AI. And by being able to devise new ways, it has pretty much equal chances for them to be bad or good, all the more that humans have a hard time enough defining clearly what is good or bad, let alone machines.

This overly "optimistic" talk just sounds like marketing babble, more so than an educated opinion. Sorry "Sir'.

Computer scientist rather than software engineer..

[fuzzyfuzzyfungus]

The theory that enormously complex software systems specifically designed to be capable of novel behavior definitely won't go off the rails seems like something that you could only embrace if you've never actually interacted with real software as written by real people.

There is also the...minor...problem that "have a human in the loop deciding" will be a feature that will have to be implemented in software; and we definitely don't have a history of either unhelpful program output or unpleasant reaction to malformed inputs; so that will go well.

So, basically...

[flargleblarg]

Killer robots will exist.

They already exist

[gweihir]

Every landmine qualifies as a very low capability "killer robot". The insane harm landmines to around the globe is a good indicator that there are by far enough people with power and money and absolutely no qualms about maiming and killing innocent bystanders and civilians in general. Hence we will definitely see killer robots of much higher capabilities, unless we get the fucked-up part of the human race under control that simply cannot stop killing others and using violence to solve disagreements.

Not easy to avoid in an arms race

[Kjella]

Side A builds robots that can't fire without human control. Side B builds jammers. Side A decides robot soldiers need to be able to act in "self-defense". Side B puts civilians in harm's way. Side A decides they need "smart robots" who can tell friend from foe by themselves. Or that we need tighter coordination between light arms, heavy arms, air support, putting down covering fire for advancing troops etc. with so tight margins that it can't be done on manual. If you're being mauled to death by a perfectly coordinated fully automatic enemy you will fight fire with fire. Maybe you're creating the world where we'll lose control of our Terminators. But in the short term if you're not playing the game you're going to lose right now.