Yoshua Bengio, a Grand Master of Modern AI, is Worried About Its Future (technologyreview.com)

Yoshua Bengio is a grand master of modern artificial intelligence. Alongside Geoff Hinton and Yan LeCun, Bengio is famous for championing a technique known as deep learning that in recent years has gone from an academic curiosity to one of the most powerful technologies on the planet. Here's an excerpt from an interview he gave to MIT Technology Review: MIT TR: What do you make of the idea that there's an AI race between different countries?
Bengio: I don't like it. I don't think it's the right way to do it. We could collectively participate in a race, but as a scientist and somebody who wants to think about the common good, I think we're better off thinking about how to both build smarter machines and make sure AI is used for the well-being of as many people as possible.

MIT TR: Are you worried about just a few AI companies, in the West and perhaps China, dominating the field of AI?
Bengio: Yes, it's another reason why we need to have more democracy in AI research. It's that AI research by itself will tend to lead to concentrations of power, money, and researchers. The best students want to go to the best companies. They have much more money, they have much more data. And this is not healthy. Even in a democracy, it's dangerous to have too much power concentrated in a few hands.

MIT TR:There has been a lot of controversy over military uses of AI. Where do you stand on that?
Bengio: I stand very firmly against.
MIT TR: Even non-lethal uses of AI?
Bengio: Well, I don't want to prevent that. I think we need to make it immoral to have killer robots. We need to change the culture, and that includes changing laws and treaties. That can go a long way. Of course, you'll never completely prevent it, and people say, "Some rogue country will develop these things." My answer is that one, we want to make them feel guilty for doing it, and two, there's nothing to stop us from building defensive technology. There's a big difference between defensive weapons that will kill off drones, and offensive weapons that are targeting humans. Both can use AI.
MIT TR: Shouldn't AI experts work with the military to ensure this happens?
Bengio: If they had the right moral values, fine. But I don't completely trust military organizations, because they tend to put duty before morality. I wish it was different.

Comments

what an idiot

[zlives]

I wish he had a library card and could check out a history book...

"build smarter machines and make sure AI is used for the well-being of as many people as possible"
"itself will tend to lead to concentrations of power, money, and researchers"
"we need to make it immoral to have killer robots"

sigh...
victims

Re:what an idiot

[Luckyo]

No, just people who specialize so deeply in their field of choice, they're utterly ignorant of everything else.

You know, scientists.

Re:what an idiot

[phantomfive]

"Grand Master of AI" is a title I have no filed along side "Technology Futurist." An extremely useful title, it tells you they will just babble mindlessly.

Re:what an idiot

[ceoyoyo]

"I wish he had a library card and could check out a history book..."

I wonder just how much history you've studied. One of my favourite courses in university was a double course entitled "The History of Human Conflict." The theme that emerged is that war at the state level is a surprisingly stylized, rigidly rule bound activity. The use of "dishonorable" weapons is highly suppressed. Which probably explains why any of us are still alive. It's also very highly conservative. Military officers study history extensively, and tradition is extremely important; "it takes the Navy three years to build a ship. It will take three hundred years to build a new tradition."

Stephen Pinker points out in "The Better Angels of Our Nature" that even guerilla and terrorist organizations violate accepted norms at their peril. Such organizations require popular support, and when they commit atrocities they tend to lose that support. The Red Brigade and IRA being prominent examples.

By the way, I know Bengio. He's very much not an idiot.

Re:what an idiot

[ceoyoyo]

"And even if we DO need to take military action, we can send the robots in instead of your kids."

You've put your finger on the likely problem. The rate of violence at all levels has been decreasing exponentially in the world for at least the last 500 years or so (actually exponentially, backed up by numbers and stats). Much of this reduction, at the state level, is associated with engagement and interdependence on other nations. You wage war for economic or political gain. If waging war is expensive because you lose all your trade benefits, you're less likely to do it.

Killer robots remove one of the major political costs, particularly in a democracy. Wars are unpopular with the citizenry, especially when body bags start coming back.

Ma Feelings

[DigiShaman]

"Some rogue country will develop these things." My answer is that one, we want to make them feel guilty for doing it..."

When dealing with an adversary - NEVER project your own morality on them. It's not a given that their values are the same as yours! Proceed on what you know to be truthful, not what you want it to be.

Chinese AI Goes Haywire

[dryriver]

Analysis: China's Communist Party Members Have Terrible Haircuts. Action Taken By AI: Inspired by Wintermute in William Gibson's Neuromancer, AI composes a mighty dub called "Tiananmen Square Boogie", generates a photorealistic-looking 3D video for it and posts it on Youtube. Disasterous Real World Consequence: The video proves so viral on Youtube that billions watch it over and over again. K-Pop crashes in popularity, and male K-Pop singers can no longer afford quality makeup, hair gel and earrings, making them very, very sad. South Korea gets very pissed with China's AI, and unleashes its own AI on China. The Korean AI hacks into Apple's manufacturing plants in China, and causes them to manufacture iPads and iPhones with a yellow Banana logo. iTunes can only play Kung-Fu movies on these devices. Siri also sounds like a transvestite with a bad cold now. Apple's stock price crashes on the Nasdaq. America gets pissed, unleashes its own AI on China AND South Korea. This in turn causes North Korea to become tittilated and take advantage of the situation by unleashing its Bang-Dong-Bong Viral Propaganda Generator AI on everybody else. Bang-Dong-Bong fills Youtube with super-viral communist anthems, causing half the world's youth to become Communism admiring messes. This in turn pisses off the Europeans, and they unleash their........

Re: Who wins?

[jd]

The country that sidesteps the problem wins.

Weapons are expensive to produce. Wars are expensive to fight. As Sun Tzu notes, the best strategy is not to fight.

If you don't produce killer AI but put your resources into out-evolving humans and AIs as individual constructs, you can't be beaten by either and can walk right over those degraded and exhausted by fighting.

That's who wins.

Re:Not really

[jd]

No, being armed simply means you're the one the gunman will shoot first to avoid being shot at. You cannot deter a gunman. That is why America has more than ten times as many as Britain, despite having only four times the population and five tines the guns.

It's also why you see more gunmen in America in places where people are more heavily armed.

It's why have a go heroes either get killed or kill innocent bystanders but rarely ever stop an attack.

If having weapons worked, nobody would fight wars. They'd turn up, count weapons, and the one with the least would go home. Doesn't happen.

Guns don't deter. That's why gun crime in countries that don't have this hero mindset have far lower gun crime, regardless of guns per capita. It is the wild west myth, the heroic gunslinger, that causes people to die.