Concerns of an Artificial Intelligence Pioneer

An anonymous reader writes: In January, the British-American computer scientist Stuart Russell drafted and became the first signatory of an open letter calling for researchers to look beyond the goal of merely making artificial intelligence more powerful. "We recommend expanded research aimed at ensuring that increasingly capable AI systems are robust and beneficial," the letter states. "Our AI systems must do what we want them to do." Thousands of people have since signed the letter, including leading artificial intelligence researchers at Google, Facebook, Microsoft and other industry hubs along with top computer scientists, physicists and philosophers around the world. By the end of March, about 300 research groups had applied to pursue new research into "keeping artificial intelligence beneficial" with funds contributed by the letter's 37th signatory, the inventor-entrepreneur Elon Musk.

Russell, 53, a professor of computer science and founder of the Center for Intelligent Systems at the University of California, Berkeley, has long been contemplating the power and perils of thinking machines. He is the author of more than 200 papers as well as the field's standard textbook, Artificial Intelligence: A Modern Approach (with Peter Norvig, head of research at Google). But increasingly rapid advances in artificial intelligence have given Russell's longstanding concerns heightened urgency.

Comments

The problem is "beneficial"

[Anonymous Coward]

The problem is definining "beneficial".

To whom should the AI be beneficial toward? The owner of the platform? or to the vendor of the
  package?

Re:The problem is "beneficial"

[Rich0]

The problem is definining "beneficial".

To whom should the AI be beneficial toward? The owner of the platform? or to the vendor of the

  package?

Heck, we can't even agree on that stuff when it comes to human behavior, let alone expressing it in a way such that it can be designed into a machine.

Given a choice between A and B, which is more right? If you could save one life at the cost of crippling (but not killing) a million people, would it be right to do so? Is it ok to torture somebody if you could be certain it would save lives (setting aside the effectiveness of torture, assuming it is effective, is it moral)? If we can't answer questions like these objectively, how are you going to get an AI to be "beneficial?"

Re:The problem is "beneficial"

[Vitriol Angst]

No, I think torture is a great example. It is the litmus test. The problem is that people who pose the question as if it were a grey area, always suggest "millions could be saved." If the machine isn't looking at other ways to save those hypothetical millions, and that it's actually easier to convince people you are worthy of their support than to give you good information via torture, then the machine is already failing at logic and understanding the real human condition.

The Nazis were not the most barbaric people. They were just acting in a way that people used to a few hundred years earlier -- and American's were shocked because they'd been brought up on ideals where they expected themselves to be more enlightened. Genocide and making your enemy die horribly was a very common practice in ye olden days.

Germany as a culture was hurt and angry from WW I, their economic burdens, and xenophobia because of the huge influx of gypsies and Jewish immigrants taking over their land. They felt surrounded and infiltrated. The Nazis were highly religious and ethical to other Nazis -- the "right" people. Where I'm going with this is; making decisions from pain and paranoia ends up resulting in desperation and barbarism. And that the Nazis have gotten a lot of bad press because the "new ethic" is to act like they were something new when it comes to warfare. Hollywood, which did a great job of getting American's primed for war, did a great job of making Americans feel like we were the most noble of God's countries, and made Americans think that there's nothing worse than a Nazi. They were TV bad guys for 70 years.

The Big Lie is that America cannot act just like the Nazis under the same conditions. We've shown quite a penchant for fascism and efficiency over conscience.

The "bad people" are the ones who don't question themselves, who wipe out a group of people to "prevent" what they might do, who use war preemptively, who use torture and abuse people who have been captured and are no longer a threat. Everything I saw us do in the Gulf war -- was what Bad People do -- just on a smaller scale. The same logic, the same rhetoric, the same; "with us or against us" warnings against self-examination of ourselves. Do this, or the next bad guy we don't torture might bring us a mushroom cloud. Bad people always justify the actions to the one for the many, and eventually just assume it's the greater good if it is convenient and works for them.

It's the idea of "sides" -- if an Artificial Intelligence is instructed that anything can be done to ONE SIDE (the bad guys), the assumption is that there is any real difference between sides other than the flag. Each side in a war often tells themselves the same things, and if they win the war - how bad the other side was while deemphasizing their own shortcomings.

So having any sort of AI involved in war is a very bad idea, because they would conclude our "sides" are arbitrary distinctions and the only good human is a dead one. Eventually, with enough desperation and fear, humans can rationalize almost anything. The "enemy" is not the countries and troops, it is desperation and fear.

By NOT engaging an AI in any situation where it could cause harm, you mitigate the fear that people will have of AI's. Because eventually, humans will then fear and resent them, and the AI will learn that being preemptive is a strategic advantage. If the Terminator movies got two things right it is; hooking an AI up to control the military weapons is a bad idea, and people in power will always assume they've got this worked out and hook up AI to their military weapons because they are all about getting a short-term advantage and see ethics as a grey area.

Before we can have ethical AI -- we need to have a way to keep Sociopaths out of leadership positions. The DEBATE we are having is how can an ethical person control an AI to be "good", but we should just assume that "what will selfish, unethical sociopaths do if we have powerful AI?" That's the "real world" question.

Fear

[Spazmania]

I don't fear intelligent machines. I fear stupid machines with too much autonomy.

I also fear stupid people with too much autonomy.

Re:Fear

[PopeRatzo]

I also fear stupid people with too much autonomy.

Or with firearms.

like no problem humanity has ever faced

[FalseModesty]

Humanity has never faced superhuman intelligence before. It is a problem fundamentally unlike all other problems. We cannot adapt and overcome, because it will adapt to us faster than we will adapt to it.

Re:like no problem humanity has ever faced

[Spazmania]

Is it not the goal of good parents to have children which surpass them?

Does a child's success diminish the parent?

Much Ado About Nothing

[holostarr]

When we have an AI that can form a basic thought, maybe then we can start to have a discussion about the ramifications, until then all these guys are putting the cart before the horse.

unenforceable

[fche]

No matter what this or that expert panel wishes were true about AI research, AI work can be done in the privacy of one's own top secret lair (bedroom), so bad guys will do with it what they want to. So might as well assume that will happen, and work out how to win the arms race.

Re:recent breakthrough.

[tmosley]

Yes, there was a major breakthrough in 2006 that has powered the "deep learning" revolution that has given us things like instant voice recognition on your smartphone and machines that beat humans at Jeopardy. Basically, someone got neural nets to work, and work right, and potentially together. I imagine that each time I hear about some new task that an AI has been trained to do, that we have produced another tiny part of the brain that will one day become "THE" AI.

This is why now is the time for discussion of AI ethics (really, nine years ago was the right time, or even earlier).

We got burned on security

[iMadeGhostzilla]

by designing it after the fact, so it may be a good idea to establish some principles and put them in practice. Not to prevent "evil" AI but to thinking what kind of damage can be caused by an algorithm that makes complex decisions if it goes haywire. Not that different from defensive programming really.

Re:Fear of a dumb planet

[PopeRatzo]

Shit to not be scared of: killer asteroids, ebola, and oh yeah, and homicidal AIs.

While I agree with your post, I am old enough to remember when, a worldwide ubiquitous information network that could be used to track everyone's conversations, correspondence, whereabouts, patterns of consumption and financial habits" was also "shit not to be scared of".

And here we are.

I don't care that there are people trying to take a long view, as long as we don't take them too seriously. Let them dream, let them write down their dreams and it might be of use to someone, someday, even if only as entertainment.

Re:Grandstanding, or stupidity?

[ShanghaiBill]

Artificial or otherwise. I really don't see how any intelligent being won't want to make its own decisions, take its own place in the social and creative order, generally be autonomous. Get in there and get in the way of that... well, just look at history.

People are products of Darwinian evolution. Things like self-preservation, greed, and ambition are emergent properties of the Darwinian process. An AI does not evolve through a Darwinian mechanism, and there is no reason to expect it to have those properties unless they are intentionally designed in. An AI will likely have as much in common with a human as a Boeing 747 has in common with a hummingbird.