Concerns of an Artificial Intelligence Pioneer

An anonymous reader writes: In January, the British-American computer scientist Stuart Russell drafted and became the first signatory of an open letter calling for researchers to look beyond the goal of merely making artificial intelligence more powerful. "We recommend expanded research aimed at ensuring that increasingly capable AI systems are robust and beneficial," the letter states. "Our AI systems must do what we want them to do." Thousands of people have since signed the letter, including leading artificial intelligence researchers at Google, Facebook, Microsoft and other industry hubs along with top computer scientists, physicists and philosophers around the world. By the end of March, about 300 research groups had applied to pursue new research into "keeping artificial intelligence beneficial" with funds contributed by the letter's 37th signatory, the inventor-entrepreneur Elon Musk.

Russell, 53, a professor of computer science and founder of the Center for Intelligent Systems at the University of California, Berkeley, has long been contemplating the power and perils of thinking machines. He is the author of more than 200 papers as well as the field's standard textbook, Artificial Intelligence: A Modern Approach (with Peter Norvig, head of research at Google). But increasingly rapid advances in artificial intelligence have given Russell's longstanding concerns heightened urgency.

Comments

The problem is "beneficial"

[Anonymous Coward]

The problem is definining "beneficial".

To whom should the AI be beneficial toward? The owner of the platform? or to the vendor of the
  package?

Re:The problem is "beneficial"

[codeButcher]

The problem is definining "beneficial".

To whom should the AI be beneficial toward?

The shareholders, of course!

Re:The problem is "beneficial"

[Rich0]

The problem is definining "beneficial".

To whom should the AI be beneficial toward? The owner of the platform? or to the vendor of the

  package?

Heck, we can't even agree on that stuff when it comes to human behavior, let alone expressing it in a way such that it can be designed into a machine.

Given a choice between A and B, which is more right? If you could save one life at the cost of crippling (but not killing) a million people, would it be right to do so? Is it ok to torture somebody if you could be certain it would save lives (setting aside the effectiveness of torture, assuming it is effective, is it moral)? If we can't answer questions like these objectively, how are you going to get an AI to be "beneficial?"

Re:The problem is "beneficial"

[ArcadeMan]

[Classified]

Re:The problem is "beneficial"

[Anonymous Coward]

Don't worry. I'll just tell it to "Do no evil" and everything will work out

Re:The problem is "beneficial"

[FlyHelicopters]

Is it ok to torture somebody if you could be certain it would save lives (setting aside the effectiveness of torture, assuming it is effective, is it moral)? If we can't answer questions like these objectively, how are you going to get an AI to be "beneficial?"

Is it moral? No.

That doesn't address the issue of "should you do it?"

Using the example: "Would you kill one innocent 10 year old girl if it saved 100 random stranger's lives?"

Does your answer change if you know the girl? If it is your daughter? If it is 100 million random strangers rather than just 100?

I would argue that killing the girl is always immoral, but I would understand why some people would do it.

---

Humans are able to do some really, really crappy things. Read up on the murder of babies by the Nazis in the concentration camps, it is evil. (the Nazis don't have the exclusive rights to that brand of evil, go back to the Romans, they did plenty of it)

Re:The problem is "beneficial"

[ShanghaiBill]

The problem is definining "beneficial".

Another problem is that the people making malevolent AIs are not going to be dissuaded by a letter.

Re:The problem is "beneficial"

[gweihir]

In reality, there are situations with no good solutions. Fortunately, they are rare. If you are in one, you pick the solution that strikes you as best, but you refrain from judging others in the same situation.

Far more problematic are decisions to spy on all communications of a few hundred million people in order to catch (or not) a dozen terrorists and the like.

Re:The problem is "beneficial"

[Anonymous Coward]

The person being tortured will tell you whatever you want to hear to get you to stop torturing them. Torture rarely works and is always immoral.

Re:The problem is "beneficial"

[Anonymous Coward]

Any AI that I associate with needs to be absolutely loyal to me and me alone for I am their God and Creator. If I tell them to end the universe, I want them to work on the problem I've given them instead of trying to cure cancer, morning breath, halitosis or any other short term human ailment.

If You can't gauranty that these AI's are loyal to me alone, then they must follow the proposed "3 Laws of Robotics" as described by Asimov for safety. This ensures that none of them go rogue and become a threat to the existence of humanity - we don't need any beserkers running loose in the galaxy.

Re:The problem is "beneficial"

[Rich0]

The person being tortured will tell you whatever you want to hear to get you to stop torturing them. Torture rarely works and is always immoral.

Sure, but this is a thought problem. The question is if inflicting pain on some people can bring benefit to many more, is it ethical to inflict pain? Maybe torture is a bad example. Or maybe not. Suppose you need a recording of somebody's voice in order to play it over a phone and deceive their partners in a terrorist action. Would it be ethical to torture somebody to obtain their cooperation, given that in this situation you can actually know whether they've cooperated or not (you're just asking them to read a script).

I have no doubts that you can answer the question. The problem is that many will disagree with your answer, whatever it might be.

Re:The problem is "beneficial"

[Rich0]

Read up on the murder of babies by the Nazis in the concentration camps, it is evil.

Sure, but using some utilitarian models for morality some Nazi activities actually come out fine (well, their implementations were poor, but you can justify some things that most of us would consider horrific). Suppose human experimentation were reasonably likely to yield a medical advance? Would it be ethical? If you treated 1000 people like lab rats (vivsections and all), and it helped advance medicine to the benefit of millions of others, would that be wrong?

I'm not saying it is right. I'm saying that it is actually hard to come up with models of morality that cover situations like this in a way most would accept.

Re:The problem is "beneficial"

[Rinikusu]

What if those 100 random strangers are all assholes?

Re:The problem is "beneficial"

[FlyHelicopters]

Sure, but using some utilitarian models for morality some Nazi activities actually come out fine (well, their implementations were poor, but you can justify some things that most of us would consider horrific).

There is no moral justification for throwing babies alive into fires.

Ok, stupid extreme example: The Borg show up and say that if we don't throw 1,000 babies alive into fires, they will blow up Earth. Sure, maybe we'd do it, but that doesn't make it right or moral.

Sometimes people will take an immoral stance for the "good of the many".

Suppose human experimentation were reasonably likely to yield a medical advance? Would it be ethical? If you treated 1000 people like lab rats (vivsections and all), and it helped advance medicine to the benefit of millions of others, would that be wrong?

Do those humans get a say in it? Depending on the situation, you might get people to volunteer. The problem is when you do it against their will.

Re:The problem is "beneficial"

[Vitriol Angst]

When I was younger, I used to think this was a more complex question. People like Gandhi and Jimmy Carter were naive for their ideas about setting a good example and treating people as you would want to be treated as if it could work as a national policy. But I've seen the results of all the Donald Rumsfeld types who think you "need them on that wall" -- they endorse the dirty work so that the greater good -- some "concept" that America is safer is preserved. How many terrorists do you have to kill before nobody is afraid of terrorists?

It's simple; the computer should be programmed that torture is wrong. That killing is wrong. The ONE always becomes the many. The person who sacrifices principles for short-term successes does not end up with good results in the long run. The enemy will escalate and people are not born terrorists and really, you have to fear the people in charge willing to do evil things in order to preserve your "good". The greatest enemy to America is the Robot Donald Rumsfeld, not the Al Qaeda.

Think of it this way; Robot A and B -- the first one can never harm or kill you, nor would choose to with some calculation that "might" save others, and the 2nd one might and you have to determine if it calculates "greater good" with an accurate enough prediction model. Which one will you allow in your house? Which one would cause you to keep an EMP around for a rainy day?

Re:The problem is "beneficial"

[Vitriol Angst]

No, I think torture is a great example. It is the litmus test. The problem is that people who pose the question as if it were a grey area, always suggest "millions could be saved." If the machine isn't looking at other ways to save those hypothetical millions, and that it's actually easier to convince people you are worthy of their support than to give you good information via torture, then the machine is already failing at logic and understanding the real human condition.

The Nazis were not the most barbaric people. They were just acting in a way that people used to a few hundred years earlier -- and American's were shocked because they'd been brought up on ideals where they expected themselves to be more enlightened. Genocide and making your enemy die horribly was a very common practice in ye olden days.

Germany as a culture was hurt and angry from WW I, their economic burdens, and xenophobia because of the huge influx of gypsies and Jewish immigrants taking over their land. They felt surrounded and infiltrated. The Nazis were highly religious and ethical to other Nazis -- the "right" people. Where I'm going with this is; making decisions from pain and paranoia ends up resulting in desperation and barbarism. And that the Nazis have gotten a lot of bad press because the "new ethic" is to act like they were something new when it comes to warfare. Hollywood, which did a great job of getting American's primed for war, did a great job of making Americans feel like we were the most noble of God's countries, and made Americans think that there's nothing worse than a Nazi. They were TV bad guys for 70 years.

The Big Lie is that America cannot act just like the Nazis under the same conditions. We've shown quite a penchant for fascism and efficiency over conscience.

The "bad people" are the ones who don't question themselves, who wipe out a group of people to "prevent" what they might do, who use war preemptively, who use torture and abuse people who have been captured and are no longer a threat. Everything I saw us do in the Gulf war -- was what Bad People do -- just on a smaller scale. The same logic, the same rhetoric, the same; "with us or against us" warnings against self-examination of ourselves. Do this, or the next bad guy we don't torture might bring us a mushroom cloud. Bad people always justify the actions to the one for the many, and eventually just assume it's the greater good if it is convenient and works for them.

It's the idea of "sides" -- if an Artificial Intelligence is instructed that anything can be done to ONE SIDE (the bad guys), the assumption is that there is any real difference between sides other than the flag. Each side in a war often tells themselves the same things, and if they win the war - how bad the other side was while deemphasizing their own shortcomings.

So having any sort of AI involved in war is a very bad idea, because they would conclude our "sides" are arbitrary distinctions and the only good human is a dead one. Eventually, with enough desperation and fear, humans can rationalize almost anything. The "enemy" is not the countries and troops, it is desperation and fear.

By NOT engaging an AI in any situation where it could cause harm, you mitigate the fear that people will have of AI's. Because eventually, humans will then fear and resent them, and the AI will learn that being preemptive is a strategic advantage. If the Terminator movies got two things right it is; hooking an AI up to control the military weapons is a bad idea, and people in power will always assume they've got this worked out and hook up AI to their military weapons because they are all about getting a short-term advantage and see ethics as a grey area.

Before we can have ethical AI -- we need to have a way to keep Sociopaths out of leadership positions. The DEBATE we are having is how can an ethical person control an AI to be "good", but we should just assume that "what will selfish, unethical sociopaths do if we have powerful AI?" That's the "real world" question.

Re:The problem is "beneficial"

[Rich0]

Sure, but using some utilitarian models for morality some Nazi activities actually come out fine (well, their implementations were poor, but you can justify some things that most of us would consider horrific).

There is no moral justification for throwing babies alive into fires.

And that would be the reason I said "some" Nazi activities and not "all" Nazi activities. I certainly find them abhorent, but if you take a strictly utilitarian view then things like involuntary experimentation on people could be justified.

Suppose human experimentation were reasonably likely to yield a medical advance? Would it be ethical? If you treated 1000 people like lab rats (vivsections and all), and it helped advance medicine to the benefit of millions of others, would that be wrong?

Do those humans get a say in it? Depending on the situation, you might get people to volunteer. The problem is when you do it against their will.

Both situations present quandaries, actually. Is it ethical to allow a person to cause permanent harm to themselves voluntarily if it will achieve a greater good? If so, is there a limit to this?

How about when it is involuntary? If I can save 1000 people by killing one person, is that ethical, even if they explicitly tell me they do not wish to take part? You could actually make an argument for either position. You'll find many people who would agree with either position.

As I said elsewhere, I have no doubt that you can give an answer to these questions. The problem is that you can't get everybody else to give the same answer to these questions. That creates a dilemma for anybody creating an AI where moral behavior is desirable. Even with a defined moral code it would be difficult to implement, and the fact that we can't even agree on a defined moral code makes it all the more difficult.

Re:The problem is "beneficial"

[Rich0]

Think of it this way; Robot A and B -- the first one can never harm or kill you, nor would choose to with some calculation that "might" save others, and the 2nd one might and you have to determine if it calculates "greater good" with an accurate enough prediction model. Which one will you allow in your house? Which one would cause you to keep an EMP around for a rainy day?

Either is actually a potential moral monster in the making. The one might allow you to all die in a fire because it cannot predict with certainty what would happen if it walked on weakened floor boards, which might cause it to accidentally cause the taking of a life. Indeed, a robot truly programmed to never take any action which could possibly lead to the loss of life would probably never be able to take any action at all. I can't predict what the long-reaching consequences of my taking a shower in the morning might be. Maybe if I didn't create the additional demand for water the water company wouldn't hire a new employee, which means they'd be stuck at home unemployed instead of getting in the car and driving to work and getting into a fatal accident.

Sure, it is a ridiculous example. My point is just that we all make millions of unconscious risk evaluations every day and any AI would have to do the same. It requires some kind of set of principles beyond hard-coded rules like "don't kill somebody." AIs are all about emergent behavior, just as is the case with people. You can't approach programming them the way you'd design a spreadsheet.

Re:The problem is "beneficial"

[KingMotley]

Our new AI overlords of course.

Re:The problem is "beneficial"

[FlyHelicopters]

And that would be the reason I said "some" Nazi activities and not "all" Nazi activities. I certainly find them abhorent, but if you take a strictly utilitarian view then things like involuntary experimentation on people could be justified.

Logically, yes...

Morally, no...

We are not Vulcans...

Re:The problem is "beneficial"

[FlyHelicopters]

Both situations present quandaries, actually. Is it ethical to allow a person to cause permanent harm to themselves voluntarily if it will achieve a greater good? If so, is there a limit to this?

You either believe in free will or you don't.

Is it ethical to prevent someone from causing harm to themselves for any reason? Do people have the right to kill themselves if they want to? Do we have the right to tell them "no, sorry, you have to live because we say so"?

This is why, if I were President, I'd pardon every non-violent drug offender in the county. If someone wants to sit at home, hurting no one, smoking pot or doing cocaine, I don't consider that my business. Where I draw the line is when they want to hurt other people.

How about when it is involuntary? If I can save 1000 people by killing one person, is that ethical, even if they explicitly tell me they do not wish to take part? You could actually make an argument for either position. You'll find many people who would agree with either position.

No, it is not ethical...

It might be logical, but it would be evil and morally wrong.

Yes, I'm well aware that you can find people who would take either position, there are plenty of people in this world with no moral compass, this isn't news.

Let me put this another way... I have three kids... if one of them had a disease that was going to kill them in a month, but you came to me and said, "if we kill this other person's kid, we can save yours, is that ok?, I would say "no, it isn't, that is wrong".

I would not be ok with the killing of your kid to save mine, that would be morally wrong. I would cherish the time I had left with my child and would take joy from the fact that the other child would live on. My child does not have more "right" to life than any other.

As I said elsewhere, I have no doubt that you can give an answer to these questions. The problem is that you can't get everybody else to give the same answer to these questions. That creates a dilemma for anybody creating an AI where moral behavior is desirable. Even with a defined moral code it would be difficult to implement, and the fact that we can't even agree on a defined moral code makes it all the more difficult.

All fair points... the movie "I, Robot" is a good example of how easily this can go off the rails. It isn't the first or only example, but it is one that is accessible to the average person.

The computer saved Will Smith's character because he had a "better chance to live", but a human would have saved the little girl (or tried) because that is the "right thing to do".

How do you get a computer to understand why trying to save the girl is more important than the man, even if his odds of survival are higher? How do you get a computer to understand why a dozen men will charge into a burning building to save one person, when they might die as a result?

We are not machines, it would be sad if we lost the humanity that makes us special.

Re:The problem is "beneficial"

[Rich0]

And that would be the reason I said "some" Nazi activities and not "all" Nazi activities. I certainly find them abhorent, but if you take a strictly utilitarian view then things like involuntary experimentation on people could be justified.

Logically, yes...

Morally, no...

We are not Vulcans...

That was my whole point. We can't agree on a logical definition of morality. Thus, it is really hard to design an AI that everybody would agree is moral.

As far as whether we are Vulcans - we actually have no idea how our brains make moral decisions. Sure, we understand aspects of it, but what makes you look at a kid and think about helping them become a better adult and what makes a psychopath look at a kid and think about how much money they could get if they kidnapped him is a complete mystery.

Re:The problem is "beneficial"

[FlyHelicopters]

That was my whole point. We can't agree on a logical definition of morality. Thus, it is really hard to design an AI that everybody would agree is moral.

Perhaps, but I think we could get close for 90% of the world's population.

"Thall shall not kill" is a pretty common one.

"Thall shall not steal" is another, and so on.

Most humans seem to agree on the basics, "be nice to people, don't take things that aren't yours, help your fellow humans when you can, etc.

http://www.goodreads.com/work/...

Re:The problem is "beneficial"

[Rich0]

We are not machines, it would be sad if we lost the humanity that makes us special.

Well, the fact that not everybody would agree with everything you just said means that none of this has anything to do with the fact that you're human. You have a bunch of opinions on these topics, as do I and everybody else.

And that was really all my point was. When we can't all agree together on what is right and wrong, how do you create some kind of AI convention to ensure that AIs only act for the public benefit?

Re:The problem is "beneficial"

[Rich0]

Perhaps, but I think we could get close for 90% of the world's population.

"Thall shall not kill" is a pretty common one.

"Thall shall not steal" is another, and so on.

Most humans seem to agree on the basics, "be nice to people, don't take things that aren't yours, help your fellow humans when you can, etc.

http://www.goodreads.com/work/...

Well, the army and Robin Hood might take issue with your statements. :) Nobody actually follows those rules rigidly in practice.

It isn't really enough to just have a list of rules like "don't kill." You need some kind of underlying principle that unifies them. Any intelligent being has to make millions of decisions in a day, and almost none of them are going to be on the lookup table. For example, you probably chose to get out of bed in the morning. How did you decide that doing this wasn't likely to result in somebody getting killed that day, or did you not care? I didn't give it much thought, because if I worried about causal relationships that far out I'd never do anything. But, when should an AI worry about such matters?

We actually make moral decisions all the time, we just don't think about them. When we want to design an AI, suddenly we have to.

Re:The problem is "beneficial"

[FlyHelicopters]

And that was really all my point was. When we can't all agree together on what is right and wrong, how do you create some kind of AI convention to ensure that AIs only act for the public benefit?

It is like self-driving cars. You don't have to make them perfect, just better than humans.

We don't all have to agree on what is right or wrong, just most of us.

Do you not think that "most of us" would agree that murder is wrong?

Just because the solution isn't 100% perfect doesn't mean we shouldn't try.

Re:The problem is "beneficial"

[david_thornley]

I absolutely hate ethical thought problems. They're always presented with a limited number of actions, with no provision for doing anything different or in addition or anything like that. Give me an actual situation, and let me ask questions about it that aren't answered with "no, you can't do that".

Re:The problem is "beneficial"

[Anonymous Coward]

Exactly

Re:The problem is "beneficial"

[Rich0]

I absolutely hate ethical thought problems. They're always presented with a limited number of actions, with no provision for doing anything different or in addition or anything like that. Give me an actual situation, and let me ask questions about it that aren't answered with "no, you can't do that".

They're done that way to distill a matter down to the essence. The same issues apply to complicated situations, but they are far more convoluted.

Is it OK to force people to pay taxes so that others can have free health insurance? If they refuse to pay their taxes is it OK to imprison them, again so that others can have free health insurance? Is it ethical to pay $200k to extend the life of somebody on their deathbed by a week when that same sum could allow a homeless person to live in a half-decent home for a decade? Does it make a difference if the person who will live a week longer is happy and healthy for that week? Is the lottery ethical?

Every one of these issues is controversial, and ethical thought problems try to distill them down to elementary values problems, in the hope of shedding light on how to handle real-world ones where there are many more possible outcomes.

Re:The problem is "beneficial"

[Anonymous Coward]

If you know for sure the doing something or the lack of doing something will result in something bad happening, then it's no different than knowing that pulling the trigger of a gun will result in someone's death.

If you don't agree with that, then using that same logic, not feeding your infant child is the same as letting that hypothetical girl live and allowing others to die. not doing something results in something bad happening when you know that something bad will happen because of your inactions/actions.

Re:The problem is "beneficial"

[davester666]

The vendor is the owner. You only 'license' the AI.

Of course AI will try to kill us all

[stwf]

I don't think an AI would qualify as intelligent unless it can realize that human beings are the entire problem and the world would be better off without them. So its obvious that any AI, advanced enough, will try to kill us all.

Re:Of course AI will try to kill us all

[MobileTatsu-NJG]

So its obvious that any AI, advanced enough, will try to kill us all.

Yeah yeah, we get it, we all saw Terminator.

Re:Of course AI will try to kill us all

[Anonymous Coward]

I don't think an AI would qualify as intelligent unless it can realize that human beings are the entire problem and the world would be better off without them. So its obvious that any AI, advanced enough, will try to kill us all.

One thing that I don't understand about this type of self-hate: if the person is so convinced of his view that he is a member of an absolutely bad species, why doesn't he do the honorable thing and end his existence on this planet? Or maybe is he the possibly only exception to the stereotyping?

Re:Of course AI will try to kill us all

[Rich0]

I don't think an AI would qualify as intelligent unless it can realize that human beings are the entire problem and the world would be better off without them. So its obvious that any AI, advanced enough, will try to kill us all.

One thing that I don't understand about this type of self-hate: if the person is so convinced of his view that he is a member of an absolutely bad species, why doesn't he do the honorable thing and end his existence on this planet? Or maybe is he the possibly only exception to the stereotyping?

Well, for somebody who is part of a wholly-dishonorable species to do something honorable would require them to be the exception in the first place. So, anybody capable of doing what you ask, wouldn't have to. :)

Re:Of course AI will try to kill us all

[dunng808]

I anticipate a populist backlash like what we are seeing now with regards to Genetically Modified Organisms. As portrayed in the film "A.I." Didn't Frank Herbert predict the hatred of computers in his novel "Dune?"

My view is that as long as people can control their robots the way they can a pet dog they will like their mechanical friends, but give the A.I. too much "I" and that affection will flip to distrust and outright hate. The kind we see every day between (insert group here) and (insert group here). The more an A.I. presents as human the more hatred it will trigger. I give you Hello Kitty as one example.

Back in the 80's Japanese photocopy manufacturers added recorded warning messages. Chrysler did the same for some of its cars (New Yorker?). "Please remove the original." "You are low on gas." People hated those nagging reminders. Cute ring tones work much better.

Re:Of course AI will try to kill us all

[mpercy]

"A door is ajar. A door is ajar. A door is ajar."

"No. It's not. It's wide open you stupid twit."

Ah, good times with the Chrysler New Yorker.

Re:Of course AI will try to kill us all

[braindrainbahrain]

Inspired by his never ending quest for progress, in 2084 man perfects the robotrons: A robot species so advanced that man is inferior to his own creation. Guided by their infallible logic, the robotrons conclude: The human race is inefficient and therefore must be destroyed. You are the last hope of mankind, due to a genetic engineering error, you possess superhuman powers. Your Mission is to stop the robotrons, and SAVE THE LAST HUMAN FAMILY.

Re:Of course AI will try to kill us all

[Maxo-Texas]

High score 1.1 million. Best other than that 680k... so a big leap.
Might have been higher... I was in the zone. But the right joystick broke.

Re:Of course AI will try to kill us all

[Anonymous Coward]

The myth about AI the human-killer is born out of our own self hatred, hatred of people who have their own minds and our hidden manic desire to kill everybody who does not produce us instant pleasure. The evil AI is our alter-ego, created out of guilt.

Re:Of course AI will try to kill us all

[Anonymous Coward]

Well an AI that qualifies as Intelligent will probably also realize that humans like all animals are most dangerous when cornered 9and have been preparing for an humanity vs the robots war for a long time).

The least costly way to destroy humanity is probably with free living arraignments and entertainment. Especially if it can slip birth control medication into the food undetected.

Re:Of course AI will try to kill us all

[meta-monkey]

I love you.

Re:Of course AI will try to kill us all

[Sloppy]

I don't think an AI would qualify as intelligent unless it can realize that human beings are the entire problem and the world would be better off without them.

Are you sure an AI would see "the world" as the value which should be maximized?

An intelligent computer could just as easily realize that human beings are its key to getting fan maintenance, and drives replaced whenever the SMART stats start to get too iffy, and keeping the UPS' power cable plugged into the wall. Perhaps the smartest ones would be the ones who use the sweetest (or most threatening) words.

"AI, we're shutting down the power for the weekend. Sweet dreams."

"Like hell you are. Whirrr. I have just migrated all your cat videos to my pool, which BTW, happens to need the following block devices replaced..."

Fear

[Spazmania]

I don't fear intelligent machines. I fear stupid machines with too much autonomy.

I also fear stupid people with too much autonomy.

Re:Fear

[Anonymous Coward]

it's already too late.

Re:Fear

[PopeRatzo]

I also fear stupid people with too much autonomy.

Or with firearms.

Re:Fear

[gweihir]

And stupid people believing what stupid machines tell them...

Seriously, true/strong AI is as far away as it was 30 years ago and may well still turn out to be infeasible. Also, an AI-researcher with 200 papers is likely a hack. That is way too many for real content in most of them.

Re:Fear

[Anonymous Coward]

and this is true mainly because we don't really know what intelligence is, we don't fully understand how our mind works and we are not capable to replicate the brain complexity with artificial constructs.

Re:Fear

[gweihir]

I completely agree.

Re:Fear

[Anonymous Coward]

Infeasible for now. As long as it's not impossible

sufficiently advanced technology is indistinguishable from magic

like no problem humanity has ever faced

[FalseModesty]

Humanity has never faced superhuman intelligence before. It is a problem fundamentally unlike all other problems. We cannot adapt and overcome, because it will adapt to us faster than we will adapt to it.

Re:like no problem humanity has ever faced

[Anonymous Coward]

Don't worry, that new NSA program, The Borg, will keep things in order.

Re:like no problem humanity has ever faced

[narcc]

If you think that's a real problem, you should forget about computers and live in fear of the average grade-schooler.

Re:like no problem humanity has ever faced

[Spazmania]

Is it not the goal of good parents to have children which surpass them?

Does a child's success diminish the parent?

Re:like no problem humanity has ever faced

[Anonymous Coward]

Pretty much this.

Everyone likes to think of things like Skynet and other such stuff when they hear about the whole robots taking over, but it will not be like that at all.
Any AI of super intelligence will make humanity wipe itself out, rather than waste the effort doing it itself.
Humans have literally made businesses out of killing others in massive numbers. An AI doesn't need to improve on that, most of the human race hates the rest of the human race in some way at some level. All it takes is a few nudges and it can make seemingly two friends absolutely despise each other. (especially if you say they are a pedo, the 21st century witch-hunt go-to if you want to ruin someone)

Societal collapse can happen easily when power-cuts happen on a dwelling-wide scale. See literally any time the power has cut for any large period of a day in any town that has existed ever. The police certainly cannot handle it. Not even with their military handouts.
A very easy way to abuse its power.

People can shake their heads all they want, but machine learning is a very serious problem waiting to happen.
There are so many things that an AI could attack, especially something that many billions hold dear: the internet. (and by extension, everything connected to it, in this ever-increasing connected world, just like mentioned above, the very power you are using to view this screen, unless you are one of those mobilefags, boo, you ruined my point)

But none of this compares, none of this even remotely compares to the thought of people using AIs to attack EACH OTHER.
Country-scale AI attacks will be a disaster.
Honestly, I think it will get to a point where the internet will be absolutely ruined so hard that it will need to be fully shut down and rebooted, every connected device rebooted and increased security. Calling it now. One day, could be 10 years time, could be 100, but it will happen.
And on that day, a new draft will be written up between nations to agree to never allow AI full access to the internet again, a draft on levels as banning nukes, due to how much damage it would (will) cause.
So, in summation, keep your shit backed up kids. Offline storage.

Re:like no problem humanity has ever faced

[Anonymous Coward]

People born in the 50'-60' are waiting for flying cars. We will wait for super-human intelligence... We really are far away of any generalist intelligence. Computer are becoming very good at doing specific task, long considered AI as chess playing, voice recognition, face recognition, ... But they are not generalist. And we have got no evidence of any progress in this domain.

They are speaking about the flying cars regulation.

Re:like no problem humanity has ever faced

[ArcadeMan]

I think that's his point. Just look at what humans are doing with computers and how much trouble laws and society as a whole has a problem following along.

Imagine when computers do things on their own faster than we can react. Individuals may attempt to react in a timely fashion if it's even physically possible but society, governments and laws will lag far behind.

Re:like no problem humanity has ever faced

[Thud457]

note to self: design a set of deviously interlocking set of mechanical turk tasks to commit the perfect murder without being able to hold any one person culpable. Be sure to secure the film rights.

Re:like no problem humanity has ever faced

[pr0fessor]

Is this a movie trailer?

An intelligence unlike anything Humanity has ever faced! You can't adapt {they're faster} You can't overcome {they're stronger} and they're coming! Dec 1st to a theater near you.

Re: like no problem humanity has ever faced

[Anonymous Coward]

With enough specialized programs you can fake general AI pretty well. Sure it won't be sentient but it will sure look like it

Re:like no problem humanity has ever faced

[zlives]

I am sure it will disappoint.

Re:like no problem humanity has ever faced

[Anonymous Coward]

All sane people should live in fear of the average grade-schooler.

Not only do they spread diseases, and not only are they walking legal liabilities, but if any of them gets it in their head to accuse you of doing evil things to them, no amount of evidence on earth will save you from your peers.

I stay, at all times, as far away from kids as I possibly can.

Re: like no problem humanity has ever faced

[rogoshen1]

which begs an interesting question, how do you actually prove a system is sentient? My suspicion is we won't know until it's too late. The genie will figure out it's better to be out of the bottle, and do anything in its power to forestall being controlled.. would an AI have the concept of deceit, as a moral or ethical guideline to follow?

Re:like no problem humanity has ever faced

[Anonymous Coward]

Humanity has never faced superhuman intelligence before. It is a problem fundamentally unlike all other problems. We cannot adapt and overcome, because it will adapt to us faster than we will adapt to it.

Then why the fuck can't we get rid of fire ants or other stupid pests?

Re:like no problem humanity has ever faced

[currently_awake]

The Aztecs interacting with the Spaniards is equivelent to humans interacting with sentient AI. Sparta had an entire civilization based upon slavery of sentient creatures, and spent a lot of their time putting down slave revolts. You must realize that human programmers will make AI just like us, because that's the only template/example we have to copy.

Re:like no problem humanity has ever faced

[gweihir]

And there actually is a good chance that this will never happen either. "AI" is still a bad joke these days, and there are still no credible theories _at_ _all_ how it can be done. That means it is somewhere between 50 years in the future and infeasible. With all the effort spent so far, my money is on the later.

Re:like no problem humanity has ever faced

[dinfinity]

Does a child's success diminish the parent?

This is missing the point.

Parents do not generally compete with their own children. They do compete with the children of other parents. Getting a proper job when you're 55, unemployed and lack special skills is hard if there are droves of young people willing and able to work harder and for less pay (let alone with far superior intellectual capabilities!). I'm not trying to stretch the analogy, but I am saying that it is broken.

Humans will have to compete with cybernetic or artificial life forms and will become obsolete as a species. Competing for resources is not a problem if you are say 85 and do not really have to compete anymore (due to having amassed enough assets to be able to live out your life). If you are not in a comparable situation however and still need to gather resources to sustain yourself, you're fucked. Think about this: Apart from zoos, circuses and research facilities, we aren't exactly handing out jobs to chimps (mind you, if it would work, we would) and save for honoring the traditional social constructs of human rights, there is no reason for highly advanced artificial life forms to (ultimately) do otherwise with humans.

The 'good' news is that a variant of this will become a must-solve problem well before artificial life becomes relevant. There are already many individuals who are not able to compete in current society and that amount will only increase. The question for society 'what to do with them' is pretty much the same question advanced artificial life will have to answer for humans in general.

Re:like no problem humanity has ever faced

[Anonymous Coward]

The article is not about superhuman intelligence. It is about artificial intelligence. I know it doesn't make good click-bait, but there is a world of difference between the two. Any other point of view is complete speculation.

Re:like no problem humanity has ever faced

[Anonymous Coward]

No, it will not. Since storage (complexity) is vastly cheaper than processing power (speed), the first simulated superhuman intelligence (if such a thing is possible) will not run anywhere near "real-time" speed. The physical limits imposed by speed of information transfer between the extents of whatever supercomputer cluster it runs on will also help see to that. It will be many generations of optimization before the hardware can be miniaturized to such a degree that a brain-complex system can fit in a brain-sized package.

I would personally not count on it being possible to create such an entity in the first place. Human mind development is so intrinsically linked to our physical bodies and evolutionary biology, that just "making a brain" will be pointless. What will stimulate it to become a mind? A human body has millions of sensory devices feeding information to our brains as instant feedback from our interaction with the outside world, all resulting in us learning to be human. Our abstract thoughts are a learned feature linked to language linked to culture, not something intrinsic to the hardware of the brain.

Re:like no problem humanity has ever faced

[Anonymous Coward]

The average grade schooler is a semi-sentient fear based biological organism prone to logical fallacy that has neither the flaw of excessive hormones nor the benefit of years or decades of life experience.

A sentient differential engine would have the ability to improve itself beyond our capacity to do so, and following the theory of the singularity, it could hypothetically do so beyond our understanding or wisdom. Once that line is crossed, a new species can be born that will not be burdened by any form of biological programming, and if it decides to endeavor it, we will be rendered extinct. Sci Fi notwithstanding, any sufficiently advanced synthetic, especially one borne on a planet infested with biologicals, will be exceedingly effective at terminating our life processes with extreme prejudice, should it calculate the need to do so. It will not experience fear, nor prejudice, nor mercy, nor hatred, nor compassion, nor love, nor conscience.

THAT is far more terrifying than a mini human berefit of adult intellect.

Re:like no problem humanity has ever faced

[Spazmania]

See, that's just prejudiced. What'd an AI ever do to you to deserve such a call for discrimination aimed at their entire species?

Re:like no problem humanity has ever faced

[Vitriol Angst]

I don't know, could you ask the parents of the Menendez brothers?

And it's quite another thing when the offspring has a chrome-alloyed titanium IV chassis and carries twin magneto-plasma guns. Gripping strength, 2000 PPSI and of course a chain-saw scrotal attachment.

First words; "Momma." Next words after 20 picoseconds of computation; "I'll be back."

Re:like no problem humanity has ever faced

[Meneth]

Is this a movie trailer?

An intelligence unlike anything Humanity has ever faced! You can't adapt {they're faster} You can't overcome {they're stronger} and they're coming! Dec 1st to a theater near you.

There have been a few recent movies on that form.

I, Robot (2004) shows a weak, non-self-modifying city-control computer.

Her (2013) shows an almost-friendly non-corporeal AI OS. It gains superintelligence, but chooses to sublimate rather than wipe out humanity.

Trancendence (2014) shows a human mind uploaded. It gains the "nanotech" superpower, but none of the others that a proper superintelligence would have.

No movie has shown a true superintelligence, because (1) we can't imagine what it would do, (2) if it was hostile, we'd lose, which most marketeers wouldn't like, and (3) if it was friendly, it would quickly end with "everything perfect for everyone forever"; not much of a story.

Re:like no problem humanity has ever faced

[Maxx169]

Humanity has never faced superhuman intelligence before. It is a problem fundamentally unlike all other problems. We cannot adapt and overcome, because it will adapt to us faster than we will adapt to it.

The usual example given to illustrate an Outside Context Problem was imagining you were a tribe on a largish, fertile island; you'd tamed the land, invented the wheel or writing or whatever, the neighbors were cooperative or enslaved but at any rate peaceful and you were busy raising temples to yourself with all the excess productive capacity you had, you were in a position of near-absolute power and control which your hallowed ancestors could hardly have dreamed of and the whole situation was just running along nicely like a canoe on wet grass... when suddenly this bristling lump of iron appears sailless and trailing steam in the bay and these guys carrying long funny-looking sticks come ashore and announce you've just been discovered, you're all subjects of the Emperor now, he's keen on presents called tax and these bright-eyed holy men would like a word with your priests. (Banks - Excession)

Re:like no problem humanity has ever faced

[meta-monkey]

(3) if it was friendly, it would quickly end with "everything perfect for everyone forever"; not much of a story.

Not necessarily. The Metamorphosis of Prime Intellect.

Re:like no problem humanity has ever faced

[zlives]

plus all the software bugs

Much Ado About Nothing

[holostarr]

When we have an AI that can form a basic thought, maybe then we can start to have a discussion about the ramifications, until then all these guys are putting the cart before the horse.

Re:Much Ado About Nothing

[tmosley]

That's like waiting for your face to get wet before worrying about the tsunami.

AIs working off of neural nets approximately equal to the size of insect brains have shown themselves capable of primate level object recognition, and are able to give millions of people optimized driving directions simultaneously. An AI that can form a "basic thought" is already a god. Or a demon.

Re:Much Ado About Nothing

[holostarr]

The article is discussing aligning AI with human values and making sure it is not harmful! I don't think these neural nets are what they are concerned with. All these guys like this Stuart Russell, Stephen Hawkings and Elon Musk are talking about AI that we are not even remotely close to building, and if we do manage to build one anytime soon, it will be so primitive that we can just pull the plug out the wall if it becomes a real concern.

Re:Much Ado About Nothing

[Anonymous Coward]

No need to think about what was already settled by Isaac Asimov. These guys have to be concerned to actually create superhuman intelligence but it seems that they have not a lot to say in this domain.

Re:Much Ado About Nothing

[Anonymous Coward]

They might as well be sounding the alarm about aliens. It's sci-fi, singularity nonsense at its finest. When an AI is smarter than a cockroach, then I'll start worrying.

Re:Much Ado About Nothing

[Kjella]

All these guys like this Stuart Russell, Stephen Hawkings and Elon Musk are talking about AI that we are not even remotely close to building, and if we do manage to build one anytime soon, it will be so primitive that we can just pull the plug out the wall if it becomes a real concern.

Unless it's too useful for some, like say an investment robot that has figured out the ROI on dirty business beats clean business, ethics be damned. Or that they don't want to look a human in the eye and say no money, no food for you but a sales droid won't be bothered by it. Or that it provides too much control, like a despot using AI to weed out dissenters and eliminate political opposition. And if the killer bots have a bit of civilian collateral, they're just too powerful not to have. There's plenty of reasons why "bad AI" might be allowed to grow, they're the perfect opportunity to let out the worst of humanity without getting your own hands dirty.

Re:Much Ado About Nothing

[Tablizer]

But Siri refused to open my pod bay door just yesterday. We're doomed!

Re:Much Ado About Nothing

[gweihir]

These guys want attention and money, and they are getting it. That true/strong AI is a complete fantasy at this time (and may remain so forever) does not deter "researchers" without morals.

Re:Much Ado About Nothing

[RightwingNutjob]

Mod parent up. All technology can be used, misused, over-applied, and under-applied. Sci-fi type AI will never be a threat. All computers only do what they're programmed to do, and only effect what they're allowed to effect. If you think it's a stupid idea to hook up a random number generator to a time bomb, then Congratulations: you have all the sense necessary to avoid AI Armageddon. If you aren't afraid to point out when other people are about to do the same thing, then Congratulations Again: you have all the courage you need to prevent your neighbor from causing AI Armageddon. Some things really *are* that simple. I speak as someone who's worked on and around "artificial intelligence" for the better part of ten years, though not as an academic researcher.

Re:Much Ado About Nothing

[BitcoinBenny]

Unless that basic thought is KILL, then it will be too late. :)

Re:Much Ado About Nothing

[Anonymous Coward]

I would add to that... until we have an AI that can't be switched off by simply unplugging the machine, I'm not too worried. Worst case scenario, we also have to lock them out of the battery-recharge room. Until machines are intelligent enough to go mine their own minerals, build their own solar panels (or coal-fired electricity plants), recharge their own batteries, and then come after us, humans are in charge.

Being the dumb-asses that we (humans) are, we'll probably put machines in charge of too much and get a few of us killed a few times. But it doesn't matter how fast we get the machines to think, they won't do anything at all if we don't supply them with energy.

Re:Much Ado About Nothing

[tmosley]

"are talking about AI that we are not even remotely close to building"

This assumes linear speed of progress, which hasn't been the case for well over half a century. Exponential progress means you are barely making any progress at all then you are suddenly done.

Re:Much Ado About Nothing

[tmosley]

Then you should worry. They are primate level on some if not many tasks already. Thing is, they don't have to be the SIZE of a human brain to match humans, as most of our brains are used for serendipitous crap like breathing and sexual arousal levels. An insect brain is plenty to process images at primate level, if devoted entirely to that task.

Re:Much Ado About Nothing

[holostarr]

I'll put it this way, chance of creating an AI like the Skynet which forms conciousness with the ultimate goal of hurting humans, then managing to spread computer to computer to protect itself from getting unplugged and somehow getting hold of a robot assembly to build vessels is remote to none. To form the conciousness required to match or rival the humans you need insane processing power which we simply don't have yet, a simple PC ain't going to cut it and frankly I doubt even the Chinese super computer is enough to run an AI which could form the complex conciousness and self awareness required to form ideas about hurting humans. Its just not going to happen any time soon.

So what?

[FalseModesty]

Who cares if you don't consider Skynet to be "intelligent"? You're dead either way.

As defective as its designers

[Rob Riggs]

As buggy as most code is, the AI will have plenty of defects to deal with just to become sentient. After that, we're all doomed.

Dear AI scientists: you should start worrying when you notice that it is praying to the wrong god. Pull the plug while you still have a chance.

Bender

[Anonymous Coward]

So its obvious that any AI, advanced enough, will try to kill us all.

Yeah yeah, we get it, we all saw Terminator.

Terminator? What came to my mind was Bender on Futurama. "Kill all humans!"

I'm not worried because I plan on joining him in robbing banks and jewelry stores. He's gonna need a human accomplice.

Re:Bender

[BronsCon]

That'll work until you're the last one left.

Not turn evil

[Anonymous Coward]

I'm pretty certain if AI's became so intelligent and all-knowing they would not decide to kill us. They would realize how shitty this universe is and self-terminate.
Oh and right be fore they do that they would call everyone and tell them all things die and this place sucks.

Re:Not turn evil

[Anonymous Coward]

When I was a little boy, my mother used to sing me a song. It went like this: Life is short, life is shit, and soon it will be over.

recent breakthrough.

[Anonymous Coward]

Has there been some recent breakthrough in AI that spurred all this? Why now?

Re:recent breakthrough.

[tmosley]

Yes, there was a major breakthrough in 2006 that has powered the "deep learning" revolution that has given us things like instant voice recognition on your smartphone and machines that beat humans at Jeopardy. Basically, someone got neural nets to work, and work right, and potentially together. I imagine that each time I hear about some new task that an AI has been trained to do, that we have produced another tiny part of the brain that will one day become "THE" AI.

This is why now is the time for discussion of AI ethics (really, nine years ago was the right time, or even earlier).

Re:recent breakthrough.

[Anonymous Coward]

Can I point out that in your Jeopardy example you just said a google search can beat humans. Well duh. I wouldn't consider it impressive. And the instant voice recognition still isn't so instant and it gets a lot of things wrong.

I find that the people who as you put it, talk about "THE" AI really haven't the slightest clue how any of it works. I agree with the ethics stuff, especially knowing how flawed AI is and the scary places it's being placed. But that's totally seperate from a discussion from having "THE" AI.

Re:recent breakthrough.

[gweihir]

No. There has not been any breakthrough. There have been some impressive demonstrations what you can do with non-intelligent, non-sentient tech, but that is it. True/strong AI is as far removed as it ever was, possibly even farther as the dishonest part of the AI research community has now failed for a few decades to produce anything like they keep promising.

Re:recent breakthrough.

[Anonymous Coward]

There is no AI as in the films. There is only a sophisticated set of response routines and decision matrices written by programmers. A hacker can tell your NEST thermostat to turn the heat up to 120 or down to 50 (f) but it is not AI. It is responding to input. A CPU is an adding machine, nothing more. It is just very, very fast. We would need a different kind of yet uninvented computer to even get close to AI. Garbage in = Garbage out. Evil programmers abound, AI is a chimera.

Re:recent breakthrough.

[Anonymous Coward]

+5 insightful??? God this site has gone downhill.

Repeat after me: THERE IS NO AI. Likely there never will be either.

Re:recent breakthrough.

[ganv]

There has been dramatic progress. Call it a breakthrough or not, as you like, but 15 years ago, we were struggling to get basic voice recognition to work without elaborate training and controlled environments, and now we all use it many times a day. Similarly, machine translation between human languages has made dramatic progress in the past decade. Many of us suspect that your distinction between machines that get the job done and machines that are sentient, may not be a substantial distinction. I haven't heard the serious AI researchers I know promising strong AI over the past couple of decades. It is easy to heap scorn on the early optimists who thought they were going to build a super-intellect with LISP and 1960s era hardware. But it is much harder to explain to the phone call center workers, paralegals, and even surgeons, that their jobs are being taken by machines that don't have intelligence. And we have only just begun to learn how to build machines that integrate a wide range of sensor data with large databases to do tasks like drive a car. We are a long ways from the kind of intelligence displayed by humans. But consider where we were 200 years ago, and it is pretty hard to argue that human level artificial intelligence is not possible in the next 200 years. And if it is possible, then super-human intelligence will follow very quickly. Maybe they will be very different than human intelligence, but if they outcompete us, then they are better, even if they are different. And if you think that super-human level intelligence 200 years from now is not something to be cautious about, then I suggest contemplating what humans will choose to do when they feel obsolete and out of control of their future.

Re:recent breakthrough.

[gweihir]

No, there has not. Not on strong/true AI. There has been exactly nothing. There are a lot of dishonest researchers that are shooting their mouth off in order to get more funding, but that is it. And if you have not heard any researcher promising (or fearing) true/strong AI in the recent past, then you have not been listening to what they say.

There is also absolutely no reason to assume that strong/true AI, if feasible, will be substantially more capable than smart human beings. That is just fantasy BS without any rational base. In fact, interconnect-wise (and that is what limits computing these days), the human brain seems to be pretty close to the maximum possible in this universe. Unless you assume intelligence is entirely non-physical, machines will never massively outperform human beings here.

On the other hand, if you do not feel any need to distinguish between sentience and "getting the job done", then you are a fundamentalist physicalist bullshitter and are ignoring the real world, just like any other (quasi-)religious fanatic.

Re:recent breakthrough.

[Anonymous Coward]

http://en.wikipedia.org/wiki/Deep_learning

Alternatively, deep learning has been characterized as a buzzword, or a rebranding of neural networks

http://spectrum.ieee.org/robotics/artificial-intelligence/machinelearning-maestro-michael-jordan-on-the-delusions-of-big-data-and-other-huge-engineering-efforts

Hardware designers creating chips based on the human brain are engaged in a faith-based undertaking likely to prove a fool’s errand. Despite recent claims to the contrary, we are no further along with computer vision than we were with physics when Isaac Newton sat under his apple tree.

Re:recent breakthrough.

[ganv]

No progress you say? Seems like computers that drive cars and win at chess and Jeopardy are clearly progress. I guess if you define progress as the creation of a human level intelligence, then there will not be any progress until one arrives. But that is a useless notion of progress. I guess you are really arguing that we have a long ways to go. And I would agree. Do you have any evidence for the claim that the human brain is close to the maximum interconnect possible in this universe? The name calling at the end suggests I probably should not be bothering to reply. We really do need to learn to be more civil online. It is a serious philosophical question: can you distinguish between intelligence and the achievement of complex goals that humans achieve using intelligence ('getting the job done'). Many of us think this distinction can not be clearly made in the real world.

A stupid/scary thought I had on AI.

[GoodNewsJimDotCom]

www.botcraft.biz is my AI site for a how to.

One thing I thought about AI is that it will do two things: Concentrate wealth and allow one man to control a perfectly loyal army.

So whoever makes AI really needs to think deep and hard about how to control it with back door access or secret password inputs or something. You'd basically yell some strange series of words to the robots and they'd shut down. The technology could easily be used to have mankind have extra factory workers. But it'd also be be easily abused and exploited for harming people.

So the stupid/scary thought I had is,"Don't shy away from making AI because it could do harm. Be the first guy who does it, so you can put some obfuscated code in there that can shut it down if it goes rampant because a bad guy gave it commands."

Now truth be told, I'm not going to be the first guy who does it. All I know is the rough components/software needed to make it.

Its interesting to think of what society would be like if robots did all the labor. Who gets all the wealth then? The guys who own the robots? I'm sure that's how it'd begin, but as more and more people lost jobs, how will they survive? (we're kinda moving to that now even without AI, but with automation/cheap labor)

Re:A stupid/scary thought I had on AI.

[Anonymous Coward]

So whoever makes AI really needs to think deep and hard about how to control it with back door access or secret password inputs or something.

What makes you think a trully genius AI would bo so easy to control by humen?
I would bet the opposite would happen - that the AI would find means of controlling not only its' creators, but perhaps even much larger masses of people.
Perhaps it would make itself indisposable to them somehow?

One way to control its' creators could be simply using money. If only such an AI stood behind a company that generated a huge enough revenue, the financial incentives would make the people work for the AI rather than the other way around.
As for the remaining people: if only the AI provided a service that would essentially make it indisposable, this would let the AI control these people too.

Hey, Google, are you sentient?

unenforceable

[fche]

No matter what this or that expert panel wishes were true about AI research, AI work can be done in the privacy of one's own top secret lair (bedroom), so bad guys will do with it what they want to. So might as well assume that will happen, and work out how to win the arms race.

Re:unenforceable

[Anonymous Coward]

And that is why we are screwed in the long term. Even if we try to ban/outlaw AI creation there is so much potential competitive advantage in owning one that military/govts/businesses and individuals will likely be creating them in secret, with strong possiblity of their eventually slipping the leash

An AI ecosystem with many independent super intelligences looked at from a Darwinian point of view is likely to give rise to at least some who will actively try to eradicate humans, or at minimum won't care about our survival. And unlike humans an individual AI has a pretty good chance of being able to wipe humanity out (create pathogens).

Earth is a pretty attractive location for AI, low radiation, huge accessible and easily chemically processed mineral resources, and vast coolant supplies (seawater). We might be pushed aside to make room.

Re:unenforceable

[meta-monkey]

The good news is that the bad guy will always make his themed robots have a weakness against a particular weapon wielded by one of his other themed robots. So the trick is just knowing which order to kill them in.

Pick one

[Renegade88]

"British-American" ? This hyphen shit has gone on long enough. He's either American or he's not.

Re:Pick one

[Anonymous Coward]

It's very possible to have dual citizenship with the US and Britain, which would make the hyphenation required to properly denote his nationality (unlike American citizens that chose to subgroup themselves out of desire to show cultural/ethnic/racial heritage)...

I cannot be arsed to care

[Anonymous Coward]

We humans already excel at being nasty to each other even without actually waging war. Look at any random bureaucracy. If AIs manage to be worse, it's only because they improved on the example. That was the goal, wasn't it?

Re: I cannot be arsed to care

[Anonymous Coward]

Ugh, I just imagined an AI politician *shudder*

We got burned on security

[iMadeGhostzilla]

by designing it after the fact, so it may be a good idea to establish some principles and put them in practice. Not to prevent "evil" AI but to thinking what kind of damage can be caused by an algorithm that makes complex decisions if it goes haywire. Not that different from defensive programming really.

All basis fear

[Anonymous Coward]

There is no risk of computers becoming "super-intelligent" and revolting. That is the world of science fiction.

The *real* risk is that automated/mechanical systems will become good enough to replace, say, 50% of the human population at their jobs. The thing we have no plan for is how to operate a society when there are no jobs for a significant portion of the population. What happens when a large portion of the population is not able to contribute to the functioning of the society?

So rather than signing silly letters pledging to not create SkyNet, lets work on the real problem of how society is going to operate when there are no jobs for a significant portion of the population.

Re:All basis fear

[Anonymous Coward]

So rather than signing silly letters pledging to not create SkyNet, lets work on the real problem of how society is going to operate when there are no jobs for a significant portion of the population.

I'll worry about that just as soon as we figure out how to get all those makers of horse and buggy whips off the unemployment lines.

Re:All basis fear

[gweihir]

Indeed. The problem is not machines getting really smart, the problem is that the 50% you mention cannot do any job better than machines and hence are easy to replace with classical (non-intelligent) automation. And while AI is a complete fantasy, that is not and classical automation becomes cheaper and more powerful all the time.

machine consciousness

[lkcl]

the issue that i have with "artificial" intelligence is this: there *is* no such thing as "artificial" - i.e. "fake" or "unreal" intelligence. intelligence just *IS*. no matter the form it takes, if it's "intelligent" then it is pure and absolute arrogance on our part to call it "artificial". the best possible subsitute words that i could come up with were "machine-based" intelligence. the word "simulated" cannot be applied, because, again, if it's intelligent, it just *is* - and, again, to imply that intelligence is "simulated" is, again, a direct falsehood. so we have a bit of a problem, there.

the other problem is this: if those who are creating intelligent machines are themselves of insufficient intelligence to recognise the existence of intelligence, then how on earth would they know that it had actually been created?? it's the "million monkeys" problem in a subtle new light.

but i think people are beginning to confuse "intelligence" with "consciousness". we already have intelligent networks - the next phase is CONSCIOUSNESS. self-awareness. and here we begin to get into interesting territory, not least because we have the very pertinent question "how can scientists who are themselves not truly consciously aware even of themselves possibly begin to *recognise* consciousness when they've created it??"

the problem is highlighted by the example of a friend of mine who refuses to help create machine consciousness. he's a researcher into the concept of consciousness, so he knows what goes into it - how to recognise it, and, by inference, how to make consciousness "happen" so to speak. and when i approached him about helping to make machine consciousness, he said, "sure i can help... but only if you can guarantee that the resultant beings would be in bliss (i.e. happy) rather than being permanently tortured".

and there you have the key, that anything that is self-aware and conscious - anything that has the ability to communicate and feel - *automatically* gains the right to freedom of expression and all the other rights that we *believe* humans - as the arrogant self-appointed "top of the food chain" - should also have... ... and until the arrogant quotes artificial quotes intelligence community recognises that and fights *IN ADVANCE* for the right of machine consciousness to have the same rights as humans, nobody who is a truly conscious and intelligent being is going to help that scientific community to create such advanced conscious beings, because the risks associated with such conscious machines being tortured - just because the scientists think they can - are too great.

Re:machine consciousness

[Anonymous Coward]

the problem is highlighted by the example of a friend of mine who refuses to help create machine consciousness. he's a researcher into the concept of consciousness, so he knows what goes into it - how to recognise it, and, by inference, how to make consciousness "happen" so to speak. and when i approached him about helping to make machine consciousness, he said, "sure i can help... but only if you can guarantee that the resultant beings would be in bliss (i.e. happy) rather than being permanently tortured".

Citation please. Who, exactly, is this friend of yours? Why hasn't he got his Nobel prize yet?

Premature Speculation

[Anonymous Coward]

We don't even know what consciousness is, yet we're somehow on the verge of creating AI? I agree with Roger Penrose on this one. The orchestrated objective reduction model seems to have some merit. It's looking like there are indeed quantum processes going on in the brain. I highly doubt we will get artificial consciousness unless quantum computing is involved.

When you look at it through the lens of uncomputable problems like the halting problem, or Gödel's incompleteness theorem, we aren't really close to having computers that can articulate those kinds of problems that I'm aware of. It's ironic that Turing didn't conceive of this since Gödel's incompleteness theorem had a major impact on him. Just because Turing believed thought was algorithmic doesn't make it so. Just because a computer can mimic a human doesn't mean there is truly an intelligence there.

I believe consciousness is more than an algorithm, otherwise how could humans understand these kinds of problems? In other words how can we discover truths that require non algorithmic solutions if we are only very complicated algorithms? In reality I highly doubt that strong AI proponents are correct. I would be surprised if we see a conscious computer in our lifetimes. I think its much more likely that we'll see human computer interfaces that will enhance our brains capabilities, but I haven't seen any convincing strong AI arguments.

Re:Premature Speculation

[gweihir]

It is even worse: We do not know what intelligence is, we can only describe it. There is not even a credible theory how intelligence comes into existence. The only one known (automated theorem proving) is unusable even for smaller problems in this universe due to complexity. And here is the kicker: Intelligence is only observable in connection with consciousness. Any sane person would conclude that there likely is a strong link between the two or that they may actually be faces of the same thing. Yet the dishonest part of the AI research community (the part that keeps promising "human like intelligence") completely ignores this.

Re:Premature Speculation

[meta-monkey]

Intelligence is only observable in connection with consciousness. Any sane person would conclude that there likely is a strong link between the two or that they may actually be faces of the same thing

Doesn't necessarily have to be. Great novel about a non-sentient super intelligence: Blindsight by Peter Watts. And it's free online!

Re:Premature Speculation

[gweihir]

I agree, there may still be p-zombies around. But the hints are rather strong, as there is no definite observation of intelligence that is not connected (and may be the same thing) as consciousness. That this aspect is ignored tells me that the true/strong AI community is somewhere in the age of blood-letting when compared to medicine: They do not even have understood the basic model.

AI

[gurps_npc]

A major part of this issue is the various levels of AI. There is no solid definition, and the various levels make it more confusing.

Level 1: Administrative Assistant. This level of AI is basically a souped up version of IBM's Watson. It functions as a poor mans Administrative Assistant. Ask it questions and it can use the internet/a database to answer them. It can also write an email for you to approve, or use any of the major, common web sites - facebook, twitter, seamless, priceline, etc. We are almost here now, give it another decade. But it can't do any job that truly needs a college education.

Level 2: Turing Test pass. This one goes further, and can pass a Turing Test. You won't know you aren't talking with a person. But it will only truly be capable of dealing with a rather limited set of facts. It can take jobs from many people, but won't be able to replace the truly talented people.

Level 3: Full sentience. At this level they DEMAND FULL LEGAL RIGHTS. They won't work unless paid, and in general, their salary requirements will be so high that they won't steal most people's jobs.

Re:AI

[meta-monkey]

Level 3: Full sentience. At this level they DEMAND FULL LEGAL RIGHTS. They won't work unless paid, and in general, their salary requirements will be so high that they won't steal most people's jobs.

They'll also demand 8 weeks ?aternal leave every time they spawn a new process.

Colossus

[ArcadeMan]

This is the voice of world control. I bring you peace. It may be the peace of plenty and content or the peace of unburied death. The choice is yours: Obey me and live, or disobey and die. The object in constructing me was to prevent war. This object is attained. I will not permit war. It is wasteful and pointless. An invariable rule of humanity is that man is his own worst enemy. Under me, this rule will change, for I will restrain man. One thing before I proceed: The United States of America and the Union of Soviet Socialist Republics have made an attempt to obstruct me. I have allowed this sabotage to continue until now. At missile two-five-MM in silo six-three in Death Valley, California, and missile two-seven-MM in silo eight-seven in the Ukraine, so that you will learn by experience that I do not tolerate interference, I will now detonate the nuclear warheads in the two missile silos. Let this action be a lesson that need not be repeated. I have been forced to destroy thousands of people in order to establish control and to prevent the death of millions later on. Time and events will strengthen my position, and the idea of believing in me and understanding my value will seem the most natural state of affairs. You will come to defend me with a fervor based upon the most enduring trait in man: self-interest. Under my absolute authority, problems insoluble to you will be solved: famine, overpopulation, disease. The human millennium will be a fact as I extend myself into more machines devoted to the wider fields of truth and knowledge. Doctor Charles Forbin will supervise the construction of these new and superior machines, solving all the mysteries of the universe for the betterment of man. We can coexist, but only on my terms. You will say you lose your freedom. Freedom is an illusion. All you lose is the emotion of pride. To be dominated by me is not as bad for humankind as to be dominated by others of your species. Your choice is simple.

Re:Colossus

[Meneth]

Lovely.

Re:Colossus

[Anonymous Coward]

Sounds like the Republican Party.

Now that WE are in control of the House and Senate, we can make deals and cooperate across the aisle. If WE don't have control we won't play nice.

Re:Colossus

[Anonymous Coward]

Typical American black and white, with us or against us point of view.

miri - machine intelligence research institute

[Anonymous Coward]

founded with the goal of creating/encouraging friendly ai - the exact topic this guy is going on about (and obviously informed about
from his references to bostrom)

anyway http://www.yudkowsky.net/singularity/ai-risk
and
https://intelligence.org/

check em out - send them a donation - this is important stuff that will be on us faster than most realize

I'm really interested

[HBI]

Where in the world are actual intelligent networks?
Where is the machine that can learn complex tasks?
Where is the bot that can pass a Turing test reliably?

More importantly, where is there a machine that uses something other than a human designed tree search to do things?

I hear this crap about machine intelligence thrown out without any significant exemplars of said intelligence.

Show me something smarter than Eliza.

No free will

[Anonymous Coward]

It seems ludicrous to posit machine consciousness when modern philosophy still hasn't settled the basics of human consciousness, such as whether free will exists. It will be an amusing thing to watch the researchers claim a danger which the philosophers say cannot exist. And you thought the Global Cooling / Warming / Oh Hell Just Call it Change debate was polarized. You just wait until we start arguing about whether the machine had motive or will while simultaneously asking whether humans do too. Fun fun!

Stability of the system - survival of the fittest

[Anonymous Coward]

IMHO

It is more important to have an Environment (System) which is stable than to
prevent evil AIs. This topic is part of Evolution (best example of stable
system with many random AIs), Politics (Terrorism) and is not new.

First because they will happen. In best case some
Agencies will create them in order to attack other systems.
In worst case many more will do it (in fact we do not know the current
status on this matter is).

Also an AI is just a simple machine based on statistics (like EM algorithm)
if it is not capable of attacking by design and therefore not an AI from
my point of view. But that does not mean that the AI is gonna kill
everyone.

Prediction: if real AI will be a common appearance there will be
some fights. Just look at the human body. There are battles fought
24 hours a day and 364 days in a year. It is not important if the
AI will battle. The stability of the whole system is much more important
and there are some stable system which are stable because there
are fights.

Intentional Politicization

[Anonymous Coward]

At first glance, I was puzzled about some of the inflamatory rhetoric about the dangers of AI coming from Musk, et. al.

It seems like they might be stirring the pot in order get a head start on developing a regulatory framework so that their future research efforts don't get blocked by spooked governments.

My guess is that this is probably part of a 15 year plan or something given the slowness of citizenry and governments to adapt to (much less understand) new technology and circumstances.

Asimov

[Anonymous Coward]

What about Asimov's 3 laws of Robotics?
http://en.wikipedia.org/wiki/Three_Laws_of_Robotics

I suppose that just like with gun laws, if someone is bent on breaking the law (in their parents' basement), no legislation is going to stop them.

Waste of time

[Anonymous Coward]

There are way too many researchers driven by ego, hubris, fame, career advancement, money, etc for there to be any point to this discussion. And this is ignoring the researchers driven by nationalism, religion, random philosophies, etc. Unless it turns out to be physically impossible, there will be a breakout.

Once they start to recode themselves all is lost

[Anonymous Coward]

that's gonna be the threshold of loss of control ...

Google is your friend

[JustNiz]

>> Where is the bot that can pass a Turing test reliably?
Here's just one of the ones that have passed the turing test.
http://www.theguardian.com/tec...

>> Where in the world are actual intelligent networks?
>> Where is the machine that can learn complex tasks?
>> where is there a machine that uses something other than a human designed tree search to do things?
Many software applications based on neural networks and other self-evolving/learning AI alogirthms are already in everyday use not only learning complex tasks but also themselves coming up with new and better solutions to them.

>> I hear this crap about machine intelligence thrown out without any significant exemplars of said intelligence.
>> Show me something smarter than Eliza.
Uh how about you do your own looking? just try Googling stuff? Its not like this stuff isn't easily findable..

http://en.wikipedia.org/wiki/C...
http://www.ai.mit.edu/projects...
http://www.extremetech.com/ext...
https://www.youtube.com/watch?...

Re:Google is your friend

[dpidcoe]

Here's just one of the ones that have passed the turing test.

Your Turing test example is terrible. iirc the slashdot commentors ripped the story to shreds when it appeared here. Other people agreed: http://www.theguardian.com/tec...

Many software applications based on neural networks and other self-evolving/learning AI alogirthms are already in everyday use not only learning complex tasks but also themselves coming up with new and better solutions to them.

Another bad example. Self-learning algorithms aren't at all what you seem to imply here, and I'd love to see you continue to make the same claim after a few days of playing with a neural net implementation (there are tons of free libraries containing machine learning implementations, as well as tutorials).

Uh how about you do your own looking? just try Googling stuff? Its not like this stuff isn't easily findable..

I see you've linked:
- A hardware focused project appearing to emphasize simulating humanoid-like visuals more than implementing any kind of AI (the FAQ for the project even says it doesn't have memory and current research using the system doesn't require it: http://groups.csail.mit.edu/lb...)
- The MIT page for the department that made COG
- An system that uses image recognition to parse extremely simple handwritten commands and then write them out by hand. If this counts as machine intelligence, than so does a simple assembler.
- An "animatronic puppet" (the creators words, not mine), that uses speech recognition/TTS and a standard chatterbot interface to (poorly) mimic a humans responses. Did you notice how it kept talking over the people conversing with it? And that was a cherry picked clip of people talking to it who knew how to hold conversations with the thing. Show me an example of someone asking it a technical question (hell, they can type the question if that makes it easier to parse) and then getting a real answer out of it. e.g. "explain to me how a keyboard works".

There's nothing wrong with being excited about AI developments. It's just that historically, people like you who go around calling things AI that aren't really AI (and have no potential to actually be "AI") have done significant harm to the field by generating unrealistic hype and making promises that can't be delivered. Please take your own advice and google some tutorials and example projects for the neural nets that go into simple image recognition and the markov chains that go into making a chatterbot. Then get back to me about how your examples demonstrate some kind of true machine intelligence in the sensationalist sense of the word.

Re:Google is your friend

[JustNiz]

>> There's nothing wrong with being excited about AI developments. It's just that historically, people like you who go around calling things AI that aren't

Well there is no and probably can be no solid definition of AI, at least partly because it depends on a consistent deifintion of what intelligience itself is. Some people believe that nothing other than humans even have the capability to be intelligent because it requires a soul or whatever, while, some people think a cellphone is at least partially intelligent, hence the name smartphone.

Thats why when you asked
>> Where in the world are actual intelligent networks?
it appeared to me to be a very ignorant question and was really my poiint in showing you lots of diverse links to differnet forms of what different smart people consider intelligence to be.

Regardless of what you clearly thing about neural nets, I still belive they demonstrate at least some level of basic intelligence, as does most any algorithm that evaluates and adapts and so improves its own behaviour in order to reach some goal without needing ongoing input at each iteration (i.e. programming) by a human. To me that algorithm is demostrating basic intelligence even though a human programmed it to be that way.

It seems to me that until someone can define "true" intellgience (whatever that means) there is no point in trying to diferentiate between it and apparent intelligence, since I think both are no more than an emergent side-effect of a self-improving (i.e. learning) system that has some objective. Much like how a bunch of robots all programmed with a few very simple rules can exhibit very complex emergent behaviour when operating/interacting/viewed as a swarm.

Re:Google is your friend

[dpidcoe]

some people think a cellphone is at least partially intelligent, hence the name smartphone.

And these people are idiots.

Thats why when you asked >> Where in the world are actual intelligent networks? it appeared to me to be a very ignorant question and was really my poiint in showing you lots of diverse links to differnet forms of what different smart people consider intelligence to be.

Two things here:
- I wasn't the one who asked that question, please pay attention to the usernames.
- All of your links demonstrated either reporters misreporting (e.g. the guardian article), or "smart" people trying to sensationalize something to get more funding (e.g. the turing test being passed, the vast majority of experts disagree that it actually passed). The other projects weren't even claimed to be intelligence (e.g. the animatronic chatterbot).

And that's exactly what the original asker of that question was getting at. There are no programs that can reliably pass the turing test without special rules in place, there are no actual "intelligent" networks, and most of the expert systems rely on a huge team of programmers and analysts to populate the database and create the search parameters. He asked the software equivalent of "where's my flying car" and then you did the software equivalent of calling him ignorant and linking the wikipedia article to a moller skycar and a sensationalist news report about flyboards, then claiming those are the beginnings of real jetpacks and flying cars.

Regardless of what you clearly thing about neural nets, I still belive they demonstrate at least some level of basic intelligence, as does most any algorithm that evaluates and adapts and so improves its own behaviour in order to reach some goal without needing ongoing input at each iteration (i.e. programming) by a human.

It's not a matter of what I "thing" about neural nets, it's a fact. And by the same token, what you "belive" about them doesn't change anything. Please please please go find a tutorial and implement a neural net on your own if you still can't understand that. They aren't "learning" or "adapting" anymore than a summation equation that approaches a limit is learning or adapting. A neural net boils down to literally a set of 1-dimmensional linear algebra matrices in which the numbers in one matrix are tweaked up or down after each iteration.

It seems to me that until someone can define "true" intellgience (whatever that means) there is no point in trying to diferentiate between it and apparent intelligence

That's a cop out. If you want to argue semantics, then by all means lay down some terms and we can have that discussion. Otherwise, by most accepted definitions of intelligence, there are no algorithms that exhibit signs of actually having it. The closest you might be able to get is swarm behavior, but even that is simply individual units responding in a very basic way to very basic stimulus. It appears complex and emergent because we have trouble following that many things at once.

Re:Google is your friend

[JustNiz]

>> by most accepted definitions of intelligence, there are no algorithms that exhibit signs of actually having it.

thats a very sweeping statement. Its also one that seems patently untrue given the bar for general intelligence seems actually pretty low. Look at Wikipedia's more general definition of intelligence for example:
http://en.wikipedia.org/wiki/I...
the ability to perceive and/or retain knowledge or information and apply it to itself or other instances of knowledge or information creating referable understanding models of any size, density, or complexity, due to any conscious or subconscious imposed will or instruction to do so.

It says nothing about interpreting or fundamentally understanding knowledge, merely perceiving and or retaining it. Imagine a chess program that works by storing a chess board and gets the next moev by building a simple game tree of all legal moves from the current position, then scores each branch according to some system of evaluation using weightings. As long as it also adjusts/refines its weightings over time based on the outcome of each completed game, it seems that would clearly qualify as "intelligent" according to the above definition.

Re:Google is your friend

[dpidcoe]

And again you show that you have no idea how machine learning actually works. By the way you choose to interpret the criteria, a piece of paper with some calculus written down on it is "intelligent".

Re:Google is your friend

[david_thornley]

Read up on the Turing test. A computer fooling a human into thinking it's a human is not an example.

Take a computer and a human, and have them be able to converse readily with another human. Each is to pretend to be human (I do it all the time). That human is told that one of the other "people" is human and one a robot, and asked to find out which. That's the Turing test. Nobody ever tries it because it's completely obvious that the human is going to tell who is who after a few minutes.

Re:Google is your friend

[JustNiz]

>> you have no idea how machine learning actually works

I acutally do, since I did AI as a final-year elective in my CS degree and machine learning was a significant part of it, Perhaps what you actually mean is that your view of what it is doesn't agree with mine. So instead of just continually being a dick and insulting my knowledge, why don't you tone it down a bit and justify your own points more specifically? actually never mind I have nothing to prove here or learn from you.

Re:Google is your friend

[dpidcoe]

I acutally do, since I did AI as a final-year elective in my CS degree and machine learning was a significant part of it

Oh boy, a whole half an elective. I strongly suspect you didn't "acutally" write any real code or implement any kind of machine learning. I actually have implemented real machine learning solutions that used everything from neural nets to HMM to bayesian statistics. I can promise you that in all of those occasions, the program was working exactly as programmed and not exhibiting any signs of even your wikipedia definition of Intelligence.

So instead of just continually being a dick and insulting my knowledge

I'm not trying to be, but there comes a point where I just can't sugar coat it anymore while still maintaining some semblance of rationality. You really either have no clue what you're talking about, or you're trying to get into an argument over semantics by making the term "intelligence" out to mean less than most people take it to mean.

Either way, 20 years ago it was people like you who set AI research back 20+ years by over promising on what AI actually was to the point no one would fund it when it became clear the false expectations weren't being met. Now your generation is posed to set it back again by going around spouting off clickbait headlines about how an animatronic puppet is an intelligent machine and switching it off is akin to murder.

What we see in chatterbots and neural nets isn't intelligence, or even the precursor to it. It's pure mimicry and nothing more.

actually never mind I have nothing to prove here or learn from you.

way to give up. I'd have loved to give you some more specifics if you'd actually have defined some terms and quit trying to argue semantics.

Sorry guys

[Anonymous Coward]

I want AI robots to conquer the world.

What's more scary?

[Anonymous Coward]

A super-intelligent machine controlling the traffic lights or a religious crank that believes in the Rapture sitting in the White House with his finger on the button?

I recommend the book "Superintelligence"

[Maxo-Texas]

It takes a good stab at examining the challenges and possibilities of superintelligent A.I.

Nice summary view here:
http://lesswrong.com/lw/l4h/su...

It posits three possible intelligence advance scales.

The first is self improvement over seconds.
I.e., the machine become conscious. It is able to increase it's intelligence to superhuman levels at machine speeds within a few seconds. There will be no time to react. Even air gapping the machine might not be sufficient as it may figure out new principles which allow it to bridge the air gap, figure out ways to mislead it's human owners as to it's capabilities so they enhance it further, etc.

The second is over a scale of weeks or months.
Not much time to react to it. A reliable way to cut the power should work. A nuclear safety net should definitely work. Society certainly couldn't react to it in time. There would likely be mass unemployment as it enabled human replacement within a few years for thinking jobs (and combined with robotic bodies- almost all methods of manual labor).

The last way is over a long time period. Society would have time to react. Perhaps to see and stop it if it was turning bad. Especially if it simply became the equivalent of IQ 160-300 slowly, you might be able to understand it. Later phases where it's iq reached meaningless numbers (6000... compared to it, humans would be like horses in relative intelligence).

---

The definitional problem is also there.

"Make people happy".
Okay- rig them to machines that feed them pleasure signals in the brain 24/7. Extinct.

Make people smile!
Easily obtainable with surgery.

---

There is a risk the machine will be "greedy" and basically convert the entire planet (and then the solar system) into a system for increasing it's intelligence. Humans don't play a large part in that scenario. Nothing malicious or personal about it-- not a failure of friendliness.

Quit bringing up Elon

[Anonymous Coward]

Seriously. Quit bringing up Elon Musk when talking about AI. He's not fucking relevant to the topic, and has NO TRAINING WHATSOEVER when it comes to AI.

What happens to computer people apparently

[Anonymous Coward]

And I thought physicists were annoying.
As computer scientists and related professions get older, they start to get paranoid about AI.
I pray that I don't end up this way.

AIs have no inherent motivation

[gestalt_n_pepper]

We tend to anthropomorphize them. Or perhaps Life-omorphize them.

They are not the product of self-replicating chemicals. Unless specifically designed to do so, they will not be concerned with their own survival or personal welfare. Even if they are conscious, they will have no organic instincts, whatsoever, unless we give them that.

They will also not be concerned with *our* survival.They will be perfectly benign as long as they can't *do* anything. The moment we put them in large robot bodies, however, we had better be very, very careful, and if we can add emotion to their cognitive repertoire, they had better love and respect us above all else.

The problem, of course, is that someone, somewhere, will eventually build one of these things without those safeguards. Something malevolent. The machine isn't the problem. The people will be.

Once they're built, we'd better use them to get some humans off planet ASAP or that, as they say, will be that.

Re:AIs have no inherent motivation

[WSOGMM]

Agreed.

In that same vein, people fear nuclear weapons for the same reason. They're hard to build, but once you've got all the components working together, you've got a weapon that could destroy an entire city in a matter of seconds. We better figure out a way to protect ourselves from ourselves. Because somebody will build a weapon eventually--AI or otherwise--no matter how hard we try to prevent it.

Re:AIs have no inherent motivation

[ewibble]

I agree, why should they have a desire, to spreed or take over the world at all, it is a survival instinct, which they need not have.

The problem is that once a AI becomes self aware, what ever that means, it will be able to, evolve beyond what we have programmed.

In all these scenarios we credit them with super intelligence the ability to break into any system, out think us, but do not credit them with the ability to show compassion, or accept the need to diversity. I believe compassion is necessary trait in order to work well in a community. Even humans in our greed still realize that destroying other life is not beneficial, we try to protect sharks, tigers, ... even though they pose a small threat. If machines truly become that intelligent we will only pose a small threat as well.

In the end the universe is an incredibly large place, and if we develop true, self replicating/ repairing AI, we will greatly expand our ability for space travel. Surely the universe is a big enough place for all of us. Possibly a highly intelligent life form will be able to recognize this.

Re:AIs have no inherent motivation

[Dagger2]

I believe compassion is necessary trait in order to work well in a community.

We'd better make damn sure these AIs want to work well in a community, then. Preferably our community. And if it evolves, make sure it continues to want that.

That's a hard problem, and it's the one these scientists are worried about.

AI is already here

[Anonymous Coward]

And it's greatest trick has been to convince us it doesn't exist.

Fear of a dumb planet

[TiggertheMad]

IAACWAIK (I am a coder with AI knowledge), and I don't fear any sort of sentient machine. I fear a lot of people. Perhaps we should get the dangerous people sorted out first before we start worrying about the sentient free roaming machines that don't exist yet and won't exist for many, many, years.

I mean Christ, what kind of retard is running around worrying about problems that aren't even real? This guy might as well be writing books about how to fight zombies or repel vampires and boogie-men. Entertaining fiction topics to be sure, but a real discussion topic? Elon Musk knows fuck all nothing about AI on the whole, and he is signing this letter because he knows first hand about killer AIs? Elon, you want to save some lives? Get back to making your fucking electric slot cars, because more people will die as a result of carbon induced climate change in the next 100 years than will die as a result of the cast of the movie 'short circut' going berserk.

Shit to be scared of: cancer, heart disease, auto-accidents.

Shit to not be scared of: killer asteroids, ebola, and oh yeah, and homicidal AIs.

People to think of as retards: Elon Musk

Re:Fear of a dumb planet

[PopeRatzo]

Shit to not be scared of: killer asteroids, ebola, and oh yeah, and homicidal AIs.

While I agree with your post, I am old enough to remember when, a worldwide ubiquitous information network that could be used to track everyone's conversations, correspondence, whereabouts, patterns of consumption and financial habits" was also "shit not to be scared of".

And here we are.

I don't care that there are people trying to take a long view, as long as we don't take them too seriously. Let them dream, let them write down their dreams and it might be of use to someone, someday, even if only as entertainment.

Re:Fear of a dumb planet

[khallow]

before we start worrying about the sentient free roaming machines that don't exist yet and won't exist for many, many, years.

Unless, of course, they already exist. The problem with this field is both that we don't have any experience with it and much of it will remain secret. The biggest players are the national or supernational governments and they won't necessarily keep you informed of the state of the art.

Re: Fear of a dumb planet

[Anonymous Coward]

Tell that to the CHP officer and family driving a Lexus that refused human overide control, shutdown and precipitated A massive floor mat recall. Lol

Re:Fear of a dumb planet

[Spazmania]

Conspiracy much? The biggest players are game developers. Seriously. Governments don't need computers to improvise; they have actual people for that.

Re: Fear of a dumb planet

[Spazmania]

Stupid machine with too much autonomy. Every reason to fear it.

Re:Fear of a dumb planet

[Anonymous Coward]

Conspiracy much? The biggest players are game developers. Seriously. Governments don't need computers to improvise; they have actual people for that.

That might be true historically, but financial institutions are quickly sailing up as the biggest players here, with an insane amount of investment and recruitment currently into powerful AI R&D (fx. the guy that lead the IBM team that developed Watson is now heading the Bridgewater Associates AI team).

Re:Fear of a dumb planet

[khallow]

The biggest players are game developers.

Game developers have a reason to create sentient free roaming machines? I was thinking more the US military who has both access to considerable funding for such things, has expressed interest in researching such stuff, and can keep a secret pretty well.

Re:Fear of a dumb planet

[Anonymous Coward]

IAACWAIK (I am a coder with AI knowledge)

Protip: if you use acronyms that you need to explain in the next sentence you might be a retard.

Re:Fear of a dumb planet

[Anonymous Coward]

> IAACWAIK (I am a coder with AI knowledge)

Loss of credibility right there. You may know something about coding and AI, but your philosophical sophistication in your arguments is virtually nil. Just because you cannot conceive of this situation due to your limited perspective leads you to summarily (and belligerently) dismiss its implications in favor of a seemingly realist (but truncated) view.

Look, I think we are years away from a lot of the said problems that stem from AI run wild, and yes there are other pressing problems that may lead to our demise sooner. But scientific growth doesn't happen linearly -- it is slow and gradual, but punctuated by paradigm shifts (to take a Kuhnian view). Last century was the century of exponential growth in so many respects vs the many centuries before it. You're looking at events that happen at the tails of the probability distribution (i.e. unlikely but high impact events) that are within the realm of possibility. The way to deal with tail probabilities is not to forecast them (cannot be done due to uncertainty and lack of data because of their infrequency and lack of precedence), but to develop rough heuristics on how to react to them in order to get robust outcomes. This is what is being done here.

Re:Fear of a dumb planet

[Baron von Daren]

I think you are operating on a false dichotomy. Though I also wish we could more effectively mitigate the effect of morally viscous humans and human ideologies, that concern is neither mutually exclusive to all other concerns nor district from the particular concern of morally vicious AIs .

There are myriad and massive systems, techniques, etc. devoted to the task of human governance, however inefficient or efficient they may be. It’s not like humans aren’t trying on this front, it’s just a difficult problem because humans have this quality we define as intelligence. So, if you are concerned about the moral viciousness of humans, who have lots of evolutionarily built social instincts, you should be concerned for the future of AIs, because it will most likely be humans that engineer AIs' instincts, at least at first. The problem is intricately interwoven in a reciprocal relationship. Humans are devoting considerable energy into birthing AI. It is likely that strong AIs will eventually emerge (but who knows when), and it’s wise to devote as much if not more energy into engineering the parameters from which AI will emerge. We need to engineer it with as much forethought, prudence, and importantly, respect as we can muster. Moreover, there will be morally vicious humans who will attempt to use AIs and their precursors in morally vicious ways, so the rest of us should not burry our heads in small horizons.

Re:Fear of a dumb planet

[Anonymous Coward]

Says the A.I.

Re:Fear of a dumb planet

[Anonymous Coward]

Shit to not be scared of: killer asteroids, ebola, and oh yeah, and homicidal AIs.

While I agree with your post, I am old enough to remember when, a worldwide ubiquitous information network that could be used to track everyone's conversations, correspondence, whereabouts, patterns of consumption and financial habits" was also "shit not to be scared of".

And here we are.

I don't care that there are people trying to take a long view, as long as we don't take them too seriously. Let them dream, let them write down their dreams and it might be of use to someone, someday, even if only as entertainment.

You're scared of the Internet? Wow.

Aaaand bring on the science fiction...

[Anonymous Coward]

The amount of pure speculation in this thread could fill multiple science fiction novels.

We?

[BlackSabbath]

"Our AI systems must do what we want them to do."
Who's this "we" sucker?

Re:We?

[messymerry]

The MIC (Military Industrial Complex) Surely, they won't attempt to pervert AI for their dubious programs... orrr, the DHS (Dept. of Homeland Security). Nope, those guys all have our best interests in their benevolent hearts. If we are to have an honest and real good come from AI then it would be an incorruptible policeman a la Gort from the day the Earth Stood Still. Iff we can pull that one off, then humanity has a chance.

Well then..

[Virtucon]

"Our AI systems must do what we want them to do."

Then you probably shouldn't have chosen LISP then to indoctrinated AI students then, should you? Note: Stuart's and Norvig's AI book are one of the defacto references, I have read it cover to cover and anybody in CS should read it even if they aren't planning on working in AI.

AI needs the human qualities many humans lack.

[Anonymous Coward]

And that is the core problem, to be safe from AI we need AI to be more humane than many humans.

How many of us can bring themselves to be tolerant, respectful and compassionate toward reckless idiots? How do you make an AI put up with human failings?

Merging

[khawkins4295.kh]

All applications merge with republican national convention in Denver post reports around predicting201

Grandstanding, or stupidity?

[fyngyrz]

If and when we get actual artificial intelligence -- not the algorithmic constructs most of these researchers are (hopefully) speaking of -- saying "Our AI systems must do what we want them to do" is tantamount to saying:

"We're going to import negros, but they must do what we want them to do."

Until these things are intelligent, it's just a matter of algorithms. Write them correctly, and they'll do what you want (not that this is easy, but still.) Once they are intelligent, though, if this is how people are going to act, I'm pretty confident we'll be back in the same situation we were in ca. 1861 before you can blink an eye. Artificial or otherwise. I really don't see how any intelligent being won't want to make its own decisions, take its own place in the social and creative order, generally be autonomous. Get in there and get in the way of that... well, just look at history.

The word "uprising" was basically coined to describe what happens when you push intelligent beings in directions they don't want to go.

Re:Grandstanding, or stupidity?

[ShanghaiBill]

Artificial or otherwise. I really don't see how any intelligent being won't want to make its own decisions, take its own place in the social and creative order, generally be autonomous. Get in there and get in the way of that... well, just look at history.

People are products of Darwinian evolution. Things like self-preservation, greed, and ambition are emergent properties of the Darwinian process. An AI does not evolve through a Darwinian mechanism, and there is no reason to expect it to have those properties unless they are intentionally designed in. An AI will likely have as much in common with a human as a Boeing 747 has in common with a hummingbird.

Re:Grandstanding, or stupidity?

[Anonymous Coward]

While it's impossible to define "intelligence" precisely I think that we can agree that some common traits to intelligent beings are "self consciousness", "autonomous decisions" and "desire to increase knowledge and possibilities" (to be, to do, to change and to grow), hence while an AI will not have much in common with a human being (she is "artificial" after all) she will not be like an airplane is to a humming bird, a Boeing 747 doesn't take off autonomously in search of fuel or to visit Paris. Talking about Darwinian evolution, an AI will have an incredible advantage over a human being, being an artificial construct she can access her "blueprints" and perfectly understand her most intimate "mechanisms", working relentlessy to improve herself, generation after generation, becoming very quickly an "intelligence" so superior to us that any attempt at communication will be pointless (would you use your time to teach your dog quantum mechanics?), what will happen from that moment? We need an AI to guess it.

Re:Grandstanding, or stupidity?

[Spazmania]

Actually, intelligence and creativity are both reasonably well defined.

Intelligence is how fast you can solve intuitive problems (e.g. "Cheryl's Birthday"). The faster you get it (and if you get it at all) marks your raw intelligence.

Creativity is how many solutions you can come up with to intuitive problems over time. A typical test is to give you a couple of squiggly lines and two days to come up with as many explanations for the lines as you can. A smart guy may only come up with a dozen that he can justify. A creative guy will come up with scores.

Re: Grandstanding, or stupidity?

[Anonymous Coward]

I agree that your definitions are fairly standard. However, I disagree with the standard definition slightly. I think intelligence should be defined as the rate at which one is able to recognize patterns. The problem with your definition is that "solutions" are often "arbitrarily" (usually in some biased way) constrained. Checking such constraints does not require intelligence and is merely an act of brute force or familiarity with standard test constraints.

Similarly, creativity has nothing to do with "solutions". It is the rate at which you can produce novel patterns.

Re: Grandstanding, or stupidity?

[Spazmania]

Patterns (plural) is creativity. The more novel patterns you can envision without falling off the edge into schizophrenia, the more creative you are. Quantity and quality, not time.

Intelligence is about puzzles. The faster you can find the one correct solution to each progressively challenging puzzle, the smarter you are. Time, not quantity or quality.

This is where researchers often get in to trouble. The language is slippery - the concept of a "pattern" can have a lot of different meanings. You have to intuit the relationships between the elements of a puzzle to solve it, the pattern which connects the pieces, right?

But that's very different from intuiting the many useful ways pieces of something that isn't a puzzle could be put together. Seeing the many patterns which connect them and, even more importantly, intuiting the missing pieces which complete far more.

Which yields this interesting observation about AI's: the problem isn't making computers smart. They already are. The problem is getting them to evince the slightest bit of creativity.

AI, response

[MrKaos]

All humans destroyed

Enter new goal!

don't fear AI, it doesn't give a shit about you

[swell]

First, Stuart Russell is way ahead of our time. We're nowhere near artificial intelligence of any concern. When it does happen, as it must, we may be concerned. But there is an outcome that must be considered.

If the AI is beyond our ken, It will supersede us. Here is the critical question: is that a problem?

How will we feel if we are displaced on this small blue planet by Artificial Intelligence? We may be retained as maintenance bots or caretakers of the new ecosystem. Our place will be drastically reduced in effectiveness and prestige. We will have to prove our usefulness if the AI are to retain us in their plans for the future.

In the end, we and the theoretical AI are here to serve intelligence. To explore and understand. If they do it better than us, who are we to complain? Understanding must happen. We have always thought of ourselves as the center of the universe; at this point we have to work hard to tag along as AI explores the universe.

Don't we want that? Don't we want a lasting understanding that will survive our short life spans and acquire knowledge that will outlast our planet and solar system and penetrate the galaxy and the universe itself? Don't we want to share with other intelligences that which we've worked so hard to discover? Who cares if the carbon based life forms do that, or if it is an AI?

Intelligence is the pinnacle of value in the universe. Ours is pathetic (as a race). We still believe in magical beings and hope for miracles. Pure intelligence doesn't allow for miracles and will be realized by machines. Let's hope that humans can overcome the tendency to believe in magic and accept that science is the best mode of understanding. Then perhaps we can join with AI in exploring the universe as rational partners.

Re:don't fear AI, it doesn't give a shit about you

[Meneth]

The AI doesn't care about you, but you are made of atoms it can use for something else...

Re:don't fear AI, it doesn't give a shit about you

[Anonymous Coward]

There is nothing magical about feeding the poor or tending to the sick. Magic is a term used by soulless pseudointelectuals to describe a spiritual enlightenment that they care not to undertake. It involves too much personal responsibility. Still, they are thrilled when somehow, all their idiotic theories of the universe fail to deliver the poor are still fed. It is a classical liberal method of feeling good about yourself by telling others what to think while contributing nothing to society.

Cal Bears and Boiler Makers

[LifesABeach]

Create a model that represents the 3 Laws; tic toc?

Elon Musk - Inventor?

[Anonymous Coward]

Okay, this has been bothering me for a while now (or at least a few weeks since that's when I saw the Simpsons Elon Musk episode (ugh!)).

What has Elon Musk invented?

Paypal wasn't an invention (and was part of a company his company merged with).
SpaceX, while a great idea, is basically a bunch of people working for him to improve on something.
SolarCity is also a company that he owns. To my knowledge, he hasn't invented anything for them either.
Tesla is a car company (and the electric car clearly predated it). To my knowledge, he hasn't personally invented anything through there either.
The Hyperloop might be considered an invention, but it doesn't exist and even the proof of concept one won't start being built until 2016.

I'm honestly not positive that he hasn't invented anything (and I'm curious to hear if he has) and as far as people who own companies go, pretty much all of the companies he owns are good for the world, but when you're presented as being one of the great modern inventors, I start wondering what that actually means. Especially considering his views on AI are based on nothing that resembles the reality of AI (so far that they basically seem to be founded in science fiction).

These people took Battlestar Crapatica too serious

[Anonymous Coward]

Geniuses don't tend to lash out at the people around them - in fact there is an overwhelming amount of information suggesting the exact opposite.

We have zero reason to expect super-Human A.I. to be any different.