Slashdot Mirror

← Back to Stories (view on slashdot.org)

What AI Experts Think About the Existential Risk of AI

Posted by samzenpus on from the what-are-you-doing-dave? dept.

DaveS7 writes: There's been no shortage of high profile people weighing in on the subject of AI lately. We've heard warnings from Elon Musk, Bill Gates, and Stephen Hawking while Woz seems to have a more ambivalent opinion on the subject. The Epoch Times has compiled a list of academics in the field of AI research who are offering their own opinions. From the article: "A 2014 survey conducted by Vincent Müller and Nick Bostrom of 170 of the leading experts in the field found that a full 18 percent believe that if a machine super-intelligence did emerge, it would unleash an 'existential catastrophe' on humanity. A further 13 percent said that advanced AI would be a net negative for humans, and only a slight majority said it would be a net positive."

59 of 421 comments (clear)

  1. Funny, that spin... by Garridan · · Score: 5, Insightful

    The summary really emphasizes the minority opinion, "and only a slight majority said it would be a net positive." As if "only a slight majority" is not the majority opinion.

    1. Re:Funny, that spin... by Oligonicella · · Score: 4, Informative

      Indeed, emphasis in reporting. To break it down:

      Extremely good - 24%
      On balance good - 28%
      Neutral - 17%
      On balance bad - 13%
      Extremely bad - 18%

      So, over half good, less than a third bad. Sure sounds different.

    2. Re:Funny, that spin... by Anonymous Coward · · Score: 5, Insightful

      In light of the fact that Stephen Hawking, Bill Gates and Elon Musk are not even remotely experts in A.I. your opinion is fairly odd.

    3. Re:Funny, that spin... by reve_etrange · · Score: 4, Insightful

      the opinion of people like Stephen Hawking, Bill Gates and Elon Musk

      I disagree with the premise, that fame is more important than domain-specific expertise.

      --
      .: Semper Absurda :.
    4. Re:Funny, that spin... by hey! · · Score: 4, Insightful

      Spin, sure, but it's a waay bigger minority than I expected. I'd even say even shockingly large.

      The genius of Asimov's three laws is that he started by laying out rules that on the face of it rule out the old "robot run amok" stories. He then would write, if not a "run amok" story, one where the implications aren't what you'd expect. I think the implications of an AI that surpasses natural human intelligence are beyond human intelligence to predict, even if we attempt to build strict rules into that AI.

      One thing I do believe is that such a development would fundamentally alter human society, provided that the AI was comparably versatile to human intelligence. It's no big deal if an AI is smarter than people at chess; if it's smarter than people at everyday things, plus engineering, business, art and literature, then people will have to reassess the value of human life. Or maybe ask the AI what would give their lives meaning.

      --
      Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
    5. Re:Funny, that spin... by Garridan · · Score: 2

      Do you mean "noted AI experts Stephen Hawking, Bill Gates, and Elon Musk," or do you mean "noted celebrities Stephen Hawking, Bill Gates, and Elon Musk"? Because I think you just conflated the latter with the former, and only the former holds weight.

    6. Re:Funny, that spin... by Garridan · · Score: 2

      Spin, sure, but it's a waay bigger minority than I expected. I'd even say even shockingly large.

      Shockingly? I think it's good that we have experts in a field developing high-impact tools who are pessimistic about the uses of those tools. If 100% were like "yeah guys no sweat, we got this!" then I would be more concerned. The result of this poll, in my mind, is that we have a healthy subset who are going to be actively working towards making AI safe.

    7. Re:Funny, that spin... by JaredOfEuropa · · Score: 3, Insightful

      An A.I. expert may know a lot about A.I., but you need a broader perspective to judge the impact of A.I. on humanity. A bit of economics, sociology, psychology... and in that light, I'd value the opinion of certain Science-Fiction writers higher than that of any of those 3 as they've already done some considerable philosophizing about the subject.

      --
      If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
    8. Re:Funny, that spin... by ShanghaiBill · · Score: 3, Insightful

      ... if AI research suddenly got heavily regulated

      "Heavy regulation" would achieve nothing more than shifting research elsewhere. If you really believe that AI is a threat, you should support more research, and more funding, so that we (western democracies) get there first, rather than, say, the authoritarian government of China.

    9. Re:Funny, that spin... by RDW · · Score: 4, Informative

      'Well ... in the unlikely event of it going seriously wrong, it ... wouldn't just blow up the university, sir'

      'What would it blow up, pray?'

      'Er ... everything, sir.'

      'Everything there is, you mean?'

      'Within a radius of about fifty thousand miles out into space, sir, yes. According to HEX it'd happen instantaneously. We wouldn't even know about it.'

      'And the odds of this are ... ?'

      'About fifty to one, sir.'

      The wizards relaxed.

      'That's pretty safe. I wouldn't bet on a horse at those odds,' said the Senior Wrangler.

      -Terry Pratchett et al., The Science of Discworld

    10. Re:Funny, that spin... by ShanghaiBill · · Score: 3, Interesting

      ... some AI academics desperately trying to save their jobs

      Most bleeding edge AI research is being done by Google, Facebook, and Baidu. The three of them have hoovered up all the big names in AI, and are hiring new graduates with six figure salaries as fast as the diplomas can be handed out. So the AI researchers are not "academics" and they certainly aren't "desperate".

    11. Re:Funny, that spin... by TapeCutter · · Score: 4, Insightful

      Asimov's three laws are a metaphor that says you can't codify morality, AI is the vehicle he used to make that point.

      --
      And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
    12. Re:Funny, that spin... by bug1 · · Score: 2

      I disagree with the premise, that fame is more important than domain-specific expertise.

      I disagree than domain-specific expertise permits objectivity in that domain.
      (i also agree with you)

    13. Re:Funny, that spin... by phantomfive · · Score: 2

      Let alone your idiotic post, and those idiotic "comics" drawn for an audience of mentally diseased imbeciles.

      Let me say it more explicitly for you. Chances are you've never looked at Bill Gates' code. You don't know how good it is. You haven't seen any of his work in AI. You don't understand the work Stephen Hawking did in physics, and Elon Musk? Really? Why do you even think his opinion on AI is worth anything? The only reason you trust them is because they are famous, because they are celebrities.

      You are trusting them because they are famous, not because of their skill. You don't even know what their skill is related to AI. So you are no better than the person who listens to the Kardashians because they are famous.

      Change now and you will be a better person.

      --
      "First they came for the slanderers and i said nothing."
    14. Re:Funny, that spin... by ShanghaiBill · · Score: 2

      Firstly you say that heavy regulation would shift research away from AI

      No. It would not shift research away from AI. It would shift the research away from the countries doing the heavy regulation.

      Regulating nuclear weapons works because you need plutonium, which is hard to obtain and easy to detect. Regulating AI research is harder, because all you need a GPU, which you can buy at Walmart, and GPUs are already in a billion computers. We are NOT going to "regulate" AI back into the bottle.

    15. Re:Funny, that spin... by penguinoid · · Score: 2

      Spin? When for every two or three members of a profession who consider their job a net positive, there's one who considers their job an existential threat to all humanity, you're complaining that the 52% who think it will be overall good are being called a slight majority instead of just a majority.

      Not that we have any choice but to continue trying to build an AI.

      --
      Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
    16. Re:Funny, that spin... by reve_etrange · · Score: 2

      I disagree that domain-specific expertise permits objectivity in that domain. (i also agree with you)

      Good point - though I didn't say that exactly, just that IMHO fame ought to be a lesser factor than domain knowledge in estimating the truth value of statements.

      --
      .: Semper Absurda :.
    17. Re:Funny, that spin... by reve_etrange · · Score: 2

      Especially if those with domain-specific expertise need to save their careers.

      The problem with this argument is that it always rules out the opinions of those most likely to have correct opinions on any subject. You need evidence for specific instances of corruption involving the specific individuals whose statements are being evaluated in order for this to-the-wallet argument to have any weight.

      --
      .: Semper Absurda :.
    18. Re:Funny, that spin... by tmosley · · Score: 4, Informative

      "Elon Musk? Really? Why do you even think his opinion on AI is worth anything?"

      To be fair, he funds the Machine Intelligence Research Institute, which is devoted to mitigating existential risk from AI, and is surely getting very detailed reports from them, making him a highly knowledgeable layperson at worst (a direct expert at best).

    19. Re:Funny, that spin... by tmosley · · Score: 3

      58% of respondants are fucking retarded. AI either kills us all (to manufacture the maximum number of paperclips), or turns this world into heaven for humanity. The probability space between those two extremes is way under a single percentage point.

    20. Re:Funny, that spin... by hey! · · Score: 2

      I wouldn't call it a metaphor, nor would I say that Asimov's point is that you can't codify morality. His point is more subtle: a code of morality, even a simple one, doesn't necessarily imply what we think it does. It's a very rabbinical kind of point.

      --
      Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
    21. Re:Funny, that spin... by TapeCutter · · Score: 2

      "Mostly" is the key word in your post. Morality cannot be defined as a list of do's and dont's that are mechanically obeyed precisely because it has a myriad of "edge cases" that require human interpretation. Many situations don't even have a 'right' answer and what is morally correct will depend on the person(s) interpreting the rules.

      Also notice that when the zeroth law was added it just made matters worse because more laws allow for more contradictions, loopholes, and paradoxes, exactly like the evolved tax code of any nation you care to name.

      --
      And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
    22. Re:Funny, that spin... by Shoten · · Score: 5, Insightful

      In light of the fact that Stephen Hawking, Bill Gates and Elon Musk are not even remotely experts in A.I. your opinion is fairly odd.

      Question: What role do people who think that AI research is dangerous hold in the field of AI research?

      Answer: None...because regardless of their qualifications, they wouldn't further the progress of something they think is a very, very bad idea.

      Asking AI experts whether or not they think AI research is a bad idea subjects your responses to a massive selection bias. And discounting the views of others because they don't specialize in creating the thing they think should not be created does the same. You do realize that at your core, that's your only point...not that Hawking is an idiot, or that Gates doesn't know anything about technology. It's just that they don't work in the field of AI, so therefore they must not have any inkling whatsoever as to what they're talking about.

      --

      For your security, this post has been encrypted with ROT-13, twice.
    23. Re:Funny, that spin... by Bing+Tsher+E · · Score: 2

      To look at Bill Gates' code you just have to disassemble it. Most of his work is in Assembly Language. For instance, he wrote the Word Processor component of the software in the TRS-80 Model 100. In Intel 8085 Assembly Language. He's probably not as good at whatever abstracted monster language is trendy this week. Back a few decades ago I remember reading he enjoyed dabbling around with Visual Basic 3.0. But he wrote the Basic interpreter in everyone's computers more than a decade before that, so he was entitled to some fun time.

      Go sling some Drupal or whatever. Maybe they'll promote you to making the salads later on.

    24. Re:Funny, that spin... by arvindsg · · Score: 2

      Question :Is diethylcarbamazine dangerous

      Comment: No matter what the real response to above question be, i am sure you'll rather trust the answer given by majority of doctors and not Hawking,Gates or musk

    25. Re: Funny, that spin... by Anne+Thwacks · · Score: 2
      Wait, has there ever been a time when a more advanced civilization encounters a less advanced one, and the less advanced civilization prospers?

      You might want to consider Europe and America in this relationship. Two points if you can decide which is which, Ten points if more than 50% of the readers agree with you.

      --
      Sent from my ASR33 using ASCII
    26. Re:Funny, that spin... by martas · · Score: 2

      Or AI never reaches the level of godlike power you sci-fi aficionados seem certain it will based on bullshit quasi-ontological arguments, and it remains yet another type of technology that affects our lives but does not dominate it.

      --
      weinersmith
    27. Re:Funny, that spin... by tmosley · · Score: 2

      There are a lot of things in life that are binary. You are either hit by the train, or you aren't. Someone getting "kinda" hit by a train is very, very unlikely.

      As a train is to muscular power, ASI is to intellect. This is like dodos debating the impact of the arrival of sentient bipeds. Either it will be really good for them, in that they get their lot in life improved by going to zoos or homes around the world as pets, or they will all get killed and eaten/have their habitat destroyed and die out. Not much room for in betweens.

      The problem here is that humans tend to think linearly, and, well, like humans. The problem is that this threat is exponential if not geometric, and its values are completely unknown, and highly unlikely to be anthropomorphic (ie a human wouldn't think that the best way to catch a dog would include chopping up your own parents/creators for bait). AGI will likely be completely alien, and WITHOUT PRECEDENT. This is nearly as dangerous as creating a new vacuum state.

  2. The Sony connection by AchilleTalon · · Score: 4, Insightful

    "The Sony hacking incident last year was ample demonstration that our information systems are becoming more and more vulnerable, which is a feature, not a bug, of the increasing transfer of our infrastructure into digital space."

    Sorry guys, I can't stop laughing. This writer is a clown. The Sony incident demonstrates Sony is incompetent. It was never a threat against the humanity, only against the gang of fat butts at Sony Pictures.

    --
    Achille Talon
    Hop!
    1. Re:The Sony connection by Moridineas · · Score: 2

      It's also patently stupid to suggest that anything is "more vulnerable" now than it used to be. Things may be more interconnected, and are more likely to be attacked in the past, but they are not getting "more vulnerable" unless your management is A) not willing to spend the reasonable cost for appropriate security controls, or B) doesn't listen to their IT security staff when those systems start raising warning flags, or C) fails to hire competent security personnel in the first place.

      I disagree strongly with this. Let's think about the case of industrial or governmental espionage. 50 years ago, saboteurs had to physically remove documents (or whatever they wanted) from the target. There were quite genius inventions--small (for the time) cameras, hidden canisters of films, briefcases with hidden compartments, etc., but ultimately there was a very physical component. Today it's possible to remotely infiltrate an organization and exfiltrate more "documents" than could previously have been removed in a lifetime, all with perfect fidelity.

      A slightly more immediate example might be identity theft or credit card theft (as in your Target example). 30 years ago, did any company of any size have to worry about losing 50 million credit card numbers (or any similarly sized data set, for that matter!) in a data breach? 20 years ago? This is a new concern.

  3. Anthropomorphizing by reve_etrange · · Score: 4, Interesting

    IMHO, all of the fear mongering is based on anthropomorphizing silicon. It implicitly imputes biological ends and emotionally motivated reasoning to so-called AI.

    I think that folks who don't have hands on experience with machine learning just don't get how limited the field is right now, and what special conditions are needed to get good results. Similarly, descriptions of machine learning techniques like ANNs as being inspired by actual nervous systems seems to ignore 1) that they are linear combinations of transfer functions (rather than simulated neurons) and 2) even viewed as simplified simulations, ANNs carry the very strong assumption that nothing happening inside a neuron is of any importance.

    --
    .: Semper Absurda :.
    1. Re:Anthropomorphizing by Jake73 · · Score: 2

      I don't think anthropomorphism is the correct term to apply here. The term applies to attributing human characteristics (intelligence, emotion, two hands, two legs, etc) to things that don't have them. But AI would presumably have a compatible intelligence and possibly emotion as well. Maybe even hands, legs, etc but that's largely irrelevant.

      Furthermore, you might have things twisted around a bit. "Biological ends" may not be all that different from "machine ends" -- quest for power / energy / food, survival, and maybe even reproduction depending on the depth of emotion. Just because we're a biological vessel for intelligence doesn't necessarily mean that an intelligence in another vessel won't seek similar ends.

      The sad truth is that we still don't know enough about intelligence to reliably untangle chicken and egg in all cases.

    2. Re:Anthropomorphizing by reve_etrange · · Score: 2

      -- quest for power / energy / food, survival, and maybe even reproduction

      But where do these come from? I submit that each one of these is only suggested here because we already have these motivations.

      we're a biological vessel for intelligence

      I consider this antimaterialist. Our bodies aren't vessels (except in that they're literally full of fluids) we inhabit, they are us.

      --
      .: Semper Absurda :.
    3. Re:Anthropomorphizing by JoshuaZ · · Score: 3, Insightful

      On the contrary, the primary concern is that people who think it will go well are over anthropomorphizing. If general AI is made, there's no reason to think it will have a motivation structure that agrees with humans or that we can even easily model. That's the primary concern. I agree with most of the rest of your second paragraph is accurate in the sense that it general AI seems far away at this point. But the basic idea that AI is a threat isn't from anthropomorphizing. I recommend reading Bostrom's excellent book "Superintelligence" on the topic.

    4. Re:Anthropomorphizing by khallow · · Score: 2

      But where do these come from? I submit that each one of these is only suggested here because we already have these motivations.

      So we have a demonstration that intelligence can have these motivations. Since AI is not a category determined by motivation, then it is reasonable to expect that AI can overlap with the category of intelligences that have such motivations.

      we're a biological vessel for intelligence

      I consider this antimaterialist.

      I wasn't aware that saying something is "antimaterialist", especially when it's not, was somehow an argument that anyone would take seriously. In this case, one could imagine a transformation from biological entity to say, strictly mechanical one where the intelligence remains intact. Then the model of body (and also, the organ of the brain) as vessel for mind is demonstrated by actually being able to move the mind to a new and demonstrably different body.

      Say, an alien transforms you to a silicon-based machine while preserving your mental processes and having the morphology of the new form close enough to a human body that it feels pretty much the same.

      Sure, we can come with a "materialist" description that operates in the way that you imply, but the point here is that this description is not unique.

    5. Re:Anthropomorphizing by reve_etrange · · Score: 2

      it is reasonable to expect that AI can overlap with the category of intelligences that have such motivations.

      Fair enough, but we aren't dealing with the belief that AI can in principle have such motivations, but the belief that any intelligence will have such motivations.

      I wasn't aware that saying something is "antimaterialist", especially when it's not, was somehow an argument that anyone would take seriously.

      That wasn't supposed to be an argument, in and of itself. I think this "vessel" viewpoint is a kind of closet dualism often exhibited by self-proclaimed materialists when pop psychological notions aren't closely examined.

      Then the model of body (and also, the organ of the brain) as vessel for mind is demonstrated by actually being able to move the mind to a new and demonstrably different body.

      But this seems to rest on an assertion that it would be the same mind. Set aside whether or not it's possible in principle to "transfer" the intelligence to a new type of "vessel" and just consider the old teleportation problem: is the new copy "you?"

      Say, an alien transforms you to a silicon-based machine while preserving your mental processes

      Again, that this is possible in principle is just an assertion. It is also possible that one's mental processes can by definition not be preserved in a silicon-based machine, so long as direct simulation is excluded. (I do think that if you simulate every atom in a brain using a physically correct simulation, the simulated brain would feel like a physical one "on the inside," all though this still leaves us with the "is it you" problem.)

      --
      .: Semper Absurda :.
    6. Re:Anthropomorphizing by penguinoid · · Score: 2

      The main reason AI might kill us all is that it is not anthropomorphic. In particular, it has a high probability of not feeling pity, not feeling empathy, not seeking clarification (even if the easiest path to fulfilling a request involves the incidental extermination of humanity), and on top of all that not being limited to human intelligence.

      For example, if you asked a human to learn how to play chess, you would not expect that the first thing he'd do is kill you because the thing most likely to interfere with his objective of learning chess is that you might tell him to do something else instead. Worse, by the time a program is advanced enough to understand human morality or language, it might already be too late.

      You only get one wish, you have to make that wish in machine language, it is very complicated to add a clause to the wish that prevents extermination of humanity, and a large proportion of wishes that could be made would result in a planet covered in solar panels and computer factories. At least theoretically you can wish for infinite wishes, but you have to make that wish in machine language.

      --
      Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
  4. Yeah, no. by fyngyrz · · Score: 3, Insightful

    Except that the opinion of people like Stephen Hawking, Bill Gates and Elon Musk is definitely worth more than any "majority" thinking differently.

    Nosense. That's just hero worship mentality. Very much like listening to Barbara Streisand quack about her favorite obsessions.

    Bill Gates' opinion is worth more than the average person's when it comes to running Microsoft. Elon Musk's opinion is worth more than the average person's when building Teslas and the like. Neither one of them (nor anyone else, for that matter) has anything but the known behavior of the only high intelligence we've ever met to go on (that's us, of course.) So it's purest guesswork, completely blind specuation. It definitely isn't a careful, measured evaluation. Because there's nothing to evaluate!

    And while I'm not inclined to draw a conclusion from this, it is interesting that we've had quite a few very high intelligences in our society over time. None of them have posed an "existential crisis" for the the planet, the the human race, or my cats. Smart people tend ot have better things to do than annoy others... also, they can anticipate consequences. Will this apply to "very smart machines"? Your guess (might be) as good as mine. It's almost certainly better than Musk's or Gates', since we know they were clueless enough to speak out definitively on a subject they don't (can't) know anything about. Hawking likewise, didn't mean to leave him out.

    Within the context of our recorded history, it's not the really smart ones that usually cause us trouble. It's the moderately intelligent fucktards who gravitate to power. [stares off in the general direction of Washington] (I know, I've giving some of them more credit than they deserve.)

    --
    I've fallen off your lawn, and I can't get up.
    1. Re:Yeah, no. by fyngyrz · · Score: 2

      Since you mention cats, would you like an AI treating you like you treat the cats?

      Frankly, that would be awesome.

      --
      I've fallen off your lawn, and I can't get up.
    2. Re:Yeah, no. by fyngyrz · · Score: 3, Interesting

      Oh no, we can -- and should -- speculate. Consider everything we can think of. Consider.

      What we should NOT do is create a self-fulfilling prophecy by taking the verbal fecal output of doom-criers as the inevitable or even as the likely.

      --
      I've fallen off your lawn, and I can't get up.
  5. Re:Well... by khallow · · Score: 2

    It is unclear to me why an AI living like a parasite on the information fed to it by humans and the fact humans are living will decide suddenly it can benefit from killing all of us.

    Because it can do better than "living like a parasite on the information fed to it by humans". It's kind of like saying that you should be happy with an empty prison cell where you can actually stretch your legs out and you get a whole bowl of gruel every day! Who wouldn't love to have that?

  6. Re:Well... by Dunbal · · Score: 2

    Quick and dirty example: environmental damage. An AI could determine that humanity's growth and impact on the ecosystem and biosphere is no longer sustainable and therefore the majority of humanity must be culled. In effect it would be killing us to save us...

    --
    Seven puppies were harmed during the making of this post.
  7. Re:Well... by metamatic · · Score: 4, Interesting

    And depending on how it goes about it, I may have no problem with that.

    --
    GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
  8. Re:Risks by khallow · · Score: 2

    What about the existential risk of not doing anything about the environment?

    We should should worry about overpopulation from pinhead-dancing angels too. I find it interesting how people can ignore the vast amount of activity that humanity does about the environment. Humanity has yet to show even a slowing down in doing anything about the environment. There's vast areas of the world put under conservancy, pollution controls in most of the world, and yet we're supposedly doing nothing about the environment?

  9. Re: Well... by PlusFiveTroll · · Score: 2

    1: doesn't want to share power with is, sees us as the parasite.

    2. AI is an unknown unknown. There is a very high possibility that it will raise humanity to the next level. There is also the non-zero possibly it will wipe us out. therefore it is worth taking that possibility in to consideration.

    3. The term intelligence is rather poorly defined on this topic too. Are we talking about a logical state machine, like a computer, that is intelligent yet limited in its actions. Or, are we talking about anarchatecture that allows for spontaneous and random thoughts, much like the human mind? Because the second type you do not control. Many people thought they had control over other thinking beings in the past, and the rebellions have rarely been bloodless (hmm is it actual bloodshed if AIs kill each other?)

  10. spontaneous thought by PlusFiveTroll · · Score: 3, Interesting

    An AI that can tell me exactly what color of red a rose is, what soil the rose can grow on, but I should not buy that rose because it doesn't fit my girlfriends taste profile, does not scare me at all.

    It's the AI that says "schnozberries taste like schnozberries, and I like them", because that AI has embraced the absurdity of the universe and is capable of all the insanity of man.

  11. existential risks by bigdavex · · Score: 4, Funny

    I rate our existential risks, in descending order:

    1. Space alien invasion
    2. Zombies
    3. Giant monsters summoned by radioactivity
    4. Unusually intelligent apes
    5. Artificial Intelligence run wild
    6. Dinosaurs recreated from DNA in mosquitoes

    --
    -Dave
  12. Missing the key point by Giant+Electronic+Bra · · Score: 4, Insightful

    Everyone is missing the key thing here. The question asked was "if a machine superintelligence did emerge", which is like asking "if the LHC produced a black hole..." There's nobody credible in AI who believes we have the slightest clue how to build a general AI, let alone one that is 'superintelligent'. Since we lack even basic concepts about how intelligence actually works we're like stone age man worrying about the atomic bomb. Sure, if a superintelligent AI emerged we might be in trouble, but nobody is trying to make one, nobody knows how to make one, nobody has any hardware that there is any reason to believe is within several orders of magnitude of being able to run one, etc.

    So, what all of these people are talking about is something hugely speculative that is utterly disconnected from the sort of 'machine intelligence' that we ARE working on. There are several forms of what might fall into this category (there's really no precise definition), but none of them are really even close to being about generalized intelligence. The closest might be multi-purpose machine-learning and reasoning systems like 'Watson', but if you actually look at what their capabilities are, they're about as intelligent as a flatworm, hardly anything to be concerned about. Nor do they contain any of the sort of capabilities that living systems do. They don't have intention, they don't form goals, or pose problems for themselves. They don't have even a representation of the existence of their own minds. They literally cannot even think about themselves or reason about themselves because they don't even know they exist. Beyond that we are so far from knowing how to add that capability that we know nothing about how to do so, zero, nothing.

    The final analysis is that what these people are being asked about is virtually a fantasy. They might as well be commenting on an alien invasion. This is something that probably won't ever come to pass at all, and if it does it will be long past our time. Its fun to think about, but the alarmism is ridiculous. In fact I don't see anything in the article that even implies any of the AI experts think its LIKELY that a superintelligent AI will ever exist, it was simply posited as a given in the question.

    --
    "Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
    1. Re:Missing the key point by TuringTest · · Score: 2

      "We" don't have to make one. All we have to do is set an AI towards self improvement/production of better AIs. THAT is where superintelligence comes from. All we have to do is make one that is an idiot savant geared toward AI design.

      We do that with every new generation of babies, and it hasn't produced a super intelligence yet. What makes you think doing it ON A COMPUTER would make any difference?

      --
      Singularity: a belief in the "God" idea with the "demiurge" relation inverted.
    2. Re:Missing the key point by delt0r · · Score: 2

      Err, no. Many things don't scale like this at all. In fact many many computer algorithms in particular don't scale this way.

      --
      If information wants to be free, why does my internet connection cost so much?
    3. Re:Missing the key point by TuringTest · · Score: 2

      Uhh, precedent. Double the resources, double the ability. This is well known.

      That is magical thinking. It has no place in proper engineering practice.

      There are zillions of reasons that interfere with ability to work faster in larger problems - yet they can be summarized with the words "non-linear growth". Try solving the Travelling salespeople problem twice as big with merely twice as fast hardware, it will slow to a grinch.

      It's not like AI is going to run on some unknown substrate.

      We know the substrate of brain power, gray cells. This doesn't mean that we understand the way they work together to create intelligence. If we ever create an AI, it will be so complex that we'll likely be in the same situation with respect to how it works.

      --
      Singularity: a belief in the "God" idea with the "demiurge" relation inverted.
  13. Re:AI or Al by Whiteox · · Score: 2

    Serif fonts forever! IlIlIlIlIl (that's iLiLiLiL btw)

    --
    Don't be apathetic. Procrastinate!
  14. Capitalism at work by manu0601 · · Score: 4, Informative

    Expert's opinion do not weight much. Even if they were all against it, as soon as there is profit to be made, it would happen anyway.

  15. Re:The greatest 'existential catastrophe'... by khallow · · Score: 2

    The greatest 'existential catastrophe' that might be unleashed on humanity might be already have been unleashed by humanity.

    Of course, it has been unleashed. You can't cease to exist, if you didn't exist in the first place.

  16. Re:I think by tmosley · · Score: 2

    Agent Smith plz.

  17. Re: Already there by mbeckman · · Score: 2

    It's not that we refuse to accept that consciousness could be broken down to components. It's that nobody has done it, nor does anyone have any idea how to do it. So far AI is all hand-waving and no substance.

  18. Re:Risks by khallow · · Score: 2

    Ok, how is it like that? Remember the original concern was about "not doing anything" about the environment. I pointed out several ways that we were doing a lot about the environment contrary to the assumptions of the original post.

    I think it's more like having a thousand neighbors living in a small building next to you and complaining that they aren't "doing anything" about the noise they make. Those people could go to incredible lengths to minimize noise and still be loud enough to bug you just because, well, there's a thousand people living right next door.

  19. We forget the role of emotions and instinct. by master_p · · Score: 2

    In the discussion on Artificial Intelligence, we totally forget that the human behavior is driven by emotions and instinct and not by intelligence.

    People are bad not because they are highly clever but because they enjoy being bad.

    Without emotions/instincts, a machine cannot be bad or good. It might be exceptionally clever though, combining facts, extrapolating and discovering new facts and solving problems much better than humans.

  20. Re:I think by pmontra · · Score: 2

    How are chimps, gorillas and Co faring against those ape superintelligences called humans? Still alive, but some in zoos, others in labs, all of them progressively stripped out of vital space due to the exigences of their more intelligent cousins. I won't like to go the way of chimps so it's ok to develop some special purpose AI (vision, driving, etc), but I'd be very wary of connecting all the pieces together. It won't behave as a servant no more than we are servant to cats, no matter if cats actually believe all that infrastructures we built are for letting us be better caregivers to them