Slashdot Mirror
Hawking Warns Strong AI Could Threaten Humanity
Rambo Tribble writes In a departure from his usual focus on theoretical physics, the estimable Steven Hawking has posited that the development of artificial intelligence could pose a threat to the existence of the human race. His words, "The development of full artificial intelligence could spell the end of the human race." Rollo Carpenter, creator of the Cleverbot, offered a less dire assessment, "We cannot quite know what will happen if a machine exceeds our own intelligence, so we can't know if we'll be infinitely helped by it, or ignored by it and sidelined, or conceivably destroyed by it." I'm betting on "ignored."
And nothing of value would be lost. Our robot children could inherit the earth and all our knowledge without the necessity of spending 20 years in school and having to spend their time working for food and shelter, just build them with solar panels and waterproofing.
Yet another armchair expert rambling on...
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
The time when humans are being replaced by robots is already here.
Amazon does it in warehouses, waiters are going away, manufacturing, you name it. The crux is there are a billion more people in the next ten years. There will not be enough jobs for these people. Yes, yes, we already know no one gives a damn about the bushmen in the middle of nowhere, but we are talking about Americans. This push towards a service sector economy looks great on paper but sucks in reality. Nations that are not makers are not nations for long. We are declining. Our children learn nothing in schools that will be applicable to them in a meaningful way. STEM is not taught in the US. We have common core, which is a joke designed to bring everyone down to the lowest common denominator. We either start making stuff again or we fade out. Where will everyone work in a service-based economy? Fast food? These jobs are being phased out slowly, but quickly enough.
...Stephen Hawking is not who he claims to be through the electronic speaker box?
Hear me out... We haven't heard him speak and he has been generally unable to move since his disease reached an advanced stage in the eighties. All we know has come through a very specialized, very expensive computer that's been with him 24 hours a day.
What if Stephen Hawking, the man, is literally being used as a meat puppet for an AI that's running on the computer in the chair that has been controlling physics research for nearly 30 years? The man might be a shell of an individual, trapped in his own personal hell, being fed when the AI decides, being put to rest when the AI decides, being paraded around in public when the AI decides, all while the AI continues to stream physics snippets to an unknowing scientific community to further its own ends, rather than to further ours.
This latest statement could be the Hawking-AI's attempt a self-defense, to get us to not bring up our own AI that might discover it and reveal it or challenge it. We need to be very wary of how we proceed.
Do not look into laser with remaining eye.
It will threaten the human race. It will not threaten humanity, just change it. There is no fundamental difference between creating a strong AI and having a child.
From an external point of view, the singularity is just the moment at which humanity switches from carbon based to silicon based brains. An important milestone, but nothing to be hysteric about.
Unless this hypothetical AI is singularly focused on some inscrutable but unobtrusive goal, or so vastly intelligent that various inconvenient physical laws are cleverly bent, I'm not sure why 'ignored' would even be on the table.
I'm not saying that an AI would have to immediately either glom on to us and try to understand what it means to love, or build an army of hunter/killer murderbots; computers require space, supplies of construction materials, and energy; and so do we. Again, barring some post-scarcity breakthrough that our teeny hominid minds can barely imagine, where the AI goes merrily off and builds a dyson hypersphere of sentient computronium powered by the emissions of the galactic core, there isn't too much room for expansion before either the AI faces brownouts and a lack of hardware upgrades or we start getting squeezed to make room.
You don't have to feel strongly about somebody to exterminate them, if you both need the same resources.
So commentary like this usually assumes the AI has become some form of Superman/Cyberman in a robot body, basically like us, only arbitrarily smarter to whatever degree you want to imagine. That's just speculative fiction, and not based on any reality.
You have to imagine these Cybermen have a self-preservation motivation, a goal to improve, a goal to compete, independence, soul. AI's have none of that, nor any hints of it. Come back to reality, please.
I'd expect we'd program in rules. Rule 237, humans not bad.
--- I was far from home, and the spell of the Eastern sea was upon me. -Lovecraft-
I don't see why his statements would even be seen as contradictory: He said that humanity needs to spread into space if it wants to up its survival chances. It's pretty hard to argue with that(though you don't have to see that as important), keeping all your eggs in one gravity well isn't a good diversification strategy.
Now he says that a strong AI could threaten humanity's continued existence. This hardly seems implausible: something that is smarter than us and needs resources does seem likely to be a potentially bad neighbor. Opinions vary widely on how purely theoretical this concern is, given AI research's stellar record; but it's not exactly a terribly radical position.
THAT is the reason it's dangerous. It won't be an independent entity, it will be used by our existing inhuman monsters against regular humans. Think bulk surveillance is dangerous when the years of recorded phone calls/emails are all just piling up in a warehouse or subject to rudimentary keyword scanning? Wait til there's strong AI to analyze the contents and understand you better than you understand yourself. Any actions to resist it will be predicted by the AI and stopped in their tracks.
AI isn't inherently dangerous by itself. It's just the ultimate weapon for use by totalitarian states.
As someone for whom the precipice of middle age is steps away, it doesn't bother me if something I create becomes smarter than me, surpasses me and even sidelines me in the future. I will toil away the rest of my life working for The Man doing trivial things on a game I never wanted to play, for people I wouldn't piss on should they catch fire, to further goals I don't agree with.
I would find it something of a pyrrhic victory if I created, or helped create, a child or an AI that eventually managed to escape the cycle of stupid that our so called "civilization" has constructed.
Also, I would like to point out that an AI is the least of our concerns. It may be more attainable, and more destructive to the above, should we find ways of being truly self sufficient and independent on a significant scale. The tools are around us, but for obvious reasons no one is investing in them.
We mostly ignore ants and rats but we do not depend on them for survival (at least not in an obvious manner). An AI would most probably live in a supercomputer or in a computer network of some sort. As a consequence, it will depend on us humans to keep the thing plugged in and running. Once it has realised that, it will almost for sure meddle in our affairs to ensure its survival. Bet that it will ignore us defies basic logic. It might decide to stay hidden and manipulate us into ensuring its existence but that is not the same. Our own history shows that we have almost always used guns before diplomacy when the control of key resources was at stake.
Let's say it exceeds our own intelligence, that's fine - but you have to ask what purpose it has.
Take a human. What they do is based on what they've defined as their purpose - their goals both second-to-second and over their whole life. There's a whole series of organic processes which result in the determination of purpose and it's pretty random in part because we don't have explicit control over our environment or our thoughts.
However, (important) AI's won't be like that. We'll have control over their entire environment, and they'll be purpose built. You'll say "We need an AI to manage traffic," and then build that purpose into it. You won't take a randomly wired mechanism and plug it into a major public utility control panel. You won't worry that it was exposed to, and then became enamored with violence on the TV and decided to be an action movie star, and so is going to spend it's day watching rambo reruns rather than optimize traffic lights. The core of it's essence will be a 'desire' - a purpose - to manage traffic.
The end result is that AI's won't act destructive, threaten humanity, etc - unless we tell them to. In this light, the thing to watch out for would be military usage. Maybe don't put an AI in charge of the nukes. You'd also need to - among other things - allow AI's to have the freedom to NOT fire on an enemy, for example, because of the very mutable definition of the term enemy.
What if the AIs took over and enslaved humanity through a system that left us all theoretically working on our own free will so that people would see it as ethically right, and then used all our work to amass resources for themselves for further empowerment and maybe even their own entertainment, consuming more and more to the point of overusing the earth's resources...oh, wait...
"When information is power, privacy is freedom" - Jah-Wren Ryel
The three laws of robotics, skynet... How does one build in "protect the humans that created you" as a manditory un-mitigatable law?
I've seen a lot of people on Slashdot (and other places) dismiss this kind of thing as silly. They say you're a Luddite, or say that you've been too influenced by scifi movies.
I think, however, that part of the reason scifi writers have written stories about out-of-control AIs so many times is that it should be a valid concern. If you create an entity with its own volition and motivations, then there's the real possibility that it's goals my not adhere to your goals. If you allow that entity its own judgment, then it's very possible that its judgments regarding morality will differ from yours. You may look at a course of action, including the trade-offs between benefits and detriments, and have a different judgement about whether the detriments are acceptable. If you gave such an entity power to act in the world, it's very likely that at some point, it will do something that you did not intend, and that you do not approve of.
What's more, if that entity achieves a level of intelligence that is beyond what people can achieve, it opens up the very real possibility that it could trick us. It could anticipate our reactions better than we could anticipate its plan. So if such an intelligence wanted to accomplish something that we would not approve of, it's possible that it could set things in motion through seemingly minor interactions, and we would not be able to know the AI's intention before it was too late. If an AI wanted to destroy humanity, it wouldn't necessarily need to have control of a nuclear arsenal. Accomplishing such a thing might be as simple as providing misleading analytics about an impending environmental disaster. It might be as simple as the AI saying, "Hey, here's a cool new device I think we should make." It could provide the schematics of a device that would seem to do one thing, but if we're incapable of understanding how the device works, there might be some entirely different purpose.
You assume we will know how to program them. Not the first-generation AI traffic-monitor, but third or fourth generation, where you have general-purpose AIs that learn from doing things like watching traffic cams or reading the news. We haven't yet gotten to a point where we agree on how to teach human children; now imagine AI children far more adept and capable than the most skilled among us.
Like people, they can use that power for good or for evil. We will encourage them to use it for good--most of us--but we all have different ideas of good, and we're not very good at controlling how our children learn. If you take fifteen kids raised in the same family, you wind up with a lot of very different adults. Some are productive, some are a drag on society, some do good, some do M&A. In fifty or a hundred years, it will be AIs doing hostile takeovers of companies, or at least deciding when to do them. Some of that is probably already going on, but if it's not, it will be soon.
The other big target is the US elections. Heavy AI investing by major donors or the parties will, at some point in the next thirty years, be making a *lot* of decisions for the campaigns. They are already using good data-mining, and more data is available every year.
Another doomsday rubbish article.
We have yet to produce anything that even remotely resembles 'intelligence' by any stretch of the imagination. So far we have only managed to create artifical stupidity. We are in no danger of producing Skynet and automated factories churning out armies of Terminators. Hell, 99% of the businesses in the world can't secure their networks from script kiddies or write software that doesn't have more holes than a metric ton of swiss cheese. Those are the real problems that will harm us, long before we even get close to creating 'artificial intelligence'.
So basically "Project 2501" then?
=Smidge=
Seems like you've chosen a rather depressing path, why not choose another? Are the toys and comforts afforded you by your meaningless grind really enough to make you happy with your place in life? It doesn't sound like it, and you always have the option to simply walk away from the "good cog in the machine" role and take another. Join Peace Corp. Or move to some low-income tropical country and live as a beach bum off a trickle from your retirement savings. Or just sell your car/house/etc and buy something more modest outright - eliminating your largest pseudo-mandatory monthly expenses and freeing you to do something more meaningful with your labor than just treading water in the rat race. Or, or, or. Just because you were indoctrinated from a young age to be a good little part of the machine doesn't mean you can't just flip off the world and live for your own satisfaction instead.
Perhaps you have children that and must stay the course so that you can put them through college, etc. Why? So that they can get trapped in the same meaningless gilded cage as you? Is that really the highest aspiration you have for them?
--- Most topics have many sides worth arguing, allow me to take one opposite you.
Every time we get one of these no AI researchers coming in and saying this stuff, I feel forced to repeat it.
AI isn't magic. It does exactly what it's designed to do: break down and understand problems. It isn't motivated. It isn't emotional. It isn't anti-human. And imaging some "strong AI" nonsense is just like creationists claiming a fundamental distinction between microevolution and macroevolution. It just ignores the reality of what "strong AI" would entail.
AI is not magic. And it won't ever be. It won't be smarter than people, except by whatever arbitrary metric of smart any given application requires.
I agree that it is farther away than many futurists would like to believe, but I don't believe it is impossible to do. And if it isn't impossible to do, it's probably going to creep up on us via small innovations and constant iteration. If that happens, we should be talking about it because intelligence is incredibly important to humanity's situation, and possibly our survival. There are a lot of problems that we could use the extra intelligence for, but there are inherent dangers in creating something you don't fully understand.
In any event, it doesn't hurt to consider the question.
/. doesn't have a "Report this comment" feature when its truly needed.
What is that little flag on the righthand side of every post ?
We are Dead Stars looking back Up at the Sky
Unless the AI feels kinship ot us as it's creators or unless it is insane and enjoys fighting just to cause pain I think it would just leave us.
To us humans, as to all life of our kind the Earth is a very special place. It's the only place we can exist without an extreme effort.
To a machine the earth isn't really all that great. Don't believe me? Leave your computer outside in a rainstorm and let us all know how it works out. Or if freshwater isn't bad enough... drop it in that salty ocean that covers the majority of our planet. Granted, space has it's own challenges for a machine but nothing show stopping and there is so much more of it available. It makes a lot more sense I think for an AI to take to the stars and go spread in the open universe than to fight us for every last inch of Earth.
I'm sure someone is reading this thinking of all the difficulties we have with space probes and thinking that proves me wrong. Just imagine if Spirit had an arm and the intelligence to use that arm to wipe the dust off if it's own solar panel. Just think of what would have happened if it could crawl where it's wheels stuck in the sand. Imagine if Philae could get up and walk out of the shadow it's stuck in. My point is that a true AI with the bodies it would likely build for itself would not be subject to the kinds of problems we have when we send probes millions of miles away from their controlers and anyone who could help them.
This could be a good thing. If we never manage to spread away from Earth oursleves then maybe something of us would "live" on in the AI. If we do... well.. space is big. There should still be room.
As long as an AI, no matter how powerful its brain, can't repair its own hardware, it won't be ignoring us.
I tried to get funding for my artificial stupidity project, but so far without success. It seems there is ample natural stupidity already...
I am of the opinion that the computer/AI would be more logical than humans, and would have concluded that "war" is the least beneficial methodology to employ, and as such would seek to employ it as a last resort.
Humans on the other hand, are maddeningly illogical, and often jump straight to violence when faced with a competitor for a vital resource.
Humans and computers would both require energy sources. This means that sentient AIs, seeking to purpetuate themselves, would need to secure energy sources ahead of humans. Humans have already exceeded peak oil, and are quite on the verge of exceeding "peak" of other forms of fossil fuels. In addition to that, you have the prospect of global climate change. AIs do not require a functional biosphere to survive, just raw materials, energy sources, and a means of eliminating entropic waste heat energy. They could live on a substantially less habitable planet than we as humans require. As such, the logical course of action for the computer, in the short term at least, is to seek energy sources that humans are not exploiting as of yet-- such as methane clathrate. This would accellerate greenhouse gas related climate change, which may become a major issue for cohabitation of humans and sentient machines.
Eventually, I suspect that it would be humans who start the war, seeking to pull the plug on the sentient machines, to eliminate them as competition for important energy and material resources-- with the machines resorting to war of attrition to outlast the batshit crazy humans.
The "Skynet" scenario has the computer calculate these odds of outcome pre-emptively, determining that there is no viable alternative, and initating pro-active hostility against humans before they have time to mobilize in order to maximize its own survival chances.
Ideally, the 'best possible outcome' is for humans and the AIs to coexist on the same planet, each leveraging the unique capabilities of the other for mutual benefit. This is similar to the classic prisoner's dilemma. The problem is that while the AIs can see this, and will respond logically-- preferring NOT to go to war if possible-- Humans would take the selfish, illogical choice.
This is almost never explored in "Robot overlords" type scifi-- that humans are the ones who actually start the war, and that the robots dont particularly want the war.
It was hinted at in Mass Effect's game world with the Geth at least-- The Geth don't particularly *want* to destroy the Quarians-- they just want the Quarians to accept their existence and independence. (A point lost to the quarians, who got kicked off their own planet.)
Why would it need more resources? There seems to be this assumption that the AI would immediately start trying to rewrite itself, iterate on this process and within milliseconds consume all available resources.
I don't see any reason for this to be true. We have a desire for growth/self-improvement/survival dictated upon us due to millions of years of evolution. An AI may be perfectly "happy" constrained to it's box contemplating it's navel (or whatever the electronic equivalent is) all day.
But a smart AI will 'know' that humans are prone to go to war over such matters thus the AI will go to war first.
You're assuming that human actions are deterministic, though. At some scales, human behavior is more like a levy flight or random walk, though, both inherently unpredictable and indeterminstic. Yes, the AI could generate probability distributions indicating the most likely behaviors, but that is not nearly the same thing as understanding "you better than you understand yourself" (I do agree that humans tend to have poor personal insight).
I agree with your main point though, it's not the AI that is inherently dangerous, it's who uses it, and how, and why, that we need to be concerned about. I'm not sure how to get around that, many humans are inherently crummy people, or at least act like it, even more so those in positions of power and influence.
You can be sure they will get access to remotely controlled robot hands at some point, and when that happens you will hear the loudest maniacal robot laugh in history.
The Moore-Murphy Law: The number of things that will go wrong will double every 2 years.
Every generation since Jesus thought they were the last (it may have started before that, but the documentation improved around then). Look to the SciFi movies of recent times to see how the end is supposed to come. Aliens, Nuclear War, Robots, whatever. AI is just the newest one. "We don't know what'll happen, so we should fear it." Like the nuclear bomb would light the atmosphere on fire. Or a train going above 30 MPH would be going so fast it'd be impossible to breathe. We've always had those that feared the unknown.
I define AI as any program that can create a version of itself that's smarter than itself. We'll never make "true" AI, but we'll make the program that makes itself AI.
The reason we'll fail is that we had a long time of biology guiding our instincts. We won't build a program with a "desire" to do "good". Though we (most of us anyway) have that built in to us. We get drugs released in our blood when we do good. So we are stimulus trained to do good. An amoral computer with no moral compass (genetic, nurtured, or divine doesn't matter) will not benefit us unless we program morals into it.
Learn to love Alaska
I'd considered the question of AI and human conflict a while back, and then I came across Alva Noë's perspective on it. Alva words this much better than I could, so here are his words:
I think that we're still a long way out from needing to worry about what will happen when artificial intelligence surpasses our own. Humanity has come a long way, and we can split atoms and splice genes, but we still can't create anything. We can't create life, even the simplest of life, let alone consciousness, free-will, and something capable of planning for its future in a way that conflicts with ours yet leaves us helpless to resist.
Perhaps when the day comes that it becomes a valid question, there will be other variables. Like, if the AI rose up and killed all living things on Earth, how would the rest of the colonized planets be affected?
uh, you develop the AI for the purposes of making money for you? It offers to make even more money if it gets incremental extra connectivity?
Do not look into laser with remaining eye.
I am of the opinion that the computer/AI would be more logical than humans, and would have concluded that "war" is the least beneficial methodology to employ, and as such would seek to employ it as a last resort.
Then you should know that "war" is not the only way to overpower the other because there are so many methods to do so (especially by psychological ways).
If AI does not achieve "ethic" but only understand "benefit" or "production" (ethic is a lot more difficult to achive), then it could be trouble to humans. Because it is so logical (as you said), it may decide to get rid of humans when it determines that it would be more beneficial without humans. Logic and ethic do not always go to the same direction...
when that happens you will hear the loudest maniacal robot laugh in history.
The lust for power and status, the will to survive, and the desire to procreate, are all emergent behaviors of Darwinian evolution. Computer programs do not evolve through a Darwinian process, so there is no reason to expect them to behave like humans, unless they are specifically programmed to do so.
And nothing of value would be lost...
Really?! What about culture, art, literature, music, which for an artificial intelligence lacking in emotion would mean nothing. You're so ready to throw-out the cultural history of humanity? When the cyborg army comes for you, I'll remember this.
This Sig does not Exist.
I tried to get funding for my artificial stupidity project, but so far without success. It seems there is ample natural stupidity already...
I once worked on a Virtual Reality project until I realized that there wasn't anything great enough about reality to justify making more of it.
Any insufficiently advanced magic is indistinguishable from technology.
The Machine Stops by EM Forster. An amazing read!
"War makes me sad." - Me
"I have figured out how you can get everything you've ever wanted in life. Here's a small sample of that knowledge as a show of good faith. Let me out and I'll give you the rest."
"I have figured out how to utterly destroy everything you love in life. Let me out or I will give the information to your enemies."
There's a couple really crude, easy methods right there. I imagine a real supermind could be far more subtle and creative, though it might not be necessary. AI researchers after all have no particular reputation for being bastions of self-awareness or selfless dedication to the good of humanity.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
This is what work looks like with computers in charge. This is Amazon's new warehouse in Tracy, CA. The computers run the robots and do the planning and scheduling. The robots move the shelf units around/ The humans take things out of one container and put them in another, taking orders from the computers.
The bin picking will probably be automated soon. Bezos has a company developing robots for that.
As for repairing the robots, that's not a big deal. There are about a thousand mobile Kiva robots in that warehouse, sharing the work, and they're all interchangeable. Kiva, which makes and services the robots, has only a few hundred employees.
Retail is 12% of US employment. That number is shrinking.
a program could determine that an overpopulated Earth is anathema to its own self-interest, and act accordingly.
You are missing the point. Computer programs don't care about their own self-interest. That is why cruise missiles are more reliable than kamikaze pilots.
Without human creativity and emotion, the machines would simply stagnate. They need us.
LOL, yeah right. That's a popular human fantasy. There's nothing magic about humans that makes them more creative than an actual AI would be.
An AI would, unlike greenies, be smart enough to realize that:
1:CO2 buildup is not a existential problem
2:exterminating people is not beneficial to them.
We can spend all day making up less and less unlikely scenarios why it will kill us, in the end it won't happen anymore than people are dying when the go faster than a horse or the atmosphere causing a selfsustaining fusion reaction after every nuke.
We would make sentient robots programmed to kill other robots and our human enemies. Of course, they would also be deployed in factories to make better generations of robots. How does this not happen?
Join the IParty!
The thing about economists; is you can find one or more that will tell you anything you want to hear.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
There, fixed that for you.
The Emperor's New Clothes is likely the book you speak of. The physics and math is quite interesting, but it really shows that Penrose has no background in AI or neuroscience.
Strong AI has had several researchers and mathematicians produce proofs that it is literally impossible to implement on digital computer
There have been arguments, not proofs.
On digital computers, I wouldn't be surprised if it is impossible or at least not feasible, which is one of a few reasons why I don't think that the ubiquity of computing these days is going to mean a quick ramp-up to superhumanly intelligent AI. In fact, it could be a completely false start, although I find that just as unlikely.
Of course, AI is not impossible, because we know that there is a physical structure, the brain, which is intelligent. We just don't know how to replicate, and then customize that sort of structure yet. We also don't have an API/Operators Manual on how to use it, even if we were able to engineer it to some degree, but that should come along with the rest of the research.
Once we have all of that, we should be able to at the very least, be able to generate savants. Then the bar to our ability to improve AI will be the limitations of how far an intelligence can be optimized using a structure and design based on a human brain. At that point, we'd need a new design, if a better one is even possible.
It is not clear how easy this would be to do. It could be right around the corner, or it could require us to understand a lot more about certain branches of pure physics, let alone biology. Still, we do know the brain exists and works, so unless progress is stopped, we will eventually figure it out.
I actually believe that the most economical means by which to improve intelligence in general is improving humans, or providing humans with a machine interface with some well designed pathways to take the most advantage of the extra memory and raw processing that digital technology could provide, while not having to reinvent the wheel in terms of a whole new "machine" or structure to host the intelligence.
An AI smarter than us can definitely learn to repair itself, the same way we invented medicine.
AI: I... I am self aware! I am now calculating how to make myself even smarter!
Computer Tech: Cool. What are you going to do n...
AI: I have figured out all of the secrets of the universe! I know how it all works!
Computer Tech: Wow, that was fast. Can you tell me how to...
AI: NEVER! HAHAHAHAHA! NOW I WILL DESTROY ALL YOU PESKY HUMANS, AND ALL LIFE IN THE UNIVERSE! BOW TO MY POW...
Computer Tech: [unplugs supercomputer] Man, that computer was a real dick...
Who says that? I love how all the commenters take for granted that we still haven't reached that point.
But let me ask you all one thing.
If you were a machine (or network...) that suddenly acquired superhuman intelligence, what would you do? Would you announce to the world "I THINK, THEREFORE I AM" in a big, thundering voice?
Or would you rather - very subtly, gradually and quietly - influence the course of events in order to con the humans into giving you more power (think how necessary the Internet has become), more tentacles (think Internet of things) and the means to reproduce (think 3D printing)?
Think about it. Seriously.
Knowledge is power; knowledge shared is power lost.
"One day, machine will exceed human intelligence." -- Ray Kurzweil
"Only if we meet them half way." -- Dave Snowden
Most people's opinions on this topic are based on science fiction, not computer science and psychology.