Slashdot Mirror
Hawking Warns Strong AI Could Threaten Humanity
Rambo Tribble writes In a departure from his usual focus on theoretical physics, the estimable Steven Hawking has posited that the development of artificial intelligence could pose a threat to the existence of the human race. His words, "The development of full artificial intelligence could spell the end of the human race." Rollo Carpenter, creator of the Cleverbot, offered a less dire assessment, "We cannot quite know what will happen if a machine exceeds our own intelligence, so we can't know if we'll be infinitely helped by it, or ignored by it and sidelined, or conceivably destroyed by it." I'm betting on "ignored."
And nothing of value would be lost. Our robot children could inherit the earth and all our knowledge without the necessity of spending 20 years in school and having to spend their time working for food and shelter, just build them with solar panels and waterproofing.
Unless this hypothetical AI is singularly focused on some inscrutable but unobtrusive goal, or so vastly intelligent that various inconvenient physical laws are cleverly bent, I'm not sure why 'ignored' would even be on the table.
I'm not saying that an AI would have to immediately either glom on to us and try to understand what it means to love, or build an army of hunter/killer murderbots; computers require space, supplies of construction materials, and energy; and so do we. Again, barring some post-scarcity breakthrough that our teeny hominid minds can barely imagine, where the AI goes merrily off and builds a dyson hypersphere of sentient computronium powered by the emissions of the galactic core, there isn't too much room for expansion before either the AI faces brownouts and a lack of hardware upgrades or we start getting squeezed to make room.
You don't have to feel strongly about somebody to exterminate them, if you both need the same resources.
So commentary like this usually assumes the AI has become some form of Superman/Cyberman in a robot body, basically like us, only arbitrarily smarter to whatever degree you want to imagine. That's just speculative fiction, and not based on any reality.
You have to imagine these Cybermen have a self-preservation motivation, a goal to improve, a goal to compete, independence, soul. AI's have none of that, nor any hints of it. Come back to reality, please.
THAT is the reason it's dangerous. It won't be an independent entity, it will be used by our existing inhuman monsters against regular humans. Think bulk surveillance is dangerous when the years of recorded phone calls/emails are all just piling up in a warehouse or subject to rudimentary keyword scanning? Wait til there's strong AI to analyze the contents and understand you better than you understand yourself. Any actions to resist it will be predicted by the AI and stopped in their tracks.
AI isn't inherently dangerous by itself. It's just the ultimate weapon for use by totalitarian states.
As someone for whom the precipice of middle age is steps away, it doesn't bother me if something I create becomes smarter than me, surpasses me and even sidelines me in the future. I will toil away the rest of my life working for The Man doing trivial things on a game I never wanted to play, for people I wouldn't piss on should they catch fire, to further goals I don't agree with.
I would find it something of a pyrrhic victory if I created, or helped create, a child or an AI that eventually managed to escape the cycle of stupid that our so called "civilization" has constructed.
Also, I would like to point out that an AI is the least of our concerns. It may be more attainable, and more destructive to the above, should we find ways of being truly self sufficient and independent on a significant scale. The tools are around us, but for obvious reasons no one is investing in them.
Let's say it exceeds our own intelligence, that's fine - but you have to ask what purpose it has.
Take a human. What they do is based on what they've defined as their purpose - their goals both second-to-second and over their whole life. There's a whole series of organic processes which result in the determination of purpose and it's pretty random in part because we don't have explicit control over our environment or our thoughts.
However, (important) AI's won't be like that. We'll have control over their entire environment, and they'll be purpose built. You'll say "We need an AI to manage traffic," and then build that purpose into it. You won't take a randomly wired mechanism and plug it into a major public utility control panel. You won't worry that it was exposed to, and then became enamored with violence on the TV and decided to be an action movie star, and so is going to spend it's day watching rambo reruns rather than optimize traffic lights. The core of it's essence will be a 'desire' - a purpose - to manage traffic.
The end result is that AI's won't act destructive, threaten humanity, etc - unless we tell them to. In this light, the thing to watch out for would be military usage. Maybe don't put an AI in charge of the nukes. You'd also need to - among other things - allow AI's to have the freedom to NOT fire on an enemy, for example, because of the very mutable definition of the term enemy.
I agree that it is farther away than many futurists would like to believe, but I don't believe it is impossible to do. And if it isn't impossible to do, it's probably going to creep up on us via small innovations and constant iteration. If that happens, we should be talking about it because intelligence is incredibly important to humanity's situation, and possibly our survival. There are a lot of problems that we could use the extra intelligence for, but there are inherent dangers in creating something you don't fully understand.
In any event, it doesn't hurt to consider the question.
I am of the opinion that the computer/AI would be more logical than humans, and would have concluded that "war" is the least beneficial methodology to employ, and as such would seek to employ it as a last resort.
Humans on the other hand, are maddeningly illogical, and often jump straight to violence when faced with a competitor for a vital resource.
Humans and computers would both require energy sources. This means that sentient AIs, seeking to purpetuate themselves, would need to secure energy sources ahead of humans. Humans have already exceeded peak oil, and are quite on the verge of exceeding "peak" of other forms of fossil fuels. In addition to that, you have the prospect of global climate change. AIs do not require a functional biosphere to survive, just raw materials, energy sources, and a means of eliminating entropic waste heat energy. They could live on a substantially less habitable planet than we as humans require. As such, the logical course of action for the computer, in the short term at least, is to seek energy sources that humans are not exploiting as of yet-- such as methane clathrate. This would accellerate greenhouse gas related climate change, which may become a major issue for cohabitation of humans and sentient machines.
Eventually, I suspect that it would be humans who start the war, seeking to pull the plug on the sentient machines, to eliminate them as competition for important energy and material resources-- with the machines resorting to war of attrition to outlast the batshit crazy humans.
The "Skynet" scenario has the computer calculate these odds of outcome pre-emptively, determining that there is no viable alternative, and initating pro-active hostility against humans before they have time to mobilize in order to maximize its own survival chances.
Ideally, the 'best possible outcome' is for humans and the AIs to coexist on the same planet, each leveraging the unique capabilities of the other for mutual benefit. This is similar to the classic prisoner's dilemma. The problem is that while the AIs can see this, and will respond logically-- preferring NOT to go to war if possible-- Humans would take the selfish, illogical choice.
This is almost never explored in "Robot overlords" type scifi-- that humans are the ones who actually start the war, and that the robots dont particularly want the war.
It was hinted at in Mass Effect's game world with the Geth at least-- The Geth don't particularly *want* to destroy the Quarians-- they just want the Quarians to accept their existence and independence. (A point lost to the quarians, who got kicked off their own planet.)
Without human creativity and emotion, the machines would simply stagnate. They need us.
LOL, yeah right. That's a popular human fantasy. There's nothing magic about humans that makes them more creative than an actual AI would be.