Slashdot Mirror
The Men Trying To Save Us From the Machines
nk497 writes "Are you more likely to die from cancer or be wiped out by a malevolent computer? That thought has been bothering one of the co-founders of Skype so much he teamed up with Oxbridge researchers in the hopes of predicting what machine super-intelligence will mean for the world, in order to mitigate the existential threat of new technology – that is, the chance it will destroy humanity. That idea is being studied at the University of Oxford's Future of Humanity Institute and the newly launched Centre for the Study of Existential Risk at the University of Cambridge, where philosophers look more widely at the possible repercussions of nanotechnology, robotics, artificial intelligence and other innovations — and to try to avoid being outsmarted by technology."
... it is still bound by energy requirements and the laws of nature. All this fear mongering is bs. If you look at the evolution of life on earth, even tiny 'low intelligence' beings can take out huge intellectual behemoths like human beings.
Not only that, you have things like EMP and nukes, not even the best AI is capable of thwarting getting bombed or nuked. Intelligence is a rather demanding, costly and fragile thing in nature. All knowledge perception has costs in terms of storage, time to access, problems of interpreting the data one is seeing and whatnot.
Consider the recent revelations by the NSA spying on everyone, there are plenty of easy low tech measures to defeat high tech spying. The same way there will be plenty of easy low tech ways to cripple a higher intelligence which is bound by the laws of nature in terms of resource and energy requirements. Anything that has physical structure in the universe requires energy and resources to maintain itself.
Humanity's biggest enemy is humanity itself. And maybe space rocks.
It would be spiffy except that your new robotic overlords won't welcome you.
Ezekiel 23:20
when a group of people have control of business and finance and commerce and information that their corruption and greed and power causes them to abuse the system in which they were trusted with to feed their fascist kleptocratic empire and when they are caught they lose the trust of the rest of the world that trade in this global market and then nobody wants to have any more business dealings with this corrupted greedy power hungry group of people anymore so they end up collapsing from the weight of their own greed & stupidity (much like what the USA/UK/Israel will do within the next few years
are you listening NSA?, i hope so because this message is for you too...
Politics is Treachery, Religion is Brainwashing
to try to avoid being outsmarted by technology.
The humanity can, of course, ban all machines that are smarter than humans. But that only artificially impedes the progress. Given that there ought to be an approximately infinite number of civilizations in this Universe, all paths of development will be taken, including those that lead to mostly machine civilizations. (We are already machines, by the way, it's just we are biological machines, fragile, unreliable, and slow.)
Civilizations that became machines will have no problem with FTL because they can easily afford a million years in flight by just slowing the clock down. So they will come here, to Earth, armed with technologies that Earthlings were too afraid to even allow to develop. What will happen to Earth?
Well, of course the doom is not guaranteed; but I'm using this example to demonstrate that you cannot stop the flow of progress if you only have local control, even if that. (How many movies have we seen when mad geniuses break those barriers and, essentially, own the world?)
IMO, it would be far more practical to continue the development of everything. If humanity in the end appears to be unnecessary and worthless, it's just too bad for it. The laws of nature cannot be controlled by human wishes (unless magic is real.) Most likely some convergence is possible, with human minds in machine implementations of bodies. Plenty of older people will be happy to join, simply because the only other option for them is a comfortable grave.
Even if it's bound by the laws of physics as we understand them (Stross-universe-like "P=NP"-powered reality modification aside) there are plenty of dangers out there we're well aware of which computing technology could ape. Nanoassemblers might not be able to eat the planet, but what if they infested humans like a disease? We're already having horrible problems with malware clogging up people's machines, and they're coded by humans; what if an artificial intelligence was put in control of a botnet, updating and improving the exploiters faster than anyone could take them apart?
Drones. Sure, probably not much of a threat if you're living in the West. But in the same way that the history of cybernetics begins with walking sticks and hearing aids, the history of man vs machine is going to start with the murder by Americans of unconvicted, if highly tanned, individuals in Africa and Asia.
We'll manage to do it long before we are able to make an intelligent machine.
We can't even make a word processor that doesn't shit the bed every two hours. Super-intelligent machines my ass.
Instead of asking questions like that, why don't you build Skype and any other software you're working on to NOT have backdoors
That way, if ever the machines DO try to take over the world, they won't have a bunch of convenient control channels in all the important software to do so.
I don't know the meaning of the word 'don't' - J
Surely they would need newscasters.
rewriting history since 2109
The typical way to mitigate such threats is to not put it in control of all of our weapon and defense systems, and give it vague orders like 'purge the infidels.' Seriously, humanity can build silicon life any way it wants, billions and trillions of permutations and forms and functions....and what do we do with it? We put a gun on its head, lasers in its eyes, and tell it to go out there and kill the other humans we don't like. It's not the machines we need to be afraid of, it's ourselves; we're the ancient enemy that is always trying to annihilate itself.
I am John Hurt.
Aside from the apocalypse, that is one of the things I worry about. Shills are bad enough today, but imagine if they could be deployed programmatically; just about any form of online speech could be drowned out with ease. That is assuming that the government/corporations aren't already using AI to accomplice pervasive censorship.
Before this gets out of hand, we need to head it off by deploying peer to peer communications systems with a pervasive trust model. This doesn't necessarily preclude anonymity or AI participation, but they would have a significantly more difficult time of gaining trust in the first place.
Let me put a "scientific" answer to your "oh piss off" answer.
All of this talk of how computers will take over humanity ignores one fact. Namely that computers once are smart they will be dumb as crap!
Yes yes sounds contradictory, but in fact it is not. The real problem with humanity is that not our lack of intelligence. Frankly we are pretty bloody intelligent. Put context, we humans are pretty quick at figuring things out even if it is entirely orthogonal to most things. The issue is that we humans come up with too many answers.
In Science there is one answer. A rock falls on the ground on planet earth and we know that is called gravity. You can't deny it, you can't fight it, it is what it is. Now throw in a question, "should the people look after other people" and you get a bloody maze of answers. Humanity has what I call the stochastic conditioning. Namely when presented with the same identical conditions, you will receive different answers. Science does not work that way. We work the way we do because of our wiring. Namely as we became more intelligent we also became more opinionated. I am not talking about Fox opinions. I am talking about deduction and how we think we know what the future holds and thusly we should not do things today.
Our intelligence actually does get in our way. In the way way way back days as we were animals it was about water holes and finding that watering hole. If you found the watering hole you survived, if you did not find the watering hole you died. These days, we have to bloody analyze the watering hole. We have to concern ourselves with the ethics, morality, and so on of that watering hole. I am not dissing our humanity for we are where we are because of our intelligence. However, often enough our intelligence gets in our way of getting things done due to the conflicts.
Now imagine two robots with superior intelligence getting together. Do you really think they will come to the same conclusion? Sure Hollywood likes to think that, but the reality is that intelligence breeds opinions, and how things will happen in the future. And it is at that point robots become as stupid as we are. One robot will say white, the other black! We will have a Hitchhikers Guide to the Galaxy type situation. Or if you want to use serious sci-fi, the closest that I have ever seen in pop scifi is "The Matrix". You have good algo's battling bad algos and they all want and desire things.
So like you, my thinking is that these institutions are "producing fucking nothing of value".
"You can't make a race horse of a pig"
"No," said Samuel, "but you can make very fast pig"
With about ten nations armed with nuclear weapons, I wonder how machines will take over every one of them. You have to take over Russia, China, US, France etc. but some nation may trigger nuclear war as a desperate move, or the machines may deliberately accept nuclear war in a bid they survive it, while not necessarily having a goal to kill us all.
Instead maybe machines will try to take over politically in every country, one by one. It would be funny if tech superminds can rise to power through democracy in fair and respected elections. Either way I like to think that super machines holding most high level political power is probably a desireable outcome, we could end up living in some kind of new USSR but without corruption and with respect for the environment and life. Machines would take care of energy production and storage, and close down all oil wells and coal mines for us. They will even put us to work, hopefully on voluntary terms, if they determine some physical and intellectual activity is beneficial to us.
Machines should rule us and not the other way around, I guess that will be better than to be ruled by the suits, ties and kings like it is today.
The other question is, what's a supermind, what about superminds competing with each other, and especially : how do you compare two vastly different superminds, independantly originated? They will be as strongly or more strongly different between each other than between one of them and a human. It will be a mess. Each supermind, or at least the first one will have to run that same inquiry that "Oxbridge" is doing. We also have no fucking idea if a supermind can be governed by a "prime directive" of some sort : if Skynet emerges at the NSA will it stay true to them for ten minutes, ten years or eternally, or will it betray the organization that hosts it? potentially committing suicide in the way.
How can the supermind deal with backups, copies and archives of itself? Will it suffer dementia, schizophrenia or even addictions. No idea, I'll bail out myself by saying it's all unpredictable.
Nerve impulses travel along nerve fibers as pulses of membrane depolarization. Within our brains and bodies, this is adequate speed for thinking and control. However, relative to the speed of light, our nerve impulses are laughably slow.
The maximum speed of a nerve impulse is about 200 miles per hour.
The speed of light is over 3 million times that fast.
Now consider what will happen when we create a sentient, electronic being that has as many neurons as we do, but its nerve impulses travel at the speed of light.
In terms of intelligence, that creation will be to us as we are to worms.
Even Rats have empathy. Self aware machines will too. Lacking irrational emotions, hyper Intelligent machines will be more ethical and fair and nice than humans. You don't have to worry about sentient machines running amok. You have to worry about pre-sentient kill bots programmed by the same assholes that do shit like PRISM.
"... really good at achieving the outcomes it prefers," he says. "So good it could steamroll over human opposition. Everything then depends on what it is that it prefers, so, unless you can engineer its preferences in exactly the right way, youâ(TM)re in trouble."
we have a national philosophers strike on our hands.
Bert
These people must have watched the Terminator series and the "self-aware" AI system Skynet. IMO, the threat of nuclear war triggered by malfunctioning defense computers is way greater. There are several well-documented instances of nuclear near-misses caused by machine failure .
Are machines more dangerous when they become super-intelligent, or when they stay "stupid" and flawed?
Yeah, you pretty much nail it. I've thought about this a lot, and i'm also sure that together with intelligence comes what we so far call 'human error'. Intelligent machines will sure share that trait
CLI paste? paste.pr0.tips!
We're likely to see this in the private sector first. A likely application would be a machine learning system used by investment funds, to decide how to optimally vote stock proxies. What that means is a machine that decides when to fire CEOs. If some fund starts getting better returns that way, it will happen.
Don't assume than malign supercomputers will wipe us all if that can be adequately done by human's stupidity
That's ridiculous. How can you possibly know what a machine intelligence capable of destroying humanity is going to look like? We're nowhere near the algorithms that could produce that type of intelligence.
Maybe it's a dumb algorithm simply caught in a self-replication loop. Maybe you'll never see two robots arguing over "white" or "black", because there's only one single "intelligence" spread over the internet - that seems more likely with the rise of cloud computing.
There may be plenty of reasons to dismiss this type of institution, but "human intelligence doesn't work that way, so machine intelligence won't either" isn't one of them.
Last post!
I've read many comment in this thread. Instead of answering them one by one. I just post one aggregated comment.
First, the possibility of intelligent machines is glimpse. All our present technology is not able to achieve intelligence. This is mainly because we do not know what that is. Furthermore, to be dangerous they must be equipped with greed and (the illusion of) a free will. It is most unlikely that someone would build that on purpose or by accident. In short, I think it is impossible to built such machine.
Second, if an alien race succeeds in this task, against all odds, and gets eradicated by their machines. And these machines would thrive out through the universe. It is most unlikely that they attack us. A) If they are greedy and logical, they will see no gain by attacking us. They would spread and multiply and die when they cannot generate enough energy to continue. B) If they are illogical, then they will start fighting with each other to gain short term gains. They could be dangerous, however, they would never reach us. C) Either way, if such civilization exists, the universe is that big, that at sub-light speed, they would need millions or even billions of years to reach Earth. Therefore, I do not assume that they are a real threat.
A real threat are NSA and their friends around the world. They suck the most. Maybe we should shot them to the moon. We could use ESAs ATV, which does not have any reentry capability.
I've always though our main future threats are AI and manmade viruses. If AI wins, we'll be relegated to zoos. If manmade viruses win, we'll all die. I'm rooting for AI.
Studying (and trying to create) hard AI is my day job.
I just want to let people know that not everyone shares the opinions or urgency of the people in the story.
I for one am trying hard to condemn humanity to death and/or enslavement at the hands of intelligent machines, and I know a number of AI researchers trying to do the same.
So don't worry too much about these guys - they are definitely in the minority. Everyone will get their chance to (as individuals) welcome our new robotic overlords, however briefly.
It's a very narrow definition of intelligence that assumes it must manifest itself as human like thought. An ants nest is an intelligent and efficient entity in it's own right but it doesn't have any thought processes, ant's themselves are basically mindless automatons, They don't think about the complexities of building nests they just do it, some species such as soldier ants build the ant equivalent of NYC every few days, shifting up to four tons of soil at a time. The octopus is another fine example, obviously a highly intelligent creature (can solve the "screw top lid" puzzle faster than any ape except man) but it's brain is nothing like that of a mammal. It has no left/right hemispheres and the neurons are distributed along it's arms rather than concentrated in a central organ.
Machines can now learn from weakly structured and contradictory data sources such as pages on the net and answer trivia questions better than humans (re: IBM;s Watson). To me this indicates we already have AI that surpasses the logical (left hemisphere) of human intelligence, our right hemisphere is the "in the moment" intelligence that we share with the Octopus, computers simply don't have the scale of sensory input that our right hemisphere thrives on and until they do their "thought processes" will rely on an artificial "right hemisphere" (such as whatever if finds on the internet). That doesn't mean it won't appear to have human like responses, after all most duck hunters know how to imitate a duck.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
I'm just amazed at the researched who actually managed to find a job where they actually get paid to sit on their asses and dream up this type of drivel...
Hmm, the humour and sarcasm seem to have been be lost on you.
Project Chess. 'nuff said.
Sigs? We don't need no steekin Sigs!
Some posters have already touched on this, and I might have modded them up instead of posting myself if I had mod points right now, but, since I don't...
I'm thinking about this as a secular humanist/Darwinist not a believer in some form of Zoroastrian/Hindu/Judeo-Christian-Islamic religion, so, what do I expect in a million years? Humans like myself still running the world? Evolved super-humans? Or aritificial intelligences that owe their existence to human beings and are the heirs of humans as much, if not more, than humans are the heirs and owe their existence to the first primates.
Are we supposed to have machines of superior intelligence that take care of us? Keeping us in super high tech zoos that are like earthly paradises to us? Are we supposed to have merged with the machines somehow?
I don't know the answers to these questions, but I suspect humans will be only a memory, hopefully a a grateful memory. Hopefully, there is some 'point' to exiistence, and our heirs will be moving towards fulfilling that 'point'. ( I'm being as deliberately vague about this point as I can. If you don't get what I mean by it, don't worry, it's not important.)
In theory, theory and practice are the same; in practice they're different. (Yogi Berra & A. Einstein)
What will smack us hard on the chin is being forced to change basic beliefs and attitudes. Normal employment will vanish quickly. We will be forced to confront facts that we do not like to deal with. As the clarity of information becomes more and more pure and reliable how can we handle it. For example does anyone want to seriously discuss CO2 levels and the effect of human reproduction? How about pollution and population sizes? Right now we can rebuild portions of New york hit by a hurricane and pretend that even worse storms will not strike there all too soon. We can pretend we have evacuation routes when we already know that we can not evacuate many areas unless we have months of forewarning.
Knowledge can kick our asses really hard.
Meh. Maybe. Assuming we don't keep centralizing everything like we're currently doing with governments, networking, communication networks, and media producers. The trend is more and more concentrated power in bigger and bigger machines. Server farms have given way to big iron and virtualization. The Internet is evolving from millions of loosely connected web sites to Google, Facebook, Amazon and a few others in control of most content. So maybe you get Verizon's AI in conflict with Comcast's, but that doesn't mean they won't agree that meatspace humans are nothing but a drag on their efficiency.
The other thing that these guys never consider is that as AI improves (if it ever does to the level they contemplate), it will be natural that people will want to imbue those systems with the own intelligence and personality, rather than some generated artificial version. The human-machine interfaces are advancing at least as fast as the ability of computers to make autonomous decisions, and it's an easier path to immortality than trying to extend the lifespan of a meatspace body. So you could just as easily end up with human intelligences occupying the same machine resource space as the AI entities. So maybe the real conflict will be between the virtual space humans and the meatspace humans. After all, you can support a lot more virtual humans with Earth resources than you can meatspace humans...
"Somebody has to do something. It's just incredibly pathetic it has to be us."
--- Jerry Garcia
That’s just Muphry's Law in action.
Sig Battery depleted. Reverting to safe mode.
He could be advocating becoming your very own personal robotic overlord. Who wouldn't jump at that chance.
I do not want your cheap brainburning drugs. They are useless for work. And I am a working man today.
computers have been working on taking over man since the electronic brain era of the 1940's
guess what
they still cant get past the if, else if, else logic developed 65+ yeas ago, its all still programmed by man, who cant translate all its brainpower into a simple T/F test of simple facts presented to the computer
I feel safe for now, my damn computer is multiples of magnitude more powerful than when this research started, still cant complete a fucking update without waiting 14+ hours for me to hit a god damned button
..... might be beneficial here - we can see technological evolution as something related to sociocultural evolution (the evolution of socially transmittable behaviors). The industrial revolution creating machines which produce copies of another artifact or even tool. Ours is a Technological and SocioCultural as well as Genetic Ecosystem with interdependency, and slowly we approach the point where some machines reproduce themselves - indeed if you see software as a virtual machine and GMOs as biotechnology than this is already happening.
Now all Ecosystems tend to have fragility; organic networks can also have fractal degree distributions with massive hub points which introduce the possibility of catastrophic tail events. Man made networks have had a tendency to be even more skewed distributions than other organic systems. So for me the intelligence of the technology is less relevant to its Virulence and its Evolutionary and Ecological impact on the Biosphere, Technosphere and Nusphere.
I wrote about the CSER last year at http://www.thisiswhyweredoomed.com/2012/12/europeans-will-doom-us-all.html - if you take this and combine it with the news that the EU is building the world's most powerful laser, you'll wonder why the movie version of Skynet even bothered with a time machine in the first place...
(oh yeah, they already HAVE a Skynet - https://en.wikipedia.org/wiki/Skynet_(satellite)
Perfectly Normal Industries
I think you are both right and wrong. Yes we are intelligent. We can figure out all the smart things but we cannot figure out how to get a bit more peace on this planet or how not to completely destroy our planet while making another attempt to produce more of more tasteless burger or more of more useless gadgets so no we are not all that smart just smart enough to figure that life in the cave with fire is better than on the tree and without - not much intelligence needed for that. There is also another take on this - one of these gays said at Economist conference that we are the dumbest of to get the technology going simply by the virtue of inventing technology as soon as it was possible for our simple glue based brains. In a sense something of a law of nature.
Another thing they said is that this machine apocalypse does not have to be due to intelligence difference per se but due to autonomous agents that can execute our orders too well without looking at consequences. There are many ways the progress can go wrong and quite frankly all the crises that this has caused are not new - there many civilizations before our modern one which used up resources to move forward which just destroyed their livelihoods. Back then there was always another place to go which in a world that is so interconnected is not really true.
I am not sure if the institution in question produce no value. In our history there have always been a lots of 'waste' spent on producing things and ideas of no apparent value yet after years gone buy suddenly such ideas started appearing useful. In other words maybe - if you only do things that satisfy your current direct needs and optimize all your economy and society in order to do that, in good case you achieve system that is unstable and prone to collapse when pushed by changing of environmental conditions or internal structure.
well consider the shitty code that my group is churning out every day that I have the misfortune to test and maintain parts of I consider their job actually inspiring and worthwhile and more useful for the society. Even if it were not so it would not amaze me at all that they got funding. In country I live in there are parties that are or were part of the nationwide government that failed to show any sign of intelligent activity except drive for good money after the years 'serving' the nation and I have intelligent hard working friends supporting them with their money - how amazing (and scary) is that?
I'm sorry, but I think you're completely wrong. Intelligence doesn't breed opinion, emotion does. Without the desire to be correct, all answers to the same problem will eventually be the same. The desire to hold on to a theory based on flawed data is an emotional response that machines would and should be without.
We don't compete for the same resources. Also, machines could simply be programmed to not want to kill humans. There is not reason to think they would resent this any more than humans resent being programmed to not want to kill humans.
Wow. AI doomsayer neo-luddite idiots are here too. Those "institutes" are neo-luddite "think tanks" that are trying to spread FUD about AI research.
To hell with them!
--exa--
I saw this in the article:
"A super-intelligent machine could be given a straightforward goal â" such as making 32 paper clips or calculating pi â" but "could pursue unlimited resource acquisition if there were no relevant cost to the agent of doing so"."
The first thing I thought was "hey, isnt that just like T.S. Eliot at his banking job?"
The second thing I thought was 'does this remind any of you of Bomb in the movie 'Dark Star'?
The third thing I thought about was the Keith Laumer stories with artificially-intelligent bureaucratic devices that were waaaaay too sophisticated for their users' own good... and of course, right after that, up came the consequences from Woody Allen's toaster being networked...
My concern is that all the techniques humans use to influence organic computers would be used on cybernetic machinery, and there is a considerable amount of science fiction (good and bad) that deals with the consequences of teaching intelligent machinery the wrong things, or the expedient things, or the mistakenly contradictory things, that produce neuroses, prejudice, and even outright misanthropy in the existing intelligent systems that are people. HAL was just fine until he had to process conflicting agendas phrased as directives; so was David Mace's Nightrider. (And we haven't quite gotten to the WOPR or Skynet -- and we're still stuck dealing with the anthropic fallacy in assuming how evolved 'intelligent' systems might manage what 'intelligence' constitutes.
Seems to me that a major difference between most machines and most organisms we currently define as life is natural selection. Many "human" traits derive from the drive to survive, procreate, and adapt, because that's how living things got to this point. Most of the machines we've created, however, have been created by us to exist as designed (intentionally or not). If they're designed to replicate, they're designed to replicate exactly. A few people out there are creating machines that evolve, but not many. If we want to understand how smarter machines are going to behave, then we need to design them to adapt in order to survive. Then we'll start to see emergent behavior that we can understand.
Some have suggested that the reason we haven't detected intelligent life outside our solar system is that once a civilization advances to the point it can reach out, it has advanced to the point that it destroys itself. Suppose that it's the machines that turn on the organic beings and wipe them out. Then the machines go covert, hiding their "civilization" to protect it from outsiders. The machines were not created through natural selection, so they have no basic drives to explore or otherwise reach out beyond their planet.
However, surely some living thing thought of this elsewhere and tried it. Apparently it didn't work out, since we haven't heard from them. Oh well.
it will be natural that people will want to imbue those systems with the own intelligence and personality, rather than some generated artificial version.
True. However it's entirely possible that man will not directly create the superhuman AI, but that it may emerge unintentionally from the interaction of systems created for other purposes.