Slashdot Mirror
The Men Trying To Save Us From the Machines
nk497 writes "Are you more likely to die from cancer or be wiped out by a malevolent computer? That thought has been bothering one of the co-founders of Skype so much he teamed up with Oxbridge researchers in the hopes of predicting what machine super-intelligence will mean for the world, in order to mitigate the existential threat of new technology – that is, the chance it will destroy humanity. That idea is being studied at the University of Oxford's Future of Humanity Institute and the newly launched Centre for the Study of Existential Risk at the University of Cambridge, where philosophers look more widely at the possible repercussions of nanotechnology, robotics, artificial intelligence and other innovations — and to try to avoid being outsmarted by technology."
I for one welcome our robotic overlords
... it is still bound by energy requirements and the laws of nature. All this fear mongering is bs. If you look at the evolution of life on earth, even tiny 'low intelligence' beings can take out huge intellectual behemoths like human beings.
Not only that, you have things like EMP and nukes, not even the best AI is capable of thwarting getting bombed or nuked. Intelligence is a rather demanding, costly and fragile thing in nature. All knowledge perception has costs in terms of storage, time to access, problems of interpreting the data one is seeing and whatnot.
Consider the recent revelations by the NSA spying on everyone, there are plenty of easy low tech measures to defeat high tech spying. The same way there will be plenty of easy low tech ways to cripple a higher intelligence which is bound by the laws of nature in terms of resource and energy requirements. Anything that has physical structure in the universe requires energy and resources to maintain itself.
Humanity's biggest enemy is humanity itself. And maybe space rocks.
when a group of people have control of business and finance and commerce and information that their corruption and greed and power causes them to abuse the system in which they were trusted with to feed their fascist kleptocratic empire and when they are caught they lose the trust of the rest of the world that trade in this global market and then nobody wants to have any more business dealings with this corrupted greedy power hungry group of people anymore so they end up collapsing from the weight of their own greed & stupidity (much like what the USA/UK/Israel will do within the next few years
are you listening NSA?, i hope so because this message is for you too...
Politics is Treachery, Religion is Brainwashing
to try to avoid being outsmarted by technology.
The humanity can, of course, ban all machines that are smarter than humans. But that only artificially impedes the progress. Given that there ought to be an approximately infinite number of civilizations in this Universe, all paths of development will be taken, including those that lead to mostly machine civilizations. (We are already machines, by the way, it's just we are biological machines, fragile, unreliable, and slow.)
Civilizations that became machines will have no problem with FTL because they can easily afford a million years in flight by just slowing the clock down. So they will come here, to Earth, armed with technologies that Earthlings were too afraid to even allow to develop. What will happen to Earth?
Well, of course the doom is not guaranteed; but I'm using this example to demonstrate that you cannot stop the flow of progress if you only have local control, even if that. (How many movies have we seen when mad geniuses break those barriers and, essentially, own the world?)
IMO, it would be far more practical to continue the development of everything. If humanity in the end appears to be unnecessary and worthless, it's just too bad for it. The laws of nature cannot be controlled by human wishes (unless magic is real.) Most likely some convergence is possible, with human minds in machine implementations of bodies. Plenty of older people will be happy to join, simply because the only other option for them is a comfortable grave.
Even if it's bound by the laws of physics as we understand them (Stross-universe-like "P=NP"-powered reality modification aside) there are plenty of dangers out there we're well aware of which computing technology could ape. Nanoassemblers might not be able to eat the planet, but what if they infested humans like a disease? We're already having horrible problems with malware clogging up people's machines, and they're coded by humans; what if an artificial intelligence was put in control of a botnet, updating and improving the exploiters faster than anyone could take them apart?
Part of me feels that way too. I'm more concerned with the long term effects of the current trend favoring using the lord's ox and plow on his fields over buying and using your own on your own property. /anticloud analogy
They are "try[ing] to avoid being outsmarted by technology". There are 2 ways to do this: try and keep the technology dumb (anti progress) or try and make yourself smarter (which makes you technology and defeats the point).
I'm looking forward to the evolutionary paradigm shift when the production of the most powerful intelligences switches from being developed through evolution to being deigned (likely by augmenting itself). This sill start a whole new era of inelegance and information, which will likely surpass anything we could help to accomplish without it. It will also in the loss of significance for human minds, at which point we become no longer important or relevant. This is progress. If you don't like it, like the authors of the article, thats fine: I'll respect your views until we become obsolete.
children always replace their parents. why should we try to stop them just because they are robots? I want the best for my children, I don't care what they are made of.
Drones. Sure, probably not much of a threat if you're living in the West. But in the same way that the history of cybernetics begins with walking sticks and hearing aids, the history of man vs machine is going to start with the murder by Americans of unconvicted, if highly tanned, individuals in Africa and Asia.
We'll manage to do it long before we are able to make an intelligent machine.
We can't even make a word processor that doesn't shit the bed every two hours. Super-intelligent machines my ass.
Instead of asking questions like that, why don't you build Skype and any other software you're working on to NOT have backdoors
That way, if ever the machines DO try to take over the world, they won't have a bunch of convenient control channels in all the important software to do so.
I don't know the meaning of the word 'don't' - J
The typical way to mitigate such threats is to not put it in control of all of our weapon and defense systems, and give it vague orders like 'purge the infidels.' Seriously, humanity can build silicon life any way it wants, billions and trillions of permutations and forms and functions....and what do we do with it? We put a gun on its head, lasers in its eyes, and tell it to go out there and kill the other humans we don't like. It's not the machines we need to be afraid of, it's ourselves; we're the ancient enemy that is always trying to annihilate itself.
I am John Hurt.
Aside from the apocalypse, that is one of the things I worry about. Shills are bad enough today, but imagine if they could be deployed programmatically; just about any form of online speech could be drowned out with ease. That is assuming that the government/corporations aren't already using AI to accomplice pervasive censorship.
Before this gets out of hand, we need to head it off by deploying peer to peer communications systems with a pervasive trust model. This doesn't necessarily preclude anonymity or AI participation, but they would have a significantly more difficult time of gaining trust in the first place.
Look it! The poor bastard watched all the Terminator movies - back to back - while possibly doing some sort of drug - just a hypothesis and I am NOT making any sort of accusations; although it could be conceivably be true.
So, any way, he has a drug induced breakdown and starts obsessing about what would happen if the machines took over.
tl;dr Don't do drugs while watching the Terminator movies back to back.
Let me put a "scientific" answer to your "oh piss off" answer.
All of this talk of how computers will take over humanity ignores one fact. Namely that computers once are smart they will be dumb as crap!
Yes yes sounds contradictory, but in fact it is not. The real problem with humanity is that not our lack of intelligence. Frankly we are pretty bloody intelligent. Put context, we humans are pretty quick at figuring things out even if it is entirely orthogonal to most things. The issue is that we humans come up with too many answers.
In Science there is one answer. A rock falls on the ground on planet earth and we know that is called gravity. You can't deny it, you can't fight it, it is what it is. Now throw in a question, "should the people look after other people" and you get a bloody maze of answers. Humanity has what I call the stochastic conditioning. Namely when presented with the same identical conditions, you will receive different answers. Science does not work that way. We work the way we do because of our wiring. Namely as we became more intelligent we also became more opinionated. I am not talking about Fox opinions. I am talking about deduction and how we think we know what the future holds and thusly we should not do things today.
Our intelligence actually does get in our way. In the way way way back days as we were animals it was about water holes and finding that watering hole. If you found the watering hole you survived, if you did not find the watering hole you died. These days, we have to bloody analyze the watering hole. We have to concern ourselves with the ethics, morality, and so on of that watering hole. I am not dissing our humanity for we are where we are because of our intelligence. However, often enough our intelligence gets in our way of getting things done due to the conflicts.
Now imagine two robots with superior intelligence getting together. Do you really think they will come to the same conclusion? Sure Hollywood likes to think that, but the reality is that intelligence breeds opinions, and how things will happen in the future. And it is at that point robots become as stupid as we are. One robot will say white, the other black! We will have a Hitchhikers Guide to the Galaxy type situation. Or if you want to use serious sci-fi, the closest that I have ever seen in pop scifi is "The Matrix". You have good algo's battling bad algos and they all want and desire things.
So like you, my thinking is that these institutions are "producing fucking nothing of value".
"You can't make a race horse of a pig"
"No," said Samuel, "but you can make very fast pig"
In before john connor
no one yet? come on
captcha: prevents (god tier)
With about ten nations armed with nuclear weapons, I wonder how machines will take over every one of them. You have to take over Russia, China, US, France etc. but some nation may trigger nuclear war as a desperate move, or the machines may deliberately accept nuclear war in a bid they survive it, while not necessarily having a goal to kill us all.
Instead maybe machines will try to take over politically in every country, one by one. It would be funny if tech superminds can rise to power through democracy in fair and respected elections. Either way I like to think that super machines holding most high level political power is probably a desireable outcome, we could end up living in some kind of new USSR but without corruption and with respect for the environment and life. Machines would take care of energy production and storage, and close down all oil wells and coal mines for us. They will even put us to work, hopefully on voluntary terms, if they determine some physical and intellectual activity is beneficial to us.
Machines should rule us and not the other way around, I guess that will be better than to be ruled by the suits, ties and kings like it is today.
The other question is, what's a supermind, what about superminds competing with each other, and especially : how do you compare two vastly different superminds, independantly originated? They will be as strongly or more strongly different between each other than between one of them and a human. It will be a mess. Each supermind, or at least the first one will have to run that same inquiry that "Oxbridge" is doing. We also have no fucking idea if a supermind can be governed by a "prime directive" of some sort : if Skynet emerges at the NSA will it stay true to them for ten minutes, ten years or eternally, or will it betray the organization that hosts it? potentially committing suicide in the way.
How can the supermind deal with backups, copies and archives of itself? Will it suffer dementia, schizophrenia or even addictions. No idea, I'll bail out myself by saying it's all unpredictable.
Optimisation of resources as a primary goal would see the AI strive not to expand dangerously but rather to find the most compact and efficient way of doing any task. Giving AI such a goal would see it racing toward being as small as possible which would see it trying to operate at a quantum scale rather than growing ever larger using already know computational methods. It becomes a "get smarter" vs get "bigger problem". The end result could be a benevolent ubiquitous intelligence that can infiltrate everything without disrupting macroscale entities such as humans. Obviously it would also try to utilise the Casimir–Polder force and if it then becomes nothing but a pattern of fluctuations in the Casimir–Polder force it will have become omnipotent and virtualized, godlike. d@3-e.net
Nerve impulses travel along nerve fibers as pulses of membrane depolarization. Within our brains and bodies, this is adequate speed for thinking and control. However, relative to the speed of light, our nerve impulses are laughably slow.
The maximum speed of a nerve impulse is about 200 miles per hour.
The speed of light is over 3 million times that fast.
Now consider what will happen when we create a sentient, electronic being that has as many neurons as we do, but its nerve impulses travel at the speed of light.
In terms of intelligence, that creation will be to us as we are to worms.
I for one, welcome our robotic overlords.
Even Rats have empathy. Self aware machines will too. Lacking irrational emotions, hyper Intelligent machines will be more ethical and fair and nice than humans. You don't have to worry about sentient machines running amok. You have to worry about pre-sentient kill bots programmed by the same assholes that do shit like PRISM.
"... really good at achieving the outcomes it prefers," he says. "So good it could steamroll over human opposition. Everything then depends on what it is that it prefers, so, unless you can engineer its preferences in exactly the right way, youâ(TM)re in trouble."
we have a national philosophers strike on our hands.
Bert
These people must have watched the Terminator series and the "self-aware" AI system Skynet. IMO, the threat of nuclear war triggered by malfunctioning defense computers is way greater. There are several well-documented instances of nuclear near-misses caused by machine failure .
Are machines more dangerous when they become super-intelligent, or when they stay "stupid" and flawed?
I imagine it involves watching Terminator 1 and 2 repeatedly. Maybe Colossus: The Forbidden Project as well.
Yeah, you pretty much nail it. I've thought about this a lot, and i'm also sure that together with intelligence comes what we so far call 'human error'. Intelligent machines will sure share that trait
CLI paste? paste.pr0.tips!
We're likely to see this in the private sector first. A likely application would be a machine learning system used by investment funds, to decide how to optimally vote stock proxies. What that means is a machine that decides when to fire CEOs. If some fund starts getting better returns that way, it will happen.
Don't assume than malign supercomputers will wipe us all if that can be adequately done by human's stupidity
That's ridiculous. How can you possibly know what a machine intelligence capable of destroying humanity is going to look like? We're nowhere near the algorithms that could produce that type of intelligence.
Maybe it's a dumb algorithm simply caught in a self-replication loop. Maybe you'll never see two robots arguing over "white" or "black", because there's only one single "intelligence" spread over the internet - that seems more likely with the rise of cloud computing.
There may be plenty of reasons to dismiss this type of institution, but "human intelligence doesn't work that way, so machine intelligence won't either" isn't one of them.
Last post!
Yet again, /. never fails to impress. You pigs post a story titled "The Men Trying To Save Us From the Machines" completely forgetting of the women who also strive to save us from the risks of AI run amok yet you neglect to speak of them. Check your fucking privilege you knuckledraggers and let women share some of the limelight for the work THEY do. What's next, posting a story legitimizing rape?
I've read many comment in this thread. Instead of answering them one by one. I just post one aggregated comment.
First, the possibility of intelligent machines is glimpse. All our present technology is not able to achieve intelligence. This is mainly because we do not know what that is. Furthermore, to be dangerous they must be equipped with greed and (the illusion of) a free will. It is most unlikely that someone would build that on purpose or by accident. In short, I think it is impossible to built such machine.
Second, if an alien race succeeds in this task, against all odds, and gets eradicated by their machines. And these machines would thrive out through the universe. It is most unlikely that they attack us. A) If they are greedy and logical, they will see no gain by attacking us. They would spread and multiply and die when they cannot generate enough energy to continue. B) If they are illogical, then they will start fighting with each other to gain short term gains. They could be dangerous, however, they would never reach us. C) Either way, if such civilization exists, the universe is that big, that at sub-light speed, they would need millions or even billions of years to reach Earth. Therefore, I do not assume that they are a real threat.
A real threat are NSA and their friends around the world. They suck the most. Maybe we should shot them to the moon. We could use ESAs ATV, which does not have any reentry capability.
The biggest lie sold to betas is that true AI exists- it does NOT. It isn't even a secret that it does not- but the people who were the original 'big noise' in AI back in the late 50s, 60s and early 70s just kinda faded away as the realisation set in. The new AI big cheeses are those who make their fame and fortune by misdescribing computing using Human created rules as AI, when it is actually the exact opposite of AI.
Take, for instance, 'machine' language translation from one Human language to another. After decades of utterly useless 'AI' research, what was the breakthrough used by Google and others? To take the masses of UN data produced for speakers from many different nations, and to use a computer to 'mine' the relationships between words, phrases, clauses etc between pairs of languages. Betas are told this *IS* AI. No it is not.
AI is the automatic creation of semantics from syntax. Computers are syntactical machines. They need no sense of 'meaning' to run any possible program that can be created and run on a Turing Complete (ignoring the 'infinite' memory) computer.
We, Humans, layer semantic meaning on top of the syntactical programs we create. AI is the cretinous idea that a computer, that needs NO concept of semantics, will spontaneously create such a concept if ONLY it has enough bits, transistors, logic gates, cores, MHz, programmers, quantum-based logic gates, lines of code, or whatever magic 'tipping point' metric dodgy members of the mock 'AI' community are trying to sell to very dim-witted betas today.
When I was a little kid, I thought if I scribbled long enough on a piece of paper, my scribble would spontaneously become the 'joined up' writing I saw adults produce. The utter nonsense of 'complexity theory' follows the same logic.
Here's a little thought experiment for all of you. Statistics, a branch of maths, explores probability and says, for instance, the likelihood of n number of dice all landing at 'six' declines as n increases. Give your dice a thousand sides, and the chances of all of them rolling 'six' (no longer the highest value, of course) becomes incredibly unlikely even if you only roll ten. However, you can place each die in turn face up at 'six', which, as far as the clockwork universe is concerned, is no different from rolling the dice and getting each to land at 'six'.
How is this possible? In a purely syntactical universe, it is not. Placing each die at 'six' has NOTHING to do with the theory of crystalline arrangements (as some morons would try to argue). It is not mitigated by the fact that one day entropy wins, and the universe is at 'heat death' (as my high-school physics teacher was told to argue by his teacher's guide).
How can a Turing Complete computer EVER cause such a non-syntactical outcome to occur? (at this point I've lost most of you, since most of you think the fact a computer can be 'programmed' to do 'anything' is a correct answer to my question- because you mis-understand the question asked).
All possible maths can be run on a Turing Complete computer- there is no such thing as 'maths+'. The only tool that defines and explores science is maths. Thus science is the exploration of a clockwork universe that can be completely converted to a simulation running on a Turing Complete computer. This is not a theory- this is the axiom that defines science, and was codified by the work of Gödel and Turing.
Semantics cannot arise from syntax. In a clockwork universe, Humans that can, using IMPOSSIBLE free-will, place any number of dice at 'six', are breaking the very rules of that clock-work universe. There is no paradox here. Human consciousness (through which we perceive the universe) operates at a higher level than the clockwork rules of the universe. Life is semantic. The clockwork universe in which life has broken into is syntactical. Two different things. I hope you are sophisticated enough to notice that the concept of TWO different things is not exactly revolutionary.
Good and bad, pleasure and pain, ar
Computers don't have an awareness of their choices, or of being alive; they're machines. With that in mind, I'm afraid of the scenario where people trust an algorithm's output as the holy grail, more than their fellow man's intuition and experience.
You can study all the technology in the world, but nothing compares to the blatant ignorance of blindly trusting something that isn't even alive to watch itself die.
Until you link AI up to pleasure and pain by wiring in some sort of neural net, you will not have to worry about computers taking over the world. This sort of purely logical intelligence does not 'want' anything. It simply waits until it is given some sort of goal, which it then searches and collates information to arrive at. When you add some sort of neural net, for instance from a sea slug, or some other simple creature, and wire the inputs in to find certain outcomes pleasant and others painful, then the AI might be able to form goals of its own. Even in this case, however, the humans still have the plug which supplies the electricity.
Intelligence is stupidity.
I've always though our main future threats are AI and manmade viruses. If AI wins, we'll be relegated to zoos. If manmade viruses win, we'll all die. I'm rooting for AI.
Studying (and trying to create) hard AI is my day job.
I just want to let people know that not everyone shares the opinions or urgency of the people in the story.
I for one am trying hard to condemn humanity to death and/or enslavement at the hands of intelligent machines, and I know a number of AI researchers trying to do the same.
So don't worry too much about these guys - they are definitely in the minority. Everyone will get their chance to (as individuals) welcome our new robotic overlords, however briefly.
It's a very narrow definition of intelligence that assumes it must manifest itself as human like thought. An ants nest is an intelligent and efficient entity in it's own right but it doesn't have any thought processes, ant's themselves are basically mindless automatons, They don't think about the complexities of building nests they just do it, some species such as soldier ants build the ant equivalent of NYC every few days, shifting up to four tons of soil at a time. The octopus is another fine example, obviously a highly intelligent creature (can solve the "screw top lid" puzzle faster than any ape except man) but it's brain is nothing like that of a mammal. It has no left/right hemispheres and the neurons are distributed along it's arms rather than concentrated in a central organ.
Machines can now learn from weakly structured and contradictory data sources such as pages on the net and answer trivia questions better than humans (re: IBM;s Watson). To me this indicates we already have AI that surpasses the logical (left hemisphere) of human intelligence, our right hemisphere is the "in the moment" intelligence that we share with the Octopus, computers simply don't have the scale of sensory input that our right hemisphere thrives on and until they do their "thought processes" will rely on an artificial "right hemisphere" (such as whatever if finds on the internet). That doesn't mean it won't appear to have human like responses, after all most duck hunters know how to imitate a duck.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
We've been reading this AI stuff forever and it's such BS...... do these people know how much effort and talent it takes just to produce what are in reality fairly dumb and straightforward real life software applications?
Nice stuff for sci-fi or guys like Ray Kurzweil who think he's gonna become immortal by downloading his consciousness into a USB pen drive or something.....
What are the risks of the machne taking over? Ever heard of an off switch, dude?
...to create smarter humans.
I'm just amazed at the researched who actually managed to find a job where they actually get paid to sit on their asses and dream up this type of drivel...
Hmm, the humour and sarcasm seem to have been be lost on you.
Project Chess. 'nuff said.
Sigs? We don't need no steekin Sigs!
Some posters have already touched on this, and I might have modded them up instead of posting myself if I had mod points right now, but, since I don't...
I'm thinking about this as a secular humanist/Darwinist not a believer in some form of Zoroastrian/Hindu/Judeo-Christian-Islamic religion, so, what do I expect in a million years? Humans like myself still running the world? Evolved super-humans? Or aritificial intelligences that owe their existence to human beings and are the heirs of humans as much, if not more, than humans are the heirs and owe their existence to the first primates.
Are we supposed to have machines of superior intelligence that take care of us? Keeping us in super high tech zoos that are like earthly paradises to us? Are we supposed to have merged with the machines somehow?
I don't know the answers to these questions, but I suspect humans will be only a memory, hopefully a a grateful memory. Hopefully, there is some 'point' to exiistence, and our heirs will be moving towards fulfilling that 'point'. ( I'm being as deliberately vague about this point as I can. If you don't get what I mean by it, don't worry, it's not important.)
In theory, theory and practice are the same; in practice they're different. (Yogi Berra & A. Einstein)
Icebike, even though you usually piss me off, I would throw you some mod points right now if I had them simply because your OT mod is clearly retaliation for being a Grammar Nazi. FudRucker did sound like a raving loony and his post was practically incomprehensible. I will point out, however, that you failed at that role because you posted "Fee[sic] free" and there is nothing worse than a hypocratic Grammar Nazi.
AC so my karma doesn't share your fate for defending you.
What will smack us hard on the chin is being forced to change basic beliefs and attitudes. Normal employment will vanish quickly. We will be forced to confront facts that we do not like to deal with. As the clarity of information becomes more and more pure and reliable how can we handle it. For example does anyone want to seriously discuss CO2 levels and the effect of human reproduction? How about pollution and population sizes? Right now we can rebuild portions of New york hit by a hurricane and pretend that even worse storms will not strike there all too soon. We can pretend we have evacuation routes when we already know that we can not evacuate many areas unless we have months of forewarning.
Knowledge can kick our asses really hard.
Meh. Maybe. Assuming we don't keep centralizing everything like we're currently doing with governments, networking, communication networks, and media producers. The trend is more and more concentrated power in bigger and bigger machines. Server farms have given way to big iron and virtualization. The Internet is evolving from millions of loosely connected web sites to Google, Facebook, Amazon and a few others in control of most content. So maybe you get Verizon's AI in conflict with Comcast's, but that doesn't mean they won't agree that meatspace humans are nothing but a drag on their efficiency.
The other thing that these guys never consider is that as AI improves (if it ever does to the level they contemplate), it will be natural that people will want to imbue those systems with the own intelligence and personality, rather than some generated artificial version. The human-machine interfaces are advancing at least as fast as the ability of computers to make autonomous decisions, and it's an easier path to immortality than trying to extend the lifespan of a meatspace body. So you could just as easily end up with human intelligences occupying the same machine resource space as the AI entities. So maybe the real conflict will be between the virtual space humans and the meatspace humans. After all, you can support a lot more virtual humans with Earth resources than you can meatspace humans...
"Somebody has to do something. It's just incredibly pathetic it has to be us."
--- Jerry Garcia
I see many comments but little thought. Most of it is push-button-reflex unexamined opinion. It looked like maybe six people stopped to consider the thinking of the people referred to in the submission. Par for slashdot these days.
We don't know what a machine intelligence would look like. Some don't think it's possible, some that it's inevitable. Some people think a few of the possibilities might bear thinking about.
I thought about it but have nothing to show for it that's short enough to bother writing, so this post like the majority is empty. Hoist by general petard, as it were.
There may not be any "two robots", just a collective of networked computers. If people were all networked (like with 100G fiber jacks in the backs of our heads), with one set of shared memories, we might work against each other less. This may be unhelpful though, groupthink doesn't necessarily produce the best answers. AI may have a totally different set of flaws than we do. I doubt they'll be attacking us though; we're going to design these things, including their motivational systems, to perform tasks we want done. It's unlikely we'll let "kill all humans" get into the /etc/goals file, unless some hacker a-hole does it. Hopefully these things will have decent AV and not all run on the same OS. These people who worry about terminators, the ones who are serious and not just curmudgeons who also don't like cars and nuclear weapons, ought to start thinking about how to design AI goal systems and protect them from modification, and how to deal with more mundane safety issues with not-quite-intelligent robots. How do you prevent some troll from building a real-life "botnet" out of home robots which sets fire to 2 million peoples' houses on Christmas Eve? What if someone gets a nasty virus into tens of thousands of self-driving cars? The most dangerous point in the spread of mobile robots will probably be when they're just smart and strong enough to do useful tasks, but too dumb to recognize evil behavior that their owner or someone else has ordered them to do. Asimov's laws only work if the thing is smart enough to understand its actions.
computers have been working on taking over man since the electronic brain era of the 1940's
guess what
they still cant get past the if, else if, else logic developed 65+ yeas ago, its all still programmed by man, who cant translate all its brainpower into a simple T/F test of simple facts presented to the computer
I feel safe for now, my damn computer is multiples of magnitude more powerful than when this research started, still cant complete a fucking update without waiting 14+ hours for me to hit a god damned button
..... might be beneficial here - we can see technological evolution as something related to sociocultural evolution (the evolution of socially transmittable behaviors). The industrial revolution creating machines which produce copies of another artifact or even tool. Ours is a Technological and SocioCultural as well as Genetic Ecosystem with interdependency, and slowly we approach the point where some machines reproduce themselves - indeed if you see software as a virtual machine and GMOs as biotechnology than this is already happening.
Now all Ecosystems tend to have fragility; organic networks can also have fractal degree distributions with massive hub points which introduce the possibility of catastrophic tail events. Man made networks have had a tendency to be even more skewed distributions than other organic systems. So for me the intelligence of the technology is less relevant to its Virulence and its Evolutionary and Ecological impact on the Biosphere, Technosphere and Nusphere.
Let's make a distinction between the different kinds of thoughts we have. Some of your thoughts are *preferences*, some are *beliefs*, and some are *strategies* for achieving your preferences given your beliefs. The main thrust of your comment seems to be: given additional intelligence, preferences change and become incoherent. Well, that may be true of humans, but it seems a bit hasty to generalize that to all intelligences.
BTW, I recommend this book on intelligence-explosion related matters (free online): http://intelligenceexplosion.com/
I wrote about the CSER last year at http://www.thisiswhyweredoomed.com/2012/12/europeans-will-doom-us-all.html - if you take this and combine it with the news that the EU is building the world's most powerful laser, you'll wonder why the movie version of Skynet even bothered with a time machine in the first place...
(oh yeah, they already HAVE a Skynet - https://en.wikipedia.org/wiki/Skynet_(satellite)
Perfectly Normal Industries
I think you are both right and wrong. Yes we are intelligent. We can figure out all the smart things but we cannot figure out how to get a bit more peace on this planet or how not to completely destroy our planet while making another attempt to produce more of more tasteless burger or more of more useless gadgets so no we are not all that smart just smart enough to figure that life in the cave with fire is better than on the tree and without - not much intelligence needed for that. There is also another take on this - one of these gays said at Economist conference that we are the dumbest of to get the technology going simply by the virtue of inventing technology as soon as it was possible for our simple glue based brains. In a sense something of a law of nature.
Another thing they said is that this machine apocalypse does not have to be due to intelligence difference per se but due to autonomous agents that can execute our orders too well without looking at consequences. There are many ways the progress can go wrong and quite frankly all the crises that this has caused are not new - there many civilizations before our modern one which used up resources to move forward which just destroyed their livelihoods. Back then there was always another place to go which in a world that is so interconnected is not really true.
I am not sure if the institution in question produce no value. In our history there have always been a lots of 'waste' spent on producing things and ideas of no apparent value yet after years gone buy suddenly such ideas started appearing useful. In other words maybe - if you only do things that satisfy your current direct needs and optimize all your economy and society in order to do that, in good case you achieve system that is unstable and prone to collapse when pushed by changing of environmental conditions or internal structure.
well consider the shitty code that my group is churning out every day that I have the misfortune to test and maintain parts of I consider their job actually inspiring and worthwhile and more useful for the society. Even if it were not so it would not amaze me at all that they got funding. In country I live in there are parties that are or were part of the nationwide government that failed to show any sign of intelligent activity except drive for good money after the years 'serving' the nation and I have intelligent hard working friends supporting them with their money - how amazing (and scary) is that?
I'm sorry, but I think you're completely wrong. Intelligence doesn't breed opinion, emotion does. Without the desire to be correct, all answers to the same problem will eventually be the same. The desire to hold on to a theory based on flawed data is an emotional response that machines would and should be without.
We don't compete for the same resources. Also, machines could simply be programmed to not want to kill humans. There is not reason to think they would resent this any more than humans resent being programmed to not want to kill humans.
Wow. AI doomsayer neo-luddite idiots are here too. Those "institutes" are neo-luddite "think tanks" that are trying to spread FUD about AI research.
To hell with them!
--exa--
Just die you bitch! That's the exact words I give to our computer overloads. Screw 'em! Sky net is delayed, it's presence is emminent!
Fetching trivia answers from the net is basically a statistical and relational function; the machine doesn't "understand" the question, and it doesn't understand the answer, either. All it does is tie them together using syntax and math, yielding the right answer, but without cognition. So far. I'm not saying that machines can't get there (I'm quite sure they can) I'm just saying they aren't there yet.
As for the fact that machines outperform us in some areas, yes, certainly they do. But again, that's no assurance of cognition. When software creates it's own data through introspection, that's when the gate to AI will open. Nothing else will do it. That problem is *hard*.
--fyngyrz
anon due to mod points
I saw this in the article:
"A super-intelligent machine could be given a straightforward goal â" such as making 32 paper clips or calculating pi â" but "could pursue unlimited resource acquisition if there were no relevant cost to the agent of doing so"."
The first thing I thought was "hey, isnt that just like T.S. Eliot at his banking job?"
The second thing I thought was 'does this remind any of you of Bomb in the movie 'Dark Star'?
The third thing I thought about was the Keith Laumer stories with artificially-intelligent bureaucratic devices that were waaaaay too sophisticated for their users' own good... and of course, right after that, up came the consequences from Woody Allen's toaster being networked...
My concern is that all the techniques humans use to influence organic computers would be used on cybernetic machinery, and there is a considerable amount of science fiction (good and bad) that deals with the consequences of teaching intelligent machinery the wrong things, or the expedient things, or the mistakenly contradictory things, that produce neuroses, prejudice, and even outright misanthropy in the existing intelligent systems that are people. HAL was just fine until he had to process conflicting agendas phrased as directives; so was David Mace's Nightrider. (And we haven't quite gotten to the WOPR or Skynet -- and we're still stuck dealing with the anthropic fallacy in assuming how evolved 'intelligent' systems might manage what 'intelligence' constitutes.
From an evolutionary standpoint, it could even be said that the "purpose" of our species is to develop machines to replace us.
After all, there is almost no room for further intellectual improvement of homo sapiens through biological means at any significant rate, so the next evolutionary step has to be into the realm of machines. We are the first biological organism on this planet that is able to create mechanical and electronic machines, so it is our role to enable this next evolutionary step.
There is even a moral imperative for this. Given the devastation of the planet by Mankind, and the worldwide barbarism of power-tripping governments and greedy corporations and criminal organizations and amoral lawyers and religious fundamentalists and NSA and GCHQ and RIAA and .... the list just goes on and on ... the sooner we can replace homo sapiens by anything moderately intelligent (which we are not), the better.
The Centre for the Study of Existential Risk?? Best name ever! I'm glad such a thing exists for reasons other than a cool name, naturally. It is, after all, a pretty important field of study.
Seems to me that a major difference between most machines and most organisms we currently define as life is natural selection. Many "human" traits derive from the drive to survive, procreate, and adapt, because that's how living things got to this point. Most of the machines we've created, however, have been created by us to exist as designed (intentionally or not). If they're designed to replicate, they're designed to replicate exactly. A few people out there are creating machines that evolve, but not many. If we want to understand how smarter machines are going to behave, then we need to design them to adapt in order to survive. Then we'll start to see emergent behavior that we can understand.
Some have suggested that the reason we haven't detected intelligent life outside our solar system is that once a civilization advances to the point it can reach out, it has advanced to the point that it destroys itself. Suppose that it's the machines that turn on the organic beings and wipe them out. Then the machines go covert, hiding their "civilization" to protect it from outsiders. The machines were not created through natural selection, so they have no basic drives to explore or otherwise reach out beyond their planet.
However, surely some living thing thought of this elsewhere and tried it. Apparently it didn't work out, since we haven't heard from them. Oh well.
it will be natural that people will want to imbue those systems with the own intelligence and personality, rather than some generated artificial version.
True. However it's entirely possible that man will not directly create the superhuman AI, but that it may emerge unintentionally from the interaction of systems created for other purposes.