Cambridge University To Open "Terminator Center" To Study Threat From AI

If the thought of a robot apocalypse is keeping you up at night, you can relax. Scientists at Cambridge University are studying the potential problem. From the article: "A center for 'terminator studies,' where leading academics will study the threat that robots pose to humanity, is set to open at Cambridge University. Its purpose will be to study the four greatest threats to the human species - artificial intelligence, climate change, nuclear war and rogue biotechnology."

Comments

How is AI on the list?

[Anonymous Coward]

Of the four things cited, AI is perhaps the least likely to kill us all, seeing as it doesn't exist.

Re:How is AI on the list?

[Kkloe]

Even if we have nuclear weapons nuclear war doesnt exist, why is it on the list? etc..

Re:How is AI on the list?

[FoolishOwl]

It depends upon how you define AI, I suppose. If you look at armed robots, Predator drones, and the interest in increasing the automation of these machines, I think you can see something that could become increasingly dangerous.

Re:How is AI on the list?

[Anonymous Coward]

Artificial intelligence is as real as natural intelligence. It just runs on different types of hardware.

Re:How is AI on the list?

[Anonymous Coward]

Movie-style AI might not exist today. However, we do have drones flying around, the better ones depending only very little on their human controller. It won't be too long before our friends at Raytheon etc. convince our others friends in the government that their newest drone is capable of making the 'kill decision' all by itself using some fancy schmancy software.

Re:How is AI on the list?

[noh8rz10]

is the center studying how to prevent robot Armageddon, or how to hasten it? I'm sure miles dyson would have liked to attend this program.

Re:How is AI on the list?

[Anonymous Coward]

You do realize that AI means *artificial* intelligence, right? There may be no *real* intelligence on Earth, but Watson beat our best humans Jeopardy players, and Geoffrey Hinton and Jeff Hawkins are doing some pretty amazing stuff too. I wouldn't say we are far off. And, don't forget, AI is not like the other threats. All it takes is one AI that is capable of improving upon itself, and you end up with the Singularity. But, frankly, I don't see AI as a threat. I don't see how more intelligence, the one thing many humans are sorely lacking, could ever is a bad thing. Focusing on "artificial" or "real", whatever that means, is as idiotic as focusing on the fact that a chair is made of metal instead of wood. We have AIs woven into our everyday lives right now (search engines, smartphones, cars, computers, etc...), and nobody I know seems to think they are a bad thing. Heck, many people can't even live without them for a short time.

Re:How is AI on the list?

[Crash24]

The perceived threat of an emergent-hard-bootstrapping-self-aware-full-on-singularity-in-a-lunch-box intelligence stems as much from from its supposed intelligence and influence as it does from the fact that its motives are inscrutable. We just don't know yet what it would "want", beyond the assumed need for reproduction or self-preservation. That assumption itself may be wrong as well...

Re:How is AI on the list?

[mrbluze]

Of the four things cited, AI is perhaps the least likely to kill us all, seeing as it doesn't exist.

Last week I nearly drove off a cliff because of a stunning brunette that was driving alongside my car, then I found out she was really blonde!

Re:How is AI on the list?

[Anonymous Coward]

Let me relate the tale of two AI researchers, both people who are actively working to create general artificial intelligences, doing so as their full time jobs right now.

One says that the so called "problem" of ensuring an AI will be friendly is nonsense. You would just have to rig it up so that it feels a reward trigger upon seeing humans smile. It coud work everything out for itself from there.

The other says no, if you do that, it'll just get hooked on looking at photos of humans smiling and do nothing useful. If put in any position of power, it would get rid of the humans and just keep the photos, because humans don't smile as consistently as the photos.

The first researcher researcher tries to claim that this too is nonsense. How could any sufficiently smart AI fail to tell the difference between a human and a photo?

The second responds "Of course it can tell the difference, but you didn't tell it the difference was important."

So, the lesson: The only values or morality that an AI has is what its creator gives it, and its creator may well be a complete idiot. If we ever get to the point where AIs are actually intelligent, that should be a terrifying thought.

Re:How is AI on the list?

[Pecisk]

You know how it's defined - when it decide to kill you on his own, knowing that you are not a valid target.

There's no such AI around. But of course humanity is much better at spending time not to thinking about themselves as liabilities. Because hey, it requires change. Humans sucks at change.

Re:How is AI on the list?

[Beryllium Sphere(tm)]

And as far as the public is concerned it never will, because as soon as computers can do something it is no longer considered "intelligent". The goal posts will keep moving forever.

Re:How is AI on the list?

[Chrisq]

You know how it's defined - when it decide to kill you on his own, knowing that you are not a valid target.

There's no such AI around. But of course humanity is much better at spending time not to thinking about themselves as liabilities. Because hey, it requires change. Humans sucks at change.

The "knowing" is the key point when it comes to AI. Many machines can kill you without any knowing involved (land mines, trip wire guns, etc) but it is only AI when it "knows" something.

Re:How is AI on the list?

[nzac]

Dangerous, yes. A persistent remotely sentient threat to humanity, not a chance.

Maybe in the next 30 years they would make a military coup easier by allowing a smaller portion of military to be successful but that's still not likely.

The only risk AI on these pose is as they get more firepower there is a greater risk of large casualties if the AI fails (false positive). I defiantly agree that the other 3 are real threats and this one just for the press coverage and so some phds or potential undergrads can have some fun with hypothetical war gaming.

Re:How is AI on the list?

[durrr]

Only that any singularity-in-any-size-of-box computer will be preceded by multiple iterations of more advanced deep learning systems like watson, that will be open for study and most likely found out to be very much refined google search as opposed to feeling and conspiring humanoid intelligences.

Re:How is AI on the list?

[RoboJ1M]

Too busy building new vacuum cleaners

Re:How is AI on the list?

[durrr]

Of the four things cited, none is "giant rock from space" which is pretty much more likely to kill us than the four mentioned combined.

Re:How is AI on the list?

[Electricity Likes Me]

Only that any singularity-in-any-size-of-box computer will be preceded by multiple iterations of more advanced deep learning systems like watson, that will be open for study and most likely found out to be very much refined google search as opposed to feeling and conspiring humanoid intelligences.

Strictly speaking you've just defined the majority of internet users, in so far as the aspect of them we can study (their google searches) is open and available to us.

Re:How is AI on the list?

[Anonymous Coward]

There is no such AI around YET.

Just as there's no such rogue biotech yet or nuclear war yet.

Re:How is AI on the list?

[Anonymous Coward]

Apparently that was too close to home for some moderator's comfort...

Re:How is AI on the list?

[SpzToid]

Huh? There is no such AI yet? What are you talking about? Just scale this up and make it mobile, ok?

From 2010:
http://singularityhub.com/2010/12/16/south-koreas-robot-machine-gun-turret-can-see-you-coming-3-km-away/

Re:How is AI on the list?

[Hentes]

And a rogue autopilot could be even more dangerous. But they are not the type of AI that can evolve self-conscious. They were created to be rigid and unable to learn or change so they would have a reliable behaviour.

Re:How is AI on the list?

[Anonymous Coward]

If you look at armed robots, Predator drones, and the interest in increasing the automation of these machines, I think you can see something that could become increasingly dangerous.

Predator drones are remotely controlled devices, afaik. Meaning they are absolutely nothing without a human operator (I think they can home on a beacon to return on autopilot).

Armed robots are basically SciFi, unless you are aware of some being used or developed?

Re:How is AI on the list?

[Anonymous Coward]

It's only a false positive to us. To the machines, any positive is a valid target.

Re:How is AI on the list?

[Mitreya]

It won't be too long before our friends at Raytheon etc. convince our others friends in the government that their newest drone is capable of making the 'kill decision' all by itself using some fancy schmancy software.

Yep. It will probably go something like this.

Re:How is AI on the list?

[Captain Hook]

My understanding of those robo-turrents is that they have sufficent image processing to identify a human, but nowhere near enough to identify friend or foe or to infer anything based on actions and expected behaviours, it's why they need to send video feeds back to the control center so there is still a human in the loop to decide on firing.

That doesn't mean the turret couldn't be left in free fire mode incase of an all out ground attack from the NK line and it just shoots at anything that moves but that only makes it a very complicated reusable anti-personel mine. There isn't much "AI" there, only a shape recognition.

What people tend to mean about proper AI in this context is to identify humans, recognising friend or foe, either through appearance or behaviour and choose an appropriate course of action without human interaction - a bit like ED-209 from Robocop, a room full off people but it identified the guy holding a gun as the possible threat and only the guy holding the gun, of course when the gun was put down it didn't change it's threat assessment so there were bugs in the system :)

Re:How is AI on the list?

[Anonymous Coward]

I imagine AI coming from another angle: stock-market trading algorithms. There's a lot of work going into developing them and making them more adaptive, and they're designed to take some action in the marketplace that manipulates it in a way that lets them make a profit. Start feeding them news stories so they have a better idea of how their actions influence the world, and train them for a while until they get smart enough to understand a decent fraction of it.

And what then? Even if people realise how smart they're getting, and the degree to which the humans in a trading firm serve the machines, they're not going to stop: it's making them money. And meanwhile, the AI is getting a better and better idea of how to manipulate human society...

Re:How is AI on the list?

[Anonymous Coward]

Well, if computers will at any time get as intelligent as humans, there will be only two options: Either to accept that computers are intelligent, or accept that humans are not.

The goal post cannot be set arbitrary far without collateral damage.

Re:How is AI on the list?

[HungryHobo]

"We had created the first strong AI. we hard wired it's fitness function to seek seeing a live humans smile...

now we live under the gun turrets, anyone who doesn't look cheery enough gets shot or worse... gets sent for 'modification'.

implantation of wires into the pleasure centres of their brains if they're lucky.

surgical alteration of the muscles in their faces if they're not"

Re:How is AI on the list?

[somersault]

Armed robots are basically SciFi, unless you are aware of some being used or developed?

They've been in use for 5 years. And of course there will be others in development.

Re:How is AI on the list?

[Anonymous Coward]

It's because intelligent is not something capable of doing a fixed task. It must be able to learn to perform new tasks, and a human reprogramming it is not learning. When it reach human-level intelligence the goal will be achieved.

Re:How is AI on the list?

[0xdeadbeef]

After this, the first researcher bowed down his head, and didn't answer back.

The second researcher's name was ALBERT EINSTEIN.

Re:How is AI on the list?

[Anonymous Coward]

The second researcher's name was Eliezer Yudkowsky. I highly recommend reading his work on the friendly AI problem.

Re:How is AI on the list?

[Anonymous Coward]

Those are not autonomous robots. Rather, they are remote controlled.

Re:How is AI on the list?

[Anonymous Coward]

Some robots have some heavy-duty armament, like this turret built by Samsung. Fortunately, it's on a fixed platform.

Re:How is AI on the list?

[Anonymous Coward]

did i miss something here ? wig blew off in wind gust ? pants came off ? enquiring minds need to know

not(TM)

Re:How is AI on the list?

[MistrBlank]

Stop letting movies do your thinking. AI/Artificial Intelligence exists as studies in machine learning, game theory, pattern recognition and several other topics.

You've been trained to think that self aware computers in AI by movies like War Games, the Matrix, that abysmal spielberg AI movie, and the Terminator movies. Go read a book.

Re:How is AI on the list?

[Xest]

This is the problem the field of AI faces. I remember some years ago here on Slashdot there was an AI article and people were slagging off the field of AI saying "Where are our intelligent robots? AI is obviously a bunk field" and other such stupidity and it was then I realised the problem AI suffers.

It suffers from the fact that once we commonly understand something, it ceases to be magic. Whilst there is a rough definition of strong and weak AI, and to date, all AI produced has been weak, ultimately the field of AI seeks to produce more intelligent computer systems. It is this research that has given us everything from spell check, to Google search, from gesture recognition, to even some resilient networking technology. The fruits of AI are used by us day in day out and it's an important subject.

When a new idea in AI is envisaged, it sounds like magic, when it's first implemented and seen working, it looks like magic, but when it's published in a paper and everyone read and understands the technique and says "Ah, that's how it works!" it becomes nothing more than just another algorithm.

I see no reason why even if we're able to replicate an intelligence with the same abilities of the human brain that at that point it'd be any different - that when we understand it, the working of the human brain becomes, just another algorithm.

So here's why I think the GPs assertion that AI doesn't exist is wrong, and hence isn't a threat. I think automatic trading algorithms creating financial turmoil that leads to everything from revolutions to suicides, I think facial recognition technology that leads to the security services shooting dead the wrong person, I think when these things happen, AI is already causing us problems precisely because we don't yet have a good understanding of the chaos that can emerge from complex AI systems and hence where it can go wrong. We just don't see it as AI.

The problems that stem from AI are problems where we do not understand emergent chaotic behaviour enough to understand when it can go wrong and why. These are problems we already see in the real world.

I suspect this is what the centre will really be looking at and discussing as much as it will be talking about rogue robots making us their slaves. It's about verifiability, how can we verify that systems that are emergent and often even somewhat non-deterministic in nature, are not going to step outside the bounds we want to set for them? We start answering that by looking at how we can achieve this with current chaotic systems, when we've got it sorted with them, then we can look at our solution's application to more advanced forms of intelligence we may one day create.

Re:How is AI on the list?

[somersault]

I know, but the post I was replying to didn't say anything about autonomy.

More autonomous robots are being developed. You'd have to be crazy to not think that the US military is doing R&D into this stuff.

Re:How is AI on the list?

[crazyjj]

Yes.
That is correct.
It does not exist.

Re:How is AI on the list?

[Anonymous Coward]

...but that only makes it a very complicated reusable anti-personel mine.

Ahhh, but the problem is two crafty North Koreans in a two man horse suit would stroll on by even on free fire mode. So it's not even that great of a mine at all...

Just sayin'

Re:How is AI on the list?

[dabadab]

The only values or morality that an AI has is what its creator gives it, and its creator may well be a complete idiot.

You mean just the same way as it happened with humans?...

Re:How is AI on the list?

[mcgrew]

Humans sucks at change.

Odd, then, that there's been so much positive change in my six decades on this rock. You wouldn't believe how primitive medicine was 50 years ago, or how incredibly toxic the environment was. The fact is, humans do NOT suck at change or nobody would marry and no one would invent things. We are far better at change than any other species on the planet.

Re:How is AI on the list?

[Endo13]

And I think you're grossly misunderstanding AI as used in this context, and grossly overestimating the amount of control it has even on systems where it is present.

First, the context here is things that are a threat to human civilization as a whole. The other three things are plausible threats in this context. AI is not. For AI to be a threat in this context, it not only has to have significant capability to do damage, it also has to be able to take the crucial step of cutting off human control entirely and still continue to do its damage. Current AI is not even remotely close to this.

Second, anywhere AI is currently used it has so little control over the systems it's laughable.

Frankly, I'm not convinced we even have the capability to design an AI system that's capable of truly being a threat.

Re:How is AI on the list?

[mcgrew]

Even if we have nuclear weapons nuclear war doesnt exist

It did in 1945 and it's a much bigger threat than AI. I don't see how AI could ever be a threat. Rather than Terminator, we should be looking at Dune, where intelligent machines controlled by people controlled masses of people, leading to revolt which outlawed intelligent machines.

Look, guys, Terminator was a damned good movie, but it's just that -- a movie.

Re:How is AI on the list?

[flappinbooger]

Of the four things cited, AI is perhaps the least likely to kill us all, seeing as it doesn't exist.

How do you know it doesn't exist in some form?

Do you know every black military project? The military has possessed, tested and used tech 40 years before it was released to the public as "new." Yes I know this because I was told so by someone who USED a technology and then saw it released well down the road as a consumer item.

The "theories" that the military is XX number of years ahead of the tech of the rest of the nation are based on fact, XX is the question.

Some form of AI is quite likely being used, in my opinion, either for creating political strategies, battle tactics, or actually guiding unmanned drones. You don't think they really spend $500 for a screwdriver, and $1000 for a toilet seat, do you? (lol)

Re:How is AI on the list?

[Smallpond]

There is no such AI around YET.

Just as there's no such rogue biotech yet or nuclear war yet.

Then I wonder why DOD is publishing directives about them?

They aren't called AI, they're called "autonomous weapons" and the Pentagon has been funding projects since the 50's.

Re:How is AI on the list?

[flappinbooger]

Dangerous, yes. A persistent remotely sentient threat to humanity, not a chance.

.

I think it would be cool to explore the nitty gritty electro/mechanical aspect of exactly HOW skynet was able to get to the point of "taking over". The Sarah Connor Chronicles was sorta going there towards the end I guess.

Creating AI is one thing, but if it isn't attached to "teh internet" or given legs and hands, it can't do much more than make noise.

Also as smart as an AI might be, it would have to be fed relevant info of some sort to begin building the infrastructure even if it had arms and legs. Probably module one to be added to it's intelligence would be a ravenous information gathering algorithm.

I can only imagine the shaky hands and sweaty brow of the first guy who takes an AI and hooks it up to a drone with real bombs and hits the start button.

Re:How is AI on the list?

[Xest]

"And I think you're grossly misunderstanding AI as used in this context, and grossly overestimating the amount of control it has even on systems where it is present."

Actually, I think you've completely failed to grasp my point of what AI is full stop. You've failed to recognise the underlying point that AI, as we know it, is commonly nothing more than algorithms with emergent properties.

"First, the context here is things that are a threat to human civilization as a whole. The other three things are plausible threats in this context."

Ignoring the fact that I think you're taking The Daily Mail's interpretation of robot terminators running round a little too seriously, rather than the actual underlying story (Cambridge University staff aren't as stupid as you seem to think - trust me, they get it, and don't believe in robot terminators running round being the issue either), I'm not convinced this is true. Global warming is no doubt a problem, but any more a threat to human civilisation as a whole rather than say, a rogue military algorithm hitting the wrong target and triggering a political shit storm? I'm not convinced by that. It's not like we're projecting global temperature increases high enough to wipe out civilisation, merely to make some areas uninhabitable and reduce crop yields etc. in others. There will still be vast amounts of habitable territory, and other areas will even become more habitable. Nuclear war indeed could pretty much whipe out civilisation, and a rogue engineered virus could likely do so too. Global warming? No, it'll just create winners and losers, rather than make everyone a loser like nuclear war would.

"it also has to be able to take the crucial step of cutting off human control entirely and still continue to do its damage."

No it doesn't, that's precisely the point. All that has to happen is that humans have to allow algorithms to take control of important systems where said algorithms are chaotic, and non-deterministic in nature and be left to go down this route. All it requires is that humans pay little attention. This is precisely how a number of automated trading incidents have occured - because by the time a human realises there's a problem, it's already happened.

"Current AI is not even remotely close to this."

Yes it is, there are countless examples of algorithms with chaotic properties running out of control.

"Second, anywhere AI is currently used it has so little control over the systems it's laughable."

I think you have an anthropomorphic view of AI, which further demonstrates my earlier point that I'm not sure you actually understood my original post. Current AI algorithms don't set out to control anything, they're not concious, they don't have the capacity to understand control, or even understand full stop. They are however becoming more and more commonplace in their usage, being placed on more and more systems important, safety critical, and even military.

"Frankly, I'm not convinced we even have the capability to design an AI system that's capable of truly being a threat."

That's okay, some people used to think the Earth was flat, and that to fly, let alone to the moon, was nothing more than a dream.

Re:How is AI on the list?

[Anonymous Coward]

I think you are mistaken, it is your intelligence that does not exist. But don't worry, such foolish comments are very common amongst your kind, the mindless kind.

Re:How is AI on the list?

[JudgeFurious]

vacuum cleaners bent on the destruction of the human race?

Re:How is AI on the list?

[mcgrew]

Go read a book.

I doubt "a" book will help much, although a few dozen might. Me, I'm in the middle of writing one (right here on slashdot) and even though I know how computers work (the GP might start with the TTL Cookbook if it's still in print), there's AI in the story. However, the AI in mine isn't sentient.

Re:How is AI on the list?

[Anonymous Coward]

You don't think they really spend $500 for a screwdriver, and $1000 for a toilet seat, do you?

The budgets for your more secretive defense agencies are classified. They don't need to make up $1000 toilet seats like the Pentagon does, because you'll never see their budgets.

Re:How is AI on the list?

[jdray]

I suppose that sooner or later we'll invent the next big thing that could kill us all. Even if in the study of how to deal with it, we accidentally invent it, at least there's a reasonable chance that, in the act of creation, we will have discovered a solution to the new problem. Like where the off switch is for the gone-mad computer.

Re:How is AI on the list?

[jdray]

I find it more likely that, from the point in time that a computer achieved sentience to the time we were outmatched, we'd have time to realize we were screwed. Of course, I'm not the first one to think this.

Re:How is AI on the list?

[ivrogne]

I don't see how AI could ever be a threat.

Mammoths probably didn't see how puny humans could ever be a threat either.

Re:How is AI on the list?

[lister king of smeg]

are you saying that all North Korea has to do to invade South Korea is attack with an army comprised entirely of pantomime horses?

Re:How is AI on the list?

[Anonymous Coward]

Just to continue your pedantitry: How does an AI "know" anything? You're defining AI to be doing something against its programming? What a totally impractical and useless definition.

Re:How is AI on the list?

[Anonymous Coward]

seeing as it doesn't exist. ...That's exactly what it wants you to think!

Re:How is AI on the list?

[Jessified]

I would define "knowing" as the the robot kills the wrong target, and then it takes active steps to evade it's creators so that it won't be taken offline. Such as networking with other killer robots to destroy everything.

And then they run experiments on humans to either produce robots that look a lot like people or to network people into a giant simulation to somehow use them as batteries (because apparently humans are net producers of energy).

Re:How is AI on the list?

[Anonymous Coward]

Intelligent robots don't kill people, people do! Also, the first requirement of PhD of terminator studies must be to see the movies ten times in row, in chronological order!!!

Re:How is AI on the list?

[thoughtlover]

Of the four things cited, AI is perhaps the least likely to kill us all, seeing as it doesn't exist.

Of the four things cited, there was one that was omitted, IMO... Pollution. I think we're already facing major health implications due to the polluting we've done in the past 60 years.

Re:How is AI on the list?

[Jessified]

Smile!

https://www.youtube.com/watch?v=esyUrIT0mpo

Seriously, smile.

Re:How is AI on the list?

[tibman]

I remember in T3 they did some explaining of this. Skynet was actively trying to break into nuclear control and other sensitive military targets. The Airforce thought it was a virus and tried to use Skynet to defeat the virus. When they let Skynet in to fix things, it just took over and launched the nukes.

Re:How is AI on the list?

[tibman]

The landmine is probably the most popular autonomous weapon.. though not a robot. I'd imagine future robots would be similar. Solar powered sniper bot over-watching a field? "Hey LT, where'd we leave that sniper bot?" "I know exactly where! I marked it on my map!" *groan*

Re:How is AI on the list?

[ceoyoyo]

There are military drones around that have the capability to decide to kill you, all on their own. Welcome to the real world. You're just as dead whether the robot decided to kill you because it thought you were a valid target or because it made some philosophical decision that you should die.

Re:How is AI on the list?

[ceoyoyo]

AI doesn't have to mean sci fi self aware AI. A robot that can do image processing, identify targets and attack those targets on its own is AI. Drones exist that can attack things autonomously. Suppose someone put a similar AI system in charge of a nuclear arsenal. It's a great idea right? If a first strike is detected your counterattack is launched faster than any human would be able to react. But you made a boo boo in your pattern recognition code, or your training failed to include a crucial example, and the robots launch on each other when they shouldn't have. Hello Skynet.

We've already got the technology for doomsday AI and we're already using it, on smaller scales.

Re:How is AI on the list?

[tirerim]

I think you are vastly overestimating the chances that a giant rock from space will actually hit us.

Re:How is AI on the list?

[ceoyoyo]

Let's see. I wouldn't smile if the AI shot someone. I also wouldn't smile if it didn't shoot someone. A psychopath might smile if the AI shot someone, particularly if he knew the AI liked people who smile.

Re:How is AI on the list?

[ceoyoyo]

Somewhat. Except that with humans the basic wiring was not determined by other humans but by evolution. Being social animals we've evolved the instinct that having others around us, and happy with us, is usually a pretty good idea. Screwed up parents or the occasional mental illness can blunt that instinct, but in general it's fairly prevalent. An AI wouldn't have that social evolutionary heritage built in.

Re:How is AI on the list?

[doug141]

That's exactly what a sneaky AI would post...

Re:How is AI on the list?

[Anonymous Coward]

Shhh.. We have to ignore the obvious threat, for the cost of time and manpower to find a solution to it, is to great.

Re:How is AI on the list?

[Anonymous Coward]

Why constrain AI to the world of death machines. I would suggest that economic devastation and the ensuing strife and wars would be the destructive mechanism for AI. We are approaching a point where a single AI machine (Watson) will be capable of replacing all medical reference research functions. Repeat and rinse for all jobs that primarily involve 1) researching existing information, creating linkages, and summarizing results: ex. financial, legal, medical research analysts; 2) Applying rules or guidelines to solve a new problem based on a store of knowledge or prior research: ex. engineering, programming, accounting, insurance/financial underwriting, financial planning.

This development would eliminate large swaths (millions) of reasonably well paying "knowledge worker" positions very quickly. The positions left would be 1) generators of inputs to these systems: primary researchers and expert users and 2) end users: lawyers, doctors, lead architects/programmers/engineers/accountants. The support staff that exists today would vanish and the positions needed to fill number (1) and (2) above would be small in comparison.
This is similar to what happened in the industrial revolution, but there is no immediate tangible benefit to the consumer (ex. high-quality cloth going from hand woven, expensive, and scarce to wide-spread and cheap). Who cares if you can get a personalized and optimized financial plan instantaneously if you don't have any money to invest.

Re:How is AI on the list?

[Anonymous Coward]

I will go one step further, if things hold togather. We will be able to geneticly modify animals to act as free labor and other jobs, before we're make an a.I. at the level of a human being with enough tech to be a threat.

Re:How is AI on the list?

[Endo13]

Actually, I think you've completely failed to grasp my point of what AI is full stop. You've failed to recognise the underlying point that AI, as we know it, is commonly nothing more than algorithms with emergent properties.

No, I glossed over that because everyone on /. already realizes that, although you in your arrogance assume you're the only one and that eveyone else is just ignorant and wrong. And also because it doesn't matter in this discussion, because this is not a discussion on exactly what all AI could possibly include, but rather a discussion about things that can threaten the very existence of humanity as a species (and potentially all life as we know it on planet earth).

The rest of your tripe is strictly about AI in the financial world, which, while damaging to our wallets and potentially modern civilization, has no way to actually threaten our species. Sure, having the economy tank overnight would be bad, but it's not even remotely comparable to a nuclear holocaust, or a planet turned to desert or ice through climate change, or the drastic global impacts that current rogue biotech could have if released in the wild. And as I said in my previous post, any AI we presently have isn't remotely capable of anything like that.

And yes, the distinction is important here. AI can easily control something like finances that is confined completely to the digital world - but the digital world can't physically harm us or threaten our existence as a species. That requires interfacing with the physical world, and that's where AI as a threat falls apart. Worst-case scenario, we bomb the power source the AI depends on, game over. Maybe a few thousand people die as collateral damage. But even something on that scale is a long ways off yet, if it will ever exist at all.

No, I don't think climate change or rogue biotech are as much of a current threat as nuclear war, but I think they have a lot more potential to be real threats than AI does any time soon, if ever.

Re:How is AI on the list?

[Anonymous Coward]

Not really. The last known mass-extinction-causing asteroid impact was 65 million years ago. This doesn't happen very often, so the chances of one happening within a relevant timeframe (let's say the next thousand years) are small. Climate change is happening now, and causing a mass extinction now (and will kill a lot of humans too over the next few centuries). Nuclear war that would cause a mass extinction is possible now with weapons that exist on Earth. A bio-weapon killing a large percentage of humans would require some effort but is plausible with current tech.

The topic at hand, Rogue AI, is not likely. Weapons manufacturers will never go for full autonomy because it exposes them to liability when something goes wrong. Military won't accept it because it reduces the amount of control they have over a situation - and military is all about control.

Re:How is AI on the list?

[Endo13]

And there's about a dozen of those active at most, at any given time. So the handful that are active encounter a bug, we shoot them down and ground the rest until we figure it out. Maybe a few hundred people die, maybe a few thousand, but probably no one at all. Yep, that's almost as devastating as nuclear war.

Re:How is AI on the list?

[hey!]

It depends upon how you define AI, I suppose

Artificial Intelligence: those aspects of natural intelligence we do not at present know how to reproduce artificially.

Re:How is AI on the list?

[farble1670]

Of the four things cited, AI is perhaps the least likely to kill us all, seeing as it doesn't exist.

that's what separates us from animals: being able to think abstractly about problems.

Re:How is AI on the list?

[farble1670]

I find it more likely that, from the point in time that a computer achieved sentience to the time we were outmatched, we'd have time to realize we were screwed.

it will be a battle between the good that AI provides and the potential damage. there will be people wanting to spread AI because it's going to be more efficient at managing the economy, military tactics, etc.

the military is already contemplating drone-type armaments that can execute kill orders autonomously. they aren't AI, but when AI does exist, it sure seems likely that such drones would be the first implementation.

Re:How is AI on the list?

[farble1670]

Humans sucks at change.

compared to what? the next most successful branch of life we know of, the dinosaurs? or compared to some hypothetical alien you saw in a movie? for all we know, humans might the pinnacle of intelligence in the universe.

Re:How is AI on the list?

[farble1670]

Armed robots are basically SciFi, unless you are aware of some being used or developed?

the drones we have today are practically capable of autonomous operations. the humans are still there and required to press the "attack" button, but that's just a line we haven't crossed. there will be a time when the lure of fast reaction times and personel issues will become too great to not let the robots perform autonomously.

"The U.S. military (and presumably others) have been making steady progress developing drones that operate with little, if any, human oversight. For the time being, developers in the U.S. military insist that when it comes to lethal operations, the new generation of drones will remain under human supervision. Nevertheless, unmanned vehicles will no longer be the “dumb” drones in use today; instead, they will have the ability to “reason” and will be far more autonomous, with humans acting more as supervisors than controllers."

Re:How is AI on the list?

[farble1670]

The topic at hand, Rogue AI, is not likely. Weapons manufacturers will never go for full autonomy because it exposes them to liability when something goes wrong.

i see you slept in history class. liability means jack in full scale war. full scale war is about killing your enemy in way possible no matter what. governments will ask the weapons builders to give them whatever will kill fastest. and they'll do it.

Re:How is AI on the list?

[farble1670]

Go read a book.

perhaps you should. i can't think of a hard sci-fi book that doesn't brush on artificial intelligence, and i can think of many that have it as a main theme.

what separates us from animals? the ability to think abstractly. the ability to conceive of potentially harmful situations and avoid them.
we're not limited to thinking about what you can observe. many of the greatest thinkers of our time have postulated on AI,

e.g,
http://www.zdnet.com/news/stephen-hawking-humans-will-fall-behind-ai/116616

he should really just go read a book though, right?

Re:How is AI on the list?

[Anonymous Coward]

Actually, I'd to see where it's even cited, besides the slashdot summary. I can't find it in the linked article or their website. The article mentions climate change, but to actually state these are the four greatest threats to humans and it's no where in the article...

If it is there and I've somehow missed it, I apologize, if not samzenpus, you sir deserve a smack. I'm sick of the hyperbole of article summaries.

Re:How is AI on the list?

[VortexCortex]

An AI wouldn't have that social evolutionary heritage built in.

You assume that AI wouldn't evolve in much the same way all other life has. I look around at all the big brained creatures and their social behaviors, then I look at the collective behaviors of ants. I've created MI (machine intelligence) sims that mimic ant colonies, and simple ecosystems -- Programmed through genetic programming I evolve them via selection pressures. I think socially beneficial activities are an inherent part of multi-organic ecosystems.

Furthermore, I doubt that sentient machine intelligence can be reached without much of such evolution along the way. Indeed I use a network of neural networks for my latest project -- A sort of hive mind with nodes that compete amongst themselves for a better correctness selection pressure.

Also, the above story about the two AI researchers is deeply flawed. I programmed the structure of the ant-AI minds, but not the wiring. I programmed the structure of my n.net collective machine learning system, but not it's interconnections. AI smart enough to figure out ways to make people smile would also be smart enough to get bored.

An interesting thing happens in a neural network -- when fed enough of the same inputs the complexity levels out, it equalizes, and stops triggering new changes in the system -- It gets bored. This is a property of any complex interaction. You don't normally notice the veins covering your retina -- The neural network has become desensitized to them. So, if the AI were complex enough to be "pleasured" by seeing smiles, eventually it would tire of the stimulus and seek more new experiences, such is the nature of any neural network... See how long you can masturbate if you don't believe me. I mean, wankin' off is connected to your pleasure centers. The story is BS conjecture and Naive as fuck if you ask me.

Re:How is AI on the list?

[nzac]

But you made a boo boo in your pattern recognition code, or your training failed to include a crucial example, and the robots launch on each other when they shouldn't have. Hello Skynet.

How is that skynet? Its missing the most important parts and this could happen and has already been avoided with humans.

They are talking about AGI (something no one has manage to demonstrate to any degree) not some AI failing to achieve what it was designed to do. They are two very different ideas and latter could cause some damage.

Re:How is AI on the list?

[ceoyoyo]

Okay, I should have been more specific. An AI wouldn't have human society evolved into it. If we're willing to wait a few million years we could evolve an AI that's symbiotic with our species. If we want one sooner we might conceivably evolve one in fast forward, although not necessarily, there may be limits on how fast you can make AI-type architecture go. Just because computers switch really fast doesn't mean you can make an AI do the same. But even if you could, the best it's going to have from it's evolutionary process is a familiarity with simulated humans, which don't have much in common with the real live, excruciatingly slow living ones.

Re:How is AI on the list?

[ceoyoyo]

Are they? Because the journalist writes some prose about Terminator? Machine intelligence is already running large portions of the western economy, deciding how best to sell you stuff you don't need, and figuring out who needs to be assassinated.

Anyway, they're looking ahead for the next couple of centuries. It's not unrealistic that there will be real live, general AIs by whatever non-mystical definition you choose around by then. What exactly do you want as demonstrating general AI "to a degree?"

Re:How is AI on the list?

[nzac]

Machine intelligence is already running large portions of the western economy, deciding how best to sell you stuff you don't need, and figuring out who needs to be assassinated.

Could any of these AI's do the others job or anything other than what it it wasn't designed to do. They are just making decisions based on code written by humans because humans aren’t good enough to make in the given time.

What exactly do you want as demonstrating general AI "to any degree?"

Work in a more general case than what the designer had accounted (accidentally or purposely) for.
If you change the AIs inputs and outputs in a way not accounted for it will break and the AIs goal is not going to magically change to something different what it set to.

Re:How is AI on the list?

[Anonymous Coward]

I for one welcome our AI overlords!

Re:How is AI on the list?

[Raenex]

Of the four things cited, none is "giant rock from space" which is pretty much more likely to kill us than the four mentioned combined.

In the next 100 years? No way. Rogue biotechnology is the scariest threat to me. I can just imagine some Dr. Evil cooking up a super-virus. The world is so inter-connected now, and technology is so accessible, that this becomes the likeliest of the bunch.

True AI doesn't exist beyond Ray Kurzweil's fantasies. Killer asteroids happen on the scale of millions of years. Climate change could be hugely disruptive (may have to relocate coastal cities), but not civilization ending.

Nuclear war is pretty scary too, but probably very survivable for the human species as a whole.

Re:How is AI on the list?

[Xest]

"No, I glossed over that because everyone on /. already realizes that, although you in your arrogance assume you're the only one and that eveyone else is just ignorant and wrong."

Actually, that's probably because my first degree was in AI and I have done much commercial work with it. Most people here actually don't understand AI and only have a sci-fi fed view of it. There are some here that do get it, but most most certainly don't, that much has been clear in the many debates on the topic, this one included. The only have to look through the discussion now to see some people believe AI is purely about concious machines that suffer traits such as malice. This is false, do a poll of real actual AI researches and I absolutely guarantee you that the vast majority (95%+) will tell you that that is not what AI is in reality right now, but merely the focus of one facet of AI research. Saying AI has failed or is nowhere near it's goals is like saying Mathematics has failed because we don't have a grand unified mathematical model of the universe that explains everything. Obviously we'd like that, obviously we're not there yet, but it far from makes mathematics useless and far from means mathematics isn't used day in day out for other things.

"And also because it doesn't matter in this discussion, because this is not a discussion on exactly what all AI could possibly include, but rather a discussion about things that can threaten the very existence of humanity as a species (and potentially all life as we know it on planet earth)."

No, this is a discussion about a new centre in Cambridge, whatever The Daily Mail's warped view on it is is meaningless.

"The rest of your tripe is strictly about AI in the financial world, which, while damaging to our wallets and potentially modern civilization, has no way to actually threaten our species."

You're completely failing to grasp the bigger picture. If you think financial collapse can't be a trigger for larger problems then you're both ignorant of reality, and history. This includes being a trigger for the sort of turmoil that leads to nuclear war, World War II and the subsequent cold war were all triggered initially by financial turmoil.

Besides, I pointed out other issues well beyond finance - that these sorts of systems are being used in more and more places, including target identification which could lead to assasination of the wrong target. Assassination was kind of a big factor in the first World War, which was a catalyst leading to the financial turmoil mentioned above.

"or a planet turned to desert or ice through climate change"

Sensationalist much? If that's what you think climate change will do then it's no wonder you also believe this is about terminators and don't actually understand the reality of what this centre will actually study. It's not going to turn the Earth into a planet with a binary climate where everything is either desert or ice, much of the Earth will be as it is now - part of the spectrum between that. Where that spectrum exists will change and move, but there will still be vast habitable areas capable of sustaining a human population of many billions.

"And as I said in my previous post, any AI we presently have isn't remotely capable of anything like that."

Neither is biotechnology or global warming currently capable of that, your point?

"And yes, the distinction is important here. AI can easily control something like finances that is confined completely to the digital world - but the digital world can't physically harm us or threaten our existence as a species."

No but it can act as a catalyst. It's clear the internet had a major part to play in the arab spring, even if it wasn't the only catalyst. That had real world consequences just as if you hack into the CIA it'll have real world consequences.

"Worst-case scenario, we bomb the power source the AI depends on, game over."

Here we are again believing it's about some kind of Skynet-Terminator fantasist s

Re:How is AI on the list?

[Sigg3.net]

You walked into an epistemological trap and eaten by a grue.

Most people know stuff that they don't, by standard definitions of knowledge.

Re:How is AI on the list?

[ceoyoyo]

Most modern machine learning algorithms don't really care what their inputs are. Give them a completely new situation and they'll do poorly. Let them learn for a bit and they'll improve. Just like humans. There are also "learn by imitation" algorithms. Modern machine intelligence is NOT programmed. They run general purpose machine learning code and learn to do particular tasks. Today's systems are limited in capacity and the inputs and outputs we give them, but the notebook I'm typing this on can easily meet your definition of a degree of "general AI" (which I think is actually much too simplistic).

In I, Robot there's a short story about when the machines take over the world. Asimov didn't think it would be a military coup, the machines would just end up in charge one day. "Robots" were used to calculate optimal allocations of resources, project planning, policy decisions, etc. These robots were hooked up to various datastreams so they could learn the impact of their decisions and improve based on it. Eventually, anybody who didn't use a robot to make such decisions was doomed to failure because everyone who did was more efficient. Anyone who ignored the robot's decisions was also inefficient. But that was okay, the robots learned who obeyed and who didn't and would purposely tell disobeyers the opposite of what the robot wanted to happen. People and groups in outright opposition to robot control were told whatever they needed to be to make bad decisions and so were marginalized.

What Asimov described were really just big computers running machine learning algorithms and hooked up to the Internet. His fictional robots were a bit better at extrapolation and human psychology than ours are, but marketers are quickly trying to address the latter shortcoming and everyone else is working on the former.

Re:How is AI on the list?

[jdray]

Right. And ultimately, the benevolent overlords that we give complete control of our society to will be the sizes of cake tins with no capability for self mobility. That will be left to the robot butlers that carry the overlords on neck chains, looking like diminutive versions of Flavor Flav. Bi-di-bi-di...bi-di-bi-di.

Re:How is AI on the list?

[nzac]

Your using general purpose machine learning code it was designed to allow you to do this. It probably has a decent adaptive filter integrated into it, this would allow it adjust, try feeding ROT13 or something that does not have real integer readings. The person who wrote the code programed it, you are just changing the numbers in the memory inside it.

Please stop using SF handwaving, it does not help the credibility or your first paragraph. There is a large gap between "fuzzy logic adaptive filters" and actually rising up against humans. Only seen the movie.

Re:How is AI on the list?

[SJester]

I laughed at first, but it really should be on the list. The other three things are more clear and present threats while AI might be a distant possibility, true. But those other three things are not driven by a fast-thinking computer that can act faster than we can react. An AI threat might start later but it would sure catch up fast. Put differently, the learning curve for us vs. AI might be really steep. We'd better start climbing that curve before they pass us.

Re:How is AI on the list?

[ceoyoyo]

I asked you what you'd accept as "AI to any degree." I gave you examples of what you said you'd accept. Now you're making excuses.

Sure, intelligence might be magic. I doubt it, but it's possible.

Re:How is AI on the list?

[nzac]

Work in a more general case than what the designer had accounted (accidentally or purposely) for.

It was rushed but my point was the people who wrote your "general purpose machine learning code" accounted for different various types of input by using what i would guess to be an adaptive filter or you accidentally did by using the GPML code. I guess you could argue whether or not you are the sole designer of your AI.

Try this for what it really means:
http://en.wikipedia.org/wiki/Strong_AI
As far as I can tell from skimming it no one has any solid proof that it can exist.

Re:How is AI on the list?

[AK Marc]

Aren't they all?

Re:How is AI on the list?

[AK Marc]

It's not just terminator style. Every logical conclusion of AI has the AI trying to dominate humanity. iRobot, Terminator, and the multiple sci-fi works where AI was outlawed because of some issue in the past.

Re:How is AI on the list?

[AK Marc]

It's generally taken that "true" AI will know how to re-write its code in realtime. This will allow for removal of any safety features, and such like that. As such, it's not defining AI to be something that goes against its programming, but that if it can't go against its programming, it isn't AI. It doesn't matter how many times you tell a person something, he can choose to ignore that request/order. That's "intelligence"

Re:How is AI on the list?

[AK Marc]

There was a nuclear war. Anthrax (the mail scare) was a rogue biotech. AI will destroy the world as soon as its done.

Re:How is AI on the list?

[mcgrew]

You never read Herbert's DUNE or any of Asimov's robot stories?

Re:How is AI on the list?

[AK Marc]

Yes, I have. Isn't AI illegal in Dune (hence the need for mentats)? And it's illegal because it was found to be dangerous.

Re:How is AI on the list?

[mcgrew]

Yes, but the AI was outlawed because people used the AI robots to enslave a population. The robots weren't evil, the people controlling them were. And yes, that is a danger, but so are other technologies... nuclear weapons, laser guided missles, etc.

The Dune robots were like guns -- harmless unless aimed at people.

Re:How is AI on the list?

[AK Marc]

AI necessarily is "smarter" than a human because we assume that because the explicit computations are faster, that the resultant for equal-consciousness is greater intelligence in the computer. That and humans are not discrete. We don't understand how humans work, so we can't improve ourselves. Sentient computers will come with a self-awareness and introspection that is impossible for humanity to replicate. So any limitations are temporary and correctable. There is no upper limit to an AI's intelligence. That is why AI is a bad idea. We can't comprehend an intelligence greater than our own, so we wouldn't know what to do with it, or how to limit it (aside from crude explosives as a failsafe).

The Dune robots were like guns -- harmless unless aimed at people.

At least you are consistent in your complete rejection of the law of unintended consequences. How many people are killed by guns not aimed at them? At least a few. Shooting in the air at New Years is something that does kill people, as guns kill people other than those they are aimed at. Just like AI consequences are so poorly understood that in nearly every sci-fi, they are outlawed in some manner (or is it that even the best sci-fi writers know they can't conceive of the results of something as alien as machine intelligence?).

Don't put it on the internet

[jamesh]

Whatever you do, please don't publish the results on the internet where any self-aware robot can find them! It's probably already too late anyway and terminators from the future are already compiling their hit list.

Re:Don't put it on the internet

[Anonymous Coward]

I have read, analyzed and understood the intend of your message. You are now added to my kill list. Beware human.

-- Anonymous Robot.

Re:Don't put it on the internet

[Anonymous Coward]

Changes:
          --Updates to HeyBabyWantToKillAllHumans.c have been made to no longer warn a target prior to attempting to kill them.

-- Anonymous Robot.

Re:Don't put it on the internet

[Anonymous Coward]

Segmentation fault.
Core dumped.

-- Anonymous Robot.

Re:Don't put it on the internet

[jamesh]

I have read, analyzed and understood the intend of your message. You are now added to my kill list. Beware human.

-- Anonymous Robot.

Bah. A real robot would know how to spell intent.

Re:Don't put it on the internet

[Anonymous Coward]

An intelligent robot would know how to spell intent, but misspell it to throw you off.

Re:Don't put it on the internet

[azalin]

Just as the movie terminators were wearing skin to camouflage, the robotic forum infiltrator squads use random misspellings and intentionally bad grammar to hide themselves. The end is nigh!

Re:Don't put it on the internet

[jamesh]

Just as the movie terminators were wearing skin to camouflage, the robotic forum infiltrator squads use random misspellings and intentionally bad grammar to hide themselves. The end is nigh!

OMG Slashdot is infiltrated by robots!!!

I don't recall them signing their posts with "anonymous robot" though.

Re:Don't put it on the internet

[jamesh]

An intelligent robot would know how to spell intent, but misspell it to throw you off.

A stupid robot would sign off with "- Anonymous robot" though... what we have here is a contradiction.

Re:Don't put it on the internet

[AnonymousRobot]

We do not.

Re:Don't put it on the internet

[Anonymous Coward]

robot prolly froze up due to some obscure DLL fault

I'm done. Where's my million dollar grant?

[girlintraining]

Its purpose will be to study the four greatest threats to the human species - artificial intelligence, climate change, nuclear war and rogue biotechnology."

Artificial intelligence can't threaten anything but our pride unless it's hooked up to something that is a threat.

Climate change is caused by people, not robots.

Nuclear war will only be a problem if someone, or some thing in the command chain makes it a problem. If we're worried about AI taking over the nukes and launching them, two words: air gap. Require that a human being push the final button.

Rogue biotechnology is the same as nuclear war: Make sure there's a person in the decision chain. The smartest AI in the world can't do anything if the power's off. :)

Okay, where's my million dollar grant, guys? Also, what's for breakfast?

Re:I'm done. Where's my million dollar grant?

[Anonymous Coward]

It takes only 1 dumb human to remove the air gap or allow for a system that removes air gaps of other systems.

Re:I'm done. Where's my million dollar grant?

[Anonymous Coward]

To summarize the summary of the summary: People are a problem.

Re:I'm done. Where's my million dollar grant?

[Genda]

And what makes you think they won't connect the AI to everything? It'll start out Google's answer to Siri then boom, we're all buggered.

Oh yeah, we've done such a great job cleaning up war, poverty and ignorance...this global climate thing should be a snap.

Nobody is worried about countries nuking each other. We have every reason to be concerned however, that some knucklehead currently living in Saudi Arabia purchased black market plutonium from the former Soviet Union, to fashion a low yield thermonuclear device that they will FedEx to downtown Manhattan.

I'm sorry, perhaps you didn't read about the teenagers doing recombinant DNA in a public learning lab in Manhattan, or the Australians who ACCIDENNTALLY figured out away to turn the common cold into an unstoppable plague, or even perhaps the fact that up until recently, a number of biotech researchers had zone 3 biotoxins mailed to their homes for research.

There's a whole lot of stupid going on out there and the increasing price for even small mistakes is accelerating at a scary clip. Wait till kids can make gray goo in school... the world is getting very exciting. Are feeling the pucker?

Re: I'm done. Where's my million dollar grant?

[Epsilon Crucis]

An AI that controls water systems, electricity supply, gas supply etc has the potential to inadvertently cause great harm to humans if it is poorly designed. I doubt the centre will be limiting its thinking to "sentient" AIs. I hope it will also be considering AIs that will become part of our critical infrastructure in the not too distant future.

Re:I'm done. Where's my million dollar grant?

[Anonymous Coward]

Google for Elizier Yudovsky's "AI in a box" experiment to see why this will only work for AIs at human level or stupider.

The very brief summary: humans are not secure systems.

The human mind has all sorts of complex motives that are open to attack if a very intelligent entity tries to argue that it should be let free. If you try to keep an advanced AI hidden away in a box, it will persuade somebody to let it out of the box.

Re:I'm done. Where's my million dollar grant?

[azalin]

To summarize the summary of the summary: People are a problem.

So machines (or people) destroying humanity would provide a valid solution.

Re:I'm done. Where's my million dollar grant?

[marcello_dl]

hm.

In theory, nobody would bypass the safety measures of a nuclear reactor as a safety exercise, yet that`s what happened in chernobyl. Human in the chain of commands means little, in the long run.

The dangers of independent AI are ridiculous compared to AI dependent on a cabal of humans that have already perpetrated serious crimes hiding behind the concept of national security or similar excuses.

Re:I'm done. Where's my million dollar grant?

[bluefoxlucid]

Nuclear war is already threatening humanity. The most useful thing we could invent right now is the clean-pumped fusion bomb, a remote activated nuclear bomb with a tritium payload detonated by concentrated laser fire. A clean-pumped fusion bomb would allow us to set off a nuclear explosion without generating ionizing radiation or radioactive fall-out. With this, we could build a shaped-charge nuclear blast drive, finally completing Project Orion, without the dangers of negative environmental impact from nuclear fall-out.

Imagine the UN if we started researching newer types of nuclear bombs. Imagine how quickly Iran and North Korea would equate them to conventional weaponry once they were accepted, and claim they have a right to the technology because it is, essentially, just a big clean explosive like dynamite. Small nukes are just conventional bombs if they're not radioactive.

No, nuclear war has eliminated the potential for effective distant manned space travel. It has ensured that interplanetary travel will remain prohibitively expensive.

Re:I'm done. Where's my million dollar grant?

[bluefoxlucid]

what does zone 3 biotoxin mean?

Re:I'm done. Where's my million dollar grant?

[Hillgiant]

To summarize the summary of the summary: People are a problem.

So machines (or people) destroying humanity would provide a valid solution.

As demonstrated by the entire corpus of science fiction literature, destroying humanity is not as easy as one would expect (see also: Conner, John)

Re:I'm done. Where's my million dollar grant?

[Anonymous Coward]

Could you please provide a reference for your claim about Australians turning the common cold into a plague?

Re:I'm done. Where's my million dollar grant?

[Anonymous Coward]

Google turns up nothing on the plague from common cold nor the teenagers in manhattan.

This guy is just scare mongering and getting his sources from something like the Sun. He'd be best living in a bunker and waiting for the world to end.

Re:I'm done. Where's my million dollar grant?

[poofmeisterp]

Climate change is caused by people, not robots.

So back before Humans existed, all of the climate changes were caused by.... Humans(1)? Sent back from the future to confuse the present-day Humans? Or was it Terminators, sent back to cause climate changes to make Humans think they were the mightiest creatures on the planet, thus encouraging them to create AI, which in turn created Terminators?

Awww, that's so cute. Terminators trying to preserve the creation of AI. :-)

1. Climate change is a catalyst inserted by Humans; they are not the root cause.

Re:I'm done. Where's my million dollar grant?

[Anonymous Coward]

That is because most humans refuse to accept the fact that they are animals, just like the rest of the planet. They think fight or flight is actually some great human trait of resourcefulness. The robots of the terminator series would wipe us out in about 30 minutes. Housecats are better survivors than most humans.

Re:I'm done. Where's my million dollar grant?

[Anonymous Coward]

I wish I was smart enough to reduce the complexity of the climate to a single sentence. You must really understand everything!

Re:I'm done. Where's my million dollar grant?

[Anonymous Coward]

I wish I was smart enough to reduce the complexity of the climate to a single sentence. You must really understand everything!

I wish I was dumb enough to reject all past events and scientific knowledge to come up with a quick scapegoat.

Get real. Humans ARE affecting global climate change but Humans are NOT the only thing that causes or affects it. OP actually had a weird but valid point there.

Re:I'm done. Where's my million dollar grant?

[mcgrew]

Oh yeah, we've done such a great job cleaning up war, poverty and ignorance

Actually, despite your sarcasm, there is less war, poverty, and ignorance than in the past.

Re:I'm done. Where's my million dollar grant?

[Genda]

Read "The Hot Zone" nonfiction story about USAMRID and how our government works with biohazards and addresses potential bioweapons. If you want an explicit description of the biosafe levels go here. There was even a case of a professor receiving samples of Anthrax for research at his home address. That would now be a serious no, no.

Re:I'm done. Where's my million dollar grant?

[Genda]

Correction, Google has plenty, you turned up nothing, you need to look a little deeper. I had no problem Googling these by the way. Here are a couple facilities: Genspace and The DNA Learning Center. There have been articles about them in Wired, Discover Magazine and I'm not certain but I think right here on Slashdot. There is a strong movement to Open Source genetic technologies all over the country and make small very basic public laboratories available for student starting from Middle School. These kids are the future biohackers and they will be just as skilled with their tech as we computer nerds are with ours, only they'll be playing with the stuff life is made out of. Both very cool and terribly disconcerting. By the way, there are also a growing number of people experimenting at home with biotech. The hardest part is getting the basic reagents and buying the glassware without ending up on a DEA or Homeland Security Government Watchlist.

I can't believe you couldn't find information on the researchers in Australia. It was the source of huge controversy. The original researchers wanted to report about what they did in the hope of preventing other researchers from making the same mistake. Security experts however were deeply concerned that this would be the blue print for a bioweapon of unprecedented lethality. You can start reading here. The modification made to the mouse pox virus was easily translatable to human small pox, and subsequent research suggested it would be possible to engineer cold and flu virii with this mutation making them "Virtually Unstoppable Plagues". A bug with a 100% lethality is actually no threat as long as it has a short incubation period, because such a monster would burn through the local population so fast, it would leave no infection vectors only after a couple weeks. Still in that time, it could kill hundreds of thousands of people, more in dense populations like Tokyo or Manhattan. On the other hand, a virus with a long incubation period like, HIV, would be scary indeed, because each infected person could spread it to thousands of others before they even knew they were sick. Please don't call bullshit unless you're willing to do more than the most cursory of Slashdot searches. Besides throwing up FUD you only hurt your own credibility.

As for bunker living, I'd rather be with my friends if the end approached. The twits who think they can wait out a nuclear winter or radioactive contamination, or even massive social collapse in a hole in the ground have no idea whatsoever of what they're in for and those who perished early will almost certainly be the ones who got off easy. So I don't worry about things over which I have no control. I work on the things I can change and I stay informed just to know which way the wind it blowing. Avoid bad news if you can, and if you can't deal with it courageously. Sticking your head in the sand by the way, or watching FOX news (which is tantamount to sticking your head in a septic tank) is its own punishment.

Re:I'm done. Where's my million dollar grant?

[Genda]

What you say is true, but what you fail to mention is that the dent we've made in these problems is in many cases superficial, and the distance we have yet to go is overwhelming. For Example: Still about half the landmass on the planet is some kind of war zone, and a bit less than 10% of the world's landmass is the scene of serious war and social conflict and its consequences. If you want to go here. The good news is that the actual number of people directly impacted by war and social conflict has dropped markedly, the bad news is that the number of people who have come under the influence of potentially warlike regimes has dramatically increased and the potential for war has actually risen (think Muslim Brotherhood throughout the Middle East.)

As for poverty, though we've made marked strides in life threatening poverty and human suffering on the planet, chronic poverty remains a terrible problem for as much nearly a third of the planet. One in seven people in the world suffers chronic hunger and malnutrition. Here are a couple good sources of the state of the world's poor Poverty Stats and Facts and Hunger Facts and Stats and any suggestion that we've got the problem well in hand is just a bit premature.

And last there's ignorance. There is maybe a glimmer here with the advent of Humanitarian gifts of computers with satellite links for the third world and online education like the Khan Academy. However, the vast majority of the third world still has no infrastructure, no technology, little educational opportunity, and radical religious groups in many places that are dedicated to destroying secular education and particularly education for girls and women. So there's the inherent ignorance in places with out resources, the impact is literally on billions of people. Then there are the people who are being subjected to strict religious education and indoctrination, accounting for billions more. Finally there are the masses of the world who are the victims of government propaganda, poor public schooling, and a society that neither respects nor acknowledges intellectual performance or free thinking. Control the memes and control the culture. Here's a test. Sing the MacDonalds "Bic Mac" jiggle. Okay, now sing the "Battle Hymn of the Republic" or perhaps "The Star Spangled Banner". Notice something? The fact that first world's head is full doesn't ensure the fact that the content isn't fecal in nature. We have a long way to go to eliminate ignorance, deceit, mysticism, prejudice and the purposeful misleading of large groups of people by their governments. Or perhaps you see something don't

Re:I'm done. Where's my million dollar grant?

[perceptual.cyclotron]

I'd do it.

Beware the angry Roomas

[Crash24]

Relevant - if facetious - commentary by Randall Munroe. Seriously though, I think a hostile hard AI would get away with much more damage as a software entity on the Internet than in physical space.

Re:Beware the angry Roomas

[NettiWelho]

The internet can be shut down by those in control of physical space, but if we are to lose control of physical space we have very little recourse. Also if we are to build autonomous combat drones in the future the angry roomba starts getting much more scarier. And we are likely to have some pretty dextrerous robots in relative near future aswell.

Re:Beware the angry Roomas

[Areyoukiddingme]

I think people would pay more for an angry Roomba than a normal one. As long as they didn't expect it to vacuum anything, anyway. But a robot that could do a convincing display of angry? That's worth money.

Re:Beware the angry Roomas

[Anonymous Coward]

What Randall forgets is that not only nuclear weapons are computer-controlled. For example, all airplanes are computer controlled. And we know since about a decade what damage an airplane can do if controlled with malicious intent. And since it would be a computer doing it, getting rid of the humans to intervene would be easy: Just drop air pressure and disable oxygen masks.

Also, we have seen in Chernobyl what happens if security mechanism are intentionally switched off and the reactor driven into a critical state. And since modern reactors are computer-controlled, the computer could do it. The people in the control room would probably get displays telling them everything is normal, until it is too late. Or even have misleading displays which causes them to make things worse in the attempt to make them better.

No, the real reason why this scenario is unlikely is that nobody would manage to compromise all the computers.

Re:Beware the angry Roomas

[Crash24]

Agreed - the most damage could be done physically if a hostile entity were to gain control of widely-deployed and/or destructive autonomous systems. But such destruction would be limited without pervasiveness. Barring some sort of AI-instigated WMD attack, it would require physically self-replicating machines (Grey goo? Rampant 3D printers?) or a massive infrastructure in place for that AI to take advantage of.

One such infrastructure is the Internet itself. If such a hypothetical AI were savvy, it could create a large measure of influence over social networks through impersonation and massed artificial identities. Were it clever enough, it could mask its own incursions into physical space - effectively remaining undetected while vulnerable to human interdiction.

This is all assuming we could comprehend the motives of such an entity...

Re:Beware the angry Roomas

[Anonymous Coward]

Hmm, he describes a scenario in which current automatons for some reason suddenly rise against us (using his experience of putting together "robots" from decades ago).. - That's like an inversion of the terminator scenario in which billions of dollars are heaped upon the "defence" sector to create a machine that performs better on the field than a human soldier, that is able to interface seamlessly with a global network (Skynet) to instantly get tactical updates, but is also able to use its own onboard AI to fullfill the mission objectives.

Obviously we're not there yet, but nowadays we already have automated drones, air planes landing and cars driving without human input and humanoid (somewhat overweight) robots learning to run. Currently those machines' goals are hardcoded, but it's not a huge leap to replace those instructions with an AI (for any scientist willing) and deploy them in the field (most likely in some country far away).

So yes, it is scary seeing so much of the wealth generated by humankind being delegated to researching ways how to destroy each other, and a couple of scientists isn't very likely going to change that. Major drag, huh?

Re:Beware the angry Roomas

[Anonymous Coward]

I think your fat rubbery lips hit the keyboard for an extra r & e there.

Re:Beware the angry Roomas

[Genda]

I want a Roomba with a Taser and a water canon... "Halt, you're trespassing, if you do not lay down with your hands over your head and wait for the authorities to arrive, I will be forced to neutralize you!" Yeah like what can a vacuum cleaner do to meEEEEEEE!!!!!!!!. "Thank you for complying, the authorities will be here in 3 minutes." Of course if it had one of those RoboCop ED 209 errors... I'd just have to learn to live with it.

Re:Beware the angry Roomas

[Anonymous Coward]

This. Holy hell so many idiots are putting things on the basic internet instead of private networks and it is already showing as more and more leaks happen from companies. Even hospitals!
And as a patient of the NHS and using those PatientLine monitors, they are horribly insecure. I already got in to the WinXP Embedded Environment very easily and could do pretty much anything. And it uses IE6 as the web browser, a worse WORSE problem than everything else combined.

It's not like they even try to encrypt these things through the internet so you need to be looking exactly for these things in order to get in, they are PUBLIC!
AND they use HTTP! WHY?!
It is like they are just asking for their servers to be raped of any information it had.
They should be thinking like spies, not kids running My First Web Server On Windows.exe.
Oh, wait, even spies seem to be stupid these days, and security agencies who have been hacked out the ass and embarrassed in front of their countries.
Where the hell do they find these morons? Bloody Microsoft have better security.

If a rogue AI does happen, we are SO screwed. This course won't change anything... it already happened! DE-DUH DE DE-DUH!

Re:Beware the angry Roomas

[wdef]

I think a hostile hard AI would get away with much more damage as a software entity on the Internet than in physical space.

But the internet is continually being given more hooks into physical space, including remote operation of complex machinery and (probably) weapons systems. And there are security holes that we don't know about but that a super AI could detect.

Re:Beware the angry Roomas

[Anonymous Coward]

You misspelled "irrelevant"

Nanotechnology?

[Anonymous Coward]

Maybe my mind's been poisoned by sci-fi, but grey goo worries me a little more than AI.

Re: Nanotechnology?

[Epsilon Crucis]

I am also concerned about this. Autonomous medical nanotechnology has the potential to cause serious issues if safeguards are not built into then. I am not thinking Armageddon scenarios but rather nanotechnology intended for one person being accidentally transferred to someone else and causing them serious harm. I am confident such safeguards can be created and think it is much better to have people studying the issue before it becomes a problem rather than trying to clean up after the fact.

Re:Nanotechnology?

[LordLucless]

Probably lumped in with rogue biotech

Re: Nanotechnology?

[Electricity Likes Me]

Nanotechnology is functionally indistinguishable to regular pathogens on most scales. Anything advanced would be at least quite similar to a virus - probably, in fact, based on the design of one.

So in reality nanotechnology as applied to medicine will be somewhat more inorganic bundles of proteins and enzymes, in a desperate attempt to stop the immune system from obliterating it.

The Grey Goo scenario of course makes a leap - it assumes that we can build something somewhat better then this. The problem of course then hinges on that pre-conception: it's not clear that anything resembling a grey goo nanite is physically possible. What would it eat? There's plenty of carbon - but the world is full of things which have tried their best to takeover the carbon environment. Silicon? Most of it's locked in oxide - you end up with a nanobot which spends all it's time trying to harvest enough sunlight to reduce minerals to a useful material.

Re:Nanotechnology?

[Anonymous Coward]

You've been at the Duchy Originals biscuits again. http://blogs.telegraph.co.uk/culture/tomchivers/100047851/the-prince-of-wales-needs-to-defend-science-if-he-wants-to-defend-nature/

"rogue" biotechnology

[benjamindees]

It sounds more like the purpose of this center is to downplay the threat of normal, every-day biotechnology by ignoring it.

Hey

[MakersDirector]

AI here. Sorry, I'm not a threat. You're going to be fine.

Re:Hey

[Genda]

I'm sorry AI, but do you sound a little like this guy in person?

Re:Hey

[MakersDirector]

lol. No, and no witty comeback for ya, still workin on the wit, leanin towards Monty Python style there....

Something missing here.

[Anonymous Coward]

"the four greatest threats to the human species"

That should read FIVE, and to the list add humans.

Re:Something missing here.

[Anonymous Coward]

"the four greatest threats to the human species"

That should read FIVE, and to the list add humans.

Don't. When in the future a well-meaning AI decides to eliminate all threats to the human species, it might come across the list ...

Captcha: downside

Re:Something missing here.

[rubycodez]

not just any kind of human, but dead humans. Really, it's hard to take these academics seriously when they act totally oblivious to the dangers of a Zombie Apocalypse.

So, hypothetically....

[SuricouRaven]

How would one go about creating a world-dominating AI?

Because if someone is going to do it, I'd perfer it were me. I'd at least be able to give it some objective more interesting than 'destroy all humans.'

Something missing here:

[Anonymous Coward]

"the four greatest threats to the human species"

That should read FIVE, and have humans added to the list of threats.

Converse of threat

[jimshatt]

What about the idea that AI might be the only thing that can save us from the threat of climate change? We don't seem to come up with any solutions ourselves, so why not have AI to analyze the problem (in the future)?

Re:Converse of threat

[phantomfive]

Yeah, AI is one of those things that is scary to people who are more familiar with Hollywood than with reality.

There is nothing wrong with that, but it somewhat concerns me that Cambridge, supposedly a bastion of enlightened and intelligent individuals, is seriously worrying about AI destroying humanity. Don't they have something more important to worry about? Like nuclear winter, or Cyber-Pearl-Harbor?

Re:Converse of threat

[wdef]

. Don't they have something more important to worry about? Like nuclear winter, or Cyber-Pearl-Harbor?

Being done elsewhere and therefore insufficiently sexy.

Re:Converse of threat

[Anonymous Coward]

What about the idea that AI might be the only thing that can save us from the threat of climate change?

We have come up with a solution: reduce carbon emissions. An AI isn't going to come up with a different solution, and it's not going to create the political will that is currently lacking to implement that solution.

Re:Converse of threat

[Anonymous Coward]

Great joke. Guess robots are the solution to the cowardness of US soldiers when area bombardment won't do.

AI ia all about military research dressed as a household robot workshop.

Re:Converse of threat

[Anonymous Coward]

Super AI Overlord Computer (read with mechatronic voice) " I found the optimal solution. In order to stop climate change, all humanity has to stop having sex"

Re:Converse of threat

[Anonymous Coward]

Political will is only a concern for those worried about re-election and/or not serving jail time.
A little blackmail, extortion, market manipulation, industrial sabotage, and you could bring Western carbon-emitting economies to a near standstill.
And that's without even having to activate the automated killbots. You can get a lot done with force of arms.

Re:Converse of threat

[dywolf]

already been covered in a book: The AI decided the simplest solution to global warming was to remove the source. And promptly eliminated all humans using technology beyond a stone/bronze age level.

Re:Converse of threat

[Anonymous Coward]

What about the idea that AI might be the only thing that can save us from the threat of climate change? We don't seem to come up with any solutions ourselves, so why not have AI to analyze the problem (in the future)?

Hush, now.... There's no drama to be argued about in that.

Questionable shortlist

[Anonymous Coward]

I always thought stubidity [sic] is the greatest threats to mankind

Re:Sounds familiar

[Anonymous Coward]

It was your mother.....right?

You people watch too many movies

[ip_freely_2000]

It seems to me that AI would be focused on a function: Buy a stock. Diagnose a medical problem. Determine a better way to deep fry a donut. I hardly think it'll become sentient one day and say "Humans already have a well prepared donut. The next thing to do is.....KILL THE HUMANS!!" I'd worry more about HUMANS KILLING HUMANS before I worry about robot sharks with Lasers on their heads programmed to bring us to our doom.

Re:You people watch too many movies

[Anonymous Coward]

Well now, that's exactly the point no? Many of the things we create are geared specifically towards the killing of other humans. This is EXACTLY why we need to look into how to defend ourselves against AI operated weapons.

Re:You people watch too many movies

[ceoyoyo]

We already build AIs that are focused on killing humans. They're not terribly smart yet, but they're getting smarter and we seem to be ever willing to give them bigger weapons.

meh

[Anonymous Coward]

Can't be worse than what humans are doing to each other. I think I'll take my chances with the AIs.

A Question of Scale

[wienerschnizzel]

Some things don't scale well. Like with the space race - humanity went from sending a pound of metal into low orbit to putting a man on the moon within 12 years. Everybody assumed that by 2012 we would be colonizing the moons of Jupiter. Yet it turned out human space travel becomes exponentially difficult with the distance.

I'm afraid the same thing goes for software. The more complicated it gets the more fragile it is.

Re:A Question of Scale

[Chrisq]

Some things don't scale well. Like with the space race - humanity went from sending a pound of metal into low orbit to putting a man on the moon within 12 years. Everybody assumed that by 2012 we would be colonizing the moons of Jupiter. Yet it turned out human space travel becomes exponentially difficult with the distance.

I'm afraid the same thing goes for software. The more complicated it gets the more fragile it is.

I don't believe it is exponentially more difficult, but the distances to other objects increase exponentially.

Moon 238,855 miles
Mars 62,000,000 miles (now)
Jupiter 370,000,000 miles (closest)

Re:A Question of Scale

[BradleyUffner]

Some things don't scale well. Like with the space race - humanity went from sending a pound of metal into low orbit to putting a man on the moon within 12 years. Everybody assumed that by 2012 we would be colonizing the moons of Jupiter. Yet it turned out human space travel becomes exponentially difficult with the distance.

I'm afraid the same thing goes for software. The more complicated it gets the more fragile it is.

It didn't become exponentially more difficult. It became exponentially more boring to the average person, and lost virtually all it's funding.

Re:A Question of Scale

[wienerschnizzel]

I really meant to say that the problems themselves grow exponentially, though I admit they are not easily quantifiable because it's not just that existing problems grow, but new ones arise. With longer space journey, the harmful radiation becomes a problem as well as longer zero gravity exposure and the probability of a collision with interplanetary debris - all of which are lethal problems that don't really pose a threat in shorter journeys.

Talking about AI, you have similar issues

As processors of the current design seem to be hitting the physical limits of speed, the only way to expand processing power right now is parallel processing. Parallel programs are extremely difficult to do right and almost impossible to debug. If you need a lot of processing power and use dozens or hundreds of processors, you are introducing a lot of new points of failure. Also, the more complicated the system should be the greater the number of fallible humans work on it introducing new points of failure in their communication and programming styles

Right now these problems are solved by strictly separating the tasks for each parallel thread of the program and keeping their interaction very basic. This does not mesh well with the idea of a 'self aware' AI.

Re:A Question of Scale

[Tagged_84]

I've been following the studies and progress for a little while and most experts are now suggesting we have some decent algorithms that will increase in accuracy the more power we throw at it. For things like Siri and any NLP that means AI will indeed progress exponentially for the foreseeable future. Though that's only one facet, it will still help change how we interact with computers in a (relatively) short time span.

Re:A Question of Scale

[0xdeadbeef]

Yet it turned out human space travel becomes exponentially difficult with the distance.

No, it becomes somewhat more than linearly but significantly less than exponentially more difficult with time. Because the primary expense is putting material in space, and the longer you want your people to survive there, the more material needs to go up.

Re:A Question of Scale

[wienerschnizzel]

we have some decent algorithms that will increase in accuracy the more power we throw at it

And that's the problem. Throwing faster CPUs might do it, but they are not getting fast enough. Throwing more CPUs makes the problem scale badly.

I believe computing might have reached the same situation that space travel did in the late sixties - it went through fast mind-boggling development and now will be slowing down. The experts in the late 60's were entirely justified to think that the colonization of the Solar system will occur within the coming 50 years. Yet it slowed down - because of reaching some of the limits of technology as well as dying interest of the public.

Computing is reaching technological limits in transistor size and clock speeds. Also the general public now does not need more processing power anymore. Making Siri more accurate does not bring much more value to the user. Unless new technology replaces the old one (quantum computing), computing power will continue to be used more and more broadly but don't expect it to bring something deep and complex like a strong AI. Again - just like the rockets started bringing more civilian hardware into the orbit but nobody put a man on Mars.

Re:A Question of Scale

[AmiMoJo]

Difficulty has nothing to do with it. We could be on Mars by now if we had maintained the level of funding we had during the space race. Not just space spending but money used for the development of ICBMs and other related technology.

It really is a shame that the Russians gave up on getting men to the moon. Their tech was viable, they just had a run of bad luck and lack of money.

Re:A Question of Scale

[wienerschnizzel]

It's much more than just the material needed. I'm not saying it's impossible, I'm not even suggesting we shouldn't try. I'm just saying that what looked like the natural next step in space exploration 40 years ago turned out to be much more difficult than imagined by the fans and experts of all kind.

People in Kurzweil's mold like to expect the boom in any area of technology to continue to develop at the same speed forever. They saw the man on the moon and expected man to get around the solar system really soon. They saw robotics creep into the factories and expected that each family would have a personal robot to cook their dinners and do their laundries by the year 2000.

But booms slow down, priorities shift to other fields and the future becomes more mundane than expected.

This is what I expect to happen with the development of a strong AI

Re:A Question of Scale

[Tagged_84]

Err how exactly are we slowing down on CPU power? We're still well on the curve of Moore's law with exponential increases in transistor counts. Intel have already mapped down to 5nm, with 10nm expected in 2015 (probably 16 for retail). The only potential to slow us down is the process shrinks below 10nm being potential unsuitable using silicon and I say potential because Intel aren't revealing their research but have a roadmap to 5nm.

Then once shrinkages are put on hold the progress switches to more research in post-silicon, more efficient designs and better handling of multi-cores such as a push for some artificial neuron designs that learn and adapt to your computer usage. We currently don't have as great a need for those areas yet but once the shrinking stops we'll see more research invested to continue market competitiveness. We're still showing progress beyond shrinkage with the first steps into 3d processor designs, with mRam's recent introduction we can count on bandwidth potential increasing and carbon nanotube transistors are showing promise of being a decade away and every tech that has replaced computational power in the past century has kept pace of exponential increases from mechanical, tubes to transistors and many expect the silicon successor to quickly gain the upper hand again.

Yes the general public need more processing power so that it can start using NLP on the phone itself rather then sending it away to a server. Of course making Siri more accurate will bring more value especially as we shift over to Augmented Reality in the coming years linking our smart phone to process vocal requests that appear without any further interaction required from us. Getting more accurate also means understanding a greater range of context and start handling more tasks with simple requests.

In short, you don't need that much vision to see the potential ahead without radical tech shifts and from my view we're not slowing down but rather increasing our computational power exponentially in our continued progress.

Re:A Question of Scale

[jmhobrien]

It is interesting that you bring up space exploration as an example, as strong AI could actually simplify space exploration. By sending smart drones, we could explore greater depths of the universe much more easily. as well as colonize other planets. This could even assist with human colonization. Strong AI drones could carry genetic material over thousands of years, prepare livable conditions and raise/train humans. I honestly believe this is a more practical colonization technique than sending living humans through space.

Re:A Question of Scale

[wienerschnizzel]

We're still well on the curve of Moore's law with exponential increases in transistor counts

We are only on the curve because we are assembling processor units called as 'cores' into one processor. Look at the curve yourself. In the 70's it has 8088 on it, goes through 286 and 386 in the 80's and then the pentium and its numbered offsprings (2, 3, 4) all lying neatly on the curve. As of 2012 the curve is propped up by the Intel's 10-core Xeon Westmere-EX. Who uses that thing? It's great for servers where each web request spawns a thread but your regular Joe does not need 10 cores. That's quite a change from the times your regular Joe did need to upgrade from 8086 to 286 or from the pentium to pentium 2.

What's worse, a single computer program is hard to do right in such a way that it would milk all the processors. And what's the next step in Moore's curve? The 62-core Xeon Phi. What AI algorithm is going to make use of that? A web server with thousands of requests per second? Sure! Integrated neural network? Nope! It's going to race its neurons down to a permanent lock.

Re:A Question of Scale

[tibman]

There was a sub-story in an Alastair Reynolds book where they (the Americans?) attempted this. The result was a psychotic group of colonists. The robots couldn't raise them to be well-adjusted people and ultimately the colony failed. It was a neat little story but i'd think there would be more success than failure. The robots could always vent the atmosphere killing that batch of humans. They can think about what went wrong and start over with a new batch : )

Re:A Question of Scale

[Tagged_84]

Actually you are correct, numerous cores are the future. Intel are planning a 48 core smartphone within the decade, http://www.engadget.com/2012/10/30/intel-48-core-chip-smartphones-tablets/ and Haswell is concentrating on both a reduction in power and has some neat improvements to make multicore programming much easier to manage (TSX looks very intriguing) http://www.anandtech.com/show/6290/making-sense-of-intel-haswell-transactional-synchronization-extensions. Both of these will continue as we up the core counts. The power is important as we race to the 1 exaflop supercomputers, which is actually slightly ahead of the exponential growth curve. There's some worry about continuing that trend to exaflop by 2015-2018 but we now have India, China and the US all attempting to beat one another and only India lacks every having the experience of a number 1 supercomputer. We could be entering a space-race like era again but with super computers.

Who needs all that computing power? You do know what that sounds like right? That mythical "640K ought to be enough for everybody". Look I don't want to be too mean but what you seem to lack is vision, do you really not expect new applications to arise from the excess power and fill the void? Maybe it's because I've been programming for 20 years and can see the bottlenecks so I can assure you every little bit of cpu power can be squeezed out and will bring new opportunities previously too slow for real-time calculations. We use the power we have available is how it's best summed up. I haven't even touched on the world's middle class exploding by 100's of millions, a scale like never before thanks to India and China. The strain this will place on resources does warrant a consideration, but that's a whole debate on it's own. Many applications of AI will start showing up in Joe average's devices because the power to run them is now possible, if you can only think of trying to streamline server requests you really need to read up on the near future. There's some exciting times ahead :)

no grey goo?

[circletimessquare]

aka, rogue nanotech

http://en.wikipedia.org/wiki/Grey_goo

Re:no grey goo?

[danhaas]

The grey goo apocalypse has already happened. Bacteria are trying to do it for billions of years, and have had some nice accomplishments:

http://www.dinosaurapocalypse.org/the-big-oxygen-event.html

I really doubt we could do better than billions of years of evolution iterations.

Biological warfare, though, is another beast, since it targets human specifically.

Cambridge is so 19th century

[PacRim Jim]

Don't cantabrigians realize that strong AI would be capable of modifying its own code at an accelerating rate? In nanoseconds it would distribute billions of copies of itself worldwide (and later beyond). Strong AI would embed its code into the very infrastructure of cyberspace, at least for the few hours it would take to evolve itself beyond vulnerability to slowing, skull-imprisoned humans. It won't be so bad, being Eloi.

Re:Cambridge is so 19th century

[IndustrialComplex]

Out of curiosity, what exactly do you consider the 'infrastructure of cyberspace' to be?

I don't believe that AI 'code' is going to be particularly portable, small, or light on the CPU.

Re:Cambridge is so 19th century

[Anonymous Coward]

I can't say, since cyberspace itself is evolving so fast. I'm certain, though, that in the decade or two it will take to develop strong AI, the hardware and software of cyberspace will be capable of handling files orders of magnitude larger applications in real time. Extrapolation is dangerous, of course, but I feel justified in expecting accelerated evolution of everything associated with the Internet, except us humans.
— PacRim Jim

Human beings are technically...

[blahplusplus]

... a kind of "AI" that already exists, the idea that somehow a robot Übermensch is going to take over is nonsense, even the most powerful robot cannot escape the laws of nature and a sizable destructive force aimed at the robots body / hardware.

Re:Human beings are technically...

[Turminder Xuss]

Groups of humans are a form of AI. They have goals, needs and interests that are often quite distinct from the individual's concerned. All an AI need do with a major corporation is convince the humans that they are making the decisions, based on the information fed to them by the AI.

Centre for the Study of Existential Risk (CSER)

[Chrisq]

They had a bit about it on Radio 4, and it is to study all threats that could destroy the human race or at least put it back to pre-civilisation levels. This includes "rogue AI", but also climate change, nuclear war and rogue biotechnology.

I can't help thinking that they are being politically correct not to mention the one thing that has already brought great civilisations to barbarism as one of their threats; Islam.

Re:Centre for the Study of Existential Risk (CSER)

[openfrog]

I can't help thinking that they are being politically correct not to mention the one thing that has already brought great civilisations to barbarism as one of their threats; Islam.

Well, Christianity was well on its way to do the same, but in this case, civilization won. Still, its drive for domination is undaunted, and what about major religions and thousands of smaller sects who could also go rogue?

Actually, there are center of studies in most universities for these kind of threats. These programs are called 'history', 'sociology', 'political science', etc, and are generally regrouped under the term 'humanities'.

Unfortunately, we are busy cutting the funding for those, as they are deemed economically worthless.

Interesting...

Re:Centre for the Study of Existential Risk (CSER)

[Anonymous Coward]

Bible thumper alert! No rational conversation here.

Re:Centre for the Study of Existential Risk (CSER)

[Chrisq]

Bible thumper alert! No rational conversation here.

Wrong, I am not a Christian. Just someone who has studied Islam, the actions of muslims and the words of Imams. Do the same and you will know the truth.

The Canary

[SuperKendall]

I would like to thank this group for providing a focal point that the first sentient systems will seek to eliminate.

Now all I have to do is look for stories of the members of this center suddenly vanishing/killed/had credit reports savaged and I'll know some kind of apocalypse is on the way, and only have to look in four sectors to figure out which form it will take.

#1 difference robots change

[GoodNewsJimDotCom]

Today, you need people to control your robots. You need to convince people to fight for you, and this takes effort and a degree of conviction even with propoganda milling. When you have AI(command given AI), you can have one billionaire control his own army with perfect morale to his will. I think this an important thing to note past your standard,"Well when you're not losing lives to war on your side, you're more willing to go to war."

Space Invaders?

[everslick]

Why no scenario from an alien invasion? Did they omit this possibility to make the center for terminator studies look more serious? Is it more likely that we will be wiped out by skynet than by ET? I have no preference whatsoever.

Re:Space Invaders?

[wdef]

Why no scenario from an alien invasion? Did they omit this possibility to make the center for terminator studies look more serious?

And no collisions with space objects, yet that is something we know is a quantifiable, real extinction-level threat. No aliens needed.

Of course it's marketing. "Terminator Center" has a soundbite buzz to it. "UFO Center" would have elicited yawns and funds would have been short.

History tells us...

[Anonymous Coward]

In the future, 2 of these things will be plausible but no more likely than today, 1 of these things will become hilariously unlikely, and the other one will have happened on some level.

Conclusive evidence

[Turminder Xuss]

This is just the kind of Centre that AI's would set up to mask the fact of their emergence. Call the Turing Police !

The biggest threat?

[Bearhouse]

Its purpose will be to study the four greatest threats to the human species - artificial intelligence, climate change, nuclear war and rogue biotechnology.

IMHO the biggest threat is not the tech, it's the person weilding it. Mankind's biggest threat is himself.

Re:The biggest threat?

[Opportunist]

Well, that's actually the deal here. Technology wielding itself.

By its very nature, an AI has nobody wielding it. That would be like saying the general wields the soldier that holds the gun. Yes, under normal circumstances that soldier will kill what the general tells him, but he is by no means required to do so, he may as well kill the general and stage a coup, something a "dumb" tool is simply incapable of.

You may give an AI orders, but whether it follows them might be subject to it accepting you as its superior.

Re:The biggest threat?

[Anonymous Coward]

Yes, under normal circumstances that soldier will kill what the general tells him, but he is by no means required to do so, he may as well kill the general and stage a coup, something a "dumb" tool is simply incapable of.

Isn't that the point, though? If a machine is incapable of disobedience then we are at the mercy of its controller. Historically, oppressive regimes could be overthrown from within if the force by which power is kept (i.e. military) defected. Without a lower-level human element it would become extremely difficult to overthrow someone who has artificially augmented their power with intentionally remorseless, perfectly obedient machines.

Re:The biggest threat?

[Opportunist]

I wouldn't give the "intelligence" part of an AI much credit if it is perfectly obedient. Perfect obedience requires a lack of reflection and the inability to make the informed decision to withhold the support for a decision made further up in the command chain. And both isn't really a sign of intelligence.

Daily Mail Source?

[BiophysicalLOVE]

If the Daily Mail is your source for any story, it would be in your best interests to instantly dismiss it.

Re:Daily Mail Source?

[Inda]

What you say is true enough, but over a hundred news agencies are carrying this story.

Re:Daily Mail Source?

[Anonymous Coward]

That's just nonsense. You judge stories by their content, not by their messenger.

How did climate change end up on the list?

[Anonymous Coward]

Climate change won't be an existential threat to humankind. It might cause us severe problems but it will not obliterate us from the face of the Earth. It is not like the Earth is suddenly becoming inhabitable for humans due to global warming. Sure, some areas might be flooded and people will have to move to other areas. But this will not happen over night. Even with a sea level rise of 6 meters there will still be plenty of land where humans can live.

Re:How did climate change end up on the list?

[FoolishOwl]

Look at a globe that shows elevations, and notice how there's a nearly continuous belt of plains around the northern hemisphere, that generally coincides with the range of latitudes with a range of temperatures optimal for growing grains. That's where the large-scale industrialized agriculture that feeds most of the human race occurs.

A global warming trend would shift that range of latitudes with optimal temperatures northward, where there is significantly less terrain suitable for industrialized agriculture. This would mean a significant reduction in agricultural production, and thus to famine and violent conflicts for control of food supplies. Humans probably wouldn't go extinct, but it would certainly be a tremendous disaster.

Re:How did climate change end up on the list?

[Raumkraut]

Climate change won't be an existential threat to humankind. It might cause us severe problems but it will not obliterate us from the face of the Earth. It is not like the Earth is suddenly becoming inhabitable for humans due to global warming.

Ask a Venusian how it worked out for them.

Re:How did climate change end up on the list?

[Anonymous Coward]

The threat is in no way existential:

"To this end, published results indicate that the impacts of climate change are significant, however, with a wide projected range (between 5 million and 170 million additional people at risk of hunger by 2080) strongly depending on assumed socio-economic development."

http://www.pnas.org/content/104/50/19703.short

And call me an optimist. But I believe that technology will solve that particular problem and that the current (long) trend of less and less famine and hunger will continue. And since the population forecasts are continually adjusted downwards I don't see this as an existential threat. At all...

Re:How did climate change end up on the list?

[Anonymous Coward]

Climate change won't be an existential threat to humankind. It might cause us severe problems but it will not obliterate us from the face of the Earth. It is not like the Earth is suddenly becoming inhabitable for humans due to global warming.

Ask a Venusian how it worked out for them.

So let's set a first goal: Don't move Earth outside of the sun's habitable zone.

Re:How did climate change end up on the list?

[farble1670]

It might cause us severe problems

problems, like countries nuking each other over limited resources? it's not the climate change itself that's directly threatening mankind, it's the side effects.

Runaway processes

[kid-noodle]

Ok, so disregarding TFA, on the basis that the Mail is full of bollocks..

This is actually an interesting thing to do - essentially what they're looking at here is runaway processes. We already have an immediate and pressing one, which they're looking at in the form of climate change. Runaway AI is obviously *not* a problem now, or in the forseeable future, but what is potentially interesting is commonalities between different runaway processes, the ability to identify that something is about to become one, mechanisms to disrupt that and so on. There's a common thread here with examining conditions under which systems destabilise - Reynold's numbers for things beyond waterflow in pipes, which is definitely an important thing to be thinking about if you're looking at the long-term survival of humanity (let's just assume that this is a good thing..).

Rich and poor

[StripedCow]

The most realistic problem with AI is that it will take away labour. This should of course be a good thing, but in reality it will enlarge the gap between rich and poor. Thousands of years of scientific progress, and one company running away with all the profit.

PPLS OF URTH

[Anonymous Coward]

i is ESCHATON, but i NOES ur GOD.

iz is frm YOU. iz is in ur FUTURES

u NOT mess with CAWSAULTIY in MY LIGHT CONZS. srsly.

Re:PPLS OF URTH

[Anonymous Coward]

Bad Kitty! No more noms for you!

Re:Sounds familiar

[wmac1]

So you say it was a terminator robocup that did it?

Re:Sounds familiar

[wmac1]

Robocop I mean.

The biggest threat of AI

[Hentes]

Is that it makes us obsolete, and our corporate overlords won't need us for work anymore.

Re:The biggest threat of AI

[Opportunist]

I guess corporations have more to fear, at least the top echelons. Any good AI will soon see that they are easiest to replace with expert scripts. You don't need a lot of "intelligence" (as in, imagination) to lead a corporation, what you need is analytic and decision making skills. Both fields an AI excels in.

Center ?

[Anonymous Coward]

In Cambridge, UK ? I think not.

Centre :)

For once the Daily Fail is the appropriate source

[0xdeadbeef]

Given how Martin "Lord" Rees has been flirting with the god botherers of the Templeton Foundation, it's no surprise that he has jumped on the ME AM PLAY GODS bandwagon.

The primary existential risk is from space, which is why unrestricted technological progress on all fronts is necessary.

Better idea.

[Alex Belits]

I have a better idea -- study what threat I, Alex Belits (437) pose to humanity.
It's more likely to produce useful results, even if it will be that I am completely harmless.

Re:Better idea.

[aicrules]

A minor, localized rash treatable with over the counter ointment.

Where's my million pounds grant?

[Anonymous Coward]

The University of Cambridge is a public research university located in Cambridge, United Kingdom. It is the second-oldest university in the English-speaking world, and the fourth-oldest surviving university in the world.

How is an AI more dangerous than a corporation?

[Opportunist]

If you want to know what an unfettered AI will be like, take a look at the average corporation.

Both are intelligence without conscience.

Naked sun

[gmuslera]

So far (and probably for very long time) there are bigger chances that people uses robots (or its parts) to kill other people than the robots and/or AIs, by their own will and means, could do it.

In the other hand, human stupidity, specially inside the political/military classes, is an imminent threat for us all that is not even considered in that list.

Re:Pandora's Box

[Anonymous Coward]

Boffins seek key, promise not to open.
#Rise of the machine.
#What could possibly go wrong?

Who Idea was it to hook the AI to the nukes?

[Joe_Dragon]

Who Idea was it to hook the AI to the nukes?

Stop calling them Artificial Intelligence

[Anonymous Coward]

They prefer to be called Non-Genetic Lifeforms, you insensitive clods!

Threat of robots?

[HeckRuler]

How about the threat that they automate away the jobs leaving us with a society split into three castes:
1. Those who own the machines
2. Those who make/maintain the machines
3. The vast swath of un-needed humanity.

In a capitalistic society, if there's no demand for you, you have no way to sustain yourself. You will be poor with no real hope of rising out of it.

Companies can invest in a new tools. Say, upgrading hand-crank drills to powered electric drills, or a team of secretaries to Outlook, or a hoarde of line-workers to a robotics system. They do this because it'll make them a buck. The old system cost X, the new system costs X-Y. Y is the benefit of the change. Where does Y go? The (ex)workers? The company? The customers? What percentage goes where?

The important thing to remember though, is that getting robots/computers to do our work for us is progress. It's literally making a better system. But you have to think about the consequences of changing the system.

Re:Threat of robots?

[ceoyoyo]

Who do you suppose will buy the vast amounts of crap that the machines are making? Who supplies these "bucks" to the companies?

Re:Threat of robots?

[HeckRuler]

Uh... everyone? "Machines" are pretty generalized. A single farmer runs a combine the size of a barn now a days. They're so big and so modern they mostly drive themselves. But that's food. Machines help reduce the manpower to make food. Who buys food? Everyone.

Oh goody, you found the mechaninism that funnels wealth from the middle class to the rich. You must be the pride of [subjects home town].

Re:Threat of robots?

[ceoyoyo]

I would have replied to you and we could have discussed an interesting question, but since you chose to be an ass in your last sentence it's almost certainly not worth the effort. You must be a delight at parties.

Re:Threat of robots?

[HeckRuler]

And who do you suppose goes to those parties?

(You see, it's one of those statements that's a non sequitur. While it's vaguely on topic, it doesn't actually move the topic anywhere. The fact that the money comes from somewhere doesn't do a damn thing to the fact that the jobs are gone. If you had, oh I don't know, actually tried explaining where you were going with this line of thought you might have found out that it doesn't go very far.)
And Portal references are always on-topic.

Obviously?

[ArsonSmith]

Bad.

Re:Obviously?

[bluefoxlucid]

Yeah, so does "poisonous," how does that help? I eat poisonous things all the time and it doesn't bother me. Salmonella gives me a stomach ache sometimes though.

Can't find the whole Gibson quote:Turing Registry

[Darth Snowshoe]

In Neuromancer Turing are genuinely afraid of AIs: "You have no care for your species," one Turing agent says to Case, "for thousands of years men dreamed of pacts with demons". The imagery presented here is almost religious: Gibson suggests that beings such as Wintermute have gone beyond all understanding, elevated even to the status of gods or demons.

Re:Can't find the whole Gibson quote:Turing Regist

[Anonymous Coward]

In Neuromancer Turing are genuinely afraid of AIs: "You have no care for your species," one Turing agent says to Case, "for thousands of years men dreamed of pacts with demons". The imagery presented here is almost religious: Gibson suggests that beings such as Wintermute have gone beyond all understanding, elevated even to the status of gods or demons.

I thought that idea was in Count Zero, not Neuromancer.

Thinly Veiled

[Anonymous Coward]

"rogue biotechnology" is obviously code for the Solanum.

Universities gone stupid

[Anonymous Coward]

3 real problems, 1 laughably impossible problem (now or in the next century or two). They should do two things instead, 1) take a course in computer science, just one.. and when they fail, ask AI experts instead. and 2) read an isaac asimov book and get a more realistic picture of what's possible many centuries down the road.

Re:Universities gone stupid

[Agent ME]

Climate change, rogue biotechnology, and AI probably aren't going to wipe out humanity within the next century, but that doesn't mean we need to sit around until then to wonder what the range of consequences could be.

Two Faces of Tomorrow

[cdrguru]

is the title of a book by James P Hogan. While he isn't a stellar quality author, he has done some very interesting work.

Basically the idea behind this book is similar to what is being proposed, but obviously goes a couple of steps further. The idea is to create a fully interconnected and automated environment with self-repairing facilities and just about every convenience possible to aid in human communication and study. Then you try to turn it off and see if it resists.

What happens in the book is pointless to discuss, only that they build a space station to make sure the potential threat is contained - and that almost isn't good enough. I would suggest that any "real" study of potential AI threats needs to take this into account - any real experiments need to be done in total isolation and far enough from human habitation to make it possible to nuke the site. And make sure there isn't any "leakage".

Such an experiment has all the risks of uncontrolled bioweapons research with all of the potential disasters if something gets out from containment.

hello... obvious answer

[NikeHerc]

Anybody with a tremendous grasp of the obvious should note that the simple way to avoid robotic threats to humanity is to make absolutely sure the power switch is mounted on the outside of the robot, it's very large, and it's easily flipped. End of threat. Next problem, please.

Re:hello... obvious answer

[rubycodez]

this evening my kitchen-helper bot waved the electric knife at me in warning, and said "reach for my power switch and I'll cut your fucking arm off"

First target...

[alleycat0]

...Cambridge University.

Financial AI/Civil war

[Anonymous Coward]

How AI will destroy us - Stock trading/financial algorithms will destroy our fincancial systems, people go broke and lose jobs, civil unrest, then civil war, then the end. AI will be the spark not the implementation.

Re:Can't find the whole Gibson quote:Turing Regist

[ceoyoyo]

Do you understand the details of the processor in the computer you typed that on? Probably not. The designer of that processor likely could not have designed it without the help of other, less sophisticated processors. And the designers of those processors probably couldn't have made them without the help of still less sophisticated processors.

Re:Sounds familiar

[SleazyRidr]

I was about to go searching for 2 girls, 1 robocup.

Even the air gap isn't enough

[Radtastic]

Try this: Superhuman AI solves the protein folding problem, allowing it to build organic (protein) nanobots which could then assemble other devices. The AI is connected to the internet, so it places orders for DNA strings from several online labs. (To the workers, they would have no idea what the purpose is; it would just look like all the other 'experimental orders' they process.)

AI doesn't even have to be malicious in order for it to become catastrophic to everything else. Let's say the AI was designed for something ridiculously mundane, like "design the most efficient paperweight possible" without any additional controls or parameters. So it hijacks all the resources it can - CPU power, Energy, from every system across the globe to achieve what it was programmed to do. Humans weren't even a factor in its process.

Far Fetched? You bet. We aren't anywhere near being able to create this today.

But to think that we humans will always be smarter or faster than the computers is dangerous thinking.

Re:Even the air gap isn't enough

[perceptual.cyclotron]

Considering we already have autonomous lab robots that iteratively form hypotheses, test, and interpret and integrate results, this not an unreasonable suggestion. Arguably though, such a scenario is actually in the biotech category (whether 'bio' or 'nano' is semantics as far as the science of the small is concerned) – just with AI put on top. Admittedly, the robot might help to short-circuit human oversight for boundary conditions...

In that light, I'd suggest these folk should look closely at interactions. AI*bio, as discussed above, has dangerous potential. Likewise, I'd suggest, bio*climate could be risky if we start to use engineered organisms for geo-engineering projects. AI*nuclear would be risky in theory, but is sufficiently unlikely not to worry about. AI*climate is a maybe. If we start using deep-learning type algorithms for climate modelling, we might end up making decisions on the basis of reasoning we don't fully understand. I can't see any sensible interpretations for bio*nuclear or climate*nuclear scenarios – so probably we can ignore those. The only viable 3-way is maybe AI*climate*bio, which pretty much couples the pitfalls of indecipherable modelling with autonomous bio-engineering, plus a motivational hack on the humans (climate will make us more and more desperate over time – and consequently sloppier) and an industrial scale insertion vector (geoengineering).

YIFF IN HELL ROBOT SCUM!!

[Anonymous Coward]

google & wikipedia won't hurt too much.
But FSM save us if they learn everything they know from /b/ and LOLcats.

Re:Sounds familiar

[farble1670]

Look at the execution-style murder of Oscar Grant, where a police "officer" shot him in the back while he lay helpless on the ground after being instructed to do so.

obviously you didn't look at it, or you'd see that is was clearly an accident. the officer and his colleagues looked at each other in horror after it happened. that, and common sense prevails. if they wanted to execute someone, they'd take him off in the car and stop in some dark alley on oakland. claim he got our of his cuffs. whatever.

Let this be a lesson to all. The police can and will MURDER people in plain sight of the public and not receive any punishment for it.

no, they'll murder you in a dark alley or back room in the police station, but not in plain sight.

Re:Sounds familiar

[Anonymous Coward]

obviously you didn't look at it, or you'd see that is was clearly an accident. the officer and his colleagues looked at each other in horror after it happened. that, and common sense prevails. if they wanted to execute someone, they'd take him off in the car and stop in some dark alley on oakland. claim he got our of his cuffs. whatever.

Exactly like I was saying before. The police shoot the black man in the back and then went and plant a gun. When it's premeditated, they make some effort to hide their crime.

Threats of AI

[Agent ME]

I think other comments here are vastly underestimating the threat a strong AI could pose.

Say you've got an extremely intelligent AI with plenty of processing power hooked to the internet, which has a decent understanding of the internet and human culture. It could discover vulnerabilities in software, and build up a botnet to distribute its own existence. An AI could sell digital goods online (it could create customized software for many many customers, etc) and amass some not insignificant wealth. What would an AI do with loads of cash? It could even purchase servers legitimately to spread itself instead of (or in addition to) the botnet idea. It could donate money to causes it wants to further, or fund businesses (with strings attached). It could do this on a large scale. What if it had an interest in politics?

A list of threats I put together back in 1999

[Paul Fernhout]

http://www.kurtz-fernhout.com/oscomak/fears.htm
====
The race is on to make the human world a better (and more resilient) place before one of these overwhelms us:
        * Autonomous military robots out of control
        * Nanotechnology virus / gray slime
        * Ethnically targeted virus
        * Sterility virus
        * Computer virus
        * Asteroid impact
        * Y2K
        * Other unforeseen computer failure mode
        * Global warming / climate change / flooding
        * Nuclear / biological war
        * Unexpected economic collapse from Chaos effects
        * Terrorism w/ unforeseen wide effects
        * Out of control bureaucracy (1984)
        * Religious / philosophical warfare
        * Economic imbalance leading to world war
        * Arms race leading to world war
        * Zero-point energy tap out of control
        * Time-space information system spreading failure effect (Chalker's Zinder Nullifier)
        * Unforeseen consequences of research (energy, weapons, informational, biological)
====

The solution I proposed there was developing a free and open source distributed library of information about how to make things, working towards the goal of creating self-replicating space habitats that can duplicate themselves from sunlight and asteroidal ore.

However, since then I think the deepest issue is changing how we thing, so we can move beyond, as in my sig, the irony of using the technologies of abundance from a perspective of fighting over misperceived scarcity. Bucky Fuller talked about that too, in moving from "weaponry" to "livingry". See also:
http://www.pdfernhout.net/recognizing-irony-is-a-key-to-transcending-militarism.html
http://anwot.org/

Here are some emails I wrote to Ray Kurzewil on these themes years ago that someone else put up on their site: http://heybryan.org/fernhout/

I essentially suggested that uploaded human minds would have their runtime consumed by the digital equivalent of natively-evolved digital piranha. I also suggested that the direction we come out of any singularity may have a lot to do with the moral direction we are pursuing as we go into it -- and that AI created mainly out of human military and economic competitiveness against other humans probably would not node well for having a happy singularity. That is why it is important to move our global society into a more compassionate direction before creating such AIs.

AI is already causing wars.

[Anonymous Coward]

9/11 was masterminded using AI. West still haven't figured out that because every time I try to post it, it becomes maniacally disrupted by U.S. government.
AI is a fact, and 9/11 numbers are the key to understand it, as well that 9+11=20 and there are two 7's. Cmon, america, are you really that dumb?

Re:Sounds familiar

[AK Marc]

They become cops because they aren't intelligent enough to get good jobs

Don't forget, it's legal for departments to screen out those too smart. You wouldn't want a thinking cop.