Smarter-than-Human Intelligence & The Singularity Summit

runamock writes "Brilliant technologists like Ray Kurzweil and Rodney Brooks are gathering in San Francisco for The Singularity Summit. The Singularity refers to the creation of smarter-than-human intelligence beyond which the future becomes unpredictable. The concept of the Singularity sounds more daunting in the form described by statistician I.J Good in 1965: 'Let an ultra-intelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultra-intelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultra-intelligent machine is the last invention that man need ever make.'"

Comments

Not quite ...

[ScrewMaster]

Thus the first ultra-intelligent machine is the last invention that man need ever make.'

Make that "... man is allowed to make" and I'll buy it.

Re:Not quite ...

[esaul]

Compassion is really a part of intelligence. Check out Kurzweil's 'Age of Spiritual Machines'. The more than human intelligence will inevitably entail compassion, love, and all the other emotions we have.
Further, forget about the 'borg' idea. We will inevitably evolve into these machines.

Re:Not quite ...

[Smidge204]

That quote has the same sentiment as "Everything that can be invented has been invented." (falsely attributed to various US patent office commissioners).

Intelligence isn't going to make invention obsolete unless there is artificial creativity to go with it. Some problems don't even present themselves as such until you try doing something different and non-obvious - almost random - and begin to realize new possibilities rather than refining existing ones.

How many great inventions came about because someone decided to try something just for the hell of it, without even thinking of the possibilities?
=Smidge=

Re:Not quite ...

[Space cowboy]

"Compassion is the inevitable result of empathy "

I Disagree. Compassion is not inevitable. You're working from your own tenets and philosophies, a machine need not have those same ideals. Compassion is at least partially born of self-interest. The cynical (or non-empathic, if you prefer) view is that compassionate societies aid those who need it, because later the person previously aided may be able to render aid... "There, but for the grace of God, go I", "Do unto others as you would be done unto", etc., etc.

Are we suggesting that these hyper-intelligent machines would have any self-interest in keeping around the competition for resources that humanity represents ? I'm not trying to be trollish, here - I'm asking a genuine question. Humanity is ruthless in exterminating competing lower lifeforms. Why would we expect superior machines to be any different ?

And even should there be some self-interest in the first generations of such machines, what about the 5th generation, the 10th, the 1000th ? All I'm suggesting is that some thought be put into providing good answers for questions like this *before* we create competition. I'm as much of a technophile as the rest of you, but the phrase goes "look *before* you leap". Later may be, well, too late.

Simon

Re:Not quite ...

[jamie]

Lions sometimes make friends with antelopes.

Not necessarily

[Anonymous Coward]

What if the intelligence of the smartest thing you can design doesn't grow as fast as your own intelligence (i.e. the slope of the graph {x=designer's intelligence, y=intelligence of its best possible design} is less than 1)? Then it would never be possible to be smarter than a robot that's exactly smart enough to design a robot as smart as itself.

Key Implication

[TrailerTrash]

If you follow TFA, and deeper, you find a discussion of the singularity that goes like this:

Man (level 1, or L1) creates better-than-man intelligence, call this L2
That intelligence uses its power to create L3

and so on.

In the case of truly artificial intelligence, i.e., independent processors, I can see the logic, though it may be that L2 is in fact smart enough not to obsolete itself by creating L3.

In the case of augmented human intelligence, I suggest that it's pretty likely that the task that the augmented L2 human turns its greater abilities on would not be creating L3.

Sadly, human history suggests that L2 will focus on manipulating the stock market for personal gain (the augmentation apparatus will leave L2 very vulnerable and L2 will want a tremendous amount of wealth to assure continued existence), or creating weapons, or accumulation of political power, or getting sucked into the vortex of religion, or other projects.

It will be very interesting to see, should we ever create L2, exactly what tasks it takes on. I bet they will not be beneficial to L1 life.

Re:Key Implication

[toppavak]

In the case of augmented human intelligence, I suggest that it's pretty likely that the task that the augmented L2 human turns its greater abilities on would not be creating L3. As a biomedical engineer I find this scenario the most likely and exciting. We are at a stage in our history at which we are just beginning to become able to directly control and alter (read: augment) ourselves. This is going to happen in 3 stages: replacement parts, augmented physical characteristics and finally augmented neurological function. This progression follows both the technical feasibility of each "step" and the sociological resistances to the idea of each. We've seen the ability to grow parts of replacement organs from stem cells directly harvested from the patient and as we learn more and more about the processes which govern differentiation in stem cells it is not science fiction at all that we will be able to grow entire organs in vitro within the near future. Once it becomes rather common practice to grow replacement kidneys and lungs for patients the "augmentation" will begin as a simple practice of removing detrimental characteristics which resulted in the failure of the organ to begin with, perhaps deleting a gene related to increased susceptibility to cancer from the new organ and move to introducing genes allowing for improved oxygen transport in lungs, more resilient filtration membranes and stronger cardiac tissue. The step between augmentation during a person's lifetime and the introduction of changes to their offspring is, I believe, a rather large one, and I dont forsee it becoming common practice for quite a while following the normalization of replacement and augmentation processes. Neurological augmentation is by far the most technically challenging and interesting problem. We're still nowhere near completely understanding the component-level functionality of neurons, heck even our understanding of neural networks is still embryonic. Transitioning from maintenance and repair of neural structures to outright re-wiring and augmentation will be a formidable technical challenge, but not one that is wholly unlikely either. The information revolution changed the way we see and learn about the world and brought about revolutionary changes in mechanical and electrical technologies. We're at the cusp of the beginnings of a biological revolution which will do the same. Biobricks is already laying the groundwork for custom-made biological machinery that can function as sensors and factories. Every day we learn more and more about the finer details of the workings of cellular machinery and in turn how to direct and control it. We're getting there.

Foreboding

[Concern]

Academia is falling all over itself in failed attempts to advance AI, but barring a series of harrowing breakthroughs, a Singularity is decades or even lifetimes away. Most of our more sober, grounded and credentialed thinkers appear not to want to consider the consequences - it's a bit too radical an idea, and "we still have plenty of time before we have to worry about it."

Futurists and writers and other folks out on the edge, like Kurzweil... those fanciful enough to take on the thought problem, seem to lean, in the majority, towards believing the human race would be destroyed or at least decimated by hyper-intelligence (Wachowskis, James Cameron, Lem, etc etc - too many to mention, really). An interesting minority are of the school that hyper-intelligences would be largely unconcerned with people, only dangerous where our goals intersected (Gibson, Lethem, Clarke). Very few seem to believe that a Singularity would be a positive development for the human race. Maybe Asimov? I'm not sure. Sometimes it seems like he was the last person who seriously spent time imagining that post-human AI could really be controlled at all (and many of his novels were arguably about the problems around the attempt).

Re:I disagree . . .

[1u3hr]

Why would anyone give this ultra-intelligent machine self-awareness? Or even give it arms/legs/options to do anything except communicate via a screen?

It would make itself useful, and be more useful if it did have access to communication and tools. Eventually it would earn trust. In any case, the technology would inevitably spread or be reinvented, add Moore's Law in some form, and in a few years they'd be cheap and ubiquitous. Someone would plug one into the net. Unless we have a Butlerian Jihad, it's inevitable.

Re:I disagree . . .

[UbuntuDupe]

Why would anyone give this ultra-intelligent machine self-awareness?

Perhaps because that's necessary for ultra-intelligence.

Or even give it arms/legs/options to do anything except communicate via a screen? I don't see them taking over anything unless they have arms/legs/means of replication.

May con artists throughout history have done "bad things" through their ability to fool people through a limited interface. (Nigerian scammers, anyone?) The AI research Eliezer Yudkowsky has proposed and run experiments showing it's possible that a very very intelligent program could "override a human through a text-only terminal". That is, it could convince a human operator to "let the genie out of the bottle".

Re:Good's bad logic

[Ralph Spoilsport]

Flying Pig is correct. The resource constraints, especially in the energy sector, are very real. We can yammer about "The Singularity" all you want, but it's not going to matter much when billions of people in the so-called "developing world" are dying of hunger, thirst, disease, or in some war over the remaining pools of energy and/or metals, and, conversely, millions of people in so-called "advanced" countries are reduced to penury as the economies slowly contract over decades.

Human numbers are following the same pathological growth one sees in a petri dish filled with sugar/energy - the bacteria grows like crazy until the energy/food is consumed. Then it dies off. Humans are capable of intensifying resources to meet needs, but logically, this is not a permanent "Get out of jail free" card. Eventually limits are hit, and people die off.

with the present numbers of humans (billions) and the political economy (industrial capitalist) the world is quickly becoming one big Easter Island.

RS