Doh, I should have seen this earlier, just posted this under another thread, but Bill Joy, a co-founder of Sun Microsystems, has also spoken out on issues regarding the dangers of emerging technology such as AI.
http://www.zdnet.com/zdnn/stories/news/0,4586,2461 470,00.html?chkpt=zdnntop
If you think about it, whether you're imagining the shape of an automobile or using your knowledge of chemistry to perform an assay, much of what we experience as our intelligence stem from attempts at fusing, structuring, searching through information collected through our senses and responding to that information through our appendages, so one should not be surprised if increased ability to process input and control interaction with the environment also happens to increase one's ability to think creatively and perform many other intellectual feats.
Personally, I've believed since my sophomore year in high school (94-95) that we are living in one of the the most promising and dangerous period in human history.
The potential for revolutionary beneficial technological change and annihilating disaster are equally probable.
It's an exciting time to be alive and to be working in science!
In fact, the airforce and the navy are planning to field unmanned air combat vehicles that will be autonomous to a degree where they will be able to complete their missions without human input (although security protocols dictate that human beings have to give the command to fire).
It is conceivable that military networks, presented with the need to monitor an increasingly complex battle field, may depend more and more on processing by artificial intelligence data that humans cannot analyse in a timely manner.
Of course, with proper safeguards, humans should still be in charge, but when situations become desperate enough, human beings tend to take more desperate measures,so it's not inconceivable that some operational control may be handed to AIs.
Discounting military AIs, industrial AIs and genetic algorithms used in product design (yes, there are now genetic algorithms that can do some design work) are taking on duties traditionally done by human decision makers. That's what they're designed for.
What Hawkings is worried about is probably the emergence of civilizations in which human beings are irrelevant, a point when human beings become so dependent on AI, and AIs become so smart that we're no longer much more than pets maintained by AIs out of interest.
Besides, there's always the possibility that ambitious individual will augment their own intelligence with artificial intelligence, creating a new aristocracy.
If we're as close to large scale quantum computing as the amount of research and money being devoted to it in the physics community indicates, there might be reason to worry.
Many physicists suspect, though they might not publicly state so, that quantum mechanics is not complete without a proper theory of conciousness and vice versa (I'm not kidding, this is stuff is seriously discussed by some of the field's most recognized practitioners when the questions about the whys of quantum mechanics come up).
Even if a concious AI can't emerge in modern computers, would quantum computers be able to attain conciousness? No one knows.
What if one day machines become so complex that we are no longer able to maintain a working understanding of them without enhancing our own intelligence? Seems to me like that, at least, is very possible.
It is interesting to speculate how one might go about merging human and artificial intelligence. The first step is to construct computers that can duplicate the functionality, communication protocols, and organizational structure of neural networks in the human brain. (This in itself would be a monumental achievement, as science as almost no understanding of how neurons actually function. For one thing, data within the past decade has shown that individual neurons perform sophisticated calculations using undetermined machinery. One hypothesis, notably advocated by Oxford mathematics and physics professor Roger Penrose, proposes that computation or possibly quantum nature might be performed inside cellular microtubules.)
The second step would be to construct the physical interface between the computer and the human brain.
Step three would be to program the computer to enhance human intelligence. This is a job best left to the human brain itself. If one constructs the computer in a way that replicates the organizational structure of a developing human brain, then the brain itself would be able to program the computer given proper stimuli. Essentially, instead of grafting a preformed artificial intelligence onto a human mind, a blank computer capable of supporting an AI would be attached on to the human brain. The brain would then be given extra stimuli, such as a network of sensors that give a richer amount of information than existing human senses, and extra appendages, such as robotic vehicles that can manipulate the environment in more intricate ways than human arms and legs. In time, as the brain attempts to accommodate added demands, it will program the newly grafted computer to become an extension the brain of the human being it's connected to.
Step four would be subtle modifications to allow for future extensibility.
The final step would be to create redundancy in such a way that the enhanced human intelligence will no longer depend on the presence of the original biological human brain to function. That is, make it so that the human intelligence (and one would hope, consciousness) will be able to survive the destruction of the biological body.
The above steps, if feasible, could turn human beings into virtual deities that would be practically immortal and extraordinarily powerful. Today, the physical manifestation of human intelligence is a system of systems containing an astronomical number of biological components that process and exchange information on both local and system wide scales and regulate the flow of resources within the physical boundaries of the human body. In the future, the physical manifestation of a human being may very well expand to include numerous networked sensors and robotic platforms connected to the human brain as integrally as our eyes, ears, arms and legs, organized along biological principles based on those that allow our bodies to function, distributed over a large region of physical space and interacting with the environment with far greater complexity on a far wider spectrum of energies and length scales. Indeed, the sum of resources available to entire nations today may become insignificant to those available to a single human being tomorrow.
On a side note, with regards to whether we should be worried about super AIs, the problem is not necessarily one of enslavement, but one of relevance. If the emergence of rapidly developing artificial intelligence is possible, and it's not obvious that it is, then we face the possibility that within one or two centuries, neither our power nor our intelligence will be significant compared to that possessed by the dominant species in our solar system, namely, artificial intelligence. Un
There's no reason why a more complex intelligence would be more benevolent than we ourselves are. We therefore might look to our own treatment of other species on earth as an indication of what an advanced AI might do to us. All except for the most naïve optimists should be profoundly disturbed by this possibility.
It is interesting to speculate how one might go about merging human and artifical intelligence. The first step is to consruct computers that can duplicate the functionality, communication protocols, and organizational structure of neural networks in the human brain. ( This in itself would be an monumental achievement, as science as almost no understanding of how neurons actually function. For one thing, data within the past decade has shown that individual neurons perform sophisticated calculations using undetermined machinery. One hypothesis, notably advocated by Oxford mathematics and physics professor Roger Penrose, porposes that computation or possibly quantum nature might be performed inside cellular microtubials.)
The second step would be to construct the physical interface between the computer and the human brain.
Step three would be to program the computer to enhance human intelligence. This is a job best left to the human brain itself. If one constructs the computer in a way that replicates the organizational structure of a developing human brain, then the brain itself would be able to program the computer given proper stimulii. Essentially, instead of grafting a preformed artificial intelligence onto a human mind, a blank computer capable of supporting an AI would be attached on to the human brain. The brain would then be given extra stimulii, such as a network of sensors that give a richer amount of information than existing human senses, and extra apendages, such as robotic vehicles that can manipulate the environment in more intricate ways than human arms and legs. In time, as the brain attempts to accommodate added demands, it will program the newly grafted computer to become an extension the brain of the human being it's connected to.
Step four would be subtle modifications to allow for future extensibility.
The final step would be to create redundancy in such a way that the enhanced human intelligence will no longer depend on the presence of the original biological human brain to function. That is, make it so that the human intelligence (and one would hope, conciousness) will be able to survive the destruction of the biological body.
The above steps, if feasible, could turn human beings into virtual deities that would be practically immortal and extraordinarily powerful. Today, the physical manifestation of human intelligence is a system of systems containing an astronomical number of biological components that process and exchange information on both local and system wide scales and regulate the flow of resources within the physical boundaries of the human body. In the future, the physical manifestation of a human being may very well expand to include numerous networked sensors and robotic platforms connected to the human brain as integrally as our eyes, ears, arms and legs, organized along biological principles based on those that allow our bodies to function, distributed over a large region of physical space and interacting with the environment with far greater complexity on a far wider spectrum of energies and length scales. Indeed, the sum of resources available to entire nations today may become insignificant to those available to a single human being tomorrow.
On a side note, with regards to whether we should be worried about super AIs, the problem is not necessarily one of enslavement, but one of relevance. If the emergence of rapidly developing artificial intelligence is possible, and it's not obvious that it is, then we face the possibility that within one or two centuries, neither our power nor our intelligence will be significant compared to that possessed by the dominant species in our solar system, namely, artificial intelligence. Un
There's no reason why a more complex intelligence would be more benevolent than we ourselves are. We therefore might look to our own treatment of other species on earth as an indication of what an advanced AI might do to us. All except for the most naïve optimists should be profoundly disturbed by this possibility.
Slashdot Mirror
It could be doable, but is extremely risky. We should definitely try commando raids first.
Doh, I should have seen this earlier, just posted this under another thread, but Bill Joy, a co-founder of Sun Microsystems, has also spoken out on issues regarding the dangers of emerging technology such as AI. http://www.zdnet.com/zdnn/stories/news/0,4586,2461 470,00.html?chkpt=zdnntop
If you think about it, whether you're imagining the shape of an automobile or using your knowledge of chemistry to perform an assay, much of what we experience as our intelligence stem from attempts at fusing, structuring, searching through information collected through our senses and responding to that information through our appendages, so one should not be surprised if increased ability to process input and control interaction with the environment also happens to increase one's ability to think creatively and perform many other intellectual feats.
Bill Joy, Sun Microsystem's co-founder has harped about the dangers of AI and other technologies for years.
4 61 470,00.html?chkpt=zdnntop
http://www.zdnet.com/zdnn/stories/news/0,4586,2
Personally, I've believed since my sophomore year in high school (94-95) that we are living in one of the the most promising and dangerous period in human history.
The potential for revolutionary beneficial technological change and annihilating disaster are equally probable.
It's an exciting time to be alive and to be working in science!
Good points.
In fact, the airforce and the navy are planning to field unmanned air combat vehicles that will be autonomous to a degree where they will be able to complete their missions without human input (although security protocols dictate that human beings have to give the command to fire).
It is conceivable that military networks, presented with the need to monitor an increasingly complex battle field, may depend more and more on processing by artificial intelligence data that humans cannot analyse in a timely manner.
Of course, with proper safeguards, humans should still be in charge, but when situations become desperate enough, human beings tend to take more desperate measures,so it's not inconceivable that some operational control may be handed to AIs.
Discounting military AIs, industrial AIs and genetic algorithms used in product design (yes, there are now genetic algorithms that can do some design work) are taking on duties traditionally done by human decision makers. That's what they're designed for.
What Hawkings is worried about is probably the emergence of civilizations in which human beings are irrelevant, a point when human beings become so dependent on AI, and AIs become so smart that we're no longer much more than pets maintained by AIs out of interest.
Besides, there's always the possibility that ambitious individual will augment their own intelligence with artificial intelligence, creating a new aristocracy.
If we're as close to large scale quantum computing as the amount of research and money being devoted to it in the physics community indicates, there might be reason to worry.
Many physicists suspect, though they might not publicly state so, that quantum mechanics is not complete without a proper theory of conciousness and vice versa (I'm not kidding, this is stuff is seriously discussed by some of the field's most recognized practitioners when the questions about the whys of quantum mechanics come up).
Even if a concious AI can't emerge in modern computers, would quantum computers be able to attain conciousness? No one knows.
What if one day machines become so complex that we are no longer able to maintain a working understanding of them without enhancing our own intelligence? Seems to me like that, at least, is very possible.
It is interesting to speculate how one might go about merging human and artificial intelligence. The first step is to construct computers that can duplicate the functionality, communication protocols, and organizational structure of neural networks in the human brain. (This in itself would be a monumental achievement, as science as almost no understanding of how neurons actually function. For one thing, data within the past decade has shown that individual neurons perform sophisticated calculations using undetermined machinery. One hypothesis, notably advocated by Oxford mathematics and physics professor Roger Penrose, proposes that computation or possibly quantum nature might be performed inside cellular microtubules.)
The second step would be to construct the physical interface between the computer and the human brain.
Step three would be to program the computer to enhance human intelligence. This is a job best left to the human brain itself. If one constructs the computer in a way that replicates the organizational structure of a developing human brain, then the brain itself would be able to program the computer given proper stimuli. Essentially, instead of grafting a preformed artificial intelligence onto a human mind, a blank computer capable of supporting an AI would be attached on to the human brain. The brain would then be given extra stimuli, such as a network of sensors that give a richer amount of information than existing human senses, and extra appendages, such as robotic vehicles that can manipulate the environment in more intricate ways than human arms and legs. In time, as the brain attempts to accommodate added demands, it will program the newly grafted computer to become an extension the brain of the human being it's connected to.
Step four would be subtle modifications to allow for future extensibility.
The final step would be to create redundancy in such a way that the enhanced human intelligence will no longer depend on the presence of the original biological human brain to function. That is, make it so that the human intelligence (and one would hope, consciousness) will be able to survive the destruction of the biological body.
The above steps, if feasible, could turn human beings into virtual deities that would be practically immortal and extraordinarily powerful. Today, the physical manifestation of human intelligence is a system of systems containing an astronomical number of biological components that process and exchange information on both local and system wide scales and regulate the flow of resources within the physical boundaries of the human body. In the future, the physical manifestation of a human being may very well expand to include numerous networked sensors and robotic platforms connected to the human brain as integrally as our eyes, ears, arms and legs, organized along biological principles based on those that allow our bodies to function, distributed over a large region of physical space and interacting with the environment with far greater complexity on a far wider spectrum of energies and length scales. Indeed, the sum of resources available to entire nations today may become insignificant to those available to a single human being tomorrow.
On a side note, with regards to whether we should be worried about super AIs, the problem is not necessarily one of enslavement, but one of relevance. If the emergence of rapidly developing artificial intelligence is possible, and it's not obvious that it is, then we face the possibility that within one or two centuries, neither our power nor our intelligence will be significant compared to that possessed by the dominant species in our solar system, namely, artificial intelligence. Un
There's no reason why a more complex intelligence would be more benevolent than we ourselves are. We therefore might look to our own treatment of other species on earth as an indication of what an advanced AI might do to us. All except for the most naïve optimists should be profoundly disturbed by this possibility.
It is interesting to speculate how one might go about merging human and artifical intelligence. The first step is to consruct computers that can duplicate the functionality, communication protocols, and organizational structure of neural networks in the human brain. ( This in itself would be an monumental achievement, as science as almost no understanding of how neurons actually function. For one thing, data within the past decade has shown that individual neurons perform sophisticated calculations using undetermined machinery. One hypothesis, notably advocated by Oxford mathematics and physics professor Roger Penrose, porposes that computation or possibly quantum nature might be performed inside cellular microtubials.) The second step would be to construct the physical interface between the computer and the human brain. Step three would be to program the computer to enhance human intelligence. This is a job best left to the human brain itself. If one constructs the computer in a way that replicates the organizational structure of a developing human brain, then the brain itself would be able to program the computer given proper stimulii. Essentially, instead of grafting a preformed artificial intelligence onto a human mind, a blank computer capable of supporting an AI would be attached on to the human brain. The brain would then be given extra stimulii, such as a network of sensors that give a richer amount of information than existing human senses, and extra apendages, such as robotic vehicles that can manipulate the environment in more intricate ways than human arms and legs. In time, as the brain attempts to accommodate added demands, it will program the newly grafted computer to become an extension the brain of the human being it's connected to. Step four would be subtle modifications to allow for future extensibility. The final step would be to create redundancy in such a way that the enhanced human intelligence will no longer depend on the presence of the original biological human brain to function. That is, make it so that the human intelligence (and one would hope, conciousness) will be able to survive the destruction of the biological body. The above steps, if feasible, could turn human beings into virtual deities that would be practically immortal and extraordinarily powerful. Today, the physical manifestation of human intelligence is a system of systems containing an astronomical number of biological components that process and exchange information on both local and system wide scales and regulate the flow of resources within the physical boundaries of the human body. In the future, the physical manifestation of a human being may very well expand to include numerous networked sensors and robotic platforms connected to the human brain as integrally as our eyes, ears, arms and legs, organized along biological principles based on those that allow our bodies to function, distributed over a large region of physical space and interacting with the environment with far greater complexity on a far wider spectrum of energies and length scales. Indeed, the sum of resources available to entire nations today may become insignificant to those available to a single human being tomorrow. On a side note, with regards to whether we should be worried about super AIs, the problem is not necessarily one of enslavement, but one of relevance. If the emergence of rapidly developing artificial intelligence is possible, and it's not obvious that it is, then we face the possibility that within one or two centuries, neither our power nor our intelligence will be significant compared to that possessed by the dominant species in our solar system, namely, artificial intelligence. Un There's no reason why a more complex intelligence would be more benevolent than we ourselves are. We therefore might look to our own treatment of other species on earth as an indication of what an advanced AI might do to us. All except for the most naïve optimists should be profoundly disturbed by this possibility.