Slashdot Mirror
Vinge and the Singularity
mindpixel writes: "Dr. Vinge is the Hugo award winning author of the 1992 novel "Fire Upon the Deep" and the 1981 novella "True Names." This New York Times piece (registration required) does a good job of profiling him and his ideas about the coming "technological singularity," where machines suddenly exceed human intelligence and the future becomes completely unpredictable. " Nice story. And if you haven't read True Names, get a hold of a copy, plenty of used ones out there.
I don't think many people back then had any idea, that it would suddenly become "normal" for people to execute untrusted data with full privledges. The concept is still mind-boggling even today, let alone 1992.
OTOH, it's more of a social issue than a technogical one. I guess it doesn't take much vision to realize: People are stupid.
---
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
I don't know why you are claiming failure. Actually there has been quite a large amount of success. The problems have certainly been many, and there appear to be many remaining, but so?
... So what about web-based intelligent agents? (I don't know of any advanced ones... and that might require more computation than would be practical.) A web connected computer could live on the web in a way that would be only indirectly related to how a persons would experience it. Would their ability to learn the environment, and to figure out (i.e., calculate) how to navigate to their desired destination be considered intelligence? I doubt it. People wouldn't see it. And they wouldn't want to believe it (except for some small number of boosters who would cheer even simple responses as proof of intelligence).
If you will recall, last year was full of people denouncing Mozilla as a failure. It took a bit longer than they expected. But I no longer use anything else when I'm on Windows. (True, on Linux I more frequently use Konqueror, but I use Mozilla whenever I'm on the Gnome side of things.)
Possibly people's ideas of how a project should work have been overly influenced by movies and popular stories. (Though in Asimov's Foundation series, the bare framework of the Seldon plan required the entire lifetime devotion of the principle architect, as well as extensive commitment from dozens of others, so not all popular fiction is of the "quick fix" school.)
Relativity took many years to be developed to the point of presentation, then it took decades of testing, and it's still being worked on. Special Relativity is now reasonably soundly grounded, but General Relativity still needs work. But people don't call it a failure. Why not? The A-Bomb was as much of a brute-force effort as Deep Blue was. Both were successful demonstrations, and in their success they highlighted the weakness of the underlying theories.
But when it comes to AI, people keep moving the markers, so that whatever you do isn't really what they mean. I wait for the day when the hard version of the Turing test is passed. I firmly expect that at that point AI will be redefined so that this isn't sufficient to demonstrate intelligence. Already in matters of sheer logic computer programs can surpass any except the most talented mathematicians. (And perhaps them, I don't track this kind of thing.) It's true, most of these programs require a bit more resources that is today available on most home computers. But that's fair. Neural net programs can solve certain kinds of problems much more adeptly than people can. And they learn on their own what is an acceptable solution (via "training" and "reinforcement", etc.). And expert systems and capture areas of knowledge that are otherwise only accessible to experts in the field. (For some reason, experts are often a bit reluctant to cooperate.)
Now it's true, that these disparate functions need to be combined. It's true that the world is quite complex, and the only way to understand it may be to live in it.
The real problem with AI, is that nobody has a satisfactory definition of the 'I' part. Artificial is clear, but nobody can agree on a testable definition of Intelligence. The one real benefit is that it may get rid of those silly multiple choice IQ tests, and Standardized Achievement Tests. It would be easy for an AI to learn how to get the highest score possible (though it would require a bit of training, but then that's what they've turned grade-schools into -- training grounds for multiple choice tests).
Caution: Now approaching the (technological) singularity.
I think we've pushed this "anyone can grow up to be president" thing too far.
This says more about when/what you choose to read than about what has been written.
In certain decades it is "fashionable" to be optomistic. In others to be pessimistic. (The reasons have much to do with the age spread of the population, of the writer, with whether the author feels that things are getting better or worse NOW, etc.) During the late 50's up through the mid 70's optimism dominated. Then there was a reaction (Vietnam war, etc.) and the trend turned to pessimism (this started in Britain for some reason...I don't know why, I wasn't there).
But there are always contrary voices. When Asimov, and the well engineered machines that favored humanity were dominant, then Saberhage introduced the Berserkers (intelligent robot war machines designed to reproduce, evolve, and kill all life.)
I can't remember which are current, but novels with robot servents (sometimes almost invisible) aren't that uncommon even now. They just aren't featured characters anymore. They've become common, expected.
OTOH, another of Vinge's postulates is coming to pass, whether through fashion or necessity, the proportion of fantasy to science fiction is increasing. Fairly rapidly. Fantasy used to be uncommon (although it was common before WWII). In the 50's and 60's it was usually disguised as science fiction. It started emerging again in the 70's. And now it is the predominant form. But a large part of this may be fashion. OTOH, Vinge predicted that as the future became more incomprehensible, the proportion of fantasy to science fiction would increase. So. Not proof, but evidence.
Caution: Now approaching the (technological) singularity.
I think we've pushed this "anyone can grow up to be president" thing too far.
Vinge is not the only one to notice that the rate of growth in computer devices, if extrapolated for a few decades, will eventually exceed the capacity of the human brain, both in terms of strorage capability and in terms of processing speed. Indeed, this very notion forms the basis of many of Joy's and Kurzweil's recent discussions.
:"think" using some (probably very complex) set of algorithms with comparable computational efficiency as the human brain, if one indeed had a computer with similar processing and storage ability as the human brain. That logic is quite flawed, due to the assumption of computational efficiency.
However, in doing this extrapolation, one is making a few assumptions. Most notably is that one can teach a computer how to
What do I mean by computational efficiency? Roughly speaking, the relative performance of one algorithm to another. For instance, in talking about the singularity (as Vinge puts it), one often neglects to notice the fact that human beings, with their neurons clicking away at petacycles per second, can only do arithmetic extremely poorly, at less than a flop! Logical puzzles often similarly vex humans (witness the analytic portions of the GRE!), where they also perform incredibly poorly. Significantly, human beings are very computationally inefficient at most tasks involving higher brain functions. We might process sound and visual input very well and very quickly, but most higher brain functions are very poor performers indeed.
One application of a similar train of logic is that human beings are the only animals known to be capable of performing arithmetic. Therefore, if one had a computer comparable to the human brain, one could do arithmetic. Heck, by this logic, we're only 50 years away from using computers to do integer addition!
The main point here is that, with regards to developing a "thinking" machine, WE MIGHT VERY WELL have the brute force computational resources available to us today. The hardware is not the limitation, so much as our ability to design the software with the complex adaptive ability of the human brain.
Just WHEN we will be able to develop that software, no one can really say, since it is really a fundamental flaw in our approaches, rather than in our devices. (It is similar to asking when physicists will be able to write down a self-consistent theory of everything. No one can say.) It could happen in a decade or two, or it could take significantly longer then 50 years. It all depends on how clever we are in attacking the problem.
Science, like Nature, must also be tamed, with a view turned towards its preservation.
But very rarely in the ways you expect. Look at the predictions people were making for life in the year 2000 back in 1800, or 1900, or 1950, or even 1990. You'll see that a lot of it didn't happen. Some did, and some things that people hadn't even considered happened as well. But a lot of it just didn't take place.
Regardless of whether advancement takes place, the link that Vinge assumes between computer hardware performance and computer intelligent does not exist. If true machine intelligence comes about within the next thirty years it will not be as a direct result of improved hardware performance. There aren't any systems out there that aren't intelligent, but could be if we could overclock their processors to 150GHz.
++ Say to Elrond "Hello.".
Elrond says "No.". Elrond gives you some lunch.
Progress in computer hardware has followed this curve and continues to do so. Progress in computer intelligence however, hasn't. Computers are still stupid. They can now be stupid more quickly. This isn't going to produce super-human intelligence any time soon.
Dr Vinge reminds me somewhat of that most mocked of AI doomsayers, Kevin Warwick.
++ Say to Elrond "Hello.".
Elrond says "No.". Elrond gives you some lunch.
So, this idea is introduced in book 2 (or 3, depending on how you count) of "Across Realtime", a Novel(s) of his (it was origianlly 2 short novels and a novella, I think).
The idea is that technology progression is asymtotic, and will eventually reach the point where one day of technological progress is equal to all that of human history, and then, well... there's the next day. He doesn't cover exactly what it is, because by definition, we don't know yet. But, it's catastrophic in the novel. A good read (actually the first part which basically just introduces the "Bauble" is a good read alone).
He sort of refined the idea into something maintainable in Fire Upon the Deep by introducing the concept of the Slow Zone which acts as a kind of buffer for technology. If things in the Beyond get too hairy, the Slow Zone always remains unaffected, and civilization can crawl back up out of the "backwaters" (e.g. our area of the galaxy).
He's a good author, and I love his take on things like cryptography, culture (A Deepness in the Sky), religion, USENET (Fire Upon the Deep), Virtual Reality and cr/hacker culture (True Names).
--
Aaron Sherman (ajs@ajs.com)
I'm still waiting for humans to exceed human intelligence...we're all so obsessed about what the "robots" will do in the future when they get smarter than us. The present sucks already.
scold-mode: off
It's 10 PM. Do you know if you're un-American?
Another strong possiblity (for lack of SETI) is that intelligent races prefer virtual reality to real reality, in much the same way that the human race prefers to sit inside watching TV instead of going outside for a walk in the woods and grasslands where we evolved.
When we have better than Final Fantasy rendering in real time, most of the human race will probably choose to spend most of their day living and interacting there, in virtual-reality cyberspace... in much the same way that many of us today spend most of their day in an office environment, living and creating economic value in ways incomprehensible to our hunter and farmer ancestors.
When this happens, the planet may seem empty in many ways... in much the same way that suburban streets in America seem empty to a third-world visitor used to bustling and noisy street life.
This phase (human race moves into and settles cyberspace, become less visible in the physical world) is not the same as the Singularity. For one thing, it is not at all dependent on future advances in artificial intelligence... we just need ordinary number-crunching computers a few orders of magnitude faster than today.
If the AI naysayers are right, and machines never get smart enough, then the Singularity will never happen... but the "ghost planet" scenario will inevitably happen in our lifetime... either as a result of progress, or as the unhappy result of plague or nuclear war.
One of the more interesting ideas I've read about is the Knowledge Crash. I'm not entirely sure how feasible a theory it is, but it was proposed in a science fiction series I read a while back. (The first book was called something about Charon... Ring of Charon? Moon of Charon? Something like that.)
The idea is, basically, that every year it costs more to educate someone. In order to be able to expand our collective knowledge, or even to utilize the machines and operate the systems of the present, it will cost a certain amount of money in the education process.
In addition, we can quantify the amount of output a single human creates in his or her lifetime. For instance - if she works for thirty years at a Power Planet or something, we can determine the value that she has contributed to society.
As systems become more complex, more education is required. The education costs more money. At some point, if this continues unchecked, we will be faced with a situation where the cost of education exceeds the value brought as a result of that education.
That's called the Knowledge Crash. (Or it was in the books.)
While I'm not convinced that this is true, it's certainly an interesting theory. It seems to me that, on average this can't happen, as one of the points of creating more and more complicated (generic) systems is to facilitate simpler and simpler controls, and thus dumber and dumber operators. While the creators of those systems may have 'crashed knowledge,' it seems that the whole point of that would be to hurl some value at the workers.
But then you have to consider that, inherent in the value of a designer, the ease of use is part of the entire value analysis versus education, and then that'll crash...
Because of stupid, but fast, computers, we are headed toward being able to hack our DNA (and/or proteins). This will certainly produce incremental gains in lifespan and health...perhaps it will produce dramatic ones.
Because of stupid, but fast, computers, we can simulate physical processes to enable us to engineer better widgets. Perhaps this will make routine space travel economical.
Because of stupid, but fast, computers, we are heading toward having the bulk of human knowledge instantly available to anyone net connection. How will this leverage technical progress?
"Rub her feet." -- L.L.
Another thing has to do with this "let's fear AI" genre of SciFi in general. Why does no one challenge the assumption that when artificial creatures develop intelligence and a personality, that personality will inevitably be indifferent, power-hungry and cold? Isn't it just as easy to imagine that artificially intelligent creatures/machines will strike us as being neurotically cautious, or maybe friendly to the point of being creepy? Maybe they'll become obsessed with comedy or math or music. Or video games.
Realistically, I think the first machines which we take to be intelligent will be very good at means-to-ends reasoning, but will not be able to deliberate about ends (i.e. why should one sort of outcome be preferrable to another). I would argue that even we humans can't really deliberate about ends. At some point we hit some hard-wired instincts. Why, for example, is it better that people are happy rather than suffering? The answer is just a knee-jerk reaction by us, not some sort of a reasoned conclusion.
When we create AI we will have the luxury of hard-wiring these instincts into intelligent machines (without some parameters specifying basic goals, nothing could be intelligent, not even we). Humans and animals are basically built with a set of instincts designed make them survive and fuck and make sure the offspring survive. There is no reason to think AI creatures would necessarily have these instructions as basic. I'm sure we could think of much more interesting ones. The consequence is that AI creatures might be more intelligent than we are, but in no way sinister.
Technological Singularity by Vernor Vinge -- available online at http://www.ugcs.caltech.edu/~phoenix/vinge/vinge-s ing.html -- is the scariest and yet most inspiring document that I have ever read on Artificial Intelligence -- which is being implemented slowly but surely on SourceForge at http://mind.sourceforge.net in JavaScript for Web migration and in Forth for robots, evolving towards full civil rights on a par with human beings and towards a superintelligence beyond any human IQ, as described so eerily and scarily by Vinge. It used to be that I did not like Vinge's science fiction, but right now I am thoroughly enjoying A Deepness in the Sky by Vinge.
--
The most succinct Vinge quote that I can think of is:
If you were blocking sigs, you wouldn't have to read this.
Ray Kurzweil seems to be making Vinge's singularity his life's work:
, 00.html
http://www.kurzweilai.net/news/
And then there's the non-profit corporation, the Singularity Institute for artificial Intelligence, which is determined to bring the Singularity about as soon as possible:
http://www.singinst.org/
There are a lot of good Vinge links on that page too, btw
Singinst seems to be the brainchild of this guy:
http://www.wired.com/news/technology/0,1282,43080
who has a lot of interesting docs here:
http://sysopmind.com/beyond.html
Don't miss, the FAQ on the meaning of life, it's great reading.
AN DIEGO -- VERNOR VINGE, a computer scientist at San Diego State University, was one of the first not only to understand the power of computer networks but also to paint elaborate scenarios about their effects on society. He has long argued that machine intelligence will someday soon outstrip human intelligence.
But Dr. Vinge does not publish technical papers on those topics. He writes science fiction.
And in turning computer fact into published fiction, Dr. Vinge (pronounced VIN-jee) has developed a readership so convinced of his prescience that businesses seek his help in envisioning and navigating the decades to come.
"Vernor can live, as few can, in the future," said Lawrence Wilkinson, co-founder of Global Business Network, which specializes in corporate planning. "He can imagine extensions and elaborations on reality that aren't provable, of course, but that are consistent with what we know."
Dr. Vinge's 1992 novel, "A Fire Upon the Deep" (Tor Books), which won the prestigious Hugo Award for science fiction, is a grand "space opera" set 40,000 years in a future filled with unfathomable distances, the destruction of entire planetary systems and doglike aliens. A reviewer in The Washington Post (news/quote) called it "a wide-screen science fiction epic of the type few writers attempt any more, probably because nobody until Vinge has ever done it well."
But computers, not aliens, were at the center of the work that put Dr. Vinge on the science fiction map -- "True Names," a 30,000-word novella that offered a vision of a networked world. It was published in 1981, long before most people had heard of the Internet and a year before William Gibson's story "Burning Chrome" coined the term that has come to describe such a world: cyberspace.
For years, even as its renown has grown, "True Names" has been out of print and hard to find. Now it is being reissued by Tor Books in "True Names and the Opening of the Cyberspace Frontier," a collection of stories and essays by computer scientists that is due out in December.
"True Names" is the tale of Mr. Slippery, a computer vandal who is caught by the government and pressed into service to stop a threat greater than himself. The story portrays a world rife with pseudonymous characters and other elements of online life that now seem almost ho-hum. In retrospect, it was prophetic.
"The import of `True Names,' " wrote Marvin Minsky, a pioneer in artificial intelligence, in an afterword to an early edition of the work, "is that it is about how we cope with things we don't understand."
And computers are at the center of Dr. Vinge's vision of the challenges that the coming decades will bring. A linchpin of his thinking is what he calls the "technological singularity," a point at which the intelligence of machines takes a huge leap, and they come to possess capabilities that exceed those of humans. As a result, ultra- intelligent machines become capable of upgrading themselves, humans cease to be the primary players, and the future becomes unknowable.
Dr. Vinge sees the singularity as probable if not inevitable, most likely arriving between 2020 and 2040.
Indeed, any conversation with Dr. Vinge, 56, inevitably turns to the singularity. It is a preoccupation he recognizes with self-effacing humor as "my usual shtick."
Although he has written extensively about the singularity as a scientific concept, he is humble about laying intellectual claim to it. In fact, with titles like "Approximation by Faber Polynomials for a Class of Jordan Domains" and "Teaching FORTH on a VAX," Dr. Vinge's academic papers bear little resemblance to the topics he chooses for his fiction.
"The ideas about the singularity and the future of computation are things that basically occurred to me on the basis of my experience of what I know about computers," he said.
"And although that is at a professional level, it's not because of some great research insight I had or even a not-so-great research insight I had. It's because I've been watching these things and I like to think about where things could go."
Dr. Vinge readily concedes that his worldview has been shaped by science fiction, which he has been reading and writing since childhood. His dream, he said, was to be a scientist, and "the science fiction was just part of the dreaming."
Trained as a mathematician, Dr. Vinge said he did not begin "playing with real computers" until the early 1970's, after he had started teaching at San Diego State. His teaching gradually shifted to computer science, focusing on computer networks and distributed systems. He received tenure in 1977.
"Teaching networks and operating systems was a constant source of story inspiration," Dr. Vinge said. The idea for "True Names" came from an exchange he had one day in the late 1970's while using an early form of instant messaging called Talk.
"Suddenly I was accosted by another user via the Talk program," he recalled. "We chatted briefly, each trying to figure out the other's true name. Finally I gave up and told the other person I had to go -- that I was actually a personality simulator, and if I kept talking, my artificial nature would become obvious. Afterwards I realized that I had just lived a science fiction story."
Computers and artificial intelligence are, of course, at the center of much science fiction, including the current Steven Spielberg film, "A.I." In the Spielberg vision, a robotic boy achieves a different sort of singularity: parity with humans not just in intelligence but in emotion, too. "To me, the big leap of faith is to make that little boy," Dr. Vinge said. "We don't have evidence of progress toward that. If it ever happens, there will be a runaway effect, and getting to something a whole lot better than human will happen really fast."
How fast? "Maybe 36 hours," Dr. Vinge replied.
Dr. Vinge's own work has yet to make it to the screen, although "True Names" has been under option for five years. "It's been a long story of my trying to convince studio executives to really consider the work seriously because it seemed so far out," said David Baxter, a Hollywood writer and producer who is writing the screenplay with Mark Pesce, co-creator of Virtual Reality Modeling Language, or VRML. "But as time has passed, the world has started to match what was in the book."
In the meantime Dr. Vinge has been providing scenarios in the corporate world as well. He is one of several science fiction writers who have worked with Global Business Network in anticipating future situations and plotting strategies for several major companies.
Mr. Wilkinson, the co-founder of Global Business Network, said that Dr. Vinge's work with the group provided "an unbelievably fertile perspective from which to look back at and reunderstand the present."
"It's that ability to conceptualize whole new ways of framing issues, whole new contexts that could emerge," Mr. Wilkinson said. "In the process he has contributed to the turnarounds of at least two well-known technology companies."
Dr. Vinge, shy and reserved, is hardly a self-promoter. He scrupulously assigns credit to others whenever he can. And although he insists that much of his work is highly derivative, his fans do not necessarily share that view.
"The thing that distinguishes Vernor is he's a scientist and all of his stuff makes sense," Mr. Baxter said. "It's all grounded in the here and now."
Dr. Vinge is now a professor emeritus at San Diego State, having retired to devote his time to his writing and consulting. Over lunch at a restaurant not far from the university, he described a story he was working on.
"Well, there's a recovering Alzheimer's patient," Dr. Vinge began, before being interrupted and asked how one could be a recovering Alzheimer's patient.
His eyes brightened. "You can't," he said, and a sly smile crossed his face. "Yet."