Slashdot Mirror
Bill Joy On Extinction of Humans
e3 writes "The Washington Post is running a provocative article in which Bill Joy is quoted as, "...essentially agreeing, to his horror, with a core argument of the Unabomber, Theodore Kaczynski -- that advanced technology poses a threat to the human species." " As it stands, the title sounds sensationalistic - but read the article, and think about what point he's trying to make. Bill Joy's a pretty level-headed guy, and I think we need to consider these issues /now/ so that they don't come true.
Try as one might, genetics and nanotechnology are not easy fields for individuals to work on their own in. They require extensive amounts of equipment, much of it high-tech since much of the work has only developed over the past twenty years.
Most things become easier in time. An eight year old with a chemistry set today does things incomprehensable to the greatest minds of the 1st century, and doesn't think much of it. At one time, the 'hello world' program was a big deal (especially when it had to be wired in). Now, it's literally child's play.
It's not time to head for the hills by any means, but these things CAN come to pass. The best hope is that the same technology can be used to avert disaster. The nasty self-replicating robots will be destroyed by 'good' self replicating robots, for example.
How soon we forget.
There was a time when incineration of much of the civilized world was always 20 minutes away, not 20 years. Whether secondary effects (so-called Nuclear Winter) would have led to eventual extinction or not seem rather beside the point--the world as we knew it would have ended. That it did not happen was, more than many of us realize, a matter of shear blind luck.
There were, and are, only two powers in the world who could bring about such a global catastrophe. The reason for this limitation is more a matter of the enormous cost of producing nuclear weapons than the technological difficulty of doing so. For now, and for the near future, nuclear physics is too expensive for more than just the US and Russia to put civilization at risk.
What Bill fears, I think, is the development of technologies as threatening as those which came from nuclear physics, but without the economic barriers. Consider: what if Moore's law applied to nuclear weapons as well as integrated cicuitry? What if it does apply to the next destructive technology? Or: what if a chain reaction of self-replicating agents--whether biological, nanotechnological, or self-organizing--proves much cheaper than the nuclear variety? By harnessing the existing biological, molecular, or technological environment to its ends, could a technology be created where such replication to worldwide scale came (from the creator's perspective) essentially for free?
The cheaper it becomes to develop the technical means to threaten humanity, the more likely it will be that a state, group, or even person will be insane enough to exploit it. It's the change in economics that increases the danger. Economics explains why New York isn't hip-deep in horse manure just as it explains why basement-lab nuclear weapons don't exist, even though the knowledge necessary to produce them is readily available. Cheaper, faster alternatives became available in the first case. Are we ready for such alternatives in the second case?
Here's a little quote from "The Difference Between the Sexes", by E. Balaban (ed) and R. V. Short (ed):
"Perhaps the lifespan of a species is inversely proportional to its degree of intellectual development? The probability that a species that has evolved to be as intelligent and all-conquering as ours could survive for long is remote indeed. We may live in a silent universe for a very good reason. Paradoxically, evolution may have ensured that we have one of the shortest survival times of any species, since it has made us, effectively, our own executioner."
You can't build an artificially intelligent computer unless you have a damn good idea of those things. You can't build something with desires, emotions, etc. unless you know, in detail, what desires and emotions are, at a far deeper level than we do now.
Your entire argument is based on the premise of top-down design - that the Right Way to build an AI is the classical engineer's approach of designing the thing as you would design any other machine or piece of software.
Fortunately, most people now recognise that this approach is doomed, for the exact reason that you point out: an "intelligence" of any sort is much more complex and less well-understood than anything we've ever had to design.
So, what's the alternative? Automated bottom-up design. Specifically, the idea is to first work out the building blocks - the equivalents of neurons - and then have a GA or somesuch start trying to put together a "brain" out of these neurons, which is fit for a specific purpose. Note that this alternative doesn't require one to understand in excrutiating detail (or at all) the high-level abstractions which we consider as "intelligence" - it only requires a good GA and a good understanding of the brain at the cellular and subcellular level.
Now this I don't consider far-fetched at all.
(Of course, it's always worth mentioning that we could go the other way - first using nanotech to completely redesign ourselves into super-intelligent cybergods, then analysing our own new brains and replicating them to create completely new, fully artificial intelligent beings.)
To the editors: your English is as bad as your Perl. Please go back to grade school.
Assuming that advances in technology continue, I think it is reasonable to postulate that at some stage we will create sentient beings. Whether this is done in software, or uses nanotechnology, or biotechnology or whatever, it raises some interesting ethical questions. Is such an entity permitted to value it's own self-preservation? What if this leads to conflict with humans?
Do we have a right to construct entities that place human well-being above their own well-being? (Asimov's 'Laws of Robotics' or similar)
If we do this, aren't we dangerously close to building slaves?
These comments do not neccesarily reflect the views of the author.
Before all the geeks in the world go hurtling themselves off their rackmounts, let's take a look at some of Bill's assumptions.
Artificial Intelligence
A lot of Bill's thesis is based on the assumption that we'll be able to create sentience in machines. Yes, computers are getting faster and yes they can even seem to think sometimes, but folks, we don't even understand how our own brains work, much less have the power to create artificial ones. Things like thought require a much deeper understanding than we're likely to achieve in the next 20 years. Don't get me wrong, I think someday we'll be able to do it, but the trials will be long and hard, and the people who do it will really understand how to make it right. I also don't think I'll see it in my lifetime (I'm 22 now).
Replication
In terms of machines, a lot of this has to do with artificial intelligence. The creative leap required to construct something and change it is pretty huge. As for nanorobots in our blood stream, they need to find the parts, and they most likely won't be in the same environment in which they were created. Genetics is more fearful, of course, because living things already have the ability to recreate, but most work done in genetics is done under the constant shadow of "what bad things can this bring". I don't think genetics is all that easy a field for an individual to work in as a radical either. It takes an extraordinary amount of time and equipment. The most likely disaster of bioengineering is something that causes the death of a significant member of the planetary cycle (like trees or bees, for instance), which has been a constant concern from day one.
The Free Radical
Try as one might, genetics and nanotechnology are not easy fields for individuals to work on their own in. They require extensive amounts of equipment, much of it high-tech since much of the work has only developed over the past twenty years. It's still much more likely that some nut is going to get his hands on some plutonium leaking out of an impoverished former superpower and create some home-made nuclear weapon than it is that someone is going to create a killer replicating robot.
And Bill ignores a lot of other ways we can kill ourselves. Civil strife, environmental pollution, global warming, and, my personal favorite, contact with a hostile alien species (didn't Independence Day look real?). The fact is, since day one, humans have been faced with causing their own extinction (overhunting, overfarming, overpolluting, travel spreading disease, etc. etc.) and we've done just fine recognizing and adapting to these problems. The one thing that nobody ever seems to factor in is the human response to adversity. We can change our environment, and once we've changed it, if something's wrong, we can change it further (not back), so that we can live in it.
p.s. And did anyone notice that Bill was called 'phlegmatic'? I thought they meant 'pragmatic', but that's one helluva typo.
Two points.
(1) If anyone here remembers their history, the'd remember that the environmental problem du'joir in the 1970's was global cooling, not global warming. The truth of the matter is that the evidence is still out about global warming--the best we can say is that we have some interesting localized weather patterns, but there is no evidence of any sea levels rising or any non-natural weather patterns changing. (And those who provide "statistical evidence"--if you look closely enough, they're cooking the books combined with weather simulations which they believe will predict the weather beyond the normal 7-14 days most simulations actually work.)
My point is that if you listen real carefully, even global warming is in the "disaster which will wipe us out in 10-20 years" category--far enough away that it seems possible (especially on warmer spring days), yet close enough to actively fear.
By the way, you forgot the ozone hole--though there are those who are starting to think it ain't the problem it once was, only because ground-level UV levels have not changed one iota. But there are those who still believe that in 10-20 years we're going to have to go out in the sun with SPF 5000 or die.
That's okay; I still remember when I was growing up in the 1970's that we were to run out of oil by 1990. That is, we would deplete all of the world's oil reserves by 1990, and because of it, civilization would collapse, causing wars a'la "Mad Max" to break out throughout the world as people struggle to find the last little caches of horded gasoline.
I have a real hard time believing in any disaster that will kill us in 10-20 years unless someone comes up with some really hard facts--like perhaps a photograph and orbital plot of the asteroid that is suppost to kill us all. I just remember too many disasters that were to wipe us out in 10-20 years while growing up (oil depletion, population explosion, global cooling, etc)--and we're still alive.
Such ideas are almost always based on linear trends. Just like the guy in the early part of the 19th century who projected that New York would be hip deep in horseshit by the year 2000. That's what the trend showed, after all.
This is not to say that we shouldn't worry about the downsides of technological progress, but for the most part, these "global extinction" thoughts are fueled by accentuating the negative and ignoring the positive.
Bad things will almost certainly happen in the future. Maybe even very bad things. But destroy the human race? Not likely. Slow it down, even? Probably not. The worst global disaster with real evidence behind it we have to face right now is global warming and while global warming could cause a lot of discomfort, with the sea-level rising and weather changing, the human race would certainly survive.
The cake is a pie
The error is in thinking that AI is just a matter of getting enough transisters together. Hardly! The real problems in AI are not hardware speed so much as what to do with that hardware to make it intelligent. This is not a trivial problem. it is an extremely difficult problem, IMHO probably the hardest problem the human race has ever faced.
The question nobody even has a coherent theory for right now is: what would an (artificially) intelligent computer do? What would be its desires? Would it also have emotions? If so, what would it feel?
And this is really the key thing. You can't build an artificially intelligent computer unless you have a damn good idea of those things. You can't build something with desires, emotions, etc. unless you know, in detail, what desires and emotions are, at a far deeper level than we do now.
The cake is a pie
Self-replicating machines? Nanotechnology run amok? Machines that become smart and enslave humanity? Please, this is reality, not an episode of star trek.
Finally, he argues argues, this threat [machinery] to humanity is much greater than that of nuclear weapons because those are hard to build.
HAHAHA!
Please. We can't even write a web browser within three years, much less program sentient robot roaches that could destroy our planet.
There's only like, what, forty thousand nukes extant on earth, each capable of wiping out millions of lives in five minutes? Many capable of poisoning an entire planet for millenia if detonated close enough to the ground? ALL of them are owned by warmongering, jingoistic, pathologically disturbed political entities who have NO QUALMS whatsoever about using nuclear warheads whenever it is convenient?
Nuclear weapons, traditionally developed viruses, lethal bacteria, political unrest, riots, the complete disruption of climate, economic decay, and plain old steel bullets fragmenting children's skulls into explosions of bloody brain and bone (just like the children of Kosovo who the entire world is eagerly attempting to exterminate) are ALWAYS going to be more of a concern to me than sentient computers messing with my tax return. This article sucked. Perhaps the real thing will explain stuff better.
The most dangerous aspect of living on earth is that we are sentient. If we weren't, we wouldn't give a shit what happens in the long run. (which we don't, when it gets down to it)
-troll taker