Jaron Lanier Takes On "Cybernetic Totalists"

Stefan Jones writes: "VR pioneer Jaron Lanier has written "Half a Manifesto" -- a long and considered rant taking on notions favored by Extropians, Singularity fans and others -- on the exclusive salon for long-hairs, 'Edge.' Lanier believes that the totalists are not only promulgating an irresponsible and inhuman ideology, but indulging in bad science. The site also features fascinating and spirited reactions by a slew of luminaries, including George and Freeman Dyson, Bruce Sterling, Lee Smolin, Rodney Brooks and Kevin Kelly. Good stuff, no matter where you stand on this issue." Oh c'mon -- no one around here would fetishize technology per se, would they?

Comments

Re:Jaron

[Admiral Burrito]

I have a hard time taking this guy too seriously. I'm sure he's bright and all, and may even have some interesting things to say, but he pretty much turned me off in the early 90s when he was going on about how the net would (could?)bring about this "utopian" age.

Read the article. Apparently he's learned from his utopian-fantasy mistakes. The article questions the transhumanist theories/fantasies.

In a nutshell: Computers aren't going to change the world that radically because we don't know how to write the required software.

I'm kind of undecided on that one. Most of the extropian/transhumanist/singulatarian ideas assume strong AI. It's obviously possible to produce intelligence- we are living examples (Aren't we? Aren't we?!?! :). But there there is no clear path from what we have now to strong AI.

I'll take the wait-and-see approach. :)

Re:Huh?

[Alien54]

So far it is not bad at all. Being a technology type, I presume that you are used to reading books without pictures, etc. This definitely requires at least that level of literacy. Also, there is a specialized vocabulary that needs to be sorted out.

Since this is "regular english", people do not realise that the fancy words used there form a jargon of their own. In a tech manual, you know how to handle this. But in this context it is easy to blank out, and watch your eyes glaze over. Snap out of it and grab yourself a philosophy tech manual so that you can at least get a hold on the jargon. [smile]

- - - - - - - -
"Never apply a Star Trek solution to a Babylon 5 problem."

Glorious exercises in hand waving

[w00ly_mammoth]

I used to read Wired magazine some years ago, and I used to take it seriously. After some time, I began to notice a trend in its articles.

There's this clique of "digerati" who keep popping up on its pages and in similar forums/magazines/books, explaining the future in all its robotic nanotech cybernetic glory. The same names keep appearing over and over, repeating visions of a future so vague and full of popcorn sci-fi visions that you can't quite pin down anything specific, but can debate about it for weeks.

You may have seen these names before, waving hands and talking about the amazing future - Nicholas Negroponte, Marc Andreesen (even makes the cover of wired recently, all for having co-written mosaic w/ eric bina), Lanier, Kurzweil, Ted Nelson, Gelertner, etc. Most or all of these have been "has beens", who never quite produce anything useful, except visions of the future that are lapped up by journalists and viewed as the gospel.

Sure, it was interesting to read about futuristic visions of tomorrow, but after 5 years of this crap, and hardly any progress in bandwidth, usability, AI, speech rec, home automation, etc., I have had enough of reading about this thing. Anybody can write vague gee whiz stuff, full of buzzwords that nobody quite agrees upon.

Trust me, after you're read this sort of stuff for a while, it begins to enter the territory of the social sciences - it's always full of controversy, you can never prove anything wrong with it, but its proponents can always make their point with something so vague that it sounds profound.

Give it a rest. It's not even as entertaining as campy 70s futurism.

w/m
PS - Funny how REAL contributors to technology of the future never write articles like this, maybe because they have something at stake? For instance, you don't see crap like this from Drexler, peter shor, or seth lloyd.

And for all the people falling over themselves trying to write serious posts. If you think you're l33t at using buzzwords, try Alan Sokal

Re:Glorious exercises in hand waving

[Wellspring]

You may have seen these names before, waving hands and talking about the amazing future - Nicholas Negroponte, Marc Andreesen (even makes the cover of wired recently, all for having co-written mosaic w/ eric bina), Lanier, Kurzweil, Ted Nelson, Gelertner, etc. Most or all of these have been "has beens", who never quite produce anything useful, except visions of the future that are lapped up by journalists and viewed as the gospel.

I think that the worst of the bunch are the anti-technological lit crit types. Alan Sokal's experiment last year came to mind while I read this, and I am glad you linked to it, since I couldn't remember Sokal's name. Here is the whole archive.

The problem is that the technological process is composed of normative and positive components. Normative components are pure opinion and value setting (eg that education is more important than national security, or that spam is something to be discouraged). Positive components are purely factual (eg that a technology can be projected to earn X dollars for the industry, or that a black hole's event horizon is at radius Y. Or that a component's magnitude cannot exceed that of its whole).

Ideally, we use Positive techniques to achieve Normatively determined goals. Positive methods are evaluated by reason and experiment. Normative beliefs are evaluated using persuasion and politics. We need both! But as Sokal's experiment warns us, we also need to keep them separate. Science, like journalism, is our society's information-gathering apparatus. Politics, religion, the marketplace, and the media (including slashdot) are our decision-making apparati.

It is really, really tempting to try to mix the two. In fact, it is pretty much inevitable, since much of what we think is fact is really widely-accepted opinion. Facts, like opinions, are often in dispute. However, I think much of the lit crit world is intentionally blurring the distinction for two reasons:

1: Facts have a special weight in society. When these become subject to revision, one can manipulate opinions by manipulating the perceptions of what a 'fact' is. Also, anyone can have an opinion. By placing policy judgements into the domain of positive analysis, you make opinions the exclusive province of the Credentialled Academic. Noam Chomsky, for instance, is a giant in the field of linguistics. However, many people who agree with him seem to think that his academic standing makes him a better economist, for instance, than real live economists. This is an extreme example, but often people who are good at describing a phenomenon are considered somehow to be specially endowed with the power to judge something. You don't need a PhD in military history to be able to say that wars are terrible. Or an economist to say that economic growth is good. Walling something away from the generally educated public is not just bad, it is a form of subtle tyranny.

2. The very lit-crit people who are trying to remove the objectivity from science are themselves just trying to move all the sciences under their banner (or as Lanier puts it, 'campus imperialism'). On a more meta level, the lit crit set actually study persuasiveness and persuasion for a living. They are experts in its techniques and uses. So why not make persuasiveness the basis for all scientific discourse? If everything in the sciences becomes a matter of who has the most witty barbs in the social scene, or whose critiques are the most cleverly worded, or whose syllables-per-word average is the highest, then of course the lit crit people would win. You can't blame them for trying.

Our prototype for the Real Scientist should be Richard Feynman. He was a total iconoclast. He was a fiercely creative, but intellectually disciplined person-- willing to throw almost any notion away in the face of hard evidence. He wasn't political, and resigned from the National Academy of Sciences because he saw them as an organization devoted entirely to determining who was worthy of being a member. I'd say that that describes the lit crit crowd pretty well. And while everyone has opinions, when it comes to science and factual data, you have to bend over backwards to ensure your own objectivity. Lanier's article challenges one part of this threat, and I hope that people recognize the problem.

Re:Subjective Experience

[Tackhead]

> [because the things we build out of silicon today break down sometimes, and thus don't aren't the idealized Turing machines we claim they are] We kid ourselves when we think we understand something, even a computer, merely because we can model or digitize it.

Nope, we don't understand conservation of momentum because we can't build frictionless surfaces. The pucks on the air-hockey table aren't good enough.

Nope, we don't understand Newtonian gravity because we can describe it by an inverse square law.

Nope, we don't understand special relativity because we can describe how clocks slow down when put into accelerating frames of reference.

Nope, we don't understand consciousness when we can model it and build a machine that passes the Turing test.

The first two statements ring hollow because we developed the technology to test our models. The third statement is untested because we have neither the models nor the technology.

I said untested, not untestable.

To make a long story short, a cybernetic totalist believes that at some point, we will develop models of consciousness that allow us to describe machines that posess it, and that we will also develop technology that will allow us to build said machines.

I'm one of 'em. At present, I take that last paragraph on faith - I'm on the Minsky side of things; a brain is a computer made of meat. It should be possible to build one.

Where I take issue with Jaron Lanier (aside from his IMHO preposterous assertion that not being able to build "ideal" computers equates to our not understanding computer science!) is that he believes that CTs see the eschaton as immanent. I don't. I see it as a possibility, but we have so many serious technological hurdles to jump over between "here" and "there" that I don't worry about it. Building nanopaste is highly nontrivial.

(I do agree with him that the blind acceptance of criticality is a problem - I'm also one of those people who "looks forward to it" - but even I acknowledge that criticality might not be as good a thing as I think it will be.)

So finally on to subjective experience.

I have never seen a credible argument that subjective experience doesn't exist. I believe it exists. I experience it 24/7.

But likewise - I have never seen a credible argument that subjective experience requires anything other than a sufficiently complex network of inputs, outputs, and some sort of feedback going on in the middle.

I can't explain how that works. Lanier would call it "soul". I call it "mind", and view it as an epiphenomenon of "brain". The fact that it is an epiphenomenon of "brain" doesn't make it any less real.

When Jaron says "But don't you experience your life? Isn't experience something apart from what you could measure in a computer?", I'd counter with:

"Yes I do, but as for the second question, I don't know, because I lack the tools to measure experience. It may be, as suggested in Godel, Escher, Bach, that my brain lacks the capability to understand said measurements, owing to some Godelian "loopiness" in that it's hard to emulate a brainputer on a brianputer. But at present, that hypothesis is untested, and I have to proceed as though it were measured.

As for ad hominem arguments, the notion that Turing developed the notion of machine sentience in order to deal with his own personal anguish is almost beneath contempt (I say "almost" because on Edge, "where ideas come from" is a legitimate topic for discussion), and I won't dignify it with a reply. Whatever the origin of the idea, the idea is IMHO valid, and Lanier does himself a disservice when attempting to criticize it on the grounds of its origins, not its merits.

Feh. A long rambling rant from Tackhead.

LionKimbro, regardless of our agreement or disagreement about Lanier's paper (I, too, found it a wonderful read; while I've taken a few slices out of him here, it's an excellent articulation of the non-CT point of view, contains much that is of merit that I've ignored in this post, and provides lots of food for thought), we're in solid agreement on one thing:

Those of you taking pride in not understanding the paper should seriously reconsider your position. Ignorance is not something to take pride in.

To those saying that Edge is "just a load of intellectual crap", would you also agree with some skript kiddie saying "That RMS guy at gnu.org, hes stupid, all he duz is write lots of stupid essays talking big intellectual crap about free beer versus free speech! Wut da fuk he talking about? N-E-1 with cl00 know that free software means #warez, d00d!!!"

Damn straight.

[Electric Angst]

This is a wonderful article. I remember my first experience with "Extropians" was at a dinner with some of the members of a large, international Extropian mailing list. I think that I realized exactly how far off the ideology was when I sat there and watch three men badger another man to sign a "Cryogenics Contract" where he would agree to be frozen when he died. Not only was it bad science, but it was the same kind of groupthink atmosphere that permiates the type of movements I was trying desperatly to get away from.

On a related note, you might wat to check out The Guy I Almost Was by Patrick S. Farley. I think that it gives one of the best descriptions of techno-fetishism among geeks, and the ways that we are being manipulated by it... (Also, it's a damn good story.)
--

Re:This man is right - heed his warning

[Mr. Slippery]

What other reasons can there be for what amounts to turning yourself into some(one|thing) else?

Most of us want to turn into something else. If I was exactly the same now as I was at the age of twenty, ten years would have been wasted!

We want to learn new things; we want to improve our bodies; we want to change elements of our personality and emotional makeup. That's why people go to school, read books (or even /.), go to the gym, try different diets, see therapists, take anti-depressants, whatever. Many of these means are misguided, and one might even argue that the goal itself is a mistake (after all, you are already a Buddha), but there's no dening that part of the human condition is the desire for personal transformation.

It's a scary prospect for the future, one in which people will willingly rush wholesale into abandoning their humanity for the promise of an artificial dream.

This doesn't have to be some Faustian bargain. Using nanotechnology to augment my physiology doesn't take away any of my "humanity"; nor would providing me with an AI butler/manservant/gal friday.

However, the techoskeptics are right about at least one thing: it isn't a given that such technology will be sued wisely. The example given about people borrowing money they don't need in order to give good inputs to a credit rating algorithm is an excellent one. OTOH, that's not so much an issue of technology as of bureaucracy - the algorithm could just as well be implemented by men with ledgers and quill pens as by big iron.

Re:Entirely different kinds of boundaries

[JimMcCusker]

I have this doubt that human beings are the optimum form of life, and if we are, then that's pretty sad.

Of course, that's assuming that there is an optimum form of life. Most people look at evolution and natural selection as having some purpose or goal. It doesn't. It's simply organisms reacting with the environment, trying to continue to exist. There is no optimum form of life, just like there is no external meaning to it. Meaning and goodness are applied to life by intelligence, trying to make some sense of it. There is nothing intrinsically better about being a human, or being a cyborg. If a human thinks they are better off being a human, then that's fine. If a human thinks that they're better off being a cyborg (or a pure computer) then that's fine. Just don't force everyone else into it! This singularity may be able to think faster, or be more creative, or whatever. It doesn't matter. Progress is defined by ourselves. It's not needed, and it isn't even really very important. It's just a filter through which we see change. Yes, it's a good way to measure movement towards a goal, but always remember that these goals are never external, never fated, always created by ourselves.

This man is right - heed his warning

[flatpack]

As enamoured as we all are with the state and progress of modern computing I think we need to take a step back and really examine the underpinnings of our beliefs about computers and the very paradigm of computing that has pervaded so much of our cultural and scientific thinking over the last few decades. This article raises some excellent points about the things that many of us hold to be self-evident, even when we don't conscioussly think about them in these terms, and these beliefs affect the way we think and act in everyday life.

The "eschatological cataclysm" that Lanier talks about occurring in the near future is truly a bleak picture for mankind as a whole, and yet it is one that I see talked about as if it were inevitable and a good thing. Just because we like computers are we really ready to throw away our humanity for a set of perceived benefits? We are what we are, and any gross changes in the state of our existance cannot help but fundamentally alter who we, as a species, are. There is no "intangible transition" between what we are and what we will be.

I personally am rather fond of the current modality of human existance, and I see anyone that believes in or even worse desires such a dramatic and unnecessary change as being at best a foolish idealist and at worst someone unable to accept the reality of their own existance. What other reasons can there be for what amounts to turning yourself into some(one|thing) else? No combination of bio/nano/computing technology is ever going to be me for the simple reason that it will have different boundaries from me, and our idea of self is shaped from these boundaries.

It's a scary prospect for the future, one in which people will willingly rush wholesale into abandoning their humanity for the promise of an artificial dream. I truly hope that I won't be around to see the fragmentation and then the end of humanity's promise.

Re:This man is right - heed his warning

[interiot]

Our boundaries are expanding all the time. Ability to go really fast. Ability to fly. Ability to see and measure distant stars. Ability to leave the earth. Ability to sort through an entire library of information in a second. Ability to send a message at light speed to the other side of the world. Ability to karma whore from thousands of people all over the globe.

Okay, it's a large leap from "our boundaries are constantly expanding" to "it's okay if our only boundaries are memory and CPU usage". But, unless you give a strong argument as to why we should force our boundaries to remain static, I don't know why it would be bad to change them arbitrarily.
--

Slashdot confuse-o-vision?

[British]

OOh, -1 penalty for not mentioning "The Well" or "Abbie Hoffman". C'mon, there's about 8 words here I have never seen before. Does ANYBDOY understand this? If so, please post a translation.

Stefan Jones writes: "VR pioneer Jaron Lanier has written "Half a Manifesto" -- a long and considered rant taking on notions favored by Extropians, Singularity fans and others -- on the exclusive salon for long-hairs, 'Edge.' Lanier ....

metaphor is easy; science is hard

[sethg]

Back in the Victorian era, a number of highly respected physicians said that a woman who want to college would become infertile, because her brain would become more developed at the expense of her reproductive organs. Why did otherwise intelligent people fall for such bogosity? Because it used the law of conservation of energy -- one of the, ahem, hot new scientific theories of the age -- as a metaphor for what was going on in a woman's body. These doctors treated the metaphor as sufficient proof for the theory, instead of looking for hard evidence that would demonstrate its truth or falsity.

I think a lot of flag-wavers for memetics, evolutionary psychology, etc. have fallen into the same trap. It's easy to construct a Just So Story to link your pet idea with the latest scientific trend, and then you can plaster your story all over the Internet and accuse doubters of being trapped by outmoded ways of thinking. It's much harder to do your homework, collect evidence that supports your theory or refutes alternatives, and then convince skeptical and educated peers.
--

causes of techno-infatuation

[peter303]

Why do people project more into technological innovation than may be there?
(1) $$$$$ Money attracts hype. People call themselves prophets. Not all that different than new age religion, health fanatics, etc.
(2) Generational rebelliousness: young guys understand tech and old farts don't. Nah-nah-nah-nah-nah.
(3) Religious instinct: people search for the ultimate beyond themselves. The old religions are dead. Technology has the answer.

What the guy is talking about.

[Apuleius]

Extropians: Randists, only
more so.

Singularity: Explained here.

Read all of that and digest. It's fun.

Subjective Experience

[LionKimbro]

I am elated to finally find a paper that so clearly elucidates my position and observations. I am also saddened to realise that people who would benefit the most from it are also the least likely to read it and understand it.

3) That subjective experience either doesn't exist, or is unimportant because it is some sort of ambient or peripheral effect.

Item 3 in particular hit home; I have had the exact same conversation and thought process ("Perhaps the person I'm talking with doesn't have a subjective experience?"), for the last five years. The last time I had it was a few days ago while talking with a fellow engineer here at LithTech.

Subjective experience is not an easy problem; in fact, it is a very hard problem, but there is something in too many scientist's minds that makes them want to treat the subject as a superstitious topic, and treat those who find subjective experience difficult to fit within a computational framework as religious or spiritual zealots. Larson has correctly identified the currently popular model of the world, "Cybernetic Totalism."

By the way; Not understanding his paper is not something to be proud of. Ignorance about *anything* is not something to be proud of. Use a dictionary or a search engine, whatever it takes, and understand these words.