Indirect Documents At Last

BarryNorton writes "In a world that increasingly takes the WWW, its pages and the other documents we exchange in the electronic world as given - and knights Tim Berners-Lee without an understanding of the pre-WWW background of stateless client/server document architectures (e.g. Gopher) and hypertext (e.g. Xanadu) on which he built - there still beavers away a forgotten figure, Ted Nelson, eager to more fully achieve the original hypertext vision. In recent communications Nelson says: 'The tekkies have hijacked literature- with the best intentions, of course!-) - but now the humanists have to get it back. Nearly every form of electronic document- Word, Acrobat, HTML, XML- represents some business or ideological agenda. Many believe Word and Acrobat are out to entrap users; HTML and XML enact a very limited kind of hypertext with great internal complexity. All imitate paper and (internally) hierarchy. I propose a different document agenda: I believe we need new electronic documents which are transparent, public, principled, and freed from the traditions of hierarchy and paper. In that case they can be far more powerful, with deep and rich new interconnections and properties- able to quote dynamically from other documents and buckle sideways to other documents, such as comments or successive versions; able to present third-party links; and much more. Most urgently: if we have different document structures we can build a new copyright realm, where everything can be freely and legally quoted and remixed in any amount without negotiation.'"

Comments

Trans (complete text)

[BarryNorton]

To respect Prof. Nelson's licensing, it's necessary that I post the whole text, from which I quoted. I'll do so in a reply to this, in the hope that that means it will fold up as comments come in below. (This version is probably the same as the one online, but just to give proper credit, this text was sent to the Advanced Knowledge Technologies (AKT) project, with which I'm partially associated)...

Re:Trans (complete text)

[BarryNorton]

trans© 2005 T. Nelson, stable at hyperland.com/trollout.txt
and xanadu.com/trollout.txt
Permission is given to redistribute this but only in its entirety.

Dear World:

The tekkies have hijacked literature- with the best intentions, of
course!-) - but now the humanists have to get it back.

Nearly every form of electronic document- Word, Acrobat, HTML, XML- represents some business or ideological agenda. Many believe Word and Acrobat are out to entrap users; HTML and XML enact a very limited kind of hypertext with great internal complexity. All imitate paper and
(internally) hierarchy.

For years, hierarchy simulation and paper simulation have been imposed throughout the computer world and the world of electronic documents.
Falsely portrayed as necessitated by "technology," these are really just the world-view of those who build software. I believe that for representing human documents and thought, which are parallel and
interpenetrating- some like to say "intertwingled"- hierarchy and paper simulation are all wrong.

This note is to announce a very special and very different piece of open-source software you can download and use now, for electronic documents radically different from anything out there- and a bigger plan.

I propose a different document agenda: I believe we need new electronic documents which are transparent, public, principled, and freed from the traditions of hierarchy and paper. In that case they can be far more powerful, with deep and rich new interconnections and properties- able to quote dynamically from other documents and buckle sideways to other documents, such as comments or successive versions; able to present third-party links; and much more.

Most urgently: if we have different document structures we can build a new copyright realm, where everything can be freely and legally quoted and remixed in any amount without negotiation.

It's time for an alternative to today's document systems, and we the loyal opposition have a proposal.

>>>Humanists please jump to transliterature.org, since what follows will be somewhat technical.

But first, some background. This will take a while.

BEFORE THE WEB, A GREATER DREAM

Long before there was a World Wide Web, there was a project with greater intent. This was Project Xanadu*, a bunch of clever, cynical idealists who believed in a dream of world-wide hypertext- somewhat like the web, but deeper and more powerful and more integrated, rooted in literary ideas, and mindful from the beginning of the copyright problems that would come. The project started unofficially in 1960 when I began to think about world-wide screen publishing, but grew to involve about a hundred participants and supporters over the last half-century.

(Note that I flip between "we" and "I" because this piece culminates work and ideas shared by a number of others over the decades; but I am presently acting alone, so whenever appropriate I am including those others by pronoun.)

Even from the beginning, we planned on unrestricted publishing of hypertext by millions of people; but web-like documents were only the beginning, only one possible form.

The Xanadu project asked at the beginning- not, "How do we imitate paper?", but "What if we could write in midair, without enclosing rectangles? What new ways can thoughts be connected and presented?" Many ideas and screen maneuvers came to mind, but they always sharpened down to this question:
"How can electronic documents on the screen IMPROVE on paper?" And our key answer was: "Keep each quotation connected to its original context."

This idea (now called "transclusion") is the center of our work and the center of my own beliefs. I

Re:Trans (complete text)

[Dun Malg]

"Written text has the very interesting property of linearity, which matches it to the linear processing of spoken discourse, for which we have hardwired functions in brain. How could you "improve" on that?"

When you start to read the WWW, do you start with TBL's original pages? No, hypertext is something different... so why should this only apply (inadequately) between documents, and not within them?

Because at some point you have to start feeding the brain information in the linear, spoken format it's designed to interpret. Linking and indexing is great for finding information, but not so good for consuming it. When you find the page you're looking for in the encyclopedia or on the web, you stop dealing with indices and hyperlinks, and start reading linearly. That's where the real gruntwork of information comprehension happens. There's no mystical transcendent mode of "uber-literacy" that allows one to absorb information better than the linear, serial way around which our human languages are designed, and for which we have trained ourselves to process since birth.

Re:Trans (complete text)

[Dun Malg]

A page from the OED is a great contrary example - have a look... I don't find a page, then read the whole thing from the first line - there are all kinds of cues (font, font size, colour, indentation) to the ability to read across a document, rather than linearly through it. I go back and forth over these structures within a page, not just to get there.

Christ Almighty, try to understand the greater meaning of my point rather than fixating on the literal meaning of my specific choice of words. When I used the word page, it was not to imply that once we reach the "page" level of organization, we start reading linearly. I used that word because it was a convenient point of similar terminology between encyclopedias and web sites. I shouldn't have to, but I will explain the point I was trying to make: Once you have found the what you are searching for (the encyclopedia entry, the web article, the OED entry, etc.) you start reading in the classic linear, serial fashion. This is the way human language works.

Re:Trans (complete text)

[schon]

We might (for the most part) read a sentence word to word (in fact we don't, if you look at experiments on eyeball tracking by psychologists, but let's ignore that).

Actually, I'd like to address this, because even though your direct argument ignores it, it seems to be included in your bias...

The eye pattern on some people may skip all over the page while reading, but that is irrelevant to the fact that the eye patterns aren't synchonous to the understanding of the written word. Each word has context within the sentences they make up, and each sentence has context within the paragraphs they make up.

This was what I was referring to by posting gibberish. If heirarchy and linearity really didn't matter, then you would have been able to understand what I'd written (as it was, you couldn't even determine the number of sentences or paragraphs.)

not all pages are arranged as series of paragraphs

Which is irrelevant.

HTML tries to force this

No, it doesn't. HTML doesn't try to do anything besides provide a way for the author to mark up their document. It doesn't force you to arrange your text in any way you don't wish to.

Here is where I think you're mistaken; it's the crux of my point, and :

people arrange their HTML pages as paragraphs of text because that's the most effective way of presentation. They don't do it because they're forced to, they do it because they want to.

You seem to be making the assumption that people's online writings are in paragraphs because they're forced to - when in fact it's the other way around: we interpret information in a particular fashion, and people write the way they read.

non-linear paragraphs/sentence in (...) diagrams are just as communicative in some situations. (emphasis mine)

Yes, but this misses two important points: first is my emphasis on "some" - it's not as communicative in *all*, and (as some might argue) that it's not as communicative in *most* situations. Second, HTML in no way forces you to lay out your words in any fashion. People just do it because they want it to make sense when others read it.

As an asize, I find it amusing that your assertions about how things should work are directly contradicted within the first few lines of the article by Dr. Nelson. To whit:

Permission is given to redistribute this but only in its entirety.

If it's so important to have paragraphs (or even sentences) be free-form, why doesn't he want you to post snippets of his article?

why Ted is doomed to obscurity

[Anonymous Coward]

"Most urgently: if we have different document structures we can build a new copyright realm, where everything can be freely and legally quoted and remixed in any amount without negotiation."

Alas, that is why Ted is doomed to obscurity. He has a decent point and then transitions into some hippiesh b.s. that won't even play well to his core utopian audience.

The form should not dictate the comment. And that point is where the techie utopians fail.

Re:New slashdot saying

[BarryNorton]

Sorry, the first sentence is supposed to be the summary. I offered the second part, as a quote, if the editors wanted to reproduce this (as they do for book reviews etc.), but they've chosen to just bang it all together to make one of the longest summaries I've ever seen!

*head explodes*

[Anonymous Coward]

Arrgh... That summary was just waay too abstract for me.
Just give me an implementation of whatever you are thinking of, and I'll try to judge it, OK? :-)

Is a document format the answer?

[ReformedExCon]

The problem is not a lack of information. The primary reason we can't have a fully transparent, infinitely linked "web" is that our puny human brains are incapable of absorbing and filtering that much information.

Consider the difference between Wikipedia and Everything2. Wikipedia is written by people who are interested in the topic at hand, and as such they link to relevant pages that are of interest to them. On the other hand, Everything2 seems to automatically link each "interesting" word to a seemingly random internal E2 page. The result is a useful and interesting encyclopedia in the former case and a jumbled, irrelevant mass of random information in the latter. Although this is just one case, it is very simple to extrapolate this result with any sort of grander version of E2 (e.g. Semantic Web).

What we need is a better way of presenting information and an easier method of linking sites of interest to the data we generate. What we don't need is some way to make everything a link.

This guy is complaining about ideological agendas?

[geekplus]

A few choice quotes from the leader:

"I propose a different document agenda"
There's that word agenda, in the first two sentences of his solution)

"I believe we need new electronic documents which are transparent, public, principled, and freed from the traditions of hierarchy and paper"
Every humanist I know who's objecting to the ways of tekkies (love that spelling) starts off by proposing, "I believe we need new electronic documents". "freed from the traditions" also kinda sounds like someone with, umm, an agenda.

"Most urgently: if we have different document structures we can build a new copyright realm"
This one was priceless. He's going to build a realm. So he can finally call himself a *real* DM...

I believe...

[markov_chain]

I believe we need new electronic documents which are transparent, public, principled, and freed from the traditions of hierarchy and paper.

And I believe that I need 10 million dollars by noon tomorrow. Unfortunately, in both cases, there is a "2. ???" step that needs to be filled in.

A lack of substance

[aussie_a]

Q: You have said that we have settled for less basically. Because I have been brought up with computers the way they are, I can't see this difference or quite comprehend what you are talking about. What would it mean for me if we had what you're suggesting.

[snipped]long ass answer that doesn't answer the question[/snipped]

Q: You haven't answered my question yet. How would life be different for me if we had?

A: I don't know.

So what's this guy talking about? All I can seem to pin down is he wants links to flow both ways (track-backs? Yeeesh. Haven't blogs taught us that these are horrible?) and he wants open-source document standards. Oh, and there's some talk of a license in this, he (again) doesn't mention any specifics, but the impression I get is his "new system" would have all content licensed under the one partiuclar license (which allows people to do whatever they like with it, from what I understood of his ramblings anyway).

He doesn't say HOW this is going to happen, he doesn't mention any benefits to it. Only that it would be a good thing.

Has he been more coherent and specific elsewhere? Or is he always like this?

Slashdot bigotry at it's highest proof...

[Hosiah]

Geez, a couple months ago I dared to suggest (1) that Tim-Berners-Lee was not - in fact - God almighty, and (2) the whole web thing is just one way to do the internet...it's the standard we ended up adopting mainly because, like so much else in the technology field, it was in the right place at the right time. Dozens of other multiple implementations could have formed. For pointing out all of the above, I got flamed from (I lost count) about 20 different directions. Now another guy, who, like me, was hanging around in computer rooms before most of you were out of diapers voices a hankerin' to make a new internet...something (yeah, he WAS kinda hazy on that point). He gets dismissed as a crotchety old man. And neither one of us are even all that old.

Guess everybody is too busy kissing the status-quo's ass to consider that things might change? What, something that's only been around for 30 years is all of a sudden hewwed in stone? Well, surprise, the technology you're married to now WILL crumble to dust eventually, as will your own dear bones, be it in a decade, a century, or a millenium. And other things WILL replace it. Be it by a new twist on an old scheme dreamed up out of some codger's half-gone imagination, or the fresh, new idea of young blood. Momento mori....

Re:Slashdot bigotry at it's highest proof...

[Lumpy]

Welcome to humanity. this is how 90% of your bretheren are.

People resist change. Everyone knew that the WWW and http was nothing but a sloppy hack to begin with, but it filled a hole. coming up with something better will upset everyone that is used to the current standard (see the insane flamefests and screamfits that happen every time a new http standard is proposed) Humanity hates change, really hates having to learn something new and will lash out against anyone even considering changing what they do or how they do it.

IPV6 should have been here 5 years ago SMTP/POP email should have been replaced with something more robust and spoof-proof 6 years ago...

both are still on old broken systems because people do not like to change anything because of good old fear.

Quote Me

[Doc Ruby]

I'm appalled that, in 2005, we still have to jump through hoops to include arbitrary objects in arbitrary documents. Why can't HTML include a tag, with an "HREF" argument, that points at any object at any URL? Like a text object that is maintained by the server, not necessarily the one maintaining the document in which the document is embedded. To do so now, I have to use IFRAMEs, which have all kinds of quirks and cross-platform differences. How about email, where the Content-Disposition MIME header has, since at latest 1997, let us include a message body from an arbitrary URL, rather than always including every (often huge) object inline, such as "attachments"?

While we're at it, I'd like servers to keep a "reference count" of objects they serve, so documents which refer to their objects can (optionally) register. I'd like servers to keep a database of all their referrable objects and their URLs, so an object whose URL changes (moved internally, externally or deleted) can simply return the response code so indicating. Servers like the "Internet Archive" could be much more useful if they accepted archives of low- or old- refcount objects from elsewhere. Other servers wouldn't be able to "disappear" objects without notice, which is extremely important now that publishers often deny some publications that have such an important effect on politics and business, revising them without notice to coverup various deceptions without accountability.

Many of the problems with making and using Internet documents in WWW and email are solved directly with those two "embedded reference" technologies. This Internet is starting to get old, without outgrowing some of its basic limitation. I want to quote any object (or fragment) from any document in any other, without copying it - just include a reference. We don't need to make a quantum leap to Nelson's Xanadu just to get some things right. Where are the versions of Evolution or Firefox that just use these simple technologies to do that?

A perspective on Ted Nelson

[yawgnol]

I've met and talked with Ted Nelson a few times, and I would never presume to speak for him or explain his ideas for him, but I think I can give a little perspective that might help clear up Ted's "thing".

Ted Nelson is personally an incredibly scattered individual, and his whole thought process seems to be like a million mixed-media post-it notes flying around in a tornado through space and time. That is basically why he makes no sense to people (and vice-versa I'd guess). I truly believe that his driving motivation is to create a system of information that WORKS LIKE HE DOES. I don't in any way mean that to be insulting, it is pretty amazing really and I am strongly PRO Ted Nelson. But with that in mind, he needs everything to connect to everything in every single way and be visible from every different angle. In his brain, he doesn't have to leave one program and export his thoughts to another program, and negotiate the copyrights so that he can think properly. And he KNOWS that it's possible, but not too many people are really looking at the big picture. I don't think he's saying there's anything WRONG with the internet, he's just looking about 50 years into the future and wants to get there... sooner.

Remember, this is a guy who thought up hypertext and micro payments at a time when people were literally telling him he was insane. In the next thirty years they went from saying "that could never physically happen" to "even though it's probably technically possible people won't want that to happen" to "oh, yeah, that's obvious and totally unavoidable. Duh Ted. Why are you even talking to me about this ?". So the guy is a visionary and a long term thinker.

Though I do admit that sometimes it seems (like all visionaries) he doesn't seem to have enough respect for the people who are actually creating useful and IMPLEMENTABLE technology. Still, we've been exploring this stuff for 20+ years now, and major "conceptual" advances are just going so unbearably slowly.

So maybe that adds some perspective. It's just my opinion anyway...