Slashdot Mirror
Web Pages Are Weak Links in the Chain of Knowledge
PizzaFace writes "Contributions to science, law, and other scholarly fields rely for their authority on citations to earlier publications. The ease of publishing on the web has made it an explosively popular medium, and web pages are increasingly cited as authorities in other publications. But easy come, easy go: web pages often get moved or removed, and publications that cite them lose their authorities. The Washington Post reports on the loss of knowledge in ephemeral web pages, which a medical researcher compares to the burning of ancient Alexandria's library. As the board chairman of the Internet Archive says, "The average lifespan of a Web page today is 100 days. This is no way to run a culture.""
Really, is there a reason to archive everything in the world? Sure, your 4 year old has some pretty drawings, but should they be put in a library someplace?
100 years from now, should anyone be forced to accidentally stumble over goatse? (which is very disturbingly archived on archive.org)
...which means that with that ISBN I can refer to the book and find it at libraries or bookstores. Why don't we setup a sort of unique web page number if articles of interest or knowledge are published there. Then it would be easy to track an article if its moved to another site or whatever just by looking up a sort of catalog for these numbers.
honestly, the transient nature of webpages makes it an unsuitable medium for the long term establishment of "culture" our categorization happy, buzz-word ridden nature so commonly prevalent will have to find a new term for what is the web. boo-freaking-hoo.. meanwhile i'll keep doing my thing, posting pics for my family to see, putting calendar events up on the web so my homebrew-club will know when we're meeting and not worry about any "culture" i might be potentially creating then destroying when i take stuff back down.
man i need coffee, insomnia is a bitch...
You probably shouldn't be quoting any kind of "Bob's World of Great Scientific Insight" type pages anyway. I mean, the majority of sites that go under in less than 100 days are the one person operations that one should identify as bad sources anyway. So it might seem obvious that quoting someone's blog in a research paper is just a plain stupid idea, but it happens way more often than you might think.
====
Crudely Drawn Games
People are worried about losing the information on the web: but all that is really happening is that the URLs are no good after a while, you lose the snapshot. The information is not necessarily going anywhere. If there is a need or a want, someone will throw it up, or another will host it. That's the beauty of the web, you get the good with the bad, but time has a way of getting rid of the chaff.
;)
What would be interesting would be a website that archives those snapshots for posterity. Well, what do you know, there are several such sites already! Looks like we're in good shape. The sky is not falling.
Auto-reply to ACs: "Truly, you have a dizzying intellect."
It's not just the short lifespan of a webpage... it's also the fact that the source isn't always reliable. Web publications are rarely given the same strict editorial process as most journal articles. The content might be just as good - or better - but they're also not given the same credibility.
I'm a recent grad of a University... my freshman year, profs wanted us to start using the Internet more so we were asked to submit at least x number of references from Internet sources. By my senior year, they were trying to get us to stop using the Internet. Using a URL as a reference was sometimes forbidden by the professor.
That matters in part because some documents exist only as Web pages -- for example, the British government's dossier on Iraqi weapons. ...
"It only appeared on the Web," Worlock said. "There is no definitive reference where future historians might find it." Much like the WMDs themselves then
Music is everybody's possession.
It's only publishers who think that people own it.
Fuck Beta
~John Lenno
Any extra effort required to make web pages and their URL's preserved for eternity makes it more difficult for people to create them in the first place, which will mean less knowledge available, not more. Something unobtrusive that goes around preserving pages for posterity, like the Internet Archive, is the best soplution.
Energy: time to change the picture.
This is why every time I use a web reference I make a hardcopy of it and include it in my research folder. It did not take long for me to figure out that web pages are no more useful than manufacturer catalogs - once the year is up, you might never get that tidbit of information back. If it's too large to want to print, I'll hardcopy the couple of pages I need, and PDF the whole thing for digital storage.
Having a hardcopy (1) documents the information and it's (purported) source, and (2) allows offline access for comparison and validation.
Is it just my observation, or are there way too many stupid people in the world?
besides if you want to see old pages just go the the the wayback machine. Between that and backup tapes, everything you ever wrote still lives (in many cases I wish it didn't !).
The grass is only greener, if you don't take care of your own lawn.
I've personally been working (internally so far) on a website of modern-day Orthodox-Jewish responsa to various issues of Jewish law, so this is an issue I've given some thought to.
To say this is some kind of problem specific to the web is misleading. There are old, well-quoted sources of Jewish thought whose texts are simply lost to us in this current day and age. Example: a famous and extremely popular commentary on the Talmud and Torah, Rashi, is missing for at least a few chapters of Talmud. That would be the equivalent of IEEE misplacing some standards papers and then NO ONE having copies, just lost to the sands of time. Yet it did happen, proving this at least _was_ a serious issue.
However, these days, with such things as the Way-Back Machine and Google caching, actually LOSING entire web pages doesn't happen very often, and, I'd bet, it happens far less frequently than the loss of books.
-Erwos
Plausible conjecture should not be misrepresented as proof positive.
i really hope you have some evidence to back that up.
Nah. There was a time when only very very few could even read, let alone write, let alone keep any kind of records...
But get your point. Too bad there are some restrictions on copying the web pages you are referencing...
There should be some service, a bit like google's cache, you could use to store the referenced pages. I submit the page to the service, then provide two links in my own document, one to the original page (which will likely expire eventually) and one to the cached version. I wonder if they could get around copyright issues the same way google cache gets around them, even though this is a bit more permanent storage than google cache... Most web page authors certainly would not have any problem with having their pages archived there, quite the opposite, most would be happy to have their work referenced by others...
Ummm, maybe only as applies to this topic, which is to say that web pages are a poor place to keep records.
I'd contend that researchers & scientists in general would be quite silly to site an electronic-only resource in their publications, because the persistence of that resource relies on too many factors (the whim of the webmaster, backups or lack thereof, fiber seeking and grid seeking backhoes, etc).
I think that will all sort itself out and real scientists will continue or return to citing more traditional resources.
What I think is much more disturbing and disruptive is the pseudo-science and mis-information that is overly abundant on the web. Too many web sites, personal and commercial, spout 'facts' in such great detail that they have the appearance of authority. Too often, novice/amatuer scientists can be seriously mis-lead by some of the crap that can be found on the web masquerading as 'science'.
I had some evidence to back it up but all the links are long dead ;P
I found that out years ago.. :P
:)
From a researcher's perspective, I used the web primarily as a quick "google" to get some ideas on where I might do further research. For instance, while a particular paper may have been taking offline regarding my search, many times the search will proffer an author's name. Take that name to the library's database (or googling it, too), and you might can get a list of more publications that the author has penned. Even better: sometimes, you can get a valid email address from other links and you can write and ask the original researcher himself about various publications, many times they have copies on hand and can send them to you. My research involves the web, but does not end with the web, which is where many people find themselves hung.
Hey, guys. See that big building with those obsolete books? Lots of chicks hang out there.
If you were me, you'd be good lookin'. - six string samurai
Usability expert Jakob Nielsen addressed the issue of linkrot in a column already in 1998: Fighting Linkrot.
That was why Tim Berners-Lee wanted URL to stand for ``Universal'' (not Uniform) Resource Locator.
;)
The problem is, few people have formal training as librarians, or understand how to file away a document under such schemes (whether or no pages like this are worth preserving is another issue entirely).
Then there's the technical issue---where's the central repository? Who ensures things are correctly filed? Who pays for it all?
With all that said, I'll admit that I use Google's cache for this sort of thing---it lacks the formal hierarchy, but the search capabilities ameliorate this lack somewhat. It does fail when one wants a binary though (say the copy of Fractal Design Painter 5.5 posted by an Italian PC magazine a couple of years ago).
Moreover, this is the overt, long-term intent behind Google, to be the basis for a Star Trek style universal knowledge database---AI is going to have to get a lot better before the typical person's expectations are met, but in the short term, I'll take what I can get.
William
Sphinx of black quartz, judge my vow.
Why would we want to archive 99.9% of today's web content ?
Does anyone archive CB radio traffic ??
It's not a permanent storage medium, never could be, too many points of failure between your screen
and the server holding the data.
Anything worth publishing digitally should be recorded in a more permanent medium.
I constantly backup all my digital photos because they are important to me. I also print the best ones for placing in photo albums, distributing to friends, etc.
The website they are published to is just a delivery medium, and not even the primary one. It can disappear and I wouldn't care. People who know me can always get access to them. Scientists should view their work the same way.
Nostalgia isn't what it used to be.
Printed media, while having a low data/pound ratio, has managed to survive and span generations for centuries. I think the need for paper libraries cannot be forgotten. The challenge is distilling out what is worth keeping, and this challenge is better met now rather than later because we have more or less a good idea of what is significant information, and what is crap.
The ephemeral nature of the web is a very real problem, but it's important not to overstate it. The reason so much more information is lost these days is partly a reflection of the fact that we produce so much more of it. The Library of Alexandria was the distilled knowledge of an entire civilisation; it was unique, irreplaceable and massively important information. The web is full of information that is of low quality, often massively redundant (thousands of pages explain the same thing in different ways) and certainly replaceable (the web is not the final repository of the information: it's a temporary place where that information is published). In the same way, for centuries, newspapers have produced thousands of redundant issues with a lifetime of just a few days. The reason no one decries the loss of our newspapers is because the publishers themselves still archive the information, even if this is somewhat hard to get to. The same is true of web pages, only the number of publishers is vastly larger.
:-)
Individual newspapers had their own ways of making their archives public (in many cases for a fee) because storing that information is a cumulative, ever-increasing cost. On the web that cost is much lower, but still present. In addition, there's the question of relevancy: www.mysite.com/index.html may contact valuable information, relevant enough to be on the front page today, but in a week's time you don't want it to still be there. So what we need is archiving, for the web.
But manual archiving is inefficient and a pain to maintain, since it involves constantly moving around old files, updating index pages, etc.. Plus linkers don't bother to work out where the archive copy is eventually going to be: they link to the current position of the item, as they should.
So what the web needs is automatic archiving. One way to do this (a solution to which was the partial subject of my final year project at uni) is to include additional a piece of additional metadata (by whatever mechanism you prefer) when publishing pages; data that describes the location of the *information* you're looking for, not the page itself. So mysite.com/index.html would contain meta-information describing itself as "mysite news 2003.11.23 subject='something happened today'". User-agents (browsers) when bookmarking this information could make a note of that meta-data, and provide the option to bookmark the information, rather than the location (sometimes you want to bookmark the front page, not just the current story). Those user agents, on returning to a location to discover the content has changed, could then send the server a request for the information, to which the server would reply with the current location, even if that's on another server.
Of course, this requires changes at the client side and the server side, which makes it impractical. A simpler but less effective solution is for the "archive" metadata to simply contain another URL, to where the information will be archived or a pointer to that information will be stored. This has the advantage of requiring only changes to the client-side.
Suggestions of better solutions are always welcome
The article states that the average life for a website is 100 days, but wouldn't journals and formal publications (the most often cited documents in research) last longer than the average? Also, is the average skewed because websites are more likely to contain 'current information'? "Average lifetime" is misleading, does this mean the average time the page stays the same, or the average time before the information in the page is unavailable?
Then DOWNLOAD the pages from your web citations.
For example, a short time ago, I did a white paper on power scavenging sources. About 1/2 the articles I read were HTML or PDF sources. Rather than just citing the URL, I downloaded/saved every online article I referenced. If someone wants the source and cannot find it, I'll just provide it to them. If your paper is going to be read by a number of people, it makes good sense to have those sources on-hand; it never hurts to cover your arse.
Hard drive/Network/Optical space is virtually unlimited, so storage isn't a problem. Paper journals are archived by most libraries, anyway, so until they start archiving technical sources, I'm going to have to do my OWN archiving.
That's a fairly reductionist view if taken too far. Not all researchers are tech whizzes (no pun intended), and I've seen a number of, in my case, professors of English Literature who run the same sort of, "Throw up ten pages with Under Construction signs, test publish a few papers, and let the site sit for years, one day to mysteriously disappear," web site lifespan that "Bob's World" might as well.
Perhaps even more interestingly, it doesn't always really matter if you've done great, repeatable research in the "soft science" fields or outright humanities. You don't have to be a literature expect to have a good insight on "Bartleby the Scrivener". A grad student's blog, as an example, might contain excellent contributions to the conversation.
Now that said, in the context of the article -- dealing with "a dermatologist with the Veterans Affairs Medical Center in Denver" -- I would tend to agree with you heartily. Hard science needs to pull, in my layman's view, from research that the article's author researched well enough to see that it wasn't a few 0's and 1's that might be pulled later, in general.
And heck, what's the harm in saving the pages on your drive and contacting the original author if they disppear? Hard drive space is cheap. If you take yourself seriously, you might want to grab a snap, even if it is technically illegal (not that I know that it is; Google seems to do it right often).
It's all 0s and 1s. Or it's not.
May I remind everyone to read and understand TimBL's Cool URI's don't change. It's not that hard to design systems where you do not have to change the URI every 100 days, folks.
Employee of Inrupt, Project Release Manager and Community Manager for Solid
Proper URL citations include the date. I'm not worried so much about the page being taken down (since it is presumably archived), as much as changing. If you don't record which version your were referring to, the content can change dramatically.
:w
I started posting usenet in the late 1980s. These g*dd*mn things are still are still on the net. I was less guarded at that time. Everyone *knew* them becase disk space ws so scare that usenet postings would disappear in 7-14 days.
I definitely wouldn't trust someone named "Horny Smurf" enough to click the link.
"If you think you have things under control, you're not going fast enough." --Mario Andretti
it's not simply webpages that are the problem. it's digital storage in toto.
because we as a generation are quickly moving away from our previous long-lived forms of storage, and toward digital management of archives, it's trivial for someone to decide to unilaterally delete (not backup?) a whole decade of data in some area of our history.
i remember the photographer who found the photograph of bill clinton meeting monica lewinsky 10 years ago. he was in a gaggle of press photographers, but nobody else had this picture because they were all using digital cameras and he was still on film. most of their pictures from that day had been deleted years ago since they weren't worth the cost of storing. but this guy had it on film.
yes. websites are disappearing. but there's a greater problem lurking in the background. the cost of preserving this stuff digitally, indefinately. who's going to pony up the cash for that? unfortunately, no one. and we'll all ultimately pay dearly for that... (hell -- we already have trouble learning from the past.)
Easy come, easy go... here's another cliche: Give and Take. What's great about the web is that it has effectively demolished the barriers to entry in publishing. Everybody and their grandmother has a blog now - you can't compare webpages to magazine articles or newspapers. There's just so much more information being published now that its average lifespan is bound to go down. So what?
Publications that cite [web pages] lose their authorities? Who the hell told you to cite a webpage? Might as well cite a poster you saw downtown. If the webpage is a reputable source in the first place, it'll keep it around permanently. Still better than scientific journals that are squirrelled away in the basements of university libraries - anyone can get to a webpage.
This is no way to run a culture. Last time I checked, nobody ran our culture... It kinda runs itself. The proliferation of accessable, ephemeral webpages over permanent, priveliged paper publications (wah, too many p's!) is a sign that our information culture has moved on into a new era. Liked the old one? Tough! Now information has to maintain its own relevance in order to be permanent... and I for one welcome that change.
-3Suns
~~~~
The Revolution will be Slashdotted
Look at DSpace, the mission of which is "To create and establish an electronic system that captures, preserves and communicates the intellectual output of MIT's faculty and researchers."
Each data set (collection) has a handle, suppoosedly longer lasting than URNs. We're talking about long term data storage here.
There's an implementation of it at Cambridge University, and my organisation will be evauluation it as soon as the SuSE Linux Enterprise Server software lands on my desk and I've installed my server.
Tom.
Oh arse
As the board chairman of the Internet Archive says, "The average lifespan of a Web page today is 100 days. This is no way to run a culture."
To the contrary, I think this is highly typical of the culture we have today, where everything is a transient fad in the media, technology and politics.
And it is also self feeding, I think, since market forces need to clear out the old to make room for the new in order to meet sales forecasts and shareholder expectations. And this is very true for pop, news and technology, which explains the lack of staying power of pop icons these days and becomes interesting when you want to ask yourself if you really need that new 3GHz machine just to surf the web.
And it is highly convenient in politics where a politician doesn't have to be accountable for what he said 100 days ago.
And so, the lack of long time life on the web is simply symbolic of all the rest here really, even if it is highly questionable.
Use genguid (or other tool) to make a globally unique number
and place that number at the bottom of your
page a link with google's "I'm feeling lucky"
searching for the GUID.
Well, I guess we know what Paul McCartney will be doing on the cover of his next album..
Webpages aren't replacements for books. Or rather, you shouldn't use them that way.
If they're lasting on average 100 days, that puts them somewhere between transient culture, like spoken conversation, and printed culture, like newspapers. Big deal.
We want to preserve culture for future generations, no doubt. But we don't want to preserve all culture for future generations. Anything that is lasting for 100 days and isn't being persisted... well, relatively that's not worth much to future culture.
I don't remember the exact saying, but there is a Native American saying to the effect of "We don't write things down. If we don't remember it, it's not worth remembering." Now, they're not the last word (no pun intended) in wisdom traditions, but there is a certain amount of enforced vitality necessitated by forgetting the details.
We'd better get used to the idea. We're only going to be forgetting more and more of the details as we generate more and more useless information.
URIs don't provide content-based addressing (like a hash of the document). They rely upon trustworthy name registrars, which is an assumption that might have been valid when Berners-Lee was doing his early work, but is not now. They rely on someone willing to continue hosting the original document -- not necessarily the case.
You can link to a article which is then changed by the original publisher (or someone else). With scientific papers, you can't do that -- and such behavior is probably not desireable.
On the up side, if you're currently using cited references, you should be able to build such a system without too much problem -- follow links to PDFs or automatically crawl HTML documents (and check images) and serve all papers that you refer to with your paper. It'd be big, but it provides better reliability than do current paper schemes.
Another feature that might be useful is signing of the content (assuming RSA doesn't get broken in the future).
Basically, if you put up a SHA-1 (Gnutella), MD4 (eDonkey), or similar reference, you can host the original referred-to documents as well as the original host.
If Freenet didn't have as a specific drawback the inability of someone to guarantee that a document remains hosted as long as they are willing to host it, Freenet would be a good choice for this.
One possibility is that, with a bit of manual work, one can frequently find an academic work by Googling for its title. At least for now, as long as you host the original papers as well, Google should pick up on this fact. Of course, it does nothing to prevent modification of that paper by another party...
A good system for handling this would be to have a known system that is willing to archive, in perpetuity (probably hosted by the US government or other reasonably stable, trustworthy source [yes, yes, cracks at the US government aside]). This system would act like a Tier 1 NTP server -- it would only grant access to a number of other trusted servers (universities, etc) that mirror it -- perhaps university systems -- which would keep load sane. These servers (or perhaps Tier 3 servers) then provide public access. Questions of whether there would be a hard policy of never removing content or what would be allowed (especially WRT politically controversial content) would have to be answered.
There could be multiple Tier 1 servers that would sync up with each other, and could act as checks in case one server is broken into. I'm partial to the idea of including a signature on each file, but I suppose it isn't really necessary.
Specific formats could be required to ensure that these papers are readable for all time. Project Gutenberg went with straight ASCII. This would probably have to be slightly more elaborate. Microsoft Word and PDF might not be good choices, and international support would be necessary.
May we never see th
Contributions to science, law, and other scholarly fields rely for their authority on citations to earlier publications. The ease of publishing on the web has made it an explosively popular medium, and web pages are increasingly cited as authorities in other publications.
;)
:)
For true scientific work, this should never happen. Because you should only cite reviewed sources. Such as books, articles or conference papers. This is no guarantee for quality, but at least the review process sorts out the most obvious nonsense. And, if the reviewer is good, it may even increase the quality of the work. Plus, those sources are permanent.
As always, there are sources that are more respected (IEEE, ACM etc.) than others. And using respectable sources is a good thing, because normally you want to prove a point and you base your argument on those publication. So if your basis for your argument is faulty... well
Furthermore, there is hardly any information that can be found on the web but not in a reviewed form. Note that there are (accepted) scientific reviewed journals using the web for publishing. Without a printed edition. And you can quote them. And, as many before me have said, the articles and links do not vanish (the URL is usually not quoted anyway - these articles are listed just like printed articles).
This is just my personal opinion on scientific work. Let's see if my head is still on my shoulders tomorrow
My cats ate my karma. They also wrote this comment.
Maintaining a links page for my wife's business' site has always been a low priority, and finally, I put up a MySQL/PHP page to do the majority of the work.
So I've been going through all the old links, and every link request we've gotten in the business' 7-year history. Of the 120 messages in the timeframe of 1997-1999, only about 15 sites still existed. Of those, two-thirds had forwarded URLs -- often from AOL or Homestead to their own brand. A couple still existed, but had totally different content.
Many just plain didn't exist at all. A fair chunk found the server, but no such page. A few had blank pages or nearly no content. The true annoyance though, is the number of domains that are owned by spamdexers/linkfarms that have no content of their own and beg you to set your homepage to them.
I've still got to cover the rest of 2000-2003 link requests, but I expect that anything pre-2001 will be very sparse.
Design for Use, not Construction!
I'm amazed that anyone doing a professional article would even think of citing a web page as a web page.
Why not just print it out?
Not only are web pages transient, but the facts they have are subject to change. This gets back to your "pseudo-science and mis-information" comment.
If you're going to use it in your work, print a copy or save an image of it or something.
Which brings up to "fair use" and copyrights and all kinds of other crap.
An easy system would be for a server to provide each document it houses with a unique meta-data identifier. Then, when a document, story or paper moves from the "main page" into an archive section, you can still refer to the FileID. This ID should be searchable, so that an article could be linked via something like:
http://www.cnn.com/?2001EXCJA2
The IDs could be system generated and handled by a file system that supports meta-data or they could be designed to mean something and handled by a content management system.
Implementation is the difficult part. Getting everyone - or at least news sites, magazines, and colleges/universities - to set up FileID searching and then document the linking process on their site is no small task.
That what was all this school was for... to teach us how to solve our own problems. -- janeowit
This is a real problem. When Vannevar Bush conceived the Memex system, his goal was to facilitate the exchange of scientific research. Later, Doug Englebart built on Bush's ideas as did Ted Nelson (the guy who coined the term "hypertext") and Tim Berners-Lee. While the web today has become a vast sinkhole of pop-up ads, crappy web stores and inane blogs it is important to not forget that its inception was in aiding scientific research.
Yet, that is not possible without some kind of permanence. Probably what is needed is some way to integrate the web into university library collections. If there was some way of indexing web pages the way libraries currently use the Library of Congress scheme to index their physical collections, then web pages could be uniquely numbered with this number incorporated into the URL. If then universities and the Library of Congress itself were to mirror (permanently) these pages, if the original URL were to become unavailable, one could try just about any manjor university or the LOC and retrieve the page. Of course, with the current political climate here in the US I don't forsee this ever happening.
If we are to say that not everything is worthy of archiving, who, then, is to decide what is? The 'net shouldn't be just another memory hole when there is the potential to create a respository of information that far exceeds the scope of anything possible before. That said, people who wish to cite to information published in an electronic form should be careful to cite only to sources that are reputable not only for veracity but also for longevity.
There was Cowboy Neal at the wheel of a bus to never-ever land.
Do you think because you print it out it suddenly becomes a more stable reference? Sometimes people doing professional articles have to cite web pages because that's where the information they are talking about is.
Can I mod something +1 Scary if it's true but I wish it weren't?
You make a good point about the abundance of mis-information on the web, and that's another problem that needs to be looked at, but I disagree with "this will all sort itself out and real scientists will continue or return to citing more traditional resources." We have an incredible resource here (the internet) for diseminating information, and to ignore it would be something that's really not going to happen. We need to solve problems like this so we can take advantage of the benefits offered by the internet.
Can I mod something +1 Scary if it's true but I wish it weren't?
Hmmm. I'm not sure most scholary works are allowed to just cite arbitrary URLs for inline references or footnotes.
The idea is that you generally have to cite peer-reviewed, published and presented articles; criteria which the majority of web published material simply does not satisfy. Web reading would fall under the "course reading", and would have to be backed up by a "real" reference.
According to my GF (currently working on a Masters in Anthropology) there is a lot of confusion on how to use the web for scholary references. Many people cite URLs in citations that are really just online archives of previously-published work. In this case, noting the URL is like saying which library you checked the article out, and what shelf it was on. If you are an undergrad and cite a URL, it is almost a sure thing that the prof or the TA's will take marks off for improper citations.
There are a few peer-reviewed journals that are (partly or completely) published online, in which case the URL might be a valid citation. This is likely to changed, and it seems the original article was suggesting that we need to handle this case now, before we lose more good work.
In a much smaller way, this is the kind of thing that those involved in the whole blog phenomenon are trying to resolve; making sure that their blog-rolls, trackbacks and search-engine cached pages stay historically maintainable.
-- clvrmnky
he Washington Post reports on the loss of knowledge in ephemeral web pages, which a medical researcher compares to the burning of ancient Alexandria's library.
The main difference being that most of what was in ancient Alexandria's library was considered to be of importance to at least a sizeable group of people, if not the majority, whereas most of the web pages that disappear every day are simply dross.
The solution suggested seems perfectly reasonable to me.
Having an archived copy allows the references to be valid and in context, whilst giving the original link allows for the updated and refreshed page to be expanded upon.
All it takes is a header on the archive stating that this snapshot was taken at a certain time, and from a certain URL.
I'm not sure if archive.org already does similar, but the action of merely *searching* the archive for a page should send the scan bots out onto that page. This way it becomes a simple operation.
I would push for the archive to be compulsory and above copyright - ALL the content continues to be the property of the original owner. Nobody should be able to remove data from the archive for any reason - if you posted it publicly, then you expect it to be cached.
liqbase
"Do you think because you print it out it suddenly becomes a more stable reference?"
Yes. Because now you have a copy of the source that you're citing.
"Sometimes people doing professional articles have to cite web pages because that's where the information they are talking about is."
And the article was about how the web pages don't stay live so you can't reference them later so the information is not available later.
So, if you're going to use web pages as a citation, you need to have a means of referencing them after they go off-line.
What better way is there than to have a copy of them yourself?
Unless it gets /.ed - then its lifespan might be measured in minutes!
I don't necessarily see a problem here, as long as serious academic research is maintained online by trusted, stable parties. That's not demanding any more than we have up to now now with a print-based distribution system, since that depends on the continuity of a large network of brick-and-morter libraries (and associated infrastructure) to function effectively. Imagine how difficult things would look if we were going in the opposite direction technologically!
As for the volume of dreck available on the web... Well, that's been equally true of print media, something I'm reminded of whenever I stand in a grocery checkout line. Credibility will always be judged by the trustworthiness of the source.
This has been a real problem for a long time. But the web is distributed. The only real solution is for people to realize that moving stuff around all the time breaks links, and avoid it. One thing that would help is a translation layer in the web server, that separates the URL from the server's filesystem. This is basic software engineering common sense.
2 50" is a much better permanent URL for this story, than exposing the details of some perl script called "article.pl" that takes a parameter named "sid", and it will be easier to adapt to all future versions of Slash or other software, or to simple archive as a static file someday. Using the PATH_INFO CGI variable you can make a CGI like "article.pl" use URLS like that above.
Non-transparent CGI, PHP and ASP scripts are even worse, they tend to change all the time. Instead they should be using the "path info", or be in the server (mod_perl, etc.)
Example: "http://science.slashdot.org/article/03/11/24/127
The idea that the basic job of a webserver is to pull files off your disk is incomplete: it's job ought to be to take your URL through *any* kind of query lookup, which might map to the filesystem and might not. The HTTP RFC's imply this as well.
reed
VOS/Interreality project: www.interreality.org
The article claims that "the average life span of a web page is 100 days". This is a very misleading statistic. What it really means is that the average web page is updated every 100 days, not that the page dies and goes away after 100 days.
Moreover, as you can imagine, authorative sources (the type that people are likely to quote) are updated much less frequently.
Law journals have tried to tried to cope with the proper weight of authority to grant web pages by trying to follow the Blue Book, a citation manual.
The general rule has been that whenever you can find something in print, cite to that, but add an internet cite when either it is available and would make it easier to find, or if it is only available online.
Things that are only available online are surprisingly common in citation. The leading court reporter services (WestLaw and Lexis Nexis) both have cases that aren't "officially" printed, but are available online.
Also, many journal articles will cite to web pages such as a company's official description or press releases.
In general, these citations are treated for their functional purpose and not their form of media -- online cases are grouped (last) with other cases, and information from most web site is considered a pamphlet or other unofficial publication.
This system seems to deal with the fact that they are ephemera pretty well. The citations really are only used to make a point that is merely illustrative or is easily accessible to legal practitioners.
It's implications go way beyond web pages, which are just one of the first manifestations of our electronic culture creating records that never touch paper, or other more established and permanent mediums.
Businesses typically only have to archive material for around 7 years legally, although some industries like pharaceuticals have to preserve data considerably longer. This is fine when records are primarly paper based, with some nice computers to speed our current business along. When records are totally electronic from start to end, ("born digital"), we start to have problems, legally and culturally. Some researches are talking about a digital dark ages, where many of our records today will simply vanish from history, totally inaccessible and unpreserved.
This is about storage, migration and emulation. It's about persistent identifiers. It's about technology obsolesence leading to cultural obsolesence.
Matt Palmer Digital Preservation Department UK National Archives.
slightly off topic but related is the efforts of researchers to create Public Knowledge Projects (PKP), such as John Willinsky , were the effort is to make research, that effects the public, accessible and understandable to the public. Stablity of links to documents and opening up citations is key to trying to develop these sites. So this is a challenge. You would almost need a completely self contained site - meaning you somehow provide duplicates of necessary links
I read an interesting article a few years ago about how even our hard copy (books, magazines, musical scores, etc.) won't be nearly as useful to future historians.
Why?
Current historians learn a lot about each writers creative process, and how writers evolved their ideas, from drafts and corrections. Music scholars pore over every scratched-out note, every furious scribbled comment, in Beethoven's draft scores. Writing music was laborious and hugely frustrating for Beethoven, unlike Mozart, who hardly stopped to think and made few if any corrections.
Future scholars won't know any of this stuff, looking back at our work. We use software to edit our work... so when we fix our errors they are gone forever. We change our minds and the original idea disappears in a puff of electrons. An electronic score of a Beethoven symphony only differs from a Mozart concerto in the musical style -- all of the other data is gone.
It's a sobering thought. Where else are we going to get this data? Not letters, because we write emails now, and regularly delete them (intentionally or not). Diaries? Some people still keep them on paper... but many store them on computer, or publish them in blogs (which as discussed will mostly be gone).
Sobering thought isn't it? It's not neccessarily hubris to say we ought to be saving more of this stuff; people a few hundred years from now should be able to learn from our failures, as well as our successes.
There are only 10 types of people: those who understand decimal, those who don't, and, uh, 8 other types I forget.
The article is not about archiving "everything in the world." It's specifically about references in scholarly papers, which, for the past three or four centuries, have been part of the essential fabric of scientific research. In a research paper, everything you say is either supposed to be the result of your own direct observation, or backed by a traceable, verifiable, and critiquable authority.
You don't just say "Frotz and Rumble observed that the freeble-tropic factor was 6.32," you say "Frotz and Rumble (1991) observed that the freeble-tropic factor was 6.32." Then, at the end, traditionally, you would put "Frotz, Q. X and Rumble, M (1991): Dilatory freeble-tropism in the edible polka-dotted starfish, Asterias gigantiferus (L) (Echinodermata, Asteroidea), when treated with radioactive magnesium pemoline. J. f. Krankschaft und Gierschift, 221(6):340-347."
Then if someone else wondered about that statement, they'd go to the library and pull down volume 221 of the journal, and see that Frotz and Rumble had only measured that factor on six specimens, using the questionable Rumkohrf assay. If they had more questions, they'd write to Frotz at the address given in the article, asking them whether they remembered to control for the presence of foithbernder residue.
This sort of thing is absolutely essential to the scientific process and makes science self-correcting.
The article says that these days, the papers are published online, the references are URLs, and that an awful lot of them are stale. If so, this cuts to the very heart of the process of scientific scholarship.
"How to Do Nothing," kids activities, back in print!
It's not just the short lifespan of a webpage... it's also the fact that the source isn't always reliable. Web publications are rarely given the same strict editorial process as most journal articles. The content might be just as good - or better - but they're also not given the same credibility.
The problem *is* the short lifespan of web pages. Even "reputable" publications move their pages around, or remove them entirely, breaking all links. I'm talking about major newspapers, scientific journals, etc. It's these people, the supposedly reputable ones, who need to do a better job. The way they're doing things now is indeed, "no way to run a culture."
Recently a colleague of mine published a paper in an online peer-reviewed journal which contained a trivial error (transposition typo) that however would change, in fact reverse, the interpretation results. They were permitted to fix this, months after the article had first been posted. Does this aid Progress, or is it Revisionist?
While it's obvious that not every URL is appropriate for a research paper, papers in high energy physics have used URL-references to preprints at arxiv since 1991. It's not surprising to see some less technical fields like anthropology further behind in understanding and using the technology, and high energy physics has a particular advantage in that the web was originally created for disseminating information in that field.
People interested in the evolution of an electronic knowledge architecture that's gradually replacing the print one in some scientific fields will likely find the articles Creating a global knowledge network and Can Peer Review be better Focused? interesting. Both are by Paul Ginsparg, who started the preprint archive 12 years ago at LANL.
It's also worth noting that free, public access to preprints has democratized physics research, as all researchers have access to timely information instead of only a few who had the right connections to get early copies of preprints before 1991. It also provides affordable access to physics articles to researchers at institutions whose libraries can't afford the 5-figure subscription fees of many modern scientific journals.
Since publishers register DOIs as soon as the electronic version of the article is available online, the article is citable using DOI way before the print journal goes to press. And since the DOIs are persistent, links will work even if the journal changes ownership/publisher.
In addition to providing free DOI resolution, CrossRef also provides a free metadata lookup for libraries (or it will provide it for free soon I think). Libraries will be able to lookup DOIs using article metadata as needed.
Many publishers also participate in variety of archive initiatives, where a copy of every electronic article is made available in large or national libraries for safekeeping. In case the publisher goes out of business, the library or institution has the authority to make the stored archive available to public. With persistent DOIs this will be very easy since the existing links will not break even if the servers are different.
I agree. The point of copyright is mainly to encourage the production of commercial works, to enrich the public domain. It was never intended to force a work to remain out of print.
We need to change copyright law so that it doesn't prevent saving of lost works, and so that it can't be used to force a work to moulder away because it's in someone's best interest that it not be for sale. (For instance, old movies that studios don't want cutting into new movie revenue.)
I'd like to see a short total-rights-reserved copyright, ten years or so maybe, and a longer commercial-rights copyright. I really see little reason why Warner Brothers, for instance, should be able to use Mickey Mouse in their cartoons, but fanfic, kids pictures, and other such uses should be allowed. It's part of our culture and to deny us the right to participate is rude, and short-sighted.
Few of today's creators grew up isolated and started creating original works immediately. Instead, they built on the culture they saw around them as they grew up. Children today won't have this ability. We're raising the bar, requiring them to create something that's safe from even an over-zealous lawyer and look-and-feel cases, as their first works.
Tolkein would never have gotten started in our current legal climate. He intentionally built on previous stories and myths, something that wouldn't be legal to do now. Hell, for a while, TSR was trying to sue people who used their monster names in fantasy works, even where their names were derived from Tolkein.
archive.org provides an essential service to counteract the short lifespan of the typical webpage. It also allows for permanent links to webpages that might be gone soon. I personally think that academia should either pour money into archive.org or create their own specialized archive for academic websites.
In the later case, the service would archive sites of scholarly interest on its own and it would have a feature that would allow someone writing an academic paper to request that a particular page be archives. The page that he references in his work would be a http://academicarchive.org page, not the original.
Cool URIs don't change
A bit over-idealistic, but worth aiming towards even if you don't achieve 100% non-URI-breakage in practice.
I feel that search engines should slightly penalize sites that have a history of breaking links or making them redirect to a completely irrelevant page: partly because there is just less chance that the link you follow from the search engine will have the content you want, and partly because even if you do get to a correct page, its usefulness as a bookmark or a link from your own dcuments is reduced.
-- Ed Avis ed@membled.com
Backups will contain the drafts in the future. Some them will surely survive.
There's a weird kind of paradox involved in what will survive, though.
Digital media has that wonderful property that it can be reproduced *perfectly* -- such that the copy is indistinguishable from the original -- but it must be copied or it will die.
You can burn your vacation videos to CD so your grandkids will be able to see them -- but that CD won't be readable anymore in a decade, never mind a century. If you faithfully make sure they're recopied every once in a while, though (and possibly converted to whatever new video formats are invented), your descendants 500 years hence will be able to see you waving from behind that sandcastle in California, as if it were filmed yesterday. No more flipping through yellowed photographs or crumbling newspaper clippings.... Imagine it! A scientist may use your video to prove his point about how the sunsets on the west coast have improved since California sank into the ocean.
He has to use family videos, though, because two decades of scientifically-recorded data on weather patters was all wiped out when a massive electromagnetic bomb was set up by terrorists in 2012.
Yeah, far-fetched example. I don't want to force the point, and definitely lots of stuff will survive... but our progeny won't be making the same kinds of attic discoveries that we can today.
"Hey, viddy all these ancient discs that Old Grampy Limp Devil had cached away up here! Can you run them? Nothing, huh? Oh, well."
There are only 10 types of people: those who understand decimal, those who don't, and, uh, 8 other types I forget.