Slashdot Mirror
Researcher Warns of "Digital Dark Age"
alphadogg writes "A assistant professor from the University of Illinois at Urbana-Champaign is sounding a warning that companies, the government and researchers need to come up with a plan for preserving our increasingly digitized data in light of shifting document management and other software platforms (think WordPerfect and floppy disks). Jerome P. McDonough, who teaches at the Graduate School of Library and Information Science at the University of Illinois at Urbana-Champaign, says there exists about 369 exabytes worth of data, and that includes some pretty hard to replace stuff, including tax files, email and photos. Open standards could play a key role in any preservation effort, he says. 'If we can't keep today's information alive for future generations, we will lose a lot of our culture,' McDonough said. Even over the course of 10 years, you can have a rapid enough evolution in the ways people store digital information and the programs they use to access it that file formats can fall out of date.'"
In 100 years, I won't care.
...digital dragons burninating the digital countryside.
And who needs to store pictures and movies on their computers anyway? In fact, I think the world would be a better place without them!
Now if you excuse me, I'm going back to watching Iron Man on my wrist watch.
We can just store everything in the cloud! Problem solved!
I am the richest astronaut ever to win the superbowl.
I'm not so sure that every megabyte of those old data disks is worth preserving. What of the past centuries' romance of the lost maps that had told of hidden treasure? Let there still be space for legends in future generations. Let the sleeping floppies lie :).
I still have floppies of Windows 3.1 ...
It's only because people are so anal these days. Who gives a shit? It's not like anyone in the future's going to miss anything. Even today with items like the Rosetta stone it's not worth much more than a Trivial Pursuit question - we'd not be any more educated or intelligent if stuff from 2000 years ago hadn't gone missing. Sure, there's a certain entertainment value in it all, but the idea that in 2000 years time anyone's going to be remotely bothered about the loss of websites, games and so on from the late 20th century is just ridiculous.
Major file formats and how to decode them. Problem solved?
I'm more concerned about the freedom of the wires than preservation of data at this point.
expandfairuse.org
"I often ask, 'Everyone in the audience who thinks they're going to be using the same word processor in ten years, raise your hand.' No hands go up. 'Everyone who has data around that's going to have value in ten years?' After a minute's thought, every hand goes up. The lesson is clear: information outlives technology."
- Tim Bray
Parity: What to do when the weekend comes.
The cultural loss isn't something that should be overlooked, some can bemoan it but the value of culture is that it exists, and that different ones existed in the past. Culture changes from moment to moment but without some action the real meat of the early 21st century will be lost forever. That is the big thing here, and that is justification for working for truly readable digital archival methods. There is a project of making minisuce indentations, but that requires a lot of technology to see much less decode. Continuous duplication, by transfer of all old data across all mediums as they rise and fall, by printing content and storing it in climate regulated warehouses, etc. We relish seeing things from thousands of years ago. This is humanity, that is our legacy. We need to leave a legacy for our grandchildren.
The only motivation for a company to invent new ways to preserve data long term is to provide it as a service so they can profit from it. Other than that, a companies main goals are deleting everything it legally can. Anything that no longer exists can't result in a lawsuit.
Everything that is preserved is a potential liability. For items requiring indefinite retention because they are critical to the business... They will be stored, redundant, and backed up appropriately. As the systems that provide those qualities age, they will be replaced in regular maintenance and upgrade schedules as economics and timing come together in the right proportions. In that way, reliability and long-term survivability are maintained - nothing stays on ancient systems that are unmaintainable forever. When systems go out of support, everybody has already been looking to the next solution to migrate to.
So what's wrong with this approach? Its essentially what all "big" companies are currently doing. I don't believe in this proprietary format FUD either - if the proprietary format is no longer supported, you migrate. Potential of future cost to migrate is the only concern, not survivability.
Migration is todays solution to long term storage and I see no reason it should be ignored. Like security, data retention is an ongoing objective that requires maintenance - its not some end-state. Dreaming of a solution that will just last forever seems archaic, no?
Overclockers
Try '10 minutes'. That format is simply not stable.
If you want to talk US politics, have you considered checking out one of the thousands of US political discussion forums? Slashdot is an international technology website, for people around the world who want to check out articles about technology.
Because the average photos of your dog and the penis enlargement spam in your inbox will be culturally important in two hundred years.
About as important as this very comment.
Most of the text in most word processing documents are easily available to be parsed out even without the specs. The formatting would be lost, as would any embedded objects or images.
Open formats would improve it, but I would be more concerned about encrypted documents and media loss than not being able to recover data (text/images/video/music/etc) from available files. There are a lot of clever people that can do amazing things with deciphering proprietary formats.
I rarely read replies, it's my opinion and if you thought about your opinion a little more, I'm OK with that.
The anonymous poster has succeeded in compressing the shiteating troll into one line, yet without sacrificing shock value. A masterpiece. Can we look forward to the BSD is Dying or nullification trolls similarly refined with Webernian brilliance?
With so much data i would say that the darkness start when you get blinded by the light. Search engines do a great job, but still, the relevant, unbiased and accurate data could be very hidden by the amount of the opposite kind of data that exist in big numbers.
Urbana-Champaign gets to inventing HAL, I'd say they should stop wasting their time with this sort of thing...
"...Well, there's egg and bacon; egg sausage and bacon; egg and spam; egg bacon and spam; egg bacon sausage and spam..."
Comparing todays Internet content to Library of Alexandria or something... Come on, the VAST majority of the data out there is comprised of pirated movies, mp3s, and pr0n. The authors concern that in case of the sudden demise of our emails, tax files and robust pr0n collections the future generations would be somehow deprived of CULTURE is laughable. When I was growing up, CULTURE came from books, museums and other things that did not involve a floppy or a hard drive.
I will not be pushed, filed, stamped, indexed, briefed, debriefed or numbered. My life is my own.
The problem existed for a while. Can you read 8" discs? Do you know how to build a device to read that "data drums" IBM used to store data?
Create documented hardware and use documentes formats to store your data. Dump everything proprietary because chances are good you don't get the whole information you need to recreate the formats or the hardware flawlessly. If you know how to build it, you can build it. If you can build it today, you sure as hell can build it in the future with better technology.
The only problem that remains after that is data deterioration due to "disc rot". I.e. the medium used to store the information failing due to age. And this is anything but a new problem. Ask your librarian what he thinks of post-1700 paper and the ink used in that age which contained some sort of acid IIRC.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Amazing as it sounds, I still have very VERY old data that goes as far back as 7th grade when I started using computers. I know of no converter for Professional Write that will convert Professional Write documents into ODF, or even MS Word 97/2000/2003.
The only hope I have is that I can use strings to extract the text elements of the data.
From the article -
âoeIf we canâ(TM)t keep todayâ(TM)s information alive for future generations,â McDonough said, âoewe will lose a lot of our culture.â
Hardly.
Apparently none of our culture is stored in books anymore?
Sure if every piece of data was wiped out the world would lose a lot of information... but a lot of valuable and useful information is still put on paper. I don't think that is our biggest cause for concern.
However I do agree that the world really needs to agree on more open / non-proprietary ways of storing data. Sure, I can open a .wav of Blackadder talking about 'sticking a Christmas tree' somewhere from 1992, but I have a bit of trouble opening .ra (real audio) video files from a few years ago.
And working in government everywhere I go the electronics file storage is just a discordant mess. Anything important we have to print and store hardcopies because our electronic systems are just unreliable.
Everyone thinks of changing the world, but no one thinks of changing himself. -- Leo Tolstoy
According to 50% of the USA, they wish we did lose our current culture. That's aside from sages wanting us to not repeat history.
Guess which side it is? Trick question.
all that becomes forgotten is useless stuff anyway, we have loads of pointless data that holds no value whatsoever. anything worth holding onto will surely survive time and the rest is best forgotten.
Open standards could play a key role in any preservation effort, he says
The way I see it there are two approaches to the problem. The Quixotic fight consisting in changing the world and forcing in a dictatorship of openness regarding file formats, which doesn't solve the problem for the past 50 years of computer history.
Or let a few hundred people around the world worry about file format parsing or, in the worst case, even emulators to do whatever old computers did. In a hundred years from now, you'll have very complete emulators for our modern PCs. Considered that a 1994 PC is quite comparable to a 2008 PC (and presumably a 2015 PC) from an emulation point of view, you know that's a given, and even then, in case there was no such emulator, you know you could find a good such emulator for machines from the 2040s, which themselves would be well emulated by machines from the 2070s, and so on.. that's what we already do. There's hardly any program you used 20 or 30 years ago that you couldn't use today.
You just got troll'd!
Yea, I have to reboot my OOXML like 4 times a day.
Paying taxes to buy civilization is like paying a hooker to buy love.
Most of the garbage that we have now just isn't worth keeping. The biggest problem is filtering out the junk we have so that we know what is really valuable. That would be things like great music; writing; the origins of software freedom; works of history and biography etc. Then we could store that, but the problem is we mostly store SOX inspired lies for compliance audits. This garbage takes away from any effort to store serious stuff long term. Who could we trust to do the filtering? The govt? (no please don't answer that :-)
=~ s,(.*),<sarcasm>$1</sarcasm>,g if any_point_you_wish();
Words: 50 Swears: 9 Facts: 1 (It's an open standard) Relevant Points: 0 You're a little light on the swears section for typical interweb poasts.
Apparently no one really cares about preserving data.
I bought a copy of Ami Pro on eBay to resurrect some of my old documents so I've already experienced this firsthand.
Is this a problem we really want solved? Look at the ever growing piles of data governments and companies are collecting on people. Is this something that we really want preserved forever?
The only change I can believe in is what I find in my couch cushions.
...the more they stay the same. Here's something I posted back in 2006 about this same issue: http://slashdot.org/comments.pl?sid=207582&cid=16922754
Quantum mechanics: the dreams that stuff is made of.
How many records do we have from ancient Assyrians? From the Egyptians? Romans? British Empire?
Entropy and loss happens. Most of the data deserves to be lost. How much do I care that Asuk the Assyrian was assessed two goats in taxes? Not a hell of a lot. How much will someone five years from now care about this post? About as much.
We work and write and live for today. That anything travels down the road of time intact is a miracle. What gets carried along is random. This is part of life. Get over it.
That is all.
Just mandate that all physical media has to contain an extra partition with schematics of a drive to read the media (the schematics themselves can be saved as a .DWG). Likewise, mandate that all file contain a binary blob which defines the file format's specifications. That way all media and all files have within them the key to read them.
Problem solved!
The English word fart is one of the oldest words in the English vocabulary.
...industrial document management system that is GNU or BSD?
I have binary only test programs for a PLC, I have test procedures, I have C source code, I have binaries, I have customer manuals and I need to manage them better.
I'd go on a Vegan diet but the delivery time from Vega is too long. --brownkitty
How's that for timing? PALGN just interviewed Eric Kaltman, cataloger at the Stanford University library about his role in cataloging game-related material and the challenges that DRM and MMOs present. Stanford's part of the "preserving virtual worlds" project, along with the University of Illinois mentioned in the article. He's also the guy who writes on the How They Got Game blog, where he documents his findings.
It's an interesting field. Far more challenging than I would have thought.
This is your life, and it's ending one minute at a time.
Government agencies and archivists are starting to wake up to the fact that this is an issue -- I think the Office 2007 file format change was a big factor that is getting it on the radar.
Minnesota, California, Massachusetts and New York definitely have people studying the issue. Unfortunately, there are no easy answers when it comes to these things.
In my opinion -- which is not necessarily the opinion of my employer -- one of the major problems is that there are far too many records being preserved.
If you looked at the archives of a government or corporate office 30 years ago, only official memorandums, some meeting minutes and policies were retained. Today, technology like email has improved communication somewhat, but has also encouraged sloppy office practices so that it is nearly impossible to figure out what is useful and what isn't.
To compound matters, the courts are now mandating document retention and email archiving which encourages the retention of even the most banal communication.
IMO, the period 1990-2020 will be a black hole in history.
Conformity is the jailer of freedom and enemy of growth. -JFK
*rolls eyes*
Asimov said the same thing in the Foundation series.
That makes this extremely old news.
He consorted with the Devil so he could make that posts.
Confess sinner or be condemned to eternal Dial-up speeds.
oh, and I hear he weights as much as a Duck.
Sorry about the writing. Robot fingers, you know? Cliff Steele in DOOM PATROL #23
The article talks about two very distinct and different problems--hardware and file formats. The author has a point about the hardware--if the media goes bad or if there is no way to read the data, then the data is lost. However, the author is completely off-base when it comes to file formats...
The author specifically mentions WordPerfect files. Bad example! The default file format in Wordperfect X4 (released in April, 2008) is the same as what was used in WordPerfect 6--which came out in 1993 (DOS and Windows). While I can't speak for OpenOffice or Google Docs, MS-Word can read those files (and WordPerfect 5.x files) with a simple File/Open. Excel opens Lotus 1-2-3 files as well. So, Word can open popular formats in use since 1988 (WP 5.0) and Excel can open some formats in use since 1983 (1-2-3 r1a). You can also buy programs like FileMerlin to convert old documents.
Frankly, when it comes to file formats, conversion apps will exist for a LONG time. For DOS apps, you could even go so far as to create a v/m or use Dosbox, load up your obsolete word processor (I miss "Leading Edge Word Processor"!) and copy/paste the text into Word or Notepad...
Image files, sounds, & videos are no exception... GIF has been around since 1987, JPEG has been around since the early '90s (opening those on a 10Mhz 8088 was slow!), and MPEG/WMV/AVI/Quicktime videos are easily openable...
Finally, the more people that are affected by obsolete files, the more interest there is in some way to convert the data... But don't forget that a LOT of the data is junk--do you really care about your 7th grade paper you wrote on Hong Kong in 1989?
Windows 3.1x calc: 3.11 - 3.10 = 0.00
Make copyright last 5 years. Then everything worthwhile will be backed up by someone who cares about it.
The thing that I've started to dislike is the requirement that you license formats in order to use them. I fully understand where this is coming from, but there was never a need to license IP to build a microfilm reader, CD player, or VHS player (I may be oversimplifying here).
But if you want to play a Blu-ray disc, or Dolby Digital TrueHD audio, suddenly you can't just buy a bunch of off the shelf parts and build something that'll read that data.
We need to do something about making formats become open. I have no idea what that something is, though.
Everyone here seems to be missing the point -- Businesses don't need help preserving data. Anything that's really valuable and needs to be preserved will eventually be put on a laptop and lost in an airport. But what about your wedding photos? What about that book you've worked for three years on, and saved it in word doc format?
The problem of data preservation is not one business needs to address -- there's a million geeks (hi slashdot) that will be eager to earn their pay coming up with washing-machine sized solutions for business, in black cases with a stylish logo on the front. But what about me -- the person who makes less than $30k a year, keeps all my files on a laptop and an external drive, and doesn't have a lot of cash?
Let's say I want to put it in a safety deposit box and forget about it for 10 years? 20? 50? What are my options for preserving photos, videos, and text cheaply? And by cheaply, let me say less than a grand, since "cheap" seems to be relative here.
My father (dead, retired 20 years ago as a curator of a technology museum), was bothered as were others in the field, back in the 80's. He had seen microfiche come and go, apart from the *new* digital stuff that was already being junked. He was relying on high quality long term photographs in nitrogen canisters. It only worked because he was storing a visual media such as a sheets of paper. Only the important ones, but about a million of them existed.
.mic image format. I have a few files still of those on my computer. You can see what the picture is if you thumbnail it. But when you try to get a full sized image, Windows says it cannot recognize the file format. It is now a .mock format. Is there a term for operating systems no longer being able to recognize their own past? 'Osheimers' for example.
As for Wordperfect and floppy disks: yep. That's a problem in our home. We are having to migrate WP files now and then. It is not sufficient to have old computers that run the programs. I had WP on my computer (but didn't use it.) A series of glitches when upgrading to SP3 had as a side effect the corruption of WP on my computer. Whatever the problem was, I could not even re-install it. We are now down to one computer that can read it.
I, when I worked in IT, migrated library data. Getting it into any sort of readable text form was a trial. We have even been sent old Macintosh computers in the hope that we could get stuff off them. Usually we could, but it wasn't been done economically, and I cursed the Education system that had highly paid administrators who did not even dimly consider that a data storage system had a finite lifetime. Not even 20 years after my father retired on under half their salary.
The core solution is as the original article says - for all government software, mandate that data export to a widely used open standard be available within the package at no extra charge. I do not know of any impediment to this worth considering. Where there are privacy issues, it is simply exported encrypted and funds are established that allow a few facilities to decrypt and migrate the data. If you cannot sell to government, including any educators, then you are marginal. OK, so some games will be unavailable to future generations. That is inevitable. But then that will be a reason to collect and maintain the hardware if you are a hobbyist.
As for large corporations, it may be sufficient that the auditors require that data be accessible for forensic and liquidation purposes. That is, not readily, but if need be in extreme circumstance.
In short, the immediate solution is an administrative one. Software and hardware is the relatively easy bit.
My own prize example of a dead data format - the Windows
The only proven lasting method for recording history.
Let's face it, nothing else comes close.
99% of data that is worth accessing is accessed enough to ensure it is in a readable format. As far as the other 1%, it will give our children of the future something to do. Sounds like another stupid excuse for everything to be open and free. Some things are better proprietary and expensive.
If you store digital information (wether you are a library or not), make sure that as long as you have information stored in format "x", that you have the proper equipment for reading format "x". Ideally, if you get new equipment that uses a new format "y", be sure to *both* keep the old equipment that you knew worked properly (and not just one set - keep several, if you can, and make sure that new employees/members know how to use it), but also try to find such new equipment that is capable of both reading the old format "x", but also capable of copying/transferring information from the old format "x" to the new format "y". Try to convert all of your information before there is any likelyhood that the format "x" equipment reaches its end of usable/supported life.
Oh, and if at any point this means you have to bypass access keys or encryption, damn the torpedos(DMCA) and go straight agead.
This is one of those fairly bogus, highly overblown stories that keeps cropping up every so often. A similar one is the supposed shortage of scientists and engineers in the US, which has never existed, and is always supposed to be coming Real Soon Now; in fact, the data to support this claim are always either nonexistent or wrong. (E.g., they compare Indian college graduates with US college graduates, but the Indian degree they're comparing with a U.S. bachelor's is more equivalent to an AA degree in the U.S.)
First off, the concern about incompatibility of physical media was valid 30 years ago, but it's a false analogy to try to apply it to today's situation. Thirty years ago, I had data on a mixture of 8-inch floppies and 9-track tapes. I can't read an 8-inch floppy anymore, and although 9-track tapes still exist, most 9-tracks from that era are no longer readable due to physical deterioration of the media. But that was all in an era when hard disks were expensive, and the internet didn't exist. Today, I have all my data on hard disks of various computers, and I use file synchronization software to keep them all in sync. If one of my hard disks dies, I replace it, and I haven't lost any of my data. (I also have backups on optical media, but I basically never need those.)
There's also the concern about formats. People tend to bring up, for example, the image of rooms full of physically deteriorating 9-track tapes with data from old NASA space probe missions. The formats are often not documented. The thing is, most of our data isn't at all analogous to the raw data from Mariner or Voyager or Viking. Those were unique historical events, and the only way to get more data like the data they collected is by sending another space probe. (People also tend to vastly overestimate the value of scientific raw data. It's extremely uncommon for raw data to be of interest decades later.)
Most of the world's data isn't in some obscure NASA format, it's stored in formats that are used by tons of people, and are extremely well documented. Sorry, but I just don't believe that the knowledge of how to decode Adobe Acrobat format is going to be lost to future generations. Ditto for html, jpeg, and mp3.
Another thing to keep in mind is that nowadays you can emulate old computers with excellent performance. For instance, my first home computer was a TRS-80. I can still run my old TRS-80 games on my linux box, using an emulator. Sure, emulation isn't perfect, and some information may be lost. But the claimed threat of data loss is vastly overblown.
The biggest threat to the preservation of information isn't technological change, it's copyright. The most likely reason that I wouldn't be able to get back an old piece of digital data is that the people who tried to preserve it and put it on the web got sued by the people who own the copyright -- the same people who let it go out of print. The economic incentives are to hold on to your copyrights (because that doesn't cost you any money) and send out DMCA notices to anyone who puts it on the net (because that doesn't cost you any money either), all in the hope that your content will be worth eleven cents fifty years from now. This is exactly what we see happening, for instance, with ROMs for old video games, which you can play in MAME, except that you have to find an illegal source for the data, because the owners of the copyrights aren't willing to sell you a copy.
Find free books.
Most of our actual culture is stored in file formats that can be read by free software. These will all be readable for the foreseeable future. Of course there are other problems like media longevity that have been covered on Slashdot before, although those are mostly solved. But his file format concerns are bullshit.
find some dimonds-sheets and carve the data on it.
I was thinking the same thing. I thought this was /. where privacy was very important 11111!!!!!!!!!!!.
Look, he's an Assistant Professor, not an Associate Professor. He just got his PhD a coupola years ago and somehow he managed to land a job. He needs to publish something, anything. He needs tenure. So he's saying the library (OK: 'Data' if you will) is on fire and we need a government rule to protect it. The librarians are going to nod wisely and agree with him (I'm a librarian and I've seen way too many wisely nodding librarians in my time.) It's all a bit of a smoke and mirrors thing and he'll be able to milk this for a few more articles to put on his c.v. He's whoring for points just like on /.
Meh?
How about a moderation of -1 pedantic.
So, here's a market for remotely accessable virtual machines with a vast array of software that once was.
Remote into the vm host, send a file to the specified storage directory, open it in the VM and save to some other format.
It's not the access to the data that's that hard. I mean shit, if it ever ran on an x86, there is SOME illegal software archive out there on the internet. I have windows 1.1, NT 3.1 etc etc because of this. Illegal probably, immoral? I'd argue not because there is no means to legally acquire them, and I dont actually use them.
This is why I think copyright should end X years after support from the provider does.
From then on, it should be a free and unsupported download.
Actually, if it's MS software, the MSDN library has it all.
Gotta give MS some credit.
What about radioactive iron man? Which isotope is he?
The Australian National Archives seem to think that its stable enough for long term archiving of documents.
Think of that data as the "Deleted Scenes" on a DVD. Does anyone really want to see that stuff again anyway?
The important stuff will survive.
Over 100 replies, and no jokes about porn?
Anybody want my mod points?
this means you have to bypass access keys or encryption
This is going to be a big problem. I have CAD files, code manuals and other engineering data that cannot be accessed with anything other than the proprietary CAD apps or browsing software. Some of these apps have been 'orphaned', in that the applicable versions are no longer supported by the vendors. Activation keys are locked to a particular machine, so trading in that Windows 98 machine for a nice new XP system is out of the question.
I make sure that none of my contracts oblige me to maintain electronic versions of deliverables or that any delivered to the clients will be accessible beyond the completion of the contract. Its rolls of blueprints or nothing.
Have gnu, will travel.
The incorrect implication is that we are for the first time in danger of losing something because of media which isn't as persistent as previous generations. The correct emphasis is that for the first time we are capable of persistence in perpetuity due to digital media. Regardless of the persistence of cave drawings, books, etc., exabytes of information have not persisted from previous generations, certainly not from previous eras.
Given enough time and money you can read "anything". But if the data is gone, no amount of effort can recover it. What's happing today is most people simply do not do any kind of backup, none at all. You can argue that maybe the world in 200 years will not care if it has a picture of your kid at the beach. But the world can't care. People do, one of them at a time.
For example I'm working on collecting some old family photos, many of them shot at the turn of the last century, around 1900. They hold some history and put faces on names. But in 2108, a hundred years from now will there be any hundred year old photos? Forget about not being able to read a file format. They WILL be able to read anything they want to. But will the file be there? I doubt it.
"Dark Age" is kind of an exaggeration. Presumably it's a reference to the period right after the Fall of Rome (475 AD) when most classical literature was lost because existing information technology (hand-transcription of documents) got too expensive for what passed for an economic system. This time around, if we lose much more, it's because we have a lot more to lose. But how much of it matters? If my USB drive dies and takes the last surviving copy of Debeee Does Dingos or the collected bloggings of Joey Joey, it's not that big a deal. But anything that really matters (the complete works of Shakespeare, the Beatles, the user's manual for Ultima IV) is going to be saved in multiple places in multiple formats, and it just not going to get lost.
I think the big problem is the exact opposite of what TFA warns about: too much preservation of stuff that isn't worth preserving and doesn't really represent our culture. Future generations wading through the digital crap we leave behind — blog rants, porn, advertising, spam, internet rumors, Star Trek flame wars and fan fiction — will be hard put to sift out our serious accomplishments.
Classical Greek civilization is probably the most influential in all of human history. And yet you can buy a single CD containing every single surviving work from the entire civilization! It's quality, not quantity, that defines a cultural heritage
Deep inside I was hoping for Mermaid Man and Barnacle Boy.
http://spongebob.wikia.com/wiki/Mermaid_Man_and_Barnacle_Boy_V
Don't be apathetic. Procrastinate!
will only be able to pick up our trash (because that's the only thing that lasts) and try to figure-out our lives from ads in newspapers or flyers and empty tv-dinner packagings...
Personally, I'm sure not much will survive from our generation.
We're a throw-away-society, for a large part (not only the US) and most stuff is done to last as short as legally possible.
Sometimes I think about making some black and white prints from my digital photos (they conserve much better over decades) - but I never get around to actually do it.
But let's face it: most of the "information" around today is barely worth preserving. It may be useful for future generations to figure out exactly how we got them into the mess they are in (then), but depending how big the mess is, they might not care at all...
Windows 2000 - from the guys who brought us edlin
So just because, it's always been that way... you think it should continue to be that way? That we shouldn't bother trying to do something about it?
There's lots and lots of information floating around today that future historians will probably wish they could have a look at. Why not make it possible for them?
I'd like to see movies and video games and news reports and government documents and everything else, preserved for as long as possible (indefinitely?). Even if no human looks at it again in the next 500 years, it can go into a giant database to be scraped and indexed and searched and cribbed from by increasingly powerful computer systems.
Why not try to preserve the totality of human knowledge, in whatever forms we've got it in now, for future generations to datamine and examine (even if its only for curiosity's sake)?
If you wish for your grandchildren and great-grandchildren to have some shred of knowledge of your existence, you should make certain that you have black-and-white photographs taken and printed on the highest quality, assured permanence, stock that you can find. Those prints should be stored in a fashion that protects them from decay so that your grandchildren and great-grandchildren at least may see what their ancestors looked like. From personal experience, I would not have known what my parents and grandparents looked like when they were children had it not been for the relative permanence of the black-and-white printing process.
Sig this!
Well, whatever an exabyte is, I'm sure OpenOffice is the starting point from whence all data crunching and conversion will go... although you know RTF has been around forever... but you know the great thing is that OOo can be modified for ANYONE's purposes, especially as far as opening ancient file formats and reformatting them goes, since that's just a matter of plugins--probably even the community would be willing to help?
Think of all the information that was produced by men and women since they began drawing pictures on cave walls and carving symbols on clay tablets. Most of it is lost, because the physical media was lost, broken, erased, reused, burnt or somehow destroyed. We lost plays by Shakespeare and Moliere, most of the writings of the classical greek philosophers and writers, and countless other valuable works. Digitizing data has its own problems (all the openness and format stuff, sharing and related legal issues, ...), but it makes it a lot easier (and practically free) to quickly duplicate and share any piece of information between different persons in different places of the world. Even with the various problems that still have to be faced, I think current digital data is much safer than anything that was written or drawn a few centuries ago.
I do agree that we will see some 'cultural disintegration', but not for the reasons cited in the article (which I, of course did not read). The reasons? New media models that require "monthly access fees" (yes Blizzard, Sony, I'm looking at you), and DRM protected media. Sure, some companies will 'do the right thing' and open their media to the public once they are not actively using it as a revenue source, but they will be in the tiny minority. My kids will probably never be able to dust off the World of Warcraft DVD, insert it into their holo-reader and find out what our generation did for fun. Likewise with the millions of songs that are stored precariously on iPods throughout the world. Once the iPod breaks, and the iTunes servers are switched off for the last time, that music is lost forever to the people who loved it dearly, but were foolish enough to accept a 'limited rights' version of their media. Looking back, we can still enjoy art from the entire history of humanity - cave paintings, books, canvas and sheet music, just to name a few. Apart from the physical disintegration of the medium, little can destroy these expressions of our culture. With our new encoded, protected and limited DRM-riddled media, there will be very little to look back on from an individuals point of view. I expect that organizations will spring up to restore these lost works of art, and efforts will be made to make our current culture accessible in fifty or a hundred years. But where does that leave the young kid who finds the suitcase full of DVD's, or Blue-Ray discs in his attic, left to him by his grandfather? Will he or she be able to take a glimpse into history, in the way that our generation has been able to dust off the old vinyl record player, and reverently remove that piece of vinyl from its weathered cardboard cover, to listen to a crackly rendition of Muddy Water's 'Baby Please Don't Go' I doubt it.
I'm reminded of this story from a few years ago, where a 500 year old Leonardo drawing inspired improvements in mitral valve heart surgery.
And the story is marvelously content-free about what the insight and new repair technique actually WERE. (As usual for modern, dumbed-down news media. But I suppose it's better than their previous approach, where they attempted to report it but always got it wrong...)
So that story is additional evidence for the need to archive the important stuff, since the mass-distributed versions are useless. B-) (Or is it B-( ?)
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
We have had the solution for some time.
3d holographic crystals can be used. They will not lose the data, are not subject to EMP vulnerability, and provided they are of sufficient hardness, are unlikely to be damaged by severe natural disaster.
Additionally, The data can be stored in digitally readable formats, or as images of human readable documents (they would be recoverable by simply reading them like microfilm and trascribing them, meaning the loss of every single computer on earth would not affect the recovery of human knowledge)
I'm pretty sure they were abandoned because of efficiency or expense issues, but for the purposes of preservation, they do exceedingly well.
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
Garbage isn't the problem.. the problem is that we have millions of copies of the same data. Think of the 50gb of video games you may have installed.. 10 million people have the same games as you. Music? Unless you performed it yourself or it's sub-underground, chances are millions of people each have multiple copies of it. The anime you've torrented has 10,000 downloads. .
No, see.. actually I'm just keeping a back up for the RIAA in case they lose their copy. PLus I keep it all transcoded to the next generation formats at no charge. And on top of that it's forward deployed for easy re-distribution without bottlenecking their servers. I even paythe lectric bill on the disks and internet connection. So copies are a good thing.
Some drink at the fountain of knowledge. Others just gargle.
IF our languages have not disappear, that's good enough. Throw away everything else. Do we not have enough crap around us already? The real question should be where are we going to find the resources to keep up all the personally important but ultimately non-essential materialism around us, lest we disappear leaving nothing more than stone heads.
This may be the only hope for all the dumb kids who have been posting videos/pics of themselves all over the internet (or even just sending them to equity irresponsible friends and lovers) doing the kinds of things kids have always done as teens or in college but didn't publish for the whole world.
There aren't too many of us who HAVEN'T done stupid reckless embarrassing things, and the fact that it was harder to broadcast/save/share those moments with the world has been a blessing.
If you have files stored in ancient word processor formats they can can be stored alongside said ancient word processor application running on top of ancient operating system. Package all of this stuff into an open virtual machine format (OVF) and run when needed on VMware, Xen, KVM, Virtual Box, Parallels, or your favorite other virtualization software.
with a self-preservation instinct
-and the ability to seek more recent and
more reliable storage sites
on the net based on stats and newness of
net-connected storage site
-and the ability to copy themselves to
multiple sites and keep track of how vital
a community of copies they are and panic
and breed if they get too low in number
-ability to rewrap their encryption to newer
standards
-open standards for metadata and data formats
and permanent URN for the program & version
needed to interpret the data
-and don't even think about trying to patent this.
I wrote it up 10 years ago.
It would only stand a chance if it is open.
Where are we going and why are we in a handbasket?
I work with Greek inscriptions on marble still accessible 2500 after recording the data.
Beat that, techies!
In the province of Quebec Canada, the only format that is allowed for archiving electronic documents is the TIF format. Have other countries, states, provinces regulated on a specific format?
I was an avid WordPerfect user until I switched to Linux about two months ago, I recently got the newest version from my university. It didn't run in WINE so I've just been using OpenOffice, but the point is that it's still being developed, and people are still using it (including the employer I've had for the past two summers).
Comment removed based on user account deletion
The Australian National Archives seem to think that its stable enough for long term archiving of documents.
Actually, the National Archives of Australia is using Open Document Format (ODF) to store wordprocessing, spreadsheet and presentation file formats.
They use proper openly specified file formats - https://sourceforge.net/projects/xena/?abmode=1 for more information
I know, because I work there (in the Digital Preservation area, actually)
I saw this coming almost three years ago and I'm not a fancy 'researcher'. http://www.dakwegmo.com/digital-dark-ages
The best example I can think of are personal letters. Usually we judge these by the importance of the person who wrote them, but in some cases we can (today) look at the letters written by ordinary people to their loved ones and gain great historical insight into the events of the time. Take, for example, Ken Burns' "The Civil War". Some of the most compelling information in the documentary was found in the letters written by ordinary soldiers.
Somehow I doubt we'll have records of the emails today's soldiers are sending home 150 years from now.
We can't judge what future generations are going to find valuable in the mountains of data we're generating today. We should find a way to preserve as much of it as we can. I hope someone is working on good, open compression algorithms to go along with the data storage.
> I miss "Leading Edge Word Processor"! Best out there; quick, even on a slow computer (though mine at the time had a "fast" toggle) Great stuff, and all amber. You never forget your first. Ben
"And I'm pretty confident that our future cousins will be able to build a current computer with their future technology, as long as they know WHAT they should build."
I hate to sound pessimistic but it could go the other way. Social and technological progress doesn't always have to go forward. Look at the Dark Ages, civilization both before and after was much more advanced. The way things are going right now we could have another one. Just before the Dark Ages, barbarians were invading Rome. Now we have fundies (Christian, Jewish and Islamic) trying to undo the way things are today. Part of the decline of the Roman Empire was the growing gap between the rich and poor. It's not too different today. There are a lot of depressing parallels between the declining Roman Empire and the times today. I just hope it's just a coincidence.
I'm guessing no. Of that ridiculous amount of estimated data, I'm thinking there's not really that much that needs to be saved for very long.
Most of what we know of Shumer history is form their tax and business records.
I'm more concerned about losing the culture from the 20th century.
Everyone born after 1975 hates the RIAA, doesn't pay any attention to whatever they say, and file-shares gigabytes without a thought to the music industry definition of 'piracy'. This is as it should be. It means that the music and movies of the (for now) young people is safe because it is widely circulated outside the control of those who have deluded themselves into believing that they own it.
It's all the stuff from the first 2/3rds of the 20th century that will disappear. Because the people who like it are in their 50's, 60's, and 70's now and don't have the technical skills to copy and distribute it. Plus they actually trust the corporations will preserve it. I mean all the books, music recordings, television shows, movies, and plays from the first half of the 20th century. The stuff that is under 'infinite copyright' and will never be in public domain because the corporations will simply pay off the politicians to endless extend the copyright period, as they do now.
As soon as all this stuff stops selling (and who nowdays is paying money for the book that was #3 on the New York Times BestSeller list of Oct 28, 1936?), and can't be legally copied because it can't enter public domain, then the corporations will just destroy it. Pulp the books; convert the film stock to ethanol to power their SUVs; dump the magazines in the oceans or in nuclear waste sites to absorb neutrons. When that happens, all this culture will be gone and historians 200 years from now will have little idea about how civilized people actually thought and acted in the critical early years of the modern technological age.
You can talk to the old people about the need to preserve their culture by making 'illegal' copies of the books, magazines, and movies that were important to them, but they are just simply and completely clueless about the extent that their culture will die as they do.
Good thing I write everything in VI. Dodged that bullet.
I really don't see the problem, because I have yet to come by an old fileformat which can't be read by modern programs.. there are so many conversion utilities around and also so much information on old formats.. I really don't see the problem.. This just sounds more like a cry for attention..
google.. they'll know what to do !
Even if I'd be a little inconvenienced to dig up a copy of Corel Draw
It is in a business's interest to use open formats. Most have no clue that this is the case, so never bother. They'll figure it out eventually.
Deleted
Your mother.
Hey, I finally got my first freak! Took you long enough!
Similarly for the BD/HD-DVD DRM.
Or it will be illegal.
It's a Digital Supernova Age and they're bitching. Think of your parents' or grandparents' generation, and try figuring out how much information exists about them. Sure there's the basics like birth certificates, marriage certificates, property records and other big things, there's probably some pictures and maybe they're mentioned in some books but I doubt there's any real record of how their daily life was and what they were doing. I know I have chat logs and such from my youth that are probably way, way more accurate and uncensored records than anything my parents have, even if they kept a diary which they didn't. If I get over how immature I was at the time, that's easily something I could release for research in 50 years time. With blogs and myspace and twitter and facebook and whatnot you can do a lot more, in a lot more detail with pictures and whatnot today and capture a large part of that as it happens.
The only thing happening here is that a few historians look at all this trivia which was always there, but never in a form to be captured and go "We should preserve ALL of it!" in a historygasm. If you preserved 0.001% you'd still preserve more than any generation of humanity to date. It's a case of diminishing returns, we don't truly need 24/7 live footage of 8 billion people as an historical record. It's certainly important to catch some sample of daily life and not just the big historical events and mainstream media, but I have no doubt that more than enough of this will be preserved anyway. Maybe we're in deep shit if humanity nukes itself out of existance but otherwise I'm sure it'll be kept as collectables or antique information from hundreds of years ago. Can you imagine in 2544 saying "It's a original (=bit exact) 2008 CD by [Artist]"? That's not going away no matter how crappy it is. And if we do nuke ourselves out of existance, I'm not REALLY concerned with what alien archeologists think of us anyway.
Live today, because you never know what tomorrow brings
The tech community has an almost diametrically opposed attitude to permanence than exists in the art world. Artists have always had great concern for the permanence of their work. The reason you can still see paintings that were created hundreds of years ago is because their creators rendered them on...and with....media that was mixed with permanence uppermost in mind.
This is why fine art paints have such catchy names as "burnt umber". They don't describe the color, they describe the chemical process used to create them. All modern paints have numbers that actually rate their lightfastness and permanence. The most permanent colors are ones made from naturally existing, inorganic materials. If you are going to spend hundreds of hours working on a painting, you don't want it to fade in the first flash of sunlight.
Contrast this with programming, where no one can be bothered to even comment their code for the guy in the next cubicle, and everything is obsolete within a decade. I have CD's (the format that was reputed to last a hundred years) that have gone bad in just a few years, the data on them often irrevocably lost. It was only after the complaints rolled in that the printing companies started making inks that lasted more than a year. Half of my perfectly serviceable hardware gets shunted into the closet every time Steve Jobs has a new idea for something newer and faster. The whole "planned obsolescence" meme inherent in our commercial culture seems to have become magnified in the tech world, where the tools we all now use to do our work are designed as much by marketers as by engineers, and their utility is entirely dependent upon the continuing existence of the monopolies that sell them.
This is a bad, bad thing.
I am already a slave to the whims of Adobe. My artwork is in constant peril of destruction if I do not take great pains to insure a multitude of copies exist on it in a variety of media. I have lost more than one collection of art to the decision of my toolmaster to quit supporting some older, proprietary file format. Many digital works I have printed have already faded. I recently had to buy a PC for no other reason than I needed to be able to work with files a client had. I've been backed into unwanted software upgrades for the same reason. And it took less than three months of Vista to make it obvious just how long all those video games are going to last.
My traditional artwork still sits quietly in a box, as legible as the day I created it. The brush I bought to paint with over twenty years ago still works as well as when I bought it...even on new paper.
I think there is an increasing disconnect with the natural world and modern living that will eventually bite everyone in their collective behinds if we don't come to terms with it, and our latest virtual utopia is the most conspicuous manifestation of that disconnect. There is great power to be had with all of this new technology. But it needs to be employed with an eye on the future, not the company bottom line.
Otherwise, we stand to lose all that we have created with it.
... Wordstar documents and old BBS transcripts created on my 8-bit CP/M-based Osborne 1 in 1982. The Osborne is long since dead, but the same files reside on the new computer I just got this summer. Over the years since, I've generated zillions of files in WordPerfect and MS Word format, and now OO. I can read the WS docs with WordPerfect, and everything else with OO. This whole collection of docs and pics has traveled down the years with me, and have passed through 5 computers and several operating systems since then. Miraculously, nothing has been lost. I wonder how long I can keep it going... guess I ought to burn some DVD's.
WTF was that!?!?!
"I'm just here to regulate funkiness."
Charles Stross's novel Glasshouse
A good read, too! I finished it yesterday, incidentallly.
Say out loud: I'm an Aspie and I'm somewhat proud, I guess. Uh. Can I write an email in all caps instead? Hm...
Micro$oft's deprecation of .tif is a prime example. Sure it creates security vulnerabilities, but when you have millions of .tif documents, do you spend more money converting it, or do you retain a couple of old platforms around that can read them?
Confidentiality, Integrity, Availability: without Availability the other two are assured, as is Bankruptcy.
Not to detract from your point, because I believe it is valid, but keep in mind that remaking older technology using newer technology is not necessarily easier because our new technology is faster and 'better'. For instance, in this story about remaking the first video game, they mention that it took 4 people 3 months to put something together that originally took one man a little over two weeks, because of the way the technology used had changed over fifty years.
For those moderators who don't get the point:
If someone weighs as much as a duck (Monty Python- The Holy Grail), and African Swallow (Monty Python- The Holy Grail), I said "African Duck" bringing both concepts together within a 'digital dark age*' (RTFTitle), then you can see that:
A. It's got humour
B. It is on topic (sort of)
C. It is relevant to the posts above
D. Ifonography was also on-topic as well.
-------
*Dark Age - Monty Python was set in the Dark Age.
Don't be apathetic. Procrastinate!
You need to go and get a copy of 1434: The Year a Chinese Fleet Sailed to Italy and Ignited the Renaissance by Gavin Menzies. Don't laugh: he makes a very good case. He argues that Leonardo and his contemporaries were merely copying manuscripts that had been brought to Venice by the Chinese.
DNA is a Turing machine. You, however, being dynamic and emergent, are not.