Slashdot Mirror
How Long Do You Want Digital Media To Last?
spamfiltertest writes "CNET asks 'Would you like your digital-storage media to last 20 years, 25 years, 30 years, 35 years or 40 years?' If you're an organization or government agency, the U.S. government and an optical-disc industry group would like you to answer that question in a quick survey. I would think that we would like our data to last forever, but maybe it's just me."
Would you like your digital-storage media to last 20 years, 25 years, 30 years, 35 years or 40 years?
If you're an organization or government agency, the U.S. government and an optical-disc industry group would like you to answer that question in a quick survey.
I work in the records department of a two year tech college. We use document imaging hardware and software to store student files on WORM optical media permanently and then we destroy the physical paper files over time.
We expect that our digital media will far outlast what we have on other permanent storage mediums, such as microfiche, which go back to 1972. If the "antiquated" microfiche can hold up that long why not our records stored on the digital media?
We realize that no storage method is 100% foolproof (i.e. you can misfile microfiche, lose physical files, misplace pages, etc) but we have put a lot of faith into the setup we currently have. If time has a negative effect on both the originals and backups we could find ourselves reverting to tried and true methods used in years past.
It's mildly humorous to me that long term data integrity (i.e. "forever") is never mentioned when companies present you with all the benefits of a digital setup. The benefits of the system are great (such as easy access to student information at various sites without any reproduction necessary, security features, etc) but will our microfiche outlast our digital media? I may never know but currently, based on recent discussions about the degradation of digital media over time, it appears that it may.
I feel sorry for the poor bastards that would have to go back to storing and reproducing everything to and from microfiche if and when we find out that digital media might not have the necessary longevity we require.
I would think that we would like our data to last forever, but maybe it's just me.
My company recently started deleting our email after 90 days. One of the reasons I heard was to protect us in lawsuits.
Any other questions?
Yeah, I'd like my digital media to last forever.
While they're at it, can they make my car run forever? I also want to stay young forever, if that's not too much trouble.
Make it last as long as possible. Any media set to self destruct after a set date is no use to anyone. Make the best you can and keep inproving it.
I like muppets.
Sometimes I think it would be great to have optical storage last forever. But then I think about my grandchildren going through my CDs years from now and stumbling on all my porn. Hmm... not good.
I always save my last mod point to mod up a good troll. You people are too serious.
Ya I really want my grandkids finding the 60 year old pr0n pix/vidz of grandma the day before we go into the home.
i want it locked up in some archaic and obsolete drm so that i can't get at it anyway.
sum.zero
So that the media will destruct at the moment I die. This will save my heirs from a lot of unnecessary work and embarrasment.
Sig for sale or rent. One previous user. Inquire within.
...are they asking how long I want the rights to use it? Or how long the file should retain its integrity? Or ... something else?
I guess the intent of the question is irrelevent. In all those cases, if I paid for it I expect it to last at least as long as I do.
"I have never won a debate with an ignorant person." -Ali ibn Abi Talib
In 25 - 30 years, the data on that disk probably won't be readable by the current software available. Just like that 8-track that you will never find a car to use in. To keep your data current you'd have to convert and rearchieve every so many years.
Wouldn't it be better to switch to a RAID style hard drive system? As long as the data can be transferred quickly (no CD swapping) I don't need the hardware to last for decades if I can move the data over to another system without a problem before it fails. The whole point of digital data is so that it can be replicated and transfered rather than for the hardware to last forever. In the future, we could just have multiple personal petabyte data archives in various places that store all of our personal information where the physical system isn't such a big deal because bandwith makes it easy to move the data to my PDA or to my bank's digital data vault.
--
Or try a free Nintendo DS, GC, PS2, Xbox. (you only need 4 referrals)
Wired article as proof
The length of time isn't terribly important, as long as it doesn't make the cost of new media too high (e.g. DVDs aren't too expensive, so if I have to reburn them every five years or move to the next media format at that point, that is a good use of money and time).
The company I work for uses USGS data going back to about 1900. It is interesting to think that data collected 100+ years ago may outlive data currently being gathered....
putting the 'B' in LGBTQ+
It must at least last until you are sure you don't need the data anymore.
If you mod this up, your slashdot background will turn into a beautiful sunset!
At least until the FBI gives my servers back to me. They DO give them back, right?
If you're a business that specializes in the massive backup or translation of data from one medium to another, you probably want media to last a few years at most. That way, organizations are constantly coming to them with stores of data that they need re-recorded.
Pretty much the only way to make your media last forever is to have it stored in a solid state (like being etched into the surface of a DVD) and then sealed and stored in a moistureless, airless, lightless temperature controlled environment. But with all the talk about self-destructing DVDs and CDs that the recording industries are trying to push, I don't think even that's possible.
CONVERT ALL YOUR DATA INTO BINARY AND THEN LASER ETCH IT INTO GLASS! THEN SANDWICH IT BETWEEN TWO OTHER LAYERS OF GLASS AND HIDE IT ON A MOUNTAINTOP! YOU MUST SAVE YOUR PR0N COLLECTION FOR ALIENS TO DISCOVER AFTER WE'VE BLOWN OURSELVES TO ATOMS!!
This rant was brought to you by the Reynolds Society for Tin Foil Hats... Remember, only Reynolds Wrap brand tin foil can protect you from the strongest of the alien mind-control rays!
While data is obviously stored on media, talking about the lifetime of data is not the same as talking about the lifetime of media. So, the original poster's "forever" comment is unrelated to the survey he links to.
If you have media that you know won't last over 30 years, just copy it onto new media at the 20-25 year point. In most cases, that's not that big of a deal. Besides, by the time that 20-25 year mark rolls around, it's very likely that you'd want to convert to a faster "online" media anyway, like holographic storage.
You have enemies? Good. That means you've stood up for something, sometime in your life. --Winston Churchill
I'm not sure it's realistic. One nice thing about digital storage is you can copy it to new media with no loss at all. A book, or painting, or photograph, might last longer (in theory). But when it does wear out it can't be magically duplicated like bits can.
So if you want stuff to last forever, each generation of people needs to convert the old stuff into a new format. But if you are only doing this once a generation, it's not that big of a deal. You could even make it a family tradition, the passing of the old to the new. Assuming of course that you actually care about keeping something 'forever'.
I'd like the media to last at least a few years after the copyright protection expires. Only that way we can legally guarantee that many great works don't disappear alltogether, as the copyright owners keep them in storage, and their media become unusable before enthusiasts can legally get and preserve them for the future. So currently, I'm looking for a roughly 100 years media lifetime.
53.3 years... because 640 months should be enough for anyone.
I was going to pontificate about usefulness of data and a bunch of crap like that until I realized how simple the answer to this question really is:
42
Earlier slashdot story regarding NIST study about potential lifespan of CD-Rs and DVD-Rs.
I will not "archive" materials. If it's important, it stays online, migrated & backed-up. If it's no longer important -- delete. Online (HD) isn't that expensive. Archives can get lost or corrupted. Or readers may no longer be available.
I think that the story makes a good point, namely that some Data / Format migration is inevitible.
Therefore, optical storage producers would be smart to offer several "levels" of guaranteed life, and you could purchase based on how long you think you need you need your data to live. e.g. price per unit... 5 years: 1 dollar, 10 years: $1.50, 20 years: $2.00 etc.
CDs last 3-5 years
Floppies last 4-5 years
The problem isn't storage, it's READING the data stored in an old format. We have many miles of census data stored on punch cards and paper tapes, but don't have the machines to read them anymore - at least not in quantity.
So making it last isn't important - I can still play my records, but it's hard to find needles to play them.
-- Tigger warning: This post may contain tiggers! --
Stone - lasts about a million years. Clay - 100 years - (10,000 years if burned!!) Parchment/Vellum - 1000 years unless eaten by bugs. Papyrus/Paper - 500 years, MUCH longer if kept dry. Acidic Paper - 100 years or less. Notice the trend - it is NOT toward longer-lived media. Volitility seems to trump Archivability every time, and possibly for different reasons in each age.
There is not nearly enough love in the world, but there is far too much trust.
Why would the porn industry be interested in conducting a survey of the lifetime of digital media?
I say porn industry because its DVDA.org... Double vag... come on, you all watch southpark.
-- -=innocent ramblings from the mind of an insomniatic programmer=-
When is the last time you saw a 5.25" disk drive? How easy is it to find a Jaz drive these days? WORM reader? Something that will read your old files stored on analog cassette tape? I could go on naming defunct storage media solutions for half the day.
The only real solution for long-term storage is to keep the files "live" on a system someplace. Under and other arrangement even if the *media* the bits are stored on doesn't go bad, there's a pretty good chance that the hardware to read that media will go the way of the dodo when you're not looking.
So, once again, good planning and systems administration proves to be the answer. Set up a reliable system in a RAID mirroring setup and cut backups on a regular basis.
This became a major concern to me once I switched over to all digital photography. I have a Linux fileserver running a RAID-1 setup that serves up all my important files. Once a month, I cut three sets of backups to DVD -- one gets stuck a CD tower in my apartment, one gets taken to work and the other one goes to a storage area I have (I figure if anything ever happens to take out all three at once, losing my data will be the least of my worries). I'm up to four DVD's to back up all my data now, thanks mostly to digital photos.
It's important to be able to rely on your media over a fairly reasonable term, but in any long-term situation live filesystems are the only way to fly.
Every year during my review, I just pray the words "slashdot.org" aren't mentioned.
So for me I think of it this way. My parents and grand parents have only a few pics of the gererations that came before. Some really old picutres we have came from around 1910. The pictures are for the most part not in very good share. I see these pictures of these people who were loved deeply by the people I love and I wish I could know them better.
Now I have a nice digital camera(Canon Digital Rebel) that was expensive, but I got it for a good reason. I am about to get married and do the whole family thing. I hope someday that a great-grand kids over maybe even a further down the line will be able to look at all the pictures I will take and maybe understand a little better where they came from, what the world was like, and how pretty there great grandma was:)
Professional archivists tend to recommend that data be turned over onto new media every 5 years regardless of how well it's weathering the years.
But the truth is that, paradoxically, the most critical data tends to be the least likely to be refreshed, because access to it is typically quite limited.
Our own department of defense doesn't know where it stashed all of it's nuclear materials over the years. Why? because they recorded it on a magnetic tape, put the tape in a vault, and had someone stand in front of the vault with a gun for 40 years, and now the tape has turned to goo, and in other cases the tape seems readable but there is no technology available to read it.
We should always strive for and recommend rigorous archival policies, but we should also strive for media that can possibly withstand the ages should some knucklehead put it in a concrete box or just forget about it completely for a few decades instead.
This is just like television, only you can see much further.
Yes! Someone would be interested in company X's records! A lot of good history is done with business records: look at Cronan's Nature's Metropolis about the history of Chicago. And let me bring it back to the digital world: I talked to Pop Top software while I was working on my thesis about how computer games present history, and they used old records/manifests from railroad companies in the 19th and 20th century to make Railroad Tycoon!
Your experience is the opposite of mine. 1.2MB floppies use the majority of their theoretical storage capacity and as such are quite fragile. The most durable floppies, in fact, are 360kB DS/DD 5.25", as they store the least data per unit of area. (Or, of course, single sided 180kB discs, which are basically the same thing.)
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
That is why I have always used plain text for the most part.
The files I wrote in the late '70s are still around today, and completely readable and editable.
Whatever editor I used (I used edlin for a very short time, and then WordPerfect and MS Word - as well as several no-name apps - on DOS machines - later I found vi and emacs under Unix - and have dabbled in OpenOffice and Abiword on my Linux systems) I made sure it had the option of saving the files as plain text - or I quickly stopped using it.
Nowadays I am using XML for anything significant (that I think I may want to publish - on the web or in print) - and plain text for everything else (and XML is really plain text from a software standpoint).
I don't have any software incompatibilities because I don't use proprietary formats to begin with.
Everyone doesn't think like that, however I am trying to educate as many as possible. Some yahoo sent me a Visio drawing the other day; I sent him a message saying, "save it as jpeg or png so I can read it". He did, and I was happy (not to mention I could easily incorporate his drawing in my own documentation/notes or translate it to some other format if needed).
This happens all the time, someone sends me a Microsoft Project file, or some other format that I do not have software for. I force them to change it to an open format - and after awhile they learn (at least to send me data that fits my open model - or that I can translate to something open). When their tools are dead and their files are useless, I will still be able to reference information that happened in the past.
So, your argument about software is only valid if you use proprietary file formats (only readable by one software application). I do not - and so your argument is not valid for me.
No one should use proprietary file formats for this reason - proprietary file formats hinder the migration of data from one technology to another (hardware or software).
Lodragan Draoidh
The more you explain it, the more I don't understand it. - Mark Twain
Some people have already approached the problem of making some data readable after a very long period of time - The Roseta Project. While their medium isn't digital, it is extremely durable and technology independent. It only takes a conscious observer to be able to (gradually) read it. Great idea.
What about Decentralized (P2P) Replication of data? Perhaps the sheer number of nodes on the internet today combined can replicate all data to the point that a file's SHA-1 or whatever can be used to retrieve that file at any point in the future. And, 160 bits isn't hard to physically write out to another medium, either.
The answer to every question is Decentralized P2P. Or at least I think it is.
http://pixelcort.com/