Slashdot Mirror
Bit Rot Stalks Your Digital Keepsakes
axlrosen writes "The NYTimes has an article about the problems of digital archiving. How many of your digital memories will still be around 50 years from now, considering lost disks, incompatible formats, hard drive crashes, fading CD-Rs, etc.? Unfortunately Peter Briggs' solution won't work for most of us. The only real way to make sure that your grandkids get to see your digital photos is to make real photographic prints from them. (When I bought my Mom a digital camera I installed Picasa for her, and made sure she knows to order real prints of all the pictures she wants to survive through the ages...)"
Half of my 5.25" floppies don't work anymore!
move your stuff to the next "permanent" media
"REAL Photos" wear out too.
Short of having titanium punchcards with your data bits punched in (and even then...) you are simply going to have to keep backing up and backing up. I'd rather have my data on 2 new hard drives than a dozen decade-old ones.
That's why I still use punchcards.
"Backups are for wimps. Real men upload their data to an FTP site and have everyone else mirror it."
--Linus Torvalds
E pluribus unum
But you have time to read a story on
Alien v Predator script saved by Internet pirates
Amazing anaecdote from Peter Briggs, the author of the screenplay for Alien Versus Predator.
User "gaygaygay"
Pass "gaygaygay"
Hope that helps.
I am going through similar problems right now. I have about 30 floppies containing drafts of my mother's first novel. She wrote it in the early nineties on an IBM, using some early version of wordperfect.
I decided to recover them and save the data on a CD, and I realized I didnt have a floppy drive installed on any of my machines! Somewhere in storage I had a USB floppy drive, but I cant get any software to read her files.
My solution: buy antiquated hardware.
I use Microsoft Word to print out all my MP3s, which I then store in a 3-ring binder. If I ever lose my digital copy, I can use text recognition to restore my MP3s from the paper backup.
Let's just hope there isn't a fire or a flood.
But the main problem is not the "end of life" of media used for storage, is the format in which the information is. In 50 years, will be an application that opens/process that information? One of the advantage of having information in open formats is that in the worst case, you can have all the information to be able to process them. But if you stored your information using an applicaiton with its own propietary/closed format, and the company just decided to not support that format anymore, or just closed, you could have lost your information, even if the media where it is stored still retains it well.
The fact that digital data rarely goes from "Perfect" to "Ok" to "umm not so good" to "What is that?".. it tends to go from "Perfect" to "Gone/Maybe not gone but very expensive to retrieve," makes it's worth discussing the finer points of digital archival versus analog.
Who gives a shit? I'm 39, and too mentally ill to attract a wife, so no kids. What am I going to leave behind? A collection of snotty and angry online postings? I just want to retire early and pursue my long denied hobby of global agitation.
And why doesn't the posting preview here work reliably with Firefox?
--- Ban humanity.
Phisical Data such as paper, stone, ... will demish the more that it is handled and there is often some loss when it is copied, but you can keep it in a safe box for hundreds if not thousands of years. Digital Data is the oposit, In order for electronic digital data to survive it needs to be moved around and each copy is the same as it was before. That is why the music indrustry hates MP3 way more then copying Tapes. With MP3 each copy is as good as the first. With tapes they can only be copy only a fiew times before the quality gets really bad. And there is only a limit on how many times the master tape can be played. But Data just as long as it is moving it is more protected.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
The link ( http://www.boingboing.net/2004/11/06/alien_v_preda tor_scr.html )
to the info on Peter Briggs has porn ads, for those to whom it matters.
Couldn't you have warned us?
there isnt a problem with digital records so long as you keep your formats up to date and have backups. generally, it seems like you should revisit data that is two years old to check if the format needs to be brought up to date before its too late. another good idea is to avoid anythign proprietary, including weird Microsfot implementations of common standards. saving digital photos as simple .jpgs is a better idea than saving them as photoshop documents for example.
also, dont forget that scripting can be your friend. i use all kinds of applescripts to manage and batch-process my colleciton 1000's of digital photos so that i dont have to drop them into something proprietary like iPhoto.
Posting links that require login isn't particularly new. Do you complain about them EVERY time they're posted?
Use http://www.bugmenot.com/
Thank me later.
Yes, I am a smart ass; it's better than the alternative.
I don't think the basement really qualifies as being a separate house. I mean, what if the whole place goes up in flames?
Sounds like FUD put out by Kodak, or maybe Epson, and not "news".
Photos, slides and negatives don't last forever, just one look at the slides my Dad had in his house in Hawaii will illustrate that. But moving them to a new form of media is a lot more cumbersome moving 5 CD-Rs to a single DVD.
"Printing" is a bad way to save a picture, inkjet printouts degrade faster than true photos. You'd need to output to a real photo to get the same lifespan as a photo. Oh, and if you do, keep the digital copy, it's going to be better than a scan of the photo that's been sitting on the mantel.
Are there many consumers out there with more than 120GB of family digital photos? A spare hard drive is cheap these days as an additional place to store a copy.
Want to have your photos at home as well as somewhere safe in case of fire? It would be pricy to made dupes of all your slides or photos, but a second set of CDs pretty cheap.
There might be people who saved digital photos on floppys ( like those who got the cheesy Sony floppy cam ), but that media is not opsolete yet and for $20 you can have a USB floppy drive to let you move them to a CD.
Old media meant that the cost of the dupe was pretty much the same cost of the original. This doesn't lend itself to redundant copies at multiple locations for most people. Digital lends itself to duplication, just ask any movie pirate.
There are films from the 20's that are lost forever. Thanks to DVD pirates, we have enough redundant copies of Star Wars that it will never be gone.
Does this make the case for parity archiving?
Damien
-----
Free P2P Backup, Windows & Linux
I store my data on redundant arrays of disks in two geographical locations (my house and my parents' house, synced nightly via rsync).
Do you run rsync with --delete? If not, how do you deal with moved files? If so, how do you deal with accidental deletion?
I grant that you've solved the decaying media problem, but I've lost more data to screwups than to bitrot.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
I have three categories that I put data into, that I've decided to call Red, Yellow and Green.
Red data is the most important and irreplacable -- things like financial data, things I've written, important emails, family videos, etc.
Yellow data is not as important, but would be in inconvenience to replace -- things like purchased software, esoteric software drivers, etc.
Green data is data that I like to have on hand but that could be easily replaced or that are updated frequently -- things like Linux distributions, freeware, etc.
For the Red data, I create PAR2 parity files and burn 3 copies (with the PAR2 data). One is stored at home, one at work and one in a safety deposit box. Sensative data (like financial data) is encrypted with a key located in the deposit box.
For the Yellow data, I burn 2 copies. One is stored at home and the other at work.
For the Green data, I burn 1 copy.
I will still need to keep an eye on the Red data and check the copies once every year or two, perhaps reburning to the most current technology, but I feel fairly confident that this data is safe, having three copies in different geographic locations, each with redundant parity files (with 10-20% redundancy) that can be used to reconstruct damaged data.
YMMV. Hopefully my scheme works; I haven't had any catastrophic events that affect my data yet.
1) Put a picture of a naked chick in the background of all your photos.
2) Put photos on the internet.
In a few days, your photos will be archived on and available on digital and print media throughout the globe, and will never disappear until mankind goes the way of the dodo.
- rouftop
QAExpress: Solid bug tracking for you. Graphs and reports for your PHB.
Is there a service where you can copy your color negatives to three b/w negatives, one for each color layer, so they can be recombined later to make a full color image? This strikes me as the best long-term analog solution to losing precious color pictures.
I just have to say, as a formally trained photographer, anyone who thinks color prints will last 100 years are dead wrong. The caustic nature of the chemistry used in traditional color printing, especially your typical drugstore-type one-hour photos, pretty much guarantees the utter destruction of the prints within a few years.
Hand-printed black and white paper can be much more archival - the medium lends itself to it. BUT it is still highly dependent on the process, care and storage of the prints. You must store them in acid-free environments, out of direct sunlight, etc etc.
I have a degree in photography and emphasized on archival techniques and I STILL have issues with color photographs that have degraded in 10 short years. Take that fwiw.
There's a photo shop in the mall around me that can take all or most digital media containers and print out only the prints you want. If you want more than eight or so pictures, it's something like 1/3 the cost of doing it on my home printer.
And they have 1/2 hour delivery, so they'd be done by the time you meander to the opposite end of the mall and back. For true convenience, there's internet sites that allow you to upload your pictures for printing, then they mail them to you.
I don't read AC A human right
For those of you who really need permant storage, drop whatever you want preserved into a black hole. The gravational waves produced will carry information (heavily encrypted!) into eternity...
Retrival may be a challenge.
HA! I just wasted some of your bandwidth with a frivolous sig!
Until Shutterfly either goes belly-up or changes their user policy. Then you're screwed.
I make a backup of everything important once a year and take copy to my parents and the cottage. I take an incremental backup with me anytime I go visiting.
My kids will have bigger computers and any digital photos will just live on by being on their computers. And their grandkids computers and ...
Recently one of my aunts scanned in and touched all my grandmothers photo album. Now that album lives on CD and Hard-Drives of most of her 13 kids and 35 grandkids. Now nobody really cares who gets the original album.
Digital medium is SOLVING the problem of the loss of this type of heirloom data -- not introducing a problem.
Oh boy, a field day for the /.ers that have been squirreling away all that "obsolete" hardware. Quit running Linux servers on those machines, boys! They'll be too valuable when Kinkos is ready to buy those "antiques"!
I knew I should've held on to my Apple II...
It's such a fine line between stupid and clever.
There already are.
If you figure that most people's data is under a gig. We're not talking about system images here, just the "my documents folder" and it's ilk. Less than a hundred megs/month with most people, including photographs, unless they really love their mpeg home movies.
With decent broadband and some system to do the backups during non-peak hours, you can easily do tens gigabytes a month. Will it cost? Yes, but it's like doing your own car repairs. Unless you have a garage equiped like the shop, they can do it quicker and easier than you. Is it worth paying $20 a month for not having to worry about backups for a couple hours a month, would you seriously consider it at $20/year?
2nd paid advert for "web backup" on google
I don't read AC A human right
You should pick up a Catweasel. It's a universal floppy controller for old media which can read Commodore, Amiga, Mac 800k, and other formats directly with modern floppy drives.
:)
The new Catweasel apparently also includes joystick/paddle ports and HardSID functionality. Yesss!
As far as beating bitrot by multiplying the data: You can also use software FEC encoding to add check blocks to the data, growing it by less than an integer multiple. Repairing the errored bits is automatic, whereas storing multiple copies of the file still gives you no easy way to tell which copy is correct.
Periodically rewriting the data and correcting for small errors that occur will prevent the accumulation of errors too large to be corrected. In RAM this is known as memory scrubbing and is used on some high-end servers to counteract cosmic rays and bit-rot.
It's also a good way to detect impending media failure. Your drives should have SMART enabled, so you know when they're covering up a growing problem, and can get your data out of harm's way. This only protects against gradual deterioration however, and is no substitute for a backup in case of catastrophic drive failure.
These questions are dealt with all the time by serious archivists. Storing metadata to provide context is important too. Historians of the future will probably have a thousand copies of "Driller.d64" but will they know what the original floppy label looked like?
On it's original media, even! My second PC was, luckily, a Mac 512K. I've still got the system disks for it, with the original MacPaint and MacWrite disks. I've still got the first doodle I've done in MacPaint on 3.5" 400K diskette, and my PowerMac 6100/60 still reads it fine. When my all-singing, all-dancing Linux-based windows/appletalk/NFS/novell server is up and running, I'm going to back up everything onto RAID, then optical. As long as I keep cycling backup strategies, and keep offsite backups in a safety deposit box, all my data should be secure for quite a long time...
My Other Computer Is A Data General Nova III.
Maybe not a direct comparison, but for quite a while Sony's digital music players supported ATRAC in lieu of MP3. They got tired of falling way behind the iPod. People were unwilling to have to convert their existing tunes to ATRAC and to not have their ATRAC files be swappable with others.
It was nice example of the market moving a big company away from a proprietary format. It probably won't happen with MS as more things out there keep adding support for WMA, but it at least shows it can happen.
Enough people shied away from Divx to kill that format as well in lieu of real DVDs.
So for the moment, I'll have a little faith.
The major downside was that it used a very unusual disk form factor: 3" - while the rest of the world standardized on 3.5"
Yes! That's exactly what I was trying to think of, but my brain is suffering from bit rot, and the data couldn't be recovered... good thing I had your brain to use as a backup. :)
~REZ~ #43301. Who'd fake being me anyway?
The problem that I've seen with these online photo albums is that they don't let you download the original uploaded file...you can only view an oversized thumbnail. Doing a right-click -> Save As won't get you the original quality of the image.
Mystic (dunno about the others) does save them for a time being, but you have to pay like $.50 to download the hi-res image. If you wanted to do this to hundreds of images, that's hardly cost-effective.
If there are indeed sites that allow it, I'd be all for using them, but due to costs, I doubt they'd be offered for free. In fact, my mom has been doing that with Ofoto, and she got a message that since she hasn't bought anything in a long time, they may delete her albums. One would be better off paying for cheap web hosting with a lot of storage space.
Here is a place to start with...
-Phlack
Yes, yes. There the insane ones.. Why wouldn't women date guys like us? High IQ's, steady paychecks, dashingly good looks from 200 yards, No noticable odors from 300 yards, ego's the size of large cities. Man I tell you, I am this close to cloning myself , but switching the sex to female. Yeah, that would show them. That would show them All.
Well.. maybe. Or Maybe not. But Definitely not sort of.
This page has a number of public domain schemes that might work. Some are fault more fault tolerant than others, but many of them should work.
Serious use of these schemes would require some kind of "Rosetta Stone" document or sculpture to make breaking the codes easy. If the archivist was to act carefully, I bet it would be possible for great-hoevermanygenerations-grandkid to break the bar code scheme, just by knowing a that the pattern is a rational symbology and by having enough repetition, of course. Automating the process would of course take some Perl scripting genius, or whatever they will be using.
Quality paper can last a really really long time.
Why do I have this? I don't smoke.
Word. Saying an actual picture will be more permanent than a digital copy is ridiculous. Maybe more permanent than ONE digital copy, but that's the whole point of digital, being that you can copy it.
In a billion years everything you observe around you is going to be blown away when our Sun becomes a red giant and engulfs our planet, making Mars the new Mercury. If I were you I would take all my CDs and DVDs of family photos and have them launched into orbit around Neptune. There they will chill for a huge period of time until the Arturians finally reach the source of all that bad programming that was radiated into space, our Solaris system, just in time to watch the fireworks. There they will find the pathetic works of a now extinct civilization that looked up the stars but kept getting distracted by his boot on his fellow human's head. The few remaining trinkets and images they discover will be taken back to a museum on their world along with various Pioneer and Voyager spacecraft, only to be lost in a freak warehouse fire a month later due to faulty wiring. A hearty laugh they will give as they note to each other, "see, their little image makers stole their souls after all."
"Beware of he who would deny you access to information, for in his heart, he dreams himself your master."