Slashdot Mirror
Best Way To Store Digital Video For 20 Years?
An anonymous reader writes "My kid is now 1 year old and I already have 100G of digital video (stored on DVDs, DVD quality) and photos. How should I store it so that it's still readable 10 to 20 years from now? Will DVDs stil be around, and readable, 10 years from now? Should I plan for technology changes every 5 to 10 years (DVD->Blue-ray->whatever)? Is optical storage better, or should I try to use hard drives (making technology changes automatic)? And, if the answer is optical, how do you store optical disks so that they last?"
no they're not. ever hear of cd rot?
store everything on hard drives, with duplicate backups stored off site.
Pictures: Backed up to HDD, DVD and Flickr. For $24.95, it's cheap offline backup and the grandparents love it.
Movies: Taken on MiniDV, backed up to HDD.
The only worry I have is that the MiniDV's and HDD are in the same house although they are stored in separate locations. But every picture is backed up offsite.
Claim up to 300 years.
http://www.smarthouse.com.au/Home_Office/Storage/U9P4F7L2
HDDs are so cheap. Buy an external one with like a terabyte of space. Fill it up, rinse, repeat.
Crackin' Wise - Blogging about whatever we want
Currently, There is no better way than store a backup on DVD and store the main data on a raid-1 disk set. Move the raid disk set to new disks every few years.
All the other technologies are more expensive, and even possibly more dangerous (loss of data due incompatibilies or for any other reason).
If you strive to keep it all accessible all the time, you will move with format changes as they occur. US networks are not capable of HD video streaming, so I put OGG Theora in my video blogs with links to better quality for those who want it. Disk storage will improve in time to keep up with your vorracious demands. Raid would be good to have. Optical storage media that has to be loaded one disk at a time is a last ditch archive that you should keep in a seperate physical location, just in case.
Intellectual property was the desert property of the twenth century.
Depends on the manufacturer and dye formulation. Some have failed in as short of a time as eight months while others are good after nearly ten years. For very important stuff, it is far too risky to be relying on the manufacturer. It's probably safer to make it a habit of regularly make multiple backups your data.
ELOI, ELOI, LAMA SABACHTHANI!?
In the department I work for we typically use archival DVDs in a temperature and humidity controlled room (also used to store photos, slides, and vellum). For the really important ones I'll copy the disc onto a server in the same room as an ISO. Every month I mirror the data drive onto an offsite server in another building on campus. It's not fool-proof and it's pretty expensive but it has worked for about 8 years now.
"Never let your sense of morals prevent you from doing what is right" - Salvor Hardin
Well why not optical platters? We have 50 year retainment requirements for certain documents and were looking at Plasmon optical devices. They claim it will still be readable and are the only type of backup media that survived both 9/11 and Katrina. Although when I asked if it was the same cartridge that survived both, the vendor gave me a dirty look. I think though you would be fine with dvd-r and just make a new copy every 5 years.
Only wimps use optical media, _real_ men just upload their important stuff on ftp and let the rest of the world mirror it.
Store the media on whatever the current standard is. Think about it, what if you had a closet full of tape reals that had all of your old sweet groovy 60's music? What would you do with that now?
-FizzGiGG (Geek)
Build a simple storage array with RAID from a barbones PC, your favorite Linux distro, configured for fault-tolerant RAID. It doesn't have to be complicated, and it doesn't have to be powered on unless you're actually pushing data to it.
Every couple of years, you can add an extra couple of drives. With drive capacities increasing as fast as they are, cost shouldn't be a huge issue.
512 MB RAM, 20 GB disk, 200 GB transfer, five datacenters. $19.95/month.
Regardless of the methods you choose, I would highly recommend using at least two different media.
If these videos are important enough to be stored for 10 to 20 years, then they are important enough to be backed up - it is always difficult to foresee long term failures in any technology. If you read the article on tin whiskers they mentioned that some failures can not be tested using short time span methods.
a few years ago, this would have been exorbitantly expensive overkill, but this stuff keeps getting cheaper by the day
with raid5, your videos will last forever, as long as someone keeps replacing the dead drives
any other media format is physically static, which can degrade. raid5 ensures that the files live on after the physical components degrade, as long as new drives are continually added to the system
and when the technology becomes ancient and archaic, simply move the files over and upgrade (obviously to a new file format as well)
as long as some continually performs low level maintenance, your videos will last forever
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
My kid is now 1 year old and I already have 100G of digital video
/.
Riiiight, this is
So people: How can this guy be sure that in 10/20 years time his Bangbus collection will still be readable?
Use multiple different media, with redudancy.
Store it on HDDs. Mirrored RAID like RAID 1 or RAID 10 is preferred. but even RAID 5 buys you some extra integrity protection.
Then back it up. CDs. DVDs. BluRay. Tape. Whatever. Multiple times, multiple ways. Every few years do some copies onto new media.
Keep at least one copy off of your premises. A safe deposit box might be good.
My blog
If you can afford it, I'd recommend a utility computing platform, like Amazon S3 or whatever Google's offering in that space. Verify that they're built out for long-term, fault-tolerant storage (ie: replication + automated verification and repair.)
I wouldn't trust that 100%, though, so keep them locally as well.
Video I posted 20 years ago is still there....
Unix, an obscure operating system developed by bored researchers in an attempt to get a better game playing experience.
nope. All magnetic storage have a (relatively) short storage life. Optical is much better if they are stored properly (ie. cool dry place and not touched much to avoid scratching)
Common Sense isn't as Common as people think...
Instead of hiding behind the camera the whole time, actually interact and play with your kid. The videos and memories aren't as interesting as who the kid will become.
1. Rename to "xxx 18yr old bj strip"
2. Upload to P2P protocol of choice.
Let it proliferate around the internet and retrieve it when necessary.
As for storage, I would personally go through and put together maybe a movie and and picture viewer DVD for each year. And then have those professionally mastered onto pressed discs. Keep those in your fire-proof storage and use burned copies for everyday (I hope not) use and sending to relatives and what not.
Shift happens. Fire it up.
As the other guy mentioned, CDs are still readable, almost 20 years later. However, they didn't have a viable alternative until about 10 years ago. I think that you will easily be able to find a DVD drive for many years to come, at least the next 20. The problem becomes ensuring that the actual media doesn't get scratched. I wouldn't trust DVDs to last that long, even if you just leave them on a shelf, away from the sunlight. If I was really interested in saving the stuff, I would put it on hard disks with at least 1 redundant copy, if not 2, stored in different places, and transfer over every 3-4 years. Still, it's going to be a lot of data. Your kid is only 1, and you already have 100 GB of stuff. Just think about how much that will balloon to once the kid has an attention span of more than 43 seconds. The first hockey game, all the school plays, all the other junk you could record.
Personally, I just don't bother with recording much. My wife gets on my case for not taking a lot of pictures with the kids, but I'd rather be interacting and paying attention, rather than trying to ensure we have everything recorded. Sure sometimes like during school plays you can record and not miss anything, but a lot of times, I find when I'm trying to take videos, or photos, I end up missing out on the actual fun.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
isn't long term storage, though it may have it. The strength is that you can reproduce it with high fidelity to the original numberous times.
The best way to store digital vidio for 20 years is to make numerous copies of it. 10Gigs is about 3 DVD's at the lowest density. Add a dvd of checksum files (something like a PAR) and you should still be able to make five sets for under $20 if you are shopping around for DVD media.
Once a year or three, load up one of the sets and run it through the checksums. Correct any errors discovered via the checksums and copies from the other sets, and make another five sets.
Volia. Repeatable as long as there is any sort of cheap digital recording media that can easily fit your files out there.
The real question is how you do this when you have 1,000 Gig to backup.
Step 1: Review video footage.
Step 2: Carve memorable/important parts into stone.
Step 3: ??? (commandments?)
Step 4: Prophet!
"Never let your sense of morals prevent you from doing what is right" - Salvor Hardin
Quit being creepy by chronicling every time your kid goes number 2. He'll thank you, as will everyone else who knows you.
If you're not trying to go cheap, get a tape drive (DLT, LTO, or AIT, not the quarter-inch or DAT crap). If your time isn't worth that much, migrate from optical format to optical format every few years. Either way, keep your backups off-site.
Hard drives and just not suitable for (home) archiving - one robbery, fire, or natural disaster and everything's gone forever. If you add backup to those hard drives, then we're back to "what format?".
Socialism: a lie told by totalitarians and believed by fools.
As I understand it, holomagical disks will be loaded with so much DRM, nobody will be allowed to view the contents. Period.
With S3 you'd pay $15/mo (+bandwidth) to have it hosted online, instantly accessible. Will it still be around 20 years from now? One can't be certain, but if not, I'm sure you'll have enough warning to copy things off to another medium, and I'm sure there'll be similar services to take its place if need be.
I was under the impression that unspun drives tend to deteriorate relatively quickly - the heads clashing with the platter or some such nonsense. Just spin them up once a month and you're fine, from what I've heard.
How are sites slashdotted when nobody reads TFAs?
Only professional CDs have that sort of shelf life, because they're physically stamped. The consumer grade ones use a type of photosensitive dye that DOES decompose in less than a decade.
ad logicam Claiming a proposition is false because it was presented as the conclusion of a fallacious argument.
The only way this could be true is if the data were rewritten.
Reading alone has no effect on the data.
Newegg has Quantum DLT SATA drives (160GB native capacity, 35 GB/h throughput) for about $700, so it won't break the bank to get proven multi-decade shelf-life media of reasonably size and speed for a 100GB dataset.
Every real OS has tape backup support (though you may have to hunt for drivers). If you're stuck with Windows, type ntbackup at the command line - it doesn't suck for home use.
Socialism: a lie told by totalitarians and believed by fools.
CD bronzing is a specific variant of CD rot, a type of corrosion that affects the reflective layer of audio CDs and renders them unreadable over time. The phenomenon was first reported by John McKelvey in the September/October 1994 issue of American Record Guide.[1][2] Affected discs will show a uneven brownish discoloring that usually starts at the edge of the disc and slowly works its way towards the center. The top layer is affected before the bottom layer. The disc will become progressively darker over time; tracks at the end of the disc will show an increasing number of audio problems due to disc read errors before becoming unplayable. CD bronzing seems to occur mostly with audio CDs manufactured by Philips and Dupont Optical (PDO) at their plant in Blackburn, Lancashire, UK, between the years 1988 and 1993. Most, but not all of these discs have "Made in U.K. by PDO" etched into them (see image). Discs manufactured by PDO in other countries do not seem to be affected. A similar, if considerably less widespread problem occurred with discs manufactured by Optical Media Storage (Opti.Me.S) in Italy. PDO acknowledged that the problem was due to a manufacturing error on its part, but gave different explanations for the problem. The most widely acknowledged explanation is that the lacquer used to coat the discs was not resistant to the sulphur content of the paper in the booklets, which led to the corrosion of the aluminium layer of the disc, even though PDO later said it was because "a silver coating had been used on its discs instead of the standard gold."[3] Peter Copeland of the British Library Sound Archive confirms that silver instead of aluminium in the reflective layer of the CD would react with sulpheriferous sleeves, forming silver sulphate, which has a bronze colour.[4] A combination of the two factors seems likely because, as Barbara Hirsch of the University of California points out, the oxidation could only have occurred if the protective lacquer did not seal the metal film and substrate well enough.[2]
Those were from Wikipedia, fact is, though CD rot can be a problem, it isn't as bad as people make it out to be.
Taxation is legalized theft, no more, no less.
Honestly, you're probably not going to look at most of the pictures in 20 years anyways. Sure it's good to have a lot, but with 100 GB of pictures/videos, that's pushing it a lot; babies don't exactly do much. Go through the collection and cut out the bad/redundant pictures, print off the especially good ones, and put em on the wall: they'll get more use that way. But... with what you do save, try burning to CD/DVD/blu-ray, since they don't degrade too much over time, and if you have space/money, archive it to tape: it'll last the longest. HDD's won't last the 20 years, but if you want to, try a RAID, but it'll have to be recopied and replaced every so often. And... with whatever media you choose, keep it away from light/dust/too much humidity.
01110000 01010111 01101110 00110011 01100100
No one's brought it up yet, so I will... As the price/convenience/long term compatibility and viability of storage goes down and down, I wonder to what end we will end up keeping this stuff? How many hours of video that you're paying (in time, money, security against fire/damage/loss, etc) to keep up you're actually going to watch? Sure, it's nice to have every single event in your child's life on demand at the touch of a button/click of a mouse, but aren't just plain old memories ok? Does his entire life have to be recorded and watchable?
At some point, I came to the realization that I had downloaded over 6 solid months worth of music. This doesn't include CD's, LP's, or 7 inch records, of which I probably have 1000 total. If I were able to put all that music on a big loop, and not repeat anything, I'm thinking it would last over 12 months. Some of these I'll probably never listen to. I'm thinking the same is true for the submitter's videos.
My parents have a big box of photographs from their childhoods, as well as those of their parents. There are some great photos in that box, and I could and have spent hours going through them. Each time I do, I make a mental note that one day I'll scan them and make them digital. Then I realize that we only drag out that box once or twice a year, and never do anything with the photos anyway, and resign to scan them once it gets even cheaper.
100G of video in the first year? I guess you should archive it...he can show it to his therapist when he gets older. One way to solve the archiving problem is to do some editing (serious editing) so there's much less to store.
People who say HDD have their heads in the sand. 20 years. Think about that. 1988. SCSI-1 40 pins. Nearing the end of MFM/RLE. Parallel.
People who say CDs and DVDs again have their heads in the sand. That's the Floppy Era.
The best format IMHO is the "current" format. DVDs + HDDs along with a live copy on your computer. DVDs and HDDs should be at two of your friend's houses.
5-10 years later, once one of the formats is obsolete (EXT3 is now EXT8, DVDs are now expensive again in drug stores), it's time to copy these to the new "current" format, and repeat the process.
So in his first year of life, you've recorded around 34-35 hours of footage? (going on single layer dvd capacities and mpeg2) Ask yourself, when are you going to watch all of that?
I don't have a single CD that has succumbed to "CD rot". I've had some rendered unuseable by scratches or by being left in the heat/sun too long by accident, but other than that all my CDs, even from the late 80s and early 90s are completely fine. The data CDs I burned in the early-mid 90s are also still fine.
I'm not saying that it doesn't happen, but people make far more of a big deal about it than is really warranted.
That said, anything I wanted to make sure was still good I'd "refresh" every 5 years or so.
"Growing old is inevitable; growing up is optional."
Collude with a friend to email back and forth encrypted copies of your photos. Arrange for them to be perpetually stored in transit on somebody-else's mail server awaiting delivery.
Better still - uucp them over some convoluted circular path back to yourself.
Or rig up an ultra-high-speed moonbounce communications system...
Just keep them all in motion and they won't get lost.
Nullius in verba
Duct, Electrical, Masking or Transparent?
It must have been something you assimilated. . . .
Print the bits out on paper to be scanned later, as necessary. You should make several copies and store them in different locations incase of fire or water damage. To answer your next question: Land in Montana and the Yukon territory is cheap.
If anyone knows of a way to read my old CD-R's, even if it is [b]one[/b] time only, let me know.
Unfortunately there isn't a guarantee on any technology. CD/DVDs were supposed to last 100 years until that pesky mold and poor quality make them unsuitable for long term storage. HD-DVD was promising until it lost the format wars. HDs reliability varies with manufacturer and model. My suggestion is to back it up every 5 - 10 years onto new media to keep ahead of the curve. It's more work but you'll make sure it gets saved.
8mm -> VHS -> DVD -> Bluray -> Profit!!Well, there's spam egg sausage and spam, that's not got much spam in it.
Qualilty digital tape will last 30 years. There's "100 year" digital optical media (if you believe it), but it's very expensive per-byte.
DARPA did a research project to create a storage medium that would last for centuries after a nuclear war, and be readible with very low-tech gear. They invented a metal punch-tape format - very cool.
Socialism: a lie told by totalitarians and believed by fools.
Forget Wikipedia, ask the people who spend their lives trying to figure this out.
http://www.digitalpreservation.gov/you/digitalmemories.html
http://www.archives.gov/preservation/technical/guidelines.html
http://www.archives.gov/preservation/family-archives/digitizing-photos.html
http://www.archives.gov/preservation/storage/
I suspect that in a few decades, estate planning will include what to do with the family terabytes.
Printing movies on paper is crazy but not unprecedented. It's not helpful to the OP, but Hollywood places full prints of its movies (actual prints of each frame, not barcode) onto paper for submission to the Library of Congress. This way they can fully copyright the material, yet leave it in an otherwise useless form. Leave it to Hollywood to think of something like this :-p
Chisel what you want to keep onto stone tablets, or use clay tablets then encase them in a clay envelope. Then bury them in a pit in your back yard.
Steve's Computer Service, Hobbs, NM
Is it just me, or do you find that besides a few photos meant to invoke memories, its better to remember something than to record it? I find If I over-indulge in 'capturing the moment', all I seem to have left was the content and I forget what it was like to -be- there.
Bye!
Man, you got really upset over that RAID suggestion. With a name like 900ftJesus, I would have expected a more kinder, gentler approach.
CD-Rs are not the same thing as stamped CDs. With CD-Rs you're lucky if they last 5 years. Stamped CDs, if taken care of, will last practically forever.
Skiffy is Spiffy, but Ort is tort.
As a general rule, for long term storage any technology that relies on magnetic data storage has a much shorter lifespan than say optical. Over time the strip looses its magnetic charge, causing data loss and corruption. I would go for optical storage (DVDs a big yet cost effective). They dont have near the base degradation over time that Magnetics do. A few things to keep in mind for DVD backup: -Buy Good Quality. Cheap disks tend to flake off and loose the back coating. I dont know how many movies and anime Ive lost that way. -Store them somewhere cool and dry, much like any electronic. -Be careful of wallets. Ive had several wallets and binders that cause friction on the back surface, causing scratches and flaking of the foil coating. Id go with spools. -Dont mess with them. The more you touch them, take them out to look at, etc, the more chance of them getting damaged. You also run the risk of getting dirt in the storage (wallet or spool, etc) which will cut down their life again. The recommendations of multiple backups is also a good idea.
Common Sense isn't as Common as people think...
More accurate would be "CD content still readable". CD rot is only an issue if you made too few copies and didn't re-copy to newer media periodically.
If it's really, really important:
--Save the stuff on reasonably reliable, name brand media. Make more than one copy, saved separately.
--Make the secondary copies on a different brand media, just to cover your ass.
--Copy over to identical media type a year or two later ( save originals )
--Copy/consolidate to the newer, cheaper media ( like CDs to DVDs) when the price on the new media drops a bit. Include drivers, codecs or even a player or two on each piece of media that consolidated several smaller pieces of media ( 5 CDs per DVD for example ). (Save originals )
--Repeat the re-copy to same type of media, repeat the consolidation to newer/larger media.
If original format is getting old or unusual, convert/transcode to newer format in the least lossy format. Save transcoded copies in addition to the originals ( don't throw away your negatives )
With the advent of consumer digital video/audio there's no reason to lose anything. Even saving all these CDs, DVDs, Blu-Ray disks would take less physical space than my Dad's box of 35mm slides that cover the same 20 year period.
Hell, why not go the whole hog and bury a small PC with the drive and DVDs? Though, I do like the other suggestions of using a tape drive. Some of my CDRs from the 90s degraded after just a few years packed in my closet.
Back then we had to hand flatten the metal parts ourselves and burn them using a magnifying glass and the sun. And unless you had like 3 espressos, the only speed you could do it at was 1X.
"But this one goes to 11!"
CDs, perhaps. CDR's? No, they're not.
I paid a ton of money for one of the first 1x CDR drives for PCs back about 14-15 years ago. Recently I've been moving all the data I've built up over that time onto HDs for longer term storage.
What I've found is nearly all of the discs from back then I can read -- the $20 a pop gold discs.
Starting with the discs from the very late 90's, I'm getting about a 50% failure rate (on discs stored in climate controlled conditions away from light). With some brands (and not necessarily low end ones), I'm getting nearly 90% failure rates after just seven or eight years.
(And I consider failure to mean a clean disc, at least one file can't be copied anymore...)
Since he's not going to be pressing glass masters and casting pressed CDs, I'm not sure non-recordable media longevity matters one bit.
True that. I have CD's I burned in 97/98. I pulled 'em out recently to cull the data and put it onto a DVD only to find that it was garbage. The disc couldn't even be read. There are some that are better than others. If you google for archive quality media you'll find countless discussions on it.
You are using English. Please learn the difference between loose and lose; they're, there, and their; your and you're.
I was under the impression that unspun drives tend to deteriorate relatively quickly
Quickly? No.
- the heads clashing with the platter or some such nonsense.
Head crashes have been a non-issue since the late 1980s.
The real problem is the lubricant in the tiny motors. It can get gummy, and then the read arm won't move.
Just spin them up once a month and you're fine, from what I've heard.
I'd say bi-annually.
"I don't know, therefore Aliens" Wafflebox1
Build a simple storage array with RAID from a barbones PC, your favorite Linux distro, configured for fault-tolerant RAID.
If you're not a Linux person, that *IS* complicated by default.
Not that it would really be less complicated with Windows, but only a Linux person wouldn't recognize the inherent complication of RAID.
paintball
Oh are we allowing theoretical products? In that case I would back up to a specialized bacteria. I would encode the data in it's DNA and allow it to replicate regularly. Even better news, after 20 years, I'll have billions of copys to give to fanily and friends! It's win-win!
Copyright 2010. All rights reserved. This comment may not be copied in any way including, but not limited to caching.
Well , you could hire webspace to store the files. A good contract would ensure regular backups and redundancy . That way , you don't have to worry about it , but it's probably the most expensive option. And off course , in 20 years , that company might go bankrupt , leaving you with nothing
Slipping shoelaces ?
Has anyone mentioned Key drives or SSDs yet? Technically those items are limited by the amount of writes you can do to them, which is sometimes an obscene number. So far, the key drive I had for over 5 years is still working, even though its been run over by a car.
Thoughts?
A morning without coffee is like something without something else.
A few good pictures and a handful of short videos become "precious memories".
A slag heap of hundreds of hours of raw material become a burden that someone will eventually stop maintaining because it is such a chore.
Lots of pictures are less of a problem than video both because they are smaller, but because you can look at them faster to see if there are any worth copying, printing etc.
Send copies of your "best of" to friends and family so that you have off-site storage should your house burn down.
This issue is a bit more complicated than you think.
Raid is good for keeping the data alive, it is a backup in the sense of avoiding failure of devices causing major data loss. Raid 10 or a Mirror are your best bets for redundancy. However, a RAID is not going to be a preventative measure against other forms of data corruption, virus, batch file run amuck, accidentally deleting a folder. So you need a separate storage medium, you can pick your favorite, everyone has ones that work well for them. There are several alternatives depending on the size of data you are backing up. For me, I use online storage through a respected vendor MOZY for my home use, which kicks the ass of any other back up medium I have found, including tape. Tape is a dying backup medium in my opinion. I think what most people will be using within the decade are personalized net storage solutions.
BTW if you worked for me you would be fired for being an ass and pretending to know what you are talking about.
CS: It is all sink or swim...oh and did I mention there are sharks in that water?
Uh, IDE dates to 1986 which was 22 years ago by my calculations.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
My wife and I have taken thousands of digital photos and videos since 2002, but the cold, hard truth is that the vast majority of them just aren't very good. I suspect the OP is in the same boat. The solution is to take the time to identify the best images and videos. This should result in a relatively compact archive only a few hundred digital photos and videos. This "best of" collection will be much easier to duplicate in different format and making physical prints on archival quality paper won't put a huge dent in your pocketbook. Remember that just because you can take 2000 photos of your spittle-covered toddler doesn't mean that you need to keep them all.
Chances are pretty good that your great grandchildren aren't going to give a damn about inheriting a massive archive of pictures and videos starring weirdly dressed dead relatives they've never met, and if it's physically large it stands a good chance of ending up in a garden shed or unprotected lunar storage pod. If you condense your family album into something more manageable, it will be more accessible and enjoyable for all - now and in the future.
Your experience is very interesting because mine is the opposite. I make (at least) quarterly backups of my data and have since mid 1993 (CDs since 98, floppy before that). This spring I got bitten by the curisoity bug and started going through all my old backups looking for forgotten and interesting things. Every CD older than two years had at least one unrecovereable read error. Every CD older than five years, except for one, was completely unreadable. Between two and five years the number of read errors grew with many files being lost and several CDs being unusable. The 3-1/2 floppies were all 100% readable.
In that time period I've been through probably a dozen CD burners, both expensive varieties and cheap ones, and I've used at least as many brands of media. All the CDs have been kept stored in dark, dry, clean places. I tried reading the "unreadable" CDs on multiple computers and met some limited success accessing additional data. I didn't try any recovery software.
Fortunately for me I kept most of these backups out of habit and I didn't really care about much of the older ones outside of curiosity.
I smurf everything and everything I smurf is perfect.
CD-Rs are not the same thing as stamped CDs. With CD-Rs you're lucky if they last 5 years.
This very week I pulled out some CDs I burned back in 1999 with old email on them. The CDs were in beautiful condition, all of the data was perfect. If you want to be sure that data will still be there, burn it 3 times, put each one in a different Case Logic book. Keep two of them in different areas of your home and a third off-site. Upload all your photos to Google's Picasa Web Albums as a 4th backup if you want. I'd be willing to bet that 10 years from now that Google will still be operating and will not have lost any of your data. The service may have changed, but old data will be brought along to the new service, etc.From what I can discern from this, it seems that even if 'CD rot' does affect certain CDs, all of these CDS are of the pressed, as opposed to burnt, variety. AFAIK, there have been no reports of Philips CD-Rs failing in a similar way, and the manufacturing (including the data pressing/burning process) methods for both types of disc are different enough to rule it out as a cause for concern, aren't they?
I have CDs from back in 1998, when I've coaxed my boss to buy a 4x TraxData burner.
They still mount better and copy and open easier than some printed ones I got with various magazines over the years.
Aaah... but back then - a writable CD was about 5$ apiece and they only did up to 4x as did the writers.
I've also had (and still have) a large number of silverbacks burned by various people over the years.
Some of them were unreadable or had problems mounting the moment I've put them in the drive.
Just because it says somewhere on the sticker that it is 52X compatible or capable - it does not mean that it is.
That rule works for both disks and drives.
Also, whether it is cutting costs in manufacturing, packaging (I've found fingerprints on some "fresh" disks), quality control, transport or just plain lying about the performance - the ultra-cheap ones are cheap for a reason.
Mit der Dummheit kämpfen Götter selbst vergebens
Any storage medium you choose will degrade over time. You should plan to transfer the data every few years. Choose a storage medium that is well-supported, cheap and relatively durable - DVD or CD for now. You should also pay attention to the format of the data. If you use a video format that is rare now, chances are you won't have a way to convert it to whatever new format you need in 5 years. When it's time to copy (probably 5 years for CD & DVD, to be safe), use the same guidelines to choose the new medium.
I'm amazed that no one mentioned it. Just get 16gb usb flash disks.
It has theoretically unlimited life for archiving. The only time it deteriorate is when you continuously write/erase it.
Low grade garbage consumer CD-R's do that. you can buy high end CD-R's that have a gold substrate and a permanent dye that are guarenteed to last decades. I have a couple of TDK archival quality CD-R's from the very early 90's that were burned on a god-awful-expensive 1X CD burner that are still readable.
Do I store them on my car's dashboard? nope. I store them in a cool climate controlled media safe. I can still buy high end archival quality CD-R and DVD-R disks that I am sure will last a long time.
And if the 3.5" floppy drive is any example, CD and DVD drives will be around for another 10 years at least.
Do not look at laser with remaining good eye.
YMMV.
I have CDs from 1998 burned with SCSI cd burner and used cheap $1.00/CD bought in bulk that still read today.
if you steal from one source, that is plagiarism, if you steal from many, well, that's just research.
SCSI
I can read a SCSI-I drive in a Ultra 320 controller easily.
Do not look at laser with remaining good eye.
WW900FTJD?
Personally I keep 2 sets of copies a secondary hard disc, originals & edits. I also keep a master copy on standard Maxwell DVD+R that I check periodically generally when I add another volume, roughly 6 months time. I also keep another set on my gf's computer on the network, keeps her out of my hair in Photoshop
On the Oregon Cost born and raised, On the beach is where I spent most of my days
Wow, a misspelled spelling flame. It's like digging into the ground and pulling up a USENET post from 1993. Let me introduce you to a 21st century technology that might help you in future: it's called a search engine.
It's Taiyo Yuden, according to their own web site.
AMPAS recently had a report called "The Digital Dilemma", which the NY Times wrote about:
If not operated occasionally, a hard drive will freeze up in as little as two years. Similarly, DVDs tend to degrade: according to the report, only half of a collection of disks can be expected to last for 15 years...
What are film archives doing? Where possible, studios are making long-lasting, non-fade B&W pan separation YCM polyester negative film backups, even when the film is mostly or totally "born digital". Then you put it under a mountain somewhere.
Government video archives worldwide are moving to LTO tape, typically using JPEG 2000 video encoding, with the recognition that every few years they will have to migrate their tapes up a generation of LTO. I suspect there may be a move from lossy JPEG 2000 to lossless JPEG 2000 and eventually uncompressed video as tape speeds and capacities ramp up.
RAID5 is beaten by:
A) Disaster: Fire. Flood. Earthquake. /" can happen to the best of us.
B) Carless mistake or Idiocy: "rm -r
Nothing but an offsite backup will do.
I'd burn a copy to good quality DVD's in an offsite location (an office drawer at work does the trick for me) as you accumulate data. Replace the media every 5 years, as new media come out.
I already have instructions on how to access my archives, what is available to whom, and what to destroy as part of my will.
Fnord.
While in school for photography, I learned that CD's are not a long-term archivable medium. I wish I had the documentation in front of me to back up my claim, but unfortunately I don't (Yes, I kept all the information material from every class in college). Since the CD is light sensitive, similar to silver-halide film (don't take this analogy too seriously), the CD will degrade over time. DVDs are very similar to CDs in that they are 'laminated' layers of CDs and, as a result, will suffer more than CDs. I would never use DVDs to archive my photos. Kodak USED to make archivable CDs that supposedly lasted a lifetime+. Buy HDDs and don't look back...especially if the data is something you treasure.
WW900FTJD?
Obviously he would use a tape.Let me introduce you to a concept I call down-engineering. What happens is something is made really well. So well that the thing lasts a long, long time. This is bad for profits. So, the company has a choice of two:
1) develop something new
2) make the thing less reliable
So, the company making the thing begins to use poorer quality material to increase the failure rate over time.
What lasted 10 years soon only lasts 5, then 3, then 1 year. And people buy more.
There is no "-1 offended" or "-1 you don't agree with me" mod options for a reason.
must... stay... awake...
you're collecting 100 GIGS per YEAR?
When do you plan on WATCHING this stuff?
Odds are, by the time he's three, you'll be so sick of watching him grow up through a camera viewfinder you'll toss the camera into the back of the closet.
And if you ever have another kid, he'll grow up thinking he's adopted, because he can't find any photographic evidence of his childhood.
I speak from experience :)
It's supposed to be completely automatic, but actually you have to press this button.
Ok, I just went looking for the handbook I got with my photography course, there's a paragraph about optical media: The CD's with a greenish look are guaranteed up to survive for 1-3 years. The siverish CD's last about 10 years. And there are also more expensive CD's with a gold color, and a black protection layer on top, that last +/- 100 years.
It also mentions there is no durability data about DVD's yet. This seems strange to me, and it's maybe outdated.
It might be wise to get some advice at a photography store, I'm sure they get a lot of those questions.
"It's too bad that stupidity isn't painful." - Anton LaVey
I'm amazed that no one mentioned it. Just get 16gb usb flash disks. It has theoretically unlimited life for archiving. The only time it deteriorate is when you continuously write/erase it.
I'm amazed this got modded up. If you look at the data sheets of most flash parts, data integrity is typically rated at 10 years.Is that the serial number for a new western digital/fujitsu hybrid drive???
Grab a LTO drive off of eBay... tapes are not expensive (LTO2 tapes and drives are easy to come by as everyone upgrades to LTO3)... The media has a 30-year shelf life, which, I would imagine, can be extended with temperature/humidity control.
If your files are small enough to fit on DVDs -- I've got individual video files over 50GB.
Sure, you can split them, but then you have to make damn certain you don't lose or otherwise damage a disk, or you have to generate a parity disk or twenty...
Its good for some stuff, not for massive amounts of data.
You're not thinking in a sufficiently long term. Yes, my wife and I only watch videos of our kids when they were babies a couple times a year. Our kids are 4 and 5 now, and they love seeing themselves as babies, that's one of the best parts of the experience.
*But* - the real value of those videos will come much farther on. My wife enjoys seeing what I looked like as a baby and a young kid, and I enjoy seeing what she was like then, too. Our kids' future spouses may enjoy seeing their baby videos, but even that isn't a long enough term
The real value of those videos will come long after the OP is dead and gone. For example, my mom's dad died relatively young (his early 50s), before my parents even knew each other. He was an extremely skilled hunter and fisherman, and I marvel at the stringers of fish I see in pictures of him with his friends, for both the size and the quantity of the fish. He hunted all sorts of birds, raccoons, just about anything but deer. My mom says he wouldn't hunt deer because too many deer hunters would shoot at anything that moved in the bush without even seeing what it was. But he shot enough raccoons that my mom and my grandmother both had raccoon coats (a fashion at the time, but theirs were all made from coons my grandfather shot with his side by side 12-gauge).
I'm the only one in my family who fishes. I taught myself. I would have loved to have learned to hunt, too, and I'm sure I would have learned from my grandfather if he'd lived longer. I love all those old pictures of him. Sadly, my mom sold his old fishing gear, and shotgun, and marbles (what a collection! Like I've never seen before or since) when I was too young to even realize that I could/should object and say "Hey, keep that stuff! I want it!" It all went to an antique dealer. A bamboo baitcasting rod. Original Creek Chub Bait Company lures from the 1920s and 1930s, most still with their original boxes. His Pikie Minnow was in near-new condition, I remember.
So (everyone) by all means, preserve those family videos on a number of media. Hard drive. DVD. Blu-Ray. Whatever comes after Blu-Ray. DLT (been around a long time, and will be for a long time to come), flash drive, etc. If possible, pass down to your children not only the media, but devices capable of reading them.
In the even longer term, like hundreds of years from now, if a lot of video from the present day is preserved, the archeologists of the future will have a much easier time seeing what our times were like than archeologists today have of seeing what times just a few hundred years gone were like.
And yes, that means I think humans will be around for a long time to come. We're the most successful species in the history of the planet and we're not going away. I'll even make a bold prediction: 20 years from now, the air and water will both be cleaner than they are today. If anyone doubts this, let me tell you that I grew up in southern California in the 1970s, and despite the fact that California's pupulation has roughly doubled in that time, the number of cars on the roads has more than doubled, the air is better now than it was then. We've yet only scratched the surface of alternative fuel vehiclesf, and emissions of internal combustion engines can still be improved. 20 years from now, vehicles that use only an internal combustion engine for power will be in the minority. They might even be downright rare. And the air then will be as much better than the air now as the air now is better than the air in the 1970s in LA.
Don't know if I'll be around to see it, since I'm almost 50, but it's the world I want to leave for my kids. Along with their baby videos :)
Lets see the *AA deal with that, imagine the copyright violations!
Though you have to worry about mutations, drift and such. Eventually the picture of your cat will turn into a lolcat, just by pure weight of evolution.
A patriot must always be ready to defend his country against his government. -edward abbey
Okay, so I'm having trouble understanding the solution you propose. Let me see if I get this right...
You should "constantly archive" which means that you regularly make copies of your data for long term storage (preferably off-site). But you shouldn't use any removable media. So you're left with Hard Drives as your only option (remember, USB keys are still removable storage).
Did I get that right? Or did you skip the idea of off-site storage all together?
What "formats you will always be able to use" are there? Hard drives? And what makes hard drives so special? They still require an interface to read the data. Someone posted that IDE has been around over 20 years. That's great, but it's also currently end of life. How long is SATA going to last? Fibre Channel? Hopefully your Linux distro in 2030 will still be able to read NTFS5 partitions. Hard drives are heavy and delicate.
Or are you talking about replicating your entire system to ensure that you can read the data? And then shipping the whole server to Iron Mountain? Heck, while you're at it you might as well plug the thing in, give it an internet connection and set it up for on-line fail-over.
Or maybe you have a different definition of "archive" than the rest of us.
Mediocrity knows nothing higher than itself; but talent instantly recognizes genius. -- Sir Arthur Conan Doyle
Two things to think about:
RAID 5 suffers catastrophic failure when two drives fail within a few hours of each other. The stress of rebuilding the replaced drive from the working ones makes this more likely to happen than some storage vendors want to admit. RAID 6 should do better because three failures in a short time are a lot less likely.
As far as I know, hardware RAID is risky if a RAID controller fails because one brand of RAID controller often can't read an array created on another brand. Software RAID on a Free operating system is more portable.
CDs and DVDs are just not suitable for the task. They're costly, slow, difficult to verify and have a short shelf life.
Really, the way to go is hard disks. Get like 4 of them, set them up to be mirrored into 4 identical drives.
Then separate the two pairs, periodically pull them and verify that all the files still match their original checksum and replace the ones as needed.
You're not likely to find an option which is that inexpensive and reliable. Definitely not with optical media of any sort.
It's not perfect, but something very much along those lines is going to be the best bet for most people.
You can relatively easily switch them to newer disks as time goes on, and whenever a new controller technology comes out, you can easily buy a add in card and transfer them like that.
The most memorable time I saw it explained was when I watched this 20 minute video which will help you think about your effect on the planet:
http://www.storyofstuff.com/
I have found the best way to find disc rot is to use Emsa disk check which is a free and fast little Windows utility. It will find when a disc is going bad LONG before it actually starts spitting errors. Anything important I burn in triplicate(1 for use,two for backups) and store in separate cool dry places. With DVD blanks being so cheap in bulk it costs hardly anything. I wouldn't use cd though simply because of the huge amount of discs you're talking about. With DVD it'll be easy to set aside a "checking day" which with Emsa goes quite quickly. Since it involves a loved one an easy way to remember checking day might be the day before or after their birthday.
Anyway this is what has worked for me and I have CDs going back to I think '97 and the DVDs start with some 1x,so whenever the first burner media came out. And so far knock on wood I haven't lost a single file to bit rot. But that is my 02c,YMMV
ACs don't waste your time replying, your posts are never seen by me.
I don't think you would be interested in videos of my kids.
I'm a paedophile, you insensitive clod!This is a really common argument and it's simply not true. People want to buy cheaper items, and they do it accepting that the failure rate will be higher than it potentially could be. If people really wanted superb reliability then they would get it - the fact is, despite what they say, they don't.
[FUCK BETA]
Well, if you want longevity, I suggest papyrus over CDs. However, finding Egyptian Teletypes that can still read punched papyrus is murder. The Computer History Museum has only one, and it's constantly in use by some odd fellow wrapped all in rags and things. He keeps muttering 'ahnksen ahmen" and burning tanna leaves, too. Queer duck. I think he programs in COBOL.