Slashdot Mirror
Best Way to Back Up Photos and Video?
jsalbre writes "I do a lot of digital video work, and my wife is a professional photographer. With raw DV from the video camera using up 11GB/hr, and raw images from the digital SLR using 7MB I'm quickly using up a lot of space. I currently back up all my important files each night from one harddrive to another, but I now have over 200GB of irreplaceable data (more than just DV and photos, but those make up the largest chunk) and I'm having to exclude the "less important" irreplaceable files as my backups have started failing. Several people have suggested backing up vital unchanging files to DVD (video, images,) and continue backing up frequently accessed files to harddrive, but with recent studies showing that optical media doesn't last very long I don't want to come back in a few years and find that all my backups are useless. Not to mention that some of my DV files are larger than even a dual-layer DVD, and it would be near impossible to automate backup to DVD. How do other Slashdotters back up their important data? I'd appreciate distinction between methods for frequently accessed files and for infrequently accessed files. Any suggestions will be highly appreciated!"
(i have no experience in the matter) you should place all the un-changing files on a hard-drive which will then sit in a draw and is only plugged in when required. I have been lead to believe that this will reduce the likelyhood of harddrive failure to close to 0. Then you can setup a RAID type setup for you changing files.
Not from TFA, from TF synopsis:
"Several people have suggested backing up vital unchanging files to DVD (video, images,) and continue backing up frequently accessed files to harddrive"
They've already considered hard drives. Since he's dismissed hard drives and seemingly all forms of optical media, the only thing that I can think of for this article getting posted is that the submitter *really* wants Slashdot to tell him that "Yes, it's ok to mortgage the house to buy that new Network Appliance SAN you've been drooling over."
Agreed. I realise tapes aren't the new hotness, but they're the most reliable, and they have good storage capacity. In addition, I'd consider a larger capacity storage server. Together this stuff may not be as cheap as tossing everything on DVDs, but apparently this is for people who work in digital media for a living. From that perspective, its worth investing in your profession.
Perhaps better than slashdot, they're bound to have a huge network of friends in the profession who have already crossed this bridge. It couldn't hurt to ask how people specificaly in these professions manage their media storage.
This is the pot calling the kettle black, though. Is there a support group out there?
I keep reading comments about how CDs/DVDs are unreliable. Here's a great trick i use to make sure my data is safe: i always include 50-100mb of parity files on each DVD. The disc would need to be REALLY messed up to be unrecoverable.
First that comes to mind is Tape backup.
So what you'd suggest is that he downloads the video from the MiniDV tape to the computer, then archives it onto backup tapes. Why not just keep the original MiniDV?
"I've got more toys than Teruhisa Kitahara."
Could be a number of reasons to do the seemingly illogical double-hop method.
First, the new data may have been processed (edits, color correction, etc).
Second, the backup media may be better rated for long term storage. I'm not familiar with MiniDV, the stuff I work with is all DLT and HCART2 under Veritas Netbackup, at 200GB raw/400GB compressed per tape.
Third, it may be helpful to have the indexing done for him by a good backup program.
However, as I say, I work with Netbackup. To say it's pricey is an understatement... but it's changed my views on what's a "workable" backup system to only liking enterprise grade stuff.
This might sound obvious.. I'm a photographer.. I have a 2 bay firewire drive set I use for "HD" backup of my photos and video. Its 700 gig. I also burn DVDs.
When I backup my stills onto dvd I use jpeg 2000, its lossy but really not that bad once the image is in a good state.. I did some tests in college on jpeg/jpeg2000 vs tiff (uncompressed) of the smae image to see how much is lost. Not a lot it turns out. I love uncompressed images, but the loss when storing as jpeg isn't so great to matter unless you do a lot more manipulation. I'm also still shooting film which can always be rescanned at a later date.
However, you shouldn't backup all the DV (raw video) you dump on the computer. The original tap e can act as the backup. its still on the tape even after you dump it into the computer. Label it and set the right protect notch. Voili, instant backup footage.
I'm assuming you edit this down and give the client a dvd/video. Just keep a DVD copy for yourself. Thats all they can really ask you for. If they come back at a later date, because the dvd is bad try yours. If that doesn't work you have to go back to the tape and redit and recharge.
I have been in the same position the Author discussed, and I have come to ONLY negative conclusions. In a few words, and I hate to say this, but buddy:
WE'RE FUCKED.
Digital is a loser's proposition. backing up to analogue or even digital data on analogic substrates (such as DV tape) fail. Simply nad purely.
The *only* thing that comes close is some kind of RAID, and those, even with the plummeting price of storage, are still too expensive given the needs.
Also, a RAID assumes a continuity of several things that are not likely to be continuous:
With Video:
Framerate, number of lines, colour depth, aspect ratio, file format, compression format, Operating system compatibility, etc etc etc. All of these things are variables.
With Audio:
sample rate, compression format, bit depth, file format, etc.
Basically all of it points to very bad places.
I am fairly well convinced that our age will simply disappear. They will find our garbage, the few books not pressed on acidic paper, our paintings (fat lot of good the abstract stuff will mean to them) and drawings, that's about it. the rest will just be shiny little bits of crap in the landfill.
Since we will have used up all the dense energy forms, they will be appalled at the energy requirements just to get the few remaining museum piece devices to work. Archiving the 21st century will be impossible. To the 25th century, the 21st century will be seen as a dark age - not only for the holocaust of the die caused by the failure of the petroleum based economy, but from the simple fact that very little of the information formats we are totally geared into will survive, including this note on /.
His problem of saving personal video is just the tip ofthe iceberg. His problem is the problem of our very civilisation, writ small.
That's why I am abandoning video, and going back to painting. In 500 years, my painting CAN survive. the video simply won't.
RS
Shoes for Industry. Shoes for the Dead.
These are the only ones I can trust to be around in 100 years or more.
*All* digital images get written to CD-Rs are are stored in a commercial document-control facility. But the ones I really want to keep get written to film.
Best Buy can have you arrested
You people don't get it do you. RAID is not for archival.
It is to survive a hardware failure &/or increase speed. It
is not meant as a backup device to archive data. Accidentally
delete a file and its gone from your RAID. Accidentally
overwrite a file with same name, the original is toast. Lose
two drives in your RAID array, good chance your data is gone.
As others have said, optical may not be as reliable as once
thought, and is not practical for large files. While tape
suffers from drive obsolescence, the media aging rate is
fairly well known and less random than optical and can be
planned for.
Someone needs to mod this one up to 5 just from the video angle alone. I know a ton about video production, and I'm constinently amazed about how many people don't know they can back up their full res video files back to MiniDV tape via something typically referred to as "Print to Tape." You can print your edited footage back to MiniDV via Firewire just like you downloaded them off of the camera to begin with. This saves the FULL RES video files, not some crappy MPEG2 DVD dumbed down low bitrate crap -- the actual NTSC video. Not a complete backup solution, but when it comes to video, nothing beats this because you are getting 30-40 gigs worth of video backup for the price of a single MiniDV tape.
Infomation now is much more perminant than it was in the past, and digital has improved this a great deal. The amount of information we generate these days is enormous, far more than ever before the digital age. Thus it's not supprising much of it gets destroyed. For that matter, most of it isn't worth saving anyhow.
Books are not such a perminant media as you might think. They wear out, and can be destoryed. A good example is the Mayan Codices. Records seem to indicate there were thousands, however Spanish priests burned them as "works of the devil" during the European conquest of the Americas. Today only 4 remain.
Digital data can be so perminant because it is so easily copied. Perminance of data does not come form trying to make a single, eternal copy, but from having many copies all over the world. Digital data can be copied for essentially zero cost very easily. Thus it's easy to give it a great deal of robustness. Also, as new formats come out, you simply copy and convert the data. I have data on my harddrive today that orignally existed on 5.25" floppy for the Apple II. It has simply been copied and converted a number of times.
Finally, it's not like book are going away. On the contrary we publish millions of works a year amounting to billions of books.
You seem to have a false sense of perminance, as though in the past things were archived forever. That's not the case, actually, most data was lost, that's one of teh reasons we have such an incomplete picutre of history. You don't even know all that was lost, because the record of it even existing, if there was one, is also lost. What has survived is by chance, or by effort, not because we had some wonderful archival system.
You don't have to have something on an immutable, indestructable medium for it to survive. The Nordic Legends weren't written down for centuries, yet today we still have them. They were passed down, as an oral traditon for generations. There was no perminance to them other than stories in people's minds, yet they've durvived thousands of years.
The video files that your NLE uses are exact duplicates of the data your camera writes to DV tape. Take a hint from that and just save your DV tapes. All modern NLEs work with EDLs (edit decision lists), so save your session files, overlays, transition parameters, etc to a CDR and push the lock-tab on your master tapes. Keep your tapes labelled and organized so you wont have a problem finding them again. It's trivial to recreate your project at that point, and it thankfully isn't MPEG-compressed on a video DVD.
Alternately, all modern NLEs have 'export to tape' functions. Just record your final product back out to your DV deck or DV camera and make a master archive on tape.
No.
They are a minimally acceptable backup media for short-term storage.
Consider the fact that with tapes, you really just have to worry about tape errors. If the tape drive fails, you can use another.
With hard drives, you have to worry not only about errors on the drive, but about hardware failures in the electronics as well.
In 10 years, that hard drive will probably be dead no matter what you do. But a properly stored tape backup would still be around.