Slashdot Mirror
A Storage Solution for Lots of Digital Photos?
Duizendstra asks: "I've been asked to explore the digital storage possibilities for a professional photographer. One of the characteristics is the rapid growth of the amount, and size of pictures. At the moment, one photo session produces about 2 GB of raw data. He has an Apple - Power Mac G5, and he currently uses DVD as his storage medium. However, he has lost quite a few photos because of DVDs that can't be read anymore. I would like to know if any Slashdot readers have any experience in creating a solution for such a problem? Any help/idea(s) would be greatly appreciated!"
One approach is to burn three copies and then you can recover the data by averaging the signal between them. This requires multiple drives but it is better than having to give up on archived data. Manufacturers suggest: "Store your recordable DVDs vertically, protected from sunlight, in a room that avoids wide variability in temperature and humidity."
http://tinyurl.com/4ny52
This is a system for a professional photographer, storing the digital photos that are the lifeblood of the business. (Note: most professional photographers charge a nominal fee for a session, but then make the real money by selling prints. No negatives or no files = no $.) This is most emphatically NOT the place to try to do things on the cheap. It's an absolutely necessary and tax deductible business expense.
I work for Apple, and while I'd prefer that this place purchases an Apple-based solution, I am not wedded to a particular OS or brand of hardware. However, you get what you pay for -- either through hiring a skilled professional building an open-source based storage system or by paying for a commercial solution (such as Apple's XServe RAID unit). Be sure to include the necessary system maintenance in the budget for such a complex setup, including off-site backups, on-call support, and making sure that it stays up and running during successive system updates and upgrades.
Given that the photographer is already using an Apple G5, I suspect an XServe RAID solution will suit the situation quite well. One unit can provide 7TB of storage, which at 2GB/session works out to about 3500 sessions at current resolutions (also allowing plenty of headroom for growth as resolutions increase). Apple offers professional services, on-call support, and training for server administrators. In addition, if you're looking for an Apple consultant with the necessary skills in your area, check the Apple Consultants Network.
--Paul
Your friend could probably get about 6:1 compression I'd guess
I *highly* doubt that. It's unlikely he'd get any significant compression, and very possible that compressing the files would actually increase their size.
We're talking about photos here, which are already compressed. Even RAW photos are compressed heavily (though losslessly). For example, a Canon EOS-1Ds Mk II takes RAW photos at a resolution of 4992 x 3328 with 36 bit per pixel. An uncompressed image would be 4992 x 3328 x 36 / 8 bytes, which is about 71MiB. The image files produced by the camera, however, are 14.6MiB, a compression ratio of nearly 5:1. The file system compression isn't going to get much more. On RAW files from my camera, bzip2 -9 only averages about 0.1% reduction in file size, and bzip2 -9 is very good -- and very slow -- compression.
Disk drives are the best way to safely store large volumes of data, especially when you add some redundancy, but don't expect to get any help from compression of already-compressed data.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
One last time.
Say it with me.
"RAID is not a backup solution."
Again,
"RAID is not a backup solution."
That being said, RAID helps to overcome failure of the drive. Do yearly or bi-yearly DVD backups, or back up to another offline harddrive... etc..
Karnal
No! These LaCie drives are a bad idea. They put multiple disks in one enclosure, but they're not RAIDed. That means if one disk fails, your entire life's work is gone!
I would store these photos on a dedicated server that has good RAID. I don't think full tape backups are an option, but remember that RAID will handle a single drive failure -- not you accidentally typing rm -rf * . Maybe there's some service that will mirror your important data off site somewhere. That's probably expensive, but if this data is what puts food on your table, it may be worth it.
My other car is first.
There should be two levels of file storage, a live copy, and a long term copy. EACH of these needs a backup. Once you need more room on the "live" server, you move it to long term storage. you want to keep as much "live" as possible to go back to.
Live Copy - I would get a RAID system with as much storage as you can afford or need. If price is no option, then you get a few XServes since they are on OSX. This is where you keep your files from today on backwards till the RAID is filled. You keep a backup of this RAID by tape, HD, optical whatever.
Long Term Copy - There are problems with EVERY type of long term storage. The most reliable would probably be hard drives. You use two(or more) NEW and different brand drives, copy the data to be archived to each, verify each. Store each drive in a static bag and some sort of case. Put one at your house, one in the bank or somewhere else. A bank's safe deposit box might be expensive, but it is climate controlled and "safe". Using tapes and optical disc are problematic because the mediums break down with age. The hard drives will last much, much longer. Yah, hard drives fail. But if you use different brands, you increase the chance of a mfg being better than another.
I have HD's that are 15 years old that still work fine after thousands of hours of use. These archive drive will only be used long enough to format, test to make sure they aren't DOA, and to write data to. They should outlast the drive interface technology that they use.
As time goes on, drive get bigger so more data can be live, and archiving becomes easier. You can always go back and re-archive the data to a "better" medium that holds more.
Back in the day, people would back up their 20MB HD's to floppy, then they bought a 100MB drive, used tape to back it up, then CD, now DVD. Times change, so your backup strategy must change too. What you do TODAY, will be easier in a few years.
Fear Is the Only God
Two phrases I don't like seeing anywhere near each other.
Aim more for "redundant" and "widely tested" for starters.
I'm Rick James with mod points biatch!
You are talking out of your ass. I'm seeing your butt cheeks move in sync with the words.
Big ass SATA RAID
SATA? Do you also act like you're on the same level as F1 race mechanics, because you checked the oil in your Ford Pinto?
using a mix of software and hardware RAIDs
Yes, that's a strategy! In something very nuanced, complicated, and with potentially disasterous consequences, let's mix the two together for even more complexity. Maybe this is worth considering in some cases, but without an expert there to come to that conclusion, and certainly without any stated reasons for this, THIS IS A BAD IDEA. If this is what you eventually decide to use, do not hire parent poster to do the job. Oh, and since you'll want someone who knows what they're talking about to do it, it's going to cost more than his estimate.
A cheap, sane alternative would be to compress your photos. JPEG really is good enough
Just when I thought you couldn't be any dumber. "Yes, for long term storage of your incredibly hi-res pics meant for professional photography and graphics, where every single lost bit seems to count, why not print them out on acid-saturated paper with my crusty inkjet printer that's out of yellow?" I mean, my god. It's an ask slashdot, people are supposed to be stupid. It can't be helped, but damn. There are sea urchins with more advanced cognition.