Slashdot Mirror
A Storage Solution for Lots of Digital Photos?
Duizendstra asks: "I've been asked to explore the digital storage possibilities for a professional photographer. One of the characteristics is the rapid growth of the amount, and size of pictures. At the moment, one photo session produces about 2 GB of raw data. He has an Apple - Power Mac G5, and he currently uses DVD as his storage medium. However, he has lost quite a few photos because of DVDs that can't be read anymore. I would like to know if any Slashdot readers have any experience in creating a solution for such a problem? Any help/idea(s) would be greatly appreciated!"
Would be to buy a few external hard drives. With the storage capacity of hard drives these days you can't go wrong. I bought an enclosure for one of my internal drives and now I don't know how I could live without it. Having a portable drive like this is an amazing convience, especially with a capacity of 160GB.
He should get an Xserve RAID, of course.
It'll just work, it's well integrated with his G5, and it's cost effective.
Try one of these
2GB/session isn't really enough information to design a storage solution but I'll dump out some generic big, reliable and cheep storage suggestions.
S hareShow.asp?ID=1764600
For large scale reliable storage I dislike both optical and tape. They both quickly become more work to manage than it's worth and have serious reliability issues. Hard drive based is the way to go and since hard drives do fail and that is a bad thing, it's best to use RAID. It's especially a good idea since RAID is getting easier, since hard drives are getting cheaper per unit and since SerialATA is making it easy to hook them up right.
Heres a basic design that I'm actually working on for a home server for myself:
http://secure.newegg.com/NewVersion/WishList/Wish
It's a 3U rack mountable 2TB storage server. Put a Linux distro on it with some small RAID1 boot partitions and a software RAID5 storage partition, throw samba and some email-home config to notify of drive failures and you've got a decent place to store up to 1000 of those 2GB sessions. Zip up the old ones if needed for more space. If rack-mounting isn't desirable there are cheaper desktop cases that would probably be appropriate.
If this is overkill a 4 drive RAID5 array or even a 2 drive RAID1 array is much much easier to accomplish. Standard case, motherboard, power supply and drives with a Linux distro and you're done. Hardware RAID is also an option but since software RAID's high CPU usage wouldn't be an issue here I'd go that route.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
WiebeTech makes a desktop RAID enclosure that looks interesting. It's actually two 5-bay RAIDs in a single unit. This is essentially a desktop equivalent to Apple's XServe RAID. WiebeTech has plenty of good high-capacity disk solutions.
Most everyone is recommending hard drives and I'm definitely part of that crowd. Most everyone says "RAID" as well. I'll tell you what I do at work, not for photographs necessarily, but for all our data. I have two servers for user data, one on each side of campus. One is the "active" server with RAID drives, the other is a backup. Each night (I have the luxury of a quiet network at night), I run a network backup to toss stuff over to the second server with RAIDed drives. This prevents the accidental "rm -rf" users that just 1 server with RAID wouldn't prevent against (of course other types of attacks WOULD kill two server solution, which is why important data goes to an external firewire drive from the backup storage server).
Granted, not everyone has the cash to blow on all this, but my stepfather, who is also a professional photographer, has finally taken that all important step toward moving to digital. He's been backing up to CD and he usually gets away with a session on one or two CDs, not counting any editing he does in Photoshop (he still prefers to touch up photos by hand). Anyway, he has been watching his bookshelf fill up with CDs, much like all his file cabinets that store all his old hard copy negatives and select prints. Any long time photographer probably deals with the same stuff, which was a problem before digital ever came around.
What I've been working with him on is what will most likely be a big storage server... even at 2GB a session, you could shoot every day of the year and only use 700GB, which will cost you about $300-400 in a non-RAID solution nowadays (based on me just purchasing 4x 300GB drives at $110 each and my friend buying a 400GB for $200). A small server with a few drives will be all the online backup one should need, plus to be extra safe, either that backup server, or just a few external drives.
If you backup to the external drive once a week or so, this should save anyone from the accidental rm -rf (my stepfather once deleted all the pictures on his laptop by accidentally dragging the wrong folder to the Recycle Bin -- naturally all his photos were too big for the trash and were instantly deleted, luckily he had all his CDs to restore from). Plus, as long as you're backing up regularly, it should be obviously that the hard drive is working or not working. If you start hearing clicking, or feel something funny -- get it replaced.
I guess my summary of all this is to have two backups. If one is your "online," primary storage, it should be obvious if it's failing or not failing, and assuming you're backing up to your second backup regularly, there shouldn't be any danger of you not realizing it's failing, because you are using it all the time. with DVDs and CDs and other media of that type, it's because you set it on a shelf and forget about it for years that is where the danger is caused.
Sorry this was long.
Sig!
Compared with the alternatives, such as IBM's entry-level system http://www-132.ibm.com/webapp/wcs/stores/servlet/C onfiguratorDisplay?storeId=1&catalogId=-840&langId =-1&site_type=public&oiId=null¤cy=USD&base=1 722-6LU&x=11&y=12/, Dell http://www1.us.dell.com/content/products/compare.a spx/das_storage?c=us&cs=04&l=en&s=bsd/ or NetApp http://www.netapp.com/, add the cost of the redundant power and cooling, and ignoring the cost of trying to maintain a Linux system alongside their OS-X, Apple's price is not bad for the storage supplied.
Sometimes, it really is time to call in professionals, and spend the money to do it right. I've built some storage solutions on my old job, to support our compute clusters. They worked from an OS point of view, but even buying what were supposed to be well-regarded components resulted in more downtime than we considered acceptable, mostly from heat. In the end, as soon as we had the money, it was entirely worth it to be able to call IBM, buy their servers for disk storage, and get the better engineering in terms of cooling, drive access, and remote management, as well as the three year warranty with onsite service.
It is highly unlikely that you can screwdriver together a system with as good of airflow as major vendors who employ real engineers. As the poster said, it's a business, therefore when your data matters is not the time to hack something together with parts from NewEgg.
the more accurate the calculations became, the more the concepts tended to vanish into thin air. R. S. Mulliken
Hmm.. $13,000 plus the cost of whatever server you hook it up to. Hardly a good value. Maybe they'll make up for obviously overpriced hardware with important yet intangible benefits. :)
Uhhh...where are you getting this from? You don't need an additional server. An XServe RAID can hook directly to a G5 tower, and with fibre channel you can locate it far enough away (such as in a closet) that noise isn't a problem. Check your facts, dude.
--Paul
I don't ever see any tape systems touted for long term storage, but I'd love a recommendation for a system that I could trust for more than 10 years.
SuperDLT. It's what we use for SARBOX data retention compliance.
Unfortunately, that's "enterprise" tech, which means Big Bucks.
"I don't know, therefore Aliens" Wafflebox1
It works. Reliably. In my previous job, we pretty much depended on it. A single faulty tape could cost us from $50k and up. And we didn't do backups of data on it... The tape drives were used continuosly 24/7.
If you can afford it, is an entirely different question. I think it's about $30k...
Film. No, seriously, there are labs that will project your digital files onto slide film. Usually you can find them if you look for presentation preperation companies. They usually advertise this service for converting computerized (read: PowerPoint) presentations to slides for showing in venues that do not have video or (SX-/X-/S-)VGA projectors. Good film has proven that it lasts for a long time. Go check your 30 and 40 year old slides.
Video Production Support
Well, (and this is me just talking out of my ass here), you could maybe invest in a film recorder (we called it the slide-shooter). Think of it as a digital projector that projects onto film for later development. We used to use one in my lab to tranfer presentations from PowerPoint to slides (for scientific conference presentations), but I imagine that, as a last resort backup solution, it might work well for photos. The slides are definitely off-line, and I presume could be used to recover the photos in the event that all else fails.
The backup scenario I imagine would be:
Now I imagine that this is not an ideal solution, but it does provide you with the 30-plus year proven backup (we've all seen 30-year-old slide shows). The downsides are that shooting the slides is a time-intensive process, and it adds in the overhead of purchasing and developing the film. I also imagine that you lose some quality going from digital to slide and back, but, you do have the photos in the event of hard-disk failure, and slides are fairly condition-tolerant and physicially small as a backup. Just make sure you don't drop the box.