Slashdot Mirror
A Storage Solution for Lots of Digital Photos?
Duizendstra asks: "I've been asked to explore the digital storage possibilities for a professional photographer. One of the characteristics is the rapid growth of the amount, and size of pictures. At the moment, one photo session produces about 2 GB of raw data. He has an Apple - Power Mac G5, and he currently uses DVD as his storage medium. However, he has lost quite a few photos because of DVDs that can't be read anymore. I would like to know if any Slashdot readers have any experience in creating a solution for such a problem? Any help/idea(s) would be greatly appreciated!"
This is definitely a good recommendation. ATA over Ethernet has comparable performance to a similar SCSI configuration. Additionally the price tag is not quite as hefty. Coraid's prices range from $2,000 for 2 Terabytes - $4,000 for 7.5 Terabytes. That does not include the price of a NAS server. Like the parent said you should be able to homebrew this. You could possibly set it up on the Mac G5 you currently use. Making the assumption that you can easily set up an NFS server in Mac OSX.
If storing all that data was important to my livelihood, it would be hard for me to not justify this investment.
Other options, are optical storage, which is not working out for you currently. Of course, you could drop a couple thousand on a good tape-drive, and spend several hours backing up, and restoring. Then you have to deal with worn tapes, and eventually a worn out tape drive. Of course, you may outgrow your tape solution and have to invest a few more thousand in a larger capacity tape-drive.
On the other hand, the biggest problem in a fault-tolerant RAID is taking a few seconds to unplug a dead drive, and replace it.
/^([Ss]ame [Bb]at (time, |channel.)){2}$/
My problem is that I'm finally starting to see decaying CD's. I've got some backup CD's from 1998 or 1999. (I can't access the media anymore to check dates, but somewhere in there.) I put them in the other day to look for some old data and they wouldn't read. When I pulled them out of the drive, the silver media was peeling away from the disk. I've run across about 6 of my backup cd's so far this year that are doing the same thing.
No, I think a good harddrive array is going to be your best bet. Get several harddrives and mirror the data. The cost of gigabytes is dropping on a daily basis. You should find that when you need more room, it will be easly upgradable and cheaper as the years go on.
I'm a photographer and I'm looking at over a terabyte of external firewire drives piled up on my desk and spindles of DVDs backing them up. Right now the options for reliable long term reasonably prices storage pretty much suck if you generate around six gig of files a day. Lots of hard drives is fine as far as price goes, but they aren't an answer for long term storage. If anyone has an idea for storage in the 30 year range, I'd like to hear about it. My experience with tape back in the DC250 days was pretty dismal, and I don't ever see any tape systems touted for long term storage, but I'd love a recommendation for a system that I could trust for more than 10 years. I assume that a product to address this market will show eventually but I'd like to have something now
S ection=pressreleases&department=maxellusa_pr&Line= datapr&Open=datapr410 608/105586/?ST=english
For what it's worth I'm looking forward to these when they come out.
http://www.maxell-usa.com/Content/Pages/Page.asp?
or these
http://techon.nikkeibp.co.jp/english/NEWS_EN/2005
buy an external 300 GB firewire drive every couple months, label the drive by the time period. If you are really worried, have 2 every couple of months, they are pretty cheap. Drives don't fail sitting on a shelf, at least not like dvds do. The last thing you wan't is a raid system thats active every day, it only increases the likelyhood of failure. raid is great for data you must access now, but a waste if you are just using it to back up data you only need once every so often.
If it is really important, use tape backup, make redundant copies, and send one off to a data storage place. As others noted, a backup solution should be part of the cost of the job, and is not really that expensive when divided over the different projects.
-I just work here... how am I supposed to know?
Even verifying CDs after burning isn't enough. I've had one batch of CDs that I burned stuff onto and verified that the sha1sums of the files I had burned on them matched. ONE week later the CDs became unreadable (I got a whole lot of I/O errors trying to even read the CDs). These CDs weren't even in the sun or hot car or anything. And these CDs weren't scratched and didn't have a spec of dust on them that I could see. They were in a cool desk drawer and untouched for that week and lost their data in a single week.
"RAID is not a backup solution."
No.. but it is many many times less likely to fail in comparison to storing a single copy on DVDs which is what it is being compared to here.
In general though, the need to do backups is greatly reduced with RAID and in some cases, a single copy on a RAID array is "good enough" which is to say it eliminates the need to create a separate backup copy. To claim otherwise is to not fully understand data backup.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
If the imperative is protecting his data, then he should do what professionals do with any other kind of critical data: put it on tape. Sure, you probably could put together an optical solution, but the tape technology is compact, proven, convenient, stable and scalable. I'd avoid anything proprietary aimed at the consumer level (if anybody still IS aiming tape technology at consumers), and look at technologies such as DLT which are popular for critical applications among professionals. Support for technologies adopted this way is measured in decades.
Given that this is his life work, he really should invest a few thousand dollars and put together a strategy that will protect him from media and system failure, localized disasters such as fires, and possibly even regional disasters. With a little thought, while it is not going to be cheap, it will be a bargain.
Supposing he's willing to put four or five thousand dollars into this. He can get a SDLT tape drive with a 160GB native capacity (don't count on compression for photos), and 16Mb/s native transfer rate. That day's photo session takes two minutes to back up. am deacj tape stores possibly up to half a year of work. He'll have enough money to buy a good number of tapes, so with a a little thought he'll have a good system for archiving his old stuff, one that is not vulnerable to single tape failures and has an offsite (important!!!) component too. And he may have enough money left over to buy a fire resistant media safe that could buy his data at least a couple of hours of time. Depending on the economic value of his work, he could also send backups to an offiste media storage facility that provides a very high degree of security against regional disasters as well.
I'll tell you a story I tell all my clients when the cost and inconvenience of a well designed backup program comes up.
Years ago I had a client who drove up with what looked like a huge piece of burnt toast in the back of his nice Mercedes sedan. He was was a CPA, and this was three weeks before tax day; the burnt toast was a minicomputer that had all his client's tax work on it. He'd been doing backups daily to tape, but contrary to our advice he had stopped bothering to take them off site. Under the circumstances, if he'd had an offsite backup, we'd have lent him everything he needed, even the office space if necessary. He'd have been back on track with maybe two days down time on the outside. When tax season was over he could have moved to a new office, bought new equipment from the insurance settlement, and his biggest worry would be decorating. But all this depended on the offsite backup he didn't have.
There's a small chance that some of his data mightbe retrieved nowadays, by firms specializing in this sort of thing. But they didn't exist in the early 80s, an in any case I wouldn't want to bet on it. The computer had obviously taken major heat; the interior wiring and connectors weren't just smoke damaged, they were brittle from cooking. We did the best we could, removing the drives, stripping and swapping the electronics on them, cleaning all the connectors on the drive with tetracholoride and so forth. After a few hours of work it was clearly futile, but we spent another day on it trying pointless and hopeless things, just to make him feel like we'd done everything possible. None of this would have been necessary, but for want of a simple step he was fully equipped to take, but seemed like a bit too much bother at the time.
The lesson is that while people comprehend small disasters like misplacing a file, large disasters are sometimes so horrible to contemplate that they discount them altogether. If your client is lucky, he'll be irritated with being saddled with having to swap tapes every morning and perhaps rotate them offsite every few days. Maybe labelling the tapes will be a chore. If he's unlucky, you'll be a hero.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
No one does, because, frankly, digital photography hasn't been around that long--so there are no solutions that were around in the 70s, that are now still working, to demonstrate that they are reliable over that kind of time period.
I'm unsure if you think this is a digital photography problem, or if you just believe digital photography is the only possible reason someone would need massive, long-term, reliable storage.
Either way, it's an "ask slashdot" at least twice a year, for all sorts of reasons. It's a general computing problem.
The only possible solution as I see it, is to quit sissying around with firewire drives... they're nice when you need a little extra storage, and don't want to dick around with opening the case. Get a real fibre channel card for $50. Get a fibre channel enclosure for $500. And another $5000 or so will get you decent, lowspeed FC drives, with a few spares. Over the course of 20 to 30 years, you'd have to constantly rebuild it... a 140gig drive from today won't be replaceable in 16 years unless you have a spare (even if you did, would it be reliable itself?). It could easily cost alot, but then, maybe you're already spending quite a bit?
Also, if I hear SATA one more time, I think I'll puke. This guy is asking about 50-ton dump trucks, and people are talking about riced out Honda Civics.