Slashdot Mirror

← Back to Stories (view on slashdot.org)

Long-Term Storage of Moderately Large Datasets?

Posted by timothy on from the rodents-of-medium-size dept.

hawkeyeMI writes "I have a small scientific services company, and we end up generating fairly large datasets (2-3 TB) for each customer. We don't have to ship all of that, but we do need to keep some compressed archives. The best I can come up with right now is to buy some large hard drives, use software RAID in linux to make a RAID5 set out of them, and store them in a safe deposit box. I feel like there must be a better way for a small business, but despite some research into Blu-ray, I've not been able to find a good, cost-effective alternative. A tape library would be impractical at the present time. What do you recommend?"

4 of 411 comments (clear)

  1. Amazon AWS? by TSHTF · · Score: 4, Interesting

    It might not be the cheapest option, but with Amazon's AWS, you can snail mail them a copy of the drive with the data and they're store it in S3 storage buckets.

  2. Re:Exactly what you're doing by forgottenusername · · Score: 5, Interesting

    I don't think it's a great solution. You're storing relatively fragile hard drives in a raid5 configuration in a lock box? It's not like you can tell if one of the drives goes bad and needs to be replaced when it's sitting in a box. You'd have to regularly pull the data sets out, fire them up and make sure everything is still functional.

    I'd at least want to do 2 complete sets of mirrored drives.

    Tape storage does store better.

    Depending on how important the data is, I might do something like a local mirrored drive set in storage and an online copy at something like rsync.net - stay away from s3, it's not designed to protect data, despite what AWS fans may say.

  3. Never us DVDs as long term storage. by strangeattraction · · Score: 4, Interesting

    Repeat never use DVDs as long term storage. I have seen them go unreadable anywhere from 2-5 years. I have fired up disk drives 10 years later with no problems. They are cheap reliable and fast. Don't try and get fancy just compress and store data sets over multiple volumes. Don't use RAID.

  4. Re:GMail Drive by 0100010001010011 · · Score: 4, Interesting

    That's what ZFS is for.

    mount -t gmailfs /disk1 -o username=gmailuser,password=gmailpass
    mount -t gmailfs /disk2 -o username=gmailuser,password=gmailpass
    mount -t gmailfs /disk3 -o username=gmailuser,password=gmailpass
    mount -t gmailfs /disk4 -o username=gmailuser,password=gmailpass
    mount -t gmailfs /disk5 -o username=gmailuser,password=gmailpass

    zpool create gzfs raidz1 disk1 disk2 disk3 disk4 disk5

    Actually.... I think I just found my project for the evening. I mean it's already been done with 12 USB drives