Ask Slashdot: DIY NAS For a Variety of Legacy Drives?

An anonymous reader writes "I have at least 10 assorted hard drives ranging from 100 GB to 3 TB, including external drives, IDE desktop drives, laptop drives, etc. What's the best way to setup a home NAS to utilize all this 'excess' space? And could it be set up with redundancy built-in so a single drive failure would cause no data loss? I don't need anything fancy. Visibility to networked Windows PCs is great; ability to streak to Roku / iPad / Toshiba etc would be great but not necessary. What's the best way to accomplish this goal?"

Comments

Not worth it.

[Anonymous Coward]

Those older drives are probably failures just waiting to happen. With the cost of the hard drive space continually dropping, just use new drives. It's not worth screwing around with old ones for anything other than salvaging old data off them, even though the urge to do so is strong in the more frugal among us.

Re:Not worth it.

[ZeroSumHappiness]

I agree to an extent. Take anything SAS or SATA that's 1TB or greater and re-think the project with just those. Sell or recycle the rest of the drives. Depending on your needs the remainder should be RAID-1, 5 or 6'd (using software RAID if speed isn't an issue) and then put on an OpenFiler or FreeNAS box. Anything non-replaceable should then be backed up to a respectable backup provider in addition to your home-grown solution.

We need more information though -- what are your actual drive sizes and what do you want to put on this NAS?

Re:Not worth it.

[King_TJ]

Yep.. I agree. "Not worth it." sums it up nicely.

Seriously, I completely understand the desire to re-use unused equipment you've got lying around. Seems like the smart thing to do, reclaiming as much of that unused storage space as possible and pooling it together so even the smaller drives add up to something worthwhile. But as a FreeNAS user myself, trust me on this one. It's NOT really a good idea.

As other already pointed out, most RAID configurations are limited by the size of the smallest drive in the array, so that would create major problems for you right there. But even assuming you skip RAID (or set up multiple RAID pools, each consisting only of very similar sized drives -- and then join all of them into a virtual master storage "device"), you're still in a situation where the lower capacity drives probably have slower data xfer rates than the newer, larger ones. That will drag the overall performance of the server down, whenever something gets loaded or saved to the slower/older disks.

Even if all of THAT doesn't discourage you? I have to ask what your time is worth, and to a lesser extent, what your data itself is worth? Old drives as small as 100GB capacity have got to be at least 4 -6 years old by now. Unless you bought them new and just stored them in a box this whole time, chances are, they've seen a lot of hours of operation already. They don't have a resale value more than $20 or so these days, so you're simply not out much money to throw them away or give them to a recycler. Meanwhile, you'll probably get into a much more complex and time consuming NAS configuration, trying to best utilize them in your drive pool. Even if you only make $10/hr. at your job, that means 2 hours of time spent messing around with this is worth the entire value of one of those old drives!

I'm kind of a pack-rat for computer hardware (since I have an on-site repair business besides a day job in I.T. and computers as a spare time interest too). But even I started throwing away IDE or SATA drives under 250GB a while ago. I keep a *couple* small ones around, but only for odd situations (like someone who wants to revive a really OLD PC with a BIOS that can't recognize larger drives properly). Otherwise, everyone who wants to go to the trouble of swapping an old/dead drive out for a replacement may as well spend the relatively small extra amount of money for a current model of much larger capacity, AND a full warranty still on it. Your data is usually worth it!

Re:Not worth it.

[19thNervousBreakdown]

It depends--is there a total of 6 TB of drives that doesn't include the 3 TB drive?

Take each disk, make an LVM physical volume from it. From those physical volumes, logical volumes. You don't have to make all of them the size of your smallest drive, you just have to be careful. Say you have the following:

1: 3 TB
2: 2 TB
3: 1 TB
4: 1 TB
5: 750 GB
6: 750 GB
7: 150 GB
8: 150 GB
9: 100 GB
10: 100 GB

On your 2 TB drive, make partitions matching the drives under 1 TB.

On your 3 TB drive, make the following partitions:

1 TB: RAID-5 with #3 and #4
750 GB: RAID-5 with #2 and #5
750 GB: RAID-5 with #2 and #6
150 GB: RAID-5 with #2 and #7
150 GB: RAID-5 with #2 and #8
100 GB: RAID-5 with #2 and #9
100 GB: RAID-5 with #2 and #10

You'll end up with the following volumes:

1: 2 TB
2: 1.5 TB
3: 1.5 TB
4: 300 GB
5: 300 GB
6: 200 GB
7: 200 GB

Then take those, LVM the RAIDed LVM volumes (fairly certain you can stack [traditional meaning of "stack"] as a contiguous disk, just use an easy FS like ext3, I've run into problems with stack size [programming meaning of "stack"] using XFS on LVM). You end up with 6 TB total space, and, just like normal RAID-5, you don't lose anything unless two disks from one of those groups die. That is, if a disk in 200 GB #6 dies, and a disk in the 1.5 TB #3 dies, you still haven't lost anything. Even if your 3 TB drive dies, which is clearly the worst case since it has data for every array, or the 2 TB which is nearly as bad, you'd still need to lose a second disk to lose any data, so for failure rates it should be the same as a 10-drive RAID-5 array, which isn't quite advisable although it's not murderously bad, but this isn't work and the primary motivation is probably maximizing space with a decent reliability increase, not making next to certain it never goes down. I'm sure it feels really weird, but I don't think you're actually increasing your odds of failure at all over the 10-disk-all-same-size RAID we're used to, other than not trusting older drives--and I'm not so sure those are much more likely to fail than new ones. After all, they've lasted this long, and I've had brand new drives die within weeks. But in point of fact, there's some 2-drive failures that don't take anything down, so I think overall you're doing slightly better than the 10-same-size disk case.

Now, your disks probably won't divide up as nicely, and you might end up having to either leave some space on the floor or subdivide in weirder ways or both, but with very careful partitioning (never put two stripes of the same array on the same disk), you can do this. Set all the arrays to verify weekly (mdadm can do this) and e-mail you on a failure. Don't set up an audible alarm, you're not going to lose a second disk at 3 AM (but you will wake up to fix it, and be worthless at work the next day for probably nothing) and even if you did lose another disk, you're not using RAID as a replacement for backups, right? Right?

ZFS would be really nice if it did all this complex stuff for you, but do you have enough control/is it smart enough to allow you to ensure that you get as good or better reliability? It'd be ridiculously easy to make a bad mistake in layout with the above scenario. Because overall, I agree with the title: It's just not worth all this effort so you can use that crappy 100 GB disk. Once it goes down, now you have to replace it.

the 2 main choices:

[gbjbaanb]

FreeNAS or OpenFiler.

I think FreeNAS (the BSD based one) is lighter and easier, as OpenFiler seems to be going in a more "fully featured" direction with less support for older hardware, but they're both good.

Re:the 2 main choices:

[McKing]

ZFS does this much more simply with no ugly hacks. You can have mismatched drives when you build a mirror (the mirror is the size of the smallest drive in the mirror set), and then you stripe across the mirrors. As the older, smaller drives fail, replace them with newer, bigger drives and the pool magically gets bigger. 100GB + 500GB mirrored (100GB usable). 100GB dies, swap in a 750GB drive and now this pool is automatically resized to 500GB. Get 2 more drives? Mirror them and add them to the pool and your pool expands with no one the wiser.

Seriously, if you haven't played with ZFS before, download FreeNAS and give it a whirl. When I was a Solaris admin, ZFS was the most fun thing to work with by far.

The mega surplus continues!

[digitalsushi]

Ah ha! Who else amongst you has a huge surplus of huge hard drives going unused, now that netflix streaming has displaced 60% of all the crud you had spinning idle in a closet the 3 years before you signed up?

My storage requirements went from about 3 terabytes to about 30 gigabytes over the past 2 years. I believe I am the archetype and that I am doing the same thing as the average geek. I suspect there are piles of huge disks sitting offline because of this streaming displacement.

It cost me about 18 dollars a month to leave my x86 file server online, idle (killawatt meter, nh rates); netflix is cheaper than that.

Come on, who else has a comment related to this.

Greyhole!

[gregthebunny]

Why am I the only one saying this? Setup Greyhole, throw a bunch of disks at it, and enjoy! And to all those saying "those drives are going to die soon", you can actually tell Greyhole that you consider a drive "broken" and it will still use most of its storage (albeit redundantly) until it does die and have to be removed.

StableBit DrivePool + WHS 2011

[alexpi]

Full disclosure: I am the developer

Check out: http://stablebit.com/DrivePool

It's a software disk pooling solution that combines any number of disks of any size into one big virtual pool. You can designate certain folders to be duplicated on the pool. Any files placed in duplicated folders will be stored on 2 disks at the same time.

The implementation is a hard core NT kernel driver with a virtual disk. There is a full NT kernel storage stack, no user mode hacks here.

Unlike RAID and similar solutions, all your pooled files are stored as standard NTFS files on each individual disk in the pool. This means that you can simply plug in any pooled disk to any system that can read NTFS to get at your files in case disaster strikes.

It's commercial software, $20 USD per server.

take look at amahi.org

[JonySuede]

look at amahi.org, it is a turn-key Home Server based on fedora and greyhole as it's replication engine.
Dump anything less than a TB except one drive and you are set.
You set the replication level by share and it keeps a full copy on each drive until the replication count is reached for that file on that share.

Example:
you have 4 1TB drives and 1 500Gb drive.
You have the share photo configured to replicate on each drive.
You have replication off on the video share.
You have a replication level of two on the mp3 share.

When you store a photo greyhole write it to your 5 drives.
When you store a video it goes on a random drive.
When you store a mp3 it goes to 2 random drives.

So if you lose a drive you should loose about 25% of your videos, 6.25% of your mp3 and 0% of your pictures.

Just throw them away

[zmooc]

Powering 10 old harddrives for some time is going to be much more expensive than just getting a new one. A modern drive uses about 5W on average. these oldies probably use much more. 10 drives using 10 watts at $0.10 per kwh will set you back $87 per year. You do the math.

Use BtrFS or Drobo

[digitaltraveller]

Use drobo if you are time poor and money rich, use btrfs if you are time rich and money poor.

Btrfs's capabilities are nothing short of amazing. Here is a vid about it:
http://www.youtube.com/watch?v=9bQc_z-Cb7E