Slashdot Mirror

Data migration and expanding RAID containers is a major PITA. I absolutely loath the task!

That's why you don't use RAID. Instead, use something more flexible. I've been running Greyhole for a while now. Adding storage doesn't require shifting files around (unless you want to rebalance storage), you can use drives of different sizes, and you can control the level of redundancy you use (more for important files, less for stuff that's easily replaced). You can yank a disk out of a Greyhole installation and read all of the files off of it with standard file-copy tools.

Important stuff that doesn't take too much space (documents, Git repos, etc.) is backed up daily to Tarsnap. Less-important stuff (movies, music) and larger files (photos) get dumped to BD-R and are stored in binders in my desk at work; images are prepared with dvdisaster for added error recovery capability and are burned to single-layer BD-R HTL media.

RAID-5 uses up 1 disk worth for striping, so net space in an 8-drive array is 7-drives worth (about 27TB using 4TB drives). The problem with RAID-5 is that you are 2 disks away from failure and rebuilds often kill the disks.

RAID-6 uses 2 disks worth for striping, so net space in an 8-drive array is 6-drives worth (about 23TB using 4TB drives). Is able to survive a double-disk failure before data loss. Still has some of the same issues as RAID-5.

I use Greyhole for media and document storage. It handles disks of unequal size (currently running one 3TB and two 1.5TB drives), and you can choose the level of redundancy you need. In my case, movies, TV shows, etc. get a single copy (one file exists on one drive), while documents and photos get two copies (one file exists on two drives). If a drive goes bad, you only lose the files on that drive...and only for the files for which you selected no redundancy. With redundancy, extra file copies are recreated on the remaining drives from the surviving copies; this process is most likely less stressful on the disk set than a RAID rebuild.

My movies, TV shows, and music are backed up to BD-R, stored in a binder at work. They hold ~20GB each, as I'm using dvdisaster to guard against media errors. When a 2TB drive failed, I brought the backup (currently about 190 discs) home and restored the files that had gone missing. Backup and restore are managed by scripts, with information about what files are on what discs held in a MySQL database that gets periodically backed up off-site as well. The initial backup took several months (on and off) to finish, and the last time I needed to restore, it took about a week, but now I just burn a disc when I have about enough new data to fill one. Burning and verifying takes a few hours, but it's something you can start and walk away.

Do ya'll buy commercial NAS systems, or does anyone here do the FreeNAS type thing as a full custom solution?

I'm running Greyhole on my home server. It aggregates storage across multiple (possibly dissimilar) drives in one or more pools. You can set varying levels of redundancy for each pool; you can have two (or more) copies of documents so that they're safe in case of drive failure, while disabling it for your video library (which is OK if you have it backed up). You can pull a drive out of a Greyhole box and access the files written to it.

I've had a disk fail recently, and another was on its way out (smartctl reported it had no more replacements for bad blocks). In the latter case, migrating data off the old drive onto a new one was easy. The other drive failed outright. My documents and photos were all safe. Some video and music files needed to be restored from backup...a minor pain. Overall, I think it's worked "as advertised" and would recommend it to others.

Granted, this is talking about RAID 5, so let's naively assume that doubling the parity disks for RAID 6 will halve the risk... but then since we're trying to duplicate 24 terabytes instead of twelve, we can also assume the risk doubles again, and we're back to being practically guaranteed a failure.

Bottom line is that 24 terabytes is still a huge amount of data. There is no reliable solution I can think of for backing it all up that will be cheap. At that point, you're looking at file-level redundancy managed by a backup manager like Backup Exec (or whatever you prefer) with the data split across a dozen drives. As also mentioned already, the problem becomes much easier if you're able to reduce that volume of data somewhat.

Use Greyhole.
http://www.greyhole.net/

It distributes files across multiple storage locations, with a user defined level of duplication (lots of options for per file type and location duplication levels). Full drive failures aren't terribly common, but read errors are. To survive a drive failure AND a read error reduplicating/redistributing files to recover from the failed drive, then you would need 3x redundancy. If you used a 3x redundancy with 4TB drives, it would be 18 drives, which is a lot but not unreasonable.

Really though, that's for more of an onsite backup. To duplicate the data for taking offsite you'd only need six 3TB drives because you only need a single copy in the backup. Set up a system with the six drives, and tell Greyhole to store one copy of every file on that system. When you're ready, shutdown that system and take it offsite. For best results have two of these systems that you rotate. When you hook it up to the network, the main onsite Greyhole system will see it show up and start syncing files to it. The best part is that a full drive failure on the backup doesn't impact files on the other drives. Just replace the failed drive and wait for everything to sync back up.

In a crisis where the onsite location is destroyed, and all but one backup drive fails, you can still hook up the one working drive to any system and copy the files off.

Why am I the only one saying this? Setup Greyhole, throw a bunch of disks at it, and enjoy! And to all those saying "those drives are going to die soon", you can actually tell Greyhole that you consider a drive "broken" and it will still use most of its storage (albeit redundantly) until it does die and have to be removed.

-What happens if you lose a disk?
So you look to install raid
-what if all your disks aren't the same size?
and
-what if you want to upgrade just one disk? Or add a new disk? (I know both are possible with the raid-5 tools, but adding new disks takes HOURS, if not DAYS, depending on the size of your array.... not something I'd call usable to a home user)

MS drive extender and Unraid both have a home-user solution that open source does not match right now. I hope this changes soon!

Possibly Greyhole, which can be found in practice in the WHS alternative, Amahi.