Slashdot Mirror
Best Solutions For Massive Home Hard Drive Storage?
i_ate_god writes "I download a lot of 720/1080p videos, and I also produce a lot of raw uncompressed video. I have run out of slots to put in hard drives across two computers. I need (read: want) access to my files at all times (over a network is fine), especially since I maintain a library of what I've got on the TV computer. I don't want to have swappable USB drives, I want all hard drives available all the time on my network. I'm assuming that, since it's on a network, I won't need 16,000 RPM drives and thus I'm hoping a solution exists that can be moderately quiet and/or hidden away somewhere and still keep somewhat cool. So Slashdot, what have you done?"
Do something like this. Put it in a case / box / cabinet of your own design since you don't need the rackmount capability.
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/
ZFS + Solaris.
I have a standard ATX case with 4-in-3 adapter from Newegg. I didn't get the more expensive ones with trays because I didn't need to hot swap.
I have 2x1TB drives in ZFS mirror for boot. 5x1.5TB drives in RaidZ as a tank and 2x200GB drives in mirror with a virtual block device for Xen Debian and Windows 7.
OpenSolaris is amazingly simple to use, if you're just doing your home network.
At the most basic level:
zpool create tank c5t0d0s0 c5t1d0s0
zfs sharenfs=on tank
zfs sharesmb=on tank
zfs shareiscsi=on tank
Now your new drives are all shared over NFS, SMB and iSCSI.
I keep looking for the old school full height 'desktops' at a bargain store or so. Search newegg. 3.5" external works just as well as internal.
This has 11 3.5" bays and 3 x 5.25" bays. With a 4 in 3 linked above you could have 15 hard drives in a case for $100. Or if you care about hot swappability This one has 20 hot swap bays (at 3x the cost).
If you want more performance, get some SSDs to work as the ZFS "cache".
I have a similar setup to previous post. Server running Linux with a 3ware RAID card running RAID 5. 8 500GB disks with ~3TB of usable disk space. This has been running flawlessly for over 5 years. I have a a movie collection, music and network share for 3 HTPC in the house. Works very well but I wish my RAID card supported the ability to power down the disks and save of power/heat when not in use.
I am almost at capacity on the RAID volume, so to expand I have another RAID card that I can put in the server and create a new volume or replace the 500GB drives with 1TB or now 2TB disks. Replacing the disks would save power and heat, but I would need to backup and restore 3TB of data. Adding another RAID card is easy, but crates more dives that I can't turn off and eat up power.
I am actually thinking about building a new server with the thought of being able to add an many SATA ports as possible (via SATA cards) and then us port multipliers. The use a software based file system or RAID that allow me to add drives of different sizes to the volume. Similar to ZFS but more open. This would make growing the system much easier and allows me to power down drives when not in use. I would still get plenty of performance for my needs.
The other thing that I am doing that most people don't think about, is I backup my entire NAS to another server. I took another old PC that I had and put a 4 port SATA card in it and four 1 TB disks and run Linux and software RAID on it. Each night it powers up and runs a script to back up the primary NAS. I do this just in case something catastrophic happen to my primary NAS and I also use it when I moved to larger disks on the NAS previously. I use rsnapshot to look for changes on the primary NAS' file system and only back up data that has changed. It also keeps the lat 3 months of files that have been changed or deleted, if I need to recover something. When the script is finished, it powers down the backup NAS and wait until the next night to run again.