Slashdot Mirror
Best Solutions For Massive Home Hard Drive Storage?
i_ate_god writes "I download a lot of 720/1080p videos, and I also produce a lot of raw uncompressed video. I have run out of slots to put in hard drives across two computers. I need (read: want) access to my files at all times (over a network is fine), especially since I maintain a library of what I've got on the TV computer. I don't want to have swappable USB drives, I want all hard drives available all the time on my network. I'm assuming that, since it's on a network, I won't need 16,000 RPM drives and thus I'm hoping a solution exists that can be moderately quiet and/or hidden away somewhere and still keep somewhat cool. So Slashdot, what have you done?"
Do something like this. Put it in a case / box / cabinet of your own design since you don't need the rackmount capability.
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/
I personally set up a downloadserver that also functions as a media server to stream the content to other devices. I put in a couple 5400 RPM 1,5 TB drives, they use less power, generate less noise and heat than a regular 7200RPM drive but since you're not running any applications of them, you won't really notice the difference in performance. Prices have gone down a bit so the sweet spot for $/GB might be at the 2TB mark now. If you don't want to go for an entire computer, maybe a NAS solution would be best for you, with the same 5400RPM drives. A NAS will have less room for the disks if you really want *massive* amounts of storage, and also you usually must purchase one + the disks. The PC you can build from spare parts lying around. I personally put gentoo linux on mine, but you also don't exactly need top of the line equipment for a nice windows XP install. The NAS however will have outputs directly for your TV and will take up less room and power.
Still, the key is 5400 RPM + 1,5/2 TB.
Good fucking god, $700 for the Drobo FS?
You could build a capable home server box AND buy some of the drives for that much.
Mod me down, my New Earth Global Warmingist friends!
I've tried a variety of approaches, but overall I've been happiest with just buying a NAS box.
I have a Synology DS209, and I've been very satisfied. It's a relatively cheap way to get 2 TB RAID 1 storage with really simple backup to an external USB drive. If you need more storage, you can buy NAS devices with more than just two bays.
This. Even ready-made resellers have pretty small devices built on Win Home Server that can take a LOT of drives. Mine supports 4, but there's many models that can take 8, 12, or more drives. The OS is rock solid and has a lot of neat features, like being able to access your network from an SSL secured web app (built in) from anywhere with indexed search, and its easy to develop plugins for (though there's a ton available already) to extend it.
You made no mention of a budget. I'd go with a Drobo - probably the DroboFS. http://www.drobo.com/products/drobo-fs.php
Informatus Technologicus
http://en.wikipedia.org/wiki/ATA_over_Ethernet
This is something I've always wanted to play with. It's a little expensive (for a home user) to get into, but it's extremely scalable. If I moved all my DVDs and such to on-line storage, I think this is what I would opt for. It can be run in all sorts of RAID configurations, doesn't require matched sized hard drives, and it can all be racked up very nicely.
Even if you "need' a 16000 RPM drive, just make it for your local drive that you play your videos directly off of. Use 5400 for all the other ones. Just move your file before watching it. Sure, if you're an impatient baby and want to watch something within 5 seconds of it entering your mind, then you might have to wait 5 minutes if the file is 4.5G. Then again, it's the type of waiting you can go pee or make your snack during.
-Clio
Karma: Bad (mostly from not giving a fuck)
Blog: http://clintjcl.wordpress.com
ZFS + Solaris.
I have a standard ATX case with 4-in-3 adapter from Newegg. I didn't get the more expensive ones with trays because I didn't need to hot swap.
I have 2x1TB drives in ZFS mirror for boot. 5x1.5TB drives in RaidZ as a tank and 2x200GB drives in mirror with a virtual block device for Xen Debian and Windows 7.
OpenSolaris is amazingly simple to use, if you're just doing your home network.
At the most basic level:
zpool create tank c5t0d0s0 c5t1d0s0
zfs sharenfs=on tank
zfs sharesmb=on tank
zfs shareiscsi=on tank
Now your new drives are all shared over NFS, SMB and iSCSI.
I keep looking for the old school full height 'desktops' at a bargain store or so. Search newegg. 3.5" external works just as well as internal.
This has 11 3.5" bays and 3 x 5.25" bays. With a 4 in 3 linked above you could have 15 hard drives in a case for $100. Or if you care about hot swappability This one has 20 hot swap bays (at 3x the cost).
If you want more performance, get some SSDs to work as the ZFS "cache".
There are large capacity "green line" drives from some manufacturers, 5400 RPM, that might be perfectly enough.
Do they work in RAID? or do they randomly stop responding to the raid controller and then get dropped from the raid, triggering a rebuild, to show up a few minutes later, to trigger another one? http://en.wikipedia.org/wiki/TLER
All of the above was encrypted with a Quad ROT-13 method. Unauthorized decryption is in violation of the DMCA.
I've got a $20 case, $50 500W power supply and $40 motherboard with SVGA and ethernet, its 5 PCI slots each stuffed with $25 4x SATA cards, 20 $100 1TB HDs. Running Linux, network mountable drives and ssh login.
That's $210 PC + $2000 HDs for 20TB. That's a lot of porn storage for you.
--
make install -not war
I have a similar setup to previous post. Server running Linux with a 3ware RAID card running RAID 5. 8 500GB disks with ~3TB of usable disk space. This has been running flawlessly for over 5 years. I have a a movie collection, music and network share for 3 HTPC in the house. Works very well but I wish my RAID card supported the ability to power down the disks and save of power/heat when not in use.
I am almost at capacity on the RAID volume, so to expand I have another RAID card that I can put in the server and create a new volume or replace the 500GB drives with 1TB or now 2TB disks. Replacing the disks would save power and heat, but I would need to backup and restore 3TB of data. Adding another RAID card is easy, but crates more dives that I can't turn off and eat up power.
I am actually thinking about building a new server with the thought of being able to add an many SATA ports as possible (via SATA cards) and then us port multipliers. The use a software based file system or RAID that allow me to add drives of different sizes to the volume. Similar to ZFS but more open. This would make growing the system much easier and allows me to power down drives when not in use. I would still get plenty of performance for my needs.
The other thing that I am doing that most people don't think about, is I backup my entire NAS to another server. I took another old PC that I had and put a 4 port SATA card in it and four 1 TB disks and run Linux and software RAID on it. Each night it powers up and runs a script to back up the primary NAS. I do this just in case something catastrophic happen to my primary NAS and I also use it when I moved to larger disks on the NAS previously. I use rsnapshot to look for changes on the primary NAS' file system and only back up data that has changed. It also keeps the lat 3 months of files that have been changed or deleted, if I need to recover something. When the script is finished, it powers down the backup NAS and wait until the next night to run again.