Slashdot Mirror
Best Solutions For Massive Home Hard Drive Storage?
i_ate_god writes "I download a lot of 720/1080p videos, and I also produce a lot of raw uncompressed video. I have run out of slots to put in hard drives across two computers. I need (read: want) access to my files at all times (over a network is fine), especially since I maintain a library of what I've got on the TV computer. I don't want to have swappable USB drives, I want all hard drives available all the time on my network. I'm assuming that, since it's on a network, I won't need 16,000 RPM drives and thus I'm hoping a solution exists that can be moderately quiet and/or hidden away somewhere and still keep somewhat cool. So Slashdot, what have you done?"
Do something like this. Put it in a case / box / cabinet of your own design since you don't need the rackmount capability.
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/
So Slashdot, what have you done?
Why? What have you heard??
I used to work for ABC news and we never kept archive footage always accessible like you want. If we wanted something that was really old we'd have to dig it off a tape, an unplugged hard drive or powered off computer, or we'd have to find another news agency that had the footage and grab it off of a satellite feed. And this was a 24/7 TV news station responsible for national news programming where we would be tracking stories for years. If we didn't need a system where everything was instantly accessible then you needing it on an individual level might be overkill in my opinion.
I have over 30TB of music, movies, and raw video footage on my home computers and I just keep everything on separate external hard drives. I label the drives, back them up twice each, and then keep an index in a .txt file that is easy to search through. So if I want a 1080p backup copy of Blade Runner I search 'Blade Runner' in the .txt file and I see it's on drive 'A' and then I plug in drive 'A' and dump the movie on my computer. I also keep an external drive that has backups of every TV show I own on DVD. So if I want to watch The Wire then I plug in the external drive labeled 'TV' and have at it.
Don't worry, we'll be right over and take care of everything. You'll never have to worry about it again.
MPAA
P.S. My sister, Riaa wants to know if you're into MP3s
I download a lot porn, and I also record a lot of masturbating videos. I have filled two computers with porn already. I want access to my porn at all times, especially since I maintain a porn site. I don't want to have swappable USB drives, I want all my porn available all the time on my network. I'm assuming that, since it's on a network, I won't need 16,000 RPM drives and thus I'm hoping a solution exists that can be disguised or stashed away and not overheat. So Slashdot, what have you done?
Having done that in the past, I'll say that buying a Drobo was worth the cost. Granted, I hunted around a bit to get a good sale price (it's not too difficult... though the FS is brand new so maybe not on that model yet), but unless you really enjoy tinkering with getting samba shares set up and working properly, sometimes it's just easier to buy your sanity.
Don't get me wrong - I wish they were cheaper. But their system worked better and more reliably than anything I ever put together, and I'm by no means incompetent. And their BeyondRaid tech, while proprietary, is pretty damn cool and works incredibly well. Being able to mix drives and not waste tons of storage space is a huge advantage that (as far as I know) I'm not going to get anywhere else.
Just a happy customer, not an employee or anything like that.
How are sites slashdotted when nobody reads TFAs?
How much data constitutes "massive"?
640K of memory should be enough for anybody.
I have a Drobo and a DroboShare. The DroboShare runs a slimmed down version of Linux so a network attached Drobo uses typically uses samba. The benefit of having one is NOT the ease of software set up. The reason I love it is the ease of drive management and small hardware size.
Due to the small size and slick style I keep mine in my TV cabinet. I've done the measurements and no PC case on newegg can fit in this same space, never mind something that can house 4 harddrives.
The other thing that is so valuable about a Drobo is how well it manages it's RAID array. They call it BeyondRaid but I hear it's just a as many normal RAID arrays as it needs to organize the drives to both optimize space and maintain redundancy. Also you can pop harddrives in or out while it's on and it will automatically restructure the RAID on the remaining drives to still be redundant with out any need to shutdown or stop sharing data. I recently needed to test this out for my self. I popped out my 4th drive, plugged it in to my PC, formatted it and started moving data from my Drobo to the harddrive I just removed from it while the Drobo was still restructuring. I expected a huge mess, but everything worked exactly like the advertised. I was kinda shocked.
FYI the reason I did that swap out was because I foolishly formatted my Drobo as NTFS. This worked ok but I had one to many problems talking to it from my Linux PC. The permissions were all messed up over samba. New folders and files I created on the Drobo were root access only for some weird reason. So I decided to format it as ext3. Since the DroboShare runs Linux this is the best option for a shared drive and works fine while talking to mac and windows as long as you do so over the network.
When I hear a question like this, I usually recommend heading over to the NCIX forums. There's some crazy guy over there - death_hawk - building a 100TB array.
What I did was a bit less ambitious. A regular old NAS running off a cheap non-RAID SATA card in a case with lots of HDD bays.
For interest, I'll throw up a build that easily scales to 12TB. Since you mentioned noise, I'll prioritize that instead of capacity. I'll use a case geared for silence, a fanless mobo/cpu, a quiet PSU, WD Green HDDs, and a ridiculously cheap SATA card.
Case - 8 bays: http://www.ncix.com/products/?sku=51277&vpn=6900654&manufacture=Fractal%20Design *1
Motherboard/CPU - Silent: http://www.ncix.com/products/?sku=50891&vpn=AT5NM10-I&manufacture=ASUS *2
DDR2 - 1GB: http://www.ncix.com/products/index.php?sku=18584&vpn=VS1GB667D2&manufacture=Corsair&promoid=1114 *3
PSU: http://ncix.com/products/?sku=33357&vpn=CMPSU-400CX&manufacture=Corsair&promoid=1114 *4
SATA Card: http://ncix.com/products/?sku=19892&vpn=SY-SA3114-4R&manufacture=Syba *5
HDD - 2TB 4KB http://ncix.com/products/index.php?sku=49591&vpn=WD20EARS&manufacture=Western%20Digital%20WD&promoid=1114 *6
HDD - 2TB 512b: http://ncix.com/products/index.php?sku=36130&vpn=WD20EADS&manufacture=Western%20Digital%20WD&promoid=1114 *7
OS: FreeNAS, Ubuntu, Win7, Other *8
*1 Only six will be filled. 6 SATA ports.
*2 Case still requires fans/airflow.
*3 A NAS probably only needs 512MB, but 1GB is cheap. A Win7 NAS may benefit from 2GB.
*4 Must be capable of spinning up 6-8 HDDs at once.
*5 Must be flashed with new non-RAID BIOS to avoid silent data corruption for > 1.0TB HDDs; disk read/write speeds around 30MB/sec, in my experience, on ext2. (but running with a VIA CPU - not dual-core Atom)
*6 Must be specially formatted under Windows and Linux. (Most distros only support 4KB sectors when the drive reports 4KB - these report 512b to maintain XP compatibility)
*7 May have longevity issues. (too early to say right now - lots of complainers, which reminds me of the 7200.10 days. A heck of a lot of those chirping barracudas perished early)
*8 Please verify SATA card support first. Ubuntu and FreeNAS work fine with this card, but I've never checked if Win7 has drivers. Do note that you'll have to flash it. *9 If that's a problem, buy a more expensive card. (which may give better performance, and SATA2 support) Promise makes nice non-RAID SATA cards.
*9 Flashing the PCI SATA card requires making a DOS boot CD: http://www.hiren.info/pages/bootablecd
Please note: A solution like this will take 12+ hours to set up. It's highly likely you'll blow a whole weekend, even if you know what you're doing. You may have to try multiple distros to get proper Atom D510 support, unless you go with Windows. When I put mine together, atoms weren't available affordably, so I went with a cheap VIA board. Ironically, Ubu