Slashdot Mirror
Data Storage For Home?
kuom asks: "Every couple of years, I face the same problem: running out of hard drive space. No matter how big of a hard drive I get, I seem to find ways to fill it up within a few months. The size of my hard drive grew from 2.5G to 13G, to 20G, to 40G, 80G, 120G, and most recently, 200G. Today, I have a combined hard drive space of 280G, but I again find that I only have about 2G of free space left. My collection of family photos, web site content, TV episode captures, music files, and my archive of ISO files for various operating systems, they just eat up my hard drive space so fast. I could get a 400G hard drive, but I figure maybe it's time to think about something long term, something like EtherDrive or StorageWare. But the price tags are definitely out of my range. Slashdot readers, what do you suggest for home data storage?"
$100!? $40 nets you a nice one off Newegg. Dual layer, too.
It's only an insult if it's not true.
save up the cash and put together a 2 terabyte SATA storage server (the mobo doesn't have to be great, nore the ram)
That all depends on how important your data is. High quality parts should be of top concern. Also, don't skimp out on the ram -- the more ram you have, the more the server will have to cache recently accessed files.
2 TB should be enough space. MAKE SURE YOUR NETWORK THAT THE SERVER IS CONNECTED TO IS 100 MBIT (preferably the client transfering files to/from as well). It might be a good idea to maybe invest in gigabit.
2 TB will fill up rather quickly, especially if the server is used to store high definition video (like HDTV). Also, 100 Mbit is a huge bottleneck -- that only gives you roughly 12 MB/s, which isn't enough to keep up with highspeed DVD burners or even play full resolution HDTV. Furthermore, if you're using ethernet, remember that usage above 60%, especially with multiple clients (say, two people pulling/pushing lots of the server at once), suffers from rather high latency and packet collision. Moving the same amount of data over a 1000 Mbit connection reduces collisions and congestion by 99% as data moves 10 times as fast down the wire. Using gigabit (at least when ethernet is concerned) is really the only option with decent performance.
Be relentless!
Just don't use a sharpie pen to write labels on your disks. I did this on some CD-ROMs I used to backup some digital video and 2 years later the disk was badly corrupted. When you write on the silvery label you're actually writing on the back of the recording medium. The solvents from the sharpie leak right through the thin layer and corrupt the data. Printing out labels or using a special pen made for labeling CD's and keeping the Cd's out of the sun will help the data remain secure.
I'm a believer in segregating storage from the machines I'm likely to use/want to reload etc.
:)
And I've spent thousands of dollars on my home network and attendant PCs, to solve the problems that the original poster will only have if he manages to actually get enough storage for his needs.
Presently I have four identical storage servers, with the following characteristics:
Athlon64/3000, 1GB RAM, Gigabyte K8VM800 motherboards, 4 Hitachi 7k250s (RAID5 on 3ware ), 2 Hitachi 7k400s (soft RAID0, stores a daily snapshot of the RAID5, which is the data that is actually shared), 1 Samsung SP1614 Boot/OS drive, a 3Ware 8506-4LP, Intel Gbit NICs.
These machines run a series of scripts that collect and copy (pictures or MP3s) or move (video) whatever I happened to have dropped on my various workstations (each have between 300GB and ~1.7TB) to appropriate filesystems on the various servers (one for porn, one for ripped DVDs and TV shows, one for music, one for pictures); those filesystems are then exported via NFS to another Linux machine where the whole mess is presented back to all my machines as a single file system.
Getting enough storage is simply a matter of applying money. 250GB drives are quite reasonable nowadays and 160GB drives are downright cheap, but dealing with dinky little disks make getting enough SATA ports problematic. Sub-$100 2- and 4-port SATA controllers from the likes of Adaptec, Promise and Highpoint have their own problems. Most don't do online volume management and REALLY only do RAID through a driver, rather than an actual onboard processor. They're fine for storage expansion, a JBOD or RAID0 (note: RAID0 is normally a VERY stupid thing to do, since most people aren't doing STR-intensive things with their PCs and the chance of losing data is substantially higher than for any single disk), but as with everything else, you get what you pay for, and ports on a proper controller are probably worth more than the disks you're attaching to them.
RAID5 is also kind of a bad deal for write-intensive data - lots of little files that get updated a lot, while I'm at it. Do RAID1 or RAID10, (or maybe RAID3 if you can find a controller that supports it) for data you care about. Spend money.
USB2 and Firewire enclosures are NOT a good solution for adding primary storage most of the time. Generic enclosures frequently have difficulty with larger drives, and often have VERY cheap fans that either fail quickly or detriorate to the point that they sound like a penny in a vaccuum cleaner. Additionally, the performance and CPU utilization of USB2 enclosures both tend to be god-awful. Brand-name enclosures have a few different problems: many use 2.5" disks, which in my experience are rather delicate. Others are not properly cooled, and almost all of them are sealed enclosures. Better to put a drive inside a computer if possible. I tend to think of USB2/Firewire drives drives as backup devices only.
Disk-wise I tend to prefer Hitachi 250 and 400GB models, or Samsung 160s or 200s, and SATA over PATA when possible. The Hitachi 500GB models get too damned hot, and it's the only one that's out (available for purchase) at the moment. Seagate and Maxtor ATA products tend toward tepid performance, and in the case of Maxtor, quality hasn't been good since the Quantum merger in 2001. I will not purchase a Western Digital drive for any reason, but specifically I avoid the geek-favorite Raptor 360GD; I was party to the construction of a small 20-drive SAN using Raptors (client's spec, not my idea) a couple years ago where the drive failure rate was approximately one drive every 33 days.
-- I wanna decide who lives and who dies - Crow T. Robot, MST3K
Where'd you learn math? 200 DVD's times 4.5GB (actually more like 4.4) per disc is only 900GB, less when you consider you can rarely fill the disc up completely due to odd files sizes. Plus most of those disc changer robots run to the thousands of dollars, you're better off dropping $6-700 on 4 or 5 300GB hard drives and a cheap PC to serve up files from them, I also suggest a seperate smaller drive (20-40GB is more than enough) to boot your OS off since software raid arrays aren't bootable. Put together a cheap system for $2-300 to strap the drives in and you've got over a TB of live networkable storage for less than $1000, spend a hundred more on a tuner card and install MythTV on the machine and you just got a massive PVR system as a bonus.
Labels can be bad too.
There are also some other good tips for archiving CD/DVD type media in that article.
-- Pete.
Monochrome - Probably the UK's largest internet BBS
100 Mbit equipment is so cheap now, every PC that's at least 100 MHz should have it. If you have some older workstations, 100 Mbit cards may be hard to find, but these would have to be pretty old boxes to not come with 100 Mbit ...
That's not true if you've got a switch rather than a dumb hub, and switches are pretty much the norm now. Switched networks have almost zero collisions, no matter how busy they are. (Granted, they should have zero collisions, not almost zero, but switches do occasionally have bugsSure, latency does go up when you're pounding on the network, but on a 100 Mbit network it's not that much -- just a few miliseconds at most. And for moving lots of data around, latency doesn't matter much anyways -- it's throughput that matters.
Not quite. You won't actually see 120 MB/s with today's hardware, but you might see 20-30 MB/s, maybe a little more on really serious hardware, which is still a lot better than the 12 MB/s you'll get out of 100 Mbit ethernet. And it's relatively cheap now -- I recently got gigabit PCI cards for $5 and a 5 port gigabit hub for $20 at Fry's.This is true for CDs, but not DVDs. They have the data layer in the middle of the plastic bulk, whereas CDs have it on the label-side surface.
You can see this clearly if you microwave coasters of each :) The shiny layer of CDs begins to strip away, whereas that of DVDs stays inside the plastic.
Escher was the first MC and Giger invented the HR department.
delete most of your porn. It mostly looks alike anyway, so just ditch it. You should then have close to 200 GB free.
Some people are like slinkys. They're useless, but it puts a smile on your face to push them down the stairs.
SGI has a neat solution to this problem called DMF. It will automatically migrate infrequently-accessed files onto tape, and 'recover' them when accessed (takes a little longer than normal) automatically when there's a tape library, and I expect it will prompt an operator to load a tape if the tape isn't online.
I've no idea what the cost is, or if there's a low-end solution.
Max.