Slashdot Mirror
Data Storage For Home?
kuom asks: "Every couple of years, I face the same problem: running out of hard drive space. No matter how big of a hard drive I get, I seem to find ways to fill it up within a few months. The size of my hard drive grew from 2.5G to 13G, to 20G, to 40G, 80G, 120G, and most recently, 200G. Today, I have a combined hard drive space of 280G, but I again find that I only have about 2G of free space left. My collection of family photos, web site content, TV episode captures, music files, and my archive of ISO files for various operating systems, they just eat up my hard drive space so fast. I could get a 400G hard drive, but I figure maybe it's time to think about something long term, something like EtherDrive or StorageWare. But the price tags are definitely out of my range. Slashdot readers, what do you suggest for home data storage?"
First off, burn your shows to DVDs and your ISOs to CDs. Why have them on the hard drive if you aren't going to use them?
I'd set up an older computer system (P2 or newer or older..) with a RAID configuration. Something like RAID1 for simplicity and redundancy. Then you just store everything on that central server and your client machines don't matter.
Or get a bunch of chinese kids in your basement to memorize strings of 0's and 1's.
Death by snoo-snoo!
Seriously, 200GB is peanuts. My MP3 collection alone is 100GB. I also back up all my computers (including the one with the 100GB MP3s) to hard drive, so I have two copies of all this stuff.
I just bought one of these: LaCie Bigger Disk for my Mac, which is 1TB RAID for under $1000.
Just buy a TB or two, and if you fill that up any time soon, then, well, start deleting stuff. My personal files take up megabytes, my MP3s take up gigabytes, so 1TB is fine for me. 1TB ought to be enough for anybody.
I'm even thinking of buying another one of these, syncing the first one up to it, and then putting it in a safe deposit box and repeating every 6 months.
It sounds like you may be a digital packrat. If you are, I sympathize as I was one too after I got broadband. I stopped after realizing how much time/money I was wasting on crap that really didn't improve my quality of life. Now I buy new hard drives for my file server once every three years instead of every three months. Following are some of the things that helped me.
1) Download less porn/warez. Or put off downloading more until you've watched/run/played what you have. If you're just one person cranking through that much space that quickly, then you're downloading things just to have them. Stop that.
2) Go through and 'rm -rf' files and directories that you haven't accessed in a year. Don't keep obsolete versions of operating systems around, because you won't use them. As soon as you download a CD image, burn it and rm the ISO.
3) Archive on external media anything that's sentimental but rarely accessed.
4) Make it a routine to burn stuff to CD/DVD at least once a week. Eventually, you'll get tired of wasting time burning crap that you don't use and this will help you realize that you really don't need it at all.
5) If you do a lot of video editing or webmastering that requires huge amounts of data, and you're making money at it, then you need to invest in a proper server to keep all that. Be sure to make backups too. If you do this for a company, have them take responsibility for this.
6) Take a page from Linus's book: Upload it to an FTP server and let the world mirror it.
Labels can be bad too.
This is true. I had several CDs I had burned a few years ago that were unreadable. I removed the labels by soaking the disc in isopropyl alcohol and after that, I had no problem reading the discs.
I'm the urban spaceman babe, but here comes the twist... I don't exist
I have two OpenBSD systems (Compaq D5S SFF boxes), each with a 400GB hard disk. One sits at home in a segregated part of the network running Samba. This serves up files for a Powerbook (OS X) and my wife's Dell (Win XP). The other system sits at my best friend's house. Both systems allow SSH with public key auth only. I tunnel rsync over SSH from the master box at home to the other system to sync files. This happens once a day at 2am.
The master box at home has a cheap DDS4 drive which I got from eBay for about US$65. I get brand new tapes for about $4 a piece. I have a 4 set cycle on which I do a full backup every week. After completion these tape sets get taken to a different friends house and she keeps them for me. Incrementals during the week I keep hold of.
My Powerbook has a superdrive and I do a set of DVD+RW's every once in a while. I copy my critical files such as accounts etc additionally to a 512MB USB key, which is encrypted and I keep in my bag. That USB key contains the data that I simply could not do without.
As for storage needs, when I begin to outgrow my 400GB allocation and tape & DVD backups/archival becomes too time consuming, then I will move to a RAID 10 solution. However, for me this probably won't be until 2008. At that point I will probably do another home brew solution. I have started looking at what those options are. There are quite a few companies that offer rack mountable chassis with lots of storage bays. My idea was to put a stripped down version of OpenBSD booting from industrial grade CF card, similar in concept to how one might go about making a firewall device. I'd go for a good quality server motherboard and then add a well supported SCSI or SATA RAID Card. Seagate Barracuda drives have a 5-year warranty on them, so I'd probably go for some of those. I'd then add striped and mirrored storage as required as my storage needs expanded. This is probably going to be a fairly costly exercise, but I shall be doing an audit of my filesystem at some point this year to clear out the old files and pornography with which I have become bored.
The most important thing to do is work out what data is actually worth protecting. If like me you have ripped your Futurama and "24" DVD episodes into DivX for easy access and local copies whilst on the road, bear in mind that you can always re-rip them. If the house goes up in a puff of smoke, it would probably be cheaper to re-purchase the DVD set with the insurance money rather than keep archiving copies of stuff onto media which you have to continually protect, maintain and renew. The stuff I'm paranoid about is the data that I *can't* buy again - i.e. pictures of holidays, family and honeymoon, my accounts files and scans of receipts for my business, plus logs of chats and text messages.
Good luck.
The bigger the disk, the longer it takes before you get the dreaded 'disk full' message, but it will take exponentially longer to recover from it.
Who cares? The last disk I added took three days to be fully on-line. But my investment in the process was about 30 minutes. It took 20 minutes to write the script to iterate through my RAID-5 volumes, run "pvmove" to migrate the data off of each volume, remove it from the volume group, rebuild it with the new disk and add it back into the group, then another minute or two each time I checked on the progress. In the meantime, my data was always on-line, and always safe from up to two (consecutive) disk failures.
The problem you describe is only a problem if you have lousy disk management tools.
Oh, just a nit: your use of the word "exponentially" is inaccurate. Even without LVM or the like, recovery time increases linearly with total storage, not exponentially.
there is no shops open to get new disks or all your storage enclosures are already filled to capacity with the biggest disks you can get
Yes, you could have a problem if you allowed this situation to arise. So don't do it. I keep a goodly chunk of my available space unallocated to any logical volumes, so when I actually run out of space, I simply run "lvextend" to allocate the space, and then set about getting more storage. The bigger problem is when there's no more room in the box for additional drives. The solution to that is simple: Don't let it get that way. Always make sure you have room for one more. When your box is almost full, and you need more storage, you have to either add another enclosure or, more likely, replace a drive with a bigger one. The scenario where they're already the biggest drives available is pretty unlikely. My file server can accomodate six drives so filling it up with the currently-largest drives would be 3TB of storage. That's a *lot* of home movies.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
See my signature. And that was seven to nine months ago, so it might very well be cheaper now, and/or possible to get more capacity with the newly-available 500GB drives.
Yes, $4100 is a lot of money. But I built it for exactly the reason you mentioned; I've always been of the philosophy of "do it once and do it right," and this way I've taken care of my storage needs for years to come.
When you write on the silvery label you're actually writing on the back of the recording medium.
Of CDs: true. Of DVDs: False. DVDs sandwich the recording medium between two layers of substrate, rather than having it sprayed on the one side of the disc. This has several benefits:
On labels, we have a problem here at my workplace in that we receive data from a certain other department on DVD. Regulations require that the DVD be labelled in a particular fashion (doesn't say use a label, does say $wordy_legal_boilerplate must appear on the disc) and the other department refuses to cease using sticky labels to do this. Our department all use laptop computers (because we all have to work weekends from home occasionally) and the sticky label causes the disc to warp slightly as it heats up in our machines, thus causing it to stop being legible after about 15 minutes unless you keep it 100% spun up the entire time. Labels on optical discs are evil
www.wavefront-av.com