Slashdot Mirror
Home Network Data Storage Device
It happened again- a machine on my home network died. Taking with it tons of data. It's mostly backed up. No huge loss. But I finally think it's time to get some sort of network raid disk. A unified place to safely store data accessible to the numerous machines on my home lan. So now I pose to Slashdot readers- what are your recommendations? I'm looking for something with RAID and SMB sharing. At least a quarter TB, probably a half, but with some room to grow. What have you used? What works? What fails?
As CmdrTaco, I'm sure you have money coming out of your ears that you've harvested from the pseudo-religion that is Slashdot.
... I know, I know, I'm going to catch hell for using such a crappy generic product. And I know many people who will tell you that VIA is crap when it comes to RAID controllers. Maybe you're one of them. If you are, I hear that the brand Promise provides excellent RAID controllers, you'll just pay a whole lot more for them. A couple of these babies in RAID 1 and you're set.
But for those of you with fewer fiscal resources, I will tell you the stories of my friend and me, a.k.a. The Master Rebaters.
My story is a simple one. I love music. I have over 1,000 CDs and have spent a lot of time meticulously ripping them with my friend CDex. So, I have some 350-400GB of data that I would like to archive. There are a multitude of possibilities but, since I'm short on cash, I opted for a simple $13 RAID 1 controller
My friend, however, opted for a huge and expensive RAID 6 array controller made by Promise. Then he waited and waited until there was a 250 GB Maxtor rebate at CompUSA or Outpost and went in and bought five with cash. Then he filled out the rebates for relatives and played the waiting game. Huge initial investment but he received a lot of money back slowly. Result, a 1.1 ~ 1.2 TB RAID array. He got a lot more storage and more efficient use of the disks since a RAID 6 with striping allows for drives to be rebuilt in the array.
What he wasn't planning on was the logistics of what he would have to do to his Antec case as a result of all these drives. Fans. Airflow. Heat. These all became huge issues for him--especially in the summer. I'm not sure what your situation is with a case but I made no alterations to my case.
Now, there's a lot of things I skipped over that you can take into consideration, like SATA or ATA? 7,200 RPM or 10,000 RPM? 8MB or 16MB buffer? Striping size? etc. Honestly, those issues aren't worth my time to mess with. Sure sure, I'm losing precious ms seek/read time on my disks but I'm not that motivated.
In the end, if you're only looking for half a TB, do what I did. Those 500 GB drives will only get cheaper and if one blows, just pop another in. And if you really need that room to grow, grab the nice RAID controller that supports RAID 0-6 and just use two 500GBs leaving the other three slots open for the future when you might buy them and RAID 6 it.
What fails? The old IBM Deathstars. Beware!
My work here is dung.
First I'd recomend using a size formating in your question that better fits your situation like "At least 250GB, probably 500GB, but with some room to grow".
On to solutions. Buy yourself a big case (you can do rackmount or regular "large" ATX cases) and stick a decent computer in there. Add Gigabit NIC. Add an 8 port 3ware SATA Raid controller (configured to RAID5). Add 4 120GB 7200RPM SATA Drives (or what ever you can find cheap, even 200GB drives are relativly cheap these days). Install Linux, share your harddrive using Samba. Done.
You have 4 extra ports to expand your RAID if you need too, or you could get bigger harddrives. I think 3Ware cards can support up to 2TB of HD space - so that gives you some expandability. Plus you have a RAID5 which has fault tollerence built in.
snowulf.com
Tera Station
Everything you need probably. I saw a 1TB version for $700 at Fry's the other day.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
There are dozens of products out there to do this. Linksys alone makes several. You obviously didn't search slashdot, google, etc. The fact this article got accepted... Words fail me.
-Foxxz
If you have the funds, this looks promising.
:)
LaCie Biggest F800 1.6TB RAID Storage 300943U (USB 2.0, Firewire 800).
It's only CAD $2,599.99, so that's like US $100
How can you dupe so many stories yet fail to dupe your own data? :)
I've been looking on-line trying to find this sort of possibility and the only prefab system I've found that has configurable RAID in a consumer NAS is the Buffalo Terrastation. I've seen lots of NAS devices but basically they are all just a single hard drive with a network connection.
I have not used one of these and do not know if it's any good, but like I said, I haven't seen any other options for a prefab system. I've priced out what it would cost to roll my own system like this and it ends up being only a tad more expensive to get a prefab device. Actually, I think the price dropped on the terrastation so I'm not sure that's true anymore.
Also, if you get something like this, you should seriously consider upgrading to gigabit Ethernet if you haven't already. I have a network mounted share for most of my files and it works pretty well, but when I try to do things like synchronize my ipod against it, it totally crawls. Having a networked file server works better if it doesn't feel like your files are on a network.
This sig has been temporarily disconnected or is no longer in service
My condolences on your recent loss.
Couple questions:
1. SMB only? NFS is faster and plain better, but only for mac/linux.
2. Noise/size/power constraints.
3. Price.
SMB only, moderately cheap, quiet and small, go for a teraserver from buffalo networks. Easy to setup, runs decently, 4x250 drives that can be raid-5'd into a 750 array. Costs about $800.
A good midlevel solution is an nforce4 motherboard, with 4 250 sata drives, total cost around $600 w/ cpu mem, etc. You need a decent case though, and it will be noisier and louder. Plus side is better performance, full customization, and ability to use it as a router or such. You will have to configure it yourself, and likely throw windows on it because the nforce raid support is tricky on linux for a novice.
I use a heavier 2tb solution myself with a HW raid card, but for most purposes a sw raid is better, and the performance difference is almost never noticable. Personally I recommend the buffalo if you don't need nfs, just for the size, quietness, and convenience.
The first rule of USENET is you do not talk about USENET.
Never trust your data to any one box.
As for the solution, the cheap and easy option nowadays is to simply use stock motherboards - most will accomodate 4 SATA drives and up to 4 PATA drives with no extra work - and run Linux with software RAID on them. It's still a problem to boot from a RAID disk, so one can be set aside for that purpose. Motherboards have GigE nowadays, so speed is not limited by the network link. 300 GB drives are cheap, making a 1.5 TB server affordable if you acquire it piecewise over the course of a year or two.
Now duplicate this setup into 2 boxes and you're good to go.
[an error occurred while processing this directive]
RAID could help with downtime, but is not a substitute for backup, really. Tape backup is still very expensive (high inital cost), and DVD's are limited in both quality and storage capacity. Well, I use both, but then my storage needs are slight since I burn my most important data to a DVD-RAM disc every night.
What OpenBSD thinks about RAID:
The only non-obvious thing (i.e. a lot of people are telling you to do the wrong thing) is that you should use software RAID instead of hardware RAID. The cheapest CPU that you can buy, will still be 99% idle.
A less non-obvious thing (but some people still forget it) is that you want a well-cooled machine, because heat is what kills hard disks. Get a nice case; pretend you're building a machine that you wanna overclock like an 31337 h4xx0r, but then of course, don't really overclock it.
Oh yeah, and keep an eye on /proc/mdstat -- when your first disk dies, you want to know it happened, instead of finding out a year later when your second disk dies. (I use a lil' python script that displays the array status on a VFD using lcdproc. But there are lots of other ways to deal with it. Just make sure you deal with it somehow.)
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
Recently I was also shopping around for a storage solution. At the store, I saw a promising looking device called the Netgear SC101. You pop any two IDE drives into it, plug an Ethernet and power cable in the back, and you have yourself a NAS. Because you can pick out your own drives, you can even do a terabyte in a cheaper and much smaller unit than 4 x 250 GB units like the Buffalo Terastation.
Unfortunately, where this device failed for me was that it doesn't just share the stuff as a SMB share like a real NAS box does. It uses some weird proprietary protocol, and only machines with the right drivers installed can talk to it at all. Such drivers aren't available for Linux, or Mac, or BSD... even versions of Windows that are old (98, ME, etc.) or 64-bit won't work. It has to be a 32 bit version of Win 2k3, XP, or 2k with the right service pack level for the drivers or no data for you.
No self-respecting geek would want a device with such limited compatibility. If a piece of network equipment only lists Windows in its compatibility, that normally means the manufacturer only officially supports Windows, or maybe you need Windows to set up and administer the thing. When even many versions of Windows can't access the device, it's a junker. I took it back the next day, and will start researching hardware purchases more carefully in the future.
In short, Netgear's short-sighted decision to use some strange proprietary protocol instead of SMB turns this unit from something I would have strongly recommended into that gets a definite thumbs down.
I own the X6 and love it.
- It's GBE is very fast.
- It supports raid-5 with up to 4 drives. (mirroring on 2 drives)
- You can just keep adding bigger drives. so it'll be highly expandable down the road.
- Supports SMB, NFS, FTP, etc.
It's $600 for the unit with no drives.
Check out the toms networking review, it's linked from Infrants site.
You can hook up several hard drives (or other usb toys) via a usb hub. Performance is not great, but totally fine for storage of music and movies if you only have a few users on your network. It supports samba, ftp, nfs, http, probably any other way you'd like to access the files. You could do software raid or some other type of mirroring/backup if you'd like.
The main reasons I really like this thing for an at home server:
I was amazed at how quiet my office became after replacing my PC file server with this guy and PC firewall with a wrt54g. I could actually hear the gf talking again, which is the only downside so far.
-Lod
100 gmail accounts, GmailFS installed on your PCs, and highspeed innuh-net! ;-)
LinkSys NSLU2. Plugs into your home network. (10/100) Then you get yourself 2 IDE drives and 2 USB 2.0 enclosures then plug them in. Then you can set it to periodically back-up one drive to the other. Sure, it's not as bullet-proof as RAID5. But it's dead simple, cheap, and it just works. Failure recovery is dead simple. Also, the system is has some of the same flexibility as the Buffalo Teraserver. (Plug in your friend's USB 2.0 drive when he comes over.)
Also, with this scheme, you can delete a file and change your mind. (Recover from the back-up before the weekly copy job.)
And, if this is too simple for your geek quotient, it's Linux-based and hackable!
Mod parent up.
I can't believe anyone would recommend anything else for a geek besides the NSLU2!!
It runs based on Linux, so you can replace the firmware
Not only do you have a NAS device, which you can mirror disks on, but then you can basically add on whatever you want, eg Firewall, web/mail/file server, music center, VOIP PBX, use NFS as well as Samba etc.
Tom's Networking has a little howto on this.
And if you're interested in more information, CmdrTaco I've found this other site where you can often find some good information from users about techy related stuff that matters.
So does Anonymous Coward have good karma?
So I have an NSLU2 at home. Had it for about a year. The length of time the thing has been actually useful is maybe two days. Let me give you the counterpoint...
- Silent operation, no fans in the nslu2 and you can get fanless enclosures for the HDs
Make sure it's an aluminum case at least. And be prepared to try several different ones until you find one that works well.
- Takes very little space away from your home office
No, other than the six thousand cords you've got hanging off the back of it to plug in these external drives.
Oh, and don't accidentally disconnect a cord. The NSLU2 doesn't support anything approaching to Plug and Play. You'll likely damage data on the drive, but the most annoying thing is you gotta shutdown and restart the whole thing.
- Very small power draw
True.
- Easy to add/remove drives without any reboots
Not in my experience.
- Can power off drives that aren't used frequently, then turn them on when needed
Again, not in my experience. This is most likely going to lock up the whole thing so it stops responding.
The other problems with the NSLU2 besides the speed(might as well hook it up to a 10baseT hub, cause it can't fully utilize 100baseT), is that if you do try to transfer a large amount of data(say 15 gigs of MP3s) more likely than not the whole thing will lock up on you.
In short... The NSLU2 is unreliable, for a variety of reasons mostly having to do with software, but also having to do with the external drives and the lack of support for hot plugging USB devices. The NSLU2 is slow. The NSLU2 is a pain to manage on the table because of all the cords hanging out of the thing. The NSLU2 is not well supported by Linksys, they periodically release firmware updates but 9 times out of 10 they don't help. The NSLU2 is particular about what type of USB enclosure you use, as well as even what drive, so it's hit or miss whether it will work.
To be fair, I did look at buying a Netgear SC101, and everything I have read indicates that it's even worse.
I ended up just taking my drives and sticking them on my computer and leaving them there. I thought it would have been nice to have this running all by it's lonesome in another room with some batch scripts periodically replicating data over to it. But it's simply not reliable enough.
I've been meaning to try to sell my NSLU2 on ebay. Maybe someone who wants to install their own copy of the nslu2 Linux on it can have some fun. But it's not a good device for a SOHO server, that's for certain.
The ratings and reviews on their homepage http://www.infrant.com/ say it all. This thing blows a Terastation away in terms of ease of use, supported protocols, and goodies. Buy an empty ReadyNAS X6 from http://www.eaegis.com/ for $579 (no tax, free shipping). Fill it with two of whatever drive is dirt cheap this week (cough-newegg-cough). Here's the kicker...ReadyNAS will expand the drive array automatically each time you add a drive. So buy a couple 300GB's for $100 each and you'll have 300GB of mirrored storage. A few months from now, you run out of room, you just drop in another 300GB drive and now you've got 600GB of redundant storage. Add another drive and you'll have 900GB with redundancy. Still need more room? Replace those 300GB drives one at a time with higher capacity drives and watch it automatically resize the set to use the extra space. Without ever having to rebuld the array! Trying to backup a TB of data so you can move your NAS from 300GB drives to something higher really sucks the big one.
Of course it does CIFS(SMB). But it is one of the only NAS products to support Apple File Protocol, which is a must for networks with Mac/OS X users that insist on using filenames with colons, slashes and question marks and other things that make CIFS/SMB explode. It also supports NFS and rsync for the UNIX/Linux crowd and both FTP and HTTP for the web browser crowd (hi, grandma). It also streams in both flavors of home media server protocols (UPnP and the HMS) so you can buy a $100 Linksys media extender and watch anything you have stored on your RAID. It also has a SlimServer plugin for streaming music to those SlimServer devices that you can hook up to your stereo or a cheap pair of speakers.
It's also supports Gigabit with Jumbo Packets (write only currently) so you can copy 200GB of HD camera footage to the NAS in a couple hours instead of a couple days. The RevB case is cable-less with just thumbscrews between you and swapping a drive. It also holds the drives vertically because who is the idiot who thinks stacking heat factories horizontally on top of each other is a good idea. Also, I can't tell you how many RAID products only lets you specify an alert SMTP server name but no authentication information, which means e-mail alerts don't get delivered (boo Promise, boo 3Ware). ReadyNAS has its own MTA so the mail gets through without a problem, and it can also let you set login/password to authenticate to your ISP's SMTP server. It looks nice, clean, and it certainly not the noisiest thing I've had in my room, although I will be happy when future firmware lets you put the drives to sleep so the case fan can be completely turned off when you aren't using it.
I spend three weeks shopping for a NAS for my network, and I'm glad I looked past everyone telling me Terastation. I've had this ReadyNAS X6 for a few weeks now and I love it. I'm already shopping for a second so I can recycle the old drives from all my other rag-tag household systems into one nice neat package.
-JoeShmoe
-- I wonder which will go down in history as the bigger failure: the War on Drugs or the War on Filesharing
Once you understand that RAID is reliability strategy and are prepared to have appropriate backup measures in place, then RAID 5 becomes an attractive option for the home network. I've recently looked at several options.
In my case, a three disk RAID 1 solution proved more appropriate than RAID 5. I value high reliability on the home system and wanted to use a rotating third disk as a backup in the event of catastrophic data loss (e.g. house burns to ground). FWIW, I also use a DAT for differentiatial backups. For many users this may be overkill -- sacrificing three disks plus fixed hardware costs to greatly reduce potential data losses -- but for priceless coding projects and digital pictures, this might be good for you as well.
For some users working with video or having large audio collections, much larger disk systems may be desired. First make sure that you have an appropriate mechanism for backing up a terabyte or three. Then, the Vanguard V5 may be an excellent solution if the $2-3k price is acceptible.
Given one hour to live, the student replied: "I'd spend it with professor FP who can make an hour seem like a lifetime."
Write performance: insigificant. He said it was for archival use, so presumably it's a lot of reading and not so much writing. Besides, any reasonable RAID should be faster than a single disk, and with just two or three drives you'll be fast approaching the upper limit of gigabit ethernet (I'm presuming that Taco's house isn't wired with infiniband, though I suppose it might be).
Multi-disk failure: Well, you can still lose your RAID-10 if two disks from the same linear array fail, so you're spending a lot of money and not really gaining a whole lot - the 33.3% figure only applies to a 4-disk RAID-10.
If you've got 4 disks and are concerned about 2 failing, go RAID-6. You get the same capacity as the RAID-10 would get you (capacity * (n-2)), and you also have a 0% chance that 2 failed disks will take the array down. To increase capacity, you just need to add one disk at a time, too (after the initial 3), as opposed to the RAID-10, where you have to add in [at least] pairs.
I don't know RAID 6 and 50 well enough to explain them, so the link will do the job. While I'm at it, they can do 10 (compare it to 0+1 to understand better)
RAID 6
RAID 10
RAID 50
-Turkey
I've been putting together the specs for such a beast. I decided to go with SATA for cheap drives and "SATA-II" (or whatever you want to call it, since there isn't a standard name for NCQ and 3.0Gbps support) for future-proofing.
1) The natural first choice was 3ware. 12 port SATA-II controller (9550SX-12), for about $800. 3ware products are very well supported on Linux. The only downside is that it's a PCI-X device (this is NOT "PCI Express"!), and PCI-X busses are generally only found on very high end motherboards for servers and workstations. Any athlon motherboard or single-processor opteron board claiming to have PCI-X is lying, they really mean PCI express (AMD chipsets did not support PCI-X at all until around the time dual opteron motherboards were being created)
So since I didn't want to spend $500 on a motherboard that had built in scsi raid, support for 16GB of ram and dual opteron processors just to use that $800 card, I looked around some more...
2) And found a serious contender, the 12 port Areca 8x PCIe ARC-1230 (also about $800). While most low end motherboards don't provide an 8x PCI Express slot, they DO provide a 16x slot which will work just fine for this card (after all, this will be the fileserver, so a motherboard with crappy built in video will do, we're not playing Doom 3 here). Linux drivers are provided as source, even including a kernel tree patch which will build the driver into the kernel rather than as a module, making booting directly from the RAID controller easy.
Slap the Areca into Tom's Hardware's 37 watt computer (motherboard has built in GigE, but pentium-Ms are 32 bit processors, making giant files/filesystems a pain. An Athlon 64+cheap mini-ATX can be had cheaper, but uses more power), add in a stack of 10 watt 400GB WD Caviar Raid Edition 2 drives, and you're set for a very low power fileserver with a lot of storage.
Now, my turn to "ask slashdot":
Where do I get a 250-300 watt powersupply with 12 SATA power connectors?
Alternatively, do the SATA drive cages (like 3ware's RDC-400-SATA (pdf) have their own SATA power connectors built in and use standard molex connectors on the outside? Do I need special cages to support 3Gbps drives (ok, not a serious problem for now, but futureproofing)? 3ware's website says it'll work, their product PDF doesn't.
If I have been able to see further than others, it is because I bought a pair of binoculars.
I have mod points, but I feel it's more important to just correct you. He already has everything backed up and the LVM idea doesn't do anything to help his situation.
He does care about downtime. Downtime = time spent restoring. With a RAID level > 0, all he has to do is replace a drive and tell the raid to rebuild. He's done in 5 minutes. It would take that long just to queue up a restore job for the tape.
You don't need 1000 CDs.
You don't need 500 DVDs.
You don't need hours and hours of shaking, badly focused home videos.
You don't need 5000 bad pictures.
(if you really do you know I am not referring to you).
Nobody is going to watch all that crap, and unless you have not got a life, you are included on that select group.
Prune your digital trash.
You will find that a moderate amount of disk space is more than enough to hold all your files.
If you want to make a datacentre of your home, go ahead and enjoy, but the lamest excuse is to house all those GBytes of data that are never going to be seen again.
IANAL but write like a drunk one.
I don't know about all you wackos with the 600$ CPU's and 600$ RAID controllers at home, but I have better things to do with my money. Like invest it rather than spend it on useless trinkets, theoretical seek-time figures and unused gigaflops.
/proc/mdstat is wrong.
Here's my brew:
1. Old PC. Any one would do, probbably even a good'ol P1. 128MB RAM is more than enough. I consider this FREE. I run a dual-PIII-450MHz that I have lying around.
2. 4x[BIG-SATA-DRIVE]. How big? When I built mine, highest bang-for-buck was 250GB. So I went with 4 of those.
3. 1x PCI SATA controller.
4. 1x PCI GbE NIC.
[3] and [4] are peanuts. [2] is worth, what, 500$?
The entire rig will easily give you ~10-25MB/sec, which is, for any home use I can consider including pumping 10GB files over the network, plain enough.
Plug any crap old 2GB or greater IDE harddrive in for sport (or two and do yourself a RAID1 configuration).
Install Linux.
Install SAMBA.
Configure RAID.
Set up healthchecks that email you if something in
[OPTIONAL]
1. Grab several old IDE drives. Not neccesarily same sizes.
2. Stick them in some other box (I did it on my windoze box cuz that's where I had case space).
3. Configure a RAID0, or better yet, a spanned volume. Use windoze dynamic disks, use LVM, whatever makes your boat float. Set up a compressed filesystem if you think that would help any. Usually, with the kind of things people store on huge arrays at home, it won't.
4. Do a daily dump of everything from your RAID to your backup array.
DONE. Forget about it and go do something better with your time.
-