Slashdot Mirror
Creating Large, Safe, and Cheap Network Attached Servers?
davco9200 asks: "I am looking to create a large data server for all my digital media files. The usage is the 'pro-user' category, to use the media from multiple stations in my house and at work. I value space (150+ gb would be nice), accessible from multiple platforms (Win, Mac), but perhaps most of all, some security (e.g. RAID 0 doesn't cut it). Total write or read access isn't that high of a priority. I have looked at things like the Snap 4100 that offer 160 gb or 300 gb and good raid options but the price seems high ($2,700 and $4,500 respectively). Has anyone had any experience making their own low-end NAS? Has anyone looked at the Adaptec IDE RAID Controller? This seems like a reasonable way of getting data parity so if one drive goes down your entire collection isn't lost. I figured Slashdot readers would have some good solutions. Information on specific cases, drives, and other pertinent facts would be helpful."
Well, if you don't care too much about performance, don't even bother with a 'real' IDE RAID card, just do software RAID. The reason why people use the hardware RAID cards like the higher end IDE RAID cards and the SCSI RAID cards is because it's faster.
So you can cut the cost of a good IDE raid card and just put an extra IDE controller card in your psuedo-NAS box so you can have 8 drives. Or you can put that off until later.
You should also consider getting a DAT drive and a bunch of DAT tapes, to back things up, just in case something massively bad happens to your system. RAID 5 is not perfect, and if your system catches on fire because of too much dust in the power supply, it'll be helpful.
Gentoo Sucks
I am working on a similar project but in the >1TB range same thing applys to http://staff.sdsc.edu/its/terafile/ On that page they have a link to another page with stuff about ide-raid. >http://www.research.att.com/~gjm/linux/ide-rai d.html The 3ware cards are the way to go as they do raid 0,1,5 in hardware, and support things like hot swap and hot spare. I priced out a system that was just over 1TB of raid 5 for around $5,000 while the prebuilt stuff is $20,000.
One day people will learn the folly of Winbloze, Linux Rules!
One great place to look for an old tower is your local Computer Goodwill or used bulletin board. Get an old box with a 300W power supply, take out the motherboard, start stacking SCSI drives in a daisy chain, and then bring it back to a SCSI controller on your main box.
Obviously SCSI is more expensive than IDE, but you get a little bit more. Just food for thought.
Troll Like a Champion Today
Tom's Hardware has an article on their main page about software RAID under Windows 2000. Apparently it supports striping and spanning, and unlimited drives as long as you have enough controllers for them. The boot HDD has to be standalone but the others can be RAID-linked. The best thing is that it supports multiple interfaces. You can have 3 IDE drives and a SCSI drive and they'll all RAID together.
I don't know if an MS product is what you'd want to use, but it's out there.
J.W. Koebel
We use them in our servers and they support 50+ users without problems.
---- Put Sig here:
I would absolutely recommend the 3Ware Escalade IDE RAID cards. I'm using a 6200 right now with a couple 75GXP's striped, and can pull 50MB/s easily. And for less than $120, it was a great value!
redundant raid offers availability, but it doesn't replace a backup system. If you're worried about losing your precious collection of MP3s you should archive your collection on cdrom, or tape. If you can't afford to have any downtime to restore your data, then you should use a redundant raid scheme in addition to maintaining a backup/archiving system.
I don't know what the failure mechanisms are... but it sure is appealing to just buy two of the biggest IDE drives you can afford. Fill up one, copy all the data to the other, and then just turn the power off to the backup drive.
Last I checked, Fry's was advertising big disks at under $2 per Gigabyte. That's cheap backup.
I've build a box just for that requirement a few months ago using a 3ware IDE controller, 4x100gb drives in raid 5 mode and linux :) gives a nice 300 gb of space.
..
..)
I think the overall costs were around 2000 Euro
(controller, drive bays, BIG case, etc
I'm wondering if you're not letting buzzwords get in your way?
KISS
For home use, right? Just buy (a bunch of) gigundo SCSI drives, and cram them into an Intel system running your choise of free/open source OS. I'm fond of the *BSD family, but YMMV.
Run samba and you'll see the files as if they were on drives native to Windows/Mac.
Or am I missing something?
Display some adaptability.
One way to go might be an inexpensive, but not underpowered PC, with a PCI Firewir-- er, IEEE-1394 card.
Buy a bunch of cheap, identical IDE HDs, and put them in IEEE-1394 cases (~$150/ea.). Compile yourself some bleeding-edge Linux-1394 support, plug in your HDs, run XFS as the filesystem, and use software RAID. Because you said this is just for storage and media access, you probably don't need the currently limited FireWire hot-plug support and possibly still currently limited RAID hot-swap support.
For more on software RAID, IBM has a nice two-part article (1, 2) on it.
This is my receipe for an "homebrew" Snap4100
1) Get:
- 1U 4bays rack mountable chassis from Sliger Designs
- 3WARE 6410 Escalade IDE controller (Choice of 0/1/0+1/5 Raid) on a 90 PCI riser card
- 4 x 75/100GB ATA100 drives (maybe DiamondMax)
- MicroATX mainboard with NIC and Video integrated on board (invest in RAM not in processing power - 750/850MHZ should be more than sufficient)
- Minimum Linux/*BSD OS booting from a read-only 16 to 64MB flash IDE device, loading kernel and a customised Ramdisk root filesystem, mounting Raid devices in R/W mode, starting SAMBA (and/or Netatalk).
A good starting point is Linux Bootdisk HOWTO
2) Choose 0+1 Raid and you get quick and completely redundant 150/200GB storage that can survive the full failure of one disk.
3) Want remote grafical managment from a standard web browser? Go for Webmin or SWAT.
Don't be too cheap here. Power supplies do go out (or catch fire because of dust or pet hair that blew into them) and even a "good" unit might have problem maintaining its rated capacity after a number of years.
Keep the chassis, but replace the power supply. Besides, since this is a server this sounds like a good place to use one of those units with a built-in UPS, even if box is hooked up to an external UPS. (I've had them fail because I unwittingly overloaded them, because of poor designs that allowed me to accidently turn off the UPS protection but not the power, etc.)
For every complex problem there is an answer that is clear, simple, and wrong. -- H L Mencken
Yeah, probably about half the people on Slashdot. ;-)
There's really nothing to it. Get a x86 PeeCee, 4 cheapo 80GB ATA drives, and Linux or xBSD. Put a bootable+usablesystem partition on each drive (since you never know which drive is going to fail first) and use the rest of each drive as a slice for a RAID5. You'll have a about 220-240 Gig of storage that can survive media failure. You can do it for somewhere in the $1500-$2000 range.
For larger non-media failures, you're still screwed, though. Backup technology just hasn't kept up. :( Unless you have Big Money to spend on this (and maybe even then), you will end up using many tapes for a single backup. That fact will tend to influence you toward not backing up at all, unless you are unusually disciplined.
For the exporting to the other machines, the stuff you need should come with just about any Linux distro. I use NFS (for Unix and Amiga) and Appletalk (for Mac). I suppose Samba would work for wintel boxes.
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
I'm not too sure but, I know you want to try to keep the mirros and the data on 2 separate controllers
I believe this would be an optimal solution get 2 cards
RC = Raid Controller
4 drives
RC1-----RC2
D1------M1
+--------+
M2------D2
8 drives
RC1-----RC2
D1------M1
D2------M2
+--------+
M3------D3
M4------D4
You need to have the mirrors on different cards
you will get better read/write performance
data for a drive on 1 controller and the mirror on the other, and keep the mirrors and data split between the controllers Ide or scsi, I'm not sure what functions are needed on the card or even if the 3ware cards will support it. I'm not sure how to configure this on the SW side either. But for my new DB server I will configure it this way.
You can lose 50% of the drives(if they are the right 1's it not the data and the mirror)
with raid 5 you can loose 1 drive, then the spare(if installed) must recreate the data before you can lose another drive.
-- Tim
TKrabec Pahh
People also choose hardware RAID for greater reliability, OS independence, OS simplicity, and in some cases hotswap-ability of drives.
I have built several RAID5 servers for clients. The largest used 10 80-GB drives (8 data, 1 parity, 1 hot spare). I used a dual P3 motherboard with a decent ATA controllers. I also used two Promise Ultra66 controllers. I booted Linux from a 20 GB /dev/hda. I place two disks on the secondary motherboard controller and two on each of the four Promise channels. Software RAID5 and an oversized UPS. The Promise controllers are a little flaky if you don't make sure that they have their own interrupt, but performance has been fairly good. Rates are typically 40 to 50 MB/s. Also, the Intel P4 motherboard seems to do a better job than P3s, so the new RAID boxes are going to be P4 instead.
I have not been at all that impressed with IDE hardware RAID solutions in the past. One box we bought from Arena had internal IDE drives and a SCSI interface on the back. Performance from it is horrible!