Slashdot Mirror
Experiences w/ Software RAID 5 Under Linux?
MagnusDredd asks: "I am trying to build a large home drive array on the cheap. I have 8 Maxtor 250G Hard Drives that I got at Fry's Electronics for $120 apiece. I have an old 500Mhz machine that I can re-purpose to sit in the corner and serve files. I plan on running Slackware on the machine, there will be no X11, or much other than SMB, NFS, etc. I have worked with hardware arrays, but have no experience with software RAIDs. Since I am about to trust a bunch of files to this array (not only mine but I'm storing files for friends as well), I am concerned with reliability. How stable is the current RAID 5 support in Linux? How hard is it to rebuild an array? How well does the hot spare work? Will it rebuild using the spare automatically if it detects a drive has failed?"
The idea is that in order to write data to any sector on one of the drives, the sectors from six of the other drives need to be read, all XOR'd together, and then the result written to the remaining drive.
Your logic eludes me. The blocks do not need to be read, as we are in the process of writing. We already have the data, because we are writing, so why would we re-read the data?
Furthermore, block sizes default to 4k, though you could go to 8k or 32k block size. At any rate, you don't need a gig of RAM to handle this.
Finally, XOR is not that expensive of an operation, and a 500Mhz CPU is going to be able to handle that faster that any but the most expensive controller cards.
So unless you are actually a RAID kernel developer, I don't buy your story.
I just posted in another thread about 3ware and mysterious drops of seemingly good drives. Even with the ultra-paranoid drive dropping, we have never lost data on 3ware.
Other than that, 3ware has been decent for us. We are about to put into service a new 9500 series 12 port SATA card.
I wish I could say our ACNC SATA to SCSI RAIDs have been as reliable. We have three ACNC units, two of them went weird after we did a firmware upgrade that tech support told us to do, lost the array.
We call tech support and they say "oh we didn't remember to tell you when you upgrade from the version you are on, you will lose your arrays".
I've had enough abrasive sigs. Kittens are cute and fuzzy.
Normally a drive crash anonunces itself some time before... use the smartctl tool.
/dev/hda every week and a -t long every month or so ...
l es/MAXT OR-10.txtm ples/MAXT OR-0.txt
that tool checks the SMART info on the disk about posible failures..
I do a lot of software raids and with smartctl, no drive crash has ever surprised me. i always had the time to get a spare disc and replace it on the array before something unfunny happened.
do a smartctl -t short
read the online page of it:
http://smartmontools.sourceforge.net/
A example of a failing disc:
http://smartmontools.sourceforge.net/examp
a example of the same type of disc but with no errors:
http://smartmontools.sourceforge.net/exa
Software raid works perfect on linux... and combined with LVM the things gets even better
Putting a windows cd backwards, plays evil messages, but it gets worse, putting it right, installs windows.
Your logic eludes me. The blocks do not need to be read, as we are in the process of writing. We already have the data, because we are writing, so why would we re-read the data?
That would depend on the nature of the write. If you're writing the initial data it's unlikely that you'll require reading. However when you go to update the date you may have to perform reads in order to calculate the parity required for the update.
Software RAID 5 is very reliable but does suffer a performance hit. Not because of the XOR computations like many here are suggesting. It's because each logical write needs to be translated into physical reads/writes...which consume time.
The beauty of software RAID, at least software RAID implementations such as Veritas, is that it allows you to spread the RAID across a number of controllers.
Listen to this guy...he knows more than the others who consider the XOR computation the slow link in software RAID 5. It's not.
Software raid is fine for simple configurations, but if you want to "do it right" - especially considering that you just dropped about a kilobuck on HDDs, go Hardware. A good, reasonably priced true hardware RAID controller that will fit the bill for you is the 3Ware Escalade 7506-8. It has 8 IDE ports, 1 for each drive - you don't want to run two RAID drives in master/slave mode off of a single IDE port; it will play hell with your I/O performance. It's true hardware raid, so you don't have to worry about big CPU overhead and being able to boot with a failed drive (a major disadvantage to software RAID if your boot partition is on a RAID volume, certain RAID-1 configurations excepted). You can buy them for under $450. provantage.com price [provantage.com] is $423.48 (I have no relationship with them other than I've noticed that their prices tend to be decent).
Hardware RAID5 is fine if your sole goal is reliability. If you need even an iota of performance, then go with software RAID5. The 3wares have especially abysmal RAID5 performance, specially older series like the 75xx and 85xx cards. 3ware's admitted it, and something targeted for fixing in the 95xx series (haven't gotten my hands on those yet, so I don't know).
As for software RAID reliability, I find that Linux's software RAID is much more forgiving than even the most resilient of hardware RAIDs. I've lost 4 drives out of a 12 drive system at the same time, and Linux has let me piece the RAID back together and I've lost nothing. Was the machine down? Yes. Did I lose data? No. Compare that with a 3ware hardware RAID system where I lost 2 drives. Even thought I probably could have salvaged 99% of the data off that array, the 3ware just would not let me work with that failed array.
Also, on any reasonably modern system, the software RAID will be faster. You just have a much faster processor to do the RAID processing for you. The added overhead of the RAID5 processing is nothing compared to a 1-2GHz processor.
> well i was talkign about more or less cheap controllers. in the arena of raid controllers 160USD is fucking cheap!
/boot / /tmp and /var /home
/, I did the same thing (pair of RAID 1s) and put a RAID 0 on top of that, for a net of 40gb redundant and fairly speedy storage.
/tmp and /var I made a simple 10gb RAID 0 for each. Not a whole lot of need for redundancy here, I make a point of backing up the important /var stuff.
/home
Not compared to $0.
You see, the typical budget RAID 5 builder just wants to store his collection of MPEG4s, MP3s, and other downloads or perhaps uncompressed hobbyist video. It's not a database, it's not a 150+ employee corporate file server, it's just personal. Performance is not a concern.
And if performance is a concern (say he wants / on these disks) then the cheap way to go is software RAID 0, 1 or 1+0 (aka 10) *COMBINED* with a RAID5.
For instance, I just built myself a new system with four 300gb drives and partitioned each one like so:
50mb -
1gb - swap
20gb
5gb -
For the 50mb, I made a bootable RAID 1 of four drives (grub can boot this, dunno about lilo)
For the 1gb swap, I made a RAID 1 with two drives and a RAID 1 with the other 2. Thus I have a net of two 1gb swap partitions, with redundancy so my system will never crash due to drive-induced paging errors. This is essentially a RAID 0+1, though I let the kernel's swap system handle the RAID 0 aspect by giving them equal priorities.
For the 20gb
For the 5gb
With the four equal-sized partitions that were left, I made the RAID 5 for
Don't you see what a great cost-effective approach this is?!?
Maybe you work for some company with plenty of money lying around for $160 RAID controllers. But I'm in business for myself, and I don't see the sense in spending money where it isn't needed. Besides, the flexibility of software RAIDs (per-partition, not per-drive) would be well worth it to me even if something like the SX4 were cheaper.
The unofficial
This happened to me. The card was sorta still working... could read, with lots of errors usually recoverable, but writing was flakey.
Luckily, even after about 3 years, 3ware (now AAMC) was willing to send me a free replacement card. They answered the phone quickly (no long wait on hold), they guy I talked with knew the products well, and he had me email some log files. He looked at them for about a minute, asked some questions about the cables I was using, and then gave me an RMA number.
The new card came, and my heart sank when I saw it was a newer model. But I plugged the old drives in, and it automatically recognized their format and everything worked as it should.
This might not work on those cheapo cards like Promise that really are just multiple IDE controllers and a bios that does all the raid in software. Yeah, I know they're cheaper, but the 3ware cards really are very good and worth the money if you can afford them.
PJRC: Electronic Projects, 8051 Microcontroller Tools
Vinum on FreeBSD absolutely rocks! You're old 500MHz machine will run FreeBSD beautifully too.
Anybody here remember Walnut Creek's huge ftp archive at "cdrom.com" which back in it's heyday of the late 1990's used to be the biggest, most highest traffic ftp download site on the planet? They used a combination of Vinum software raid and Mylex hardware raid to handle the load. I remember reading a discussion article from them once that until you get a totally ridiculous volume of ftp sessions hammering away at ther arrays, that Vinum was actually a slight bit faster than the hardware array controller.