Slashdot Mirror
RAID Controller Shoot-Out
mikemuch writes "ExtremeTech has a comparison with benchmarks of three RAID controllers from Adaptec, LSI Logic, and Promise, and along the way gives you a little refresher course on RAID in general and why you want to use it: faster throughput, longer uptime, and improved data security. Motherboard RAID controllers do well when there's 'very little or no load on the CPU, I/O bus, and memory bandwidth. But with heavy traffic and processor loads, the limitations of the shared bus and the benefits of intelligent RAID's integrated IOP and memory cache have a more significant impact.'"
Print version here
Annoying "next page" articles...
According to the OpenBSD i386 supported hardware website, out of the cards reviewed, only Adaptec and LSI cards are compatible with OpenBSD.
However, Adaptec has refused to provide documentation so that the OpenBSD project may improve the drivers.
"Note: In the past year Adaptec has lied to us repeatedly about forthcoming documentation so that RAID support for these (rather buggy) raid controllers could be stabilized, improved, and managed. As a result, we do not recommend the Adaptec cards for use."
Other *nix variants might support the Adaptec and Promise cards a little more, but the hardware fully supported by OpenBSD is generally well-supported across all *nix variants.
Out of the cards reviewed, only the one from LSI is worth buying. Adaptec may have a little support, but it's not a good idea to purchase any RAID cards from them until they start providing better documentation.
LSI has a much newer (and presumably) faster SATA/300 card now. If you were buying a new RAID card, that's what you would buy. Why did they use the old generation?
Where's Areca? They're the performance leader in this market, and their pricing is now in line with competition.
Where's 3Ware?
What about other host-based RAID solutions? Broadcom? Marvell?
Don't even get me started about what they tested. This is just not a serious RAID review. I strongly urge folks who are interested in this subject to Google for better.
3Ware has been a mixed bag for us. The 7xxx/8xxx series are running in a number of our production servers (the little kind - the kind slated to become VMWare images in a year or three). They weren't exactly top performers, but they have been very reliable and easy to manage, good with notification of problems, and a breeze to rebuild. We had a bad run in one time with one of the original 9000 series models dropping zeros to the disk and causing us some bad database corruption (and it was a pain in the ass to track down). We're using a pair of the 9550-SATA models for two of our backup servers and those new models are top-notch and fast as hell. I think they've matured. We use them on linux and windows - they always perform far better in linux no matter how much tuning we do.
;)
I'd still like to sink my teeth into an Areca at some point for comparison, though.
Hell is being intelligent in a world full of idiots.
Anyone care to shed some light on how rebuilding arrays compares when using intelligent vs host-based controllers?
Sure.
All the BIOS RAID interfaces I've seen (mostly MegaRaid and Adaptec) suck hard. About as friendly as a cobra, and slightly more dangerous if you do the wrong thing.
Software RAID interfaces can do better - But few actually do.
However, I wouldn't suggest choosing one or the other based on the friendlyness of rebuilding - Whichever you choose, when you eventually need to replace a drive in the array, you go step-by-painful-step following the manual, and calling tech support if you see any ambiguity at all. You don't trust Clippy to tell you "So it looks like you need to rebuild your RAID..."
I personally choose BIOS managed over software managed for one simple reason - You don't need to screw with drivers, and while the host OS can still wipe out your data, it at least has to do so at the filesystem level rather than by breaking the underlying RAID.
I've got 2 250GB WD SATA drives RAID-1'd with an LSI MegaRAID SATA150-2 controller. It's an add-in controller, but I've got no idea how "intelligent" it is. I just had to rebuild the array (unplugged one drive for several days without knowing it) and it took 70 HOURS. From the BIOS interface for the card. Masochism indeed.
Ok, I realize it's a bad metaphor because you actually can throw a motherboard quite a distance. But here's another example of where things can go horribly wrong: How do they handle error conditions? On my desktop system, I'm running RAID-0 (with WD Raptor drives) for speed. Yes, I know what I'm doing (famous last words). No, I don't store any important data on my desktop (it's on a RAID-5 array on a server). Originally, I was using the Silicon Image 3114R on-board RAID controller included on my Asus A8N-SLI "Premium" motherboard. Eventually one of the drives died. The SI3114R responded to the problem by freezing and becoming unresponsive when a disk error occurred. Under DOS, Linux, or WinXP - the problem is not OS specific. The rest of the system works fine, but once it hits an error the SI3114R just stops working and returns nothing but errors to the OS. Now, since Asus doesn't update the SI3114R BIOS in their mobo BIOS updates (and I'm too lazy to hack my own), I don't know whether it's bad silicon, bad BIOS, or a bad design (my guess would be the latter). Accessing the drive's S.M.A.R.T. data indicates that the warning numbers were screamingly bad and probably were for some time.
So apparently the SI3114R doesn't monitor S.M.A.R.T. data, and it's error-handling capabilities fall somewhere between "shitty" and "non-existant". No big deal for me; I was only inconvenienced by having to re-install operating systems and applications.
The moral of this long-winded story is that you generally get what you pay for. This isn't the first bad experience I've had with on-board RAID controllers. If your data is important, then spend the appropriate money (think in terms of data replacement cost), do the appropriate research, and invest in a RAID setup that's right for your situation. If your protected data consists of anything more important than your Oblivion saved games, your mobo's RAID controller (or the $39 Fry's special) is probably the wrong choice.
And if anyone cares to know, I'm now using the NVRAID on the mobo (we'll eventually see if it handles failures more gracefully), and I use an Areca ARC-1110 on my server. I can attest that the Areca card does handle failures extremely well, albeit noisily.
Help save the critically endangered Blue Iguana
I spent about $8000 for a complete rig, including fifteen 500GB disks and a couple of cat6 crossover cables for the links. I'm not currently doing multi-host simultaneous access so I don't need a fancy file system, it's just a regular block device to the OS.
I'm using it to back up a couple of terabytes nightly with rsync --link-dest. (See Mike Rubel's site if you're not familiar with that trick).
Performance feels about the same as the $200,000 (US dollars) fiberchannel SAN array sitting next to it, but I haven't actually measured.
I finally broke down and bought an Areca card for one of my home-office servers (I had read some nice reviews and wanted to test one myself before recommending it). Seems reliable (at least from my single, lonely sample point) - it handled a drive failure perfectly (that is, it caught ugly S.M.A.R.T. statistics and notified us before the drive actually failed completely) - and it's very fast. Their Linux driver is BLOB-Free, well-commented and 100% GPL. Prices are reasonable, but it'd be nice if they were available through mainstream distribution (Ingram, TechData, etc) - not yet, apparently.
Help save the critically endangered Blue Iguana
...and a proprietary striping format, so when you have controller problems you have to use the same vendor's software & hardware to recover your data.
Malike Bamiyi wanted my assistance.
I don't know why the meme that software (or pseudo-hardware) RAID5 "sucks up" CPU cycles continues to propogate.
A 300Mhz Pentium 2 has a RAID5 checksumming speed (in Linux) of about 800M/s. So at the more realistic speeds of 50 - 75M/s your average PC's RAID5 array can write at before the bus and physical drives hit their limits and on any remotely modern CPU, the processing overhead of checksumming is insignificant.
There *are* valid reasons why a hardwar RAID controller should be faster[0] - particularly for large arrays and/or the typical, bus-starved PC, but they have nothing to do with the "CPU overheads" of RAID5, nor even the "intelligent caching" some cards boast about.
[0]And even though they *should* be faster, they frequently aren't.
This article is pretty good, though a few months old now. I bought an Areca ARC-1120 based on this review, and have been very happy with it. 100% GPLed driver. Wish they were a little easier to find in online stores though.
I'll second the advice on Adaptec. I have the ATA Raid 5 cards. The software they provide for Windows is ok. The BIOS software is ok. The Linux software? 2.4 kernel software was a nightmare for anything other than Red Hat, SCO, and possibly one more distro. Everything else was a fucking nightmare of patching the kernel and rebuilding, then hoping it worked. Debian, it didn't. As for 2.6, forget it. No support for 2.6 yet, don't think there's any planned unless you are buying a current SATA card. The ATA cards with 2.6 kernel? There's no way of monitoring the array while the system is up. The only way to check the status of the array is to shutdown and boot up again, then get into the RAID BIOS. Last time I took one system down just to check the array, I found one dead drive, with the hot spare having kicked in to replace the dead drive. Since its a production system, its now running raid 5 without a hot spare. With the luck we've had on drives, between deathstars, bad batches from more than one manufacturer, etc., the production system with the missing hot spare and no way to monitor it is keeping me tense until I can schedule some downtime to replace the hot spare. The way it is now, I lose two drives over 100-150 days (has happened before), then I have a dead production system until it can be restored from backups.
I saw the situation with OpenBSD recommending against Adaptec, and I ignored it, crossing my fingers. Big mistake. OpenBSD is correct. Only stick with companies and hardware which OpenBSD and others in the community can get documentation on. Documentation is what OpenBSD is looking for, so they can write their own drivers. They aren't looking for the source code to the firmware, just the documentation to the registers and other internals, so no company is giving away trade secrets, precious ip, or competitive info.
Raid cards are used for a very specific reason. The safety of your data. Reliability. Especially when you consider how much the RAID hardware cards cost. Listen to OpenBSD. Stay far away from Adaptec. Far, far away from Adaptec. They are flipping the community the bird. And I don't mean penguin.