What Kind Of Software RAID Are You Running?

ErikZ asks: "Lately, I'm having issues with my RAID. Specifically, closed source drivers for my RAID card that only support Red Hat 9. So I've decided to Ebay the card, and try to figure out how to turn 4 SATA drives into a software driven RAID 5 setup. Yes, I know I'll lose all the data, and I'm not worried about it. Finding a 4 port (or more) SATA controller card, that's well supported under Linux, has been difficult. Everyone wants to slap on their own RAID chip and charge you another 100$ for the pleasure. Where can a guy get a highly recommended, well supported, 4 port SATA card for Linux? The Rocket 1540 cards have vanished off the face of the earth. There are a few motherboards out there that have 4+ SATA connectors on them, but they also add RAID and some other cutting edge features that aren't well supported under Linux. So, I thought I'd try another route and ask Slashdot: What are you using for your Linux software RAID needs? What do you suggest?"

Comments

I'm using md, aka Linux Software Raid

[eakerin]

Why not just let linux handle the raiding of the drives? No special hardware needed ourside of the drive controllers you already need to hook the drives up.

For more information check: man md

Also RAID 5 is distributed parity raid, no data loss if only one drive goes. it takes two failures to lose data on a raid 5 array.

Re:I'm using md, aka Linux Software Raid

[TheWanderingHermit]

If it's not included, the full package is mdadm. There are a number of tutorials on the web for it. It's easy to set up, and easy to run (just ignore it). I don't remember if it can work with hot swapping (I don't need that yet), but I'm using it on several systems. I set it up, and I haven't had to worry about it since.

When I first got it, I stuffed a lot on the raid drive, disabled it, wiped out one disk, and re-activated the raid. It rebuilt it and worked fine.

I asked this question on a Debian (or Debian based) user list at a time when a lot of experienced admins were around, and overall the feeling was that there was no need to go hardware and the software raid would do the job.

Re:I'm using md, aka Linux Software Raid

[gregmac]

Why not just let linux handle the raiding of the drives? No special hardware needed ourside of the drive controllers you already need to hook the drives up.

Another point to this: there are three kinds of raid setups:

• hardware raid - where the OS speaks RAID-specific commands to the controller
• firmware raid - where software RAID is implemented in the firmware of the card
• software raid - RAID done by the OS

From my reading on forums and other various articles, there's almost no (if any) benefit to using firmware-based raid over true software raid. In fact, it's possibly worse, as you're relying on the (possibly closed) software in the firmware of the card, as opposed to the well-tested and open implementation of mdadm (or whatever raid implementation you use). Chances are you'll also spend some time struggling with the driver and getting your kernel to recoginize it.

If you're going to go hardware raid, go for real hardware raid. Otherwise, I'd suggest sticking to mdadm and doing it in the OS. You get nice email alerts if there's a problem, and all the software will work so long as linux sees your IDE and SATA controllers (probably). You can even mix and match RAID across your IDE/SATA/SCSI/USB/1394 drives.

Re:I'm using md, aka Linux Software Raid

[kzadot]

Unless you want to boot, in which case you will need a seperate boot drive because the BIOS cant load the kernel off a software raid.

Which defeats the purpose, as the performance gains from RAID are going to be greater on the system drive (swap space, loading programs, libraries, program resources etc) than on the data drive, which is typically multimedia data where performance isnt a factor, as when you save it, your download speed is the bottleneck, and when you play it, the multimedia files have set bitrate.

I just dont get software raid.. Its the system drive that has the most to gain from RAID yet software raid doesnt work on system drives. Who gives a shit if your mp3 drive is faster it only has to deliver a couple of hundred kbps.

Re:I'm using md, aka Linux Software Raid

[kzadot]

That all applies to the average home user of course, I could see roles for people doing video editing and compressing, dvd authoring etc, they could do with a speed improvement on their data drive where software raid would do, but if your a video editing pro then you will want the very best anyway... Once again hardware raid.

Re:I'm using md, aka Linux Software Raid

[TheWanderingHermit]

Those are good points, and ones I had not considered, since all the data on my raid is text or tables for MySQL. So far I haven't seen any indication of a bottleneck at all. The boot drive is separate (and both RAID and boot are backed up and stored on a separate system which will be offsite one day).

I also am confused -- you say data drives are mostly multimedia, and point out that they have set bitrates for reading, then get vulgar about someone getting good performance for mp3 players. It seems you almost contradict yourself there.

Most of the people I know storing data on a raid drive are storing business data -- such as data files, documents, etc. The only case I know of where someone is storing multimedia on raid is where I had to temporarily store my ogg files on my raid while removing one old system from my LAN and preparing the new one.

Re:I'm using md, aka Linux Software Raid

[TheWanderingHermit]

Anyone doing serious video editing or DVD authoring will not be using Linux. While Cinelerra offers a good many professional features, overall video tools on Linux are not developed enough for professional use. I keep checking on them every few months, in the hope that when my business is at the point of moving toward video production, the programs on Linux will be able to support professional video work. While there are programs that let you do DVD authoring, they're not yet at the point where one can focus more on the authoring work than on how to use the tools.

That means if you're going to do professional video, you're likely to use a Mac (or possibly a Win32 based system). Networked drives aren't fast enough (although gigabyte ethernet speeds may be fast enough, I've never tried them), so you're best off storing your files on the system you do your capture and editing on. If I'm wrong on this, then things have changed, and I'd be glad to hear about those changes.

Re:I'm using md, aka Linux Software Raid

[kzadot]

I dont know about that, I am not a professional myself, but I have done it as a hobby, and mroe importantly I have set up such stuff for professionals who were dissolusioned with the mac and win32 solutions. I havent heard of cineralla, I just set the guy up with some home made guis over the top of software like dvgrab and transcode..He is totally happy with it and many other such professionals in the town want such a simialr system because the mac and windows solutions are so inferior.

Re:I'm using md, aka Linux Software Raid

[kzadot]

Sorry, I just inferred that the typical home user will be using his large data drives for audio and video files. Very broad generalization I know... And I didnt mean these drives have fixed rates but the files themselves, a 128kbps mp3 is only going to require so much performance from a drive.

Re:I'm using md, aka Linux Software Raid

[cowbutt]

Actually, the RAID cards you refer to as 'firmware RAID' perform most of the RAID operations in the driver of the OS (though some, at least, provide enough BIOS support in order to get the OS booted). Promise and Highpoint's cheaper cards work like this, and Linux's ide-raid drivers used to support some of these chipsets (along with the manufacturer's own drivers).

IMHO, the only reasons to use these as anything other than bog-standard ATA controllers are a) if you have a pre-existing RAID setup that you wish to continue to use b) to migrate from or c) because your OS doesn't provide its own software RAID (XP Home, Windows 9x). As bog-standard ATA controllers, they're as good as any other (I use a Promise MBUltra 133 with two 200G Seagate discs and Linux md myself).

Re:I'm using md, aka Linux Software Raid

[TheWanderingHermit]

It'd be interesting to find out what they're using for editing. You can do capture and playback on Linux, but the only package I've found that I would even consider using for editing is Main Actor, by Main Concept. I think it's either $200 or $300. For the price, when compared to Adobe Premeire, it's pretty good, but last time I tried the demo, I found it lacking in features I was used to using regularly. I've been watching open source editors, and I think it'll be several years before there's one with all the features of your standard professional analog switcher.

I looked at Cinelerra, but I couldn't stand it because of the GUI. It felt counter-intuitive. I've heard a few people say they liked it, but I couldn't stand it.

Part of what frustrates me is that I look at audio programs like Audacity and others on Linux, and see how you can do almost anything with sound files on Linux, but most Linux video editors do only one or two transitions (if you're lucky, they'll do a dissolve, many aren't even frame accurate, which is something I NEED as a pro) and don't have features like keying, although they may allow titles.

I'd be interested in finding out what the video people you set up are doing. By professional, are they working in the low end wedding market? Do they do promos, commercials, photo transfers, depositions, training videos or other work for corporations? If I were only doing depositions (some people make quite a good living doing that!), for example, I wouldn't need frame accuracy, but that doesn't involve much skill, just setting up the camera, reading the legal stuff at the start, making copies, and sometimes cutting out parts the lawyers later say they don't want.

Re:I'm using md, aka Linux Software Raid

[onkelonkel]

No you didn't. You IMPLIED that.

Dictionaries are great. Not only do they spell the words for you, they also tell you what they mean.

Re:I'm using md, aka Linux Software Raid

[kzadot]

haha your a dick:
infer Audio pronunciation of "inferred" ( P ) Pronunciation Key (n-fûr)
v. inferred, inferring, infers
v. tr.

1. To conclude from evidence or premises.
2. To reason from circumstance; surmise: We can infer that his motive in publishing the diary was less than honorable.
3. To lead to as a consequence or conclusion: "Socrates argued that a statue inferred the existence of a sculptor" (Academy).
4. To hint; imply.

I actually looked up both words, as I do to every single word I ever type in a word forum (after all web forums are the epitome of literary prestige) and concluded that infact imply was the inferior choice.

Re:I'm using md, aka Linux Software Raid

[drsmithy]

haha your a dick:

[...]

I actually looked up both words, as I do to every single word I ever type in a word forum (after all web forums are the epitome of literary prestige) and concluded that infact imply was the inferior choice.

I suggest you look up the words "your" and "you're". See if you can find "infact" anywhere as well.

Re:I'm using md, aka Linux Software Raid

[drsmithy]

From my reading on forums and other various articles, there's almost no (if any) benefit to using firmware-based raid over true software raid.

Yes, there is - the same major (practically *only*, these days )advantage hardware RAID gives you:

Transparency.

Both hardware and "firmware" RAID present a single block device to the OS and BIOS, making it feasible to install and boot the OS on that RAID device and not worry about a hardware failure crippling the machine (a distinct possibility with software RAID if the primary drive dies).

That said, Linux's software RAID is excellent. My recommended configurations (where large data stores are required) is:

Hardware or "firmware" RAID1 for the OS/System/Applications.

Software RAID1/5/10/50/whatever for the data (which RAID level you choose is dependant on the objectives).

This gives you the best of both worlds - your OS install benefits from the transparency of the hardware/firmware RAID and your data benefits from the extra flexibility, robustness and (usually) performance of software RAID.

Re:I'm using md, aka Linux Software Raid

[Asgard]

Don't ignore it, be sure to run mdadm in monitor mode to tell you when something fails.

Re:I'm using md, aka Linux Software Raid

[TheWanderingHermit]

Actually, that is setup. IIRC, it's automatically set up in Debian, but I don't remember it if was setup or if I set it up.

Re:I'm using md, aka Linux Software Raid

[onkelonkel]

The first three examples tell you that infer is not the word you want. Enough bozos confuse infer and imply that we have to add the 4th definition (Gresham's Law ... I think?). Def # 4 is "deprecated"

Re:I'm using md, aka Linux Software Raid

[ErikZ]

Because first I would need a Motherboard with 4 SATA ports on it. Which I don't have, so I need to get a SATA card.

Software raid...

[eviltypeguy]

Software raid? Only on non-production boxes. Otherwise I use some ole' AMI MegaRaid controllers. No software raid for me thank you very much.

What experience I've had with it under Linux systems makes me flee in terror.

Re:Software raid...

[Anonymous Coward]

I use some ole' AMI MegaRaid controllers. No software raid for me

Huh? First you say you use AMI MegaRaid, then you say you don't use software RAID - please make up your mind!

Re:Software raid...

[Wakko Warner]

um... has anyone taught you the difference between software and hardware yet?

Re:Software raid...

[eviltypeguy]

Thanks for moderating down someone's opinion just because you don't like it.

The device-mapper works fine here.

[woolie]

It's been running my raid for a couple of years...I think. I cannot honestly remember when I first configured it.

There are some things to be aware of. If you want to mount / as a raid, it can be tricky. The initrd needs to be properly configured, or the drivers must be built into the kernel.

Sometimes, the raids don't shut-down completely. I've never been able to completely solve this problem. Most of the time it's OK, but some machines have trouble. The most common culprit has been NFS.

GRUB & LILO need some special care when you plan to mount root from a raid mirror. It does work and it has all of the appropriate relibility features.

Re:The device-mapper works fine here.

[rhythmx]

I went through the same thing you did. I bought and retured several RAID cards. One was unsupported and the others had binary only drivers. So, I just got an extra drive controller and let Linux-2.6 work it's magic. On my worstation, I have been using 3 plain-ATA drives in both a RAID5 and RAID-0 configuration for about a year now. I highly recommend this method unless you have some need to access your disks from a lesser OS.

The logical disk's read speeds with RAID0 are (as expected) three times faster than normal (162 MB/s). The RAID5 partition reads at almost 2x, but writing is slow because of the parity. RAID5 writes also take up about 5% of my CPU which is an Athlon 2500+. Not really a heavy load for a modern processor.

Using pure linux also gives you much more control of the organization of your data. As the device mapper operates at a block device level, you can RAID partitions or even files instead of whole drives like with a card. This is what enables me to use 0 and 5 by partitioning the drives.

General Infomation

Re:The device-mapper works fine here.

[Short Circuit]

That brings up an interesting point. Why demand a four-port SATA card, when a pair of two-port cards should work just as well? The PCI bus bandwidth bottleneck doesn't get any tighter whether you run four drives through one slot or one drive each through four slots.

There might even be an advantage, since the drives won't necessarily be on a shared IRQ.

Re:The device-mapper works fine here.

[arkane1234]

The same goes with the IDE expansion cards they are selling. It basically adds another IRQ setup, along with 2 other IDE channels to plug into. Essentially giving you 2 more drives to put into an IDE software raid array. (just keep slapping the cards in)

Re:The device-mapper works fine here.

[ErikZ]

I looked into that. And it's impossible to tell if two 2 port cards will play nice with each other when installed in the same system, without actually experimenting with them.

it doesn't really matter, does it?

[Anonymous Coward]

First of all, don't use software raid when you can get, e.g., a nice 3ware card that works well.

But, sometimes software raid is fun to play with at home. For that purpose I just have a couple Promise cards stuck in my machine, each with two SATA plugs. Works fine.

If you're using software raid, it seems to me it doesn't really matter what kind of hardware you are using.

Re:it doesn't really matter, does it?

[itzdandy]

linux runs a very VERY good software raid that can and will outperform many hardware raid solutions. i get better performance on my software raid array than i did on the same server with hardware raid. btw, hardware raid card died ;`(

also, software raid is migratable! any linux machine with software raid modules will read the partitions and use them where hardware raid ties you to a specific chipset and even card model.

hardware raid has its place but linux software raid it VERY VERY GOOD at SIMPLE raid tasks. IMHO i dont think it is quite ready for full hotswap and hot rebuilds of raid5 and whatnot but it will get their someday.

Re:it doesn't really matter, does it?

[Gilk180]

Actually, it is ready in my experience.

I raid5 5 external scsi drives. When I want to swap on out, unplug it, plug in the new one, add it to the raid set and your done. No downtime. Simple. Checking /proc/mdstat will tell you how long it will take to fully integrate the new drive.

Re:it doesn't really matter, does it?

[innosent]

3ware is definitely the way to go. We use them exclusively where I work for SATA RAID, and Adaptec for SCSI. Works in Linux, FreeBSD, and Windows, has REAL hardware RAID (unlike the Promise cards), and gives you great management tools, both in firmware and software. The only possible issue you might have is that they are certainly not the cheapest solution. Then again, how much is it worth to you not to be called about an array failure at 4 AM?

Also, since you're using SATA, why only 4 drives? Even with the Raptors, that doesn't seem like a high-performance setup. The minimum we have in a more than RAID 1 (simple mirror) setup is 6 drives, and those are 15k U320 drives, for SATA we have at least 8 drives. If you're going to eat the time on rotational latency, at least make it up with more spindles, and don't forget to have at least one hot-standby drive (which would also require you to have a decent card/software RAID setup, cheap cards don't do hot-standby).

Re:it doesn't really matter, does it?

[ErikZ]

"Also, since you're using SATA, why only 4 drives?"

Because 4 drives fit perfectly behind the 120mm fan in the front of the case.

Because I can power everything in the case with a common power supply, also with a 120mm fan.

Because that's all I can afford.

It doesn't have to be high performance. It's my linux box for the house. The RAID is for keeping drive images, mp3s, video recorded from TV, etc.

megaraid

[Johnny Mnemonic]

I like this one: MegaRAID SATA 150-4. Admittedly, I've only used it under OS X Server, as it's apparently what Apple uses in their OEM; but they do have linux drivers and I can only assume that they work as well, if not better. Straightforward setup on the CLI, and not too expensive.

Personally, for $300 I wouldn't screw around with a software raid unless this is your own personal box and the drives only have MP3s.

Re:megaraid

[klui]

We use this one, too. There was actually an article a while back here and quite a few folks liked software RAID because CPUs are now fast enough to support it without causing significant degradation. After skipping over the low end Adaptec (the $150 product), Promise, and Highpoint; the middle ground came up and the LSI MegaRAID SATA 150-4 made a lot of sense. We currently use it under Windows 2000 and the product is supported under Linux. Unfortunately it's not supported under *BSD. We ran into a problem where copying a large set of files result in that Windows Explorer window freezing up and the disk activity light was on solid for around 30-60 seconds. Tech support said this was normal, but several months later, a new BIOS/driver update acknowledged and fixed this problem. Don't know if this is good or bad. Bad that tech support doesn't talk to their developers; good that their developers scour their tech support database and fix bugs. I originally wanted to buy this product from Germany but they only sell there and it's not available in the U.S. 3ware was a candidate but I recall that some parts of their performance was not as good as the LSI.

Re:megaraid

[croddy]

noooo! noooooooo!

the linux megaraid drivers are pretty terrible. we have a couple of machines at work with *unbelievable* throughput problems due to the poor linux megaraid drivers.

Re:megaraid

[j-turkey]

Although it costs an extra $80, the MegaRAID SATA 150-6 is really one of the best of its kind on the market simply due to its available battery backup support. IME, battery backup can really make the difference in terms of reliability, especially when you have controller caching enabled (particularly if you are using a DB/transaction work with RAID 5). Just a thought.

Uhmm, what's your problem really...

[joto]

Lately, I'm having issues with my RAID. Specifically, closed source drivers for my RAID card that only support Red Hat 9. So I've decided to Ebay the card, and try to figure out how to turn 4 SATA drives into a software driven RAID 5 setup. Yes, I know I'll lose all the data, and I'm not worried about it. Finding a 4 port (or more) SATA controller card, that's well supported under Linux, has been difficult. Everyone wants to slap on their own RAID chip and charge you another 100$ for the pleasure.

But can't you just use your raid card as a SATA card, and ignore the raid functionality? Why do you absolutely need it to be non-RAID? I'm sorry, but I'm having real trouble understanding what's the difficulty here...

Re:Uhmm, what's your problem really...

[ErikZ]

I didn't think it was possible to access the drives connected to a RAID card without having a working driver to access that card.

It's the Fastrack S150 SX4.

Promise

[IIDX]

I've got a Promise TX4 (4 port). I use it under windows for software raid 5, but it has been good to me.

It lists Red hat and Suse support, I thought that it was one of the cards with wide spread linux support though.

Re:Promise

[IIDX]

It does have the driver source code available, so you should be able to use it with your favorite distro.

Re:Promise

[Isao]

And I've been running it for several years, using ATA100 hot-swap sleds, with no difficulties. Works, too - had a transparent drive failure about two years ago.

I don't quite understand how you're going to boot a mirrored root volume automagically via software raid (much less a striped one). Or is it acceptable to have to reconfigure to boot?

Re:Promise

[Gilk180]

"booting" from a mirrored volume works just fine with most distros. If you are running custom kernel, you should be careful to make sure to compile in all the needed drivers, but if you're running a raid system you would probably do that anyway.

The only real restrictions are that /boot must be on a non-raid or mirrored raid partition. This is because the bios and boot loader need access to that heirarchy. Other than that you can software raid anything. The reason you can use a mirrored partition is that bios and grub treat it as read only and since it's mirrored it looks just like a normal partition to them.

I have been running small non-raid /boot with a striped root partition under Fedora for some time and haven't seen any problems.

Promise still makes good multiport non-raid boards

[r_naked]

I am using the Promise Ultra133TX4 combined with raidtools. Granted this is a 4 port (8 devices) PATA controller but I think the 4 port SATA board they have would suit you fine. Setting up software raid under Gentoo was a breeze. I have 6 160gig Hitachi drives in a raid5 configuration and it has been rock stable for over a year (only shutdown for the huricanes in Florida and a kernel upgrade).

ARECA has worked nice for me

[cs668]

This is not software raid it is a SATA-RAID card so I am not exactly answering your question. Bet this card has worked well for me and they perform well too.

# hdparm -tT /dev/sda /dev/sda:
Timing cached reads: 2052 MB in 2.00 seconds = 1026.16 MB/sec
Timing buffered disk reads: 380 MB in 3.01 seconds = 126.27 MB/sec

Re:ARECA has worked nice for me

[Anonymous Coward]

What drives (and how many) are you using on that card?

Re:ARECA has worked nice for me

[cs668]

4 x WD2500JD

Vanilla Western Digital SATA 250 GB

this is just me but...

[serialhex]

personally?

i'de get this board, not worry about the SLI features (unless your gonna use it) and not care about the hardware RAID... this way if you get really paranoid you can stick 4 more drives in and mirror the bitches! or alturnativly stick in more drives to make it go faster!

but thats just what i'de do. if it was just for me servin files to myself & friends. anything more i'de start to look into more $$$

Re:this is just me but...

[pugdk]

Yeah, let the poor soul buy an NFORCE based board when he's already griping about his hardware not being supported... ehm...

I assume your comment was sarcastic... right?

-pug

Whoever I want

[Anonymous Coward]

I raid whoever I like.

Sincerely, BSA

3ware, 3ware 3ware.

[zsazsa]

The best option for real hardware SATA (or IDE) RAID in Linux is 3ware, bar none. Their drivers have been in the official Linux kernel since the early 2.4 days, and they just work. Highly recommended.

Why real hardware RAID? Say, for example, your boot drive goes out in a software RAID configuration. Your system is suddenly unable to boot, requiring manual intervention for a rebuild. With hardware RAID, the BIOS built-in to the card handles things smoothly and your system can boot without a problem.

Re:3ware, 3ware 3ware.

[MBCook]

Bingo.

This month's Linux Magazine has a review of a four port 3ware hardeware RAID5 controller that is (duh) supported under Linux. They gave it 5/5 Penguins.

Now the card is $440, which may be more than you are willing to spend, but that would solve your problem.

Re:3ware, 3ware 3ware.

[ColaMan]

Say, for example, your boot drive goes out in a software RAID configuration.

Hmm, I've never really had that problem with bootable mirrored software RAID's that I've setup.
There was a HOWTO about bootable software raid somewhere.... but it's what I used in the following.

I had a lab server on a remote site set up with two mirrored drives and the BIOS set to boot the first drive...then the second. That way, if one died, as they are a mirrored pair, everything still reboots fine. md detects the dud/missing drive and soldiers on with the other one. You shut it down, replace the dead drive, power up and use mdadm to configure your replacement.
And it apparently did work, 'cause one week I went to visit the site, and found that one IDE cable had been pulled out, by person(s) unknown(!!).

Of course, this is just with a mirrored drive. All those other raid configurations are a bit more difficult/impossible to deal with, but it's handy to do if all you've got is a shedful of old parts and you need RAID.

That site had a lot of people with the "windows" mentality - unfortunately, I couldn't secure the server anywhere safe :
Luser: "Hmmm, I can't reach the file on the server... it must be stuffed!"
(luser hits reset button on front of server)
(again)
(again)
(again)
(luser notices other PC's connect to server fine)
(decides to restart windows on their PC)
Luser: "oh, it works now!"
Me(reading system logs remotely): "WTF?"
(makes mental note to unplug reset button on next visit)

Re:3ware, 3ware 3ware.

[Malor]

The 3ware cards are well-supported, but they're SLOW. I have an older generation 8500, one that is supposedly 'optimized' for RAID-5. When under a heavy write load, the machine essentially grinds to a near-halt, making interactive use of the system very painful. Just abysmal performance. RAID-10 was reasonably snappy, but no speed demon. They make big speed claims, but they just don't deliver on them, in my experience.

It works great, however, in JBOD mode (Just a Bunch of Disks), running software RAID on my Athlon 1900+. I get good throughput, even on writes (25 megs/second or thereabouts, maybe faster... I'd have to look at my bonnie numbers, which I don't have right now). And the interactive response stays excellent, even when stressing the system. But there are far, far cheaper ways to get 4 SATA ports.

Just as an aside, I do know what a RAID card should look like. I have an ICP Vortex SCSI RAID Controller that kicks ass and takes names, relatively speaking. I believe the write performance was around 40 megabytes/second on six 10KRPM SCSI drives, and read performance was on the order of 80 megs/second. For a PCI controller, that's exellent, and THAT is what a good RAID controller should look like.

The 3Ware card, on the other hand, was probably giving me 6-8 megs/second on writes, and rendering the system unusable to do it. The 8000 generation is woefully underpowered. Hardware RAID is really nice when it works, because you can hide the details from the OS, but the 3Ware implementation is so bad that you're far better off with JBOD mode and software RAID. $400 will buy you a motherboard with 4 SATA ports, possibly 8... along with a new CPU and RAM to boot!

The 9000 series may be better, but they look to have pretty similar specs. The 8500 was bad enough that I'm not likely to buy 3Ware again until I KNOW they're delivering on their performance claims. I love their Linux support, but earlier generations of their cards didn't come CLOSE to delivering what they promised.

Re:3ware, 3ware 3ware.

[Jon Chatow]

I'm sorry, but this is commonly repeated, but wrong.

3ware's SATA implementation is ugly; it's effectively a bridge from their PATA one, so it doesn't support NCQ.

Personally I use Areca cards - a 16-way card that can run RAID '6' (RAID 5 but with two parity discs) and a hot-spare, has its own Ethernet port for remote access to the firmware is rather good. Oh, and it has (unofficial) kernel sources suitable for 2.3 - 2.6.

Very good.

Re:3ware, 3ware 3ware.

[zsazsa]

Wow, their cards looks pretty sweet. I'll be sure to check them out if I need to build a big server again.

Re:3ware, 3ware 3ware.

[slithytove]

Either you are expecting too much, or there is something wrong with your setup. I havent done a lot of direct comparison of ide software RAID, so I wont speak to that. Obviously its not scsi, but if you need a lot of capacity for a reasonable price, SCSI blows.
I have a 7810 running 4x250G in the XP2600 box I'm typing this on. The card is in a normal pci slot. The fs of the single 702G (usable) partition is xfs (with su and sw set properly). I'm running kernel 2.6.11 w/ cfq io scheduling, udev, and a gentoo ~x86 system. This box is always running kde3.4, slimserver (to serve a squeezebox and softsqueeze on another box), serves high bitrate video over the network and runs aMule and azureus at all times. Oh, and it has intrusion detection running (and constantly writing to a db). I can't speak to benchmark performance, but I can tell you that it runs smooth as silk.
I also have 3 servers at work that I built with 3ware cards running 2.6 kernel gentoo systems. The cards range from 7000 series to a brand new 9500-8. Needless to say, I'm quite pleased with 3ware.

Re:3ware, 3ware 3ware.

[Malor]

I don't think there's a whole lot to get wrong here. I was running the most current firmware and most current utilities, and I manually patched the most recent driver into the 2.4 kernel. (at the time, the 2.6 kernel wasn't supported by the monitoring utilities, so I stayed on 2.4). I was probably using ReiserFS, though I don't remember for sure. I didn't start using XFS until later.

All I had to do to make interactive use of the machine exceedingly painful was to start dumping a big file from my Windows box onto the RAID. Everything would come to a near-halt, stuck on I/O... the CPU wasn't loaded, but good luck running any new programs or doing much of anything with the disk. This was on an Athlon 1900, I believe with a KT333 chipset.

I had the same results on a server machine at work. That one was on a gigahertz P3 with an Intel chipset... very similar results/symptoms. RAID-10 was okay on both machines, but didn't come anywhere near the speed I was expecting. hdparm reported about 30mb/second, instead of the 60 or so I was expecting off a four-drive RAID-10. It didn't get bogged down or give me problems in RAID-10 mode, but it wasn't fast.

I don't think it was the chipsets, since both the Intel and the VIA failed to perform well. And I had all the most recent code for everything. It just sucked terribly at writing.

I've read multiple reviews that seemed to support this view... they certainly didn't shake it any. RAID-5 on the 8500s and earlier is fine for reading... if that's mostly what you'll be doing, it's okay. (I see, for instance, that your described system load is mostly reading.)

I get much better performance on both reads AND writes using the md driver. I'm very happy with the machine's performance now. It feels very snappy, even under a substantial load.

And, of course, the ICP Vortex absolutely kicks ass and takes names.. and you can get those used for not too much money. The disk space is still relatively expensive, but if you want it to run really fast, SCSI is still the way to go, and will be for awhile longer yet. You can get reasonably priced enclosures from Cremax that fit into open drive bays on your existing computer... you can get 3-in-2 and 4-in-3, among others, with supplemental fans and failure alarms on both the fans and on temperature. They're quite noisy, so not good for a desktop, but they're a great way to build a solid SCSI server, including hotswap capability, without spending much money at all.

If, however, you need it both big and cheap, then software RAID over either SATA or PATA seems to be where it's at.

Re:3ware, 3ware 3ware.

[DrZaius]

I agree. 3ware cards are great. They have excellent linux support and an excellent reputation. I use them under linux and windows.

Two areas you may run into problems with:

1) The cards require a lot of power and riser cards can be troublesome.

2) They aren't the fastest cards on the market. They do use a custom PATA=>SATA bridge, even on the 9500 cards. That being said, they are still blazingly fast and reliable.

I'd recommend getting the 9500 boards. The 8500's didn't support onboard cache and there is a significant speed improvement on the 9500 boards.

We have a 9500 in our db server. We were using the Western Digital raptor drives. It is a RAID10 of 6 drives. Over the course of 8 months, we lost 12 drives. That's right, 12 drive failures. Through this, we didn't experience a single second of downtime.

I'm yet another satisfied 3ware user :)

Re:3ware, 3ware 3ware.

[GoRK]

The new 9xxx series 3ware cards are much better than the old series and support command queueing and all that jazz that you get from a native SATA implementation. They're not even much more expensive either. The driver has finally made it into 2.6 too, but you gotta patch it into 2.4. You can get really really good performance on them for the cost - I run one with 4 WD Raptor 10K RPM 72GB drives and don't get any of the same kind of slowdowns I used to get on 3ware's older PATA cards, especially during heavy writes.

Thanks for the heads up on Areca cards. I'll keep that filed under 'check out for next time' :)

Re:3ware, 3ware 3ware.

[rweller]

Over the course of 8 months, we lost 12 drives.

Tried cooling them?

Re:3ware, 3ware 3ware.

[DrZaius]

That wasn't really an option. They were in a 2U chassis that was already loaded up with fans.

My personal experience with WD has been terrible. I swore off the drives many years ago, but it was hard to turn down 10K RPM drives. We took a risk getting WD drives and it didn't pan out. We've since (using 3ware's CLI) swapped out all of the WD drives for Seagates. Again, no down time, plus we were going to larger drives.

We had 3 failures of WD drives in the first 3 months of operation. It's been 4 and no seagate drives have failed.

WD's website says:

Enterprise storage systems built using WD's new 10,000 RPM Enterprise Serial ATA hard drive are highly reliable, fast, low-cost systems that meet the budget and storage needs of today's IT manager.

I figured "highly reliable" "enterprise serial ATA" drives could live in a typical "Enterprise" Chassis in a well air conditioned colo center. In retrospec, we didn't notice any speed/performance loss when we went to the 7200RPM drives.

Re:3ware, 3ware 3ware.

[asdfghjklqwertyuiop]

A RAID controller with its own ethernet port and protocol stack all the way up to a telnet service? That is just stupid.

The right way to do this is either just get one of these or one of the many more expensive/featureful alternatives. Or better yet, just get a real server that has a real serial console (or if you run windows and/or have more money than brains, get some KVM over IP thing)...

A RAID controller with its own telnet service for remote access to the firmware... *shudder*...

Re:3ware, 3ware 3ware.

[drsmithy]

I had a lab server on a remote site set up with two mirrored drives and the BIOS set to boot the first drive...then the second. That way, if one died, as they are a mirrored pair, everything still reboots fine.

I don't know if you've actually battle-tested this, but you'll almost certainly find it won't work because /boot (or wherever your kernel and initrd are) can't be on a RAID partiition.

Unless you're "manually" syncing up two copies of /boot onto each device, of course, but that's a rather ugly hack just asking for trouble :).

Re:3ware, 3ware 3ware.

[Jon Chatow]

Umm. Riiight. You are aware that the two options you suggested both require restarting the machine to access the firmware of the RAID card, however "cool" you think it is?

Restarting = downtime = bad. No, really.

The Areca card is for proper environments where you just don't have the option to take the system down to pull a disc or three from the array. Hotswap discs, hot-rebuilding of RAID arrays, etc. It's a proper RAID card.

Re:3ware, 3ware 3ware.

[asdfghjklqwertyuiop]

Umm. Riiight. You are aware that the two options you suggested both require restarting the machine to access the firmware of the RAID card, however "cool" you think it is?

Restarting = downtime = bad. No, really.

The Areca card is for proper environments where you just don't have the option to take the system down to pull a disc or three from the array. Hotswap discs, hot-rebuilding of RAID arrays, etc. It's a proper RAID card.

No... "proper" RAID cards have software that you can use to administer and monitor the array. "proper" RAID cards have allowed you to pull disks, hot swap and rebuild the array online for decades. No "proper" RAID card has a network connection and has you use telnet to into the controller to administer it... how dumb. No "proper" system will have you using telnet or any plaintext protocol to administer anything.

And if the array is so fucked up that you can't get into the server to see what the array is doing via software, well then bringing the server down to get into the firmware setup isn't hurting much.

Re:3ware, 3ware 3ware.

[arkane1234]

I don't know if you've actually battle-tested this, but you'll almost certainly find it won't work because /boot (or wherever your kernel and initrd are) can't be on a RAID partiition.


Sure you can, since grub will boot off a raid1 partition easily. I do it on all of my remote servers, separate raid "array" for /boot, /, and a backup OS array just in case.
As long as the kernel has support for raid built in (or you have the modules in an initrd), you'll be fine with a RAID1 /boot with grub.

I've had major problems with Promise cards...

[Futurepower(R)]

I've had major problems with Promise cards under Windows XP. Promise RAID is part software, part hardware. RocketRAIDs have had the same problems. There seems to be basic problems, like Microsoft may not want other RAIDs competing with their Windows 2003 software RAID. Windows seems to have timing problems that confuse RAID cards.

The problem seems to be detecting when the RAID array is broken. This problem has gotten much worse with faster motherboards, because the timing window is shorter. If so, then Linux would be affected, also.

In any case, under Windows, the Promise and RocketRAID cards would often detect a broken mirror or RAID when it was not actually broken.

These problems occur apparently only when booting from the RAID array. When using RAID only for data, there seems to be no problem.

You know....

[porcupine8]

You know you've seen one too many memes on LiveJournal and other such places when you just read that "What Kind of Software RAID Are You?" and think somebody's posted a link to a quiz.

Re:You know....

[jimboisbored]

I know what you mean, because that's what I thought too.

Highpoint-Tech SATA Raid Cards

[seanbry]

Highpoint makes great cards!
check out their controllers and they work under every OS, well at that!

Re:Highpoint-Tech SATA Raid Cards

[DrSkwid]

for some values of *every*

Re:Highpoint-Tech SATA Raid Cards

[Anonymous Coward]

Indeed, I have a ATA rocket 1540 here who's firmware requires you to use out-dated kernels and highpoint's drivers or you ned to unplug the disks before you boot and as soon as LILO loads up you plug them back in.

I am now using a brownbox SiL3114a card.

3Ware has the best reputation.

[Futurepower(R)]

Of all the PC RAID card manufacturers, 3Ware has the best reputation. However, you cannot boot from one drive in a 2-drive mirror. If for some reason you don't have a working 3Ware card, you cannot get your data. It is lost.

If you use 3Ware cards, keep one or two spare cards.

HighPoint RocketRAID cards won't boot XP reliably.

[Futurepower(R)]

HighPoint RocketRAID cards do not function well when used as the boot device for Windows XP. This was verified by HighPoint technical support. We did not try them under Linux. But read my comment above about timing issues.

my setup

[william_w_bush]

5x Maxtor 6 B200M0 sata 2 -- ~$120
in a supermicro 5 slot sata hotplug raid enclosure. -- ~$100
connected to a highpoint rocketraid 1820a 64/133 pcix 8-slot sata card. -- ~$150
2x xeon 2.0Ghz @ 533 fsb ... -- def not priceless, but it has pcix and is spare *shrug
integrated intel 1000MT gige controller on mb
500? watt ssi ps, forget the brand... -- ~$75

gentoo linux, kernel 2.6.9, highpoint card has a proprietary driver which sucks, and proprietary software raid module which gave me problems i'll explain later but would probably be workable if you are into that kind of thing.

linux md raid-5 across all 5. 800Gb resultant /dev/md0. formatted reiserfs 3.6.

my opinion:
works well enough, performance is not amazing but is very solid under most/all loads (mostly fileserving on nfs/samba)

my biggest problem is that the hotplug enclosure tends to cause at least 1 drive to fail to start up properly on reboot, needing a removal and reinsertion and rmmod/insmod to get all the drives online, which is a royal pita. This also means I can't use the built-in software raid module, which doesn't bother me that much, except I lose the hotplugging (not something I'd use). Once I'm up and running tho it's rock-solid for as long as I need it.

md has been beautiful to me so far, too bad the drives support ncq and the card does not. Highly recommend something similar, unless you can get a better controller. I read lots of reviews and few hw sata raid cards came close to sw raid in linux, and the flexibility of md is a MUST-have. Don't think I'm hitting any bottlenecks yet. Really hard to beat the storage/price point right now, and at that size backups become non-trivial.

note: samba and nfs clients will probably require mild to severe tuning for best performance, ymmv. For mysql/apache loads this is possibly overkill, but md is very good with cache/paging if you have the ram for it. Most/all other linux hw raid controllers get killed on cache management alone from what ive read.

Also have a 4x 6y250m0 250Gb raid for backup purposes, but that is usually offline and uses a secondary raid controller.

pricing is half-guess/half-bs

Re:my setup

[william_w_bush]

Honestly if i was going to do it again I'd prolly go with a good nf4 board with an el asso wipo silicon image chip onboard, just do sw raid that way. nf4 is a great sata 'set, and you end up with 8 ports total, plus gig'e/dual gig'e. hell of a lot cheaper nowadays too.

nf4 is mostly supported under recent kernels, and you end up with a much better overall system also, especially if youre looking for an apache/mysql combo.

Also, adaptec 2610 6 pt hw card, about $200 on ebay. uses the common linux i20 adaptec driver. just make sure its returnable because a lot of the reboxed dells are either broken or very badly supported by nobody.

please don't do hw raid tho... that path leads to the dark side... and costs a f*ing lot too, when supported.

Why not the 1640 cards?

[zymurgy_cat]

Why not a rocketraid 1640? They support 4 SATA drives and support (so they claim) Linux. I've run a Highpoint card under FreeBSD with no problems whatsoever...well, the management software won't work, but, hey, I can check things with the command line....

Re:Why not the 1640 cards?

[leifw]

Why not a rocketraid 1640?

I bought a RocketRaid 100. While I had no problem getting it to work under Windows, I was unable to get it to work under any of a number of flavors of Linux. Of course, my ineptitude at compiling a patched linux kernel may have led to my difficulties.

I wound up using the card as a plain old IDE interface and then build software RAID on the drives connected to it. In retrospect, I should've bought a 3ware card, despite its significantly higher cost because it would've saved me significant time.

What Kind Of Software RAID Am I Running?

[happymedium]

A righteous and just one against those godless peer-to-peer commies, thank you very much.

From my notes when setting up my Soft-RAID server

[RabidSquirrel]

I took some notes when I was setting up my home 4x250GB RAID 5 server. I found there to be three categories of RAID solutions. Might be helpful for you in deciding. Copied them below and added a few extra comments.

It really depends on what you are using the server for. I ended up going for the pure Software RAID option. Its for home and I'm a cheap. If you're not cheap or it is for a work server, I'd stick with the pure Hardware solutions.
________

Hardware RAID:
The expensive Adaptec, 3ware, etc SCSI cards found in most servers.

Pro - Offloads XOR calculations from the CPU to internal processor.
Pro - No manual intervention required in case of a raid failure.
Con - Expensive.
Con - Third Party and/or closed source drivers often required.

Semi-Hardware RAID:
These are typically the SATA RAID controllers that are built into motherboards but includes cheapo add-on cards. Generally if less than 150 bucks, not full hardware RAID. I believe all of the RocketRAID cards fall in this section.

Pro - No manual intervention in case of a disk failure.
Pro - Cheap.
Con - Minimal or No CPU offloading.
Con - Third Party and/or closed source drivers often required.

Software RAID:
Use Linux and plain old SATA/PATA controllers to handle all of your RAID needs.

Pro - Very cheap.
Pro - No worry about driver incompatiblity or closed source drivers.
Con - No CPU offloading. You essentially trade CPU power for Disk speed/redundancy... and its a significant trade.
Con - Manual intervetion required in case of disk failure.
Con - PATA Only. Must be one drive per channel! NO SLAVES! Apparently data loss can occur on both drives in the chain if one goes bad. http://www.tldp.org/HOWTO/Software-RAID-HOWTO-4.ht ml#ss4.1
Performance is also hurt in a Master/Slave combo.

________

Use the Onboard SATA and bypass the Hardware RAID

[OmgTEHMATRICKS]

The general consensus among linux kernel engineers and software RAID users is:

1. As long as the onboard SATA chip is well supported on your linux kernel, use the onboard chip.

2. Don't worry about the "hardware RAID" built into the motherboard. You don't have to use it. In fact, most people bypass it.

3. Use the non-BIOS SATA driver for your motherboard. Some motherboards have two different chips. Mine (an Epox 8RDA+Pro nForce Ultra2/400) uses both the common Silicon Image SIL3114 which supports 4 SATA drives and an additional 2 SATA drives provided by the onboard nForce 2 Ultra Gigabit MCP chipset. Quite nice for RAID and I still have normal PATA IDE drives 0 - 3.

4. Quite often the SATA RAID hardware only supports RAID 0,1 and 10 (or 01 depending). If you're looking for RAID 5 then you'll have to buy a more expensive outboard solution. The problem with outboard solutions are that they will eat into your PCI bandwidth. If you will be using PCI-X then you will probably also be paying significantly more for your outboard solution. Most people have a ton of CPU lying around, so handing off the I/O doesn't really buy you that much.

5. When it comes down to it you might as well just use software RAID because you have more control over it. You can use the onboard SATA controllers which allow you to take advantage of the increased on-motherboard bandwidth as well as having a significantly less expensive solution.

6. Another advantage to using Linux software RAID is that you don't have to learn a new RAID management system everytime you upgrade your machine and controller. You can also connect to your machine remotely and manage your raid system through a firewall. Sometimes you can do that with your hardware RAID system and sometimes you need to manage it from the BIOS itself.

7. Once you get comfortable with software RAID you can experiment with mixing and matching various I/O systems underneath it. One of the things I'd like to play with would be using software RAID with Firewire 800 external drives in a pseudo-SCSI arrangement.

8. The LVM2 system doesn't need software RAID, but it works very nicely with it none-the-less and gives you snapshot support etc.

9. Personally, I'm going for RAID 10 (striped mirroring) because drives have gotten very inexpensive and I don't mind burning a few more to get higher I/O rates. Remember, if you go with a mixture of RAID 0 and 1 then you want a striping over mirroring -- that way if you have a single drive failure the array keeps going.

Have fun and don't use RAID instead of backups. Backups save the stuff that you deleted intentionally but need to recover.

Re:From my notes when setting up my Soft-RAID serv

[Hardwyred]

If I might add a comment to your last point. IDE drives take a performance hit when you are working with master and slave devices period. This is not a tradeoff of software raid but just a general issue with ide. You can run software raid across master and slave devices just fine, I do right now both at home and on 168 linux boxes where I work, and we hold the service those boxes provide to 5 9's. The only issue I have ever had with software raid has been throughput. You just cant beat a nice fiberchannel enclosure and a massive controller card to get the data rates up, but that is why we put our databases on the SAN and use software raid for the frontends. Now, yes the howto says not to have both of your drives on the same ide channel in case the bus itself freaks out and starts puking bits all over your drives, but how many people have you met or heard of that have lost data because their ide bus died? Not saying it can't happen, just saying that for most of us, this really isn't a huge concern. Besides, we all have backups, right?

Re:From my notes when setting up my Soft-RAID serv

[Matt Clare]

Thanks for the great post.

I would add to software RAID:

Pro - More inspectable than a cheap RAID card

My $0.02
I've setup a few small office boxes with RAID1 - the average CPU today has power to spare serving RAID1 over 100MB connection - most can do a 1G connection too. Save hardware RAID for big time servers or the gamers.

Re:Use the Onboard SATA and bypass the Hardware RA

[OmgTEHMATRICKS]

I don't usually like responding to my own posts, but I forgot to add something.

Whatever software RAID setup you choose, install the test failure driver in the kernel. That way you can force a simulated failure and make sure the system takes a licking and keeps on ticking.

Good luck...

Re:From my notes when setting up my Soft-RAID serv

[Short Circuit]

Software RAID:
Use Linux and plain old SATA/PATA controllers to handle all of your RAID needs. ...

Con - PATA Only. Must be one drive per channel! NO SLAVES! Apparently data loss can occur on both drives in the chain if one goes bad. http://www.tldp.org/HOWTO/Software-RAID-HOWTO-4.ht ml#ss4.1
Performance is also hurt in a Master/Slave combo.

Um...why PATA only? I've done software raid using loopback devices. I can't see why SATA drives would be any less likely to work.

Promise SATA150 TX4

[lysander]

I learned about the Promise SATA150 TX4 from the aaltonen forums. The card is just a disk controller and support is in the 2.6 kernel. I'm using it in a software raid5 configuration and haven't had any problems. It's about $75 at newegg.

I use straight-up Linux md

[scottfk]

Nothing fancy... just a RAID-1 set, using Linux md.

Since a desktop/workstation machine does mostly reads anyway, I am getting the benefit of striped reads. I don't really care that my writes incur a slight penalty.

Granted, hardware RAID would use less CPU time... but hardware RAID is tied to a particular card. What happens if you move your disks to a new machine? You have to move the RAID card. If you go with an integrated RAID solution on your motherboard, that's tough.

With Linux md RAID, that is not a problem. Just plug the drives into your new machine and go.

Failure question

[wfeick]

I've considered setting up software raid on my Linux server, but I haven't found any doc yet about what happens in the event of an unexpected crash or poweroff part way through writing a RAID-5 stripe.

Suppose I have 4+1 disks in a RAID-5 configuration, and during a write to a stripe of the disk only two of the disks are written to before the system crashes. This leaves me with 2 disks with new content, 2 disks with old content, and a useless parity.

I found a page at RedHat that indicates that as of 2001 there was no multistage commit. Has anything changed since then? Do the Linux MD tools address this?

Doesn't anyone use striping anymore?

[SanityInAnarchy]

I like striping on my desktop. If one of my drives decides to fail, I'm hoping to get plenty of warning, and besides, I have the essential stuff backed up to gmailfs.

I'm doing this because the mirroring is a performance AND a disk space hit, and is only worth it if I am planning for the disk to fail. With striping, I lose $20 for buying two 80 gig drives instead of one 160 gig drive, and I get twice the speed.

The annoying part is that I have to redo stuff if I want to add to the array. That's the one advantage of mirroring -- you can hotswap drives and add them to the array, no big deal, whereas with striping, you have to backup/restore. But if I don't have to restore once in awhile, I'll never get my backups right.

Re:Doesn't anyone use striping anymore?

[Anonymous Coward]

"With striping, I lose $20 for buying two 80 gig drives instead of one 160 gig drive, and I get twice the speed."

I found on my system that the difference between 2 x 80gig drives and a 160 was only about 33% faster, due to the higher data density and associated transfer rate on the larger platters.

Still, if you're going to stripe you can still challenge the data transfer rates of much more expensive hard discs... I'm going to trade up to a brace of 250's before too long.

Re:From my notes when setting up my Soft-RAID serv

[RabidSquirrel]

Sorry that was just a confusing line. In my original post change "PATA Only" to "This Con only applies to PATA drives". Then it should make more sense.

RTFP

[Andy Dodd]

Read his post - He's trying to find drive controllers that support 4+ SATA drives but *do not* include hardware raid.

He's not asking how to set up the software, he's asking about hardware that contains the features he needs for software RAID (many ports) but not redundant features that reduce compatibility and/or add significant cost. (hardware/firmware RAID).

2 problems

[Andy Dodd]

a) Cost
b) There's a good chance that even the non-RAID capabilities of the controller have been compromised by lack of documentation to write a good driver

Re:2 problems

[cowbutt]

Check http://linux.yyz.us/sata/faq-sata-raid.html and http://linux.yyz.us/sata/sata-status.html. There's a good chance you'll be able to use your existing SATA "RAID" card with open drivers included in the standard kernel, and without any performance penalty.

SiI3114 4 port card

[Anonymous Coward]

http://www.newegg.com/app/ViewProductDesc.asp?desc ription=15-124-020&depa=0

Syba 4 port cart based on the SiI3114 - $30. Supported by libata... I'm using one right now in my big storage server.

I'm pretty sure you can boot from it

[RabidSquirrel]

Although I have never set it up myself so it is possible I'm talking out of my ass.

From what I read about the Persistent Super-Block and from what The Software-RAID HowTo says, you can boot from a software RAID setup.

http://www.tldp.org/HOWTO/Software-RAID-HOWTO-5.ht ml#ss5.9/

Re:I'm pretty sure you can boot from it

[kzadot]

Yeah I think it works with mirroring(raid 1), but wont work with striping (raid 0), wont work with RAID 5 or 10 (Heck most software raids can only do 0 or 1 anyway)

Re:I'm pretty sure you can boot from it

[Curtman]

you can boot from a software RAID setup

I don't see the problem with putting a 100MB partition for /boot on one of the drives.. It's a heck of a good idea anyway. Put grub, and your kernels on it. Put tomsrtbt on there too.

Re:I'm pretty sure you can boot from it

[Anonymous Coward]

Just remember to mirror the boot drive and grub, or you're going to really find youself SOL when the primary drive fails.

Re:I'm pretty sure you can boot from it

[Curtman]

or you're going to really find youself SOL when the primary drive fails

SOL? Knoppix, any other LiveCD, or install CD with md support in its kernel will get you booted again if you loose /boot.

Re:I'm pretty sure you can boot from it

[Curtman]

Err... Lose /boot I mean. ;)

Re:I'm pretty sure you can boot from it

[drsmithy]

From what I read about the Persistent Super-Block and from what The Software-RAID HowTo says, you can boot from a software RAID setup.

You still need /boot on a non-RAID device, so your system is still vulnerable to disk failure.

Re:I'm pretty sure you can boot from it

[the_crowbar]

Booting from a Linux software RAID drive works just fine. I currently run a RAID1 (mirror) array for my / partition.

I have two 160GB SATA disks. Partitioning is like this:

/dev/sda1 15G Fat32 (0x0c)
/dev/sda2 15G Fat32 (0x0c)
/dev/sda3 15G JFS (0xfd)
/dev/sda4 rest of disk

sdb is an exact copy of sda. I boot fine from /dev/sda3 (the grub boot block is on /dev/sda1, but that is only to dodge the 1024 cylinder limit, the actual kernel file is on sda3)

To sum up booting from a /dev/mdX device is easy. Just have a recent 2.6 kernel and compile in md. Pass boot=/dev/mdX and let it figure things out for itself.

Cheers,
the_crowbar

I suggest moving to Windows

[Anonymous Coward]

That way, you won't have to deal with the lame support and archaic capabilities of the steaming pile that is the LinSux codebase. Not to mention, you'll be more secure and enjoy more stability under Windows than you would with LinSux. LinSux is for zealot losers with a sweatly, precarious grip on reality.

Re:I suggest moving to Windows

[computersareevil]

Soda machine out of Diet Coke again? Dammit!

Re:From my notes when setting up my Soft-RAID serv

[jamesh]

> Software RAID:
> Con - Manual intervetion required in case of disk failure.

You can get around this for some failure modes, as long as your boot partition is always raid1. I do this at home, /boot is raid1 across 5 disks, the rest is raid5. Read performance for a 5 disk raid1 would probably be fantastic :)

The success of this depends on the disk either failing so badly that the system can't see it anymore and so boots off another disk, or that the part of the failed disk that holds /boot is still readable.

I have a Linux RAID question

[Curien]

OK, I'd like to piggyback on this question.

I have a Promise FastTrak100 Lite controller built into my MB, and I've been using it for firmware RAID for about three years now. It worked fine in Windows (using the Promise SCSI emulation drivers) and in Linux 2.4 (via /dev/ataraid/d0pN). But Linux 2.6 can't see it. I've done some reading and from what I can tell it doesn't support the /dev/ataraid tree anymore.

Is there any way to get a 2.6 kernel to see the array while leaving the data intact?

Re:I have a Linux RAID question

[cowbutt]

dmraid, apparently. Once you've migrated the data off your array, do yourself a favour, and rebuild it using Linux md RAID. :-)

Re:From my notes when setting up my Soft-RAID serv

[mbyte]

i don't know what kind of kernel you run, but my 2.6.11 supports md with SATA and PATA mixted together just fine:

Personalities : [raid0] [raid1] [raid5] [multipath]
md0 : active raid1 sda1[1] hdc1[0]
120060736 blocks [2/2] [UU]

sda is a sata, hdc is a pata drive ...

Re:From my notes when setting up my Soft-RAID serv

[RabidSquirrel]

Never said they couldn't be mixed. You're probably referring to the last line about the PATA problems. See my other reply to Short Circuit. Basically that part of my notes just reads confusing. The notes were meant for my reference. Didn't really write them for others to read. That last Con only applies to PATA configurations. I'm not saying that you cant mix them, nor am I saying that Soft-RAID only works with PATA. IF you use PATA, you must be aware of that Con.

Re:HighPoint RocketRAID cards won't boot XP reliab

[Peter Lustig]

HighPoint RocketRAID cards do not function well when used as the boot device for Windows XP. This was verified by HighPoint technical support. We did not try them under Linux. But read my comment above about timing issues.

I have a Abit KT7A-Raid board with onboard HPT370 Controller. Attached to this one are 3 Seagate 80GB and one Maxtor 20GB drive. The Seagates are combined via Linux Software Raid5 (using Raid1 for the /boot partition). My Athlon TB1333C with 512MB Ram has no further problems with generating the parity for Raid5.

Otherwise, Windows seems to have serious problems with the Highpoint-Controller. Quite often, i found the Windows partition on the Maxtor scrambled, so I had to use my backup to regenerate it. Attaching this disk to the onboard VIA-controller does not generate these problems. If it wasn't for the games, I would leave the Windows world entirely.

Re:HighPoint RocketRAID cards won't boot XP reliab

[seanbry]

I never boot off of raid so i've never had this problem I use raid for things that need the disk speed, like multi-video streams and games. Other than that I find booting from raid a danger and a waste of time.

Re:HighPoint RocketRAID cards won't boot XP reliab

[Peter Lustig]

Why should booting off a raid be a waste of time? Takes as long / short as booting off a single disk. And if you've ever lost something (perhaps personally) valuable due to a disk failure, you appreciate any kind of security you can get which includes storing your data on a raid array.

Networked RAID, anybody?

[graf0z]

I wonder if someone has done RAID mixing local harddrives and network block devices like GNBD or iSCSI? Should be ok on gigabit speed, right?

I know DRDB, but that's more for HA pairs and cannot sync drives in background while mounted.

/graf0z.

FUD

[adolf]

On my systems, I have a software RAID-1 "boot drive."

If one drive in the pair fails, things keep ticking along smoothly. They're really just identical partitions with identical data on different disks.

LILO merrily writes boot code to the array without episode. Meanwhile, the machine's BIOS is happy to boot from disks other than primary-master, all by itself.

I've booted the system after randomly unplugging devices. It works just fine.

Why do all of you 3ware goons think that the world wants to buy hardware which offers no clear advantage over having no hardware at all? (As if I want to add -more- potential points of failure to my systems . . .)

Re:FUD

[zsazsa]

Sorry, I wasn't trying to spread FUD. It's good to know that newer BIOSes can cope with booting from drives other than primary master. The only problem I can see is if the drive first in the boot order is in a failure state and the motherboard's BIOS isn't aware and tries to boot from it anyway. This may be something people can live with to save the $400 on a 3ware card.

Re:FUD

[GoRK]

The ol' software raid 1 boot trick depends highly on the behavior of your BIOS under a failed drive condition. This is not the same thing as you get when you unplug a drive. Some BIOS may boot fine; some may not boot from the 2nd hard drive if the first is still attached and failing. It may also depend on how your drive has failed. If the drive electronics are failed and shorting the wrong pins on your IDE controller, then you may not get past the drive detection code in the BIOS at all.

This is simply one advantage to using a real hardware raid card like the 3ware. There are plenty of other reasons too: Does your chipset/hardware support hot swapping? If you use SATA, does it support command queueing? Do your drives? How much cache does it have? Does it have cache? Can it tolerate all types of hardware failure? Does it have *ahem* 16 ports with individual controllers for each drive? It's not like the BIOS/IDE chipset makers write out in their specs how their hardware performs under drive failure conditions so you have the overhead of testing each configuration to make sure it works proeprly before you have to rely on it. It's not so much a performance difference between hardware and software raid (until RAID-5 anyway) but an issue with how the hardware will respond when something goes wrong, which is one of the primary reasons for using anything above RAID-0 anyway.

Yes, running a 3ware card costs more. There are times when that $400 costs a lot less than the time spent configuring and testing an alternative software-only implementation. There are times when it doesn't and spending another $400 doesn't make a lot of sense. I have run both setups. I have machines deployed with both IDE software-only RAID arrays, IDE 3ware arrays, SCSI software RAID5's, SCSI Adaptec RAID's etc.. it's all application specific. There's no reason to call somebody a goon for recommending 3ware hardware. It's really good hardware; maybe you should try it sometime.

Re:FUD

[drsmithy]

On my systems, I have a software RAID-1 "boot drive."

How do you deal with /boot (or wherever your kernel and initrd are) not being able to be on the RAID device ?

Re:FUD

[arkane1234]

How do you deal with /boot (or wherever your kernel and initrd are) not being able to be on the RAID device ?

It does... try it sometime. One caveat, it has to be RAID1.

Beware of a company that is so sloppy:

[Futurepower(R)]

Beware of a company that is so sloppy that it won't go to the Amigo Guest House in Taipei and find a U.S. citizen and native speaker of English to help it with the English on its web site: http://www.highpoint-tech.com/

Some tips

[obi]

hardware RAID has these advantages:
1) offloads operations to the controller, so eats less CPU/IO bandwidth.
2) can have battery backed cache
3) often looks like "just a scsi controller" to linux and the boot loaders, so booting from f.e. a RAID5 set is often easier.

software RAID has these advantages
1) is cheaper
2) CPU time lost makes hardly any difference
3) has well-tested and supported tools to manage your raid setup. (imagine if you could only set up your raid sets by rebooting and entering the raid bios)
4) disk-layout is non-proprietary (controller died? don't have the same brand lying around? manufacturer left the market? no problem!) - so all-around more flexibility.

Look here for properly supported sata disk controllers:
http://linux.yyz.us/sata/

Some of these cards come with BIOS smarts that provides you with software raid which offer you the advantage of point 3) of hardware raid, ie: bios and boot loader support for your raid.

however, this does mean that the on-disk layout has to be recognized in linux, so linux can make sense of it and set up the raid sets properly. In linux 2.4 there were some drivers that did that themselves, however for linux 2.6 there's now a little userspace program that recognizes a whole bunch of on-disk layouts, and sets them up using the device-mapper facility (part of LVM2).

The advantage of this is that you can use the same well-tested and -supported linux drivers mentioned on http://linux.yyz.us/sata/ , but still use the (bios) facilities provided by the hardware. Another advantage is that this program will probably be used by all ATARAID ("mostly-software-raid") devices on linux, so it is, or will be well-tested and -supported in itself.

You can find this program, called DMRAID here:
http://people.redhat.com/~heinzm/sw/dmraid/

So if you decide to go the SW-RAID way, think and decide if you want the advantage of dmraid. I haven't tried this myself yet, and the only aspect I'm unsure of is the management aspect of it (like with HW-RAID drivers) - DMRAID doesn't use MDADM, so how can you properly monitor, hot-add, ... your raid sets? I don't know, you'll have to investigate this yourself.

MDADM itself isn't going away any time soon either, if I understand correctly. (And even if it does, it's probably very likely that they'll make DMRAID understand the MDADM on-disk layout to provide an upgrade path.)

If however you decide to go the HW-RAID way, make sure you get a reliable and reputable manufacturer - with open source drivers (!), preferably with a known on-disk layout, and be prepared to spend money. I've heard a lot about 3ware, but I have no direct experience with them myself, so I can't vouch for them.

Booting is not an issue

[Anonymous Coward]

Booting is not an issue with software raid. Simply use one cylinder of each drive as a boot partition. Mirror the partitions with RAID-1. Booting is a trivial problem.

You surely CAN boot from it

[adinu79]

I'm booting off my RAID 1 drive (mirroring) using Software RAID. For linux, if you want RAID 0 or any other version, I guess you can put the /boot on a NON-RAID partition, create an INITRD that loads the raid modules and set the real-root to the RAID partition and it should work like a charm. So you only have you kernel and modules on the non-RAID device and the rest of the system on the RAID device, so you don't lose the advantages of RAID for system files.

RAID 1 with Windows XP *shudder*

[Anonymous Coward]

I have 1 250Gb IDE HDD and one 160Gb. Part of the 250Gb drive is used to mirror the other. All this under Windows XP (SP2) which according to M$ cannot be done!

Performance is not noticably slower and I get piece of mind... well, as much as you can do running XP.

:D

• http://www.microsoft.com/resources/documentation/w indows/xp/all/proddocs/en-us/sag_diskconcepts_17.m spx
• http://www.tomshardware.com/storage/20041119/index .html

The real issue

[Intron]

Why aren't we discussing the real issue? Should we allow "ebay" to be used as a verb.

Re:The real issue

[Rob Riggs]

Dude, typing "sell" requires just way too much effort.

Need more information

[gseidman]

You can't just ask what RAID solution to use; you need to specify your optimization criteria. For example, if you are going for inexpensive, high capacity, and good redundancy but don't care about hotswapping, you can go with a reasonably cheap PATA or SATA hardware RAID. If you are willing to skip high capacity you can get an external SCSI SCA rack, an inexpensive SCSI card, and a bunch of small SCSI SCA drives. If you want high capacity and high performance without too much cost, get a bunch of large PATA drives, a similar number of FireWire or USB 2.0 (or both) enclosures, as many FireWire or USB 2.0 interface cards as necessary, and do it that way (which also gives you hotswap). If you are willing to spend lots and lots of money, get a Network Appliance server.

Personally, I'm not worried about performance beyond being able to play full-motion video. I have a PPC 604 180MHz from 1997 with a SCSI card and a RAID rack. 8x18GB at RAID 5 gives me 118GB or so of redundant storage, and I serve it over NFS to my other machines. Just for kicks, I have it going through a cryptoloop, too (LVM on cryptoloop on Linux RAID5 on SCSI). The initial cost was low (the drives were $15 each, the rack was around $100 on eBay, the trays were given to me, the SCSI card was under $40 on eBay, the 100Mbit ethernet card was about $20, and the computer had been a doorstop until I put Linux on it). The ongoing (electricity and cooling) costs are a little high (they are 10K drives), but that's life. I can play an MPEG or AVI from two machines on the network at once without hiccups, so I'm happy.

If I were going to build a RAID server today, I'd probably buy a Mac Mini, four large PATA drives, and four FireWire enclosures. Assuming 160GB drives, I'd have 320GB of RAID5 storage available over NFS (with a spare drive to swap in) for an investment of under $1200, and I can vary that cost with the size and number of drives. Yes, I'd be daisy-chaining FireWire, which means that each drive has only a portion of the total bandwidth. Then again, my network card will only manage 100MBit, so 3/4 of the FireWire bandwidth will be of minimal use anyway (except for reducing latency due to readahead and such, of course).

Re:Need more information

[ErikZ]

RAID 5 isn't supported by OS/X, I had the same idea.

There was a 3rd party software implementation of RAID5 for OS/X, but they went out of business.

Re:Need more information

[gseidman]

Who said anything about OSX? I figured on loading Debian on it.

Don't get a four port controller w/ software RAID

[iamcadaver]

I'm not the only one that tested, and concluded, that software RAID outperformed a 3Ware card.

The problem you run into is bandwith.

Picture four drives on a single PCI controller running in a software RAID5. For every block written, four commands must be issued to the PCI card, one for each drive.

This works great w/ a hardware RAID controller, because they emulate a single SCSI drive, thus only one write command.

Even so: we saw better throughput via software raid ( tested via Bonnie++ on a knoppix 'toram' boot )

BTW: I tested every filesystem format with various block sizes that knoppix offers. As was reported earlier, JFS showed the most impressive jack-of-all-trades performance. Too bad I later discovered you only get two options with RedHat Enterprise WS: ext2 or ext3.

FYI

[Anonymous Coward]

The link in your sig is broken.

whooosh

[Anonymous Coward]

went over your head didht it ya fucking moron.

hes saying the card doesnt have any onboard raid - theyve just implemented software raid in the drivers for the card to make ot look like hardware raid.

dunno if its true, but that's what he's saying.

you numshit.

Re:whooosh

[arkane1234]

Pretty much almost all non-scsi raid cards are software raid in a driver.
Except for 3ware, they make a damn fine card. You pay the money, though.

I use software raid on our production field servers where the machine losing a drive would be catastrophic due to the fact that it'd take at least a day to get out there and fix. I use raid1, and it's worked flawlessly. I've had a few failures (western digital.... I'm going to maxtor) and all of those times it's just purred along nicely.

I guess I shoulda put this in a separate post.

Re:HighPoint RocketRAID cards won't boot XP reliab

[Anonymous Coward]

I had no problems booting a trial of Windows 2003 server on my server back in 2003. Not only did the server use a RocketRAID 404 card with the recently released RAID 5 firmware, it used an VIA EPIA motherboard.

It was a little pokey, but it was rock solid. I booted off the RAID 5 array. :)

I did not have problems for a long time.

[Futurepower(R)]

I also did not have problems. Some motherboards would work correctly for weeks. Then there would be an unexplained failure of the mirror. HighPoint tech. support said they were not able to understand why the failures were occurring. (Promise told me the same thing.) Highpoint said that the failure was common.

A Few Question From The Linux-RAID Newbie

[xsbellx]

I have a fairly extensive background with raid setups but not within Linux nor on an X86 platform so if someone would fill in some blanks, I would be very appreciative.

1) The original poster is looking to setup a 4 drive array, RAID-5 prefered and is looking for a 4 port SATA adapter. My recommendation would be get two adapters with two drives each to provide greater redundancy. I am guessing this can be done with stock PCI SATA controllers. Would a configuration such as this have a negative performance impact based on the characteristics of the PCI bus?

2) Assuming a straight software RAID config, what happens during the boot process if a drive is lost? For example, let's assume the following RAID-1 configuration:
Controller 1:
Disk 0
Disk 1
Controller 2:
Disk 2
Disk 3
/ is mirrored across disk 0 and disk 2 - partition 1
/boot is mirrored across disk 0 and disk 2 - partition 2
/home is mirrored across disk 1 and disk 3 - partition 1

No swap - too confusing for this discussion.

Let's say disk 0 on controller 1 fails. I would hope the system will generate some type of message/alert and then continue functioning normally until it could be shutdown to replace the failed drive (too cheap to go hot-swap). Eventually, I get a new disk and shut the system down to replace the failed disk. I reboot and something happens. Ideally I should see the system come up normally with the possible exception of a warning message and then I can remirror / and /boot when convenient. Can anyone confirm that this is what actually happens?

3) Based on the configuration described above, should /etc/fstab refer to "md" devices or "disk labels"? If either device names or labels can be used, is there a recommendation for one over the other?

I would be quite thankfull for any information regarding these question. Thanks.

MOD PARENT UP

[menscher]

I'd mod you up myself, but then people would just think you were just modded up for saying something that *might* be true. Instead, I'd rather say that it *is* true. We've been rather pissed off at the pitiful performance from our megaraid arrays. Stuff like 35M/s writes. You could almost beat that with a single disk. Sheesh.

For cheap setups, I go with 3ware. For more expensive ones, we use an external raid array with a scsi uplink to the computer. The cache, battery backup, and simplicity of host communication are important advantages to consider. Software raid, while fast and, just scares me.

Re:MOD PARENT UP

[Anonymous Coward]

Even worse, the drivers for RedHat Enterprise 3 are buggy. I'd had numerous lockups (on several machines) when running disk intensive stuff. Back when I was setting these servers up (may 2004), running bonnie++ with a 1GB test size was enough to reproducably crash the server. Pathetic.

Stay away from Megaraid.

Re:MOD PARENT UP

[menscher]

We weren't seeing problems that severe (also using RHEL3 on a megaraid machine), though we did have to run a cron job to restart the megaraid daemon (for email notifications of drive failures). It had a habit of crashing after exactly 8 hours of execution. At least they've fixed that annoying bug.

In July 2004 we had a disk fail, and the entire partition got knocked offline. Not exactly what you'd expect from a raid5 array, is it? We had to rebuild the array offline (ie, system downtime) and then fsck (where there was plenty of corruption) to get it back up. We didn't lose anything important (hooray for tape backups) but it doesn't leave you with a warm fuzzy feeling when something like that happens.

Re:From my notes when setting up my Soft-RAID serv

[drsmithy]

Con - No CPU offloading. You essentially trade CPU power for Disk speed/redundancy... and its a significant trade.

Maybe if you're using a 486.

The CPU overhead of software RAID 5 is insignificant on any remotely modern machine. Even "ancient" ca. 500Mhz P3s have checksumming speeds over 1GB/sec

RAIDcore/Broadcom

[tbuskey]

http://www.broadcom.com/products/Enterprise-Small+ Office/Storage+Solutions

It's semi software but it has several advantages:

Linux support (I think the driver is OSS or slated to be)

Transform from RAID 1 to RAID 5 (or 10 or 50) on the fly

Can support boot drive RAID nicely

Hot swap

We use software RAID-1 here...

[phorm]

One thing to note: slower writes, faster reads.

And one thing I've been wondering about, obviously you would keep the drives on different IDE channels if possible (hda, and hdc here).

If you also have non-RAID drives on hdb, and a CD-ROM on hdd... that should not overly influence the data speeds of the RAID drives except when there is actual data transfer on hdc/hdd, or would a machine automatically split the available pipe upon having two IDE devices (master/slave) on a given IDE channel?

Re:Use the Onboard SATA and bypass the Hardware RA

[glazed]

The Sil3114 on my Asus K8N-E has the 4 sata ports and will do a raid 5.

How I built a 2.8TB RAID storage array

[Yeechang Lee]

On Usenet I posted a detailed description of how I built a 2.8TB RAID storage array for under $4100.

Re:From my notes when setting up my Soft-RAID serv

[RabidSquirrel]

Wow! You're absolutely right. I completely misinterpreted my bonnie++ results. I assumed the high CPU usage was related directly to the fact that it was software raid. My head hurts from slapping it now.

The md0_raid5 daemon takes something like 4-9% of the CPU during the write tests, 0% during reads, and 1-2% during the file create/delete tests.

Man this changes my whole view of software raid. I'd give you a +1 Informative if I had a spare point. How bout just a thank you instead?

My bonnie++ results are below if interested (mangled by slashdot):
"bonnie++ -s 1024"
Size 1G
files 16

------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
33986 88 47744 6 25865 6 28655 73 92221 11 369.2 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
/sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
1291 14 +++++ +++ 596 6 712 9 +++++ +++ 1185 13

CPU: 2.6GHz Celeron D
RAM: 512MB DDR400
Drives: 4x250GB 7200rpm 8mb PATA Maxtor
FS: Raid5 512k chunk, XFS

Those high %CPU marks from bonnie threw me off. Guess I need to update my notes a bit.

Hardware RAID not always the fastest.

[JungleBoy]

At work we've been testing RAID 0 (strip) setups for performance on a number cruncher machine. Hardware raid isn't always the best.

Our first setup was 2x15kRPM U320 SCSI drives on an LSI MegaRAID controller. Apparently the 2.6 kernel driver has serious issues, because we can't get read performance over 50MB/s. This is slower than reads off a single drive on a vanilla SCSI controller.

Our second setup was the same two drives on an LSI U320 SCSI HBA. The HBA has a 'simple' stripped raid via firmware. This worked well for two drives, we got 140MB/s reads. This is about twice the single drive speed. We added a third drive to this setup and it must be swamping the firmware, because the read speeds went _down_ to 110MB/s with three drives. But when we switch to linux md software raid we got 215MB/s. This told us that we weren't yet swamping the scsi channel or PCI-X bus.

We're hoping to be able to software raid 4 15K rpm drives on a single scsi channel. This should put us around 280-300MB/s. If this works we'll try a 2 channel U320 controller with 4 drives on each channel. By offloading the raid operations from the HBA to the CPU we should be able avoid swamping the HBA processor. The only other issue to consider is weather or not we'll swap the PCI-X133 bus. If this happends well use 2 single channel controllers on seperate PCI-X133 buses. Our goal is to get over 500MB/s read on a software raid array. Really, this is an insane read speed. It is over a quarter of memory speed on these systems. We're doing this on a Dual Opteron 250.

Re:From my notes when setting up my Soft-RAID serv

[ErikZ]

"Con - PATA Only. Must be one drive per channel!"

Woah! Wait, what?

SATA is one driver per channel.