Slashdot Mirror
Dependable SCSI RAID Controllers for Linux?
"I have been considering ICP Vortex RZ and RS series and AMI Megaraid as possibles, along with the Mylex line of controllers. I would like some opinions, praises and even nightmare stories on any of these. I am not wanting to invest $350-$1500 per controller on another nightmare like Adaptec/DPT line. It should be obvious but cost is not primary, reliability and to a lesser degree performance are the key issues. In addition I run my controllers in RAID 5 with a hot spare, so suggestions should be for controllers that can do that RAID mode and ones that can be administered from a running Linux system so I can do hot swapping. I would also like controllers whose manufacturer keeps current patches available for the stock kernel tree or is in the kernel tree (for both 2.2 and 2.4, I use 2.2 mostly due to issues with 2.4) as I never use a canned kernel after the install is done. If you run Windows or some other truthfully Adaptec supported OS look for a few *good* DPT or Adaptec controllers on eBay when the swap-out is all over."
I would blame a really large portion of Slashdot's downtime, and the recent downtime with Freshmeat, on those controllers. Outside of a Megadrive that I used at the Virtual Hospital, those are probably some of the worst pieces of Hardware I have ever ran into.
I would never recommend that anyone ever use those cards. Flaky hardware is one issue, but those cards have consistently been the root of a lot of sleepless nights for me fixing the mess that they have caused.
You can't grep a dead tree.
an LSI Megaraid should work quite nicely in Linux, I have an older AMI Megaraid Enterprise 1200 (model 428) with a couple of small disks as my linux disks (and one of them is my windows swapfile :)
never had any problems with it whatsoever.
So what is replacing it? I know you did not just suggest that IDE is actually worth anything.
I would really like to see you run servers on IDE.. HaHa. You are not just a troll, but a stupid one.
SCSI will not die any time soon. If it does, it will be replaced by Fibre Channel.. You couldn't pay me to use an IDE disk anymore, except maybe to boot legacy (x86) hardware with so I can boot from an NFS sever.. for a workstation.
For a server, IDE has no place. Half-duplex, cpu intensive, unreliable, do I need to say more? Oh, and incrediably limited in the number of disks. Raid5 array of IDE? yeah right. You can only have 8 IDE disks in a system, all of which use interrupts.
We have two IBM Netfinity servers that use IBM ServerRAID 3L. The cards are not that great as they only have 4 MB of cache, but they run reliably under 2.4.13.
The drivers are maintained in the kernel, so there is now patching or downloading of drivers.
I think IBM has other models that come with more cache, so you could try calling them.
Does anyone have any thoughts about IDE raid, especially the offerings from Promise Technology? They've got cards that do RAID 5 with regular IDE drives, including hot failover capability. They've also got subsystems that put a full 8 disks into a RAID array, but presents it to the controller as a single SCSI device.
Advantages: Cheap drives.
Disadvantages: Speed, maybe, though since it's all going directly into the PCI bus, I'm not sure this is an issue.
Anyone used these? Comments? I figure with their SuperTrax controller and a bunch of 80 or 100-G drives, you could have half a terabyte in your basement for under two grand.
Don't discount the Compaq line of SmartArray controllers. I've been using one for 2 years without a hitch. Supports everything you need them to do (I'm using the Smart/2P controller in my server). Never had a single problem with it. You can find these on eBay really cheap too.
You can accomplish anything you set your mind to. The impossible just takes a little longer.
Aight, sorry - I do agree with you, and have a stack of IBM DDYS disks to show for it - BUT, you said some things that were wrong:
Just about all PCI ide controllers can use DMA, which cuts the cpu intensiveness down to almost the scsi-level.
Further, Not all drives have to have their own individual interrupt - this depends on the ide chip and how they are arranged on the pci daughterboard, or on the motherboard. (interrupt sharing, etc.) Promise chips will use one interrupt for two interfaces.
SCSI does offer a whole slew of advantages, disconnect, command queueing, etc. These are advantages in a RAID setup. IDE does suck, but not because it's cpu intensive or gobbles interrupts.
fnord.
Compaq's SMART array controllers work well under Linux, and support RAID 0, 1, 5, and 0+1 with hot spares. I have been running RedHat 7.1 on one of these for months, in a production environment, with great success.
I work for a medium sized web hosting company which sells dedicated/managed servers to customers. We will only put ICP Vortex cards in them. These are the only cards we put in our own servers as well, I would say we have at least 40 of them in our datacenter and they work great. Not to mention if a drive fails you can easily hear them beeping from outside the datacenter, even with all the server/air conditioner noise.
Great cards, great speed, and a not so bad price. They work flawlessly in Linux and Windows.
Open Source Time and Attendance, Job Costing a
Since cost is not your primary worry, can you run a test? Get one of each, configure each with several harddrives and see what happens to each under load. While there is a large difference between the way system behave when running 50GB over 7 harddrives and the way systems work when running with 50TB in a real raid system, anything that doesn't work with the smaller systems will fail in the large ones too.
I wonder if your hardware isn't going bad. I too run DPT SmartRAID controllers. A 2654U2 to be exact (2-channel version) in a 440LX P2/266 (we're network bound, not CPU bound) which is used for fileserving about 50 people in a file-heavy office environment. Before that, it was a SmartRAID V with the hardware cache/RAID card (which is in use ino another, heavier-hit webserver).
Zero stability issues on both. The 2654U2 has a 5-drive RAID5 + hot spare (UW2 SCA drives) on one channel and a 6x24 DDS-3 on another. I've done some pretty I/O intense things on this controller (including rebuilding the array during office hours) with no problems at all. This is on kernel 2.4.17. The SmartRAID V is on a 2.2.14 system which has about 50 colocated web and mail sites (it does a pretty good job of keeping the T1 busy). It runs a RAID1+0 array with really old Seagate Baracuda SCSI-1 drives and a single external DDS-3 for backup. Again, zero stability issues. I'd buy these again without hesitating.
Perhaps you need to delve deeper into the problem. The 2654U2 did not like the original P90 system the server used to be; we had bad issues there and the tech basically said that the original PCI spec was not good enough for the card. Upgrade the motherboard and all was fine. If you're running 2.4, make sure you're in the .16/.17 kernels, as earlier 2.4 kernels had issues with all manner of things, but not specifically the DPT I2O drivers, IIRC.
Both of these systems run the kernel drivers and use the dptutil software that DPT used to have (which you're right, has gone the way of the dodo after Adaptec's assimilation of DPT); what specifically can you do to cause problems? I don't think it's the card/drivers in general but if you give me a test or two I can run to see if I'm affected as well we might be able to fix this.
Slashdot's braindead lameness filter is not letting me post my dptutil -L output. Sorry.
While I'll agree that SCSI is superior for most applications, IDE is no slouch nowadays.
On one of our production servers we have twin 18 Gig 10krpm Ultrawide SCSI drives for the database, and a pai rof 80 Gig IDE drives for the static data like web content.
The pair of U2W SCSI drives in a RAID1 can be read at about 48 Megs a second by bonnie, while the pair of 80 gig IDEs can be read at about 28 Megs a second.
pgbench, a little benchmarking program for postgresql, gets about 150 to 200 transactions per second on the dual SCSI drives, while it gets about 100 to 120 on the dual IDE drives.
the problem is, even under it's heaviest loads, that machine never handles more than 10 or 20 transactions every second. Both sets of drives are plenty fast enough to hand the load.
For servers that need hundreds of gigabytes of storage but only have to provide static storage for a medium, to small group, the money you'd spend on SCSI is probably better spent on other options for that server.
For a database server handling hundreds of concurrent users, SCSI (via electrical cables) is a good choice, but maybe a SCSI over FC-AL setup would be needed.
Engineering isn't about which component is the absolute best, it's about which component makes the most sense for what you're doing.
--- It is not the things we do which we regret the most, but the things which we don't do.
I've been using a Mylex Acceleraid 150 for just about a year now with zero problems. The box
had about 235 days of uptime until I shut it down to add memory.
What precisely was your problem with Mylex?
Random kernel crashes, forgets about disks... just about every kind of error that I have seen.
You can't grep a dead tree.
There are three great options to get your servers out of the RAID-controller business. One is NAS (Network Attached Storage), the second is using native SCSI or IDE controllers with RAID provided by your OS. And lastly, you can buy a box that already is a RAID but just looks like one big fat drive and plug it in.
:-)
i nd ex.asp?fileid=R28825
At work run all our linux boxen at work with kernel mirroring and it uses almost NO CPU even under pretty heavy parallel load. Great for the base OS with SCSI or IDE, since the only thing they'll do once they boot is swap to these. Striping your swap space across multiple drives really helps when a server starts running low on memory.
I have mirror sets running at 48 Megabytes a second on two year old 18 Gig 10k SCSI drives for streaming output, and can provide very good performance under parallel load as a database disk set.
I've never had the kernel RAID drivers act flakey since I started using them over two years ago, and I've done various things like hot insert a raid disk in both RAID 1 and RAID 5 (both were pretty easy to do.) and typed the respected, yet undocumented --really-xxxxx (xxxxx=a 5 letter word not mentioned here!) flag a few times.
A friend is in the process of building NAS servers in 2U units with multiple IDE cards and ~500 Gigs of storage for ~$3500 or so. SCSI versions would be a bit more, bigger, and probably need more cooling, but be faster too. Right now the IDE ones are fast enough with a RAID 5 configuration.
The IDE ones can flood a 100 Base-TX connection, so performance isn't really an issue for anything on less than gigabit, and even then the IDEs will use up a goodly chunk of that.
The external RAIDS are often the fastest for databases, offering fibre optic connections. they're not cheap, but if you're running EBay's database, cheap isn't the point anymore.
If you have to have a RAID card, I can recommend the AMI Megaraid 428, which used, on Ebay, goes for $100 right now. Not that fast (I never got more than 20 Megabytes a second from one) but very solid and reliable, and they can hold up to 45 SCSI hard drives if you can afford the cooling and electrical for them. Plus the first channel looks like a regular SCSI card to anything other than a hard drive, like a tape drive or CDROM, so you don't need another SCSI card if you want a tape drive to back it up.
While the Megaraid site no longer has configuration software available, this site:
http://domsch.com/linux/#megaraid
points to this site:
http://support.dell.com/us/en/filelib/download/
on Dell where you can find management software for the MegaRAID controllers.
--- It is not the things we do which we regret the most, but the things which we don't do.
In my albeit limited experience w. RAID controllers, you cannot buy a better controller than ICP Vortex. Support is good, and they make a solid product. And you pay for it!
IDE RAID is fine for workstation's and home use....as far as I am concerned, it has no business in a corporate server environment. Anyone who tells you differnet is shaving pennies, and hasn't a clue. Of course, your opinion may differ!
Can you buy hot-swappable IDE enclosures? I've never seen any.
Performance-wise, these cards aren't top-notch. They have a very small amount of cache. Modern SCSI RAID cards take DIMMs and can be easily upgraded to more cache if necessary. These things have soldered-on memory.
For mass storage, they're great. For high-performance mass-storage, I'd still look to SCSI. Where else can you get 15000 RPM drives with 5-year warranties?
- A.P.
"Remember when the U.S. had a drug problem, and then we declared a War On Drugs, and now you can't buy drugs anymore?"
SCSI is SCSI wether it be fibrechannel or the 50/68 pin connector.
Fibrechannel is just another connector.
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.