Slashdot Mirror
Dependable SCSI RAID Controllers for Linux?
"I have been considering ICP Vortex RZ and RS series and AMI Megaraid as possibles, along with the Mylex line of controllers. I would like some opinions, praises and even nightmare stories on any of these. I am not wanting to invest $350-$1500 per controller on another nightmare like Adaptec/DPT line. It should be obvious but cost is not primary, reliability and to a lesser degree performance are the key issues. In addition I run my controllers in RAID 5 with a hot spare, so suggestions should be for controllers that can do that RAID mode and ones that can be administered from a running Linux system so I can do hot swapping. I would also like controllers whose manufacturer keeps current patches available for the stock kernel tree or is in the kernel tree (for both 2.2 and 2.4, I use 2.2 mostly due to issues with 2.4) as I never use a canned kernel after the install is done. If you run Windows or some other truthfully Adaptec supported OS look for a few *good* DPT or Adaptec controllers on eBay when the swap-out is all over."
I would blame a really large portion of Slashdot's downtime, and the recent downtime with Freshmeat, on those controllers. Outside of a Megadrive that I used at the Virtual Hospital, those are probably some of the worst pieces of Hardware I have ever ran into.
I would never recommend that anyone ever use those cards. Flaky hardware is one issue, but those cards have consistently been the root of a lot of sleepless nights for me fixing the mess that they have caused.
You can't grep a dead tree.
The parent post would seem to be an opinion, and perhaps even a nightmare story, about the Mylex line of controllers. Way to read, read-boy.
Well I don't post on usenet, I decided to ask slashdot. Perhaps SCSI users don't have time to read usenet, I know I have not read a newsgroup except in digest form in about 3 years, much less posted.
an LSI Megaraid should work quite nicely in Linux, I have an older AMI Megaraid Enterprise 1200 (model 428) with a couple of small disks as my linux disks (and one of them is my windows swapfile :)
never had any problems with it whatsoever.
So what is replacing it? I know you did not just suggest that IDE is actually worth anything.
I would really like to see you run servers on IDE.. HaHa. You are not just a troll, but a stupid one.
SCSI will not die any time soon. If it does, it will be replaced by Fibre Channel.. You couldn't pay me to use an IDE disk anymore, except maybe to boot legacy (x86) hardware with so I can boot from an NFS sever.. for a workstation.
For a server, IDE has no place. Half-duplex, cpu intensive, unreliable, do I need to say more? Oh, and incrediably limited in the number of disks. Raid5 array of IDE? yeah right. You can only have 8 IDE disks in a system, all of which use interrupts.
We have two IBM Netfinity servers that use IBM ServerRAID 3L. The cards are not that great as they only have 4 MB of cache, but they run reliably under 2.4.13.
The drivers are maintained in the kernel, so there is now patching or downloading of drivers.
I think IBM has other models that come with more cache, so you could try calling them.
fyi, you can have a couple ofo ddetail.html?prodkey=ASR-3410S&cat=%2fTechnolo gy%2fRAID%2fTwo+and+Four-Channel+SCSI+RAID">THE SE</a> is preferable.. especially for a server environment :)
but yeah, one of <a href="http://www.adaptec.com/worldwide/product/pr
well, THAT fucked up... links == SCSI RAID IDE RAID
Does anyone have any thoughts about IDE raid, especially the offerings from Promise Technology? They've got cards that do RAID 5 with regular IDE drives, including hot failover capability. They've also got subsystems that put a full 8 disks into a RAID array, but presents it to the controller as a single SCSI device.
Advantages: Cheap drives.
Disadvantages: Speed, maybe, though since it's all going directly into the PCI bus, I'm not sure this is an issue.
Anyone used these? Comments? I figure with their SuperTrax controller and a bunch of 80 or 100-G drives, you could have half a terabyte in your basement for under two grand.
Don't discount the Compaq line of SmartArray controllers. I've been using one for 2 years without a hitch. Supports everything you need them to do (I'm using the Smart/2P controller in my server). Never had a single problem with it. You can find these on eBay really cheap too.
You can accomplish anything you set your mind to. The impossible just takes a little longer.
As always, your mileage may vary...
I'm having a lot of success with my Adaptec 29xx (2940 for SE like CD or external SE device, 2944 for LVD) and 39xx series cards. We don't use anything else in any of our operating systems (unless they are built-in to a motherboard). Granted, I'm not stressing my systems 24 hours a day... more like a few hours spread out over a regular business day.
I'm sure there are plenty who will readily disagree, but I don't think I've found, end-to-end, better hardware for SCSI controllers. Sure, getting the AAA-133 RAID controllers to work can be a challenge, but we've been nothing but happy with the rest.
We also have a lot of success with Mylex RAID controllers on several critical production boxes, though those are not *nix machines (NT 4.0 SP6).
fwiw, we pulled the DPT cards we have and replaced them with Adaptecs.
Amateurs discuss tactics. Professionals discuss logistics.
Aight, sorry - I do agree with you, and have a stack of IBM DDYS disks to show for it - BUT, you said some things that were wrong:
Just about all PCI ide controllers can use DMA, which cuts the cpu intensiveness down to almost the scsi-level.
Further, Not all drives have to have their own individual interrupt - this depends on the ide chip and how they are arranged on the pci daughterboard, or on the motherboard. (interrupt sharing, etc.) Promise chips will use one interrupt for two interfaces.
SCSI does offer a whole slew of advantages, disconnect, command queueing, etc. These are advantages in a RAID setup. IDE does suck, but not because it's cpu intensive or gobbles interrupts.
fnord.
Compaq's SMART array controllers work well under Linux, and support RAID 0, 1, 5, and 0+1 with hot spares. I have been running RedHat 7.1 on one of these for months, in a production environment, with great success.
I work for a medium sized web hosting company which sells dedicated/managed servers to customers. We will only put ICP Vortex cards in them. These are the only cards we put in our own servers as well, I would say we have at least 40 of them in our datacenter and they work great. Not to mention if a drive fails you can easily hear them beeping from outside the datacenter, even with all the server/air conditioner noise.
Great cards, great speed, and a not so bad price. They work flawlessly in Linux and Windows.
Open Source Time and Attendance, Job Costing a
I have used Mylex RAID controllers very successfully with the older kernels on Linux (I run debian-stable) and on FreeBSD. Hot-Swap worked fine, and the on-board BIOS could be used for all configurations, plus there was adequate information from the kernel on RAID state. So, unless they have become significantly worse recently, I would at least consider Mylex.
But, you might want to consider one of the alternatives like RaidTec or its ilk. These are large boxes with RAID controllers built in and capacity for a fair number of disk enclosures. The RaidTec, for instance, can take 512GB+ (maybe 768GB+ now) and has options for redundant controllers, either fiber channel or SCSI. Just shows up as drive space. I haven't yet had a RaidTec unit up with Linux, but they claim it's fine. There are many others, with the EMC units being at the top of the cost heap.
Since cost is not your primary worry, can you run a test? Get one of each, configure each with several harddrives and see what happens to each under load. While there is a large difference between the way system behave when running 50GB over 7 harddrives and the way systems work when running with 50TB in a real raid system, anything that doesn't work with the smaller systems will fail in the large ones too.
we just put together a system here at work with a dual athlon xp tyan board and a 64bit adaptec raid card rinning raid 5 + 1 hot spare, and have not had any trouble what soever suse 7.3 detected it all on its own no driver disk needed. I am not sure of the exact model number (and iam not going to go pull the cover off one of our main production servers) tho dmesg gives this Loading Adaptec I2O RAID: Version 2.4 Build 5. I dunno why this poster has had such trouble maybe he should upgrade his distro to suse :-)
PAGERANK++ Robsell.com
We just bought a system that has a usable Terabyte of disk space and payed about $7500 for it.
We're using 10 Maxtor 130gig drives on 2 3ware 7510 Controllers. We could still put 6 more drives on the two controllers.
YHBT YHL HAND
I wonder if your hardware isn't going bad. I too run DPT SmartRAID controllers. A 2654U2 to be exact (2-channel version) in a 440LX P2/266 (we're network bound, not CPU bound) which is used for fileserving about 50 people in a file-heavy office environment. Before that, it was a SmartRAID V with the hardware cache/RAID card (which is in use ino another, heavier-hit webserver).
Zero stability issues on both. The 2654U2 has a 5-drive RAID5 + hot spare (UW2 SCA drives) on one channel and a 6x24 DDS-3 on another. I've done some pretty I/O intense things on this controller (including rebuilding the array during office hours) with no problems at all. This is on kernel 2.4.17. The SmartRAID V is on a 2.2.14 system which has about 50 colocated web and mail sites (it does a pretty good job of keeping the T1 busy). It runs a RAID1+0 array with really old Seagate Baracuda SCSI-1 drives and a single external DDS-3 for backup. Again, zero stability issues. I'd buy these again without hesitating.
Perhaps you need to delve deeper into the problem. The 2654U2 did not like the original P90 system the server used to be; we had bad issues there and the tech basically said that the original PCI spec was not good enough for the card. Upgrade the motherboard and all was fine. If you're running 2.4, make sure you're in the .16/.17 kernels, as earlier 2.4 kernels had issues with all manner of things, but not specifically the DPT I2O drivers, IIRC.
Both of these systems run the kernel drivers and use the dptutil software that DPT used to have (which you're right, has gone the way of the dodo after Adaptec's assimilation of DPT); what specifically can you do to cause problems? I don't think it's the card/drivers in general but if you give me a test or two I can run to see if I'm affected as well we might be able to fix this.
Slashdot's braindead lameness filter is not letting me post my dptutil -L output. Sorry.
While I'll agree that SCSI is superior for most applications, IDE is no slouch nowadays.
On one of our production servers we have twin 18 Gig 10krpm Ultrawide SCSI drives for the database, and a pai rof 80 Gig IDE drives for the static data like web content.
The pair of U2W SCSI drives in a RAID1 can be read at about 48 Megs a second by bonnie, while the pair of 80 gig IDEs can be read at about 28 Megs a second.
pgbench, a little benchmarking program for postgresql, gets about 150 to 200 transactions per second on the dual SCSI drives, while it gets about 100 to 120 on the dual IDE drives.
the problem is, even under it's heaviest loads, that machine never handles more than 10 or 20 transactions every second. Both sets of drives are plenty fast enough to hand the load.
For servers that need hundreds of gigabytes of storage but only have to provide static storage for a medium, to small group, the money you'd spend on SCSI is probably better spent on other options for that server.
For a database server handling hundreds of concurrent users, SCSI (via electrical cables) is a good choice, but maybe a SCSI over FC-AL setup would be needed.
Engineering isn't about which component is the absolute best, it's about which component makes the most sense for what you're doing.
--- It is not the things we do which we regret the most, but the things which we don't do.
I've been using a Mylex Acceleraid 150 for just about a year now with zero problems. The box
had about 235 days of uptime until I shut it down to add memory.
What precisely was your problem with Mylex?
Random kernel crashes, forgets about disks... just about every kind of error that I have seen.
You can't grep a dead tree.
I recently built a server with an Escalade 7410 (four drives, RAID 10) and I think 3ware made the right decision to go with the 64bit PCI support. It's a pretty sweet setup, and again, great support under Linux (which is what I use.) The 3ware Escalade line comes highly recommended from me, but there are a few things on my wishlist.
- Upgradable cache memory. Not just 1 or 2MB with the $100 difference between the 7x10 and 7x50 series that is standard.
- Different Port Configuration. I recently found that in a 4U server case (with a 7410) having 4 drives (stacked horizontally) with the vertically placed IDE connectors on the Escalade board itself a challenge to wire. Those wide parallel ATA cables just got in the way too much. I eventually choose to buy some very convienient rouded IDE cables (from Bigfoot Computing.)
- ATA/133 Support. I imagine this will come with time, but again, I'm looking forward to improved performance (I wonder, will they be called 8xx0s?)
I don't deny the very positive influence SCSI has had on the industry (recent experiences with 400Mbit/s IEEE 1394 based networking have been great, and 1394/Firewire/iLink is essentially a serial-based SCSI protocol and physical media) but I definately have reaped the benefeits of ATA/IDE based RAID.Pete
There are three great options to get your servers out of the RAID-controller business. One is NAS (Network Attached Storage), the second is using native SCSI or IDE controllers with RAID provided by your OS. And lastly, you can buy a box that already is a RAID but just looks like one big fat drive and plug it in.
:-)
i nd ex.asp?fileid=R28825
At work run all our linux boxen at work with kernel mirroring and it uses almost NO CPU even under pretty heavy parallel load. Great for the base OS with SCSI or IDE, since the only thing they'll do once they boot is swap to these. Striping your swap space across multiple drives really helps when a server starts running low on memory.
I have mirror sets running at 48 Megabytes a second on two year old 18 Gig 10k SCSI drives for streaming output, and can provide very good performance under parallel load as a database disk set.
I've never had the kernel RAID drivers act flakey since I started using them over two years ago, and I've done various things like hot insert a raid disk in both RAID 1 and RAID 5 (both were pretty easy to do.) and typed the respected, yet undocumented --really-xxxxx (xxxxx=a 5 letter word not mentioned here!) flag a few times.
A friend is in the process of building NAS servers in 2U units with multiple IDE cards and ~500 Gigs of storage for ~$3500 or so. SCSI versions would be a bit more, bigger, and probably need more cooling, but be faster too. Right now the IDE ones are fast enough with a RAID 5 configuration.
The IDE ones can flood a 100 Base-TX connection, so performance isn't really an issue for anything on less than gigabit, and even then the IDEs will use up a goodly chunk of that.
The external RAIDS are often the fastest for databases, offering fibre optic connections. they're not cheap, but if you're running EBay's database, cheap isn't the point anymore.
If you have to have a RAID card, I can recommend the AMI Megaraid 428, which used, on Ebay, goes for $100 right now. Not that fast (I never got more than 20 Megabytes a second from one) but very solid and reliable, and they can hold up to 45 SCSI hard drives if you can afford the cooling and electrical for them. Plus the first channel looks like a regular SCSI card to anything other than a hard drive, like a tape drive or CDROM, so you don't need another SCSI card if you want a tape drive to back it up.
While the Megaraid site no longer has configuration software available, this site:
http://domsch.com/linux/#megaraid
points to this site:
http://support.dell.com/us/en/filelib/download/
on Dell where you can find management software for the MegaRAID controllers.
--- It is not the things we do which we regret the most, but the things which we don't do.
In my albeit limited experience w. RAID controllers, you cannot buy a better controller than ICP Vortex. Support is good, and they make a solid product. And you pay for it!
IDE RAID is fine for workstation's and home use....as far as I am concerned, it has no business in a corporate server environment. Anyone who tells you differnet is shaving pennies, and hasn't a clue. Of course, your opinion may differ!
SCSI is SCSI wether it be fibrechannel or the 50/68 pin connector.
Fibrechannel is just another connector.
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
Maxtor (and Western Digital) are pretty bad these days. We quit using them in our servers at all awhile back (I work for a VAR). IBM and Seagate (with a couple of notable individual model exceptions) are the best. Quantum being middle-of-the-road of course, though that may change since Maxtor bought them a year or so ago.