Slashdot Mirror
Dependable SCSI RAID Controllers for Linux?
"I have been considering ICP Vortex RZ and RS series and AMI Megaraid as possibles, along with the Mylex line of controllers. I would like some opinions, praises and even nightmare stories on any of these. I am not wanting to invest $350-$1500 per controller on another nightmare like Adaptec/DPT line. It should be obvious but cost is not primary, reliability and to a lesser degree performance are the key issues. In addition I run my controllers in RAID 5 with a hot spare, so suggestions should be for controllers that can do that RAID mode and ones that can be administered from a running Linux system so I can do hot swapping. I would also like controllers whose manufacturer keeps current patches available for the stock kernel tree or is in the kernel tree (for both 2.2 and 2.4, I use 2.2 mostly due to issues with 2.4) as I never use a canned kernel after the install is done. If you run Windows or some other truthfully Adaptec supported OS look for a few *good* DPT or Adaptec controllers on eBay when the swap-out is all over."
I would blame a really large portion of Slashdot's downtime, and the recent downtime with Freshmeat, on those controllers. Outside of a Megadrive that I used at the Virtual Hospital, those are probably some of the worst pieces of Hardware I have ever ran into.
I would never recommend that anyone ever use those cards. Flaky hardware is one issue, but those cards have consistently been the root of a lot of sleepless nights for me fixing the mess that they have caused.
You can't grep a dead tree.
The parent post would seem to be an opinion, and perhaps even a nightmare story, about the Mylex line of controllers. Way to read, read-boy.
Let me be the first to say "I don't give a fuck!" and "Please shove hot grits down the front of my pants please!"
One wank or two?
Fat people need sex too!
StorageReview officially confirms: SCSI is dying
Yet another crippling bombshell hit the beleaguered SCSI community when recently IDC confirmed that SCSI accounts for less than a fraction of 1 percent of all drives. Coming on the heels of the latest StorageReview survey which plainly states that SCSI has lost more market share, this news serves to reinforce what we've known all along. SCSI is collapsing in complete disarray, as further exemplified by failing dead last in the recent Sys Admin comprehensive networking test.
You don't need to be Davin Shearer to predict SCSI's future. The hand writing is on the wall: SCSI faces a bleak future. In fact there won't be any future at all for SCSI because SCSI is dying. Things are looking very bad for SCSI. As many of us are already aware, SCSI continues to lose market share. Red ink flows like a river of blood. UltraSCSI is the most endangered of them all, having lost 93% of its users.
Let's keep to the facts and look at the numbers.
SCSI leader Eugene Ra states that there are 7000 users of SCSI. How many users of UltraSCSI are there? Let's see. The number of SCSI versus UltraSCSI posts on Usenet is roughly in ratio of 5 to 1. Therefore there are about 7000/5 = 1400 UltraSCSI users. fibre channel SCSI posts on Usenet are about half of the volume of UltraSCSI posts. Therefore there are about 700 users of fiber channel SCSI. A recent article put SCSI2 at about 80 percent of the *BSD market. Therefore there are (7000+1400+700)*4 = 36400 SCSI users. This is consistent with the number of SCSI Usenet posts.
Due to the troubles of SCSI, abysmal sales and so on, Western Digital went out of business and was taken over by Seagate who sell another troubled Drive. Now Seagate is also dead, its corpse turned over to yet another charnel house.
All major surveys show that SCSI has steadily declined in market share. SCSI is very sick and its long term survival prospects are very dim. If SCSI is to survive at all it will be among RAID hobbyist dabblers. SCSI continues to decay. Nothing short of a miracle could save it at this point in time. For all practical purposes, SCSI is dead.
FACT: SCSI is dead, long live ATA
an LSI Megaraid should work quite nicely in Linux, I have an older AMI Megaraid Enterprise 1200 (model 428) with a couple of small disks as my linux disks (and one of them is my windows swapfile :)
never had any problems with it whatsoever.
We have two IBM Netfinity servers that use IBM ServerRAID 3L. The cards are not that great as they only have 4 MB of cache, but they run reliably under 2.4.13.
The drivers are maintained in the kernel, so there is now patching or downloading of drivers.
I think IBM has other models that come with more cache, so you could try calling them.
Does anyone have any thoughts about IDE raid, especially the offerings from Promise Technology? They've got cards that do RAID 5 with regular IDE drives, including hot failover capability. They've also got subsystems that put a full 8 disks into a RAID array, but presents it to the controller as a single SCSI device.
Advantages: Cheap drives.
Disadvantages: Speed, maybe, though since it's all going directly into the PCI bus, I'm not sure this is an issue.
Anyone used these? Comments? I figure with their SuperTrax controller and a bunch of 80 or 100-G drives, you could have half a terabyte in your basement for under two grand.
Don't discount the Compaq line of SmartArray controllers. I've been using one for 2 years without a hitch. Supports everything you need them to do (I'm using the Smart/2P controller in my server). Never had a single problem with it. You can find these on eBay really cheap too.
You can accomplish anything you set your mind to. The impossible just takes a little longer.
As always, your mileage may vary...
I'm having a lot of success with my Adaptec 29xx (2940 for SE like CD or external SE device, 2944 for LVD) and 39xx series cards. We don't use anything else in any of our operating systems (unless they are built-in to a motherboard). Granted, I'm not stressing my systems 24 hours a day... more like a few hours spread out over a regular business day.
I'm sure there are plenty who will readily disagree, but I don't think I've found, end-to-end, better hardware for SCSI controllers. Sure, getting the AAA-133 RAID controllers to work can be a challenge, but we've been nothing but happy with the rest.
We also have a lot of success with Mylex RAID controllers on several critical production boxes, though those are not *nix machines (NT 4.0 SP6).
fwiw, we pulled the DPT cards we have and replaced them with Adaptecs.
Amateurs discuss tactics. Professionals discuss logistics.
Compaq's SMART array controllers work well under Linux, and support RAID 0, 1, 5, and 0+1 with hot spares. I have been running RedHat 7.1 on one of these for months, in a production environment, with great success.
I work for a medium sized web hosting company which sells dedicated/managed servers to customers. We will only put ICP Vortex cards in them. These are the only cards we put in our own servers as well, I would say we have at least 40 of them in our datacenter and they work great. Not to mention if a drive fails you can easily hear them beeping from outside the datacenter, even with all the server/air conditioner noise.
Great cards, great speed, and a not so bad price. They work flawlessly in Linux and Windows.
Open Source Time and Attendance, Job Costing a
I have used Mylex RAID controllers very successfully with the older kernels on Linux (I run debian-stable) and on FreeBSD. Hot-Swap worked fine, and the on-board BIOS could be used for all configurations, plus there was adequate information from the kernel on RAID state. So, unless they have become significantly worse recently, I would at least consider Mylex.
But, you might want to consider one of the alternatives like RaidTec or its ilk. These are large boxes with RAID controllers built in and capacity for a fair number of disk enclosures. The RaidTec, for instance, can take 512GB+ (maybe 768GB+ now) and has options for redundant controllers, either fiber channel or SCSI. Just shows up as drive space. I haven't yet had a RaidTec unit up with Linux, but they claim it's fine. There are many others, with the EMC units being at the top of the cost heap.
Since cost is not your primary worry, can you run a test? Get one of each, configure each with several harddrives and see what happens to each under load. While there is a large difference between the way system behave when running 50GB over 7 harddrives and the way systems work when running with 50TB in a real raid system, anything that doesn't work with the smaller systems will fail in the large ones too.
we just put together a system here at work with a dual athlon xp tyan board and a 64bit adaptec raid card rinning raid 5 + 1 hot spare, and have not had any trouble what soever suse 7.3 detected it all on its own no driver disk needed. I am not sure of the exact model number (and iam not going to go pull the cover off one of our main production servers) tho dmesg gives this Loading Adaptec I2O RAID: Version 2.4 Build 5. I dunno why this poster has had such trouble maybe he should upgrade his distro to suse :-)
PAGERANK++ Robsell.com
We just bought a system that has a usable Terabyte of disk space and payed about $7500 for it.
We're using 10 Maxtor 130gig drives on 2 3ware 7510 Controllers. We could still put 6 more drives on the two controllers.
I wonder if your hardware isn't going bad. I too run DPT SmartRAID controllers. A 2654U2 to be exact (2-channel version) in a 440LX P2/266 (we're network bound, not CPU bound) which is used for fileserving about 50 people in a file-heavy office environment. Before that, it was a SmartRAID V with the hardware cache/RAID card (which is in use ino another, heavier-hit webserver).
Zero stability issues on both. The 2654U2 has a 5-drive RAID5 + hot spare (UW2 SCA drives) on one channel and a 6x24 DDS-3 on another. I've done some pretty I/O intense things on this controller (including rebuilding the array during office hours) with no problems at all. This is on kernel 2.4.17. The SmartRAID V is on a 2.2.14 system which has about 50 colocated web and mail sites (it does a pretty good job of keeping the T1 busy). It runs a RAID1+0 array with really old Seagate Baracuda SCSI-1 drives and a single external DDS-3 for backup. Again, zero stability issues. I'd buy these again without hesitating.
Perhaps you need to delve deeper into the problem. The 2654U2 did not like the original P90 system the server used to be; we had bad issues there and the tech basically said that the original PCI spec was not good enough for the card. Upgrade the motherboard and all was fine. If you're running 2.4, make sure you're in the .16/.17 kernels, as earlier 2.4 kernels had issues with all manner of things, but not specifically the DPT I2O drivers, IIRC.
Both of these systems run the kernel drivers and use the dptutil software that DPT used to have (which you're right, has gone the way of the dodo after Adaptec's assimilation of DPT); what specifically can you do to cause problems? I don't think it's the card/drivers in general but if you give me a test or two I can run to see if I'm affected as well we might be able to fix this.
Slashdot's braindead lameness filter is not letting me post my dptutil -L output. Sorry.
While I'll agree that SCSI is superior for most applications, IDE is no slouch nowadays.
On one of our production servers we have twin 18 Gig 10krpm Ultrawide SCSI drives for the database, and a pai rof 80 Gig IDE drives for the static data like web content.
The pair of U2W SCSI drives in a RAID1 can be read at about 48 Megs a second by bonnie, while the pair of 80 gig IDEs can be read at about 28 Megs a second.
pgbench, a little benchmarking program for postgresql, gets about 150 to 200 transactions per second on the dual SCSI drives, while it gets about 100 to 120 on the dual IDE drives.
the problem is, even under it's heaviest loads, that machine never handles more than 10 or 20 transactions every second. Both sets of drives are plenty fast enough to hand the load.
For servers that need hundreds of gigabytes of storage but only have to provide static storage for a medium, to small group, the money you'd spend on SCSI is probably better spent on other options for that server.
For a database server handling hundreds of concurrent users, SCSI (via electrical cables) is a good choice, but maybe a SCSI over FC-AL setup would be needed.
Engineering isn't about which component is the absolute best, it's about which component makes the most sense for what you're doing.
--- It is not the things we do which we regret the most, but the things which we don't do.
I've been using a Mylex Acceleraid 150 for just about a year now with zero problems. The box
had about 235 days of uptime until I shut it down to add memory.
What precisely was your problem with Mylex?
Random kernel crashes, forgets about disks... just about every kind of error that I have seen.
You can't grep a dead tree.
I recently built a server with an Escalade 7410 (four drives, RAID 10) and I think 3ware made the right decision to go with the 64bit PCI support. It's a pretty sweet setup, and again, great support under Linux (which is what I use.) The 3ware Escalade line comes highly recommended from me, but there are a few things on my wishlist.
- Upgradable cache memory. Not just 1 or 2MB with the $100 difference between the 7x10 and 7x50 series that is standard.
- Different Port Configuration. I recently found that in a 4U server case (with a 7410) having 4 drives (stacked horizontally) with the vertically placed IDE connectors on the Escalade board itself a challenge to wire. Those wide parallel ATA cables just got in the way too much. I eventually choose to buy some very convienient rouded IDE cables (from Bigfoot Computing.)
- ATA/133 Support. I imagine this will come with time, but again, I'm looking forward to improved performance (I wonder, will they be called 8xx0s?)
I don't deny the very positive influence SCSI has had on the industry (recent experiences with 400Mbit/s IEEE 1394 based networking have been great, and 1394/Firewire/iLink is essentially a serial-based SCSI protocol and physical media) but I definately have reaped the benefeits of ATA/IDE based RAID.Pete
There are three great options to get your servers out of the RAID-controller business. One is NAS (Network Attached Storage), the second is using native SCSI or IDE controllers with RAID provided by your OS. And lastly, you can buy a box that already is a RAID but just looks like one big fat drive and plug it in.
:-)
i nd ex.asp?fileid=R28825
At work run all our linux boxen at work with kernel mirroring and it uses almost NO CPU even under pretty heavy parallel load. Great for the base OS with SCSI or IDE, since the only thing they'll do once they boot is swap to these. Striping your swap space across multiple drives really helps when a server starts running low on memory.
I have mirror sets running at 48 Megabytes a second on two year old 18 Gig 10k SCSI drives for streaming output, and can provide very good performance under parallel load as a database disk set.
I've never had the kernel RAID drivers act flakey since I started using them over two years ago, and I've done various things like hot insert a raid disk in both RAID 1 and RAID 5 (both were pretty easy to do.) and typed the respected, yet undocumented --really-xxxxx (xxxxx=a 5 letter word not mentioned here!) flag a few times.
A friend is in the process of building NAS servers in 2U units with multiple IDE cards and ~500 Gigs of storage for ~$3500 or so. SCSI versions would be a bit more, bigger, and probably need more cooling, but be faster too. Right now the IDE ones are fast enough with a RAID 5 configuration.
The IDE ones can flood a 100 Base-TX connection, so performance isn't really an issue for anything on less than gigabit, and even then the IDEs will use up a goodly chunk of that.
The external RAIDS are often the fastest for databases, offering fibre optic connections. they're not cheap, but if you're running EBay's database, cheap isn't the point anymore.
If you have to have a RAID card, I can recommend the AMI Megaraid 428, which used, on Ebay, goes for $100 right now. Not that fast (I never got more than 20 Megabytes a second from one) but very solid and reliable, and they can hold up to 45 SCSI hard drives if you can afford the cooling and electrical for them. Plus the first channel looks like a regular SCSI card to anything other than a hard drive, like a tape drive or CDROM, so you don't need another SCSI card if you want a tape drive to back it up.
While the Megaraid site no longer has configuration software available, this site:
http://domsch.com/linux/#megaraid
points to this site:
http://support.dell.com/us/en/filelib/download/
on Dell where you can find management software for the MegaRAID controllers.
--- It is not the things we do which we regret the most, but the things which we don't do.
In my albeit limited experience w. RAID controllers, you cannot buy a better controller than ICP Vortex. Support is good, and they make a solid product. And you pay for it!
IDE RAID is fine for workstation's and home use....as far as I am concerned, it has no business in a corporate server environment. Anyone who tells you differnet is shaving pennies, and hasn't a clue. Of course, your opinion may differ!
Maxtor (and Western Digital) are pretty bad these days. We quit using them in our servers at all awhile back (I work for a VAR). IBM and Seagate (with a couple of notable individual model exceptions) are the best. Quantum being middle-of-the-road of course, though that may change since Maxtor bought them a year or so ago.