Slashdot Mirror
Hardware IDE/SCSI RAID for Windows 2000 Servers?
reezle asks: "Mostly I was wondering what other sysadmins have been doing for Mirroring or RAID-5 in their w.2000 servers. I really don't like the M$ 'Enhanced' disks that allow for RAID, since I've actually lost a volume during the conversion from 'basic' to 'enhanced', and also I worry that I will get locked out of the volume if the OS goes belly-up on me. There is also the idea that software RAID is much slower, but it's cheap, and so are some of my customers. What kinds of solutions are being used successfuly? What kind of recovery nightmares have people run into? Is IDE RAID ready for the real-world server market yet?"
Seriously, Ask Slashdot question for this??
Toms Hardware IDE RAID review
IDE RAID without hardware
Exercise left up to reader: Finding SCSI RAID reviews
What would it cost for your company to reconstruct the lost data?
Risk Analysis argument over!
Seriously, the Win2K volume management and "enhanced disk format" you worry about are a subset of the Veritas VM, licensed by MS. It's crippled without many of the data-recovery features, and doesn't include the file-system enhancements.
When you convert a Windows volume from "Basic", you are essentially performing the same operation as "Encapsulating" a native volume with Veritas on Solaris or HP/UX.
"Flyin' in just a sweet place,
Never been known to fail..."
First - 3ware makes an excelent line of IDE hardware raid cards if your too cheap for SCSII.
Secondly - Windows Software raid will blow up in your face - especially Microsoft's version. If 'lost' three RAID arays to Promise, and four to Microsoft before getting a clue and forever forsaking crappy software. Windows Software RAID sucks so hard, that even if they fix it now, it's suckyness will caryover for years.
So you really have two choices for Windows RAID - SCSI or 3Ware.
Aside: Too bad Microsoft and Promise are too stupid to review NetBSD's RAIDframe - this is software RAID done right. Totally abuseable - you can pull out an IDE cable and it just keeps chugging along. Easy to set up as well - no guessing if it's going to work, it just does.
Moneyed corporations, non-working 'poor' and criminal prisoners are turning productive citizens into tax-slaves.
$300 will buy you a decent 3ware Escalade 7410 card, which comes with both Windows and Linux support.
Promise IDE RAID is a lot cheaper, but unreliable; I would get kernel trap exceptions all the time and it wasn't worth the trouble. Asides from a problem setting it up where the onboard motherboard ATA-100 driver was conflicting with the 3ware card, I haven't had so much as a hiccup. There's an erroneous report that says they only work on 64 bit PCI, but they work fine on 32 bit as well.
With CPU speeds being what they are, IO is really the bottleneck in your average computer. I've seen dramatic since the card went in -- I'd guess compilation time has halved.
If you're starting fresh, see if you can get a Tyan motherboard and 64 bit PCI and you should have no problems for the forseeable future.
Software RAID-1 is actually quite fast. In my benchmarks it is as fast or faster than a hardware RAID-1 solution on Linux. I'd expect that MS's implementation performs similarly. It is very cheap to implement :)
If you're doing raid 5 (or 10) you could benefit from more horsepower. You have a few options. They are (in rough order of cheaper cost --> better performance and reliability)
- Buy faster CPUs to make up for the overhead of software RAID-5 or RAID-10. They will still not be as fast as a hardware solution, and it might be a real pain to deal with in a disaster situation. Make sure you have lots of backups.
- Use the 3ware 7850 card to get you cheap IDE RAID-5. Obviously the benefit of this is that you can save a ton of money on disks. In my experience the card performs reasonably well and is stable, but I have to admit I've only been using it for non-critical fileservers over the past 6 months. It may not be a mature solution for all uses.
- Buy a classic SCSI hardware raid card (like a Mylex AccellaRAID 320) with a large battery-backed RAM cache. This type of card will give you the highest performance, and you can safely enable write caching as well, which will tremendously improve your RAID-5 write performance if that is the RAID level you want to use. It's a rock solid, but expensive solution when you count the cost of the scsi drives.
Some pitfalls:
Don't use IDE (hardware or software) RAID with Promise controllers. I don't really have any proof, just lots of annecdotal first and second-hand reports of craptapular performance and instability.
There are Windows drivers, Linux drivers, and the FreeBSD kernel also has a driver (twe) for it too. You can also find the management software for FreeBSD, though not through 3ware.
Beware, Nugget is watching... See?
I use Promise FastTrack RAID controllers in a mirror configuration (two drives, one on each cable) in 15 Novell servers. I have both 66 and 100TX2 models in service (most of them for over a year) with no problems.
I also used one on my workstation (striped, two 7200RPM 20GB drives) for the better part of last year and it sped up my computer substantially with no problems.
When a server has gone down then usually both hard drives have good, valid data. When one hard drive goes down the other keeps trucking until I replace it (offline - I didn't get hot-swap enclosures, it happens so infrequently it's not worth it).
So for the low end (ie, CHEAP) hardware RAID from Promise is right on the money. If you want something without such bad anecdotal evidence (as attested by other posts in this story) then you will have to pay more.
As always, your customers get what they pay for. So far my company's investment has paid off over and over again - I don't have to recreate the entire server from the ground up (or from a backup) when at least one hard drive is good. I've had to replace 4 servers in the last year and one or both hard drives have always survived whatever caused the server to go down. (These are low usage, but physically punished servers)
-Adam
The real answer is to use server class hardware from a real supplier. Both Dell and Compaq have rather solid server offerings with hardware raid built in. No need to worry with MS's implementation. And if you're doing RAID 5, you really should be in hardware anyway, or performance will kill you.
If this is for home use, or for fun, then play all you want. But since you're paying for windows anyway, you might as well poney up for real hardware. Your life will be happier for it.
Zapman
If you're working for a company just get a decent server with hardware RAID. Dell's servers use Perc2 or 3 cards which can support upto a metric buttload of drives (12-16? not including 2-4 external connectors). Win2k can use all of the cards nicely and will let the controller do its thing. Most Linux distros come with drivers for them too if you want to make sure you aren't stuck with win2k.
Poweredge 2550's can carry 4 drives and are fairly cheap by company standards (~$5k decked out).
Notice: I do not work for Dell, but I am a Windows admin for a company that does buy from Dell
The following pages provide information about ATA RAID and hot swapping:
- Adaptec 2400A - FAQ
- 3Ware 7500-series controller - Datasheet
- 3Ware ATA Drive Cage - Product Specs
- Promise SuperTrack SX6000 - Datasheet
There are other products out there that do support ATA RAID and do provide hot swap facilities and capabilities.It supports online capacity expansion, hot-spare and hot-swap (chassis required), and all major operating systems.
Just look for hardware RAID cards with Windows 2000 compatibility. A monkey could do this for you, if you feel like training the monkey for a few hours.
Here is what my monkey found.
"Remember when the U.S. had a drug problem, and then we declared a War On Drugs, and now you can't buy drugs anymore?"
There have been some bumps with the hardware, but 3Ware has been responsive to our bug reports, and our current revs of the drivers/firmware have been solid. The drivers have been incorporated into the main kernel, and you can download the latest driver from 3Ware's site.
The 3dm software is great for managing and monitoring the arrays. It has a web interface, email notification, and SNMP access (I think).
Give them a fair evaluation and you should be impressed.
Method of processing duck feet
You know, I have seen lots of people say the exact same things as you. "IDE is so bad! Never use it!" "Use brand $x, it's the only REAL brand!" and overall, "Spend more money than you want to"
I'd like to respond in general to these things.
#1) IDE bad SCSI good.
The most common argument I hear is because of CPU resources. Now let's think about this. We'll go with the largest drive that each interface has. SCSI: 181GB @ @1000 IDE: 160GB @ $222 That is a price difference of almost $800 for $800, you can buy yourself TWO intel 2.4GHz processors. So if you arn't already running the fastest processor out there, you'd be better off (price-wise) getting IDE and purchasing a faster processor (or two, or whatever) This result is even more valid if you have more drives. (bigger savings) Quality of the drives? In many cases, they are the exact same drive with different electronics attached to them. The quality is the same. Also, there are IDE raid cards that have their own CPU. But you can just do software raid with the faster CPU. BTW, people: RAID does NOT improve performance. It hurts it. Read some benchmarks if you don't believe.
#2) ALWAYS buy the best you can afford.
I've got 4 servers that were the top of the line, most reliable hardware that are 5 years old. They are all working just fine. They cost $8k each back then. I've also got about 10 desktop computers flipped on their side, with 'server' written on them in crayon. They were about $2k each. They all still run just fine. If we had purchased all of them as desktops, I could have paid myself $24k extra. That money was wasted. Sometimes, (very seldom) it pays to buy the best. But if something is redundant anyway, get cheap! If it breaks, replace it. You've still saved the money. If it can be down, just keep it backed up, and buy cheap. You'll save money (a LOT of money) in the long term.
My basic idea here is that spending more money isn't always the best thing to do. Yeah, it's a lot more fun to play with a new Sun220R than a used P450 desktop from "Mikes Computers" but with a $10k price difference, there needs to be some VERY good reason to buy the expensive stuff.
Nathan Brazil?
There are also ATA drive chassis available that have some onboard electronics that allow the drive, mounted in the chassis, to be hotswapped into the appropriate recepticle, although I am not as familiar with these as I am with the SCSI drives (I have 3 of them in my desktop machine that I built and learned a great deal about SCSI in the process of getting everything working).
UNDER NO CIRCUMSTANCES SHOULD ANY DRIVE, ATA OR SCSI, BE "HOT SWAPPED" IF IT PLUGS DIRECTLY INTO THE BUS AND POWER SUPPLY. THIS WILL RESULT IN DAMAGE TO YOUR DRIVE AND QUITE POSSIBLY YOUR DRIVE CONTROLLER.
Friends don't let friends use multiple inheritance.
As an interface, the difference between SCSI and IDE is small. Yes, Scsi has a few more controllability and asynchronous features, but these are not a big deal. The difference is that the manufacturesers use Scsi is a marker for a generally higher level of build quality and testing. Just as PCs marketed as servers are built better than desktop workstations, SCSI drives are simply better built than IDE ones. The price difference is not the trivial/zero cost of the different interface, it is better bearings, stronger actuators, more rigid cases, bigger buffer rams, cleverer firmware, extra levels of ECC, more vibration testing and so on. Check the MTBF figures - when I last looked, SCSI drives had 5 times the MTBF of comparable IDE drives from the same manufacturer. Basically, IDE is designed down to minimum cost for the cutthroat desktop/home market, while Scsi is designed up to beat the competition in the less price sensitive server market. [Most of this derived from talking to the tech support of a major disk manufacturer]
Which means that if you really, really, want your data to stay there, the delta of SCSI is probably worth it. OTOH, I would go for Raid-ed IDE before non-Raide-ed SCSI - drives fail, even the best.
There is no technical reason why IDE cannot be made host-swap - but not in an ordinary PC case. You need a mounting enclosure designed to make/break contacts in the right order, and a controller designed for hot swap. These cost money, and people tend to put that money alongside the premium features already in Scsi rather that the minimum-cost IDE.
Consciousness is an illusion caused by an excess of self consciousness.
I'm not a RAID hater. I run it on my home system, and 4 servers at work. But it's NOT for performance. RAID0 generally doesn't hurt performance much but RAID1 does. You have to write twice as much to disk! (and a lot of RAID cards actually don't 'mirror' they do parity for even one drive. RAID1 and RAID3 end up being the same thing. (saves on programming costs)) RAID5 ALWAYS hurts performance.
Any combo-raid (10, 0+1, 15, 51) hurts performance. I hate to tell you this, but you are simply wrong.
Yes, there are 10k and 15k SCSI drives, while IDE has no models over 7200. But there are MANY 7200RPM SCSI drives. Do some research sometime. Please.
For your information, servers are not there just to have lots of hardware. They are there to *do* something. If your application doesn't require more than 2 drives (and at 160GB for IDE drives, that's a pretty hefty chunk of data) then who cares about the 2 drive limit? Also on those lines, who cares about external connectors when all of your drives are internal? Why pay for features that you arn't going to use? Do all of your servers have GeForce 4 4600 cards in them? What about sound cards? Why not? You want external SCSI connectors on them that you won't use, so why not add other things you won't use.
The maximum curent transfer rate on IDE is 133MB/s, and SCSI is 320MB/second sounds amazing, right? Well, the drives themselves are around 55MB/second average maximum sustained transfer rate. (this is on the Seagate Cheetah X-15, the 15000RPM drive) You arn't saturating the bus with this. Yes, there is cache on the drive, and you can burst much faster, but If you ever do anything over 4MB in size, you are out of cache, and are dealing with the sustained transfer rate. On a 160GB drive, in order to ignore the sustained transfer rate, you'd need to have 40k files minimum.
I know exactly how to create and run a proper high availability server setup. I've done it multiple times. I also know how to not waste money on features and equipment that arn't needed. (which you do not) If you need the features, and you need the speed of SCSI (Yes, it *is* faster, I never said otherwise) then go for it. Spend as much as you need! But if you don't, you can save a lot of money buying IDE, or other things. (perhaps a second machine) I'd be interested to see a comparison of a single large SCSI RAID webserver with severral smaller IDE webservers.
What you shoould have gotten from this whole message is that when purchasing hardware (esp. disk drives) you really need to think about what the hardware is going to do, and buy hardware based upon requirements. People who automatically go out and buy the best they can are in part responsible for many companies going broke. Before I learned this, I spent $750k on hardware at my last start-up. We never used more than 1% of the capabilities of that equipment. I could have (should have) purchased the *right* equipment, not the *best* equipment, and saved half a million dollars.
Nathan Brazil?
Someone recently accused me of looking for the trolls on /. and responding to them.
:) I guess I'm questioning your concept of "top of the line".
Guilty as charged!
BTW, people: RAID does NOT improve performance. It hurts it. Read some benchmarks if you don't believe.
RAID is faster on reads than writes. It depends on your application, but most servers tend to do more reads than writes anyway. But the prime motivating factor usually is improving reliability of the server as a whole.
I wouldn't recommend any RAID system which didn't have a hot-spare and the drives weren't hot swappable. Might as well keep your server up and running when you have a drive fail, right?
I've got 4 servers that were the top of the line, most reliable hardware that are 5 years old.
I've got a server like that under my desk. It's an older production system that they were throwing out because it was no longer under a maintenance agreement from Compaq. I now use it for testing different software configurations.
They cost $8k each back then.
Oh, I'm afraid my Proliant 5000 with quad PPro-200, 1 Gig of RAM and five 9.1 Gig 10k drives with a Smart array controller cost quite a bit more than $8k in it's day. My guess is more like $40k. It's worth maybe $1000 today, how's that for depreciation!
Companies and people tend to be cheap when they start out, but as you grow, at some point you need to mature and get past that tendency. I used to have the same attitude as you, but numerous failures have taught me a lesson.
I have a Highpoint Rocket Raid 100 in my stereo, working fine so far, with 4 drives in 0+1 config for 160 Gb under NT 4sp6a.
Haven't (yet) tried to replace one of the drives, but the initial install was ez.
Cheap customers hmmm...
Give me a second, I'm having trouble with the concept of Windows(AnyVer) and reliability.
Nope, can't do it 2K is more reliable than Win3x or Win95 or Win98... But I just can't put reliability with it.
Have you considered using a *nix system for backup of the Win system?
Cost being a deciding factor, the trade is going to have to be in potential loss of data,
depending on the load you could give the customer tiered options for recovery of data.