Slashdot Mirror
IDE RAID Examined
Bender writes "The Tech Report has an interesting article comparing IDE RAID controllers from four of the top manufacturers. The article serves as more than just a straight product comparison, because the author has included tests for different RAID levels and different numbers of drives, plus a comprehensive series of benchmarks intended to isolate the performance quirks of each RAID controller card at each RAID level. The results raise questions about whether IDE RAID can really take the place of a more expensive SCSI storage subsystem in workstation or small-scale server environments. Worthwhile reading for the curious sysadmin." I personally would love to hear any ide-raid stories that slashdotters might have.
I ran an IDE RAID, one of the first, a few years ago. It was a 3ware RAID-1 controller. I thought it would be useful because I had gotten sick of losing data on a drive failure. I didn't have the money (or patience :) for a good backup solution and Linux RAID hadn't matured.
... never have I seen 2 drives go down simultaneously. Nor have I seen a controller malfunction in a way that damaged the drives (though I've heard of it from other people).
Everything was fine for awhile. After a few months I lost a drive, replaced a drive and it remirrored fine. Same thing happened a year or so later.
Then one day my controller fried. Nothing else in the system went down, but some kind of surge hit the 2 drives from the RAID controller. The controller still worked but neither drive was accessible, either as RAID drives or as single drives. Tried numerous tricks, eventually gave up.
I've run SCSI RAID in boxes I admin at work
All in all, I decided it wasn't worth it. I am currently doing Linux mirroring in combination with journaling filesystems on one box, and Windows mirroring on another.
It is more productive to voice thoughtful opinions (reply) than to judge (moderate) others.
I don't quite understand where this Inexpensive crap came from. RAID was around long before IDE RAID controllers started showing up and of course SCSI RAID arrays almost always use very expesive disks. It's Redunant Array of Independent Disks, always has always will be.
For performance reasons, I haven't seen a single vendor that actually expects you to put two drives on a single interface, and infact, I've found that the 3ware Escalade controllers just won't let you. When they advertise that it can two drives, it usually means it has two dedicated interfaces, therefore have the potential for completely saturated a single port all by itself (which is hard to do with ATA/100 and ATA/133 drives that cannot even burst that high -- get good drives! big caches too!)
Pete
There is also the reliability factor. SCSI drives tend to be more robust.
Brevity is the soul of wit
-- Polonius
I work in a 24x7 shop and we've been using the 3Ware Escalade series as well (using Slackware here). They're really nice: the drivers are in the kernel - none of this add a binary driver that might or might not work with later kernel updates stuff. Painting oneself into a corner is for the birds.
Oh yeah, I used to have this configured in RAID-5 mode, and had a drive fail. It took just about 24 hours for 120GB to reconstruct onto a spare disk.
I was fortunate enough to have also purchased a Fry's Electronics Instant Exchange guarantee for all the hard drives. So I popped in to Fry's to exchange it, and got a replacement after waiting for two fricking hours. I swear the poor guy had to run around for 20 different manager signatures.
Fry's Instant Exchange is not so instant.
Adaptec 2400A - $350
Two 3ware Hotswap drive bays - $340
Four 120GB western digitals (7200rpm) - $920
With Linux 2.4.9-SGI-XFS, filesystem writes were pretty damn slow -- maybe 12MB/sec on RAID-5.
You are wrong.
The orignal meaning was inexpensive.
When RAID was invented disk size was scaling up more slowly than demand and there was a huge price premium for the largest drives available. Economies of scale meant that smaller drives meant for the PC market were rather cheap, while larger drives remained very expensive. The epiphany of the RAID inventors was that since the price/storage unit was so much lower with smaller drives, it made sense to eschew large drives and stack multiple smaller drives together to achieve the same space with higher performance and lower price.
Thats because one of the major limitations of current generation IDE is that only one device on a channel can "talk" at a time. So even if you're using a RAID card with two devices on a channel, it will be no faster than a standard IDE connection, since only one drive read/write can be done at a time. With SCSI, all of the drives on a channel can talk at the same time until the 160 MB/s that SCSI can handle is saturated.
Most onboard RAId solutions and add-in cards under ~140$ are like this. You have to replace failed ones with cards using the same chipset in order to recover the data or use the array again. Onboard Promise and Highpoint RAID controllers have add-in counterpart cards that use the same chipset, and thus can be used to recover data if the on-board chip decides to die.
I don't quite understand where this Inexpensive crap came from. RAID was around long before IDE RAID controllers started showing up and of course SCSI RAID arrays almost always use very expesive disks. It's Redunant Array of Independent Disks, always has always will be.
It probably comes from the original reseach paper... A case for redundant arrays of inexpensive disks in the Proceedings of SIGMOD International Conference on Data Management, 1988. (Pages 109-116.) SCSI drives were an inexpensive option compared to other storage technologies that offered high performance and fail over safety.
Over time the acronym expansion was changed to become "redundant array of independent disks" as RAID become more popular (and affordable) for smaller systems.
Some references: here, here and here
I'm not the original poster but here goes. This is with a three 27gig drive Linux software RAID5 (md). This box has been up and running for about 2 years as my personal file server. All three drives are identical (Seagates, I'm too lazy to lookup model numbers)
/root> hdparm -t /dev/hda /dev/hda:
/root> hdparm -t /dev/md0 /dev/md0:
[10:40pm]
Timing buffered disk reads: 64 MB in 2.70 seconds = 23.70 MB/sec
[10:41pm]
Timing buffered disk reads: 64 MB in 1.40 seconds = 45.71 MB/sec
These are read tests of course but this is a mostly-read file server so that's what matters to me.
Also, it is a highly patched RedHat 6.2 install with 2.4.17 kernel, dual p2-400mhz, 1/2 gig of ram.
I would not suggest it to the average Joe, unless they run Windows...
/. will pay for your controller.
/dev/md0) than it did in Windows98SE/ME/2000/XP. And if anyone HAS used IDE RAID on Windows every last one of the controllers have the same problems. They CAN'T maximize the throughput without the lost of stability & reliability.
I've used IDE RAID since I purchased an Abit KT7-RAID. I've used the HighPoint HPT370 onboard as well as an Adaptec 1200A, the HPT-372/372A and now I'm using a Promise IDE RAID on an MSI KT3 Ultra.
With all the buzz words in this article for the techno geek, shuffle your buttocks to closest place to get one of these bad boys, spend that $100 bucks. If that controller benchmarks better than the SAME IDE DRIVES setup on your normal IDE controller using Linux RIAD..... well
I've went so far as to purchase 4 identical drives (as close as I could for your obssesive perfectionist boneheads). 4 Maxtor 40G ATA-100 7200 RPM drives, put them on the IDE RAID (RAID-0) controller and benchmarked them using SiSoft. Put the same drive on Mandrake 8.2/Redhat 7.3 and it showed higher drive throughput under Linux (using hdparm -Tt
For those who've lose multiple drives in IDE RAID, your victim to another reason IDE will never overtake SCSI in ANY insured business. HEAT. If you can't take the HEAT get the hell outta the real server market.
Those are the facts from someone who USES what the article "tests"
I've nothing of importance to say, now go away before I taunt you with a second sig!
I'm using IDE raid on my home desktop right now, but I'm using software raid as opposed to a hardware controller. I have two Seagate Barracuda ATA IV 40GB hard drives hooked up as masters to my primary and secondary motherboard IDE ports. I also have a DVD-ROM hooked up as secondary slave, and a Promise Ultra133TX2 controller with a CD-RW hooked up to its first port. Both hard drives are sectioned into a 3GB primary 1st partition and a 34GB (yes, the drives are only 40GB when you're in marketing land) 2nd primary partition. Windows 2000 is installed on the first drive's 3GB partition, and redhat linux 7.3 is installed on the second drive in the same place. Both OSs share the combined 68GB RAID 0 set, which is formatted with NTFS, made from the combined second partitions. The only problem is that linux can't write to the array because NTFS write support under linux is currently "DANGEROUS" according to the driver's author and I keep important data on there. (Yes, I know about the dangers of using RAID0 and I back up regularly.) It'd sure work a whole lot better if that driver were finished, though. (hint hint, Legato Systems, Inc.) ;)
:D After a quick format with NTFS (the partiton was too big to format with FAT32), I was in business.
/dev/md0
/dev/hda2 /dev/hdc2
/etc/raidtab, ran raid0run /dev/md0, and added a line to /etc/fstab. (I read online that WinNT 4.0's software raid driver uses 64K chunks.)
;) As a bonus, I also get to keep my standard partition table as well as compatibility with non-M$ disk editing/management/recovery tools.
Getting the two OSs' software raid drivers to play nicely together was an "adventure", mostly due to Win2K's insistance on turning the disks into "dynamic disks" before letting me use its built-in RAID functionality, meaning it wanted to wipe out my old partition table, replace it with a single partition taking up the entire disk, and create a new system of partition organization inside the dummy standard partition. After a lot of reading, I found out that Windows NT 4.0 supported "stripe sets" using standard partitions, and that Windows 2000, when installed over an old copy of NT4, would support the "legacy" software RAID drive. Windows 2000 would not, however, allow me to create new legacy stripe sets for compatibility with other OSs. Stupid Micro$oft. So all I had to do was fake Win2K into thinking it had been installed over an old copy of NT4 which had been using its stripe set functionality.
The first thing I had to do was create partitions. I opened up linux fdisk and allocated 3GB on each disk to my OSs, one for linux and one for windows, and created two partitions, each one taking up the rest of the space on its disk, and set their types to 87h (NT stripe set [thanks to whoever put the L command in linux fdisk!]). After installing Windows 2000 on the first disk's first partition, I needed to get my hands on a couple of tools that didn't come with windows 2000: Windows NT 4 Disk Administrator and MS's fault tolerant disk set disaster recovery tool, FTEDIT. After spending about 6 hours searching online, I finally found a download site for FTEDIT - MS's web site says you can get it free from them, but it provides no download link. NTDA was a bit easier. Since MS service packs replace OS files, and somewhere in NT4's history a bug or problem had been found in NTDA, that file was in the service pack 6a for NT4. Service packs check to see if you're using the correct OS _after_ they decompress themselves, and they're nice enough to display an error message telling you this ("Whoops. You just wasted a whole bunch of time downloading a huge file you didn't need. Sorry!") before they delete the decompression directory. Figuring that out took a while, but snagging the executable during decompression was easy.
I ran NTDA, which populated the "missing" DISKS key in the windows registry (Win2K stores disk information in a different place from NT4), and told FTEDIT that, yes, I really did already have a software RAID 0 set on those drives, and that windows NT had died on me and I had to restore it. After a reboot, "Drive D" appeared in my computer. 68GB and unformatted. YAY!
Getting linux to see the array was much easier. I added
raiddev
raid-level 0
nr-raid-disks 2
persistent-superblock 0
chunk-size 64
device
raid-disk 0
device
raid-disk 1
to
Btw, yes, I know linux has support for MS's dynamic disk scheme. I enjoy tweaking and doing new things, even if it means days spend reading about Windows.
"So," you're probably wondering, "why did Erpo spend all that time setting up a RAID0 set (presumably for extra performance) and then go and do a stupid thing like put a DVD-ROM drive on the same ata cable as one of the disks when he has an extra ata port on his add-in controller that he's not using?" Thanks for asking. It's because Promise's bios on the Ultra133TX2 card was broken. The company "Promised" me it would allow me to boot from CD, but in reality it only will let me do so when I want to boot from a windows installation CD. Not just any windows installation CD, either. It had to be Windows 2000 Professional or XP, which I refuse to use.
It wouldn't recognize my Windows 98 SE cd, or any of my linux distros. I didn't have a choice about the DVD drive if I wanted to install linux. Just now, months after I got the card and sent promise and email, they released a bios update that claims to fix the issue. If it works I'll be moving my optical drives around. Even with the DVD drive, the performance isn't too bad - about 80MB/sec at the beginning of the disk, and it slowly drops to 50MB/sec at the end.
Here are the hdparm results of the two machines I currently have that run Linux Software Raid.
/dev/md1 && hdparm -t /dev/md1 && hdparm -t /dev/md1 && hdparm -t /dev/md1 && hdparm -T /dev/md1 && hdparm -T /dev/md1 && hdparm -T /dev/md1
/dev/md1:
/dev/md1:
/dev/md1:
/dev/md1:
/dev/md1:
/dev/md1:
/dev/md1:
/]# hdparm -g /dev/md0 && hdparm -t /dev/md0 && hdparm -t /dev/md0 && hdparm -t /dev/md0 && hdparm -T /dev/md0 && hdparm -T /dev/md0 && hdparm -T /dev/md0
/dev/md0:
/dev/md0:
/dev/md0:
/dev/md0:
/dev/md0:
/dev/md0:
/dev/md0:
The first machine (brainstem) has SCSI Raid-5 with 18 GB drives. The second machine (heschl) has IDE Raid-5 with 120 GB drives. It's used to serve music and pictures (sorry, no pr0n - just digital camera pics) to my local network.
Machine 1 (SCSI)
-----------------
brainstem:~# hdparm -g
geometry = 58240/2/4, sectors = 141499392, start = 0
Timing buffered disk reads: 64 MB in 1.83 seconds = 34.97 MB/sec
Timing buffered disk reads: 64 MB in 1.83 seconds = 34.97 MB/sec
Timing buffered disk reads: 64 MB in 1.83 seconds = 34.97 MB/sec
Timing buffer-cache reads: 128 MB in 0.85 seconds =150.59 MB/sec
Timing buffer-cache reads: 128 MB in 0.86 seconds =148.84 MB/sec
Timing buffer-cache reads: 128 MB in 0.86 seconds =148.84 MB/sec
Machine 2 (IDE)
----------------
[root@heschl
geometry = 42304/2/4, sectors = 937765376, start = 0
Timing buffered disk reads: 64 MB in 3.14 seconds = 20.38 MB/sec
Timing buffered disk reads: 64 MB in 3.24 seconds = 19.75 MB/sec
Timing buffered disk reads: 64 MB in 3.19 seconds = 20.06 MB/sec
Timing buffer-cache reads: 128 MB in 1.09 seconds =117.43 MB/sec
Timing buffer-cache reads: 128 MB in 1.09 seconds =117.43 MB/sec
Timing buffer-cache reads: 128 MB in 1.10 seconds =116.36 MB/sec
Linux - Because Mommy taught me to Share.
Promise RAID is actually a software RAID. Don't let the fancy BIOS thiny fool ya. Here's a little story:
:) Those jokers are REALLY SLOW to recompile their kernel modules.
I had a Promise Ultra 100 controller in a system and loaded Linux on it. I tried to get the thing ot run RedHat 7.3. This was back when 7.3 was pretty new... like about 3 months old. I wanted to use the RAID controller since I had nothing else to do with it at the time and I knew Promise was supporting Linux.
Turns out they didn't have a driver for 7.3... just 7.2. I went around and around with them asking them to recompile the driver for me after I was mentioning the merits of oper sourcing drivers and such. Finally I gave up and bought some Maxtor PCI IDE controller out of the CompUSA bargain bin for like $10. It looked familiar...
I pulled the Promise card out and was about to put the Maxtor card in when I realized they were both the SAME DAMNED CARD! It was then I realized the RAID controller depended on software and not some fancy hardware thing. It was then I understood why Promise doesn't want to open source their driver!
Anyway, I put the bargain controller in and used a Linux software RAID. Short end to the story. I got my RAID and it worked. Better end, it was software and I could configure it using Webmin! That probably what you should do. Let the system see the two drives and then do the Linux software RAID... there is a redhat 7,3 driver out on the promise website... but I don't see a redhat 8.0
"I seriously doubt IDE Raid will ever overtake SCSI in any area where knowledgeable people are doing the administration."
Funny that go ask your or hell any sysadmin worth his or her salt what they use at home for redundant drives. Once Serial ATA comes in to broad adoption it will surely overtake SCSI based RAID.
3ware features.
At work we have 8x120GB 5400 RPM Maxtor drives set up as 7 raid 5 + 1 hot spare. The system is sitting in a 4U rack and just hums away 24/7. I think we've had it 5 month now.
We used bought this machine as a test replacement for some fibre channel arrays + dell poweredges the previous IT guy bought a year and a half ago. We got almost 2x the storage space for 1/10 the cost (that's fully loaded system + drives)
I could never get more than 33MB/s throughput on the fibre channel stuff regardless of how it was configured. Just terrible for 15k rpm fibre drives. However they did benchmark to 2ms access times (like we really need blazingly fast access times when working with 50GB of imagery simultaneously). The other problem is that the fibre channel arrays drives keep on dropping, we lose several drives a month and in the past month we actually simultaneously lost multiple drives in 2 of the systems!
Anyways, when first testing the 7500-8, I got 170MB/s sustained reads and 140MB/s sustained writes with bonnie++ using ext2. Moving over to ext3 dropped those numbers by about half, but the fsck time on that much space was unacceptable.
After building that raid we bought a 7500-4 and filled it with 80GB 7200rpm Seagate barracuda IV's. I couldn't seem to break past 60 MB/s reads and 35MB/s writes on that drive setup (i think ext3). We put an identical setup on a win2k box and saw about the same results.
From my experience, if you want an IDE raid with more than 4 drives, 3ware is the ONLY way to go. I definitely think we may try one of those fasttrak sx400's if we build another 4 drive array at work.
A comment on the SCSI reliability...
The primary reason for drives to crap out is from heat. In this area SCSI totally loses. At work were losing several 10k drives a month until we went through some drastic efforts to cool the drives.
In 1988 SCSI was still bloody cheap compared to, say DASD.
First off, they've failed to note that some of their contestants are in fact just IDE controllers, with the RAID functionality implemented in the software driver (WinRAID, like WinModems), whereas others are Hardware. I don't know all four products well, so I'm unsure on at least one of them as to which are which.
They tested CPU utilization, and seperately various speed tests, but never a comprehensive "loaded system" test. As expected they ranked the Adaptec (a true hardware RAID) lowest, while ranking the WinRAID's higher. This couldn't be further from the real truth. Sure, the idle P4 cpu does a great job of fast software RAID compared to the embedded RAID ASIC on Adaptec's card. However, if you had a heavily loaded server machine, where the processors were loaded down doing other things (say SSL-encrypting for an secure web server), the machine with the Adaptec would trounce the others, as the RAID processing speed will not decrease while your applications are using most of the CPU (or depending on the device driver's pre-emptability, it could be the other way around, that the CPU simply wouldn't be as available to your CPU-hungy SSL server as it's busy with the RAID).
11*43+456^2
The Seagate barracuda IV had a problem when connected in RAID-0; it actually performed worse than as a single drive. There was a problem with caching on them that made RAID controllers gag. There is a firmware fix; you can contact Seagate and they will replace your drives for you! I currently have 5 Barracuda IV drives connected to my highpoint RAID controller (abit at7) but they are running as single; I use the speed advantage of each being on their own IDE channel.
> I don't quite understand where this Inexpensive crap came from.
RAID (Random Array of Inexpensive Disks) was as opposed to SLED
(Single Large Expensive Disk). (The term "Random" means the same
as in RAM -- i.e., that you can access any part (any drive, in this
case) at any time.)
> RAID was around long before IDE RAID controllers started showing
> up and of course SCSI RAID arrays almost always use very expesive
> disks.
"expensive" is relative. (Instead of thinking of SCSI as the only
other option besides RAID, try to remember that there were larger
and more expensive disks at one time.)
> It's Redunant Array of Independent Disks, always has always
> will be.
It's not necessarily "redundant" at all; some RAIDs are done just
for performance reasons, with no redundancy. (Personally, I am
more interested in the redundancy, however.)
Cut that out, or I will ship you to Norilsk in a box.
I've passed this feedback onto the author of the ide raid-roundup - i figured i might as well post it here too.
I just thought i'd share some of my experiences with promise support.
Frankly, they have been terrible. I would not voluntarily buy another promise product again at this stage based on my experience with them.
I have been attempting to get support for the Promise FastTrack which is a popular embedded raid controller option, under Linux.
Promise indeed "support" RedHat but do so with a binary only, closed source module that in the end turns out to be useless.
Promise hard code a supported kernel version for this driver such that you can run it under say RedHat 7.3, but only the initial 2.4.18-3 kernel, which has a number of critical bugs which have been addressed in later (errata) kernel updates.
Needless to say, promise's driver will not run on any later kernel or at least they are unwilling to answer questions on how to do this.
A comparable analogy would be if they had released Windows XP drivers and then your hard drive failed to work if you installed a hot fix or a service pack because the driver is keyed to only the specific intial installed released of XP. Promise don't treat windows users this way, so why do they do this for linux users ?
I've managed to get two responses out of their support, none of which will address my problem - support the hardware under linux by releasing the source or provide updated kernel drivers for the released kernel images that will actually work.
In terms of driver support for Linux/FreeBSD, 3ware wins hands down in this group.
regards,
-jason
I can vouch for the stability and speed of one pre-built IDE-RAID Product. RaidZone OpenNAS They use a special raid controller that allows increased IDE Raid speed, and IDE Hot-Swap capability. This one had a total of 1.2TB (plus hot spare). The project involved a unique (to this project) Application that required a proprietary Database System that could only run on an MS Win2k Server, therefore we didnt fully utilize the sytems capabilities. It served as a file server for images (ranging in size from 60kB to well over 100MB per image) (and as I recall the Image Store is now around 500GB), MS SQL Server, confidential proprietary DB system (indexed images, among other things), and several small services (and the chunky MS GUI). It even has a 900GB native backup system attached. The load ranges. Since we put it into action early 2002 it hasn't missed a beat. I would recommend it highly for most applications, though there does come a time when higher speed drives are needed.
Why hasn't the ArcoIDE solution caught on like wildfire? It provides mirrored disk capability with absolutely no visibility to even the motherboard, much less the OS. I've been running it for years and it's great. Mine is the PCI slot model that simply uses the slot to get power to the card. One IDE cable from the motherboard to the card, two cables to the two hard drives.
And there's all sorts of alarming options -- LED's on the card, LED's on a front panel bezel, audible screech, Form C contacts for you industry types ...
I don't get it.
One simple rule for its versus it's
None of my ATA raid controllers requires identical drives. In the case of disimilar capacities, the array views all drives as the smallest drive in the array.
Later on in the life of the array, you can eventually replace the remaining drives and then resize the partition.
-ted
So I was surprised reading the review to see the Adaptec and 3Ware neck and neck in the RAID 5 area. 3Ware's usually have no competition in RAID-5 since their firmware and HW rock.
Then I found out WHY they were so close:
The 3Ware cards are 64-bit cards while the Adaptec's are only 32-bit. 3Ware cards can hit 70MB/sec writing and over 150MBsec reading with 8HD's! If they ever get to 66MHz, I expect their performance to go even higher.
If you want to see better benchmarks that fit with reality, check out the XBit Labs Review
Top Most Bizarre/Disturbing Error Messages
the major problem with a hardware raid is to rebuild you need an INDENTICAL drive as the others. fast fwd 2 yrs and good luck finding the spare. i being the most paranoid dude EVER, had a raid 5 over 3 disks live - with a hot spare running. plus i have a 'cold' spare in the box cabled in without the power pluged in ready for a rebuild. and if that is not enought i even have a 'frozen' spare in a cheapo media fire-safe just in case the back-up to the back-up to the back-up fails. it cost me tons of cash... but if you can't afford to lose it - don't mess around.... i do also have a DDS$ 6 pack autoloader for the twice daily back-up. ONLY THE PARANOID SURVIVE!!!
Try random access. Then you'll see the difference. Sequential is optimized by just about every cache out there - you're NOT benchmarking the drives with sustained transfer! You're benchmarking the caches!
A witty [sig] proves nothing. --Voltaire
At our business we use the 3Ware 7500 with a 3x 120GB (1200JB) Western Digital RAID5 configuration on Linux RedHat 7.2. The machine is a dual P3/1GHz on a SuperMicro 370DER motherboard. We use the machine as our primary file server/compilation box, so data integrity and fast failure resolution is critical.
The cited benchmark page has excellent information (130 graphs!), and it confirms my first hand experiences of everyday use of the 3Ware 7500. The read times in RAID5 are outstanding, but there are sometimes significant delays on file creation. In addition, it seems that IO is single piped, or serialized on writes at times.
Since the 3Ware 7500 is based on an FPGA (Field Programmable Gate Array, (see http://www.xilinx.com/), with LOTS of extra ECC processing power, the problem with writes is not likely to be related to the "CPU", but rather part of the internal IO structure of the card. I hope it is amenable to correction with a microcode upgrade to the FPGA, but it may be related to the memory architecture of the card.
Overall, I'm pretty happy with the controller, especially the ease of installation and rebuild time. I have high confidence in the data integrity, and the price is good. I also recommend the drive cage that 3Ware sells, even though it is expensive at $200, it's well worth it because it fits 3 drives in the space of 2 5 1/4 inch bays, and it is hot swap proven.
I'd like to see a shoot-out between the 3Ware and some SCSI/Fibre Channel RAID 5 configurations!!!
Comments?
---- Luke "To boldly go where no one has gone before..."
Each IDE controller can support up to two drives, a master and a slave. What happens if you hang two drives off one controller, and the "master" drive dies?
Actually, any modern standard IDE controller supports 2 channels or four devices. You are right in saying you shouldn't have more than 1 device per channel, or 2 devices on a standard controller. Most of the dedicated RAID IDE controllers like the ones review in the article have 4 or more channels. This allows you to build a pretty big RAID before you would consider putting a disk on as a slave.
Standard controllers are cheap, I just added a controller and 2 drives to my linux software RAID and it cost me less than $200 for the controller and the drives (80 GB and 30 GB). IIRC, the controller was ~$40. With prices like that, there is no need to run more than 1 drive per channel (unless you run out of PCI slots).
Enigma
There are valid performance and reliability reasons for using SCSI drives instead of IDE drives; the question is whether these gains are worth the cost, not whether they are there at all.
Reasons why SCSI might be worth it:
- Spin rate. Until IDE drives gain 10k and 15k spin rates, SCSI drives will always be king in multitasking and random-access situations. 3ms seek time is so much better than 10ms that you have to use it to believe the difference.
- Reliability. IDE drives have one year or at best three year warranties. SCSI drives have five year warranties. You can run modern 15k scsi drives stacked next to each other with zero additional case fans and expect to outlast your warranty. Try that with IDE.
- Hot swap. Does anyone here know of a hot-swap IDE raid solution? I think not.
- Tagged command queuing. A SCSI drive can collect multiple drive requests and reorder them to optimize the actual physical retrieval of the bits in question. IDE drives, even if the box lists this feature, have never done TCQ particularly well. This kind of thing is impossible to benchmark because its benefits only show up under heavy multitasking, not single-tasking benchmarks.
For most people, I would agree that you would be better off buying 2GB ram or two CPUs before spending money on SCSI. However, if you already have 2GB ram and two CPUs, and you still need more, then that's when you should look into high end SCSI.I help out a broadcast non-profit group that works with tons of audio files. As one might imagine space is a very big issue with audio and since they are non-profit price is always the most important factor.
d id =556
They had to build a very large streaming server and luckily write speed wasnt an issue. Every week, at least 2 gigs of new content gets loaded onto the server.
I did TONS of research... SNAP servers were two small, NETAPPS way too expensive and SCSI drives, for some reason, are still outrageously pricey ($700 for a 73 gig drive?!?!?!?!).
I was able to make a nice RAID 5 box with over 800 gigs of available space for around $5000. I can't imagine making such a setup using SCSI for under $12k. Since the server was built awhile back, everything is cheaper now, so there's no reason why one couldnt make such a server now for close to $4,000.
Unfortunately, the server was built a few months before the 160 gig maxtors were available... the same setup with these drives would go over a terabyte.
Here's the recipe we used:
1) get one very snazzy case with lots of fans that can hold up to 16 IDE drives:
Case:
http://www.rackmountpro.com/productpage.cfm?pro
2) since price is an issue, we of course went with AMD instead of Intel:
Motherboard:
Tyan Tiger 760 MPX S2466N DDR (AMD)
CPU
2 X AMD Athlon MP 1900+
3)RAM: 2 x 512 Megs DDR from crucial
4)HD: 9 X Western Digital 120GB drives (8 for RAID 5 + 1 spare)
5)Controller: 3WARE Escalade 7850 ATA Raid Controller
When the first 8 disks fill up, all we have to do is throw in one more Escalade, and put 8 new drives into the hotswap cages and we've doubled capacity. Total cost for an additional terabyte: $1800.
The performance of this setup using Red Hat 7.3 has been AMAZING. Our server regularly streams to over 400 listeners and we're not seeing any bottlenecks on I/O. The 3ware management tools are great... should one of my ide drives go bad on me, it sends an email.
If I had to worry about a big DB with tons of real time writing going on I would think twice about using IDE but for general file serving I can't think of a good reason to justify the price of SCSI.
One caveat: as some here have pointed out here, cabling is a bit of nightmare.
I've been working on x86-based servers a long, long time.
:-)
There are many reasons one should choose SCSI over IDE, but I want to counter a few of the arguments I've read through the many messages here:
Argument #1:
SCSI can have 15 devices per bus, but why buy more smaller and more expensive SCSI drives instead of getting fewer large IDE drives?
Answer: Bigger isn't always better. On large RAID systems (real servers, here people...not Mp3 servers) one of the concepts of RAID5 is to spread out the data among as many drive spindles as possible. This keeps each drive's load level under control, and eliminates hot-spots on individual disks. If you sit down with any SAN vendor, like EMC, they will tell you the same thing.
Argument #2
Sustained IDE Raid performance can equal SCSI
This is absolutely incorrect. This may be true on a server with no CPU load. Try this again on a server running SQL and averaging 85% load. You will NOT see the same performance out of an IDE disk layer. There is simply too much CPU overhead on an IDE-based RAID system for heavy-load systems. The idea behind a SCSI controller is that it is free of the system's CPU as a bottleneck. The money saved on non-SCSI hardware will instead need to be spent on faster CPUs.
Argument #3
IDE Disks are just as reliable as SCSI
Again, completely false. You get what you pay for. SCSI disks have logic on each disk to control the operations OF that disk. In a RAID array, you want each disk to be completely independant of the others. IDE RAID requires the controller to do all the monitoring (if there is any) of each disk, lowering performance of its primary function--controlling disk I/O. Anyone who has worked on a Compaq server and used Insight Manager will be able to see the advantages of SCSI disks directly. SCSI disks will be more reliable since they are built to be more reliable. IDE disks are meant for cheap deployment on cheap systems.
Thank you, have a nice day
-brain
Thats because one of the major limitations of current generation IDE is that only one device on a channel can "talk" at a time. So even if you're using a RAID card with two devices on a channel, it will be no faster than a standard IDE connection, since only one drive read/write can be done at a time.
Not at all true. There are a good many IDE (ATA, actually) RAID controllers out there that use one drive per IDE channel, and connect to the host via SCSI or Fibre Channel. (Of course, in this case there is *never* channel contention, and the weak spot is in the SCSI or of FC connection, both of which are using the SCSI protocol. This approach is FAR faster than almost all SCSI-based RAID systems out there, and much cheaper to boot. One of the advantages of using serious IDE Raid subsystems (not the cheezy desktop variety) is that the cost savings can allow you to replace RAID 5 with RAID 0+1 (sometime called 10) and still save money. I know because I've engineered and built multi-terabyte storage servers on this technology that are 2-3x faster and an order of magnitude less expensive than high-end storage servers like the IBM Shark or EMC Symmetrix. IDE *will* squash SCSI, it's not a matter of if, but when, mostly because SCSI will never be able to compete with the volume economics that produce IDE's 5-6x cost advantage. The performance advanstage of individual SCIS drives is already becoming marginal, and the speed of individual drives is nearly irrelevant anyway in a RAID environment where most of the poerformance comes from spanning mutiple splindles, not the speed of the individual disks. (This is why a properly configured RAID array of disks with average access time N can deliver average access times of significantly less than N.)
With SCSI, all of the drives on a channel can talk at the same time until the 160 MB/s that SCSI can handle is saturated.
Not even close. SCSI is a one-talker at a time bus architecture. This is one reason a good IDE RAID controller can so easily kick SCSI butt. The largest clusters and multiprocessor computers are all going to high performance IDE RAID arrays because of their superior cost, performance, and yes, reliability, since electrical problems in physical SCSI are one of the most common causes of data corruption in high performance environments, which is one of the chief reasons Fibre Channel has been so widely adopted. It too uses the SCSI protocol, and so has real weaknesses, but at least it avoids the hideous flakiness of SCSI's connector and termination scheme.
"The future's good and the present is nothing to sneeze at." - Roblimo's last