Slashdot Mirror

← Back to Stories (view on slashdot.org)

Hardware RAID 5 Performance Configurations?

Posted by Cliff on from the how-many-drives-per-channel dept.

gandy909 asks: "I am facing the need to replace a major server in the next few months due to both EOL status and disk I/O bottleneck issues with the array containing the data. The server is configured with a 2 channel array controller. It is a RAID 5 array and has 4 drives, 2 per channel (2 data, 1 parity, and 1 hot spare). Obvious performance benefits in replacing the server are quadrupling CPU speed and doubling the memory. Other side benefits I will gain with the new drives, that I think should help my performance, are moving from u160 to u320, and going from 7200 RPM to either 10k or 15k RPM." How would you configure a larger array to best increase its performance? "Having googled around, the consensus is that increasing the number of drives is the preferred way to attack the I/O bottleneck. What I don't find much help on is determining the configuration of the larger array. Assuming I am going to be using a 12 drive array I have come up with the following possible configurations:

1 2 channel controller, 6 drives per channel
1 4 channel controller, 3 drives per channel
2 2 channel controllers, 3 drives per channel
3 2 channel controllers, 2 drives per channel

Would any one of those configurations provide better performance than the others, or would they all even out considering other factors?"

14 of 62 comments (clear)

  1. Get away from RAID 5 by Fished · · Score: 4, Insightful
    Not enough information - is this data warehousing? Transaction processing? Mostly reads, or a lot of writes as well?

    But generally, I don't recommend RAID 5 for performance critical situations. It's great for data warehouse, but if you lose one drive their goes your performance. Also, realize that, often, the place where you can really boost performance is in the database, not the hardware. How's your query optimization? Do you have appropriate indexes? Is the code accessing the database efficient?

    --
    "He who would learn astronomy, and other recondite arts, let him go elsewhere. " -- John Calvin, commenting on Genesis 1
    1. Re:Get away from RAID 5 by chinakow · · Score: 2, Informative

      Are you kidding? Have you ever used a Hardware array controller? do you even know the difference between a 0+1 and a RAID 5?

      First of all any controller made in the last decade has a fast enough proccessor on it that it is going to be faster than software RAID, second, a 0+1 is WAY slower than a RAID 5. For example lets say I got 4 disks if I setup a 0+1 that would be 2 mirror sets and the data is even split across the 2 arrays so I ask for a file and I have just 2 drive seeking the info, and retreiving that info, with a 4 disk RAID 5 I have at least 3 head seeking and I have just increased the data rate coming of of the drives by 50% just because I have one more drive fetching the info.

      hardware RAID in a RAID 5 is only slower than a RAID 0 and they are both Lightning fast compared to mirrors and 0+1 sets , I do not know where you get the impression that tying up 4 drives and making them act like 2 drives is faster than using 4 drives as if they where 3 but I would actually be interested to see hard facts as all I am providing is logic and observed performance. but from the work I have done RAID 5 is the way to go and if you have good backups and can handle downtime RAID 0 is even better.

    2. Re:Get away from RAID 5 by Fweeky · · Score: 2, Informative

      Here's some logic: RAID-5 needs to write across all disks to update parity on writes, which slows them no matter how much fancy hardware you've got to improve them. RAID-5 also needs to rebuild data from parity after a drive failure, meaning your high volume server is going to crawl until you can replace and rebuild.

      RAID-10/01 gives you a mirrored stripe; mirrors improve read performance by letting you balance reads across drives either to increase STR or TPS, stripes improve read performance again and also give you a boost in writes.

      RAID-10/01 is generally faster (mainly on writes and rebuilds), and can sustain more drive failures than RAID-5 (which is more important to me than saving a few bucks on hardware given that these are typically on important high volume systems).

      And here's a nice big document from a fairly trustworthy source to back me up. Nice try though.

  2. Well, knowing 0 about your business requirements. by dnight · · Score: 4, Informative

    I'd put 2GB fibre HBA's in the server(s) and attach an OpenSAN RAID box from Winchester Systems with dual-homed fibre to the array. That'll take care or your bottleneck and help your future scaling requirements.

  3. i am not an expert but... by blackcoot · · Score: 2, Informative

    ... the key to bottlenecks, as i've understood, are: 1) how many physical paths are there to disk, 2) how big are the buffers.

    #1 can be fixed by adding more controllers /and/ more disks, #2 can be fixed by buying the right controllers. not /too/ too useful, but a start.

    if you're /really/ concerned about performance (enough to spend cash on it), give someone like storagetek a call --- they've got this down to a fine art. quite probably waaay overkill for what you want to do, but it's a start.

  4. Depends on usage patterns by cookd · · Score: 5, Interesting

    First, 3 drives in RAID-5 is not very useful. You get a lot of the disadvantages with few of the benefits. Having more drives really helps throughput. So go for more smaller drives over fewer faster drives for RAID-5.

    Second, RAID-5 is great for read speeds, but less great for write speeds. A good caching controller will help hide this, but a small write operation requires a read from each disk in the set before the write can be completed (in order to recompute parity for the stripe). If this is mostly reading, or if most writes are large (not small and random), RAID-5 will work fine (data storage, data mining, etc). If writes are frequent (transactions), RAID-5 is painful. RAID-10 might be better.

    --
    Time flies like an arrow. Fruit flies like a banana.
    1. Re:Depends on usage patterns by cookd · · Score: 2

      So go for more smaller drives over fewer faster drives for RAID-5.

      I meant to say larger.

      --
      Time flies like an arrow. Fruit flies like a banana.
    2. Re:Depends on usage patterns by cookd · · Score: 4, Insightful

      And to answer the question you actually ASKED...

      Fast drives can sustain about 60 MB/s nowadays, so don't put more than (max channel rate)/(max drive rate) = (320)/(60) = around 5 drives on a single channel.

      As for multiple controllers, if they aren't present for redundancy purposes, you're probably just as well off using just one.

      But then again, I'm just a Slashdotter who hasn't kept up with the latest specs, so I'm pretty much just making stuff up. Free advice is worth every penny.

      --
      Time flies like an arrow. Fruit flies like a banana.
    3. Re:Depends on usage patterns by MarkRose · · Score: 2, Informative

      a small write operation requires a read from each disk in the set before the write can be completed (in order to recompute parity for the stripe)

      Not entirely. If your parity is a simple XOR of all the chunks, then you only need to read the chunk to be rewritten and the appropriate parity chunk. XOR the old chunk with the parity, XOR that with the new chunk and rewrite the parity chunk, and finally write the new chunk. However, this still necessitates a read from the drive where the write is to occur and a read of the appropriate chunk of parity information for every write. This means that only 2 of the n drives in an array are involved. This technique works best when your parity information is striped across all the drives (to avoid a bottleneck -- which is why RAID 5 is so popular), but there is still a penalty for dealing with parity.

      --
      Be relentless!
    4. Re:Depends on usage patterns by innosent · · Score: 3, Informative

      No, internal transfer rates are now up to around 109 MB/s for the top 15k RPM drives. You should be able to do better than 60MB/sec per drive, unless your reads are all over the place. (Lots of small reads instead of a few large, contiguous reads?) If you REALLY want speed, you can't beat RAID 10. For 12 drives, I would go with 2 U320 dual-channel controllers, having 3 drives on each channel (4 would probably be better, but that's 16 drives). Probably would be best to mirror across channels on the same card, and stripe those 6 (or 8) mirrored pairs across the two cards. You could hit a theoretical write speed of 640MB/sec pretty easily with 16 drives, probably only around 550-600 with 12, with read speeds of 1280MB/sec for 16 drives, and just over a GB/sec with 12. If a GB/sec isn't fast enough, it's probably time to write new software.

      --
      --That's the point of being root, you can do anything you want, even if it's stupid.
  5. Why not forego raid5? by millisa · · Score: 2, Interesting

    If this is an EOL system and its using U160 drives . .chance are those drives are 36 gig or less . .I'd even bet they might be 18's . . . But lets say they are 36's . . four of them in a raid 5 is giving you ~120GB? Why not just get a pair of 147GB drives and run in a raid1? I mean, like others have said, without knowing what you are doing with it, it is hard to say where you are going to get the most benefit, but a lot of times, Raid5 is chosen just due to the increase in space you can get. . . There are more complex options out there that can get you better performance than raid5 (ie, raid 1+0 raid 5+0 and similar) if your controller and wallet support it . . .

  6. No we will not do your homework for you. by Photar · · Score: 2, Informative

    This sounds exactly like a question I had in Computer Architecture class.

    It looks to me that if you're upgrading from a computer that was 1/4 the speed of today's servers, then if you get a modern server machine with raid SATA you'll be fine.

    --
    He who knows not and knows he knows not is a wise man. He who knows not and knows not he knows not is a fool.
  7. More Information... by gandy909 · · Score: 2, Informative

    This is a *nix box running a Court application from a vendor that used a Synergy DE ISAM type 'database' backend. We have about 100 users, a general mix of add/change/query operations but not a lot of deletes. It is keeping Court data, and that stuff never goes away. The reads are mostly displaying screenfuls of data or small reports to the screen all the time, or larger reports to the printer several times during the day. The writes are usually writing manually entered information, a screen or paragraph at a time, so there tend to be less of them, and at a slower pace, but all day long.

    The app is a legacy non-SQL type db that is not, nor ever will be, anywhere near normalized by any stretch of the term. The largest of the data files is just over 1 gig at this point. The OS file size limit is 2gb. Due to this, and the other reasons we will likely be moving to a completely different system in the 5 year range.

    Hardwarewise, the box as I inherited it, is a Dell 6400 rackmount server with 4 700mhz P3 Zeons (only 1 activated...don't ask), 1g mem, a PERC2(AMI MegaRAID) dual channel controller, and a split(4+4) backplane. It holds 8 9 gig drives in 2 arrays. Even with these small drives there is over 50% and 70% free space on the arrays.

    My budget limit is $10k to replace it. One of the options I was looking at was a Dell 2650 with a PERC3-QC controller and one of the Storcase 10 bay Infostations they offer on the Dell site to hold the rest of the drives. The way the app is so 'interconvoluted' together I don't think I gained anything by separating the data into 2 arrays and will likely just use a single array on the upgrade.

    I hope this helps... :)

    --

    (Stolen sig) Remember: it's a "Microsoft virus", not an "email virus", a "Microsoft worm", not a "computer worm
  8. Comment removed by account_deleted · · Score: 2, Informative

    Comment removed based on user account deletion