Slashdot Mirror
Linux HW and SW RAID Benchmarked
An anonymous reader writes "A Norwegian site has written up an article with various RAID solutions benchmarked using both bonnie++ and dbench. The result shows a lot of surprises, especially when comparing low end sw RAID with high end hw RAID. The text is in Norwegian but the numerous graphs are self explanatory. It does look like a few kernel drivers need a little tweaking."
Anyone know an internet translator that supports Norwegian? Or even a Norwegian? It would be nice to have a translation so we don't have to sit around making uninformed comments about what we can't understand...
Oh, wait...
# cat
Damn, my RAM is full of llamas.
My bonnie++ was used by Norwegians, To see how fast my RAID could be, My bonnie++ was used by Norwegians, ...but was bonnie++ written in C?
Paleotechnologist and connoisseur of pretty shiny things.
here
Damn. I've been a geek too long. After all these years I know understand how my pointy haired boss feel when attempting to read a technical article.
~~ What's stopping you?
http://www.translation-guide.com/free_online_trans lators.php?from=Norwegian&to=English
Not that it's really useful. It's a *little* more readable than the original. I think.
Paleotechnologist and connoisseur of pretty shiny things.
Ok, who the fuck allowed this submission to go through? A whole 2% of Slashdot readership will probably be able to read this, the rest of us are left in the dark. Are longer bars better, or worse? WTFOMGBBQ?!
Jeg skal ikke gå så langt som å si at man burde satse på verken SATA, billige kontrollere eller software-RAID.
In english; I will not go as far as to recommend SATA, cheap controllers or software-RAID.
Seriously, is this frontpage news on Slashdot? I'm a native speaker, and the article did not impres s me much. In fact, there is nothing newsworthy about the article, and the author admits it in the conclusion. Not very insightful, the article is crearly written by an amateur. In fact, in my opinion, the only reason this was submitted to Slashdot, is because hwb.no is a new site, which is trying desperately to get visitors.
A møøse once bit my sister...
It would be nice to have a translation so we don't have to sit around making uninformed comments about what we can't understand...
Somehow, I don't think a translation would keep them away.
-"One machine can do the work of fifty ordinary men. No machine can do the work of one extraordinary man." -EH
From TFA: To innebygde gigabit-nettverkskort
That is just the coolest; I am hereby recommending everyone refer to networking as 'nettverkskort'. It might be cold in Norway, but they have some awesome sounding linguistic constructions!
PS - What the heck is nettverkskort, exactly? 'Networking'? 'Network Adapter'? Heck, I don't know what it is; I just know I like it.
This is not a very good review, they have used kernel 2.6.8, 2.6.11 has many fixes upon previous releases in regards to RAID and md (software raid) drivers.
Lets get a review that uses 2.6.11, then lets see where we are.
Hey, this is my sig, if you don't like it, STOP READING MY POSTS!
Now, give it to a non-Norwegian speaking geek, and a non-Geek Norwegian speaker.
Who do you think will have more luck making heads or tails of it?
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
I think the real surprise for me is that SW RAID is 95% as fast as HW RAID without the pricey board, not that SW RAID is slower.
Also, another surprise is that a SATA RAID (speed) performs about as well as a SCSI RAID. Whether SATA drives are as reliable is a different matter, but with the cost savings, it is easier to have more spare drives on hand.
From a system bus bandwidth perspective, it would seem that the chief difference between HW and SW RAID would be that SW RAID requires some more housekeeping bits, the biggest one being the data from the parity drive goes over the system bus for SW, but it stays local to the RAID controller for HW.
From a CPU perspective, for SW, the CPU would have to compute the XORs rather than offloading them to the dedicated hardware, which are compute cycles and pages that could be done for other tasks in a HW setup.
For me, the speed difference is kind of moot though. If I want RAID, it would be for the redundancy and spanning multiple drives, not speed. Also, I have systems with 64/66 PCI and a system with PCI-X, so that bus isn't an issue.
Drilling through the article with my utterly minimal norwegian (Prarie Home Companion + German + exposure to Danish coworkers), I think I've distilled the following:
Cache on the LSI RAID controller is 1/2 the adaptec. Performance is comparable, though not equivalent.
All of the controllers are 64-bit.
Adaptec SCSI is good for both hardware RAID and software RAID.
LSI has good hardware SCSI RAID only.
Don't use current SATA controllers (RAID or Otherwise) for best performance.
Does anybody with access to a good collection of modern hardware care to re-run this test in a language that Babelfish understands?
the more accurate the calculations became, the more the concepts tended to vanish into thin air. R. S. Mulliken
Ok, here is a rough translation:
I have wanted to test some real SATA controllers against SCSI controllers for some time now, to see how good SATA has become. I once thought that cheap controllers like Sil 3114 is cheap crap that manufacturers put on their boards simply to provide SATA-support, and that software RAID was a cheap, but insufficient solution, since I have followed the principle that hardware does the job best. "A more expensive controller, means more hardware", was my initial guess, but it seems that even the cheap controllers are worthy. Software RAID also performs very well. SATA is no longer some gag for disk systems that are supposed to perform well, and many myths have been dispelled by my test.
I will not go as far as to say that you shall place your bet on cheap controllers or software RAID. The reason is simple, in a expensive controller, there is much more functionality, that a cheap controller can just dream about. Functionality like hot-spare drives and hot-swap, just to mention some. I do not want to recommend SATA over SCSI in a while either. The lifespan of a SCSI drive is in most casese many times as long as a vanilla SATA-disk. When you choose a solution, it should last. If you have machines that has a big fat controller, RAID50, then SATA might be something for you. If you have a machine that needs redundancy on the internal drives, but where changing controllers, or even buying them in the first place has been in the way, then software RAID might be the solution for you.
I shall be careful to mock the LSI controller, as I think there might be a problem with the way the test machine talks to it. I think the new Megaraid driver in the kernel might be the problem. Either it needs to mature, or it is simply that it does not like 64-bit Linux. I have not tampered too much with the default settings, but it runs superparanoid verification algorithms when it sends and recieves data. I have not fleshed the BIOS on any of the controllers.
Adaptecs controllers do very well. Everything was not perfect with them, and the aacraid driver in the kernel was too old for both of the controllers. From their website, I found something that looked like source code (Adaptec seems to rely on 100% RPM based distros), and I could bouild my own module. After that, no problem. A little minus is that the aacraid does not report how long the controller has gotten in building the array after you have set up a RAID. By looking at the SCSI-BIOS after some hours, I got to verify that the array was built.
I want to warn everyone that is going to buy a controller. Carefully check that the controller is supported in the kernel! I use Google to check for references to the card on mailing lists, but that does not help much when you have Debian, and all that exist is binary RedHat drivers.
Now, run to your console and test your disk system. This test does only give you indications on what to choose. I allow myself to give you one final advice: Run tests for yourself.
Dvorak on Doomtech
A few years back I was responsible for benchmarking potential RAID solutions for a major computer company. We investigated both software based and hardware based solutions.
The conclusion we reached; software RAID gave greatly superior performance than the hardware RAID solutions available at the time, but the hardware RAID solutions had better feature sets and usability.
The superiority in performance that the software raid solutions showed was due to a quirk in what was then state-of-the-art in RAID and systems design.
Most RAID controllers at that time contained embedded Intel i960 processors running at around 100 MHz, and had caches that topped out in the 128 MB range. Meanwhile, systems contained 2-4 CPUs in the 1.2 GHz range, and 2-8 GB of memory. There was simply no way that the embedded processor and cache on the RAID card could manipulate the data as quickly as the primary system resources could, and the benchmarks showed it.
The "exception" to this performance was when RAID-5 was used. Because RAID-5 requires computational resources above and beyond simply moving data back and forth in order to calculate parity, the host-based RAID solutions couldn't always keep up.
It was the fact that RAID-5 required additional computational resources that led fairly directly to the "ROMB" (RAID on motherboard) solutions that some vendors today. The ROMB chip is often nothing more than an XOR engine, to accelerate parity calculations.
The major, major, shortcoming we found with software RAID solutions was that they did not work with our customer's software, if that software ran outside of an operating system that had drivers for the solution. With hardware RAID, the physical disks were completely abstracted away, and you could run in any possible environment and still be able to read/write from your RAID volumes.
All of the above commentary about hardware vs. software performance is meant to apply to a specific point in time. I wouldn't try to extrapolate those results to current technology without rerunning the experiments today.
My own comments are inside []-brackets.
Bolded text:
What controllers should you look for for a new machine? Do you need one costing 2-4000 NOK (300-500USD) to maintain uptime and data integrity without losing speed? In this test we will look at some of the options and the results when building a good system.
Published May 13.
Most modern motherboards has some form of S-ATA. Both desktop and servers. One of the most common is the Silicon Image 3114. This is a 4 port SATA with alledged RAID-capabilities.
In almost all cheap SATA-controllers they tout raid capabilities. This is a half-truth. These controllers can as just about ever other controller be used in a RAID-array, but most of the work is done in the OS. In windows, mostly in the driver. Rumor has it one should look for raid 5 capabilities if one is looking for a true hardware raid solution (as in transparent to the rest of the hardware). Whether this is correct is not known since GNU/Linux has RAID5 support in software. The road ahead should be short. [idomatic expressiob, doesn't make much sense in Norwegian either in this context.]
[2 controllers' pics]
In this test we will look at different controllers. The aforementioned Sil 3114 is one of those cheap ones with fake hardware raid. How well does it do compared to a much more expensive SCSI setup. Is SATA so good that expensive SCSI setups is only useful in special cases?
Thanks to Nextron for a machine, several controllers and other equipment. And also thanks to MPX for the loan of several Adaptec controllers.
[Next page]
[I will skip most of the redundant translating] Fire diskport -> four disk ports.
[The comment about the 1.5 GiB memory is about finding a faulty chip.]
enhet -> unit
[long text]
This pretty server has almost all one needs in its small cabinet. It comes with "speed-couplings" [hard to translate] for SATA-disks, so the test with the SCSI controllers is done with an external SCSI cabinet and a PSU. The barebone system kan be delivered with SCSI if needed or one can add this oneself.
With it's 6 angry [slightly different conotations in Norwegian] and tiny little fans I would recommend being in the same room. Noicy like a small machine room. [as in say a boat].
[Next page] David. This chip has several Goliaths to fight.
* SiL 3114
On most controllers one sees this one or it's little brother. 3112 is often used as an interface to the disks. Simple controller with no RAID caps in HW.
* Megaraid. 150-4
This one has two 3112 chips for the SATA part and 64MiB ECC cache and an intel processor. It is not low profile but has a nice space saving design. Supports Raid 0,1,5 and 10.
* Megaraid 320-1
Low profile SCSI, internal and external connector. Has the GC08302 procsessor. Supports RAID 0,1,10,5,50.
* Adaptec 2130SLP.
Low profile. Internal and external connector. Has a staggering 128 MiB DDR Cache. Supports RAID 0,1,10,5,50 and JBOD.
*Adaptec 2410SA
Low profil SATA with two 3112 chips for SATA support, comes with 64 MiB cache. Supports 0,1,5,10 and JBOD.
[Rant about "true" RAID and level 0 and JBOD with link to a guide.]
The different controllers has support for various functions. LSI controllers tout their "on the fly" changes in the array, changing of raid-type without losing data and similar. Adaptec focuses on SNMP and a lot of the same as LSI. What one needs is up to the reader. The four "external" controllers come with various cables, manuals and CDs.
[Next page] During the test we used 50GB partitions. Sata disks were almost 3.5 times as big as the SCSI ones, and under this test the file system etc should change the results due to different physical size. It's not really possible to compare it directly, since the disks are quite different, we're looking for patterns in how te configs behave, not only if SATA can compare with SCSI.
For the test we used bonnie++ and dbecnh. [links]
Nonnie++ was us
Rest in peace Malin "looxn" Kristiansen. We miss you...
A 3ware 12 port SATA card and a three port U320 SCSI card with four drives on each port both support the same number of drives. Except the SATA card will probably be 1/3 the price, the SATA drives will be 1/10 the cost per GB, and have higher transfer rates.
SATA does have real command queuing. There are real hotswap SATA drive bays. It's true the cables can't be as long, but since you only need to connect once device per cable instead of 4 or more, it's usually easier to connect. And believe me, I know my way around a SCSI cable.