Slashdot Mirror
Linux HW and SW RAID Benchmarked
An anonymous reader writes "A Norwegian site has written up an article with various RAID solutions benchmarked using both bonnie++ and dbench. The result shows a lot of surprises, especially when comparing low end sw RAID with high end hw RAID. The text is in Norwegian but the numerous graphs are self explanatory. It does look like a few kernel drivers need a little tweaking."
Surprising? No. Reading the results I can see that software raid is generally slower than hardware raid and that some of the SCSI drivers are not completely tweaked, probably because they can't get enough information from the manufacturer.
Anyone know an internet translator that supports Norwegian? Or even a Norwegian? It would be nice to have a translation so we don't have to sit around making uninformed comments about what we can't understand...
Oh, wait...
# cat
Damn, my RAM is full of llamas.
My bonnie++ was used by Norwegians, To see how fast my RAID could be, My bonnie++ was used by Norwegians, ...but was bonnie++ written in C?
Paleotechnologist and connoisseur of pretty shiny things.
here
Damn. I've been a geek too long. After all these years I know understand how my pointy haired boss feel when attempting to read a technical article.
~~ What's stopping you?
http://www.translation-guide.com/free_online_trans lators.php?from=Norwegian&to=English
Not that it's really useful. It's a *little* more readable than the original. I think.
Paleotechnologist and connoisseur of pretty shiny things.
Ok, who the fuck allowed this submission to go through? A whole 2% of Slashdot readership will probably be able to read this, the rest of us are left in the dark. Are longer bars better, or worse? WTFOMGBBQ?!
Jeg skal ikke gå så langt som å si at man burde satse på verken SATA, billige kontrollere eller software-RAID.
In english; I will not go as far as to recommend SATA, cheap controllers or software-RAID.
Seriously, is this frontpage news on Slashdot? I'm a native speaker, and the article did not impres s me much. In fact, there is nothing newsworthy about the article, and the author admits it in the conclusion. Not very insightful, the article is crearly written by an amateur. In fact, in my opinion, the only reason this was submitted to Slashdot, is because hwb.no is a new site, which is trying desperately to get visitors.
A møøse once bit my sister...
norse--
If only I had a "Learn Norse in 30days" book to advertise about now, i'd be rich
It would be nice to have a translation so we don't have to sit around making uninformed comments about what we can't understand...
Somehow, I don't think a translation would keep them away.
-"One machine can do the work of fifty ordinary men. No machine can do the work of one extraordinary man." -EH
From TFA: To innebygde gigabit-nettverkskort
That is just the coolest; I am hereby recommending everyone refer to networking as 'nettverkskort'. It might be cold in Norway, but they have some awesome sounding linguistic constructions!
PS - What the heck is nettverkskort, exactly? 'Networking'? 'Network Adapter'? Heck, I don't know what it is; I just know I like it.
This is not a very good review, they have used kernel 2.6.8, 2.6.11 has many fixes upon previous releases in regards to RAID and md (software raid) drivers.
Lets get a review that uses 2.6.11, then lets see where we are.
Hey, this is my sig, if you don't like it, STOP READING MY POSTS!
Now, give it to a non-Norwegian speaking geek, and a non-Geek Norwegian speaker.
Who do you think will have more luck making heads or tails of it?
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
I think the real surprise for me is that SW RAID is 95% as fast as HW RAID without the pricey board, not that SW RAID is slower.
Also, another surprise is that a SATA RAID (speed) performs about as well as a SCSI RAID. Whether SATA drives are as reliable is a different matter, but with the cost savings, it is easier to have more spare drives on hand.
From a system bus bandwidth perspective, it would seem that the chief difference between HW and SW RAID would be that SW RAID requires some more housekeeping bits, the biggest one being the data from the parity drive goes over the system bus for SW, but it stays local to the RAID controller for HW.
From a CPU perspective, for SW, the CPU would have to compute the XORs rather than offloading them to the dedicated hardware, which are compute cycles and pages that could be done for other tasks in a HW setup.
For me, the speed difference is kind of moot though. If I want RAID, it would be for the redundancy and spanning multiple drives, not speed. Also, I have systems with 64/66 PCI and a system with PCI-X, so that bus isn't an issue.
More info here:8 9489
http://slashdot.org/comments.pl?sid=108060&cid=91
Ok, here is a rough translation:
I have wanted to test some real SATA controllers against SCSI controllers for some time now, to see how good SATA has become. I once thought that cheap controllers like Sil 3114 is cheap crap that manufacturers put on their boards simply to provide SATA-support, and that software RAID was a cheap, but insufficient solution, since I have followed the principle that hardware does the job best. "A more expensive controller, means more hardware", was my initial guess, but it seems that even the cheap controllers are worthy. Software RAID also performs very well. SATA is no longer some gag for disk systems that are supposed to perform well, and many myths have been dispelled by my test.
I will not go as far as to say that you shall place your bet on cheap controllers or software RAID. The reason is simple, in a expensive controller, there is much more functionality, that a cheap controller can just dream about. Functionality like hot-spare drives and hot-swap, just to mention some. I do not want to recommend SATA over SCSI in a while either. The lifespan of a SCSI drive is in most casese many times as long as a vanilla SATA-disk. When you choose a solution, it should last. If you have machines that has a big fat controller, RAID50, then SATA might be something for you. If you have a machine that needs redundancy on the internal drives, but where changing controllers, or even buying them in the first place has been in the way, then software RAID might be the solution for you.
I shall be careful to mock the LSI controller, as I think there might be a problem with the way the test machine talks to it. I think the new Megaraid driver in the kernel might be the problem. Either it needs to mature, or it is simply that it does not like 64-bit Linux. I have not tampered too much with the default settings, but it runs superparanoid verification algorithms when it sends and recieves data. I have not fleshed the BIOS on any of the controllers.
Adaptecs controllers do very well. Everything was not perfect with them, and the aacraid driver in the kernel was too old for both of the controllers. From their website, I found something that looked like source code (Adaptec seems to rely on 100% RPM based distros), and I could bouild my own module. After that, no problem. A little minus is that the aacraid does not report how long the controller has gotten in building the array after you have set up a RAID. By looking at the SCSI-BIOS after some hours, I got to verify that the array was built.
I want to warn everyone that is going to buy a controller. Carefully check that the controller is supported in the kernel! I use Google to check for references to the card on mailing lists, but that does not help much when you have Debian, and all that exist is binary RedHat drivers.
Now, run to your console and test your disk system. This test does only give you indications on what to choose. I allow myself to give you one final advice: Run tests for yourself.
Dvorak on Doomtech
A few years back I was responsible for benchmarking potential RAID solutions for a major computer company. We investigated both software based and hardware based solutions.
The conclusion we reached; software RAID gave greatly superior performance than the hardware RAID solutions available at the time, but the hardware RAID solutions had better feature sets and usability.
The superiority in performance that the software raid solutions showed was due to a quirk in what was then state-of-the-art in RAID and systems design.
Most RAID controllers at that time contained embedded Intel i960 processors running at around 100 MHz, and had caches that topped out in the 128 MB range. Meanwhile, systems contained 2-4 CPUs in the 1.2 GHz range, and 2-8 GB of memory. There was simply no way that the embedded processor and cache on the RAID card could manipulate the data as quickly as the primary system resources could, and the benchmarks showed it.
The "exception" to this performance was when RAID-5 was used. Because RAID-5 requires computational resources above and beyond simply moving data back and forth in order to calculate parity, the host-based RAID solutions couldn't always keep up.
It was the fact that RAID-5 required additional computational resources that led fairly directly to the "ROMB" (RAID on motherboard) solutions that some vendors today. The ROMB chip is often nothing more than an XOR engine, to accelerate parity calculations.
The major, major, shortcoming we found with software RAID solutions was that they did not work with our customer's software, if that software ran outside of an operating system that had drivers for the solution. With hardware RAID, the physical disks were completely abstracted away, and you could run in any possible environment and still be able to read/write from your RAID volumes.
All of the above commentary about hardware vs. software performance is meant to apply to a specific point in time. I wouldn't try to extrapolate those results to current technology without rerunning the experiments today.
Yes.
First off, most SATA controllers are NOT hardware RAID although they support hardware raid options.
This is called BIOS raid and esseciantly uses software drivers in a similar fasion that Winmodems or software modems use software in their drivers to emulate hardware.
Dedicated hardware RAID devices are much more expensive, up in the hundreds of dollars for the controller. These devices use a embedded style cpu running around 200-400mhz that is specially designed for doing work like this.
For Linux MD, software raid, you generally have a 2000mhz to 3000mhz+ cpu backing it up.
So you guess which is faster.
It realy makes sense once you realise what is going on here.
There is no 'driver' tweaking needed. Linux software RAID is the bomb, it's better then FreeBSD/Windows/OS X/etc, it's something that Linux has strength in.
Supports almost any RAID configuration. 0, 1, 4, 5, 10, whatever you want. It supports many different ways to get raid using any block device...
I use 1 PATA drive on my onboard then I have 2 SATA drives on a PCI card in RAID 5 configuration. It's very nice.
In contrast those cheapy SATA 0/1 raid setups you get on cards are ASS. Always. ALWAYS, linux MD raid is faster.
What you want in hardware RAID in in Linux is RELIABILITY, not speed. If you want speed you can go with a Linux cluster using Lustre' or Redhat's GFS and get much more impressive results at a cheaper price.
Hardware RAID allows advanced error detection, working hotswapability, and other features that go beyond mere I/O.
Haesta paska suatanan peikko.
Eihä tua näytä suamelta etes murteella.
Perkele.
Bot Assisted Blogging
My own comments are inside []-brackets.
Bolded text:
What controllers should you look for for a new machine? Do you need one costing 2-4000 NOK (300-500USD) to maintain uptime and data integrity without losing speed? In this test we will look at some of the options and the results when building a good system.
Published May 13.
Most modern motherboards has some form of S-ATA. Both desktop and servers. One of the most common is the Silicon Image 3114. This is a 4 port SATA with alledged RAID-capabilities.
In almost all cheap SATA-controllers they tout raid capabilities. This is a half-truth. These controllers can as just about ever other controller be used in a RAID-array, but most of the work is done in the OS. In windows, mostly in the driver. Rumor has it one should look for raid 5 capabilities if one is looking for a true hardware raid solution (as in transparent to the rest of the hardware). Whether this is correct is not known since GNU/Linux has RAID5 support in software. The road ahead should be short. [idomatic expressiob, doesn't make much sense in Norwegian either in this context.]
[2 controllers' pics]
In this test we will look at different controllers. The aforementioned Sil 3114 is one of those cheap ones with fake hardware raid. How well does it do compared to a much more expensive SCSI setup. Is SATA so good that expensive SCSI setups is only useful in special cases?
Thanks to Nextron for a machine, several controllers and other equipment. And also thanks to MPX for the loan of several Adaptec controllers.
[Next page]
[I will skip most of the redundant translating] Fire diskport -> four disk ports.
[The comment about the 1.5 GiB memory is about finding a faulty chip.]
enhet -> unit
[long text]
This pretty server has almost all one needs in its small cabinet. It comes with "speed-couplings" [hard to translate] for SATA-disks, so the test with the SCSI controllers is done with an external SCSI cabinet and a PSU. The barebone system kan be delivered with SCSI if needed or one can add this oneself.
With it's 6 angry [slightly different conotations in Norwegian] and tiny little fans I would recommend being in the same room. Noicy like a small machine room. [as in say a boat].
[Next page] David. This chip has several Goliaths to fight.
* SiL 3114
On most controllers one sees this one or it's little brother. 3112 is often used as an interface to the disks. Simple controller with no RAID caps in HW.
* Megaraid. 150-4
This one has two 3112 chips for the SATA part and 64MiB ECC cache and an intel processor. It is not low profile but has a nice space saving design. Supports Raid 0,1,5 and 10.
* Megaraid 320-1
Low profile SCSI, internal and external connector. Has the GC08302 procsessor. Supports RAID 0,1,10,5,50.
* Adaptec 2130SLP.
Low profile. Internal and external connector. Has a staggering 128 MiB DDR Cache. Supports RAID 0,1,10,5,50 and JBOD.
*Adaptec 2410SA
Low profil SATA with two 3112 chips for SATA support, comes with 64 MiB cache. Supports 0,1,5,10 and JBOD.
[Rant about "true" RAID and level 0 and JBOD with link to a guide.]
The different controllers has support for various functions. LSI controllers tout their "on the fly" changes in the array, changing of raid-type without losing data and similar. Adaptec focuses on SNMP and a lot of the same as LSI. What one needs is up to the reader. The four "external" controllers come with various cables, manuals and CDs.
[Next page] During the test we used 50GB partitions. Sata disks were almost 3.5 times as big as the SCSI ones, and under this test the file system etc should change the results due to different physical size. It's not really possible to compare it directly, since the disks are quite different, we're looking for patterns in how te configs behave, not only if SATA can compare with SCSI.
For the test we used bonnie++ and dbecnh. [links]
Nonnie++ was us
Rest in peace Malin "looxn" Kristiansen. We miss you...
In my expirience it's more like 9 out of 10 times can you swap (or even hot-swap) a failed drive.
Out of 5 drive failures that I have expirienced only one brought the machine to a halt (assume the failed drive did something strange to the ide bus).
In all other cases the raid degraded gracefully and the machine could be shutdown cleanly to swap in a new drive.
It *should* be even better with server-grade SCA (hot-swap SCSI) drives because since these are built for hot-swap they are even less likely to confuse the bus when they go down.
So, as you might have guessed, I don't agree with you.
Swapping in a new disk is usually much less hassle than restoring from backup.
Even if the machine halts when a disk fails - at least you still have your data.
I'm a happy RAID customer.
"Avoid employing unlucky people - throw half of the pile of CVs in the bin without reading them." -- David Brent
2.4.27 still provides better md performance than 2.6.9 says Neil, not sure if this go fixed in .11.
http://cgi.cse.unsw.edu.au/~neilb/
I use md so I can span PCI buses with multiple controllers to get better performance than a single HW raid card. Also, when my controller goes south I don't have to get the same controller. If I was really desperate I could use the onboard. I can upgrade my controller without backing up and restoring the array. I could get a SATA-II controller and slowly move my drives to SATA-II. I feel like I get more control with mdadm, too. At least I can inspect, alter or wipe the drive metadata while I'm up in the OS.