Slashdot Mirror
Linux HW and SW RAID Benchmarked
An anonymous reader writes "A Norwegian site has written up an article with various RAID solutions benchmarked using both bonnie++ and dbench. The result shows a lot of surprises, especially when comparing low end sw RAID with high end hw RAID. The text is in Norwegian but the numerous graphs are self explanatory. It does look like a few kernel drivers need a little tweaking."
Surprising? No. Reading the results I can see that software raid is generally slower than hardware raid and that some of the SCSI drivers are not completely tweaked, probably because they can't get enough information from the manufacturer.
here
http://www.translation-guide.com/free_online_trans lators.php?from=Norwegian&to=English
Not that it's really useful. It's a *little* more readable than the original. I think.
Paleotechnologist and connoisseur of pretty shiny things.
Jeg skal ikke gå så langt som å si at man burde satse på verken SATA, billige kontrollere eller software-RAID.
In english; I will not go as far as to recommend SATA, cheap controllers or software-RAID.
Seriously, is this frontpage news on Slashdot? I'm a native speaker, and the article did not impres s me much. In fact, there is nothing newsworthy about the article, and the author admits it in the conclusion. Not very insightful, the article is crearly written by an amateur. In fact, in my opinion, the only reason this was submitted to Slashdot, is because hwb.no is a new site, which is trying desperately to get visitors.
try with swedish, it's almost the same.
"Større er bedre" means "bigger is better", while "mindre er bedre" means "smaller is better". That should help a lot with the bar graphs.
"nettverkskort" = "network card".
And in case it doesn't, it means "I'd like to warn everyone who's going to go buy controllers after this. Check _carefully_ that the controller is supported by the kernel."
More info here:8 9489
http://slashdot.org/comments.pl?sid=108060&cid=91
Ok, here is a rough translation:
I have wanted to test some real SATA controllers against SCSI controllers for some time now, to see how good SATA has become. I once thought that cheap controllers like Sil 3114 is cheap crap that manufacturers put on their boards simply to provide SATA-support, and that software RAID was a cheap, but insufficient solution, since I have followed the principle that hardware does the job best. "A more expensive controller, means more hardware", was my initial guess, but it seems that even the cheap controllers are worthy. Software RAID also performs very well. SATA is no longer some gag for disk systems that are supposed to perform well, and many myths have been dispelled by my test.
I will not go as far as to say that you shall place your bet on cheap controllers or software RAID. The reason is simple, in a expensive controller, there is much more functionality, that a cheap controller can just dream about. Functionality like hot-spare drives and hot-swap, just to mention some. I do not want to recommend SATA over SCSI in a while either. The lifespan of a SCSI drive is in most casese many times as long as a vanilla SATA-disk. When you choose a solution, it should last. If you have machines that has a big fat controller, RAID50, then SATA might be something for you. If you have a machine that needs redundancy on the internal drives, but where changing controllers, or even buying them in the first place has been in the way, then software RAID might be the solution for you.
I shall be careful to mock the LSI controller, as I think there might be a problem with the way the test machine talks to it. I think the new Megaraid driver in the kernel might be the problem. Either it needs to mature, or it is simply that it does not like 64-bit Linux. I have not tampered too much with the default settings, but it runs superparanoid verification algorithms when it sends and recieves data. I have not fleshed the BIOS on any of the controllers.
Adaptecs controllers do very well. Everything was not perfect with them, and the aacraid driver in the kernel was too old for both of the controllers. From their website, I found something that looked like source code (Adaptec seems to rely on 100% RPM based distros), and I could bouild my own module. After that, no problem. A little minus is that the aacraid does not report how long the controller has gotten in building the array after you have set up a RAID. By looking at the SCSI-BIOS after some hours, I got to verify that the array was built.
I want to warn everyone that is going to buy a controller. Carefully check that the controller is supported in the kernel! I use Google to check for references to the card on mailing lists, but that does not help much when you have Debian, and all that exist is binary RedHat drivers.
Now, run to your console and test your disk system. This test does only give you indications on what to choose. I allow myself to give you one final advice: Run tests for yourself.
Dvorak on Doomtech
Yes.
First off, most SATA controllers are NOT hardware RAID although they support hardware raid options.
This is called BIOS raid and esseciantly uses software drivers in a similar fasion that Winmodems or software modems use software in their drivers to emulate hardware.
Dedicated hardware RAID devices are much more expensive, up in the hundreds of dollars for the controller. These devices use a embedded style cpu running around 200-400mhz that is specially designed for doing work like this.
For Linux MD, software raid, you generally have a 2000mhz to 3000mhz+ cpu backing it up.
So you guess which is faster.
It realy makes sense once you realise what is going on here.
There is no 'driver' tweaking needed. Linux software RAID is the bomb, it's better then FreeBSD/Windows/OS X/etc, it's something that Linux has strength in.
Supports almost any RAID configuration. 0, 1, 4, 5, 10, whatever you want. It supports many different ways to get raid using any block device...
I use 1 PATA drive on my onboard then I have 2 SATA drives on a PCI card in RAID 5 configuration. It's very nice.
In contrast those cheapy SATA 0/1 raid setups you get on cards are ASS. Always. ALWAYS, linux MD raid is faster.
What you want in hardware RAID in in Linux is RELIABILITY, not speed. If you want speed you can go with a Linux cluster using Lustre' or Redhat's GFS and get much more impressive results at a cheaper price.
Hardware RAID allows advanced error detection, working hotswapability, and other features that go beyond mere I/O.
For a supposed geek site, many /.'ers shows an alarming ineptitude to find/verify information. For the controller in question, he could quite simply use Google, or go to the LSI home page. The tools used are standard, the controllers/hardware are standard, which Linux kernels used should be apparent. Understanding the conclusions, of course, means understanding Norwegian, but he should be able to interpret the output himselves.
My own comments are inside []-brackets.
Bolded text:
What controllers should you look for for a new machine? Do you need one costing 2-4000 NOK (300-500USD) to maintain uptime and data integrity without losing speed? In this test we will look at some of the options and the results when building a good system.
Published May 13.
Most modern motherboards has some form of S-ATA. Both desktop and servers. One of the most common is the Silicon Image 3114. This is a 4 port SATA with alledged RAID-capabilities.
In almost all cheap SATA-controllers they tout raid capabilities. This is a half-truth. These controllers can as just about ever other controller be used in a RAID-array, but most of the work is done in the OS. In windows, mostly in the driver. Rumor has it one should look for raid 5 capabilities if one is looking for a true hardware raid solution (as in transparent to the rest of the hardware). Whether this is correct is not known since GNU/Linux has RAID5 support in software. The road ahead should be short. [idomatic expressiob, doesn't make much sense in Norwegian either in this context.]
[2 controllers' pics]
In this test we will look at different controllers. The aforementioned Sil 3114 is one of those cheap ones with fake hardware raid. How well does it do compared to a much more expensive SCSI setup. Is SATA so good that expensive SCSI setups is only useful in special cases?
Thanks to Nextron for a machine, several controllers and other equipment. And also thanks to MPX for the loan of several Adaptec controllers.
[Next page]
[I will skip most of the redundant translating] Fire diskport -> four disk ports.
[The comment about the 1.5 GiB memory is about finding a faulty chip.]
enhet -> unit
[long text]
This pretty server has almost all one needs in its small cabinet. It comes with "speed-couplings" [hard to translate] for SATA-disks, so the test with the SCSI controllers is done with an external SCSI cabinet and a PSU. The barebone system kan be delivered with SCSI if needed or one can add this oneself.
With it's 6 angry [slightly different conotations in Norwegian] and tiny little fans I would recommend being in the same room. Noicy like a small machine room. [as in say a boat].
[Next page] David. This chip has several Goliaths to fight.
* SiL 3114
On most controllers one sees this one or it's little brother. 3112 is often used as an interface to the disks. Simple controller with no RAID caps in HW.
* Megaraid. 150-4
This one has two 3112 chips for the SATA part and 64MiB ECC cache and an intel processor. It is not low profile but has a nice space saving design. Supports Raid 0,1,5 and 10.
* Megaraid 320-1
Low profile SCSI, internal and external connector. Has the GC08302 procsessor. Supports RAID 0,1,10,5,50.
* Adaptec 2130SLP.
Low profile. Internal and external connector. Has a staggering 128 MiB DDR Cache. Supports RAID 0,1,10,5,50 and JBOD.
*Adaptec 2410SA
Low profil SATA with two 3112 chips for SATA support, comes with 64 MiB cache. Supports 0,1,5,10 and JBOD.
[Rant about "true" RAID and level 0 and JBOD with link to a guide.]
The different controllers has support for various functions. LSI controllers tout their "on the fly" changes in the array, changing of raid-type without losing data and similar. Adaptec focuses on SNMP and a lot of the same as LSI. What one needs is up to the reader. The four "external" controllers come with various cables, manuals and CDs.
[Next page] During the test we used 50GB partitions. Sata disks were almost 3.5 times as big as the SCSI ones, and under this test the file system etc should change the results due to different physical size. It's not really possible to compare it directly, since the disks are quite different, we're looking for patterns in how te configs behave, not only if SATA can compare with SCSI.
For the test we used bonnie++ and dbecnh. [links]
Nonnie++ was us
Rest in peace Malin "looxn" Kristiansen. We miss you...
In my expirience it's more like 9 out of 10 times can you swap (or even hot-swap) a failed drive.
Out of 5 drive failures that I have expirienced only one brought the machine to a halt (assume the failed drive did something strange to the ide bus).
In all other cases the raid degraded gracefully and the machine could be shutdown cleanly to swap in a new drive.
It *should* be even better with server-grade SCA (hot-swap SCSI) drives because since these are built for hot-swap they are even less likely to confuse the bus when they go down.
So, as you might have guessed, I don't agree with you.
Swapping in a new disk is usually much less hassle than restoring from backup.
Even if the machine halts when a disk fails - at least you still have your data.
?
"What the fuck, O my God, BARBECUE?!"
2.4.27 still provides better md performance than 2.6.9 says Neil, not sure if this go fixed in .11.
http://cgi.cse.unsw.edu.au/~neilb/
- sequential output per char:
- hw and sw RAID mostly equally fast
- however LSI SATA performed badly. In most tests LSI SATA performed sub par. This could be worth looking into.
- sequential output per block:
- hw RAID shines, especially SCSI.
- however sw RAID Adaptec SCSI outperformed the rest
- sequential input and output per char:
- sw mostly outperformed hw RAID
- However CPU use by Adaptec drivers hw and sw were strangely high
I hope this sumamry could be of use to the Kernel driver developer.