IO is a controller/interface issue. RAID is a parity calculation issue. They are not the same. If you can't understand that, you have serious issues. Calculating parity has nothing to do with IO. Don't understand why you don't get that.
When things become IO bound, you need more controllers, not hardware raid. If you run out of cpu cycles, you get more cpu (either in the form of hardware raid, or your main cpu). As Intel itself says - the main cpu is now fast enough to calculate parity, to fill up the IO channel. And using less than 5% of the cpu.
So, again, if your server can't handle software RAID (that Intel says is faster than hardware raid) because of a piddly 5% more cpu usage kills your server, you don't know how to size servers.
Just spoke to an EMC Solutions Architect, you know, the guy they sent round to all those Fortune 100 companies to put an EMC solution in place...
I asked my friend: Does EMC use hardware raid, or is it all software, but implemented via firmware? For what it does, it just feels weird to me that it's "hardware raid" like some dell PERC card.
Reason I'm asking - someone keeps insisting it is hardware raid and that really surprised me. Does it mean that Sun's new storage devices, the 7000 series is considered hardware raid if they put solaris in firmware...?:)
EMC Solutions Architect responded: It's essentially software, but it appears as hardware to whatever accesses the storage. If you have the right access to the system then you can rebuild the RAID sets, make storage assignments, etc. But, this is all done on the back end using the array's own operating system and controllers.
And by the way, you should really check out how ZFS works, and how it overcomes all the RAID issues (atomicity, RAID hole, write when power loss, etc).
I always like to go back to first principles. When Patterson, Gibson, Katz first described RAID, they detailed how it worked, and how you can break the data up over multiple inexpensive disks, how can you stripe data, and how to use parity. In 1987, x86 architectures were a 286 or maybe a 386. So, when they implemented parity for RAID, it had to be done in specialized hardware, because the CPUs are not fast enough. We have all seen how a Windows NT 4 server slowed to a crawl when you turn on the openGL screen saver, or used winmodems. The CPUs in those days were simply slow.
Today, a Phenom II X4 945 can compute RAID6 parity at close to 8GB/s. A throughput of 500 MB/s requires less than 1.5% of CPU time. (see: http://blog.zorinaq.com/?e=10)
Now, of course, there's some overhead, processes swapping, etc. And RAID is more than simply calculating parity. The numbers I have seen do not go above 5% of one core however.
Can specialized chips implement RAID functionality faster than an Intel CPU? Sure, that's obvious. But again - if your server cannot handle a 5% load on one CPU on top of what it is already handling - you are sizing it wrong.
Why would software RAID be better? Because you cannot easily implement new technologies like ZFS in hardware. Well, you could, after all, firmware is just software that's written to a chip instead of the hard drive, and runs whatever processes/algorithms you want it to run, on whatever processors. But when new revisions of ZFS come out, it is not as easy to upgrade.
By the way, this is assuming well implemented software raid. Crappy software raid will give you really crappy performance, obviously.
As an example - Cisco always used low powered CPUs (compared to their competitors) and always marketed their "we have specialized hardware ASICS for switching and routing" because they were too cheap to use real CPUs. And so what happens when you cross the threshold for whatever they designed the ASICs to hold/process? Your performance dives off a cliff. Or what happens when you implement IPSec? Or IPv6? Well, buy more Cisco equipment I guess. Cisco's CEO thanks you: http://etherealmind.com/poster-reassuringly-expensive/
Obviously they have non-commodity controllers, but are these "hardware raid"? So far, nothing indicates that they are from cursory reading. If you have links that say that they are, do share.
But you sound like an old storage guy, so it's obvious why you like hardware raid. But it's a brave new world out there my friend.
And which part of dealing with frivolous cases is free or does not involve tax payer money, which was your assertion? Quit backpedalling. You were caught making a stupid statement, admit it and move on, instead of doubling down and showing us what a desperate idiot you are.
My Sun x4540 box laughs at you. And if you believe NetApps and EMC use RAID "cards", well, I guess, pass the pipe. A better computer does not mean the most expensive computer. It just means you don't buy the cheapest thing from walmart.
And your motherboards using FakeRaid? And you are using that as an argument? Shows how much enterprise shit you get to play with.
Anyone building a serious enterprise storage capability knows better than to waste money on raid cards.
You think Google, Facebook, and those cloud providers use RAID cards? Seriously?
And if your servers are so constrained that your cpu or ram could not handle it, then maybe you should size your servers appropriately. Or perhaps upgrade to a better OS.
How the fuck is this insightful? The key here is whether Facebook buys from Cisco who buys from a factory in China, or Facebook buys from the same damned factory in China.
It's the same company in China doing the design and manufacturing.
Record unemployment. You seem to think each worker is the same as another. Competent workers are hard to find. Extremely hard to find. You think Billy Bob who took a class in html will be able to cut it at Google?
Do people really not know the difference between a switch and a router any more? A 24 port switch is affordable. A 24 port router is fucking expensive.
Did you forget Active Directory and Kerberos where Microsoft refused to say WTF they did in the extension field until the Kerberos working group threatened to redefine that field away and turn Microsoft's implementation incompatible?
Slashdot Mirror
Sun box is gold plated? Then wtf is an EMC box?
IO is a controller/interface issue. RAID is a parity calculation issue. They are not the same. If you can't understand that, you have serious issues. Calculating parity has nothing to do with IO. Don't understand why you don't get that.
When things become IO bound, you need more controllers, not hardware raid. If you run out of cpu cycles, you get more cpu (either in the form of hardware raid, or your main cpu). As Intel itself says - the main cpu is now fast enough to calculate parity, to fill up the IO channel. And using less than 5% of the cpu.
So, again, if your server can't handle software RAID (that Intel says is faster than hardware raid) because of a piddly 5% more cpu usage kills your server, you don't know how to size servers.
Just spoke to an EMC Solutions Architect, you know, the guy they sent round to all those Fortune 100 companies to put an EMC solution in place...
I asked my friend:
Does EMC use hardware raid, or is it all software, but implemented via
firmware? For what it does, it just feels weird to me that it's
"hardware raid" like some dell PERC card.
Reason I'm asking - someone keeps insisting it is hardware raid and :)
that really surprised me. Does it mean that Sun's new storage
devices, the 7000 series is considered hardware raid if they put
solaris in firmware...?
EMC Solutions Architect responded:
It's essentially software, but it appears as hardware to whatever
accesses the storage. If you have the right access to the system then
you can rebuild the RAID sets, make storage assignments, etc. But,
this is all done on the back end using the array's own operating
system and controllers.
By the way, you should really check out what Intel says as well:
http://www.theregister.co.uk/2010/09/15/sas_in_patsburg/
And by the way, you should really check out how ZFS works, and how it overcomes all the RAID issues (atomicity, RAID hole, write when power loss, etc).
I always like to go back to first principles. When Patterson, Gibson, Katz first described RAID, they detailed how it worked, and how you can break the data up over multiple inexpensive disks, how can you stripe data, and how to use parity. In 1987, x86 architectures were a 286 or maybe a 386. So, when they implemented parity for RAID, it had to be done in specialized hardware, because the CPUs are not fast enough. We have all seen how a Windows NT 4 server slowed to a crawl when you turn on the openGL screen saver, or used winmodems. The CPUs in those days were simply slow.
Today, a Phenom II X4 945 can compute RAID6 parity at close to 8GB/s. A throughput of 500 MB/s requires less than 1.5% of CPU time. (see: http://blog.zorinaq.com/?e=10)
Now, of course, there's some overhead, processes swapping, etc. And RAID is more than simply calculating parity. The numbers I have seen do not go above 5% of one core however.
Can specialized chips implement RAID functionality faster than an Intel CPU? Sure, that's obvious. But again - if your server cannot handle a 5% load on one CPU on top of what it is already handling - you are sizing it wrong.
Why would software RAID be better? Because you cannot easily implement new technologies like ZFS in hardware. Well, you could, after all, firmware is just software that's written to a chip instead of the hard drive, and runs whatever processes/algorithms you want it to run, on whatever processors. But when new revisions of ZFS come out, it is not as easy to upgrade.
By the way, this is assuming well implemented software raid. Crappy software raid will give you really crappy performance, obviously.
As an example - Cisco always used low powered CPUs (compared to their competitors) and always marketed their "we have specialized hardware ASICS for switching and routing" because they were too cheap to use real CPUs. And so what happens when you cross the threshold for whatever they designed the ASICs to hold/process? Your performance dives off a cliff. Or what happens when you implement IPSec? Or IPv6? Well, buy more Cisco equipment I guess. Cisco's CEO thanks you: http://etherealmind.com/poster-reassuringly-expensive/
Now go watch this: http://video.google.com/videoplay?docid=-6304964351441328559# at :34'50" they tell you software raid gave them a 20% to 30% increase over hardware raid.
This old dog has learnt to go with the times. You should too.
Color me surprised if Netapps actually use hardware raid, since everything else is handled by the software, according to http://en.wikipedia.org/wiki/NetApp_filer and articles like http://media.netapp.com/documents/tr-3298.pdf
Obviously they have non-commodity controllers, but are these "hardware raid"? So far, nothing indicates that they are from cursory reading. If you have links that say that they are, do share.
But you sound like an old storage guy, so it's obvious why you like hardware raid. But it's a brave new world out there my friend.
And which part of dealing with frivolous cases is free or does not involve tax payer money, which was your assertion? Quit backpedalling. You were caught making a stupid statement, admit it and move on, instead of doubling down and showing us what a desperate idiot you are.
the letters "cosmo".
My Sun x4540 box laughs at you. And if you believe NetApps and EMC use RAID "cards", well, I guess, pass the pipe. A better computer does not mean the most expensive computer. It just means you don't buy the cheapest thing from walmart.
And your motherboards using FakeRaid? And you are using that as an argument? Shows how much enterprise shit you get to play with.
Anyone building a serious enterprise storage capability knows better than to waste money on raid cards.
You think Google, Facebook, and those cloud providers use RAID cards? Seriously?
http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/ sure don't show any RAID cards in there.
Touché :)
And if your servers are so constrained that your cpu or ram could not handle it, then maybe you should size your servers appropriately. Or perhaps upgrade to a better OS.
I guess you haven't dealt with multiple conflicting firmware versions that wipe out raidsets then. You're lucky.
Exactly. Only morons argue against that. And we have clear evidence of that here.
Don't understand why people still don't understand this.
What a convoluted way to make sure when the shit hits, it gets spread out really evenly.
Seriously?! hardware raid cards? Which century are you stuck in?
I prefer 68. I owe her one.
Like Australia doesn't suck at America's teats.
This lack of ability to read and understand is why America is on the decline. But these damned idiots are loud. And obnoxious.
Yet another idiot. I was a H1B, and I guarantee you that I was the highest paid person on the team, beating out all the Americans.
At my current place, I pay for performance, and gave one guy a 60% bump over his last place, and another guy over 100% bump (both Americans).
However, I have had to fire people too, for non-performance.
Stop talking using some lame bullet points.
How the fuck is this insightful? The key here is whether Facebook buys from Cisco who buys from a factory in China, or Facebook buys from the same damned factory in China.
It's the same company in China doing the design and manufacturing.
Record unemployment. You seem to think each worker is the same as another. Competent workers are hard to find. Extremely hard to find. You think Billy Bob who took a class in html will be able to cut it at Google?
No wonder the US is losing its edge.
Do people really not know the difference between a switch and a router any more? A 24 port switch is affordable. A 24 port router is fucking expensive.
RAIDZ3 you insensitive clod! My pr0n is important!
Did you forget Active Directory and Kerberos where Microsoft refused to say WTF they did in the extension field until the Kerberos working group threatened to redefine that field away and turn Microsoft's implementation incompatible?
whooooosh