Slashdot Mirror
RAID Problems With Intel Core 2?
Nom du Keyboard writes "The Inquirer is reporting that the new Intel Core 2 processors Woodcrest and Conroe are suffering badly when running RAID 5 disk arrays, even when using non-Intel controllers. Can Intel afford to make a misstep now with even in the small subset of users running RAID 5 systems?" From the article: "The performance in benchmarks is there, but the performance in real world isn't. While synthetic benchmarks will do the thing and show RAID5-worthy results, CPU utilization will go through the roof no matter what CPU is used, and the hiccups will occur every now and then. It remains to be seen whether this can be fixed via BIOS or micro-code update."
it's not a bug, just errata ;)
sum.zero
If you're running raid5 it's probably in an enterprise setup. If so, why aren't you running a dedicated controller? The CPU should have little to no impact on the raid subsystem...
Seems odd to me that the inquirer is the only one reporting this. How about a real hardware review site?
Does this mean that Apple won't be using Intel chips in their Xseve's for a while?
-mrxak
Onions Will Kill You
Just because you don't use RAID in your basement doesn't the market is small.
I don't get what the problem is. Are there specific instructions used often in raid 5 algorithms that are slow on the new chips? Is it bus contention?
MidnightBSD: The BSD for Everyone
Article implies that these are the lame RAID systems that the l33t G4m3rz use...the ones that use the CPU to do the real work, rather than having the card do it.
I didn't realize they were so wide-spread...especially in government! Seems like just an additional avenue for data corruption should the software hiccup and lunch your data.
Blar.
Still, this is a problem for Intel. Their products are supposed to do what they do extremely well under all conditions. I hope that they find a way to fix this admittedly niche problem.
Information wants a fueled airplane waiting at the hangar and no one gets hurt.
Exactly. I'm baffled how a separate controller would cause such an issue. You would expect more than just raid controllers to exibit this behavior considering even non-intel cards are causing woes.
I know a lot of people are using on-board RAID-5 feature.
The point TFA makes is not that a RAID5 setup would be used on a desktop, but that real-world performance seems to suffer on this chip.
Am I halucinating to recall something happening like this a long time ago with Intel?
Yes, i tought about this too.
Using RAID5 in software (be it completely in software like Linux MD or Windows Dynamics Disks, or 99% in Software, like most Onboard RAID Controllers out there) isn't a good idea if you want to run an "enterprise" setup. It might be okay for your mom's basement, or for test systems.
But productive systems should be using real raid controllers, equipped with half a gig of cache memory, a battery backup in case of a power failure for the cache, and dedicated processor for the raid5 overhead.
Intel might've screwed this up, but it will only affect non-professional IT.
Reading the article it's all about software raid and the performance they get.
... )
The interesting question is what other peices of software that we run will get unexpectedly bad performance.
( I have > 2TB of hardware RAID 5 at home so I was wondering
You should be using a controler with a dedicated processor, anyway.
I totally agree. If this is actually a RAID-5 setup, then it requires at minimum 3 drives. Most onboard (intel) RAID controllers are only setup for 0,1,0+1, or 10. And not RAID 5. I don't see how it could possibly be correlated to the CPU. It seems much more likely that if it is a new North/South bridge, that the problem is the with IO controller.
CPU utilization in RAID5 configurations is almost entirely offloaded to the RAID controller.
The article (including spelling errors) fails to mention a lick about the RAID controller. Only that "it's a cpu problem."
If you don't care about data integrity, throughput, or CPU utilization, like me. Most users will be buying the core 2 for household heating, and not superfluous stuff like data access.
This test is interesting for two reasons:
Not to mention that most workstations and home PCs don't run RAID 5. If the Core/Core2 chip sets are targeted for machines that don't run RAID, it's not a big deal. If you are running RAID 5, it's likely in a server environment where you would probably have a RAID controller and a Opteron or Xeon based chip.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
Remember that all a HW raid controller is, is a low end (compared to Xeons, etc) embedded CPU running software not unlike what your software raid solution would run.
This extra coprocessor for RAID is great when you have a box doing many different things like rendering, etc. But on a dedicated fileserver you'll be better off using the really fast CPU rather than the much slower raid controller chip to do the RAID logic.
Python/perl/java have not suffered in any tests I've seen. I guess that leads me to question these findings even more.
Intel might've screwed this up, but it will only affect non-professional IT.
Or those with budgets.
Because it's often slower to do so. We ran tests on a good Adaptec u320 raid controler about a year back and though cpu usage was good. We got much better performance out of Linux softraid5. I would suspect this was because the host cpu was faster than that on the controler.
Not to mention there is a huge cost savings in going with a softraid solution.
Raid 5 is 3 hard drives at the minimum. Most gamers will have 2 that run in striped configuration as that is the best price/performance ratio saving money for the ever expensive video cards. Corporations on the other hand have servers with dedicated raid cards.
Short answer: Move along nothing to see here.
If it truely is a problem with the CPU, then that's what Intel has the Microcode Upgrades. A simple little upgrade and voila... no more AMD people shouting and pointing "OHH! OHHH!!!! Intel made a MISTAKE!!!!! Lookie Lookie! HAHA!!!! AMD NEVER MAKES MISTAKES!!"
Of course, this is definately a case of the pot calling the kettle black. I mean, name one processor in the last 20 years that HASN'T had a bug in it.
Cliff Claven
K.E.G. Party Chairman
Founding Leader of: Koncerned for Egalitarin Governance
My personal "analysis", is that this sounds much more like a DMA issue, either in chipsets, in the processors, or in OSes. Core 2 is doing some speculative prefetching and a quite different cache management scheme, so some naive ideas would be that some piece of code or hardware got away with doing things improperly before, a very rare race condition might have become commonplace. If that's the reason, it might be easy to fix. Of course, it might also mean that the prefetching or cache sharing between the cores (or a couple of other things) are actually faulty...
I'm slightly confused.
... further implicating the mobo. However, similar mobos with AMD processors didn't experience the problem, so there's obviously something going on that's Intel's fault.
The articles are both very light on technical details, and somewhat vague as to what's really going on. (Admittedly, maybe they don't know it.) In the first article, they allude to the problems being the result of the "softmodem"-like RAID systems that modern integrated motherboards use, which would remove some of the blame from the processor. But then they also suggest that the same problem occurs with dedicated RAID controllers (IBM ServeRAIDs -- I think these are dedicated controllers), which don't cause too much CPU load at all
It doesn't seem like it would be that difficult to pin the blame down to the particular component: is it the integrated RAID subsystem utilizing the processor inefficiently? Or is it the processor itself, being slow? And if it was the processor, why wouldn't this slowness be exhibited in other situations?
Seems to me that what needs to happen, is for somebody to do a test with a Conroe processor in a motherboard that doesn't include any of the integrated, offload-work-to-the-processor type of integrated subsystems (RAID, sound, Ethernet), use a 'real' hardware RAID controller, and see what the results are. If there are still problems in that scenario, then there would seem to be something wrong with the processor, and this could be confirmed with simulative benchmarks.
As a criticism of Intel's complete "systems" (processor plus chipset) I suppose this is a valid criticism, but I'd like to see more of a breakdown as to where the performance hit is coming from.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
If the Core/Core2 chip sets are targeted for machines that don't run RAID, it's not a big deal.
Everyone's downplaying this, but nobody's asking the real question: Why does it suck at RAID? If we don't know why it's doing this, what if there are other things that will cause the Core to likewise suck?
Real world businesses use External Hardware RAID boxes, or SANs
using your CPU to play RAID5 controller is just plain dumb
now, as long as mirroring works fine, i'm happyh
Exercise caution when modding this message up: the author acts like a jerk when his karma is excellent.
The real "Libtards" are the Libertarians!
From TFA:
The reason was that there were severe problems when Woodcrest was paired with a 1E RAID field when using IBM ServeRAID controllers. The problems didn't occur just in benchmarking, it was the every-day usage model that produced unexpected errors.
ServeRAID controllers aren't some cheapo CPU-based RAID, it looks like this might be a more serious problems.
If you can't afford to do something right, don't do it.
You use XOR to clear a register. XOR CX, CX sets the CX register to 0. It is faster than MOV CX, 0.
This sounds like a timing problem -- the processors are too fast, causing the system to slow down.
i ?id=121434)
There was a similar problem that I had to wrestle with on a Linux when runnig 3Ware RAID controllers w/ RHEL3 on fast dual-processor servers. When battery backed write caching was turned on, the fast acceptance of IO requests (by the CPU's and then by the hardware RAID controller) lead to awesome sustained performance for short bursts, but under constant load would suddenly hit a wall and then IO would practically hang. (https://bugzilla.redhat.com/bugzilla/show_bug.cg
Can Intel afford to make a misstep now with even in the small subset of users running RAID 5 systems?
No. No, it cannot. Sell your stock. Rip the CPU out of your boxen. One hundred ten billion dollars in market capitalization has disappeared in a flash with the publication of this groundbreaking article in the Inquirer.
Intel has signed its own death warrant. As goes RAID5, so goes the world.
What if you have a lot of photos, music or movies - these aren't unusual things these days. I don't want to go rummaging through DVDs to find the picture I want, I want to fire up f-spot and see it there straight away.
RAID5 provides sensible protection against data loss when using consumer hard disks - software RAID5 is readily available on linux and hard disks in the 2-300GB range are easily affordable. You can often pick them up for $50 after rebates. So I can get a TB of storage for a few hundred dollars, but to use hardware RAID5 would probably double the cost. Fine if you're an enterprise, but not fine if you're using it at home.
Every Inquirer story I've clicked through to from slashdot has been subsequently debunked.
Anyone got independant verification of this startling discovery?
No, I did not read the f***ing article!
"I agree with this. For most people, backing up your data every week is a LOT better option for data security. Users who should be using RAID 5 should also have dedicated controllers."
You're generalizing a little too much. For example: I have >1TB storage on my mythtv box (I just like to have a good selection of stuff to watch when I finally get to watch tv, and I'm never at home when the shows I like are being broadcasted), and I'm using software RAID5 on that. That is, software raid5, on shared controllers: All together seven disks off the mainboard, from a mixture of pata and sata connectors. I wouldn't do this on something like a server, but it's plenty fast enough for mythtv. It also gives a lot of protection for the array of disks, and it's a much, much better option than the weekly backup you suggest (first of all, a backup would take ages, cost waay more in disks (which wouldn't even fit in the HTPC), and last but not least: without raid5, if one disk dies, I could lose up to 7 days of recordings...).
--- Hindsight is 20/20, but walking backwards is not the answer.
The problem persists even if you use a dedicated controller.
Not any more. With even consumer-level boards offering embeddedRAID5 and RAID 1+0/0+1 support at the $100-$150 price level, and with hard drives being uber-cheap nowadays, there is absolutely no reason you won't see an explosion of growth in the use of RAID5 in at least higher-end home and SOHO machines.
The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
I have installed a software Raid5 at work for online backups of workstations. 250GB SATA disks cost nothing (~80?); it'd pain my anus to fork out a kilobucks or two to pilot them. Sorry if that's not enterprisy enough for you!
If you can't afford to do something right, don't do it.
;-)
hope you're happy with the way your life is going
People replying to my sig annoy me. That's why I change it all the time.
I agree, it seems on slashdot (and actually, some of my friends) that you're an idiot if you're not running RAID but your equally dumb if you're running RAID5 because it's not a backup solution. It's as if there can't be any gray area in the matter. People make it seem like RAID5 has no purpose or benefit and everyone should just be using striping+backup. To me, the point of RAID5 or other redundancy RAID setups is it's your first line of recovery for a disk failure. If a disk fails, you replace it and you've suffered little downtime. If something major happens then yes, you restore from backup.
My other issue is with people forgetting the idea behind being sensible about what needs to be protected and how much it should cost. There is no reason why my personal collection of photos, music and video should cost me so much. Software RAID is way more than adequate for providing a cheap way to store my files. If data protection AND peak performance are what you need, then yes you need to go full hardware. WHERE'S THE MIDDLE GROUND PEOPLE?
I go to school in Washington DC and I live the rest of the time in Southeast Pennsylvania. Both places have been hit by floods recently, which should alert you all to use things other than RAID for backup. RAID does have its palce, as a devestating hard drive failure can become a minor 45 minute annoyance.
Information wants a fueled airplane waiting at the hangar and no one gets hurt.
-*The above statement is printed entirely on recycled electrons*-
This isn't about life. It's about a professional decision as someone working for a company. Before deploying half-assed solutions, it's usually better to deploy nothing at all.
It probably should be pointed out that many software RAID systems use a dedicated channel for every drive. RAID-5 on a SCSI hardware RAID adapter doesn't do this. The more drives you operate on the same channel, the bigger the issue can get. This could be an easy explaination as to why software RAID would be faster in your circumstance.
I, for one, intend to run RAID 5 on my next home system, and not just for Geek Pride. Three hard drives with the advantage of mirroring for reliability, striping for performance, and the loss of only 1/3d of my hard drive space for this redundancy, verses 1/2 for mirroring alone, and no redundancy with striping alone. And since I'm doing this for improved performance, I want that performance. Three modern moderate performance hard drives are hardly expensive, and RAID 5 can make up for average performing of 7200rpm drives otherwise.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
If you'd read TFA you'd see that the problem has shown itself with a Woodcrest (the next Xeon) CPU using an IBM RAID controller.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
Actually the market has become so diluted with everyone's jumping into the RAID game (thanks to Highpoint Tech and Intel with their hybrid solutions) that it's becoming increasingly difficult to discern the true hardware RAID controllers from the hybrid models. Of course there are the companies that won't so much as touch software RAID (namely 3ware) but Promise, Koutech, and even Adaptec all are very slick with their descriptions of the controllers and make it unclear as to whether or not their products are actual RAID controllers of if they offload the processing to the CPU. If you want to give a small business (mom & pop or a larger business with a tightass PHB who sees IT as solely a cost center rather than an essential tool to keep things going) better assurance of data integrity than a single HDD will provide, and they're NOT willing to back up regularly, and obviously won't spend $300-$700 for an entry level GOOD RAID5 controller, then a hybrid solution may be all that you can offer them. Given that these controllers are being implemented on motherboards more and more now, the performance they provide has to be reasonable, without hogging the processor.
Also, when you do find a hardware controller: will it run in your board? In other words, if it's PCI, do you actually have a PCI slot to fit it? This is especially a problem in a high-end consumer box or in a lower-end workstation, where you might have one or two PCI slots and the rest are PCI-E x1 slots. Where you're likely going to have a GOOD sound card and a capture card in your legacy PCI slots, or maybe a multi-port Firewire 400 card, where is the hardware RAID controller going to live? Obviously the solution is going to be to go with an embedded solution on the motherboard, hopefully with a model that doesn't totally suck.
The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
You seem to be resisting the group-think... surely you didn't read TFA, as your sig suggests. Here ya go.
I'd also like to see how a Pentium D would perform in the same system seeing as how it's socket compatible with Conroe. This will help isolate whether it is indeed the CPU or if it's more to do with the I/O subsystem(chipset).
This isn't really reasonable. Software RAID can be as good as many hardware RAID configurations. Modern CPUs are very, very fast, and in many cases can calculate parity faster than dedicated controllers, at the cost of some CPU overhead. However, the cost of the CPU can be much less than the controller. Also, large disk cache helps reduce the read-pairty-write overhead from RAID 5, but most systems have more cache than the drive controller you would pair them with. Finally, if my computer has a UPS, the value of battery backed cache is limited.
Obviously there are many cases where hardware raid is desirable, but to say that any "real" system must use the most expensive hardware possible is just wrong.
Did you get a refurbished drive that was once part of a RAID-5 array? ;)
Join Tor today!
I tried Raid5 once with 5 hard drives but one drive had a failure while I was swapping out one and the whole thing went kaput so I had to do the image all over again.
For saftey's sake, I just used 6 hard drives with 2 pairs striped and then had those drives mirror each other with 2 extra that would go to a drive when one failed. So in theory I could loose 3 drives instead of two and still keep my data.
And yes... This was on my personal setup for no good reason other than a big ego, but in reality Raid5 isn't that useful ore efficient unless you are using enteprise applications that requires 100% uptime and you have way more than 3 hard drives (just in case two of them fail on you at once for no particular reason) and then you should have that server mirrored by another one so if one server fails because of a bad motherboard/powersupply etc, you'll have another server ready to go.
"I am the king of the Romans, and am superior to rules of grammar!"
-Sigismund, Holy Roman Emperor (1368-1437)
No kidding. That part caught my eye, as we use IBM where I work, and everything is RAID-5. If there's some kind of problem with even the lower end xSeries machines, this will affect our purchasing.
Sorry, my bust, I'm illiterate.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
Your looking for "improved performance" out of raid5?
Never used it, or even read about it I'm guessing.
Raid5 while excellent for redundancy, is quite slow when you are talking about alot of writes.
"I could lose up to 7 days of recordings..."
Oh the humanity..
Because if your dedicated controller goes you have to find the same make & model of controller. On no notice. Possibly a few years after that make and model has been discontinued.
With software RAID-5, any controller that works with your host bus (PCI) and HDD bus (ATA, SATA, or SCSI) will do just fine.
Most onboard (intel) RAID controllers are only setup for 0,1,0+1, or 10. And not RAID 5. I don't see how it could possibly be correlated to the CPU.
That's because you can do RAID 0, 1 or any combination of 0 and 1 without needing parity data. The performance killer on RAID 5 (and any other form of RAID that requires parity) is in the XOR operations used to compute and verify the parity information. In order for RAID 5 to perform at a satisfactory rate and not totally bog down your CPU, the XOR calculations should be handled on a dedicated hardware controller, not in software.
However, for non-parity RAID setups the amount of CPU overhead is almost trivial, so referring to "fake RAID" or "software RAID" with the integrated RAID controllers on most motherboards is a misnomer. That being said, at least one of these articles is talking about servers using third-party RAID controllers.
RAID5 for less than 100$? Perhaps if you go with PATA (or those dinky 2 drive cards), but all SATA RAID (or SCSI) cards I've seen were all a LOT more than that...
Yes, Tekram has a TR-824 card for 50$, but it's not RAID5, and it's only old PCI (not PCI-X or PCIe) so not too fast.
Find me a 4 port PCIe SATA RAID card that does level 5 for ~100$ and I'm buying 4 today for myself. Willing to pay a bit more than that for more ports or a nicer/better card, but not the extorsion prices for the 3ware and such cards.
Why does Slashdot still accept article submissions from that piece of shit excuse for a website..?
Given hwo cheap disks with rebates are (and since we probably all have extra computers) a few hundred dollars probably IS doubling the cost of such an array. As for the cheap cards? Well have fun finding 3 cheap 333+ gb drives to get that 1TB array, because I doubt any of them will support more than 4 drives per controller.
Because hardware controllers add a failure point.
If your not going for ultimate disk performance, and your cpu isnt overly burdened, the use of software raid is sensible.
If you're running raid5 it's probably in an enterprise setup. If so, why aren't you running a dedicated controller? The CPU should have little to no impact on the raid subsystem...
The first of the two referenced articles talks about them using Woodcrest CPUs (the Conroe-based replacement for the Xeon server CPUs) on IBM systems that used IBM ServeRAID controllers. They obviously aren't talking about the Intel RAID controller integrated into the chipset. And while I agree that the CPU should have little to no impact on the RAID subsystem, especially if they are using third-party controllers, apparently that's not the case here. As to whether the problem lies in the CPU or in the mainboard chipset itself isn't mentioned, but it's certainly possible that there is a chipset bug related to high-bandwidth operations. The FSB architecture is relatively starved for memory bandwidth to begin with. Generally speaking, for a given number of hard disks a RAID 5 array will have the highest level of read throughput of the commonly deployed RAID versions. Could so much data be flooding the bus?
They specifically stated this was seen with an IBM ServerRAID controller. Not exactly a non-dedicated controller.
It IS the Inquirer, though, so the required grain of salt has been taken at my end.
It's almost a certainty that this is a software problem of some sort. Driver bugs are the most common source of "hardware" instability, particularly on Windows. Drivers are often written by clueless intern-level engineers, and quickly forgotten once the drivers initially pass based Windows hardware quality tests.
...are shite. Use an HP (compaq) SmartArray 64xx or an LSI MegaRAID 320 series controller. They're fast and quite reliable. I especially like how the MegaRAID will store it's configs on each and every array member hard drive as well as the card's eeprom itself.
Check out http://baarf.com/
BAARF = Battle Against Any RAID Five
Doesn't seem like a good idea anyway.
It wouldn't explain why he was faster if he was doing software RAID with SCSI, though.
I've done software and hardware RAID-5 with SCSI, using the same drives and cable, and can confirm that software RAID can be faster than a dedicated hardware controller, all drives and cabling remaining the same. Some hardware controllers can be pretty darn slow.
"3ware card for a few hundred bucks "
The nice 3ware cards for 100 bucks are NOT hardware raid, they use the CPU to calculate the RAID, it might even say it is in the literature but working at company (tech support) who sells servers that use 3ware for 80% of it's business, I can definitely tell you this isn't the case.
You CAN get a hardware based 3ware card, but then you are looking at 400-500 bucks (+some for the battery backup unit).
Plus if you read the parent correctly, 4 300GB hard drives for 50 bucks totals 200 bucks, a "3ware card for a few hundred bucks " WOULD double the cost.
Intel hints that Conroe is going to release at B-2 Stepping as Intel Core 2 Duo processor. As for the previous version, a problem was found to make the system full loaded. It's only solved in the new stepping. We don't encourage anyone to buy the engineering sample from the web. The retail version is going to release in the end of this month, and it's much stable.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
I've never heard it called FAKERAID maybe it should just be called FAID? I'll file that one back for use later...
Anyway, it's not entirely a hw/sw combo. These types of raid controllers are entirely software based. They consist basically of an ata or sata controller and an interrupt handler. When the disk is being accessed in legacy bios mode (ie during an os install, etc) the cpu pulls the interrupt to write to the disk and the BIOS calls the software stored on the card. This software is executed by the BIOS on the CPU and this code basically does whatever reading or writing to disk is necessary to keep the array consistent. Performance is improved after an OS's native driver is loaded since the software raid is done entirely in the driver.
It is arguable that software raid done at the OS level actually can have a performance advantage since it can know about file tasks at a much higher level than the driver itself, which basically only knows about block accesses. In the example presented in the article it seems that most of the tricks that software raid has been using do not perform well on the Core2 chips; I don't really see what the big deal is though; nobody has had a chance to write optimized routines for Core2. Probably their legacy BIOS handling ability is poor; so be it. There are a lot of alternatives out there from completely managing the array inside of software to full on ASIC-based hardware RAID.
"Modern CPUs are very, very fast, and in many cases can calculate parity faster than dedicated controllers,"
Especially given the fact that most CPU usage is less than 18% when calcualting parity for RAID 5, so compare 18% of your CPU cost and see if it is worth it to lose that much over head (or buy the next higher model) OR pay 400+ for hardware RAID.
I'd say RAID 5 is the best solution for home backups, you have redundancy for disk failure and with monthly or even quarterly backups you should be fine. (who wants to spend 12 hours a week backing up to tape?).
Can you niggers post real stories instead of spam/trash/rumors of beta software from spam/trash/rumor websites like theinquirer.net?
"Of course there are the companies that won't so much as touch software RAID (namely 3ware)"
Until the 9500 (and MAYBE 9000) series ALL of 3ware's offerings were software RAID. this is why you couldn't initalize a RAID array until the OS booted (and 10 minutes after that), it even says this in the manual.
I worked in tech support for a company that sold these for 2 years and talked with the 3ware techs quite extensively on the subject.
As a company they don't officially admit it, but from a technical standpoint it is obvious they only supported software RAID.
Excuse me, but isn't the function of the CPU to efficiently execute software written in its instruction set? If some CPU instructions are more equal than other CPU instructions, Intel should have said so a long time ago. That's like saying that its the fault of the road because some cars traverse it better than other cars. If the road is optimized for certain car types, it should be publicly disclosed.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
Although if you really want someone to do your legwork for you I'd point out that there are cards in the 100 dollars range with 4 channel hardware raid 5 from LSI, Highpoint, and Promise. I can't really speak to the quality of these cards as I use 3ware cards myself. But as with most things the more $$$ the more features, support, and reliability. Then again I don't really understand the need for super high storage speeds for TB+ systems in a home environment. Most TB+ home users are probably going to use that space for a big ass file dump where they toss all their music and video files and perhaps stream them to other devices... This isn't really heavily IO intensive...
-*The above statement is printed entirely on recycled electrons*-
Because such controllers use a proprietary disk format. So when (not if) your controller breaks, you need to get a new one to be able to read your data. Whereas when using a free (libre) operating system and software raid, you'll always be able to get to your data.
Error: password can't contain reverse spelling of ancient Chinese emperor
"Why aren't you..."
May be you should ask Intel?
The D975XBX motherboard have on-board RAID5 and sound. Is it an enterprise setup or not?
I dunno about that, with U160 and U320 you would have to use at least 5 drives or more to max out the bandwidth that U160 provides and upwards of 8 or more for U320.
No RAID systems use one channel per drive, maybe you are thinking connector, in which case that would be SATA, and possibly IDE to maximize bandwidth.
Most RAID cards have more than one channel per card, but multiple connectors per channel (7-15 on SCSI, up to 12 that I have seen on SATA, and 2 per channel for IDE).
RAID5 is extremely common in the home media appliance/server area. It's a small part of the market, but also a part of the market fairly likely to buy a core duo - they're lower-power than most competitors and they have enough horsepower to do full-HD video.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
It makes more sense to use ordinary software RAID than to use half-hardware RAID. Put the OS on its own (small) disk and stuff all your data into the software RAID. The funny thing is that software RAID can often give better performance than hardware RAID, so long as the box isn't using all its CPU for something else, because your primary CPU has more powerful than almost any RAID controller.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Linux actually has a number of different algorithms for doing the software RAID5 XOR operation and tries which one performs best on the given CPU during bootup. On certain other operating systems, this is typically implemented in each device driver, so the driver author may have chosen an algorithm that performs well on Pentium4, but not on Core2.
"I could lose up to 7 days of recordings..."
Oh the humanity..
True, it's only TV, but still, better with raid5 than trying to do weekly backups...
ps: TV does become a whole lot more interesting when you can choose from a large set of recorded shows, plus when you can play it back at 1.4x real-time speed with mythtv's fantastic 'time stretch', which speeds up TV shows without distorting the sound.
--- Hindsight is 20/20, but walking backwards is not the answer.
Oh wait, they did. Not only did they say so a long time ago, they publish documentation and maintain a compiler to help you optimize for the way their processors work.
I rarely criticize things I don't care about.
I'm just trying to be real here folks....
-*The above statement is printed entirely on recycled electrons*-
I just built a new file server for home use, using 4 300GB SATA drives and the slowest socket-939 CPU I could get (1.8Ghz Athlon 64) in a RAID5. I'm getting pretty close to the limits of the individual drives during testing, with just over 100MB/sec writes and just under 200MB/sec reads. CPU usage remains comfortably under 50% (that's the spikes; average is more like 20%) in that configuration, and dmesg reports my RAID6 checksumming speed at over 4GB/sec.
With the drives on the PCI SATA controller, I'm bus-limited (writes much faster to a degraded array, because only 3/4 as much data is actually being written to disk). With the drives on the chipset controller, I seem to be limited by the drives themselves. Hopefully later this summer I will be building a larger array and will be able to do more tests, but even with 8 drives I doubt I will be bottlenecking on the CPU.
High-speed Road Trip (18.000KPH)
RAID is not a backup solution. Anyone who tells you differently is either a fool or trying to sabotage your data.
robert
See www.baarf.com
Sorry, I don't know about the whole line, but the 7500 and 8000 series are definitely hardware. The 7500 page specifically talks about the hardware XOR engine:
0 3.pdf pages 25-27
http://www.3ware.com/products/parallel_ata.asp
The 8000 page also says: "True hardware RAID protection for all your valuable data"
http://www.3ware.com/products/serial_ata8000.asp
Not to beat a dead horse, but the Escalade 7000/8000 confiuration manual specifically discusses using the BIOS utility (invoked through Alt-3 during boot) to create/modify arrays:
http://www.3ware.com/support/UserDocs/7000IG_0429
Where do you see that any of their products are software?
Derek
Don't Panic...
No kidding? Did you read his comment? RAID is a good first line of recovery against disk failure. Then he goes on to say that if you have something more serious you restore from backups.
If it happens with a dedicated controller such as ServeRAID, then my first hunch would be that the chipset isn't handling memory contention very well. We used to see this at Dolphin a lot; the Intel chipsets at the time would behave terribly if there was any kind of serious memory traffic coming from the "far side" of the memory controller. This could also be a problem on the "softmodem-like" RAID controllers, where one core is trying to bring previously DMAed data in for its XOR while the other is trying to do its normal stream of memory accesses. It wouldn't be quite the same kind of thrashing as in the previous case, but it's very easy to imagine that it would still occur.
Slashdot - News for Herds. Stuff that Splatters.
"It's as if there can't be any gray area in the matter."
No, the problem is that there's no grey matter in the area.
...is a dvd writer. If you're generating more than 8 gig of backup-worthy data a week as a home user then you need to put down the digicam and get another hobby.
We run very "enterprise" setups where I work and you're right, it's "enterprisey" to go full on hardware RAID.
:-/
And deal with all the headaches that leads to:
1) Poor OS integration
** 3Ware
** Dell PERC
** Adaptec
need I go on?
You get alarms coming from inside the unit and you have no idea what the fuck is going on until you reboot and drop into the BIOS.
It's only recently that 3Ware started shipping a WBEM data source that could let you know what was going on with an array from MMC, etc. etc.
With Dell you have a propietary MMC snap-in which is a pain in the ass to keep straight depending on the particular PERC flavor you're using and firmware revision of your card.
*nix or BSD support? A shitty CLI or nothing. Or maybe a web management tool (WTF?)
There is no standard for this sort of thing and it pisses me off to no end. Hell, there's a fucking standard for removable drive enclosures, but not for offloaded RAID volumes.
2) Shitty firmware / embedded system design
I've been burned one too many times by firmware bugs and NVRAM failures.
3) Propietary on disk layout
* No way to swap drives between systems with different RAID engines
* Have to prepare unlabeled disks in BIOS/firmware tool before it can be used
* Boot floppies for OS installs onto array. God, what is this, 1992?
The only RAID offload I ever use is complete offload. Seperate NAS/SAN devices with an embedded OS that present the volumes as LUNs or network shares. I've found these to be engineered to a much higher tolerances and you can always connect a TTY to it or SSH in and figure out what's going on with it.
Other than that we completely standardize on software RAID. CPU performance these days is sufficient enough that RAID-5 isn't a particularly notable hit. We'll actually buy 3WARE 95xx cards, and put in 12 JBOD disks just so we can have hotswap.
It's really nice to be able to pull a subset of disks set up for softraid in Linux, and plug it into any old machine and have it magically appear.
So yeah, software RAID is your friend.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Damn, let me download some!
No, it's aFRAID array. Oops, forgot the space.
RAID5 can cause extra bus traffic to and from the controller. You need to check to see if your bottleneck is the card itself when you like 8+ drives attached.
It helps if you have more than one controller on seperate busses and you can stripe across them RAID50-style.
(Recommended on Opteron based systems with multiple CPUs and MCPs)
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Even 1gbps is a whole fucking lot of sustained disk traffic.
The example is hyperbolic.
You're better off using multiple RAID5s instead one large one, or better yet, using JFS or something that understands stripe sizes so it doesn't cause unnecessary parity reads for partial writes.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Backup is a simple issue and a huge issue, and the playing field drifts. Intuitively simple, but very complex to manage when you grow into the enterprise space.
An interesting issue for us, because the moment we did away with MT backup and replaced them with nice little SEAGATE USB external hdd's (we're *very* distributed) they started getting nicked, so the security people keystone locked the little items next to the server. Result is we're keeping the backup media next to the server, even after one site burnt to the ground. Comedy is not pretty. I suggested burying a unit in a floor safe underground (cable through the sealed cash slot) but the only people who thought that was a good idea was the Fire Department. RAID ain't backup, it's hardware fault-tolerance. There's a difference.
Do not mock my vision of impractical footwear
No, no, no, no. The processing overhead of parity calculations is miniscule on any remotely modern CPU (even a paltry 300Mhz Pentium 2 has a parity throughput of ~700M/sec).
The performance killer on parity-based RAID configuration is the additional disk reads required to calculate the parity, *not* the parity calculations themselves. Which is why modern software RAID is typically faster than hardware RAID until you get into larges numbers of disks and/or machines with limited bus bandwidth.
This "RAID 5 is slow because of parity calculations" meme must die (although, admittedly, it's a good indicator of whether or not someone really understands what's going on).
It's got nothing to do with CPU power. It's because the OS almost certainly has vastly more memory available (how much RAM in the system vs on the controller ?) and is able to better utilise that to cache data and avoid physical disk activity.
The (relative) slowness of RAID 5 does *NOT* come from calculating parity, it comes from the additional physical disk activity getting the data to caclulate the parity (and the subsequent disk activity to update parity and/or data blocks on disk) entails.
Users who should be using RAID 5 should also have dedicated controllers.
Unsupported assertion.
Here's mine: "People who are self-described social conservatives should be beat with bricks."
I like mine better.
The fact is that my friend, who knows more about RAID than everyone else reading this (if you tie his knowledge you are very, very good) told me that RAID 5 in software doesn't take that much CPU and hardware RAID is pointless for me in my NAS.
Look at what the Internet Archive uses (no link because if you can't find it, you are probably a social conservative and should report immediately to the brickyard).
CPU usage is 100% when calculating anything. In fact, at any given time, CPU usage is either 0% or 100%. It's either doing something or it's not. Same with network links - an ethernet card is either transmitting a packet, or it's not. It's only when you average these things out over periods of time that you see percentages like 18%. In the meantime, your other processes wait until the interrupt handler finishes or the scheduler gets to them. A 100% idle system is not merely 122% as responsive as an 82% idle system. Also, as another poster mentioned, software RAID5 may potentially be slower for reasons other than the CPU.
BTW, regarding one of your other posts, I own an old 3ware 6410, which you allege is software RAID, and I distinctly remember going into its BIOS to create and initialize its RAID5 array. The Linux 3ware driver source is public - show us the part where it does the parity calculations.
Consider using rdiff-backup to automate syncing to an external hard drive.
and I just spent all this money on my Core 2. Or wait . . did I get the core Duo . . . er . . maybe the Core 2 Duo. Double Duo 2?
Conserve Oil, Recycle, Boycott Walmart
SAS and SATA RAID cards have one driver per channel
look at Tekram ARC and 3ware
"Before deploying half-assed solutions, it's usually better to deploy nothing at all."
You obviously aren't a manager who "spearheads a new e-commerce initiative in customer motivation" while telling the staff to buy $299 servers from Dell or whatever, refreshes the ol' resume the week before the system goes into production, and "transitions" to a new job using the well-worded but half-assed system as a major bragging point during the interview.
most of the on board raid and some raid cards are part hard ware and part software also you can not boot form software raid 5 useing windows build in raid.
I'm inclined to think that flooding (or fire, hurricane, or other insurance-termed Acts of God) will cause problems for most backup mediums, not just RAIDed hard drives. In fact, if what they say about the life of a burned DVD is true, then hard drives are likely the best solution for most home users. Any solution can naturally fail, but hard drives under normal use have pretty good lifespans, and that being combined with a parity bit or proper mirroring should be quite reliable within the home. I don't bother with RAID in any form, though if I were to set it up on my fileserver, I would go for RAID5. I've got enough of my stuff replicated in six places that I'm not especially worried. Hell, even if my house burned down and cooked every bit of data I posess, a decent amount of my photos and whatnot would be recoverable as they're all uploaded to photobucket, and then Google Cache has probably made a mirror of them from wherever they were posted.
How are sites slashdotted when nobody reads TFAs?
What you described has come to be called FakeRAID. This is basically when there is some additional software in the BIOS, but that also requires a special Driver in the OS to handle it. Software raid is on the partition level. FakeRAID can be done on the disk level.
There's no "middle ground", there's cost-benefit analysis.
I.e., is it worth my time to spend $50, $100, $200, $500, etc, and an hour a week to mirror a pr0n collection? Some people would say $50 and 5 minutes, and others would say $500 and 6 hours a week. And some would say, "Chunk it. If the disk dies, I'll just download it all again."
"I don't know, therefore Aliens" Wafflebox1
I hear you talking but it sounds like gibberish.
Here http://www.newegg.com/Product/Product.asp?Item=N82 E16816116036 a nice 3ware, hardware RAID controller for ~$100.00. I use this.
Software RAID is good for speeding up your disks, but I would never do Software RAID for data redundancy. Nor would I use it for east of recovery after a failure. I have used it for several years and the failures are nothing but trouble to get going again.
Hell there are only a few OS that can boot from software raid partitions. And your own personal education level will have to go way up to get it working anyway.
This $100 hardware raid card is no worries. Disk crash? put new disk in, reboot, tell RAID in BIOS to add disk to array. Finished. BTW, you get an email when the disk is acting funny or when it drops from array. Also supports smart. Can't be beat.
These 3ware cards are definitely hardware RAID. You are spreading FUD.
The parallel card is the $110 on newegg.
From Newegg StorSwitch switched architecture delivers the full performance benefit of Parallel ATA's pointto- point architecture up to 133MB/sec per port On-board processor provides true hardware-based RAID and intelligent drive management functions BIOS set up utility and 3ware Disk Management (3DM) web-based management software Bootable array support for greater system fault tolerance
http://3ware.com/products/parallel_ata.asp
That's a crock.
Every non-trivial decision needs some form of cost-benefit justification.
Sure, U320 SCSI controllers with a GB of batter-backed RAM are fast and durable, as are 15K RPM 300GB Cheetahs, 4Gbps fiber and Hitachi 9980 SANs with 128GB of cache.
However, not all data deserves to be stored in an Hitachi 9980V. And not everyone can afford them. Some organizations can only afford FAKERAID.
And some managers are stupid, cheap bastards.
"I don't know, therefore Aliens" Wafflebox1
Yes and no.
Yes, because the mathematics is trivial.
No, because, as you said, in order to do the calculations, all N disks must be read.
That's why, though, Big Controller Cache makes RAID-5 speed just about on par with RAID-0.
"I don't know, therefore Aliens" Wafflebox1
Folks here are dissing software RAID a tad too much. This isn't the dark old days of critically limited CPU and bus bandwidth. With dual core CPU's, more often than not CPU processing capacity is far in excess. So, for many reasonably light loads, why not save a few bucks on the RAID controller? Heck, for the cost of a "real" hardware RAID card (never mind a battery backed one), you can buy a 2nd CPU which will be more flexible since it'll help both RAID speed calculation times and regular processing tasks. Sure, you wouldn't run top end database servers on software RAID 5, but there are many applications where speed is not important, but downtime is.
Sure, an adapter card.
SCSI disk controllers let your RAID-5 sets stripe across channels to get extra bandwidth.
Maybe this is just a DEC feature, though.
"I don't know, therefore Aliens" Wafflebox1
Buyer beware: That's a IDE (rather than SATA) controller card.
That's big-E Enterprise.
I'm talking about sub-1m yearly IT budgets here *eye roll*.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
HTX based _anything_ some day.
That being said, that means every device would have to have carry a PCI bus emulation layer with it.
OS's don't know how to negotiate resources unless it looks like its connected on a well-known logical bus.
Unless you've got some magic "platform" driver that probe-detects it and works without resource allocation (you'd have to negotiate your IRQ for OOB notification through some OS-negotiated process... Hello PnP!)
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Well that's certainly possible. The card only had something like 128mb of onboard ram where the server had 2gb. However, I don't see how that could be a major factor. If I do a random read and that data is cached on the system, I get it from cache regardless of whether I am using softraid or hardware. Same for a write, it's gonna use system ram for a bit and flush it out when it's good and ready regardless of the underlying hardware.. In otherwords, the raid should still benefit from the 2gb of ram even in a hardware raid configuration.
I disagree. Usuable RAID-Controllers don't cost that much. They are in the price range from 750 to 1000CHF.
That's not free, but it isn't expensive either. Think about it this way: An Ultrium 3 Tape Library costs about 20'000CHF. Does it really matter if you spend 1k CHF more on your server, when your Backup costs 20kCHF anyway?
You use XOR to clear a register. XOR CX, CX sets the CX register to 0. It is faster than MOV CX, 0.
wrong !
The instruction is shorter (2 bytes instead of 3 or 5, depending on the word size). But it sets flags on return while MOV CX,0 will keep them intact, so the CPU cannot reorder instructions around a XOR. In the best case (no dependency), they will all need 1 cycle. In the worst case, XOR will stall the pipeline while MOV will not.
Err, NO! It's about FAKERAID, which is a H/W S/W combo.
RAID stands for Redundant Array of Inexpensive/Independent Disks. Nowhere does it say "Controlled By A Dedicated CPU" ("RAIDCBADC"? Doesn't quite sing like "RAID"). Software RAID is as much RAID as a top of the line server RAID controller with RAM and a battery backup. It isn't as fast, sure, and it loads the system CPU, but it is still RAID. Calling it "FAKERAID" is just pretentious and misleading. The data integrity benefits are still present, as are some performance benefits in some circumstances (in fact, Linux RAID is demonstrably faster in some workloads than a top end Adaptec hardware RAID controller, though this is the exception rather than the rule)
That said, I hate pretty much all RAID controllers (whether software or hardware). Linux software RAID means that I can drop the disks into any PC and access the data. Every RAID controller from Promise, Adaptec, and Tektronic requires me to use their disk format, and if I lose the controller I lose the data until I can get another controller. Sure, in high availability environments, you keep a spare...but with Linux software RAID, every PC in the office is a spare controller. That's my kinda redundancy. I've even had two identical Adaptecs with different firmware lead to pretty massive data loss during a server migration. Thankfully there were good backups. I've never had similar problems moving Linux software RAID disks into a new Linux box.
We create snapshot based online backups from our system nightly. Takes about 6 hours to backup our systems to an ultrium 3 library with multiple drives. Data is about 1.5TB now. You can access the system at that time without problems, performance is a bit limited though.
The advantage comes from fewer disk reads necessary to do parity calculations during disk writes. With more cache, there's more chance all the disk blocks that would otherwise need to be read and/or updated, are already in the cache. So, when your system has 16x as much RAM as your disk controller, it's far, far more likely the software RAID won't need to do as many physical disk operations due to the disk blocks necessary for updating parity already being cached.
It's not the data on the filesystem level being cached that's delivering the benefit (although that is somewhat relevant, as software RAID can use knowledge of the filesystem to improve caching, whereas hardware cannot), it's the data on the disk block level.
You would be surprised how much difference this can make. One of the main reason SANS and (good) NASes are so fast (relatively speaking) is because of the massive amounts of cache memory they have in them negating the additional disk activity inherently required for parity-based RAID configurations.
This isn't new. Hyperthreading puts a performance hit on software RAID 5 as well, but the impact isn't nearly as noticeable. The problem is that shared caches do poorly with the kind of workload that RAID 5 imposes on a system. HPC users have been disabling HyperThreading for years for the same reason. The higher degree of separation between execution threads on a multicore chip magnifies the problem several times.
They might be able to improve the dynamic cache sharing policy with a microcode update, but they should probably also work with OS vendors to optimize their RAID 5 drivers to be friendlier to chips with dynamically shared caches.
There's no failure quite as dissatisfying as a complete and total solution to the wrong problem.
This is all rather silly. Nobody who needs RAID performance uses software raid, especially the crappy software raid found in most MB chipsets. It's just plain stupid. I wouldn't trust that junk even if I didn't care about performance! Use a hardware raid solution like a 3ware card or something similar (in particular cards with battery backup capabilties) and wash your hands of the whole affair.
-Matt
>I've never heard it called FAKERAID
It's WinRAID, like WinModems, WinPrinters, and other pretend technology.
This can go both ways. A 0% idle system can be nearly as responsive as a 100% idle system. A purely CPU bound task (such as SETI) will not interfere with an interupt driven task (such as IO) (neglecting cache flushing effects).
Regardless, for most non-desktop uses, the latency caused by scheduling overhead is a negligible contribution to total system performance. The important factors are CPU throughput and IO latency. 18% parity overhead reduces throughput 18% (or requires a 22% faster CPU to maintain throughput) and the IO latency and bandwidth requirements of RAID5 are independent of whether the pairity is done on a controller or by the system CPU, though they *do* depend on where the cache is.
Personally, I use software raid5, raid1 and single disk systems which rsync cross-backup each other to provide tolerable limits on downtime due to disk failure as well as backup for data corruption/deletion failures. I have considered getting 3ware cards on occasion, but for the uses I have it just didn't make sense. Plus, software raid has the advantage of being independent of controller type.
Implications: "Home users shouldn't use RAID5" because its not "backup". Enterprise customers should use "hardware RAID5" because its somehow superior.
What utter bunk.
I like my data; I back up my data; and I use RAID5. NOTHING is more important than my data. Now, the big thing about RAID5 is computing the checksums. And how fast is that? Lets have a look (from ganymede, my storage box):
Adding RAID1 or RAID5 is a graid5: measuring checksumming speed
8regs : 430.800 MB/sec
32regs : 219.600 MB/sec
8regs_prefetch: 430.800 MB/sec
32regs_prefetch: 219.600 MB/sec
pII_mmx : 532.000 MB/sec
p5_mmx : 554.800 MB/sec
raid5: using function: p5_mmx (554.800 MB/sec)
md: raid5 personality registered as nr 4
Look at that carefully -- we are computing checksums at a rate of 555 MB/sec. And what is this mysterious box?
A Pentium II/266. That's right, a bleeding Pentium II/266.
Quite fast enough to do "software RAID". So, would adding a dedicated RAID5 card help? Nope, you would never see the difference. We are faster than the drives, and the network. The bottleneck is NOT the RAID5 software. Go back and look at the machine spec again.
What happens if I put a RAID5 card in this system? Um... right, another potential point of failure. By Crom! Lets not do that, and increase reliability! If you need 24x7, you would need redundant hardware anyway (including CPU and disk controllers). If you DON'T need 24x7, do what I do, and invest in a "cold spare".
Ratboy
Just another "Cubible(sic) Joe" 2 17 3061
Running RAID5, you NEED to do the parity calculation when writing to the array.
When reading? The data rolls off FASTER because it is effectively striped. And... heres the kicker. UNLESS there is an error during reading, you don't need to do the parity calculation.
So, for reading, RAID5 is almost on par with RAID1, *not* RAID0.
For writing? The largest bottleneck in a RAID5 is (normally), the need to read the entire block for a small update. Why? because the checksum has to be recalculated.
And this is where a "software raid" can have a MAJOR advantage over "hardware raid". Specifically, if the FILESYSTEM layer knows that a block is "fresh", is doesn't have to be read before the checksumming. A hardware implementation can never know this, and is forced to do the block reads.
Ratboy
Just another "Cubible(sic) Joe" 2 17 3061
Going into the BIOS prooves nothing. I have promise cards that are software RAID and yet I can go into the BIOS and initalise an array. In fact, it is through this BIOS that software raid cards are bootable.
:-) )
The trick is that the card has an "extenion ROM" on it that hooks interrupt 13H (yes, this is all old-school real mode stuff!) which is the interrupt used when BIOSes, real-mode OS's and most (all?) bootloaders call to load a sector off disc.
This hooked int13 of course runs the code stored on the card's ROM which can then do any parity calculations/mirroring or whatever else. This piece of trickey is what lets you boot from a software raid card as the BIOS uses INT 13H to get the boot process underway, and what lets MS-DOS or any other such OS access the raid without any special drivers.
However, once your OS transitions to protected mode, it's usually quite a hack to continue to use the real mode 13H interface, so your OS specific protected mode driver normally does the parity calculations. And even if possible, there is a performance hit.
(I believe that Windows 3.1 did do something like this as it still used MS-DOS for disc access. "32bit disc access" in Windows 3.11 was all about using protected mode disc controller drivers instead. I think that NTLDR, the WinNT/2k/XP bootloader, may also call INT 13h from protected mode mainly to maintain compatibility with situations like this.
You're right though that auditing the linux driver will proove the matter. But I am aware that some linux drivers for such cards just exposed the IDE interfaces of the card as block devices to the OS - effectively, only supporting JBOD. Linux software mirroring could then carry it the rest of the way to get it all working.
(An extension bios is a rom that has a certain magic checksum and is thus automatically executed by the main system BIOS during POST. Often video cards have one too, and they perform rudimentary checking at boot time and/or display annoying pictures. When initially executed, the extenion bios may hook an interrupt allowing the extension bios to intercept calls to that interrupt in the future.)
In theory, a true hardware RAID card could hide all the details from the OS, making it look like just another HDD controller as far as the OS can see - so no specific driver is needed.
"If you're running raid5 it's probably in an enterprise setup. If so, why aren't you running a dedicated controller?"
'Cos it's cheaper not to, that's why. I've seen plenty of installations that have large disk arrays running under Veritas Volume Manager and the performance is perfectly adequate for the client's needs. Disks are cheap, controllers are not. It's RAID - Redundant Array of INEXPENSIVE Disks - not RAIDVEC (RAID with Very Expensive Controller).
Lol, beware? I guess its kind of a problem that my old computers don't die. I bought the Harddrives first. I think I ran into a drive size barrier on my computer so buying the hardware PATA RAID card solved two problems.
Beware: They have SATA versions as well. Not all of their SATA support hot plug though I don't think. Pay more get more...
Beware! Not all IBM ServeRAIDs are dedicated RAID controllers. In fact most of built-in ServeRAIDs are software ones: the (in)famous ServeRAID "e" series (ServeRAID 7e and ServeRAID 8e, that you can find in most xSeries 206/206m) as you can see in the ServeRAID Red Book:
http://publib-b.boulder.ibm.com/Redbooks.nsf/Redb
I don't think a real dedicated controller (like ServeRAID 6m) could have some impact on the CPU apart from saturating some I/O channel.
Argh. As I posted elsewhere, the presence of a BIOS does NOT proove they are not software RAID. In fact, the BIOS is integral to how software RAID systems boot! (At boot time, the BIOS hooks Int 13H allowing the card's BIOS - which is a program that runs on the main CPU - to do specific processing for the RAID, such as parity calculations or mirroring or whatever - but invisibly to the main system BIOS, operating system bootloader or old-skool OS's like MS-DOS.) Effectively, it hooks in its own disc controller driver.
8 1&cid=15674142
http://hardware.slashdot.org/comments.pl?sid=1904
(In fact, this same trick of hooking INT 13h is what allows boot roms on network cards to work - and that's 100% software as the disc access requests are being directed to an image in ram!)
A hardware XOR engine doesn't proove it either - it's possible that the hardware XOR engine runs under control of the OS driver (driver says load this block from this disk, driver then says xor that with this) which is somewhere between fully hardware and software. Not saying this is the case, just that it's theoretically possible.
Now, if you give me a RAID card that works under Linux, Windows, NetBSD and whatever else operating system because it emulates a standard SATA or IDE controller at the hardware level (so much so that you use the same driver for the RAID card as you do for an ordinary controller) *then* you definitely do have hardware RAID. So far, I'm yet to see a card that will work with standard drivers though. What also is definitely hardware raid is external boxes that pretend to be a single disk to a SCSI controller despite containing many drives internally.
I use one of the GMail file storage front ends, which is great for critical documents.
Information wants a fueled airplane waiting at the hangar and no one gets hurt.
Easy enough to put the O/S on top of RAID1 as well (assuming that we're talking Linux).
After working with Software RAID for a while, I definitely prefer it over more proprietary RAID solutions (dedicated RAID controllers). Software RAID is extremely flexible, easily recovered (move the disks to another box) and my RAID array doesn't depend on a single piece of hardware that may be difficult to replace if it fails. For smaller companies who can't afford to buy three of everything, that's a big advantage.
Mdadm also does a good job of letting you run without a config file. That means you can move a disk from a failed controller to a spare port on another controller and immediately get back up and running.
Wolde you bothe eate your cake, and have your cake?
What about hotswapping? AFAIK, you still can't do that with software RAID and SATA drives... (at least with Linux 2.6.x - never tried with Windows or Mac OS X). A hardware (SATA) RAID controller is your only option, unless you spend $$$ on a SCSI disk subsystem
Wrong.
Same Make, maybe, but not same model.
I didn't have to do anything when I upgraded from a Compaq Smart Array 2 to a Smart Array 4200 to a Smart Array 5302, it just worked.
Been there, done that. Then you have to hunt for one on EBay. NOW. Ask the seller to send it via hand-delivered overnight courier. NOW. For $200 extra.
And the controller that got wasted had its RAID config stored in NVRAM and not on the disks... And your predecessor did not write it down somewhere... Which makes the new controller useless....
Fortunately switching NVRAM chips works. Never been glad to see NT 4.0 boot in my life, except for that one single day.
The dangers of excessive individualism are nothing compared to the oppressiveness of excessive collectivism
I used to run linux RAID 1 and RAID 5 at home for pretty much the reasons you cite. I now only use RAID 5 and that will be replaced when I'm able to do so. Why? Too many corrupted files. One of the "faults" with RAID is that it provides no protection from file system errors. Please note that I didn't rely on the RAID for data security, I have a regular backup plan as well to DVD that provides a historical archive.
Because I was using software RAID 1 in linux converting to an rdiff solution was trivial (more or less turn off the raid, setup rdiff). That is one thing I really do like about software RAID 1. When I set it up I didn't have excess storage space to mirror, format and copy back. I got a second drive, told linux it was now a RAID 1 setup and it "repaired" to the new drive. Great stuff.
The reason I used RAID 5 on one system was I had a supply of 9 GB SCSI drives. RAID 5 offers more efficient redundancy so... but unfortunately those 9 GB SCSI drives are aging. I've had two of them die and am running low on spares. I can, at a fairly reasonable price, buy a couple of ATA drives and have redundancy (either RAID 1 or using rdiff) with even greater storage capacity. And fewer drives taking less power and generating less heat.
In sum? Although you can setup a RAID for a home setup it isn't necessarily the best way of introducing redundancy (all the file system eggs in one basket). Using something else, like rdiff, while not as efficient as RAID 5 offers greater redundancy and historical data.
thoromyr
Cite? I'm looking at Newegg's site now and the cheapest 3ware card I can find is $175 and doesn't even do RAID5.
... And so it comes to this.
RAID has a history independent of the expansion of the acronym. That history means that RAID is associated with and means hardware unless the term RAID is modified with the term software. You may not like it, but you do not get to rewrite history just because of your preference.
In the context of this alleged Intel problem, it is misleading to call this just a RAID problem and not specify that you mean software RAID.
on the relative slowness of RAID-5:
your right that it's not the parity calculations soaking CPU time that cost you, however, it's not the read latencies that get you. You calculate parity on write, from data you hold in memory. What kills you is every block written to the logical drive must first be written to memory, parity calculated, and only then written to disk. It isn't the CPU usage that is the problem here, but the latency you incur from those (albeit fast) calculations and memory operations. TO contrast that, RAID-1 requires only that each block be written to 2 physical drives instead of just one. The only added latency here is that you must wait for both drives to complete the write and you will always wait for the slower one.
Gee, I've been buying dell and hp comntrollers for years. Never had a problem with doing controller swaps. Both of them write the configs to disk as well as nvram, so its 2 minutes spent in the contorller's bios and i'm back on line. Plus I spare controllers, and in the 6 years i've run this shop, I have only 5 raid controllers in my inventory. I've even got in the habbit of ordering servers with hot spare drives in them, so i just rebuild and not worry. I've been hot sparring in my SAN for ages, and I sleep better with it. the short and sweet of it is, I'm not knocking linux software daid, but I just dont trust software at all for that stuff. and in all the years i've been doing this stuff in an enterprise environment, i've never had a major problem running hardware raid controllers.
For less than $100 I can the same things with mdadm and smart. Software RAID is built into the kernel so it's more likely to work better than the drivers for whatever card you have.
Case in point. I ran software RAID on a file server at work for years. I started running of space and decided to upgrade the machine. I decided now was the time to "do it right" and ordered a hardware RAID card and new disks. After 2 months of running ok I started suffering from random system failures, one, two or more disks would fail at once. Rebooting the system would bring them back online. After 3 months of messing around with Dell and Adaptec I ripped out the card and started using software RAID again. I got better performance and no more crashes!
I've also suffered disk failures under software and hardware RAID and honestly, neither works any better than the other one for recovery. The one really nice thing about hardware RAID however is the RAID set appears to be just one drive to the system and you're able to install without any funky paritioning or initrd's.
Like I said, it DOES say in their documentation that it is hardware, they can SAY anything, the proof is that if you read further into the documentation, you will see that any initialization occurs 10 minutes AFTER the OS has booted.
The hardware RAID is just marketing BS, from years of supporting these cards I can definitely tell you they are NOT hardware based.
Yes you can "create" an array in the BIOS, will it be initialized or useable until the RAID driver has loaded into the OS? No.
Raid5 is expensive for all but servers and you need hot swappable drives to replace a bad drive and continue on.
For cheaper you can purchase a DVD rw and Roxio that can archive your system. I have never had a hard drive fail on my home computers. I dont leave them on 24x7 but alot of my drives are older adn therefore better built.
How many disks do you need for a raid5 setup? Its been years since I was in IT but I remember seeing at least 4 disks. Also joe six pack doesn't want to open his box and try to guess which drive failed in his raid and replace teh drive himself. So raid is not recommend for all but workstation users who know what they are doing or work in a place with an IT department.
Raid5 also slows down access with all the mirroring and ecc parity stripping going on. This would annoy a home user and slow it down considerable. Fine for servers if you have a fiber connection and 5 drives spinning simultanously.
http://saveie6.com/
You are confusing channels with controller ports. SATA raid cards have one driver per controller port but multiple ports on a "Channel".
From the user manual of he 7000-8000 user guide (pdf) found on page 42:
11 Press F8 to save your configuration and reboot the system.
The rebuild will start within a few minutes of the 3ware driver
loading, once the operating system has booted.
The 3ware 7506 does not do hardware RAID by itself.
I have talked extensively with 3ware engineers and can assure you this is the case. They might have aprocessor on the card but it is using the CPU to do RAID calculations.
If you want to compare marketing blurbs with real life experience diagnosing and troubleshooting these products, then by all means go ahead, but saying I am spreading FUD is not informative.
BTW, regarding one of your other posts, I own an old 3ware 6410, which you allege is software RAID, and I distinctly remember going into its BIOS to create and initialize its RAID5 array."
t h=/download/Escalade7000Series/7.7.1/3w-xxxx.tgz
Correct, you setup the array in the BIOS, but it will not build or rebuild the array until the OS is booted and the driver is loaded:
From the user manual of he 7000-8000 user guide (pdf) found on page 42:
11 Press F8 to save your configuration and reboot the system.
The rebuild will start within a few minutes of the 3ware driver
loading, once the operating system has booted.
I know this is from a later model, but I am certain it is the same way for your controller, they didn't start hardware RAID until the 9000 series.. (I have talked with their engineers extensively and worked with their hardware in real life situations for years).
I'm sorry but I won't be spending any time pouring over code to find where they do the calucations, but that would be a great place to look, any coders out there wanna give it a try?
http://www.3ware.com/support/windows_agree.asp?pa
Just click agree to get the 7000 series linux 2.4 source.
Should have read "one drive per controller port".
Daniel, Fran, What say you guys? Is it the usual management chain crap, or is there a tech problem? Cache thrash? More test monkeys would help, automatic testing is only good for regression, not for new issues and unfound problems. Have Steve pull Melissa off, and put her on something she would be competent at.
Well apparently what they previously published has changed radically, since the code in question seems to have no problems running on earlier processors. Does Conroe need everything recompiled just to run efficiently on it now? Do we need multiple binary distributions to run on different Intel IA86 processors? Did you even get the first point in the original post?
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
If you're not spending more than $1,500 USD on a HW RAID controller, cross your fucking fingers :)
Sticking feathers up your butt does not make you a chicken - Tyler Durden
The next issue is what else doesn't perform as expected. Is this processor actualy backwards compatible? Will any other thing creep up causing the system to not perform as expected. While this is more of an "lets hope not" or maybe a "not likley to happen" type situation, anyone making buying decisions should be aware that it could be a potential problem. Think about it as if you bought a new dell poweredge server using a couple of these processors and ended up getting less performance averall then if you had recycled a P133 workstation and converted it. Now, if your the one doing the selling (consulting firms resell these) and this problem creeps up, it can damage your reputation as a provider. If it makes your applications become unavailible, it can damage the business as well as COST for upgraded applications that are optimised for this processor. Sometimes this can be in the area of more then the server itself. The last acounting package I helped upgrade (because of XP incompatabilities) cost around $25,000 for software along then we had to puchase a support contract on top of that to get some data transfered to the new system. I don't think a company would want to do this because they purchased an intel dual core based server instead of an outdated intel P4 (itanium, P3, whatever) based server or even an AMD operatron based server.
There are probably more scenarios to it then this. But these do seem to provide a big headacke potential. I guess the likley hood of any one of those scenarios happening are minimal but it is a potential disaster for someone. It could devestate small consulting firms if thier new server they provided didn't work correctly. It could also potentialy set back any other companie who was forced to upgrade this software because of this.
Calling it "FAKERAID" is just pretentious and misleading.
AFAIK, in the linux kernel driver code, it *is* called "fakeraid".
Issues with the chipsets or much more likely their drivers haven't been eliminated, and that's entirely possible because they're new too. These are much more likely as candidates than a CPU being mysteriously slower than it ought to be.
But, that being said... of course what they published previously has changed radically. That's what happens when you throw the old architechture out and introduce a new one. What did you expect? How could they or any other company ever make advancements if they weren't willing to change things? They have an obligation to maintain compatibility with previous x86 chips, but in no way can they be expected to give them identical performance characteristics. Netburst's performance characteristics suck; that's why they're throwing it away.
No more than for every other implementation of the x86 ISA. Some performance sensitive things like video codecs have different code paths for Athlons, Athlon 64s, Pentium 4s, Pentium Ms, etc. This isn't exactly the first time a new iteration has been released. For most things it doesn't matter, and where it does matter the developers are used to having to perform new optimizations every few years.
The more you optimize software for a particular chip, the more you'll run into problems on different chips. That's how optimizations work.
I rarely criticize things I don't care about.
What you said was valid... 3 years ago (the page you cite is 3 years old.) Since that time, Linux has implemented write barriers to precisely fix the problem you describe. With write barriers, the fs layer is now able to force data blocks to be committed to the device to enforce write ordering constraints (because the disk cache might choose to flush data in random order). And if that fails, it knows it and the journalled fs can act accordingly without corrupting the fs. The MD layer also implements write barriers, and advertises the capability to filesystems.
All in all, as of today, raid cards with battery-backed cache are not needed for Linux to implement reliable filesystems over software raid arrays. These cards are only useful for other OSes whose filesystems to not use write barriers.
What about hotswapping?
I don't do it, and I've never been in a situation where I felt like I needed to be able to do it. If it's a critical server, it has a hot spare disk (which is supported by Linux software RAID) so I can schedule downtime at the earliest convenient time. It it's not critical, I schedule downtime roughly immediately (i.e. end of business day, or whatever fits). I don't have any systems that must always run, forever and ever, amen. I just don't. I'm sure there are situations where it's necessary, but I've never seen one.
I used to believe in hardware RAID, but I've come into enough situations where the prior admin had setup critical systems with hardware RAID and were using whatever controller Dell shipped them--and they didn't have extras on-site. When the RAID controller failed, and I've seen them fail enough to consider it a real threat, the wait was hours or days while Dell got a new controller on-site. Sometimes at great expense.
Thankfully, I don't have to worry about that sort of thing these days (might have to again in the future, but IT admin is in my past for now). But at least a few of my most hair-raising IT experiences were due to problems with hardware RAID controllers. I'm not saying they don't do their job, or that a good RAID controller doesn't do a better job (for some definitions of "better") than software RAID. I'm saying that if you don't have a spare for every RAID controller in your office available, on-site, you're going to regret it one day. Even if you do, you might still regret it--as I mentioned, I've met two identical model Adaptec controllers that were incompatible due to different BIOS versions. Data was lost and had to be recovered from backups. I've also managed to screw up the RAID table due to human error (sometimes those Sunday late night upgrades go awry) in ways that would have been less likely to happen and more easily repaired in Linux, but the limited nature of the OS available on a hardware RAID controller made it far easier to screw up and more difficult and time consuming to repair. Having a whole OS with excellent partition and array management tools and a real shell to explore the situation with is a mighty fine thing when trying to figure out what stupid thing you did that led to a system not finding any system disks on boot.
If you need hot swap, you'll need a controller that supports it...though I feel obligated to mention that hot swap is a part of the SCSI and SATA standards, not any magical RAID feature. It can be implemented on any SCSI or SATA controller, regardless of RAID capability, and some Linux drivers probably support it (though I've never needed it, so I don't know for sure). Though it's probable that the controllers that most often have support for the hot-swap features are the ones also equipped with RAID capabilities.
BAsically I got hardware RAID because its so easy to use on failure. I don't have time to worry about bootsectors and which OS can boot in RAID and which not. Hardware is flawless.
I might also point out that you too have drivers for your drive controllers. And you have drive controllers. Using Hardware RAID does not inject more hardware into the equation than what you have.
Besides, Software RAID is not the same as hardware RAID. one RAIDs partitions, one RAIDS drives. Sorry about yoru bad experience but I have been using software RAID on Redhat for about 4 years now. The hardware RAID setup was much simpler and have not had any issues as yet.
I should also say that FakeRAID offers the worst of both worlds...
http://www.newegg.com/Product/Product.asp?Item=N82 E16816116036
Since i am using a 3ware card I am fully aware of this. I say your spreading FUD because you are saying its not a hardware RAID card.
If you want to say its not the best because it uses more of the CPU that others, then OK. I'm not trying to make out like this card is up there with the more expensive cards...
Interesting, thanks. I wonder why it didn't come up when I browsed through the 3Ware category.
Regardless, this one wouldn't help the situation mentioned in TFA because it doesn't do RAID5.
... And so it comes to this.
Wait until that software raid 5 (in windows at least) fails a disk and then you see the CPU price you pay. Even duing rebuild you are hurting.