Slashdot Mirror
Wear Leveling, RAID Can Wipe Out SSD Advantage
storagedude writes "This article discusses using solid state disks in enterprise storage networks. A couple of problems noted by the author: wear leveling can eat up most of a drive's bandwidth and make write performance no faster than a hard drive, and using SSDs with RAID controllers brings up its own set of problems. 'Even the highest-performance RAID controllers today cannot support the IOPS of just three of the fastest SSDs. I am not talking about a disk tray; I am talking about the whole RAID controller. If you want full performance of expensive SSDs, you need to take your $50,000 or $100,000 RAID controller and not overpopulate it with too many drives. In fact, most vendors today have between 16 and 60 drives in a disk tray and you cannot even populate a whole tray. Add to this that some RAID vendor's disk trays are only designed for the performance of disk drives and you might find that you need a disk tray per SSD drive at a huge cost.'"
This assumes that RAID controller manufacturers won't be making any changes though.
RAID for years has relied on millisecond access times. So why spend a lot of money on an ASIC & Subsystem that can go faster? So taking a RAID card designed for slow (relatively) spinning disks and attaching them to SSD of course the RAID card is going to be a bottleneck.
However subsystems are going to be designed to work with SSD that has much higher access times. When that happens, this so called 'bottleneck' is gone. You know every major disk subsystem vendor is working on these. Sounds like a disk vendor is sponsoring 'studies' to convince people not to invest in SSD technologies now knowing that a lot of companies are looking at big purchases this year because of the age of equipment after the downturn.
As a rock-in-roll Physicist once said, No matter where you go, there you are.
RAID means "Redundant Array of Inexpensive Disks".
Wear Leveling, RAID Can Wipe Out SSD Advantage for enterprise.
While it may not be efficient to slap together a platter of 16 SSDs, it is worthwhile to upgrade personal computers to use an SSD.
He's got a point - the embedded RAID controllers in boxes like the HP MSA70 just aren't up to the challenge of sustaining the IOPS of SSDs. They weren't designed for that, so you can't get a million I/Os per second by accident. You have to know what you're doing and build out an architecture that can support it.
OTOH: Who pays 100K for one of those? That has to be including the Enterprise 120GB SSD's at $4k each, right?
What 200k IOPs might look like (not mine).
Help stamp out iliturcy.
Scaling works both ways. Often technology that benefits larger installations or enterprise environments gets scaled down to the desktop after being fine tuned. It's not uncommon for technology that benefits desktop or smaller implementations to scale up to eventually benefit the 'big boys'. This is simply a case of the laptop getting the technology first as it was the most logical place for it to get traction. Give SSD's a little time and they'll work their way into RAID as well as other server solutions.
:)
The real advantage of solid state storage is seek time, not read/write times. They don't beat conventional drives by much at sustained IO. Maybe this will change in the future. RAID just isn't meant for SSD devices. RAID is a fix for the unreliable nature of magnetic disks.
It's not the performance that's no faster, it's the writing. So he should either say "...and make writes no faster than a hard drive's" or "...and make write performance no better than a hard drive's". Whenever I read this kind of redundancy, I can't help but imagine the author having trouble with indirection in a programming language, writing things like foo_ptr > *bar_ptr.
This study seems to have a very bad case of "unconsciously idealizing the status quo and working from there". For instance:
"Even the highest-performance RAID controllers today cannot support the IOPS of just three of the fastest SSDs. I am not talking about a disk tray; I am talking about the whole RAID controller. If you want full performance of expensive SSDs, you need to take your $50,000 or $100,000 RAID controller and not overpopulate it with too many drives. In fact, most vendors today have between 16 and 60 drives in a disk tray and you cannot even populate a whole tray. Add to this that some RAID vendor's disk trays are only designed for the performance of disk drives and you might find that you need a disk tray per SSD drive at a huge cost."
That sounds pretty dire. And, it does in fact mean that SSDs won't be neat drop-in replacements for some legacy infrastructures. However, step back for a minute: Why did traditional systems have 50k or 100k RAID controllers connected to large numbers of HDDs? Mostly because the IOPs on an HDD, even a 15K RPM monster, sucked horribly. If 3 SSDs can swamp a RAID controller that could handle 60 drives, that is an overwhelmingly good thing. In fact, you might be able to ditch the pricey raid controller entirely, or move to a much smaller one, if 3 SDDs can do the work of 60HDDs.
Now, for systems where bulk storage capacity is the point of the exercise, the ability to hang tray after tray full of disks off the RAID controller is necessary. However, that isn't the place where you would be buying expensive SSDs. Even the SSD vendors aren't even pretending that SSDs can cut it as capacity kings. For systems that are judged by their IOPS, though, the fact that the tradition involved hanging huge numbers (of often mostly empty, reading and writing only to the parts of the platter with the best access times) HDDs off extremely expensive RAID controllers shows that the past sucked, not that SSDs are bad.
For the obligatory car analogy: shortly after the début of the automobile, manufacturers of horse-drawn carriages noted the fatal flaw of the new technology: "With a horse drawn carriage, a single buggy whip will server to keep you moving for months, even years with the right horses. If you try to power your car with buggy whips, though, you could end up burning several buggy whips per mile, at huge expense, just to keep the engine running..."
... researchers have found that putting a Formula One engine into a Mack truck wipes out the advantages of the 19,000 rpm.
Why is it so hot? Where am I going? What am I doing in this handbasket?
HP MSA70 is junk anyways
Super Sonic Device. They're hard drives that spin so fast the edge of the platter goes faster than sound.
Not a sentence!
That guy from Samsung (?) who had a billion SSDs RAIDed up for a demo didn't seem to be doing too bad... right?
and use something along the lines of "http://en.wikipedia.org/wiki/UBIFS"
Because SSDs aren't spinning platter drives, what if we skip the part in making the SSDs try to impersonate them.
Thoughts?
OTOH: Who pays 100K for one of those? That has to be including the Enterprise 120GB SSD's at $4k each, right?
That $100K gets you more than bare drives, You get the flexibility to carve out partitions however you like, configuring them for maximum performance or whatever level of redundancy you need. You get snapshot backups, offsite replication, etc.(At additional cost of course...)
And, of course you also get the letters 'E', 'M', and 'C'.
Any insufficiently advanced magic is indistinguishable from technology.
So does anyone know if this applies to software RAID configurations?
Just curious...
And we don't have to use Highlander Rules when considering drive technologies. There's no reason that one has to build a storage array right now out of purely SSD or purely HDD. Sun showed in some of their storage products that by combining a few SSDs with several slower, large capacity HDDs and ZFS, they could satisfy many workloads for a lot less money. (Pretty much the only thing a hybrid storage pool like that can't do is sustain very high IOPS of random reads across a huge pool of data with no read locality at all.)
I hope we see more filesystems support transparent hybrid storage like this...
If you use ZFS with SSDs, it scales very nicely. There isn't a bottleneck at a raid controller. You can slam a pile of controllers into a chassis if you have bandwidth problems because you've bought 100 SSDs - by having the RAID management outside the controller, ZFS can unify the whole lot in one giant high performance array.
Was I the only one who saw the words leveling, raid, and wipe, and spent several seconds thinking the story was somehow related to WoW?
Even the highest-performance RAID controllers today cannot support the IOPS of just three of the fastest SSDs.
In the old days, raid controllers were faster than doing it in software.
Now a days, aren't software controllers faster than hardware? So, just do software raid? In my very unscientific tests of SSDs I have not been able to max out the server CPU when running bonnie++ so I guess software can handle it better?
Even worse, it seems difficult to purchase "real hardware raid" cards since marketing departments have flooded the market with essentially multiport win-SATA cards that require weird drivers because they're non-standard?
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
Firstly, "$50,000 or $100,000 RAID controller"? I think the author means Storage Array. Regular RAID controllers cost nowhere near that number. In fact, most enterprise Storage Arrays cost far more than "$50,000 or $100,000".
Secondly, they are also typically only certified for vendor provided disks (at $ludicrous), which seldom include SSDs as offering.
Thirdly, no one in their right mind is going to be using very expensive SSDs for sequential load applications, which regular disks are perfectly capable of for a fraction of the price. The only load that makes sense at that price point for the enterprise are database applications and others that utilize heavy random i/o workloads. Once you have that type of load, the performance of each SSD is going to be a fraction of the top sequential speed, but still far faster than a regular disk.
The article is FUD.
Any idiot newb knows this. Whenever you raid and are not the right level, you invariably get wiped out. Duh.
MSA are low performance crap anyway. here's a quarter kid, get yourself an EVA
I suppose it would be more important for enterprises, but personally, I wouldn't see speed as the primary purpose of having a RAID setup. Obviously it wouldn't be cool if it was really slow, but isn't data redundancy the primary purpose?
Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
Is that the EVA that caps out at 8 SSD drives? That doesn't sound like it's going to get the IOPs.
Help stamp out iliturcy.
My understanding is that pretty much all the serious storage appliance vendors are moving in that direction, at least in the internals of their devices. I suspect that pretty much anybody who isn't already a sun customer doesn't want to have to deal with ZFS directly; but that even the "You just connect to the iSCSI LUN, our magic box takes it from there" magic boxes are increasingly likely to have a mix of drive types inside.
I'll be interested to see, actually, how well the traditional 15K RPM SCSI/SAS enterprise screamer style HDDs hold up in the future. For applications where IOPS are supreme, SSDs(and, in extreme cases, DRAM based devices) are rapidly making them obsolete in performance terms and price/performance terms are getting increasingly ugly for them. The costs of fabricating flash chips are continuing to fall, the costs of building mechanical devices that can handle what those drives can aren't as much. For applications where sheer size or cost/GB are supreme, the fact that you can put SATA drives on SAS controllers is super convenient. It allows you to build monstrous, and still pretty zippy for loads that are low on random read/write and high on sustained read or write(like backups and nearline storage), storage capacity for impressively small amounts of money.
Is there a viable niche for the very high end HDDs, or will they be murdered from above by their solid state competitors, and from below by vast arrays of their cheap, cool running, and fairly low power, consumer derived SATA counterparts?
Also, since no punning opportunity should be left unexploited, I'll note that most enterprise devices are designed to run headless without any issues at all, so Highlander rules cannot possibly apply.
A lot of those features are available for a lot less than $100,000. But what you don't get, usually, is the same level of support.
Bandwidth is the limiting factor for some SSD RAIDs today, but it doesn't "wipe out the advantage" of SSDs. 8 mirrored pairs of 15K RPM hard drives would have about 150*8=1200 random writes a second. A *single* second-generation Intel X-25M has 6,000 write IOPS a second, and a single 6Gbps SATA RAID connection can handle at least 60K IOPS assuming 4k blocks and every block gets sent out twice (software RAID).
The way to deal with wear leveling, and the otherSSD controller problems the linked article raises, is to get an SSD with a good controller and large write cache; Intel has the best, then Indilinx. (You can see for yourself by looking at the SSD performance charts at Tom's Hardware or any number of comparisons out there. Note that controller maker != brand on the SSD box; you have to Google a bit.) The good SSDs aren't much more expensive per gig than the JMicron ones, so there isn't much excuse.
And sure, it would be great if RAID cards understood SSDs' nonstandard SMART statistics and used them to autoprovision spares for the drives most likely to fail next, but if you really need thousands more IOPS -- i.e., your database is crashing under crazy load -- and the cost doesn't stop you, then a little thing like hardware autoprovisioning of spares won't stop you.
Why on earth does the article even mention RAID-5 or 6 with SSDs? If you want SSDs or even 15K disks, you certainly don't want RAID-5 or 6, because your RAID performance will be limited by the speed of the parity disks. End of story.
Finally, as other commenters mentioned, enterprise disk interfaces are certainly gonna catch up as disks get faster.
The article's tone sounds like your basic kneejerk contrarianism -- "everyone says SSDs are great; here's why they're wrong" -- but it's mostly just incomplete (and, as always, posted to Slashdot with an even more fragmentary/contrarian/exaggerated summary) rather than outright wrong; you should certainly think about your SSD controller maker and RAID card before your company goes and shells out 200 Benjamins for big fast SSD arrays for your main and backup DB servers. But reports of the death of enterprise SSDs have been greatly exaggerated.
In other news, if you *really* want a reason to consider holding off on SSDs, weigh their cost-effectiveness against the other ways to keep your app running nicely under load: getting more RAM or paying employees to add caching and tune their DB accesses, or maybe even doing scale-out with tons of DB servers (which has plenty of expense of its own in development time). What's right for you mostly depends on the size of your data and working set, your workload, and how expensive scale-out and optimizations would be on the software side.
All I want to know is who is making RAID cards that cost $50,000 to $100,000? Or is he describing a complete system and calling it a RAID card?
kernel based software raid or zfs gives much better raid performance IMHO. The only reason I use hw raid is to make administration simpler. I think there is much more benefit to be had letting the os govern partition boundaries, chunk size and stripe alignment. Not to mention the dismal firmware upgrades supplied by closed source offerings.
boycott slashdot February 10th - 17th check out: altSlashdot.org
http://www.fusionio.com/
We used these to solve a problem with a horrendously mismanaged (but exceedingly crucial) MySQL DB. We compared solutions on a dollar-per-IOPS basis and these came out ahead by far. For about $17k we got 320GB of space but at well over 100,000 IOPS. The fastest arrays we could cram into a server would only reach into the low tens of thousands.
If using RAID for mirroring drives, well, you must also consider the fail rate of drives, as it is all about fault tolerance, no? It is reported that SSDs are far more durable, so the question should be, what does it take to match the fault tolerance of HDD RAID with an SSD RAID, and only after that, can we truly compare the pros and cons of their performance sacrifices.
On a side note, you can now get a sony laptop that comes equipped with a RAID 0 quad SSD drive.
http://www.sonystyle.com/webapp/wcs/stores/servlet/CategoryDisplay?catalogId=10551&storeId=10151&langId=-1&categoryId=8198552921644570897
I assume you would only do this with SSDs, given that they have a much lower failure rate than HDDs.
Disks are cheap. There's no reason to use the full GB (or TB) capacity, especially if you want fast response. If you just use the outside 20% of a disk, the random I-O performance increases hugely. ISTM the best mix is some sort of journalling system, where the SSDs are used for read oparions and updates get written to the spinning storage (or NV RAM/cache). Then at predetermined times perform bulk updates back to the SSD. if some storage array manu. came up with something like that, I'd expect most performance problems to siomply go away.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
We haven't purchased 15k disks for years. In most cases, it is actually cheaper to buy 3x or even 4x SATA spindles to get the same IOPS. Plus you get all that capacity for free, even when you factor in extra chassis and power costs. We use all that capacity for snapshots, extra safety copies, etc. If your enterprise storage vendor is charging you the same price for a 1TB SATA spindle as a 300GB 15K spindle, you need to find a new vendor. Look at scale-out clustered solutions instead of the dinosaur "dual fiber controllers and a bunch of disk" offerings.
just put regular spinning disk in it. Point is working for an HP VAR I've had the misfortune to set up many database and middleware systems on MSA and performance is appalling compared to EVA
Your dead on. Fibre channel drives are dead, they will cease to exist in the near/medium term future. SAS & SATA will live on. Fibre Channel as a transport (i.e. SANS) will be dead in the medium to long term future, giving way to the expansion of 10Gb CEE (maybe holding on in FCoE for a while).
The problem in 'the enterprise' is not the ability to find the different technologies (SSD, FC, SAS, SATA) for your workloads... the problem is finding which workload belongs on which of your technologes. Every application vendor & DBA I've ever dealt with wants raid 10 for everything, and in a shared SAN environment, in most cases it's unnecisary and in some cases it's counterproductive.
What we're seeing from some of the enterprise hardware vendors is two fold. a) using SSDs in the disk subsystem as a form of second stage cache for cache friendly workloads and b) intelligently reviewing every block by use and moving each block to the appropriate technology (SSD, Sata, FC, etc) to best service IO. Sounds promising, but I'll believe it when I see it.
Getting business & application folk to 'classify' there data for IO usage & throughput, especially before they've installed or written the app, is like herding rabid cats. So you'll end up buying SSDs for an app that will never leverage them or SATA for an app that needs SSDs, depending on what budget these folk could justify to there PHBs.
While the ioDrive may offer great performance, I hate their marketing.
http://www.fusionio.com/products/iodrive/
"It's not SSD + RAID, it's solid state memory in parallel channels!"
No, it's not X25-M's on an Adaptec card. However, it is NAND flash with a bunch of parallel channels. It's the exact same idea behind SSD + RAID, it's just above the level that you'll get with "regular" SSD + RAID.
He's clearly talking about SAN controllers like EMC Clariion or IBM DS5000; if you don't look too carefully you might mistake them for RAID controllers.
How about PCM (Phase Change Memory)?
No wear leveling, much longer life predicted, possible even higher density.
Most of the big players see as their future.
Just based on the face that it says "$50,000 or $100,000 RAID controller." Ummm what? Where the hell do you spend that kind of money on a RAID controller? A RAID controller for a few disks is a couple hundred bucks at most. For high end controllers you are talking a few thousands. Like Adaptec's 5805Z which has a dual core 1.2GHz chip on it for all the RAID calculations and supports up to 256 disks. Cost? About $1000 from Adaptec. Or how about the 3Ware 9690SA-8E, 8 external SAS connectors for shelves with 128 disk support. Going for about $700 online.
So anyone who's trying to pretend like RAID controllers cost 5-6 figures is just making shit up. Yes, you can pay that much for a NAS, but you aren't paying for a RAID controller. You are paying for a computer with custom OS, controllers, shelves, disks, monitoring and so on. A complete solution, in other words. Also, if you are spending that kind of money, it is a really serious NAS. We bought a NetApp 2020 and it didn't cost $50,000.
Then, as you say, it is not bad to hit the performance limits. While on a small scale you may be mostly buying RAID for performance reasons, that isn't the reason on a large scale. The reason is space. We got our NetApp because we need a lot of reliable central storage for our department. Yes, it needs to have reasonable performance as well, but really the network is the limit there, not the NAS. The point of it is that it holds a ton of disks. So, if we filled it full of SSDs and those were higher performance than it could handle, we'd not care. Performance with magnetic disks is already as good as we need it to be.
My (possibly incorrect?) understanding was that 3-4x 7200rpm drives arent a drop-in replacement for a 15k in all situations-- the slower drives still have a higher rotational latency, do they not? Even if you throw 50 slower drives at the problem, there are still situations where the 15k drive will respond faster simply because of its rotational latency.
Correct me if i am incorrect.
A point I wouldn't argue. Like I said, the MSA is inadequate for this task - as is the EVA. We're talking about an entirely different tier of performance storage here. You're not going to hit 250K IOPs in an EVA, no matter what you put in it.
Help stamp out iliturcy.
My (possibly incorrect?) understanding was that 3-4x 7200rpm drives arent a drop-in replacement for a 15k in all situations
Not all but I would think most.
If you are comparing a big array of 15K drives with an even bigger array of 7.2K drives I would think it likely that the application in question is one that is capable of generating large numbers of requests in parallel (most likely some kind of database server).
note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
Henry Newman may know SSD drives but he doesn't know enterprise storage. Henry, enterprise shops don't talk about MB/s unless they are streaming video or working on their laptop.
All IO in the a storage networked enterprise are random. Most important IOs are usually small block (databases). There is no concept of MB/s of bandwidth except to gauge channel capacity. Any one who does enterprise storage works in IOPS. SSD drives smoke for random IOPS to the tune of 50x for writes and 200x for reads (MLC vs same size 15k RPM drives). These are significant numbers. Even if we lost 1/2 the write IOPS to wear leveling, that would be 25x faster. Want your database to scream.
RAID controllers will only be able to do RAID 10. Most RAID controllers can do RAID 10 in their sleep. The bottle neck will now be the channels in and out of the controllers. The first roll out of SSD storage in the enterprise will be direct attached SSD trays to bus attached controllers with the most external channels (bandwidth).
SSD drives are going to choke SAN channels. In a couple of years when administrators want to network their SSD drives there will be a really big push to get better pipes in the SAN. I wonder if inifiniband will get back in the mix?
This kind of disruptive technology keeps us employed.
machinator omnis sine licentia
There may exist such applications, but I haven't seen them before, and I would think they have already been moved to SSD.. The average service time for an IOP with SATA is indeed longer, but most applications (even simple file servers) issue a queue of many requests, so aggregate throughput matters more. For writes, cache makes the difference unimportant. For databases, file servers, etc. throughput is king.
I always wondered why you couldn't put another set of heads on the opposite side of the platters in a 7,200 rpm drive. The sector you want to access would then never be more than 1/2 turn away, giving the equivalent of 14,400 rpm. It shouldn't cost more than twice a regular drive, but you get to keep your 1-2TB. can you even get 1TB 15k drives?
404: sig not found.
Why keep the RAID controller at all, it's likely the slowest CPU you have in your system anyway. SSDs and smart SW based RAID aware filesystems allows build a new type of storage HW, with no need of a dedicated RADID controller. You can already today with OpenSolaris and combination of SATA drives and a few SSDs build storage solution with very good performance. And if you need the pure SSDs IOPs and low access time, just replace your spinning drives with SSDs for even better performance. Your host CPU/CPUs will have a lot better chance of keeping up with your SSD based RAID.
Yeah, I love my X8DTH-6F.... its a beast. Pair of E5520's on it. Made a great upgrade from Q6600.
Now if only I had the SSD's and LSI cards to go with it...... LOL
Actually, I'd settle for OSX drivers for the LSI 2008 and the intel 82576... Oh well, maybe soon.
Well, most of the raid controllers out there can't even keep up with the throughput of a half dozen modern magnetic drives either. Our application eats bandwidth, and its been a real struggle to find controllers that can sustain higher transfer rates. Getting much more than 1GB/sec out of a RAID box is pretty much impossible no matter the cost. In the end we use fairly low end RAID disks ganged together via high end FC switches and let our software do the stripping across them. AKA we gang a bunch of 600MB/sec RAID arrays together to get multiple GB/sec.
Trolling since 1998?
IANAL but write like a drunk one.
That was tried before, many years ago, but abandoned as far as I can tell. I imagine they had a good reason. Probably the fact that two head motor assemblies means they would have to make the platter smaller to fit in a standard form factor case.