Slashdot Mirror
Costly SSDs Worth It, Users Say
Lucas123 writes "When you're paying $30,000 for a PCIe flash card, it had better demonstrate an ROI. While users are still struggling with why solid state storage cost so much, when they target the technology at the right applications, the results can be staggering. For example, when Dan Marbes, a systems engineer at Associated Bank, deployed just three SSDs for his B.I. applications, the flash storage outperformed 60 15,000rpm Fibre Channel disk drives in small-block reads. But when Marbes used the SSDs for large-block random reads and any writes, 'the 60 15K spindles crushed the SSDs,' he said,"
Small (and cheap) 32GB SSD for my desktop...
Big powerful 12TB file server using traditional disks for the bulk of my data.
Performance for the stuff where the SSD makes a difference (program files), cheap storage for the stuff where it doesn't (just about everything else).
And if that 32GB drive dies (unproven technology.. MTBF is still a guess) .. I'll buy another cheap (probably cheaper at that point) one and restore from my daily backup.
except it's already on the market. http://www.seagate.com/www/en-us/products/internal-storage/momentus-xt-kit/
For example, when Dan Marbes, a systems engineer at Associated Bank, deployed just three SSDs for his B.I. applications, the flash storage outperformed 60 15,000rpm Fibre Channel disk drives in small-block reads. But when Marbes used the SSDs for large-block random reads and any writes, 'the 60 15K spindles crushed the SSDs,' he said,"
So when you need lots of small, random reads, 3x SSDs beat 60x HDDs. Most of the time is spent seeking the file on the HDDs, your ~4.6 ms random seek time is an order of magnitude or more slower than the flash-based drives. No surprise here.
When you are just transferring large files, most of the time is spent actually transferring data. A modern SSD might manage 300-400 MB/s read, but 20x as many HDDs are still going to beat the crap out of them.
The only mildly surprising part is that part about the HDDs winning for all writes, but I guess that really depends on how the test is set up - unless you are actually writing to random parts of the HDD, it is basically a straight-up write operation, so only throughput matters - and again, 60x HDDs are going to beat 3x SSDs (though it is important to note that SSDs are significantly slower at writing than reading in general, although still much faster than an HDD on an individual basis).
Am I supposed to just run out and buy SSDs for the whole load?
No. I think that in the open source world you're expected to write the update yourself.
"Our two-party system is like a bowl of shit looking at itself in a mirror." - Lewis Black
Not quite. Seagate's tech is a simple block cache, where the most frequently accessed blocks get thrown on the SSD portion. It is no doubt quite effective, but we should be able to do better if we move the logic into the OS.
The OS has intimate knowledge of the filesystem, and can easily profile its use. Files that are small or randomly accessed should get put on the SSD, while large sequentially accessed files should get put on the HDD.
In addition to the Momentus XT that several others mentioned, there is also SSD caching on Intel's Z68 motherboards allowing you to designate a 20GB+ drive as an automated cache of whatever data is traveling to and from your hard drives. The effects are quite noticeable and it seems like the system is pretty smart. Plus, it is done in software that is easily updated should a more efficient algorithm be found.
To me the cost per GB of an SSD really makes it tough to justify in a lot of cases especially if you want to store large programs like games where you'll need at least a 100GB SSD. However one area place I have started to use low capacity (8 or 16GB) SSDs are in low noise and/or low power environments. If you team them with an ITX Atom board and the right power supply you can build a small computer with no moving parts whatsoever. And the computer will have a very low power usage for applications like HTPCs or network appliances (like firewalls) where the machine might always be powered on.
...and would require integration with multiple OS's, require drivers (the quality of which we don't know), etc. Making it transparent to the OS means you don't have any of those problems. It's a trade off, and for a first gen drive, probably the better way to go.
Commercial SANs have had such tiered capability for years. Multiple levels of performance from bulk, long term storage on spinning disks to short term storage on SLC flash and finally a big memory cache. ZFS for Solaris and FreeBSD offers something similar with the L2ARC, allowing a cheaper but slower way to provide a large, high speed memory cache.
Im running this drive on my gaming comp. Its a 4GB flash 'mirror' of all the files you use most frequently. All the logic and mirroring is handled by the drive controller so its OS-agnostic. It runs great, especially for the price and considering i built my gaming rig before z68 was available. The drive HDD tachs at 90MB/sec, besting my old Raptor 10k 74 GB.
Good-bye
Of course. OS awareness of hardware capabilities lets you avoid workarounds like TRIM. Instead we have TRIM because the nature of the hardware (NAND flash) is hidden behind an interface designed for rotating media that behaves differently.
Well, if you integrate it into the block layer, then it can be utilized by the filesystem with one layer of abstraction. Abstract it completely and you have to hope that the drive logic is capable of making good decisions, otherwise you'll end up with your swap occupying the SSD and not get any sort of speed increase.
No, you are supposed to start a flame-war on lkml about how SSD cache is a stupid idea that will never amount to anything. Next hundreds of kernel developers will start develop the code to prove you wrong.
If my comment didn't sound as good in your head as it did in mine, then I guess we all know who's to blame
If you think about it, the outcome of this test is 100% in favor of the SSD.
Think about it:
The tester was willing to test only 3 SSD's versus *60* 15K drives. So the tester thought that 20 times fewer drives was a fair test for the comparison. What is the tester actually saying here? I think I have a feeling I know. :-)
Anyway, 15K drives are not long for the market. Soon, all that will be left are economy class, 10K, and SSD's.
Only a little more maturity, and the enterprise flood gates will open. When that happens, the hapless victim will be the short stroked IOPS environment, where total IOPS was always the requirement, and that requirement was for more IOPS than capacity. I.e., if a 15K drive offers 400 IOPS, and you need 400,000 sustained, but don't have to store very much at all, your only current choices are buying a lot of 15K drives. Or a bazillion less SSD's.
The switchover point is only a heartbeat away.
Bye 15K drive. I'll miss you.
C//
Just as an info dump for anyone who's not familiar with why SSDs perform so much better: SSDs have far better seek performance.
A normal HDD takes about 10ms to seek (3ms at the very high end, 15ms at the low end- 10ms is a good rule of thumb), which means you've got a princely 100 seeks per second per spindle (i.e. HDD). SSDs don't have seek limitations. Looking up a contiguous block of data vs not looking up a contiguous block of data makes no difference to an SSD.
It turns out that 100 seeks isn't a lot in serving infrastructure or, in some cases, on a desktop. When you go to read a file off disk multiple seeks are involved- you need to look up the inode (or equivalent), find the file and a large file will probably be in many different chunks require separate seeks to access them.
Even on a desktop you'll frequently be seek bound not throughput limited. Lets say you are starting up a largish java application (Eclipse might be a good example). It references a huge number of library (.jar) files which are certainly large enough to require many seeks to access. And those libraries are often linked in to system libraries which also have their own dependencies and may have additional dependencies all of which require further seeks. Plus with Eclipse it will look up the time stamps on files in the project... and so on.
During boot of a system is another time when HDD are usually seek bound- lots of different applications/services/daemons are starting at the same time, loading lots of libraries causing lots and lots of seeks.
On server infrastructure a highly utilized database will probably be seek bound not throughput limited.
The article is kind of stating the blindly obvious- if you are seek bound SSDs are better. And 60 drives gives ~6000 seeks. A typical modernish desktop HDD can get in the order of 100MB/s data transfer (average sustained), more expensive HDD can get quite a lot more. If we take 3.0Gb/s as a ceiling (i.e. the SATA 3.0 max transfer rate) then at 6000 seeks/second you are getting 3000MB/6000seeks=0.5MB per seek. So the result makes perfect sense if you looking up data that is either entirely non-contigous or smaller than 500kB- an SSD will beat you every time on seeks (since it has no seek time).
The limitations on SSDs are: they have throughput limitations, just like HDD and more importantly their write performance is usually significantly worse than a HDD (writing on an SSD often involves reading and re-writing large chunks of data, even for very small writes). You can easily construct tests where HDD perform better than SSDs (particularly something like a 60 spindle array of HDD where an awful lot of writes can be cached in the on disk's ram buffer, which is common on hire performance drives- often battery backed so they can "guarantee" the write has been committed without having to wait for a write to the magnetic media).
Of course SSDs other obvious application are where you want robustness and silence, i.e. laptops. Oddly enough their power performance isn't that much better than a normal HDD (although that might have changed since I last read about it).
15K enterprise drives cost around ~$1/gigabyte ... not all that much cheaper than SSD's which cost around ~$2/gigabyte (MLC) or ~$5/gigabyte (SLC)
Now, the comparison in the summary is between 3 SSD's and 60 15K HDD's.. in other words, the HDD solution was enormously more expensive. (and thats NOT counting the cost of the stack of Fiber Channel raid enclosures, let alone the power that 60 stack draws)
You dont seem to know what you are talking about. SSD's arent much more expensive per gigabyte than HDD's in performance enterprise environments, and always significantly outperform for equal investment, with less power costs. The only place the "cheaper per gigabyte" argument is true is when you can get away with inexpensive HDD's.. in other words, you heard people talk about one thing but didnt know that it didnt apply to another.
When you dont know what you are talking about, act like it.
"His name was James Damore."
. . . a Storage Review experiment from over a year ago:
http://www.storagereview.com/western_digital_velociraptors_raid_ssd_alternative
They put WD Raptors in RAID 0 to form a high performance (yet still affordable) platter drive setup, and then faced them off against Western Digital's new (at the time, first) SSD. Makes sense, right? Except that WD's first SSD was a complete joke, an underperforming, laughably expensive POS that I forgot about a couple days after Anand's review. When I first read about it I couldn't help but think that WD was deliberately setting it up to fail. It was at the bottom of every benchmark yet priced higher than any other (MLC) SSD. They even put a jmicron controller in it for fuck's sake (not the infamous original one, but still . . .)! Storage Review's calling it a "mid-range" SSD is very generous at best.
Even so, this supposedly screaming platter drive setup could only occasionally hang with the bottom of the barrell of SSDs, and mostly lagged behind it. And as I said, this was over a year ago. It goes without saying that they didn't worry much about heat, noise, reliability (of RAID 0), or power consumption.
Anand doesn't even list platter drives in his benchmark results anymore because they'd skew the charts so badly.
As a previous poster said, a winning strategy is to get a SSD boot drive just big enough for your OS and programs, and use platter drives for everything else. And since the SSD takes care of your performance needs, you can get the cheapest, slowest, coolest, quietest platter drives. There are some cases where both high performance and high capacity are needed at once (like video editing) but they're not the norm.
The difference is even starker when you take rack space into account. The largest 15k drive I could find was 600 gb. Enterprise SSDs on the other hand(if you dont want to go the PCIE route) are right now approaching 1 TB for a 3.5" drive, and the difference in density between the two is only going to grow. The reduced amount of rack space SSDs take up is going to further decrease operating costs.
Monstar L
A tiny number of SSDs out perform a huge number of spinning disks except in certain situations. Story right now.
Paying taxes to buy civilization is like paying a hooker to buy love.
real-time video capture(or any sort of real time capture of streaming data really) is about the WORST application for SSDs. SSD writes can potentially take a REALLY long time to write if they need to clear the sector before writing. While TRIM lessens the probability of this happening, it by no means removes it. There is still the possibility of writing to a TRIMed sector again between when the TRIM command was issued and when the SSD does garbage collection, which can be delayed considerably if a lot of data is being written to the SSD. In a real time system, you just do not want to have to deal with this unpredictability, esp. when long sustained writes to SSDs arent that much better even when the sectors are clean.
Now compare this with spinning disks. Provided there is no additional disk contention(which if you are capturing HD video, there really shouldnt be), the write times for long sequential writes to a disk are VERY predictable as the disk doesnt care what may or may not have been there before, it just over-writes it.
Monstar L
I moved a small 4TB database from 24x 256G 15k SAS drives to 24x 240G OCZ Vertex 3 SATA3 drives. I ran a few queries on the old and the new. same data, same parameters, same amount of data pulled. Both were hooked up via PCIe 8x slots.
the SSD crushed the SAS. Not just a mere 2x or 3x crushing. A _FIFTEEN TIMES FASTER_ crushing. This was pulling about a million rows out. 12 seconds (SSD) vs 189 seconds (spindles)
Cost difference? under $50 per drive more expensive for SSD. I think our actual rate was around $10 per drive more. However, the system as a whole (array+drives+computer) was $12k less. No contest... for our particular application, SSD hands down makes it actually work.
we'll be moving the larger database (same data, same function) to SSD as soon as we can.
-- Who is the bigger fool? The fool or the fool who follows him? --
...when your selection criteria is cost-per-IOPS. For heavy, random IOs (like transactional databases) SSD presents a cheaper solution.
Anyway, that's my two-sentence rewrite of TFA.
People go to great lengths to minimize read/writes; putting their games and apps on a HDD, changing system/user folder locations to a HDD, etc. So basically is it worth it to merely load the operating system a little bit faster? Why buy an SSD if you aren't going to use it to load your games and apps faster?
On "why does NetApp sell their PCIe NAND flash card $30k?", here is your answer, Chris Rima: http://blog.zorinaq.com/?e=37
In a 3 words: because NetApp can.
It's not the components or engineering behind the card that cost $30k. NetApp prices it so high because the card boosts the performance of their filers by about the same amount as a ~$50k shelf of SAS disks (click that link and go read NetApp's own marketing documentation). They have got to have price points that make sense to customers.
(I know a fraction of you will think "No way!". Well, arbitrary price markups on enterprise gear do exist. This NetApp Flash Cache is effectively priced $150/GB. How do you think that certain competitors can even sell _enterprise_ flash at well below $10/GB? We are not talking 25 or 50% less, but a whole order of magnitude less expensive!)