Slashdot Mirror
IBM Flash Memory Breaks 1 Million IOPS Barrier
alphadogg writes to tell us that IBM is claiming a victory on the flash storage front. Their new research project "Quicksilver" is claiming data transfer speeds of more than 1 million input/output operations per second (IOPS). "IBM said Quicksilver is two and a half times faster than its own SAN Volume Controller coupled with IBM's DS4700 storage. It would also be two and a half times faster than technology from Texas Memory Systems, which says it has the world's fastest storage with an IOPS rate of 400,000. "
It has a limit to the number of writes, the number of reads is pretty unlimited. The expected average lifetime is similar to a hard drive, and in some cases better.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
It's the 1 MIOPS /mark/. If it was a barrier, you wouldn't be /able/ to break it.
Hail Eris, full of mischief...
E pluribus sanguinem
The bigger the flash drive, the more area in which to spread the wear, the longer it lasts.
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
You can normally assume that for that level of IOPS they are 4K blocks, so 4GB/s, pretty damn impressive as that's saturating 4*10Gb/s links.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
By the way, for a workload with a lot of independent reads or writes you'd be surprised how slow a hard disk is. With a 512byte block (common on hard drives) you get a maximum throughput of around 50KB/s for a random access pattern on a cheap drive, going up to around 125KB/s on an expensive one. Even very cheap flash can do better than this, so for moderate sized databases (a few GBs) with a very heavy access load flash works out a lot cheaper.
Oh, and for reference each of the ops in this test was up to 640KB, giving a maximum of around 640GB/s data transfer.
I am TheRaven on Soylent News
MB/s is only a measure that's meaningful for sequential reads where the data can be prefetched. Most enterprise storage is based on applications that read randomly all over the disk (like databases and email servers). The benchmark measurement for this type of application is in the number of operations you can do per second. A single hard drive spindle can do between 80 and 150 IO/s, which would generate the number of IO/s times the size of the IO blocks per second.
How does this translate into normal transfer speed units like MB/s? Otherwise I have no point of reference to tell if I am impressed or indifferent.
I'll try to help.
MB/s is a measure of IO throughput. Often this isn't the most relevant figure for 'enterprise' storage. Certain applications do a lot of random access IO so IOPS becomes more important than throughput.
Today a typical desktop disk is capable of about 100-150 IOPS. That's a rule of thumb range that varies based on operation size, cache, etc. It works pretty well usually. You can aggregate disks and get almost linear scaling; 12 disks, for instance in a device like this, will give you a maximum of 1200 IOPs, roughly. A common USB Flash device can break 1000 IOPS with certain access patterns.
The second graph on this page illustrates the extreme IOPS advantage of Flash for certain applications. Disks are limited by head actuation and rotation latency. This is why enterprise storage vendors have been pursuing Flash aggressively. That's what this story is all about.
The dream is to host the same IOPS in with an order of magnitude less physical space, power, heat, etc. If you don't need thousands of IOPS (and most PC users don't) then it isn't very interesting. If you happen to run an OLTP system with thousands of reads/write per second it means a great deal.
Lurking at the bottom of the gravity well, getting old
Obviously, there's not a whole lot of independent research out there, but here are some of the claims:
http://talkback.zdnet.com/5208-11408-0.html?forumID=1&threadID=39451&messageID=725468&start=0
http://www.linuxdevices.com/news/NS7676844023.html
http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9007518
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
I stand corrected, from the talkback link I followed a trail to an IBM blog with a LOT more details here, and this is the 70/30 SPC-1 benchmark numbers with cache disabled. This is freaking phenomenal performance! The storage is only 4TB, but if you put your logs, flashback, and temp tables on this beast and pinned your busiest tables in ram you would have a screaming OLTP database. I guess it's now just a matter of price, but a rack of x-series boxes with flash card's shouldn't be THAT expensive. Unless IBM asks for a crazy markup it should be affordable for most enterprises (ok, pretty much a given with IBM but still).
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
It's not freaking 640GB/s, there's only one switch in existence that can do that much (Cisco 7000, the Brocade DCX-Backbone is the only other one that's close and it's 6.4Tb/s total per chassis). It's ~4.4GB/s, 1.1M peak IOPS * 4KB chunks...
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
...the more you wear it down by using it. (eg. bigger files, more files used.)
Isn't that the whole point of a bigger drive?
Any sufficiently advanced intelligence is indistinguishable from stupidity.