Slashdot Mirror

You could use Managed Flash Technology to improve write performance and drive lifetime as well (Windows and Linux). For MLC or SLC drives. http://managedflash.com/home/index.htm

People (read: vendors) now frequently refer to flash storage as superior when IOPs are the main issue.

From what I've been able to discern this is actually true only in read-mostly applications and applications where writes are already in neat multiples of the flash erase block size.

If you're doing random small writes your performance is likely to be miserable, because you'll need to erase blocks of flash much larger than the data actually being changed, then rewrite the block with the changed data.

Some apps, like databases, might not care about this if you're able to get their page size to match or exceed that of the underlying storage medium. Whether or not this is possible depends on the database.

For some other uses a log-oriented file system might help, but those have their own issues.

In general, though, flash storage currently only seems to be exciting for random read-mostly applications, which get a revolting performance boost so long as the blocks being written are small enough and scattered enough. For larger contiguous reads hard disks still leave flash in the dust because of their vastly superior raw throughput.

Vendors, however, make a much larger margin on flash disk sales.

This article (PDF) may be of interest:
Understanding Flash SSD performance
(google text version).

On June 15th, Mtron will start shipping the 1000 series MLC drives. Put these in an array with the right software and you end up with price/GB parity with 36GB 15K 2.5" SAS drives and about 12x the random IO performance.

HDD Array:

8 Seagate Savvio 2.5" HDDs: $350ea $2,800
    configured raid-10
1 SAS raid controller $600
Total cost for 144 GB $3,400 or $23.61/GB

SSD Array:

6 Mtron 1025-32 2.5" SSDs: $290ea $1,740
    configured raid-5
1 SATA raid controller $250
MFT Software License $1,250
Total Cost for 144 GB $3,240 or $22.50/GB

HDD Performance:
    4K and 8K read IOPS: 250/2000 (single-threaded/multi-threaded)
    4K and 8K write IOPS: 1200

SSD Performance:
    4K read IOPS: 8000/48000 (single-threaded/multi-threaded)
    8K read IOPS: 6000/36000 (single-threaded/multi-threaded)
    4K write IOPS: 40000
    8K write IOPS: 22000

These performance numbers are with the MFT driver in place. Without MFT, the 4K random write performance is about 140 IOPS (>250x slower).

Endurance for these SSDs in this configuration is good enough to overwrite the entire array with random data three times a day (500GB of random updates/day) for about five years.

These drives make a wicked mail server (EasyCo just moved one of it's mail servers mirrored to MLC flash and the difference is amazing).

Sorry for the blatant advert, but SSDs are here now.

Doug Dumitru
EasyCo LLC
http://managedflash.com/
+1 610 237-2000 x2

My apologies for a long post. There will be some adverts embedded, but I will try to keep things informative.

The reason that Flash SSDs act "wierd" in benchmarks is that they have asymmetric performance patterns when reading and writing. Particularly with random operations, this asymmetry is huge. Here are a couple of example "drives":

* Mtron 7000 series: >14,000 4K random reads. ~130 4K random writes.
* SanDisk 5000 series: ~7,000 4K random reads. 13 4K random writes.
* Cheap CF card or USB stick: ~2,500 4K random reads. 3.3 4K random writes.

This is a 100:1 performance deficit when doing random writes versus the random reads. This has some really weird impacts on system performance. For example, if you run Outlook and tell it to "index" your system, it will build a 1-4 GB index file in-place with 100% random writes. If you do this on a hard disk, the job takes a long time and drags down your laptop, but the operation is still pretty smooth. Do the same think on an SSD and the system slugs to molasses. One of our customers described it as "totally unusable" with 2+ minutes to bring up task manager. What happens is that the fast reads allow the application to dirty write buffer faster and this then swamps system RAM, you get a 100+ deep write queue (at 13/sec), and you want to throw the machine off of a bridge.

This fix as some have described it is not some magic new controller glue or putting the flash closer to the CPU. It is organizing the write patterns to more closely match what the Flash chips are good at. Numerous embedded file systems like JFFS do this, but they are really designed for very small devices and are more concerned with wear and lifespan issue than performance.

Now here comes the advert (flames welcome). A little over 2 years ago, I wrote a "block translation" layer for use with Flash storage devices. It is somewhat similar to a LogFS, but it is not really a file system and it does not play be all of the rules of a LogFS. It does however remap blocks and linearize writes. Thus it plays well with Flash. It also appears to be an "invention", and thus my patent lawyer is well paid.

The working name of the driver layer itself is "Fast Block Device" (fbd) and the marketing name is "Manged Flash Technology". And what this does is to transparently map one block device into another view. You can then put whatever file system you want into the mix.

In terms of performance, it is all about bandwidth. Build a little raid-5 array with 4 Mtron drives and you will get over 200 MB/sec of sustained write throughput. With MFT in place, this directly translates into 50,000 4K random writes/sec. Even better, you tend to end up with something that is much closer to symmetric in terms of random read/write performance.

MFT is production on Linux (it has actually been shipping since last summer) and is in Beta test on Windows. It works with single drives as well as small to medium sized arrays. It does work with large arrays, but the controllers don't tend to keep up with the drives, so large arrays are useful for capacity but don't really help performance a lot. Once you get to 50,000 IOPS it is hard for the controllers to go much faster.

Consumer testing with MFT tends to produce some laughable results. We ran PCMark05's disk test on it and produced numbers in the 250K range. This was with a single Mtron 3025. Our code is fast, but we fooled the benchmark in this case.

There are several white papers on MFT posted in the news link of our website:

    http://managedflash.com/

My apologies for the advert, but I see a lot of talk about SSDs without actually knowing what is going on inside.

I am happy to answer any questions on-line of off.

Doug Dumitru
EasyCo LLC
610 237-2000 x43
http://easyco.com/
http://managedflash.com/
http://mtron.easyco.com/

My apologies for a long post. There will be some adverts embedded, but I will try to keep things informative.

The reason that Flash SSDs act "wierd" in benchmarks is that they have asymmetric performance patterns when reading and writing. Particularly with random operations, this asymmetry is huge. Here are a couple of example "drives":

* Mtron 7000 series: >14,000 4K random reads. ~130 4K random writes.
* SanDisk 5000 series: ~7,000 4K random reads. 13 4K random writes.
* Cheap CF card or USB stick: ~2,500 4K random reads. 3.3 4K random writes.

This is a 100:1 performance deficit when doing random writes versus the random reads. This has some really weird impacts on system performance. For example, if you run Outlook and tell it to "index" your system, it will build a 1-4 GB index file in-place with 100% random writes. If you do this on a hard disk, the job takes a long time and drags down your laptop, but the operation is still pretty smooth. Do the same think on an SSD and the system slugs to molasses. One of our customers described it as "totally unusable" with 2+ minutes to bring up task manager. What happens is that the fast reads allow the application to dirty write buffer faster and this then swamps system RAM, you get a 100+ deep write queue (at 13/sec), and you want to throw the machine off of a bridge.

This fix as some have described it is not some magic new controller glue or putting the flash closer to the CPU. It is organizing the write patterns to more closely match what the Flash chips are good at. Numerous embedded file systems like JFFS do this, but they are really designed for very small devices and are more concerned with wear and lifespan issue than performance.

Now here comes the advert (flames welcome). A little over 2 years ago, I wrote a "block translation" layer for use with Flash storage devices. It is somewhat similar to a LogFS, but it is not really a file system and it does not play be all of the rules of a LogFS. It does however remap blocks and linearize writes. Thus it plays well with Flash. It also appears to be an "invention", and thus my patent lawyer is well paid.

The working name of the driver layer itself is "Fast Block Device" (fbd) and the marketing name is "Manged Flash Technology". And what this does is to transparently map one block device into another view. You can then put whatever file system you want into the mix.

In terms of performance, it is all about bandwidth. Build a little raid-5 array with 4 Mtron drives and you will get over 200 MB/sec of sustained write throughput. With MFT in place, this directly translates into 50,000 4K random writes/sec. Even better, you tend to end up with something that is much closer to symmetric in terms of random read/write performance.

MFT is production on Linux (it has actually been shipping since last summer) and is in Beta test on Windows. It works with single drives as well as small to medium sized arrays. It does work with large arrays, but the controllers don't tend to keep up with the drives, so large arrays are useful for capacity but don't really help performance a lot. Once you get to 50,000 IOPS it is hard for the controllers to go much faster.

Consumer testing with MFT tends to produce some laughable results. We ran PCMark05's disk test on it and produced numbers in the 250K range. This was with a single Mtron 3025. Our code is fast, but we fooled the benchmark in this case.

There are several white papers on MFT posted in the news link of our website:

    http://managedflash.com/

My apologies for the advert, but I see a lot of talk about SSDs without actually knowing what is going on inside.

I am happy to answer any questions on-line of off.

Doug Dumitru
EasyCo LLC
610 237-2000 x43
http://easyco.com/
http://managedflash.com/
http://mtron.easyco.com/

SSD's do rather poorly for random writes; even a decent MTron drive's only rated for something like 130/second, and for the cheaper ones aimed at portables you can drop a zero off that. To actually get decent write performance you have to start jumping through hoops to convert the random writes to sequential.

Flash devices are fast at small reads but very slow at small random writes. Swap is usually a 50-50 mix of reads and writes with no pattern to be seen. Thus if you swap to Flash, at least traditional Flash, then you will be very unhappy.

Your comment about bandwidth needing to go to a spindle seems strange as well. I have a 4 drive raid-5 flash array here that just tested at >400 MB/sec on reads and >150 MB/sec on random writes ...

      http://managedflash.com/news/papers/07-12-01_mtron-benchmarks.pdf

I don't know of many "spindles" that can keep up with that. Plus total power draw is 12 watts operating and 2 watts idle (actually the raid controller draws another 15, sorry).

If you want to see how Flash "can" be used for swap, see:

      http://managedflash.com/

Even compact flash cards are effective for swap yielding about 2000 4K random read/write IOPS.

ps: sorry for the advert. at least it was short.

Flash devices are fast at small reads but very slow at small random writes. Swap is usually a 50-50 mix of reads and writes with no pattern to be seen. Thus if you swap to Flash, at least traditional Flash, then you will be very unhappy.

Your comment about bandwidth needing to go to a spindle seems strange as well. I have a 4 drive raid-5 flash array here that just tested at >400 MB/sec on reads and >150 MB/sec on random writes ...

      http://managedflash.com/news/papers/07-12-01_mtron-benchmarks.pdf

I don't know of many "spindles" that can keep up with that. Plus total power draw is 12 watts operating and 2 watts idle (actually the raid controller draws another 15, sorry).

If you want to see how Flash "can" be used for swap, see:

      http://managedflash.com/

Even compact flash cards are effective for swap yielding about 2000 4K random read/write IOPS.

ps: sorry for the advert. at least it was short.

Random write performance to bare drives is usually quite bad. Most "reputable" vendors do publish random write figures. SanDisk quotes 13 IOPS in the spec sheets. Mtron quotes 120 IOPS. I have not seen quotes from Samsung, but have tested their old drives at 27 IOPS. I even tested one drive at 3.3 write IOPS.

On the other hand, random writes issues are "fixable". My company just published tests for various Raid-5 Flash SSDs setups. For 4 drives testing with 10 threads on Linux 2.6.22 using our MFT "driver", we get:

4K random reads 39,689 IOPS 155 MB/sec
4K random writes 29,618 IOPS 115 MB/sec

These are real numbers and the application does see the performance improvement.

For full details on drive performance see:

    http://managedflash.com/news/papers/index.htm