Software SSD Cache Implementation For Linux?

Annirak writes "With the bottom dropping out of the magnetic disk market and SSD prices still over $3/GB, I want to know if there is a way to to get the best of both worlds. Ideally, a caching algorithm would store frequently used sectors, or sectors used during boot or application launches (hot sectors), to the SSD. Adaptec has a firmware implementation of this concept, called MaxIQ, but this is only for use on their RAID controllers and only works with their special, even more expensive, SSD. Silverstone recently released a device which does this for a single disk, but it is limited: it caches the first part of the magnetic disk, up to the size of the SSD, rather than caching frequently used sectors. The FS-Cache implementation in recent Linux kernels seems to be primarily intended for use in NFS and AFS, without much provision for speeding up local filesystems. Is there a way to use an SSD to act as a hot sector cache for a magnetic disk under Linux?"

Comments

isn't 40 GB enough for applications?

[owlstead]

Is there really a need for this? Intel 40 GB SSD still has a read speed of 170 MB/s and costs about 100 euro here in NL. Why have some kind of experimental configuration while prices are like that? OK, 35 MB/s write speed is not that high, but with the high IOPS and seek times you still have most of the benefits.

I can see why you would want something like this, but I doubt the benefits are that large over a normal SSD + HDD configuration.

Re:ZFS L2ARC

[Anonymous Coward]

Not Linux per se, but the same idea is implemented nicely on ZFS through its L2ARC: http://blogs.sun.com/brendan/entry/test

Swapcache on DragonFly BSD 2.6.x was implemented for this very reason IIRC.

http://leaf.dragonflybsd.org/cgi/web-man?command=swapcache&section=ANY

Re:I don't get it

[owlstead]

A fast SSD is not 1 Gb/s under ideal conditions. A fast SSD is up to 2 Gb/s (about 250 MB/s) under real life conditions (while reading). Anyway, it still makes sense to cache network content to disk if the other side of the connection is slow or not reliable.

FSCache would work except...

[Jah-Wren Ryel]

I have a similar problem and I tried the FSCache approach:

I've got two raids.
One is optimized for big ass files read contiguously and has raid6 redundancy.
The other is a much smaller JBOD that I can reconfigure via mdraid to anything that linux supports in software.

The problem is that 5% of the big ass files need read-only random access and that kills throughput for anything else going on. It takes me down from ~400MB/s to 15MB/s.

So, I thought I'd use the FSCache approach and use the JBOD as the cache.
I did an NFS mount over loopback and pointed the fscache to the JBOD.
It worked great got practically full throughput for contiguous access, for about 10 hours and then crashed the system.

Apparently NFS over loopback is well known to be broken in linux and has been since, essentially, forever.
I was stunned, it had never even occurred to me that NFS over loopback would be broken. Its freaking 2010 - that something I had been using on Sun0S 3 a bazillion years ago didn't work on linux today had not even entered my mind.

I've also tried replicating the files from the raid6 to the jbod, but that quickly turned into a hassle keeping everything syncronized between the files on disk and the applications that create the files on the raid6 and the apps that use the files on the JBOD. Plus, it doesn't scale out past the size of the JBOD, which I also ran into.

So now, I'm looking at putting the apps that need random access reads to the data in a VM and NFS mounting it with cache to the VM hoping to avoid the NFS-broken-over-loopback problem. I haven't had time to implement it yet, and personally and leery of doing so since I have to wonder what new "known-broken" problems will bite me in the ass.

So, if there is a better way, I am dying to hear it, unfortunately solaris/freebsd is not an option...