Linux Support for Hybrid Hard Drives?

christoofar asks: "HHDD (Hybrid Hard Disk Drive) technology has been receiving some buzz lately. The concept is not new, but Samsung has been working on a consumer version of HHDD that everyone can use. HHDDs are disk drives that carry onboard RAM (in this case, NAND flash) which is non-volatile and offers to speed boot times and writes to the disk. This carries enormous benefit to laptop users who need to keep their disk activity to a minimum in order to preserve battery life. Given that Microsoft is adding support for Hybrid Hard Drives in their upcoming Windows Vista release, what efforts are being undertaken in the Linux realm to use this new storage technology?"

Comments

It should just work

[saskboy]

Why aren't the drives designed in a way that the drive technology worries about the details of retrieving and writing files, and sends them the way USB, IDE, SATA, or SCSI drives already do? Why would these drives need special drivers?

Re:It should just work

[mrchaotica]

I would guess that it's because having a software-based queuing and caching system is more flexible. For example, if you want to optimize operating system startup, then you need some way to specify that the operating system is what you want in the flash memory, instead of some other data.

Or if you've got a laptop, you maybe want to cache writes to the flash if you're using the battery, but write directly to the disk if you're hooked up to the mains. Since the operating system knows about this but the drive doesn't, the operating system needs to control the drive.

Re:It should just work

[AKAImBatman]

I'm guessing that it's not so much that it can't work that way, as it is that it would kill performance.

Today's operating systems try to do exactly what this drive is doing by pumping all reads and writes through the paging system. The problem is that the OS can't take a full performance boost from this or data will be lost in a crash or power loss. You may remember that EXT2FS could easily lose all the data you'd recently written if it wasn't unmounted properly. (It ticked me off when I tried to move RedHat 5.2 packages to a special partition, then found out that they had all disappeared on reboot because I hadn't explicitly unmounted the disk. *grumble*)

Since this technology has little to no chance of losing data in a power failure, the OS can be modified to write the blocks immediately. This could easily result in a performance increase of 2 to 3 times what you normally see today. The improvements in writing meta-data alone could easily provide this increase.

Re:It should just work

[Nethemas the Great]

If access to drive contents are bound to runtime drivers then that content can be made bound to DRM. Am I the only one noticing a nice little trend happening...

--Neth

Re:Is it me or does this seem pointless?

[msbsod]

Well, they only produce the isolinear chips for Star Fleet.
Seriously, the NAND flash memory stores the data immediately. It is a perfect write cache, because it does not forget the data. Almost perfect, because its write cycles are limited. But a more reliable file system would do the trick, too. Journaling is one way of making things more reliable. But, you have a good point. It is important that some data get written ASAP onto the medium. Unfortunately Linux file systems are not exactly the right choice if this is of importance for you. VMS would be a much better alternative. In order to utilize these hybrid disks it is also important to ensure that the important data gets written immediately, and that the hybrid disk knows which data needs to be cached in the flash memory. The disk allocation tables and directories are obviously important. But some databases might be important too. Bottom line is, the disks simply cannot be smart enough to handle all file systems and all applications. Therefore I agree with you, the whole thing is pointless. It would be much easier to back up some of the computers power lines with a battery, as part of the power supply, not as external UPS (stupid HV-LV-HV-LV conversion - HV=High voltage 110/230V, LV=3.3-12V). There are a few power supplies available which do exactly that (internal LV backup).

Re:Is it me or does this seem pointless?

[Anonymous Coward]

Right like, perhaps in addition to the kernel, or other essential boot files, it would also store the journal for journaled filesystems - it will cache the writes and commit them the next time that power supply and demand warrant. Consider a 1 Gbyte flash, in addition to your 100+ Gbyte drive, that's alot of non-volatile write cache.

Re:Is it me or does this seem pointless?

[EvanED]

This could IMPROVE the situation you're talking about.

First of all, you know that right now your writes are cached in your computer's memory for up to 30 seconds before they are flushed to disk. This is done so that synchronous read calls are less disrupted by writes.

It's possible that this new technology would allow the OS to commit writes to disk immediately (or at least much sooner). The disk writes to NVRAM or flash, at which point it should be stable through power loss or crash. The disk doesn't need to commit it to the actual platter right away for it to be safely stored.

Re:Random writes needed?

[Loconut1389]

i should add, if a couple million writes is true, even if you were to randomize or even out the bits algorithmicly, I still would be afraid of failure.

What to put in there?

[aspoon]

The question is not whether Linux will support the drive -- someone will eventually write one. The question is what files to put in there to make it boot faster. Perhaps should be done is to put the entire swap partition in there, and put those "hibernation" files in the swap. So it would resume fast, and would also make normal operations faster with the faster swap.

Re:What to put in there?

[tepples]

If you're swapping while running normal applications, something is wrong with your system.

Either that, or you're just dirt poor and trying to surf the net on 5-year-old equipment.

Why wait? Hack one up yourself!

[Myself]

You shouldn't have to wait for the drives to come out. Laptop drive controllers can address a master and a slave, they just don't have a slave drive connected in normal usage. I was just looking for an adapter to let me put a pair of CF cards in place of a 2.5" hard drive. (I can only find the single-card version in a 2.5" form factor, all the dual-card ones are for 3.5" mounting.) I figure, put a solid-state card in one slot and a microdrive in the other, and I've got a hybrid-drive laptop, right?

I just got rid of an old toughbook cf-25 that would've been perfect for this, as the drive mounting is gel and would easily accomodate an oddly shaped adapter instead of a regular drive. Or for the truly insane, a CF card piggybacked on a regular 2.5" drive! All I need is the ability to home-brew those little flex cables, and I'd be in business.

Nice to see

[Telvin_3d]

With all of the improvements in RAM, on-die cache and processor speed, the bottleneck in performence is HD speed. Anything that helps boot that is a welcome improvement.

Other good uses for this

[nial-in-a-box]

I think the real demand for flash-cache hard drives should be in the area of data loss prevention. Anyone who has any doubts about the fact that laptop hard drives are still way too fragile for the job needs to work at a help desk for a week. Any way to keep those drives spun down as much as possible should be seen as a good idea, since speed does not really matter much if you lose your data and/or need a new drive every three months simply because you like to move your laptop while it is running.

I know this just adds another point of failure to the mix with the addition of flash memory. However, with the apparent improvements in the quality of flash memory, I would expect one of these drives to outlast a current laptop drive by at least 50%. (Note: this is just idle speculation, but I don't know of any solid real-world statistics on laptop hard drive lifespan. I'm guessing laptop makers don't want us to know, either.)

I think that the point of drivers is, therefore, valid. There needs to be some sort of intelligence behind this system to allow frequently needed files to be held in cache in order for this to work effectively. Sure, you could build a drive that could try this on its own, but odds are you would totally throw out any performance or power advantage by doing so.

Another use?

[agentcdog]

So I see a completely different picture here. There are a lot of files on the HDD that are never rewritten. System files, etc. This is where having hybrid drives really helps. Put the main boot files, executables and libraries which are accessed when staring things up in there. So no fast saves, but no load time.

Re:Write caching in flash...

[greg1104]

Samsung's OneNAND design uses some interesting implementation details to run faster than is normally expected with flash. Their specs at http://www.samsung.com/Products/Semiconductor/OneN AND/index.htm suggest writes at 9.3MB/s and reads at 108MB/s; plenty fast for many applications even without running multiples in parallel. It's certainly much slower than writing directly to the drive if the drive is active. However, if the drive isn't spun up at the moment and the amount of data to be written fits in the NAND flash cache, I could see this being a net performance boost over the spinup/seek time combination of the hard drive.

They also spec 100,000 erase cycles before it's worn out. As was noted by an underrated poster the last time this came up, intelligent flash controller designs like this can cope with bad bits and assuring level usage of the memory much better than what you normally see in random hunk of flash.
An analysis at http://www.sudhian.com/printdocs.cfm?aid=686 suggests 33 years of usage for a typical worker. When you run the numbers it doesn't sound that difficult to create a design that would likely outlast the mechanical parts of a standard hard drive.

What I'd rather see anyway.

[IBitOBear]

I'd like to see linux have support for (and have hardware makers create) what SUN used to call NFS Accelerators. Basically add-on NV (or battery backed) memory sufficent to transparently cache write information. On my laptop I'd put it in my ExpressCard(tm) slot.

Then again I have been considering using an SD type card to contain the journal for my ext3 file system.

Actually, that combination; an ext3 file system in full data write-back journal mode, a solid-state flash device for that journal, and a "large" flush interval (more than the default five seconds) could accomplish essentially the same thing.

I wonder what the flash wear rate would be...? Do they make NAND SD cards?

(etc. od nausium. ahmen.)

Re:Simple solution ... still no special driver nee

[Skapare]

Maybe someone doesn't want their operating system on the RAM-ified part. Maybe they want swap space there instead. Maybe they want the reserved journal space of a journaling filesystem that is frequently updated to be there. Of course all of these needs could be addressed in some way (at least I know some journaling filesystems let you put the journal in a different partition). You'd have to choose which sectors (don't think in cylinders anymore, it's all fake anyway) would be RAM-ified in a way that would fit various needs in various operating systems, including that one with the largest market.

loss prevention

[Elwood P Dowd]

Most common source of data loss.

Laptop hard drives: the new floppy

It looks like the answer is "yes"

[NetRAVEN5000]

I've got the 2.6.15.4 kernel and there's a spot to enable support for "Memory Technology Devices".

Here's the description:
"Memory Technology Devices are flash, RAM and similar chips, often used for solid state file systems on embedded devices. This option will provide the generic support for MTD drivers to register themselves with the kernel and for potential users of MTD devices to enumerate the devices which are present and obtain a handle on them. It will also allow you to select individual drivers for particular hardware and users of MTD devices."

It's got options for a whole bunch of things, such as "FTL (Flash Translation Layer)", "NFTL (NAND Flash Translation Layer)", and "INFTL (Inverse NAND Flash Translation Layer)".

Given that it says the hard drives will use "a one-gigabit OneNAND flash chip" (according to the article), it sounds like it will work.