The Benefits of Hybrid Drives

feminazi writes "Flash memory is being integrated with the hard disk by Seagate and Samsung and onto the motherboard by Intel. Potential benefits: faster read/write performance; fewer crashes; improved battery life; faster boot time; lower heat generation; decreased energy-consumption. Vista's ReadyDrive will use the hybrid system first for laptops and probably for desktops down the road. The heat and power issues may also make it attractive in server environments."

Comments

Maximum Writes for Flash Memory?

[graphicartist82]

Most flash memory i've seen (such as the USB keychain drives), have a rated maximum writes before the memory starts having problems.

Am I missing something here? How are they going to overcome this if they plan on using the same type of memory for disk cache?

Re:Maximum Writes for Flash Memory?

[Eivind]

You're missing that the typical comercial flash-module is built to withstand 1 million writes or more.

A 1GB flash-module bein written to *constantly* (24 hours a day, 365 days a year) with a sustained speed of 5MB/s would thus wear out sometime after 6.5 *YEARS* of continous operation.

I'm guessing you can see why this problem is purely hypothethical for 99.99% of all laptops out there. You don't write to disc *constantly* and even if you did, you don't typically use the laptop 24/365, and even if you did, having a laptop-drive fail after 6-7 years is normally not a showstopper.

If, more realistically, the laptop is used 8 hours/day 250 days/years, and writes to disc 10% of the time when turned on, then the 1 million writes to flash will get reached after aproximately 30 years.

Even these numbers are high -- my laptop is heavily used as a developer workstation, and it certainly does not write to disc 10% of the time it is turned on.

Re:Finally...

[grammar fascist]

This is not a new idea, nor is it new technology... This has been a long time coming.

The prices finally fell to where it's economically feasible.

Personally, I like Intel's idea better (embedding the flash memory in the drive controller), because it should work just fine with existing drives. It might also be upgradeable, but I'm not holding my breath.

A good idea

[apathy maybe]

This is a good idea (even if it is old). In fact flash memory is so small that you could scrap hard drives altogether if you had enough money.

Imagine twenty 1 gig flash memory cards in a row ... less space then the equivelent hard drive.

another "benefit"

[plasmacutter]

another benefit of integrating flash memory onto the motherboard is the ability of hackers to hack your motherboard independently of the OS, and for friendly companies like microsoft to protect you from yourself by placing code they control in places you cant access on your machine.

no, I dont like this one bit, it's just a huge security hole begging for exploitation by hackers and DRM vendors.

Re:another "benefit"

[plasmacutter]

I hate to reply to my own post but look, it's not offtopic.

flash memory is persistant. Unless you provide open apis to allow anyone to develop applications to wipe it, there is no real way to confirm anything that gets stored on it is actually removed.

Every platform, but especially windows, has a history of security exploits, and now the viruses will have somewhere to hide where they will be much harder to dig out, and anyone wanting to implement DRM could build an OS designed to hide critical components of it by burying it on the flash memory.

Re:Finally...

[NitsujTPU]

I don't think that's what he's driving at.

People have been talking about doing exactly this technique for quite a while. It just never hit the mainstream. I even think that there were a couple commercial implementations of this, but I'm not sure on that last point. It is definitely talked about in research papers on filesystems that I have read.

Not applicable to server environments

[pestilence669]

"...The heat and power issues may also make it attractive in server environments..."

Not necessarily... perhaps during boot time. These potential savings are reserved for end-users who aren't doing anything data intensive. Last time I checked: database, web, email, and file servers are all data intensive... meaning that the drives will have to be spinning.

Hybrid drives do less in a server environment than a RAM disk. They can help boot faster, which is great for disaster recovery. If heat & power are a huge concern, flash drives, that are here now, solve those problems.

Catastrophic Failure of Flash Memory

[reporter]

The technical specifications of the flash memory in my USB drive says that it is guaranteed to work for, at most, 100000 (i.e., one followed by 5 zeros) writes. People do not talk about this limitation, but I have seen this limitation written into the technical specifications of the flash memory in many devices.

The hard drive in my Compaq x86 workstation has been humming nicely for more than 5 years. Due to the nature of my work at the institute, the number of writes to the hard drive have easily exceeded 100000 during that time.

Using flash memory as a fast cache for the hard drive will increase the performance of the drive but will decrease the overall life of the drive. Someone will be awfully upset when she makes a final save of her million-dollar PowerPoint presentation for the CEO and discovers that the save is the 100001st write to the hybrid drive.

Hopefully, the engineer who designed this hybrid drive has, at a minimum, integrated an LCD counter and a tiny speaker into the drive. The counter shall display the running total of the number of writes to the flash memory. The tiny speaker shall beep like crazy when the total exceeds 99900.

Re:Catastrophic Failure of Flash Memory

[flooey]

Using flash memory as a fast cache for the hard drive will increase the performance of the drive but will decrease the overall life of the drive. Someone will be awfully upset when she makes a final save of her million-dollar PowerPoint presentation for the CEO and discovers that the save is the 100001st write to the hybrid drive.

The thing to note is that that limitation is per flash block, not for the whole thing. So for a 1 GB flash component, given perfect block mapping, you can write around 100 TB of data to it before it wears out. With a 150MB/sec transfer rate, it would take more than a week of continuous writing to write that much. As well, modern flash can withstand a couple million writes, extending the life to several months of continuous writing. Given that this would generally be containing operating system components, which are read often but written to rarely, the lifespan of the memory should be no worry at all.

Re:Catastrophic Failure of Flash Memory

[Osty]

But, on the other hand, how often do you write to your windows folder? There's the monthly update, the occassional reg hack, but all in all, once it's established, that's a pretty static area of your drive. I could see this as an incredible benefit to system files, which, as has been discussed oft here before, the big reason for this.

Depends on what you're doing. For example, if you run IIS, your log files (by default; you can change this) are in %WINDIR%\Sytem32\LogFiles. That's going to have a lot of writes. Any new hardware or software installation may cause writes to %WINDIR%. There's a lot of other stuff that legitimately writes to %WINDIR% like installing a new printer (think roaming -- you may print to a different printer every day), the .NET Global Assembly Cache, Visual Styles and themes, and a whole lot more. Whether these things should be in %WINDIR% or not is a different question. The point is that using flash for %WINDIR% under the assumption that you'll not write there very often is a little naive. Perhaps Vista reorganizes %WINDIR% somewhat so that fewer processes need to write there.

There should be some level of safeguard built in that anything user created should be stored to the magnetic part of the drive, my documents, program files, but they should have this anyway. I mean, nothing like the last save and then having to call Dell because your drive is spitting out an Error Code 7...

All of this is a moot point anyway, because this use of flash is only as cache. Anything written to the flash drive should eventually be flushed to the hard drive. Similarly, if you've exhausted your write cycles and try to write to the cache, it should seamlessly catch the fault and go directly to hard drive. In that case it would be nice to give an occasional notice that your flash chip is exhausted and you need to replace it, but you should not risk losing any data. I'm not a big fan of on-board flash simply because it may be unreplaceable. Any onboard flash chips should not be surface-mounted, but socketed like RAM, CPU, or the clock battery. That will require some standardization on sockets, but as long as there are only two or three different options and the designers of said options let others build chips using that interface (*cough*Sony*cough*) it shouldn't be a problem.

In the long run, I think computer manufacturers will love this. How likely do you think your parents will be to replace their onboard flash when they run out of write cycles? The average consumer will just buy another PC for a couple hundred dollars rather than buying a new flash chip and installing it (or paying someone to install it).

How soon do you think the conspiracy theories will start up that manufacturers like Dell are intentionally shortening the life of onboard flash through factory "testing"?

Not in most servers...

[evilviper]

The heat and power issues may also make it attractive in server environments.

No, it won't. Servers have large ammounts of system RAM, which is far faster than flash on the hard drive bus could ever be. They also have battery-backed RAID controllers, meaning flash would be a step down, not a step up.

This is only really useful in notebooks.

Are standard file formats fine for use on flash?

[Chonine]

Are the standard NTFS or Ext3/Reiser/Whatever optimized for use on hard drives? If flash drives start appearing as main system drives, would new or modified versions of file systems help in any way? Or are modern file systems abstract enough to where they dont deal with all the little fiddly-bits? I don't know enough about this area, but it would seem to me that a new hardware device to store files may benefit with a change in the way the OS uses it.

Where to put the flash?

[Michael Woodhams]

The article discusses this. Intel want to put it on the MB, the drive manufacturers want to put it in the drive. A third option is to attach it separately and externally (e.g. a USB flash drive.) A final option would be to (e.g.) have a compact-flash-card (or similar) socket on the hard-drive, and users provide their own flash.

To my mind, the logical place to put it is on the drive. This is where the useful caching information is most easily available. (Which sectors are read/written how often? Which reads are often delayed by waiting for the disk to spin up?) This is also where you can make the process most transparent. The drive's firmware can make the system "just work", like a standard HD, but faster - whatever the OS, no drivers needed. (Although you'd possibly like to have drivers to give the OS more control over what is flash-cached.)

Re:Where to put the flash?

[Locutus]

how about using CF and a CF->IDE adapter for desktops and hang the adapter off the IDE bus. CF based PCcard adapters in laptops maybe? You can do this today and IIRC, CF cards have builtin wear leveling. move your /boot onto this for quicker boots and/or put your swap here and have quicker restores from hibernates.

LoB

Re:Are standard file formats fine for use on flash

[I kan Spl]

Most filesystems are in fact optimized for use on magnetic media. Ext3 uses algorithms to place data on the disc in order to minimize the amount of waiting done for data.

There are research filesystems that are optimized for this kind of a hybrid environment. These were written for MEMS insetead of flash, but the basic ideas are nearly the same.

http://www.ssrc.ucsc.edu/proj/mems.html

Disclaimer: I work there. I may be biased.

Suspend to disk + flash for better boot times.

[Chonine]

It seems like there is a huge demand for faster booting systems, but so few people use suspend to disk (hibernate for windows, goes by other names too). Shutting down and booting are faster, and it uses *no* power when off. It seems to me that some people are overly fixated on faster boot times, so long as no interesting software tricks such as suspend are used. Why is that? Many people want a faster booting computer, but refuse to do so with anything other than a traditional boot. I understand the limitations and the need to do a traditional reboot from time to time anyways, but suspending to disk is a great feature that is here now, that doesnt even need ACPI or any sort of power management doo hickery.

Flash is great but even with its random access speeds, the throughput isn't much better than drives, and so I don't see such a huge boost in boot times from flash alone. To have your cpu do all of that work every boot seems a bit rediculous. Reading a 512+ MB file into memory and a few adjustments, you are back to where you were.

(As an aside, can anyone tell me how BeOS was able to boot in only around 7 seconds for me a decade ago, to a fully usable desktop? From fully off to fully usable, that was nuts.... what can modern operating systems do to approach this?)

Massive disk cache

[nmg196]

We seem to be going backwards. About 10 years ago, I had a vesa local bus HDD controller which took SIMMS to use as cache. You could shove up to 32mb on it and it would remain powered even when the system was shut down. This meant you could load DOS and even Windows 3.11 entirely from the disk cache after rebooting. As far as I'm aware, there are no SATA controllers which can take DIMMS or similar to use as a large cache. PLEASE correct me if I'm wrong.

Why doesn't this exist today? I think it was a really good idea. The closest thing I've found is Gigabyte's iRam, but this isn't really the same thing - as it's purely a RAM drive and doesn't persist to hard disk.

I think that slow booting is the one of the biggest annoyances of computers and the primary reason many people never turn off their machines in an office environment (hiberating on XP rarely works reliably in my experience - usually due to driver issues not reinitialising the hardware properly rather than there being any problem with XP itself).

If people's machines booted to the desktop in under 10 seconds, far more people would turn them off at the end of the day and worldwide power consumption would be significantly reduced.

Write Limitation

[xdxfp]

100,000 writes is only a median of the distribution. Some will be higher and some will be lower, so a counter would be useless. I'm sure it's made of a higher quality RAM than your typical flash drive. 100,000 writes would last about one day on a server, and probably less than four days for your typical PC [if you assume one write per second for a busy server, which is not unreasonable]. I'm not sure a flash cache makes a hell of a difference. Why not just use RAM, and have a battery to keep the memory state for 20 or 30 minutes if the power were to shut off? Plus even the fastest static RAM is no where near the fastest regular RAM. If they can get the size, speed, and reliability up to par perhaps it could be useful.

Re:Finally...

[Fordiman]

Still, it might not be a bad idea to have flash journalling; have the controller record a disk-to-disk write and return immediately, intelligently handle disk reads, even if the data hasn't been relocated yet, etc. The flash chip just stores a list of actions (like in a journalled FS) and the controller performs them. They can be suitably small (1 block) so as to keep state granularity high.

No, seriously. Sure, lots of filesystems journal, but how many can journal with separated control? In a normal journalled fs, the journal is stored on the same media as the disk - and thus suceptable to the same corruption.

It's like the issues that put journalling in place originally; sure, some data can fail and we're ok, but if the filesystem structures fail, everything is effectively lost.