Slashdot Mirror
The Benefits of Hybrid Drives
feminazi writes "Flash memory is being integrated with the hard disk by Seagate and Samsung and onto the motherboard by Intel. Potential benefits: faster read/write performance; fewer crashes; improved battery life; faster boot time; lower heat generation; decreased energy-consumption. Vista's ReadyDrive will use the hybrid system first for laptops and probably for desktops down the road. The heat and power issues may also make it attractive in server environments."
This is not a new idea, nor is it new technology... This has been a long time coming.
I'm not fat, just big boned...
Old news.. but still, it all depends on how well the drive caches. If it's writing useless data it's worthless... a swapfile to "slower" memory could be nice.
Will they increase fuel economy as well?
I remember in 2002 when I was enrolled in the CISCO CCNA classes, we were playing around with a hybrid console. It was a small HD maybe 5 gig and small flash drives working in conjunction. Ok so it wasnt EXACTLY the same but it worked on the same principal. I only wish I could remember the model number...
have you seen my sig? there are many others like it but none that are the same
Most flash memory i've seen (such as the USB keychain drives), have a rated maximum writes before the memory starts having problems.
Am I missing something here? How are they going to overcome this if they plan on using the same type of memory for disk cache?
MRAM would have been a better choice.
This is a good idea (even if it is old). In fact flash memory is so small that you could scrap hard drives altogether if you had enough money.
... less space then the equivelent hard drive.
Imagine twenty 1 gig flash memory cards in a row
I wank in the shower.
another benefit of integrating flash memory onto the motherboard is the ability of hackers to hack your motherboard independently of the OS, and for friendly companies like microsoft to protect you from yourself by placing code they control in places you cant access on your machine.
no, I dont like this one bit, it's just a huge security hole begging for exploitation by hackers and DRM vendors.
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
Plug in a USB or Flash drive and mount it as a non removable drive (drivers exist for this purpose...google them) then set your page file and temp files, etc. to the flash drive.
The solid state portion of the drives are really only good for data that will not change often. That section suffers from limited number of re-writes before the data integrity degrades. The hybrid disks work well mainly for the primary system OS disk and that is really just about it. The kernel and main OS components will rarely change (patches and kernel updates are the only times). This is why boot times are increased using these disks, because the OS and kernel is contained on the faster solid state memory...
Again, in an environment where data is constantly being written and deleted, these disks will fail a lot sooner.
We were all warned a long time ago that MS products sucked, remember the Magic 8 Ball said, "Outlook not so good"
I'd prefer something longer lasting (and faster) than flash memory.
Haiku for you!
"...The heat and power issues may also make it attractive in server environments..."
Not necessarily... perhaps during boot time. These potential savings are reserved for end-users who aren't doing anything data intensive. Last time I checked: database, web, email, and file servers are all data intensive... meaning that the drives will have to be spinning.
Hybrid drives do less in a server environment than a RAM disk. They can help boot faster, which is great for disaster recovery. If heat & power are a huge concern, flash drives, that are here now, solve those problems.
What about increased reliability? I realize a lot of this might depend on how the flash memory is interfaced, but it would be awesome to have a small built in flash chip capable of live backups of critical data. With say a spare gig of memory on the hard drive, it should be more than feasible to have data of certain folders (e.g. My Documents and system folders) in the off chance that your hard drive actually does fail. Being able to boot directly to the flash chip would be great in emergencies, and a copy of DSL/Puppy Linux/*Your favorite recovery tool* would be perfect to store there. Bonus points if you can easily (i.e. without a soldering iron) swap the flash chip to a fresh drive and do a Stage 1 Gentoo reinstall from scratch.
Come to think of it, the possiblities of RAIDing these things together could be interesting as well. With a RAID 1, all but the most paranoid wouldn't need to include the flash memory in the mirror. Or, should the flash memory get sufficiently large (say, 20-25% of the hard drive size), you could use the flash memory as dedicated parity in a RAID 4 array. Obviously this means squat if you can't interface the flash memory properly, but hey, at least the possibilities are there.
I only mod funny =D
The hard drive in my Compaq x86 workstation has been humming nicely for more than 5 years. Due to the nature of my work at the institute, the number of writes to the hard drive have easily exceeded 100000 during that time.
Using flash memory as a fast cache for the hard drive will increase the performance of the drive but will decrease the overall life of the drive. Someone will be awfully upset when she makes a final save of her million-dollar PowerPoint presentation for the CEO and discovers that the save is the 100001st write to the hybrid drive.
Hopefully, the engineer who designed this hybrid drive has, at a minimum, integrated an LCD counter and a tiny speaker into the drive. The counter shall display the running total of the number of writes to the flash memory. The tiny speaker shall beep like crazy when the total exceeds 99900.
Another benefit of hybrid drives is, you can use the carpool lane even if you're by yourself.
I think that this would have been a much relavant product if it was released 10 years ago, when flash memory was much harder to come by. But I would say we're less than one upgrade cycle away from completely solid state drives anyway, the ones that several manufacturers have been showing of over the past year and due very soon. I say don't bother with this concept that is a decade too late and soon to be obselete anyway. Why Linux sucks #42104: [SHIFT] + ["] ! = "
No, it won't. Servers have large ammounts of system RAM, which is far faster than flash on the hard drive bus could ever be. They also have battery-backed RAID controllers, meaning flash would be a step down, not a step up.
This is only really useful in notebooks.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Run, Vista! Run! Don't let those bullies Linux and OS X catch you!
It's not offtopic, dumbass. It's orthogonal.
Haven't any of you been playing with the Vista betas? Vista has a sort of swap file / prefetch feature that you can enable on USB flash drive. Vista first benchmarks the device, to determine if it is fast enough. Then you can create a sort of swap file on it, as big as you like.
l eID/48085/48085.html
It's part of the Vista SuperPrefetch.
http://www.windowsitpro.com/Windows/Article/Artic
Are the standard NTFS or Ext3/Reiser/Whatever optimized for use on hard drives? If flash drives start appearing as main system drives, would new or modified versions of file systems help in any way? Or are modern file systems abstract enough to where they dont deal with all the little fiddly-bits? I don't know enough about this area, but it would seem to me that a new hardware device to store files may benefit with a change in the way the OS uses it.
The article discusses this. Intel want to put it on the MB, the drive manufacturers want to put it in the drive. A third option is to attach it separately and externally (e.g. a USB flash drive.) A final option would be to (e.g.) have a compact-flash-card (or similar) socket on the hard-drive, and users provide their own flash.
To my mind, the logical place to put it is on the drive. This is where the useful caching information is most easily available. (Which sectors are read/written how often? Which reads are often delayed by waiting for the disk to spin up?) This is also where you can make the process most transparent. The drive's firmware can make the system "just work", like a standard HD, but faster - whatever the OS, no drivers needed. (Although you'd possibly like to have drivers to give the OS more control over what is flash-cached.)
Quattuor res in hoc mundo sanctae sunt: libri, liberi, libertas et liberalitas.
Hey mods next time look at the timestamp before you mod redundant. Just because the other posts replied to a post before mine doesn't mean that the information contained in those replies came before my information did.
We were all warned a long time ago that MS products sucked, remember the Magic 8 Ball said, "Outlook not so good"
I like the part best where these hybrids fund the R&D for pure transistor drives without moving parts.
--
make install -not war
At we host many Asterisk VPSes, and because our customers rely on us for their phone service, any downtime, even planned well in advance, is something we try very hard to avoid. Because downtime from security patches is inevitable, this would be a big win for us, and a good selling point, so using it seems to be a no brainer.
My question is - what kinds of support can/will the linux kernel have for this? We run Gentoo Linux as our host OS, and I cannot see us migrating to Windows for the forseeable future. I searched a bit online, and found this ask slashdot story, but no authoritative answer. Anyone?
Most filesystems are in fact optimized for use on magnetic media. Ext3 uses algorithms to place data on the disc in order to minimize the amount of waiting done for data.
There are research filesystems that are optimized for this kind of a hybrid environment. These were written for MEMS insetead of flash, but the basic ideas are nearly the same.
http://www.ssrc.ucsc.edu/proj/mems.html
Disclaimer: I work there. I may be biased.
My UID is prime and so is this number: 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0.
Well, I guess we know why the preview button's there... good thing I code better than I post, otherwise I'd be fired!
Damn. The RSS feed made me think this might be about hybrid _cars_, not hard drives. I was already dreaming of making clever comments about how cool it is to own a Toyota Prius. Now I make whiny comments about getting it wrong instead. Damn. Mod me down for futility and insignificance.
Continuous positive slashdot karma since... uh, maybe next year.
Flash is great but even with its random access speeds, the throughput isn't much better than drives, and so I don't see such a huge boost in boot times from flash alone. To have your cpu do all of that work every boot seems a bit rediculous. Reading a 512+ MB file into memory and a few adjustments, you are back to where you were.
(As an aside, can anyone tell me how BeOS was able to boot in only around 7 seconds for me a decade ago, to a fully usable desktop? From fully off to fully usable, that was nuts.... what can modern operating systems do to approach this?)
ofcourse what will be written in the flash memory will not be the data, but program code what is seldom changed.
has lots of benefits! http://www.shadetreemechanic.com/allison_hybrid_dr ive.htm/
1)Faster startup times.
2)Lower noise.
3)Fluid-coupled technology.
3)Better power efficiency.
FTA: "This is pretty incredible technology."
By using more of these we could reduce energy usage, pollution, and global warming!
Coupled with an HOV lane, it may even result in faster transfer rates!!
I thought Digg was bad.. then i saw this guy.
I hope the flash goes straight on the motherboard, and can be made to operate independently of the hard drive. Damn Small Linux only takes up 64 MB on the disc and ~100 MB of RAM (I could be off). With that 512 MB of flash straight on the motherboard, I'd be able to boot up and suspend instantly! Now, if they'd start making laptops with OLED screens, just think of they battery life!
On high end systems its been fairly common to write the journals to nvram. Since journals have to be synchrous writes, this being available on commodity drives could greatly improve the performance of ext3 and other journalling file systems. Otherwise, your stuck waiting for journal commits to get to disk before you can move on.
nor is it a brand new technology - but it's only going to happen and appear on the site of your favorite component supplier because MS has decided to support it in it's new OS.
A whole load of new hardware tech never takes off as it's a bit chicken and egg - I'm not buying a PhysX card until I actually find my software will support it and the software's not going to be made until I buy the card etc.
People might bitch and moan about MS, but it looks like they can actually make new stuff happen.
MS decide they're going to support a new type of drive -> All the drive makers start making the new sort of drive -> Linux/OSX will get drivers to support the new drive -> We all benefit.
See MS is good for something.
640mb of hard drive flash cache, should be enough for anyone.
I've not been able to figure out why flash RAID setups aren't more popular in portable devices. 10 4GB flash disks in a RAID would give you what, 20GB drive space with 100Mb/s bandwidth. Not bad for memory that doesn't have any moving parts and doesn't vanish without power. And a 'drive' like that would be about the size of a typical laptop network adapter card. I'm guessing cost is the main problem?
A-Bomb
We seem to be going backwards. About 10 years ago, I had a vesa local bus HDD controller which took SIMMS to use as cache. You could shove up to 32mb on it and it would remain powered even when the system was shut down. This meant you could load DOS and even Windows 3.11 entirely from the disk cache after rebooting. As far as I'm aware, there are no SATA controllers which can take DIMMS or similar to use as a large cache. PLEASE correct me if I'm wrong.
Why doesn't this exist today? I think it was a really good idea. The closest thing I've found is Gigabyte's iRam, but this isn't really the same thing - as it's purely a RAM drive and doesn't persist to hard disk.
I think that slow booting is the one of the biggest annoyances of computers and the primary reason many people never turn off their machines in an office environment (hiberating on XP rarely works reliably in my experience - usually due to driver issues not reinitialising the hardware properly rather than there being any problem with XP itself).
If people's machines booted to the desktop in under 10 seconds, far more people would turn them off at the end of the day and worldwide power consumption would be significantly reduced.
I've always wondered why MS hasn't considered putting Windows on a memory chip that could plug into a custom Operating System port. They could fully document the hardware, so that other OSes can use the technology without them being accused of a monopoly. I'm sure it would make instant booting more of a reality. And with full access to the windows installation on the chip you wouldn't have anything to copy over on an install, it'd just be a matter of detecting the hardware and making some settings files (overly simplified, but you know what I mean); in fact not having the Windows eat up hard drive space could be a selling point for some users. But I suppose this hybridization will help a bit, but I don't think it was neccesary to incorporate it into the Hard Drive. I'm more picturing a port on the front of your case where you plug in your OS with a chip emblazoned with a Microsoft, Apple, Debian, Red Hat, Gentoo etc. etc. logo.
100,000 writes is only a median of the distribution. Some will be higher and some will be lower, so a counter would be useless. I'm sure it's made of a higher quality RAM than your typical flash drive. 100,000 writes would last about one day on a server, and probably less than four days for your typical PC [if you assume one write per second for a busy server, which is not unreasonable]. I'm not sure a flash cache makes a hell of a difference. Why not just use RAM, and have a battery to keep the memory state for 20 or 30 minutes if the power were to shut off? Plus even the fastest static RAM is no where near the fastest regular RAM. If they can get the size, speed, and reliability up to par perhaps it could be useful.
HRESULT WinAPIGetSystemProcessThreadMetricsMenu...
LibraryVolumeModuleHandlePtrEx(PHSPTMMLVM PHndl);
Oh wait. Hybrid Drives? Never mind.
http://www.microsoft.com/emea/itsshowtime/sessionh .aspx?videoid=173
For quite a while I've been curious about a similar application of flash. Suppose someone builds an NVRAM card or drive, which is RAM with a battery backup; the battery holds enough power to save the RAM to flash should the power go out. Between the interface, battery, controller chip, flash and RAM it should cost on the order of $100-$150/GB, retail, if it became reasonably popular; if Dell put a half-gig chip in every outgoing workstation, it would probably cost them $20-$50. It could be in a PCI card for servers, or it could go into high-performance disks. Furthermore, if the controller were clever, it could improve price/performance by using flash as a clean cache for slowly-changing files, much as these hybrid drives do; if the power went out, this cache would be overwritten with the contents of the RAM, but that would merely cause a performance penalty on the next boot.
...) would be more performant: writes could be batched without violating transaction semantics, and metadata could be updated in place. Writes in general would have much lower latency and higher burst bandwidth, as they could be made to NVRAM first. It would be easier to incorporate Sun's "RAID-Z" technology to improve performance for higher-levels of RAID, and because of the transaction log, RAID5 would no longer carry the corruption risk that it currently does.
If such a chip became standard would revolutionize filesystem storage. In half a gig, you could cache the entire filesystem's metadata (locate would become obsolete), most of the boot sequence, a transaction log, and a cache of most recently accessed files. And since it would be writing to RAM first, it would be much, much faster than disk, and it wouldn't wear like flash (since the flash is only used when the power goes out, or when you shut down).
You would be able to build transaction-oriented filesystems much more easily can be done with disk alone. Write-anywhere filesystem layouts (ReiserFS, Netapp,
Database designers currently use 10kRPM (and higher) disks for their transaction logs. Why can't they use NVRAM?
I'm sure someone has thought of this before, and I've heard of NVRAM cards in high-end servers, but why doesn't it go to the desktop or laptop market?
I hereby place the above post in the public domain.
NOR flash (like the BIOS chip in your PC) is good for 1M writes or more.
But NOR flash is low density. An 8MByte NOR flash is large.
The flash that is being integrated into these drives is NAND flash. NAND flash is the kind of flash you use in your digital camera. NAND flash is high density.
And it is crap.
SLC NAND flash is good for 100,000 writes. But SLC is on the way out because it's only half as dense as MLC NAND flash. MLC NAND flash is good for 10,000 writes.
Are you scared yet?
That's a statistical measure, so often cells last longer than 10,000 writes before crapping out. And systems that use NAND flash use ECC (error correction codes) and wear levelling to try to hide the flash wearing out. It's complex, but it does work pretty well.
But a coworker made a flash burner app to wear out some flash on purpose. It wrote constantly. He able to wear it out in a couple days. It didn't wear out the entire flash chip, but that's when the flash started to develop sectors that were unusably bad, even with ECC.
http://lkml.org/lkml/2005/8/20/95
Flash has a limited no. of write cycles (10^5 - 10^6 depending on tehchnology) so when the flash memory has been written too many times, then you have to replace the hard drive too. Great idea only if you're an HD manufacturer. Costly for consumers.
P roducts_Others_Storage.html.
Its a much better idea to keep the flash separate, say on a PCI-e card. So you could replace it when it burns out without replacing the drive. Even better would be something like this but with a bigger battery:
http://www.gigabyte-usa.com/Peripherals/Products/
Another benefit of memory on a system bus rather than the ATA bus is I/O performance.
Yeah, and what happens if your flash chips "wear out" well before your hard-drive platter? Does your performance subtly degrade, or do OS that "require" this hybrids suddenly refuse to operate? Will the flash be replaceable?
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
C:\Windows\pagefile.sys is a very high traffic file. Now, you can manage that with a separate drive ... but few people do.
There's nothing preventing you from creating a separate partition for the swap file in Windows. But personally, I think the Windows way is more flexible. Why use up a large chunk of hard drive space for that one time you happen to use an obnoxious amount of swap?
Besides, only in rare circumstances will the swap file fragment - if your swap file is grows automatically, it usually won't fragment badly unless the drive is mostly full and fragmented. For most people, slashdot crowd excepted, hard drives are ridiculously large, so the swap won't fragment.
I probably wasn't old enough to care about anything other than games when the controller you describe was sold... however, I'm going to guess that it was just too expensive a feature when computers got cheaper and cheaper. If you want a drive controller with DIMMs for cache, just check out the server RAID controllers - each card cost more than a cheap Dell workstation!
IMHO, hibernate works reasonably well. The biggest problems in my personal use relate to network activities that can't be worked around with the "cached" concept. Network connections die and often programs don't know what to do.
My idea for saving power in the office would revolve around a smarter use of Wake-on-Lan. Maybe a central server to boot up all the PC's at 8:00AM or so. But that's another story...
> > The technical specifications of the flash memory in my USB drive says that it is
> > guaranteed to work for, at most, 100000 (i.e., one followed by 5 zeros) writes.
>
> I thought I'd seen specs an order of magnitude larger than that in many cases, but
> the problem still may not be as bad as you think in many cases even if it is as bad
> as 100000 writes. The reason? Flash devices have systems built in to their
> controllers specifically to deal with these problems. The mechanisms may vary, but
> the ones I know about are wear leveling and excess capacity (beyond the capacity
> that the device reports to the operating system) that can be pressed into service
> when a block fails.
Wear levelling can only work, if the changes are done to tiny portions of the whole memory capacity. It can't work, when the whole device is changed.
Why? It's simple. Lets say a cell can be written 100000 times. A device with 32*8 million of those cells (32 MB), can accept a total of 100000 * 32*8 million single bit writes. Or half that amount of 2-bit writes. Or 100000 writes of a 32*8 million bit block.
The idea of the hybrid harddisk is that a RAM cache holds all the changes. Once RAM is full, changes are flushed to the magnetic media. The write-acceleration stems from the fact, that several individual writes (*) can be combined to single operations, and write order can be optimized according to the relative positions of head / platter.
For this to work, RAM utilization has to be high. High RAM utilization also means that lots of RAM must be dumped to FLASH on power failure. But in the ideal situation (100% RAM utilization), FLASH write size is 100%, too. That is, no place for wear levelling. No worn blocks can be exempt from such a write. All blocks have to join, sorry.
On the other hand, 100000 writes mean that you can power-cycle your computer once per minute, 24h per day, during almost 3 months. Or once every hour during 10 years. I bet the power supply unit bails out first..
Marc
(*) Operating systems already try to combine writes of course, but they can only do so to a certain extent. For example, journaled filesystems must write the journal first, then the data, and then journal, and so forth. It's not possible to combine several (related) journal updates within the scope of the operating system, but a hybrid harddrive can easily do so.
I don't have a problem with the way Windows does things, but I do think Linux is more flexible. And as mentioned, my pagefile literally disintegrated. I'd be tempted to turn off VM, reboot and delete the pagefile.sys and turn VM back on if I thought I could guarantee it worked afterwards, but it's a company machine so it is a risk I don't feel comfortable taking.
I am suprised that no-one has considered the fact that using flash storage would remove one more moving part from your computer - the spinning hard disk. Admittedly, disk noise has reduced considerably over the years, but has anyone had a listen to a HDD that has been running in a server for a year or so? You get that annoying high pitched whine (feel free to insert joke here...). I must also admit that I am a bit of a scavenger when it comes to IT equipment - anything my company is throwing out is usually only a few years old and still usable, but I tend to draw the line with HDDs - they are just too noisy and I'm better off getting something new.
Wanted: A better sig than this one. I have neither the wit nor motivation...