Revisiting the Five-Minute Rule

In 1987, a study published by Jim Gray and Gianfranco Putzolu evaluated the trade-offs between holding data in memory and storing it on a disk. Known widely as the "five-minute rule," their research was updated and expanded 10 years later. Now, as jamie points out, Communications of the ACM is running an article by Goetz Graefe with another decennial update, evaluating the rule using hardware and software typical of 2007, with an eye toward how flash memory will affect the situation. An excerpt from Graefe's conclusion: "The 20-year-old five-minute rule for RAM and disks still holds, but for ever-larger disk pages. Moreover, it should be augmented by two new five-minute rules: one for small pages moving between RAM and flash memory and one for large pages moving between flash memory and traditional disks. For small pages moving between RAM and disk, Gray and Putzolu were amazingly accurate in predicting a five-hour break-even point two decades into the future. Research into flash memory and its place in system architectures is urgent and important. Within a few years, flash memory will be used to fill the gap between traditional RAM and traditional disk drives in many operating systems, file systems, and database systems."

Comments

for those wondering:

[wjh31]

http://en.wikipedia.org/wiki/Five-minute_rule

"The 5-minute random rule: cache randomly accessed disk pages that are re-used every 5 minutes."

Not to be confused with

[salahx]

The more useful 5 second rule.

Re:Not to be confused with

[michaelhood]

I'd rather be a disgusting American than a naive European with no sense of humor..

Re:What article?

[amorsen]

The article I read spent a good deal of time talking about flash memory. What article are YOU referring to?

The article treats flash as something you place in between hard drives and memory. This turned out not to happen (with a few exceptions). SSD's simply replace hard drives. Hybrid systems are rare, and it doesn't look like they will become more common -- either you can live with the slowness of hard drives, or you can't. The mainstream will switch to SSD's for everything except backup applications.

There are some hybrid SAN's, but they're damn expensive. At that price they have a hard time competing with simpler pure-flash SAN's.

Re:What article?

[wwwillem]

Hybrid systems are rare, and it doesn't look like they will become more common.

You're probably right when we talk about desktop PCs and laptops. I'm sure the latter will be SSD only in 5-10 years time, and desktops are also losing terrain quickly against laptops.

But when we look at datacenter grade enterprise storage, hybrid systems are currently picking up fast. The advantage is that because of the fast 'flash memory cache' you can use SATA disks instead of the FC/SCSI drives, where the former are both much bigger and much cheaper. Instead of 300 146GB 15K FC disks, you only need 30 1.5 TB 7200 RPM SATA disks. For the same capacity this results in much lower power bills, less DC floor-space costs and much better performance.

If you say "There are some hybrid SAN's, but they're damn expensive.", have a look at what [shameless plug-on] Sun is doing, and yes, I work for Sun [plug-off]. But other storage vendors (NetApps, EMC, IBM, etc.) are starting to do similar things.

So the whole "storage-stack" gets more and more hybrid and integrated. It consists of the full gamut of DRAM, flash memory, hard drives and finally tape. Each of these have their own strength and are used best in combination.