Slashdot Mirror
Revisiting the Five-Minute Rule
In 1987, a study published by Jim Gray and Gianfranco Putzolu evaluated the trade-offs between holding data in memory and storing it on a disk. Known widely as the "five-minute rule," their research was updated and expanded 10 years later. Now, as jamie points out, Communications of the ACM is running an article by Goetz Graefe with another decennial update, evaluating the rule using hardware and software typical of 2007, with an eye toward how flash memory will affect the situation. An excerpt from Graefe's conclusion:
"The 20-year-old five-minute rule for RAM and disks still holds, but for ever-larger disk pages. Moreover, it should be augmented by two new five-minute rules: one for small pages moving between RAM and flash memory and one for large pages moving between flash memory and traditional disks. For small pages moving between RAM and disk, Gray and Putzolu were amazingly accurate in predicting a five-hour break-even point two decades into the future. Research into flash memory and its place in system architectures is urgent and important. Within a few years, flash memory will be used to fill the gap between traditional RAM and traditional disk drives in many operating systems, file systems, and database systems."
http://en.wikipedia.org/wiki/Five-minute_rule
"The 5-minute random rule: cache randomly accessed disk pages that are re-used every 5 minutes."
The more useful 5 second rule.
I couldn't quite figure out if the article willfully ignored the advent of SSDs or was written before they were available and not updated to include them (but it appears the article was updated to include other current technology).
Given the fact that SSDs are likely going to replace rotational media for most applications in the future, it makes this article basically meaningless, at least insofar as the fact that flash memory and the disk are/will be synonymous. As the article is basically predicated around the entire fact that flash memory will change the 5 minute rule to a degree, it invalidates the entire article.
To be relevant, the article really needs to include the current state of SSDs and a likely projection (10 year) of where the state of the art in SSDs.
I do, however, suspect we may see a shift from drives all together at some point (perhaps more than 10 years, but perhaps not) and the computer will just have persistent storage for everything in MRAM or some other technology that obliterates the line between RAM (for speed) and drives (for storage) - it's just one big pool that's hyper fast and persistent.
So really, I don't think this article has held up in even the intervening two years since 2007, and it certainly won't hold up for another 10 years.
The article I read spent a good deal of time talking about flash memory. What article are YOU referring to?
five minutes is an awful long time for food to remain on the floor before you pick it up to eat it...
Ask Me About... The 80's!
This article indicates that Flash Memory (AKA SSDs) are only going to be an intermediary between rotational media and RAM.
If your handheld device or subnotebook PC has only an internal SSD and no internal hard drive, then you will store any data that doesn't fit on your SSD on a hard drive plugged into a Hi-Speed USB port, copying it to the SSD when it is needed. For example, you'd keep the video footage that you are editing on the SSD and other projects on the hard drive. That sounds to me like a memory hierarchy, albeit one that occasionally requires manual intervention to connect the (offline) long-term mass storage to the machine.
I propose a physical switch to toggle between read/write and read only.
And have social engineers disguise malware as OS updates or dancing bunnies, prompting the home user who doesn't understand risks to flip the switch to see the dancing bunnies.
The article I read spent a good deal of time talking about flash memory. What article are YOU referring to?
The article treats flash as something you place in between hard drives and memory. This turned out not to happen (with a few exceptions). SSD's simply replace hard drives. Hybrid systems are rare, and it doesn't look like they will become more common -- either you can live with the slowness of hard drives, or you can't. The mainstream will switch to SSD's for everything except backup applications.
There are some hybrid SAN's, but they're damn expensive. At that price they have a hard time competing with simpler pure-flash SAN's.
Except if you're using ZFS. You can put a (SLC or MLC) SSD drive into just about any system and tell it to act as a (write or read) cache.
MSDN? Noted. The author of that article has a vested interest in an operating system prone to such exploits.
Ad hominem.
the Linux user is somewhat less likely to a: have admin rights or to b: grant admin rights to a program if he has them.
In the case of Linux on the home desktop, the owner of the PC has admin rights, and I don't see how granting setuid is any harder to social-engineer out of an inexperienced Linux user than out of an inexperienced Windows user.
When I look at bunnies, I see food.
What about dancing scantily clad people of the appropriate sex?
What I love about slashdot is its scalability. The discussion ranges anywhere from the design of a Google data center in 2015 to some guy's psychological stance toward his next netbook purchase in 2009. Sometimes it's unclear which end of the spectrum is under debate, but the discussion happily progresses in a state of astral superposition. When this gets too confusing, even for slashdot, the moderation system helps to sort things out. For example, if the comment
Flash memory is set to replace rotational media.
is moderated +1 insightful, then we know we're talking about some guy's future netbook purchase. Or if the same comment is moderated -1 troll, then we know we're talking about Google data centers in 2015.
Flash memory begins to fade - ZDNet.co.uk from 2005
"The scaling laws are not favourable to flash," said Tom Lee, an associate professor of electrical engineering at Stanford University and a founder of Matrix Semiconductor, which makes a 3D memory chip that performs flash-like functions. "The noises are getting louder now, so it looks like manufacturers are already in that new age of diminished gains."
Numonyx Breakthrough Delivers First 45nm NOR Flash Memory Chips from Jan 2009
"Numonyx engineers overcame major scaling limitations by developing new process techniques to produce the 7th generation MLC NOR flash on the industry's most advanced 45nm technology, and to be the first to bring the cost and performance benefits to our customers." ..."
...
"At a time when the entire industry grapples with the scalability of all flash memory technologies,
Brewster Kahle
I think Brewster Kahle is going to jump off a bridge when he learns that Seagate is exiting the disk drive business in 2010. If you think CERN or EOS cost a lot of money, try updating the budget with SSD specified as the primary storage layer.
A useful way to view this transition is the long tail on steroids. 99% of the world's stored information will be held by a few hundred mega-scale institutions (NASA, Google, CERN, GenBank) on rotating hard drives, while 99% of the world's gadgets have no hard drive at all.
The same thing happened in software. The C language represents a tiny sliver of source code written over the last ten years, but if you could measure the number of machine instructions executed by language of origin, C would continue to represent a very large slice of the pie. A major factor in the success of scripting languages is that the problems these languages don't handle well can be off-loaded to a well established compiled language. If you cherry pick your niche, it's amazing how much more convenient it looks compared to the ancestral technology which didn't.
I thought the paper was quite good, and more relevant than 99% of what I read these days. I'm always interested in analysis of hybrid solutions. In the engineering world, there is a de facto allergy to hybrid solutions. We tend to achieve the best result by scaling a single virtue to the max, rather than engaging in the jello-like trade-offs involved in balancing complementary virtues. I first began to think about this when ethernet trounced ATM by the simple measure of vastly over-provisioning bandwidth.
The exception to this is on the large scale where operational costs exceed all other costs, such as major data centers.
This is one of the reasons why progress in ecology is so painfully achieved: ecological systems almost always demand hybrid solutions, and we're not terribly comfortable with this. Engineers prefer monarchy. In ecological systems, life is complicated, and you can't just sit there and
Since this is slashdot, I'd bet most will pick bacteria over carpet fuzz any day ... after all, if it doesn't look fuzzy ...
or this ...