How To Use a Terabyte of RAM
Spuddly writes with links to Daniel Philips and his work on the Ramback patch, and an analysis of it by Jonathan Corbet up on LWN. The experimental new design for Linux's virtual memory system would turn a large amount of system RAM into a fast RAM disk with automatic sync to magnetic media. We haven't yet reached a point where systems, even high-end boxes, come with a terabyte of installed memory, but perhaps it's not too soon to start thinking about how to handle that much memory.
For those of you who don't have Adblock: Printerfriendly Version
One important thing to consider, is that if using a ramdisk for important stuff, what happens when the power dies?
For example, will the stuff synced from magnetic media be stored elsewhere? If so, what happens to the speed?
-B
You wrote: "We haven't yet reached a point where systems, even high-end boxes, come with a terabyte of installed memory" - this is not true. Sun's E25k can go over 1TB of memory.....
Linux gobbles free RAM to add to the buffer cache. This is already a large RAM disk with automatic sync. In embedded systems, you can even decouple the buffer cache from any physical media and just live in a variable size RAM disk, which means that Linux finally catching up to AmigaDOS.
If you run MS SQL Server and don't manage the RAM then it will use it all just for the fun of it.
If you find this in any way strange, wrong or confusing, perhaps you should read up as to what the primary purpose of a frikkin' DATABASE SERVER is.
Here's a hint: the more data it can keep readily accessible (that is, in RAM) the better it will perform. And as you mentiones, you can of course set it to use less RAM if you have to. It's just that it's optimized for performance by default.
Quality, performance, value; you get only two, and you don't always get to pick.
posix_fadvise() technically does allow you to do what you want. You can use posix_fadvise(POSIX_FADV_DONTNEED) to evict the buffer cache in the IO loop of the program. See http://insights.oetiker.ch/linux/fadvise.html for the ugly details. Unfortunately you can't just make one system call and have it effect an entire file or process. POSIX_FADV_NOREUSE is supposed to be the default in the kernel buffer cache management so it is implemented as a no-op.