Slashdot Mirror

← Back to Stories (view on slashdot.org)

How To Use a Terabyte of RAM

Posted by kdawson on from the every-factor-of-25-helps dept.

Spuddly writes with links to Daniel Philips and his work on the Ramback patch, and an analysis of it by Jonathan Corbet up on LWN. The experimental new design for Linux's virtual memory system would turn a large amount of system RAM into a fast RAM disk with automatic sync to magnetic media. We haven't yet reached a point where systems, even high-end boxes, come with a terabyte of installed memory, but perhaps it's not too soon to start thinking about how to handle that much memory.

17 of 424 comments (clear)

  1. Add-Free one-page Version of the story by saibot834 · · Score: 4, Informative

    For those of you who don't have Adblock: Printerfriendly Version

    1. Re:Add-Free one-page Version of the story by Freedom+Bug · · Score: 4, Informative

      Here's a much better link on Jon Corbet's own site, the famous Linux Weekly News:

      http://lwn.net/Articles/272011/

  2. Power Failure by Anonymous Coward · · Score: 3, Informative

    One important thing to consider, is that if using a ramdisk for important stuff, what happens when the power dies?

    For example, will the stuff synced from magnetic media be stored elsewhere? If so, what happens to the speed?

    -B

    1. Re:Power Failure by itsjz · · Score: 5, Informative
      There's about three paragraphs in the article discussing this. Basically, use a UPS:

      If line power goes out while ramback is running, the UPS kicks in and a power management script switches the driver from writeback to writethrough mode. Ramback proceeds to save all remaining dirty data while forcing each new application write through to backing store immediately.
  3. uh - there is at least one system with 1TB of RAM by Anonymous Coward · · Score: 5, Informative

    You wrote: "We haven't yet reached a point where systems, even high-end boxes, come with a terabyte of installed memory" - this is not true. Sun's E25k can go over 1TB of memory.....

  4. nothing new here by dltaylor · · Score: 3, Informative

    Linux gobbles free RAM to add to the buffer cache. This is already a large RAM disk with automatic sync. In embedded systems, you can even decouple the buffer cache from any physical media and just live in a variable size RAM disk, which means that Linux finally catching up to AmigaDOS.

  5. Re:1 TB of memory... by Digi-John · · Score: 2, Informative

    emacs is a Lisp interpreter, an editor, a games package, an irc client, many things, but its memory usage is just a drop in the bucket compared to the monstrosities I mentioned above. Of course, there's quite a few complete operating systems that can boot in the amount of RAM required by emacs :)

    --
    Klingon programs don't timeshare, they battle for supremacy.
  6. Re:The problem with giving Windows 1TB... by W2k · · Score: 3, Informative

    If you run MS SQL Server and don't manage the RAM then it will use it all just for the fun of it.

    If you find this in any way strange, wrong or confusing, perhaps you should read up as to what the primary purpose of a frikkin' DATABASE SERVER is.

    Here's a hint: the more data it can keep readily accessible (that is, in RAM) the better it will perform. And as you mentiones, you can of course set it to use less RAM if you have to. It's just that it's optimized for performance by default.

    --
    Quality, performance, value; you get only two, and you don't always get to pick.
  7. Re:You only need 16GB of RAM for this to be useful by exley · · Score: 2, Informative

    Things like this (somewhat smaller scale) already are (somewhat bigger scale) being done.

  8. Re:1TB of RAM is available today in a server by thinduke · · Score: 2, Informative

    IBM p595 can have 1TB of RAM too. And yes, they run Linux.

  9. Re:1TB of RAM is available today in a server by thatskinnyguy · · Score: 2, Informative

    Something like this one?

    --
    The game.
  10. cachefs by argent · · Score: 2, Informative

    A fully caching file system that could be layered on top of your network or disk file system. Sun did this for dataless workstations and it worked pretty well.

    Another historically interesting ram file system was the Amiga Recoverable RAM Disk. You coudl even boot off it.

  11. Re:uh - there is at least one system with 1TB of R by Dammital · · Score: 2, Informative

    IBM's just-announced z10 mainframe, with 1.5 TB memory.

  12. Re:8 GB by Gr8Apes · · Score: 2, Informative

    I run Oracle on Linux - it barely fits into 4GB. Add in a few other daemons, and I can easily fill 8GB.

    --
    The cesspool just got a check and balance.
  13. Re:You only need 16GB of RAM for this to be useful by Daniel+Phillips · · Score: 2, Informative

    Yeah, imagine, then, to be able to use such a fast disk as your swap device! That'll make your system swiftz0rs. Bingo. That is one way you can use the Violin 1010 without needing any special backing to disk at all. In fact, this is a nigh-on perfect use of the device, because the 2x8x PCI-e bus connection, while fast, is still not as fast as main memory. But the swap subsystem knows now to manage that latency increase quite nicely. Such a swap arrangement will even tend to bring things back in balance as far as the Linux VM goes, since in the good old days when swap was invented, disk was only two or three orders of magnitude slower than memory instead of 5 orders like today.

    Or, hey, wait a minute... In all honesty, though, I don't really get the point of this. Isn't the buffer cache already supposed to be doing kind of the same thing, only with a less strict mapping? Indeed, less strict, and it also does not know how to flush dirty cache to disk and switch to synchronous mode when running on UPS power, or how to fully populate the cache as fast as possible on startup. Indeed, buffer cache can be taught these things, but I have already created a block driver to do it, which has the added benefit of supplying a nice general interface that will no doubt be repurposed in ways I did not think of. Maybe after Ramback is really solid I will create a variation that sits right in the VFS, though actually there are other more important projects in the pipe so anybody who wants to do that, be my guest.
    --
    Have you got your LWN subscription yet?
  14. Re:You only need 16GB of RAM for this to be useful by shapor · · Score: 3, Informative

    posix_fadvise() technically does allow you to do what you want. You can use posix_fadvise(POSIX_FADV_DONTNEED) to evict the buffer cache in the IO loop of the program. See http://insights.oetiker.ch/linux/fadvise.html for the ugly details. Unfortunately you can't just make one system call and have it effect an entire file or process. POSIX_FADV_NOREUSE is supposed to be the default in the kernel buffer cache management so it is implemented as a no-op.

  15. Re:1 TB of memory... by Anonymous Coward · · Score: 2, Informative

    It does indeed, the implementation of vi for emacs is called viper, there might be another one as well, but I'm not sure.