Slashdot Mirror

← Back to Stories (view on slashdot.org)

How To Use a Terabyte of RAM

Posted by kdawson on from the every-factor-of-25-helps dept.

Spuddly writes with links to Daniel Philips and his work on the Ramback patch, and an analysis of it by Jonathan Corbet up on LWN. The experimental new design for Linux's virtual memory system would turn a large amount of system RAM into a fast RAM disk with automatic sync to magnetic media. We haven't yet reached a point where systems, even high-end boxes, come with a terabyte of installed memory, but perhaps it's not too soon to start thinking about how to handle that much memory.

19 of 424 comments (clear)

  1. Add-Free one-page Version of the story by saibot834 · · Score: 4, Informative

    For those of you who don't have Adblock: Printerfriendly Version

    1. Re:Add-Free one-page Version of the story by Freedom+Bug · · Score: 4, Informative

      Here's a much better link on Jon Corbet's own site, the famous Linux Weekly News:

      http://lwn.net/Articles/272011/

    2. Re:Add-Free one-page Version of the story by Anonymous Coward · · Score: 1, Informative

      Or you could just read the story on LWN.

      Out of curiosity, why the link to PC World when the summary specifically mentions that it's on LWN? If Jon get paid for the PC World version because of the ad revenue, that's fine with me. However saying that it's an LWN piece and linking to another source is a bit misrepresentative.

  2. Power Failure by Anonymous Coward · · Score: 3, Informative

    One important thing to consider, is that if using a ramdisk for important stuff, what happens when the power dies?

    For example, will the stuff synced from magnetic media be stored elsewhere? If so, what happens to the speed?

    -B

    1. Re:Power Failure by itsjz · · Score: 5, Informative
      There's about three paragraphs in the article discussing this. Basically, use a UPS:

      If line power goes out while ramback is running, the UPS kicks in and a power management script switches the driver from writeback to writethrough mode. Ramback proceeds to save all remaining dirty data while forcing each new application write through to backing store immediately.
  3. uh - there is at least one system with 1TB of RAM by Anonymous Coward · · Score: 5, Informative

    You wrote: "We haven't yet reached a point where systems, even high-end boxes, come with a terabyte of installed memory" - this is not true. Sun's E25k can go over 1TB of memory.....

  4. nothing new here by dltaylor · · Score: 3, Informative

    Linux gobbles free RAM to add to the buffer cache. This is already a large RAM disk with automatic sync. In embedded systems, you can even decouple the buffer cache from any physical media and just live in a variable size RAM disk, which means that Linux finally catching up to AmigaDOS.

  5. Re:1 TB of memory... by Digi-John · · Score: 2, Informative

    emacs is a Lisp interpreter, an editor, a games package, an irc client, many things, but its memory usage is just a drop in the bucket compared to the monstrosities I mentioned above. Of course, there's quite a few complete operating systems that can boot in the amount of RAM required by emacs :)

    --
    Klingon programs don't timeshare, they battle for supremacy.
  6. Re:The problem with giving Windows 1TB... by W2k · · Score: 3, Informative

    If you run MS SQL Server and don't manage the RAM then it will use it all just for the fun of it.

    If you find this in any way strange, wrong or confusing, perhaps you should read up as to what the primary purpose of a frikkin' DATABASE SERVER is.

    Here's a hint: the more data it can keep readily accessible (that is, in RAM) the better it will perform. And as you mentiones, you can of course set it to use less RAM if you have to. It's just that it's optimized for performance by default.

    --
    Quality, performance, value; you get only two, and you don't always get to pick.
  7. Re:You only need 16GB of RAM for this to be useful by exley · · Score: 2, Informative

    Things like this (somewhat smaller scale) already are (somewhat bigger scale) being done.

  8. Re:1TB of RAM is available today in a server by thinduke · · Score: 2, Informative

    IBM p595 can have 1TB of RAM too. And yes, they run Linux.

  9. Re:1TB of RAM is available today in a server by thatskinnyguy · · Score: 2, Informative

    Something like this one?

    --
    The game.
  10. cachefs by argent · · Score: 2, Informative

    A fully caching file system that could be layered on top of your network or disk file system. Sun did this for dataless workstations and it worked pretty well.

    Another historically interesting ram file system was the Amiga Recoverable RAM Disk. You coudl even boot off it.

  11. Re:uh - there is at least one system with 1TB of R by Dammital · · Score: 2, Informative

    IBM's just-announced z10 mainframe, with 1.5 TB memory.

  12. Re:8 GB by Gr8Apes · · Score: 2, Informative

    I run Oracle on Linux - it barely fits into 4GB. Add in a few other daemons, and I can easily fill 8GB.

    --
    The cesspool just got a check and balance.
  13. Re:You only need 16GB of RAM for this to be useful by Daniel+Phillips · · Score: 2, Informative

    Yeah, imagine, then, to be able to use such a fast disk as your swap device! That'll make your system swiftz0rs. Bingo. That is one way you can use the Violin 1010 without needing any special backing to disk at all. In fact, this is a nigh-on perfect use of the device, because the 2x8x PCI-e bus connection, while fast, is still not as fast as main memory. But the swap subsystem knows now to manage that latency increase quite nicely. Such a swap arrangement will even tend to bring things back in balance as far as the Linux VM goes, since in the good old days when swap was invented, disk was only two or three orders of magnitude slower than memory instead of 5 orders like today.

    Or, hey, wait a minute... In all honesty, though, I don't really get the point of this. Isn't the buffer cache already supposed to be doing kind of the same thing, only with a less strict mapping? Indeed, less strict, and it also does not know how to flush dirty cache to disk and switch to synchronous mode when running on UPS power, or how to fully populate the cache as fast as possible on startup. Indeed, buffer cache can be taught these things, but I have already created a block driver to do it, which has the added benefit of supplying a nice general interface that will no doubt be repurposed in ways I did not think of. Maybe after Ramback is really solid I will create a variation that sits right in the VFS, though actually there are other more important projects in the pipe so anybody who wants to do that, be my guest.
    --
    Have you got your LWN subscription yet?
  14. Re:You only need 16GB of RAM for this to be useful by shapor · · Score: 3, Informative

    posix_fadvise() technically does allow you to do what you want. You can use posix_fadvise(POSIX_FADV_DONTNEED) to evict the buffer cache in the IO loop of the program. See http://insights.oetiker.ch/linux/fadvise.html for the ugly details. Unfortunately you can't just make one system call and have it effect an entire file or process. POSIX_FADV_NOREUSE is supposed to be the default in the kernel buffer cache management so it is implemented as a no-op.

  15. Re:1 TB of memory... by Anonymous Coward · · Score: 2, Informative

    It does indeed, the implementation of vi for emacs is called viper, there might be another one as well, but I'm not sure.

  16. cosmology simulations by Anonymous Coward · · Score: 1, Informative

    Speaking of voxels, octrees, and a terabyte (TiB that is; 2^40 bytes):

    Cosmology simulations take as much memory as you can throw at them. You think a few exabytes is big, but that's just peanuts to space!

    This one is one is the most complex (by most measures) cosmological simulation ever run, using just over 10 billion particles and nearly the entire terabyte of the computer's available memory:
    http://www.mpa-garching.mpg.de/galform/millennium/

    They ran it on a 512-CPU IBM p690, which took about 28 days of parallel runtime. (2^40) / (10^10) ~= 100 bytes per particle, which is high by maybe 50% but accounts for overhead and is a reasonable value. The paper reports ~200 Gflops sustained, which is certainly memory bound in any distributed N-body simulation, giving ~5*10^17 total floating-point ops.

    They only did total output at 64 output timesteps because each of those took about 300GB for ~20TB total, which filled up a decent sized storage rack 3+ years ago (and still). The contemporary highest-detail galaxy clusters were modeled with as many as 3 million particles, but the simplest interesting "structures" (the dark-matter halo detection threshold was 20 particles) contained 20~100 particles.

    Now for the cool stuff:
    - there are ~400 billion galaxies in the observable universe, even if galaxy surveys (Sloan, etc.) catalog only a small fraction of them
    - reasonable (i.e. "useful") resolution for typical structures of interest (quasars, typically) is ~1~10 million particles; better if you can get it

    So:
    You have to model dark matter halos with useful precision. Even if you are satisfied with modeling whole galaxies as point-like objects:
    (4*10^11 galaxies) * (100 bytes/particle) = 4*10^13 bytes ~= 40 TB
    (~10^12 CDM particles) * (100 bytes/particle) = ~100TB
    => allowing a little slush, 200TB of RAM would get you some very valuable theoretical simulation data of an object/construct/whatever that approaches literally the size and complexity of the observable universe at a galactic scale. More memory will just make that even better, too.

    Desktop memory doubles say, every 2 years; look at the typical "consumer" systems the last few years: ...
    1994: 16MB (P1... wow; 16 is a little high for '94 I guess)
    1996: 32MB
    1998: 64MB
    2000: 128MB
    2002: 256MB
    2004: 512MB
    2006: 1GB
    2008: 2GB

    We want to see when we have about 100,000 times as much memory as now, which is about 16.5 doublings, each of which take around 2 years, so we should see this some time around 2035-2040 on consumer grade machines. A purpose-built supercomputer can contain, say, 1000 times the consumer amount, so today's 4~50TB supercomputers need to double, drumroll.... 2~5.5 times. The top supercomputers will probably have 200 terabytes of ram within 2 years, and the lowly supercomputers cosmology simulations have to settle for will probably have that within 10-12 years; i.e. by 2020 (yeah, it's getting closer!).

    It may even be possible to do quad-precision floats in hardware, which would be beneficial for cosmological simulation (or any N-body simulation), and that will cost a doubling too. And after a petabyte or so, the resolution gets *really* interesting because you can start modeling macro-scale systems of micro-scale (you know, individual atoms or stars) as long as you know the force laws, and you can start finding those out numerically if you don't. So there's plenty of use for *truly* astronomical, dizzying amounts of memory way, way beyond one measly terabyte.