Slashdot Mirror
How To Use a Terabyte of RAM
Spuddly writes with links to Daniel Philips and his work on the Ramback patch, and an analysis of it by Jonathan Corbet up on LWN. The experimental new design for Linux's virtual memory system would turn a large amount of system RAM into a fast RAM disk with automatic sync to magnetic media. We haven't yet reached a point where systems, even high-end boxes, come with a terabyte of installed memory, but perhaps it's not too soon to start thinking about how to handle that much memory.
Finally, I'll have enough space to run Firefox, OpenOffice, and Eclipse *all at the same time*! As long as I don't leave Firefox running too long.
Klingon programs don't timeshare, they battle for supremacy.
Given that the core components of an OS are only a few GB, even 8GB systems might be able to do this, today.
GPL Deconstructed
For those of you who don't have Adblock: Printerfriendly Version
I would think that, since we aren't even close to having boxes with more memory than we actively use, and RAM isn't growing any faster than we are using it up, that using it as a "disk" is even further off than the article would seem to imply.
Whale
One Terabyte ought to be enough for anybody.
See also, Windows 7 minimum requirements.
Is it just me, or do you hate it when people say "Is it just me..."?
Is that the recommended or minimum requirement?
I have 8 GB of RAM and rarely use more than four of it unless I'm playing a 64-bit game which eats it up (Crysis). Yes, I am running both 64-bit Linux and Windows.
One time, I opened up more than a thousand tabs in Firefox just because I could.
Colin Dean Go a year without DRM
One important thing to consider, is that if using a ramdisk for important stuff, what happens when the power dies?
For example, will the stuff synced from magnetic media be stored elsewhere? If so, what happens to the speed?
-B
...I might be able to run Vista!!! (I wonder how many people have written this prior to me already?)
It's a lot of RAM and at today's computational speeds, it's not likely that it could be used for anything beyond a RAM drive.
Is it too soon to think about how to use that much RAM? NO! It's the lack for forward thinking that caused a lot of artificial limitations that have been worked around in the past. We're still dealing with limitations on file systems and the like. I've got an old Macintosh that can't access more than 128GB or something like that because its BIOS can't handle it... I had to get another PCI controller installed to handle larger drives.
What it is time to think about is now to code without such limitations built-in. This would better enable things to grow more easily and naturally.
The System Tray would end up filling most of my dual monitors with all the crap Microsoft will inevitably find "necessary" to run the OS, leaving me with a small, 640x480 patch and approximately 640k for applications.
You wrote: "We haven't yet reached a point where systems, even high-end boxes, come with a terabyte of installed memory" - this is not true. Sun's E25k can go over 1TB of memory.....
char *ptr=malloc(1099511627776);
memset(ptr,1,1099511627776);
Linux gobbles free RAM to add to the buffer cache. This is already a large RAM disk with automatic sync. In embedded systems, you can even decouple the buffer cache from any physical media and just live in a variable size RAM disk, which means that Linux finally catching up to AmigaDOS.
I'm using regular ramdisks initalized with data on bootup, composited with temporary, empty disk partitions using unionfs and synchronized back to their real partitions on powerdown, so that I have an extremely fast read time for most things contained on such a disk and conventional write-reread times. However, the problem is that for the upper layers of the kernel, those ramdisks are not RAM at all, just some other block device around - and when it comes to loading executables and libraries, they are copied, well, from memory to memory. What's missing is some way to tell the damn thing to use the data pages that already are there and issue a copy-on-write only when required. If this mechanism can do that - well, I'll be in as soon as they make it a little bit more fault-tolerant.
This is Slashdot. Common sense is futile. You will be modded down.
Current high-end server boards support up to 64GB of RAM (16 slots, 4GB DIMMs).
By Moore's Law, we should hit 1TB in a high-end server 6 years, high-end desktops (assume 8GB of RAM, currently selling for $180 CAD) in 10.5 years, and the average midrange desktop (assume 2GB of RAM, currently selling for $45 CAD) in 13.5 years.
We might be a while off in consumer applications, but for high-end servers, 6 years doesn't seem very far away.
IBM p595 can have 1TB of RAM too. And yes, they run Linux.
Something like this one?
The game.
Well I can do cock push-ups.
Check out the specs on the Motorola (formerly BroadBus) B-1 Video Server:
http://www.motorola.com/content.jsp?globalObjectId=7727-10991-10997
Sounds like a good use for a terabyte of RAM to me.
Disclosure: I currently work for Motorola, but I don't speak for them, and don't have any involvement with this product beyond salivating over it when it was announced that we were buying BroadBus.
Ten years ago, my PC had 8 megs of system RAM. My laptop now has four gigs of RAM. In ten more years, I am sure we'll have a terabyte of RAM.
A NYC lawyer blogs. http://www.chuangblog.com/
If you are planning on having a few minutes' worth of UPS backup then why would you need to write to the hard drive continuously? Keep the hard drive spun down (saving power). If the system is being shut down, or AC power fails, then spin up the drive and make a backup of your ramdisk, thus being ready to restore when the power comes back up.
Next step beyond that: stop using a filesystem at runtime. Just assume your data can all fit in memory (why not, if you have a terabyte of it?) This simplifies the code and prevents a lot of duplication (why copy from RAM to RAM, just to make the distinction that one part of RAM is a filesystem and another part is the working copy?) But you will need a simple way to serialize the data to disk in case of power-down, and a simple way to restore it. This does not need to be a multi-threaded, online operation: when the system is going down you can cease all operations and just concentrate on doing the archival.
This assumption changes software design pretty fundamentally. Relational databases for example have historically been all about leaving the data on the disk and yet still fetching query results efficiently, with as little RAM as necessary.
Next step beyond that: system RAM will become non-volatile, and the disk can go away. The serialization code is now used only for making backups across the network.
Now think about how that could obsolete the old Unix paradigm that everything is a file.
Geez, I wrote a floppy disk cache driver as a programming homework exercise in the 1980s. Talk of re-inventing the wheel...
Excuse me, but please get off my Pennisetum Clandestinum, eh!
A fully caching file system that could be layered on top of your network or disk file system. Sun did this for dataless workstations and it worked pretty well.
Another historically interesting ram file system was the Amiga Recoverable RAM Disk. You coudl even boot off it.
Use a large geographically distributed cluster with the ability to pxe boot off each other.
Then a power outage wouldn't be an issue. Power comes up, machine PXE boots off a machine in a neighboring town, state, country, whatever.
I know--not really feasible, but you'd be the king of basement dwellers if you could pull it off...
There's no place like
Still think the floating point voxel octree version of Google Earth will use that memory before any ramdisk gets it.
IBM's just-announced z10 mainframe, with 1.5 TB memory.
How it seems to work:
Actual "ramdisk" -- that is, like /dev/rd -- that is, appears as a block device. You can run whatever filesystem you want on it, but it's still serializing and writing out to... well, RAM, in this case. No sane way for the kernel to free space on that "disk" that's not actually used.
How I wish it worked:
No Linux that I know of has used an actual ramdisk in forever. Instead, we use tmpfs -- a filesystem which actually grows or shrinks to our needs, up to an optional configurable maximum size. It'll use swap if available/needed. It's basically a RAM filesystem, instead of a RAM disk.
Even initrds are dead now -- we use initramfs. Basically, instead of the kernel booting and reading a ramdisk image directly to /dev/rd0, it instead boots and unpacks a cpio archive (like a tarball, but different/better/worse) into a tmpfs filesystem, and uses that.
So, how I would like this to work is, use a tmpfs filesystem -- as I suspect it will be faster, and in any case simpler, than a ramdisk -- and back it to a real filesystem on-disk. The only challenge here is that it's not as deterministic -- it would be more like a cp than a dd.
An even better (crazier) idea:
Use a filesystem like XFS or Reiser4 -- something which delays allocation until a flush. In either case, it would take a bit of tweaking -- you want to make sure no writes, or fsyncs, block while writing to disk, so long as the power is on -- but you'll hopefully already be caching an obscene amount anyway, so reads will be fast.
In this case, forcing everything out to disk could be as simple as "mount / -o remount,sync" -- or something similar -- forcing an immediate sync, and all future writes to be synchronous.
Conclusion:
Either of the two ideas I suggested should work, and could perform better than a traditional ramdisk. If it is, in fact, a simple disk-backed ramdisk (not ram filesystem), then it's both not as flexible (what if your app suddenly wants 50 gigs of RAM in application space?) and a bit of a hack -- probably a hack around traditional disk-backed filesystems not being able to take advantage of so much RAM by themselves.
In fact, glancing back at TFA, it seems there are some inherent reliability concerns, too:
Now, true, this should never happen, but in the event it does, the inherent problem here is that the ramdisk doesn't know anything about the filesystem, and so it doesn't know in what order it should be writing stuff to disk. Ext3 journaling makes NO sense for a ramdisk when the ramdisk itself knows nothing about the journal -- the journal is just going to slow down the RAM-based operation. Compare this to a sync call to XFS -- individual files might be corrupted, but all the writes will be journaled in some way, so at least the filesystem structure will be intact.
This gets even better with something like Reiser4's (vaporware) transaction API. If the application can define a transaction at the filesystem level, then this consistent-dump-to-disk will happen at the application level, too. Which means that while it would certainly suck to have a UPS fail, it wouldn't be much worse than the same happening to a non-ramdisk device, at least as far as consistency goes. (Some data will be lost, no way around that, but at least this way, some data will be fine.)
Don't thank God, thank a doctor!
You better skip the memory test.
now we need to go OSS in diesel cars
Thats the only reason i can see to have that much ram. Unless our current crop of so called programers bloat their code to fill the expansion for yet another worthless feature.
---- Booth was a patriot ----
Well, closer to 1.2 TB. 40 systems with 32Gb each. Want to know what it's used for? Disk cache... It's virtually all I/O buffer.
All RAM is used as cache anyway. When an application allocates some RAM, it's in lieu of directly manipulating the permanent (disk) storage because it's horribly horribly slow. That's really an operating system failure. Network file systems, disk, RAM should all be completely transparent, the OS should abstract all that away and allow application programmers to handle it simply as storage.
Deleted
Chip design apps (and I imagine a number of other ones) will likely need 1TB in a year or so. I already know of several companies using boxes with 64G of RAM and the apps are consuming like around 40-50 of it. Designing (& analyzing) those multi-billion transistor designs eat memory. My sw package was designed to allow for ~80G per cell in the hierarchy. Since my system allows 128K cells, thats about 10TB of RAM that could be used. I have even wondered if the 80G limit needs to be increased in the near future.
I really don't like the idea that this is racing with the UPS. When the battery gets old it's ability to hold a charge drops and a timeout that was sufficient in the old days won't be now. I've also had situations where the battery was supposed to tide you over till the generator kicked in; but the system was never tested for the generator failing at exactly the wrong moment.
I'm pretty sure the answer to this is a simple generation number on the blocks so you can use a database checkpoint scheme.
1) Every write to the ramdisk brands that block with an ever increasing number (transaction number).
2) When you initiate a checkpoint the driver finds all blocks that have changed since the last checkpoint and writes them to a "physical log", followed by the checkpoint marker.
3) The same blocks are then written to the actual disk area; nb application writes to these blocks must be diverted.
4) The "physical log" is cleared.
5) Block diversions from (3) are cleared.
Using this well known scheme the disk is always either in a consistent state or easy to get there.
Note the "diversions" may mean that clean blocks must be discarded from the ramdisk/cache to prevent the applications being blocked by the checkpoint.
If you want the ability to have the system 'roll forward' after a crash you need a transaction log where the updates are written to the log as they happen, because this is linear it happens at the maximum transfer rate of the disk; but it's still limited in performance.
This also looks a lot like doing a backup from a volume snapshot.
The question should be not WHAT to fill it with, but how to read/write gargantuan amounts of data quickly.
Upward mobility is a slippery slope - the higher you climb the more you show your ass.