Other Uses For The Linux RAM Disk?

Dante_J asks: "Recently I discovered an old Amiga DOS 1.3 Manual I had lying around. While thumbing through it I remembered all the joyful days of good fun hacking. One thing I particularly remembered was how anyone with 3Mb of ram was considered especially blessed with resources because they could copy all their system files into the ram disk and have a 'trans-warp' fast machine on their hands. In this age of more Ram than sense why are Ram Disks only used for Linux installation floppies? Sure buffers are great, but why not mount /tmp to a Ram Disk, and the cache directory for Web browsers too? Does Linux support dynamically reseizing Ram Disks? Surely they would be vital in remote booting, diskless thin clients."

Comments

Dynamic Sizing of Ramdisks

[Jason Straight]

The 2.4 kernel does support dynamic ramdisks.

Solaris already does

[larien]

At least, the /tmp bit. Under Solaris, /tmp is effectively part of virtual memory (used to be just swap space, now includes physical RAM). Causes several problems as:

• files disappearing from /tmp on reboot which users didn't expect
• large files eating up swap space

Personally, I find it a great idea; just some admins don't like it as much.
--

ramdisk vs. hard disk

[VoidOfReality]

Where I work, we use a bunch of Linux boxes to serve our website. Currently, all of our content is located in ramdisk, as well as a data cache used by the web applications that we run. I'm currently on a project to evaluate the merits of using hard disk for this as opposed to the ramdisk that we're currently using.

The results of the performance test that I ran were somewhat surprising - it seems the machine with the hard disk actually performed _better_ than the machine with the ramdisk. I'm not a kernel hacker so I don't know exactly why this is the case, but I know that the buffer caching in the kernel really kicks ass (we're running 2.2.10) and I suspect having a ramdisk hampers the kernel's ability to manage the buffer cache (i.e., it takes up space that could be used for buffer cache). Just my $.02...

-VoR

Re:ramdisk vs. hard disk

[Mr Z]

The results of the performance test that I ran were somewhat surprising - it seems the machine with the hard disk actually performed _better_ than the machine with the ramdisk. [...] and I suspect having a ramdisk hampers the kernel's ability to manage the buffer cache.

You're partly right. The other reason is that it forces more pages out to swap, since the RAM disk can't be paged out (I'm pretty sure). Placing your data in a traditional Linux RAM disk has two bad effects:

• It reduces the total amount of RAM available to applications. This results in more paging activity.
• It reduces the size of the buffer cache, meaning that files outside the set you've placed in RAM are more likely not cached. This also includes filesystem metadata, such as block bitmaps and the like.

Even if the Linux ramdisk can be swapped out (I think the new ramfs may be capable of this), it will still likely be slower than a traditional filesystem if you push into swap, because swap gets fragmented over time. In contrast, ext2 resists fragmentation pretty well and so will perform better as a result.

My 0x02 cents...

--Joe
--

Re:ramdisk vs. hard disk

[Arker]

The results of the performance test that I ran were somewhat surprising - it seems the machine with the hard disk actually performed _better_ than the machine with the ramdisk.

This is exactly what I would suspect - I'm glad you posted this because I don't have time today to test it myself, but this is the result I would bet on.

The reason ramdisks aren't very useful with Linux is that the kernel has very good buffer/caching code - the effect is the same as having a ramdisk, except that the kernel can dynamically determine the contents based on actual usage. If you stick commonly used data on the ramdisk, you should be able to beat any caching algorithm, in theory, but this requires that there is certain data that you know will always be the most frequently accessed. In the real world, this rarely works out.

Say you put 16 megs of what you think is the most commonly used data on a ramdisk. Say further that you are right - over time, that 16 megs of data is the most frequently accessed data in your system, by far. If your box is hitting that data constantly, every bit of it, every few cycles, you might get a small performance boost. But more than likely some parts of it will not be hit all that frequently, and there are also likely to be times when it's not being hit at all. Put the same data on hard disk and let the kernel have the ram to manage, and it will manage things on the fly, responding to the actual demands of the system... it's very hard to beat.

Now, if the caching algorithm was more primitive, something like smartdrv for instance, then you can get a big performance boost out of the ramdisks. I used to play that game quite frequently. But this only worked because smartdrv really isn't very smart.

Re:ramdisk vs. hard disk

[WNight]

Yup. Having a ramdisk limits the ammount of ram available for caching and caching caches the files actually used, not just the one you specify.

There's one time ramdisks are good... If you have a small set of files (relative to your total ram) that you don't use very often, but when you do, you want them to load with as little delay as possible...

Not really something you'll run into with a webserver where a 10ms lag in HD access will be hidden under at least 50ms of network lag and probably 250ms of rendering lag.

But, if you run a machine who is doing realtime data sampling and you need to run various transforms on the data, you might want to keep some of the essential tools in ram. (Just a contrived example...)

The best way to do this though would be to ask the caching subsystem to keep some, higher priority (specified by you, and by overall usage patterns) in ram even when they might have been dumped, because they're either timing critical, or likely to be used a lot in the future. (The way an ASM programmer could hint to the CPU which branch will be taken by making a seldom taken loop fall-through when skipped and a often-taken loop fall-into when taken.)

The OS in ROM

[Consul]

There's one other option, as well. Why not place the entire operating system on an EEPROM? Large-sized EEPROMs are getting pretty cheap these days.

Using an EEPROM would allow you to upgrade/patch the OS as necessary. Also, some clever engineering would make it all but immune to viruses (putting the OS in a true ROM would do wonders for virus protection, but make it difficult to upgrade you system software).

Hell, you could put Linux and X-Windows with the Window manager of your choice all on an EEPROM and have a superfast, instant booting machine.

I'm sure this is being done somewhere. Any ideas or links anyone cares to share?

Re:The OS in ROM

[hey!]

I'm not sure that your boot up time would be all that much faster. After all, most things that boot from ROM boot fast because they are pretty simple and small. Perhaps better to put /etc/rc.d on diet.

Read only media are a good idea though.

For security, if super fast boot time is not an issue, then you might consider booting from CD-ROM or othe read only medium on some machines. If your home page and apache configuration files are on CD-ROM, then your home page cannot be defaced no matter how clever the cracker. Likewise even if somebody does manage to use a root kit on your box they can't replace one of the regular utilities with a trojan if the directory it's in resides on a CD-ROM.

Anybody know a Linux how-to doing this? I've seen it done with the BSDs.

I am just waiting for...

[Anonymous Coward]

...I am just waiting for the inevitable suggestion that the ramdisk be used for "swap" :)

no benefits

[austad]

Just because it's in Ram doesn't mean it's faster. You're still putting a filesystem on it and you have the overhead of the filesystem, it still uses flock(), and if your box dies you lose your data on it.

I looked into this awhile back. Go ahead and make a 200MB ramdisk on a machine with lots of ram. Make an ext2 fs on it and run some benchmarks on it. Then run the benchmarks on your disk. In most tests, my crappy IDE disk outperformed the ramdisk (1GB of mem in the box, so I wasn't swapping). CPU was much higher when testing the ramdisk, at 100%. I think it's because there was no DMA controller being used when using the ramdisk.

Unix/Linux does this for you automatically

[IGnatius T Foobar]

Unix/Linux does this for you automatically. The disk caching functionality will keep the disk blocks belonging to recently used programs in memory -- so if you have a lot of memory, you'll simply find that once you've typed a few commands, the machine doesn't have to go to disk to fetch them on subsequent runs.

This actually reflects the perfect way of doing this: add optimization, but don't bug the users about it -- it's not their problem.
--

Re:Good idea, but...

[Grab]

This is exactly why we don't use RAM-disks on modern machines.

The stock A500 had 1/2Meg of RAM, so most stuff was designed to run in that memory space. Most word-pros and spreadsheets would run in this, but didn't have much room spare for file data, so more serious users got a 1/2Meg RAM upgrade for this (and even now, you can store a lot of text in a 1/2Meg file). If you had the money for a 2Meg (or more) expansion card, the world was your oyster. You could then run 2 or more heavy-duty programs simultaneously, and use any space left over to cache your frequently-used commands in a RAM-disk. Well cool at the time.

Now back to today. It's no longer strange to run several heavy-duty applications together - at any one time in Windows (sorry, but that's what I use at work), I may have Word, Excel, Access, DevStudio, Outlook, Matlab, Acrobat and IE all running together. At this point, the Amiga would have reached the "heavy heavy heavy, man" stage and died with a Guru Meditation error. We have vast stacks of RAM now, but our expectations have risen too, and so have the program sizes. You could still sit down and code a graphics app in Intel assembler if you really wanted to (as one Amiga developer did to get fastest performance and minimum code size), but I wouldn't recommend it.

Also, the purpose of a RAM-disk has pretty much vanished. When we used floppies, the disk access time was enormous and slowed things down considerably, but modern hard drives are so fast that disk access time isn't as big a deal as it was then. Even then, if you had a HDD (20Megs was state-of-the-art then!) then you didn't really need to use a RAM-disk.

Grab.

Linux and unused memory

[PureFiction]

In Linux, all unused memory is used for filesystem caching. In general, linux does this caching mcuh better than you could by mounting specific disks and files in a RAMDISK. Linux chooses things which are accessed frequently, or very recently, among other things, into this FS chache.

By creating a RAM disk to do this, you would force a much smaller subset into memory, which would be great for what you are using there, but would hinder performance on other things which linux does not have the room to cache now.

So, unless there is something very specific that needs to be cached, there is no rationale for this, and the chances are that linux will cache it for you anyway if its that much of a performance hit.

Last but not least, the biggest reason RAM disks are slightly faster than average (when linux does cache) is because they never have to synch to a physical medium. If you dont care that all the data you have written there is gone *poof* once a crash occurs, or if the system is shutdown, then thats ok. If you have a file there, and 'oops' forgot to write it to disk, its gone.