Slashdot Mirror
Other Uses For The Linux RAM Disk?
Dante_J asks: "Recently I discovered an old Amiga DOS 1.3 Manual I had lying around. While thumbing through it I remembered all the joyful days of good fun hacking. One thing I particularly remembered was how anyone with 3Mb of ram was considered especially blessed with resources because they could copy all their system files into the ram disk and have a 'trans-warp' fast machine on their hands. In this age of more Ram than sense why are Ram Disks only used for Linux installation floppies? Sure buffers are great, but why not mount /tmp to a Ram Disk, and the cache directory for Web browsers too?
Does Linux support dynamically reseizing Ram Disks? Surely they would be vital in remote booting, diskless thin clients."
The problem with allocating that much ram to just hold cache for a web browser or similar program is that they're designed to expect that disk cache to be very VERY slow.
So you'd be better off just using that memory to allow the OS to buffer disk accesses and your programs to do their own in-memory caching than to have it act as a ramdisk.
Doug
Venn ist das nurnstuck git und Slotermeyer? Ya! Beigerhund das oder die Flipperwaldt gersput!
The 2.4 kernel does support dynamic ramdisks.
That being said, in some respects, its a denial of service attack waiting to happen, though probably no more than a malloc() loop...
Using a ram-disk to cache web-content seems a bit strange.. Most browsers I have seen already have options for setting how much ram you want to use for cache, and all you would have to do is increase it. When it comes to /tmp, if you have enough ram, Linux will just write-behind cache the file until its use is over.
I was under the impression that disk caching renders ram disks obsolete except for the cases where you want to be absolutely certain that the data stays in memory (such as when your files are on removable media).
- files disappearing from
/tmp on reboot which users didn't expect
- large files eating up swap space
Personally, I find it a great idea; just some admins don't like it as much.--
Where I work, we use a bunch of Linux boxes to serve our website. Currently, all of our content is located in ramdisk, as well as a data cache used by the web applications that we run. I'm currently on a project to evaluate the merits of using hard disk for this as opposed to the ramdisk that we're currently using.
The results of the performance test that I ran were somewhat surprising - it seems the machine with the hard disk actually performed _better_ than the machine with the ramdisk. I'm not a kernel hacker so I don't know exactly why this is the case, but I know that the buffer caching in the kernel really kicks ass (we're running 2.2.10) and I suspect having a ramdisk hampers the kernel's ability to manage the buffer cache (i.e., it takes up space that could be used for buffer cache). Just my $.02...
-VoR
Using an EEPROM would allow you to upgrade/patch the OS as necessary. Also, some clever engineering would make it all but immune to viruses (putting the OS in a true ROM would do wonders for virus protection, but make it difficult to upgrade you system software).
Hell, you could put Linux and X-Windows with the Window manager of your choice all on an EEPROM and have a superfast, instant booting machine.
I'm sure this is being done somewhere. Any ideas or links anyone cares to share?
-----
"You spilled my egg... I needed that egg."
There's no reason to put the netscape cache in swap. There's already a setting for memory cache as well as disk cache. Personally, Netscape takes up enough memory already. Besides, do you really want 20-30M of RAM wasted when Netscape isn't even running?
You might have an otherwise busy computer
/tmp on solaris
that is serving thousands of httpd requests
from harddisk and your filesystem cache is
flushing itself over and over again, and you
still want netscapes cachefiles to be in a
speedy environment?
Of course, having more RAM than disk is a fine
solution when the OS buffers with your free mem,
but obviously everyone can't have that.
Having a dynamic ramdisk (like
and the aforementioned amiga-ramdisks) is quite
a big win speedwise for many kinds of situations.
As long as you have free mem, tempfilecreation
will go nearly as fast as ram allows, and whenever
it gets full, you get swapspeed, which is more or
less what you would have gotten in the first place
with the cache on local disk anyhow.
-- I'm as unique as everyone else.
- - - - - - - -
"Never apply a Star Trek solution to a Babylon 5 problem."
"It is a greater offense to steal men's labor, than their clothes"
...I am just waiting for the inevitable suggestion that the ramdisk be used for "swap" :)
Just because it's in Ram doesn't mean it's faster. You're still putting a filesystem on it and you have the overhead of the filesystem, it still uses flock(), and if your box dies you lose your data on it.
I looked into this awhile back. Go ahead and make a 200MB ramdisk on a machine with lots of ram. Make an ext2 fs on it and run some benchmarks on it. Then run the benchmarks on your disk. In most tests, my crappy IDE disk outperformed the ramdisk (1GB of mem in the box, so I wasn't swapping). CPU was much higher when testing the ramdisk, at 100%. I think it's because there was no DMA controller being used when using the ramdisk.
Need Free Juniper/NetScreen Support? JuniperForum
I an effort to speed up some calculations I was doing I directed the I/O to/from a ramdisk, rather than the usual HDD. I figured, RAM is much faster than HDD, so I will remove any I/O blocking time. (This particular application was very I/O intensive. Basically, two programs communicating via files. I.e., program A runs a calculation, program B reads the output, generates new input for A, and so on. Very messy...)
Anyway... I found that there was NO speed up. None. My conclusion, the Linux file system caching whatever (I am a chemist, not a kernel hacker), was extremely good. I imagine that these smallish files were rarely ever actually written and/or read to disk. They were just stored in some cache. Essentially the normal use case and the ramdisk case were basically identical in that all of the I/O was in RAM anyway.
Now, these files were small, and updated frequently, and I have tonnes of RAM (0.5 GB), and what not (2xPIII450, SCSI disk,...) so YMMV... just my experience with ramdisks, for what it is worth.
linux 2.4's ramfs dynamically resizes. The traditional method of creating an ext2fs on a ramdisk does not.
__
In all honesty, the kernel knows much better than you do which files it's always accessing, so it can optimize itself better than you.
Type in "free" and you'll see that almost all your RAM is in use -- that's because it's got a RAM buffer of most recently accessed files so they can be accessed again faster. In fact, if you create a temporary file and then delete it, often that file will never touch the disk.
This, of course, is why you have to unmount disks - the unmounting writes the buffers to the disk so that the changes won't be lost.
So, it's already done for you, assuming you want a RAMdisk of your most frequently accessed files.
Sometimes, people want a rarely used file to be easily accessed to reduce load times, which is something that buffering won't help with. So, you just flip the sticky bit on the file, and it's done for you.
By making a RAMdisk, you're taking away from the available RAM that the kernel could be using for intelligent buffering, and actually slowing down the machine.
It's a fairly simple and straightforward sysadmin exercise to make a script (well, one line, really) to delete all files in /tmp that haven't been accessed in say 3 days or so. (It involves reading the manpage on 'find'.)
/tmp at one point, just to ensure that students wouldn't fill up /tmp and bring down the machine.
Heck, at cc.purdue.edu they put quotas on
---
At least mafia-owned pizzarias make excellent pizza. Compare to Bill Gates.
One of the Fundamental differences between all Unixes and every other OS ever invented is the use of memory to buffer the filesystem. There are no direct writes to a disk, ever! All file IO is done to the buffers in memory, and then eventually the bdflush daemon runs and syncs the disk to the image in memory. Notwithstanding the recent journaling file-systems, and the sync-write IO calls, Unix today still does all its file-IO in this manner...which is why a RAMdisk is redundant. You already HAVE a RAMdisk.
Mac, Windows, VMS, MVS, Amiga, et.al all do direct and/or syncronous writes to the disks. That's why a RAMdisk has such an effect.
Linux boot floopies use a RAMdisk because they can't put all the needed files onto a 1.44MB floppy without compressing the image. The RAMdisk is simply the "disk" to which the decompression runs its output. If the root filesystem could fit entirely onto a floopy, there'd be no need for a RAMdisk upon install. See Redhat verion 3.x
Maybe someone should do a dist that's optimized to use a 100 megabyte (or so) RAMdisk to speed up the loading of the most used and slowest loading apps. Netscape would probably qualify. All the gnome stuff... star office... The field's wide open.
Of course, that space might be better used as disk buffers...
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
For remote-booting thin clients I would recommend the network block device. It allows you to mount any filesystem on a TCP connection, although you can't use it for swap space (at least not in 2.2-series kernels).
Besides, on NT, the reason having the GDI subsystem in kernelspace sucks is because the drivers tend to be buggy. The http-kernald only serves up text pages; anything more hefty it tries to send to the actual httpd. So although it IS a security and stability risk, it's a lot easier to debug (all a httpd does, at it's basic level, is parse a text request, read a file from disk, and print it out to a socket.)
Vintage computer games and RPG books available. Email me if you're interested.
Unix/Linux does this for you automatically. The disk caching functionality will keep the disk blocks belonging to recently used programs in memory -- so if you have a lot of memory, you'll simply find that once you've typed a few commands, the machine doesn't have to go to disk to fetch them on subsequent runs.
This actually reflects the perfect way of doing this: add optimization, but don't bug the users about it -- it's not their problem.
--
Tired of FB/Google censorship? Visit UNCENSORED!
Actually, every modern OS has buffering options. But you really do need to have the option to have syncronous I/O. Databases, for example. Why do you think Oracle prefers to have raw disks? Filesystems get in the way, and async i/o is a real bummer for data integrity. :-)
Vintage computer games and RPG books available. Email me if you're interested.
The last few versions of rxvt have "transparency" support. You may have to compile your own version if your distro doesn't compile it with that option...
"Free your mind and your ass will follow"
No faster, I've tried it.. Compiling is processor-bound, even on a quad Xeon 450. Shaved less than a minute off of 'make World, which normally takes just over 48 minutes. I think the Linux ramdisk has processor usage issues, because a ramdisk should in theory be faster than the UW/2 SCSI drive I normally compile on..
.sig: Now legally binding!
When the third drive died on my powerbook 180, I created a seven floppy boot-set. I had the thing maxed out at 14mb, so I could install system 7.1, word 4 (I'd been using 5, but backtracked), and excel 3, along with my usual utilities, and still have enough room left to work . . .
Of course, given that batteries were only good for 1:40, and that some genious diddn't include a capacitor to back up ram while changing . . .
hawk, who still has the pieces of that machine
The same is probably true, today. Simply because few people will be able to test and refine the code on extreme memory machines, RAM Disks will probably still be the way to go.
There is -one- other case for RAM Disks that I can think of. =VERY= Extreme RAM cases. Where the size of RAM is comparable to, or exceeds, the size of the HD(s), it is not efficient to keep swapping to and from drives. They're slow. It's much faster to simply dump the drive(s) to RAM and write-through to disk a you go. All actual read operations, after boot, would be to RAM.
Beyond that, fast IDE's or SCSI's, with decent on-board cache, are all-round a better idea than RAM disks.
Going back to the extreme memory case for a moment, this would be ideal for a laptop. Non-volatile RAM is going to eat batteries far less than a mechanical drive. (Especially on power-up, or where there is extensive disk activity.)
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Oh boy! Virtual Memory! I'm going to make a HUGE ramdisk!
--
My word processor was written by Stanford Professor Donald Knuth. Who wrote yours?
A machine with 512k that is a personal work station coul dhave 256k for memory, 128 for /tmp and 128 for swap. This could add in that necessary performance increas I need for video.
I don't want a lot, I just want it all!
Flame away, I have a hose!
Only 'flamers' flame!
The only real place I can imagine where this might help is one where there is a lot of disk writing going on on a particular filesystem, and the system is so busy it never has a chance to flush it's cache without causing incoming requests to wait, such as a way overloaded mail server (I've seen this). Of course, the problem there is that it's not a night and day performance difference anyway when you're talking about modern disks with built in caches and OS's with decent lazy write techniques. And of course the little tiny issue that you're completely compromising your data integrity since if the system dies, everything in that file system goes with it. A far better solution, if you feel you must second guess the OS's caching/paging/vm system, is a controller card with on board cache and a battery backup. This maintains your data integrity, and will give you a little performance boost for busy disks with lots of writing. Of course, you'd still probably see just as much performance improvement by adding memory...
Sorry, no. You'll have to run Windows 98SE for that pleasure. Linux is unnecessarily stable for such a task.
Mr. Ska
All of the data stored on your disk is already proactively cached, most recently used is much more likely to be in cache longer.
Full blown RAM disks have limited applications, especially when the buffer cache can almost always do a better job. If you have gobs of RAM you're not using, increase the size!
This is exactly why we don't use RAM-disks on modern machines.
The stock A500 had 1/2Meg of RAM, so most stuff was designed to run in that memory space. Most word-pros and spreadsheets would run in this, but didn't have much room spare for file data, so more serious users got a 1/2Meg RAM upgrade for this (and even now, you can store a lot of text in a 1/2Meg file). If you had the money for a 2Meg (or more) expansion card, the world was your oyster. You could then run 2 or more heavy-duty programs simultaneously, and use any space left over to cache your frequently-used commands in a RAM-disk. Well cool at the time.
Now back to today. It's no longer strange to run several heavy-duty applications together - at any one time in Windows (sorry, but that's what I use at work), I may have Word, Excel, Access, DevStudio, Outlook, Matlab, Acrobat and IE all running together. At this point, the Amiga would have reached the "heavy heavy heavy, man" stage and died with a Guru Meditation error. We have vast stacks of RAM now, but our expectations have risen too, and so have the program sizes. You could still sit down and code a graphics app in Intel assembler if you really wanted to (as one Amiga developer did to get fastest performance and minimum code size), but I wouldn't recommend it.
Also, the purpose of a RAM-disk has pretty much vanished. When we used floppies, the disk access time was enormous and slowed things down considerably, but modern hard drives are so fast that disk access time isn't as big a deal as it was then. Even then, if you had a HDD (20Megs was state-of-the-art then!) then you didn't really need to use a RAM-disk.
Grab.
What about hardware RAM disks?
I've seen some (don't remember where) that were designed to be used like a normal disk but on the insides they were a bunch of battery-backed RAM. I believe the high-end models even had the ability to automagically sync to a physical disk in the enclosure on power loss/off/shutdown and restore to RAM.
I think the advantage of these solutions over a conventional host-based RAM disk was that by treating the RAM disk as a SCSI device you could make it much faster than conventional host RAM (special controllers, interleaving, etc).
I don't remember the name of the company that was selling this and they don't appear to be around anymore. Maybe the cost of RAM relative to the sizes people needed just made it commercially impractical.
Linux installation from floppy uses the RAMdisk to store the installation filesystem. This is not only quicker than running from a floppy, but allows the RAMdisk image to be compressed. Debian and Slackware do this, and I presume other do as well.
When I've used RAMdisks in the past on other systens, it has always been when other media was slow. A common one was under DOS on a floppy only 8086, copying COMMAND.COM and to a small (30K) RAM disk (stored in spare RAM on the non-standard video adaptor IIRC), and setting COMSPEC accordingly. Saved me having to swapp floppies just to load COMMAND.COM on program exit.
The only advantage I can see on a non-embedded Linux is if you have a some data or executable that you need to guarantee is in cache, and pre-load this into RAM disk before-hand. Faking benchmarks springs to mind here.
My parents have a Performa 6116; it has a PPC 601 @66 mhz and a 33 Mhz bus. Running Netscape 3 is quite a task for the poor machine (Netscape 4 is right out :).
It was even more of a task before I did just what you described and made a RAM Disk and told Netscape to use it. Sped it up by about 3 times.
Libertarianism is rich wolves and poor sheep playing gambler's ruin for dinner.
Yup. I did that too. A500, Workbench 2.1, 20 meg SCSI hard disk.
Even though the 880k of RAM could have been better used for program space, I found that the performance penalty of having the system files on a slow band-stepper hard drive was such that the memory hit was less obtrusive.
And, I had a later-revision A500, into which I'd plopped a Fat Agnes chip. With a little hacking, of course, it was possible to set up the memory expansion on the bottom of the A500 to become an extra 512k of Chip RAM; the 2 megs in my hard disk controller setup was the Fast RAM.
The only reason I bring this up is that, if nothing else, the Amiga is a great source of inside jokes, like "Guru Meditation Errors" and "volatile memory"...Or, if you've been downloading too much off the local high-speed (2400 baud!) Warez BBS, "Your Amiga is alive..."
Ya know, a Mac says, "Sorry, a System Error has occurred"; Windows tells you that "This program has performed an illegal operation" or any number of other nasty things.
But, besides Eudora 3.x ("Eudora is tired of waiting for the system to respond."), can anyone else think of any really nasty or sarcastic error messages like the Guru Meditation Error?
Fire and Meat. Yummy.
In Linux, all unused memory is used for filesystem caching. In general, linux does this caching mcuh better than you could by mounting specific disks and files in a RAMDISK. Linux chooses things which are accessed frequently, or very recently, among other things, into this FS chache.
By creating a RAM disk to do this, you would force a much smaller subset into memory, which would be great for what you are using there, but would hinder performance on other things which linux does not have the room to cache now.
So, unless there is something very specific that needs to be cached, there is no rationale for this, and the chances are that linux will cache it for you anyway if its that much of a performance hit.
Last but not least, the biggest reason RAM disks are slightly faster than average (when linux does cache) is because they never have to synch to a physical medium. If you dont care that all the data you have written there is gone *poof* once a crash occurs, or if the system is shutdown, then thats ok. If you have a file there, and 'oops' forgot to write it to disk, its gone.
Even though the lack of a swap file capability for the Amiga meant you sometimes you didn't have enough memory to do some things it did give you a certain philosophy that worked well.
If you can run it, it will run well. If an app _has_ to use lots of memory because of it's fundimental nature (such as image processing) it will intelegently do the swapping itself.
Because memory was not considered vitually unlimited people developed with an eye towards keeping memory requirements down.
The requirements of programs for OS's with swap files have rocketed over the last few years.
It's sad to see the Amiga style of operation disappear without any debate as to the merits of it. People made swap files because they could, not because they were essential.
Bill Gates reputedly once said '640k should be enough for anybody'. In my own mind I think that 64 meg should be enough for most people, yet I routinely do tasks on 64meg machines that are swapping merrily away. It's not because I'm not considering the possibilities of things that could be done (Like Bill did) but rather that I feel that those things could be done in 64Meg. I fell like my memory is going to waste.
I have ICQ running at te moment. It's using 6 meg, I haven't a clue what it actually puts in that 6 meg. If memory were not considered unlimited how much would it be using?
-- That which does not kill us has made its last mistake.
You have to admit that if I somehow got an "Ask Slashdot" posted where I asked the question: "How do I get my Windows 98 box to recognize 4 ethernet cards without crashing?", I'd get the following five answers:
1. Works fine for me - under Linux.
2. Why are you using Windows?
3. Linux supports theoretically infinite ethernet devices.
4. Natalie Portman!
5. FUCK YOU AND YOUR WINDOWS BOX, YOU WHORE
"Beware he who would deny you access to information, for in his heart he deems himself your master."
A friend of mine uses a RAM disk for his Netscape cache. It saves him the trouble of having to clear the cache out manually on every boot. (He is on a Mac.) I have been considering doing something similar on my Linux box, but too many things are already needing to be done.
"Trademarks are the heraldry of the new feudalism."
The reason ramdisks were so useful on your old Amiga was the absence of a good disk caching program. Linux has excellent dynamic disk caching, you're better off letting the kernel have that memory to play with instead of locking it into a ramdisk.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Friends don't let friends enable ecmascript.
Every filesystem type comes with a special set of options you can use when mounting one of those filesystems. For Solaris' tmpfs, you can set the maximum size of the "partition" so that it won't actually use up all of swap.
Most admins bitching about the large files problem aren't aware of this option. (I wasn't for a while.)
You cannot apply a technological solution to a sociological problem. (Edwards' Law)
E is derived from FVWM?
One of the Fundamental differences between all Unixes and every other OS ever invented is the use of memory to buffer the filesystem.
Actually, disk caching is NOT a unique idea at all.
Macs have supported a disk cache for performance since at least System 3.0, in 1986. You can see a history of the old Mac OS here. However, I'm not sure if this is a read cache only and what form of cache writing scheme it supports if any nowdays.
While I can't really say about the DOS-based Windows variants, the NT versions of the Win32 API has lots of support for asynchronous file I/O. By default, all normal disk writes are written to a disk cache which is lazily flushed. You can specify certain options when opening a file handle with Cre ateFile() to force it to write straight through to disk rather than lazily cache it. In fact NT gets its asynchronous packet-based I/O subsystem design from VMS. (The designers of the NT kernel were ex-VMS designers.)
Finally, while I can't speak about the Amiga, I can speak about MVS's descendant OS, OS/390, which can handle asynchronous file I/O. I can't find you a good link, but most of the references I could find on this talk about OS/390's UNIX services. Apparently around release 2 of OS/390, they began to comply to the XOpen definition of a UNIX, so I guess that doesn't help that much.
If it's for-profit but free, you're not the customer -- you're the product (e.g., the Slashdot Beta's "audience").
The purpose of this is to let the hard drive in a laptop spin down. It is one of several suggestions in an old Linux laptop power reduction list...which I can't find at the moment.
I want my PIII to boot like a C64!!
-- Periodically spray diskettes with insecticide to prevent system bugs from spreading.
Obviously this will be vulnerable to failure, but for something that collects massive quantities of statistics, such as Ifile, it can be worthwhile.
With Ifile, an early edition stored stats in DBM files, and would do simply massive numbers of increments to entries. On disk, this meant that for a relatively small mail spool, the analysis would take hours.
If you're not part of the solution, you're part of the precipitate.
Well, yes but all the fvwm code is long gone...
"Free your mind and your ass will follow"
I think his point was that you wouldn't have any writable media in the machine which would prevent what you're talking about.
Of course the corollary to that is if you could create a ramdisk then you can write to it. So if you leave out ramdisks and all network filesystems - you'd be set. Just log everything remotely via syslog (or something better/custom) and you'd be pretty much defacement proof... won't stop 'em from bringing down the server - and because you're running on ROM you won't be able to fix the problem very quickly so they could just keep performing the same exploit over and over... haha!
But if you've ever USED a ramdisk before you'd know that it stores non-active executables in the ramdisk or files you want memory resident without actually being in use. On MacOS you can stick Word and IE into the ramdisk and they will start pretty darned fast as they don't need to access the disk in order to open. The cache is for stuff thats already been used, ramdisk is for things you might use.
I'm a loner Dottie, a Rebel.
To get the best of both worlds, it would be very handy if you could use logical volume management (or maybe something simpler) to ensure that the /tmp filesystem started out using RAM disk, then migrated onto a second physical volume on disk.
/tmp filesystem caching to be much more aggressive, using more memory, this would be self-tuning (i.e. expanding to disk when needed) and probably work better all round. Though perhaps a special filesystem would still be needed to avoid writing to disk unless you've run out of RAM.
But perhaps the real issue is filesystem-specific caching parameters - if you could configure the
but you can't unmount root.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
I agree. fvwm2 is my wm of choice. I was just stating a fact.
BTW I AM Ed Gein, muahahaha.
"Free your mind and your ass will follow"