Slashdot Mirror
How Much Virtual Memory is Enough?
whitroth asks: "Ten years ago, Received Wisdom said that virtual memory should be, on the average, two to two-and-a-half times real memory. In these days, where 2G RAM is not unusual, and many times is not that uncommon, is this unreasonable? What's the sense of the community as to what is a reasonable size for swap these days?"
Under Windows it seems it'll swap out whether the free RAM is needed or not, no matter what (there's a registry setting to change this though). Under Linux, you won't swap much anyway unless you need it.
I run a Core Duo laptop with 1GB of RAM and have never swapped out in Linux, no matter what I was doing.
"You can either have software quality or you can have pointer arithmetic, but you cannot have both at the same time."
Back when I had 512MB of memory, I had a 512MB swap partition, but I noticed that I never came close to using all of it.
When I got my new machine with 1G, I never bothered to make one at all, and I've never had a problem with it. If I do ever find myself in a situation where I need some swap space, I could always just create a swap file. It's a lot more convinient because it wouldn't have to be a fixed size, doesn't take up space when I don't need it, and I have one less partition
Especially if you have 2G or more, I don't see a real reason to use swap
2X physical memory for under 2G RAM
2G swap for up to 8G RAM
+1G swap for every 4G RAM beyond that
According to MS, it's 1.5 times the total RAM. I assume you're asking because you're trying to avoid a fragmented page file. While the benefits of an unfragmented page file are dubious at best (since it will be randomly accessing different parts of the page file), it's better to err on the side of caution: If you have 2GB of memory, you likely have an equally compensating-for-something hard drive, so you probably won't miss 3GB of space, or even 4. It's better to waste a little space than have Windows run out of Virtual Memory. Otherwise, just let it do its dynamic page file adjustment thing.
If you're asking about creating a swap partition for Linux then 1.5X is also recommended. Just be generous, unless -- for some reason -- you've got 2GB of RAM and a 50 meg hard drive. Too much is always better than not enough.
https://www.eff.org/https-everywhere
One of the real advantages of using swap isn't to avoid memory exhaustion at all; by moving infrequently accessed pages from memory you make more room for the disk cache, thereby possibly improving overall system performance by reducing hard drive reads.
just let windows set it for you.
My strategy generally is to use a file for swap rather than a partition, even in linux. I figure that if memory has to be swapped in from disk, it's already crappy going to disk so the extra overhead doesn't matter much, and I have freedom to adjust it up or down depending on my needs. (This is a desktop/laptop circumnstance). I generally start at 512MB or so, increasing maybe if IO is faster on the drive. I view swap like a rumble strip on a road before a stop sign. With no swap, you don't realize a process leaked memory until it's too late, with swap, while it eats through your swap the performance will degrade and you'll see the end coming ahead of time, and may be able to head it off with a kill. It may be well an good your 4GB of ram is technically capable of handling the same load your 1GB RAM+1GB swap handled in the past, but having some noticable impact when things start going wroing is nice. I realize theoretically there are better approaches, but nothing gets in your face like poor performance and tons of disk accesses.
On a production server or a problematic system where I want support and the OS likes to dump a core to swap, I'll ensure a generous swap partition is available (generally observed active swapx1.5+physical memory size). In this case a file-backed swap may depend on layers of the kernel that are in an invalid state, and a swap partition is more likely to be reliably writable. The only system I would even theoretically hibernate on is my laptop, and I only ever suspend to ram or shutdown completely, so I don't consider my laptop as needing a swap partition of any significant size.
XML is like violence. If it doesn't solve the problem, use more.
I have 4GB of physical ram (ddr2-6400) and 4gb of swap. There are actually a few reasons for this, YMMV (obviously I think the answer to this question depends on what you do).
I have a lot of things running which, usually, are doing nothing. For instance, apache2, mysql, postfix, and courier-imapd-ssl are always running, but they're rarely actually *doing* anything. (If I get a hit or an email, it's relatively rare as I hardly have very little hosted off of my home box - nevertheless, I do want these running). So I'm happy to let these get swapped out. When I start up matlab, and start dealing with huge datasets, I know it's going to swap most of these out. That's good. It will also swap out some of my matlab data that's loaded but not currently being used (and yes, it's quite possible to have >4gb in your workspace). For me, I have the swap because I need it. Figure out what you need, and you will have the answer to your question.
Disk is always far cheaper and more plentiful than memory. If you have four gigs of memory, what's wrong with carving eight gigs of swap out of your terrabyte RAID? If you have that much memory in the first place, then you're probably running large apps. Do you and them a favor and give them a little breathing room.
Dewey, what part of this looks like authorities should be involved?
But... but... the rule of thumb says to have twice as much swap as RAM!
It's a pet peeve of mine that so many system administrators appeal to "rules of thumb" about decisions such as this, instead of actually thinking it through. Sys admins pass around these nuggets of wisdom with unquestioning reverence, like they were handed down from some bearded UNIX guru sitting on a mountaintop. These rules either 1) happen to reflect reality, 2) do not reflect reality, or 3) reflected reality 20 years ago but nobody got around to issuing some sort of "revocation rule of thumb". :)
My experience is that very little swap is needed these days, and the rule of thumb falls into category #3. Long gone are the days that the OS demanded swap space for all process memory.
If I have a machine with 1GB of RAM, I'll usually give it 512MB of swap or so. As discussed elsewhere in this thread, a little bit of swap is good for pre-emptive swapping and for emergencies (to avoid the dreaded Linux "oom killer".) Also, if you're going to use hibernate, you'll want at least as much swap as real memory.
My strategy generally is to use a file for swap rather than a partition, even in linux.
What I find curious is that you have a strategy. On what relevant experience do you base this strategy? 1 GB of disk space costs less than $0.50. Set up 3 GB of VM if it makes you feel good. The latte you drink while you set it up costs more than the extra disk space!
So go for it!!! Who cares what you do? Heck, give yourself 10x the RAM and see if it actually makes any difference!!! (it won't)
This is sort of like asking: "Which goes faster: the yellow Pacer or the red Pacer?"!
I have no problem with your religion until you decide it's reason to deprive others of the truth.
The equation stays about the same though the scale of memory sizes involved increases. If one ran a set number of processes that all fit within core (RAM) memory and did not increase in size over time, one wouldn't need virtual memory at all. When using a properly sized computer of a given generation, the typical set of processes being run fits in or almost fits in core memory, so a virtual memory size equal to the core size provides ample protection against memory exhaustion (both core and virtual memories full). Exactly how much memory this is increases as the sizes of those typical processes increase. These days 2gb of core is usually large enough to avoid the need to use virtual memory, but it can be consumed pretty quickly by either large numbers of typical processes or a few memory intensive ones. Memory exhaustion is a very unpleasant situation and leads to data loss and service outages. The computer does not react well to having literally no room to think. So given this, and that virtual memory (disk space) gets cheaper at (somewhat) the same pace as core (RAM), it is much safer and cost effective to err on the side of caution and make the virtual memory bigger than necessary for day to day operation. Regardless of the scale of the current generation memory sizes, a virtual memory space equal to one or so times the core space of a properly equiped machine is the right size. For small core machines, the larger the core memory deficit, the more times larger the virtual memory space must be to avoid running out of total memory. A machine running the latest Windows environment in 512mb of core would need a virtual memory much larger than one or two times that size to be safe. Said machine would still perform very poorly due to the cost of continually accessing the virtual memory, but it would avoid crashing due to memory shortage. Systems with much more than average core memory may be able to do safely with less or even no virtual space, but it is arguably a foolish place to conserve since disk space is cheap and maintaining at least a one times core sized virtual memory space is insurance against the pain of memory exhaustion.
Or distilled: less RAM than average needs more than two times that for virtual, average RAM needs one to two times that, and lots more RAM than average can probably get away with less than one times or even none but probably should use one times anyway.
Again note that average refers to the RAM size of a current generation machine configured to run the typical number of typical current programs with reasonable performance.
What the original article didn't mention, and none of the replies seemed to go into, is the fact that with current CPUs, effectively all RAM is 'virtual':
Only on-chip memory, i.e. cache, is "real" these days, and all accesses to DRAM will be handled in paging units of 64/128 bytes or so. If this sounds familiar, it should! CPUs with 1 to 4 MB of real memory and lots of virtual memory is what the mainframes and minicomputers had about 20-30 years ago.
What this means is that now, just like then, all performance-critical code needs to be written to keep the working set within the amount of "real" memory you have available. When you passed this limit, you needed to make sure that you handled paging in suitably large blocks, to overcome the initial seek time overhead.
Today this corresponds to the difference between random access to DRAM and burst-mode (block transfer) which can be nearly an order of magnitude faster.
In the old days, when you passed the limits of your drum/disk swap device, you had to go to tape, which was a purely sequential device. Today, when you pass the limits of DRAM, you have to go to disk, which also needs to be treated as a bulk transfer/sequential device.
I.e. all the programming algorithms that was developed to handle resource limitations on old mainframes should now be ressurected!
"those who forget their history, are condemned to repeat it"
Terje
"almost all programming can be viewed as an exercise in caching"
I recently attended IBM's "Performance Tuning with AIX" course. It could basically be summed up as "Don't Use Paging Space. Ever." Then it went into lots of detail about AIX memory management techniques and the VM subsystems, with a brief foray into network performance.
It is a very sickening feeling to go and power-cycle a production system which is completely halted due to running out of memory. Almost as bad is a system which is hitting the swap and responding like molasses.
Look at the work you need your server to do, then put the RAM in it you need to get the job done. I've not worked with Linux in a full-on production environment, but I will go look into its systems for dealing with OOM errors. I'm sure it will be interesting.
-- Flaw
Yes ram is incredibly cheap, and any amount of serious swapping is to be avoided. On the other hand, once in a while you do something stupid like having VI load a 2GB log file into RAM, or whip firefox into an 800mb frenzy and then load that 16kx32k image into GIMP, or do that database query that uses *way* more ram than you'd expected.
In general, I'd rather have my system slow to a crawl than blow up in my face when something like that happens. At least, then, I've got the choice of what I want to kill/stop, rather than having random (critical) processes die on me and have no choice other than a post mortem.
If you're that worried about your system slowing to a crawl when you start eating into swap space, then put instrumentation onto the system that alerts you when swap gets over 100MB. At least that way, you keep uptime and some hope of a controled recovery. With the price of hard disk storage being what it is today, it's not having a few spare gigabytes of backup VM resources that seems like a bad idea.
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
I do - any time I'm running Mozilla with a lot of tabs open and it decides to go into annoying-swapping-mode (on WinXP and predecessors) for no obviously good reason, so I've got to wait for Mozilla to swap itself in or out before I can see the web page or other application I want. It doesn't help that I mainly use it on a laptop, where the drive is slow and the RAM is a fairly large 384MB, but it also happens on my home desktop, where the drives are faster and there's 640GB of RAM, which ought to be enough for anybody.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks