Slashdot Mirror
How Much Virtual Memory is Enough?
whitroth asks: "Ten years ago, Received Wisdom said that virtual memory should be, on the average, two to two-and-a-half times real memory. In these days, where 2G RAM is not unusual, and many times is not that uncommon, is this unreasonable? What's the sense of the community as to what is a reasonable size for swap these days?"
lots
Under Windows it seems it'll swap out whether the free RAM is needed or not, no matter what (there's a registry setting to change this though). Under Linux, you won't swap much anyway unless you need it.
I run a Core Duo laptop with 1GB of RAM and have never swapped out in Linux, no matter what I was doing.
"You can either have software quality or you can have pointer arithmetic, but you cannot have both at the same time."
My rule of thumb these days is 1.5x RAM, unless you're at 2GB, in which case I go with 2GB swap as well. This is for *gasp* Windows, though.
Back when I had 512MB of memory, I had a 512MB swap partition, but I noticed that I never came close to using all of it.
When I got my new machine with 1G, I never bothered to make one at all, and I've never had a problem with it. If I do ever find myself in a situation where I need some swap space, I could always just create a swap file. It's a lot more convinient because it wouldn't have to be a fixed size, doesn't take up space when I don't need it, and I have one less partition
Especially if you have 2G or more, I don't see a real reason to use swap
640k should be enough for anybody.
2X physical memory for under 2G RAM
2G swap for up to 8G RAM
+1G swap for every 4G RAM beyond that
I disable virtual memory on computers with more than 1GB of ram unless the user is going to be manipulating large images. Never had a problem yet.
If you use LVM (which you should, it's great!), you can expand and contract your swap partition as needed.
If you really want to know, I use 1-2 GB swap with 1GB ram and the same for 512MB ram.
However, you might just do what I do and try out different values to figure out what works. If you're talking about a linux system a real-time memory/swap usage graph can be added to most window managers so that you can see what's happening. You could also try to estimate usages based on what the machine is expected to do.
-Tim Louden
I use 4x750 GB hard drives (RAID), purely for virtual memory. It increases the speed on the RAM preprocessing directive, but demodulates the core processing utility monitor. I find it to be a good setup, especially for running Naibed Linux.
Stupidity is like nuclear power, it can be used for good or evil. And you don't want to get any on you.
According to MS, it's 1.5 times the total RAM. I assume you're asking because you're trying to avoid a fragmented page file. While the benefits of an unfragmented page file are dubious at best (since it will be randomly accessing different parts of the page file), it's better to err on the side of caution: If you have 2GB of memory, you likely have an equally compensating-for-something hard drive, so you probably won't miss 3GB of space, or even 4. It's better to waste a little space than have Windows run out of Virtual Memory. Otherwise, just let it do its dynamic page file adjustment thing.
If you're asking about creating a swap partition for Linux then 1.5X is also recommended. Just be generous, unless -- for some reason -- you've got 2GB of RAM and a 50 meg hard drive. Too much is always better than not enough.
https://www.eff.org/https-everywhere
One of the real advantages of using swap isn't to avoid memory exhaustion at all; by moving infrequently accessed pages from memory you make more room for the disk cache, thereby possibly improving overall system performance by reducing hard drive reads.
just let windows set it for you.
Linux has futzed with this a lot (and lets the user tweak VM behavior a lot, but /proc/sys/vm/swappiness goes a long way...), both linux and Windows will swap well ahead of not having free memory (for good reason). Just wanted to go into detail because I keep seeing people complain that they see swap used in linux or windows when they still have free memory, not realizing this isn't a bad thing generally.
There are generally two strategies:
-The common-sense one where you swap when you run out of memory. This makes a lot of practical sense on systems with limited write cycles (flash based swap, though you really never ever should do that anyway), and systems that want to spin down drives to conserve power for battery conservation. Performance wise (this may surprise people who haven't spent time thinking about it), this can often be bad. Avoiding swapping is generally only good on systems where resource utilization is carefully managed and you know it won't swap ever (the IO operations of unneeded can interfere with the productive activity of a constantly busy system). This is actually a vast minority of systems in the world (no matter how l33t one may think themselves, they most certainly don't have a usage pattern that would be impacted by the extraneous IO operations of occasional write to swap.
-Pre-emptive swapping. When the IO subsystem is idle and the system can afford to copy memory to swap area, it does so (depending on criteria). Generally speaking it will select memory not accessed much and write it to disk, but leave the memory copy in place if the physical memory is not immediately needed. A fair amount of swap used in an apparently underutilized system is duplicated in physical memory and swap space. The benefit here is that if the process reads back that memory, it doesn't incur any penalty in reading it back despite it being also in swap (the system may make certain decisions on what is the best swap candidate and write to disk different data). The benefit of writing this stuff to swap even when not needed is clear when an application comes along that allocs more memory than the system has free in physical space. In the first strategy, this means the malloc blocks while data is written to disk, and the new application starting or needing a lot of data is severely impacted. In the pre-emptive swap case, system notices the condition, knows what memory it has a backup in swap of that hasn't been used lately, and can free that memory and satisfy the malloc pretty much instantly.
To those who have 1GB of RAM or so it becomes less likely that the system will have to flush memory from physical RAM, but there is a balance to be struck between memory used directly invoked by applications, what the application memory access pattern is, and what ram you can use to buffer filesystem access. If your total application memory allocation is 75%, it still may make sense performance wise to only keep 50% of your physical memory dedicated to the applications, (the other bit relegated to swap), and 50% of the memory to buffer disk I/O.
XML is like violence. If it doesn't solve the problem, use more.
You're full of it.
1. The 128 KB Mac did not have a HD (though there were some companies that made disks that plugged into the floppy port).
But more importantly:
2. There was no "swap" (Virtual Memory) for the Mac OS until System 7, which wouldn't run anything less than a Mac Plus.
I think it was one of the Live Journal guys at OScon that said, "If your server starts to swap, you've lost the battle".
With all of our 64bit 4GB of ram minimum hosts floating around, there is no longer a point to having swap -- if you server really is swapping, it's under a huge load and the io is making the problem worse. Let the OS kill a few processes to get it back under control
-- DrZaius - Minister of Sciences and Protector of the Faith
OP poses wrong question. Virtual Memory is built into the OS and cannot be turned off. What OP means is Paging or Swap File (i.e. simulating memory using HD space). The rest of this reply will ignore this difference.
Very simply, if you use windows and use it heavily (run some intensive tasks or need performance), turning off the page file will give you a nice performance boost.. or rather will not take away from performance.
I have 1GiB of physical memory on my laptop, and reaching the limit in Windows when my paging file was off, posed a challenge (in other words, it worked perfectly well without it)
This is because Windows attempts to use the paging file whenever it can (proactive), unlike Linux, which uses it only when there's no other way (reactive). Depending on the applications you're running, one of the approaches will be better than the other, though from what I've seen, I don't like what Windows does...
Caveat Lector: this might be because I wasn't seeing the slowdowns which might've been caused by reactive approach. I've still yet to formulate an opinion on it - but so far it looks very reasonable.
If using Linux, keep the swap partition and forget about it.
In Windows, the best way to figure out if you need your page file is to load up as many apps as you normally load, maybe a few more - and check the memory usage (don't trust "VM usage" in windows task manager, it doesn't show you what you think it shows you!). If the usage is lower than your physical ram by a [few] hundred MiBs, turn off the page file and don't look back. If it's closer, set the page file to a small size, usually no more than 512MiB. If you set the file, make its size static, so that Windows doesn't try to adjust it all the time (it's too stupid to understand that you want to keep it as small as possible)
Interesting to note that the paging file is not used for hibernation, even though you'd think it were almost tailor-made for that purpose. I've heard that early betas of Windows 2000 woke up from hibernation in a few seconds - I bet they were using the paging file for hibernation then... but I digress
HTH
Did you know that "FTW" ("for the win") is a direct translation of "Sieg Heil"?
My strategy generally is to use a file for swap rather than a partition, even in linux. I figure that if memory has to be swapped in from disk, it's already crappy going to disk so the extra overhead doesn't matter much, and I have freedom to adjust it up or down depending on my needs. (This is a desktop/laptop circumnstance). I generally start at 512MB or so, increasing maybe if IO is faster on the drive. I view swap like a rumble strip on a road before a stop sign. With no swap, you don't realize a process leaked memory until it's too late, with swap, while it eats through your swap the performance will degrade and you'll see the end coming ahead of time, and may be able to head it off with a kill. It may be well an good your 4GB of ram is technically capable of handling the same load your 1GB RAM+1GB swap handled in the past, but having some noticable impact when things start going wroing is nice. I realize theoretically there are better approaches, but nothing gets in your face like poor performance and tons of disk accesses.
On a production server or a problematic system where I want support and the OS likes to dump a core to swap, I'll ensure a generous swap partition is available (generally observed active swapx1.5+physical memory size). In this case a file-backed swap may depend on layers of the kernel that are in an invalid state, and a swap partition is more likely to be reliably writable. The only system I would even theoretically hibernate on is my laptop, and I only ever suspend to ram or shutdown completely, so I don't consider my laptop as needing a swap partition of any significant size.
XML is like violence. If it doesn't solve the problem, use more.
This is completely wrong.
/tmp.
In Windows, your RAM is saved to a file called "hiberfil.sys" which is the exact size of your physical RAM. Your swap file stays exactly the way it is, otherwise you'd lose the data that was swapped to it.
In Linux, it depends on what program you are using to suspend, but typically, it's a file in
I have 4GB of physical ram (ddr2-6400) and 4gb of swap. There are actually a few reasons for this, YMMV (obviously I think the answer to this question depends on what you do).
I have a lot of things running which, usually, are doing nothing. For instance, apache2, mysql, postfix, and courier-imapd-ssl are always running, but they're rarely actually *doing* anything. (If I get a hit or an email, it's relatively rare as I hardly have very little hosted off of my home box - nevertheless, I do want these running). So I'm happy to let these get swapped out. When I start up matlab, and start dealing with huge datasets, I know it's going to swap most of these out. That's good. It will also swap out some of my matlab data that's loaded but not currently being used (and yes, it's quite possible to have >4gb in your workspace). For me, I have the swap because I need it. Figure out what you need, and you will have the answer to your question.
This depends - see other comments for most situations. However if you have a large Sun, HP, Fujitsu, IBM, etc. with 16+ CPUs and say 2 to 8 Gb per CPU (not uncommon in the big systems), then at minimum you need 3 times the --per CPU-- memory, becuse if one of the CPU's goes bad, the hot swap mechanism is going to use the swap space to keep the processes (at least on some of these systems) for moving them to the other CPU's as it marks the one as bad. You certainly don't need 2 times the total memory, or several hundred Gig. This is assuming the kind of NUMA architechure that I think all of these systems still have.
Generally we just used to use, say, 36 Gig local drives as (mirrored) swap for simplicity. In this environment you are probably on a SAN and people will say to move everything there, and that might be more true now than a year or two ago.
Many people think that you should save the fastest part of the HD to allocate as swap. But after thinking about it no matter where your swap is on a hard disk it's going to be noticeable when the system pages. what is not noticeable is the perceived difference in human terms of the VM at the outer most of the platter or the innermost. And if your allocating a gig of VM or more you're wasting space for system files or applications files where things like time to boot and application launch speed can be faster. (Along with prefetch data) In Linux you can use riser for your system root to further optimize this since most system application files are small in size. (Along with a small block allocation size.) Also, if you're using a secure VM file you may notice a difference if you move the paging file to a secondary drive with the file allocated at the outermost section. No matter what the system I make a page file. If it needs to be secure you should lock the applications pages to ram memory. I any event when using a disk you're going to notice paging and at that point you'll be waiting anyway... I doubt you'll care or notice at that point the page finsihed 2ms faster or not.
||| I still can't believe Parkay's not butter.
all this swapping talk is giving me nightmares.
a few years ago, we had a customer with multiple colocated servers complaining that sometimes they crashed for no apparent reasons.
after much debugging, we figured out one of their script was leaking memory. eventually consuming all ram (2.5G) + all swap (1-2g).
now the real problem is this. those were LIVE processes so the system was constantly paging back and forth, using 90%-95% cpu just to swap the damned things in and out and starving the actual processes.
linux 2.4, linux 2.6 (early 2.6). same deal. amazingly, the distro made a difference, redhat was pure hell, debian slightly better (though still not acceptable)
freebsd was much smarter, it just killed the offending processes.it sure wasn't ideal, but at least the server was still serving its clients
to this day, i never put more than 256m as swap even on servers with 4G of ram. that's where we had the least problems.
Sure when you had 128MB of ram, and you had a 256MB swap.
But dude, my next box will have two GIGABYTES of RAM!
Every one of your usage options assumes you'll run out of physical ram. Maybe if the OS is wasting it on pointless disk caching, but don't you think the programs in memory should have priority over blind disk caching?
Lest a foolish reader believe your two options (swap immediately, or swap as lazily/late as you can) are the only two possibilities, how about swapping when, say, only 20% of physical RAM is left? That way my Firefox and Eclipse don't swap to disk and take twenty seconds to swap in when I have 500MB of GODDAMN FREE RAM!
Hey, I'm just your average shit and piss factory.
On Mac OS X, swap is stored (by default) in files in the /var/vm directory on the boot hard drive, instead of on a separate partition. So there's no limit to how much is used, nor a predefined minimum amount of space used, the swap space expanding and contracting as needed. That seems reasonable.
Signature.
Disk is always far cheaper and more plentiful than memory. If you have four gigs of memory, what's wrong with carving eight gigs of swap out of your terrabyte RAID? If you have that much memory in the first place, then you're probably running large apps. Do you and them a favor and give them a little breathing room.
Dewey, what part of this looks like authorities should be involved?
A note regarding Windows, though: from XP and on, it's very rary that the complete hiberfil.sys is used. Pages are swapped out aggressively to general swap or whatever binary file that's backing read-only pages. The remaining pages are compressed. However, when all these decisions are made, it would be impossible/inconvenient to realize that the file was really too small, so a worst-case allocation is made.
You have a GUI to run: 600 MB for firefox, 1800 MB for OpenOffice.org, 100 MB for X, 100 MB for desktop odds and ends, 300 MB for Evolution or Thunderbird, and 10 MB for old-style stuff running in the background.
Total: 2910 MB
Yep, you need a gigabyte of swap. OpenOffice.org was made 64-bit clean for a reason. If you plan ahead, not wanting to reallocate disk space in the next few years, you'll allow for this:
2 GB for firefox, 5 GB for OpenOffice.org, 1/2 GB for X, 1/2 GB for desktop odds and ends, 1 GB for Evolution or Thunderbird, and 10 MB for old-style stuff running in the background
That's 9.01 GB. You're exactly 7.01 GB short, so you'll be needing that swap space before you know it.
swsusp stores the data in the swap partition.
Suspend2 can write it to a file instead.
See http://www.suspend2.net/features
But... but... the rule of thumb says to have twice as much swap as RAM!
It's a pet peeve of mine that so many system administrators appeal to "rules of thumb" about decisions such as this, instead of actually thinking it through. Sys admins pass around these nuggets of wisdom with unquestioning reverence, like they were handed down from some bearded UNIX guru sitting on a mountaintop. These rules either 1) happen to reflect reality, 2) do not reflect reality, or 3) reflected reality 20 years ago but nobody got around to issuing some sort of "revocation rule of thumb". :)
My experience is that very little swap is needed these days, and the rule of thumb falls into category #3. Long gone are the days that the OS demanded swap space for all process memory.
If I have a machine with 1GB of RAM, I'll usually give it 512MB of swap or so. As discussed elsewhere in this thread, a little bit of swap is good for pre-emptive swapping and for emergencies (to avoid the dreaded Linux "oom killer".) Also, if you're going to use hibernate, you'll want at least as much swap as real memory.
One thing to consider is whether or not you're using tmpfs for /tmp. For performance, I recommend using tmpfs for /tmp, and basically treat the swap partition as your /tmp partition. It may seem counterintuitive, "why would it be faster than a filesystem when it's backed to disk anyway, and my filesystems caches just file if need be?" The answer is that tmpfs never needs to worry about consistency. On the kernel.org machines, we have seen /tmp-intensive tasks run 10-100 times faster with tmpfs than with a plain filesystem. The downside, of course, is that on a reboot, tmpfs is gone.
whereas linux can swap however you wish, swap partitions are only frequently used because they are the best performance wise, no overhead dealing with the filesystem. Linux can and will use basicly any type of storage you can imagine to swap on though.
:)
If I ever encounter a linux box without swap, a quick dd creating an empty file and then using swapon on the file fixes all, you can also use multiple swap files/partitions if necessary. same deal.
only main annoyance with mac os vm is that it swaps out way before necessary, not as bad as windows but still it does some strange things which my mac tech friends cannot fathom why it does that way. Also it (appears) to not be changable behaviour. Correct me if wrong on that since said friends are quite annoyed with it
Man, it's utterly depressing to see the same useless "rules-of-thumb" still in effect when the original question is asking if the rule-of-thumb is a good idea.
1) Page space is not swap space. There's a small distinction that's generally lost (and generally ignored). Page space is used to move memory pages to and from disk. Swap space is technically to move entire processes out to disk. The difference is mainly based on when your OS was created (i.e., technological underpinnings) and no need to get into it now... but the difference is meaningful.
2) Page space is not *free*. There's a misconception that if you have 500G of disk space then "how does it hurt" to put 8G of swap on 4G RAM. Depending on your OS, the size of the page table can grow remarkably depending on how much memory (RAM + VM) is allocated. This means that adding 2G of page space may not cost anything, but adding 2.5G may suddenly take up another chunk of real, non-pageable memory because the page table cannot itself be paged. This means that if your app is thrashing, then adding page space may make it worse.
3) Even with lots of RAM, it's still often a good idea (depending on your usage) to have some page space. Modern OSes will still page out unused pages to use RAM for better stuff. I.e., if you have a huge file open in a graphics application, but are not actively using that application for a length of time (an hour, say) then the OS will page it to disk. This makes better use of your physical RAM. On some OSes the OS will use page space even if free RAM is available. It can then toggle a page out by flipping a bit in the page table and not have to do an expensive write.
4) In some systems you can overcommit memory. Applications tend to request a lot more memory from the OS than they'll actually use. This is useful in many instances but it again depends on your usage. If you're running a single application that doesn't dynamically allocate memory then you can run pageless. If a new app requests memory that's not available then it will get a failure on malloc request. This can be desirable in some circumstances.
5) There are benefits to running page space on a separate disk, but for the vast majority of home users, the difference is negligible. This applies to Windows and Linux. Once you start stressing the VM subsystem then a separate disk is highly desirable.
6) You can create page files on Unix/Linux. It's not desirable generally because of the extra filesystem overhead and possibility of fragmentation. But hey, in a pinch it works.
7) Why this 2x RAM rule? A lot of it comes from old VM subsystems that needed a "picture" of the entire memory space. This made the page-out algorithms easier to code. Newer algorithms don't require the 2X RAM.
KL
The 2.5x case comes from two simple rules of thumb. Firstly, you need enough to be able to hold the whole of what is in RAM now, plus everything you want to swap in, plus enough to minimise fragmentation and cover overheads. Secondly, the more swap space you have, the more metadata you need to manage it AND the greater the latency to perform any kind of swap AND the more swapping you need to do to run all active processes. Too much virtual memory is a Bad Thing. Having 2.5 x RAM was considered a good compromise and it is one I use to this day.
Today, both rules of thumb still hold. The largest single object you can have is one that fills ALL of RAM after the kernel, and you absolutely must have sufficient swap space to be able to dump that object to disk. If you don't, then the kernel will either panic, kill the process or cause any other activity to behave unpredictably. It won't have the resources to behave correctly. Any number of these objects could, in theory, be swapped out - but remember that they don't run when on disk, only when in memory, so the more you have, the smaller the timeslice each will get - and the sum of those timeslices will go down, as you need to allow time for the swap to take place.
However, today isn't quite the same as yesterday. The difference in performance between hard drives and RAM has changed. There is better caching on the drive. The swap algorithms are smarter and there is more understanding of what metadata is useful and what really has no value. Process handling is also smarter, so processes aren't necessarily run in order - round-robin scheduling is used for some time-critical stuff on Linux, but most applications use a more relaxed system.
Also, programming has changed. There is greater re-use of tools and libraries - well, sometimes - and this means that the largest object you really have to handle at a time is much smaller than the size of RAM. A certain fraction of what's left will be used by shared libraries and shared resources.
Lastly, because hard drives are reasonably cheap and most PCs can handle several at the same time, you are far far better off getting a drive and dedicating it to swap. This is good for many reasons, not least because the drive won't have to move the read heads from data space to swap space and back. You eliminate a vast chunk of seek time, reduce the stress on the drive AND can experiment with different swap sizes without risking losing data.
I would therefore STRONGLY advise using the classic 2.5x and a different hard drive, but if you can't do this for some reason and want an updated formula, here is what I would suggest:
The meaningful RAM will be equal to the total RAM minus the space used by the kernel and vital, non-swappable resources/daemons. Multiply this by three for 7200RPM hard drives or by five for 15000RPM hard drives. Multiply by one and a quarter for basic swap schemes, or by one and an eighth for profiling/intelligent swap schemes. Add the size of the hard disk cache, if the cache uses a high water mark to control operations. Subtract the size of the hard drive cache (unless this takes the size below zero) if the behavior is controlled by a low water mark only. Add one megabyte per simultaneous user. Add one megabyte for each large -or- long-running application likely to be running simultaneously. Subtract the total size of all the shared libraries likely to be loaded in the case just considered.
This is a LOT more complex than 2.5x, so much so that I generally wouldn't bother using it except
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
There's no real hard and fast rule anymore. And setting it against a static value (like physical memory) is incredibly wasteful.
It's a much better idea to set it interactively. Use the system without adjusting the Virtual Memory for a while. Then take a look at your usage and set your virtual memory against that usage.
For instance.
If you're in a Windows machine, let it run normally for a few days.
Run everything the way you normally use it.
Multiple apps, multiple instances, games out the ass, everything.
Then open up the Task Manager and look at the Performance tab.
Take a look at the Peak value under "Commit Charge".
Set your virtual memory, min and max, at about 10% above that value to leave yourself a little headroom.
Normally this will be enough to deal with your maximum swap requests.
If, somehow, you begin bumping against virtual memory limits again AFTER that, bump it another 10%.
If you still have problems, keep bumping it in 10% increments, and start looking for apps that are memory leaking.
Chas - The one, the only.
THANK GOD!!!
Nope. There is a separate file called hibernate.fil (I think) that's stored in the root folder of the same drive containing your Windows folder.
Don't you hate meta-sigs?
My strategy generally is to use a file for swap rather than a partition, even in linux.
What I find curious is that you have a strategy. On what relevant experience do you base this strategy? 1 GB of disk space costs less than $0.50. Set up 3 GB of VM if it makes you feel good. The latte you drink while you set it up costs more than the extra disk space!
So go for it!!! Who cares what you do? Heck, give yourself 10x the RAM and see if it actually makes any difference!!! (it won't)
This is sort of like asking: "Which goes faster: the yellow Pacer or the red Pacer?"!
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Before talking about swapping, pagging, and virtual memory please learn and understand this equation: Virtual Memory = Physical Memory + Swap(or Page) file. I let the OS (windows) manage my page file. The current generation of windows OS's (2k, xp, & 03) mange the swap file much more efficently than windows 9x did. All this mumbo-jumbo about tweaking the swapfile came about because these old versions of windows needed to be tweaked, they had memory problems and tweaking the swapfile would improve performance and stability. But the current generation of windows has no need for tweaking the swapfile, save for some special server situations. Its a waste of time to personally manage the swapfile for a desktop/development computer.
I never user more than 512Mb of swap. If you have a runaway process, you can let it live but you avoid a lot of trashing. If more than one process start consuming RAM like crazy you, actually want them to die from an out-of-mem error otherwise your whole system will grind to a halt while it spends most of its time unswaping one and the other. At 512Mb you can do a little excess of memory usage but won't go beyond what you can unswap in a time quantum (mostly).
Smarter per-process ressource quotas would probably be better and it would be nice to have a trashiness function according to the disk speed but so far 512Mb sounds like the limit between using the resset button or just taking a coffee break when you see the HD led blinking like a strobe.
It is just easier to try the approach where you consume a lot of RAM first and to re-code if it doesn't work. I work in bioinformatics and we often have huge datasets, I alway try to load the whole thing and to make the computation in RAM. Only when I get and out-of-mem error do I segment the dataset and try a smarter approach. That might explain my choice for 512Mb and the right threshold for other people might be bigger or lower but I'm pretty sure that its bellow 8Gb.
That raises the question: is swapping obsolete? Or to put it more explicitly, has the speed difference between modern CPUs and hard drives become so large, and RAM so cheap, that it's better to consider running out of RAM to be indicative of a software failure? That way you end up with a system where one or more processes may fail (or be terminated) but at least the machine remains usable and doesn't swap itself into non-responsiveness.
In my experience, the answer is yes: with 2GB of RAM in my machine, I never need to swap, and in the few instances where swapping did occur, it was because of buggy software (memory leaks) and manually terminating the offending processes is what I needed to do resolve the memory shortage. So why not just have the OS do that automatically?
Or to put it a third way, is there any situation where swapping is helpful, anymore?
I don't care if it's 90,000 hectares. That lake was not my doing.
...I see your swap is as big as mine...
"Memory is like an orgasm. It's a lot better if you don't have to fake it."
-- Seymour Cray, on virtual memory.
You are not entitled to your opinion. You are entitled to your informed opinion. -- Harlan Ellison
Yes, swap is useful in any situation when you don't know if you'll have enough RAM to run everything.
And RAM can be so "cheap" as you say, but disk is still far cheaper.
With swap, you also have some way to find out that you're running out of memory. You can monitor it, and you can also sometimes see a performance decrease (if it's a desktop), though you'll probably not notice it with SCSI disks. But you still have the monitor, right ?
Or to put it a third way, is there any situation where swapping is helpful, anymore?
:)
Sure. Consider Andrew Morton's logic:
http://kerneltrap.org/node/3000
In your average program, most code never gets executed, and most data is never used. For a long-lived process, swapping out the unnecessary bits frees the memory for disk cache.
While you may improve overall performance, by minimizing the average completion time for operations, the downside is responsiveness. As a user, I don't care if Firefox reads cached images a few milliseconds faster (by reading from cache instead of disk) if I have to wait 3 seconds for Thunderbird to respond to my clicks (because it has to swap in) after I've been browsing for a while. Average speed be damned!
Having said that, I just set my swappiness to 100.
The equation stays about the same though the scale of memory sizes involved increases. If one ran a set number of processes that all fit within core (RAM) memory and did not increase in size over time, one wouldn't need virtual memory at all. When using a properly sized computer of a given generation, the typical set of processes being run fits in or almost fits in core memory, so a virtual memory size equal to the core size provides ample protection against memory exhaustion (both core and virtual memories full). Exactly how much memory this is increases as the sizes of those typical processes increase. These days 2gb of core is usually large enough to avoid the need to use virtual memory, but it can be consumed pretty quickly by either large numbers of typical processes or a few memory intensive ones. Memory exhaustion is a very unpleasant situation and leads to data loss and service outages. The computer does not react well to having literally no room to think. So given this, and that virtual memory (disk space) gets cheaper at (somewhat) the same pace as core (RAM), it is much safer and cost effective to err on the side of caution and make the virtual memory bigger than necessary for day to day operation. Regardless of the scale of the current generation memory sizes, a virtual memory space equal to one or so times the core space of a properly equiped machine is the right size. For small core machines, the larger the core memory deficit, the more times larger the virtual memory space must be to avoid running out of total memory. A machine running the latest Windows environment in 512mb of core would need a virtual memory much larger than one or two times that size to be safe. Said machine would still perform very poorly due to the cost of continually accessing the virtual memory, but it would avoid crashing due to memory shortage. Systems with much more than average core memory may be able to do safely with less or even no virtual space, but it is arguably a foolish place to conserve since disk space is cheap and maintaining at least a one times core sized virtual memory space is insurance against the pain of memory exhaustion.
Or distilled: less RAM than average needs more than two times that for virtual, average RAM needs one to two times that, and lots more RAM than average can probably get away with less than one times or even none but probably should use one times anyway.
Again note that average refers to the RAM size of a current generation machine configured to run the typical number of typical current programs with reasonable performance.
What the original article didn't mention, and none of the replies seemed to go into, is the fact that with current CPUs, effectively all RAM is 'virtual':
Only on-chip memory, i.e. cache, is "real" these days, and all accesses to DRAM will be handled in paging units of 64/128 bytes or so. If this sounds familiar, it should! CPUs with 1 to 4 MB of real memory and lots of virtual memory is what the mainframes and minicomputers had about 20-30 years ago.
What this means is that now, just like then, all performance-critical code needs to be written to keep the working set within the amount of "real" memory you have available. When you passed this limit, you needed to make sure that you handled paging in suitably large blocks, to overcome the initial seek time overhead.
Today this corresponds to the difference between random access to DRAM and burst-mode (block transfer) which can be nearly an order of magnitude faster.
In the old days, when you passed the limits of your drum/disk swap device, you had to go to tape, which was a purely sequential device. Today, when you pass the limits of DRAM, you have to go to disk, which also needs to be treated as a bulk transfer/sequential device.
I.e. all the programming algorithms that was developed to handle resource limitations on old mainframes should now be ressurected!
"those who forget their history, are condemned to repeat it"
Terje
"almost all programming can be viewed as an exercise in caching"
In my opinion the rule of thumb still applies. No matter what you are going to do with your system, it never hurts to have (more than) enough swap space. Like several people mentioned earlier: the more free RAM you have, the more RAM is available to the OS for the disk and buffer caches.
There is, however, a potentially severe case if you have two processes accessing the same resource simultanously. Every good informatician and computer programmer knows that such a case is the ultimate no-no in software engineering. Unfortunately, there are scenarios thinkable, in which it will happen nonetheless.
Back when I was still working on my Bachelor's degree I and a couple of friends of mine tried to simulate this theoretical possibility and see what happens. We had two processes, called 'ss1' and 'ss2', accessing the same resource at the same time:
ss1 would create a file sized X and go into an endless loop writing random bytes at random positions in the file. ss2 would open that file and mmap() it. That way it would be in the buffer cache as long as data was written to it (and since data was written to it by the other process, that was actually the case). The result of the mmap() was a character array and ss2 would write random bytes to that character array at random positions.
We tested this on the following OSes: Linux 2.0, Linux 2.2, Solaris x86 (can't remember which version), FreeBSD 3.3, Irix 4.0.5, 5.3 and 6.2 and Windows NT 4.0 Workstation. We ran the application with administrative or superuser privileges.
As long as the size (X) did not approach half the physical amount of RAM present in the machine, there were no problems whatsoever. However, as soon as X passed that threshold, bad things started to happen. The only exception was Windows NT, which simply aborted the process with a page fault and an out-of-memory error.
All the aforementioned machines that were running Linux or a variant of UNIX, suffered the same problem: a non-responsive system. The processes could only be terminated by doing a hardware reset of the machine. A kill -9 of the two processes did not work, because they were in a non-interruptable sleep. And the reason they were was that the OS was trying to fullfill the resource demands of the processes by swapping out other stuff, including, as we theorized, other parts of the file that were not "hot" at that time.
This piece of intentionally bad-written software and intentionally bad system operatorship of course proved that, while it was highly unlikely to happen, it could happen and would have dire consequences for the system.
Ordinarily, one should never run programs as a privileged user unless one absolutely has to and the two competing processes would have been terminated by the OS had they not run as root on the Linux and UNIX machines. But regardless of whether the OS in question uses the optimistic or pessimistic approach when allocating resources for a new process, the net result of having such a (in our case intentionally) badly written piece of software is the same: the system becomes non-responsive.
In this case, it does not matter much how much swap space you have, the only difference is that if you have only a little amount of swap space the "dreaded" OOM killer starts to kill of processes at a very early hour instead of when it is already too late (and virtually incapable of functioning properly and actually do its job).
Personally, I would still recommend using at least the same amount of swap space as you have physical RAM, and preferably at twice the amount. Bad things happen all the time, and it is better to be prepared for it. Therefore, the rule of thumb still applies.
And on the Eighth Day, Man created God.
The VM Size listed in Activity Monitor is not the size of the swapfiles. I'm running on a machine with 1 GB RAM, my VM size is 6.53 GB, but I only have 128 MB of swapfiles. You might find it interesting to intall MenuMeters http://www.ragingmenace.com/software/menumeters/ to keep track of what's going on.
Oh, please. How do you quantify "more responsive?"
Sorry to say this, but the witch-doctory of computer maintenance is not engineering, it isn't science, it isn't even common sense
If you get "thrashing" in your system it is because you don't have enough RAM for the applications you are using. At this stage, the application has one of two options, use user space files to reduce the "in RAM" data size or quit with an out of memory error. Most all applications, in this scenario, perform better when the OS provides virtual memory. (Obviously not all because the default swap algorithm can be a worst case, but I digress.)
The Macintosh's early virtual memory (pre-OS/X) and old OS/2 1.x virtual memory systems really created a lot of lore about the woes of virtual memory. The reality is that the modern VMMs as implemented in the Pentiums and later are very good, and the OS support for them are excellent, and a system is almost always better off with VM than without.
In an honest discussion it is hard to speak with absolutes, so no one can say any one way is any better than any other way 100% of the time, unless it is an obvious choice like drink poison or drink pure distilled sterilized and safe water in moderate amounts when you are thursty and have balanced electrolytes.
That being said, unless you have a specific and quantifiable case where virtual memory hurts performance dramatically, use virtual memory, you will be better off.
And is 1000 times faster ?
So, to answer the original question :
Optimal amount of Swap ? 0 !
Even my old PIII 1Ghz takes 2 gigs of ram. Newest system we have at the office takes 64 Gigs.
=> right now I consider the Linux systems @ work as having a problem if they use swap...
Adding just a wee bit of RAM to your system and seeing swap disappear means your perfomance just exploded on this particular task...
Best Regards,
D.
It takes 40+ muscles to frown, but only four to extend your arm and bitchslap the motherfucker
Before I started this I didn't care about the answer, but now I realize that this question is as important as "Ultimate Answer to Life, the Universe, and Everything" and as I read I have realized that we humans are not powerful enough to answer this question. We should build a super computer with more VM and VA and more processors, yes, then we can ask it the question.
Just kidding all, but it was good reading
My own perspective is from UNIX servers. As I keep telling people any use of swap/paging spaces is bad for performance so the ideal solution is to add RAM. That's not always practicable, so the real answer to how much swap space to allocate is "enough".
I still get software suppliers, (mostly SAP AG) moaning that we've got to allocate 3.5xRAM, which is arrant nonsense. It might have been necessary years back when 2GB was a lot of memory. Now I've got servers with 10s of GBs and I really don't want to waste 100s of GBs of disk on swap space which simply isn't going to be used. Sure: disk is cheap but it all adds up. One of the larger servers I support has 128GB of RAM and 32GB of paging space, (only 1% is actually used at the moment). A few servers like that and you're saving TBs of disk space.
Of course, if you're going to keep your swap space to a minimum you need to have good monitoring in place so that you can extend it before it becomes a problem if something unexpected happens, and it's sensible to be a bit generous about it. We do occasionally have problems when processes suddenly start writing vast amounts of data to memory but I doubt that having loads more swap space would help in those cases, as there are usually bugs in the code. Fortunately root can usually still get in, (if you're patient), identify the offending processes and kill them.
It also helps to have an OS that makes effective use of memory. What I know best is AIX and a few years back, (quite a lot of years in IT terms!) the memory allocation processes were changed so that even if you requested an enormous amount of memory it wasn't really allocated until you actually started to use it, (i.e. put some data in there). That made a considerable difference. I would expect any modern and efficient OS to do something similar.
Paging can be dreadful for performance as you get a multiple hit: the process that needs swapped-out pages runs slow as it waits for data to be paged in; your system as a whole also runs slowly as CPU cycles are taken up servicing the paging requests; your I/O subsystem suffers as it spends time reading and writing to/from paging spaces rather than actually doing useful I/O. It's one of the first things I always target when I'm investigating performance problems on a server, just as it was a couple of decades ago when I was doing the same things with MVS.
Power Mac G5
OS X.4.7
3GB physical RAM
64MB swap file, which has never grown bigger since I added the extra RAM
...so, no, at least on OS X there's no point in having 6GB swap files.
For Windows XP the geniuses at Microsoft removed this ability and the whole system runs much worse because of it.
Every time you do something which reads big files from disk on XP all your apps get paged out to disk. I don't know in which fantasy world this is supposed to be an "improvement", but it's one of my favourite reasons to hate XP.
No sig today...
We've found that 512 Megs of swap is more than enough for our 2 and 4 Gig machines. Why even have swap? Here is an example:
1) On a system with zero swap, when apache gets slammed (say you get to the top of digg or slashdot), apache starts consuming lots of memory to handle new inbound requests. When it runs out, the machine grinds to a halt because it can't allocate more and requires a power cycle. (Setting a low max children really only helps if you are happy denying traffic to the people who are trying to see your site...it's best to plan for capacity and put quite a few servers load balanced).
2) On a system with any appreciable swap (IMHO, more than 128 Megs, up to 512 Megs), if you're monitoring the system (watch -n 1 df -h, for example) and all of a sudden it starts using swap, the machine is on the edge of dying. This gives you an early warning that maximum machine performance/throughput is occurring. You can restart apache or shut it down or similar, you can do something to temporarily lower or remove load from that machine. This doesn't give you *much* time, but it gives you some.
In our real world experience, at digg and slashdot loads you have about 10-15 seconds to stop apache once it starts swapping. After that, the performance degrades so bad that the machine becomes catatonic, the same as #1, requiring a power reset (obviously because virtual memory on HD is magnitudes slower than RAM, as numerous others have suggested). The key here is that you must realize that some swap is good for allowing unused programs to be swapped out, such as login terminals that just sit there. It's great for detecting problems, but if your heavy app is the one utilizing swap, your machine is about to crash anyway.
I think most people who think that swap should be in terms of multiples of physical RAM are missing the point.
How much swap you have should be related to the longest you are willing to wait for stuff to be swapped in and out.
Adjust your swap so that your computer is as slow as you can tolerate when it runs out of memory.
For example: if you have a typical ATA drive, random read transfers would be about 10-15MB/sec. So if you ever need to swap in 400MB of stuff, you'd have to wait about 30-40 seconds before all of it is read in.
What complicates things is there are some applications/programs that allocate memory that they will practically never use, so you'd may want to add swap for that.
So the swap amount would be something like: total swap = "permanently swapped out unused stuff" + (seconds willing to wait * random read speed).
Of course virtual mem doesn't really behave exactly like that - when you are low on RAM the computer will be continously reading the program it needs in, while writing the stuff it thinks it is less important out, but basically you're kind of reliving the old days of "drum/disk memory" - where you running stuff from drum or disk. And that's really slow.
The problem with running out of memory is that under some conditions some operating systems (e.g. Linux) can mess up and kill the wrong process to free memory. I think this has improved somewhat - but Linux used to be pretty stupid and kill pretty important stuff...
This is mainly because of the default overcommitting of memory. With overcommit, the O/S can say "fine" even if there really isn't enough memory, but when it turns out you really do need it all, the O/S goes around looking for stuff to kill...
If you turn off overcommit things can become safer, but you'll need enough memory to hold all allocated memory even if unused.
I recently attended IBM's "Performance Tuning with AIX" course. It could basically be summed up as "Don't Use Paging Space. Ever." Then it went into lots of detail about AIX memory management techniques and the VM subsystems, with a brief foray into network performance.
It is a very sickening feeling to go and power-cycle a production system which is completely halted due to running out of memory. Almost as bad is a system which is hitting the swap and responding like molasses.
Look at the work you need your server to do, then put the RAM in it you need to get the job done. I've not worked with Linux in a full-on production environment, but I will go look into its systems for dealing with OOM errors. I'm sure it will be interesting.
-- Flaw
Yes ram is incredibly cheap, and any amount of serious swapping is to be avoided. On the other hand, once in a while you do something stupid like having VI load a 2GB log file into RAM, or whip firefox into an 800mb frenzy and then load that 16kx32k image into GIMP, or do that database query that uses *way* more ram than you'd expected.
In general, I'd rather have my system slow to a crawl than blow up in my face when something like that happens. At least, then, I've got the choice of what I want to kill/stop, rather than having random (critical) processes die on me and have no choice other than a post mortem.
If you're that worried about your system slowing to a crawl when you start eating into swap space, then put instrumentation onto the system that alerts you when swap gets over 100MB. At least that way, you keep uptime and some hope of a controled recovery. With the price of hard disk storage being what it is today, it's not having a few spare gigabytes of backup VM resources that seems like a bad idea.
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
I do - any time I'm running Mozilla with a lot of tabs open and it decides to go into annoying-swapping-mode (on WinXP and predecessors) for no obviously good reason, so I've got to wait for Mozilla to swap itself in or out before I can see the web page or other application I want. It doesn't help that I mainly use it on a laptop, where the drive is slow and the RAM is a fairly large 384MB, but it also happens on my home desktop, where the drives are faster and there's 640GB of RAM, which ought to be enough for anybody.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Zero. ZERO.
Zero swap. Buy enough ram, deactivate swap, watch your computer run as fast as it should.
Making laws based on opinions that stem up from false informations leads to witch hunts.
I never use swap (for about six years) and had a 256MB ram machine with win98.
Thing is i run into out of memory errors,when running alot of stuff,though rarely(windows takes 35MB by itself here).Now with 512MB i could run practically anything.
My advice:Turn off swap,buy more ram.