Slashdot Mirror
How Much Virtual Memory is Enough?
whitroth asks: "Ten years ago, Received Wisdom said that virtual memory should be, on the average, two to two-and-a-half times real memory. In these days, where 2G RAM is not unusual, and many times is not that uncommon, is this unreasonable? What's the sense of the community as to what is a reasonable size for swap these days?"
lots
Under Windows it seems it'll swap out whether the free RAM is needed or not, no matter what (there's a registry setting to change this though). Under Linux, you won't swap much anyway unless you need it.
I run a Core Duo laptop with 1GB of RAM and have never swapped out in Linux, no matter what I was doing.
"You can either have software quality or you can have pointer arithmetic, but you cannot have both at the same time."
2X physical memory for under 2G RAM
2G swap for up to 8G RAM
+1G swap for every 4G RAM beyond that
not creating a swap partition at all is a bad idea, imo...
you never know when some runaway process is going to eat all yer RAM and need to use swap... no matter how much RAM you've got.
I typically just make a 1 or 2 GB swap partition since I've got more than enough space to spare. I mean, back in the days when 128MB of RAM was considered a lot, and a 5GB drive was considered huge, no one would consider using 20% of their storage space for swap. Now, it's not unusual to have 300GB of storage, so what's 1% of that being used for swap?
I've also got a serious collection of 2-6GB harddrives kicking around, now, so I've been using them for swap. It's really pointless to have a 4GB partition for data, so I just use the entire 6GB drive for swap on some machines.
my primary server right now has a 4GB swap partition and 1.25GB of RAM... a piece of bad AJAX code that ran overnight wound up using all the RAM and had some seriously detrimental effects on the performance of the server. it took 25 minutes to ssh in in the morning and when I finally got in, I found that the load averages were at over 100 (I've NEVER see that before).
my point is that even if you have a LOT of RAM, it's still handy to have some spillover available.
...spike
Ewwwwww, coconut...
I use 4x750 GB hard drives (RAID), purely for virtual memory. It increases the speed on the RAM preprocessing directive, but demodulates the core processing utility monitor. I find it to be a good setup, especially for running Naibed Linux.
Stupidity is like nuclear power, it can be used for good or evil. And you don't want to get any on you.
One of the real advantages of using swap isn't to avoid memory exhaustion at all; by moving infrequently accessed pages from memory you make more room for the disk cache, thereby possibly improving overall system performance by reducing hard drive reads.
If you've got a 300GB primary drive, it's foolish to use a 5GB drive for your swap. While you gain the benefit of having that drive separate from the primary (and potentially not contending for the bus), those drives are so far apart technology wise that you'd probably be better off with a swap partition on your most modern disk.
That 2/5/6GB drive may have a 20MB/s sequential rate at OD and half that at ID. Modern drives more than double that sequential performance (or triple), which is what's critical when swapping in/out a large job. Many drives in that generation don't support UDMA either, and talk with PIO, meaning you get no data checksum on your transfers.
You can span generations when you're using a cost reduced modern drive (fewer heads, same formats) but the drive that was stretching to make 5GB across 6/8 heads will be a real POS compared to modern drives performance wise.
Thrashing is bad, but thrashing to a slow disk I'd think would be worse. It is even compounded since that 5GB drive is probably PATA, meaning you're going to have your swap drive and primary drive sharing a cable, which will basically nuke most of the savings of 2 disks since they'll be reselecting master/slave at almost every command.
More data, damnit!
Linux has futzed with this a lot (and lets the user tweak VM behavior a lot, but /proc/sys/vm/swappiness goes a long way...), both linux and Windows will swap well ahead of not having free memory (for good reason). Just wanted to go into detail because I keep seeing people complain that they see swap used in linux or windows when they still have free memory, not realizing this isn't a bad thing generally.
There are generally two strategies:
-The common-sense one where you swap when you run out of memory. This makes a lot of practical sense on systems with limited write cycles (flash based swap, though you really never ever should do that anyway), and systems that want to spin down drives to conserve power for battery conservation. Performance wise (this may surprise people who haven't spent time thinking about it), this can often be bad. Avoiding swapping is generally only good on systems where resource utilization is carefully managed and you know it won't swap ever (the IO operations of unneeded can interfere with the productive activity of a constantly busy system). This is actually a vast minority of systems in the world (no matter how l33t one may think themselves, they most certainly don't have a usage pattern that would be impacted by the extraneous IO operations of occasional write to swap.
-Pre-emptive swapping. When the IO subsystem is idle and the system can afford to copy memory to swap area, it does so (depending on criteria). Generally speaking it will select memory not accessed much and write it to disk, but leave the memory copy in place if the physical memory is not immediately needed. A fair amount of swap used in an apparently underutilized system is duplicated in physical memory and swap space. The benefit here is that if the process reads back that memory, it doesn't incur any penalty in reading it back despite it being also in swap (the system may make certain decisions on what is the best swap candidate and write to disk different data). The benefit of writing this stuff to swap even when not needed is clear when an application comes along that allocs more memory than the system has free in physical space. In the first strategy, this means the malloc blocks while data is written to disk, and the new application starting or needing a lot of data is severely impacted. In the pre-emptive swap case, system notices the condition, knows what memory it has a backup in swap of that hasn't been used lately, and can free that memory and satisfy the malloc pretty much instantly.
To those who have 1GB of RAM or so it becomes less likely that the system will have to flush memory from physical RAM, but there is a balance to be struck between memory used directly invoked by applications, what the application memory access pattern is, and what ram you can use to buffer filesystem access. If your total application memory allocation is 75%, it still may make sense performance wise to only keep 50% of your physical memory dedicated to the applications, (the other bit relegated to swap), and 50% of the memory to buffer disk I/O.
XML is like violence. If it doesn't solve the problem, use more.
One thing to consider is whether or not you're using tmpfs for /tmp. For performance, I recommend using tmpfs for /tmp, and basically treat the swap partition as your /tmp partition. It may seem counterintuitive, "why would it be faster than a filesystem when it's backed to disk anyway, and my filesystems caches just file if need be?" The answer is that tmpfs never needs to worry about consistency. On the kernel.org machines, we have seen /tmp-intensive tasks run 10-100 times faster with tmpfs than with a plain filesystem. The downside, of course, is that on a reboot, tmpfs is gone.
Man, it's utterly depressing to see the same useless "rules-of-thumb" still in effect when the original question is asking if the rule-of-thumb is a good idea.
1) Page space is not swap space. There's a small distinction that's generally lost (and generally ignored). Page space is used to move memory pages to and from disk. Swap space is technically to move entire processes out to disk. The difference is mainly based on when your OS was created (i.e., technological underpinnings) and no need to get into it now... but the difference is meaningful.
2) Page space is not *free*. There's a misconception that if you have 500G of disk space then "how does it hurt" to put 8G of swap on 4G RAM. Depending on your OS, the size of the page table can grow remarkably depending on how much memory (RAM + VM) is allocated. This means that adding 2G of page space may not cost anything, but adding 2.5G may suddenly take up another chunk of real, non-pageable memory because the page table cannot itself be paged. This means that if your app is thrashing, then adding page space may make it worse.
3) Even with lots of RAM, it's still often a good idea (depending on your usage) to have some page space. Modern OSes will still page out unused pages to use RAM for better stuff. I.e., if you have a huge file open in a graphics application, but are not actively using that application for a length of time (an hour, say) then the OS will page it to disk. This makes better use of your physical RAM. On some OSes the OS will use page space even if free RAM is available. It can then toggle a page out by flipping a bit in the page table and not have to do an expensive write.
4) In some systems you can overcommit memory. Applications tend to request a lot more memory from the OS than they'll actually use. This is useful in many instances but it again depends on your usage. If you're running a single application that doesn't dynamically allocate memory then you can run pageless. If a new app requests memory that's not available then it will get a failure on malloc request. This can be desirable in some circumstances.
5) There are benefits to running page space on a separate disk, but for the vast majority of home users, the difference is negligible. This applies to Windows and Linux. Once you start stressing the VM subsystem then a separate disk is highly desirable.
6) You can create page files on Unix/Linux. It's not desirable generally because of the extra filesystem overhead and possibility of fragmentation. But hey, in a pinch it works.
7) Why this 2x RAM rule? A lot of it comes from old VM subsystems that needed a "picture" of the entire memory space. This made the page-out algorithms easier to code. Newer algorithms don't require the 2X RAM.
KL
There's no real hard and fast rule anymore. And setting it against a static value (like physical memory) is incredibly wasteful.
It's a much better idea to set it interactively. Use the system without adjusting the Virtual Memory for a while. Then take a look at your usage and set your virtual memory against that usage.
For instance.
If you're in a Windows machine, let it run normally for a few days.
Run everything the way you normally use it.
Multiple apps, multiple instances, games out the ass, everything.
Then open up the Task Manager and look at the Performance tab.
Take a look at the Peak value under "Commit Charge".
Set your virtual memory, min and max, at about 10% above that value to leave yourself a little headroom.
Normally this will be enough to deal with your maximum swap requests.
If, somehow, you begin bumping against virtual memory limits again AFTER that, bump it another 10%.
If you still have problems, keep bumping it in 10% increments, and start looking for apps that are memory leaking.
Chas - The one, the only.
THANK GOD!!!
"Memory is like an orgasm. It's a lot better if you don't have to fake it."
-- Seymour Cray, on virtual memory.
You are not entitled to your opinion. You are entitled to your informed opinion. -- Harlan Ellison
We've found that 512 Megs of swap is more than enough for our 2 and 4 Gig machines. Why even have swap? Here is an example:
1) On a system with zero swap, when apache gets slammed (say you get to the top of digg or slashdot), apache starts consuming lots of memory to handle new inbound requests. When it runs out, the machine grinds to a halt because it can't allocate more and requires a power cycle. (Setting a low max children really only helps if you are happy denying traffic to the people who are trying to see your site...it's best to plan for capacity and put quite a few servers load balanced).
2) On a system with any appreciable swap (IMHO, more than 128 Megs, up to 512 Megs), if you're monitoring the system (watch -n 1 df -h, for example) and all of a sudden it starts using swap, the machine is on the edge of dying. This gives you an early warning that maximum machine performance/throughput is occurring. You can restart apache or shut it down or similar, you can do something to temporarily lower or remove load from that machine. This doesn't give you *much* time, but it gives you some.
In our real world experience, at digg and slashdot loads you have about 10-15 seconds to stop apache once it starts swapping. After that, the performance degrades so bad that the machine becomes catatonic, the same as #1, requiring a power reset (obviously because virtual memory on HD is magnitudes slower than RAM, as numerous others have suggested). The key here is that you must realize that some swap is good for allowing unused programs to be swapped out, such as login terminals that just sit there. It's great for detecting problems, but if your heavy app is the one utilizing swap, your machine is about to crash anyway.