Slashdot Mirror
Is Swap Necessary?
johnnyb writes "Kernel Trap has a great conversation on swap, whether it's necessary, why swapless systems might seem faster, and an overall discussion of swap issues in modern computing. This is often an issue for system administrators, and this is a great set of posts about the issue."
When I was running Linux on my 350 mHz Pentium II with 128MB RAM, you can dang well bet I wouldn't have made it without a swap partition. I probably would have gone back to Windows if swap hadn't existed.
All the docs on Linux and swap amounts to use are from the days of 386s and 4 megs of ram!
I want to know how much swap I should REALLY be using for a system with 1 gig of ram.
Same for some of the kernel compilation docs. Maybe on a 4 meg system compiling that extra option might cause slowness but on a 500 meg system does an extra 30k in the kernel matter?
Can we get some docs that aren't from the mid 90s!
People like to claim that swap can always improve performance, by swapping out unused sections of memory, allowing for more memory to throw at apps or disk cache.
Well, *most* apps won't just arbitrarily consume memory, so endless amounts of memory won't help. And disk cache gets you greatly diminishing returns.
One of the machines I use has 3 gigs of memory. It will swap out unused programs, in an attempt to free up more memory. The joke is that it simply can't use all three gigs. After half a year of uptime, there's still over half a gig completely unused, because the apps don't take memory, and there's not that much to put in disk cache.
Obviously, that's a pathological case. And there are pathological cases at the other extreme. But as memory prices keep dropping over the long run, swap does become less and less useful.
steve
Oh, you're not stuck, you're just unable to let go of the onion rings.
As long as users can eat up more memory than they have available, and as long as hard drive space is cheaper than RAM space, swap will always be necessary.
isn't 640k enough for everyone?
people constantly make this joke, but seriously, at the time BG said this, it was probably true.
if today i say "1 gig ought to be enough for everyone" it is true, but in 10 years from now you will be laughing at this.
he never claimed it would 'ALWAYS' be enough (unless there is more to this quote???)
Marge, get me your address book, 4 beers, and my conversation hat.
Does anyone out there want to run a series of benchmarks with a few standard applications to prove/disprove whether disabling swapping improves performance?
I'm tired of just hearing antidotal evidence on this. Everyone has their stories about turning off swap files and improving performance, but in what cases? Are there some users this would harm?
Most applications today have unnecessary or rarely used portions of code or data - bloat. These get swapped out first. Also there are various memory leaks here and there, which means the programs sometimes forget to release allocated memory they do not need any longer.
Look at the size of your X server, or mozilla, or apache, or pretty much anything else and you will see over the course of a few weeks that it has grown beyond reasonable operation demands.
The memory lost this way is never accessed from there on, but the system cannot release it without the program telling it to, so it does the next best thing and shoves it in the swap. Not a solution since eventually swap gets full, but since the leaks are slow to begin with, at least it prevents them from affecting system performance too early.
I've built many servers, embedded systems, and even desktop systems that don't use any swap at all. Many more I limit the amount of swap greatly. The overall responsiveness is much better if you don't use swap and I find system stability to be better. Really it doesn't matter what the systems are used for or how many apps are being ran.. it's just how much memory you're going to use compared to the amount of physical memory you can afford. You can run out of memory just as easily using swap as you can while limited to physical memory.. the main difference being that the recovery of the sitution is much worse in the case of using swap. Quite often the system starts to churn and then grinds to a halt. Without swap those tasks just die and everything else keeps running. Setting memory limits on tasks is a good way of ensuring which tasks are killed first but I'd like to see better control of this given to the admin.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
"Why not more? Because that's the largest a swap partition can be"
Just a side note: you can have multiple swap partitions. [not that you need them, but you can have them].
--- d'oh
Apparently a new feature (mentioned by a network engineer workmate), is to have the IOS reserve a portion of memory for administrative tasks (like supporting the login process and configuration shell).
A feature like this, that "reserves" a portion of RAM so that if something really fubars your system, you can still login to fix it - would be great for Linux/BSD.
Man watching 6 MSCE's around a sun box, looks alot like the opening scene's of 2001:space odyssey...
For the kinds of complaints about Linux swap I've been seeing of late, it would be bogus to call swap the issue, really. People looking to eliminate swap entirely on desktop machines are cutting off the arm for the sake of a finger.
The issue with swapping in a desktop system is that perception of system responsiveness is almost as important as real performance, and swapping in (actually, it's paging in, but that's semantics) causes high latecy. This is especially noticeable when returning to an idle machine. So we want to cut latency.
People say "the kernel shouldn't swap unless it can't fit everything it needs in system memory." Duh! And it doesn't! It's swapping to increase the size of the file cache, a huge performance win. If the file cache gets too small (say, because this Wal-Mart PC only has 128 megs of RAM, and you've turned off swap, so Moz is eating it all) then you wind up with disk seeks for harddrive intensive applications, causing the same latency as swap.
What's clear to me from these complaints is that the file cache isn't smart enough. People with lots of RAM want to cut down on all these disk reads - that's why they got gobs of RAM. (Ain't it funny that the same Linux heads who say that Linux makes a little machine fly also say that a desktop has no reason to have less than 512MB or 1GB of RAM). At the same time, smaller machines should still be supported, and even folks with gobs of RAM don't want to elimiate swap, otherwise disk bound apps suffer the same latency they're trying to eliminate.
The Linux file cache seems too aggressive for most users. Ext2 loves a file cache like no other filesystem, and this probably influenced the design. If the file cache can be smarter about when to swap to grow itself, and when it should just be content to use up all available system memory, then lots of these latency issues can be fixed in a way which will scale across both hardware and multiple use environments.
A swapless system won't be faster for the same workload, usually the contrary, in fact, since lack of swap denies the system the opportunity to optimize RAM hit ratios. What a swapless system can do is force admission control on new processes in the system, thus enforcing a no-overcommit policy on RAM, and therefore increasing responsiveness at the expense of global throughput.
Swap thrashing in a desktop environment is usually the sign of a workload that is too high for available memory, e.g. trying to run far too many apps simultaneously. No amount of OS smarts is going to compensate for overbooking RAM with too large a working set. The solution is to increase RAM or not run as many apps simultaneously.
Swap thrashing in a server environment is usually the sign of improper server configuration. Naive administrators configure too many processes, thinking they will avoid a bottleneck if all server processes are busy, but all they achieve is turning RAM into the bottleneck rather than the server processes themselves. If you have a web server and configure Apache to have too many running processes, these processes will spend their time contending for RAM instead of doing useful work. Too many cooks spoil the broth. A swapless system would prevent excessive Apache processes from starting in the first place, thus alleviating the problem (at the expense of high error rates, which is probably not acceptable), but performance won't be anywhere as good as a system with swap and properly sized Apache process limits.
Swap is not a panacea. It should not be used to protect against runaway processes (setrlimit is here for that). It is useful in absorbing sporadic spikes in traffic without causing denial of service, and to shunt away uselessly allocated virtual memory (ahem, memory leaks).
As for the idea of putting swap on a RAMdisk, it is completely brain-dead (unless you have exotic memory arrangements such as NUMA) - the kernel is going to waste a lot of time copying memory from the active region to the ramdisk region and back. A straight swapless system will be preferable.
There is no hard and fast rule for sizing swap, it depends on your workload, such as the average ratio of RSS to SIZE. The usual rule of thumb is between 1x and 2x main memory.
Intelligent memory managment only affects performance if you have swap space. Swap space could be defined as storage which is slower than main memory. If all your storage is the same speed, memory management is trivial.
Ironically, I have the same challenge for theorem #1 that you used for #2. #1 states that having swap is better than not having it. Clearly having swap increases the amount of allocations programs can make before their malloc's fail. But improve performance? That is only true if the OS can predict what data is needed for the future operations. If it predicts wrong, the usage of swap can degrade performance.
Of course, the Kernel will have no idea about watching movies, but it stil can distinguish this "unimportant" data from data that do needs to be cached. The most important way to distinguish this data from data that does need caching is how fast it is needed in the first place.
When I do a grep on the kernel tree, you'll find that your harddisk speed is bottle neck; it is worth caching this data as grepping from memory would enhance its performance.
When I play a movie, the harddisk is not the bottleneck, in fact, NOTHING is a bottle neck, as my movie would be stuttering and unwatchable otherwise. This data is not worth caching (atleast not worth caching so much of it to the point of swapping out all else).
This goes for most media streams, but also for interaction with the internet (downloads/uploads/p2p). There's no need to keep a 1 GB file cached when that file has "accumulated" cache space in the course of an hour or more (ie, slow I/O); if it was important enough to warrant caching, I'd think the harddisk would have been the bottleneck in the first place...
--Swilver
Surely if your system runs out of RAM it shouldn't die? The runaway process, sure, but the OS should be able to reclaim some RAM from that and manage to carry on, no?
When you're DMAing large amounts of memory, memory fragmentation becomes an issue.
This is why the 'bigPhysArea' patch to the kernel exists-- to create continuous bloakcs of memory which can be transferred without having to scatter/gather.
Note that this is independant of memory -usage-.. this is an issue with the 'block size' if you will, of memory segmentation.