Slashdot Mirror
Is Swap Necessary?
johnnyb writes "Kernel Trap has a great conversation on swap, whether it's necessary, why swapless systems might seem faster, and an overall discussion of swap issues in modern computing. This is often an issue for system administrators, and this is a great set of posts about the issue."
One can have 1GB of RAM for a fairly cheap price.
I really doubt that majority of newest desktop PCs need to swap on the HD at all.
The unused/used portions argument from the article isn't quite true. You don't have to swap every unused bit,
if you have enough RAM, leave everything there. It's R-A-M. don't access parts you don't need.
If you don't have them in the RAM, read them from the drive,
don't waste time putting them where they mostly are in the first place.
I'm willing to bet that people who need performance, don't often run 10 applications at the same time. If they do, they
surely know what are they doing.
IMHO the average user should get enough RAM and no swap, let the OS optimize things a bit.
When I was running Linux on my 350 mHz Pentium II with 128MB RAM, you can dang well bet I wouldn't have made it without a swap partition. I probably would have gone back to Windows if swap hadn't existed.
You could make a big ramdisk and swap to that!
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
All the docs on Linux and swap amounts to use are from the days of 386s and 4 megs of ram!
I want to know how much swap I should REALLY be using for a system with 1 gig of ram.
Same for some of the kernel compilation docs. Maybe on a 4 meg system compiling that extra option might cause slowness but on a 500 meg system does an extra 30k in the kernel matter?
Can we get some docs that aren't from the mid 90s!
People like to claim that swap can always improve performance, by swapping out unused sections of memory, allowing for more memory to throw at apps or disk cache.
Well, *most* apps won't just arbitrarily consume memory, so endless amounts of memory won't help. And disk cache gets you greatly diminishing returns.
One of the machines I use has 3 gigs of memory. It will swap out unused programs, in an attempt to free up more memory. The joke is that it simply can't use all three gigs. After half a year of uptime, there's still over half a gig completely unused, because the apps don't take memory, and there's not that much to put in disk cache.
Obviously, that's a pathological case. And there are pathological cases at the other extreme. But as memory prices keep dropping over the long run, swap does become less and less useful.
steve
Oh, you're not stuck, you're just unable to let go of the onion rings.
the rule is swap should be 1.5x your RAM! ;)
actually MS followed this rule, in win2k, the default swap size is set to exactly 1.5x your ram, was 176 for my 128mb system, and 384 for my 256mb system, not sure about XP though, someone fill me in
(yes, some great minds working at MS)
Marge, get me your address book, 4 beers, and my conversation hat.
I ran linux without a swap file on 128 MB of memory a couple of years go. It was an accident, I didn't create a swap partition. I never had a problem (forutnately). Of course, I wasn't doing the heavy duty stuff I am now (scientific computation).
Error 404 - Sig Not Found
If you've got 128MB of RAM you have plenty and therefore will have no need for swap space. I mean, isn't 640k enough for everyone?
As long as users can eat up more memory than they have available, and as long as hard drive space is cheaper than RAM space, swap will always be necessary.
Swap improving performance... yeah. On slow systems and low memory, every byte freed up helps. But not swapping in the first place is good too.
I'm now expermineting with replacing various tools with smaller versions, such as dropbear, udhcp, tinylogin, and buzybox. I'm also slowly writing up a "exec and restart shell afterwards" utility called PivotShell.
Hardware wize, I have swap on a CF drive. 32 megs so far, but if I can afford larger CF drives, I'll format 'em as swap and use them.
Why all of this? 40 megs swaps to HD, and on a laptop, any HD access sucks battery power. When you're using Xfree (or even Kdrive) and Firefox, you're going to swap. Period.
--
# Canmephians for a better Linux Kernel
$Stalag99{"URL"}="http://stalag99.net";
Sometimes, when a process goes haywire, it will start munching RAM. If important programs like, say, sshd or X, can't malloc when they need to, they'll die ignominiously. Swap gives you the chance to kill the rogue process before your OS goes kaput. Its slowness can actually help for this.
but today's production, heavily loaded system will still need the ability to swap to/from disk.
Already, there are systems that minimize that need, set-top boxes, embedded systems in general. But each of those is seriously modified (kernel-wise, mostly) to achieve the responsiveness, the frugality of resource treatment that a general purpose desktop computer can't expect to enjoy.
That doesn't mean that developers should stay in the same rut, assuming that hardware that confined system design in the '60s, '70s... '00s will perpetually assign similar constraints.
IMO, desktops still need to swap... for now. but let's not paint ourselves into a performance corner.
Notice how sluggish the system is after doing something disk-intensive like watching a movie. That's because the kernel is caching as much of the movie as possible to memory and swapping your running apps out. And kernel developers think this is a good thing, so it isn't going to change any time soon. IMHO for a desktop system this makes no sense, that's why I run my 1GB RAM machines with zero swap.
I use my gmail account as my swap partition. It's fully searchable and displays helpful advertisements every time I load fifty tabs in Firefox and OpenOffice goes idle. I don't know what I'd do without it. I'd probably be less of an unfunny jackass.
I also reply below your current threshold.
Does anyone out there want to run a series of benchmarks with a few standard applications to prove/disprove whether disabling swapping improves performance?
I'm tired of just hearing antidotal evidence on this. Everyone has their stories about turning off swap files and improving performance, but in what cases? Are there some users this would harm?
As I RTFA & previous comments here, I was rather suprised at how argumentive people were getting over this. Some people are saying swap is an absolute necessity & a swapless system was a broken system, while other's said swap was an obsolete solution to a problem that no longer exists (expensive RAM). This seems odd to me, because as far as I can tell, the decision of whether & how much swap to use is based mostly on two things: specific situations (and thus there is no general answer to 'Is Swap Necessary?'), and opinion. And either way, with the Linux kernel today (and for quite a while now), I can choose for myself whether or not, and how much, swap I want to use. So if I am in a situation that I think requires swap, I can use it, and in a situation that I think would be hurt by having swap, I don't have to use it. So I don't see why there's so much hoolabaloo about this: nobody is forcing anyone to do it one way or the other. And if someone else thinks it should be done different from how I would do it, that's their decision, not mine.
Join moola.com, play games to earn money.
Most applications today have unnecessary or rarely used portions of code or data - bloat. These get swapped out first. Also there are various memory leaks here and there, which means the programs sometimes forget to release allocated memory they do not need any longer.
Look at the size of your X server, or mozilla, or apache, or pretty much anything else and you will see over the course of a few weeks that it has grown beyond reasonable operation demands.
The memory lost this way is never accessed from there on, but the system cannot release it without the program telling it to, so it does the next best thing and shoves it in the swap. Not a solution since eventually swap gets full, but since the leaks are slow to begin with, at least it prevents them from affecting system performance too early.
The potential speed increase isn't seen when comparing 1G RAM vs a 2G RAM system. Its comparing a 1G RAM system with a 1G RAM system with swap.
The gist of it is: with swap you can put things that aren't being used (like mingetty, gdm, etc) into swap to free up space for things that are running now. Without swap you have to keep the little-used processes in memory and you don't have as much 'free' space to use for things like caches.
Its also important to note that the kernel will swap out code segments regardless of whether or not you have a swap partition: they get swapped out to nowhere. When they need to be swapped back in, the executable file itself is read.
In the average case code and data _do_ tend to be accessed more than once. We would all be complaining a lot more if the kernel NEVER cached... remember the huge performance boost SMARTDRV made in DOS?
So, frankly, the default kernel behavior is right.
To fix the movie/updatedb/jumbo cp/etc issues see "man madvise" and check out MADV_DONTNEED. I am hoping applications will start using this syscall sooner, rather than later. The Linux VM can take a hint, and it's pretty easy to give it one.
In the 90's, I ran a 10 line BBS on an Amiga 4000 with 16 megs of Fast ram, 2 megs of Chip ram, and 0k for the swap file. :)
I know, I know, the Amiga didn't HAVE virtual memory. Well actually it did if you had an 040 and installed a memory management program such as GigaMem, but so few people had a use for such a thing that it was practically unheard of.
Oh, and before someone jumps in saying that I wasn't able to do anything else, that is totally NOT the case.
Very often I was doing lots of stuff. The difference is developers were used to working within memory constraints, and now days developers are used to systems growing into the applications.
"Everything you know is wrong. (And stupid.)"
Moderation Totals: Wrong=2, Stupid=3, Total=5.
I haven't seen a case where disabling swap actually increases performance. I have however seen lots of cases where disregard for logic involving swap space caused serious performance problems. The old 1.5x and 2x rules for swap space are outdated and even dangerous in today's systems with ooglebytes of memory.
With less than 128MB of RAM you practically need 2x your physical memory worth of swap space. Running a full GUI environment, even a relatively lightweight one, needs quite a bit of system memory. With 64MB of RAM and a 128MB of swap space you'll be able to run a light GUI environment but have a crappy filesystem cache. The system will crawl but it won't get constant OOM errors if you're not overzealous with your app usage.
The 2x RAM rule on a system with 512MB of physical RAM on the otherhand is excessive. With 1GB of swap space most of it will end up empty unless you're running programs needing huge amounts of allocated memory. With more than 512MB or more of physical memory on a single user workstation you're pretty unlikely to run into situations where active pages are swapped out to disk.
I've seen the runaway process situation crop up on more than one system with excessive amounts of swap space. Since swap is so slow it can be troublesome to kill a process that is using so much memory that it ends up having active pages swapped to disk. The system ends up spending 99% of its time trying to handle the disk IO from the heavy swapping which can make the system totally unresponsive for local and remote users. Because the systems had way more swap space than was logical the offending processes never got OOM errors even though they were using up almost all of the system's resources.
I've pretty much set 256MB as the upper limit for my systems with 256MB or more of physical memory. That is enough swap space to hold any dirty pages or unused processes but not so much that a runaway process is going to eat up all my disk IO for a couple of hours. Once a system hits the 256MB threshold I toss out the silly 2x RAM rules for something with a little more cognitive thought.
I'm a loner Dottie, a Rebel.
If you've got kernel 2.6 you can change the "swapiness" to fit your needs/desires. People with lots of RAM could experiment by changing the swapiness value to 0 and report back with the results (be easier than installing a system without swap).
I've built many servers, embedded systems, and even desktop systems that don't use any swap at all. Many more I limit the amount of swap greatly. The overall responsiveness is much better if you don't use swap and I find system stability to be better. Really it doesn't matter what the systems are used for or how many apps are being ran.. it's just how much memory you're going to use compared to the amount of physical memory you can afford. You can run out of memory just as easily using swap as you can while limited to physical memory.. the main difference being that the recovery of the sitution is much worse in the case of using swap. Quite often the system starts to churn and then grinds to a halt. Without swap those tasks just die and everything else keeps running. Setting memory limits on tasks is a good way of ensuring which tasks are killed first but I'd like to see better control of this given to the admin.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
there's a definite pattern with regard to swap in the windows world.
for win'9x: use up ram until almost gone then start allocating swap space in anticipation of actually using it. should memory allocation still be increasing then actually use swap space. reverse the order when freeing memory.
i had 384 megs ram at the time and as long as i used less than about 350 megs total the system wouldn't be in swap.
for win 2k & xp: (when within physical ram limits) whatever amount of memory is requested, allocate between 60-80% to ram and the rest to the swapfile. even the disk cache partially goes to swap! i didn't believe it at first but all one has to do is look at the numbers in the task manager's memory/cpu window. at first i figured that all i'd need to do is throw in some more ram and the disk thrashing and absolute crawl would go away. i put in a gigabyte of ram (i never allocate more than 700 megs at most and the total system memory usage on bootup is 100 megs). even with the extra ram the problem stayed the same.
turning off swap gives me consistent fast performance, and since the disk cache isn't swapped (partially) i get 2x the throughput i had with a swapfile on large file copy operations
machine tested: duron 1.3ghz, 1 gig pc133 ram, 2x 80 gig wd800jb hdd.. os win2000 & winxp running newsbin which allocates disgusting amounts of ram in a large header grab (yeah i could have used a test program but why do that when newsbin is a real-world test for me). the os and applications are on different drives on their own ide chains
with swapfile enabled (size=1.5x system ram).
allocation time: unaffected, only the time to perform task reqested
memory de-allocation time: (by either quitting app or selecting another group) 23 MINUTES of constant disk thrashing
with swapfile DISabled
allocation time: unaffected, only the time to perform task reqested
memory de-allocation time: (by either quitting app or selecting another group) 2 seconds
if you want people to think you know what you are talking about, just put ".com" at the end of everything you say.com
Here's a real-life example of why swap is useful. One machine I manage has a gig of ram. At the time of purchase, that seemed quite reasonable. But the users are working on a project that takes 2 gig of ram. So currently it's using a gig of the swap. Yes, that's bad, and I'll be adding a second gig to it in a few days (it's in the mail). But in the mean-time, that swap space is really handy. It means the users can get their work done! Think of the first 256M of swap as being for speed. If you're regularly using more than that, then it's time to order more ram. But it's nice to have the spare gig of ram for odd jobs, or while you're waiting to install it.
I'm no expert, but I think a lot of these arguments could be resolved if people took advantage of the ulimit constraints. If you can limit how much a program can get out of control, then there's no longer a concern for a single user sending the server into swap hell. One of my current projects is to figure out reasonable limits.
Bill Gates never made the infamous "640K... enough for anyone" comment. Not only have I never seen it documented anywhere, but he was asked about it and replied that he never said that.
He didn't see the Internet coming -- he thought MSN should be like CompuServe, because that was the top info service (before the Internet became big). And I remember some wild comments he made about the truly amazing, throbbing power of the 286 chip. So he's not an amazing guru with awesome predictive powers. But people keep beating him up about this bogus quote, and I'm tired of it.
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Sheesh, evil *and* a jerk. -- Jade
> Pan is an awesome program, but seriously...when it can single handedly use > 1GB of RAM just stealing divx rips...
Think of it as a sin tax.
Sheesh, evil *and* a jerk. -- Jade
Apparently a new feature (mentioned by a network engineer workmate), is to have the IOS reserve a portion of memory for administrative tasks (like supporting the login process and configuration shell).
A feature like this, that "reserves" a portion of RAM so that if something really fubars your system, you can still login to fix it - would be great for Linux/BSD.
Man watching 6 MSCE's around a sun box, looks alot like the opening scene's of 2001:space odyssey...
For the kinds of complaints about Linux swap I've been seeing of late, it would be bogus to call swap the issue, really. People looking to eliminate swap entirely on desktop machines are cutting off the arm for the sake of a finger.
The issue with swapping in a desktop system is that perception of system responsiveness is almost as important as real performance, and swapping in (actually, it's paging in, but that's semantics) causes high latecy. This is especially noticeable when returning to an idle machine. So we want to cut latency.
People say "the kernel shouldn't swap unless it can't fit everything it needs in system memory." Duh! And it doesn't! It's swapping to increase the size of the file cache, a huge performance win. If the file cache gets too small (say, because this Wal-Mart PC only has 128 megs of RAM, and you've turned off swap, so Moz is eating it all) then you wind up with disk seeks for harddrive intensive applications, causing the same latency as swap.
What's clear to me from these complaints is that the file cache isn't smart enough. People with lots of RAM want to cut down on all these disk reads - that's why they got gobs of RAM. (Ain't it funny that the same Linux heads who say that Linux makes a little machine fly also say that a desktop has no reason to have less than 512MB or 1GB of RAM). At the same time, smaller machines should still be supported, and even folks with gobs of RAM don't want to elimiate swap, otherwise disk bound apps suffer the same latency they're trying to eliminate.
The Linux file cache seems too aggressive for most users. Ext2 loves a file cache like no other filesystem, and this probably influenced the design. If the file cache can be smarter about when to swap to grow itself, and when it should just be content to use up all available system memory, then lots of these latency issues can be fixed in a way which will scale across both hardware and multiple use environments.
A swapless system won't be faster for the same workload, usually the contrary, in fact, since lack of swap denies the system the opportunity to optimize RAM hit ratios. What a swapless system can do is force admission control on new processes in the system, thus enforcing a no-overcommit policy on RAM, and therefore increasing responsiveness at the expense of global throughput.
Swap thrashing in a desktop environment is usually the sign of a workload that is too high for available memory, e.g. trying to run far too many apps simultaneously. No amount of OS smarts is going to compensate for overbooking RAM with too large a working set. The solution is to increase RAM or not run as many apps simultaneously.
Swap thrashing in a server environment is usually the sign of improper server configuration. Naive administrators configure too many processes, thinking they will avoid a bottleneck if all server processes are busy, but all they achieve is turning RAM into the bottleneck rather than the server processes themselves. If you have a web server and configure Apache to have too many running processes, these processes will spend their time contending for RAM instead of doing useful work. Too many cooks spoil the broth. A swapless system would prevent excessive Apache processes from starting in the first place, thus alleviating the problem (at the expense of high error rates, which is probably not acceptable), but performance won't be anywhere as good as a system with swap and properly sized Apache process limits.
Swap is not a panacea. It should not be used to protect against runaway processes (setrlimit is here for that). It is useful in absorbing sporadic spikes in traffic without causing denial of service, and to shunt away uselessly allocated virtual memory (ahem, memory leaks).
As for the idea of putting swap on a RAMdisk, it is completely brain-dead (unless you have exotic memory arrangements such as NUMA) - the kernel is going to waste a lot of time copying memory from the active region to the ramdisk region and back. A straight swapless system will be preferable.
There is no hard and fast rule for sizing swap, it depends on your workload, such as the average ratio of RSS to SIZE. The usual rule of thumb is between 1x and 2x main memory.
I don't know if Linux works this way, but...
UNIX kernels have assumed the availability of swap for nearly 35 years. You cannot remove this major architecutural feature without unintended side effects.
Now with more advanced filesystems that can be mounted synchronously, using a swapfile is less of a problem -- it certainly could be done, but you still get a performance hit by having to manage a filesystem entry, rather than swapping to raw disk.
BTW, a number of databases use raw disk for exactly this reason -- to avoid the performance hit of managing a filesystem. And yes, it will make a difference to the overall performance of your database.
Once in a while I'll do something like 'grep -r "oops" /big/filetree'. The fact of the matter is that I'm probably only reading any of that data ONCE, and it's not going to all fit in memory anyways, so I don't even gain anything if I run the grep a second time.
In a situation like that, I'd like to have some sort of 'nocache' directive that says 'Don't waste the cache with this'.
Something else that might help would be to have some sort of 'minprog' directive which would tell the swapper that a certain amount of space is reserved for 'program' data (i.e. code (including shared libs) and data), -- and that that memory shouldn't be swapped out in favour of something otherwise being read from disk. I think that this might avoid the situation that I sometimes run into of a large program (mozilla/gimp) being unresponsive after I do some other disk-intensive task (like the aformentioned recursive grep).
Things like the OS enforcing things like the RSS rlimit hints would also help. (I hadn't previously realized that it didn't).
Free Software: Like love, it grows best when given away.
My boss started worrying that we weren't going to be able to deliver what the company had contracted to deliver. He was the antithesis of a PHB and so he sat down and in a few hours wrote a small driver to emulate the overall task the project had to accomplish. No detail, just broad brush emulation. He was able to demonstrate with a few lines of code that nothing we could do would hit the delivery spec. Burroughs responded by doubling the amount of RAM on the box as well as installing RAM that was twice as fast as what they had initially delivered. The combination enabled us to turn off swapping and deliver a working product.
Fast forward to 2004 and I'm working on Excel spreadsheets that have 60-70 sheets in a workbook. Saving the book is a bitch - 15-20 second wait after I hit ctrl-S. Every so often, Excel just goes away as it performs a prophylactic background save just in case Excel dies. 15-20 second pauses because the software has become so bloated that saving a 2-3 meg document is an excuse to flog the poor drive into a seek frenzy. The drive, which was about 4 years old, finally gave up the ghost. Its replacement has an 8 meg cache separate from the 512meg Windows manages - that "little" 8 meg junk of RAM belongs to hard drive alone. Night and day performance difference. The Excel swap frenzies that were induced by a simple ctrl-s are gone. 3 meg documents save in under a second - just what you'd expect from a drive that has a transfer speed in excess of 60 mbytes/sec.
My sense is that swap has always been a kludge. It's an attempt to squeeze more data into a machine that has only so much space. The working set graphs look pretty but they seldom describe what is happening day to day. Trading 2 nanosecond response for a 5 millisecond seek is seldom going to be a good trade. Bottom line from that OS class 35 years ago? Keep your working set size less than your physical memory and your machine will remain responsive. Just what the old IBM Geezers were saying in the first place.
Whilst that's strictly true, some modern languages use generational garbage collectors that segregate objects in memory according to age. Only when an age group gets full do they sweep through an age group, and move any surviving objects up to the next age group.
This heuristic works exceptionally well, and runs fantastically quickly, and triggers significant swapping hardly ever.
There are some circumstances where it runs slowly, but in the worst case the performance is similar to simply doing a full garbage collection. These situations are pretty rare; objects generally segregate very well into young/old or young/middle aged/old categories- the vast majority of objects die very young.
Sad isn't it.
-WolfWithoutAClause
"Gravity is only a theory, not a fact!"Use a swapfile instead of a partition. 2.6 kernels cache the location of the file, so there's no performance hit for swap files compared to swap partitions. I'll give a quick HOWTO:
/var/swap /etc/fstab for: /var/swap none swap sw 0 0
/proc/meminfo and see how much you're using when you've got the system strained, use twice that amount, not less than 128MB.
1. decide how much you want (you can change it later, I have 128MB on all my boxes with over 512MB RAM). The example uses 128MB
2. #dd bs=1M count=128 if=/dev/zero of=/var/swap
3. mkswap
4. edit
5. swapon -a
6. There is no step six!
But the best way to know how much swap you need is to peek at #top every now and then, or #cat
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails