Slashdot Mirror
Is Swap Necessary?
johnnyb writes "Kernel Trap has a great conversation on swap, whether it's necessary, why swapless systems might seem faster, and an overall discussion of swap issues in modern computing. This is often an issue for system administrators, and this is a great set of posts about the issue."
the rule is swap should be 1.5x your RAM! ;)
actually MS followed this rule, in win2k, the default swap size is set to exactly 1.5x your ram, was 176 for my 128mb system, and 384 for my 256mb system, not sure about XP though, someone fill me in
(yes, some great minds working at MS)
Marge, get me your address book, 4 beers, and my conversation hat.
Swap improving performance... yeah. On slow systems and low memory, every byte freed up helps. But not swapping in the first place is good too.
I'm now expermineting with replacing various tools with smaller versions, such as dropbear, udhcp, tinylogin, and buzybox. I'm also slowly writing up a "exec and restart shell afterwards" utility called PivotShell.
Hardware wize, I have swap on a CF drive. 32 megs so far, but if I can afford larger CF drives, I'll format 'em as swap and use them.
Why all of this? 40 megs swaps to HD, and on a laptop, any HD access sucks battery power. When you're using Xfree (or even Kdrive) and Firefox, you're going to swap. Period.
--
# Canmephians for a better Linux Kernel
$Stalag99{"URL"}="http://stalag99.net";
Yeah, because God forbid you should have to use the SAME amount of disk space as the ram you have installed to save a few seconds to bring the laptop up :P
Join the TWIT army now!
Linux has two properties that make swap a good thing (TM).
;) and a less recent PC running a Linux distro. The RAM threshold is realistically around the 128-512 MB range. Those who are dual-booting on a brand new machine can use 1 GB, but the rest of us put up with less than that (I for one want to avoid MBR screwups and the hassle of communicating with NTFS, so I don't dual boot. I had a nasty GRUB incident, so I'm probably paranoid).
The first thing to remember is that, for many Linux users, they have a newer PC running Windows (or a Mac
Finally, every Linux user that has compiled a kernel knows that it can really tax a system. Gentoo users also know how strenuous a XFree86 or KDE/Gnome compile can be. Being able to work on another terminal while compiling is one of the most beautiful things about *nix, and to do that on anything with less than 512 MB or 1 GB or RAM you want to have some swap.
And finally, while RAM is very cheap, so are hard drives, and how hard is it to squeeze a swap partition out of a hard drive? Can it really hurt that much to let the system use it?
As for Windows, swap is absolutely required for a lot of the games that are out there. I've heard that Unreal-engine based games in particular make heavy use of swap filing.
If my answers frighten you, stop asking scary questions.
One of the machines I use has 3 gigs of memory. It will swap out unused programs, in an attempt to free up more memory. The joke is that it simply can't use all three gigs. After half a year of uptime, there's still over half a gig completely unused, because the apps don't take memory, and there's not that much to put in disk cache.
Yes, it is swappnig because it is trying to free up "low memory", of which you have less that a gig. This is unfortunately an inevitable failure case of Intel's brain damaged (although practical) PAE schmeme.
Were you to have no swap at all, you may have had to reclaim important kernel side caches (inode, dentry caches), or even had memory alloction failures in eg. the network stack (common).
I don't know if it works for SWSUSP (the hibernation driver), but you can actually swap to a file under GNU/Linux. You can add and remove swap files while the system is running, so you might even be able to add a power management script that creates and formats a swap file right before hibernating, and deletes it after unhibernating.
one reason you'd want swap on a system is to have someplace to dump/savecore information in the case of system crashes. Kind of hard to do with volatile memory.
FreeBSD for the impatient.
The potential speed increase isn't seen when comparing 1G RAM vs a 2G RAM system. Its comparing a 1G RAM system with a 1G RAM system with swap.
The gist of it is: with swap you can put things that aren't being used (like mingetty, gdm, etc) into swap to free up space for things that are running now. Without swap you have to keep the little-used processes in memory and you don't have as much 'free' space to use for things like caches.
Its also important to note that the kernel will swap out code segments regardless of whether or not you have a swap partition: they get swapped out to nowhere. When they need to be swapped back in, the executable file itself is read.
In the average case code and data _do_ tend to be accessed more than once. We would all be complaining a lot more if the kernel NEVER cached... remember the huge performance boost SMARTDRV made in DOS?
So, frankly, the default kernel behavior is right.
To fix the movie/updatedb/jumbo cp/etc issues see "man madvise" and check out MADV_DONTNEED. I am hoping applications will start using this syscall sooner, rather than later. The Linux VM can take a hint, and it's pretty easy to give it one.
I for one want to avoid MBR screwups and the hassle of communicating with NTFS, so I don't dual boot.
The fact that you irrationally avoid two very mature and easy-to-understand technologies implies you are uninformed and compels me to ignore any suggestions you might have.
I've heard that Unreal-engine based games in particular make heavy use of swap filing.
"You've heard"? As in, someone told you to believe it but you didn't take the time to think about the claim yourself? No application can "make heavy use of swap filing". Either it uses a lot of RAM or it doesn't. And either it accesses RAM in a predictable pattern (condusive but to swapping) or not. But an application can't actually make use of swap filing: it always -thinks- it's using RAM. The OS will always decide whether it's time to swap in or out.
My take on the swap -vs- no swap.
I have no data, links or proof to back it up. Only the system responsiveness under two different setups.
Setup A) 300mhz intel, 256mb ram, 6gb hdd, swap 512mb.
Taking the swap out of that setup and the system crawled to a halt, expecially when using X. Re-enabling it and it ran just fine.
Setup B) 2GHz Intel, 1gb ram, 40gb hdd, swap 2gb.
Taking out the swap in that machine and the system ran fine. Even running Half-Life: Counter-Strike via WineX by transgaming. Re-enabling the swap, and I noticed a little performance increase, but I couldn't measure it b/c I didn't know how.
Just my take on it. Swap is generally a good thing in older machines, while in newer systems it isn't a critical thing to have. HOWEVER, I did not run many tests with and w/o swap. just basic use, and Setup B's HL:CS test was only to make sure!
YYMV, KBD.
-zoloto
I haven't seen a case where disabling swap actually increases performance. I have however seen lots of cases where disregard for logic involving swap space caused serious performance problems. The old 1.5x and 2x rules for swap space are outdated and even dangerous in today's systems with ooglebytes of memory.
With less than 128MB of RAM you practically need 2x your physical memory worth of swap space. Running a full GUI environment, even a relatively lightweight one, needs quite a bit of system memory. With 64MB of RAM and a 128MB of swap space you'll be able to run a light GUI environment but have a crappy filesystem cache. The system will crawl but it won't get constant OOM errors if you're not overzealous with your app usage.
The 2x RAM rule on a system with 512MB of physical RAM on the otherhand is excessive. With 1GB of swap space most of it will end up empty unless you're running programs needing huge amounts of allocated memory. With more than 512MB or more of physical memory on a single user workstation you're pretty unlikely to run into situations where active pages are swapped out to disk.
I've seen the runaway process situation crop up on more than one system with excessive amounts of swap space. Since swap is so slow it can be troublesome to kill a process that is using so much memory that it ends up having active pages swapped to disk. The system ends up spending 99% of its time trying to handle the disk IO from the heavy swapping which can make the system totally unresponsive for local and remote users. Because the systems had way more swap space than was logical the offending processes never got OOM errors even though they were using up almost all of the system's resources.
I've pretty much set 256MB as the upper limit for my systems with 256MB or more of physical memory. That is enough swap space to hold any dirty pages or unused processes but not so much that a runaway process is going to eat up all my disk IO for a couple of hours. Once a system hits the 256MB threshold I toss out the silly 2x RAM rules for something with a little more cognitive thought.
I'm a loner Dottie, a Rebel.
If you've got kernel 2.6 you can change the "swapiness" to fit your needs/desires. People with lots of RAM could experiment by changing the swapiness value to 0 and report back with the results (be easier than installing a system without swap).
Thats not true, if the linux kernel gets out of memory it takes the list of all processes, scores them with memory usage, runtime etc. and then simply kills the process of highest score. In your case your RAM munching app. would just be killed by the kernel.
I know that pretty for sure, since I modified that part of the kernel once for an embedded system, where we explicitly didn't want it to kill any process, but have instead to reboot in such a case. Since nothing is worse than having a half-functional system with some processes missing....
--
Karma 50, and all I got was this lousy T-Shirt.
Bill Gates never made the infamous "640K... enough for anyone" comment. Not only have I never seen it documented anywhere, but he was asked about it and replied that he never said that.
He didn't see the Internet coming -- he thought MSN should be like CompuServe, because that was the top info service (before the Internet became big). And I remember some wild comments he made about the truly amazing, throbbing power of the 286 chip. So he's not an amazing guru with awesome predictive powers. But people keep beating him up about this bogus quote, and I'm tired of it.
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Kernel compiles do -not- tax the system. They utilize it. There are relatively few process threads used during a kernel build, substantial CPU use per process, a fair amount of disk reading, and a large amount of memory swapping about and modification.
:)
:)
Its perfectly reasonable to expect a system to be responsive while compiling a large project. That's not a "taxed" system. A slashdotted machine, which spawns thousands of apache processes, is taxed, however.
Except for your statement that for "anything less than 512M or 1G of RAM you want to have some swap"... You want to have swap all the time, unless there's a power-related (ie, laptop) reason to not have it. As the article says, it's trivially demonstrated that there is a performance increase by using swap, regardless of how much RAM. It is, however, imperative to have swap if the machine doesn't have much ram, otherwise you'll run into some nasty results after a while.
I'd agree with your last two sentiments.
~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
IMHO Swap is a good idea and here's why. /tmp is mounted on swap. If you, say, have 1GB of swap space and chuck 512MB of stuff in /tmp, you've now got 512MB of swap left. Lots of Unix software dumps stuff in /tmp and, when there is available RAM, /tmp lives in RAM. This makes temp files very fast.
I admin Solaris systems, and swap on Solaris is a fine thing indeed.
You allocate a complete slice of a hard disk for swap, and you can then add and remove swap dynamically while the system is running. Need 1 GB more swap? Create an empty 1GB file, and add it as swap.
What's more
Plus, the VM subsystem also deals with the file cache so on a Solaris system, you will see the amount of RAM used always around the 100% mark. No point in having RAM there unused, it costs too much. Use it as disk cache.
In addition, when an application needs to be swapped out to disk, why bother writing to disk something that's already there - the application's code is marked as being paged out to disk and removed from RAM and when it's needed again, the code is fetched from the original binary that the application was loaded from.
All in all, these kind of modifications to the VM subsystem mean that swap is good to have and can make systems faster with it than without.
k:.
Specialist Mac support for creative pros, Melbourne
No, kswapd is for 32 bits
open4free ©
Religion is regarded by the common people as true, by the wise as false, and by rulers as useful.
It'd be nice if they had a similar control for filesystem cache.
You're missing the point. That's exactly what the swappiness setting does, indirectly. If you avoid swapping (swappines = 0), the system has less memory left over for filesystem cache. OTOH, if you set swappiness=100, the system has a lot more memory to use for file cache.
The system always tries to use all available memory, and that's a good thing. The question is whether to use extra memory for file cache or for keeping pages in memory.
Now with more advanced filesystems that can be mounted synchronously, using a swapfile is less of a problem -- it certainly could be done, but you still get a performance hit by having to manage a filesystem entry, rather than swapping to raw disk.
BTW, a number of databases use raw disk for exactly this reason -- to avoid the performance hit of managing a filesystem. And yes, it will make a difference to the overall performance of your database.
This whole discussion has ignored the only important point.
... clicking on one of your X windows audibly causes a lot of disk reads and response is very slow.
That's the algorithm for deciding which page to swap out. All systems I know of swap out the "least recently used" page when they need to swap something out.
This is what causes the bad effects caused by swap. Suppose you (like me) have a machine that you use mostly as an X server. But naturally it's also running a few other servers, a MTA, an ntpd, minor stuff like that.
You leave your desktop overnight and when you come back in the morning
That's because your servers write to log files. Of course these writes stay in memory. And when Linux needs to swap something out (which it will do when a server writes to a log file), your X sessions get swapped out, not the useless logfile pages, because the logfile pages were used more recently. This is really braindead behavior. But that's how the swap algorithms, as currently specified, work.
Use a swapfile instead of a partition. 2.6 kernels cache the location of the file, so there's no performance hit for swap files compared to swap partitions. I'll give a quick HOWTO:
/var/swap /etc/fstab for: /var/swap none swap sw 0 0
/proc/meminfo and see how much you're using when you've got the system strained, use twice that amount, not less than 128MB.
1. decide how much you want (you can change it later, I have 128MB on all my boxes with over 512MB RAM). The example uses 128MB
2. #dd bs=1M count=128 if=/dev/zero of=/var/swap
3. mkswap
4. edit
5. swapon -a
6. There is no step six!
But the best way to know how much swap you need is to peek at #top every now and then, or #cat
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
"...the OS should be able to reclaim some RAM from that and manage to carry on, no?"
yeah, pretty much. i may be openly admitting my ignorance here, but i have a gentoo box with 256 MB DDR RAM. i set up a swap partition, and set up the entry in /etc/fstab, but when i ran the box, it never touched the swap. me: "great, this RAM ownz! and the 2.6 kernel ownz at memory management! it never uses swap!"
then, stuff started dying on me during times of heavy system load. like, i'd be in Firefox and running emerge at the same time, and firefox would croak on me. or VMWare wouln't boot Knoppix. or OpenOffice would die. all this time i had something else going on on top of whatever memory-intensive program was dying. me: "wtf, mate?"
it finally occurred to me that the box still wasn't swapping, and that might be a bad thing. so i tried to run "swapon /dev/hdb2" (my swap partition) and got errors. weird. then i realized what happened...
...i had forgotten to officially make the swap partition using mkswap. i'd been running this partition schem for about 2 months, and never realized it until about 2 days ago.
all that to say, yes. the system DOES reclaim memory. by killing other stuff :)
i'm a stoopid monkey sometimes :-D
I saw it on Slashdot, it must be true!
Most commercial Unix systems have had a fully unified memory architecture for a while now; I'd say something like 10 years. Pages of executables and pages of mmap()ed files work the same way: the memory manager just maps them to a virtual address and faults them in and out as necessary. That is to say, the memory for an mmap() file is not loaded into RAM then copied (partially) to swap. Instead, the virtual memory is just backed by the mmap()ed file itself. Even if low memory forces you to evict all the pages of the mmap()ed file, no swap is needed to do this, because they are flushed directly back to the original file itself.
Likewise with binaries: in the old days, you would execute a binary, and this would cause the loader to copy the entire binary into virtual memory, possibly putting some of it on swap. But on modern Unix systems, loading a binary means simply mapping the virtual memory address space directly to pages in the executable's file (at least for the read-only portions of the executable, which are the majority). This has the advantage that if a part of the executable is never run, it's never even loaded from disk. It also means swap space is not needed used, and no copying is done. It also means that two processes can share the *same* page of the executable; even if they are two different users, they can still be executing out of the same page of physical memory because the code is read-only. (Of course, this only applies to the read-only pages of an executable, and not all of them are read-only. But the ones that aren't mostly equate to variables declared in a program and obviously those can't be stored back in the executable in the event of low memory...)
This, on the other hand, is possible. I don't have any personal knowledge of any OS that does that, but it could be done, and the unified virtual memory model I mentioned above wouldn't be able to help (in contrast to the fact that it can help with locally-backed executables).
http://msdn.microsoft.com/library/default.asp?url= /library/en-us/dngenlib/html/msdn_ntvmm.asp
Make sure to remove the space when you cut n paste
BBH
That's because many RAM reporting utilities (and
The system only does that when it runs out of unused buffers though. This means that at the time a program has to be killed all of your ram is officially used by processes.
I think the OP is probably right. I have 512 megs of ram and using my full suite of apps (let's say something like firefox, thunderbird, a seti process, xmms, irssi in an rxvt and maybe some other xterms etc) I'm using about 200 megs of ram (not counting unused pages). So there was no real need for me to have swap since it never, ever gets used (I've never actually tried to fill all the 500 megs up to the full). So on a desktop system, you rarely have enough bloat to actually need swap.
That being said, I don't understand why a swapping system would be slower than a swapless system, except maybe on bootup when you have one extra partition to load.