Slashdot Mirror
Is Swap Necessary?
johnnyb writes "Kernel Trap has a great conversation on swap, whether it's necessary, why swapless systems might seem faster, and an overall discussion of swap issues in modern computing. This is often an issue for system administrators, and this is a great set of posts about the issue."
One can have 1GB of RAM for a fairly cheap price.
I really doubt that majority of newest desktop PCs need to swap on the HD at all.
The unused/used portions argument from the article isn't quite true. You don't have to swap every unused bit,
if you have enough RAM, leave everything there. It's R-A-M. don't access parts you don't need.
If you don't have them in the RAM, read them from the drive,
don't waste time putting them where they mostly are in the first place.
I'm willing to bet that people who need performance, don't often run 10 applications at the same time. If they do, they
surely know what are they doing.
IMHO the average user should get enough RAM and no swap, let the OS optimize things a bit.
When I was running Linux on my 350 mHz Pentium II with 128MB RAM, you can dang well bet I wouldn't have made it without a swap partition. I probably would have gone back to Windows if swap hadn't existed.
You could make a big ramdisk and swap to that!
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
All the docs on Linux and swap amounts to use are from the days of 386s and 4 megs of ram!
I want to know how much swap I should REALLY be using for a system with 1 gig of ram.
Same for some of the kernel compilation docs. Maybe on a 4 meg system compiling that extra option might cause slowness but on a 500 meg system does an extra 30k in the kernel matter?
Can we get some docs that aren't from the mid 90s!
People like to claim that swap can always improve performance, by swapping out unused sections of memory, allowing for more memory to throw at apps or disk cache.
Well, *most* apps won't just arbitrarily consume memory, so endless amounts of memory won't help. And disk cache gets you greatly diminishing returns.
One of the machines I use has 3 gigs of memory. It will swap out unused programs, in an attempt to free up more memory. The joke is that it simply can't use all three gigs. After half a year of uptime, there's still over half a gig completely unused, because the apps don't take memory, and there's not that much to put in disk cache.
Obviously, that's a pathological case. And there are pathological cases at the other extreme. But as memory prices keep dropping over the long run, swap does become less and less useful.
steve
Oh, you're not stuck, you're just unable to let go of the onion rings.
the rule is swap should be 1.5x your RAM! ;)
actually MS followed this rule, in win2k, the default swap size is set to exactly 1.5x your ram, was 176 for my 128mb system, and 384 for my 256mb system, not sure about XP though, someone fill me in
(yes, some great minds working at MS)
Marge, get me your address book, 4 beers, and my conversation hat.
I ran linux without a swap file on 128 MB of memory a couple of years go. It was an accident, I didn't create a swap partition. I never had a problem (forutnately). Of course, I wasn't doing the heavy duty stuff I am now (scientific computation).
Error 404 - Sig Not Found
If you've got 128MB of RAM you have plenty and therefore will have no need for swap space. I mean, isn't 640k enough for everyone?
As long as users can eat up more memory than they have available, and as long as hard drive space is cheaper than RAM space, swap will always be necessary.
Goddammit I can't suspend my laptop because my swap partition is insufficient in size. Why should I have 2 GB of disk space *empty* all the time just so I can close the lid.
Someone please fix acpi hibernate to do something sane...
So long, and thanks for all the Phish
Does anybody have experience with running swapless Linux servers? I am looking for a best practices for setting memory and virtual memory limits (presumably, using ulimit). Can setting soft (ulimit -aS) and hard (ulimit -aH) limits guarantee that your programs will not eat up all of the system's RAM and cause an OOPS? I'd love to hear about your experiences.
Must-not-watch TV!
Swap improving performance... yeah. On slow systems and low memory, every byte freed up helps. But not swapping in the first place is good too.
I'm now expermineting with replacing various tools with smaller versions, such as dropbear, udhcp, tinylogin, and buzybox. I'm also slowly writing up a "exec and restart shell afterwards" utility called PivotShell.
Hardware wize, I have swap on a CF drive. 32 megs so far, but if I can afford larger CF drives, I'll format 'em as swap and use them.
Why all of this? 40 megs swaps to HD, and on a laptop, any HD access sucks battery power. When you're using Xfree (or even Kdrive) and Firefox, you're going to swap. Period.
--
# Canmephians for a better Linux Kernel
$Stalag99{"URL"}="http://stalag99.net";
Sometimes, when a process goes haywire, it will start munching RAM. If important programs like, say, sshd or X, can't malloc when they need to, they'll die ignominiously. Swap gives you the chance to kill the rogue process before your OS goes kaput. Its slowness can actually help for this.
but today's production, heavily loaded system will still need the ability to swap to/from disk.
Already, there are systems that minimize that need, set-top boxes, embedded systems in general. But each of those is seriously modified (kernel-wise, mostly) to achieve the responsiveness, the frugality of resource treatment that a general purpose desktop computer can't expect to enjoy.
That doesn't mean that developers should stay in the same rut, assuming that hardware that confined system design in the '60s, '70s... '00s will perpetually assign similar constraints.
IMO, desktops still need to swap... for now. but let's not paint ourselves into a performance corner.
On one hand you can have a huge amount of RAM. On another, you can have memory mapped I/O. Anyone see the future of this?
-I am an elective eunuch.
Swap files are how we are able to get those $400 computers. I don't have any idea how a modern person can put up with 128 megs of RAM and Windows XP, but some people out there do. Doing without swap is only really feasible if you aren't running many different things on your computer at once, and what you do run isn't as much of a hog as many programs are nowadays. I guess if you were to use as small of a Linux kernel as you could and didn't install X on your computer, it wouldn't be much of a problem and you could do without swap. If you really want to go light on the memory usage, just use ed.
"Anyone who attempts to generate random numbers by deterministic means is living in a state of sin." -- John von Neumann
Notice how sluggish the system is after doing something disk-intensive like watching a movie. That's because the kernel is caching as much of the movie as possible to memory and swapping your running apps out. And kernel developers think this is a good thing, so it isn't going to change any time soon. IMHO for a desktop system this makes no sense, that's why I run my 1GB RAM machines with zero swap.
Why are we whining about swap files any longer?
:-)
IMHO, anyone running a system (WIN ME - XP) with less than 256MB RAM is nuts.
I feel that if you want to remove swapping then have more memory than memory addresses. Let's see what happens when the machine has to figure out what to do with the extra memory it has.
Back to topic: Really, how much is a GIG of RAM nowadays?
I'm only just a l'il punch drunk
Sincerely, Czephyr
Seriously, I don't get it. How in the world can swap ever increase performance.
:)
Specifically, suppose I have one computer with 1GB of RAM and 1GB of swap, and another computer with 2GB of RAM and no swap. Under what circumstances will the first computer be any faster?
Now I suppose if the swap is used for other things besides memory space then I could understand it. But then it seems like a simple solution would be to allocate a fraction of RAM for those things. In effect, create a swap partition on a RAM disk
Seriously, I'd appreciate some education here, but make sure you answer my specific scenario above if you reply... thanks
This may be slightly off topic...
Running KDE 3.2.1 now, I notice it takes longer to open apps than it does in windows. Mozilla for example takes literally a few seconds longer to open each window than it did in windows. Another thing windows does is make it faster when you run an app right after you ran it then closed it. Say for example in windows I run mozilla, then close it, then open it. When it opens it the second time, it's almost instant. However in linux, it seems to take the same original amount of time to load it completely. I'm sure it has to do with an entirely different process of loading programs, but apps always seemed to open faster in windows than in linux, in my view.
Then again, graphics used to be in the NT kernel and that's what made it appear fast, but lead to a lot of problems and crashes, so maybe the longer load time is worth the wait when compared to a reboot.
I use my gmail account as my swap partition. It's fully searchable and displays helpful advertisements every time I load fifty tabs in Firefox and OpenOffice goes idle. I don't know what I'd do without it. I'd probably be less of an unfunny jackass.
I also reply below your current threshold.
Does anyone out there want to run a series of benchmarks with a few standard applications to prove/disprove whether disabling swapping improves performance?
I'm tired of just hearing antidotal evidence on this. Everyone has their stories about turning off swap files and improving performance, but in what cases? Are there some users this would harm?
I remember when I had disabled swap completely under windows XP. The first program that gave me annoying errors was photoshop. Every time it would start up it would say "Photoshop detects there is no swap, and it should be used or problems can occur with your system". (something like that). Also, some games (empire earth for one) would not even run without a swap partition. So even though you don't technically NEED one, some programs still rely on it!
None of my machines have less that 512 megs of RAM, (or more than 1.8GB) and I never use swap...never have a problem with it. It's faster because drive access doesn't have to compete w/ swap (even on SATA it's annoying).
/dev/zero > /swap.tmp /swap.tmp /swap.tmp
Okay...I lied. Does anybody else use Pan? I mean..wtf?! When I run Pan it's more like
$ cat
*wait 20 seconds*
$ mkswap
$ swapon
$ exit
# pan
Pan is an awesome program, but seriously...when it can single handedly use > 1GB of RAM just stealing divx rips...
We can just make an OS that reboots every time it runs out of memory. Oh, wait, someone already did that...
Alot of companies around here have started purchasing diskless drives for their workplace. SWAP is pretty darn useless then.
If you need to use Swap for what your doing with your computer, then you obviously don't have enough ram!
one reason you'd want swap on a system is to have someplace to dump/savecore information in the case of system crashes. Kind of hard to do with volatile memory.
FreeBSD for the impatient.
As I RTFA & previous comments here, I was rather suprised at how argumentive people were getting over this. Some people are saying swap is an absolute necessity & a swapless system was a broken system, while other's said swap was an obsolete solution to a problem that no longer exists (expensive RAM). This seems odd to me, because as far as I can tell, the decision of whether & how much swap to use is based mostly on two things: specific situations (and thus there is no general answer to 'Is Swap Necessary?'), and opinion. And either way, with the Linux kernel today (and for quite a while now), I can choose for myself whether or not, and how much, swap I want to use. So if I am in a situation that I think requires swap, I can use it, and in a situation that I think would be hurt by having swap, I don't have to use it. So I don't see why there's so much hoolabaloo about this: nobody is forcing anyone to do it one way or the other. And if someone else thinks it should be done different from how I would do it, that's their decision, not mine.
Join moola.com, play games to earn money.
load a binary newsgroup of 500K+ posts (on my system)
watch performance drop to zero.
or load one with 750K+ posts and watch you run out of memory. dmesg errors (process killed out of memory)
thats when swap and RAM were filled
Most applications today have unnecessary or rarely used portions of code or data - bloat. These get swapped out first. Also there are various memory leaks here and there, which means the programs sometimes forget to release allocated memory they do not need any longer.
Look at the size of your X server, or mozilla, or apache, or pretty much anything else and you will see over the course of a few weeks that it has grown beyond reasonable operation demands.
The memory lost this way is never accessed from there on, but the system cannot release it without the program telling it to, so it does the next best thing and shoves it in the swap. Not a solution since eventually swap gets full, but since the leaks are slow to begin with, at least it prevents them from affecting system performance too early.
In the average case code and data _do_ tend to be accessed more than once. We would all be complaining a lot more if the kernel NEVER cached... remember the huge performance boost SMARTDRV made in DOS?
So, frankly, the default kernel behavior is right.
To fix the movie/updatedb/jumbo cp/etc issues see "man madvise" and check out MADV_DONTNEED. I am hoping applications will start using this syscall sooner, rather than later. The Linux VM can take a hint, and it's pretty easy to give it one.
Eventually my persistent storage device will be solid state with ultra-efficient DRAM and integrated batteries, and my system RAM will be a big SRAM space. I will want to swap to DRAM.
--- Nothing clever here: move along now...
I for one want to avoid MBR screwups and the hassle of communicating with NTFS, so I don't dual boot.
The fact that you irrationally avoid two very mature and easy-to-understand technologies implies you are uninformed and compels me to ignore any suggestions you might have.
I've heard that Unreal-engine based games in particular make heavy use of swap filing.
"You've heard"? As in, someone told you to believe it but you didn't take the time to think about the claim yourself? No application can "make heavy use of swap filing". Either it uses a lot of RAM or it doesn't. And either it accesses RAM in a predictable pattern (condusive but to swapping) or not. But an application can't actually make use of swap filing: it always -thinks- it's using RAM. The OS will always decide whether it's time to swap in or out.
In the 90's, I ran a 10 line BBS on an Amiga 4000 with 16 megs of Fast ram, 2 megs of Chip ram, and 0k for the swap file. :)
I know, I know, the Amiga didn't HAVE virtual memory. Well actually it did if you had an 040 and installed a memory management program such as GigaMem, but so few people had a use for such a thing that it was practically unheard of.
Oh, and before someone jumps in saying that I wasn't able to do anything else, that is totally NOT the case.
Very often I was doing lots of stuff. The difference is developers were used to working within memory constraints, and now days developers are used to systems growing into the applications.
"Everything you know is wrong. (And stupid.)"
Moderation Totals: Wrong=2, Stupid=3, Total=5.
From what i gather one of the reasons swap is considered necessary is that its not possible to shift pages around in physical memory to optimize things.
:P)
(Note: this is me speaking out of my arse, please correct me flamefree
Say you have 100 meg of ram (to make things easier) and 5 applications have 10 meg of ram allocated to them at equally distant addresses. Application #1 says it needs 60 meg allocated sequentially. In order to do that 60 megs, a couple of other apps will need to be swapped out.
Now, without swap that wouldn't be possible, so the allocation would fail.
So what interests me is if you had a RAM disk set as swap would that make this possible, and the whole swap thing more efficient? Say 1/4 ram is ram-swap, and have another 512meg as disk swap at a lower priority?
when a board has more than 265MB RAM, swap may be well suited for 3 things.
1. Java
2. Python
3. Earwax
All 3 in common; when unchecked, they fall down at most inappropriate time.
=================
Q: What's your name?
A: Slim
Q: Slim Shady?
A: No, Slim Jim
Q: A Beef Jerky?
A: No, I'm just black on the outside and light in the inside.
... seen any swap usage, at all, on my Linux box... OK, granted I have a gig of ram. But I do like to use xmms, kde 3.2, mozilla, and other memory hogs. I guess what I'm saying is that for people like me, swap is pretty much necessary, because not everyone has a gig of ram.
Now, I would like to see Windows manage the above scenario... I had Win2k on the same hardware and it never used more than around 650 MB of the physical RAM, and still swapped around 200MB when I was using stuff that wanted huge amounts of ram (photoshop, etc)
I haven't seen a case where disabling swap actually increases performance. I have however seen lots of cases where disregard for logic involving swap space caused serious performance problems. The old 1.5x and 2x rules for swap space are outdated and even dangerous in today's systems with ooglebytes of memory.
With less than 128MB of RAM you practically need 2x your physical memory worth of swap space. Running a full GUI environment, even a relatively lightweight one, needs quite a bit of system memory. With 64MB of RAM and a 128MB of swap space you'll be able to run a light GUI environment but have a crappy filesystem cache. The system will crawl but it won't get constant OOM errors if you're not overzealous with your app usage.
The 2x RAM rule on a system with 512MB of physical RAM on the otherhand is excessive. With 1GB of swap space most of it will end up empty unless you're running programs needing huge amounts of allocated memory. With more than 512MB or more of physical memory on a single user workstation you're pretty unlikely to run into situations where active pages are swapped out to disk.
I've seen the runaway process situation crop up on more than one system with excessive amounts of swap space. Since swap is so slow it can be troublesome to kill a process that is using so much memory that it ends up having active pages swapped to disk. The system ends up spending 99% of its time trying to handle the disk IO from the heavy swapping which can make the system totally unresponsive for local and remote users. Because the systems had way more swap space than was logical the offending processes never got OOM errors even though they were using up almost all of the system's resources.
I've pretty much set 256MB as the upper limit for my systems with 256MB or more of physical memory. That is enough swap space to hold any dirty pages or unused processes but not so much that a runaway process is going to eat up all my disk IO for a couple of hours. Once a system hits the 256MB threshold I toss out the silly 2x RAM rules for something with a little more cognitive thought.
I'm a loner Dottie, a Rebel.
Makes me wanna go out and buy more ram just thinking about it.
Yes, it sounds crazy. But not too long ago, somebody here on Slashdot posted something about an IDE device wherein you could put RAM on that thing and access is as if it was an ordinary IDE device. YEs, compared to your memory bus, the bandwidth sucks, but its not really bad compared to hard drives, especially the hard drives' seek time.
If you've got kernel 2.6 you can change the "swapiness" to fit your needs/desires. People with lots of RAM could experiment by changing the swapiness value to 0 and report back with the results (be easier than installing a system without swap).
Comment removed based on user account deletion
1. Make a ramdrive
2. Create swap file in ramdrive
With being annoyed at a system that takes some time to load applications back into memory from disk after being idle for some time, despite not having used up available memory and the OS deciding to swap it out anyway. And there is merit in the idea that an OS shouldn't swap something out of memory unless it needs the space.
However, the idea with doing away with swap space entirely, I'm sorry but what are some of these people smoking? "I have 256MB of physical memory, and 256MB swap space, a hard cap of 512, if I add 256MB of new real memory, and could remove the swap space I'd have the same amount of memory and better performance!". No, you're wrong, you'd have MORE memory at the cost of money from your pocket and better performance than you used to, but you won't be able to run any more applications than you could before by having more memory.
I would support the idea of on demand swapping, only sending least used memory to the harddrive when absolutely necessary, instead of when convienient. It would definitely improve performance of bring back up those idle tasks, and degrade slightly the performance of opening new applications when memory is almost full. However, doing away with it entirely would be a BAD design decision for an OS to make. The whole point of swap is to increase the amount of usable memory without having to buy more memory. Just because RAM these days is seen as 'cheap' does not limit either the desirability of that feature, nor the necessity of it for those people who haven't tricked out their systems. Because you consider 4 gigs of physical memory cheap and a necessity, does not mean everyone else can afford it.
This is not a sig.
Yeah, that's happened to me in kde (quanta) in Mandrake 10 CE. I was sure glad I had a swap space then.
BTW I've got a gig of ram.
I've built many servers, embedded systems, and even desktop systems that don't use any swap at all. Many more I limit the amount of swap greatly. The overall responsiveness is much better if you don't use swap and I find system stability to be better. Really it doesn't matter what the systems are used for or how many apps are being ran.. it's just how much memory you're going to use compared to the amount of physical memory you can afford. You can run out of memory just as easily using swap as you can while limited to physical memory.. the main difference being that the recovery of the sitution is much worse in the case of using swap. Quite often the system starts to churn and then grinds to a halt. Without swap those tasks just die and everything else keeps running. Setting memory limits on tasks is a good way of ensuring which tasks are killed first but I'd like to see better control of this given to the admin.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
there's a definite pattern with regard to swap in the windows world.
for win'9x: use up ram until almost gone then start allocating swap space in anticipation of actually using it. should memory allocation still be increasing then actually use swap space. reverse the order when freeing memory.
i had 384 megs ram at the time and as long as i used less than about 350 megs total the system wouldn't be in swap.
for win 2k & xp: (when within physical ram limits) whatever amount of memory is requested, allocate between 60-80% to ram and the rest to the swapfile. even the disk cache partially goes to swap! i didn't believe it at first but all one has to do is look at the numbers in the task manager's memory/cpu window. at first i figured that all i'd need to do is throw in some more ram and the disk thrashing and absolute crawl would go away. i put in a gigabyte of ram (i never allocate more than 700 megs at most and the total system memory usage on bootup is 100 megs). even with the extra ram the problem stayed the same.
turning off swap gives me consistent fast performance, and since the disk cache isn't swapped (partially) i get 2x the throughput i had with a swapfile on large file copy operations
machine tested: duron 1.3ghz, 1 gig pc133 ram, 2x 80 gig wd800jb hdd.. os win2000 & winxp running newsbin which allocates disgusting amounts of ram in a large header grab (yeah i could have used a test program but why do that when newsbin is a real-world test for me). the os and applications are on different drives on their own ide chains
with swapfile enabled (size=1.5x system ram).
allocation time: unaffected, only the time to perform task reqested
memory de-allocation time: (by either quitting app or selecting another group) 23 MINUTES of constant disk thrashing
with swapfile DISabled
allocation time: unaffected, only the time to perform task reqested
memory de-allocation time: (by either quitting app or selecting another group) 2 seconds
if you want people to think you know what you are talking about, just put ".com" at the end of everything you say.com
For most home users who can get 200+ GB of disk relatively inexpensively what's the big deal about giving up (say) 1GB for swap?
There's also the use that if for some reason a system panics (hey, it happens) you have a place for the kernel to dump to. This can be valuable in helping debug what happened with a backtrace.
The sticky bit on Unix systems keeps the image of an excutable from abandoning the swap space image when the last user has stopped running it. It speeds startup time of selected applications in a heavy multi-user environment. Not very useful feature anymore.
What would be really cool is to repurpose this bit to tell the OS to not page out the application instead.
That way you could have all the benefits of swap, but your U.I. programs (with this bit set) could still give you snappy response because they'd always be in RAM. Call it a "half-swap" system.
Linus, you listening?
Here's a real-life example of why swap is useful. One machine I manage has a gig of ram. At the time of purchase, that seemed quite reasonable. But the users are working on a project that takes 2 gig of ram. So currently it's using a gig of the swap. Yes, that's bad, and I'll be adding a second gig to it in a few days (it's in the mail). But in the mean-time, that swap space is really handy. It means the users can get their work done! Think of the first 256M of swap as being for speed. If you're regularly using more than that, then it's time to order more ram. But it's nice to have the spare gig of ram for odd jobs, or while you're waiting to install it.
I'm no expert, but I think a lot of these arguments could be resolved if people took advantage of the ulimit constraints. If you can limit how much a program can get out of control, then there's no longer a concern for a single user sending the server into swap hell. One of my current projects is to figure out reasonable limits.
Not to be a smart ass or something, but...
I think a modern Linux desktop system with plenty of RAM needs at least half the amount of RAM as swap, to accomodate hibernating. With swsusp2 moving into the kernel 'soon' there is no need to shutdown your programs even if you shutdown your machine.
And even with 1GB of RAM you will need some swap, I happen to fill 600MB swap with a system with 512MB RAM. Just some browser apps, email and from time to time th Eclipse Java IDE.
Bill Gates never made the infamous "640K... enough for anyone" comment. Not only have I never seen it documented anywhere, but he was asked about it and replied that he never said that.
He didn't see the Internet coming -- he thought MSN should be like CompuServe, because that was the top info service (before the Internet became big). And I remember some wild comments he made about the truly amazing, throbbing power of the 286 chip. So he's not an amazing guru with awesome predictive powers. But people keep beating him up about this bogus quote, and I'm tired of it.
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Have you tried the swsusp2 patches? It uses gzip compression or the faster LZF for writing the hibernation image to swap.
They can be found here:
http://swsusp.sf.net/
...remember how horrible the swap is in Windows (including XP). It really kills the resposivness to have large parts of most apps being swapped out. Especially if you leave the computer. Why swap a bunch of stuff out just because I am not sitting there and no program actually want any more (especially since I never use any other screensavers than "blank" or "turn of the monitor").
I think the problem is basically that "seldomly used memory" is simply "a page not accessed in the last x time". I'm sure that a better system for a process can be deviced after all, and not having the startup code for the app in RAM isn't too bad, but swapping out everything needed to turn of the screensaver and start surfing/coding/gaming again really really pisses people off. Remember, it is about percieved speed.
"If I have to wait, it takes too long".
--Forgoil
Apparently a new feature (mentioned by a network engineer workmate), is to have the IOS reserve a portion of memory for administrative tasks (like supporting the login process and configuration shell).
A feature like this, that "reserves" a portion of RAM so that if something really fubars your system, you can still login to fix it - would be great for Linux/BSD.
Man watching 6 MSCE's around a sun box, looks alot like the opening scene's of 2001:space odyssey...
Just in case you need it.
Although, I don't know what the big deal is. My OpenBSD server, which has 2 gig swap and 1 gig ram, hasn't actually USED any swap for more than 2 months. The server is used for email & an intranet site, with about 50 concurrent users.
Of course, OpenBSD is dying, so what do I know...
When I found out Photoshop had it's own scratch disk, (swap file), I thought it was dumb, at first. There is a setting that sets how much available system ream Photoshop will suck up-- default is 50%. Without this, Photoshop would immediately saturate system memory and begin swapping, when you load a large image. Any program that knows it is capable of sucking up all the RAM should have a feature like this....
Ok, I'll bite. How fast is it?
Sincerely, Czephyr
I've run my Linux systems without swap for years (since 2.2) without any problems. Of course, I make sure I have way more RAM than I am likely to need (the stuff is practically free these days; but OTOH, so is HDD space.....).
/data/swap /data/swap
Simply put, you need enough 'memory' to hold all the stuff you want to run, plus caches. For a given task, you might go for a system with 512MB RAM and a 512MB swap, and I'll just go for 1GB RAM and forget the swap. The only difference is that if/when your system comes up on its RAM limit, it's going to start slowing down. When it starts using a lot of swap, it's going to crawl. But it'll still run. Until you run out of both.
Mine will run like blazes upto the 1024MB limit, then barf. No warnings like with swap.
So if you want an early warning sign, use swap. If your needs are well known and won't push beyond the limits of your hardware, don't bother.
You can always add a swap file later/only the fly as your needs change anyway.
dd if=/dev/zero of=/data/swap bs=1M count=512
mkswap
swapon
Opportunity knocks. Karma hunts you down.
- Generational garbage collectors, such as the one used in the JVM, screw up swap. It seems like there is a conflict between what the OS is trying to do with swap and what the JVM is trying to do with GC. I would rather let the GC win in this so the application runs fast.
- Memory is cheap these days. I just started up OpenOffice, widely regarded as a hog of a process, and it takes 27mb. Huge, you say? That's about $2 worth of memory these days.
- Does swap actually make things faster? The concensus these days seems to be "if your process is swapping you need more memory." Therefore... why swap?
It seems like the place where swap makes sense is that you can have a bunch of non-running/idle processes all "running" at the same time, while swapped out. I'm not sure how much this is needed on modern machines with > 1gb of ram.----------
WAP software
- The system can swap out unused portions of memory (that have been allocated, written to once, and not touched in a long time), and use that memory as a disk cache. Depending on how often those "unused" portions are needed, this can be a big win.
- As somebody else pointed out, if a process goes haywire and allocates far too much RAM, swap gives you a bit more breathing space before it becomes a problem.
- Final point. Under Solaris, you can configure the kernel so that, if it panics, it dumps the entire contents of RAM to the swap partition. On the next bootup, this memory dump is read, and put into a real file on a real filesystem. This can help track down the cause of problems. But for this to work, the swap partition must be at least as large as the amount of physical RAM you have.
Why write the contents of RAM to swap? Well, where else can it go? The kernel's just had a panic. You can't trust any significant part of the code (eg: filesystem drivers). You do, however, know where the swap partition is, and it's safe to scribble all over it; none of the apps that were running are going to continue anyway.It's all about what you want to do with the system, and making a judgement call on this. Me? I say, disk is cheap; why not have a swap partition?
..that I think is spiffy is using the partition I would normally use for /tmp as . Then, I mount a tmpfs of that size on /tmp. This makes a large performance improvement for anything that uses a lot of temp space, because everything /tmp would normally handle is done in RAM until RAM fills up, at which point we're back to using the disk.
For the kinds of complaints about Linux swap I've been seeing of late, it would be bogus to call swap the issue, really. People looking to eliminate swap entirely on desktop machines are cutting off the arm for the sake of a finger.
The issue with swapping in a desktop system is that perception of system responsiveness is almost as important as real performance, and swapping in (actually, it's paging in, but that's semantics) causes high latecy. This is especially noticeable when returning to an idle machine. So we want to cut latency.
People say "the kernel shouldn't swap unless it can't fit everything it needs in system memory." Duh! And it doesn't! It's swapping to increase the size of the file cache, a huge performance win. If the file cache gets too small (say, because this Wal-Mart PC only has 128 megs of RAM, and you've turned off swap, so Moz is eating it all) then you wind up with disk seeks for harddrive intensive applications, causing the same latency as swap.
What's clear to me from these complaints is that the file cache isn't smart enough. People with lots of RAM want to cut down on all these disk reads - that's why they got gobs of RAM. (Ain't it funny that the same Linux heads who say that Linux makes a little machine fly also say that a desktop has no reason to have less than 512MB or 1GB of RAM). At the same time, smaller machines should still be supported, and even folks with gobs of RAM don't want to elimiate swap, otherwise disk bound apps suffer the same latency they're trying to eliminate.
The Linux file cache seems too aggressive for most users. Ext2 loves a file cache like no other filesystem, and this probably influenced the design. If the file cache can be smarter about when to swap to grow itself, and when it should just be content to use up all available system memory, then lots of these latency issues can be fixed in a way which will scale across both hardware and multiple use environments.
To add memory to a server you must take it down. This is bad for uptime figures.
If the peak loads on a machine are starting to hit triggers then you schedule and outage but if the client load surges then you can add a swap file on-the-fly as a temporary fix to get the server through to its maintenance window. Sort of a limp-home mode.
Then when its maintenance time the machine can be tweaked based on its new peak load.
Swap is not necessary.
Why not simply assign a MAXIMUM amount of RAM
that normal application may used,
let say an application may never used more than 64 MB of RAM.
Application like OpenOffice, Mozilla, GCC, KDE, GNOME,
could be
nice --RAM 512MB pid
nice --RAM 0 pid = No limit
or started with a shell script with a specific amount of RAM.
If an application runs out of RAM, two possible solutions:
- Kill it "gracefully"
- Suspend it and send an email to root@host
basically, you reduce the timeslice to 0.
This would allow for instance "nuclear reactor"
to have that process use whatever it needs
and watchdog or any other "less important"
process get killed, instead of having
the entire system becoming slugish
after some amount of time.
You may also provide a 'hint' to the parent process
via some API that the child got "out of control"
and something should be done about it.
This would allow finer control over the current "statue quo".
Just a tought.
It's as if rather than truly unloading the app from memory, Windows is just decrementing a flag which says the app is no longer in memory. When it decrements it to 0, it probably runs some routine which restores the used memory to the state it'd be in if you'd just started the app. But until it finishes this, it can't increment the flag to 1 (restart the app). Whereas if it were truly reloading the app at a different memory address, it could start loading it immediately.
Given some of the behavior I've seen from users, I can actually understand making the GUI behave this way. I've seen many users who still don't get or refuse to use the concept of having multiple documents open at once. If they need to open a different Word file, they will exit Word (to close the file they're working on), then double-click on the new file they want to open. Restarting an app behind the scenes in this situation makes sense in a predictive branching sort of way.
There are thrid party cards out there that use standard dimms to make ram drives...woulnt the ultimate performance come from loading up one of these and putting your swap there? Then you would have the benifits of swap space but the speed of ram. That to me seems ideal.
Do you have a deranged amount of disk fragmentation or something? What happens if you terminate the process (task manager->processes->executable name->end process->yes) instead of closing it normally?
23 minutes to deallocate less than a gig of ram seems like you've got more serious problems than overly-aggressive swap behavior.
The general answer is yes. As always YMMV. However, the things have changed a LOT since all of the general "rules of thumb" came about. If you look back far enough (back around the late Cretaceous and please stop staring at that bone in my hair) an awful lot of research was done into VM performance. There are several equations from that period that you can use to characterize the performance of your VM system.
Fast forward to present times... if you'll apply those very same equations, I think what you'll discover is that page sizes ARE WAY TOO SMALL for current processor, disc and RAM speeds. Today we should probably have pages about 64k or 128k in size given the relative performance of things. I'm too old & lazy (and it's too late) for me to look up the references, but for the more energetic and inquisitive among you, it wouldn't be too hard to track down (Sherman, set the wayback machine for the year 1979 (or there abouts)). Having been in the business for a while, it's curious to see how cyclic things are. That said, some applications (like HTPC) probably DON'T need swap (but still want a disc to store program material on).
OK, just so this post isn't totally content free, the basic equations (for the ambitious) are based upon the cache equation (which, afterall, is what VM is). The cache equations is:
T(a) = T(r)*P(r) + T(m)*P(m)
The terms of the equation are:
T(a) - time to access an item
T(r) - access time if the item is resident
P(r) - the probability that the item is resident
T(m) - access time if it's not resident (miss time)
P(m) - the probably that a miss will occur
If you look at the access patterns of programs, generally if you increase the page size, the number of misses will go down (taken to the extreme, if the page size was the same size as large as the program, there obvious would be no misses). What we're seeing (as CPU and memory speeds go through the roof) is that the miss cost (T(m) - the time to access the backing store/disc) is (relatively) increasing. Increasing the page size will(or at least should) cause the overall cost of an access to decrease (assuming the other assumptions are correct). The rest is left as an exercise to the reader.
hahahahaha.... you lose karma for being a jackass! take that jackass! maybe next time you'll be a little less of a jackass when you decide how big your swapfile is, hey jackass? you like that, don't ya jackass! jackass! jackass!
The answer is simple
The execution of the answer is not so simple
It is good practice to establish a profile of the applications that you run (memory wise) and to then purchase adequet RAM to suit that application profile. Basically, figure out how much RAM you "normally" use, and then buy it (plus a bit extra). DB server profiles are as different from web server profiles as office profiles are from gaming profiles. Without the context the particular profile, blanket statements regarding memory and swap tuning are meaningless.
Thank you for your time,
BBH
I start a routine 3 GB compilation on my workstation with -j 6 and distcc. While I wait for it to finish in 20 minutes, I need to use the 3-GHz/1-GB PC to check my mail with KMail, perform some vmware MS-Access errands, and fill out some Web forms with Firefox. The CPU load is at 5, but one sixth of the machine should still be a 500-MHz PC, which ought to be more than enough for such human-pace activities.
No such luck! The file access kicks all the applications out of RAM. Even the mouse barely moves. The window frames become unresponsive for seconds or longer.
A few days ago I turned off swapping, and the machine has actually become usable during compiles. It still swaps out the text segments of the executables, which causes occasional delays and funny, unexpected keyboard autorepeats. To deal with that, I'm planning to move the desktop utilities to a ramdisk.
IMHO Swap is a good idea and here's why. /tmp is mounted on swap. If you, say, have 1GB of swap space and chuck 512MB of stuff in /tmp, you've now got 512MB of swap left. Lots of Unix software dumps stuff in /tmp and, when there is available RAM, /tmp lives in RAM. This makes temp files very fast.
I admin Solaris systems, and swap on Solaris is a fine thing indeed.
You allocate a complete slice of a hard disk for swap, and you can then add and remove swap dynamically while the system is running. Need 1 GB more swap? Create an empty 1GB file, and add it as swap.
What's more
Plus, the VM subsystem also deals with the file cache so on a Solaris system, you will see the amount of RAM used always around the 100% mark. No point in having RAM there unused, it costs too much. Use it as disk cache.
In addition, when an application needs to be swapped out to disk, why bother writing to disk something that's already there - the application's code is marked as being paged out to disk and removed from RAM and when it's needed again, the code is fetched from the original binary that the application was loaded from.
All in all, these kind of modifications to the VM subsystem mean that swap is good to have and can make systems faster with it than without.
k:.
Specialist Mac support for creative pros, Melbourne
I used to run a linux computer on 256 MB of RAM and no swap. Turned out it was a bad idea when playing with Gimp. I once tried to open a large image file which ended up taking on the order of 8 hours to load (I basically went to sleep and when I got back up it hadn't quite finished). From then on, I started implementing a swapfile. If nothing else, now I have swap as a "just-in-case" measure so that in case I have to deal with something extremely memory intensive the computer won't slow to a crawl. I tend to implement swap using a swapfile, though, so that I can change it more easily, and in some cases turn off swapping.
I ran my system without swap for a while. I never really needed swap until I tried playing UT2003. At the time it only had 256MB of RAM and I had been running 2.6.0-testX kernels.
A swapless system won't be faster for the same workload, usually the contrary, in fact, since lack of swap denies the system the opportunity to optimize RAM hit ratios. What a swapless system can do is force admission control on new processes in the system, thus enforcing a no-overcommit policy on RAM, and therefore increasing responsiveness at the expense of global throughput.
Swap thrashing in a desktop environment is usually the sign of a workload that is too high for available memory, e.g. trying to run far too many apps simultaneously. No amount of OS smarts is going to compensate for overbooking RAM with too large a working set. The solution is to increase RAM or not run as many apps simultaneously.
Swap thrashing in a server environment is usually the sign of improper server configuration. Naive administrators configure too many processes, thinking they will avoid a bottleneck if all server processes are busy, but all they achieve is turning RAM into the bottleneck rather than the server processes themselves. If you have a web server and configure Apache to have too many running processes, these processes will spend their time contending for RAM instead of doing useful work. Too many cooks spoil the broth. A swapless system would prevent excessive Apache processes from starting in the first place, thus alleviating the problem (at the expense of high error rates, which is probably not acceptable), but performance won't be anywhere as good as a system with swap and properly sized Apache process limits.
Swap is not a panacea. It should not be used to protect against runaway processes (setrlimit is here for that). It is useful in absorbing sporadic spikes in traffic without causing denial of service, and to shunt away uselessly allocated virtual memory (ahem, memory leaks).
As for the idea of putting swap on a RAMdisk, it is completely brain-dead (unless you have exotic memory arrangements such as NUMA) - the kernel is going to waste a lot of time copying memory from the active region to the ramdisk region and back. A straight swapless system will be preferable.
There is no hard and fast rule for sizing swap, it depends on your workload, such as the average ratio of RSS to SIZE. The usual rule of thumb is between 1x and 2x main memory.
Actually swap is just an another level of caching. Ideally swap space should behave as other caches at other levels, and just be made of the free unused disk space, or a good part of it. This is the optimal way to use ressources as found over the years at the processor level, and at the RAM level. One should not specify the amount of swap space, the sytem should be able to use whatever free disk space is left.
Above there are many arguments/illustrations for one or the other. Linux is used in a very wide variety of systems - to meet different technical and budget criteria, some will need swap, some won't.
You have the choice - configure according to your needs.
"Nick Piggin [interview] explained that swap can improve performance no matter how much RAM you have, "well it is a magical property of swap space, because extra RAM doesn't allow you to replace unused memory with often used memory. The theory holds true no matter how much RAM you have. Swap can improve performance. It can be trivially demonstrated." This said, numerous Linux users do report success running a swapless system."
^ so you're saying that having say, 512mb mounted as swap on a ramdrive and 512mb normal ram is better than just all ram?
A blog I run for the wealth
I don't know if Linux works this way, but...
UNIX kernels have assumed the availability of swap for nearly 35 years. You cannot remove this major architecutural feature without unintended side effects.
I have an XP 1800+ with 512MB on Win98SE. Disabled swap. Noticable performance increase. Cranked up a game of RTCW Enemy Territory, completely one map with great performance!!! Game crapped out when trying to load the second map and froze, not enough (i.e. zero) swap dag nabbit. Needs more RAMses I do.
Turn all you disk space into swap space and do everything in memory!
RAM versus Hard disk...
Virtual memory is one concept to free the programer from having to think about physical program size versus physical memory size, by providing a general mechanism to get stuff to and from the hard disk into physical memory.
Files/file systems are another concept, that also require the loading from hard disk to and from physical memory (ultimately).
Who needs both!
Transactions? What? yeah, yeah, yeah... Your happy to do all the file open, file close, read, read, read, but not prepared to be explicit about transaction boundaries? *sigh* use method invocation and pay the penalty, chump.
Although it may sound counterintuitive, it can (and usually does, unless you have obscene amounts of RAM) actually improve performance. Here's why: If the computer can swap something out which hasn't been used in a long time, you can use the freed physical RAM for other things, e.g. more disk cache, thus speeding things up.
HAND.
...because I *CAN*, you insensitive clod!
No, kswapd is for 32 bits
open4free ©
Then you won't have anything using hundreds or thousands of megabytes in a pathological fashion for a tiny little GUI and you don't need to worry about lack of memory on a modern machine.
If you want to run a program bigger than your available RAM, then you need swap.
...
The software I'm developing uses about 3Gb, most of our systems have 2
Not true. He's saying, that adding swap to system with some RAM can improve performance. So, 512 MB RAM + swap can be faster than 512 MB RAM without swap. But 1 GB will always be better than 512 MB RAM, regardless of having swap or not.
For example, an "interactive" task like a login could steal some hot pages from lower priority apps already running.
Well, the trouble with this is that there are rapidly diminishing returns on the size of the disk cache. At some point it becomes pointless to swap to try to increase the size of the file cache.
(However, being able to allocate more than the physical amount of memory and avoiding thrashing as long as the working set is smaller than physical RAM still counts for something if you normally work on huge data sets).
HAND.
The lessons of history teach us - if they teach us anything - that nobody learns the lessons that history teaches us.
I haven't touched Linux for several years, although I used to do serious work on it.
I take it from the tone of the discussion that Linux still uses separate swap partitions? Why? My main machine now runs OS X, which swaps into the filesystem, and that seems to work a lot better. The system can decide what it needs to use, and I don't have to make a decision. I recall that Linux supports swap to the filesystem, but it sounds like nobody actually uses this feature. I can somewhat understand a server using a swap partition, since the needs of a server would be more or less known in advance and I assume it's marginally faster, but I don't see any reason to use one on a desktop machine. Why is everybody still using dedicated swap partitions?
Mod down posts with a "Free Mac Mini/iPod" sig, they're spam!
23 minutes to deallocate less than a gig of ram seems like you've got more serious problems than overly-aggressive swap behavior.
Definitely agree with that assessment. NBPro might be a pig, but the parent poster's machine needs some serious work if de-allocating memory takes 20+ minutes. While I've seen it happen, it's only when you exceed the amount of physical memory in the machine by a goodish amount.
The parent poster should've used 1 spindle for O/S and applications. With the 2nd spindle dedicated to swap, temp and data files.
Wolde you bothe eate your cake, and have your cake?
My dekstop hasn't been rebooted in a couple of weeks, and I've had a couple of sessions of Far Cry each day, Azureus running the whole time, watched a couple of DVDs, used Outlook, Zircon, Nero, Celestia and quite a few other apps. Task Manager tells me I haven't been over 840 MB used. If/when apps and games push it over 1GB, RAM is cheap and both machines can take 2 GB.
At work, I make sure the servers I build are specced with enough RAM for the job (Windows Server 2003, SQL Server and IIS mainly). When first installed, I give them 1 GB of swap and then do protracted burn-in tests including unfeasably high test loads. Generally tehy don't go above the RAM they have, and once I've proved it I turn off the swap. If they run out of RAM, we get more RAM. We now have a few on 4 GB and 8 GB, so those 64-bit extensions are coming at an ideal time!
In summary, RAM is fast and cheap enough, disk is slow and mechanically vunerable to failure. Don't swap!
Ok, I have a responsiveness question on this subject. This annoyance has gotten worse over the years.
Install a RedHat AS 3.0 system, using generic settings (normal amounts of swap and such. Log into X and open up a terminal session. Run "free" in that terminal.
Start any other program you want, and it will probably start pretty quick. (time it)
Reboot, and open up the same terminal session and run free again.
Walk away from the system and come back the next day.
Walk up to it, and run free in that exact same terminal.
What you will see is that most of the memory is now used (I'm assuming it is the active disk cache).
Now, start the same app you started yesterday after running free
It will take at least twice as long to start (probably more).
You can write a quick program that will use all the memory in the machine (huge perl array or something). This will free all the memory, and starting other apps will be fast again, until the system sits idle for a while.
I have done this with different versions, and different systems, but the results are always the same. A system sitting idle uses up all the resources.
Now, my question. How the heck are people getting months of uptime when the performance appears so bad just sitting overnight? Also, how the heck do you stop this, and why is it designed to be so hyperactive when sitting idle? This is one of my biggest frustrations in linux, because the responsiveness goes to poop all the time (performance is poop as well).
- Take a system with X RAM and swap.
- Add Y (=X) more of RAM,
- Disable swap, and
- Limit the maximum disk cache to available space in the original X amount of RAM.
Wouldn't that enhance performance over a system with X RAM and swap?-- Gary Goldberg KA3ZYW 301/249-6501 AIM:OgGreeb Digital Marketing Inc., Bowie, MD
Both OS's have filled their RAM with completely useless cached files (part of a 1 GB+ AVI for example, that I will most likely not be watching again for several months), swapping out all the programs I have running.
Both OS's really need to learn how to deal with Slow I/O. If I/O is only being done at a rate that is a fraction of my harddisk speed (say 300-400 kB/sec), which occurs for stuff like watching a movie, playing music, serving an upload over DSL, then this data is really not worth caching for longer than a few minutes. Even if I do need it again, it will probably again be at just 300-400 kB/sec, something a harddisk can take care of quite comfortably.
--Swilver
I know this wouldn't solve everybody's problems, but: it seems like what's really needed by most people is the ability to easily place critical apps and/or data files into a reserved area of working memory that never gets swapped or paged out. I think this would differ from a run-of-the-mill RAM drive in that (I think) RAM drives aren't really part of 'working' memory, so apps/data stored in them still need to be loaded/paged/swapped in to be active. This all seems like such a pathetically obvious idea that I'm really expecting somebody in the know to say, "Duh... to do this, you just _________". Any takers?
Listen to what I say, not what I mean...
I think the problem is that application ram, file cache and mmapped files all content for the same pages. If we give these groups there own page groups this will be reduced alot. The os could shift the sises around a bit to minimise the amount of pagefaults it detects. The most common problem i see is that after i do something file intensive, other applications are horribly degraded, i suspect this is because mmap is pushing everything else to the swap file, which would only make sense if the the application is long running. But most applications i use that cause this arent long running, movieplayer, mp3s, file sharing, basicly anything multimedia. Maybe providing a bit more metadata on the kind of operations a binary does would help.
Just allow people to set a limit on how much memory can be used by various things.
So that instead of being flexible the things are hard coded...
--
IE, you have 1GB of RAM.
You set aside 100MB as disk cache.
This leaves 900MB for executables to run in.
--
The limitation of this is that it is not flexible and is fragile, if you need even 1 more byte above the 900MB you won't get it. Not even if there are 50MB that haven't even been touched yet.
But many people are coming back with, we don't care, Linux is blazingly fast upto the point that something reads through all the files on the disk indexing them, which flushes every desktop application out to disk and then everything you click on for a few minutes takes seconds to come up the next morning.
What people are saying is keep my programs in memory. They are even letting you assume that all the applications are well behaved. Because anyone setting this up is going to be able to fix it so they are well behaved.
They are willing to sacrifice the trade offs involved and figure out the mix they need for a particular use all by themselves. They are saying, let my box run applications like it has 900MB of memory, but let it access files off the disk like a computer that only has 100MB of RAM.
Creating swap space on a RAM drive is just one way they are thinking about how to reserve more memory in RAM for their processes.
Any machine that deals with large files will still need swap space... Photoshop when dealing with large image files...etc
||| I still can't believe Parkay's not butter.
Are there any tools for linux for reading/clearing swap files/partitions? And more importantly, how do you know if it is safe to clear/modify a swap?
username:oldwarez password:oldwarez
This could be incorrect, because I haven't actually sat down and figured out exactly what's happening, but, from my own experience, here's what I've found.
If you run windows with adequate ram and no swap, it runs faster and smoother than the same setup with swap. Windows appears to push stuff into swap prematurely. example: 512MB of ram. ~200MB used. Windows is already swapping.
Linux seems to not use swap until it damn near runs out of RAM. I therefore always put swap in linux boxes, for the simple reason that it's better to have performance start degrading but still work, rather than have everything ticking along just groovy and run out of memory.
Swap *always* comes down to memory, one way or the other. If the system is not using all the ram, it can't really get any faster by using DISK instead of ram.
Once in a while I'll do something like 'grep -r "oops" /big/filetree'. The fact of the matter is that I'm probably only reading any of that data ONCE, and it's not going to all fit in memory anyways, so I don't even gain anything if I run the grep a second time.
In a situation like that, I'd like to have some sort of 'nocache' directive that says 'Don't waste the cache with this'.
Something else that might help would be to have some sort of 'minprog' directive which would tell the swapper that a certain amount of space is reserved for 'program' data (i.e. code (including shared libs) and data), -- and that that memory shouldn't be swapped out in favour of something otherwise being read from disk. I think that this might avoid the situation that I sometimes run into of a large program (mozilla/gimp) being unresponsive after I do some other disk-intensive task (like the aformentioned recursive grep).
Things like the OS enforcing things like the RSS rlimit hints would also help. (I hadn't previously realized that it didn't).
Free Software: Like love, it grows best when given away.
And of course, he always tells the truth?
My boss started worrying that we weren't going to be able to deliver what the company had contracted to deliver. He was the antithesis of a PHB and so he sat down and in a few hours wrote a small driver to emulate the overall task the project had to accomplish. No detail, just broad brush emulation. He was able to demonstrate with a few lines of code that nothing we could do would hit the delivery spec. Burroughs responded by doubling the amount of RAM on the box as well as installing RAM that was twice as fast as what they had initially delivered. The combination enabled us to turn off swapping and deliver a working product.
Fast forward to 2004 and I'm working on Excel spreadsheets that have 60-70 sheets in a workbook. Saving the book is a bitch - 15-20 second wait after I hit ctrl-S. Every so often, Excel just goes away as it performs a prophylactic background save just in case Excel dies. 15-20 second pauses because the software has become so bloated that saving a 2-3 meg document is an excuse to flog the poor drive into a seek frenzy. The drive, which was about 4 years old, finally gave up the ghost. Its replacement has an 8 meg cache separate from the 512meg Windows manages - that "little" 8 meg junk of RAM belongs to hard drive alone. Night and day performance difference. The Excel swap frenzies that were induced by a simple ctrl-s are gone. 3 meg documents save in under a second - just what you'd expect from a drive that has a transfer speed in excess of 60 mbytes/sec.
My sense is that swap has always been a kludge. It's an attempt to squeeze more data into a machine that has only so much space. The working set graphs look pretty but they seldom describe what is happening day to day. Trading 2 nanosecond response for a 5 millisecond seek is seldom going to be a good trade. Bottom line from that OS class 35 years ago? Keep your working set size less than your physical memory and your machine will remain responsive. Just what the old IBM Geezers were saying in the first place.
This whole discussion has ignored the only important point.
... clicking on one of your X windows audibly causes a lot of disk reads and response is very slow.
That's the algorithm for deciding which page to swap out. All systems I know of swap out the "least recently used" page when they need to swap something out.
This is what causes the bad effects caused by swap. Suppose you (like me) have a machine that you use mostly as an X server. But naturally it's also running a few other servers, a MTA, an ntpd, minor stuff like that.
You leave your desktop overnight and when you come back in the morning
That's because your servers write to log files. Of course these writes stay in memory. And when Linux needs to swap something out (which it will do when a server writes to a log file), your X sessions get swapped out, not the useless logfile pages, because the logfile pages were used more recently. This is really braindead behavior. But that's how the swap algorithms, as currently specified, work.
Steve Jobs said that regarding the Lisa or whatever it was that came out before the Mac. He said that about 256k, I think when someone raised the issue that the new IBM PC could be expanded to 640k.
now we need to go OSS in diesel cars
RAM limits are a primary issue with machines I work on. These days I can't really do things 100% with less than 1G of memory.
What we call folk wisdom is often no more than a kind of expedient stupidity.-Edward Abbey
So fix the cause, not the symptom ... don't use bloated programs.
now we need to go OSS in diesel cars
Gates has to wait for the Linux engineers to finish innovating before he can steal their ideas.
Liberals call everyone Nazis yet they are the closest thing to it.
I really enjoy swap space
1) Your analysis is based on bad assumptions so your result is way off. 2) You're a sick bastard for fucking a horse.
Whilst that's strictly true, some modern languages use generational garbage collectors that segregate objects in memory according to age. Only when an age group gets full do they sweep through an age group, and move any surviving objects up to the next age group.
This heuristic works exceptionally well, and runs fantastically quickly, and triggers significant swapping hardly ever.
There are some circumstances where it runs slowly, but in the worst case the performance is similar to simply doing a full garbage collection. These situations are pretty rare; objects generally segregate very well into young/old or young/middle aged/old categories- the vast majority of objects die very young.
Sad isn't it.
-WolfWithoutAClause
"Gravity is only a theory, not a fact!"Disabling swap will improve responsiveness not performance. And performance would only be lowered if the system had less RAM than needed for OS + apps + disk cache.
Never put in more swap than you're willing to use.
It's "easy" with exploding disk sizes to give yourself several gigs of swap.
However, transfer rates have NOT increased in proportion to disk size.
Several years ago I could flood my 16 megs of swap in a few seconds to minutes. It would take "hours" to flood a couple gigs.
For a good time make a graph of how long it takes to "read an entire hard drive" from the 80s to today. It's gone from reading the whole device in less than a minute to taking several hours in some cases.
When you have a bug that allocates all your memory you will have to wait for the swap to flood full before the oom-killer will zap that process. A huge swap space makes a huge delay, may as well just reboot instead of waiting.
Unless you have infinite patience I would not use suggest using more than a quarter gig or so of swap.
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
The government doesn't 'tax' your income; they utilize it! ...tell me another one...
As I read the KernelTrap thread I remembered having the same discussion with either Vernon Shriver or Mark Callow at SGI something like 14 YEARS AGO!!! The comment then was something like, "Bryan, there are only 4 people in the world, all of them in North America, and 3 of them are in mental institutions, that truely understand the strong juju that is swapping." That said, I have to wonder how many people posting on the swap thread have actually read the man pages, let alone understand swap and/or remember using xenix. Not only do they lack rough consensus and running code, they lack a fundamental understanding of why an operating system needs swap (do NOT equate M$s' virtual memeory with any modern unixes' swap), or, for that matter, most of them lack a fundamental understanding of operating systems. Leave your memory management to the system level codesmiths. If you want faster swap, buy a SCSI ramdisk widget and some ram and STFU. Your only other option is to earn a clue and start spending your nights and weekends becoming a tool builder. Otherwise, the "I want a pony" feature requests, critiques, and generally retarded banter from people who think they know something have every right to continue to fall on deaf ears.
This thread is precisely why I don't contribute to those communities anymore.
Use a swapfile instead of a partition. 2.6 kernels cache the location of the file, so there's no performance hit for swap files compared to swap partitions. I'll give a quick HOWTO:
/var/swap /etc/fstab for: /var/swap none swap sw 0 0
/proc/meminfo and see how much you're using when you've got the system strained, use twice that amount, not less than 128MB.
1. decide how much you want (you can change it later, I have 128MB on all my boxes with over 512MB RAM). The example uses 128MB
2. #dd bs=1M count=128 if=/dev/zero of=/var/swap
3. mkswap
4. edit
5. swapon -a
6. There is no step six!
But the best way to know how much swap you need is to peek at #top every now and then, or #cat
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
After reading this I thought to look at my laptop. And what to my surprise I didn't have any swap. It turns out a couple of weeks ago I was playing with swsusp and it had corrupted my swap space and it wasn't loading. I never noticed but after the fact I realized some of the stuff that was going on was because of this. I tried openening a large pdf file and it was taking forever to load and seemed to almost lock up my system. Same thing happened when compiling a few programs and other things similar. I guess with 196 MB of Ram you want to have some swap :)
here's my output from free. free
total used free shared buffers cached
Mem: 904732 202036 702696 0 9416 108356
-/+ buffers/cache: 84264 820468
Swap: 0 0 0
this is typical. i hate having a swap partition. i didn't even swap with 384 megs of ram. now with a gig. this whole "need swap" argument is stupid. if adding a gig of swap would help you, then adding a gig of ram instead, would help more.
will-the ac
In both cases he is hosed. HOW IS IT DIFFERENT???????
In case A: 256MB RAM & 256MB swap is hosed
Case B: 512MB RAM is also hosed
No, I don't trust in god. He'll have to pay up front, like everybody else.
There are all kinds of facts being brought up here.
Let's remember one thing, WHY we have swap in the first place.
We have swap so that we can effectively address and use more RAM than we have physically.
That is the reason. The ONLY reason.
All other facts like how we swap out unused pages... we do that because it's more efficient than swapping out often used pages, or swapping indiscriminately. The reason is NOT because "unused pages in ram slow things down" or "they have to be moved somewhere because they are not in use".
All the fancy algorithms for figuring out what to swap and what not to are just to make swap more efficient... they are not the primary reason swap exists in the first place.
Swapping makes sense if the performance lost by swapping is less than the gains you get from the extra free physical memory. Perhaps that memory is needed for disk caching. Perhaps you need swap to get enough RAM to run several programs simulatenously, so you can switch between them, rather than load and close them all the time.
If you have enough physical memory, you don't need swap.
I post this from a Solaris perspective....
What you need from swap depends purely on the activity and needs of your applications. Generally, an NFS server has no need for swap - it only wants to feed clients and use memory for caching filsystem data. What needs to be swapped out? Nothing - file system caches can be flushed back to disk and reused, so no need to go crazy with swap.
A Sunray server on the other hand might only need say 2G of RAM to perform well enough at peak usage (purely a figure out of the air, but lets run with it). However you might have 50 users logging into it leaving sessions running, but only 10 users active at one time. Those 40 active sessions are just dozens of sleeping mozillas and other session crap. If you need memory for your active users, just throw the inactive processes to swap. So on 2G of RAM you could easily use 16G or more of swap just on idle processes. Why buy 16G of memory just so you dont have to allocate 16G of cheap disk, when 2G of memory satisfies your performance needs?
Two extreme examples, but they demonstrate the issue - other applications fall in the middle. if you need to page out stuff, you need swap. If not, why bother?
How then does a page fault compare (to paging and swapping)?
While there are benefits to having extra RAM, removing swap is not a good idea. Even if all currently loaded processes would fit in RAM, it might be a better idea (as decided by the CPU) to swap a few idle processes into swap and cache some commonly used files or pre-load some data. Without swap, all buffers and processes would have to fit in RAM, which would limit the ability of the operating system to cache stuff even if the processes stay within the available RAM.
If a swapless system beats a system with swap and the same RAM, it's because the operating system has a poor resource management.
Actually I now use kernels without swap support except in some servers. For my desktop computers, I noticed that swap would not go over 50MB usage. With 1GB main memory there is no sense in having the facility at all.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
So why not have it :)
True it's annoying if some program got swaped to disk and it's taking a huge time to come back, but that program only got swaped out because something you did required a lot of ram, something you wouln't have been able to do if you didn't have swap, only time I really notice it is when I exit a game like quake3 which has been loading more textures than I can imagine.
swapoff /dev/hdaX
no need to install a new system. (?)
Windows doesn't give a damn if you have 32MB of RAM or 2048MB of RAM, it will still use swap
Also, dont tell me you can turn swap off. Well if you look carefully after you turn it off and reboot, swap will re-appear on your installation drive...
That pisses me off, you turned it off, and it still comes back on, what kinda OS is that?
Many of the systems I've been installing and using over the past couple of years have stuck fairly closely to my "one MHz/1Mbyte" rule of thumb. My workstation is 2.4GHz and 2Gigs of RAM for example.
Despite application bloat (another discussion;) the vast majority of RAM in most of my systems ends up as disk cache. I've limited many of the systems to 10Megs of Swap with one or two spare partitions I can turn on if I'm going to do something that needs more - bringing the total to about 1/2 total RAM in most cases. Old school said you should have 2x total RAM due to need to re-build page tables when RAM got too fragmented. Old Xenix systems (that actually "swapped" as opposed to todays "paging") could get to the point where the OS had to push all but the kernel out to disk and rebuild the RAM image from scratch.
One of the things I'm looking at (2.4 kernel) is using a file in /dev/shm for "swap" with a high priority and a small (10 megs or so) size, and another larger (partition or file) at lower priority. The kernel seems to need at least a couple of Megs of swap and the /dev/shm/swapfile would satisfy that but still make it FAST! The lower priority disk cache might then soak up the need for any larger swap, and a periodic "swapoff /dev/swap ; sleep 5 ; swapon -p 32767 /dev/swap" would keep it from getting too old and crusty - using a script that would look at how busy the system was to decide if it should run or not (don't run if busy)
I'm just getting into the 2.6 kernel on a couple of servers and will play with the "swapiness" setting to see what it does in real life.
On my workstation I'm looking to keep all of the (many - 60+ virtual desktops full) terminal sessions and applications I keep open responsive as I move from desktop to desktop.
On many of my small customers' servers in many cases I need to balance the needs of a diverse range of server applications such as Samba, NFS, Mysql, EXIM, POP, Spamassassin, DHCP, etc.
The diverse file activity of the SMTP chain (EXIM, Spamassassin, POP) can soak up a lot of cache, while the need to keep buffer space available for SMB and NFS for fast response has to be balanced with the need for RAM for Mysql's tables.
As one last point, I'm wondering if the use of the old "sticky bit" (which forced the old swapping systems to keep a program in RAM even if it was no longer active on the theory that things like the shell and cron would be more responsive if they didn't have to be loaded from disk each time they were run) wouldn't give us some more control by allowing us to "nail" a particular application into RAM simply by setting it chmod 1755. I believe this would take a change to the kernel to achieve as I don't believe the sticky bit on files has much if any use on a paged system. I'd love to do this for example for rxvt/xterm.
Been there, done that, paid for the T-shirt
and didn't get it
ConservativeSwapfileUsage=1
This setting supposedly encouraged Win9x to use real memory rather than swap - I don't know if there's something similar you can do in 2K/XP...
Gentoo Linux - another day, another USE flag.
Calculate a swappiness threshhold based upon six parameters:
memory latency, bandwidth, size
swap latency, bandwidth, size
This swappiness threshhold would measure the amount of diminishing returns in terms of time to swap in and bound this above. System load could be put in terms of these same units, and would give a good indication of when you need additional physical RAM, or need to relax your swappiness threshhold to achieve expected performance.
For scientific apps, we generally use swap=8-10x main memory, and turn off overcommit. This gives us the best performance, and the ability to "fairshare" on our batch system, by suspending tasks to swap.
This is our pop server, which accepts some 60,000 messages per day and does all the spam filtering on this one machine:
Mem: 48M Active, 167M Inact, 90M Wired, 18M Cache, 60M Buf, 175M Free
Swap: 4096M Total, 0K Used, 4096M Free
It's only got 512 Megs of RAM.
"No problem. I have the capacity to do infinite work so long as you don't mind that my quality approaches zero."-Dilbert
At work, my machine routinely uses 1.5GB of memory. It has 2GB installed. The part of the filesystem that gets most of the heavy workload holds about 400MB.
When I didn't use an application for a while, it got swapped out. Now I switched to XP (have to use Windows there, sorry) and disabled the swapfile. Guess what - I'm happier now.
Why?
My physical RAM can hold all my apps AND the part of the filesystem (cached) that gets heavy workload. Now I can switch to Mozilla while it's deploying stuff and not wait around a minute for the app to come up.
And no, I didn't notice longer waits for the ressource-hungry jobs.
Trust me, I've got more where that comes from.
Windows XP is horribly inefficient when it comes to swapping. I have 1GB of memory which is more than twice as much as is in use normally (typically 300-400 MB). An efficient swapping algorithm wouldn't dream of using the page file with more than 50% of the ram still available for use. Yet this is exactly what windows does. Even if you use some of the essential tweaks (like making sure the kernel is never swapped to disk) you still end up having to wait for the disk when switching between applications. Which is rediculous considering that you probably bought that much memory to prevent swapping (like I did).
I disabled my pagefile over a year ago (after observing that upgrading from 512MB to 1GB did not reduce the amount of swapping). The effect was very noticable and permanent. At the time I promised myself to use the third memory slot the moment I would get an out of memory error. That has never happened. I run all sorts of games that are known to require lots of ram. This has never caused me any trouble. I play far cry, unreal tournament, flight simulator and lots more, all without runnning out of memory (most of these games only allocate 200-300 MB at most). Unlike with the swapfile enabled, I can switch to my browser and back to the game in under three seconds (e.g. to reply to icq messages).
Switching between applications is instantaneous, even if you are running lots of them (with swap file enabled this is downright painful). I really can't think of any good reason why I would want to go back to swapping hell.
Jilles
In windows 2000 or XP, how to disable swap file? I tried setting it extremely small so that it would know not to use much but Windows insist on increasing the size immediately!
Thanks, and sorry about poor English.
Those few days out of the year when you have to work with something twice as large as memory, in memory.
Like a seatbelt, you won't be using it most of the time, and it may be an inconvenience at times, but it can save you when you really need it.
-
GigaPixels photos of maps (travelling, routing, EDA,
..)
- Virtual Reality: millions of 3D-pieces.
- Artificial Brain: graphs of millions of nodes for A.I.
- Complex Games with quasi-full Reality: by example, playing as the Terminator II's evilman.
- Classifying & processing many videoframes in memory.
- 64 bit's life of software is of many many years, it's long time to wait for passing to 128 bit.
- Etc.
open4free ©It seems fairly clear. In many circumstances swap can be beneficial. But, there are plenty of cases where users would PREFER a message like...
"You're out of physical RAM, I can either waste 5 minutes swapping until you quit programs, or I can put this process on hold until you quick programs or get back from Fry's with more RAM"
I think between that and the 'swappieness' setting in the latest kernel and IF that system message was real, I would bitch a lot less about swap. I'm all for VM and OS managed memory, I'm just against using a HardDisk as the medium. HD's are several orders of magnitude slower than RAM.
Does anyone know if the swapfile of OSX still behave like the one in OpenStep? We used to have a very memory eating application on an OpenStep pc and after several days the OS said something like "The swapfile is getting very large (about 1 GB I think), please restart the computer!"
I heard one have to restart OSX after some days of heavy usage because everything gets slower and slower when the swapfile gets bigger. Don't know if it's true, my iBook has not that big uptime.
So, when RAM prices continued to fall (it's a 1998 computer!), I turned off the swap and bought 2 more sticks of RAM.
Linux has not hiccupped.
Is there something special about the fact that the swap is on a HDD rather than just RAM?
My vague understanding is that swap is just fake memory which is cheaper than RAM.
Anyhow, next machine will be dual opteron; 4G (2 per), and unless something really funny happens, no swap.
Paying mondo bucks for a processor, then bogging down everything with disk swapping seems silly to me. It seems smarter to go for the slightly cheaper (slower) processor and spend the extra money on RAM.
Disclaimer: The plural of "anecdote" is not "data".
"Reality is that which, when you stop believing in it, it doesn't go away." - Philip K. Dick
I have a question about the functionallity of swap space. If I am running 10 applications, 1 - 9 being critical, and 10 being a text version of leisure suit larry.. assuming I have the RAM to carry the first 9 applications, will they reside in the faster memory, leaving good old Larry to reside in swap space? only being swapped in when necessary? Or is it arbitrary, the applications being swapped so that my more critical apps are being moved to the slower, HD space. If so, wouldn't adding SWAP space actually reduce the performance of critical apps? Thanks in advance for /.'s infinite wisdom :).
~ Chris
ObliTech Consulting
especially on my 3B2 with 2MB RAM and SVR3. Also on my 16MB 486/66 that's been running FreeBSD 2.05 since November 1995. That box runs NIS, DNS, and sendmail with a total of about 5GB of SCSI disk. Current uptime is only 34 days (power failure and the UPS ran out of poop).
i never tried forcing it closed... never was able to under win9x .. the whole damn thing was frozen solid during all the thrashing an i didn't want to lose data.
:)
as far as fragmentation is concerned, a fresh ghost restore and defrag (swap was on windows partition) didn't help. newsbin and its data was on a separate drive.
when swap wasn't in the picture, disk performance was never a problem (rules out newsbin's own data files). a 4 gig file copy goes in 2.5 minutes and ghost in dos getting 300-600 megs/minute for a restore.
newsbin may have been poorly optimized as far as that goes, it didn't seem to deallocate all of the ram at once like one would expect, it was something like 4k at a time (per disk stroke)
anyway, i fixed it by disabling the swap... nobody needs it swapping when there's plenty of ram left now do they?
if you want people to think you know what you are talking about, just put ".com" at the end of everything you say.com
I used to work for BBNPlanet, and there was no problem with traffic caused by customer data, but on a clunky old Cisco 2500 w/ 2 mb of RAM and a couple of class C's, a simple scan would always smoke the box.
:)
The problem was that for each scanned IP that didn't have an ARP entry, the router would timeout and wait, then fill the cache with an "incomplete" entry. There wasn't much free memory to begin with, but that's probably where your RAM went.
I won't tell you about Code Red and the customer w/ a Class B attached to his Ethernet0 interface.
// Agent Green (Ian / IU7 / KB1JQO)
// IEEE 802.3: All 10base Are Belong To Us
Ok
I am going to make one dumb question.
Though seeing how late I am in the discussion, It may go unanswered.
Am I right to think that the limit of SWAP (SW) is such that memory (MEM) and address bus with (W) follow this equation?
SW+MEM = 2^W
So that if I have all the physicall ram that my system can adress, swap would be useless? or would cause some trouble to the kernel or would not be possible to turn on?
Off course, nowaday PC's can address from 4 to 64 Gb (32 bits and some modified to address a bit more, right?) so it is unlikely.
errera hunamum ets
Or you could just not use such an inefficient command like 'grep -r'. Recursive grep is for people who never bothered to learn how to use the find command.
/big/filetree. Maybe you are writing a huge project in C, and you're looking for something in a header file. Perhaps you know you shouldn't even bother looking at other people's file, and there is no sense grepping directories, now is there?
/big/filetree -name \*.h -type f -user myself -exec grep oops '{}' \;
Chances are, you have some idea of where "oops" would be in
find
this command would only open files that are relevant, and it wouldn't cache all that other stuff that you already know you never should have cached in the first place. Ever wonder why real UNIX systems don't support the -r option for grep? Because find has been around for decades and does the job a whole lot better.
Swap does NOT improve performance on a system with enough memory for the tasks it performs. THIS can be trivially demonstrated.
How could you possibly argue swap improving performance in this scenerio. Swap letting you use ram for most often needed memory does NOT make things faster, it makes things faster in a swapping world.
In a world without swap everything commonly accessed IS ALREADY in memory, you don't need swap to do this. Guess what, everything NOT commonly accessed is also ALREADY in memory so swap doesn't slow down those accesses. ALL the memory accesses are as fast as they can be within the limits of your physical hardware and the software controlling it (kernel).
Just write an extension to a standard login shell, and have it sitting on one of the virtual consoles. The extension being: it preemptively mallocs a bunch of memory, sets it as non-swappable, and then does its own memory management inside that fixed chunk of RAM. Use one of those rescue shells with all the simple commands like "ps" built in. Voila, emergency fallback recovery shell.
Even if you have a gig of memory and never use more memory for programs
than 300K, and even if you set the memory manager to give priority to running
programs over file blocks, you will still see NT use swap.
If you don't give it swap, you won't be able to get various stats related
to paging (as when it pages a program into memory) since the module that
shows those stats also tries to show swap stats and basically does a divide
by zero somewhere and crashes.
My linux server, on the other hand will use 'zero' swap (even if it is
allocated, provided I never try to use more memory than I have
physical). This was in the 2.4 era. Things may have changed in 2.6 to
make it more like Win XP. Dunno.
-l
Apparently bad flash memory management shut down one of the 2004 Mars rovers a few weeks after it started. Something like the directory space node lists were being incorrectly updated. The rover appeared to use up all its memory, then tried to reboot over a 100 times over two days to fix the problem. NASA evently transmitted a patch to both rovers. The rovers use a version of realtime, non-virtual UNIX called VxWorks from Wind River. Its been well tested in spacecraft for a couple of decades, but never with a huge amount of flash memory.
Now would this have happened if the Rovers had a virtual memory UNIX?