Slashdot Mirror
Comprehensive Guide to the Windows Paging File
busfahrer writes "Adrian's Rojak Pot has a nice article about the internals of the Windows paging file. It explains what a paging file is and lists the differences between a swapfile and a paging file. But first and foremost, a large part of the article deals with the various methods of optimizing the Windows paging file, thus yielding a notable performance gain for people who are not overly blessed with RAM."
One of the biggest performance helps is to keep the paging file from being fragmented, and I'm not talking about three or four fragments.
I've come across workstations where the paging file is in thousands of fragments. This happens when someone messes with the settings. For instance, they might increase the size of the paging file thinking it'll help to have more. Normally, it's not a bad idea to increase it but if the drive is heavily fragmented, Windows dutifully uses the fragments for the new space.
The only way to fix it is to completely delete (deactivate) the page file, then do a defrag, then re-create the page file (several reboots involved).
That's probably the best way to tune the page file. There, I saved you from having to take the time to read the article.
A programmer is a machine for converting coffee into code.
Would Mr. File please pick up one of the white courtesy phones? Your OS needs to talk to you.
I can't read the article now because it's slashdotted, but is there a difference between the swap file and the paging file in Linux? Does Linux even have a paging file?
And the men who hold high places must be the ones who start
To mold a new reality... closer to the heart
They might want to adjust the paging file on their web server a bit...
Given the speed of the slashdotting, I'd say he needs to add more ram to his box, and stop relying on the paging file for speed...
Online Starcraft RPG? At
Dietary fiber is like asynchronous IO-- Non-blocking!
People are going to argue this a lot, because geeks like to fiddle with things. Clearly, the thing isn't optimized until I have changed the settings !!oneoneone!!!
But the best way to optimize the paging file in Win2000 (and later) is to leave it alone. Windows will manage the paging system just fine on its own.
http://www.bysoft.com/freeram.asp
A free application that you can use to 'pre-page' out pages right before loading up your application. What it does is hog as much RAM as much as it can, forcing the OS to page out any unnessecary applications.
I've seen the standard Explorer + lsass + cwrss + all the svchosts memory footprint go from 80-ish megs to 20. Running this before your game will allow quick load-times and quicker performance.
... for people who are not overly blessed with RAM.
You mean those who's PHBs said the "minimum" requirements were good enough.
-- If god wanted me to have a sig, he'd have given me a sense of humor.
and It's called a paging file, because if they call it swap they couldn't get the patent!
Or maybe the will be figthing trying to convince the world that they came up with the idea of a swap space!
It seems like Windows' treatment of virtual memory is inefficient. Why use a file for swap -- incurring ntfs overhead and fragmentation -- when you could use a swap partition (like Linux) without exposing the user to any additional complexity (which is the usual tradeoff for efficiency gains).
I had to upgrade my laptop from the stock 512 to 1.25GB just because WinXP thrashed the paging file so much. Granted, I multitask like a demon, but it shouldn't take 30 seconds to swap Firefox back in.
thus yielding a notable performance gain for people who are not overly blessed with RAM.
She has money, she has a title, she has huge... tracts of RAM!
#include <signature.h>
It's 44 Pages!! I mean heck, the average /. doesn't read one page news stories, but a 44 Page Article!? Not in this lifetime.
Your hair look like poop, Bob! - Wanker.
c'mon, who gives a fsck?
Liberal (adj.): Free from bigotry; open to progress; tolerant of others.
Make sure it's a fixed-size page file, not system-managed.
By using 'Diskeeper,' you can also do some additional optimizations besides just defragging it; it's a nice app, though its warnings are overly-dire, and it insists of having something staying in memory all the time, which is irritating.
For God's sake, put a Printer Friendly link on your site so I can print the whole article. Having to read lots of text on my 14" EGA monitor makes my head asplode!
The site is so slashdotted that it took forever to load the first page which basically has nothing but useless history of swap files on it. Even the mirrordot doesn't appear to be working. Anyway, one sentence on that first page made me no longer interested in reading the article:
Whenever the operating system has enough memory, it doesn't usually use virtual memory.
This flies in the face of real world experience. You can have 4 gigs of RAM and nothing but Windows 2000 running and your OS will still use the swap file frequently. Don't believe me? Run Performance Monitor and monitor Memory, Pages/Sec and just click on a few things and you'll see that I'm correct.
I'm a big tall mofo.
The solution they came up with was to use some space on the hard disk as extra RAM. Although the hard disk is much slower than RAM, it is also much cheaper and users always have a lot more hard disk space than RAM. So, Windows was designed to create this pseudo-RAM or in Microsoft's terms - Virtual Memory, to make up for the shortfall in RAM when running memory-intensive programs.
Amazing how they manage to turn everything around as though Microsoft invented the world of computing...
Virtual memory, the term and the implementation have been around long before Microsoft came into existence.
To read the start of the article, you'd think there were no non Microsoft operating systems when DOS came out and that Microsoft invented the concept of Virtual Memory.
I couldn't read any more after that.
-Disgusted AC in PA
Cant get to the listed site since its totally /.ed but another interesting one re: XP, 98, me and page files
Virtual Memory in Windows XP http://aumha.org/win5/a/xpvm.htm
Windows 98 & WinME Memory Management http://aumha.org/win4/a/memmgmt.htm
and there is
How can I optimize the Windows 2000/XP/2003 virtual memory (Pagefile)? http://www.petri.co.il/pagefile_optimization.htm
--
Time is on my side
There's a good reason why people don't write about VM systems often; the combination of writing skill and the ability to understand modern VM is extremely rare.
Take most, if not all, of what the article discusses with a large grain of salt. Everything, from his history (did Microsoft invent demand paging? Hardly) to his terminology is flawed.
Just reading the first 40 comments or so here reveals that VM remains one of those "black magic" areas, where reason is overcome by superstition and people will assert the most ridiculous, irrational and unsupportable things. Regrettably, the contents of this article do nothing to improve the situation.
= Mike
I have over a gig of RAM but my laptop's HD is rather slow. How do I make it so that my computer will not use one of these files? I doubt there are many times when I am using all of my RAM, so I'd want to keep the HD use as little as possible.
The article is a little misleading. Virtual memory is not just space on a hard disk that you use when you run out of physical memory. Virtual memory is the practice of giving each process its own virtual address space that is independant of the physical address space. Doing this allows the OS to send some pages of memory to disk if it needs to, but the OS is still using the mechanism of virtual memory even if there is no hard drive at all. Each process's memory space is independant of the others'.
SysInternals (who make some of the best freeware available for Windows) has PageDefrag. It defrags your registry hives and pagefile at boot, before Windows loads. Very slick program and free to boot.
Entrepreneur : (noun), French for "unemployed"
Turn off virtual memory and see how many MS apps suddenly stop working at all. And we're not talking memory pigs, either. Some screen savers don't run without virtual memory running, no matter how much RAM you have. It's really stupid.
Boobies never hurt anyone. - Sherry Glaser.
I've run into similar problems. I have 1 gig of ram on my machine, if I have my java ide loaded up, and run a game or something like UT2k4, and play for a bit, when I exit out, my machine will thrash like made. But then I tried just disabling the page file altogether because of someone's suggestion, and ya know what? Everything works fine, no memory problems, no thrashing.
Use the free PageDefrag utility from http://sysinternals.com.
It explains what a paging file is and lists the differences between a swapfile and a paging file.
There is no difference.. He says that swapfiles would swap whole processes. I beg you pardon? Working on whole processes hasn't been the case since 'multiprogramming' on third generation computers (around 1965-1980).
btw, a good program to defrag your Windows page file is PageDefrag
Together with Dirms & Buzzsaw, you can keep your disk defragmented for free. Especially Buzzsaw is nice since it will defragment recently accessed files in the background.
Well, they tried, but I think they really missed some very commonly-held misconceptions.
I've just finished a project involving reconstructing process images from physical memory, including pages from swap, if available, so I've got a pretty good understanding of this stuff at a very low level.
Misconception 1: Swap-file usage = performance degradation
Yes, it is slower (usually by 3 orders of magnitude, not 1) to access a page (frequently 4K) from disk instead of memory. HOWEVER, effiency dictates that all available RAM be utilized as soon as possible.
For example, in addition to running processes, we also use volatile memory storage to cache file data. Clearly, we want to cache as much as possible. Page replacement policies then become important to determine how much swap space to use. But usually it is much greater than 0, because we've got process image pages that are less frequently used than a lot of file cache pages. So we've gotta remember that data, but we don't wanna waste fast RAM on it.
In other words, isn't it great that we can swap out pages from an unused process to make room for frequently accessed file data? Regardless of how much memory we have, that's a Good Thing.
Misconception 2: Virtual memory = disk space
Virtual memory is a mechanism for translating program-visible addresses to arbitrary storage locations transparently. This doesn't necessarily mean that disk space is used for swapping memory, it means, for example, that 5 processes can simultaneously be accessing address 0x5000, but the actual storage location is different for each. If the system (usually the page address translation facility on the CPU) determines that this address isn't at some location in volatile storage, it will bring in that memory from swap space and possible page out some other data. This is what the article is generally talking about.
I've seen some other questions about pages. There are a couple reasons for treating memory in page-size chunks, where a page is usually in the 1-8k range (4k for x86). First, the page address translation stuff needs to keep data on translations. It must do this in > 1 byte chunks, since keeping translation data on every byte would require way too much storage. Disk I/O and other I/O is frequently done on page-by-page basis for that reason as well as for the sake of performance.
Well, I rambled enough. Just wanted to clear a couple things up.
I found this link to the article: Swapfile_Optimization
Suncoast Linux - Sarasota, FL
Rojak is a type of salad eaten in Malaysia and Singapore. The term "rojak" is also casually used in those countries to mean "a mixture of everything."
The term Rojak Pot, I assume, is meant to portray that this website talks about a wide range of topics.
It spends a lot of time discussing what really happens in various situations and asserting that some theories that have been promoted are maybe incorrect, but does no investigative research.
Moreover, all these performance claims are bandied about, and yet I see no benchmarking.
Try harder, windows folks.
-josh
I don't usually set out to bash windows, but the windows virtual memory subsystem is as dumb as a brick.
At work I am blessed to have 1GB of ram, so I don't ever need to use any virtual memory. What I find really interesting is that windows is noticably more responsive when I turn off virtual memory entirely. Even though I'm never running out of memory, windows was always swapping things out that I needed soon when I had virtual memory enabled. And I'm talking about when I had 2/3 of my memory unused still (at least by applications, the disk cache could have potentially been using the rest). Just turning off virtual memory altogether made things much much more responsive/faster for me.
Putting http://text.burstnet.com/* into adblock makes for a slightly less annoying experience, but there's also the fact that it's 44 freaking pages long. Probably getting paid by the site hit or something.
Do yourself a favor, give this content-lite article a miss. Make a small partition with ntfsresize, put a fixed pagefile on that partition alone. Works on every single version of windows and it's zero maintenance. Tah-dah.
I am no longer wasting my time with slashdot
No sig for now.
It is always amusing to me to read how M$ "invented" technologies that were in common use by other operating systems while Bill Gates was still wearing diapers. Both paging and swapping were used on IBM mainframes dating back to the mid 1960's. What windoze (and linux for that matter) could REALLY use is the ability to deterministically dedicate portions of system resources to particular processes.
Back when a 1 MIP, 1MB machine cost $1M, operators became highly skilled at managing workloads. Today everyone just throws oodles of RAM and disk at servers and lets the chips fall where they may , so to speak. It wouldn't be a bad idea to actually put a little thought into matching workloads with machine resources, and pro-actively matching them up by deliberate choice(For example - our company is running a prime-time ad at 9pm that features our URL, maybe that's a good time to shut down the normal file backup that happens then). Chaos theory is not always a good load balancer. But what am I saying, that's as outrageous as asking kids to put their money in the bank instead of buying video games...
"Sic Semper Path of Least Resistance"
maximium 128MB swaps on linux are an old myth.. from the mkswap man page:
Linux knows about two styles of swap areas, old style and new style. The last 10 bytes of the first page of the swap area distinguishes them: old style has `SWAP_SPACE'new style has `SWAPSPACE2' as signature.
In the old style, the rest of this first page was a bit map, with a 1 bit for each usable page of the swap area. Since the first page holds this bit map, the first bit is 0. Also, the last 10 bytes hold the signature. So, if the page size is S, an old style swap area can describe at most 8*(S-10)-1 pages used for swapping. With S=4096 (as on i386), the useful area is at most 133890048 bytes (almost 128 MiB), and the rest is wasted. On an alpha and sparc64, with S=8192, the useful area is at most 535560992 bytes (almost 512 MiB).
The old setup wastes most of this bitmap page, because zero bits denote bad blocks or blocks past the end of the swap space, and a simple integer suffices to indicate the size of the swap space, while the bad blocks, if any, can simply be listed. Nobody wants to use a swap space with hundreds of bad blocks. (I would not even use a swap space with 1 bad block.) In the new style swap area this is precisely what is done. The maximum useful size of a swap area now depends on the architecture. It is roughly 2GiB on i386, PPC, m68k, ARM, 1GiB on sparc, 512MiB on mips, 128GiB on alpha and 3TiB on sparc64.
so.. this is it..
Dude, ever heard of VM/370
or maybe Multics
or, more obscure but even earlier Atlas
For example, even under Windows 3.1, the swap file only swapped in chunks of 4/8k. It's just that, on machines that were memory-poor (like we all were way back when), most users had their boxes configured with a 3/1 ratio of swap to real ram, and most of their free real ram then ended up being used to manage the swap file. 2/1 was the sweet spot.
The DOS Shell program, on the other hand, worked by swapping out whole processes, allowing DOS 5.0 and up to appear to multitask.
Pick up a copy of any decent assembler manual for the 2/386 from the early '90s, look for the instructions for swapping pages into ram. It's a hardware function. On a page fault, the cpu then goes through a look-aside buffer (8k worth of pointers each, IIRC, local and global), and then you can map pages of ram in/out as required. Just don't triple-fault, as the cpu would then go out of "enhanced" mode and/or reboot.
Must be a REALLY SLOW day for this to be "news".
I had run the Windows 2000 default defragger quite often before this, and most of the actual files were defragmented just fine, but there was tons of green slivers in the display there before I ran Diskeeper, and only a few chunks of solid green afterwards. I got rid of diskeeper because I didn't like it running in the background all the time, but its reboot defrag process was pretty good.
The Windows 2000/XP defragmentor is a limited version of the Diskeeper defragmentor that MS licensed. By definition the limited version does not defrag the MFT or Registry hives. Also, you don't have to run Diskeeper in the background. There is a configuration setting for this! Just run it when you want to defrag
Who needs a page file when you have a 10.00 GHz AMD Athlon, a 2000 MB DIMM, and a 30000 GB IDE Hard Disk http://www.amazon.com/exec/obidos/tg/detail/-/B000 22ACYK
I quote from a review:
"I was a little skeptical at first on how this thing would perform. However, when I installed and run the Seti@home program it started instantly finding alien signals, which where corrected, cleaned up and translated.
I am now listening and talking to the supreme galactic defense minister about Earths surrender. Apparently this computer is not only tapping into the sun for power but also into the mysterious dark energy and tearing the universe apart. Just comes to show how bugs always show up in technology when you least expect it."
I don't know why screen savers would stop working, but I bet the developers never planned for the creation of shared memory failing.
Larry Osterman writes in his blog that the author fundamentally doesn't understand what he's writing about. Mr. Osterman has worked at Microsoft for 20 years. How old is the author of the article?
Read Epic the first RPG novel.
Then you can put your windows pagefile into linux swap partitions. :-)
System V UNIX had no virtual memory until the late 1980s. Sure, there were specialized versions, such as BSD with VM added on, but it wasn't in stock UNIX. It was around though, and was in IBM 360 and Multics (1960s).
If you do have the memory, disabling the page file is the best thing you can do performance wise. In theory this is a bad idea, in practice it works very well.
I have 1GB of memory. This is more than adequate for browsing, games, development, photoshop etc. So, two years ago I disabled the page file after discovering that with 300MB in use, windows was still swapping like crazy (and yes I had all the popular registry hacks applied which should prevent that). So eventually I disabled the page file. The immediate result of doing this is that applications become much more responsive. Suddenly multitasking becomes easy and fast.
Normally when you work with an application for a while, all other applications get swapped to disk. It doesn't matter that you have 700MB of unused, readily available ram. So when you try to alt tab to them, windows spends a couple of seconds moving stuff back into memory. This is very annoying and totally unnecessary. The problem only becomes worse if you start to run some memory intensive programs because windows will swap aggressively then.
Disabling the page file fixes this problem. The only disadvantage of doing this is that when you need more than 1GB it isn't there. This is typically the point where things would get very slow anyway due to swapping. In any case, this doesn't happen very often and is easily resolved by closing some applications. All games are optimized to run well with 256-512 MB. Most games don't use more than that, even if it is available. Office applications and other desktop software rarely use more than 100MB. Photoshop can push the limits but unless you are doing some extreme high resolution photography stuff with it, you will not run into any problems with 1GB. It's actually quite hard to run out of memory. Most things that are infamous for memory usage like ms flight simulator, doom 3, photoshop, vmware, etc all run without problems and without swapping.
If you think about it, swapping is a really lousy solution unless you expect to run out of memory. Disk is many times slower than ram. The reason that you open programs is that you want them ready for action. Swapping them to disk is therefore undesirable. The only reason it would be desirable is if the total amount of memory used by all of your running programs is larger than the amount of memory you have. So if you have 256 MB, swap files are a nice poormans solution to the problem that you don't have enough memory. If you have 1GB you shouldn't have that problem (and if you do, buy another GB).
Jilles
The 2000 degragger doesn't defrag the MFT, but the XP defragger does.
XP still has no buit-in method of defragging the pagefile or registry hives, however.