Slashdot Mirror
Why Use Virtual Memory In Modern Systems?
Cyberhwk writes "I have a system with Windows Vista Ultimate (64-bit) installed on it, and it has 4GB of RAM. However when I've been watching system performance, my system seems to divide the work between the physical RAM and the virtual memory, so I have 2GB of data in the virtual memory and another 2GB in the physical memory. Is there a reason why my system should even be using the virtual memory anymore? I would think the computer would run better if it based everything off of RAM instead of virtual memory. Any thoughts on this matter or could you explain why the system is acting this way?"
You must be confused about virtual vs. physical memory. In modern processors, there is no penalty for using virtual memory, all translation from virtual to physical address space is done internal to the processor and you won't notice the difference.
So all the physical memory installed in your PC is used by the processor as one big pool of resources. Processes can think whatever they want and address huge memory spaces, that's all in virtual land. Virtual memory only starts impacting performance when pages are being swapped in and out, because all your processes need more resident memory than you actually have.
Swapping means accessing the disk and freezing the requesting process until its page of memory has arrived from the disk, which takes millions of processor cycles (a lifetime from the processor's point of view). It's not so bad if you swap once, as the processor can work on other processes while waiting for the data to arrive, but if all your programs keep pushing each other out of physical memory, you get thrashing and consider yourself happy if the mouse pointer is still responsive!
So you may want to change the title of your post to: "why use physical memory in modern systems?". I would point you to an article I wrote on that topic in 1990, but somehow I can't find a link to it on the web :-)
fairsoftware.net - software developers share revenue from the apps they build
Memory exists to be used. If memory is not in use, you are wasting it. The reality is that your system will operate with higher performance if unused data is paged out of RAM to disk and the newly freed memory is used for additional disk caching. Vista's memory manager is actually reasonably smart and will only page data out to disk when it really won't be used, or you experience an actual low-memory condition.
Virtual memory and pagefiles still exist so that there will be persistent, recoverable storage of your browsing and search history, illegally downloaded music, and furrie porn should anybody come a-knockin after you hit the power switch.
[/tinfoil hat]
Vista boots in 4 GB? I'm impressed!
:-)
...laura
Are you asking about modern systems or Windows?
you could create a RAM Disk and set your page file to use that.
Then all your virtual memory is in RAM.
I'll leave it to someone else to explain why that isn't a good idea.
"The stupid neither forgive nor forget; the naive forgive and forget; the wise forgive but do not forget." -Thomas Szasz
We who know what we are doing are free to take the risk of running our computers without a swapfile.
Most people are not in a position where they can be sure that they will never run out of physical memory. Because of that, all operating systems for personal computers set up a swapfile by default: It's better for joe average computer owner to complain about a slow system than for him to lose his document when the system crashes because he filled up the physical memory (and there is no swap file to fall back on).
Stop the brainwash
The other extreme point of view is that modern systems should only have virtual memory and, instead of having an explicit file system, treat mass storage as a level-4 cache. In fact, systems that support mmap(2) do this partially.
The idea here is that modern memory management is actually pretty good, and that it's best to let the OS decide what to keep in RAM and what to swap out, so that issues like prefetching can be handled transparently.
For that matter, why do we even need to explicitely "save" anymore? Why does the fact that Notepad has 2KB of text to save prevent the shutdown of an entire computer? Just save the fecking thing anywhere and get on with it! Modern software is such a disorganized mess.
The problem is whatever Vista is using for page replacement, not virtual memory itself. Let's say you're using 2 GB of physical memory, and then start up some memory-intensive program that uses another 2 GB. You're done; if you run out of space now, the OS can do nothing about it, and is forced to do something drastic like start killing off processes.
The question then is why isn't Vista making fullest use out of virtual memory? It is probably trying to proactively move little used pages out of physical memory to make space for new pages. Its an attempt at optimization--after all, if you're not using that data, why let it take up valuable physical memory?
Absolutely not true. You can even install and run Vista on a computer with 1Gb ram and no page file. And run applications. So it doesn't reserve 1Gb for itself and thus, your myth is busted. Vista's memory manager will use as much memory it can (free memory is a waste, so it will use it rather than watch it empty). But as soon as a process needs memory it will give it back.
It's time to realise that Abble's products are the biggest abomination these days. Just say NO to the dumb iAbble way!!
But, at least in Mac OS X, exited processes consume "inactive" memory - basically being kept in memory until they are launched again. If Vista has a similar implementation, your swapfile may just contain a bunch of pages left over from previously-running applications. Are you experiencing actual system performance problems? Concerning yourself too much with the numbers only can be a bad thing.
I'd assume what he's asking is: in modern systems where the amount of physical RAM is considerably larger than what most people's programs in total use, why does the OS ever swap RAM out to disk?
The answer is basically to free up RAM for disk cache, based on a belief (sometimes backed up by benchmarks) that for typical use patterns, the performance hit of sometimes having to swap RAM back into physical memory is outweighed by the performance gain of a large disk cache.
Of course, OS designers are always revisiting these assumptions---it may be that for some kinds of use patterns using a smaller disk cache and swapping RAM out to disk less leads to better performance, or at least better responsiveness (if that's the goal).
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
Virtual memory is now used for little tricks, in addition to providing more memory than is physically available.
One example is ring transitions into kernel mode which start out as exceptions. (Everyone seems to have ignored call gate, the mechanism Intel offered for ring transitions).
Another is memory mapped pointers. It is cool to be able to increment a pointer to file backed ram and not have to care if it is in ram or not.
Maybe the OP is onto something. Imagine writing Windows drivers without having to worry about IRQL and paging.
This should generate some polarized discussion.
There are two camps of thought.
One will insist that, no matter how much memory is currently allocated, it makes more sense to swap out that which isn't needed in order to keep more free physical ram. They will argue until they are blue in the face that the benefits of doing so are good.
Essentially - your OS is clever and it tries pre-emptively swap things out so the memory will be available as needed.
The other camp - and the one I subscribe to - says that as long as you have enough physical ram to do whatever you need to do - any time spent swapping is wasted time.
I run most of my workstations (Windows) without virtual memory. Yes, on occasion, I do hit a "low on virtual memory error" - usually when something is leaky - but I prefer to get the error and have to re-start or kill something rather than have the system spend days getting progressively slower, slowly annoying me more and more, and then giving me the same error.
This is not to say that swap is bad, or that it shouldn't be used - but I prefer the simpler approach.
Each process has it's own addressspace. This is necessary and usefull for several reasons. .exe files usually contain no relocation section and thus need to be mounter always at the same virtual address. .dlls) can be mapped as copy-on-write using the physical RAM only once, even if they are mounted in several process. Once a process modifies the executable it gets its own copy of the modified page, and the virtual address does not change.
1) You can get memory at any virtual address you request. This is necessary as
2) Processes are well seperated, and on a 64Bit system each 32bit process can get a memoryspace which is addressable with a 32bit pointer, but the whole system having more memory than thet.
3) executable code (mainly
And if you are talking about paged memory, it is still usefull even if you still have enough RAM, as you can reserve memory for future use. To be able to satisfy a commit of that memory at a later time the OS reserves that space in the swap file. As there is no data to write to the HDD you get no performance hit.
I can finally put my CS degree to good use, answering the same questions students would ask the TAs in basic OS and systems-level programming courses! ...except that the other comments have already answered the question. So, in true CS fashion, I will be lazy and refrain from duplicating effort ;)
Laziness is a virtue! (And that's on-topic, because a lazy paging algorithm is a good paging algorithm).
I recall back in 2002 or so, a friend of mine maxed out his Windows XP system with 2gb of memory. Windows absolutely refused to turn off paging (swap), forcing him to whatever the minimum size was. The solution? He created a RAMdisk and put the paging file there.
On Linux (and other modern systems, perhaps now including Windows), you can turn off swap. However, the Linux kernel's memory management isn't so great at the situation you hit when you need more memory than you have, but you can't swap. Usually, the memory hog crashes as a result (thankfully, Firefox now has session restore). I might be slightly out of date on this one.
A well-tweaked system still has swap (in nontrivial amounts), but rarely uses it. Trust me, you can afford losing the few gigabytes from your filesystem. Again in Linux, /proc/sys/vm/swappiness can be tweaked to a percentage reflecting how likely the system is to swap memory. Just lower it. (Though note the cons to this presented at the kerneltrap article above.) My workstation currently has an uptime of 14 days, a swappiness of 60, and 42/1427 megs of swap in use as opposed to the 1932/2026 megs of physical memory in use at the moment.
This is summarized for Windows and Linux on Paging at Wikipedia.
Use my userscript to add story images to Slashdot. There's no going back.
thinks "Virtual Memory" is the same thing as paging...
Mac Classic (OS 8 for sure) used the term "Virtual Memory" the same way Windows today uses "Page File" or unix uses "swap", so you can at least understand why some people might be confused by this.
db
I am literally 3000 tokens away from the chaotic crossbow --Stephen
Okay, so we've got most of the "you can run Vista with 4GB?!" jokes out of the way (hopefully). Here's my take on the situation.
...). With your configuration, that's when you'd start to crash.
I have Vista x64 running in a machine with 8GB physical memory, and no page file. I can do this because I'm never running enough memory-hungry processes that I will exceed 8GB allocated memory. So, while the OS may be good at deciding what gets swapped to the hard disk, in my case, there's simply no need, as everything I'm running can be contained entirely within physical memory (and for the curious, I've been running like this for a year and a half, haven't run out of memory yet).
However, if you don't have enough physical memory to store all the processes you might be running at once, then at some point the OS will need to swap to the hard drive, or it will simply run out of memory. I'm honestly not sure exactly how Vista handles things when it runs out of memory (never been a problem, never looked into it), but it wouldn't be good (probably BSoD, crash crash crash). I can tell you from personal experience that I regularly exceed 4GB memory usage (transcoding a DVD while playing a BluRay movie while
Long story short, with just 4GB, I would leave the swap file as is. Really, you should only disable the swap file if you know based on careful observation that your memory usage never exceeds the size of your installed physical memory. If you're comfortable with the risks involved, and you know your system and usage habits well, then go for it. Otherwise, leave it be.
Vista reserves 1 GB to itself, so your system will only ever have 3 GB available for processes.
Not exactly. 32-bit OSes won't normally report more than about 3.2 GiB of system RAM, as a 32-bit OS can only address 4 GiB (PAE/himem aside), and the upper addresses are reserved for hardware.
64-bit OSes (even Vista) will use and report RAM to much higher upper limits.
Or something like that.
I am literally 3000 tokens away from the chaotic crossbow --Stephen
Wrong. I routinely run my laptop in this configuration. No issues.
I think he is referring to the userspace/kernelspace split in Windows NT. On 32bit Windows XP, by default, the userspace (ring3) will have at most 2 GB of the physical RAM, and the kernel space would get the rest (some of it paged and some of it not). On systems with more than 3G of RAM (a lot by 2002 standards), it was kinda pointless to reserve that much for the kernel space, so they added a boot.ini flag that changed the split to _AT_MOST_ 3GBytes for the userspace and the rest for kernel space.
In Vista the split for 3G/1G of RAM is default. Actually on a system with 4G of RAM running in 32bit mode, you can't use all of them even if you try (in Windows XP), because right under the 4G limit you would have the PCI memory address mappings, that can be as large as 512M for a common video card with half a gig of RAM. Add to that the RAID controllers and the other hardware, and you have about 800megs of RAM unused because they can't be addressed, as their address-space is used by the installed devices.
I think that http://support.microsoft.com/kb/823440/ and http://support.microsoft.com/kb/171793/ should describe what I'm talking about pretty clearly.
UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever ones.
What is left over is the physical memory needed by the system. It seems like the OS preferred a fixed amount of memory, so it would just set up fixed space on the hard disk. So, even if all you have a 1 MB of available memory, the system would set up say 10MB, and that is what would be used. The pages that are being used will be stored in the physical ram, while everything would be stored on the HD.
If page management is working correctly, this should be transparent to the user. The management software or hardware will predict what pages were needed, and transfer those page to ram. One issue we I had was available memory was not hard disk plus physical available ram, but was limited by the available hard disk space.
So, it seems to me that virtual paged memory is still useful because with multiple applications loaded, memory can be a mess, and big fast hard drives it should not be an issue. I don't how Vista works, but it seems that *nix works very hard to insure that the pages that are needed are loaded to physical memory, and page faults do not occur. In this case, where virtual memory equals available physical memory, it would seem that since only physical memory is being used, there would be no performance hit from virtual memory. it is only there in case an application is run that need more memory. It is nice that we do not get those pesky memory errors we got in the very old days.
"She's a scientist and a lesbian. She's not going to let it slide." Orphan Black
Linux kernel maintainer Andrew Morton sets his swappiness to 100 (page as much physical memory as you can, the opposite of this Ask-Slashdot's desires), which he justified in an interview (see above link) by saying:
My point is that decreasing the tendency of the kernel to swap stuff out is wrong. You really don't want hundreds of megabytes of BloatyApp's untouched memory floating about in the machine. Get it out on the disk, use the memory for something useful.
Of course, there's another view, also presented at the above kerneltrap article: If you swap everything, you'll have a very long wait when returning to something you haven't touched in a while.
If you have limited resources, milk the resources you have plenty of; workstations should have high swappiness while laptops, who suffer in disk speed, disk capacity, and power, are probably better suited with lower swappiness. Don't go crazy, though ... swappiness = 0 is the same as running swapoff -a and will crash your programs when they need more memory than is available (as the kernel isn't written for a system without swap).
Use my userscript to add story images to Slashdot. There's no going back.
Guess what. I AM running Vista on such a setup. Your move, AC.
One word: Multics. Way too far ahead of its time. Those who forget history will have to try to re-invent it. Badly.
It should be. However, people will keep developing bloated programs that require far more RAM then is required to perform their task.
Spelling and Grammar errors have been added to this post for your enjoyment
Ram is cheap. If you are swapping, buy more ram.
Computers are cheap. If adding more ram doesn't fix the swapping on your current computer, buy a new computer that can use more ram.
Nerd rage is the funniest rage.
That page mostly talks about what virtual memory is and doesn't directly list why it is an improvement.
Some folks have already mentioned the fact that it eliminates memory fragmentation, and that it allows mapping of files and hardware into memory without dedicating (wasting) part of the address space to those uses.
Another reason is that you can have 2^64 bytes of total system memory, even if the individual applications are 32-bit, and can only address 2^32 bytes of memory. Since the 32-bit applications are presented a virtual address space, it doesn't matter if their pages are located above the 32-bit boundary.
It means that per-process memory protection is enforced by the CPU paging table. Without virtual memory you would have to reimplement something like it just for memory protection.
It means that the linker/loader don't have to patch the executable with modified address locations when it is loaded into memory.
The above two reasons have the corollary that libraries can be shared in memory much more easily.
And that's just off the top of my head. Virtual memory is a very, very useful thing.
You must be confused about virtual vs. physical memory. In modern processors, there is no penalty for using virtual memory, all translation from virtual to physical address space is done internal to the processor and you won't notice the difference.
Huh? That's totally wrong. If it were true, you wouldn't need any RAM.
It's true that address translation is hard-wired in modern processors. But that just means that figuring out where the data is is as fast as for data that's already in RAM. Actually reading or writing it is only as fast as the media it's stored on. So if you have a lot of big applications running, and there isn't enough RAM for them all to be in physical memory at once, your system "thrashes", as data migrates back and forth between the two media. That's why adding RAM is very often the best way to speed up a slow system, especially if you're running Microsoft's latest bloatware. Defragging the swap disk can also be helpful.
To answer the original question: actually, you often don't need any virtual memory. But sometimes you do. Disk space is cheap, so it makes sense to allocate a decent amount of virtual memory and just not worry about whether it's absolute necessary.
Windows makes me CRAZY about this. the OS is internally configured to use an LRU algorithm to aggressively page.
("Technical bastards" who question my use of paging and swap interchangeably in this post can send their flames to /dev/null \Device\Null or NUL depending on OS)
What I found when disabling paging on an XP pro system with 2GB RAM is that the system performance is explosively faster without the disk IO.
Even an *idle* XP pro system swaps - explaining the time it takes for the system to be responsive to your request to maximize a window you have not used in a while.
I was thrilled to have a rocket-fast system again - until I tried to hibernate my laptop. Note that the hibernation file is unrelated to the swap/paging space.
The machine consistently would blue screen when trying to hibernate if swap/paging was disabled. Enabling swap enabled the hibernation function again. Since reboots take *FOREVER* to reload all the crap that XP needs on an enterprise-connected system - systems mangement, anti-virus agent, software distribution tool, and the required ram-defragger which allows XP to "stand by" when you've got more than 1GB of RAM, plus IM, etc
I reboot as infrequently as possible and consider "stand by" and "hibernate" required functions. As a result, I live with XP and paging enabled, and tolerate the blasted system "unpaging" apps that have been idle a short time.
Poo!
But Herr Heisenberg, how does the electron know when I'm looking?
Circa 1975 through 2000 or so, the "native" (*) versions of the Pick Operating System worked exactly this way. Even the OS-level programmers - working in assembly language (!) - only saw virtual memory in the form of disk pages. When you put an address in a register, it was a reference to a disk page, not physical memory. The pages were auto-magically brought into memory at the moment needed and swapped out when not by a tiny VM-aware paging kernel. That was the only part of the system that understood that there was anything besides disk storage in the entire system.
Might sound inefficient, but I remember the first Pick system I worked on supported 14-16 simultaneous online users using only 64kb of physical memory (and it was core - REAL core - at that point).
Now get off my lawn, kids.
-TSG-
(*) "Native" meaning "with no other OS involved underneath, bare Pick on the metal". These types (Reality, Ultimate, CDI Series/1, etc) were mostly gone by Y2K in favor of more modern systems with underlying host OS (Un*x or NT) handling the hardware and Pick riding on top as a database/programming environment (Universe, Unidata, AP / D3). Though I'm sure there are still hundreds if not thousands of the old systems chugging away in back rooms of warehouses and the like to this day.
If it's there, and you can see it . . . it's real.
If you can see it, but it's not there . . . it's virtual.
If you can't see it, and it's not there . . . it's gone.
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
Because you don't just use RAM to hold your processes; you use it for caching frequently accessed data from your (comparatively slower) hard disks as well. Thus, there is never any such thing as "enough" RAM, that is, until you have enough primary storage to equal the sum of all the secondary storage you'll use in a single computing session PLUS the amount of memory needed by all your running processes.
But nobody has that much memory. It's a waste of money. So, we trust the OS to swap out inactive pages and then fill the remaining space with disk caches. Then we spec our systems with as much primary storage as we need to contain actively used memory from processes as well as a healthy disk cache for the persistent data we're working on. With modern memory management, we get substantially higher disk access performance as well as a system that's affordable because it doesn't contain a terabyte of solid-state memory that can't even remember anything through a reboot.
Yes the OP is right if you don't page to disk and go off all RAM then you will be faster. However with a good paging it will help you from things from getting slower or not working when you really need the extra Horse Power, and you probably wouldn't even notice it.
First we got the 80/20 rule where 20% of the Data is used 80% of the time. So a large chunk of data will rarely be used, being that it not being used read or written just kinda sitting there. You might as well page it to disk so you have more space free.
Next if you get a big chunk of memory request say you open a VM system that need Gigabytes of memory. Say 3 GB and you only had 2 GB free. That means before you app can run you will have to wait for 1GB of data to be dumped to the disk. Vs. say a good paging algorithm which would already have that 1 GB already paged so you can fill the RAM with the VM for a faster access then probably depending on the paging algorithm pieces will slowly get paged back to disk allowing you run say an other 512meg load on your system without having the system dump that 512meg of data. If you didn't page you would be stuck as you don't have the ram to run the application. Or a poor paging algorithm will spend so much time paging the data until it gets enough free to operate.
Drive space is relatively cheap if you are going to do some high RAM intensive apps. With a good paging you can get by with about half as much RAM saving money.
Most systems have more ram then ever but the apps use more ram then ever too. (This isn't necessarily bloat) Lets say your app does a lot of Square roots. The time it takes for it to process say 1,000,000 Square Roots vs. Dumping to memory the recalculated Square Roots values and doing a quick memory lookup of the answer. That way you get faster calculation time at cost of RAM.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
One problem is that there are relatively frequent types of disk-access patterns where caching them gives little to no benefit in return for the paging out of RAM it requires. A virus scan (touching most of your files exactly once) is one canonical example. Media playback (touching a few large files in sequential block reads) is another.
The difficult question is how to exclude these kinds of wasted caching while still retaining the benefits of caching frequently accessed files, and not introducing excessive overhead in the process.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
There has been this ridiculous notion floating around recently that swap space and paging files are relics and need to be eliminated. You can only safely eliminate them only so long as you're 100% confident you'll never use more RAM than you actually have. But there are lots and lots of memory hogging applications - video editors, image editors, scientific applications, etc. And when you consider that a web browser can eat up to 300MB of RAM, it shouldn't be hard to imagine a multitasking user running out by using too many little programs.
I note a lot of people are insisting that "virtual memory" refers to the virtual address space given to a execution context, and what the author really means is "paging".
The funny thing is that these are traditionally poorly defined/understood terms which are gaining a hard consensus for the meanings due to some recent OS books, and poor comp-sci education which insists on a particular definition. Everyone is faulting M$ for using the term incorrectly, even though the original mac OS and other OS's used the term in the same way. Wikipedia defines it one way and then goes on to give historical systems which don't really adhere to the definition. For example the B5000 (considered the first commercial machine with virtual memory) didn't even have "contiguous" working memory as required by the wikipedia definition. It had what would be more specifically called multiple variable sized segments which could be individually swapped. Again, the mac OS evolved from a single process model to muliprocess, in the same address space (look up mac switcher) and implemented "virtual memory" using a system without a MMU by swaping the allocated pieces of memory to disk if they weren't currently locked (in use) and reallocating the memory. Aka they had "virtual memory" in single fragmented address space.
The other example is people use "paging" to describe the act of swaping portions of the memory to disk, misunderstanding that paging is more about splitting an address space or segment up into fixed pieces for address translation to physical, and that disk swapping of pages isn't required for paging. Aka, your system is still "paging" if you disable swapping.
Even the term swapping is unclear because the need to differentiate between swaping pages, and swapping whole processes (or even segments) resulted in people avoided the term swapping to describe systems which were swapping pages instead of segments/regions/processes. These systems were generally called "demand paged" or something similar to indicate that they didn't need to swap a complete process or dataset (see DOSSHELL).
So, give the guy a break, in may ways he is just as correct, if not more so.
Perhaps set up a small ramdisk and pf to that?
"If you have nothing to hide, you have nothing to fear." - Every fascist, ever
4gb on OSX here and I've got 0 bytes in swap even after having both Wow and firefox up for an hour or two.
Windows is very agressive in what it puts into swap.. if you come to a machine after it's been idle for a few hours it runs like a dog while it swaps everything back.. that's a royal pain at work where we're doing database work - if mssql gets paged out it takes a *long* time to recover to a running state. I tend to issue a create database and go and make my morning cup of tea.
which is written as if it were just like any other compute task, to be scheduled as the OS sees fit (maybe with bumped up priority, but that's not enough to make sure the right thing happens). It instead has to be scheduled as a realtime task with guaranteed bounds on the response time for low-overhead user operations, which means locking stuff in RAM even at the cost of more swapping on less interactive tasks. That also means turning on realtime kernel support in systems that don't already have it active. I've thought for a while that the Linux window system should be rewritten by game programmers, who tend to have some clue about how to identify the parts of an interactive program have to be responsive, and to make those parts actually be responsive.
The amount of physical memory you have has nothing to do with the 2gb|2gb split or 3gb|1gb split. You can use your full 3gb either of physical memory either way, since the virtual mappings (which is what is split) is different for each app, so two apps can each use 1.5gb of user space and take up 3gb total memory.
The actual reason for the /3GB flag is to enable OS support for special applications which explicitly opt-in for up to 3gb of memory, usually to help boost performance (they can have a larger cache or buffer or whatever)... normal apps will still only be able to use up to 2gb because for compatibility. Apparently some apps use the highest bit of a 32-bit pointer to store a boolean, since with 2gb of user space that highest bit is always 0. Adding an extra 1gb allows that bit to be 1 which could seriously confuse the program, likely causing a crash.
SQL Server includes support for 3GB, IIRC.
I'm not sure if there's a downside to using 3gb (there must have been for it to be optional and hidden like that) especially since apparently Vista turns it on by default... however IIRC while each app gets it's own 2gb/3gb of userspace, kernel address space is shared between all apps, so your kernel only gets 1gb total memory. That includes all running drivers. I imagine the XP team thought it was better to allow for the total 2gb for drivers, especially since most apps couldn't take advantage of the full 3gb anyway (as I described above).
My understanding is that Windows uses the pagefile primarily for single-use and rarely-used stuff, a great deal of which is read only at boot-time or when programs start up. So it can indeed "fill up" even when there's plenty of RAM left over. And you don't really want rarely-used data cluttering RAM if your RAM supply is limited.
BUT... "rarely used" is a relative term. There is a registry setting (which I don't know off the top of my head tho I'm sure a search would find it) that forces Windows to keep all that "rarely used" OS-related stuff in RAM instead of paged out, and I gather this setting improves performance immensely -- for the same reasons as killing the pagefile entirely does (reads from disk are painfully slow compared to reads from RAM).
As to the slow boot -- did you have the pagefile set permanent? and preferably on its own partition so it can't become fragmented? Temporary pagefile that has to be written at boot time, plus fragmented all to hell (which does nothing for stability) due to the disk being fragmented, is about the slowest/worst way to do it.
First thing I do on any machine that's going to have a pagefile is give the nasty thing its own partition, which the user is forbidden to use for anything else. If for some reason a separate partition is impossible, I kill the pagefile and all tempfiles, defrag, then reset the pagefile at a fixed permanent size, so it won't refragment.
~REZ~ #43301. Who'd fake being me anyway?
Here are two great posts by the famous Mark Russinovich, they may give some insight:
- http://blogs.technet.com/markrussinovich/archive/2008/11/17/3155406.aspx
- http://blogs.technet.com/markrussinovich/archive/2008/07/21/3092070.aspx
The answer to why /PAE and /3GB are not defaults is 3rd party drivers.
Yes, believe it or not, there are Windows drivers that believe user-space pointers and kernel-space pointers will always and forever be in the same 2GB memory spaces they have always been in.
QNX, the real-time operating system, does not page to disk. (Not for most processes, anyway. Read on.) The GUI has a bar in the lower right hand corner which shows how much memory is in use; if memory fills up, processes have requests for more memory rejected.
The big advantage of this is consistent performance. Hard real time works. There are no pauses waiting for the disk. This is what you want for entertainment applications, like video players, audio players, and games.
It's possible to set up paging for specific programs, though. GCC has paging, so that huge programs can be compiled, somewhat slowly, on smaller systems. By default, though, paging is not used. Provided that applications aren't bloatware, this works quite well.
It's something to think about for Linux. Should programs be paged by default? Maybe that era is over.
If you consider the "REAL" memory to be that stuff that's normally labeled "cache".
Example this Athlon 64 had 1MB of physical memory on the CPU divided into "pages" of 8 bytes. If your working set is less than a megabyte you will be running entirely from this memory and running very very fast.
As soon as you start 'paging out' to the so called "main memory" your performance takes a dive.
The only differences between this and the level normally mentioned are speed, size and the fact that this level is usually controlled completely by the hardware.
Ummm... no. There are a statistically significant number of humans who aren't notably good at anything. I have unhappily encountered too many of them, both in and out of tech work. This is akin to actually believing that "all men are created equal" merely because it would be really really neat and make you feel all warm-and-fuzzy inside if it were true.
Even if your pollyanna perspective was true, being competent at some task doesn't directly equate with an absence of dumbassery. There are numerous species of "dumb" creatures that can be trained to memorize some task and then mimic (repeat) it perfectly ad nauseum... including H. sapiens. An ability to memorize and mimic doesn't equate directly with intelligence. It's a precursor, a prerequisite, perhaps, but not the Real McCoy.
A shocking number of humans, including many regarded as "average" by testing standards, never actually reach a state of true intelligence. Too many of them are profoundly ignorant and quite determined to remain that way.
If you have caches of a size smaller than your real RAM, the order in which you try to access memory really CAN make a difference because cache is many times faster than regular RAM and will try and do things like speculative readahead. If what you are working with is already in the cache by the time you request it then you won't stall for as long.
If you are forever causing the cache to become flushed and forcing the cache to be refilled with a different contents (perhaps because you are causing a large number of random memory access and the cache's readahead is getting your future access wrong so it has to be turned off) then performance will by comparison be slower than a sequential memory access workload.
The above is of course a gross simplification (and doesn't apply if what you are reading fits entirely within cache and is already there). If you have the technical chops you can read more about how order of access can have an impact on speed in Ulrich Drepper's what every programmer should know about memory on LWN.
HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management
DisablePagingExecutive
0 = Drivers and system code can be paged to disk as needed.
1 = Drivers and system code must remain in physical memory.
Ummm... no. There are a statistically significant number of humans who aren't notably good at anything.
to be more precise a lot of people are not good at anything that we know and value, they could be good at stupid things (soccer strategy) and never know it, never make others know and certainly never apply it to something good.
but teoretically anyone could have some bright (but unuseful/unused) capacity.
There are numerous species of "dumb" creatures that can be trained to memorize some task and then mimic (repeat) it perfectly ad nauseum... including H. sapiens. An ability to memorize and mimic doesn't equate directly with intelligence. It's a precursor, a prerequisite, perhaps, but not the Real McCoy.
you forget a prerequisite of intelligence evolution, THE NEED FOR IT.
if they could live their lives in a dumb way many lazy person never try to reach an higer lever of intelligence, never ever feel their lack of it!
A shocking number of humans, including many regarded as "average" by testing standards, never actually reach a state of true intelligence. Too many of them are profoundly ignorant and quite determined to remain that way.
the deepnes of their ignorance is an huge problem because they never perceive their ignorance, and many of the few that perceive it refuse to believe it and assault the condition that make them feel stupid.
that is the sad problem.
It's a FESTIVUS miracle.
You insensitive clod.
Where's my aluminum pole?
Long before Windows, virtual memory was 'invented'. Given that, the term has a specific meaning. As others have mentioned, it is a method for making programs believe they have unlimited memory space, whilst sharing the actual available physical memory between numerous programs. This 'feature' has a cost - references to memory must be translated from a virtual address to a physical one, by memory management hardware (and sometimes software). Until most recently, Intel processors used a separate chip to manage this. AMD put their memory controller onboard a few years ago. In terms of memory performance, Intel lagged for the past few years because their outboard memory controller consumed extra time to do its job. Moving the controller onboard removes an electrical interface or two, thus speeding things up and generally improving efficiencies.
The original post, I thought, was brilliant. Why are we devoting all this chip real estate (or, in the past, chips), to sharing a rare resource (memory) when that resource is no longer rare? Grant, virtual memory gives us other advantages such as ensuring one program doesn't write in the memory space of another, but surely there are other ways to do that. If we did away with virtual memory and returned to the old (ack! DOS) days of physical memory references, we could devote that chip real estate, power quota, etc to other worthwhile pursuits, like making my twitter pages load faster.
- The Kessel run is for nerf herders. I can circumnavigate the entire Central Finite Curve in a lot less than 12 parse
I know I am late in replying, but maybe you'll look down this far. The simple fact is, regardless of how much RAM you have, there are some things that Windows wants to page. Period. So, you really have to situations when the system pages (grossly simplified):
1. The system is low on physical memory (RAM). This is what people USED to encounter a lot. With RAM being cheap nowadays, this happens far less. When this happens, your system will slow down a lot, and it is considered a Bad Thing(tm). The reason the system slows down is that RAM is fast and disk is slow. Paging (aka using the swapfile) is a slow process compared to RAM. Similarly, there are wasted cycles as the system determines that it has to page, so this type of virtual memory use needs to be avoided.
2. Paging as architected. Windows, regardless of the amount of RAM, will page certain things. Even if you had 32GB of RAM, it would still page some stuff. This paging should not affect performance.
You are experiencing paging type #2, most likely. It is possible you are thrashing your system's 4GB of memory through the use of big applications, but unlikely. If Task Manager or perfmon show lots of free physical memory, you are paging normally and should not be concerned.
Of course, it is possible that you have an application improperly paging data, artificially causing situation #1, which would be bad, but this is unlikely. If you see lots of disk access and experience system slow-down, this is an indication something is wrong.
Windows never says that virtual memory is a memory paging file. They have a section titled "Virtual memory" with an OPTION for the size and location of the memory paging file, which is a feature of the virtual memory subsystem. I'm sorry if you don't understand the difference, I've explained it as simply as I can.