Slashdot Mirror
Why Use Virtual Memory In Modern Systems?
Cyberhwk writes "I have a system with Windows Vista Ultimate (64-bit) installed on it, and it has 4GB of RAM. However when I've been watching system performance, my system seems to divide the work between the physical RAM and the virtual memory, so I have 2GB of data in the virtual memory and another 2GB in the physical memory. Is there a reason why my system should even be using the virtual memory anymore? I would think the computer would run better if it based everything off of RAM instead of virtual memory. Any thoughts on this matter or could you explain why the system is acting this way?"
You must be confused about virtual vs. physical memory. In modern processors, there is no penalty for using virtual memory, all translation from virtual to physical address space is done internal to the processor and you won't notice the difference.
So all the physical memory installed in your PC is used by the processor as one big pool of resources. Processes can think whatever they want and address huge memory spaces, that's all in virtual land. Virtual memory only starts impacting performance when pages are being swapped in and out, because all your processes need more resident memory than you actually have.
Swapping means accessing the disk and freezing the requesting process until its page of memory has arrived from the disk, which takes millions of processor cycles (a lifetime from the processor's point of view). It's not so bad if you swap once, as the processor can work on other processes while waiting for the data to arrive, but if all your programs keep pushing each other out of physical memory, you get thrashing and consider yourself happy if the mouse pointer is still responsive!
So you may want to change the title of your post to: "why use physical memory in modern systems?". I would point you to an article I wrote on that topic in 1990, but somehow I can't find a link to it on the web :-)
fairsoftware.net - software developers share revenue from the apps they build
Memory exists to be used. If memory is not in use, you are wasting it. The reality is that your system will operate with higher performance if unused data is paged out of RAM to disk and the newly freed memory is used for additional disk caching. Vista's memory manager is actually reasonably smart and will only page data out to disk when it really won't be used, or you experience an actual low-memory condition.
Virtual memory and pagefiles still exist so that there will be persistent, recoverable storage of your browsing and search history, illegally downloaded music, and furrie porn should anybody come a-knockin after you hit the power switch.
[/tinfoil hat]
Vista boots in 4 GB? I'm impressed!
:-)
...laura
Are you asking about modern systems or Windows?
you could create a RAM Disk and set your page file to use that.
Then all your virtual memory is in RAM.
I'll leave it to someone else to explain why that isn't a good idea.
"The stupid neither forgive nor forget; the naive forgive and forget; the wise forgive but do not forget." -Thomas Szasz
We who know what we are doing are free to take the risk of running our computers without a swapfile.
Most people are not in a position where they can be sure that they will never run out of physical memory. Because of that, all operating systems for personal computers set up a swapfile by default: It's better for joe average computer owner to complain about a slow system than for him to lose his document when the system crashes because he filled up the physical memory (and there is no swap file to fall back on).
Stop the brainwash
The other extreme point of view is that modern systems should only have virtual memory and, instead of having an explicit file system, treat mass storage as a level-4 cache. In fact, systems that support mmap(2) do this partially.
The idea here is that modern memory management is actually pretty good, and that it's best to let the OS decide what to keep in RAM and what to swap out, so that issues like prefetching can be handled transparently.
Mental Note: Slashdot Editor Timothy doesn't know the difference between paging memory to disk and virtual memory.
For that matter, why do we even need to explicitely "save" anymore? Why does the fact that Notepad has 2KB of text to save prevent the shutdown of an entire computer? Just save the fecking thing anywhere and get on with it! Modern software is such a disorganized mess.
The problem is whatever Vista is using for page replacement, not virtual memory itself. Let's say you're using 2 GB of physical memory, and then start up some memory-intensive program that uses another 2 GB. You're done; if you run out of space now, the OS can do nothing about it, and is forced to do something drastic like start killing off processes.
The question then is why isn't Vista making fullest use out of virtual memory? It is probably trying to proactively move little used pages out of physical memory to make space for new pages. Its an attempt at optimization--after all, if you're not using that data, why let it take up valuable physical memory?
Absolutely not true. You can even install and run Vista on a computer with 1Gb ram and no page file. And run applications. So it doesn't reserve 1Gb for itself and thus, your myth is busted. Vista's memory manager will use as much memory it can (free memory is a waste, so it will use it rather than watch it empty). But as soon as a process needs memory it will give it back.
It's time to realise that Abble's products are the biggest abomination these days. Just say NO to the dumb iAbble way!!
Its Vista. Thats a feature ;)
Windows really sucks at managing large amounts of memory. I hit this all the time, where I start a build of my project, then I click in a web browser window and I have to wait 10 seconds for Windows to page back in the pages that it "helpfully" swapped out with no fucking benefit to me whatsoever. These days, disk I/O is an insanely bad bottleneck on all the Windows machines that I actually use for stuff. Paging is just another useless thing competing for my available I/O bandwidth and imposing a ridiculous latency on me whenever I switch to an app I haven't used for a while.
The only reason I haven't turned off my swap file is that I sometimes need to have more stuff running than RAM I actually have.
You seem to be confusing virtual memory with physical memory and swap space. Try this.
The virtual memory system is not just used for swap. Instead of loading whole files into memory, the system can map the file into the address space and only load the pages which the program actually accesses. This file then becomes "virtual memory" and this way of doing it is faster than not using the virtual memory system. That's why you should always keep virtual memory enabled, even when you have enough RAM.
But, at least in Mac OS X, exited processes consume "inactive" memory - basically being kept in memory until they are launched again. If Vista has a similar implementation, your swapfile may just contain a bunch of pages left over from previously-running applications. Are you experiencing actual system performance problems? Concerning yourself too much with the numbers only can be a bad thing.
http://blogs.technet.com/markrussinovich/
Note that in Windows-based machines, 'Virtual memory' is synonymous with the Page (swap) file.
Therefore, the OP is wondering why this needs to be the case at all, and it's a good question, as Windows always seems to enjoy using the disk when there's perfectly good RAM available.
I've forced my system to use no paging file, which greatly increases performance. However, any faults in your memory will show up more frequently (as is the case for me when I'm using a large portion of my RAM -- but not enough to get new sticks just yet :P )
[DISCLAIMER: This post is a work of satire and should not be misconstrued as a holy text upon which to base a religion.]
4GB should be enough for anybody!
Fascism starts when the efficiency of the government becomes more important than the rights of the people.
You're using Windows 64 ultimate and it seems slow on 4GB?
Well of course. Its Microsoft.
MSBPodcast.com The opinions expressed here are my own. If you don't like 'em... Think up your own stuff.
I think the more relevant question is why use swap if you have enough ram? If you're 100% certain that everything you do will always fit inside physical memory, technically you can do away with swap. I'm not sure how Windows will deal with this however.
I have not lost my mind... it's backed up on disk somewhere!
I'd assume what he's asking is: in modern systems where the amount of physical RAM is considerably larger than what most people's programs in total use, why does the OS ever swap RAM out to disk?
The answer is basically to free up RAM for disk cache, based on a belief (sometimes backed up by benchmarks) that for typical use patterns, the performance hit of sometimes having to swap RAM back into physical memory is outweighed by the performance gain of a large disk cache.
Of course, OS designers are always revisiting these assumptions---it may be that for some kinds of use patterns using a smaller disk cache and swapping RAM out to disk less leads to better performance, or at least better responsiveness (if that's the goal).
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
Yes, for memory protection. Oh, you mean swap. Well,
not everything mapped into a process's address space is in constant use. Swap it out, use the RAM for...
Virtual memory is now used for little tricks, in addition to providing more memory than is physically available.
One example is ring transitions into kernel mode which start out as exceptions. (Everyone seems to have ignored call gate, the mechanism Intel offered for ring transitions).
Another is memory mapped pointers. It is cool to be able to increment a pointer to file backed ram and not have to care if it is in ram or not.
Maybe the OP is onto something. Imagine writing Windows drivers without having to worry about IRQL and paging.
The algorithms in windows that decide when and what to page to disk are fucking insane. No matter what it will always do that. We have a vista64 box with 16gigs of ram and - surprise the same shit! I've tried to disable swap altogether and it just can't be done in 'doz. It will for instance turn it back on, or give crazy errors.
This should generate some polarized discussion.
There are two camps of thought.
One will insist that, no matter how much memory is currently allocated, it makes more sense to swap out that which isn't needed in order to keep more free physical ram. They will argue until they are blue in the face that the benefits of doing so are good.
Essentially - your OS is clever and it tries pre-emptively swap things out so the memory will be available as needed.
The other camp - and the one I subscribe to - says that as long as you have enough physical ram to do whatever you need to do - any time spent swapping is wasted time.
I run most of my workstations (Windows) without virtual memory. Yes, on occasion, I do hit a "low on virtual memory error" - usually when something is leaky - but I prefer to get the error and have to re-start or kill something rather than have the system spend days getting progressively slower, slowly annoying me more and more, and then giving me the same error.
This is not to say that swap is bad, or that it shouldn't be used - but I prefer the simpler approach.
Vista for me idles fresh boot at about 300-400mb. When you see large spikes it's probably prefetching applications and loading them into ram. This way when you go to load world of warcraft every day at 5:30pm it takes a fraction of the time on the initial load. The memory used to store prefetched applications appears as part of "windows' memory" before being loaded so it can make it seem like windows vista is actually using 1gb when in fact most of that is WoW, or Photoshop, or Counterstrike, etc.
The operating system will page out pages that have never been used, or used very infrequently, if at all.
If you have a 6MB program, and there is 500K of code that's used on startup, but never referenced again, let the operating system page it out, and use that RAM for other programs, or cache. The same goes for code that implements functionality that is rarely (if ever) used. Why load it, if it never gets referenced?
In a regular program, there are lots of pages that are referenced once, or not at all.
Canonical Anonymous Coward
Can a sig be more clever than it's creator?
Each process has it's own addressspace. This is necessary and usefull for several reasons. .exe files usually contain no relocation section and thus need to be mounter always at the same virtual address. .dlls) can be mapped as copy-on-write using the physical RAM only once, even if they are mounted in several process. Once a process modifies the executable it gets its own copy of the modified page, and the virtual address does not change.
1) You can get memory at any virtual address you request. This is necessary as
2) Processes are well seperated, and on a 64Bit system each 32bit process can get a memoryspace which is addressable with a 32bit pointer, but the whole system having more memory than thet.
3) executable code (mainly
And if you are talking about paged memory, it is still usefull even if you still have enough RAM, as you can reserve memory for future use. To be able to satisfy a commit of that memory at a later time the OS reserves that space in the swap file. As there is no data to write to the HDD you get no performance hit.
You also had to provide a display driver, even for devices with no display. Same deal, make a "fake one".
Engineering is the art of compromise.
I can finally put my CS degree to good use, answering the same questions students would ask the TAs in basic OS and systems-level programming courses! ...except that the other comments have already answered the question. So, in true CS fashion, I will be lazy and refrain from duplicating effort ;)
Laziness is a virtue! (And that's on-topic, because a lazy paging algorithm is a good paging algorithm).
Free Memory is wasted Memory. Even-More-Modern operating systems like Linux and BSD manage memory incredibly efficiently.
Do you mean that vista requires 1GB for loading all of it's libraries? Because I thought if you turned off services you don't need and Aero you could get it well under that.
If it actually reserves 1GB no matter what then...link or better explanation?
I recall back in 2002 or so, a friend of mine maxed out his Windows XP system with 2gb of memory. Windows absolutely refused to turn off paging (swap), forcing him to whatever the minimum size was. The solution? He created a RAMdisk and put the paging file there.
On Linux (and other modern systems, perhaps now including Windows), you can turn off swap. However, the Linux kernel's memory management isn't so great at the situation you hit when you need more memory than you have, but you can't swap. Usually, the memory hog crashes as a result (thankfully, Firefox now has session restore). I might be slightly out of date on this one.
A well-tweaked system still has swap (in nontrivial amounts), but rarely uses it. Trust me, you can afford losing the few gigabytes from your filesystem. Again in Linux, /proc/sys/vm/swappiness can be tweaked to a percentage reflecting how likely the system is to swap memory. Just lower it. (Though note the cons to this presented at the kerneltrap article above.) My workstation currently has an uptime of 14 days, a swappiness of 60, and 42/1427 megs of swap in use as opposed to the 1932/2026 megs of physical memory in use at the moment.
This is summarized for Windows and Linux on Paging at Wikipedia.
Use my userscript to add story images to Slashdot. There's no going back.
thinks "Virtual Memory" is the same thing as paging...
Mac Classic (OS 8 for sure) used the term "Virtual Memory" the same way Windows today uses "Page File" or unix uses "swap", so you can at least understand why some people might be confused by this.
db
I am literally 3000 tokens away from the chaotic crossbow --Stephen
Pst... You posted to ask Slashdot and mentioned you use Microsoft Vista? Flamebait
Flexible bare-metal recovery for Linux/UNIX
Okay, so we've got most of the "you can run Vista with 4GB?!" jokes out of the way (hopefully). Here's my take on the situation.
...). With your configuration, that's when you'd start to crash.
I have Vista x64 running in a machine with 8GB physical memory, and no page file. I can do this because I'm never running enough memory-hungry processes that I will exceed 8GB allocated memory. So, while the OS may be good at deciding what gets swapped to the hard disk, in my case, there's simply no need, as everything I'm running can be contained entirely within physical memory (and for the curious, I've been running like this for a year and a half, haven't run out of memory yet).
However, if you don't have enough physical memory to store all the processes you might be running at once, then at some point the OS will need to swap to the hard drive, or it will simply run out of memory. I'm honestly not sure exactly how Vista handles things when it runs out of memory (never been a problem, never looked into it), but it wouldn't be good (probably BSoD, crash crash crash). I can tell you from personal experience that I regularly exceed 4GB memory usage (transcoding a DVD while playing a BluRay movie while
Long story short, with just 4GB, I would leave the swap file as is. Really, you should only disable the swap file if you know based on careful observation that your memory usage never exceeds the size of your installed physical memory. If you're comfortable with the risks involved, and you know your system and usage habits well, then go for it. Otherwise, leave it be.
system requirements check it out seems that premium business and ultimate both support 1gig and the other support 512 so wtf are you smoking? Of course I would recommend that you use more than that but it still will run which debunks the OP.
split the page file in windows across multiple hard disks to create a fake RAID effect. And if one of those drives is a RAID array then all the better ;)
Vista reserves 1 GB to itself, so your system will only ever have 3 GB available for processes.
Not exactly. 32-bit OSes won't normally report more than about 3.2 GiB of system RAM, as a 32-bit OS can only address 4 GiB (PAE/himem aside), and the upper addresses are reserved for hardware.
64-bit OSes (even Vista) will use and report RAM to much higher upper limits.
Or something like that.
I am literally 3000 tokens away from the chaotic crossbow --Stephen
IBM: Z/OS Z/VM
Cordially,
Kilgore Trout
Swapping is a separate issue. You can virtualize memory with or without a swap file. Why? For starters, memory mapping gets rid of the entire problem of memory fragmentation at the system level. The list goes on from there, but eliminating fragmentation is huge. Anyone who's ever written or used a non-mapped OS will tell you in no uncertain terms why they suck!
Wrong. I routinely run my laptop in this configuration. No issues.
That's not true at all, it requires 700 Meg for the system and 300 meg for addition extras.
I think he is referring to the userspace/kernelspace split in Windows NT. On 32bit Windows XP, by default, the userspace (ring3) will have at most 2 GB of the physical RAM, and the kernel space would get the rest (some of it paged and some of it not). On systems with more than 3G of RAM (a lot by 2002 standards), it was kinda pointless to reserve that much for the kernel space, so they added a boot.ini flag that changed the split to _AT_MOST_ 3GBytes for the userspace and the rest for kernel space.
In Vista the split for 3G/1G of RAM is default. Actually on a system with 4G of RAM running in 32bit mode, you can't use all of them even if you try (in Windows XP), because right under the 4G limit you would have the PCI memory address mappings, that can be as large as 512M for a common video card with half a gig of RAM. Add to that the RAID controllers and the other hardware, and you have about 800megs of RAM unused because they can't be addressed, as their address-space is used by the installed devices.
I think that http://support.microsoft.com/kb/823440/ and http://support.microsoft.com/kb/171793/ should describe what I'm talking about pretty clearly.
UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever ones.
What is left over is the physical memory needed by the system. It seems like the OS preferred a fixed amount of memory, so it would just set up fixed space on the hard disk. So, even if all you have a 1 MB of available memory, the system would set up say 10MB, and that is what would be used. The pages that are being used will be stored in the physical ram, while everything would be stored on the HD.
If page management is working correctly, this should be transparent to the user. The management software or hardware will predict what pages were needed, and transfer those page to ram. One issue we I had was available memory was not hard disk plus physical available ram, but was limited by the available hard disk space.
So, it seems to me that virtual paged memory is still useful because with multiple applications loaded, memory can be a mess, and big fast hard drives it should not be an issue. I don't how Vista works, but it seems that *nix works very hard to insure that the pages that are needed are loaded to physical memory, and page faults do not occur. In this case, where virtual memory equals available physical memory, it would seem that since only physical memory is being used, there would be no performance hit from virtual memory. it is only there in case an application is run that need more memory. It is nice that we do not get those pesky memory errors we got in the very old days.
"She's a scientist and a lesbian. She's not going to let it slide." Orphan Black
Linux kernel maintainer Andrew Morton sets his swappiness to 100 (page as much physical memory as you can, the opposite of this Ask-Slashdot's desires), which he justified in an interview (see above link) by saying:
My point is that decreasing the tendency of the kernel to swap stuff out is wrong. You really don't want hundreds of megabytes of BloatyApp's untouched memory floating about in the machine. Get it out on the disk, use the memory for something useful.
Of course, there's another view, also presented at the above kerneltrap article: If you swap everything, you'll have a very long wait when returning to something you haven't touched in a while.
If you have limited resources, milk the resources you have plenty of; workstations should have high swappiness while laptops, who suffer in disk speed, disk capacity, and power, are probably better suited with lower swappiness. Don't go crazy, though ... swappiness = 0 is the same as running swapoff -a and will crash your programs when they need more memory than is available (as the kernel isn't written for a system without swap).
Use my userscript to add story images to Slashdot. There's no going back.
Concerning yourself too much with the numbers only can be a bad thing.
stop attacking my poor numbers! what did they ever do to you?
Guess what. I AM running Vista on such a setup. Your move, AC.
One word: Multics. Way too far ahead of its time. Those who forget history will have to try to re-invent it. Badly.
Windows splits the 32-bit process address space in half; 2GB for kernel-mode and 2GB for user-mode. There is a switch to change this to 1GB kernel-mode and 3GB user-mode. This is probably where the "reserves 1 GB for itself" myth comes from. Obviously we are talking about virtual memory here, so there is no hard physical memory requirement of 1GB. At some point you are going to run out of physical memory (512MB? 256MB? 128MB?), but I suspect 1GB would be sufficient for many uses.
Sister's laptop running Vista Home Basic came w/ 512 MB RAM installed, integrated graphics sucking 64 MB of it, so Vista had 448 MB to run the rest of the system. And before anybody asks, it was purchased new and it came this way.
Installed another 512 MB stick, so 1GB installed. Graphics auto bumped itself(can't find BIOS option to tell it otherwise, and seems to be before the OS comes up) up to 128 MB, so it runs with 896 MB of RAM.
Another "I've got less than 1 GB installed" quote.
And with this post I do thee unsubscribe from my RSS reader. Goodnight Slashdot, it was fun but your content has become moronic and useless.
Poindexter
http://java.sun.com/
I'm sure what he meant to say was 1GB of the virtual 4GB address space in 32-bit versions of the OS are reserved by the kernel (which is "Vista" I guess, as well as any kernel mode device drivers that are loaded) and cannot be used for user-mode allocations. This doesn't mean anything about your physical memory however.
In some situations this split may be 2GB / 2GB.
One very important reason to use virtual memory is to bend but not break. Suppose you have an application that suddenly starts using more memory than you expected. Before you realize it you're out of RAM, and some other completely unrelated application crashes because it was unfortunate enough to try to allocate memory.
This can be avoided if you're using swap, because the system slows down. This lets you choose to kill or suspend the applications of your choice rather than letting the OS arbitrarily choose by letting all the applications run out of memory.
Example: If you're using a 4GB RAM + 2GB page file and you're able to sustain 3.0Gbit/sec transfer speeds to your SATA drive (highly unlikely), then you'll have at least 8*2e9/3e9 = 5.3 seconds to respond to an out memory condition before you hit the 6GB wall and applications start crashing. In the real world you'll have a lot more time to react.
to the Collective. All units are directed to unify in the task of supplying a Vinculum, um Mac to the lost dro.. umm, Borg.
(Is there a difference in being ASSIMIlated into the Mac and being assIMMOlated in the ms pc, and EMUlated by the Linuces?)
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
of physical memory.
There are 10 kinds of people in the world > > Those who understand binary and those who don't
At my local BestBuy there are a half dozen 1 gig laptops running Vista, and they don't explode when you run a program on them. I wouldn't choose to buy one without the intention to upgrade to more memory, but they do work, and for very basic use you don't even feel the lack of memory.
"I zero-index my hamsters" - Willtor (147206)
The conjecture here is interesting, but this question can be answered better by just posting a link to some test results (comparing performance in Windows with and without virtual memory enabled). Does anybody have a useful link to such a comparison?
Wrong. I routinely run my laptop in this configuration. No issues.
Ha ha. This comment reminds me of the bumper sticker/t-shirt "No money, no kids, no job, no worries, RETIRED". How about "No money, no eye candy, no free time, no RAM, VISTA".
Seriously, I ran Vista for a couple months out of morbid curiosity. I turned off eye candy and /all/ the other "performance vs appearance" tweaks and left the page file intact, and just booting to the desktop tied up 800+ MB of RAM. How you might get away with that configuration and no page file is a mystery to me. Why you would even try is another question altogether.
Yeah, I've seen XP run on 64 megs of RAM too. And run. And run. And run......
There's a reason MS is in court over the "Vista Capable" program. Vista on 1 gig? Maybe. Vista on 1 gig with no page file? kill me. kill me. kill me...
Well, let's very early start the Salacious Santa jingles...
"Memory exists to be used. If memory is not in use, you are wasting it. The reality is that your system will operate with
higher performance if unused data is paged out of RAM to disk and the newly freed memory is used for additional disk caching. Vista's memory manager is actually reasonably smart and will only page data out to disk when it really won't be used, or you experience an actual low-memory condition."
Makes me think:
Bodies exist to be used. If orifices are not in use, you are wasting them. The sensuality is that your sys stem will operate with higher performance if unused attach points are paged out for RAMMing to di*k and the newly recalled memories used for additional di*k THRASHING.....
Oh, wait, i'm running out of anal-orgies for comparison...
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
I have 3GB of RAM and my swap has never been touched, in Linux at least.
Not only does it run with 1GB, it runs well, at least it did until I installed my bloatware anti-virus.
Dang. Disappearing html tags killed my Simpsons reference. [hic] kill me. [hic] kill me. [hic] kill me.
Virtual memory refers to the entire hunk of addressable space, which may reside in a number of places (often concurrently) - primarily main ram, on-chip cache, or swap.
All modern end-user systems use virtual memory/pagetable to abstract program addresses from system ram. So a better question is "Why use swap on modern systems?"
And the answer is easy: because you have to (if you have insufficient ram) or because you just haven't turned it off yet.
There is one exception; a well designed VM can improve performance by paging out inactive data, even if you have "enough" ram. Paging out inactive data frees up more physical memory, allowing it to be used as disk cache. It doesn't make sense to keep (for example) system boot code or infrequently used daemons in memory when that ram is better used as disk cache while processing archives, browser cache, etc.
A government is a body of people notably ungoverned - AC
That page mostly talks about what virtual memory is and doesn't directly list why it is an improvement.
Some folks have already mentioned the fact that it eliminates memory fragmentation, and that it allows mapping of files and hardware into memory without dedicating (wasting) part of the address space to those uses.
Another reason is that you can have 2^64 bytes of total system memory, even if the individual applications are 32-bit, and can only address 2^32 bytes of memory. Since the 32-bit applications are presented a virtual address space, it doesn't matter if their pages are located above the 32-bit boundary.
It means that per-process memory protection is enforced by the CPU paging table. Without virtual memory you would have to reimplement something like it just for memory protection.
It means that the linker/loader don't have to patch the executable with modified address locations when it is loaded into memory.
The above two reasons have the corollary that libraries can be shared in memory much more easily.
And that's just off the top of my head. Virtual memory is a very, very useful thing.
You must be confused about virtual vs. physical memory. In modern processors, there is no penalty for using virtual memory, all translation from virtual to physical address space is done internal to the processor and you won't notice the difference.
Huh? That's totally wrong. If it were true, you wouldn't need any RAM.
It's true that address translation is hard-wired in modern processors. But that just means that figuring out where the data is is as fast as for data that's already in RAM. Actually reading or writing it is only as fast as the media it's stored on. So if you have a lot of big applications running, and there isn't enough RAM for them all to be in physical memory at once, your system "thrashes", as data migrates back and forth between the two media. That's why adding RAM is very often the best way to speed up a slow system, especially if you're running Microsoft's latest bloatware. Defragging the swap disk can also be helpful.
To answer the original question: actually, you often don't need any virtual memory. But sometimes you do. Disk space is cheap, so it makes sense to allocate a decent amount of virtual memory and just not worry about whether it's absolute necessary.
Windows makes me CRAZY about this. the OS is internally configured to use an LRU algorithm to aggressively page.
("Technical bastards" who question my use of paging and swap interchangeably in this post can send their flames to /dev/null \Device\Null or NUL depending on OS)
What I found when disabling paging on an XP pro system with 2GB RAM is that the system performance is explosively faster without the disk IO.
Even an *idle* XP pro system swaps - explaining the time it takes for the system to be responsive to your request to maximize a window you have not used in a while.
I was thrilled to have a rocket-fast system again - until I tried to hibernate my laptop. Note that the hibernation file is unrelated to the swap/paging space.
The machine consistently would blue screen when trying to hibernate if swap/paging was disabled. Enabling swap enabled the hibernation function again. Since reboots take *FOREVER* to reload all the crap that XP needs on an enterprise-connected system - systems mangement, anti-virus agent, software distribution tool, and the required ram-defragger which allows XP to "stand by" when you've got more than 1GB of RAM, plus IM, etc
I reboot as infrequently as possible and consider "stand by" and "hibernate" required functions. As a result, I live with XP and paging enabled, and tolerate the blasted system "unpaging" apps that have been idle a short time.
Poo!
But Herr Heisenberg, how does the electron know when I'm looking?
Circa 1975 through 2000 or so, the "native" (*) versions of the Pick Operating System worked exactly this way. Even the OS-level programmers - working in assembly language (!) - only saw virtual memory in the form of disk pages. When you put an address in a register, it was a reference to a disk page, not physical memory. The pages were auto-magically brought into memory at the moment needed and swapped out when not by a tiny VM-aware paging kernel. That was the only part of the system that understood that there was anything besides disk storage in the entire system.
Might sound inefficient, but I remember the first Pick system I worked on supported 14-16 simultaneous online users using only 64kb of physical memory (and it was core - REAL core - at that point).
Now get off my lawn, kids.
-TSG-
(*) "Native" meaning "with no other OS involved underneath, bare Pick on the metal". These types (Reality, Ultimate, CDI Series/1, etc) were mostly gone by Y2K in favor of more modern systems with underlying host OS (Un*x or NT) handling the hardware and Pick riding on top as a database/programming environment (Universe, Unidata, AP / D3). Though I'm sure there are still hundreds if not thousands of the old systems chugging away in back rooms of warehouses and the like to this day.
If it's there, and you can see it . . . it's real.
If you can see it, but it's not there . . . it's virtual.
If you can't see it, and it's not there . . . it's gone.
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
I have 8 GB of RAM and it created a 16 GB pagefile. Yeah, that's real smart. If I ever actually did page-out 16 GB of crap the computer would trash for hours and nothing would get done. Hard drives are too slow to swap out more than 512 MB or so. Anything more than that will kill your computer if you actually used it. Consider with 4+ GB of RAM than a 512 MB page/swapfile ain't shit, then you just run with no swap file.
The way I see it, if you have a fault in your memory, wouldn't the constant swapping in/out of a page file be MORE dangerous since now it affects everything that goes into that physical address? I'd assume that without a page file, memory stays much more constant than it would with a page file--after all, there's nothing to be gained by swapping data between addresses in physical memory... (In fact, it would just waste CPU cycles). So, if you have a fault, wouldn't you want to turn off the page file so that fault has the opportunity to do the least amount of damage to data?
Windows 3.1x calc: 3.11 - 3.10 = 0.00
disable the paging executive, it then only uses the swap when the ram gets full
google it, its a simple registry hack.
Virtual memory addressing enables such beautiful tricks as Shared Memory (inter-process communication, saving physical memory on identical copies of the same DLLs/SOs), Memory-Mapped IO, etc.
Yes the OP is right if you don't page to disk and go off all RAM then you will be faster. However with a good paging it will help you from things from getting slower or not working when you really need the extra Horse Power, and you probably wouldn't even notice it.
First we got the 80/20 rule where 20% of the Data is used 80% of the time. So a large chunk of data will rarely be used, being that it not being used read or written just kinda sitting there. You might as well page it to disk so you have more space free.
Next if you get a big chunk of memory request say you open a VM system that need Gigabytes of memory. Say 3 GB and you only had 2 GB free. That means before you app can run you will have to wait for 1GB of data to be dumped to the disk. Vs. say a good paging algorithm which would already have that 1 GB already paged so you can fill the RAM with the VM for a faster access then probably depending on the paging algorithm pieces will slowly get paged back to disk allowing you run say an other 512meg load on your system without having the system dump that 512meg of data. If you didn't page you would be stuck as you don't have the ram to run the application. Or a poor paging algorithm will spend so much time paging the data until it gets enough free to operate.
Drive space is relatively cheap if you are going to do some high RAM intensive apps. With a good paging you can get by with about half as much RAM saving money.
Most systems have more ram then ever but the apps use more ram then ever too. (This isn't necessarily bloat) Lets say your app does a lot of Square roots. The time it takes for it to process say 1,000,000 Square Roots vs. Dumping to memory the recalculated Square Roots values and doing a quick memory lookup of the answer. That way you get faster calculation time at cost of RAM.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
See Mark Russinovich's (author of SysInternals tools) blog that describes how windows uses memory resources,
Pushing the Limits of Windows - Virtual Memory:
http://blogs.technet.com/markrussinovich/archive/2008/11/17/3155406.aspx
Pushing the Limits of Windows - Physical Memory:
http://blogs.technet.com/markrussinovich/archive/2008/07/21/3092070.aspx
One problem is that there are relatively frequent types of disk-access patterns where caching them gives little to no benefit in return for the paging out of RAM it requires. A virus scan (touching most of your files exactly once) is one canonical example. Media playback (touching a few large files in sequential block reads) is another.
The difficult question is how to exclude these kinds of wasted caching while still retaining the benefits of caching frequently accessed files, and not introducing excessive overhead in the process.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
It's something that's annoyed me for ages in Windows too. If I have over a gig of physical memory available, for the love of God don't start swapping out applications that I'm actually using.
I'll accept that I'm a power user, and that not everybody switches regularly between a dozen applications, but there's still no excuse for swapping stuff out when there is bucket loads of memory free. I mean, 1GB is around four times the original requirements for running Windows XP, that's not a small chunk of memory to have lying around.
And actually, this gripe was the main factor in migrating my computers over to Ubuntu a few months back. I got so fed up of waiting for windows that I decided to give it a try. It's definately a bit rough around the edges, but I haven't had to wait for a single application to get paged back from disk, it's far, far more responsive and there's no way I'm going back to windows now.
So far the longest I've gone between reboots is about 3 weeks, and at the end of that I still had under 50MB paged to disk. That's a massive difference to windows where I can have 1.5GB paged out by the end of the week, and I'm running more apps under Ubuntu too!
. . . apparently not known. That an OS that has been around for this long can still be a mystery to the herd that is /. confirms for me that computer science is akin to philosophy for many of its users. So there.
It's all history, man. -anon
There has been this ridiculous notion floating around recently that swap space and paging files are relics and need to be eliminated. You can only safely eliminate them only so long as you're 100% confident you'll never use more RAM than you actually have. But there are lots and lots of memory hogging applications - video editors, image editors, scientific applications, etc. And when you consider that a web browser can eat up to 300MB of RAM, it shouldn't be hard to imagine a multitasking user running out by using too many little programs.
this is true of many *nix style systems include macOS X.
its actually funny to watch but if you have some processes running on a server that don't access the disk you can rip the drive right out of the server and it will be fine, the kernel and programs are in ram so who cares? you could even open recently used apps without the drive in there since they will be cached in ram still as well (probably)...
trying this in windows results in BOOM
cheers
He worked waaay to hard on that statement
What's this place?
Virtual memory is for the lazy application developer. It's his linear address space which he can use for data or even blend in files, swap or other device access (e.g. graphics)
You cannot "consume" virtual memory, as every app has a fixed amount assigned anyways. Using this address space to access physical ram is just one option.
Your system will have 4GB, but a small chunk of that memory is going to be use d to store the kernel, all relevant drivers, running services for graphics, virus scanning, printing, mail, and networking. All of these are going to reserve large blocks of memory either by the drivers themselves or when the services are in use. The kernel might even make it a priority to pre-allocate memory to device drivers rather than leave it free for applications to use.
I remember upgrading my PC's memory, thinking it was going to give me a lot more memory for applications to run. To my surprise, the application took even longer to run a large data file, as the kernel was swapping out all the pre-allocated memory.
Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
...cause I sure would like one.
This question evolved from what I thought was a pretty straightfoward one ("why use a paging file") to a convoluted discussion of virtual memory, addressing schemes, coding, blahblahblah.
Wasn't the OP asking why modern Windows PCs, with gobs of RAM, even need a page file anymore except in exceptional circumstances?
And that's a good question.
I had an NT server back when with a whopping at the time 512MB RAM. I turned off the paging file since it was strictly a file/print server. Norton Corporate Edition refused to load. I had to set a paging file, and I did - 10MB worth, IFIRC. Happy now, Norton loaded and the server was happy. I've run some XP machines without paging when I had time to fool with them.
I remember when NetWare started supporting paging files. The engineers I knew essentially told me that it was a holy war within Novell, with the paging gang winning for no technical reason. Sort of the holy war over the licensing model that got a lot of people riled up and changed nothing. Paging in NetWare, back in v5 days, changed nothing. Now NetWare on Linux, paging seems to be crucial, even when you have 8GB machines.
But back to my question, I don't see the answer yet. I couldn't bear to read through the minutiae of virtual/physical one more time.
Reminds me of LIM/EMS memory. Ack! It hurts!
deleting the extra space after periods so i can stay relevant, yeah.
I note a lot of people are insisting that "virtual memory" refers to the virtual address space given to a execution context, and what the author really means is "paging".
The funny thing is that these are traditionally poorly defined/understood terms which are gaining a hard consensus for the meanings due to some recent OS books, and poor comp-sci education which insists on a particular definition. Everyone is faulting M$ for using the term incorrectly, even though the original mac OS and other OS's used the term in the same way. Wikipedia defines it one way and then goes on to give historical systems which don't really adhere to the definition. For example the B5000 (considered the first commercial machine with virtual memory) didn't even have "contiguous" working memory as required by the wikipedia definition. It had what would be more specifically called multiple variable sized segments which could be individually swapped. Again, the mac OS evolved from a single process model to muliprocess, in the same address space (look up mac switcher) and implemented "virtual memory" using a system without a MMU by swaping the allocated pieces of memory to disk if they weren't currently locked (in use) and reallocating the memory. Aka they had "virtual memory" in single fragmented address space.
The other example is people use "paging" to describe the act of swaping portions of the memory to disk, misunderstanding that paging is more about splitting an address space or segment up into fixed pieces for address translation to physical, and that disk swapping of pages isn't required for paging. Aka, your system is still "paging" if you disable swapping.
Even the term swapping is unclear because the need to differentiate between swaping pages, and swapping whole processes (or even segments) resulted in people avoided the term swapping to describe systems which were swapping pages instead of segments/regions/processes. These systems were generally called "demand paged" or something similar to indicate that they didn't need to swap a complete process or dataset (see DOSSHELL).
So, give the guy a break, in may ways he is just as correct, if not more so.
"In modern processors, there is no penalty for using virtual memory,"
Umm, would you like to bet? If it's not stored in physical RAM, there's only two other place to store, processor cache or a slower storage medium. I guarantee you a fast processor WILL CRAWL if you tried accessing the 'virtual memory' pagefile written on disks that have low bandwidth, like CF cards and older pre-NT days hard drives.
By historical use and the way Windows uses the term today, the Submitter is correct. It's been called virtual memory since the late '70s.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
Win32 apps can access more than 2/3GB of physical memory.. if they're written properly. http://msdn.microsoft.com/en-us/library/aa366527.aspx
I have been asking myself the same question lately. Try disabling the swapping altogether or set the page file size to zero and see what happens. :-)
The mechanism must be there to ensure applications get the required memory. Any amount of memory will always be too tight for somebody (ask BGates).
Still, I can see where you are coming from and I agree, there should be a way to get rid of the functionality if all you crave is speed of execution and system performance.
Many programs allocate much more memory than they use on average (AFAIC, Java does this frequently). If you have enough such processes running, the amount of headroom used by each program can add up to a significant amount of total memory. Since this happens so often, some operating systems implement overcommit, meaning they pretend they have more physical memory than they actually have.
But what if, by some coincidence, many of those processes decided to use much more than their average amounts at the same time? Without enough physical memory, the operating system would have no other choice but to kill a process to free up some resources. This could be a real pain for the user.
So the choices you are left with, are either risking that random processes die, or that you cannot use most of your memory on average. A neat solution for this is to have a lot of swap while disabling overcommit, so that you can use most of your RAM on overage, and only use the disk as memory when some coincidence happens.
Vista has a feature they call SuperFetch, which is supposed to optimize memory usage and prevent page outs that would make the system unresponsive when running background tasks (exactly the situation many complained about above - delays in application switching/maximizing after a period of non-use). It's also supposed to preload data based on historical usage of application. I haven't seen any (subjective) improvements in responsiveness over Windows XP however. Might be that when Microsoft gets something right, they make sure to get other things wrong.
First of all, Windows uses virtual memory to map things like executable pages and DLLs into memory, so they only get loaded if a particular function gets executed... parts of an EXE or DLL that aren't used are always considered "paged to disk", even though they don't take up any space in the page file and were never loaded into RAM. If you access these pages, Windows will dutifully read them from the original file into memory.
This is related to the prefetch system that attempts to say "ah, i always end up loading these other pages over the next 20 seconds of program start anyway, might as well just let the HDD keep churning while the user is moving the mouse to the open button that normally triggers the page faults and save some time". If you didn't do it this way you'd have to stare at the hourglass waiting for the entire EXE to load into memory (and other stuff to get paged out if necessary, or even dumping some read cache which may in turn make performance even worse). If you don't prefetch, then every new function invocation can potentially cause a page fault. So sure, after you see Word up and running the HDD is churning for 20 seconds. That's fine with me - Windows is just pre-caching the parts of winword.exe it knows I usually end up needing anyway.
Windows will also immediately dump bits of read cache as well as memory that is already stored on disk (such as DLLs) if there is a sudden demand for more RAM, thus allowing that memory allocation to succeed immediately instead of waiting five minutes for everything to get paged to the swap file first. Windows will also scan memory looking for bits that aren't already on disk and write them there during idle times, just so it has the option of dumping that RAM if needed. You aren't using the hard disk anyway, why not?
The caching system also plays into this, and Windows may determine that the system will be faster overall if these bits of data are cached while those bits of data are written to the page file. (See the server vs workstation defaults in My Computer properties about prioritize applications or cache).
Are these algorithms perfect? Certainly not, but they do help, and none of it is arbitrary.
Natural != (nontoxic || beneficial)
Physical memory or RAM access times measured in nanoseconds and hard drive which virtual memory or paging resides access time is measured in milliseconds so physical memory is 1000 times faster than virtual memory.
Someone can correct me if I'm wrong in Vista and most Microsoft Windows OS virtual memory is used after all physical memory is used up. Even you had the maximum physical RAM installed on your system you will need virtual memory so if for some reason physical memory runs out. Virtual memory as one advantage over physical memory, depending of the operating system, virtual memory is persistent so a power failure will most likely erase the contents of physical memory. Again, depending on the OS, this virtual memory can be recovered.
If you are running a bog standard desktop or laptop you probably have some basic requirement
* when I turn up at my computer after having a cup of coffee and hit slashdot.org into firefox I expect it to launch straight away
* when I shutdown my computer I expect it to happen instantly
* when I start up my computer I expect it to happen instantly
* when I save my music/words/picture I expect it to be reliable and nothing my computer will do after I save my file will affect it
* under no circumstance will my mouse, keyboard or icons on desktop not perform to my actions - I want no hang.
This is not the fault of disk swapping it is just an example about how 5 utterly straightforward requirements have not been addressed in over 20 years of desktop development.
First OS which has this gets my vote
Here's an analogy I find quite fitting, but first of all let's clear it up: the OP is asking about "swap space", which Windows calls "Virtual Memory".
Think of your computer as a bank, the Ram as money, applications as clients. Swap space is credit.
You might lend out 5 times more credit than you have actual money, and as long as your clients don't all claim it at once, you're fine. This way, each client can "have" a ton of money at their disposal, if and when they need it.
Well swap space is a way to "lend" memory to a process that may or may not use it all, or may not need it at that point in time. Memory that's sitting idle gets pushed into swap, to free up real memory for another program that needs it more.
If you disable the swap file(s) on your PC, you're basically forcing your computer to live on "cash". If one program needs 80% of your Ram, and another program needs 25%, one of them will get canned because you can't "borrow" any more memory. The potential problem with this scenario is that many programs allocate more memory than they actually need. Some developers do this for efficiency, since it's faster and simpler to reserve a big contiguous block of memory at once, rather than many small ones as needed. The rest do it because they're DeVry graduates.
-Billco, Fnarg.com
To increase performance, windows is paging to a ram disk.
The problem is computer scientists think they can make algorithms that are smarter than I am.
But I would WAY rather be told "You're out of memory, what would you like to do?" than 'Your computer will now freeze for 30 seconds while I shuffle things around'. Yes, disk cache is useful, but having a little bit is often plenty and then its diminishing returns after that.
I always keep a visual memory meter on and more often than not I load something and realize 'oh shit, its starting to swap' and then rush to quit a few programs to minimize the damage.
The key is *I* know which programs I'm going to use next, the OS simply 'guesses' based on usage patterns.
There's a problem you're not seeing: the logic for swapping out cold memory pages assumes that the goal is to maximize application throughput, at the potential cost of latency when doing large context or workload switches. The approach makes the applications that are currently running optimally fast, at the cost of making it costly for an idle application to become active again.
This is excellent for some kinds of workloads, but bad for user interaction, where latency of task switching is more important than maximum throughput. When I click on a minimized application window, I want that application to be ready to run instantly. That's more important to most users than having an occasional computationally intensive task take 3 minutes instead of 5--I can find something to do during those 5 minutes.
Are you adequate?
You can, but usage shows that even with 2gb of ram and no pagefile vista becomes very unstable.
Dumb question, but this is a Microsoft product we are talking about, and not one of the better ones... how do you know it hasn't gone behind your back and created a pagefile somewhere?
I am government man, come from the government. The government has sent me. -- G.I.R.
Concerning yourself too much with the numbers only can be a bad thing.
stop attacking my poor numbers! what did they ever do to you?
he's not attacking them you tinfoil hatted loon! he's NEGLECTING THEM! They would kill their first born for attention, even if it means being attacked!
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
Way back when memory was $1 a byte, and all programs ran in batch mode overnight, it seemed a super idea to make disk look like memory. The idea being that there was always more time, but lots of people could not afford another 64k bytes of real memory. It also let IBM sell a whole new set of very expensive CPU's and memory mapping boxes.
Now today when memory is about a million times cheaper and we expect millisecond response times, those thirty years of computer science and marketing are just an ancient millstone around our necks. Unfortunately everyone just expects to have unlimited, although stuttering, memory. Soo many programs read in a whole file into memory, then build a tree of it all, in memory, and they wonder why the disk light is on all the time.
Folks, it's not 1968 anymore. Go out and buy a 64-bit address space mother board for $55, splurge and buy as much REAL memory as you can afford, turn off the virtual memory backing file, and enjoy reliable memory that's all 3 nanoseconds away.
welcome to computer science. There are two types of memory, primary memory and secondary memory.
Primary memory are RAM chips, Random Access Memory, the computer uses them as a temporary storage to run the operating system and programs within it.
The secondary memory are such devices as hard drives, USB drives, SD flash memory cards, etc. When something is saved to secondary memory it is permanent unless it is deleted later.
When an operating system runs out of primary memory it has a backup called virtual memory by caching out data from primary memory to secondary memory, mostly things not recently used.
Even though you have 4 gigs of RAM in your system, it might not be enough if the operating system has over 4 gigs of data it is processing. Eventually you'd need virtual memory. Even Bill Gates was wrong in the 1980's when he said 640K ought to be enough for anybody, because the people who did graphic photo work needed megabytes of memory just to hold a photograph in memory at the time.
If there was no virtual memory on your Vista computer you'd get "out of memory" errors that used to be quite common on 1970's and 1980's 8 bit computers that had less than 64K of RAM. My old Timex/Sinclair 1000 had 4K of RAM and that 16K add-on pack didn't always connect right so my BASIC programs always got the "out of memory" errors. Back there there was no such thing as hard drives and the T/S 1000 only used a cassette tape and not a hard drive for secondary storage. For some reason the quality of the Sinclair computers in the USA were not as good as in the UK as they teamed up with Timex and had poor quality and Atari and Commodore beat them in the USA while in the UK the Sinclair series got Sir Sinclair Knighted.
But your question is quite funny in a way, as it shows your ignorance of how a computer works, and shows how low in quality Slashdot's article publish system has become. Computer Science 101 can easily answer your question. But then there are no such thing as stupid questions, only stupid people, whom the US Philosopher Ron White says cannot be fixed, but I disagree that they can be educated to be smarter.
Remember, Slashdot does not have a -1 disagree moderation, and no, troll, flamebait, and overrated are not substitutes.
Perhaps set up a small ramdisk and pf to that?
"If you have nothing to hide, you have nothing to fear." - Every fascist, ever
This UID and mod combination is in direct violation of the most fundamental scientific law of slashdot: UID and mod points granted are always inversely proportional.
It does, however, exist.
The breakdown of this fundamental law was referenced by nostradamus as the first sign of the end times, and as such I think it's time to ready my space helmet and silver broadsword.
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
One of the most onerous problems facing users of Windows are the 2GB per process and 4 GB total limits.(often only 3.5GB or so actually usable) This seems to be a corporate decision by Microsoft rater than an actual limitation of the hardware, as all of the modern processors will allow 38 bit addressing.
http://en.wikipedia.org/wiki/Physical_Address_Extension
32 vs 38 bit might not seem like a lot, but it will allow up to 64GB of ram on the OS. Note how the Windows 32 bit builds are crippled on purpose. Linux, BSD, Mac, and so on all allow it with a simple tweak($0 and a couple of minutes - bad Microsoft!) Also, most motherboards can only fit 8GB anyways. Some can fit 16, but that's it. You need special hardware to get anything approaching really large amounts of memory.
1: Get an Acard ANS9010. MWave sells it for about $250. This is a ram based drive. Not a SSD but a real DDR2 ram disk and perfect for sticking a swap file on as well as Photoshop and other swap files. It even has a battery and a CF card slot to do on the fly backups of the device when you shut down. Better than the I-Ram as once it's off, it's off - the battery only exists to power it long enough to dump the contents to the flash drive.
This solves the speed problems and the lack of extra memory slots on most motherboards at the same time.
2: Another dirty $0 trick is to put the swap file on a second drive on an otherwise unused controller. I use my single IDE port for this - it's my temp storage drive and so it sees no real use. But the difference is massive. I can quit a major gaming title and be back to the desktop in under a second. My drive with the applications on it and the paging file never fight for bandwidth or time. Yes, the second drive thrashes for 1-2 minutes while it's cleaning up all of the garbage files, but I don't slow down.
Note - Both of these together? It's astounding how fast Windows (or even Linux) runs like this.
if you can do without a paging/swapping file/parition, then just do it.
If at some point your system then crawls to a halt or is refusing to start processes (out of memory) then you know why that paging area is needed.
4GB is enough? Could still be short in some circumstances.
Atari rules... ermm... ruled.
I think the issue really has more to do with address spaces than reserving memory, although the end result is similar. Windows (and probably most other operating systems) typically maps the lower half of the address space for the current process, while the upper half in all processes is mapped to the kernel. If you think of memory addresses as signed values, positive addresses are for user mode and negative addresses are for the kernel. If you can't address the memory, obviously you can't use it, which is where the 2GB limit comes in.
The advantage of doing things this way is that the kernel is present in every process. This means when someone moves the mouse and triggers an interrupt, you don't have to switch to the mouse handler process to take care of the interrupt, you can do it from the context of whatever process happens to be running. It also lets you make system calls without having to fiddle around with the address space too much also.
The way windows handles virtual memory is the reason why I have ran with no paging file on my home Windows XP machine (1GB RAM) for the past 5 years. Prior to this, when using multiple applications that made use of most of the available physical memory there would be considerable delays whilst the previously inactive application was paged back in despite not exceeding the available physical memory. I assume that this was paged out for disk cache in order to attempt to improve overall performance. However, consistency in response time is much more important to me than possibly having cached something. In addition, if allocated memory ever exceeds the total physical memory even by a small amount then performance became so appalling that I would prefer the application that was leaking memory to be killed than to wait for 5 minutes whilst the system swaps.
Isn't it only the server versions of XP that you can tweak userspace to 3 GB?
So put it on the desktop, and make the desktop capable of showing you the contents of your files. When your desktop gets cluttered, then you set aside 15 minutes to file away or discard the items in it.
You might answer that many users' desktops would then get insanely cluttered; my reply to that is that many people's desks are insanely cluttered, so that's ok. The UI doesn't have to solve the user's character flaws.
Are you adequate?
Nice! I have an HP a1140n MCE that I bumped up to 4GB with Crucial's help. System only shows 3.11GB and I couldn't get to the bottom of it.
Thanks for that info d3vi1.
Let's be careful with the notion of virtual address space and page/swap file. Virtual address space is a hardware feature that allows any physical memory to be mapped to any address seen by the program in the granularity of pages (typically 4kb), and that part is almost free thanks to TLB. Virtual address space also allows an address to be backed by no physical memory, in which case the operating system will be notified if a program tries to access that address. If there was formerly a memory page swapped out to disk, the OS brings it in (possibly swapping another page out) and continues the program. If not, the OS aborts the program. Virtual address space and page/swap file are altogether called virtual memory management.
I could be mistaken, but I thought Windows requires each physical memory page to be backed by something on disk, either page file or memory mapped files (e.g. all executables and dlls are like that, which is why you can't remove executable files on disk when the program is still running). I think that's why the ask slashdot poster sees as much usage of "virtual memory" (here used to mean page file) as "physical memory."
I once had a signature.
Mark Russinovich, (of sysinternals fame), did a series of videos with David Solomon, and this is one of the topics they covered. So this information comes from them and that video. I suggest everyone check it out some time, those guys probably know more about Windows than most people.
1) No paging file == no crash dumps. Windows writes crash dumps to the paging file on the boot volume. So if you want a full memory dump, you'll need to match your paging file size to your physical memory size.
2) Modified private virtual memory is the only memory that gets written to the paging file. Not code. Code can be read from the same place it got it from in the first place. And only memory that isn't referenced for some amount of time gets paged. So my understanding of that is even though you disable your paging file, CODE can still be "paged out".
3) So, it's slightly more efficient to have a SMALL paging file than none at all. Some memory allocated by software may be used once then never touched again. The paging file gives Windows a place to put it, so the paging file essentially becomes write-only. Then those pages are freed up for the memory manager to use.
4) If you're going to use a paging file, the correct size to use is the sum of the private bytes of user-space programs plus the kernel private bytes. This is known as the "commit charge" which you can see in the lower left hand size of the built-in task manager. That is the "potential page file usage", not the amount of actual paged-out memory.
Personally, I've been running without a paging file for years without issues. Unless I need to do a crash dump analysis.
This split is for the process address space, it has nothing to do with the actual amount of RAM in the machine. It exists so a process can go from user to kernel space and back without having to update the page tables, resulting in a faster transition.
The only relation this split has with physical RAM is that if a process has 2 GB of address space and the computer has more free RAM than that the process cannot use it all without tricks like AWE/PAE.
which is written as if it were just like any other compute task, to be scheduled as the OS sees fit (maybe with bumped up priority, but that's not enough to make sure the right thing happens). It instead has to be scheduled as a realtime task with guaranteed bounds on the response time for low-overhead user operations, which means locking stuff in RAM even at the cost of more swapping on less interactive tasks. That also means turning on realtime kernel support in systems that don't already have it active. I've thought for a while that the Linux window system should be rewritten by game programmers, who tend to have some clue about how to identify the parts of an interactive program have to be responsive, and to make those parts actually be responsive.
I think he might actually be referring to Vista's... interesting tendency to allocate ~33-50% of your physical memory upon boot.
I have 6GB installed in my machine, and it's currently using 2.25GB to run Firefox with a handful of tabs, VLC, Windows Explorer, Winamp, and the Sidebar. God only knows how it uses that much to do so little, but I'm not fussed - it's not as if RAM is expensive these days.
An interesting note though - I worked out a while ago that these days, you pay more per chip in a packet of potato chips than you do per megabyte of RAM. Go figure.
I find this blog the most informative of all thing windows internals http://blogs.technet.com/markrussinovich/ He explains it better that anybody
In a low memory situation without paging, can't the OS reserve enough RAM to produce a pop up asking if you would like to turn on paging temporarily at least? Or even do it automatically if there is sufficient disk space?
I run one of my Linux systems with no swap space. What I did was after calculating the most RAM I expected to need, I doubled that amount. That machine has been the most stable of all of them. I should do this more often.
now we need to go OSS in diesel cars
Yes, lots of the memory mapping is done by the CPU. But it still needs to be done. One important issue is if the manipulation of addresses and page tables has a cost in cycle time. Sure, this is much less than the cost of swapping pages in and out of main memory from and to a hard disk, but it's still a cost. If every single instruction (or most of them) cost extra cycles because virtual memory is being used without any valid reason to do so, then that cost adds up fast.
I don't know of the exact timing issues in this case. It likely even depends on the processor make being used (AMD vs. Intel) and even the processor family. There may also be extra costs involved in how cache memory is dealt with. But I certainly expect that there is an overhead cost, and no real justification to doing things this way except that some programmers don't appreciate the issues involved at a low level.
I'm an American. I love this country and the freedoms that we used to have.
There usedta be a number of windows tweaking guides online that recommended setting the registry entry DisablePagingExecutive, which
"Specifies whether kernel-mode drivers and kernel-mode system code can be paged to disk when not in use"
That at least could reduce some of the paging out, but I haven't researched which O/S versions it applied to or if it is still relevant.
Other recommendations I have seen recently suggest when you have a large amount of RAM, just create a small paging file of around 512mb, to stop certain kernel processes from spitting the dummy.
If I had a DeLorean... I would probably only drive it from time to time.
With XP, I had a similar problem related to a Maxmem setting in the boot.ini. I had just added another stick of mem, but sysinfo didn't reflect it and there was major swapping. I got rid of the maxmem option, which was set by an old Matrox video driver, and all was right in the world. MS on maxmem: http://msdn.microsoft.com/en-us/library/ms791488.aspx There might be something similar in Vista.
I've turned off a few unecessary services in Vista (DWM, Themes, Server, Security Center, etc) and got the "Memory" section of the task manager to report 271MB at the lowest, right after a reboot. This would climb to around 300MB after a full day of usage (after exiting all running programs, of course).
So as to running Vista on less than 1GB it's very possible, as is running it on 512MB. Sure, to make it at all usable you have to turn off some un-needed features which requires a bit of skill, but it's still possible.
And the boot time went from three minutes to about thirty seconds.
It used to have about 300Mb of swap file after bootup, which is dog slow on the Eee's SSD. This is despite task manager showing 700Mb of unused RAM.
Whatever Windows is doing, it's doing it wrong.
Note to EeePC owners: Get a 2Gb RAM module and disable the swap file. You'll be glad you did.
No sig today...
Vista will use excess RAM to preload some bits that you might use frequently. This is particularly obvious if you upgrade your RAM from high level to higher level in Vista. My at-boot memory usage was ~30% with 4gigs installed; when I installed 8gigs, the at-boot memory usage was unchanged.
I had more RAM, so it filled more RAM. Straightforward. I've not had any issues with its memory management.
Blog
The amount of physical memory you have has nothing to do with the 2gb|2gb split or 3gb|1gb split. You can use your full 3gb either of physical memory either way, since the virtual mappings (which is what is split) is different for each app, so two apps can each use 1.5gb of user space and take up 3gb total memory.
The actual reason for the /3GB flag is to enable OS support for special applications which explicitly opt-in for up to 3gb of memory, usually to help boost performance (they can have a larger cache or buffer or whatever)... normal apps will still only be able to use up to 2gb because for compatibility. Apparently some apps use the highest bit of a 32-bit pointer to store a boolean, since with 2gb of user space that highest bit is always 0. Adding an extra 1gb allows that bit to be 1 which could seriously confuse the program, likely causing a crash.
SQL Server includes support for 3GB, IIRC.
I'm not sure if there's a downside to using 3gb (there must have been for it to be optional and hidden like that) especially since apparently Vista turns it on by default... however IIRC while each app gets it's own 2gb/3gb of userspace, kernel address space is shared between all apps, so your kernel only gets 1gb total memory. That includes all running drivers. I imagine the XP team thought it was better to allow for the total 2gb for drivers, especially since most apps couldn't take advantage of the full 3gb anyway (as I described above).
Sorry, I have hibernation on my laptop and desktop and use it several times a day. I've also turned off "virtual memory" a long time ago.
This runs most esceptionally on X SP3. Turn off your theme while you are at it. That is a resource hog too.
Good luck
Bill
bamph
Well you could always switch to an operating system that doesn't use virtual memory. Check out http://www.minix3.org./ Sure they might add virtual memory eventually but in the meantime you'd have a system without it.
You'd also have to use a processor chip that doesn't use virtual memory look ups. The extra hardware used to look up virtual addresses and map them to either physical or virtual (page file) pages requires a lot of extra clock cycles on each instruction the processor executes that involves loading or storing to or from memory addresses. We're talking a lot of extra cycles even for pages that are in physical memory.
Many micro-controllers don't use virtual memory. They also have the benefit of being dirt cheap compared to Intel's and AMD's behemoths. They also consume much less power. Check out this computer: http://www.makezine/controller.
I remember the frustration of using a Linux desktop in the morning. All night various things had been running and as they were loaded it swapped out the desktop programs. But then it stayed like that, because nobody was using the desktop programs.
When you tried to use the desktop in the morning, it was *really* slow, because it now had to swap out all that stuff it had loaded, and then swap in the desktop programs. After awhile it would return to normal speed.
I believe Windows tried to address this. They tried to get the "nightly" programs to at least write themselves out to the swap file, so that getting the desktop programs back required only 1/2 as much work (only reading them in, and not having to write out the nightly programs). Probably more significant, they keep track of the "active" process and continuously swap in that process even if it is doing nothing.
This is all from 8 or so years ago. Nowadays Linux seems to work fine. I don't know if it is because the swapping was fixed, or because a modern machine has so much memory that the desktop programs don't get swapped out. It is also possible that some of the crap that ran at night has been eliminated. Meanwhile it does seem that Windows attempts to solve this have backfired because they cause far more swap writing than is necessary.
oops... http://www.makezine.com/controller.
Here are two great posts by the famous Mark Russinovich, they may give some insight:
- http://blogs.technet.com/markrussinovich/archive/2008/11/17/3155406.aspx
- http://blogs.technet.com/markrussinovich/archive/2008/07/21/3092070.aspx
MICROSHIT VISTAIDS!
If you have a single 32 byte fragment in a 4KB page, you cannot release the page back to the OS. If you can reduce memory fragmentation by 1% you reduce the amount of memory you need to request from the OS by 1%. There is still work being done on allocators that minimize fragmentation. It is not clear how you can defragment memory after it has been allocated though. It would be very hard to safely update the pointers in a C program to point to the new memory locations after defragmentation. Possibly these memory defragmenters move around pages of virtual memory so that the kernel can allocate 8KB and larger chunks of physical memory (Kernels typically bypass VM and prefer pages that are physically adjacent). I understand that the motivation for the 4K stacks in the Linux Kernel was the difficulty in finding two adjacent free physical pages. This would be even more of an issue on Windows which apparently use 12K stacks.
Gentle answers is what 6 years in customer support teaches you.
That, or hating everyone ;-)
Why can't it be both?
If it's for-profit but free, you're not the customer -- you're the product (e.g., the Slashdot Beta's "audience").
I've always disliked the usage of "Virtual Memory" to describe virtual addressing. "Memory" is an object. Virtual Memory implies an object. It is the memory presented to an application on a system that does virtual addressing. Addressing is an action. You can't say "My computer does virtual memory." It doesn't make sense. However, saying "My computer does virtual addressing" does. (And you can't really say "My computer has virtual memory", since it's not an object IMHO.)
Baver
If an application requests 3GB of memory (your example), it is expecting to do something to that memory. Fortunately (for performance reasons), its use of that memory is still in the future.
Basically, the request can be treated as a request for an address space and a potential allocation of memory.
The actual memory doesn't have to be supplied until the application does some with it (or, hopefully, just before). This effect can be usefully exploited to amortize the cost of paging out.
In other words, the OS doesn't have to swap out on such an allocation.
Of course, the actual availability of memory has to be accounted for. Linux uses this approach, with poor availability management:
Quoting "man malloc"
" By default, Linux follows an optimistic memory allocation strategy.
This means that when malloc() returns non-NULL there is no guarantee
that the memory really is available. This is a really bad bug. In
case it turns out that the system is out of memory, one or more pro-
cesses will be killed by the infamous OOM killer. In case Linux is
employed under circumstances where it would be less desirable to sud-
denly lose some randomly picked processes, and moreover the kernel ver-
sion is sufficiently recent, one can switch off this overcommitting
behavior using a command like:
# echo 2 > /proc/sys/vm/overcommit_memory
"
Just another "Cubible(sic) Joe" 2 17 3061
I don't know about BSD, but Linux now uses the 2Q allocator. This means that the cache will be released shortly, unless it is accessed a second time. This should prevent updatedb etc. from forcing out all other pages out of RAM by trying to cache the entire contents of the harddisk.
At my previous work, a work colleague (new IT person in another location) had the bright idea of placing the paging file in Windows onto a RAM disk. This sounded pretty pointless to me, but yeah... maybe would work quite nice in Windows, since it does stupid things :)
I try not to use Windows now, Linux seems to only use swap memory when it needs to (ie: all memory is used)...
How about a quote from Seymour Cray?
"Virtual memory leads to virtual performance."
Seeing how he managed to build the fastest computers I'd say he had a point.
You are not entitled to your opinion. You are entitled to your informed opinion. -- Harlan Ellison
system.ini:
[386Enh]
ConservativeSwapfileUsage=1
Use this on WinXP to have it drop it's insecurities, and use paging only when it actually needs to.
> In Vista the split for 3G/1G of RAM is default.
Wrong, 32bit Vista is still 2/2 by default. The system needs to be booted with a specific switch, to enable the 3/1 split, which was only intended for server class apps that do most of their own caching etc (e.g. SQL Server).
Note that in 64bit Vista, 32bit processes can use all 4GB of (32 bit) VA.
No offense intended, but you need to do some reading.
> Windows XP, by default, the userspace (ring3) will have at most 2 GB of the physical RAM
Wrong. The split you refer to (2GB kernel, 2GB user) is VIRTUAL ADDRESS SPACE, not physical memory. There is no fixed allocation of physical memory between user/kernel...!
The upper 2GB of the 32bit address range is always the kernel, the lower 2GB is mapped to whatever user mode process is currently executing.
> In Vista the split for 3G/1G of RAM is default.
Wrong. The VA split in Vista 32 is still 2/2. It can be booted to run as 3/1, intended for server class apps that manage their own caches (e.g. SQL server) and want maximum VA space, and are marked as such in the .exe header.
Note that Vista 64 bit allows 32 bit user mode processes a full 4GB of VA space.
> Actually on a system with 4G of RAM running in 32bit mode, you can't use all of them even if you try (in Windows XP),
Ok, now you're talking about PHYSICAL memory, and this is not a Windows limitation.
Go work in an HMO and then call me to talk about scrapings of the gene pool ...
The problem here isn't the page cache, but applications that don't properly tell the kernel that the data they're reading in from disk are not going to be used again and so shouldn't be cached.
It's amazing how many programmers still don't know about madvise().
Not for XP only older WinX
XP doesn't need or use system.ini or win.ini
Don't be apathetic. Procrastinate!
QNX, the real-time operating system, does not page to disk. (Not for most processes, anyway. Read on.) The GUI has a bar in the lower right hand corner which shows how much memory is in use; if memory fills up, processes have requests for more memory rejected.
The big advantage of this is consistent performance. Hard real time works. There are no pauses waiting for the disk. This is what you want for entertainment applications, like video players, audio players, and games.
It's possible to set up paging for specific programs, though. GCC has paging, so that huge programs can be compiled, somewhat slowly, on smaller systems. By default, though, paging is not used. Provided that applications aren't bloatware, this works quite well.
It's something to think about for Linux. Should programs be paged by default? Maybe that era is over.
Okay, all the digs asside, the windows paging file isn't actually "real" virtual memory in the Unix/SunOS/FreeBSD/Linux/Real-Operating-System.
Let me explain...
In most "real operating systems" (e.g. modern, shared, multiprocessing, virtual memory supporting, etc) the virtual memory system has "nothing" to do with paging or swapping memory out to disk per-se. In these systems each process in the system lives in its own (usually identical) separate memory space. In this space an particular address, say five (5) is typically private to the process and may be at any physical address in electrical memory.
The fact that it may be anywhere includes the fact that it may be nowhere in electrical memory at all. That's where paging and swapping come in, the pages that haven't been used lately, and which may not be used at all can be copied out to slower storage.
Even more important, a virtual address that has never been used (yet) then they memory may never exist. So even if you never page/swap a single lick of memory, they could have allocate huge memory regions they intend to fill later. The total amount of memory allocated but not used in all the process could even exceed the total physical and available paging/swapping memory. This is called "over-committing memory".
Its important and cool and vital, but this _isn't_ what really happens in windows.
In windows the processes don't (necessarily) live in their own separate spaces. More importantly every element of the process is a "relocatable" image element. EXE, DLL, FOT, CPL and all those other "file types" are actually variants of the EXE format. And in all these formats there are a whole bunch of places in each file where, when it is loaded into the "virtual memory" the executable text and data references have to be "fixed up". This is so that a compiler can make all the DLLs (etc) the same way, but they can be loaded into the process memory "wherever the code will fit." (That's oversimplified but true enough for this explanation.)
So anyway, in a "real" os, the process definition (e.g. the theoretical model of how the process will exist in memory) makes it possible for the image of the program elements in the process (executable) text files to be "final". That is, the process elements dont need fixing up.
The benefit of this in a real environment is that pages of the program that aren't being used can be "forgotten" for free. If the process needs that piece of the program file it can just read it back in. (This is also why you cannot replace a running program without getting "text file busy" etc.)
But windows cannot "forget" a page of text for free because if it needs it again, it will have to rerun the fixup step, which may involve re-fixing up the entire process image.
So what happens in Windows is that After the fixups, the fixed up image is written to the pagefile immediately. This is why you will _always_ see windows using the pagefile. This is also why windows will tell you that the "maximum page file size" is slightly less than half the physical memory. Any one cell/page in the pagefile exactly maps to one location of physical memory. Then, after that, windows can afford to replace one (and only one) physical memory page with alternate contents.
So why must you _always_ use it?
Because if you _don't_ use the windows page file, the kernel cannot overcommit memory for processes.
See most windows programmers, by default, don't know how to tell the compier and library how _not_ to ask for huge virtual memory layouts. So without the assurance of overcommit windows will kneecap itself and prevent you from running programs that you could otherwise run because most of that overcommitted memory will never be used.
This is because there isn't an operating system on the planet that can create memory or swap space out of thin air. In linux process can be killed and process text can be dropped and overcommit can be marshaled. In windows there are none of these last-resort options (practical
Innocent people shouldn't be forced to pay for inferior software development.
--"Code Complete" Microsoft Press
If you consider the "REAL" memory to be that stuff that's normally labeled "cache".
Example this Athlon 64 had 1MB of physical memory on the CPU divided into "pages" of 8 bytes. If your working set is less than a megabyte you will be running entirely from this memory and running very very fast.
As soon as you start 'paging out' to the so called "main memory" your performance takes a dive.
The only differences between this and the level normally mentioned are speed, size and the fact that this level is usually controlled completely by the hardware.
Ummm... no. There are a statistically significant number of humans who aren't notably good at anything. I have unhappily encountered too many of them, both in and out of tech work. This is akin to actually believing that "all men are created equal" merely because it would be really really neat and make you feel all warm-and-fuzzy inside if it were true.
Even if your pollyanna perspective was true, being competent at some task doesn't directly equate with an absence of dumbassery. There are numerous species of "dumb" creatures that can be trained to memorize some task and then mimic (repeat) it perfectly ad nauseum... including H. sapiens. An ability to memorize and mimic doesn't equate directly with intelligence. It's a precursor, a prerequisite, perhaps, but not the Real McCoy.
A shocking number of humans, including many regarded as "average" by testing standards, never actually reach a state of true intelligence. Too many of them are profoundly ignorant and quite determined to remain that way.
Who modded parent up? Totally misleading.
Windows /3GB switch controls amount of virtual memory available to a process. Not physical. Virtual. The rest of virtual memory space goes to OS. Low memory (2 or 3 GB) is user process visible (ring3) memory. The upper 2 or 1 GB are ring0 memory. No need to change memory mapping (no need to flush TLB) on a simple syscall that goes back to the same process. Linux does the same thing. [OSX prefers to give userland 4GB virtual memory]
The fact you need /3GB switch illustrates that there are applications that need more than 2GB of virtual space per process. What if you run a number of these apps? Does 4GB look like a lot of memory now?
In fact, Crysis on 64bit Vista with very high settings and res can start swapping to disk because it outgrows 4GB. This is a single app plus graphics drivers.
And, the #1 reason to have virtual memory: memory-mapped files.
Extra extra: the reason 32-bit XP-SP2|Vista limit address space to 4GB (that is, PCI space combined with memory) is because they don't trust driver writers to do the right thing when DMA buffers straddle 4G boundary.
The 2GB limit is not physical ram, it is virtual address space. Each 32-bit usermode process has its own 2GB of virtual address space, which could be backed by physical RAM. The kernel's 2GB virtual address space is static across all processes.
You may not like that this article isn't flattering to Windows AT ALL, but it does reveal physical memory addressability limits, which may help your understanding of the problems: http://www.appleinsider.com/articles/08/09/03/road_to_mac_os_x_snow_leopard_64_bits_santa_rosa_and_the_great_pc_swindle.html
Encrypted swap
This is totally off-topic but can you tell me whether you came into contact with the top parts of humanity through your work in emergency services too? Is it just the proportions of the parts of humanity that you are likely to come into contact that changes compared to desk jobs?
You have this a bit confused. Let me try to explain it. On 32-bit Windows a virtual memory address is 32 bits allowing you to address up to 4GB. By default, all 32bit Windows systems split the virtual address space evenly. So 2GB of virtual address space is reserved for kernel and 2 GB for user mode. For some applications, like databases, it may be necessary to allocate more virtual memory space to user mode and less to kernel mode. The 3GB switch was created for this scenario. When using it, kernel mode will get 1 GB of virtual address space and user mode will get 3 GB. But it is important to note that this is all related to virtual memory, it doesn't matter how much physical memory (e.g. RAM) you have.
Now as far as physical memory there are separate issues. Again, on 32 bit systems, you can normally only reference 4 GB of virtual memory unless you have some extension mechanism like PAE. You are correct in that some people are seeing Windows reporting ~3 GB of physical memory when they have 4 GB installed. You are correct in stating that this can be because hardware installed on the system has reserved the physical memory (video cards are common for this) or it can be because of BIOS or other system limitations.
Of course, on 64 bit the limitations on physical/virtual memory are much more relaxed.
The topic should be why swap? On Linux you can run without swap! Don't use outdated OS ;)
If you have caches of a size smaller than your real RAM, the order in which you try to access memory really CAN make a difference because cache is many times faster than regular RAM and will try and do things like speculative readahead. If what you are working with is already in the cache by the time you request it then you won't stall for as long.
If you are forever causing the cache to become flushed and forcing the cache to be refilled with a different contents (perhaps because you are causing a large number of random memory access and the cache's readahead is getting your future access wrong so it has to be turned off) then performance will by comparison be slower than a sequential memory access workload.
The above is of course a gross simplification (and doesn't apply if what you are reading fits entirely within cache and is already there). If you have the technical chops you can read more about how order of access can have an impact on speed in Ulrich Drepper's what every programmer should know about memory on LWN.
I've run Vista on less than 1GB (512MB) and it ran fine. Vista used 800MBs on your system, because vista adjusts its memory usage based on how much memory is in the system. I think I had 400MBs used on the 512MB system, enough for simple IE sessions and a little quicken which is all the client required.
Once upon a time I worked as a salesman at Sears. Anyway, I had a guy storm out of the store because I refused to discount a television if he didn't take the remote. This was in the late 90's by the way, not the late 70s.
I have a theory that the truth is never told during the nine-to-five hours. - Hunter S. Thompson
Here is the blog post by Mark Russinovich which explains the issue with 4Gb on 32-bit Windows in more detail.
in that programs that are essentially sleeping; checking for mail or somesuch task (and there's quite a few) have to stay resident in physical memory.
That sounds ok at first, but it's often better to page them out as they're rarely doing anything so the memory they use is better off used for caching or something like that.
Windows is pretty good at side-lining such programs so it can give processor active processes more physical memory to play with, which ultimately speeds up your machine.
throw new NoSignatureException();
I am runnning swappiness=0 right now, and some stuff got swapped out when I copied a few dozen GB of files from one disk to another.
When called into help a particular company with it's IIS web systems I was able to bring the site back to life by doing a few simple things.
Of 20 Load balanced IIS servers.
1. Buy from local shop sticks of ram for 2 of the box's bringing total ram to 3Gig.
2. Disabling the Page file.
3. Only load balance the site to those 2 box's.
4. Power off the other 18.
Total time to implement 6 hours.
Site performance went from +30 sec per page to around 6 second. ( Heavy pages )
Biggest challenge convincing the architects and management that less was more even after implementation. They simply didn't believe it even after it was working in production.
2 Days after I left they undid what I did for them. The site now sucks again. Apparently I bruised some ego's and that couldn't stand.
Oh well. Some people will never learn.
Dont forget virtual memory also allows process isolation (aka protected mode) to happen. In 32-bit environment, each process gets its own copy of 2GB of virtual memory or 3GB if they are compiled with /3GB flag.
x86 system are designed in this way so they could not corrupt each other accidentally. In practice, you can have any program crashing as often as it wants, without bring down the entire system.
Also, when they say "Windows reserves 2/3GB of memory for user mode process" or "Windows reserves 1GB of memory for kernel space", it means that is the absolute upper limit for memory usage. You should read it as "each process cannot allocate more than the 2/3GB limit, trying to do so will result in 'Out of Memory' exception", rather than "That process exclusively own the 2/3GB, nobody else can use it".
Mark has a good explanation about this. http://blogs.technet.com/markrussinovich/archive/2008/11/17/3155406.aspx
I'm intrigued... tell me more...
Any sufficiently advanced intelligence is indistinguishable from stupidity.
I understanding SSD has finite write capability and having paging/swap on SSD would eat away its life. How does one overcome this? Page/swap to a regular spinning disk?
Thanks
blue_teeth
Ummm... no. There are a statistically significant number of humans who aren't notably good at anything.
to be more precise a lot of people are not good at anything that we know and value, they could be good at stupid things (soccer strategy) and never know it, never make others know and certainly never apply it to something good.
but teoretically anyone could have some bright (but unuseful/unused) capacity.
There are numerous species of "dumb" creatures that can be trained to memorize some task and then mimic (repeat) it perfectly ad nauseum... including H. sapiens. An ability to memorize and mimic doesn't equate directly with intelligence. It's a precursor, a prerequisite, perhaps, but not the Real McCoy.
you forget a prerequisite of intelligence evolution, THE NEED FOR IT.
if they could live their lives in a dumb way many lazy person never try to reach an higer lever of intelligence, never ever feel their lack of it!
A shocking number of humans, including many regarded as "average" by testing standards, never actually reach a state of true intelligence. Too many of them are profoundly ignorant and quite determined to remain that way.
the deepnes of their ignorance is an huge problem because they never perceive their ignorance, and many of the few that perceive it refuse to believe it and assault the condition that make them feel stupid.
that is the sad problem.
I have a system with Windows Vista Ultimate (64-bit) installed on it
If you install this bloathware then you have to expect that 4Gb is not enought...
Interesting posts so far, but they all assume that the host is running on the metal. What are the recommendations for running VM guests which tend to have crap disk access speeds?
I've turned swap off and had real performance improvements, but hell, occasionally it crashes (not a huge problem).
I am using a 3.25 GB RAM machine with Windows XP on it. Was using a PF size of 2048 MB. After reading the post, i lowered it to 128 MB, well below the Windows recommended 4603 MB. To my surprise, there has been no negative effect on the system performance. The resource hungry Java apps which I use frequently in fact seem to be running much faster. As a tech support pro, I've always been recommending higher PF sizes for those running apps with large mem requirements. Seems that I got it wrong till now.
And yes, Linux uses a much less amount of swap, almost a fraction of what Windows PF uses for running the same Java app under similar conditions on machines with identical configuration.
If I remember correctly, on DEC VAX systems, there were both pagefiles and swapfiles (PAGEFILE.SYS and SWAPFILE.SYS). The difference between them was that the pagefile was for individual pages of a process's working set, whereas the swapfile was used when an entire process was moved out to disk - akin to hibernating a process. You tried to avoid getting to a point where the system started swapping rather than only paging, as everything s.l.o.w.e.d d..o..w..n t...o a c.....r.....a.....w.....l
No. However last time I tried it it broke my 8800GTX, because it couldn't map 768MB of system address space to itself anymore.
Maybe I spend too much time on *NIX, but for me:
Virtual Memory = Physical Memory + Swap space.
So the question should then read "Why use a swap file..."
RAM can get fragmented to the stage where in total there's plenty of it free, but not in a big enough single chunk to satisfy the requirements of an application.
Using some silly numbers by way of example....
I went to a talk given at Microsoft Research, here in Cambridge, UK, a year or two ago, the theme of which was the forthcoming changes we can expect to see in operating systems.
One of the issues that was discussed was the use of virtual memory/swapping - the technique was invented in Cambridge I think. The idea behind virtualising resources is to be able to share resources amongst competing programs. But in a world of 8GB RAM, the point was made that RAM is no longer a limited resource which needs sharing, and consequently, except for when you are running programs like simulations which need vaaaast amounts of RAM to run, virtual memory isn't needed.
The speaker said that Microsoft had done some experimentation with turning virtual memory off on computers with large amounts of memory, but that it hadn't gone well. One problem is that some programs are written with the assumption that virtual memory is present and will be needed, so they explicitly swap pages in and out. These programs die. Unfotunately at the moment Windows is one of these.
So, good idea in principle on a modern system running a set number of tasks, but not possible at the moment in practice.
Jeff
We're way off topic here, but - try being a Math teacher some time. "Oh, I don't know how to add and subtract, my kid does that for me." WTF? Are you kidding me? So you're OK being ignorant? Gaaaaah!
One reason that an OS kernel will swap out even when it's not actually forced to do so is because it thinks it can get better overall performance by having something else in the memory instead. You don't have to run out of RAM, just have your OS decide there's something different that would be more useful to have quick access to. So whatever your OS is doing is a strategy that's intended to improve overall throughput in various common cases.
I'm sure there are plenty of replies here already that point this out. I think it's worth noting, though, that optimising for overall throughput - whilst good on a server - is not necessarily best for interactive response times.
Consider the following: every time your CPU switches between tasks, it wastes some processing power. There's an overhead that must be paid in order to do multitasking. Modern OSes interrupt tasks when a timer fires, no matter whether they're still using the CPU - this is the pre-emptive multitasking that makes our interactive experience reliable and (mostly) free of lock-ups. However, since we're interrupting tasks that could have continued to make use of the processor, we're paying a penalty in the overhead required to switch tasks at all. If we'd allowed the task to run until it was complete, or waiting for IO, we'd eliminate that overhead, since the CPU wasn't busy anyhow. We're actually wasting CPU time and therefore reducing throughput (overall work done in the system) in order to achieve a better interactive experience (amongst other things).
Paging in order to make space for caches can be like this. The OS is looking to improve the system's overall efficiency but (depending on what's running) not necessarily the user's interactive experience. For one thing, the OS paging strategy is not even guaranteed to work - it can slow things down if it makes the wrong decision. Secondly, it's not (on a general purpose OS) optimising for interactive experience, it's a compromise between that and throughput. Potentially we're actually damaging our enduser experience by tying ourselves to algorithms that are partially designed to improve throughput on a server-style or mainframe-style workload. Sometimes you actually want to reduce throughput in order to improve user experience; where you draw the line depends on your particular job. Sometimes real-world throughput isn't hurt significantly / at all by improving your response times - the pre-emptible kernel introduced in Linux 2.6, for instance.
Not using swap is and / or using a ramdisk is a hack that shouldn't be necessary if the OS understands and optimises perfectly for the use patterns you're interested in. OSes can't do perfectly in reality and they do a pretty good job. They could still be better, so questions like this are very relevant and are based firmly in reality.
If the facility doesn't seem to be necessary for your usecase, do you really want it? With modern computers, often the answer is still "Yes" but it's still right to ask.
http://blogs.technet.com/markrussinovich/archive/2008/11/17/3155406.aspx
Some of it blew over my head, but other bits were very interesting. In the article, he explains physical, paging, and virtual memory usage - and my favorite, the only REAL way to calculate what the size of your paging file should be! (How many opinions have you read on that one?!)
If you're curious, here is the short version of how to calculate your needed paging file size:
Hope that helps! I suggest you read the rest for more info on this approach.
Vista Ultimate (out of the box) is going to use a lot of memory. 4gb might not be enough here after a couple updates. Anyway, I run Ubuntu Hardy on AMD64 3300XP+ with 1gb of RAM and 5gb swap and I rarely notice any slowdown at all. The bottom line is that swap is NOT bad, if you have the disk space for swap then use it.
The major problem I see in my country (USA) is that much of our primary education system did not emphasize problem solving skills in the 50's, 60's, and 70's, based on my experience deal with people my age and older. I personally can't speak for anything later than that.
I think the reason for that was that learning to problem solve requires lots of time (in teaching and practice), is not Scantron(TM) friendly (which is still used in schools around here), and even normally bright kids would rather learn how to make high scores on the SAT or ACT to get into the college of their choice and possibly a scholarship than spend oodles of time in practicing problem solving skills.
As a result, we have tens of millions of Americans that cannot problem solve themselves out of a paper bag.
These people were not even allowed to touch a computer back decades ago and now they are on the "innerwebs" by the millions. Many of these people had AOL but they want DSL/Cable Modem and abandoned the once "idiot-proof" domain of AOL (which hasn't been seriously broken since 7.0).
Mathematically speaking:
Theorem of "LOTS OF FAIL on the Internet and LOTS of frustration at the help desk"
Let "Wide open Internet" be A;
Let "Millions of people that are not familiar to computers" be B;
Let "Millions of people that can't problem solve" be C;
Let "LOTS of FAIL!" be D;
Therefore:
D = ABC
Q.E.D.
Maybe we need to create a Bugzilla site to track the bugs in various "releases" of the human hardware/firmware/software (hey, no one is sure ) like the guy you were "quoting". Of course, I don't think the manufacturers/developers are producing any patches so it might be futile.
It's a FESTIVUS miracle.
You insensitive clod.
Where's my aluminum pole?
"Is there a reason why my system should even be using the virtual memory anymore"
I've wondered that too. I recall when DOS couldn't handle extended memory, so a used it as a fake drive and ran tha apps from there ..
davecb5620@gmail.com
Something similar happened to my phone company. Lightning struck something, which caused everyone's phones lines to stop working in the neighborhood. But they were able to reroute traffic, and had stuff working within a few hours.
If I called (via cell phone) and someone told me I wouldn't have a land line for several days, I'd like to know why your company didn't have something like this already planned through.
Don't blame me for paying $15 a month. That's how much your rate is. Do you expect the customer to voluntarily offer to pay more money? Ensuring you have a good recovery system planned, is your problem, not the customers.
I'm not denying the lawyer is an idiot, but you shouldn't begrudge him because he expects your service to work as advertised. It sounds like your company has the headless chicken people in charge. Maybe if they did their job right, you would never have to hear from the idiot lawyer.
Long before Windows, virtual memory was 'invented'. Given that, the term has a specific meaning. As others have mentioned, it is a method for making programs believe they have unlimited memory space, whilst sharing the actual available physical memory between numerous programs. This 'feature' has a cost - references to memory must be translated from a virtual address to a physical one, by memory management hardware (and sometimes software). Until most recently, Intel processors used a separate chip to manage this. AMD put their memory controller onboard a few years ago. In terms of memory performance, Intel lagged for the past few years because their outboard memory controller consumed extra time to do its job. Moving the controller onboard removes an electrical interface or two, thus speeding things up and generally improving efficiencies.
The original post, I thought, was brilliant. Why are we devoting all this chip real estate (or, in the past, chips), to sharing a rare resource (memory) when that resource is no longer rare? Grant, virtual memory gives us other advantages such as ensuring one program doesn't write in the memory space of another, but surely there are other ways to do that. If we did away with virtual memory and returned to the old (ack! DOS) days of physical memory references, we could devote that chip real estate, power quota, etc to other worthwhile pursuits, like making my twitter pages load faster.
- The Kessel run is for nerf herders. I can circumnavigate the entire Central Finite Curve in a lot less than 12 parse
There's a difference between being stupid and ignorant. There's no shame in being ignorant. We all started out ignorant. We just have to work to overcome it. Stupid is for those that refuse to try.
Why do they have to be mutually exclusive?
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
And that is that, in Windows, the paging file SHOULD be as large as your total allocated memory (physical + swap) - because it contains a continously updated image of the ENTIRE virtual memory model - including whats in your physical RAM.
This is used for situations like kernel panics (aka blue screens) to generate crash dumps... among other things...
So my interpretation of the OP is that his computer isn't using any swap space - his paging file probably stays very close to the actual allocated RAM (I'm not sure how the update mechanism works, but I imagine it's dirty-cached writes managed by the disk controller... but I might be assigning far too much intellignce to Microsoft engineers with that assumption... but if some sort of cache is involved, one would imagine there is some amount of hysterisis between the two measurements)
Ignorance is simply lack of knowledge; I think what you are griping about is stupidity; which is something people have to learn first. Look at children - nobody is more honestly ignorant than a young child; and what is it children are more than anything else? No, the answer is NOT "tender and tasty when cooked well"; they are curious. Stupidity is what you get when you believe that "curiosity killed the cat" and that sort of nonsense.
That old guy you know, he has kept his curiosity alive; and research has shown time and again that when you keep challenging your brain, it becomes more resistant to the deterioriation that so often hits you in old age. So, if you end up one day thinking that it has become simple and easy, it is probably time to go and learn something entirely new, like Chinese or playing renaissance lute.
What are peoples experiences of Windows vista's ReadyBoost?
Will this argument still be relevant in 2 years when everyone runs OS off of a SSD?
I know I am late in replying, but maybe you'll look down this far. The simple fact is, regardless of how much RAM you have, there are some things that Windows wants to page. Period. So, you really have to situations when the system pages (grossly simplified):
1. The system is low on physical memory (RAM). This is what people USED to encounter a lot. With RAM being cheap nowadays, this happens far less. When this happens, your system will slow down a lot, and it is considered a Bad Thing(tm). The reason the system slows down is that RAM is fast and disk is slow. Paging (aka using the swapfile) is a slow process compared to RAM. Similarly, there are wasted cycles as the system determines that it has to page, so this type of virtual memory use needs to be avoided.
2. Paging as architected. Windows, regardless of the amount of RAM, will page certain things. Even if you had 32GB of RAM, it would still page some stuff. This paging should not affect performance.
You are experiencing paging type #2, most likely. It is possible you are thrashing your system's 4GB of memory through the use of big applications, but unlikely. If Task Manager or perfmon show lots of free physical memory, you are paging normally and should not be concerned.
Of course, it is possible that you have an application improperly paging data, artificially causing situation #1, which would be bad, but this is unlikely. If you see lots of disk access and experience system slow-down, this is an indication something is wrong.
Why Use Virtual Memory In Modern Systems?
1. Every application on a modern system is running in its own virtual address space, and these virtual addresses are then mapped to (different) physical memory addresses. This is called "virtual memory".
You are talking about swap space, not virtual memory.
I have a system with Windows Vista Ultimate (64-bit) installed on it, and it has 4GB of RAM. However when I've been watching system performance, my system seems to divide the work between the physical RAM and the virtual memory, so I have 2GB of data in the virtual memory and another 2GB in the physical memory.
2. Just because Windows has poor memory management doesn't mean that swap space is bad in general, or is an outdated concept.
We are running AIX on a pSeries with 576 GB of main memory and lots of swap space, and it doesn't do this. It's a software problem - all you need is better memory management.
Not a helpful response. It is obvious the article uses "VM" to mean "page file" or whatever the Windows jargon for that is.
I immediately understood the question, which is a simple and natural one. "Why does Windows appear to use only 2GB of the 4GB RAM available before starting to page on a large scale?" Why start using VM mapped to disk, when there is plenty of much faster RAM (apparently) going begging?
It's typical of Slashdot that this small, and quite transparent, misuse of terminology touched off hundreds of replies arguing about tech support and how stupid users are. A good tech support person would ignore the incorrect term, answer the intended question, and only then (if it could be done tactfully) explain the difference and the importance of using exact language when asking technical questions.
I am sure that there are many other solipsists out there.
Here's exactly what you're looking for:
http://www.tomshardware.com/reviews/vista-workshop,1775-6.html
because long time mac users don't really add to technical discussions, they're that stupid and so are you for even cluttering up slashdot with this useless bit of trivia.
A reasonably accurate car analogy? Is this some kind of hoax?
Your brain is not a computer.
I asked the same question back when 64mb was 110 bucks delivered....
virtual is not virtual, it is a silently written wherever you or the os puts it, ready to be read as fast as any memory. the slowness of it all is the write. once written, who cares...if the functions are called very often, the writes and deletes of real memory unnecessary. To Free up the other "physical" called dynamical ram for dynamical things. Very simply answered? How much memory is used is NOT performance...and it is never a dumb question.
The fact is, I started off very optimistic, years ago, and believed that all men were created equal. I believed that the only obstacles were people's can-do attitude.
Years in the tech industry (specifically tech support) have definitely jaded me. I cannot help but agree with you completely! And after a while, you realize, despite your attempt to be humble about it- you can't help but realize your thoughts seem a bit conceited and elitist.
I personally try not to be elitist, or too self involved. But it's hard to ignore the facts. There are so many people who don't have basic critial thinking skills, let alone the desire to develop them, I cannot help but start to think poorly of my fellow species.
It's frustrating, and lonely, and annoying- all rolled into one. It's hard to relate to people in general.
Belief? Hope? Preference?The Existential Vortex
In all of this, I really expected to see someone comment about bad tech support experiences as a counter to these--more than half of the people I've talked to in various tech support departments are sitting in front of a screen clicking a mouse to advance to the next problem-solving step because they themselves have no clue what they are doing. And then customer billing support is even worse!
Let me start by saying my technical expertise ends at the boarder of self-built PCs and playing with Linux distros just enough to crash those same PCs many times. I do want to ask though, from a crash/recovery perspective doesn't the use of page files increase the OS's ability to reassemble data that would have otherwise been lost during an improper shutdown? Since that information resides on a HDD, it doesn't disappear with the loss of power like it would if stored in RAM (though again I don't have the tech experience to know whether it would be accessible on reboot).
I ask because that big glowing power button appeals to the less computer-inclined when things start slowing down or going wrong. Since they represent the majority of users, Microsoft (and any for-profit OS developer) would want to minimize the damage their ignorance could do, right?
A shocking number of humans, including many regarded as "average" by testing standards, never actually reach a state of true intelligence. Too many of them are profoundly ignorant and quite determined to remain that way.
...and about 50% are below average.
You need RamDoubler... Maybe even two licenses!
I know watching Palin was traumatic, but really, with a little professional help you should be able to get over it rather quickly...
Since you mention Vista, go to the horse's mouth..
There are several good whitepapers on this from NT engineers at Microsoft.com.
If you are aren't in the mood to read, go to channel9.msdn.com and look up topics on NT kernel, Vista, etc.
There are some really good explainations about VM and specifically how NT works and what Vista does differently in contrast to XP and previous version of NT memory management.
Do you need one, No.
Should you have one, well it probably doesn't make much of a difference in terms of performance on Vista. Vista doesn't use or dumpt crap to the pagefile in the 'sense' that you or many people would think based on previous OS memory management schemes.
Since you are using Vista, I would say, leave it on, as there are a few performance benefits, like speeding up hibernation, and applications that are written to 'allocate' RAM, but then dump that to the pagefile as it probably won't use it very often. (Even though memory allocation on NT is dynamic, some application developers know what they are doing and will take advantage of the pagefile to hold bitmaps or other things it seldom references, and there is no need to keep them in physical RAM, and they often don't go the extra step to check for pagefile size/presence and just assumes the system has one, as it use to be required years ago.)
On XP turning off the pagefile can help performance, as XP uses a more conservative method to dump applications to the pagefile, but again, if you are using applications like I describe in the paragraph above, you are losing RAM that isn't used for much, so then it comes down to how much 'extra' RAM you have and if you have a chance of ever hitting the celing, which is kind of a 'duh' logical conclusion.
"Any thoughts on this matter or could you explain why the system is acting this way?"
Leaky browsers.
The lower limit on Fragmentation? Not really. If you have a hundred more mallocs than frees, you'll have to track one hundred mallocs and perhaps another hundred chunks of free space. A smarter algorithm can use clever heuristics to avoid some fragmentation at the cost of performance, but if anything this would usually result in less chunks of free space. At the end of the day no matter how smart the algorithm or how many chunks it manages, if you allocate memory in 32bytes chunks and then free all but one 32 byte chunk from each 4K page, you'll end up with 99% fragmentation.
The smaller the stack size, the easier it is to allocate memory for the stack, but also the harder it is to write drivers etc. that don't overrun the stack.
Most allocations in a paging system are done by page. A 8k stack on x86 means that you need to find two adjacent free pages, finding two adjacent pages can be difficult. A 4K stack means that you only need to find one page, i.e. so long as there are pages free you can easily allocate a 4K stack. Thus there is little benefit to having a stack smaller than 4K, having a 1K stack would save a small amount of memory, but wouldn't make it any faster to allocate a new stack.
Technical terms sometimes have different meanings in different contexts. Most grown-ups can deal with this. A "turtle" in the US isn't always the same as a "turtle" in the UK, and a UK "fanny" is not the same as a US "fanny". These little linguistic inconsistencies are just part of life, and anyone who's done any tinkering at all with Windows should have come across this piece of MS terminology (virtual memory, not turtles).
And if someone doesn't have at least a passing acquaintance with the Windows settings, or with the Windows literature or technical materials (in other words, if they haven't used the product in any depth, or RTFM), then they probably oughtn't be giving out technical advice on it.
Eric Baird
Excellent advice! Another step to boost performance is to have that partition on a separate physical disk drive. Sod's Law says that the pagefile will often be most active when you're loading and saving conventional files (or launching apps), because that tends to be when things are moving into and out of memory, or when a document's edit history is being wiped. So a problem that happens is that the harddrive is being asked to read and write files to one part of the disc, while the pagefile changes are demanding that it reads and writes to a different part of the disc.
So even if the two sets of files are individually nice and contiguous, in practice the poor HD heads spend ages scooting back and forth between one region of the disc and another, and since those seek operations are pretty slow compared to the actual data transfer, the result can be a noticeable slowdown and a noisy HD desperately thrashing the heads between the two r/w areas.
OTOH, if both sets of data are on their own separate physical drives, then you have two independent sets of heads each smoothly doing their own thing. Potentially much faster, and you notice a marked reduction in rattly noises.
I wouldn't be surprised if it reduced HD wear, too -- do you remember the old IBM "DeathStar" drives that could expire after a matter of months when you ran an OS on them, because they were originally designed as "streaming" media drives and couldn't cope with the heads being continually wrenched about by a constant stream of seek requests?
Eric Baird
If your RAM requirements go up, and its difficult to upgrade a machine, you might also try the compromise solution of plugging in a fast external memory card, and using that external volume for a fixed-size swapfile.
Eric Baird
That certainly answers my question. Thanks. :)
zosxavius photography
1 - you are misinterpreting what you see, the O/S has swap (as it should) but it not using it.
2 - you have so much stuff running (or the OS does) that the excess must be paged out to swap. This is possible with VISTA.
3 - the OS is just doing a bad job handling memory. I don't see anything like this on Linux, so it's not a feature of "modern operating systems" but may be a feature of a legacy system like VISTA.
Locality of reference helps caching at the L2-cacheline size and below. So about 128 bytes. Now the virtual address can't change because that would mess up all the pointers, so you can only change the virtual address translation table... which is at a granularity of 4KB or so. Not gonna help with caching issues. Plus, just cramming all used memory together wouldn't be good anyway, sometimes it's a good idea to leave gaps to prevent false sharing (because only one core can be using a particular cacheline at once).
But usually memory fragmentation means fragmentation of heaps which are used by memory management components of most programming languages' runtime libraries. That's a different issue entirely, there's no speed advantage gained by the memory being sequential but it allows the programmer to use array indexing and pointer arithmetic.
Oh, yeah....
I have tried this and it greatly hindered performance with a 4g ram setup using vista hp 64, therefore you are busted.
Basically the current kernel memory allocator does not typically stop long idle memory being swapped out to make room for pagecache in the updatedb case (you can see the the Linux swap problem being discussed back in 2004 when the allocator had already been set). The classic "OpenOffice left overnight is swapped out by early morning updatedb run" problem is still here (because OOo memory has been idle for so long).
While there is an madvise option to make mmaped memory dodge the page cache there is a need for more. As recently as 2007 there have been attempt to try and let userspace warn the kernel that what it is going to do shouldn't stay in the pagecache, but it's unclear how far it progressed. The current solution seems to be "don't run updatedb".
Half of the slashdotters on this topic are talking things way off topic.. You all are insensitive clods! ;)
I didn't respond to most of these responses right away, as I wanted to think some more on the issue based on everyone's replies. Not sure if anyone will ever see this at this point, but here goes. . .
"what's the issue with simply saving the undo data, along with each revision".
If you can do that in a 'metadata' block in the filesystem, seperately from the actual file data, fine. But a text file does not have any concept of a journal of changes. If you create a new file format with that info, fine, but that is no longer a text file. So, you do have the issue of making sure that files remain the basic data that they are. Notepad, as our example, is designed to work with text files. Most other data file formats do not have the ability to save undo data in them (perhaps doc or odf do?).
Also, what if I am using a USB flash drive formatted with FAT32 or NTFS? Those don't support that, so I would still need the 'old style' save capability.
In fact, I would argue that for the most part, any type of functionality like this would have to be implemented at the O/S filesystem level, not the application level, although the applications *also* need to be changed so that if they detect a filesystem that supports this, they start auto-saving or continuous saving, but revert back to 'traditional' behavior when using a 'legacy' filesystem.
The biggest roadblock to this is that it requires every (well, most, anyhow) application to be updated to be compatible, before you start to see the advantages. That is, you need first the filesystem to be changed (to support the seperate-from-the-file-data change journal), then additionally the apps. Any 'legacy' application could still potentially interrupt your system reboot even with this new capability.
Do I think auto-versioning filesystems are a good idea? Heck yeah. But I'm not sure that applications having a 'save' button is really that big a deal, to worry about updating thousands of applications to get rid of it. Also, you still need to decide what directory/volume to save the file to, somewhere. Maybe when you first create a new document, before you type anything into it (or, as the case may be, record audio, record video, create graphics, create models, create level maps, etc).
I'm currently marvelling at how wonderfully puerile and completely useless that comment was.
XML is like violence. If it doesn't solve the problem, use more.