Slashdot Mirror
Why Use Virtual Memory In Modern Systems?
Cyberhwk writes "I have a system with Windows Vista Ultimate (64-bit) installed on it, and it has 4GB of RAM. However when I've been watching system performance, my system seems to divide the work between the physical RAM and the virtual memory, so I have 2GB of data in the virtual memory and another 2GB in the physical memory. Is there a reason why my system should even be using the virtual memory anymore? I would think the computer would run better if it based everything off of RAM instead of virtual memory. Any thoughts on this matter or could you explain why the system is acting this way?"
You must be confused about virtual vs. physical memory. In modern processors, there is no penalty for using virtual memory, all translation from virtual to physical address space is done internal to the processor and you won't notice the difference.
So all the physical memory installed in your PC is used by the processor as one big pool of resources. Processes can think whatever they want and address huge memory spaces, that's all in virtual land. Virtual memory only starts impacting performance when pages are being swapped in and out, because all your processes need more resident memory than you actually have.
Swapping means accessing the disk and freezing the requesting process until its page of memory has arrived from the disk, which takes millions of processor cycles (a lifetime from the processor's point of view). It's not so bad if you swap once, as the processor can work on other processes while waiting for the data to arrive, but if all your programs keep pushing each other out of physical memory, you get thrashing and consider yourself happy if the mouse pointer is still responsive!
So you may want to change the title of your post to: "why use physical memory in modern systems?". I would point you to an article I wrote on that topic in 1990, but somehow I can't find a link to it on the web :-)
fairsoftware.net - software developers share revenue from the apps they build
Absolutely not true. You can even install and run Vista on a computer with 1Gb ram and no page file. And run applications. So it doesn't reserve 1Gb for itself and thus, your myth is busted. Vista's memory manager will use as much memory it can (free memory is a waste, so it will use it rather than watch it empty). But as soon as a process needs memory it will give it back.
It's time to realise that Abble's products are the biggest abomination these days. Just say NO to the dumb iAbble way!!
But, at least in Mac OS X, exited processes consume "inactive" memory - basically being kept in memory until they are launched again. If Vista has a similar implementation, your swapfile may just contain a bunch of pages left over from previously-running applications. Are you experiencing actual system performance problems? Concerning yourself too much with the numbers only can be a bad thing.
At the very least, the amount of swap should be easily configurable like it is in Linux. I haven't actually used a swap partition in Linux for years, preferring instead to have 6 or 8gb of RAM, which is now cheap.
It is, (Right-click "My Computer")->Properties, "Advanced" tab, "Settings" under Performance, "Advanced" tab, "Change" under "Virtual memory". Almost as easy as "dd if=/dev/zero of=swapfile bs=1G count=1; swapon swapfile", spclly if u cant spel cuz u txt 2 much.
I recall back in 2002 or so, a friend of mine maxed out his Windows XP system with 2gb of memory. Windows absolutely refused to turn off paging (swap), forcing him to whatever the minimum size was. The solution? He created a RAMdisk and put the paging file there.
On Linux (and other modern systems, perhaps now including Windows), you can turn off swap. However, the Linux kernel's memory management isn't so great at the situation you hit when you need more memory than you have, but you can't swap. Usually, the memory hog crashes as a result (thankfully, Firefox now has session restore). I might be slightly out of date on this one.
A well-tweaked system still has swap (in nontrivial amounts), but rarely uses it. Trust me, you can afford losing the few gigabytes from your filesystem. Again in Linux, /proc/sys/vm/swappiness can be tweaked to a percentage reflecting how likely the system is to swap memory. Just lower it. (Though note the cons to this presented at the kerneltrap article above.) My workstation currently has an uptime of 14 days, a swappiness of 60, and 42/1427 megs of swap in use as opposed to the 1932/2026 megs of physical memory in use at the moment.
This is summarized for Windows and Linux on Paging at Wikipedia.
Use my userscript to add story images to Slashdot. There's no going back.
thinks "Virtual Memory" is the same thing as paging...
Mac Classic (OS 8 for sure) used the term "Virtual Memory" the same way Windows today uses "Page File" or unix uses "swap", so you can at least understand why some people might be confused by this.
db
I am literally 3000 tokens away from the chaotic crossbow --Stephen
Man, I hated that assumption in 2000, and I hate it in XP. It's the one that means when you bring Firefox up after it has been minimized, that the OS will have to laboriously swap in all of the memory for it from disk, which takes forever when you're talking about a slow laptop hard drive. I made it a habit of switching the paging file management to "manual" and reducing the paging size down to 2mb. It makes the whole system way more responsive when you're like me and have a bunch of applications open at once and in the background, and memory is so cheap that buying a little extra so you never run out (2GB) is easy.
I read the internet for the articles.
I think he is referring to the userspace/kernelspace split in Windows NT. On 32bit Windows XP, by default, the userspace (ring3) will have at most 2 GB of the physical RAM, and the kernel space would get the rest (some of it paged and some of it not). On systems with more than 3G of RAM (a lot by 2002 standards), it was kinda pointless to reserve that much for the kernel space, so they added a boot.ini flag that changed the split to _AT_MOST_ 3GBytes for the userspace and the rest for kernel space.
In Vista the split for 3G/1G of RAM is default. Actually on a system with 4G of RAM running in 32bit mode, you can't use all of them even if you try (in Windows XP), because right under the 4G limit you would have the PCI memory address mappings, that can be as large as 512M for a common video card with half a gig of RAM. Add to that the RAID controllers and the other hardware, and you have about 800megs of RAM unused because they can't be addressed, as their address-space is used by the installed devices.
I think that http://support.microsoft.com/kb/823440/ and http://support.microsoft.com/kb/171793/ should describe what I'm talking about pretty clearly.
UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever ones.
What is left over is the physical memory needed by the system. It seems like the OS preferred a fixed amount of memory, so it would just set up fixed space on the hard disk. So, even if all you have a 1 MB of available memory, the system would set up say 10MB, and that is what would be used. The pages that are being used will be stored in the physical ram, while everything would be stored on the HD.
If page management is working correctly, this should be transparent to the user. The management software or hardware will predict what pages were needed, and transfer those page to ram. One issue we I had was available memory was not hard disk plus physical available ram, but was limited by the available hard disk space.
So, it seems to me that virtual paged memory is still useful because with multiple applications loaded, memory can be a mess, and big fast hard drives it should not be an issue. I don't how Vista works, but it seems that *nix works very hard to insure that the pages that are needed are loaded to physical memory, and page faults do not occur. In this case, where virtual memory equals available physical memory, it would seem that since only physical memory is being used, there would be no performance hit from virtual memory. it is only there in case an application is run that need more memory. It is nice that we do not get those pesky memory errors we got in the very old days.
"She's a scientist and a lesbian. She's not going to let it slide." Orphan Black
You can also adjust the "swappiness" of a computer running linux. I've set my desktop to have a swappiness of 10 (in a scale of 0 to 100 where 0 means don't swap at all). In Ubuntu, you can do sudo sysctl vm.swappiness=10 to set the swappiness until next boot or edit /etc/sysctl.conf and add vm.swappiness=10 to the bottom of the file to make it permanent.
The default swappiness level is 60.
God is imaginary
That page mostly talks about what virtual memory is and doesn't directly list why it is an improvement.
Some folks have already mentioned the fact that it eliminates memory fragmentation, and that it allows mapping of files and hardware into memory without dedicating (wasting) part of the address space to those uses.
Another reason is that you can have 2^64 bytes of total system memory, even if the individual applications are 32-bit, and can only address 2^32 bytes of memory. Since the 32-bit applications are presented a virtual address space, it doesn't matter if their pages are located above the 32-bit boundary.
It means that per-process memory protection is enforced by the CPU paging table. Without virtual memory you would have to reimplement something like it just for memory protection.
It means that the linker/loader don't have to patch the executable with modified address locations when it is loaded into memory.
The above two reasons have the corollary that libraries can be shared in memory much more easily.
And that's just off the top of my head. Virtual memory is a very, very useful thing.
You must be confused about virtual vs. physical memory. In modern processors, there is no penalty for using virtual memory, all translation from virtual to physical address space is done internal to the processor and you won't notice the difference.
Huh? That's totally wrong. If it were true, you wouldn't need any RAM.
It's true that address translation is hard-wired in modern processors. But that just means that figuring out where the data is is as fast as for data that's already in RAM. Actually reading or writing it is only as fast as the media it's stored on. So if you have a lot of big applications running, and there isn't enough RAM for them all to be in physical memory at once, your system "thrashes", as data migrates back and forth between the two media. That's why adding RAM is very often the best way to speed up a slow system, especially if you're running Microsoft's latest bloatware. Defragging the swap disk can also be helpful.
To answer the original question: actually, you often don't need any virtual memory. But sometimes you do. Disk space is cheap, so it makes sense to allocate a decent amount of virtual memory and just not worry about whether it's absolute necessary.
I note a lot of people are insisting that "virtual memory" refers to the virtual address space given to a execution context, and what the author really means is "paging".
The funny thing is that these are traditionally poorly defined/understood terms which are gaining a hard consensus for the meanings due to some recent OS books, and poor comp-sci education which insists on a particular definition. Everyone is faulting M$ for using the term incorrectly, even though the original mac OS and other OS's used the term in the same way. Wikipedia defines it one way and then goes on to give historical systems which don't really adhere to the definition. For example the B5000 (considered the first commercial machine with virtual memory) didn't even have "contiguous" working memory as required by the wikipedia definition. It had what would be more specifically called multiple variable sized segments which could be individually swapped. Again, the mac OS evolved from a single process model to muliprocess, in the same address space (look up mac switcher) and implemented "virtual memory" using a system without a MMU by swaping the allocated pieces of memory to disk if they weren't currently locked (in use) and reallocating the memory. Aka they had "virtual memory" in single fragmented address space.
The other example is people use "paging" to describe the act of swaping portions of the memory to disk, misunderstanding that paging is more about splitting an address space or segment up into fixed pieces for address translation to physical, and that disk swapping of pages isn't required for paging. Aka, your system is still "paging" if you disable swapping.
Even the term swapping is unclear because the need to differentiate between swaping pages, and swapping whole processes (or even segments) resulted in people avoided the term swapping to describe systems which were swapping pages instead of segments/regions/processes. These systems were generally called "demand paged" or something similar to indicate that they didn't need to swap a complete process or dataset (see DOSSHELL).
So, give the guy a break, in may ways he is just as correct, if not more so.
Uh. You do realize that block of ram are not written contiguously right? You won't find it any different on Linux or MacOS or any operating system for that matter. You also realize that the access time of RAM is effectively 0 right? Yeah, the AC was right. Nothing in the KB article about ram fragmentation. That program is also one of those create "free" ram programs that I despise so much. These kinds of utilities might be somewhat marginally useful on a very resource bound system, but I can hardly see the use for this crap. Even if RAM were to be somehow "defragmented" how could it possibly make it any faster? The bottleneck isn't in accessing the addresses. An OS keeps a running tab of what is stored where. As soon as it makes the request for the data its coming off of the RAM as fast as the FSB will let it pass through. The reason defragmenting is effective on hard drives is because the hard drive has a physical dimension where the heads take actual time to move to the desired location. In RAM there is no moving parts and hence, extremely low latency, which is measured in nanoseconds versus the milliseconds they use to measure latency in hard drives.
I smell snake oil here. That is, unless you have some real science to back up the benefits of ram "defragmenting"
zosxavius photography
There was never any implication that MS has anything to do with the disk drive business; SpaceLifeForm said sales of Windows would be helped.
If the drive fails in 2 years instead of 5, the owner is likely going to go out and buy a new PC three years earlier than they need to, instead of getting the drive replaced; this generally means another sale of Windows.
If you have caches of a size smaller than your real RAM, the order in which you try to access memory really CAN make a difference because cache is many times faster than regular RAM and will try and do things like speculative readahead. If what you are working with is already in the cache by the time you request it then you won't stall for as long.
If you are forever causing the cache to become flushed and forcing the cache to be refilled with a different contents (perhaps because you are causing a large number of random memory access and the cache's readahead is getting your future access wrong so it has to be turned off) then performance will by comparison be slower than a sequential memory access workload.
The above is of course a gross simplification (and doesn't apply if what you are reading fits entirely within cache and is already there). If you have the technical chops you can read more about how order of access can have an impact on speed in Ulrich Drepper's what every programmer should know about memory on LWN.
HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management
DisablePagingExecutive
0 = Drivers and system code can be paged to disk as needed.
1 = Drivers and system code must remain in physical memory.
Does this happen in USA a lot? if a light in the fridge goes out, do you buy a new one? when a tire is blown out, do you buy a new car?
No, but it's a little different with electronics in general. First, assume here that we're talking about out-of-warranty items so that the owner is responsible for costs. Each town used to have several TV repair shops, but it came to be that it wasn't any cheaper to fix one than to replace it. The same with a clock radio; while you might be able to find someone qualified to troubleshoot and fix it, it'd probably be cheaper just to buy a new one. Well, a lot of people lump computers into the same category. If the hard drive (or CPU or RAM or video card) fails, then they figure it might be cheaper to buy a new one than to replace the bad parts.
Honestly, they're probably right. Suppose you're Joe Sixpack with a busted Dell and take it to Best Buy so their experts [1] can check it out. They quote you $147 for a new 60GB hard drive [2] plus $75 in labor. You're looking at $200+ to fix a two-year-old PC. Being the frugal type, you check out dell.com and see that you can buy a brand new one for $279 that's faster, has more storage, and has that Mojave thing so you can view photos. I won't really hold it against you for spending an extra $50 to get a new, better computer with a full warranty [3].
[1] Work with me here. ...which will run out the week before the embedded graphics chipset overheats.
[2] You could get a 750GB drive for the same price, but your computer was "designed for a 60GB drive", and they're hard to get now. Luckily for you, they were able to find one in the warehouse.
[3]
Dewey, what part of this looks like authorities should be involved?