86% of Windows 7 PCs Maxing Out Memory

CWmike writes "Citing data from Devil Mountain Software's community-based Exo.performance.network (XPnet), Craig Barth, the company's chief technology officer, said that new metrics reveal an unsettling trend. On average, 86% of Windows 7 machines in the XPnet pool are regularly consuming 90%-95% of their available RAM, resulting in slow-downs as the systems were forced to increasingly turn to disk-based virtual memory to handle tasks. The 86% mark for Windows 7 is more than twice the average number of Windows XP machines that run at the memory 'saturation' point, and this comes despite more RAM being available on most Windows 7 machines. 'This is alarming,' Barth said of Windows 7 machines' resource consumption. 'For the OS to be pushing the hardware limits this quickly is amazing. Windows 7 is not the lean, mean version of Vista that you may think it is.'"

Comments

When do people get this

[sopssa]

RAM is wasted when it isn't in use. The fact that the task manager in Windows says your RAM is used 95% tells nothing, and no it won't "result in slow-downs as the systems were forced to increasingly turn to disk-based virtual memory to handle tasks". I'm actually really surprised, and not in a good way, that "chief technology officer" of the company doesn't know this.

The new memory models in recent OS's try to utilize all the available RAM (as they should) to speed up things otherwise. It makes a lot of sense to cache things from hard-drive in low-peak usage points, and in such such way that it doesn't interfere with other perfomance. When the things that are most often used are already cached in RAM, their loading works a lot faster. This doesn't include only files, icons or such, but everything the OS could use or do that takes time.

If theres a sudden need for more RAM, the cached data can be "dropped" in no time. It doesn't matter if it averages at 25% or 95%, just that the perfomance overally is better when you utilize all the resources you can to speed up things in general.

Re:When do people get this

[Mr Thinly Sliced]

My understanding was that memory used for disk caching doesn't show up in task manager as "used".

It's been a while since I booted win7 though, so I might be mistaken.

Certainly under linux ram used as disk cache is marked "free".

It wouldn't surprise me that win7 has a heavier memory footprint though - as more applications move to .net and web browsers use lots of flash / silverlight etc - all of these things have a RAM cost.

Re:When do people get this

[dr.newton]

If all that RAM was simply being used for a filesystem cache, the system would not have to "increasingly turn to disk-based virtual memory to handle tasks" - it would just drop some cache when it needed to start a new task, as you said.

It seems that something else is going on.

Re:When do people get this

[Bazer]

I don't know how caching works in W7 but on Linux, if the system has to "turn to disk-based virtual memory to handle tasks" then the memory utilization isn't caused by buffers because buffers are never swapped out to disk. If W7 behaves in a similar manner the it's either a memory leak, system bloat or the caching mechanism is broken.

Re:When do people get this

[eldavojohn]

The new memory models in recent OS's try to utilize all the available RAM (as they should) to speed up things otherwise. It makes a lot of sense to cache things from hard-drive in low-peak usage points, and in such such way that it doesn't interfere with other perfomance. When the things that are most often used are already cached in RAM, their loading works a lot faster. This doesn't include only files, icons or such, but everything the OS could use or do that takes time.

If theres a sudden need for more RAM, the cached data can be "dropped" in no time. It doesn't matter if it averages at 25% or 95%, just that the perfomance overally is better when you utilize all the resources you can to speed up things in general.

Assuming your claims of how Windows 7 is implemented are true, then the claims from the person who actually collected all the empirical evidence must be false:

resulting in slow-downs as the systems were forced to increasingly turn to disk-based virtual memory to handle tasks.

If the memory was freed up dynamically as needed then no processes would ever be forced to resort to disk-based virtual memory. So either you work at Microsoft and are assuring us that the implementation protects against this or you're speculating against someone who has claimed to gathered a large enough to make such accusations.

No offense but I'm going to side with the guy who appears to make his living testing these sorts of things ... the guy who is offering me numbers.

Re:When do people get this

[Rockoon]

According to their website blog, they are "rethinking windows performance."

So instead of thinking about what actually effects OS performance, they are rethinking things so that they don't have to sell real solutions to their customers, "where [they] maintain several large installations of our commercial DMS Clarity Suite performance analysis solution."

Re:When do people get this

[Anonymous Coward]

I think the issue here is that the system is turning to swap. Caching stuff that may be referenced again is fine and dandy, but if the system regularly turns to swap just to keep itself afloat, then you have a problem.

Re:When do people get this

[dr.newton]

From TFA:

"On average, 86% of Windows 7 machines in the XPnet pool are regularly consuming 90%-95% of their available RAM, resulting in slow-downs as the systems were forced to increasingly turn to disk-based virtual memory to handle tasks."

Re:When do people get this

[jibjibjib]

Using more RAM doesn't use more energy. Either your RAM is powered on, or it's not. And if it's powered on it maintains its contents, no matter whether the OS has actually written anything useful to it.

Re:When do people get this

[Sockatume]

If they'd measured pagefaults, they could've reported pagefaults. They didn't. RAM usage appears to be the total basis for the article, so his concern is a genuine one. We don't know enough about the study at this stage to dismiss it.

Re:When do people get this

[snemarch]

It shows up as part of the memory commit bar - which is what regular users will look at, and then go off screaming about "OMG IT USES ALL MY SYSTEM MEMORY!1!!! one one". It's also deducted from the "free" count, since technically it isn't free (it can be freed quickly, but has to be zeroed out before it can be handed off to a new app - security and all).

The Win7 task manager does show a "cached" stat, though, so your effectively free memory is "free"+"cached". And if you want more comprehensive memory stats, you should look at perfmon.msc or SysInternals' Process Explorer.

I wonder if TFA has actually measured that disk swapping happens (easy with procexp or perfmon), or are just shouting their heads off without understanding what's going on... it's well-known that SuperFetch utilizes otherwise unused memory for disk caching, and does so proactively :)

Re:When do people get this

[jimicus]

Here we see why /. needs a "-1, Wrong" mod.

Re:When do people get this

[Mr Thinly Sliced]

Ahh fair enough. "Colour me learned something today". :-)

Re:When do people get this

[phatcabbage]

Barth acknowledged that XPnet's data couldn't determine whether the memory usage was by the operating system itself, or an increased number of applications.
So yeah, it doesn't seem like the author really knows what's going on...

Re:When do people get this

[dhavleak]

If the memory was freed up dynamically as needed then no processes would ever be forced to resort to disk-based virtual memory.

The trouble is, the TFA doesn't actually say (at least not clearly) that the Win7 machines are indeed turning to swap more regularly. It just states that fetching stuff from the swap file is a consequence of running out of RAM and causes perf degradation. So if the Win7 machines are indeed utilizing all available RAM and yet not swapping at a significanly higher rate, it means they're making more optimum use of available RAM.

Re:When do people get this

[dunezone]

Hi there.

Ive been running Windows 7 since the BETA release. I have never experienced any issues that result in I/O thrashing against the hard drive as result of all my ram being utilized. I also have numerous friends running Windows 7 none have reported any issues like this, if anything its been praise for the operating system. So I am amazed to see a 86% number being thrown out there yet never seen this problem before.

No offense but I'm going to side with the guy who appears to make his living testing these sorts of things ... the guy who is offering me numbers.

And now lets quote something from the article...

Barth acknowledged that XPnet's data couldn't determine whether the memory usage was by the operating system itself, or an increased number of applications, but said that Devil Mountain would start working on finding which is the dominant factor in increased memory use.

This single sentence makes the article rubbish. They have no clue whats causing the heavy memory usage, its just an assumption that the OS is causing it and they're yelling fire before looking through all the data or completely analyzing the problem.

Re:When do people get this

[nmg196]

Totally agree. If you don't want Windows 7 to use the 4GB of RAM you've paid for to speed up your computer, take out 2GB and put it in the drawer. Otherwise, be thankful that it's actually making the most of the RAM you're using.

What next? People complaining that games use 100% CPU to give them maximum framerate when it could just use 30% CPU and give them 10 FPS?

Re:When do people get this

[FlyingBishop]

You'll excuse my ignorance, but from college I remember that usually you have 0-2V represent 0 and 3-5V represent 1. Does a 0 have a corresponding increase in amperage so that it levels out and uses the same amount of power?

It seems natural to me that it would be initialized with zeroes on power-up, so that it would minimize power consumption.

Furthermore, more advanced chips, especially in mobile devices, have a variety of power-saving tricks. I would expect RAM would be no exception in having ways to clock down in line with requirements.

Re:When do people get this

[jernejk]

I used Ubuntu for almost a year and I think linux cachnig / virtual memory is implemented better than win7. It seems win7 cache is too aggressive and it dumps active programs from RAM to page files when it should not. Maybe it works OK for most desktop users, but it doesn't work very well for a development machine. I have nothing but my subjective feeling to backup my observations.

Re:When do people get this

[Mr Thinly Sliced]

If the RAM is filled with a lot of crap, such that the OS must keep loading of the HDD to get information, that slooooows system performance not speed it up.

Actually in modern operating systems RAM can be used as a disk cache is such a way as it can be "freed" at little to no cost when needed by programs. This in effect means you get quick access to often used programs as they are already cached in RAM.

The only situation in which "RAM is filled with crap and the OS goes to disk" (paraphrased) is when RAM is full - with programs. In this situation yes, it has to go to disk.

Re:When do people get this

[snemarch]

Yep, that would be a problem - but neither the TFA nor xpnet mentions if this is actually happening, it seems that they're looking almost exclusively at "free physical memory", which isn't a useful stat in this regard. The xpnet site does say they factor in "how often it relies on virtual memory", but not how they do this (there's multiple metrics to choose from, some fairly uninteresting) and the fact that they seem to factor this in as a part of "memory usage" rather than keeping it as a separate stat makes me pretty wary of trusting any analysis from them.

Re:When do people get this

[snemarch]

You're making an apples-to-oranges comparison, and you don't mention what software is running on either of your machines...

I've found Vista and Win7 to generally work better, giving decent enough hardware. Because of SuperFetch, Visual Studio loads faster on my Vista64 dualcore laptop with 7200rpm drive and 2 gigs of memory than on XP64 quadcore workstation with 10000rpm raptor drive and 8 gigs of memory. But this isn't a fair comparison either, since the rest of the software suite on those two machines are specced differently.

Re:When do people get this

[lcarnevale]

HDD caching and swap are two completely different things. HDD caching is loading things from the disk to RAM to speed up things. SWAP is using the HDD as extra RAM when the system doesn't have any more memory left to use. So, what I think you wanted to do was to turn off swap, not hdd caching.

Re:When do people get this

[FooBarWidget]

Definitely the part about HDD caching slowing things down. Even in the DOS age it was well known that hdd caching utilities (I forgot the names, too long ago) improve disk performance tremendously.

Linux does the same things as Windows: it caches as much stuff from disk into main memory as possible. Try running:

    cat large_video_file.avi > /dev/null

You'll see that after running the command, your memory usage jumps up by the size of the video file. Now try running the same command again, it's now an order of a magnitude faster.

On Linux things like this are stored in main memory in the form of caches and buffers. I don't know about Windows, but Linux clears some caches and buffers if applications need real memory. Caches and buffers show up in memory usage reporting tools like 'free', so it's quite normal to see Linux systems using 90% or more RAM, most of which go to caches and buffers. It seems that most people who complain about memory usage don't know how memory is managed on modern operating systems, so they go all apeshit about "OMG HELP linux is using so much memory it sux0rz!!!" and I have to explain again and again how they're not getting it. Same goes to you. Now, Windows is suffering from the same problem.

FYI, here's the memory usage of my Linux server:

             total       used       free     shared    buffers     cached
Mem:           720        702         17          0         55        510
-/+ buffers/cache:        136        583
Swap:          399          0        399

It says 702 MB of used memory. Now look at "-/+ buffers/cache", it says 136 MB. That's the amount of memory *actually* used by applications.

Re:When do people get this

[cgenman]

Read TFA. It just claims that Windows 7 consumes all available RAM. That is the "empirical evidence." System slowdown was NOT measured.

Utilizing all available RAM is a pretty well understood technique at this point. All web browsers do this now, as do many other applications. One would expect a well-designed modern OS to do this. Consuming all memory itself is not a sign of poor programming itself, so long as disk caching of things that should be in RAM doesn't occur. This is not something that the people in the article has measured.

I'm going to side with the guy who appears to make his living testing these sorts of things.

Bad science is bad irrespective of the person conducting it. And whatever the original tester said is getting filtered through the viewpoint of the gentleman writing the article. Considering that he says that "Windows 7 is not the lean, mean version of Vista that you may think it is," yet never once compares statistics to Vista (or even mentions it outside of this statement), I'd take the conclusions from these stats with a grain of salt.

Re:When do people get this

[Moryath]

You obviously don't understand memory access design. It's all about feeding the CPU. There are two sorts of relationships we can use to make this work: temporal and sequential.

Hard drives are the largest-capacity storage (well unless you want to go to tape). But they're slow. Even the fastest high-RPM SCSI or SATA drives are SLOW compared to what's above them. This is mitigated, somewhat, by putting some cache memory on the drive's controller board itself. Still, having to "hit" the hard drive for information is, as you say, a slowdown. Same goes for "external" storage (Optical media, USB media, etc).

So you try to keep as much information as possible in RAM (next step up). Hitting RAM is less expensive than hitting the H/D in terms of a performance hit. In the original days of computing (up until the 486DX line for Intel CPUs), RAM and CPU operated on a 1:1 clock speed match, so that was that.

Once you factor in the "clock multiplier" of later CPU's, even the fastest RAM available today can't keep from "starving" the CPU. So we add in cache - L3, L2, and L1. the 486 implemented 8KB (yeah a whole 8K, wow!) in order to keep itself from starving. L3 is the "slowest", but largest, L2 is faster still but smaller, and L1's the smallest of all, but the fastest because it is literally on the same die as the CPU. That distinction is important, and in general you'll find that a "slower" CPU with more L1 Cache will benchmark better than a "faster" CPU with less.

The CPU looks for what it wants as follows:
- I want something. Is it in L1? Nope.
- Is it in L2? Nope.
- Is it in L3? Nope.
- Is it in RAM? Nope.
- Is it in the H/D Cache? (helps avoid spin-up and seek times) Nope.
- Crap, it's on the H/D. Big performance hit.

Everything except for the L1 check, technically, was a performance it. The reason for pre-caching things (based on temporal and sequential relationships) is all about predicting and getting what will be needed next into the fastest available place.

Yes, I suppose you can run an entire system where it all goes into "RAM", and you'll see it as "more responsive" simply because you never have to touch the hard drive. But turning off HDD caching is a BAD idea. It makes cache misses that much more expensive because then, instead of having even the chance of finding what you needed in RAM or in the HD's onboard cache, you have to wait for the H/D to spin up and seek to the right sector.

Re:When do people get this

[Barny]

To be fair, windows 7 will swap data out, my PC atm is sitting here with 8GB of ram, 1.1GB used by programs, 5.9GB used by cache and its reporting 1.2GB free, so it still pages out data. However I have never noticed it pageing, so likely its paging out the "right data", in other words stuff that is not used, just how an OS should work :)

Re:When do people get this

[jimicus]

Maybe the part about HDD caching slowing things down?

I could be wrong there, since I'm not an expert but I remember the dark, dark days when my computer when spend 2-3 minutes just to redraw a Word document. Why? Because it was using the HDD like memory, instead of using the actual memory. It seems to me that this problem, while minimized, has never completely gone away.

Anyway telling me "you're wrong" doesn't enlighten either me, or the other readers. Please elucidate.

There's a lot of variables, but in simple terms the theory goes that something which you have recently accessed (be it an application, a document or whatever) you are likely to want again in the near future. Hence it's worth keeping a copy in memory on the offchance.

On the other hand, you really don't want to be swapping. So if a program needs more physical memory than what you have immediately available, it makes more sense to allocate memory which was recently holding cached data and just reduce the cache size than it does to start swapping, which is what any sane OS will do.

If there's any real intelligence involved in this, the OS will re-allocate an area which hasn't been used in a while.

The cache would only cause a problem in the way you describe it if the OS did not dynamically resize cache to account for other demands on system RAM.

I can't explain the differences between yours and your brother's computer but I can tell you that OEM builds of Windows tend to have so much garbage loaded at boot that they often need serious work before they're genuinely usable. Some of the builds I've seen, it is a wonder they boot at all.

Re:When do people get this

[GordonBX]

Which part is wrong?

Your old P4 is a minimum 3 - 4 times slower at raw CPU tasks than the Athlon X2 (http://techreport.com/articles.x/18448/13). The story about your 512Mb XP machine running faster than the Athlon X2 3Gb machine simply does not ring true. Perhaps you were only playing solitaire on it? No-one is denying that Puppy Linux is fast, or small, or that it fits in memory. Try running Open Office on it with only 30Mb of Ram and then tell me that it is fast.

Maybe the part about HDD caching slowing things down?

YES the part about HDD cacheing slowing things down. HDD cacheing speed things up! Memory Swapping slows things down - which is simply a question of how big your application + operating system's "working set" of memory is (i.e. the memory that simply has to be held in Ram because it is being actively read and written). 3Gb with Windows 7 is plenty for "normal" apps, and it won't be any slower than your 512Mb XP machine running the SAME applications - in fact it'll be a load faster because it won't have to swap as much and the disk is likely faster.

Re:When do people get this

[lagfest]

My current* windows 7 stats say:
Total: 2046MB
Used: 1.26GB
Cache: 634MB
Available: 743MB
Free: 132MB

So used RAM does not include cache. And 'available' is a nice way of telling grandma that RAM used for cache is actually available to apps.

* not a snapshot, i can't type that fast :)

Re:When do people get this

[snemarch]

actaully the windows 7 caching model is great. on games the difference between the first loading of a level and subsequent loads are night and day thanks to it's caching model.

That's the windows cache system generally, from way back in the NT days... Vista and later SuperFetch is more than that.

btw, regarding the article more directly: they shows no figure about the actual _swap_ usage, a thing that may or may not disprove their theory.

Indeed. The xpnet site does mention that they factor in paging somehow, but that's still pretty useless - paging activity needs to be a separate statistic. Also, simply looking at pagefile usage isn't terribly useful, an inactive app can have it's working set trimmed and pages flushed out to disk, and this won't matter much in the big picture.

What you need to look at is the rate of pagefile activity (ie., pages/second) as well as how often it happens - not just static numbers (even if having 1gig of data in the pf is probably a warning sign :))

Re:When do people get this

[lordlod]

You'll excuse my ignorance, but from college I remember that usually you have 0-2V represent 0 and 3-5V represent 1. Does a 0 have a corresponding increase in amperage so that it levels out and uses the same amount of power?

It seems you must have missed the complex electronics portion of your college.

5V TTL circuits use 0-0.8V Low and 2.2-5V High (on input), in between the high and low states is undefined. Regardless, modern RAM is almost certainly a 1.8V device externally and internally even less.

Modern RAM (DRAM) works by each bit of memory being a floating charge in a capacitor. When you read the bit the charge is released and read as either a one or zero. I wouldn't want to make assumptions about if a high voltage corresponded to a one or zero, they would choose whichever they felt worked best. This also includes whatever state they initialise the RAM to on power on.

In the ideal no-friction world, floating charge = no current = no power. In our world floating charges leak slowly and have to be topped up, so there is a degree of power being used depending on the high or low state of a given bit. That said, the refresh rate probably has more impact than the value.

In all honesty though, you are barking up the wrong tree. A far greater power issue in the modern computer is the increased power required by all the high speed external connections. Transmission line theory means that as the speeds of links like ethernet have increased the power required to shovel the bits down the line becomes exponentially linked to the speed of transmission. (EMF also becomes a serious issue: Fun game, wrap a GPS antenna in a ethernet cable then plug it in.) So to really save power you should start by unplugging your Gigabit link and hooking up some environmentally friendly 10BASE2 goodness.

Re:When do people get this

[Shakrai]

I think the issue here is that the system is turning to swap.

Turn swapping off. If you have a reasonable amount of RAM there's no reason to leave it turned on. I turn it off anyway for security considerations -- I use Truecrypt -- but really there's no reason not to do so on any XP machine with 2 gigs of RAM or Vista/Win 7 machine with 4 gigs of RAM.

Re:When do people get this

[Z00L00K]

Just because RAM is available doesn't mean that the OS should hog it. You may want to use that RAM for something different. It may be legal to use "excess" RAM for buffers, but then those buffers must be freed fast whenever necessary.

If you use large amounts of RAM for buffers you will either freeing the least used buffers and use them for the application, and then you will get memory fragmentation. This can be bad for some applications. The other scenario is that you will just kill a block of buffers and that may happen to be buffers that are heavily trafficked and has to be re-loaded somewhere else - and you have a performance penalty.

And caching a lot means more caching overhead. Not everything makes sense to cache. What about the case when you run a database, caching the database file through the OS first and then have the database engine to cache the same thing again? A complete waste of performance and resources.

So having the OS caching - it works fine for many applications, but not for every application. Some applications are also tuning themselves by looking at available memory and determining how to best allocate resources. It is of course possible to figure out how much the OS can free for application use, but it's also hard to calculate an usage budget. And if it goes wrong you will end up swapping to disk.

Re:When do people get this

[theaveng]

>>>You're making an apples-to-oranges comparison, and you don't mention what software is running on either of your machines...

Nothing exotic. The Windows OS plus Firefox browser. My P4-XP-512MB machine runs faster than my brother's AMD X2-WIN7-3GB machine. XP is more responsive.
.

>>>I've found Vista and Win7 to generally work better, giving decent enough hardware.

Well my Vista install was running Microsoft's minimum recommendation (512 megabytes) and ran like a snail through molasses. When I later upgraded it to 2 gig it did run better, but still nowhere as fast as my old 0.5 gig XP machine, which is why I eventually went back to XP.

My overall impression is that Vista is like a Beetle with a lot of chrome/pinstriping/et cetera. Looks good. Runs like crap.

Re:When do people get this

[spxero]

...or the caching mechanism is broken.

I'm inclined to think it's this, at least for my Vista machine. I currently have 6GB RAM, but at any given time with Outlook, FireFox, and a handful of Explorer windows open there isn't any more than 2-3GB showing to be in use. The rest is cached. This becomes a problem only when I need to fire up a 2GB Linux VM for testing, the VM will pause itself on startup, citing not enough RAM available. I'm no expert, but I have a sneaking suspicion that the caching mechanism is the culprit.

Re:When do people get this

[drinkypoo]

And just to swim in anecdotal waters, when I copied big piles of files to my GRiDPad 1910 via null modem cable using Microsoft's classic INTERLNK and INTERSVR for file sharing, SMARTDRV sped up the copy operations by about an order of magnitude. Life without disk caching isn't worth living.

Re:When do people get this

[Lumpy]

but really there's no reason not to do so on any XP machine with 2 gigs of RAM or Vista/Win 7 machine with 4 gigs of RAM.

Yes if you do the typical office no taxing tasks on a pc. If you do anything big it's no where near enough.

If you edit HD Video or very large Photo arrays you bump up against 4gig without effort. I hit the 16 gig mark on a regular basis.

Re:When do people get this

[afidel]

Actually what you really need to do is calculate hard pages/second which for some retarded reason isn't available as a default stat counter, ie those pages which actually go to the secondary backing store (disk).

Re:When do people get this

[sopssa]

If the system is gratuitously using 95% of the RAM nearly all the time, then it’s a completely different scenario. Everything I try to open that wasn’t cached already will force the system to dump some memory to the swap file to make room for the new application.

Uh no. The point here is that the RAM is utilized with data that speeds up things, but that can be instantly freed if needed. It doesn't need to put that in swap file.

Re:When do people get this

[afidel]

Windows gets really cranky when it doesn't have a pagefile. We tried it for performance reasons and we saw an almost 40% drop in performance despite the server not being under any kind of memory pressure.

Re:When do people get this

[Locklin]

The xpnet site does say they factor in "how often it relies on virtual memory", but not how they do this

Even that isn't ideal. It makes sense to swap out some library that hasn't been used in hours or days. The more ram available for disk cache the better. The only solution is to look at Memory used minus disk cache (like the 'free' command on Linux does).

Re:When do people get this

[TheLink]

Yeah I don't know why they don't set up the counter by default.

Anyway to set it up yourself:

Start perfmon.msc
Then add counters
go to Memory, add "Pages Output/sec".

I'm not an authority on virtual memory but from what I know:
Page Faults/sec is not usually relevant for this - the virtual memory stuff will have page fault even if it's not swapping to/from disk - it's part of how virtual memory works.
Page Inputs/sec could happen when you launch programs (then the O/S starts paging in the stuff it needs) - it's no indication of running out of memory.
Page Output/sec on the other hand is when the O/S is low and needs to copy stuff in RAM and write it OUT to disk so that it can reuse that RAM for something else. This is the one you want to monitor.

Re:When do people get this

[xigxag]

You missed the most important emphasis:

delays in I/O processing - ostensibly due to heavy virtual memory activity as Windows compensates for insufficient RAM. (emphasis mine)

Ostensibly means "to all outward appearances." In other words, they're admitting they don't really know the inner workings, the true cause of the delays. They're just supposing it's due to RAM swapping, as opposed to increased networking activity, aero glass, more concurrent programs being run on average, or any other number of other wag'd reasons. Basically, they picked two measurements that are both higher in W7, and just said, "Well, it stands to reason that A causes B." What's that phrase that internet smarty-pantses use all the time about this?

Re:When do people get this

[growse]

I currently have 81 processes running on my Win7 install. Each one of them can address 2GB (I think) virtual memory. If you turn off the page file and a few processes decide to make full use of their addressable space, then everything will stop working. That's why a page file is a good idea.

Re:When do people get this

[Eivind]

Actually, even that is inaccurate.

You see, it can make sense for the OS to swap out some not-recently-used pages of a program, to free up more memory for caching. For example. Say you're playing a game, but you've got firefox open. It could make sense to page out the entirety of firefox, so as to have more physical ram free for caching of game-content.

Life ain't so simple in a virtual world :-)

Re:When do people get this

[CrazedSanity]

The 'anecdote' to which you refer is completely plausible. Remember that the info wasn't based on any sort of raw processing power, frames per second, or disk throughput: it was based upon human perception. Don't ever throw the two together and think that they will always (or even mostly) match up.

I can tell you that a computer that has 4x the specs of another doesn't necessarily end up running better, regardless of the (Windows) OS. This can have a lot to do with hard drive fragmentation: one system that has low specs but is defragmented routinely can appear to run faster than a high-end machine that has 50% file fragmentation. I know it sounds like I'm picking on Windows, but the issue with data fragmentation--and the need to defragment--wholly resides on the shoulders of Windows users. And again, the speed difference, especially when it comes to perception, can be hugely affected by how much the hard drive is thrashing when a program is started.

Another issue in perception is with video performance. Two identical machines with the same model of high-end video card can have completely different apparent speeds if one has problematic drivers. A number of other things might be measured. I remember back when a person would say that their high-end, I-just-spent-more-on-my-computer-than-my-car computer was running slow simply because their dial-up connection was slow: the computer was running perfectly, but the user perceived it to be an issue with the computer (instead of the slow connection) because the data wasn't loading fast.

Re:When do people get this

[Bakkster]

But more page-faults doesn't always correlate to more slowdowns. An OS with better page-allocation prediction will run faster (from the user's perspective) with the same number of page-faults. It's only a problem if the page-faults are on cached data that the user is requesting at that moment.

Continuing the Firefox example: it might be one page of memory to each page you want to view. A smart OS will leave the pages with the main Firefox program and current tab in RAM and cache the others first. Then when tasks switch, the cached Firefox pages are reloaded while the user is still looking at the first page. There are page faults, but the user experiences fewer delays.

Basically, there is no meaningful conclusion we can draw just from peak RAM utilization and page fault numbers (either average or peak). To do that, we would need to measure the number of page faults for pages that were already in memory but cached to disk and that required the user to wait before continuing.

More importantly, to claim that Windows 7 is 'bloat' just because it uses more RAM and has more page faults is erroneous without additional evidence.

Re:When do people get this

[TheLink]

If stuff slows down due to that swap out, then it's still accurate enough for me.

Maybe the O/S could get it right and swap out and swap in Firefox in a way so I won't notice any slow downs. Give me an example of such an O/S please.

So far in my experience, if Windows or Linux swaps out Firefox for whatever reason, if I then switch to Firefox, I have to wait for it to be swapped back in.

Why "page out" and not "page in"?

"Page in" doesn't necessarily mean that I'll have to wait if I switch to different programs- the O/S is bringing stuff from disk to ram - I believe in some cases the O/S pages in stuff as part of running a new program - so it's not such a useful metric for "not enough memory".

But "page out" means something in RAM is going to disk - if I ever want it back in RAM, I'll have to wait.

If stuff in RAM is going to disk needlessly and causing unnecessary waits then the O/S virtual memory algorithm is getting things wrong.

Re:When do people get this

[Foofoobar]

First I said ANY GIVEN RESOURCE... not just RAM. Second, disk cache is not as fast as RAM and should not be relied upon like RAM. It is merely a safety net if you run out of RAM... like when you get a usage spike.

Re:When do people get this

[ShadowRangerRIT]

If you have a reasonable amount of RAM there's no reason to leave it turned on.

If the swapping algorithm is so bad that it swaps unnecessarily, then yes, turning off swap will help. But a good swapping algorithm remains useful even if you have 16 GB of RAM. Large sections of many processes are basically "run once, then ignore" or even "never run". Most processes have a decent amount of startup code that is never referenced after the first half second of execution, or load multi-megabyte shared libraries into process memory space to get two short functions (or, similarly, contain code that only runs under exceptional circumstances). If you disable swap, you're denying access to that memory not only to other programs (which we'll assume for the sake of argument don't run out of RAM themselves, since you use the phrase "reasonable amount of RAM"), but for I/O caching.

If you've got a program regularly crawling part of your directory structure, or you're writing frequently to files, the RAM freed by swapping out the junk parts of each program could be used for a productive purpose. Delaying the write means you can write a contiguous block all at once, allowing higher priority I/O to go through without forcing the low priority I/O program to block, and it can also mean reduced fragmentation on the disk.

Similarly, a predictive caching algorithm can improve responsiveness with that otherwise wasted RAM you've decided *must* be kept in memory. If you always start a program around 6 PM when you get home, the system can recognize this from the metrics and preload it. If you don't run it, oh well, the RAM was used just as effectively as if it held unused, unswappable contents. If you do use it, your program starts nigh instantaneously. If the program itself has a specific performance profile, where specific data files are predictably read, the computer can cache them using the RAM freed by swap, reducing loading times and increasing responsiveness. That might seem wasteful (spinning up the hard disk when it isn't needed), but in other cases it saves energy; if you're seeding a torrent, and the computer has enough memory, it may cache the whole file in memory; voila, no matter what piece a client asks for, the disk doesn't need to spin up.

That said, no swap algorithm is perfect. And if you've got 4+ GB of RAM and all you're doing is running a browser, an office suite and maybe a game, the difference will be small (or non-existent if all your programs and all reasonable file system caching can fit in memory). This doesn't mean swap is useless; all it means is that it's not perfect.

Re:When do people get this

[ShadowRangerRIT]

One other minor note: Windows's use of pre-emptive paging makes for a much faster hybrid sleep and/or hibernation. If your page file is larger than main memory, and you're not paging excessively, most of your memory is probably already paged out. Thus, the hibernate file only needs to have the unique data written to it; on a laptop with 4 GB of mostly used RAM and a relatively slow hard disk, it could take two minutes to hibernate the machine (hope your battery lasts). Every bit of memory paged out preemptively means less time to hibernate. My home machine is set up for hybrid sleep, and has 4 GB of RAM. Time from issuing the sleep command to hibernation is about 3-5 seconds, and that only works because of the page file.

Re:When do people get this

[snemarch]

Things [i]have[/i] changed since XP :) - by default it's cache system is pretty conservative. You can get around that by setting LargeSystemCache=1 (except if you're using ATI drivers - nastier-than-normal BSODs galore!). Then 32bit and the kernel/user address space split becomes the limiting factor.

With more physical RAM, a larger address-space, the cache has more room to play - and it doesn't hurt that Win7 (like Vista) is less conservative than XP was :)

Re:When do people get this

[Daengbo]

You mean your machine with 8GB RAM never hits swap? Wow. Shock. Color me surprised! O_o

Re:When do people get this

[Foolhardy]

But "page out" means something in RAM is going to disk - if I ever want it back in RAM, I'll have to wait.

On Windows it doesn't necessarily mean that. Writing a page to disk != needing to read it back from disk later.

Each process has a working set. Pages in the working set are mapped actively into the process's VM with page tables. The memory manager aggressively trims these pages from the working set and puts them into standby memory. A page in standby is not mapped for reading (and more importantly for writing) anywhere in the system. Part of putting the page into standby involves writing a copy to disk. This will show up as a page written.

From standby, the page can be used one of two ways:

1. Transitioned back. If one of the processes that originally had the page mapped touches the page, it will cause a soft page fault in which the page is simply put back in the process's page directory. There's no need to retrieve it from disk since it still has the same data from before. The disk copy is discarded. This will show up as a transition fault in the performance monitor.
2. Reused for something else. Standby pages are counted as "Available" because they can be immediately re-used for another purpose without accessing the disk. The memory copy of the page is discarded and the page is re-used for something else. No disk activity is needed at this time since there is already a copy on disk. When one of the original owners of the page want the data back and the page is no longer on standby, it has to be retrieved from disk. This will count as a page fault in the performance monitor.

The nice thing about this model is that disk activity isn't needed to either reuse pages or bring them back at the time of the demand. It helps avoid the ugly condition of paging one process out while paging another in at the same time, causing disk thrashing.

Since Vista, the memory manager will preemptively re load pages that have been bumped out of standby back into standby if there is free unused memory available. Also since Vista, each page of memory has a priority from 0-7 that determines which pages are preferred to keep in RAM. In all versions of NT based Windows, memory mapping is very similar to page file management and will use many of the same counters (including standby memory, transition and hard faults, pages in/out). Memory mapping is used by lots of components internally and for loading executable images and libraries. Also, file caching is logically based in many ways on memory mapping, although the counters are different in many cases.

Re:When do people get this

[ztransform]

Windows gets really cranky when it doesn't have a pagefile. We tried it for performance reasons and we saw an almost 40% drop in performance despite the server not being under any kind of memory pressure.

And yet when I turn off swap on my 32-bit Vista laptop performance increases 1,000 - 10,000% easily. The difference between waiting 10 minutes for the computer to stop thrashing the swap file and near-instantaneous action is immeasurable.

Typical "don't turn off your pagefile" responses are fraught with lack of experience. There are times when a system performs better with a pagefile. There are many times when a system performs so much better without a pagefile that one wouldn't dream of ever turning it back on.

Re:When do people get this

[Blakey Rat]

Doesn't require advanced knowledge to make sense of that ouput,

I'd argue that knowledge of what "wired" means in that context is advanced knowledge, required to make sense of that output. I've been working with computers for a hell of a long time, and I have absolutely no clue what "wired" means in regards to allocation of memory.

I also don't have whatever knowledge (advanced or otherwise) to understand the difference between "buffer" and "cache." And I'm only guessing that "inact" means "inactive", although I don't really get the distinction between "inactive" and "free."

My first guess would be "wired" is total amount of RAM, but from the numbers that's clearly wrong. In fact, that report doesn't seem to tell you your total amount of RAM at all... which is less than useful.

Anyway, the point is, if you think that readout is obvious, you've completely lost contact with the general public. I hope to God you don't write UIs that you expect other people to use.

Depends on what kind of memory

[A beautiful mind]

If it is filesystem cache, then it's not wasted or "maxed out". If it is application/system memory, then it is indeed a problem.

Re:Depends on what kind of memory

[Bazer]

Would a filesystem cache cause the system to swap?

Re:Depends on what kind of memory

[A beautiful mind]

It's Windows. It might be a bug.

Bogus Story

[filesiteguy]

Let's start from the story (which I *did* read) - 'Barth acknowledged that XPnet's data couldn't determine whether the memory usage was by the operating system itself, or an increased number of applications,"'

Right there I'd be suspect whether this is even an issue or not. Given Windows (which I generally regard as inferior) as an OS having lots of functionality, I wouldn't be suprised if it takes up all available RAM prior to utilizing swap. I'm on my 2GB Ubuntu system right now and am running at 18% of 2GB with just Mozilla (with two tabs) and Thunderbird. But there's also my network layer (Network Monitor), KTorrent, and my bluetooth daemon running in the background. All told, System Monitor says i have 31 processes running.

Let's do a like comparison - run the exact number of apps and processes before declaring a memory leak.

Sheesh!

It's called SuperFetch

[dhavleak]

I guess Devil Mountain or whoever don't know about SuperFetch. Or need publicity.

And I guess slashdot editors don't know about SuperFetch. Or maybe an article like this gets them more traffic, revenue, etc.

The fucking bullshit that passes for articles these days..

Re:It's called SuperFetch

[mystikkman]

True, the technical details for interested geeks are here http://blogs.technet.com/askperf/archive/2007/03/29/windows-vista-superfetch-readyboost.aspx

Trollworthy rating for posting this...

[hitech69]

Computerworld should just close up shop for this worthless piece of journalism, or at least give their author the boot for doing any work with Craig Barth who represents a team of morons. samzenpus should be given a troll rating for getting this to Slashdot.

Page Faults

[Dr_Barnowl]

The metric to count is the number of page faults, an indicator of the number of times that the OS addresses memory that isn't in RAM.

As others point out, measuring just the fraction of memory consumption is stupid. I have 6GB of RAM ; my processes are using about 1.7GB of that, but the OS is claiming that 3.8GB is consumed. So that's 2.1GB of cached data that I no longer have to wait for from disk. Hooray.

TFA hints that they may be measuring page faults, and does mention that Win7 is hitting the disk for virtual memory more often. But they should make that clearer if it's the case.

People are buying cheap PC's and upgrading old one

[Nzimmer911]

"Current generation hardware"? Seriously, how many machines in this very small sample set are using i series intel chips? The way windows 7 was marketed, I'd bet that many of these machines were upgraded XP boxes. Top that with the 32 bit memory caps and people's general hesitation to install a 64 bit desktop OS, and I am not surprised at all that many machines are hitting memory saturation. Add to that that the Windows 7 interface leads to leaving more apps open at any given time than the XP interface...

Oh come on

[megla]

First we had submitters who didn't read the stories they were posting. Then we had editors who didn't read the stories they were approving. Now we have companies who don't read the articles they put out. Seriously, it's called a file cache. That's how it's supposed to work. Nice job, idiots.

Parent is +1 informative

[Anonymous Coward]

You cannot study virtual memory performance without considering how many page faults occur.

It is perfectly reasonable to use RAM as a filesystem cache, which is why Linux has adopted this approach. The effect is that almost all of the physical RAM is always in use. The cost is that pages are more likely to be wrongly swapped out - however, in typical cases, this increased cost is tiny in relation to the huge reduction in the number of disk accesses.

Comment removed

[account_deleted]

Comment removed based on user account deletion

In other words, 86% did not disable superfetch

[postmortem]

Which is left on by default.

And that "bad performance" simply meant that user opened app that was not already cached in RAM.

But Slashdot filter was pleased: if bad article about windows submitted, then publish it on front page

Re:Anti-Virus progs

[Hierophant7]

That makes sense. It also makes sense that roughly 86% of users would enable a full scan every day.

User base increases over time.

[tepples]

If you're seeing an actual slowdown in performance, fine, worry about it.

User base increases over time. Even on an intranet server, your company will probably add users when it grows. As your user base increases, you will see slowdowns. If you can catch slowdowns before they happen, you will be more prepared for the DDOS attack that comes when your site gets mentioned in a Slashdot article or when a bunch of new employees go through orientation.

100% CPU usage is a good thing: it means there's a process that's not IO bound.

Or it could mean that you need to optimize the process that uses the most CPU time so that it becomes I/O bound. All other things equal, once all your processes are network bound, you can serve more users.

Available memory != Free memory

[TheLink]

Yeah. I don't have low mem problems with Windows 7. There's stuff I don't like about Windows 7 but "memory hog" is not on the list.

For work I'm using Windows 7 64 bit on a 4GB notebook PC with tons of windows open e.g. a few Explorer windows open, a few Excel "windows"[1], a few Word windows, one Visio doc, Notepad++, Google Chrome, Firefox, putty, Outlook (a resource hog), Communicator, MSN Messenger windows, a Virtual Box Linux vm machine, Microsoft Security Essentials (it's my work PC so it's supposed to have AV) and it typically says 1700 to 2000MB _available_ (depending on how many firefox tabs, how many word docs and virtual machines etc). But overall no mem problem.

And guess which is using the most RAM? Not Virtual Box, not Word, outlook or Excel. It's Firefox with a 173MB working set and 142MB Private Working Set!

Yes it only has 500MB free memory, but so what? The O/S says there's 1700MB available. And so far I haven't had much slowdowns due to low memory issues.

To me the relevant metric for "low on memory" is "Pages Output/sec" (go launch perfmon.msc and add that counter). If that's a constant zero when you or the O/S switches from app to app, window to window, it means it's not swapping out. If it's not swapping out and not getting "out of memory" messages, it's not low in RAM no matter what some random "expert" thinks. And it's zero for me.

The equivalent in Linux for that is the swap "so" column when you run vmstat 1 (or vmstat 2). Same thing there - stuck at zero = not swapping.

I don't think my usage can be considered "light", as it is, what are those users running that's using up so much memory? Symantec or McAfee antivirus? ;).

FWIW, my laptop is not running any of the "OEM crapware" - I did a clean install of Windows 7 months ago when I got the laptop.

If that "expert CTO" can't even give an example of one memory hogging program (or show where Windows 7 itself is using so much memory that it's a problem), then it's likely he's full of crap.

Lastly, it's true my taskbar looks messy with two rows of task buttons, but I don't see the advantage of closing and reopening documents or programs if I'm not running out of RAM yet. I close them if I really do not need them (e.g. the document is out of date and not used for comparison). Otherwise it's much faster to just click a button to show the desired doc, rather than have to reopen it again from scratch (uses less battery power too - except in the case of MS Word which seems to use CPU even when "idle" - haven't figured that one out yet).

[1] By default Excel actually just has one window which changes to display the relevant document depending on which Excel taskbar button you click, whereas Word actually has separate windows for each doc.

Re:Available memory != Free memory

[TheLink]

If you are running the sidebar you may like to look at this:

http://seclists.org/bugtraq/2007/Sep/134

See the discussion and also the pdf http://www.portcullis-security.com/uplds/Next_Generation_malware.pdf

I'm sticking to perfmon.msc, task manager, resource manager and Process Explorer, depending on the circumstances.

Windows has a bigger swapPINESS

[tepples]

If the RAM is needed for another purpose the blocks can be freed by simply changing a flag and mapping them as quickly as if they had been totally free. This does not slow things down. After a while most of the blocks you need will be in RAM when you need them. Thus caching makes the system faster, not slower.

The above describes Linux. I assume Windows works similarly.

As I understand other comments to this article, Windows runs at the equivalent of 100% swappiness, making the system far more likely to evict a process from RAM than to evict things from disk cache. Writes are slower than reads, especially on a RAID or on a laptop with a low-cost SSD. Or to express it more graphically:

8==D : Swappiness on Linux properly tuned for slow writes
8=========D : Swappiness on a Windows box

Comment removed

[account_deleted]

Comment removed based on user account deletion

No kidding.

[endus]

Installed x64 on my 4gb machine and the performance was just ridiculously bad. I am a photographer and do a lot of image editing...couldn't even keep Cap One, Photoshop and iTunes open at the same time...especially if 7 was trying to thumbnail images in a folder (thumbnailing is broken and ridiculously resource intensive despite Microsoft's claims that its more robust in this OS). Noticed that it was swapping to the disk like crazy and ordered another 4gb. Definitely much better now, though I think 12gb would not be totally out of line for x64 with heavy applications. I haven't even TRIED to edit HD video yet...that is not a prospect I am looking forward to.

I pity people running the x86 version of the OS that are maxing out at 4gb. Definitely buy 64 bit (even though its more of a beta than a real OS) if you do anything memory intensive.

The one good thing about all this is that HOPEFULLY...FINALLY...maybe this will push Microsoft to push 64 bit more. They need to abandon 32 bit and force application writers and hardware manufacturers to start making 64 bit native applications. Working in a medical environment BELIEVE me I understand the need for backwards compatability, but the fact is that the resources are just not being put in to 64 bit to make it a really viable platform and even moderate power users are going to start bumping up against the 4gb limit.

Yea, I read that thing saying that the 4gb limit is a product-based limit rather than a technical limit but either way...it appears that x64 is where MS is choosing to support > 4gb so lets get serious about it.

Re:Isn't it a good thing that it uses 100% of RAM?

[Blakey Rat]

Ok that's retarded.

Using 100% of your RAM can easily make your computer slower. Firstly, if a page is "dirty", it needs to be saved to disk before it can be used for another activity. This causes a write to disk before a read, automatically making the read slower.

While that's true, it doesn't matter because those pages would be in-use no matter how much RAM you have.

The "free" RAM (that is, RAM not being used for applications, that is, RAM pages that won't ever be marked dirty) is all caching-- disk and DLL caching. It'll never get marked dirty, and therefore never need saving to disk before being reused.

The problem is "which page do you overwrite next?" If a program is about to read from a portion of its program memory, or is about to read data sitting in the disk cache, you don't want to delete that page. It is just that the operating system doesn't know which piece of data the program will need next.

That's... somewhat true.

First of all, while the OS doesn't know what piece of data the program will need next, it certainly can (and does) make a guess based on what the program has requested in the past. Most programs allocate most of their memory when first starting up, and the OS is going to be able to predict these bits almost 100% of the time. (Unless it's the first run of that program. But it's not magic.)

Secondly, even if the guesses are bad, even if only one guess in a thousand is correct you're still better-off. Tossing away RAM from a bad guess takes zero time, for all practical purposes. In reality it takes time, but it takes less time than the disk takes to load up the data to fill that page, so in practice it takes no time.

Therefore, the penalty for a wrong guess is zero. The reward for a correct guess is significantly higher than zero. It's a win-win situation.

The result is that having empty RAM is almost always beneficial.

Wrong, for the reasons listed above.

Blindly swapping program memory to disk is not a good idea. It never really was.

True, but Windows doesn't do that and that's not what we're talking about. That would never happen unless RAM was 100% full of application data. Even if that was the situation, it's hardly "blind" swapping, as the same heuristics that allow Windows to predict what an application will load next can help it predict what an application will never touch again.

But to repeat: this isn't the situation we're talking about. We're talking about a computer filling unused RAM with disk and DLL cache. You're talking about a computer whose RAM is 100% full of application memory. Apples to oranges.

On today's computers, a memory leak can effectively stop/bring down the computer. The real function of the page file is to soften this problem. Instead of dying relatively quickly, today's computers will slow to a crawl, and then die. The page file causes this.

Ok... and? What's your point? Are you saying that if we didn't cache to RAM this wouldn't happen?

If you have good, well behaved software, and 4 GB of RAM, you will likely experience better system performance under Windows XP by turning the page file off.

Unless that software wants to memory-map a large file, in which case your performance will go straight to hell. Also, in practice, there's a ton of stuff Windows loads initially that can be (more or less) swapped-out immediately and stay there.

For example, the login screen is likely to only be displayed once, ever. And when it's needed again, there's plenty of time to load it-- why leave it in RAM? (And Windows is smart enough not to.)

The only time I'd consider turning VM off entirely is if you're running MacOS 8. Which had worse than useless virtual memory support. You're not, you're using an OS made in the 21st century, turn on the fucking swap file.