Extreme Memory Oversubscription For VMs

Laxitive writes "Virtualization systems currently have a pretty easy time oversubscribing CPUs (running lots of VMs on a few CPUs), but have had a very hard time oversubscribing memory. GridCentric, a virtualization startup, just posted on their blog a video demoing the creation of 16 one-gigabyte desktop VMs (running X) on a computer with just 5 gigs of RAM. The blog post includes a good explanation of how this is accomplished, along with a description of how it's different from the major approaches being used today (memory ballooning, VMWare's page sharing, etc.). Their method is based on a combination of lightweight VM cloning (sort of like fork() for VMs) and on-demand paging. Seems like the 'other half' of resource oversubscription for VMs might finally be here."

Comments

Re:So... is this different from Linux KVM w/ KMS?

[amscanne]

Yes, rebasing is reduced by careful selection of preferred base addresses (particularly by Microsoft for their DLLs). Yes, if DLLs are not rebased then they are shared -- I did not claim otherwise. My point along these lines is that rebasing *does* occur surprisingly often, and can hurt sharing. The actual level of sharing you achieve obviously depends almost *entirely* your applications, workload, data, etc.

By the way, as far as I know versions of Windows newer than Vista enable address-space randomization by default for security purposes. Since the starting virtual address of each DLL is randomized, preferred bases can't be respected. I don't know what impact this has on Windows memory usage post-Vista, but it seems like one can't rely on carefully curated base addresses.

I'm not saying one approach is better than the other (Linux, Windows, whatever) -- I'm only positing a possibility for why one might see better improvement with KSM. One might just as easily see better over-subscription with Windows simply due to the fact that it zeroes out physical pages when they are released, as far as I know. Those zero pages can all be mapped to the same machine frame transparently (without the need for balloooning).

Re:Leaky Fawcet

[vlm]

When disks come in 2TB sizes .... why should I sweat 8 GB?

You are confusing capacity problems with thruput problems. Sweat how poor performance is when 8 gigs gets thrashing.

The real problem is the ratio of memory access speed vs drive access speed has gotten dramatically worse over the past decades.

Look at two scenarios with the same memory leak:

With 8 gigs of glacially slow swap, true everything will keep running but performance will drop by a factor of perhaps 1000. The users will SCREAM. Which means your pager/cellphone will scream. Eventually you can log in, manually restart the processes, and the users will be happy, for a little while.

With no/little swap, the OOM killer will reap your processes, which will be restarted automatically by your init scripts or equivalent. The users will notice the maybe, just maybe, they had to click refresh twice on a page. Or maybe it seemed slow for a moment before it was normal speed. They'll probably just blame the network guys.

End result, with swap means long outage that needs manual fix, no swap means no outage at all and automatic fix.

In the 80s, yes you sized your swap based on disk space. In the 10s (heck, in the 00s) you size your swap based on how long you're willing to wait.

It takes a very atypical workload and very atypical hardware for users to tolerate the thrashing of gigs of swap...