Slashdot Mirror

← Back to Stories (view on slashdot.org)

Anatomy of Linux Kernel Shared Memory

Posted by kdawson on from the culture-by-another-name dept.

An anonymous reader sends in an IBM DeveloperWorks backgrounder on Kernel Shared Memory in the 2.6.32 Linux kernel. KSM allows the hypervisor to increase the number of concurrent virtual machines by consolidating identical memory pages. The article covers the ideas behind KSM (such as storage de-duplication), its implementation, and how you manage it.

7 of 93 comments (clear)

  1. This Is Just One Reason ... by WrongSizeGlass · · Score: 4, Interesting

    ... why OSS is the way things should be. You'll never see this type of documentation, and this type of detail, available to anyone and everyone from closed source software. I love my Mac, and supporting Windows pays my bills, but OSS is unlike any other animal out there.

    1. Re:This Is Just One Reason ... by abigor · · Score: 4, Informative

      OS X's kernel is open source (BSD license) and very well documented.

    2. Re:This Is Just One Reason ... by mswhippingboy · · Score: 3, Insightful

      far better documentation from MS and Apple than Linux has ever had.

      Have you ever looked at Amazon or InformIT/Safari or any technical documentation vendor or website? There are enough books and articles on MS and Linux to keep you reading for many lifetimes (Apple not so much, but still plenty by my estimate). It's just a FACT that there are certain things that closed source vendors do not disclose as a matter of trade secret or intellectual property, which is what I believe WrongSizeGlass was referring to. OSS does not have this limitation.

      And no, the source doesn't count if no one knows what you intended to do

      It absolutely does count, if you know how to read code. You can read documentation all day long, but if it doesn't match the code you'll be lost.
      When I get involved in a rewrite/upgrade/modification of an existing application (and I've worked on many much larger than most OSS apps, and I've been doing it for over 30 years - so it think I have a clue), I read the docs for a quick overview, but I read the code to find out how it really works.
      I've spent plenty of time with both msdn.microsoft.com (and developer.apple.com to a lesser degree) over the years and while there are loads of information there, I know "everything" is not there. There are API documentation discrepancies, quirks, or various undocumented oddities that can be frustrating to work around. These issues can only be resolved one of two ways - trial and error or by reading the code.
      With closed source there is only one way - pray the docs are correct. With open source there are usually two ways and if the documentation is shitty, you can be sure the code is correct. That makes my life as a developer easier.

      Get a clue fanboy.

      Wow. I'm sure you won him over with that jab. First an outpouring of pomposity and narcissism, then a put-down to finish it off.
      Good job dude.

      --
      Sometimes the light at the end of the tunnel is the headlight of an oncoming train.
  2. Re:First Post by Anonymous Coward · · Score: 5, Informative

    From the article:
    "Going further

    Linux is not alone in using page sharing to improve memory efficiency, but it is unique in its implementation as an operating system feature. VMware's ESX server hypervisor provides this feature under the name Transparent Page Sharing (TPS), while XEN calls it Memory CoW. But whatever the name or implementation, the feature provides better memory utilization, allowing the operating system (or hypervisor, in the case of KVM) to over-commit memory to support greater numbers of applications or VMs. You can find KSM—and many other interesting features—in the latest 2.6.32 Linux kernel."

  3. Re:First Post by Abcd1234 · · Score: 5, Insightful

    If your OS isn't sharing duplicate memory blocks already, you're using a shitty OS. (Linux already shares dup read only blocks for many things, like most modern OSes).

    Umm, no.

    Most modern OSes share memory for executable images and shared libraries. In addition, some OSes, such as Linux, support copy-on-write semantics for memory pages in child processes created with fork (note, Solaris is an example of an OS that *doesn't* do this).

    Aside from that, there is no automated sharing of memory between processes. Frankly, I have no idea where you got the idea there was.

  4. Re:First Post by Anpheus · · Score: 3, Interesting

    For now, at least. VMWare doesn't support combining pages >= 2MB because the overhead (hit rate on finding duplicates versus the cost of searching for duplicates) and I suspect other hypervisors will do the same. Additionally, Intel and AMD are both moving to support 1GB page tables. What are the odds that you'll start up two VMs and their first 1GB of memory will remain identical for very long?

    The only way I see page sharing working in the future is if the hypervisor inspects the nested pages down to the VM level, which will typically be the 4KB pages we know and love. Either that, or paravirtualization support needs to exist for loading common code and objects into a shared pool.

    Even so, there's a lot of overhead from inspecting (hashing and then comparing) pages which will only grow as memory sizes grow. If we increase page sizes, the hit rate decreases and the overhead of copy-on-write increases. It's not a good situation.

    Sources: Performance Best Practices for vSphere 4 which references Large Page Performance which states:

    In ESX Server 3.5 and ESX Server 3i v3.5, large pages cannot be shared as copyonwrite pages. This means, the ESX Server page sharing technique might share less memory when large pages are used instead of small pages. In order to recover from nonsharable large pages, ESX Server uses a “sharebeforeswap” technique. When free machine memory is low and before swapping happens, the ESX Server kernel attempts to share identical small pages even if they are parts of large pages. As a result, the candidate large pages on the host machine are broken into small pages. In rare cases, you might experience performance issues with large pages. If this happens, you can disable large page support for the entire ESX Server host or for the individual virtual machine.

    That is, page sharing involves breaking up large pages, negating their performance benefit and is only used as a last ditch when you've overcommited memory and you're nearly to the point of having to hit the disk. And VMWare overcommit is great until you hit the disk, then it's a nightmare.

  5. Re:First Post by Abcd1234 · · Score: 4, Insightful

    Aside from all the places that memory is shared between processes, theres no sharing between processes ... yea, I totally get you ...

    That's exactly right. I pointed out all the places. *All of them*. And there's *two*: shared, read-only executable pages, and the heaps of children created by COW-enabled forks. That's it. That's all.

    So any new technology for memory de-duping is impressive because, traditionally, it just ain't done. Which directly contradicts the content of your original post.

    Perhaps now you understand?