Slashdot Mirror


'Kernel Memory Leaking' Intel Processor Design Flaw Forces Linux, Windows Redesign (theregister.co.uk)

According to The Register, "A fundamental design flaw in Intel's processor chips has forced a significant redesign of the Linux and Windows kernels to defang the chip-level security bug." From the report: Programmers are scrambling to overhaul the open-source Linux kernel's virtual memory system. Meanwhile, Microsoft is expected to publicly introduce the necessary changes to its Windows operating system in this month's Patch Tuesday: these changes were seeded to beta testers running fast-ring Windows Insider builds in November and December. Crucially, these updates to both Linux and Windows will incur a performance hit on Intel products. The effects are still being benchmarked, however we're looking at a ballpark figure of five to 30 per cent slow down, depending on the task and the processor model. More recent Intel chips have features -- specifically, PCID -- to reduce the performance hit. Similar operating systems, such as Apple's 64-bit macOS, will also need to be updated -- the flaw is in the Intel x86 hardware, and it appears a microcode update can't address it. It has to be fixed in software at the OS level, or buy a new processor without the design blunder. Details of the vulnerability within Intel's silicon are under wraps: an embargo on the specifics is due to lift early this month, perhaps in time for Microsoft's Patch Tuesday next week. Indeed, patches for the Linux kernel are available for all to see but comments in the source code have been redacted to obfuscate the issue. The report goes on to share some details of the flaw that have surfaced. "It is understood the bug is present in modern Intel processors produced in the past decade," reports The Register. "It allows normal user programs -- from database applications to JavaScript in web browsers -- to discern to some extent the contents of protected kernel memory. The fix is to separate the kernel's memory completely from user processes using what's called Kernel Page Table Isolation, or KPTI."

36 of 416 comments (clear)

  1. FOOF by OverlordQ · · Score: 5, Insightful

    About par for Intel's course. Make it fast at the expense of horrible bugs.

    --
    Your hair look like poop, Bob! - Wanker.
    1. Re:FOOF by sjames · · Score: 4, Informative

      Actually, it's a reference to a hex value that could trigger a nasty Pentium bug.

    2. Re:FOOF by Anonymous Coward · · Score: 5, Informative
    3. Re:FOOF by TheRaven64 · · Score: 4, Interesting

      Having done a fair amount of both, I would disagree. The things that make software complex are dynamic: you can dynamically create threads and you can dynamically allocate memory, which makes the number of possible interactions almost impossible to statically reason about. In a CPU, all of these things are bounded and (very) finite, to the extent that it is actually possible to apply formal methods to the design (Centaur in collaboration with UT Austin do some amazing work in this area).

      The difference is that a bug in software can usually[1] be fixed after you ship, whereas a bug in silicon usually can't (though if you've got a lot of microcode you can sometimes work around it). ARM has a nice chart of the cost of fixing a bug at each stage in development, which becomes more and more terrifying, whereas for software that cost is roughly flat, so you can get away with spending a lot less effort on correctness.

      [1] Though not always easily. A colleague of mine released his first CVE a couple of years ago for a small library he wrote a couple of decades back. It turns out that most deployments of this library are in fax machines and printers, with no software update mechanism.

      --
      I am TheRaven on Soylent News
  2. How could this be abused? by jader3rd · · Score: 4, Interesting

    Sorry for the lack of imagination, but if the user space process can only read kernel memory, and can't write to it, how could one make use of this?

    1. Re:How could this be abused? by OverlordQ · · Score: 5, Insightful

      You're running in EC2 on shared hardware. Your instance can read the memory of other instances running on the same physical hardware.

      --
      Your hair look like poop, Bob! - Wanker.
    2. Re:How could this be abused? by pereric · · Score: 5, Interesting

      Cryptographic keys, information on other processes (making other attacks feasible), perhaps random number generator seeds and status, for example ...
      And the principle in general that there could be information the process is not supposed to reach.

    3. Re:How could this be abused? by EndlessNameless · · Score: 5, Interesting

      Private keys for system-level crypto and user credentials are stored in kernel space. You want everyone on the system to be reading those? If you can read a private key or a Kerberos token, you can become that daemon/system/user.

      This bug essentially destroys local security and severely compromises network security, subject to any limitations on where/when data can be read.

      I'm not a microarchitecture guru who can dig through the details and figure out the limitations of potential attacks. Perhaps only a small portion of kernel memory can be exposed via this bug. I don't really know. The naive, simple scenario where all kernel memory is exposed, though---that is pretty damned bad. Infosec doomsday bad.

      --

      ---
      According to the latest ruleset, this post should be modded as Vorpal Flamebait +5.
  3. This could be massive by Artem+S.+Tashkinov · · Score: 5, Interesting

    The developers behind the GRSecurity project measured up to 63% performance loss. If most common tasks are equally affected, Intel is sure fucked. Home users might not need to bother, but large cloud providers might be seriously affected.

    Meanwhile the Linux kernel has received the largest incremental minor patch in its history (229KiB) - perhaps kernel 4.14.11 already contains all the required fixes.

    I have a sneaking suspicion Intel shares will fall through the floor in the next few weeks because Intel CPUs might have suddenly become quite slower than their AMD Zen based counterparts.

    1. Re:This could be massive by Artem+S.+Tashkinov · · Score: 5, Informative

      Some PostgreSQL results have just been released: up to 23% performance loss. This is indeed huge.

    2. Re:This could be massive by scumdamn · · Score: 5, Informative

      Doesn't look like it's everybody. https://lkml.org/lkml/2017/12/...

    3. Re:This could be massive by grimr · · Score: 4, Informative

      Nope. Page Table Isolation is the fix and not the fault. But isolating the userland and kernel page tables means you have to switch between them each time you go from user mode to kernel mode and back. This slows things down.

      AMD CPUs don't have the bug where user mode can read kernel pages so does not require this isolation and the performance hit caused by enabling it. From the AMD email: "The AMD microarchitecture does not allow memory references, including speculative references, that access higher privileged data when running in a lesser privileged mode when that access would result in a page fault."

    4. Re:This could be massive by Qzukk · · Score: 4, Interesting

      Based on this link from Hacker News: https://cyber.wtf/2017/07/28/n... and the linked email/patch from AMD, it looks like what happens is that AMD checks memory permissions up front before allowing an instruction into the pipeline, while Intel made the memory permission check as a later part of the pipeline, apparently after the memory was accessed and inserted into the cache.

      --
      If I have been able to see further than others, it is because I bought a pair of binoculars.
    5. Re:This could be massive by Motherfucking+Shit · · Score: 4, Interesting

      I have a sneaking suspicion Intel shares will fall through the floor

      Intel's CEO agrees; a couple weeks ago he sold all the Intel stock he can. If he'd dumped any more shares, he would have had to forfeit his job. That isn't a man who's confident about the future of his company...

      --
      "BSD: Free as in speech. Linux: Free as in beer. Windows 10: Free as in herpes." --Man On Pink Corner in #52607549.
  4. Obligitory LWN link (Also affects ARM64) by sl3xd · · Score: 5, Informative

    Linux Weekly News has been covering this for quite a while.

    5% slowdown on average, with up to 30% for some particularly bad network operations.

    ARM64 is also affected, so it's not just intel

    --
    -- Sometimes you have to turn the lights off in order to see.
  5. Re:five to 30 per cent slow down by Carewolf · · Score: 4, Informative

    I find it hard to believe that a virtual memory change will result in a 5-30% slowdown for Intel processors. Maybe for a few extremely specific (likely edge-case) tasks, but if there was a legitimate 5-30% performance decrease, you can bet there would be a far different solution in the works that would suitably fix the problem.

    Virtual memory access is used in every single memory access cached or not. 5% would be lucky for trying to work around a broken system. I am guessing the flaw is probably in the TLB which is meant to accelerate these things.

  6. Re:In all fairness... by scumdamn · · Score: 5, Informative

    Looks like you missed this commit from Tom Lendacky at AMD.

  7. And, time for AMD to shine again by NuclearCat · · Score: 5, Informative

    And what is interesting, AMD is immune to that, proof: https://lkml.org/lkml/2017/12/...

  8. AMD is safe by TeknoHog · · Score: 5, Informative

    The summary is not fully explicit: this is not a flaw in Intel x86 ISA, but specific to CPUs made by Intel. AMD processors don't have the problem, so they should not need the patch.

    https://lkml.org/lkml/2017/12/...

    This could be a huge win for AMD, because the patch incurs a measurable slowdown. At the moment, though, the Linux fix doesn't seem to distinguish between manufacturers. I expect the distinction will appear later -- better safe than sorry.

    --
    Escher was the first MC and Giger invented the HR department.
  9. More info on the subject by trybywrench · · Score: 4, Informative

    some of my sys admin friends posted this on a slack channel i'm in, apparently it's a big deal

    http://pythonsweetness.tumblr.com/post/169166980422/the-mysterious-case-of-the-linux-page-table

    --
    I came to the datacenter drunk with a fake ID, don't you want to be just like me?
  10. Re:In all fairness... by nemequ · · Score: 4, Informative
    What are you talking about? That patch says AMD CPU's aren't vulnerable:

    AMD processors are not subject to the types of attacks that the kernel page table isolation feature protects against. The AMD microarchitecture does not allow memory references, including speculative references, that access higher privileged data when running in a lesser privileged mode when that access would result in a page fault.

    The (trivial) patch essential disables the work-around on AMD CPUs. I'm not going to comment on how fair GP's criticism of Intel is in general, but that link definitely isn't evidence in Intel's favor

  11. Re:five to 30 per cent slow down by Zocalo · · Score: 4, Informative

    Linux users don't even need to compile a custom kernel (although if performance *really* matters you should probably be doing so as a matter of course) as there's a boot time option in the that can be set to disable the new Page Table Isolation mode, "nopti". Without knowing the performance hit for a specific usage case and the nature of the flaw its currently impossible to say whether using it is going to be a good idea or not, but it's nice that they at least thought to include the option. Pretty sure BSD will do the same, but feel free to place your bets on commercial operating system vendors...

    --
    UNIX? They're not even circumcised! Savages!
  12. Re:In all fairness... by sjames · · Score: 5, Insightful

    That bolster's AC's point. It looks like the Intel guys were going to cripple performance for everyone until the patch from AMD removed the unnecessary crippling from AMD processors.

  13. The future by Artem+S.+Tashkinov · · Score: 4, Interesting

    I'm curious how much Cannon Lake and Ice Lake CPU architectures are going to be delayed. Since Cannon Lake is basically SkyLake on a 10nm node, Intel cannot release it with such a glaring hole which causes such a significant performance loss.

    I've been running a Sandy Bridge CPU for seven years now, and now I'm really looking forward to the second gen Zen CPUs. Viva, competition. I'm really glad AMD is still around.

  14. Intel CEO Sold a lot of stock... by Nos. · · Score: 5, Interesting

    https://www.fool.com/investing...

    Less than a month before we know the linux kernel was being patched for this bug.

    1. Re:Intel CEO Sold a lot of stock... by sl3xd · · Score: 5, Informative

      This bug has been known and reported about since early November; the original paper was presented in July of 2017, and code has been in Github since Feburary.

      Motley Fool is just noting that the Intel CEO isn't holding any more stock than he needs to.

      And there are good reasons:

      * AMD is back from the dead.
      * Intel's GPU hasn't been that successful -- they've even teamed up with AMD to put Radeon GPU's in the same die as an Intel CPU.
      * PC sales are declining as consumers shift from Intel PC's to using ARM-powered tablets & phones instead.
      * ARM is making inroads into the "desktop and laptop computer" marketplace.
      * ARM is powering most consumer electronics as well (TV's, Blu-ray players, Smart Speakers, etc)
      * Intel is absolutely nowhere in the mobile world. Mobile has one ARM to rule them all.
      * Intel missed the boat for the current generation of XBOX and PlayStation consoles.

      Intel is looking more and more like a one trick pony, and its competitors are beginning to do that one trick better too.

      --
      -- Sometimes you have to turn the lights off in order to see.
  15. ... and this is why ... by nbvb · · Score: 5, Interesting

    This is why we run our mission critical workloads on SPARC and Power along side Linux. Solaris and AIX. Diversity -- in operating system, in processor, in manufacturer - is healthy. The SPARC T8's are blazing faster, secure, and don't have this nonsense. Neither do our POWER8's. Having all your eggs in the Intel+Linux basket could be a major shitshow here... meanwhile, we'll keep chugging along.

  16. Re:In all fairness... by sexconker · · Score: 5, Insightful

    Well, I'm guessing the approach was more along the lines of "an abundance of caution with the X86 ISA" as opposed to deliberate malice towards AMD.

    Hi. Have you met Intel?

  17. Re:Don't know if serious. by Hal_Porter · · Score: 5, Funny

    https://www.mail-archive.com/l...

    2) Namespace

          Several people including Linus requested to change the KAISER name.

          We came up with a list of technically correct acronyms:

              User Address Space Separation, prefix uass_

              Forcefully Unmap Complete Kernel With Interrupt Trampolines, prefix fuckwit_

          but we are politically correct people so we settled for

            Kernel Page Table Isolation, prefix kpti_

          Linus, your call :)

    LOL!

    --
    echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
  18. Re:five to 30 per cent slow down by grimr · · Score: 5, Informative

    I don't think you understand how drastic this fix is. Every time a user mode to kernel mode transition happens and every time a hardware interrupt happens, the entire page table directory layout has to be switched. This means all the TLB caches are flushed as well and that's where the main performance hit comes from.

    So if you're doing something like crypto currency mining you're not going to see much of a hit. But if you're doing a lot of I/O (file servers, database servers, web servers, etc.) you're going to see that 25-35% performance hit.

    And that's why hardware bugs are so serious. Sometimes you get lucky and it's a microcode update with no penalty. Sometimes it's a simple fix with barely any performance penalty. But sometimes you get unlucky and the fix hurts a lot and the only way to get the performance back is to swap out the hardware.

  19. Re:Go ARM Go by andydread · · Score: 5, Informative

    really sorry to bust your ARM bubble. https://lwn.net/Articles/74039...

  20. Speculative Memory References and Page Faults by bill_mcgonigle · · Score: 5, Interesting

    From the AMD commit:

    AMD processors are not subject to the types of attacks that the kernel
    page table isolation feature protects against. The AMD microarchitecture
    does not allow memory references, including speculative references, that
    access higher privileged data when running in a lesser privileged mode
    when that access would result in a page fault.

    this can probably be rewritten in the inverse like:

    Intel processors ... allow memory references, including speculative references, that
    access higher privileged data when running in a lesser privileged mode, [including]
    when that access would result in a page fault.

    So it seems like: set up a speculative memory reference to a kernel memory structure, cause a page fault, and then get a bit of kernel memory out (and back in?). That could get you root before long. Some people have been saying this can be leveraged to get a guest into its hypervisor too.

    --
    My God, it's Full of Source!
    OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
  21. Re:In all fairness... by scumdamn · · Score: 5, Informative

    I'm saying that Intel tried to cripple all processors but the dude from AMD told him their products weren't vulnerable and didn't need to be slowed down.

  22. Re: five to 30 per cent slow down by MrKaos · · Score: 4, Interesting

    By "virtual memory" are we talking Page Files and swap space?

    Almost. The difference is a minor or major page fault. From swap space to ram to CPU cache is a major page fault whereas a memory transfer is between the CPU cache (L1,L2 or L3) and system RAM is a minor page fault.

    Disk space as memory?

    No. Between CPU cache and system RAM. In this case the issue relates to minor page faults which occur when the CPU scheduler is switching tasks, called "context switching", between processes, threads and lwt running on the system.

    The kernel maintains a summary page of the process when it switches tasks so that it doesn't have to recreate details of the process and is more efficient when it switches tasks. It is this summary page that can be attacked and IIUC addresses can be changed to access memory the process does not have permission to access.

    This means that a process running in the userspace of Ring 3 of the CPU it can modify it's summary table to access data in Ring 0, where the kernel is. Which is obviously bad because now that process can access all the memory.

    So an almost unusuable computer becomes completely unusable. Unless you're on solid state, then you get the performance of a mechanical hdd.

    Maybe, but not how you would expect. The CPU scheduler *may* make it possible to hide some of the latency created by the now crippled context switching in mechanical disk latency that you can't do with ssd because of the way IO determines *when* the scheduler will context switch. Still early days and it depends on what techniques can be devised. It depends on where the summary tables are maintained and I would expect that to be in CPU cache, which is much faster than system RAM, which is why it is a tough bug to get around.

    With that in mind it's plain to see why the Linux kernel devs want to call the patch fuckwit_ because Intel screwed up badly on this.

    --
    My ism, it's full of beliefs.
  23. Re:FUCKWIT by Waffle+Iron · · Score: 4, Insightful

    I'm not in the habit of running random binaries downloaded from the Internet

    As TFS implies, given that Javascript required to do almost anything on the web, you are most likely downloading and running random code from the internet that could potentially exploit this bug hundreds of times every day.

  24. Re:Caching is what makes CPUs fast by TheRaven64 · · Score: 4, Interesting

    Leaving stale ones is often fine. FreeBSD does this intentionally in the transparent superpage promotion. When it condenses adjacent pages into a single superpage in the page tables, it doesn't invalidate the TLB entries. The exact behaviour of this varies between CPUs. When you get a TLB miss in an adjacent address range, it's filled from the superpage entry and now you have two TLB entries for the same virtual address[1]. Intel will just discard the smaller one, Centaur will note the mismatch, invalidate both, and refill from the page table, gem5 will crash (I think we've upstreamed the fix for this), and I'm not sure what AMD does.

    Your example is highly unlikely, because the kernel typically reserves the top half of the address space for itself and an attempt by userspace to map anything in this range will fail. On x86 chips, there's a huge gap in the middle of the address space (it actually makes more sense to think of virtual addresses as signed values, with userspace ones being positive, kernel ones being negative, and the size of the number somewhat less than 64 bits [microarchitecture specific]). The kernel map, by default, will include the entire userspace portion of the address space for the current process, so that copies between kernel and userspace are cheap.

    This kind of TLB invalidate is actually cheap on AMD, because they implement a tagged TLB with the cr3 value as the tag, so swapping cr3 values implicitly invalidates the TLB, but the entries become valid again when you reset the entry. The PCID feature, which is apparently the cause of this vulnerability, is largely a result of the fact that AMD patented this technique and so Intel doesn't use it.

    [1] Some old SPARC chips would literally catch fire if you did this: the TCAMs would run hot enough to burn.

    --
    I am TheRaven on Soylent News