24 Cores and the Mouse Won't Move: Engineer Diagnoses Windows 10 Bug (wordpress.com)

Longtime Slashdot reader ewhac writes: Bruce Dawson recently posted a deep-dive into an annoyance that Windows 10 was inflicting on him -- namely, every time he built Chrome, his extremely beefy 24-core (48-thread) rig would begin stuttering, with the mouse frequently becoming stuck for a little over one second. This would be unsurprising if all cores were pegged at 100%, but overall CPU usage was barely hitting 50%. So he started digging out the debugging tools and doing performance traces on Windows itself. He eventually discovered that the function NtGdiCloseProcess(), responsible for Windows process exit and teardown, appears to serialize through a single lock, each pass through taking about 200 microseconds each. So if you have a job that creates and destroys a lot of processes very quickly (like building a large application such as Chrome), you're going to get hit in the face with this. Moreover, the problem gets worse the more cores you have. The issue apparently doesn't exist in Windows 7. Microsoft has been informed of the issue and they are allegedly investigating.

Comments

The lock cycles were avg 200 us each

[Anonymous Coward]

Not 200S each, which is off by a factor of one million. But, hey.

Windows has always been unresponsive to user input

[fustakrakich]

We just don't have priority...

I don't get it.

[fuzzyfuzzyfungus]

If there is an issue that keeps process termination and cleanup from being properly parallelized; I can understand why that might cause unexpectedly poor utilization of additional cores for computationally intensive tasks that also massacre lots of processes; but why would that cause the GUI to stop responding?

Unless moving the cursor also depends on terminating a bunch of processes; and hangs until that task is finished, wouldn't the inefficiency imposed on the build process be expected to keep the GUI more responsive; by preventing it from occupying as much CPU time as it otherwise would?

Am I just confused? Does keeping the desktop and cursor drawn actually involve lots of time sensitive process killing? Does this indeed not make sense?

Re:I don't get it.

[Anonymous Coward]

The Windows GUI interface actually uses a separate process to update the mouse on the screen. Due to various historical reasons (compatibility with old applications, mostly), it was required to recycle this process every time the mouse moved, as the process could get a memory leak (which couldn't be fixed properly, in order to preserve compatibility with the aforementioned applications). Therefore, every time the coordinates of the mouse change, the process has to be killed and replaced, therefore putting it through the same lock that this build process is hogging. Combine that with the 200 second delay to get through the lock, and the responsiveness is easily explained.

It's worth it to keep compatibility with the "After Dark" flying toasters screensaver, though.

Re:I don't get it.

[Rutulian]

It's easy to criticize from the outside, but the Linux kernel has historically had kernel locks that created similar problems, such as the "big kernel lock", removed ca 2011 (ie: not ancient history).
https://kernelnewbies.org/BigK...

As noted in the article, this particular locking problem appeared in Windows 10 and wasn't present in Windows 7, so the balancing acts between the fine-grained locking mechanisms, thread performance, and backwards-compatibility are clearly challenging to maintain. Not excusing; just observing. Windows has never been known for it's ability to support massive numbers of parallel threads, so it is not surprising that previously overlooked problems can appear or become exacerbated in these situations. Many people, even here on Slashdot, laud Microsoft for the generally excellent backward- compatibility in Windows, and criticize the Linux kernel for being generally horrible at it. But here you go, a pretty nice example to illustrate that backwards-compatibility has a cost.

Re:I don't get it.

[mvdwege]

Yes, you are making exuses, that's exactly what a tu quoque fallacy is

The big lock was removed in 2011, Microsoft produced a regression on an already bad design a lot closer in history. That's a sign of incompetence, period.

Also, the Linux kernel has bad backwards compatibility, which is why things like drivers and such should be upstreamed as much as possible and built in the main tree, but Linux userland still happily runs old Unix software, so you are overstating that case as well.

Re:Windows...

[presidenteloco]

More specifically, why are OSes not designed, and computing hardware not designed, so that the GUI cannot be slowed down by other slow processes, process switching, or I/O / virtual memory thrashing.

The most brain-dead design-avoidable situation in the computing universe is where my computer is thrashing due to some resource over-use, and the UI is inoperable so I can't fix the problem e.g. by killing processes/programs. DOH!

The UI and user input devices should be a completely separate set of processes and memory than the rest of application processing. It should operate as a service, through data pipelines, to the rest of the applications. It should be completely separate, in terms of resource management. Or failing that, certain aspects of GUI, such as program kill controls, should be highly prioritized over pretty much everything else.

Again, slow and over-used everything else should not slow the UI and user input processes. This is basic.

I remember BeOS

[rsilvergun]

being the only OS I've ever seen in my life move a window around screen w/o tearing. Yeah, it doesn't make much difference, but you'd think in 2017 my quad core CPU and 8 gigs of ram could do what a 400 mghz AMD K6 did in 1996 with 512 mb ram.

Re:I remember BeOS

[adolf]

I was accomplishing this on 486DX2 hardware using OS/2 in ~1994, and by 1995 on a P120.

Several years ago I stopped by a buddy's retail establishment. He was transitioning network to Ubuntu on more modern hardware (with OS/2 in a VM), but still had an old and crusty OS/2 machine (probably a K6-2, but maybe a DX4) on the bench by the back door.

This was the last time I ever saw such a thing in the wild.

It was remarkably snappy doing normal, productive things -- scanning documents, browsing web pages, writing and viewing proposals -- just like it was when it was built. (And what window tearing?)

Sometimes I think that the more abstraction layers we add, the slower things get. I think this coupled with programmer laziness (and/or pay based on lines of code), makes human-interactive things continue to behave just as slow as they have been for ~20 years.

Do we even use accelerated 2D desktop graphics anymore, or are we completely back to the bad old days of every application drawing into a dumb framebuffer?

Re:Not just when closing a program

[viperidaenz]

That could be related to the hardware acceleration. If I had to guess, the Windows desktop would need to wait for the game to release it's GPU resources and load its own in to the GPU memory.

Back in the XP days, going from game to desktop was very quick, but going from desktop back to game was very slow. When Vista came along, the Os started using the GPU to accelerate the desktop. Made it slow both ways

Only 24 Cores?

[Anonymous Coward]

2 Core for DRM
2 Core for DRM Protection
2 Core for Telemetry
2 Core for Telemetry Protection
2 Core for Genuine Advantage
2 Core for Genuine Advantage Protection
2 Cores for Driver Signing Validation
2 Cores for Driver Signing Validation Protection
2 Cores for Cortana
2 Cores for Cortana Telemetry
2 Cores for Cortana Telemetry Protection
1 Core for the Base OS
1 Core, at 25% for user processes

Re: Not just when closing a program

[viperidaenz]

Android is dropping to lower-res/fewer colors to accelerate the transition

Or did it swap the high-res texture for a low-res one to save memory while it was not in use?

GPU's have been able to do 32bit acceleration for a long time.

Semi-OK gaming video cards that didn't support DirectX 9 couldn't run Vista Aero because it used the DirectX 9 API, required hardware based Pixel Shader 2.0 (not emulated in the driver) and at least 128MB of RAM. Not because of bit-depth.

Re: Windows...

[TheRaven64]

This issue didn't exist, so someone went and added synchronisation around the tear-down process. Why? People don't generally add mutexes at random, so this was probably done because they found a race condition in process tear-down that resulted in either a resource leak or kernel data structure corruption. Simply removing the mutex will probably fix this problem, but make the computer either run out of some kind of resources or crash. Probably not the correct fix...

OS and process scheduling

[DrYak]

More specifically, why are OSes not designed, and computing hardware not designed, so that the GUI cannot be slowed down by other slow processes, process switching, or I/O / virtual memory thrashing.

That's why OSes such a Linux (not even over-optimized for responsivity) have an entire zoo of CPU schedulers and IO schedulers.
(with BFQ being the latest popular IO scheduler for responsivity),
and linux specifically has hte non-POSIX "CGROUPS" extension that enables it to arrange the various processes into a tree hierarchy with each node supporting its own scheduling tactics between its childern (see demos of 256 GCC compiler jobs launched in parallel and the GUI still being responive).
(That's also part of the reasons why modern complex manager like systemd are getting popular, they have modules to handle all this : session, seats, etc. concepts that POSIX lacks)

BeOS was an OS whose entire purpose was exactly that : no matter what, keep UI responsive and avoid media stuttering.
(Well, running initially on architecture with less expensive context switches did also help a lot).

The UI and user input devices should be a completely separate set of processes and memory than the rest of application processing.

Actually, in most OSes, they already are.

It should operate as a service, through data pipelines, to the rest of the applications.

That's a tiny bit less obvious. Some graphical tool-kits run their UI in the main thread.
Some software would need to have the processing moved into a background thread or process.

WebApps are an obvious counter-exemple where the UI is an entirely different process (And depending on where the sever is executed - even different machine).

Or failing that, certain aspects of GUI, such as program kill controls, should be highly prioritized over pretty much everything else.
Again, slow and over-used everything else should not slow the UI and user input processes.

And then you'd complain that any complex calculation (compression of a video) takes ages, because the process is constantly being interrupted to give time to your GUI and mouse (i.e.: to the various driver and daemons and libraries processing USB and/or Bluetooth) even if they don't need.

Balancing responsivity (i.e. constantly interrupting everything just to be sure that everyone get their share of CPU cycles and IO) and performance (running as much un-interrupted as possible so the task finishes as fast as possible) is a complex dark art.

But, yeah, Windows is significantly worse at this compared to everyone else.

Which also explains you'll never see any deployment of Windows on the TOP500 (it's nearly Linux all the way, with a few excption like BSD - i.e. other Unix-type of OSes), and Azure is the only known cloud running it.
It's also why Linux is popular in most embed systems (modems, routers, smartphone, tons of IoT gizmos, smart TVs, etc. - basically nearly anything with a CPU that is not a desktop computer is likely to run some Unix-like kernel like Linux)

It's not that Linux is magic, it's that Windows is *THAT* awfully bad at anything.