Slashdot Mirror


Generational Windows Multicore Performance Tests

snydeq writes "Windows XP, Windows Vista, and (soon) Windows 7 all support SMP out of the box, but as InfoWorld's Randall Kennedy notes, 'experience has shown that multiprocessing across discrete CPUs is not the same thing as multiprocessing across integrated cores within the same CPU.' As such, Kennedy set out to stress the multiprocessing capabilities of Windows XP, Windows Vista, and Windows 7 in dual-core and quad-core performance tests. The comprehensive, multiprocess workload tests were undertaken to document scalability, execution efficiency, and raw performance of workloads. 'What I found may surprise you,' Kennedy writes. 'Not only does Microsoft have a firm grasp of multicore tuning, but its scalability story promises to keep getting better with time. In other words, Windows Vista and Windows 7 are poised to reap ever greater performance benefits as Intel and AMD extend the number of cores in future editions of their processors.'"

27 of 228 comments (clear)

  1. So what? by fpophoto · · Score: 5, Insightful

    Are we supposed to be surprised that the leading OS vendor, who's had deep, intertwined relationships for decades with hardware makers is exploiting that hardware properly?

    Honest question: where's the news here?

    1. Re:So what? by Anonymous Coward · · Score: 5, Insightful

      The news is that nobody can say anything non-critical of Microsoft without being accused of advertising or astroturfing.

      Oh wait, this is Slashdot. Nevermind.

    2. Re:So what? by kno3 · · Score: 2, Insightful

      Ha, I can't believe that you could be so biased against Microsoft that you would even turn an example of them doing well into an insult. And yes, I have many a Linux obsessed friend that have argued with me saying: Vista's 64 bit support is terrible and it is unable to utilise more than one core. Fact is that it does so very well, and I am very glad of this article as it is a great thing to cite when I get told bulls**t again.

    3. Re:So what? by squallbsr · · Score: 2, Insightful

      I would believe that the 5000 layoff is the culling of the 'bad apples' that every organization has in their ranks. The economy is just a good excuse to do it.

      The proof is in where the cuts are being made. Are they cutting sales staff? Are they cutting consulting services? Or are they cutting developers? -- THAT is the question.

      --
      Sleep: A completely inadequate substitution for Caffeine.
  2. 118% slower by TheMeuge · · Score: 5, Insightful

    Roughly 118 percent slower than XP on dual-core

    Some great mathematics in this review... it also appears as if Vista isn't just not solving the problems presented to it, but also adds new ones to increase its own workload.

    Fascinating...

  3. And Windows XP is still faster by ameline · · Score: 4, Insightful

    And XP is still faster than vista or 7, even on 4 cores... And he speculates that it would be faster on 8 (although he didn't measure that)

    Scalability doesn't matter if you're still slower in absolute terms on systems that are available commercially at a reasonable price. (going past 8 cores these days is a very large price jump per core)

    --
    Ian Ameline
  4. Interesting by quo_vadis · · Score: 4, Insightful

    It is interesting that WinXP is still better in terms of performance than either. The article suggests that Win7 and Vista would be better on systems that hypothetically had 16+ cores.

    But nowadays, especially in tech savvy crowds like on /., the most popular thing to do is run VMs with virtual instances of Windows, which reduces all the hassles associated with dealing with win cruft. Got a worm? restore machine. Drivers made system unstable? restore machine. The vms are typically only given 1-2 cores, the exact use case where WinXP does way better than its successors.

    So even if we move to a world with 16+ core processors, if Win7 cannot do better than a 10 year old OS, in common scenarios, how can that be called progress?

    --
    Legally obligatory sig : My opinions are my own... etc etc
    1. Re:Interesting by blankinthefill · · Score: 5, Insightful

      Try running Win 98 on Vista's minimum hardware. Hell, lets go whole hog here. Try running Win 3.1 on Vista's minimum hardware. (Okay, you might have to do a lot of work to get it running, but I'm just trying to make a point). I guarantee you that both 98 and 3.1 will run faster than XP on that hardware. By a lot. If it surprises you that vista and xp run slower on the same hardware than xp, then either you're not thinking things through, or you're not very bright. As stated, they are far newer. This means they have a much higher assumed baseline of technology that they can run against, which means that they have far more assumed resources to play with. So yeah, on the same system, Vista runs slower than XP. No surprise (at all, as far as I'm concerned). Honestly, all this speed stuff is pretty pointless. The question with OSes is never really about the fastest, or we would all still be using DOS. The question with OSes is are the fast ENOUGH. This is very subjective, but it basically boils down to: will they run what we want them to run in an acceptably small amount of time. On its original release, Vista did not. However, right now Vista is certainly running fast enough for me, and I expect Win 7 will to. But you're ALWAYS going to take a performance hit moving to an OS that utilizes new technology, and I don't care what OS you use.

    2. Re:Interesting by EvilJohn · · Score: 3, Insightful

      Terrible analogy.

      Windows 98 or Windows 3.1 can't run the exact same applications. The author's point is a good one. Why does running the exact same application under VISTA or W7 cost 20 to 40 more cpu cycles?

      In business terms: Why do you have to buy 20 to 40% more hardware to get the same result as I have today?

      --

      Less Talk, More Beer.
  5. Look for the Dodgy Phrasiology by segedunum · · Score: 3, Insightful

    Not only does Microsoft have a firm grasp of multicore tuning, but its scalability story promises to keep getting better with time.

    When you see bullshit buzzwords in articles that look as if they've been written by marketing people then look out. Marketing-led, buzzword-laden people always have stories. Are we really supposed to be impressed that the richest OS developer in the world can actually create a SMP capable OS that actually works reasonably given that SMP systems have been around for years? From the tone of the article it's like they're shocked that it works.

  6. The Money Quote by d3ac0n · · Score: 5, Insightful

    Basically, this article states the obvious: Windows XP 64 is just plain faster than Vista 64 or Win7 64. By a factor of 20-40%. But to understand why, you need to read the MONEY quote. Here it is:

    In the end, it all comes down to the complexity of the execution path. With its simpler legacy kernel devoid of DRM hooks and other performance-sapping baggage, Windows XP provides a cleaner code path for the workloads to navigate as they execute. This, in turn, translates into better overall performance with lower consumption of CPU cycles.

    It's the DRM baby. You strip that out of the Kernel, and Vista and Win7 will EASILY outpace XP with their more advanced and flexible SMP capability. Until Microsoft understands that people DO NOT WANT DRM and removes it from their newer OSes, these new OSes will continue to suffer from performance problems, and thusly, acceptance and sales problems.

    Come on Microsoft. Apple has figured it out, DRM is a sales loser. Do you really want to keep wasting time on a loser technology in the midst of a global recession? You blew it with Vista, but you still have a chance with Win7. Offer people a DRM-Free kernel and Win7 will FLY off the shelves.

     

    --
    Official Heretic from the "Church of Global Warming". Proven right thanks to whistle blowers. AGW = Flat Earth Theory
    1. Re:The Money Quote by aliquis · · Score: 2, Insightful

      I've never understood why the DRM crap matters as long as you don't run anything using it but whatever. Sounds like a bad excuse.

    2. Re:The Money Quote by Anonymous Coward · · Score: 1, Insightful

      you need to read the MONEY quote. Here it is:

      In the end, it all comes down to the complexity of the execution path. With its simpler legacy kernel devoid of DRM hooks and other performance-sapping baggage, Windows XP provides a cleaner code path for the workloads to navigate as they execute. This, in turn, translates into better overall performance with lower consumption of CPU cycles.

      Does he know this for a fact, or is he just pulling a big juicy guess out from his gaping asshole?

    3. Re:The Money Quote by quo_vadis · · Score: 5, Insightful

      I have never been a "windows fanboi"( In fact this is being posted from a linux computer) and I am no defender of Microsoft's business practices. However without doing code analysis, it is impossible to say that this slowdown is because of DRM. Nowhere in the article does it suggest that they were able to do a profile analysis of the kernel codes and compare what modules on the path were causing the delays. So while it is theoretically possible(and likely) that the source of the delay was DRM related, one cannot be sure. If you possess knowledge otherwise, please feel free to cite it and correct me.

      --
      Legally obligatory sig : My opinions are my own... etc etc
    4. Re:The Money Quote by rossjudson · · Score: 5, Insightful

      This is yet another one of those times when I wish Slashdot wasn't so ridiculously hostile to Microsoft. What we need here is some informed, possibly even official commentary from someone in the know at MS. Exactly why is a workload slower on Vista? Where's that time going? Right now something like 60-70% of corporate workloads still run on Windows OS, so gaining an understanding of exactly why is important.

      When the difference is on the order of 20-40% (if the article is to be believed), we're looking at some level of system-call "tax" under Vista, or we're looking at a different _capability set_. If the workload on Vista is in a secured environment, and the same workload runs faster on XP in an unsecured environment, we're talking apples and oranges.

      It could be the case the even for workloads running as root equivalents in Vista execution times are worse...but we don't really know from what's quoted in this article, and there isn't any response from MS.

      I think Vista is a pretty important upgrade for most users. Even if its security mechanisms are intrusive, at least they're _there_, and that's a step in the right direction.

    5. Re:The Money Quote by CannonballHead · · Score: 3, Insightful

      That's an itneresting point. I wonder why they didn't test 2k, 98, and 95?

      I could test an old DOS game vs. [insert new game here] and see the FPS in [insert new game here] be so, so much lower. [insert new game here] must be horrible, DRM infested, and we should all use old DOS games.

      Granted, Vista does seem a bit on the slow side (although on my Q6600 w/ 7GB RAM, it runs pretty quickly all in all), and Windows 7 *seems* to run slightly faster even in a VM... but I'm so tired of reading the "XP (read: an operating system that came out many years ago) is faster on current hardware than Vista/Windows 7!"

      I may as well come out and say Ubuntu is so stupid, I could run [insert random distro] on my old 486 faster than Ubuntu on my modern hardware! Or whatever. The general idea is there.

    6. Re:The Money Quote by samriel · · Score: 2, Insightful

      I can't install rockbox on a newer ipod because they decided to encrypt the firmware.

      I can't use linux to sync newer ipods because they decided to increase the itunes database.

      So... Apple won't let you install a third-party program... on their proprietary device. Big whoop. When you bought something that was locked down, and YOU KNEW IT, you lost the right to complain about Apple protecting any rights they may have tied up in the iPod OS / hardware.

      The syncing on linux point is moot, because even if you can't do it right now, you WILL be able to shortly. Thank OSS.

    7. Re:The Money Quote by setagllib · · Score: 4, Insightful

      Unlike Windows, if you install a very lean Linux distribution, even one with up-to-date package versions, it will be absurdly fast as you'd expect from modern hardware. Just because Ubuntu is being burdened doesn't mean the FOSS landscape itself is.

      The algorithms and data structures in almost all open source applications have either stayed the same over the years, or gotten better, with the notable exception of some programs like GCC which have used the increase in system resources to advance compiled code optimisation rather than compile-time performance.

      This whole "of course newer is slower" thing is just Microsoft brain damage. Apple is another company which regularly improves performance and scalability in its software products, even with DRM problems similar to Vista.

      Did you know that by now hundreds/thousands of Windows *system* *calls* have built-in backwards compatibility checks to support programs whose source code is not available for maintenance? You're paying a performance penalty every time you make a system call, with no option to turn it off, just because someone somewhere might want to play The Sims. Open source solves this problem by fixing the program, rather than hacking every layer of the OS to support old bugs. You can't spell "backwards compatibility" without "backward".

      --
      Sam ty sig.
    8. Re:The Money Quote by Toreo+asesino · · Score: 2, Insightful

      The best thing about you "Linux shills" is that you make outrageous claims about Windows while in the same breath letting it known publicly you've never touched it with a 10-foot pole.

      And I include Peter Gutmann in that bracket too; find me a single example where one of his claims has been substantiated with actual hands-on testing.

      It's awesome; kinda like you build up an almost convincing argument, only to smash it down again in the next paragraph.

      --
      throw new NoSignatureException();
  7. Better than Vista, still worse than XP by Animats · · Score: 4, Insightful

    First, go to the real story, bypassing an intermediate blog and two interstitial ads.

    Second, the article says the performance of the newer OSs is worse than XP. "In fact, during extensive multiprocess benchmark testing, Windows 7 essentially mirrored Vista in almost every scenario. Database tasks? Roughly 118 percent slower than XP on dual-core (Vista was 92 percent slower) and 19 percent slower than XP on quad-core (identical to Vista). Workflow? A respectable 38 percent slower than XP on dual-core (Vista was 98 percent slower) and 59 percent slower on quad-core (Vista was 66 percent slower)."

    Third, there are no tables or graphs anywhere in these articles, and very few numbers. As a benchmarking article, this is awful.

  8. Why not tweak the XP kernel? by wonkavader · · Score: 2, Insightful

    It's great that MS was able to tune the Vista kernel to avoid locks which reduce performance on multiple cores, but I'd rather see the same work done for XP, giving us something MUCH faster on a high number of cores, rather than a pig we can compensate for with many cores.

  9. More great mathematics by Lonewolf666 · · Score: 4, Insightful

    From page 2 of TFA:

    In terms of raw application throughput, Windows XP clearly is still king of the hill. However, despite its current edge on dual-core and quad-core systems, Microsoft's 8-year-old OS is beginning to show its age. For example, when you contrast the dual- and quad-core transaction times for the ADO (database) and MAPI (workflow) workloads, you see that scalability -- in terms of a percentage improvement from dual-core to quad-core -- is capped at 265 percent for the database tasks and 32 percent for the messaging workflow tasks. While excellent by legacy Windows NT standards, these improvements pale next to the 571 percent boost witnessed for the same SQL-driven database workload under Windows 7, or the 58 percent improvement for the MAPI message store workflow task under Windows Vista.

    (emphasis mine)
    So we are supposed to believe that the database test on Windows 7 runs 571 percent faster on a quad-core compared to a dual core?
    That would be a factor of 6.71, or in terms of performance per core, a factor of 3.355. In other words, the quad core would do 3.355 times more work per core than the dual core. That sounds not very believable, considering similar tests the German C't magazine has done in the past (for Linux and Windows 2000). In those tests, both OS scaled at best linear with the number of CPUs, so the "performance boost" from going from dual to quad core was at best 100% (in most tests more like 80%).

    Maybe I'm misunderstanding what Randall C. Kennedy wanted to say. Here it would have helped if he posted his raw data and test configuration, as most reputable testers do. But as he only posted a few end results, I can only say that his numbers seem bogus. I rated the Infoworld article with 1 of 5 points.

    --
    C - the footgun of programming languages
  10. What about multi-function? by CannonballHead · · Score: 4, Insightful

    Nobody is going to argue that if you run one single application (a database something), a "small" OS will work better. There are Linux versions that are specifically geared towards doing that sort of thing, right? Ubuntu is probably slower at something than [insert other dist].

    The real question is, though... what about normal usage? Unfortunately, that's hard to measure... but how does Vista/Windows 7 affect normal user productivity and speed as opposed to simple benchmarks designed to test out efficiency at doing ONE thing?

    If Vista and Windows 7 were designed to have a lot of background processes to help the user do this or that, then why not test that, too? XP wasn't designed that way, apparently, while Windows Vista/7 are more designed that way. So give it a level playing field and test what it was designed to do.

    I don't have an answer of whether or not Vista/Win7 are slower or faster when doing other things (like, say, searching for a file because you can't remember where you put it, running multiple applications, using something DRM enabled, or whatever), but it'd be interesting to try to test it rather than a generic "XP runs a single application faster than Vista because Vista has more stuff running in the background." It'd be interesting to try to physically load the system with lots of applications and see which is better then.

  11. DRM Check by Gonoff · · Score: 1, Insightful

    How does the system 'know' when to start running the DRM? There must be something running at all times "just in case" the paying customer decides to excersise their right to play their own stuff.

    Whether it is a service, thread or whatever, it doesn't matter. Some system resources have to be used in advance. That can only drop the performance.

    --
    I'll see your Constitution and raise you a Queen.
    1. Re:DRM Check by TheThiefMaster · · Score: 2, Insightful

      How about "when the user presses the play button"?

      I'd expect it to be called from the codec's initialize function myself.

    2. Re:DRM Check by sexconker · · Score: 3, Insightful

      I don't know the details of Vista/7's implementations, but come on - just use your head.

      If the file header has the broadcast flag, or otherwise indicates the file is DRM crippled, the DRM checks begin.

      If no, then DRM checks aren't going to run.

      It's about the same level as:

      OH NO. My PC is automatically adjusting my clock for daylight saving time. MY FUCKING CLOCK CYCLES ARE BEING STOLEN!

    3. Re:DRM Check by makomk · · Score: 2, Insightful

      If no, then DRM checks aren't going to run.

      Well, the DRM checks aren't. The system integrity checks to prevent users bypassing the DRM, OTOH, still have to. For example, Windows Vista has code running to detect unauthorised changes to the kernel and reboot, even when no DRMed video is playing (officially to prevent developers from doing things that could affect system stability). I suspect there's overhead in the video card driver model too, to prevent stuff like a hacker setting up the card state so they have read access to the framebuffer via something apparently innocent. (It's much easier to put code in place to prevent hackers from changing the system state in a way that allows DRM to be bypassed, than it is to detect it reliably afterwards.)