Slashdot Mirror
Commissioning Misleading Core i9-9900K Benchmarks (techspot.com)
On Monday, Intel unveiled the 9th Gen Core i9-9900K, which will rival AMD's Ryzen 2700X when it goes on sale in two weeks. We will soon be reading reviews of the 9th Gen Core i9-9900K, which Intel claims is the "world's best gaming processor," to see how exactly it fares against its AMD counterpart. But as reviewers test the new CPU and comply with an NDA/embargo (non-disclosure agreement) with Intel, which requires them to not share performance data of Intel's new CPU for another few days, surprisingly, one publication has already made a bold claim. In a story published this week, news outlet PCGamesN said, "Intel's Core i9 9900K is up to 50% faster than AMD's Ryzen 7 2700X in games." The publication cites data from an Intel-commissioned report [PDF] by third-party firm Principle Technologies to make the claim. TechSpot explains the issues with this: So Intel can go and publish their own "testing" done suspiciously through a third party ten days before reviews, while reviewers are prohibited from refuting the claims due to the NDA. First bad sign. Scrolling down PCGamesN says the following when looking over Intel's commissioned benchmarks. "But the real point of all this is for Intel to be able to hold out the 9900K as hands down the best gaming processor compared with the AMD competition, and in that it seems to have excelled. On some games, such as Civ 6 and PUBG, the performance delta isn't necessarily that great, but for the most part you're looking at between 30 and 50% higher frame rates from the 9900K versus the 2700X."
Right away many of the results looked very suspect to me, having spent countless hours benchmarking both the 2700X and 8700K, I have a good idea of how they compare in a wide range of titles and these results looked very off. Having spotted a few dodgy looking results my next thought was, why is PCGamesN publishing this misleading data and why aren't they not tearing the paid benchmark report apart? Do they simply not know better?
Over at the Principled Technologies website you can find the full report which states how they tested and the hardware used. Official memory speeds were used which isn't a particularly big deal, though they have gone out of their way to handicap Ryzen, or at the very least expose its weaknesses. Ryzen doesn't perform that well with fully populated memory DIMMs, two modules is optimal. However timings are also important and they used Corsair Vengeance memory without loading the extreme memory profile or XMP setting, instead they just set the memory frequency to 2933 and left the ridiculously loose default memory timings in place. These loose timings ensure compatibility so systems will boot up, but after that point you need to enable the memory profile. It's misleading to conduct benchmarks without executing this crucial step.
Right away many of the results looked very suspect to me, having spent countless hours benchmarking both the 2700X and 8700K, I have a good idea of how they compare in a wide range of titles and these results looked very off. Having spotted a few dodgy looking results my next thought was, why is PCGamesN publishing this misleading data and why aren't they not tearing the paid benchmark report apart? Do they simply not know better?
Over at the Principled Technologies website you can find the full report which states how they tested and the hardware used. Official memory speeds were used which isn't a particularly big deal, though they have gone out of their way to handicap Ryzen, or at the very least expose its weaknesses. Ryzen doesn't perform that well with fully populated memory DIMMs, two modules is optimal. However timings are also important and they used Corsair Vengeance memory without loading the extreme memory profile or XMP setting, instead they just set the memory frequency to 2933 and left the ridiculously loose default memory timings in place. These loose timings ensure compatibility so systems will boot up, but after that point you need to enable the memory profile. It's misleading to conduct benchmarks without executing this crucial step.
Desperation calls for desperate measures.
Speculation on Reddit about this seems to suggest they may have enabled the gaming profile* in Ryzen Master for games that don't benefit it (threadded / multi-core friendly games), and disabled it for those (single threadded dependent games).
For a multi-threadded suddenly loosing access to 4 cores, and for a single threadded game suddenly losing access to an additional 200MHz will give you some of those gimped benchmarks.
*For those who don't know, Gaming profile disables half the cores on a Ryzen 2, specifically targetting the poorest performing cores, and then raises the boost frequency thanks to the additional thermal / power headroom available. This is of great benefit to games that don't take advantage of multi-core processors.
Modern RAM settings need to be tweaked a bit or the performance is meh. The company did the tweaks on the Intel platform but not the AMD one.
They had to know they'd be called out by the benchmarking community. That and Youtubers hungry for video content.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
It's the second time I hear this company's name in less than a month. Some days ago I read a press release by Dell about a panoplia of new products, and the entire list, ranging from laptops to server computers was full of performance improvements (vs competitors) claims. all of them referring to paid-for reviews by this same company.
I personally find their motto - "win the attention war" - amusing. Also of interest is the fact (pun setup) they interchange links with their main domain and with a redirect from my country's TLD subdomain "facts.pt" (pun successful..?), as a subtle way to include their initials as something factual, and for the unsuspecting eye to believe it's a different company or to provide credit to their reviews with such a "reputable" subdomain. Genius stuff.
These companies are the audit companies of tangible products. Usually, you have Big Four conducting external audits for finantial institutions, country elections and whatnot, gathering data only these auditors are given access. The process is usually compulsory, but still paid by the targets of the audit, and there's always the sense the best auditors are usually the more positive. Now we get these paid product reviewers acting exactly the same way, getting paid to review products before they come out so companies can make bold claims. Then just NDA every other actually independant party interested in reviewing the product. See a pattern?
>>> will you discard it and build a new one to manufacture a new chip?
Generally, yes you will. A large part of the cost of the factory is the machines in it - and generally they all need to be replaced when you move from one process node to the next.
Let's say you spent $1B to build a 40 nm fab. You start building state-of-the-art wafers in it, and (being a good businessman) after a year are running it near 100% capacity. Next year, the bleeding edge wafers need to be at 32 nm. Your choices are:
1. Shut down production, spend 6 months retooling for 32 nm, then re-open. Cost: Six months of production plus new machines. End result: One fab producing 32nm wafers.
2. Keep the plant running, and build a new 32 nm fab. Cost: New fab with new machines. End Result: you still have the old fab cranking out 40nm wafers which everyone who doesn't need the bleeding edge will be buying for the next several years. In fact, you might have 55 nm, 40 nm, and 32 nm fabs all running in parallel. And those state-of-the-art 250 nm fabs from 20 years ago? Some are still running, putting out dirt-cheap wafers for people whose needs are met by low-performance, dirt-cheap ASICs.
And the worms ate into his brain.