Slashdot Mirror
AMD Rips 'Biased and Unreliable' Intel-Optimized SYSmark Benchmark (hothardware.com)
MojoKid writes: AMD is making a stink about SYSMark, a popular benchmarking program that's been around for many years, and one the chip designer says is not reliable. Rather than provide meaningful results and information, AMD claims SYSMark unfairly favors Intel products and puts too much emphasis on strict CPU performance above all else. John Hampton, director of AMD's client computing products, explained in a video why SYSMark itself is an unreliable metric of performance. He even brought up the "recent debacle" involving Volkswagen as proof that "information provided by even the most established organizations can be misleading." Salinas says SYSMark's focus on the CPU is so "excessive" that it's really only evaluating the processor, not the system as a whole. In comparison, PCMark 8 probes not only the CPU, but graphics and subsystems as well. In an attempt to drive the point home, AMD ran a set of custom scripts it developed based on Microsoft Office and timed how long it took each system to complete them. The Intel system took 61 seconds to finish the benchmark versus 64 seconds for the AMD platform, a difference of about 6-7 percent and in line with what PCMark 8 indicated, though Sysmark shows a stark delta of 50 percent in favor of Intel with comparable CPUs.
If you're buying an AMD processor, it's for price. If you're buying an Intel processor, it's for performance.
I just upgraded from an FX-8350 to an i7-5930k
night and fucking day
Hey, it is not cheating, it is not compliance checking, so it is not like lying like VAG or Bosch. I understand that this is a synthetic benchmark that favours Intel CPUs. However, there are a variety of benchmarks out there. For example, Oracle posted some benchmarks where Sparc CPUs run faster than Intel CPUs, for a given application. Curiously enough, Oracle did not bother benchmarking agains AMD CPUs.
Intel's a member of BAPco, the SYSmark organization, and AMD isn't.
https://bapco.com/about/
On the other hand, if it's really a big deal to AMD, they should be able to find $100K or so to join BAPco and tilt the deck in their favor - total annual budget only seems to be $400K.
http://www.faqs.org/tax-exempt...
Just maybe if AMD got off their butts and made unbiased reports and reliable/fast chips and graphics they would not be in this predicament.
http://saveie6.com/
SYSMark has been known to be not particularly representative of actual performance for quite some time. In particular, they seem weirdly sensitive to memory latency, way beyond its actual impact, yet they deliberately evade caching even in benchmarks measuring something where caching is normally useful. And they do seem to unreasonably penalize AMD chips, although I'm not sure if that's malice or simple incompetence.
The review sites I frequent tend to use PCMark for the general-overview synthetic benchmarks, along with some actual-program benchmarks (usually compression, crypto, and video encode). I of course prefer the latter - nobody runs synthetic benchmarks in production, it's always some actual application. The closer you can get to benchmarking that actual app, the better.
The real outrage should be that operations in Microsoft Office are measured in seconds and minutes instead of nanoseconds and milliseconds.
I agree with AMD that most of the market is slanted towards Intel.
But I don't want a benchmark score that is dictated by a graphics card and it's driver set. I want a cpu score that is based on CPU performance, only CPU performance and perhaps taking into account the effects of memory memory bandwidth. Plenty of tools on the internet for that they could have showed instead.
What I want from AMD is a cpu in the 150$~ range with a performance equal or exceeding my old overclockedi5-2500k (2011 vintage) and be capable of gnarly overclocking. If they can deliver that, they have me for an entire socket generation.
If I benchmark a common office task, like typing War and Peace into Microsoft Word, I find that these new processors run no faster than an old IBM PC/XT with a 4.77 MHz Intel 8088.
"National Security is the chief cause of national insecurity." - Celine's First Law
My work desktop is AMD, my home fileserver is AMD, and both my parent's desktops are AMD. That's because in those use cases, AMD is "good enough". Web browsing and email don't require a lot of horsepower.
That said, my gaming/transcoding PC is an Intel i5-4690, because AMD's top line CPU can barely compete with Intel's I3 line. CMT didn't pan out, and they've been held hostage by TSMC/GloFo's failure to produce a sub-28nm lithographic process.
I love AMD's engineers, they have some impressively smart people working for them, and I hope Zen + 16nm heralds a new beginning for them. But today, they aren't "competitive", merely "good enough".
Generally I've bought AMD on price-point for various machines, but I've also played a bit with their APU's in cases where space was more of a consideration and I didn't want to run a discreet card (and Intel's onboard graphics weren't very good).
They have different power schemas. And if you didn't think you'd ever use the processor fully utilized (as you mentioned), then why did you care about power consumption under load?
Any guest worker system is indistinguishable from indentured servitude.
I like AMD and all, but we're talking about a company that ships water coolers with their i7 competitors ( the 9xxx line). My brother's running his i7 with a stock fan... I'd love to be proven wrong but right now AMD just doesn't seem like they can hang. My A10-5800k is nice and all but in games it's about the equivalent of a mid range i3, but I can replace that i3 with a 5 or a 7... With the AMD the best I could do is an 8350...
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
Actually no. I buy AMD processors for *intregity*
I haven't bought an Intel processor since they said : If you can show us you *need* it you can have a fixed Pentium (without the FDIV bug).
https://en.wikipedia.org/wiki/...
Publicly, Intel acknowledged the floating-point flaw, but claimed that it was not serious and would not affect most users. Intel offered to replace processors to users who could prove that they were affected. However, although most independent estimates found the bug to be of little importance and would have negligible effect on most users, it caused a great public outcry.
So.. long memory. Many $'s lost. Don't piss off customers (or lie about it and when you have to fess up say : Whatever..)
someone was triggered..
So you saved what? $10/year on power consumption?
Honestly, most of these benchmarks have LONG outlived the point where they provide any sort of useful information for anything.
They're basically masturbatory devices whereby people who have nothing better to do flaunt their e-peen.
They try to tell you that they correlate back to specific tasks in the system. Unfortunately this has been so utterly abstracted by now that the synthetic counterpart is utterly meaningless.
The only thing that matters is "Does the system do what you need it to do at a reasonable speed?"
And the problem with codifying any sort of benchmarks at all is that, eventually, they get gamed.
So, rather than a more organic development with general performance gains, we get a stunted development process where components that matter to benchmarks get more tweaking, and performance in other areas suffers.
The main point is, most new computers are ridiculously overpowered for 99% of all desktop productivity tasks. And the remaining 1% of video editing, graphical editing and gaming tasks are simply hyper-specializations.
Chas - The one, the only.
THANK GOD!!!
Went up in flames due to a problem with after market coolers...
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
While an AMD chip can run x86-64 code compiled for an Intel processor, it isn't surprising that the code doesn't perform as well since a lot of optimisations relate to features of the specific chip. You can't use a precompiled binary across all chips and expect them to be useful other than to say one chip can run that binary quicker than another. I remember years back having some code that was optimised for the Intel PIII and when that same code ran on the AMD Opteron it was slower despite the Intel running at a clock speed of 1.4Ghz and the AMD running at 1.7. Once I went in and had a look at the ASM I could see why - the AMD had a 64 bit bus and the code was using instructions which weren't as efficient on AMD's chip as a result of this. Once I realised that, I rewrote that section of code to account for this and the AMD ended up being 30% quicker than the old code when I rewrote four lines of C. Compiler optimisations only go so far but you still have to be aware of the underlying chip if you really want to get the most out of it.
"I have the attention span of a strobe lit goldfish, please get to the point quickly!"
AMD announced the acquisition of ATI and a few days later Intel released the Core line of processors that destroyed anything AMD could produce.
Only the State obtains its revenue by coercion. - Murray Rothbard
limited PCI-e bandwidth they need to stack more off of the open HT link on CPU2 or at least up the link to chip set to HT3.
Haveing network / storage / other slots all of that one link can slow stuff down.
On lntel each cpus has it own pci-e 3.0 links.
AMD's HyperTransport bus limits PCI-3.0 and servers chips don't have there own pci-e links from the cpu.
MNL-H8QG(7)(i)-LN4F and other super micro boards does have 2 HT links from the cpus's to the pci-e.
When AMD was KING lntel locked them out of big parts of the market and that give intel time to get back on top. And now that they are on top they are ripping people off and lacking on pci-e lanes on sky lake and cutting the number of lanes on the on higher end chips unless pay more then the last gen when the lowest one had it all.
Sure.
However, there's pretty much zero evidence that synthetic benchmarks stress a system in the same way regular use does.
Chas - The one, the only.
THANK GOD!!!
Rule 1 (good to a first approximation): All benchmarks are meaningless.
Rule 2 (for experts only): Every benchmark measures something very specific. A benchmark is only meaningful if you know exactly what it is measuring, and the thing it measures is something you actually care about.
"I'm too busy to research this and form an educated opinion, but I do have time to tell everyone my uninformed opinion."
That want an i5 or better. Arkham Knight, Just Cause 3 and some of the Ubisoft games (Far Cry 4 et al) have troubles with lower end CPUs. There's strong evidence that the reason is that the problems are due to the newer DRM encrypting the entire game and decrypting it on the fly but I haven't heard a 100% confirmation of that from anyone.
Intel's next CPUs are suppose to have hardware support for decryption. I think you're going to get stuck with a powerful CPU at some point to run games for just that reason.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
First of all, I said for anything BUT gaming AMD is a better processor. And the main reason for that is that while the each individual core is slower, it's true 8 cores (vs Intel's 6 cores on 5930k or even 4 cores which was all Intel could do for you when AMD's 8350 came out). The only way to actually use performance this high is to run poorly designed algorithms. Otherwise, the IO is much more of a bottleneck then the processing. The main reason Intel is better for gaming is not that it has faster cores. It's asymmetric L1 cache. AMD has symmetric (same input and output amount of L1 cache). So for most things that people use computers for (multitasking rather than really fast performance of small-thread-count tasks) AMD is a smoother experience.
Any guest worker system is indistinguishable from indentured servitude.
Gee, I wonder why AMD waited so long to attack that evil evil Sysmark?
Your sentiment would be something more people could get behind if AMD hadn't been bitching about Sysmark for the best part of the last 10 years.
Check out this story:
http://arstechnica.com/gadgets...
I love how this got flagged as Flamebait and Troll.
Tl;dr I hope Zen makes a comeback, because Intel needs competition, and this will force them to make even better processors.
This post was completely sincere.
I've been an AMD fanboy for over 15 years and I still am to an extent even though I'm on an Intel Platform now.
My point was, 95% of use cases for processors will not benefit an Intel over an AMD chip, and yet people, myself included, will spend multiples more for an Intel chip, when the benefit is diminished.
If I cared about power comsumption, I would turn off a light for a couple hours a day, that would offset the processor power usage.
Have you ever fallen asleep at the keybhanusdiog?
Honestly, most of these benchmarks have LONG outlived the point where they provide any sort of useful information for anything.
I agree but so has CPU speed and number of cores but we need some way of comparing apples and oranges to decide which CPU a person should buy. I find AMD's approach interesting. A benchmark written in excel should be about as good as any for testing desktop work. Many video games like minecraft and even some first person games can be accessed via scripts so it would be simple enough to create benchmarks based on actual games too. This seems like the better approach. Build the benchmarks inside the actual games. Ideally you would have the benchmarks written by a trusted third party where the exact methods are not known by AMD/Intel so they can't be gamed but the second best solution would probably be to test on a half dozen or so games/apps and made it open source so that the multiple apps make it harder to game and the open source make it so that they can game it equally.
Okay, so AMD is in the business of manufacturing and selling CPUs. Along comes a tool to qualitatively analyze CPU performance. AMD doesn't like that. What are they really trying to say?
P. S. I'm fully aware that there are all kinds of backdoor deals and benchmark fudging in the market, but as other posters have noted, you want a CPU score based on the performance of the CPU.
C. Griffin
"Can I keep his head for a souvenir?" --Max from Sam 'N Max Freelance Police
Tom's Hardware occasionally comparison-tests equivalent AMD and Intel processors. Intel always comes out on top overall, and seldom loses more than a couple of individual tests.
My particular critical application is refocus routines in GIMP, and waiting 2 minutes for a single iteration is a nuisance. I doubt that AMD would be better.
Contribute to civilization: ari.aynrand.org/donate
First off does anyone use synthetic benchmarks anymore? I mean they were proven to be cheating like a decade ago. Anyway I'd agree with AMD that it isn't a good measure of performance.
Having said that, there are different types of performance for different things. However there are limits. So while generally speaking, saving some money on a CPU, and spending it elsewhere may make sense for gaming, there are limitations. I think generally speaking, you're wrong about AMD being a "better value" for gaming. Then again, it depends on your definition of "gaming" a bit also. The only chips AMD has had in years that are a "value" for money spent, are the *very* low end, none of which are suitable for gaming (other than of the very light variety). No you do not need an i7, but a mid-range i5 for example is better bang for buck than AMD offering. As to what you spend it on, I would agree you will see immediate gains with a SSD, however that "performance" is limited to load times, which is also governed by your CPU to a degree, so that does make sense. However a SSD will not do anything for quality, or speed. For that, the GPU is the big gun, however even it can be limited by your CPU. So buying a low end AMD, and tacking a med to high end GPU to it, may not be the best solution either. As for the memory, it likely isn't needed. Additional memory is really only used either for very specific applications, or if you need to run a large number of applications all at the same time, or if trying to future proof you system against possible increases in demand (which really hasn't manifested in recent years).
Intel has a substantial advantage with process technology, which is the long-term result of having more money to utilize. As time goes on, physics is reducing the advantage that can be gained with a better process. Some day that advantage will be so small that a better architecture will outweigh process advantage, and if AMD can live till then it stands a fair chance of matching Intel's performance.
Reforming copyright (and maybe patents) is a good idea, abolishing them is not. Authors that can make a fair living when paperbacks sell for $7.00 will leave the field when every published book is available from China at $1.00 a week after release.
Contribute to civilization: ari.aynrand.org/donate
If it matters, and it should not, then you've a piddling amount of money and will be broke within five years. I'd offer to give you some suggestions but your attitude indicates you'd not listen and that it would be a waste of my time. Also, I didn't inherit mine, I earned it by selling my company. I will give you to important tips. Hire two accountants and a lawyer before you do a damned thing. Also, look into the various types of corporations and trusts.
"So long and thanks for all the fish."
Not really, Intel beats AMD so hard in IPC that even with half the cores they still tend to pull better multithreaded performance. Just like how you could have a well-designed 4-cylinder engine that outperforms a poorly-designed 8-cylinder.
I tend to run multiple VM's. And running 4 VM's on a Win 7 machine means at least 5 concurrent cores are used (if all vm's are actually in run state and so is the host operating system). And I7 used to die (slow down to a complete crawl) with 5 vm's. AMD churns along with no problems.
Any guest worker system is indistinguishable from indentured servitude.
Not if your number of actively running processes is higher than the number of cores. That means more context switches. And that means full cache invalidation and mandatory memory access. If run a lot of VM's, and you want them to actually run, you don't want Intel. Well, they have a 10 core chip now, but that's comparing apples and oranges, if you compare the generation of chips when AMD had 8 full cores and Intel had 4 slightly faster cores, Intel was awful for running multiple vm's.
Any guest worker system is indistinguishable from indentured servitude.