Slashdot Mirror
AMD Rejects SYSmark Benchmark
Deathspawner writes "In an unusual move, Advanced Micro Devices has issued a press release rejecting its endorsement for the industry recognized benchmark SYSmark 2012. Developed by BAPCo and backed by industry heavyweights such as Dell, Intel and Hewlett-Packard, AMD has stated that BAPCo both has tuned SYSmark to create bias in favor of its competitor, and that its benchmarks are not relevant for the audience it targets. Also noted is a complete lack of heterogeneous CPU+GPU testing. Techgage tears apart AMD's claims to see if they are valid, while also evaluating the overall usefulness of SYSmark and the impact it can have on consumers."
Nvidia and Via quit too.
for your convenience, here you go: http://techgage.com/print/amd_rejects_bapcos_sysmark_2012_-_should_we
-- Flame me and I will happily flame you back. Bring it!
This would've been avoided if these "industry standard" tools were released as open source. It would be interesting to see if such a development will arise from this dispute.
It seems like AMDs biggest complaint is that the benchmark isn't offloading CPU intensive tasks to the GPU. It is pretty hard to take them seriously when they are complaining that the benchmark favors their competition by actually benchmarking the CPU.
Quite a bit of Windows software is compiled using Intel's compilers, and they are intentionally made to sabotage performance on AMD chips. When looking at CPUID, instead of checking the features they want, they look for that _and_ the CPU being "GenuineIntel", and if not, the code chooses the worst possible implementation. This includes some major scientific math libraries and a part of popular benchmarks.
The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
AMD has chosen an architectural roadmap that makes the GPU and CPU part of the same APU. SYSmark does not measure 3-D graphics performance. At all. So while AMD is pursuing a path that will give its APUs greater overall performance than the CPUs they contain, they are actually hamstringing themselves in the CPU-only testing arena, because the CPU portion of thier APUs will seem relatively lower in performance at the same price point.
AMD's proper course of action should have been to promote an APU-specific benchmark. Instead, it tried to change SYSmark to do something it doesn't do.
It was denied the right to twist the benchmark in its favor. Rather than coming up with the obvious solution of spinning off a new benchmark consortium to develop an APU-specific test, it started crying and ran to its room shouting, "I hate you! I hate you! I hate you!"
AMD is, really, behind a major 8-ball right here. It has, again, put all of its eggs into a rather hopeful basket, and come up with fewer than expected. At least this time, unlike with the Barcelona debacle, it isn't doing it while roller-skating blindfolded through a car-wash. That time it cost them their fabs. They don't have much left to sell.
It's little wonder that it's not having an easy time of finding a new CEO.
A benchmark that doesn't test realistic workloads is of little use for evaluation if a system is fit to purpose.
A benchmark that isn't open about its methodology is at best worthless and at worst directly misleading.
I think they are pissy because they dont stand up well to the competition.
I think the complaints sound quite reasonable on the face of it. If it is true that Nvidia and VIA have also resigned, that just leaves Intel waving their cocks around at any rate.
Um, they do have their own compiler: http://developer.amd.com/tools/open64/Pages/default.aspx
Seems to be both free to download, and comes with source code so you can go over it if you wish.
Okay, but SYSmark isn't a CPU benchmark.
From BAPco's SYSmark page:
As stated by the GP, there are CPU benchmarks such as SPEC's. But SYSmark isn't one and AMD alleges it isn't designed to benchmark what they say they're benchmarking.
I don't expect morality, equality, consistency, or justice from the law. I expect only legality.
Intel, you mean the same Intel that got caught paying integrators not to use AMD chips?
It's a pretty gross mis-characterization to suggest that criticism of the size of Intel is based upon size rather than how they got to be so big.
I'm not sure you understand the point of benchmarking.
You can benchmark a lot of stuff. But its pointless to benchmark HD read/write speeds when what you're interested in is FLOPS. So there are benchmarks which see how many floating point ops your system can do. And there are benchmarks about hard drive performance. But there tends not to be one benchmark to rule them all.
BAPco say that SYSmark is a benchmark for real-world business app performance. But AMD say SYSmark doesn't utilize the GPU in any way.
Given that modern operating systems go to GPUs for rendering, which they're good at, and freeing up CPU time for CPU stuff, SYSmark isn't benchmarking what they claim to be benchmarking.
Which would make SYSmark a pointless benchmark.
I don't expect morality, equality, consistency, or justice from the law. I expect only legality.
...and make a compiler.
They did. It's even GPL licensed.
http://developer.amd.com/tools/open64/Pages/default.aspx
upon the advice of my lawyer, i have no sig at this time
I don't care about SYSmark telling me whether any given Intel CPU is better than any given AMD CPU or vice versa. What I really care about is finding out if the newly released Intel/AMD [insert arbitrary name here] CPU/GPU is truly better than the old Intel/AMD [insert arbitrary name here] CPU/GPU. By the time I get to the point of looking into concrete numbers from benchmarks, I've already decided whether I'm going to get an AMD or an Intel processor. The real problem that I have with all this benchmarking crap is why these manufacturers don't just provide us with coherent naming schemes for their CPU's (and GPU's too) so we as customers can fully understand the product they're trying to sell us.