Slashdot Mirror
AMD Rejects SYSmark Benchmark
Deathspawner writes "In an unusual move, Advanced Micro Devices has issued a press release rejecting its endorsement for the industry recognized benchmark SYSmark 2012. Developed by BAPCo and backed by industry heavyweights such as Dell, Intel and Hewlett-Packard, AMD has stated that BAPCo both has tuned SYSmark to create bias in favor of its competitor, and that its benchmarks are not relevant for the audience it targets. Also noted is a complete lack of heterogeneous CPU+GPU testing. Techgage tears apart AMD's claims to see if they are valid, while also evaluating the overall usefulness of SYSmark and the impact it can have on consumers."
Nvidia and Via quit too.
for your convenience, here you go: http://techgage.com/print/amd_rejects_bapcos_sysmark_2012_-_should_we
-- Flame me and I will happily flame you back. Bring it!
This would've been avoided if these "industry standard" tools were released as open source. It would be interesting to see if such a development will arise from this dispute.
It seems like AMDs biggest complaint is that the benchmark isn't offloading CPU intensive tasks to the GPU. It is pretty hard to take them seriously when they are complaining that the benchmark favors their competition by actually benchmarking the CPU.
How they never take the ability to deal with threads into account.
This is one area where things becomes problematic for Intel. This is because of hyperthreading shaes some important resources, such as the SSE related stuff. There for if you are running lots of threads making use of features with these issues, then you will actually run into notably reduced performance as the threads compete for the same resources.
This can also be a increased problem with some optimization options with GCC as it will make heavier use of SSE/etc.
AMD's largest complaint is that SM2012 doesn't represent the market well enough, employing high-end workloads that the regular consumer doesn't care about
As far as i know benchmarking is about pushing the tech to its limits to see what it can do or at least how it does against a standard heavy load. Your average user whose surfing the web, checking email, playing/streaming media or playing games probably isnt going to stress new hardware all that much. Aside from gaming, hardware from 5 years ago is still up to that task.
I think they are pissy because they dont stand up well to the competition.
Quite a bit of Windows software is compiled using Intel's compilers, and they are intentionally made to sabotage performance on AMD chips. When looking at CPUID, instead of checking the features they want, they look for that _and_ the CPU being "GenuineIntel", and if not, the code chooses the worst possible implementation. This includes some major scientific math libraries and a part of popular benchmarks.
The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
AMD has chosen an architectural roadmap that makes the GPU and CPU part of the same APU. SYSmark does not measure 3-D graphics performance. At all. So while AMD is pursuing a path that will give its APUs greater overall performance than the CPUs they contain, they are actually hamstringing themselves in the CPU-only testing arena, because the CPU portion of thier APUs will seem relatively lower in performance at the same price point.
AMD's proper course of action should have been to promote an APU-specific benchmark. Instead, it tried to change SYSmark to do something it doesn't do.
It was denied the right to twist the benchmark in its favor. Rather than coming up with the obvious solution of spinning off a new benchmark consortium to develop an APU-specific test, it started crying and ran to its room shouting, "I hate you! I hate you! I hate you!"
AMD is, really, behind a major 8-ball right here. It has, again, put all of its eggs into a rather hopeful basket, and come up with fewer than expected. At least this time, unlike with the Barcelona debacle, it isn't doing it while roller-skating blindfolded through a car-wash. That time it cost them their fabs. They don't have much left to sell.
It's little wonder that it's not having an easy time of finding a new CEO.
Fanboy fight!
Will it be AMD, the plucky underdog who always does what's best by the consumer vs Intel, the evil conglomerate who will stop at nothing to screw you over for profit?
Or will it be Intel, who are trying their best in the face of constant criticism simply for being number one vs AMD, who are just bitter about the fact that they've been playing catch-up ever since the Core2s were released?
Let's watch!
AMD is certainly correct in that most benchmarks these days are optimized for Intel cpus. That has been known for years. However, they are also wrong because Intel's SandyBridge architecture blows away AMD's phenom II architecture by 30% or better while also using considerably less power, in tests with the more cpu-neutral GCC.
As far as I can tell it basically comes down to main memory management. The Phenom II architecture can run a system call a lot faster than an Intel I7, but the moment data has to be pulled from main memory the whole thing comes grinding to a stop and Intel beats the crap out of AMD. SandyBridge also has much better bulk memory access for data sets exceeding available caches (I7 against Phenom II again, with the same DDR3 memory speeds).
Intel is able to charge a significant premium for their I7, $100 or more, verses AMD's high-end Phenom II's, and the difference pays for itself in power savings (for a server installation) in less than 2 years so it's a real problem for AMD. In previous years upgrading an AMD system was a lot cheaper due to its better socket compatibility but that isn't going to be the case for AMD's next generation. There are very few AM3+ mobos available at the moment and those that are available are wired for compatibility, meaning they can't make full use of the AM3 cpus that will be coming down the line. So at the moment there's no reason to stick with AMD.
In anycase, a lot of these benchmarks are basically designed to break AMD's cache architecture and yet still work within Intel's, which makes the numbers look even worse. But despite this AMD doesn't really have a leg to stand on right now. They had an executive shakeup and the executives who wanted AMD to compete on the high-end lost. That possibly spells the end for AMD's competitiveness because their lead in the integrated GPU arena is not all that large. Just today Intel introduced a 50-core (mini intel cpus basically) coprocessor to compete against nvidia's gpu architecture, and other things too. Intel can catch up without much effort in my view.
-Matt
...and make a compiler.
The reason Intel's compiler gets used so much is because it consistently generates the fastest code, period, when run on Intel processors (which are by far the majority). You see compiler shootouts and among others it goes back and forth what is faster at what, then at the top is the ICC, it just kills all the others.
Well, maybe AMD should do something about that. Maybe they should make their own, competing, compiler. Make it generate optimized code for ALL chips. Hell give it away for free (the ICC is not cheap).
Companies would have incentive to use it. Free, generates highly optimized code for all CPUs (even if its Intel code wasn't quite as good as the ICC), talk about a win.
However as it stands, AMD has nothing. So people can use VC++ or GCC or the like, which do an ok job, or they can use the ICC (which plugs right in to VS) and get the fastest code for Intel CPUs, including extremely good auto parallelization.
No surprise the ICC gets a lot of use.
We all know that benchmarks are designed to sell certain products based on who pays the benchmark company more. In the defense of AMD, the chips seem to be a more perfected design than intel. For instance my i7 notebook core utilization would be all over the place with certain cores utilizing more processor load than others while my AMD quad core notebook is even across the board at all times.
Reason is you are benchmarking, well, the CPU. You don't want the GPU messing with it. While the day may come when CPU and GPU fuse, that day is not now. So seems silly to bench a CPU with things that accelerate using a GPU.
Nothing wrong with benchmarking a GPU, or benchmarking things that use both, but you need to be clear about what you are testing. If the test is CPU, then that is what you want to restrict it to.
Hyperthreading doesn't share some things, it shares everything. It is the ability for a single core to run two threads at the same time in hardware. Intel isn't the only one who does things like that, Sun does, as does IBM (with more than two threads to a core). There are two benefits to this:
1) Less context switching penalty. It actually takes a fair amount of resources for an OS to switch from one thread to another. So run more threads in parallel in hardware, get more performance in heavily multi-threaded stuff. Servers in particular can benefit form this, hence why Sun and IBM do it with more than 2 threads per core.
2) Increased usage of the chips execution units. You discover that in almost all cases with a single thread on a core, come execution units sit unused some of the time. With two threads, the processor can better schedule things in to get all the units used more of the time.
Net effect is better performance for heavily threaded tasks. There is no real downside, other than additional chip complexity. It doesn't slow down or anything.
What you may be confusing is that it doesn't double performance. Going from 4 cores to 4 hyperthreaded cores, which does 8 threads does not double performance. At worst it remains the same, at best you see a low double digit percentage increase.
However it doesn't decrease performance. The only way that would happen is if you made the mistake of thinking a dual core, hyper threaded system was the same as a quad core, non-hyperthreaded system.
http://www.vanshardware.com/articles/2001/august/010814_Intel_SysMark/010814_Intel_SysMark.htm
http://www.vanshardware.com/reviews/2002/08/020821_AthlonXP2600/020821_AthlonXP2600.htm
http://www.vanshardware.com/reviews/2002/08/020822_AthlonXP2600/020822_AthlonXP2600.htm
http://www.vanshardware.com/reviews/2002/08/020822_AthlonXP2600/SYSmark%202002%20Analysis%20Presentation%20FINAL.pdf
I don't care about SYSmark telling me whether any given Intel CPU is better than any given AMD CPU or vice versa. What I really care about is finding out if the newly released Intel/AMD [insert arbitrary name here] CPU/GPU is truly better than the old Intel/AMD [insert arbitrary name here] CPU/GPU. By the time I get to the point of looking into concrete numbers from benchmarks, I've already decided whether I'm going to get an AMD or an Intel processor. The real problem that I have with all this benchmarking crap is why these manufacturers don't just provide us with coherent naming schemes for their CPU's (and GPU's too) so we as customers can fully understand the product they're trying to sell us.
Dude, that isn't a guess at the answer.
Go on, guess.
This is what I take away from RTFA. AMD thinks it's unfair to include GPGPU accelerated benchmarks because they know CUDA is better in every way (except openness), and its performance benefits are highlighted by sysmark. A more even test for the processor would be to use an nVidia GPU for both Intel and AMD benchmarks, but I'm willing to bet AMD favors the use of their (inferior) Radeon products. To that I say Boo-Hoo. Continue developing your Fusion APU and call me when it matures.