Slashdot Mirror
AMD Rips 'Biased and Unreliable' Intel-Optimized SYSmark Benchmark (hothardware.com)
MojoKid writes: AMD is making a stink about SYSMark, a popular benchmarking program that's been around for many years, and one the chip designer says is not reliable. Rather than provide meaningful results and information, AMD claims SYSMark unfairly favors Intel products and puts too much emphasis on strict CPU performance above all else. John Hampton, director of AMD's client computing products, explained in a video why SYSMark itself is an unreliable metric of performance. He even brought up the "recent debacle" involving Volkswagen as proof that "information provided by even the most established organizations can be misleading." Salinas says SYSMark's focus on the CPU is so "excessive" that it's really only evaluating the processor, not the system as a whole. In comparison, PCMark 8 probes not only the CPU, but graphics and subsystems as well. In an attempt to drive the point home, AMD ran a set of custom scripts it developed based on Microsoft Office and timed how long it took each system to complete them. The Intel system took 61 seconds to finish the benchmark versus 64 seconds for the AMD platform, a difference of about 6-7 percent and in line with what PCMark 8 indicated, though Sysmark shows a stark delta of 50 percent in favor of Intel with comparable CPUs.
If you're buying an AMD processor, it's for price. If you're buying an Intel processor, it's for performance.
Just maybe if AMD got off their butts and made unbiased reports and reliable/fast chips and graphics they would not be in this predicament.
http://saveie6.com/
SYSMark has been known to be not particularly representative of actual performance for quite some time. In particular, they seem weirdly sensitive to memory latency, way beyond its actual impact, yet they deliberately evade caching even in benchmarks measuring something where caching is normally useful. And they do seem to unreasonably penalize AMD chips, although I'm not sure if that's malice or simple incompetence.
The review sites I frequent tend to use PCMark for the general-overview synthetic benchmarks, along with some actual-program benchmarks (usually compression, crypto, and video encode). I of course prefer the latter - nobody runs synthetic benchmarks in production, it's always some actual application. The closer you can get to benchmarking that actual app, the better.
The real outrage should be that operations in Microsoft Office are measured in seconds and minutes instead of nanoseconds and milliseconds.
I agree with AMD that most of the market is slanted towards Intel.
But I don't want a benchmark score that is dictated by a graphics card and it's driver set. I want a cpu score that is based on CPU performance, only CPU performance and perhaps taking into account the effects of memory memory bandwidth. Plenty of tools on the internet for that they could have showed instead.
What I want from AMD is a cpu in the 150$~ range with a performance equal or exceeding my old overclockedi5-2500k (2011 vintage) and be capable of gnarly overclocking. If they can deliver that, they have me for an entire socket generation.
Having to pay money to remove corruption in an established organization is not the definition of ethical business in the first place.
That's called the old extortion/thug plan "pay up for protection".
BAPco deserves $0 from anyone, especially if they have a problem discriminating against people who don't give them omney.
My work desktop is AMD, my home fileserver is AMD, and both my parent's desktops are AMD. That's because in those use cases, AMD is "good enough". Web browsing and email don't require a lot of horsepower.
That said, my gaming/transcoding PC is an Intel i5-4690, because AMD's top line CPU can barely compete with Intel's I3 line. CMT didn't pan out, and they've been held hostage by TSMC/GloFo's failure to produce a sub-28nm lithographic process.
I love AMD's engineers, they have some impressively smart people working for them, and I hope Zen + 16nm heralds a new beginning for them. But today, they aren't "competitive", merely "good enough".
I like AMD and all, but we're talking about a company that ships water coolers with their i7 competitors ( the 9xxx line). My brother's running his i7 with a stock fan... I'd love to be proven wrong but right now AMD just doesn't seem like they can hang. My A10-5800k is nice and all but in games it's about the equivalent of a mid range i3, but I can replace that i3 with a 5 or a 7... With the AMD the best I could do is an 8350...
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
Well, yes, because each keypress in current versions of ms word triggers several megabytes worth of code to execute, probably inside a CLR (because somehow this makes everything better). 'Progress' is now defined by getting gigaflop spec cpus to mimic the same laggy performance we got from 286s back in the day.
While an AMD chip can run x86-64 code compiled for an Intel processor, it isn't surprising that the code doesn't perform as well since a lot of optimisations relate to features of the specific chip. You can't use a precompiled binary across all chips and expect them to be useful other than to say one chip can run that binary quicker than another. I remember years back having some code that was optimised for the Intel PIII and when that same code ran on the AMD Opteron it was slower despite the Intel running at a clock speed of 1.4Ghz and the AMD running at 1.7. Once I went in and had a look at the ASM I could see why - the AMD had a 64 bit bus and the code was using instructions which weren't as efficient on AMD's chip as a result of this. Once I realised that, I rewrote that section of code to account for this and the AMD ended up being 30% quicker than the old code when I rewrote four lines of C. Compiler optimisations only go so far but you still have to be aware of the underlying chip if you really want to get the most out of it.
"I have the attention span of a strobe lit goldfish, please get to the point quickly!"
Rule 1 (good to a first approximation): All benchmarks are meaningless.
Rule 2 (for experts only): Every benchmark measures something very specific. A benchmark is only meaningful if you know exactly what it is measuring, and the thing it measures is something you actually care about.
"I'm too busy to research this and form an educated opinion, but I do have time to tell everyone my uninformed opinion."