Slashdot Mirror
AMD Rips 'Biased and Unreliable' Intel-Optimized SYSmark Benchmark (hothardware.com)
MojoKid writes: AMD is making a stink about SYSMark, a popular benchmarking program that's been around for many years, and one the chip designer says is not reliable. Rather than provide meaningful results and information, AMD claims SYSMark unfairly favors Intel products and puts too much emphasis on strict CPU performance above all else. John Hampton, director of AMD's client computing products, explained in a video why SYSMark itself is an unreliable metric of performance. He even brought up the "recent debacle" involving Volkswagen as proof that "information provided by even the most established organizations can be misleading." Salinas says SYSMark's focus on the CPU is so "excessive" that it's really only evaluating the processor, not the system as a whole. In comparison, PCMark 8 probes not only the CPU, but graphics and subsystems as well. In an attempt to drive the point home, AMD ran a set of custom scripts it developed based on Microsoft Office and timed how long it took each system to complete them. The Intel system took 61 seconds to finish the benchmark versus 64 seconds for the AMD platform, a difference of about 6-7 percent and in line with what PCMark 8 indicated, though Sysmark shows a stark delta of 50 percent in favor of Intel with comparable CPUs.
If you're buying an AMD processor, it's for price. If you're buying an Intel processor, it's for performance.
I just upgraded from an FX-8350 to an i7-5930k
night and fucking day
When did AMD become relevant again?
Hey, it is not cheating, it is not compliance checking, so it is not like lying like VAG or Bosch. I understand that this is a synthetic benchmark that favours Intel CPUs. However, there are a variety of benchmarks out there. For example, Oracle posted some benchmarks where Sparc CPUs run faster than Intel CPUs, for a given application. Curiously enough, Oracle did not bother benchmarking agains AMD CPUs.
Intel's a member of BAPco, the SYSmark organization, and AMD isn't.
https://bapco.com/about/
On the other hand, if it's really a big deal to AMD, they should be able to find $100K or so to join BAPco and tilt the deck in their favor - total annual budget only seems to be $400K.
http://www.faqs.org/tax-exempt...
Just maybe if AMD got off their butts and made unbiased reports and reliable/fast chips and graphics they would not be in this predicament.
http://saveie6.com/
There are lies, damned lies, and {statistics,benchmarks}. -PCP
Captcha: minimize
Hrmm... why is AMD's PR department* suddenly making a giant stink about Sysmark? They aren't due to launch any new chips until the end of the year at a bare minimum... right?
Well, maybe it's because a certain company (AMD) has run some internal benchmarks with a certain chip (Zen) and they aren't necessarily coming out with miraculous results.
So what do you do? Attack the benchmark of course!
Gee, I wonder why AMD waited so long to attack that evil evil Sysmark? Maybe it's because back when the Opterons were actually much stronger than Pentium-4 Xeons, AMD actually won Sysmark benchmarks and openly bragged about it.
Here's a 13-year old example of AMD bragging about SPEC, which AMD has also attacked as being an evil conspiracy (when they don't win that is): http://www.geek.com/chips/amd-...
* Wait to call him until after Tuesday since he has to mop the hallways on Tuesdays.
AntiFA: An abbreviation for Anti First Amendment.
SYSMark has been known to be not particularly representative of actual performance for quite some time. In particular, they seem weirdly sensitive to memory latency, way beyond its actual impact, yet they deliberately evade caching even in benchmarks measuring something where caching is normally useful. And they do seem to unreasonably penalize AMD chips, although I'm not sure if that's malice or simple incompetence.
The review sites I frequent tend to use PCMark for the general-overview synthetic benchmarks, along with some actual-program benchmarks (usually compression, crypto, and video encode). I of course prefer the latter - nobody runs synthetic benchmarks in production, it's always some actual application. The closer you can get to benchmarking that actual app, the better.
Ba dum tssss
The real outrage should be that operations in Microsoft Office are measured in seconds and minutes instead of nanoseconds and milliseconds.
lies .lt. damn lies .lt. statistics .lt. benchmarks .lt. manufacture's benchmarks
I agree with AMD that most of the market is slanted towards Intel.
But I don't want a benchmark score that is dictated by a graphics card and it's driver set. I want a cpu score that is based on CPU performance, only CPU performance and perhaps taking into account the effects of memory memory bandwidth. Plenty of tools on the internet for that they could have showed instead.
What I want from AMD is a cpu in the 150$~ range with a performance equal or exceeding my old overclockedi5-2500k (2011 vintage) and be capable of gnarly overclocking. If they can deliver that, they have me for an entire socket generation.
I really like my Fx-6300. Can beat it for $100. Thanks AMD!
If I benchmark a common office task, like typing War and Peace into Microsoft Word, I find that these new processors run no faster than an old IBM PC/XT with a 4.77 MHz Intel 8088.
"National Security is the chief cause of national insecurity." - Celine's First Law
Ya know, I just recently upgraded to a i5 4690k from an FX-4300 processor. Yeah it's faster, not much.....
Does it make Windows load any faster? No
Do any games player smoother or higher FPS? Very few.
Does it use less energy? Never checked, couldn't care less.
Did I spend alot more money to get this CPU? Yeah, over 2x as much as the AMD.
Is this post full on confirmation bias? Maybe.
Have you ever fallen asleep at the keybhanusdiog?
https://www.youtube.com/watch?v=BrUuKehpGuQ
When masturbating to internet porn it takes exactly 76 seconds to finish, on both Intel and AMD processors. Bambi from club Gushers only needs 42 seconds. Bambi outperforms both Intel and AMD by 45%
"Rather than provide meaningful results and information"
" AMD ran a set of custom scripts it developed based on Microsoft Office and timed how long it took each system to complete them. The Intel system took 61 seconds to finish the benchmark versus 64 seconds for the AMD platform, a difference of about 6-7 percent and in line with what PCMark 8 indicated, though Sysmark shows a stark delta of 50 percent in favor of Intel with comparable CPUs."
LOOKS LIKE THEY PROVIDED FUCKING RESULTS AND A METHODOLOGY PRETTY GODDAMNED SOUNDLY.
Hey, MojoKid, do you bother trying to copy-edi-OH WAIT you're just like the Slashdot editors.
I don't buy AMD processors because I want less power consumption (and therefore less heat output). A few years ago I literally based the decision not on cost or performance (because quite frankly with what I was doing I would NEVER see the difference), but on total wattage consumed under load. In my case, I wanted to use a 550w power supply. The intel processor and amd processor that were equivalent on performance differed by 30 watts of consumption. Intel was 120, AMD was 150. I needed the wattage for other things, and couldn't justify 30 watts of waste when the price was only different by about 80 bucks.......
I'm giving to those who were nice to me when it was clear I was just a nobody. On other words...not you.
My work desktop is AMD, my home fileserver is AMD, and both my parent's desktops are AMD. That's because in those use cases, AMD is "good enough". Web browsing and email don't require a lot of horsepower.
That said, my gaming/transcoding PC is an Intel i5-4690, because AMD's top line CPU can barely compete with Intel's I3 line. CMT didn't pan out, and they've been held hostage by TSMC/GloFo's failure to produce a sub-28nm lithographic process.
I love AMD's engineers, they have some impressively smart people working for them, and I hope Zen + 16nm heralds a new beginning for them. But today, they aren't "competitive", merely "good enough".
Intel is a stone giant. AMD are aphids clamoring up blades of grass to get a better view of the impending foot stomp.
Hey, I have been nice to every AC I ever ran across.
Generally I've bought AMD on price-point for various machines, but I've also played a bit with their APU's in cases where space was more of a consideration and I didn't want to run a discreet card (and Intel's onboard graphics weren't very good).
I like AMD and all, but we're talking about a company that ships water coolers with their i7 competitors ( the 9xxx line). My brother's running his i7 with a stock fan... I'd love to be proven wrong but right now AMD just doesn't seem like they can hang. My A10-5800k is nice and all but in games it's about the equivalent of a mid range i3, but I can replace that i3 with a 5 or a 7... With the AMD the best I could do is an 8350...
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
are slower than their competitors. Way to go AMD.
But, but, not *as* slow as they claimed...? Great argument.
AMD used to innovate and compete. They've been the lagging pony for years now. They either need to produce a better product or be the really cheap option--not just a little cheaper, but much cheaper. Else they will be on their way out of the market.
That's why you should give it to me. I *actually was* nice to you when you were just a nobody, but neither of us knew that other wasn't just a random internet stranger until you turn up to give me $1 million.
AMD's CPU's since bulldozer only achieve about half the IPS of even a first generation Corei CPU. What does AMD expect? AMD's performance per core is stuck in the Core2 era(~2006).
The answer is still no. $0 for you. Or any charity (including open sores).
Actually no. I buy AMD processors for *intregity*
I haven't bought an Intel processor since they said : If you can show us you *need* it you can have a fixed Pentium (without the FDIV bug).
https://en.wikipedia.org/wiki/...
Publicly, Intel acknowledged the floating-point flaw, but claimed that it was not serious and would not affect most users. Intel offered to replace processors to users who could prove that they were affected. However, although most independent estimates found the bug to be of little importance and would have negligible effect on most users, it caused a great public outcry.
So.. long memory. Many $'s lost. Don't piss off customers (or lie about it and when you have to fess up say : Whatever..)
I want you to know that I've always liked you. You're not like the other Anonymous Cowards, and I have long admired the cut of your jib.
Can I get you a beer? Seriously, you look thirsty.
You are welcome on my lawn.
Get fucking fucked you fucking fuck-faced fucker.
OP here. Beer makes one fat, and I don't know what a jib is. $0 for you.
That's preposterous! Nobody gets their hands on $100MM without a passing knowledge of sailing jargon. Next you'll claim to have never been on a fox hunt...
This is not new; Intel is known for doing this from some time. As proof. there is actually an utility to search executable files compiled with the Intel C++ Compiler and disable the CPU dispatcher in detected files on the hard drive addressing an issue that causes non Intel chipsets to run these programs slower.
"The compiler or library can make multiple versions of a piece of code, each optimized for a certain processor and instruction set, for example SSE2, SSE3, etc. The system includes a function that detects which type of CPU it is running on and chooses the optimal code path for that CPU. This is called a CPU dispatcher. However, the Intel CPU dispatcher does not only check which instruction set is supported by the CPU, it also checks the vendor ID string. If the vendor string says "GenuineIntel" then it uses the optimal code path. If the CPU is not from Intel then, in most cases, it will run the slowest possible version of the code, even if the CPU is fully compatible with a better version."
In other words, it is believed that Intel uses this to make their processor faster than the competition, mainly AMD.
There is no benchmark in the world that represents my use case, and that's not the job of a benchmark anyway. The purpose of a benchmark is to give an apples-to-apples comparison of performance between revisions of a product on a specific (and possibly meaningless) workload. You can take a bunch of benchmarks and try to evaluate all the relevant dimensions of performance but that might still be meaningless.
If my performance criteria includes Intel specific instructions then no amount of AMD benchmarking can change my mind - their product does not meet my requirements.
AMD is complaining that their oranges don't look as good as Intel's apples under benchmarking. Oranges are not apples. Duh.
Honestly, most of these benchmarks have LONG outlived the point where they provide any sort of useful information for anything.
They're basically masturbatory devices whereby people who have nothing better to do flaunt their e-peen.
They try to tell you that they correlate back to specific tasks in the system. Unfortunately this has been so utterly abstracted by now that the synthetic counterpart is utterly meaningless.
The only thing that matters is "Does the system do what you need it to do at a reasonable speed?"
And the problem with codifying any sort of benchmarks at all is that, eventually, they get gamed.
So, rather than a more organic development with general performance gains, we get a stunted development process where components that matter to benchmarks get more tweaking, and performance in other areas suffers.
The main point is, most new computers are ridiculously overpowered for 99% of all desktop productivity tasks. And the remaining 1% of video editing, graphical editing and gaming tasks are simply hyper-specializations.
Chas - The one, the only.
THANK GOD!!!
Went up in flames due to a problem with after market coolers...
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
They're trying to get Reddit to buy... They have been getting a lot of product placement here for over a year now. When it happens, we can say that Slashdot is truly dead. And yes, many of the accounts are suspicious. They have filled the journal section with pure spam. I figure they don't want any side stories drawing attention from the front page.
AMD, (like Cyrix) is a knockoff. It can never catch up to the people they're trying to imitate. Intel (like Nvidia) has an effective monopoly on high performance chips. Copyright and patents must be abolished so we can progress in a more natural fashion with real competition, without having to reinvent the wheel every time. In fact, everything about society needs a major overhaul. We need a system that imparts cooperation and respect, not coercion and dominance.
While an AMD chip can run x86-64 code compiled for an Intel processor, it isn't surprising that the code doesn't perform as well since a lot of optimisations relate to features of the specific chip. You can't use a precompiled binary across all chips and expect them to be useful other than to say one chip can run that binary quicker than another. I remember years back having some code that was optimised for the Intel PIII and when that same code ran on the AMD Opteron it was slower despite the Intel running at a clock speed of 1.4Ghz and the AMD running at 1.7. Once I went in and had a look at the ASM I could see why - the AMD had a 64 bit bus and the code was using instructions which weren't as efficient on AMD's chip as a result of this. Once I realised that, I rewrote that section of code to account for this and the AMD ended up being 30% quicker than the old code when I rewrote four lines of C. Compiler optimisations only go so far but you still have to be aware of the underlying chip if you really want to get the most out of it.
"I have the attention span of a strobe lit goldfish, please get to the point quickly!"
Using a few MS Office scripts is a good way to benchmark disk latency, but will tell you very little about CPU peformance, so of course there's almost no perceptible difference between AMD and Intel.. (I would say, however, a 2 sec differnece in this case is pretty telling already, if that's the best AMD could do.)
How about a benchmark where cpu is the bottleneck, like video or image processing? AMD falls far behind fast.
And let's not mention power usage like the AMD processor underforms.
AMD just needs to stop whining about everything being against them.
limited PCI-e bandwidth they need to stack more off of the open HT link on CPU2 or at least up the link to chip set to HT3.
Haveing network / storage / other slots all of that one link can slow stuff down.
On lntel each cpus has it own pci-e 3.0 links.
AMD's HyperTransport bus limits PCI-3.0 and servers chips don't have there own pci-e links from the cpu.
MNL-H8QG(7)(i)-LN4F and other super micro boards does have 2 HT links from the cpus's to the pci-e.
When AMD was KING lntel locked them out of big parts of the market and that give intel time to get back on top. And now that they are on top they are ripping people off and lacking on pci-e lanes on sky lake and cutting the number of lanes on the on higher end chips unless pay more then the last gen when the lowest one had it all.
I've always been nice to the AC. I frequently post about how AC posts at 0 or -1 are the best posts, while the +4 or +5 are usually karma whoring trash.
A lot of stuff still require CPU or GPU performance...
is missing the point of using log values in big O notation.
Sure.
However, there's pretty much zero evidence that synthetic benchmarks stress a system in the same way regular use does.
Chas - The one, the only.
THANK GOD!!!
Rule 1 (good to a first approximation): All benchmarks are meaningless.
Rule 2 (for experts only): Every benchmark measures something very specific. A benchmark is only meaningful if you know exactly what it is measuring, and the thing it measures is something you actually care about.
"I'm too busy to research this and form an educated opinion, but I do have time to tell everyone my uninformed opinion."
That want an i5 or better. Arkham Knight, Just Cause 3 and some of the Ubisoft games (Far Cry 4 et al) have troubles with lower end CPUs. There's strong evidence that the reason is that the problems are due to the newer DRM encrypting the entire game and decrypting it on the fly but I haven't heard a 100% confirmation of that from anyone.
Intel's next CPUs are suppose to have hardware support for decryption. I think you're going to get stuck with a powerful CPU at some point to run games for just that reason.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
...and PCMark 8 that AMD uses in their comparison says the same thing. And it actually uses tests that simulate real use (browsing the web, writing text etc.) and results from those tests tend to be "even a cheap processor is almost as fast". Yes, in the overall score a faster processor helps you gain some benefit, in some of the tests where absolute CPU power helps slightly, but the difference is marginal - just like it is in the real world use.
Then it comes down to "pay twice the price for 10% more practical performance"? Sure, that twice-the-price might get you a chip that has substantially higher performance if CPU is totally maxed out, but if 95% of your system use is word, excel and web browsing, why would you care?
In most cases investing extra $100 to the CPU matters way less than investing that to a good SSD vs having a dinky hard disk.
I spat my milk out.
1. Of course a CPU benchmark will favor Intel vs AMD as compared to a system-wide benchmark. The Intel CPUs are faster.
2. Power consumption matters for overall cost. 30 watts difference x 24/365 is $30 or more per year in California. I'm still running an i3 2100 24/7 in my NAS.
3. Reliability matters. AMD is a crap shoot. Intel hardware is much less so, especially since you can just buy an Intel motherboard and be done with it.
4. Compatibility matters. Again, Intel wins.
After the poor performance, awful reliability, and horrid compatibility of my last AMD system, I'll never buy another one. Fool me once, shame on you.
Check out this story:
http://arstechnica.com/gadgets...
Honestly, most of these benchmarks have LONG outlived the point where they provide any sort of useful information for anything.
I agree but so has CPU speed and number of cores but we need some way of comparing apples and oranges to decide which CPU a person should buy. I find AMD's approach interesting. A benchmark written in excel should be about as good as any for testing desktop work. Many video games like minecraft and even some first person games can be accessed via scripts so it would be simple enough to create benchmarks based on actual games too. This seems like the better approach. Build the benchmarks inside the actual games. Ideally you would have the benchmarks written by a trusted third party where the exact methods are not known by AMD/Intel so they can't be gamed but the second best solution would probably be to test on a half dozen or so games/apps and made it open source so that the multiple apps make it harder to game and the open source make it so that they can game it equally.
Okay, so AMD is in the business of manufacturing and selling CPUs. Along comes a tool to qualitatively analyze CPU performance. AMD doesn't like that. What are they really trying to say?
P. S. I'm fully aware that there are all kinds of backdoor deals and benchmark fudging in the market, but as other posters have noted, you want a CPU score based on the performance of the CPU.
C. Griffin
"Can I keep his head for a souvenir?" --Max from Sam 'N Max Freelance Police
First off does anyone use synthetic benchmarks anymore? I mean they were proven to be cheating like a decade ago. Anyway I'd agree with AMD that it isn't a good measure of performance.
Having said that, there are different types of performance for different things. However there are limits. So while generally speaking, saving some money on a CPU, and spending it elsewhere may make sense for gaming, there are limitations. I think generally speaking, you're wrong about AMD being a "better value" for gaming. Then again, it depends on your definition of "gaming" a bit also. The only chips AMD has had in years that are a "value" for money spent, are the *very* low end, none of which are suitable for gaming (other than of the very light variety). No you do not need an i7, but a mid-range i5 for example is better bang for buck than AMD offering. As to what you spend it on, I would agree you will see immediate gains with a SSD, however that "performance" is limited to load times, which is also governed by your CPU to a degree, so that does make sense. However a SSD will not do anything for quality, or speed. For that, the GPU is the big gun, however even it can be limited by your CPU. So buying a low end AMD, and tacking a med to high end GPU to it, may not be the best solution either. As for the memory, it likely isn't needed. Additional memory is really only used either for very specific applications, or if you need to run a large number of applications all at the same time, or if trying to future proof you system against possible increases in demand (which really hasn't manifested in recent years).
It is a shame that the 68000 architecture didn't propagate-around to hang all those 8080 descendants. Imagine how much better performance could be - on ANY benchmark!
If it matters, and it should not, then you've a piddling amount of money and will be broke within five years. I'd offer to give you some suggestions but your attitude indicates you'd not listen and that it would be a waste of my time. Also, I didn't inherit mine, I earned it by selling my company. I will give you to important tips. Hire two accountants and a lawyer before you do a damned thing. Also, look into the various types of corporations and trusts.
"So long and thanks for all the fish."