ZDNet Says AMD Posts Blatantly Deceptive Benchmark

Glasswire writes "George Ou, writing in ZDNet's Real World IT blog, accuses AMD of comparing processors the company will not be shipping for months (2.6GHz Barcelona quad core) with older Intel Xeon quad cores rather than currently shipping ones which would beat the (hypothetical) score AMD claims for the future Barcelona. I guess while even the much slower 2.0GHz Barcelona is due soon AMD didn't think results from the 2.0 would look good enough — even against the slower Xeons they picked. Maybe the right comparison should be either best cpu against best cpu — or compare ones at the same price — and only shipped products."

Comments

Who trusts a vendor's benchmarks anyway?

[nweaver]

Vendor benchmarks are always considered untrustworthy, so I don't see what the big deal is.

Re:Who trusts a vendor's benchmarks anyway?

[PFI_Optix]

If we don't point out every time they use blatantly unfair product comparisons, the amount of disinformation coming out of vendors will only increase. Even though very few people (just the fanboys) place any stock in AMD's or Intel's benchmarks, it's worth pointing out flaws like this to keep them as honest as we possibly can.

Re:Who trusts a vendor's benchmarks anyway?

[CatsupBoy]

If we don't point out every time they use blatantly unfair product comparisons, the amount of disinformation coming out of vendors will only increase

Or the amount of crap product comparisons will continue to be the same no matter how much its pointed out.

Companies will continue to tout themselves as top dogs, regardless of the facts. And it never ceases to amaze me how far they go to stretch the truth in order to make themselves look good.

How else could salesmen go into a room and pitch their product? Or how can manufacturers sell their AMD products when competitors are pushing Intel? And vice versa? Its capitalism at its best.

Re:Who trusts a vendor's benchmarks anyway?

[mgoheen]

> Vendor benchmarks are always considered untrustworthy, so I don't see what the big deal is.

That logic gets you into trouble...

Politician promises are always considered untrustworthy, so I don't see what the big deal is.

Auto companies are untrustworthy, so you should expect the brakes to fail.

People are untrustworthy, so if you are robbed, it's your fault for carrying cash.

People are killed every day, so I don't see what the big deal with Iraq is.

etc.

Sheesh...wrong is wrong, no matter who is doing it. If you don't fight it, you're part of the problem.

Re:Who trusts a vendor's benchmarks anyway?

[arth1]

Still, the submission misses the point completely when it want benchmarks of only CPUs that's on the market. The reason for the benchmarks of CPUs that haven't been released yet is so OEMs and retailers know a little more of what to expect, and make plans for ordering (or not ordering) accordingly. If there's no benchmarks of unreleased CPUs, it would not hit the market, and thus wouldn't be benchmarked -- catch 22.

Who the manufacturer compares against is of course up to them, and there's nothing "unfair" about it. It's telling the world that this is the competition they strive to beat. If it's an older CPU, the new CPU is obviously intended as a replacement for these. If I had a large server farm running these Xeons, I'd be most interested to see this benchmark, well before the CPUs actually come out (if they're already out on the market, they will be off the market by the time upper management approves the budget). And remember, AMD and Intel aren't in the game to try to trick you to buy a CPU that won't work well for you -- they want you to return for your CPU needs, over and over again. That's why they publish benchmarks like these, which are relevant, just not to the GP.

Other comparisons both will and do appear once a CPU has hit the market. But for the initial pre-release vendor benchmarks, I'd rather it be the choice of the vendor, so we can see where the market position is going to be.

Move along -- nothing to see here, except for a particularly silly submission.

Re:Who trusts a vendor's benchmarks anyway?

[PFI_Optix]

Or the amount of crap product comparisons will continue to be the same no matter how much its pointed out.

You don't think it can get worse? You don't think it would get worse if there weren't people crying foul at the current comparisons?

You can use legitimate comparisons to tout a product, you don't have to unfairly match them. Look at your average car commercial (fictional example):

Ford's new truck gets better gas mileage than Dodge.
Ford's new truck has a bigger, more powerful engine than Chevy.

They just said it's better than Dodge and Chevy, but in two completely different ways. They do this all the time in marketing. If nothing else, AMD could talk up price points and power efficiency, two things they almost always have over Intel. Skewed benchmarks just make the company look inept and leave knowledgeable consumers feeling like AMD is insulting their intelligence.

Re:Who trusts a vendor's benchmarks anyway?

[Anonymous Coward]

If you read the comments on TFA

You must be new here. Welcome to /.

Re:Who trusts a vendor's benchmarks anyway?

[arth1]

If someone is lying, could you please care to name a lie?
Cause I can't see a single lie. Self-flattery, yes, and selective truths, yes, but no lies.

If you're in the business (and if you're not, this type of benchmark isn't meant for you), you know very well how to read and interpret the reported benchmarks and marketese. It's the expected format, which is helpful to those who need to know these things, e.g. because they are planning on upgrading a large Xeon farm to faster CPUs at as low cost as possible, or because they're a large OEM who needs to know the market segment this CPU is intended for, so they know both how much to order and how to market it.

Can we all stop this lynch mob now?

Re:Who trusts a vendor's benchmarks anyway?

[cbreaker]

And, not ONLY all that, all of these enthusiast sites continually post overclocked benchmarks for these CPU's.

They used to do it with the Pentium 4 all the time; You'd see a currently available Athlon versus a currently available Pentium 4 in a bechmark chart, and next to it would be a 60% overclocked P4 that requires special cooling. Yet they'd always say "BUT The OverClocked one BLOWS AMD AWAY!"

Just because this is coming from a manufacturer doesn't make it any less valid, and I don't see why AMD has to go hunting for Intel's latest CPU with the same model number (but a different revision) just to keep things fair OUT of their favor.

Besides, all this SPECint and CPU benchmark crap is worthless anyways, unless all you do with your server is run scientific calculations. In real world SMP applications, such as heavy-use VMware servers or database servers with lots of I/O and RAM, the Opterons will always kick the crap out of the Intel boxes with the Northbridge bottleneck. HyperTransport is the key to actually USING all of those system resources.

Re:Who trusts a vendor's benchmarks anyway?

[arth1]

Comparing a product that I (may) produce, 4 months from now, to one that someone else, did produce, 4 months ago, in a rapidly changing market, to imply that "My chips are better than their chips" is lying.

Really? You have a very strange definition of "lying", then. I think it shows how good a replacement the new CPU would be compared to the older one, but what do I know?

To say "well, they aren't saying anything which isn't true, my future chips do outperform their former chips" is disingenuous. It may be, literally true, but the implications of that graph are misleading and therefore a lie.

How is it misleading? It's a very good indicator on whether the future CPU would be a good replacement for the old CPU, and that is useful information to many -- both large companies and OEMs. The only misleading here seems to be people misleading themselves into thinking the benchmark is for a different purpose than it is.

This isn't helpful to some buyer, as it doens't compare two comparable items. My not yet released chips, are not comparable to their old chips. That you suggest something else, makes me wonder who you are writing for.

Me, myself and I. I refuse to join a lynch mob without thinking things through first.

It's not the deceptive benchmarks that bother me

[realmolo]

What REALLY upsets me is the fact that the writers at ZDNet actually get *paid* to regurgitate data they likely found on some other website via Google.

What a great job.

Let's all scream FIRE!

[nick_davison]

From the comments on the original article:

The graphs are from a several months old marketing promo. Suddenly there's really no story.

Claim: AMD listed a product they don't intend to release.
Truth: AMD listed a product they intended to release at the time but subsequently withdrew.

Claim: AMD deliberately used out of date Intel scores.
Truth: AMD used the most current Intel scores available at the time. Improved scores came from an improved compiler - which may well change AMD's scores too. Either way, it wasn't available at the time of writing.

Claim: AMD ignored the most recent Intel processor releases.
Truth: Those Intel processors weren't released at the time of writing and no benchmarks existed.

Journalistically, this is about on a par with finding footage from the 50's saying we'd all be driving flying cars by the year 2000 and boldly asserting there's clearly a government conspiracy to hide the technology from the people to protect big oil.

Bold claims are one thing. Making them on the back of badly researching where the information came from is a great way to look like an idiot.

Re:Not the architecture..

[TheRaven64]

Intel's Core 2 architecture is significantly better than AMD's current or past (and seemingly future) architectures In some ways, yes. The micro-op fusion stuff is incredibly shiny. They took some good branch prediction logic from NetBurst, and have a lot of neat tricks internally, particularly in the cache controller. On the edge of the CPU, AMD have the lead. They have a better interconnect (they are going to lose this lead soon, once Intel get CSI out of the door), and they have more intelligent memory controllers, which give them the edge in virtualisation and a few other things.

It's not entirely fair to say Intel is ahead of AMD architecturally. Both architectures have their strengths and weaknesses. At the moment, Intel are getting better overall performance (which means performance per Watt these days), but their architecture does have a few issues.

Re:Not the architecture..

[Ambassador Kosh]

I definitely don't agree that the intel systems scale vastly better. Most of the 4+ way benchmarks I have see with 8 or more cores go to amd pretty handily, The more memory the benchmarks need to use the worse off it gets for intel. So for desktops and very small servers where IO is not very important Intel is currently ahead in pure performance. If you need to setup an 8 core db server with 32GB of ram I would definitely go with opterons.

AMD is definitely not losing on the higher end server stuff, they are losing on the gaming desktops though since the Core 2 is a faster chip. For business work you pretty much never need something very fast. Probably the 3600+ is overkill for just about any business task and it currently as the best value of any chip I know of.

Perhaps

[Colin Smith]

They are benchmarking against Xeons because they are going to price them at the older Xeon's level rather than the newer faster faster ones...