ZDNet Says AMD Posts Blatantly Deceptive Benchmark

Glasswire writes "George Ou, writing in ZDNet's Real World IT blog, accuses AMD of comparing processors the company will not be shipping for months (2.6GHz Barcelona quad core) with older Intel Xeon quad cores rather than currently shipping ones which would beat the (hypothetical) score AMD claims for the future Barcelona. I guess while even the much slower 2.0GHz Barcelona is due soon AMD didn't think results from the 2.0 would look good enough — even against the slower Xeons they picked. Maybe the right comparison should be either best cpu against best cpu — or compare ones at the same price — and only shipped products."

Comments

Who trusts a vendor's benchmarks anyway?

[nweaver]

Vendor benchmarks are always considered untrustworthy, so I don't see what the big deal is.

Re:Who trusts a vendor's benchmarks anyway?

[baggins2001]

No kidding
I don't think the accuracy of the benchmarks is in question here.
The deceptive manor in which the benchmark data was presented is the issue. Which is really a none issue. This is advertisement, anybody who doesn't look critically at data presented by the manufacturere is really gullible.
Anybody who doesn't look critically at the data from a third party is pretty gullible
I really really don't see the problem here

Re:Who trusts a vendor's benchmarks anyway?

[CatsupBoy]

If we don't point out every time they use blatantly unfair product comparisons, the amount of disinformation coming out of vendors will only increase

Or the amount of crap product comparisons will continue to be the same no matter how much its pointed out.

Companies will continue to tout themselves as top dogs, regardless of the facts. And it never ceases to amaze me how far they go to stretch the truth in order to make themselves look good.

How else could salesmen go into a room and pitch their product? Or how can manufacturers sell their AMD products when competitors are pushing Intel? And vice versa? Its capitalism at its best.

Re:Who trusts a vendor's benchmarks anyway?

[mgoheen]

> Vendor benchmarks are always considered untrustworthy, so I don't see what the big deal is.

That logic gets you into trouble...

Politician promises are always considered untrustworthy, so I don't see what the big deal is.

Auto companies are untrustworthy, so you should expect the brakes to fail.

People are untrustworthy, so if you are robbed, it's your fault for carrying cash.

People are killed every day, so I don't see what the big deal with Iraq is.

etc.

Sheesh...wrong is wrong, no matter who is doing it. If you don't fight it, you're part of the problem.

Re:Who trusts a vendor's benchmarks anyway?

[arth1]

Still, the submission misses the point completely when it want benchmarks of only CPUs that's on the market. The reason for the benchmarks of CPUs that haven't been released yet is so OEMs and retailers know a little more of what to expect, and make plans for ordering (or not ordering) accordingly. If there's no benchmarks of unreleased CPUs, it would not hit the market, and thus wouldn't be benchmarked -- catch 22.

Who the manufacturer compares against is of course up to them, and there's nothing "unfair" about it. It's telling the world that this is the competition they strive to beat. If it's an older CPU, the new CPU is obviously intended as a replacement for these. If I had a large server farm running these Xeons, I'd be most interested to see this benchmark, well before the CPUs actually come out (if they're already out on the market, they will be off the market by the time upper management approves the budget). And remember, AMD and Intel aren't in the game to try to trick you to buy a CPU that won't work well for you -- they want you to return for your CPU needs, over and over again. That's why they publish benchmarks like these, which are relevant, just not to the GP.

Other comparisons both will and do appear once a CPU has hit the market. But for the initial pre-release vendor benchmarks, I'd rather it be the choice of the vendor, so we can see where the market position is going to be.

Move along -- nothing to see here, except for a particularly silly submission.

Re:Who trusts a vendor's benchmarks anyway?

[PFI_Optix]

Or the amount of crap product comparisons will continue to be the same no matter how much its pointed out.

You don't think it can get worse? You don't think it would get worse if there weren't people crying foul at the current comparisons?

You can use legitimate comparisons to tout a product, you don't have to unfairly match them. Look at your average car commercial (fictional example):

Ford's new truck gets better gas mileage than Dodge.
Ford's new truck has a bigger, more powerful engine than Chevy.

They just said it's better than Dodge and Chevy, but in two completely different ways. They do this all the time in marketing. If nothing else, AMD could talk up price points and power efficiency, two things they almost always have over Intel. Skewed benchmarks just make the company look inept and leave knowledgeable consumers feeling like AMD is insulting their intelligence.

Re:Who trusts a vendor's benchmarks anyway?

[arth1]

If someone is lying, could you please care to name a lie?
Cause I can't see a single lie. Self-flattery, yes, and selective truths, yes, but no lies.

If you're in the business (and if you're not, this type of benchmark isn't meant for you), you know very well how to read and interpret the reported benchmarks and marketese. It's the expected format, which is helpful to those who need to know these things, e.g. because they are planning on upgrading a large Xeon farm to faster CPUs at as low cost as possible, or because they're a large OEM who needs to know the market segment this CPU is intended for, so they know both how much to order and how to market it.

Can we all stop this lynch mob now?

Re:Who trusts a vendor's benchmarks anyway?

[arth1]

Comparing a product that I (may) produce, 4 months from now, to one that someone else, did produce, 4 months ago, in a rapidly changing market, to imply that "My chips are better than their chips" is lying.

Really? You have a very strange definition of "lying", then. I think it shows how good a replacement the new CPU would be compared to the older one, but what do I know?

To say "well, they aren't saying anything which isn't true, my future chips do outperform their former chips" is disingenuous. It may be, literally true, but the implications of that graph are misleading and therefore a lie.

How is it misleading? It's a very good indicator on whether the future CPU would be a good replacement for the old CPU, and that is useful information to many -- both large companies and OEMs. The only misleading here seems to be people misleading themselves into thinking the benchmark is for a different purpose than it is.

This isn't helpful to some buyer, as it doens't compare two comparable items. My not yet released chips, are not comparable to their old chips. That you suggest something else, makes me wonder who you are writing for.

Me, myself and I. I refuse to join a lynch mob without thinking things through first.

Re:Who trusts a vendor's benchmarks anyway?

[arth1]

You seem to have missed the part where they compared them only to older Intel CPU's, and conveniently ignored the fact that current Xeons could beat them in benchmarks. Fanboy much?

No, I'm far from a fanboy. And no, I didn't miss that part. In fact, I explained why they compare the unreleased CPU to an older Intel CPU, and why that makes sense. Apparently you either skipped this, or didn't understand it, so I guess I have to get a silver spoon and bib for you:

When a CPU hasn't been released yet, what's important to the large scale buyers is how it fits in with their business. If they now sell cheap servers with Xeons, would this be a feasible replacement that could increase sales and profits and/or reduce costs? A comparison with the "old" Xeons is thus very relevant. It also tells exactly which market segment AMD intends this CPU for, which is quite valuable information. Not getting benchmarks until the CPU has been released makes it much harder to make decisions like these. Similarly, there's little value in having a CPU compared to the "state-of-the-art", when you wouldn't purchase it as a state-of-the-art replacement anyhow.

It may be of interest to the casual hobbyist, but that's not the big market.

Re:Who trusts a vendor's benchmarks anyway?

[arodland]

Er no, quite the opposite. To state that is to be the cause of the problem. Notice that you can't even make your point without falling into your own trap. Your argument against a categorical morality is that believing in one is categorically wrong. :)

I can smell the desperation

[Urusai]

Core 2 is smoking AMD and they are panicking. Do they even have a real next gen architecture, aside from bizarre (albeit intriguing) CPU/GPU hybrids?

Re:I choose AMD for the price...

[bealzabobs_youruncle]

Or course it won't cause you to "trade my 3800+ dual core Athlon 64 for an intel Core 2 duo of the same speed and have to pay twice the price" becuase none of those circumstances are possible. I like AMD too, but they got owned this round and that is just the way it is. AMD is just as capable of evil and more guilty of whining, brand loyalty is for suckers in regards to performance desktop computing; buy the fastest gear you can get at the moment of purchase.

Re:It's not the deceptive benchmarks that bother m

[eln]

Welcome to tech journalism. All you have to do is publish companies' press releases. For "in depth" articles, you visit their offices and have the PR guys talk to you all day. For product reviews, you repeat the companies' benchmarks and then turn on your demo unit to take some screenshots (if you can't find screenshots on the manufacturer's website, that is).

As someone who once worked for a company producing a product that had major hardware issues (as well as some fairly significant software bugs) yet still got rave reviews from tech sites, I can tell you this is pretty much how it works.

Re:I choose AMD for the price...

[QuantumPion]

AMD and Intel are CPU manufacturers, not sports teams. Buy the product that is the best performing at the lowest price.

What a load of lying malarkey

[(negative video)]

If AMD was comparing one architecture to another, they MUST do so based on identical core clock to memory clock ratios.

So what are the ratios in question, ZDNet? <pull string> "Math is hard."

Then the ZDNet jerkoff has the gall to complain that AMD didn't use the latest SPEC.org numbers. Well, duh. RUNNING benchmarks means just that: running them. You get the actual machines you want to compare, scrupulously make all the software as identical as possible, and let 'em rip. You DO NOT just grab random numbers generated by random software off a random website, no matter how impressive the numbers claim to be.

Re:Not the architecture..

[Iam9376]

In the broader sense of an 'architecture' in my mind, AMDs has a more advanced one than Intel (the integrated and hypertransport IO/multi-processor strategy).

Then it seems your mind needs an update. Intel's Core 2 architecture is significantly better than AMD's current or past (and seemingly future) architectures.

Putting all the fanboy drivel aside for a moment;
Intel's processors are faster without using more transistors, indications that the architecture is more optimized and makes better use of the available transistors.
Intel's processors scale vastly better than AMD's offerings both current and future.
Also consider, the die shrink to 65nm for AMD produced little to no benefits in speed and scalability (read: you couldn't over clock them very much)
Also, if anyone remembers, the Pentium M (which the Core 2 is based off) was benchmarked a few years ago against the AMD 64bit desktop processors and spanked them, no not in all cases or by any significant margin, but the fact a low power laptop processor (32bit) matched a 64bit mid-range/hi-end processor from AMD; that should indicate the advantages of the architecture.

Just because Intel does not currently have the memory controller on board, as well as the use of the older FSB does not make the Intel architecture any less advanced, the proof is in the puddin`, check any benchmark that puts current purchaseable processors and see how wins.

You're right about Intel, they will be releasing CSI (common system interconnect) for their processors in `08, if CSI does for Intel what even half of HT did for AMD, they may be in very serious trouble.

Sorry, but the rest of your post is moot.

Re:I choose AMD for the price...

[timeOday]

I mostly agree, but customers big enough to influece market dynamics, like Dell or the US Government, should think about how awful it's going to be buying a computer in a few years if AMD falls out of the race. Personally I'm a little worried about it.

Comparison points

[Todd Knarr]

I'm interested in a side-by-side comparison at three points:

1. Best against best. How do the current top-of-the-line CPUs from each company compare.
2. Similar price points. If I'm willing to spend $X on a CPU, which company gives me the most performance for my money?
3. Similar clock speeds. This is more a techie thing, gives me an idea of which company's wringing the most from each clock cycle in their chips. With current tech it's not a really reliable guide to which CPU to buy, but it gives me an idea of how their tech stacks up.

Re:Not the architecture..

[TheRaven64]

Intel's Core 2 architecture is significantly better than AMD's current or past (and seemingly future) architectures In some ways, yes. The micro-op fusion stuff is incredibly shiny. They took some good branch prediction logic from NetBurst, and have a lot of neat tricks internally, particularly in the cache controller. On the edge of the CPU, AMD have the lead. They have a better interconnect (they are going to lose this lead soon, once Intel get CSI out of the door), and they have more intelligent memory controllers, which give them the edge in virtualisation and a few other things.

It's not entirely fair to say Intel is ahead of AMD architecturally. Both architectures have their strengths and weaknesses. At the moment, Intel are getting better overall performance (which means performance per Watt these days), but their architecture does have a few issues.

Re:Let's all scream FIRE!

[neersign]

from the summary (i refuse to read ZDNet articles):

(2.6GHz Barcelona quad core) with older Intel Xeon quad cores rather than currently shipping ones which would beat the (hypothetical) score AMD claims for the future Barcelona. I guess while even the much slower 2.0GHz Barcelona is due soon AMD didn't think results from the 2.0 would look good enough - even against the slower Xeons they picked. Maybe the right comparison should be either best cpu against best cpu - or compare ones at the same price -- and only shipped products."

I don't understand how the Xeon 5355 is "older than currently shipping ones". I'm not a server processor guru, but when I go to Intel's site and click on server processors, I only see Quad Core 3200 series and Quad Core 5300 Series. Of the 5300 series, the 5355 is the top of the line processor as shown here. So, it looks to me like AMD picked the top of the line, currently shipping processor from Intel.

And while I'd rather see benchmarks on the 2.0ghz Barcelona since they are going to release it first, I do appreciate seeing numbers on a clock-to-clock basis. This shows me that per clock, AMD's Barcelona is doing more work, so even at 2.0ghz I can expect it to do more work than an Intel quad core at 2.0ghz. But I think it's clear that AMD chose to run their Barcelona at 2.6ghz to match the clock speed of the currently shipping, top of the line Intel processor.

While any benchmarks released directly by the manufacturer need to be taken with a grain of salt, I think any story written by some one who works for ZDNet needs even more scrutiny.

Re:I choose AMD for the price...

[tcc3]

Ok so when the athlon and opterons were eating Intel chips for breakfact and exacting prices to match the story was "Amd has faster chips!"

Now that Intel got their act together and are cleaning AMDs clock its "They are cheaper / a better value / more bang per buck."

AMD dropped their prices because of the performance differential.

They both make great stuff these days and are pushing each other to higher standards. Buy what you want or need but lets not pretend alot of this isnt just blatant fanboyism.

Re:Let's all scream FIRE!

[Anonymous Coward]

No, AMD did not choose to run their processor at 2.6GHz - they chose to imagine running it at that speed. They wanted to ship at that speed but they have failed to execute. Although they may have it at 2.6GHz in a lab somewhere they have no hope of delivering at that speed this year. Intel OTOH has 3.0GHz Quad Core processors shipping, right now (and has had for several months) in Apple's Mac Pro machines. If you can buy it - it's shipping. If you can't - it's vapor - and that's a fair description of even the 2.0GHz Barcelona part right now.

You like clock-to-clock comparisons? OK, well you might like to imagine a P4 running at 10GHz. Or a PowerPC at 100GHz. Or a Z80 clocked even higher. It would be equally valid.

Opteron is fine, just overpriced

[billcopc]

I'd like to remind everyone that AMD started out selling cheap slow CPUs in the upgrade market. Remember that K6 400mhz your uncle used to have ? Well that K6 cost a whole lot less than any Intel processor at the time, and it breathed new life into old boards one or two generations behind. Then one day AMD released the Athlon, took the performance crown and didn't really know how to play their role. Their marketing was shit, and their pricing wasn't so good anymore. They had tons of experience being the underdog, but zero skill as a leader.

Now Intel has come back on top, but AMD doesn't want to go back to being #2. Instead of putting their efforts toward a new, faster architecture like Intel did, AMD is resting on their laurels, releasing outdated underwhelming puke way too late in the game. I'm sitting here with an AMD x2, I've had it for about two years and I've been running it slightly overclocked since I got it. Well my 2 year old chip is still faster than their fastest CPU today. I would love to buy a new CPU that's 30-40% faster, but they don't make one. Even my buddy's brand new Intel E6600 is faster than what I have, and he didn't pay all that much for it. That's why I'm getting an Intel Q6600 in a few weeks, when the prices drop again. AMD still won't be anywhere near releasing their first quad core processor.

AMD needs to shut up and take their place. They're really good at selling slower, inexpensive processors for the everyman. They need to stop lying to themselves and accept the fact that they just can't cut the mustard when it comes to high-end, which is fine because the big money is in the OEM market, where every dollar counts. If AMD can produce a decently fast and affordable chip, and hire a goddamned business director to get some partnerships going, they could make a ton of money. Just don't pretend the Athlon is a performance king, because we all know it's a lie and the only fool is AMD.