Intel Funds AMD-bashing Report

Jim Norton writes "The Inqwell and ZDNet are reporting that the Aberdeen group, who recently published a report attacking the Athlon XP's processor rating system, was funded by Intel to produce the report. The articles also mention that AMD claims they were never contacted for information regarding this issue." From the benchmarks that various outfits have done on the new AMD chips, their model number is actually pretty conservative.

Comments

And in other news....

[Chester K]

And in other news, business as usual continued yet again today. Analysts continue to be shocked.

PR Rating Stupidity

[Syllepsis]

When AMD released the Athlon XP 1800+, every reviewer on the planet ran a battery of benchmarks and concluded that for most applications, the XP 1800+ not only beat the P4 1800 MHz, but also the P4 2000 MHz.

When AMD released the Athlon XP 1900+, every reviewer on the planet ran a battery of benchmarks and concluded that for most applications, the XP 1900+ not only beat the P4 1900 MHz, but also the P4 2000 MHz.

When Intel released the Northwood 2000 and 2200 MHz P4s and AMD released the XP 2000+, every reviewer on the planet ran a battery of benchmarks and concluded that for most applications the XP 2000+ beat the P4 2.0A but could not quite beat the P4 2.2A

Then when AMD released the XP 2100+, many reviewers concluded that it tied or beat the P4 2.2A, although I really think that the 2.2A has the edge.

Based on this data, what really happened, what is really happening, and what disinterested parties seem to believe, I would conclude that the AMD PR Rating system provides a very nice comparison of Athlon performance relative to P4 performance at the clockspeed of the PR rating. Even though AMD says the rating is to compare the Athlon XP to other AMD products, it is incredible how well it scales athlon performance to the P4 performance at the clockspeed of the rating.

Therefore, if I wished to buy a machine, as a general purpose user, I think the best way to compare prices would be to match the AMD PR Rating against the Intel P4 clockspeed.

OTOH, comparing raw clockspeeds would give a false conclusion that an Athlon XP 2000+ would not outperform a P4 1.7 GHz. Sure, this is true if you plan on using Newtek Lightwave (where all P4s beat all Athlons), but for most tasks you would be horribly in error.

It would seem fairly obvious, that for this point in time, and with the current set of processors available, for the user who uses a variety of applications, the consumer would be better informed by using the AMD rating system than by just about any other comparison (other than carefully studying a battery of 30 different benchmarks)

However, there has been a flurry of criticism of the PR rating.

As much as I hate to cheerlead corporations, I just have to yell...

FUD!

...and anyone who disagrees with me is invited to study any of the following review sites:

Tom's Hardware
Anandtech
XBitLabs
Sharky Extreme
Lost Circuits

etc... etc... etc...

Missing the point

[CaptainAlbert]

AMD and Intel can argue for as long as they like about whose benchmarks are rigged, but it doesn't change the fact that, in the end, they are just that - benchmarks. They bear absolutely no resemblence to real life performance whatsoever. In the end, it doesn't matter what the graphs say or who claims to be faster that whom.

If you're going to start sueing people for misleading the public into buying products by presenting them with misleading data, then any hardware or software vendor who uses a benchmark in their marketing literature should be prosecuted.

The hierarchy is thus: lies, damned lies, statistics... benchmarks. :)

Re:Missing the point

[larien]

Benchmarks prove whose processor is the fastest at running benchmarks...

That said, a well written benchmark can give a guide to relative performance, and it's hard to argue that a quake 3 benchmark isn't measuring "real world" performance in 3D gaming (although the incident with ATi was a bit of an embarressment).

Re:Missing the point

[Geek In Training]

The hierarchy is thus: lies, damned lies, statistics... benchmarks.

This is true for proprietary benchmarks run by companies against their own products.

But for benchmarking apps developed by third parties and used by fourth parties to compare and review hardware, I don't think this rings true. Things like Q3bench (which checks real-world performance by running timedemos) and MadOnion 3DMark (which tests graphics capabilities and performance in various areas) are very handy for testing video cards by review sites.

I mean, a lot of people have used Prime95 and RC5-64 clients to review processors over the past four years, since they are so CPU-intensive. (Er, they take 100% of available CPU cycles.) Therefore, you can judge how well certain processors can handle certain types of operations by seeing how many blocks they can crunch in an hour.

The problem is that certain apps are going to run better on certain processors because of a harmony between the processor design and the heavily-used commands in the app. Like the fact that a G4 (with Velocity Engine! er, AltiVec unit)will totally spank a P4 twice its clockspeed in RC5.

So the *good* reviewers out there run a battery of tests, including graphics, processor-intensive apps, memory-intesive apps, etc. to get a good, holistic representation of performance. They can then replicate that battery of tests on other hardware to compare performance with a reasonable degree of accuracy.

If a company says, "We used our own IntelMark tests to determine that we are the fastest processor ever," that's marketing crap. If [H]ardOCP says "We ran the P4 and the Athlon XP though 58 tests developed by third parties, and here are the results:" ... I think you can use those numbers for educating yourself before making a purchase with some degree of confidence.

HardOCP

[rosewood]

Since I can not link directly to this article I will quote Kyle from HardOCP.com - and as of 8:18 CST March 28th, this can still be found on the Front Page of www.hardocp.com.

I just off the phone with the fine folks over at AMD and was discussing the issues over the Aberdeen Group white paper. Aberdeen, if you are unfamiliar with them, claims to be a Market Analysis company. If this is all new to you, please visit the InqWell as well as ZDNet on this issue.

There are two situations in this issue that are fun to look at. If you go read the white paper entitled AMD's Gigahertz Equivalency: Inexperienced Buyers Accept Bad Science, published here (and you will have to sign up) you will notice that Aberdeen uses flawed logic to pick on AMD's model numbering system of their CPUs.

Nevertheless, Advanced Micro Devices (AMD) last year deliberately took a step down a slippery slope of bad science when it named its Athlon XP line of microprocessor models with clock-speed gigahertz ratings equivalent to Intel's competing Pentium 4 (P4), based on a set of application
benchmarks audited by Arthur Andersen and fully described in AMD vs. Intel comparisons at AMD's Web site.

And then..

What's the flaw in AMD's equivalency ratings? There are many discussed in this Aberdeen Executive White Paper. The key flaw is that the equivalency rating is a snapshot in a moment in time -- and time surely marches on in the computer industry -- making the gigahertz equivalency subject to increasing variance over time. For example, the AMD Athlon XP 2000+ processor announced last fall runs at 1.667 GHz. The 2000+ equivalency rating is aimed at Intel's P4 2.0 GHz Willamette processor.

Aberdeen seems clueless that the basis for AMD's model numbering system is a comparison to their own TBird core CPUs and the speed they would have to run to be equivalent to a Palomino core CPU in performance. Seems as though Aberdeen did not even contact AMD in order to better understand the exact subject they were commenting on.

The other part of this situation is this. It is now being rumored that Intel paid for the white paper.

But the INQUIRER learned from Intel this morning that it paid for the report to be written, which certainly puts a different complexion on the thrust of the piece. Intel maintains, however, that fact doesn't affect the objectivity of the Aberdeen Group's findings. Cough.

While I have no idea of the validity of this claim, I will tell you this. I think of all of these "market analysis" companies as no more than paid mouthpieces of the hardware industry. Some are scumbags that make a living off taking a product and making it look good to distributors and consumers. They are spinmeisters looking for a buck and will spin their "truths" to support their clients needs. If you think market analysis companies are in this hardware industry to make sure that you, the consumer, get the truth, you are sadly mistaken. Is this to say all anylysts bad and are always wrong? Of course not, but I give them about as much credibility as a 4th grader with a Geocities site. At least the 4th grader most likely has purer motives.

I think it was one of these industry analysts that referred to sites such as our own as "homebrewed" and that we bascially did not deserve the voice that we have in the hardware community. I guess even we piss off the analysts when we uncover the truth that does not agree with their spin.

Anyway, this is all my opinion and subject to just flat being wrong but there is one thing I will tell you for sure and that is that the hardware industry has a nasty underbelly just about like every other industry in this world.

UPDATE: From the cards and letters we have gotten on this subject everyone seems to be missing the point of my little diatribe posted above. The rating methodology wars are over in my mind, and have been for a long time now as Aberdeen is way late to this party. The opinion I want you know about analyst companies is that they are simply all bought and paid for and expected to spin the agenda of their client.

Also, Intel did share with us tonight that they did finance the Aberdeen research into the AMD rating system.

As long as you guys are smart enough to form your own opinions and thoughts, which most of your are, just make sure you don't let these "analyst" companies shape your opinions as some are nothing more than a PR company that runs a couple of benchmarks. In this case Aberdeen based their entire opinion on BABPCo Sysmark 2001 and Quake III numbers.

You guys would hunt me down and whip my ass if I ever gave you a review based on that little data.

Intel Funds AMD-bashing Robot

[Hnice]

In my early-morning haze, that's how this headline reached my cortex. You can imagine how disappointed I am right now.

the real unbiased benchmarks

[Gary Yngve]

SPEC

see the 1st quarter 2002 results for CPU2000

Real world comparison

[olympus_coder]

We use dual processor machines to run simulations (particle physics). We have 3 dual 1.7gig Xeon/RDRAM setups and several 1800+ MP/DDR setups. The 1800+ setups will complete the same amount of work as the xeons in 75% of the time! I thought they were better, but I didn't think they were that much better. That is a 1.53gig machine completeing the same work as a 1.7 gig machine (with faster memmory) in 75% of the time.

Our application, as you can imagine, is very floating point intensive.

Re:TradeOffs between Intel and AMD Cpu

[BlackSol]

You missed the biggest point: Cost/performance.

AMD chips have been shown over and over again to provide greater performance at less cost than Intel.

Re:PR Rating Stupidity and lightwave benchmarks

[tcc]

I might add, this is because some Lightwave portion of the rendering pipeline are heavily optimized for SSE2. The benchmarks on tom's hardware are flawed because he uses a scene that 95% of the calculations are SSE2-based (radiosity) and it's a known fact that it's heavily optimized (so it's clearly not a balanced scenario). If they'd use standard raytracing, you wouldn't see such a jump. In fact we use lightwave where I work, I am a lightwave fan since 2.0, and I've built up a renderfarm based on Dual XP athlon solution (with tigerMP and yes it works with bios 2.03). I don't even want to touch a dual P4 solution, clearly not a good bang for the buck even if it's faster in some (clearly not all) cases.

Again, When you look at tom's benchmarks you tend to think like if the P4 would be almost 50% faster than AMD, when you'll render balanced stuff (which is most cases I've seen and besides, nobody will do a fully animated short with "radiosity" on, it takes forever to render, so you render 1 and use the "baking" function, so you revert to raytracing or standard rendering after that), the margin grows way thinner. When you calculate the costs, Intel is way out of range for price/performance. A lot of people told tom about his LW benchmark, but as usual, he didn't acknowledge nor changed his ways (there have been benchmark data available and howto's on the net for lightwave since 4.0 on multiple platform, he doesn't seem to want to follow the "standard" thus invalidating his work to the eyes of the LW community checking the benchmark numbers. But that's another story.

frowned upon ???

[Archfeld]

C'mon the only thing 'frowned' upon by a corporate world is getting caught, or failing to make a profit. Frowning on an act would indicate some remedial conscience or morals, and as we see everyday corporations have NONE. That is not to say the individuals that make up those corps are bad but if everyone 'just follows orders', Lemmings Inc. will be as dirty as they come.