The Register Finds Fault In Turion Benchmark Setup

An anonymous reader submits "From The Register, it appears that AMD has joined Intel, ATI, nVidia, and just about every other hardware manufacturer on the planet in benchmark fiddling. The benchmarks for the Turion appear to have been compared using quite different systems - a 35 watt Turion 64 with an ATI GPU versus a 25 watt Pentium M with an Intel integrated graphics processor. Sadly, it appears the original benchmarks were too good to be true."

Comments

Strage Focus

[mal0rd]

Oddly, the register article reads like an opinion piece, focusing on how AMD should care more about battery life.

Re:Strage Focus

[JPriest]

... that usually has a pro AMD slant.

Re:Strage Focus

[ameoba]

Actually, if you look at benchmarks on the fastest Pentium-M chip, they beat the high-end P4 quite consistantly. They're quite impressive chips, taking the excellent design of the old P6 core & integrating the best parts of the Netburst core. The end result is a fast, low-power CPU.

How much does power consumption differ here?

[PxM]

The article says "Reynolds said... that Turion-powered systems could have up to one-third less battery life than laptops running on Intel's ultra low voltage products." That sounds like its just a comparison of the 27W to 35W specs of the processors. However, since both companies have their own systems to dynamically scale back power by slowing down the CPU like SpeedStep and PowerNow, could the power labels be inaccurate as a measure of battery life in this case? Just like how clock-vs-clock benchmarks are no longer valid, could the same now be true about Watt-vs-Watt measurements?

*shrugs* Then again, AMD might just be flat out decieving on the benchmarks because they are thinking of their shareholders rather than their customers.

--
Free iPod? Try a free Mac Mini
Or a free Nintendo DS, GC, PS2, Xbox
Wired article as proof

overestimating the public?

[coshx]

From TFA:
This kind of trickery in the benchmark game does little for the vendor.

I have to disagree with this one. Fudging benchmarks almost always helps the vendor, except with very specialized (/.) audiences.

Remember, most people just see a bunch of random numbers when they shop for laptops, and compare processors based on GHz. They're more likely to read a blurb (or hear from the salesperson) that Turion outperforms it's competitors than they are to search blogs about the truth to the claim.

Now, by making enough of a fuss over this, we can create negative publicity, but why rag on AMD when, as the article states, all the other companies have set precedent?

Re:overestimating the public?

[Anonymous Coward]

First the Apple fanboys, then the Google fanboys, now we have the AMD fanboys. If Intel does it it's evil, but if AMD does it, well, "everybody else does it too".

Prevalent

[SteelV]

It seems like this is becoming necessary (well, maybe not fiddling, but at least adding additional support for certain applications).

For example, Half-Life 2 is a very popular game. If nVidia starts messing with their drivers to run HL2 better, but ATI does not, then guess who HL2 fans are going to buy from?

I know -- the case mentioned in this article is completely different and not a useful change -- but it just got me thinking about past occurances.

It seems like drivers for GPUs should be able to run well in general, and applications should be catered to them, rather than the other way around, but I guess it's just not a viable option.

Re:Prevalent

[bersl2]

For example, Half-Life 2 is a very popular game. If nVidia starts messing with their drivers to run HL2 better, but ATI does not, then guess who HL2 fans are going to buy from?

And this is, of course, just another one of the many wonderful reasons why graphics drivers should but never will be opened up.

And give us the goddamn specs, you bastards! Part of the reason ISAs (it'd still be an ISA, even though a GPU is not really a CPU, right?) exist is so that one does not need to know the microarchitecture to use the damn hardware!

This kind of crap has to stop.

Actualy there's a bigger one

[justsomebody]

All benchmarks are 32-bit. Turion is 64-bit.

I wonder how it will perform with 64-bit linux. Well, I'm going to see when some brand name shows up with Turion in his higher class.

Re:Total power use comparison?

[megabyte405]

Well, the AMD64 (desktop) chips include their own memory controller, which is probably what you're thinking of. I do not know if the Turion does as well, however, if it's a "mobilization" of the desktop chip (or even if it's not), it would make sense.

Seen it before...

[Datasage]

A couple years ago, I was at an AMD Press event thingy because they were giving away free stuff. At the event they computers set up with UT2003. Yes, they did the same thing. They gave they the intel computers onboard graphics, while thier computers had nvidia cards.

Games especailly are bound to the video card in terms of performance.

Fair?

Yeah, right.

different type of fiddling than ATI/nVidia

[krunk4ever]

If I recalled correctly, nVidia or ATI fiddled with benchmarks by disabling certain features which take up a lot of processing power when they detected 3dmark or other benchmarking utilities.

Technically AMD didn't falsify their benchmarks by disabling certain features. They just chose a easier opponent to fight with. (sorta like when one picks a fight with someone that looks way weaker than you to guarantee you a win).

What I find weird benchmarks usually don't just give out percentages, but actual figures. Like # of flops per second, etc. I'm not sure if any other graphs were provided, but from the links that were in the opening topic, they were all percentages. If the benchmark had listed actual results, it would've been easier to do actual comparisons with other cpus which were not benchmarked against.

Saying all that, I don't believe AMD falsified any information (unlike nVidia or ATI). What they did was purely comparing a weaker opponent. Sorta like taking the ATI Radeon 9800 and comparing it to the nVidia Geforce MX440. But comparisons like that do exist when you do a wide range of benchmarking. That's why I always make sure I know what the heck is being benchmarked or else, it'd be just throwing #s at me. It's nice that newer benchmarks tell you if higher is better or if lower is better. Sometimes it's quite easy to get confused on if something w/ a higher # is better or not.

I'm not too worried...

[kisea]

My first laptop was an Athlon 1.2, and the battery life majorly sucked (i'm talking mere minutes not even close to an hour), you may have even said "battery...what battery?" So when I enrolled in a Game Dev class I needed to upgrade due to crappy video. I got an athlon64 based system and my friend got a Pentium-M centrino system with similar video and similar processor speed, etc. and my machine will run circles around theirs. If i'm running on battery, I dim my screen, etc. Granted, I do not have quite the battery life as they do, but I feel I have a very nice balance between power and battery life.
AMD has made some major headway in the mobile processor market and I believe they will continue making improvements. Just look what they did with desktop processors and look what they have already done with their mobile processors. Its unfortunate that the test setup was flawed, but the 'centrino' package is being pushed very hard in advertising. I'm not trying to make any excuses, and I admit that I haven't done much research since I got my notebook, but are there many notebooks offered with the 25 watt pentium-M that isn't centrino based?

Re:So?

[bayvult]

...It's also noteworthy that TheRegister has a partnership with Tom's Hardware in the U.S., and some editors of Tom's have been noted as being overtly biased towards Intel

No shit, Sherlock!

I always thought TheRegister was really cozy with Intel too. Especially Ashlee Vance, who seems to be a real Intel fanboy.

Do you remember how they always gave Intel's IA-64 processor a really easy ride - by calling it a cute name like "Itanic" and running suck-up stories like Itanic: Enron's Golden Albatross, Dell 14- IBM 0: Itanic quarterly sales revealed, Itanium sales fall $13.4bn shy of $14bn forecast, Do not feed, poke or disturb the Itanic user and even Miracle cures Berkeley man of Itanic wickedness.

When you closer you begin to see a pattern. World beating Intel products like the system-on-a-chip Timna were launched first at The Register with puff pieces such as Intel's Timna has dead duck look'n'feel. Recently they've done nothing but tell us how great Intel's wireless strategy is - have a look at Ronald McDonald to save WiFi and shit like this. Pure ficking PR.

They're corporate whores - keep investigating Sivar you'll nail their asses.

Perhaps you should read more?

[melted]

Let me guess, you're an AMD fanboi.

Let's take a look at the specs of test systems side by side, shall we?

Intel system has this shitty "Extreme" integrated graphics with no dedicated RAM. If you've ever used a system with Intel Extreme integrated graphics, you know it's dog slow. Putting it into a laptop with slowish system bus doesn't make it any better.

Now let's look at AMD system. It has integrated GPU (!) from one of the leaders in computer graphics hardware with _32M_ of _dedicated_ Video RAM. I wonder if I've emphasized this clearly enough to get through your thick skull.

You don't have to be a genius to say that AMD system has an advantage when it comes to 3D graphics.

Wouldn't it be cool if instead of cooking the benchmarks AMD put the processors head to head and made them use the same Radeon Mobility 9600 card with 128M RAM? Why didn't they do this? Were they afraid of something?

I mean, come on, we all know that they make the best desktop processors at this point. Their mobile chips, however, leave much to be desired, and with the release of this Turion chip status quo remains the same.

Re:Perhaps you should read more?

[eRacer1]

Let me guess, you're an AMD fanboi.

Better to be an AMD fanboi with the facts than an Intel fanboi with the flames I guess. I mean after all your comment that "they "forgot" that Intel chip used integrated graphics" was shown to be completely false. Quite an embarrassment for a non-AMD fanboi like yourself, wouldn't you say?

Intel system has this shitty "Extreme" integrated graphics with no dedicated RAM. If you've ever used a system with Intel Extreme integrated graphics, you know it's dog slow. Putting it into a laptop with slowish system bus doesn't make it any better.

Now let's look at AMD system. It has integrated GPU (!) from one of the leaders in computer graphics hardware with _32M_ of _dedicated_ Video RAM. I wonder if I've emphasized this clearly enough to get through your thick skull.

What do you not understand? The systems were comparing integrated graphics performance? Is that a fair measure of CPU gaming performance? No. Is it a fair measure of gaming performance you can expect to get with an AMD and Intel laptop with integrated graphics? Sure is.

You realise Intel MANDATES that every Centrino using integrated graphics must use Intel integrated graphics, right? You realise that Intel has discouraged ATI and others from producing top-of-the-line integrated video chipsets for Pentium M because they are not allowed in Centrino bundles, right?

By the way "GPU" is just a fancy marketing name for graphics processor. ATI integrated graphics is no more of a GPU than Intel's integrated graphics.

Now let's look at AMD system. It has integrated GPU (!) from one of the leaders in computer graphics hardware with _32M_ of _dedicated_ Video RAM. I wonder if I've emphasized this clearly enough to get through your thick skull.

Better to have a thick skull with a brain in it than a normal one with none. In case you hadn't noticed Intel "forgot" to do the same to improve their performance. There is a tradeoff in battery performance. The good news is that you can get somewhat acceptable gaming performance without taking a much larger hit in battery life by using a discrete video card like the Mobility 9600 card with 128MB RAM.

Wouldn't it be cool if instead of cooking the benchmarks AMD put the processors head to head and made them use the same Radeon Mobility 9600 card with 128M RAM? Why didn't they do this? Were they afraid of something?

The gaming benchmarks weren't cooked as they compared the best integrated graphics chipsets on each platform in terms of features, performance and expected availability. But I do agree that benchmarks using the same video card would be interesting and more indicative of pure CPU gaming performance. As I mentioned in another post Pentium M performance is very competetive with Athlon 64 on desktop gaming benchmarks when a top-of-the-line desktop video card is used.

Re:There's a reason AMD is scared

[Gr8Apes]

It seems the tide has turned, whether Dell sees it or not. A multi CPU Opteron system blows away any comparable (number of CPUs and GHz, even though GHz is fading fast as a true indicator) Intel system in performance, and $ for $, well, there's really no comparison.

Also look at the gaming world - they're pretty much all heading for AMD 64 systems.