Slashdot Mirror
The Register Finds Fault In Turion Benchmark Setup
An anonymous reader submits "From The Register, it appears that AMD has joined Intel, ATI, nVidia, and just about every other hardware manufacturer on the planet in benchmark fiddling. The benchmarks for the Turion appear to have been compared using quite different systems - a 35 watt Turion 64 with an ATI GPU versus a 25 watt Pentium M with an Intel integrated graphics processor. Sadly, it appears the original benchmarks were too good to be true."
Oddly, the register article reads like an opinion piece, focusing on how AMD should care more about battery life.
AMD is perfect. Nothing to see here. Please move along.
Soon we will have benchmark woodwinds, benchmark flutes, and worst of all benchmark trumpets. Off course it will come together as all that benchmark jazz. . .
But hell, in the desktop market they're kings, and everybody knows that. It's too bad they had to resort to benchmark fixing for a mobile processor.
Condemnant quod non intellegunt.
It's not like you can trust the PC hardware web sites any more than you can trust the vendors anyways. There's a high road? Where?? =P
According to the article, their laptop processor beat out the intel processor only by a 'small margin'. They have to release benchmark press releases, so, being a business interested in profit, they stack things.
They make some of the best chips on the market. Doesn't keep them from being 'just another business'.
The article says "Reynolds said... that Turion-powered systems could have up to one-third less battery life than laptops running on Intel's ultra low voltage products." That sounds like its just a comparison of the 27W to 35W specs of the processors. However, since both companies have their own systems to dynamically scale back power by slowing down the CPU like SpeedStep and PowerNow, could the power labels be inaccurate as a measure of battery life in this case? Just like how clock-vs-clock benchmarks are no longer valid, could the same now be true about Watt-vs-Watt measurements?
*shrugs* Then again, AMD might just be flat out decieving on the benchmarks because they are thinking of their shareholders rather than their customers.
--
Free iPod? Try a free Mac Mini
Or a free Nintendo DS, GC, PS2, Xbox
Wired article as proof
From TFA:
This kind of trickery in the benchmark game does little for the vendor.
I have to disagree with this one. Fudging benchmarks almost always helps the vendor, except with very specialized (/.) audiences.
Remember, most people just see a bunch of random numbers when they shop for laptops, and compare processors based on GHz. They're more likely to read a blurb (or hear from the salesperson) that Turion outperforms it's competitors than they are to search blogs about the truth to the claim.
Now, by making enough of a fuss over this, we can create negative publicity, but why rag on AMD when, as the article states, all the other companies have set precedent?
It seems like this is becoming necessary (well, maybe not fiddling, but at least adding additional support for certain applications).
For example, Half-Life 2 is a very popular game. If nVidia starts messing with their drivers to run HL2 better, but ATI does not, then guess who HL2 fans are going to buy from?
I know -- the case mentioned in this article is completely different and not a useful change -- but it just got me thinking about past occurances.
It seems like drivers for GPUs should be able to run well in general, and applications should be catered to them, rather than the other way around, but I guess it's just not a viable option.
*Hugs his Pentium 4*
bloodclotjungletekno
In other news, I think oranges taste much better than apples.
The Dude abides.
All benchmarks are 32-bit. Turion is 64-bit.
I wonder how it will perform with 64-bit linux. Well, I'm going to see when some brand name shows up with Turion in his higher class.
Signature Pro version 1.13.2-3 release 83.5 beta3try7 after-breakfast edition
A fair comparison of power use would include total power used by cpu+chipset+video+memory. I remember reading somewhere that AMD cpu doesn't need northbridge, so that may give it an edge in total power use.
I think you meant Acrobat you want to boycott, not pdf. Xpdf loads the document in a snap for me.
Real programmers don't comment their code. It was hard to write, it should be hard to understand.
Press and hold the shift key as soon as Acrobat begins to launch. It'll skip the initialization process of the mostly-useless plugins. Doing this decreases the load-up time to less than two seconds on my 500mhz Celeron.
Intel shows it's thermal design power (TDP) at 27 watts for the 2Ghz chip, while AMD shows 35 watts. This is, however, an apples to oranges comparison. Intel's Prescott P4 at 2.8Ghz has a TDP of 89 watts, the same as an Athlon64 2800+. But according to this link, the P4 will actually draw 179 watts compared to the Athlon's 115. So, if the "marketing delta" holds true for the mobile line as well, we can expect the AMD solution rated at 35 watts to use roughly 45 watts of power at load, while the "27-watt" Pentium M will take 54 watts.
According to TFA, Turion notebooks might have 1/3 the battery life of Intel's Ultra Low voltage products. Now, Intel has a separate line of ultra-low-voltage Pentium M's, not to be confused with normal Pentium M's. The ultra low voltage Pentium Ms are only available at 1-1.2 Ghz, Turion's bottom out at 1.6 Ghz. So it's not a fair comparison. Regardless, with the power taken from the LCD and hard drives and stuff, I doubt it's even possible for an ultra low voltage Pentium M having 3 times the battery life of a Turion using the same battery.
So AMD and some other companies did some benchmarks with unequal systems. And?
It would certainly have been less misleading to use an Intel laptop with a similar video card at least, but it isn't like you are going to see these benchmarks on television ads or anything. I doubt any big buyers will care about benchmarks of pre-release products anyway.
Granted, this is rather shady of AMD, but it doesn't even approach the raw evil of, say, a company joining BAPCO and systematically removing all benchmarks in their Sysmark tool in which AMD wins. No, that would be unprecedented in this industry's history.
It's also noteworthy that TheRegister has a partnership with Tom's Hardware in the U.S., and some editors of Tom's have been noted as being overtly biased towards Intel, though Tom's itself seems to be getting better, having articles like the used to--real tech info rather than the sensationalized, poorly written crap which had infested my once favorite hardware site.
Granted, both AMD and Intel are "evil" for-profit companies, but something like an unfair benchmark hardly brings tears to the eye when you consider some of the staggering bullshit actions of the past.
What it all comes down to is preference--The Turion is going to be a 64-bit chip (isn't it?) with the benefits of AMD64 mode (most of which involve the fact that it has double the general-purpose registers in the chip, and not from the fact that those registers are 64-bits wide). The Turion will likely outperform the Pentium-M in most test, like the Athlon64.
The Pentium-M, however, will perform just fine thankyou, and will drain less battery power and thus be in cooler-running laptops with better battery life.
I'd pick the Pentium-M myself, since to choose a product based on anything other than overall effectiveness/price ratio set is usually either fanboyism or poor research.
Computer Science is no more about computers than astronomy is about telescopes. --E. W. Dijkstra
A couple years ago, I was at an AMD Press event thingy because they were giving away free stuff. At the event they computers set up with UT2003. Yes, they did the same thing. They gave they the intel computers onboard graphics, while thier computers had nvidia cards.
Games especailly are bound to the video card in terms of performance.
Fair?
Yeah, right.
In America we are imprisoned by our fear of them.
If I recalled correctly, nVidia or ATI fiddled with benchmarks by disabling certain features which take up a lot of processing power when they detected 3dmark or other benchmarking utilities.
Technically AMD didn't falsify their benchmarks by disabling certain features. They just chose a easier opponent to fight with. (sorta like when one picks a fight with someone that looks way weaker than you to guarantee you a win).
What I find weird benchmarks usually don't just give out percentages, but actual figures. Like # of flops per second, etc. I'm not sure if any other graphs were provided, but from the links that were in the opening topic, they were all percentages. If the benchmark had listed actual results, it would've been easier to do actual comparisons with other cpus which were not benchmarked against.
Saying all that, I don't believe AMD falsified any information (unlike nVidia or ATI). What they did was purely comparing a weaker opponent. Sorta like taking the ATI Radeon 9800 and comparing it to the nVidia Geforce MX440. But comparisons like that do exist when you do a wide range of benchmarking. That's why I always make sure I know what the heck is being benchmarked or else, it'd be just throwing #s at me. It's nice that newer benchmarks tell you if higher is better or if lower is better. Sometimes it's quite easy to get confused on if something w/ a higher # is better or not.
HD Trailers
While the article accuses AMD of comparing "Apples vs. Bananas" (their subtitle, not mine) the author includes a pro-Intel quote containing the same type of flawed comparison they accuse AMD of making. Much of the article was complaining about the lack of battery life benchmarks. Example:
"The answer is that the battery life isn't so good," Reynolds said, adding that Turion-powered systems could have up to one-third less battery life than laptops running on Intel's ultra low voltage products.
Turion is not an ultra low voltage CPU and does not claim to be. Guess why Intel's ultra low voltage products are ultra low voltage products? It's because their clock speed is limited to a mere 1.2GHz so the voltage can be lowered to reduce power consumption and increase battery life. Had AMD compared the 2GHz Turion to a ULV 1.2GHz Pentium M, yes, Turion battery life would be lower, but AMD's benchmarks would have been a legitimate 50+% higher.
1st, why is AMD naming operating techniques with that 'n' in there (so far as I know, there's cool'n'quiet and thin'n'light)?
McDonald's has a sandwich called the big'n'tasty. Does this mean that I will soon be able to get my McAthlon with fries (and greasy thermal paste on top)?
I don't know about you, but I think the future will taste delicious!
Secondly, (and more seriously) does this question the validity of the performance their other products, such as athlon64? Or have those other lines of products so far been tested true?
It makes me concerned because I was dumb enough to spend more money than I should've on an athlon64 3400+ (I should've waited until the price dropped. But, I really wanted it!)
In any case, I still prefer athlons over pentiums.
Look at the system specs yourself.
They are both absolute base systems that share clockspeed, memory and price.
It's not like we are comparing G4s to P4s here.
As for power consumption, had he bothered to actually dive into the whitepapers, he would find that particular Turion at 35 w while the Pentium M is 27.
No mention is made of the Turions available at 25w.
But he does mention the 9 watt Pentium M that runs at half the clockspeed taking it completely out of this class.
Mentions the 35,27 and 9 but not 25.
No, no, that might be a favorable data point.
Might as well be a Fox Spinner.
If voting were effective, it would be illegal by now.
One of the biggest problems the Register had with the AMD test setup versus the Intel test setup is that the AMD setup "had a graphics processor from ATI" while the Intel setup had "Intel's integrated graphics processor". It doesn't mention that the ATI video, the Mobility Radeon Xpress 200 series, is also an integrated solution.
. aspx?i=2269&p=18) Anandtech review of the chipset the the Xpress 200 integrated graphics debuted on. As expected of integrated graphics, it performs much worse than the lowest tier graphics cards ATI is currently making, the Radeon X300.
From the article, "An AMD spokeswoman insisted the company picked 'the most comparable offering from the competitor' that it could find, even though it didn't actually do that."
Well, AMD doesn't make any integrated graphics solutions, and the Radeon Xpress 200 series is one of the only integrated graphics options available. Benchmarks of the Radeon Xpress 200 can be found in this(http://www.anandtech.com/cpuchipsets/showdoc
AMD rigging benchmarks? Maybe. But the proposition that AMD did not choose the closest video to Intel's offering -- that it had availale -- is false.
The more sensational the report, the more hits the Register will recieve. It is ironic that while chastising AMD for fiddling benchmarks to sell more units, it fiddles with rhetoric to increase popularity.
(I type this at a Pentium M laptop, this is not AMD fanboyism)
My first laptop was an Athlon 1.2, and the battery life majorly sucked (i'm talking mere minutes not even close to an hour), you may have even said "battery...what battery?" So when I enrolled in a Game Dev class I needed to upgrade due to crappy video. I got an athlon64 based system and my friend got a Pentium-M centrino system with similar video and similar processor speed, etc. and my machine will run circles around theirs. If i'm running on battery, I dim my screen, etc. Granted, I do not have quite the battery life as they do, but I feel I have a very nice balance between power and battery life.
AMD has made some major headway in the mobile processor market and I believe they will continue making improvements. Just look what they did with desktop processors and look what they have already done with their mobile processors. Its unfortunate that the test setup was flawed, but the 'centrino' package is being pushed very hard in advertising. I'm not trying to make any excuses, and I admit that I haven't done much research since I got my notebook, but are there many notebooks offered with the 25 watt pentium-M that isn't centrino based?
The argument is that the wattage of the two compared CPUs was not identical, and therefore the results should not be compared.
Does this mean that it's "fiddling" to compare a high wattage Prescott core Pentium 4 with a lower
wattage Athlon 64?
Would it be "fiddling" if you matched laptop wattage overall? (The P-M needs more support chips after all). Would it be "fiddling" if you matched chips based on equal price? Would it be "fiddling" if you matched laptops based on equal weight?
No. The comparison of the chips is fair.. AMD wasn't being deceptive about which chips they were comparing. The price, weight, frequency, cache size, wattage, and instruction set support of both chips are not secret.
The Register is just making noise to get notice and readers.
1. AMD used a standard mobile chipset for it's notebook -- and ATI chipset, with integrated graphics. Then they used an intel chipset with integrated graphics. 2. It compared a 2ghz part to a 2ghz part -- it's a reasonable thing to do as an AMD sponsored benchmark. Still no fiddling. 3. The register author does a lot more speculating and throws a few w.a.g'es out there regarding battery life. They're trying to get mfrs interested in their new product. 4. We won't really know about battery life until we see fraternal twin laptops. IE: all other things being equal -- processor / chipset different. To be honest, I have no idea how it will turn out -- it'll be interesting to see. In short -- the author (dishonestly) takes a matter of opinion and presents it as a matter of principle. AMD vs Intel aside -- find some integrity buddy.
Let me guess, you're an AMD fanboi.
Let's take a look at the specs of test systems side by side, shall we?
Intel system has this shitty "Extreme" integrated graphics with no dedicated RAM. If you've ever used a system with Intel Extreme integrated graphics, you know it's dog slow. Putting it into a laptop with slowish system bus doesn't make it any better.
Now let's look at AMD system. It has integrated GPU (!) from one of the leaders in computer graphics hardware with _32M_ of _dedicated_ Video RAM. I wonder if I've emphasized this clearly enough to get through your thick skull.
You don't have to be a genius to say that AMD system has an advantage when it comes to 3D graphics.
Wouldn't it be cool if instead of cooking the benchmarks AMD put the processors head to head and made them use the same Radeon Mobility 9600 card with 128M RAM? Why didn't they do this? Were they afraid of something?
I mean, come on, we all know that they make the best desktop processors at this point. Their mobile chips, however, leave much to be desired, and with the release of this Turion chip status quo remains the same.
Actually, if you look at benchmarks on the fastest Pentium-M chip, they beat the high-end P4 quite consistantly.
? i=2342&p=21
I looked at the benchmarks and they definitely *do not* beat the high-end P4's consistently.
The Pentium-M compares to the P4 much the same way that the old Cyrix chips compared to a Pentium- they do well on non-CPU intensive tasks such as Microsoft word and internet explorer, but the weak FPU hinders its raw performance in CPU hungry tasks. The Pentium-M's floating point performance is slightly better than half of the high end P4's.
When you design a chip to be low power, you have to make tradeoffs. Intel designed a pretty efficient chip that delivers good performance for the amount of wattage it consumes, but it shouldn't be confused with more powerful desktop chips. It does well in light applications that aren't really CPU hungry and won't bog the processor.
Anandtech did a pretty thorough review of the Pentium-M and how it compares to desktop chips.
"As a mobile processor, the Pentium M cannot be beat - we've actually seen why, even in this comparison today. With a highly power optimized architecture, the Pentium M continues to deliver performance that is competitive with other mobile CPUs on the market. The problem is that in the transition to the desktop world, its competitors get much more powerful, while the Pentium M is forced to live within its mobile constraints."
http://www.anandtech.com/cpuchipsets/showdoc.aspx