Slashdot Mirror
How Many Android OEMs Cheat Benchmark Scores? Pretty Much All of Them
An anonymous reader writes "After Samsung got caught out cheating on benchmarks (Note 3, Galaxy S4) AnandTech has done a detailed analysis of the state of benchmark cheating amongst Android OEMs. With the exception of Motorola, literally every single OEM they've looked at ships (or has shipped) at least one device that does benchmark-specific CPU optimizations. AnandTech also thinks it will get worse before it gets better. 'The hilarious part of all of this is we’re still talking about small gains in performance. The impact on our CPU tests is 0 - 5%, and somewhere south of 10% on our GPU benchmarks as far as we can tell. I can't stress enough that it would be far less painful for the OEMs to just stop this nonsense and instead demand better performance/power efficiency from their silicon vendors.' The article notes that Apple doesn't do any of the frequency gaming stuff."
With the exception of Motorola...
And Apple. Apple and Motorola/Google are the only two companies that don't boost their devices for benchmark tests. If you're going to give credit to one, please do be fair and give credit to the other.
I respect both of them for that level of integrity and I hope they stick to their guns and remain honest.
I may be an Apple fanboy (and I am) but I'm really looking forward to seeing what Motorola starts releasing in about a year once Google's able to, as they said, flush things out of the system and start releasing truly Google-designed products.
'The hilarious part of all of this is we’re still talking about small gains in performance.'
The even more hilarious part is that OEMs are going to the trouble to do this when CPU benchmark scores are a very small factor in the decision of most consumers to buy these phones. I doubt that the ROI is higher than say, oh, improving the user experience of the GUI or call QoS.
The performance quoted simply is not available to apps that are not on a whitelist of benchmark applications. It literally does not represent any part of the phone's non-benchmarking performance.
No kidding!!! What do you say at this point?
Pfff. Car manufacturers tape up the air intakes and door seams on their cars to do fuel economy runs, just to eek out the every last 0.1mpg. Running your car like that for any reasonable period of time would wreck the engine pretty quick.
Benchmarks are about as useful as manufacturer spec sheets. Take both with a a few metric tonnes of salt.
Finally had enough. Come see us over at https://soylentnews.org/
The discrepancy is deliberate. For a long, long time drive capacity was quoted in the same units that the computer used for storage: binary SI prefixes, not decimal ones. The change to "1 megabyte = 1 million bytes" didn't set in until the 2000s.
Kudos to Apple for making their specs and their OS use consistent units, but it's still a marketing bullshit decision.
No kidding!!! What do you say at this point?
I was under the impression that what they were doing was ensuring that the clock speed was running at full, not slowed down for power saving, etc.
No. They are running at a clock speed that no real application will see under any circumstance, either the GPU or CPU cock increased.
It is within what the parts are rated for but not what the device was built to run at normally.
I assume some games and other applications also force the processor to full
There is no way to build a game on Android that can run at the speed the benchmarks are getting run at on each of the devices "cheating".
"There is more worth loving than we have strength to love." - Brian Jay Stanley
No, he was right. The phones CAN and DO reach that clock speed. Read the AnandTech article. The graph of CPU speed shows it quite clearly.
Actually, manufacturers do report their own efficiency numbers, and the EPA spot-checks them.
http://business.time.com/2012/12/10/more-reason-to-be-skeptical-about-new-car-mpg-claims/
If you are trying to say AMD's compiler rigs? Sorry you are full of shit, the reviewer that first found the cripple code in ICC did rigorous testing of AMD's compiler and found it was merely GCC with some extra X64 flags added to better support 64 bit, that is it. Oh and you can download the source for the ACC and see for yourself, its based on GCC and is fully GPL V2 compliant.
ACs don't waste your time replying, your posts are never seen by me.