Slashdot Mirror

← Back to Stories (view on slashdot.org)

How Many Android OEMs Cheat Benchmark Scores? Pretty Much All of Them

Posted by Soulskill on from the at-least-they're-consistent dept.

An anonymous reader writes "After Samsung got caught out cheating on benchmarks (Note 3, Galaxy S4) AnandTech has done a detailed analysis of the state of benchmark cheating amongst Android OEMs. With the exception of Motorola, literally every single OEM they've looked at ships (or has shipped) at least one device that does benchmark-specific CPU optimizations. AnandTech also thinks it will get worse before it gets better. 'The hilarious part of all of this is we’re still talking about small gains in performance. The impact on our CPU tests is 0 - 5%, and somewhere south of 10% on our GPU benchmarks as far as we can tell. I can't stress enough that it would be far less painful for the OEMs to just stop this nonsense and instead demand better performance/power efficiency from their silicon vendors.' The article notes that Apple doesn't do any of the frequency gaming stuff."

3 of 189 comments (clear)

  1. And Apple by Anonymous Coward · · Score: 5, Informative

    With the exception of Motorola...

    And Apple. Apple and Motorola/Google are the only two companies that don't boost their devices for benchmark tests. If you're going to give credit to one, please do be fair and give credit to the other.

    I respect both of them for that level of integrity and I hope they stick to their guns and remain honest.

    I may be an Apple fanboy (and I am) but I'm really looking forward to seeing what Motorola starts releasing in about a year once Google's able to, as they said, flush things out of the system and start releasing truly Google-designed products.

  2. Re:Or, alternatively by L4t3r4lu5 · · Score: 4, Informative

    Pfff. Car manufacturers tape up the air intakes and door seams on their cars to do fuel economy runs, just to eek out the every last 0.1mpg. Running your car like that for any reasonable period of time would wreck the engine pretty quick.

    Benchmarks are about as useful as manufacturer spec sheets. Take both with a a few metric tonnes of salt.

    --
    Finally had enough. Come see us over at https://soylentnews.org/
  3. Re:Or, alternatively by ftobin · · Score: 4, Informative

    Actually, manufacturers do report their own efficiency numbers, and the EPA spot-checks them.

    http://business.time.com/2012/12/10/more-reason-to-be-skeptical-about-new-car-mpg-claims/