Slashdot Mirror


Futuremark Delists Samsung and HTC Android Devices for Cheating 3DMark

MojoKid writes "Benchmarks are serious business. Buying decisions are often made based on how well a product scores, which is why the press and analysts spend so much time putting new gadgets through their paces. However, benchmarks are only meaningful when there's a level playing field, and when companies try to 'game' the business of benchmarking, it's not only a form of cheating, it also bamboozles potential buyers who (rightfully) assume the numbers are supposed mean something. 3D graphics benchmark software developer Futuremark just 'delisted' a bunch of devices from its 3DMark benchmark results database because it suspects foul play is at hand. Of the devices listed, it appears Samsung and HTC in particular are indirectly being accused of cheating 3DMark for mobile devices. Delisted devices are stripped of their rank and scores. Futuremark didn't elaborate on which specific rule(s) these devices broke, but a look at the company's benchmarking policies reveals that hardware makers aren't allowed to make optimizations specific to 3DMark, nor are platforms allowed to detect the launch of the benchmark executable unless it's needed to enable multi-GPU and/or there's a known conflict that would prevent it from running."

3 of 188 comments (clear)

  1. Re:And why do you think they are? by SuperKendall · · Score: 5, Interesting

    The name is irrelevant

    Not to Samsung phones! Which was kind of my point.

    The combination of DRM key and code identifies to the OS the precise application and whether or not it's allowed to run

    And anyone who has jailbroken a phone can easily re-sign an application bundle if they chose, renaming it or doing whatever else they like with it.

    Or if you are compiling the benchmark yourself you just change what you like.

    Apple need only boost applications according to a mathematical model

    Yeah, you see that's called "compiler optimization" and applies to all applications, not just benchmarks.

    Again, when you ship with a fast enough processor you don't need to waste time scanning for benchmarks.

    Funny how iPhone battery falls 20-30% faster and the phone runs substantially hotter when running 'official' benchmark apps

    Funny how I've never noticed that at all in my own testing. And in fact sometimes the system will get hot when playing commercial games.

    Oh wait, it's not funny at all - you're an Apple-Hater AC so we know not to believe anything you post.

    Apple betrayed by the only thing they can't control, the laws of physics.

    That's funny because Apple seems to be the only smartphone maker paying attention to such laws, not building needlessly dense displays that suck power like a kid with a juice box.

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
  2. Here's the RDF at work. by Anonymous Coward · · Score: 5, Insightful

    When Apple brought out Retina Display, that was 100% FAN BLOODY TASTIC according to Apple fans. Absolutely the best thing EVAR, and PROOF Apple are "innovative" by making displays finer in resolution than any other smartphone.

    Nokia didn't count, since they were ~12ppi lower resolution! SHUT UP!

    But now resolution is higher than Retina Display, higher resolution and better pixel count is BAD. Which, yet again, PROVES Apple are BEST EVAR because they don't waste time trying to get uselessly higher resolutions!

    7" tablets were too small when the iPad was only 10".

    But when the iPad mini comes out at 7", it's the best size for many many tasks!

    Phones were too big if they had a 4.3" screen. Until Apple brought out a bigger screen, then they had many uses!

    And so on.

  3. Re:1080P Phones by nogginthenog · · Score: 5, Informative

    Taken from the Apple/Google sites:
    16GB iPhone 5C: $549
    16GB Nexus 5: $349