Slashdot Mirror

← Back to Stories (view on slashdot.org)

Chinese Company Oppo is the Latest To Be Caught Cheating on Phone Benchmarks (engadget.com)

Posted by msmash on from the fraud-alert dept.

An anonymous reader shares a report: You can add another big name to the list of phone makers found cheating on benchmarks. UL Benchmarks has delisted Oppo's Find X and F7 phones from its 3DMark charts after testing from itself and news outlet Tech2 revealed that both devices were artificially ramping up processor performance when they detected the test by name. Oppo acknowledged that it always stepped things up when it detected "games or 3D Benchmarks that required high performance," but claimed that any app would run full bore if you tapped on the screen every few seconds to signal your actions. UL, however, rejected the justifications. It was clear that Oppo was looking for the benchmark by name and not the extra processing load involved, according to the outfit. Moreover, tapping wouldn't be an effective solution if Oppo treated apps equally -- you couldn't get consistent results. Further reading: Huawei Caught Cheating Performance Test For New Phones.

16 of 82 comments (clear)

  1. I suspect mostly everybody does that! by ls671 · · Score: 2, Insightful

    I suspect mostly everybody does that from Facebook to Volkswagen etc. so it isn't limited to technology.

    Just don't get caught I guess...

    --
    Everything I write is lies, read between the lines.
    1. Re:I suspect mostly everybody does that! by FeelGood314 · · Score: 2

      And that is why we need to call companies that do this stuff out and punish them. We need to reward companies that don't and also we need to appreciate companies like UL Benchmarks and Tech2.

    2. Re:I suspect mostly everybody does that! by thegarbz · · Score: 1

      My computer BIOS literally presents the option to cheat on 3DMark...

    3. Re:I suspect mostly everybody does that! by ls671 · · Score: 1

      Sure buddy! I have heavy artillery available at will but I tend to usually adopt a more diplomatic tone when I post on /.

      --
      Everything I write is lies, read between the lines.
  2. How to not cheat by Sarten-X · · Score: 1

    It seems to me that it'd be fairly easy to add a slider control in the settings app that would let you choose power-saving vs. performance. When full, everything gets the CPU's full capabilities. Put it right next to the control for screen brightness, which people already go to when they want a bit more battery life.

    Then you can publish any measured benchmarks you want, and claim great performance and battery life, while pitching the configurability as a feature.

    --
    You do not have a moral or legal right to do absolutely anything you want.
    1. Re:How to not cheat by fuzzyfuzzyfungus · · Score: 1

      Some devices already do that(unless they've changed it recently, Nvidia-SoC ones had fairly granular control over power vs. performance through a custom settings widget); and basically all of them do in the limited sense that they have 'battery saver' as an option that prunes power consumption in some areas.

      I suspect that the cheaters aren't primarily motivated by confusion about how to implement this feature, though; but by the desire to get impossibly good numbers: if there is a slider the reviewer knows about they'll presumably run the battery and thermal benchmarks with it on the same setting they use for performance benchmarks; which will make it abundantly clear that there's a tradeoff, quite possibly a really harsh one. If you successfully sneak the settings past them you can potentially get performance benchmark numbers that would generate horrific battery life; and battery life numbers based on the handset mostly avoiding its most power hungry modes.

  3. Nothing original by 140Mandak262Jamuna · · Score: 1

    They copied the idea from Volkswagen.

    --
    sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
  4. They must have learned from we Americans... by bogaboga · · Score: 1

    GM also lied here too. Never under estimate the reach of American companies.

    1. Re:They must have learned from we Americans... by EvilSS · · Score: 1

      Huh, didn't even know GM made smartphones.

      --
      I browse on +1 so AC's need not respond, I won't see it.
    2. Re:They must have learned from we Americans... by bogaboga · · Score: 1

      Huh, didn't even know GM made smartphones.

      Dude, it's the motivation to lie for profit.

      You must have been to school, right?

  5. The Chinese Lying and Cheating? by DatbeDank · · Score: 1

    Gosh, haven't seen that before! /sarcasm

  6. This is a mindset... by trudyscousin · · Score: 1

    ...that values being clever over doing the right thing, with a cool eye toward the consequences.

    Adding melamine to baby formula to jack up the protein content? Clever. Not so clever when babies become sick and die because of kidney failure.

    Ripping off designs from other companies? Clever. Not so much money needed for R&D. Not so clever when consumers find out there's an inferior product under that familiar user interface.

    Cheating on benchmarks? And so it goes...

    --
    Those who can, do. Those who can't, write technology blogs.
    1. Re:This is a mindset... by tlhIngan · · Score: 1

      ...that values being clever over doing the right thing, with a cool eye toward the consequences.

      Adding melamine to baby formula to jack up the protein content? Clever. Not so clever when babies become sick and die because of kidney failure.

      Ripping off designs from other companies? Clever. Not so much money needed for R&D. Not so clever when consumers find out there's an inferior product under that familiar user interface.

      Cheating on benchmarks? And so it goes...

      No, it's a mindset of GREED over the right thing. Green, as in money. The US was once a country like it, but evolved beyond it by introducing regulations that prohibit all sorts of actions.

      Melamine in milk? Why, it came about in an attempt to turn 1L of milk into 2L of milk (or 1 gallon into 2 gallons). You do it by adding water. But an analysis would show that you actually watered down the milk, so add melamine to fool the measurement devices into recording it as double the amount of milk. Thus, you make twice the amount of money out of that milk.

      Ripping off designs is similar - you see those designs sell, so you make a counterfeit version and hope to sell enough to fool people into paying money for it Also means you don't have to do a bunch of R&D in trying to figure out why widgets work the way they do. And if the product is expensive, there will be gullible folks willing to pay half price for a similar looking product. All you need to do is fool them into buying your cheap copy.

      Benchmark cheating is the same - you want to show good numbers so people feel they're getting a good phone. If the phone benchmarks the same as every other phone in the store, it won't sell, so you want to show something ridiculously big so they can put it in big numbers on the phone tag and people will buy it because it has a big number.

    2. Re:This is a mindset... by gnasher719 · · Score: 1

      Just saying: The melanine thing _killed_ babies, but also got the main perpetrators killed. That was something you can't get away with even in China. The state had to be seen to do something about it, and they did. With death sentences.

  7. Long history by gnasher719 · · Score: 1

    Years ago, when graphics cards where slow enough that text output was benchmarked, some magazine found their benchmarks on a particular test ran 3 times slower than everyone elses.

    Their test displayed the string "The quick brawn fox jumped over the lazy dog." with a spelling error. Everyone else ran the benchmark with the correct text. It turned out the driver maker checked for this particular string, and had a ready-made bitmap for this strong stored into the driver. So they could use a bitblit instead of text drawing and ran three times faster. In the benchmark only.

  8. FLASHBACK 1993: Hercules and The quick brown fox by TheRealHocusLocus · · Score: 1

    repost

    My boss slaps a folded-over InfoWorld magazine onto my desk, thick enough to kill a rat with in those days. He says with obvious glee, "How bout dem apples?"

    It is Steve Gibson's INFOWORLD column of March 8 and Gibson (with obvious glee) has caught a manufacturer of Hercules graphics cards red-handed. The standard WinBench program had conducted a series of tests --- and in one particular test of text display, in which the phrase "The quick brown fox jumped over the lazy dog's back then sat on a tack" is continuously painted on the screen --- the card performed oddly spectacularly. It was that one score that when combined with the others, ranked the card above the competition. Suspicious, Gibson changed a single letter in the test phrase and the card's score dropped to a reasonable range. The card was apparently recognizing that a test was in progress and 'cheating' by failing to actually over-write this static text repeatedly.

    I love the comment by the manufacturer when Gibson contacted them (read it!) but what intrigued the industry the most was that the cheat was not to be found in the Windows driver code, it had been embedded into the firmware of the accelerator chip. In the next Winbench version the test phrase jumped around the lazy screen's back during the test, rendering the cheat obsolete.

    --
    <blink>down the rabbit hole</blink>