Slashdot Mirror


New Analysis Casts Doubt On Intel's Smartphone Performance vs. ARM Devices

MojoKid writes "A few weeks ago, the analyst company ABI Research published a report claiming that Intel's new CloverTrail+ platform (dual-core Medfield) for smartphones was significantly faster and more power efficient than anything ARM's various partners were shipping. If you follow the smartphone market, that was a very surprising claim. Medfield was a decent midrange platform when it launched in 2012, but Intel made it clear that its goal for Medfield was to compete with other platforms in its division — not seize the performance crown outright. Further investigation by other analysts has blown serious holes in the ABI Research report. Not only does it focus on a single, highly questionable benchmark (AnTuTu), the x86 version of that benchmark is running different code than the ARM flavors. Furthermore, the recently released Version 3.3 of the test is much faster on Intel hardware than on any of the other platforms. But even with those caveats in place, the ABI Research report is bad science. Single-source performance comparisons almost inevitably are."

19 of 94 comments (clear)

  1. Re:It just kind of seems DOA by h4rr4r · · Score: 4, Insightful

    Not if you use the NDK, which most games and video applications will use for performance reasons.

    You can of course compile for both.

  2. Re:Queue by K.+S.+Kyosuke · · Score: 4, Funny

    Why would you have to queue them? I'm sure each of them has his own device, it's not like they have to contend for them with each other.

    --
    Ezekiel 23:20
  3. Clearly this can't be true by NoNonAlphaCharsHere · · Score: 2

    Intel cooking the books on a benchmark? Never happen.

    1. Re:Clearly this can't be true by BasilBrush · · Score: 2

      Whenever an analyst produces a dishonest research report, look to the company that benefited from the dishonesty for the source of funding of the report.

    2. Re:Clearly this can't be true by Synerg1y · · Score: 2

      In addition, how much did Intel pay ABI for the report I wonder?

      With Intel wanting to get a bigger share in the mobile market as bad as they do (they restructured around it), paying for a favorable research report doesn't sound all that out of scope.

    3. Re:Clearly this can't be true by NoNonAlphaCharsHere · · Score: 4, Insightful

      Intel has a VERY long history of questionable ;) benchmarks, all the way to tweaking processor designs to run benchmark code faster. Microsoft's "Get The Facts" propaganda is just a pale imitation of Intel's history.

    4. Re:Clearly this can't be true by djohnsto · · Score: 2

      Intel has a VERY long history of questionable ;) benchmarks, all the way to tweaking processor designs to run benchmark code faster. Microsoft's "Get The Facts" propaganda is just a pale imitation of Intel's history.

      Supposedly, benchmarks are written to simulate real workloads. It seems to me that tweaking processor designs to run benchmarks faster is a good idea. If you have a better idea for what applications to design a processor for perhaps you should join a processor company in the workload analysis or planning group.

      I agree that Intel, or any company with enough "muscle" (see Nvidia, ATI/AMD, MSFT, IBM, HP, Oracle, etc.), will try to influence the press and show their products in the best possible way, even dishonestly on occasion. I'm not saying that the fact that "everyone does it" makes it right, just that everyone does do it.

      I also think that these companies receive a lot of blame for articles / "research reports" like this one that they have absolutely no involvement with. To think that EVERY positive Intel article is funded or influenced by Intel would:

      1) Ignore the fact that Intel does make fairly good products most of the time
      2) Greatly overestimate Intel's marketing prowess. :)

      --
      Dan
    5. Re:Clearly this can't be true by makomk · · Score: 3, Insightful

      Wow. What's more, apparently that optimisation was added by Intel after the benchmark was developed:

      What's more, this optimization wasn't present in ICC until a recent release. Somehow I don't think that they just now discovered it has general purpose value. More likely case is that they discovered is they could manipulate AnTuTu's scores. Seems to coincide well with this third-party report appearing showing how amazing Atom's perf/W is - using nothing but AnTuTu. Or the leaked scores seen for CloverTrail+ and now BayTrail that are AnTuTu. Is this really a coincidence?

      So basically they modified their compiler to optimise away the actual benchmark, then got someone to release a third-party report based solely on the benchmark they'd just manipulated the results of.

  4. Clover Trail vs Medfield by Anonymous Coward · · Score: 5, Interesting

    I design chips and work for intel.

    Clover Trail is significantly more power efficient than Medfield because it has a lot more power control stuff in it, so more of it is turned off most of the time. This is not a big secret as far as I know.

    The rest of the phone consumes power too, particularly the screen. So YMMV.

  5. Biased benchmarks endemic in chip industry by OneAhead · · Score: 4, Insightful

    News at 11.

    I don't think there's any chipmaker (CPU, GPU or otherwise) who hasn't been caught doing it. Not that that makes it right, of course.

    For the quick readers, note that this is about Clover Trail, not to be confused with the recently announced Bay Trail. Though it does cast doubts on Intel's claims about the latter's performance...

    1. Re:Biased benchmarks endemic in chip industry by evilviper · · Score: 2

      note that this is about Clover Trail, not to be confused with the recently announced Bay Trail

      I anxiously await Intel's new "Oregon Trail" CPU.

      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
    2. Re:Biased benchmarks endemic in chip industry by Guppy · · Score: 3, Funny

      I anxiously await Intel's new "Oregon Trail" CPU.

      Might be a long wait, I heard a bunch of their engineers mysteriously died of dysentery while working on the design.

  6. Re:It just kind of seems DOA by JDG1980 · · Score: 2

    App stores make this less of an issue. If Apple wants to switch the iDevices to x86, they'll tell developers that they must have an x86 build posted by such-and-such date, or their app will be dropped from the store.

  7. Re:It just kind of seems DOA by phantomfive · · Score: 5, Insightful

    If you write your code in C, you can port it relatively easy to iPhone, Android, WP8, and Blackberry (depending how much UI code you have). If you write it in Java, avoiding the NDK, you have to do two-four times as much work to port it.

    Which would you rather do, use the NDK and recompile, or write once for each platform? "The right way" isn't always a single choice, it's usually a compromise.....

    If Intel processors become popular in Android phones, Google will probably introduce a multiple-architecture executable format, much like iPhone does with FAT and MACH (currently around 70% of apps for the iPhone have two architectures, one for ARM7 and one for ARM7s).

    --
    "First they came for the slanderers and i said nothing."
  8. Streisand effect in action! by Anonymous Coward · · Score: 3, Interesting

    What's really hilarious is that this supposedly "rigged" benchmark got almost zero press (never posted on Slashdot or mentioned on major tech websites) when it first came out. Now the ARM religion has to wage a jihad on anybody who claims that it is physically possible to use silicon that doesn't include royalty payments to the Church of ARM to run your smartphone.

    Using my own ARM-based smartphone and the Dexplorer app, I looked at Antutu and other common Android benchmarks. One interesting thing that stood out was that Antutu uses the NDK (i.e. there are C-compiled libraries for both ARM + x86 ABIs). The other benchmarks like quadrant and Linpack were pure java based. Basically what I'm seeing is that the Clovertrail+ x86 hardware absolutely is in the same ballpark as the snapdragon 600 for performance, but that the ARM vendors and Android developers have been optimizing Dalvik for the ARM architecture since 2009 while little has been done for x86. Once a real compiler like GCC gets to generate C-code for the x86 Atoms though (and also for the ARM parts BTW, it's a level playing field), then we see that Clovertrail+ puts up decent competition for modern ARM chips in the same power envelope.

    As for the usual: "Intel cheats using compilers!" whine, please take a big dose of vitamin STFU. I've been bored to death for over a decade by the drone of the ARM jihadists who claim that their architecture is so magical that you can literally knock back a fifth of Tequila and vomit up a perfectly optimized web browser before breakfast. If ARM is truly so beautiful and if the engineers at ARM are truly such geniuses, then it should be trivial for them to implement compilers that blow away x86.

    1. Re:Streisand effect in action! by Bert64 · · Score: 2

      When it comes to compiled code the situation is reversed, gcc has been heavily optimized for x86 whereas other architectures although supported have had far less work done on them. There's also Intel's compiler which generally produces faster code than gcc.

      --
      http://spamdecoy.net - free throwaway anonymous email - avoid spam!
  9. Which they do not do by SuperKendall · · Score: 5, Informative

    App stores make this less of an issue. If Apple wants to switch the iDevices to x86, they'll tell developers that they must have an x86 build posted by such-and-such date.

    Apple DOES NOT do that for older apps. It will often place restrictions on app updates or new app submissions (like must include iPhone5 screen shots) but they don't ever force older apps to do any kind of update - nor should they.

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
  10. AnTuTu assembly code analysed by edxwelch · · Score: 5, Informative

    Some guy on the Anandtech forums analysed the AnTuTu code and found that it indeed had been tweeked to favor x86 processors:
    http://forums.anandtech.com/showthread.php?t=2330027

  11. Re:Queue by ShanghaiBill · · Score: 4, Insightful

    The arm fanboys

    We are not "ARM fanboys". We are "Intel-haters". There is a difference.