Slashdot Mirror

← Back to Stories (view on slashdot.org)

Apple Hardware VP Defends Benchmarks

Posted by pudge on from the i-blame-florida dept.

Greg Joswiak, vice president of hardware product marketing at Apple, in a phone interview today, defended Apple's performance claims for its upcoming Power Mac G5, after they came under fire in the wake of yesterday's announcement. Read on for the details. Joswiak went over the points in turn, but first said that they set out from the beginning to do a fair and even comparison, which is why they used an independent lab and provided full disclosure of the methods used in the tests, which would be "a silly way to do things" if Apple were intending to be deceptive.

He said Veritest used gcc for both platforms, instead of Intel's compiler, simply because the benchmarks measure two things at the same time: compiler, and hardware. To test the hardware alone, you must normalize the compiler out of the equation -- using the same version and similar settings -- and, if anything, Joswiak said, gcc has been available on the Intel platform for a lot longer and is more optimized for Intel than for PowerPC.

He conceded readily that the Dell numbers would be higher with the Intel compiler, but that the Apple numbers could be higher with a different compiler too.

Joswiak added that in the Intel modifications for the tests, they chose the option that provided higher scores for the Intel machine, not lower. The scores were higher under Linux than under Windows, and in the rate test, the scores were higher with hyperthreading disabled than enabled. He also said they would be happy to do the tests on Windows and with hyperthreading enabled, if people wanted it, as it would only make the G5 look better.

In the G5 modifications, they were made because shipping systems will have those options available. For example, memory read bypass was turned on, for even though it is not on by default in the tested prototypes, it will be on by default for the shipping systems. Software-based prefetching was turned off and a high-performance malloc was used because those options will be available on the shipping systems (Joswiak did not know whether this malloc, which is faster but less memory efficient, will be the default in the shipping systems).

As to not using SSE2, Joswiak said they enabled the correct flags for it, as documented on the gcc web site, so that SSE2 was enabled (the Veritest report lists the options used for each test, which appears to include the appropriate flags).

5 of 1,081 comments (clear)

  1. Re:Curious by pudge · · Score: 5, Informative

    Eh, we do this sometimes, when it is appropriate. In this case, I have a PR contact at Apple who asked me last week if I wanted to talk to someone about WWDC, and we set up a call last weekend, for this afternoon. It just happened to coincide with the benchmark discussion, which Greg was eager to set straight (he had read the arguments and already compiled his responses :-). We also talked a bit about some other topics, but nothing of interest that you haven't read elsewhere.

  2. Re:Benchmarks by j3ffy · · Score: 5, Informative
    But what was even more inpressive than the spec scores was watching the powermac squash the dual xeon in several applications from 3D video rendering, to photo editing, to audio processing, to mathematical calculations.

    I'm a science guy, and for the calculations and simulations done here at the physics dept. where I work, the IBM power4 kills just about everything else. And when I saw the powermac calculate fractals with mathematica faster than the xeon box by more than a factor of 2, I was very excited (although a little cautiously) to see we will soon get power4 performance for well under $20,000

  3. Re:But he didn't refute the most damning claim! by doce · · Score: 5, Informative

    this is the funniest claim i've seen in a while. not only does apple do this, so does dell, and so does virtually every consumer-oriented company on the planet. gas companies shave a TENTH of a PENNY off gas prices to make them seem cheaper.

    a department store (was it macy's?) started this practice. the funny part is that the aim wasn't really to fool consumers into thinking it was less expensive. alas, the real purpose was to force cashiers to open the register, since the customer was almost always going to be due some change.

    --
    woof!
  4. Re:Apple: innovation or catch up? by tuxedobob · · Score: 5, Informative

    I don't think anyone's really arguing that most of Apple's improvements weren't catch up. Heck, when Steve Jobs was on stage he was saying how everything they added was "the latest PC standard". How could it not be catching up if it's already a standard on the PC?

    The reason the Mac users are happy about all this is because we already knew we were way behind, and we've been begging Apple to catch up!

    Even considering all the benchmarks, which may or may not be accurate, the simple fact remains that this Mac is much faster than the previous Mac. Which is good news for Mac users. And presumably the crowd at the keynote was full of Mac users.

  5. Photoshop WAS compiled with the AIX compiler! by kriegsman · · Score: 5, Informative
    I don't know if they still do, but for a while at the beginning of the PowerPC era Adobe was using the AIX compiler to generate its PowerPC binaries for Photoshop.

    Admittedly, this was when the PowerPC was pretty new, and the choices were the IBM/AIX compiler which was robust and produced fast code but required an AIX box in addition to a Power Mac, or the nacent Metrowerks CodeWarrior compiler which run natively on the Power Mac, but generated poorly optimized code.

    If I recall my history timeline correctly, after CodeWarrior came
    • the Apple MPW "MrC" compiler (better code than CodeWarrior 1.0, but with a wacky command-line "IDE"), then
    • gcc for PowerPC (cruddy code back then), then
    • the Motorola PowerPC compiler (better code than Apple's compiler, with NO IDE - it plugged into the CodeWarrior or MPW IDE).
    • Then Motorola inexplicably stopped selling their compiler.
    • Later Motorola bought Metrowerks.
    • Somewhere along the line, gcc learned to generate better PowerPC code.
    • Eventually, Apple pretty much shelved their "MrC" compiler, and settled on using gcc for Mac OS X
    • Monday, Apple released their "Xcode" environment -- still using gcc, I believe.
    Apple's MPW tools are still available (free) here for Mac OS 7/8/9. The new Mac OS X tools including Xcode are available here.

    As a side note, it's really nice to see Apple giving away a full development suite for free, and continuing to put development time and effort into improving it.

    -Mark