Intel Mac Performance Behind Hype

Barry Norton writes "Steve Jobs, at the MacWorld tradeshow, boasted: 'the new iMac [with] Intel processor is two to three times faster than the iMac G5.' MacWorld (the publication) has been putting the iMacs through their paces. The results are a good deal less impressive than Steve's boast, showing an average performance increase of 10 to 25 per cent while performing a series of everyday tasks with software specially designed for the new systems." Ars Technica had another perspective on the new systems earlier this week.

Comments

Newsflash!

[daveschroeder]

Company performs benchmarks to show product in best light!

From http://www.apple.com/imac/intelcoreduo.html:

2. Testing conducted by Apple in December 2005 using preproduction 20-inch iMac units with 2GHz Intel Core Duo; all other systems were shipping units. All scores are estimated.SPEC is a registered trademark of the Standard Performance Evaluation Corporation (SPEC); see www.spec.org for more information. Benchmarks were compiled using the IBM compiler and a beta version of the Intel compiler for Mac OS.

3. Testing conducted by Apple in December 2005 using preproduction 20-inch iMac units with 2GHz Intel Core Duo; all other systems were shipping units. All of the iMac and iMac G5 systems ran beta Universal version of Modo. All other applications were beta versions.

And since actual application performance has been subjective since the dawn of time, how is this surprising?

I mean, we're talking about a company that said no one wanted flash players until they made one, that no one wanted to watch video on an iPod until they made an iPod that played video, and that said all x86 architecture and CISC processors sucked until they switched to them.

And you know what? All of the above statements had significant elements of truth to them. Apple is doing nothing more than showing its products, accurately insofar as it goes, in the best possible light. Is this the least bit stunning?

Re:Newsflash!

[Jezza]

Actually Steve never said that the system was that fast, he was only talking about the chip. He said "the disks are 2 the 3 times as fast" so I don't see why we find it so amazing that the real world performance is lower than the benchmarks. It is always going depend on the application. So an application that spends most of its time waiting for the disk isn't going to see much of a speed up with a faster processor (it'll see a little because it'll move from wait-state to wait-state faster). For a lot of applications this is the norm (described as "disk bound").

What is probably more important (for home users) is actually something Steve side stepped, these new iMac should generate less heat and therefore run more quietly (because the fans won't need to spin as fast/often) for users in a domestic setting this is important.

I think most people who buy Macs (especially iMacs) are not buying it because they think it's the fastest computer around (amazing as it may sound there are other factors in the purchasing decision).

Re:Newsflash!

[Overly Critical Guy]

Here's the "crap from Jobs" you're talking about:

"Now everything's not going to run 2-3X. You know the disks aren't 2-3X faster, etc., but on the most important benchmarks, 2 to 3 times faster." - Steve Jobs from the keynote

Seems pretty honest to me. Amusingly, it's the sites like Slashdot leaping on the speed claims and obsessing over them, while Jobs himself gave them a real-world context in the keynote speech. Not that such a thing would get mentioned in the article submission...no, no, gotta get all those page hits from people bitching about Steve Jobs "lying." Sigh.

Re:Newsflash!

[Stephan Schulz]

I think people may be missing the fact that the dual core thing might be throwing a wrench into the mix,...

Exactly right. The G5 is an excellent processor, and clock for clock should approximately be able to keep up with one core of the CoreDuo. But the new iMac has a much better bus system, and, of course, the second core.

It's no coincidence that Jobs showed the SPECint_rate results that measure throughput, not the more often used plain SPECint that measures time-to-finish of a sequentially run suite of programs. So his claims are not exacly wrong...

I'll probably still wait for the second generation of new laptops before I upgrade from my TiBook.

Re:Newsflash!

[pkhuong]

"Excuse me? Why would the CISC frontend allow better ultilization of cache resources?"

Because the encoding is more compact. For example, compare adding or loading a 32 bit immediate in x86 to the same in any 32bit "RISC". It's widely known and accepted that CISCy encoding reduces the pressure on the I-cache (yes, not for the L1 I-cache on the P4, since the instructions are stored in a decoded form). There has been research into compressing the instruction stream (huffmann, gzip, ...), but I don't know if it's yielded anything in recent years.

"But assumed that a possible benefit in the CISC frontend exists (apart from more compact code on average, that is) -- what difference does it make in terms of bus accesses when the CISC commands get recoded into CISC instructions anyway? Any optimization which was done by (a) the compiler on the CISC frontend and (b) by the internal OoO scheduler on the uOps kan be done as well on native RISC ops."

That was not the grandparent's point. His point was that while, as we seem to agree, CISC offers a more compact encoding, it doesn't suffer from the encoding's complexity much, since they are decoded back in a RISCy form. In other words, the gp was not saying that decoding gives an edge to CISC, but that it allows one to use a CISC encoding while still enjoying RISC's advantages later in the pipeline. As chips are getting more and more complex, adding more logic to reduce bus pressure (or perform runtime optimisations... *cough* EPIC *cough* ;) seems less and less costly.

Re:Newsflash!

[Haeleth]

So, where are the important benchmarks that show things actually running 2-3 times faster? RTFA: they performed a wide variety of tests here, and the largest performance increase they measured was 1.84x, which is not "2-3x" by any means. And that was for system startup, hardly an "important" benchmark, given that most people I know with Macs use the suspend feature instead of switching the thing right off every night. And the average speed increase on "important" benchmarks, which I take to mean "things people actually wish were faster", was 1.2-1.5x. That's a good figure. If Steve Jobs had said "it's 50% faster", people would still have been impressed. But that's not what he said.

Look, if you go to the Apple Store right now, what you'll see is a banner that says "The 2x faster iMac". Not "The iMac that's 2x faster on artificial benchmarks, but actually only 1.2-1.5x faster in real life because most tasks are IO-bound". Apple are selling this thing as 2x faster, period - and it isn't. Call it lying, or call it marketing, as you wish, but it still doesn't reflect well on Apple.

Apples and Oranges

[Anonymous Coward]

Steve Jobs was reporting improvments in CPU benchmarks, but the article refers to application benchmarks.

The CPU is going to be doing different things from those benchmarks in those applications- and may not even be the bottleneck in any given "real world" task.

Now whether Steve should have demonstrated "real world" improvements is up for debate, but all he presented were CPU benchmarks. He made no claim about application performance.

Trolling Mac Fanatics

[SPYvSPY]

Yes, Mac users have a well-deserved reputation for being fanatical (and sometimes even for good reason). But then along comes a story like this one that smears Apple for no particularly good reason and without much of an argument, and you have to ask yourself WTF.

Re:Should have gone with CELL

[Llywelyn]

You do realize there is a *reason* no one has tried such, right?

It's an iMac.

[nsayer]

Let's reserve judgement for "Mac Pro" (that is, the pro level desktop machine) when it comes out. There will be no excuses at all if that machine does not kick serious ass.

"Twice as fast" vs. "Two equally fast cores"

[tongodeon]

The Core Duo is about twice as fast because, as Steve said, each core is about as fast as a G5 and there are two of them.

This means that for most tasks which are single-threaded (searching for text in BBEdit) there's going to be a modest or zero speed increase. For those rare tasks that are written to be multithreaded it'll be ~1.8x as fast (thread overhead, bus contention, etc.)

I'm not surprised either by Steve's stated SPEC benchmarks or real world app benchmarks. That's how concurrency works in the real world whether it's on a dual-core Mac running OSX or a dual-core Athlon running Linux.

This is boring

[Tibor the Hun]

Yes, Steve said that processors are faster in 2 specific benchmark tests.
Yes, the marketing on the website is misleading. (2x, 4x)
It's bad enough that Apple and clueless media are taking things out of their context, we don't need /. to do that.

Everyone on slashdot, I presume, knows at least the basics of how to benchmark a CPU, system, process whatever...
We don't process media feeds on IT specs as facts.

Re:Errr...

[Golias]

Um, it's pretty safe to say that even in pure cpu performance the intel processor is NOT 2-3x faster then the G5's overall.

Um, I think it's pretty safe to say that a dual-core CPU, from Intel or anybody else, is likely to be about 2x faster than the single G5 which the old iMac had.

I think it's also pretty safe to say that a dual-core Intel chip in the new MacBook Pro is going to scream past the single-core G4 (at a vastly slower clock speed) which the old PowerBooks were saddled with.

Anybody who says any different is relying more on religion than math.

PC technology, Mac prices

[Jivha]

Performance apart, it seems that good ol' Apple is charging $1300 for a machine that costs around $900(according to market research firm iSuppli) to them. A markup of around 45% in a ultra-competitive market like PC hardware!

Build cheap, claim big, advertise huge...no wonder the stock market can't get enough of Steve Jobs. I'd envy a man who has the ability to charge above market prices for a near commodity product(a PC) and in the process command a cultish following among the buyers.

Re:PC technology, Mac prices

[BearRanger]

Let's see...the hardware costs about $900. It comes with MacOSX and iLife '06. Apple sells that software for about $200 retail. Plus you get features that aren't available on most PC's, like the built in iSight camera--and the software to run it is an integral part of the OS.

I think the *value* of the Mac package exceeds the budget basement PC you're trying to compare it to. Price out the software for the PC to match the Mac and it won't even be close.

SPEC benchmark hypocrisy

[vijayiyer]

Interesting how all the WinTel fans used to use SPEC benchmarks to bash Macs and the PowerPC processor. Now, in some ironic twist of fate, the same people are using the fact that SPECmarks are fairly useless to say that Apple is lying. The bottom line is that the benchmarks are useless except for people doing specialized tasks. The amount of work you can get done in a day has not changed much unless you do serious rendering work, finite element work, or something similarly CPU intensive.

Re:Well, from what I remember from the Keynote

[TheNetAvenger]

Steve Jobs said that he was talking about the processors being faster...and he specifically said not everything is going to be faster like the hard drives and memory etc etc. Just the processors which is why he showed the SPECmarks or whatever this phantom benchmark that, to my knowledge, isn't a free download from anywhere. Or was I the only one that heard him prefacing the results?

Oh well, let the Mac bashing continue, blood is in the water.

Possibly, but then why does their web site specifically word things to make the 'average' consumer think the new computers are going to be twice as fast?

Did he not convey this to his Marketing and Web team well enough that they wouldn't use wording to mislead customers again?

www.apple.com
"iMac has always made it fast and easy to do the most amazing things. Now the fast and easy part literally doubles overnight -- because the newest iMac computers are powered by the Intel Core Duo."

This is going to be another one of the 'first 64bit desktop computer' things... Which was very closely worded, but yet mislead the majority of the Apple consumer market (to the point the UK made Apple pull ads stating such facts)

And ironically, OSX still isn't a 64bit OS before the G5 is already outdated, and now their new 'performance' Mac is running on a Dual Core 32bit processor...

What happened to leading the 64bit revolution crap we had to listen to uninformed Mac Users recite from the Apple Marketing book?

And here I am sitting with a Dual Core 64bit Notebook, that is almost twice the rated SPEC 'benchmark' of the Mac Intel Duo, and the Apple world is going crazy on how superior their stuff is already. (Even Apple's Web Site, not just the fan boys and girls.)

Lame, once again, and ONCE AGAIN - MAC Customers SHOULD BE DEMANDING MORE FROM APPLE, and instead are just drinking whatever kool-aid the Apple Marketing Buzz gives them...

There are a lot of nice things about dual core

[happyemoticon]

For one, UI responsiveness and multitasking. I know that if I've got an application soaking up all of 1 processor, I'm not going to cause it to go belly-up by shoving it in the background and surfing the web while some single-threaded app happily churns away on that thread.

<Mac Snobbery>Oh, and that reminds me of the nicest feature of OS X: That pop-ups can't take the focus away from you. I hate hitting spacebar, thinking I'm typing into Notepad, and actually I've agreed to a window that flashed up on my screen for about a half a second and I'm wondering if I just bought viagra.</Mac Snobbery>

So perhaps it's a bit of exaggeration but in the end it isn't hurting anyone.

Right on both counts, and I think these are the reasons:

People who actually will buy a top-of-the-line system because a few extra FLOPS saves them hours and hours of time running photoshop filters are going to see the improvements because by and large, the applications that they use are designed to leverage multiple processors. If they're not, they need to bitch at their vendors, because that's ostensibly why Photoshop costs x-hundred dollars.

People like me, who just want to run World of Warcraft in the foreground and have safari open to look things up on Thottbot as necessary and surf the web during transit, are going to notice the UI responsiveness. Nothing's more annoying than when I can't click on Start for 10 seconds because I'm ripping a CD, or the Java VM is starting up for the first time at the behest of a web application running in the background.

Single-threaded performance is slightly overrated. No task I do, except compilation, gaming, and XSLT transformations, is going to benefit heavily from being twice as fast, even on a single thread. If you stuck a gigabyte of ram into my circa-2001 1GHz P3, set it up next to my office 3.2GHz P4 with HT disabled, and had me take the Pepsi Challenge, I would be hard-pressed to tell the difference in most of the applications I use without getting a stop watch or running Doom 3.

MacWorld article one of the worst I've read...

[w4rl5ck]

... for some time.

Really, be serious. They take a dual core - which is much like 2 seperate CPUs - and throw a bunch of non-optimized, single-threaded applications at it.

*NO WONDER* that the CPU does not perform 2-3 times as fast as the PowerPC; one of the two cores can't on his own. Steve never told us that applications will be 2-3 times faster. He just showed some flops. If people still can't understand a benchmark *phew*

In fact, the 10-20% increase in spead is exactly the gain that one would expect who knows that MacOS X usually takes 10-20% system load when doing any transfer task (like memory-to-disk and stuff); so it seems to me that this is what happened with those programs.

Also, the article does not give any suspicions why the architecture performes so bad, no background information about the hardware at all (like, jikes, completly different motherboard architecture, different bus system).

In short: from the technical aspect, bad article.

PLEASE, guys, next time, throw in some common sense and benchmark at least one real multiprocessor optimized program, i.e. Cinema4D rendering.

Re:Is this really a surprise?

[Just Some Guy]

OS X is basically unix under the covers.

...with the exception that I/O Kit and the HFS+ filesystem seem to think a hard drive is a floppy and do their best to set its performance to that level.

I fully understand that my wife's iMac isn't an Xserve, but holy cow, the drive is slow. I'm not exaggerating when I say that the estimate stage of an Amanda backup - that is, basically running "tar --file /dev/null" - takes over an hour to complete on 20GB of content.

For a (not very) quick comparison, here's how long that process takes to run on my home directory on my FreeBSD desktop:

$ find . | wc
52788 60297 3122487
$ du | tail -n1
4270686 .
$ time gtar 2>/dev/null cf /dev/null .
gtar cf /dev/null . 2> /dev/null 0.42s user 1.04s system 89% cpu 1.635 total

On the Mac, though, we see:

$ find . | wc
34346 35311 1997441
$ du | tail -n1
1026640 .
$ time gnutar 2>/dev/null cf /dev/null .
gnutar cf /dev/null . 2> /dev/null 2.27s user 9.41s system 41% cpu 27.874 total

Even though my home directory in the Mac has 35% fewer files and directories to glance at, the tar run takes 17 times longer.

Now, I don't want to be that "a file copy takes 20 minutes!" guy, but this thing really is incredibly slow at certain operations. Just because parts of OS X have a Unix heritage doesn't mean that the whole package has Unix-like performance.

Buy a Mac because you like the OS and applications. We did. If you buy one because you think it's going to dominate all available benchmarks, though, then you're going to be sadly disappointed.

Re:Compiler?

[Krach42]

Only intel zealots would think that an intel chip would be 3 times faster anyway. POWER isn't that bad or Microsoft wouldn't have put them in xbox 360s. Another factor is that the software "optimized" for x86 hasn't been out long. Sure apple's been keeping the old nextstep port alive all these years (it ran on intel and 68k), but making it run and tuning it for the latest pentium chip are two different things.

It's not. The iMac Intel just has a dual core processor. The actual increase in speed from a G5 to a Core Duo is only about 10~25%, the rest just comes from getting two of them.

So, SURPRISE, comsumer level single-threaded apps only get a 10~25% increase, it's AMAZING.