Slashdot Mirror
MacWorld's iMac Core Duo Benchmarks Debunked?
madgunde writes "Looks like MacWorld magazine was a little premature in reporting that the new Apple iMac Core Duo doesn't live up to Apple's speed claims. The folks over at MacSpeedZone have done some performance testing of their own that debunks MacWorld's results and shows that the new iMac Core Duo DOES live up to the hype. Not only did the new iMac wipe the floor with the old model in their tests, but using MacWorld's own test methodology would allow MacSpeedZone to conclude that the new Intel iMac is almost as fast as a PowerMac Quad G5. " I see only one way to solve this: Give me one. I'll run WoW on it, and decide.
How many stories can we have about the Intel-based iMac's benchmarks?
1 2/2478
All of these "benchmarks" are true, as far as they go.
Apple's original SPEC benchmarks are "true".
Macworld's "real world" application benchmarks are "true".
And now, MacSpeedZone's further tests of various tasks also are "true".
I mean, obviously the new iMac isn't going to be 2 times faster for everything under the sun. In fact, Jobs even spoke to this fact in the keynote when he directly said that the tests were just for the CPU and that everything else, like disk I/O and other subsystems, weren't all twice as fast, but it was to illustrate the performance (and performance per watt) of the new Core Duo, which is indeed impressive by any measure.
I think it's safe to say that the new iMac running native applications is definitely faster - sometimes up to twice as fast, and sometimes even more - than the iMac it's replacing. And Rosetta is so impressive that while non-native applications will run slower, it's damned good until native versions of those applications come out, too.
And speaking of CmdrTaco's request for a WoW test on the new iMac...
http://arstechnica.com/journals/apple.ars/2006/1/
"It's fast, fast as in a superlative and not a comparative sense. One wonders why Steve Jobs didn't blow the crowd away with the saturated colors and excessive frame rates of WoW on an iMac. It loaded fast, and when the first character popped up in town, the frame rate never dropped below 60, and this was pretty much going full tilt in the settings."
The machines themselves have to be faster. If the old chips were on par with similar PC chips the very fact they are dual core increases the speed. The real problem is in applications. Even in the PC world most apps don't take advantage of the dual core architecture. Even Maya only uses multiple processors when rendering. If you have a quad machine, a dual/dual core, it will only use one processor for most functions but will use all four nodes in rendering. If I ran a benchmark that involved modelling it would show no improvement in speed over a single chip machine. If I ran a render test would clock in around 4X faster. Both tests are accurate and simply reflect how the software is designed not how the chips themselves function.
Microsoft says their software is secure...
Oracle says their database is hack-proof...
Symantec says their software protects me from hackers...
Its referring to tests that don't max out the CPU anyway, and therefore presumably have bottlenecks in some other part of the system. Here's a more rediculous example using the same theory:
Test: Compressing and sending a 16MB file over the network
iMac: 83 seconds (cpu usage 23%)
quad: 84 seconds (cpu usage 11%)
Wow! The iMac is faster than the quad! Of course, in reality it was working much harder to accomplish the same task (compressing at a bandwidth-limited speed). The articles point - and it is very poorly written, I will agree - is that this kind of test is crap.
The Macworld test used the same theories in the other direction. After all, if you perform a task that takes the old G5 iMac 20 seconds but uses 99% of its CPU, and takes the new intel iMac 19 seconds but only uses 45% of its total CPU power, I think you'd say that the iMac was more than 5% more powerful, right?
Admittedly if all you ever do is one task at a time, you wouldn't notice the difference. Considering that many people like to do multiple tasks - watching the recent keynote in a background window while doing some other work in a foreground window, for example - this is not an inconsequential point.
That brings up the example from the linked MacSpeedZone article:
Encoding one QuickTime movie:
intel dual core iMac: 97.02 seconds (87% CPU)
g5 quad core powermac: 84.85 seconds (42% CPU)
advantage g5: 14% faster
Encoding two QuickTime movies:
intel dual core iMac: 176.60 seconds (100% CPU)
g5 quad core powermac: 86.25 seconds (87% CPU)
advantage g5: 105% faster
Even that's a little misleading, since the quad still had spare processor bandwidth. This is why a lot of benchmark tests are designed to test each piece separately, spinning them up to 100%. Of course, real world tests are great as well - but only if your usage actually parallels those tests.
You're special forces then? That's great! I just love your olympics!
For most purposes, the key question for most users is not going to be how fast it is really, but how fast it "feels," in practice, say watching a QT movie with maybe a browser loading a couple of windows in the background and a Spotlight search in progress.
The OS X seems to be pretty good at spreading the load of multiple programs and the OS across processors. I remember that the dual 450 MHz Macs seemed dramatically snappier that the 800 MHz iBook, even though in most tests the iBook would come out ahead.
I doubt that Apple's move to Intel had a great deal to do with performance, and I dislike this fact being used as a key selling point for the iMac. If you refer to the "definitive" G5 vs. everyone else benchmarks at http://www.anandtech.com/mac/showdoc.aspx?i=2436 it is apparent that the G5 is largely comparable to offerings from AMD and Intel (admitedly the new Intel Core Duo is not benchmarked) and although the G5 is, in many cases, not the fastest chip, it is similar. The increases of 2-3x in performance between the G5 and MacIntel iMac are a consequence of having a dual core chip (and being a generation ahead of the G5) besides, Apple could have feasibly used the dual-core G5 chips that they've had at their disposal for a while now. Any Mac zealot will argue that their PowerPC Mac is "just" as fast as an intel based system, but performance is NOT the issue. This is why the iMac was updated first, it is a consumer product, supporting Apple's fledgling attempts to enter the living room (consider front row ) - it desperately needs Intel's brand name associated with its hardware.
The significance of this new product is long term and cannot be underestimated.
Apple finanlly has penetrated the consumer electronics market with the iPod, and their brand recognition and image could not be better. Apple has shoehorned its way into the psyche of the common man. It now has to bring its key product, the mac, to the masses. Consumers will be attracted from a design perspective and because it shares the same logo as their iPod, the OS is a little different to windows, but now at least you have the reasurrance of dual booting into windows (I'd like proof of this concept, but I'm sure it will come) and the processor gives the security of a well recognised brand name (consider brand strengh of Intel vs. AMD).
In the future, I doubt that IBM's die shrunk Power chips will share the low power consumption that I expect Intel will bring, and many concepts for great products will never be realised. I'll be interested to see if the new Intel chips can match up to the PowerPC altivec-ised vDSP FFT's , but in a way I don't care. It is an exciting time to be a Mac user, as more people join the fantastic experience that we have had for so long, and new software and hardware comes our way. Either way, they're finally here and it will be interesting to see what the future holds.
Nothing sucks like a Vax, nothing blows like a PowerMac G4
OK, lots of talk about these but here it is from someone with both G5 and Intel iMac 20" machines. For some things the Intel is faster than the G5 by a significant margin (Safari in particular feels quite snappy) but when you have to run PPC apps the G5 is much better. For the moment there are really quite a lot of apps that are not Intel native so the overall impression when using the two machines is that the Intel is no quicker, and some times much much slower. For PPC apps the Intel machine is no better than my 933Mhz iBook G4. Worse, there is significant pain at the moment in doing much that is taken for granted with the G5 iMac. Many programs do not run (we use BlueJ and Eclipse, neither work on the Intel). You still get the spinning beachball of death, and it seems quite often too. All in all, it feels just like any other previous Mac.
One thing that impressed me was the fact that Rosetta is able to run command line apps compiled for PPC. Gives a good idea of just how fast Rosetta is when running raw PPC code without a GUI. The answer is that a 2Ghz Intel chip running PPC code is about the same speed as a 500Mhz PPC. very reasonable compared with something like PearPC but still a significant drain. You get some back with the GUI as much of that code is native so something like MS Office actually feels usable. Our 2.3Ghz G5 Xserves smoke both the G5 iMac and the Intel even when the Intel is running native code at least with our apps.
So, do I recommend the Intel iMac? Probably. Would I recommend against a G5? Nope. Buy whichever you like. With the G5, you know what you are getting and it will still run software for the forseeable future. The Intel machine is pretty hard work at the moment but has the promise of getting better as more universal apps come along. Of course, there is currently no viable fast PC emulator so you can't run Windows or Linux on it. With Qemu or VPC on the G5 you can run Windows quite reasonably but not as quickly as you will be able to in say, six months when MS get off their arses and build VPC for the Intel Mac.
I can see why Apple released the iMac first, makes sense. The G5 iMac was never really a speed demon so the Intel one doesn't suffer too much overall. Same goes for the MacBook Pro which should be able to keep up with the G4 PowerBooks. It will take a while yet before slotting an Intel chip into the pro towers makes sense though.
A Mac is a Mac though, doesn't really matter what is inside chip wise.
"I have the attention span of a strobe lit goldfish, please get to the point quickly!"
Yes, but who really cares about CPU benchmarks?
My 2.6GHz Thinkpad smokes my 1.0GHz G4 iMac in CPU benchmarks. I use them both every day for many hours a day and for many common tasks. However, in spite of the CPU performance difference, the iMac seems faster (i.e. it's more responsive AND my productivity/output is higher).
Now I'm sure that there are some applications (that I never use) where the standard benchmarks do give meaningful results... However, for the applications that I use, I find most benchmarks to be a meaningless waste of time (with the possible exception of the old ByteMark).
There's an old saying about standards, but I think it applies equally well to benchmarks, so I'm going to co-opt it...
The nice thing about benchmarks is that there are so many to choose from...
The point being is that everyone has different expectations and different ways of using computers. Alternatively, the point may be that benchmarks are just another way of lying with statistics...