Gigahertz Mac Finally SPEC'd

FrkyD writes "C't magazine puplished a story with the results of a test they designed using a Mac OS X-adapted benchmark suite by the Standard Performance Evaluation Corporation (SPEC) entitled CPU2000. SPEC allows comparisons to be made within a certain framework with the Intel competition. They compared the G4/1 GHz running Mac OS X with a PIII/1 GHz (Coppermine) running Windows and Linux."

Comments

It all comes down to application

[Strog]

Until a more optimized compiler comes along then it looks like for general purpose applications clock speed will mostly be the indicator of performance.

Having said that, there will always be applications that are optimized enough to kick some butt on a G4 like Photoshop, etc. If you are a programmer then it is nice to not be limited on registers on a RISC cpu. Choose the right tool for the right job. If it comes down to a push then use your favorite. :P

Linux vs. Windows

[crow]

I found this from the article to be interesting:

With a SPECint_base value of 306 Apple's 1 GHz machine under Mac OS X ran almost head to head with the equally clocked Pentium III, combined with Linux and GCC, with a SPECint_base value of 309. Under Windows, the bad quality of Microsoft's run-of-the-mill compiler, which pushed the system down to a SPECint_base value of 236

That means Linux is over 30% faster than Windows!

Too bad they didn't give similar floating point numbers (or at least I didn't find them in the article), especially seeing as how the Mac is faring so poorly against the Linux PIII in that area.

Re:Linux vs. Windows

[foobar104]

That means Linux is over 30% faster than Windows!

Of course it doesn't. It means that GCC is somewhat better at compiling the SPECint_base benchmark than Visual Studio is.

I won't pretend to be educated about the inner workings of SPECint, but one would suppose that, because it's purported to be a hardware benchmark rather than an OS benchmark, it is completely independent of the standard C library, or any other OS-level service. One would expect the compiled benchmark to just run pure code inside the CPU, without any system calls or any of that stuff.

So the same benchmark compiled with the same compiler but run under two different OSs should return exactly the same result, within a certain statistical margin.

Somebody with more time on their hands could either test this hypothesis, or confirm that it's already been done by somebody else.

Re:Linux vs. Windows

[leviramsey]

Of course, since everything on a standard Linux machine (kernel, libraries, X, etc.) is compiled with GCC, while I imagine that everything that Windows runs is compiled with Visual Studio's compiler, if GCC consistently turns out code that's faster than Visual Studio, we could extrapolate that Linux is significantly faster than Windows.

Re:Linux vs. Windows

[RevAaron]

we could extrapolate that Linux is significantly faster than Windows.

No, you couldn't. Writing code in C is generally faster than writing it in Java. However, surely you've seen a C program do something slower than a Java program. For example, starting up Nautilus compared to running a hello world program in Java. While this example is taken to an extreme, it still holds true- there is a big difference in the way hello world in Java and Nautilus does things, the same with Linux and Windows, and the speed of the C compiler says absolutely nothing about the amount of abstraction, the amount of bloat, the amount of optimization, the sheer amount of code or the quality of code written as a part of neither Linux nor Windows. Among plenty of other factors.

Re:Linux vs. Windows

[foobar104]

[I]f GCC consistently turns out code that's faster than Visual Studio, we could extrapolate that Linux is significantly faster than Windows.

Maybe you could, if GCC could be thus characterized. But there's no evidence in this article that points to that conclusion. Rather, this article says that GCC did a better job of compiling the SPEC benchmarks. As everybody knows-- or should know-- benchmarks are to real applications as fish are to bicycles.

Re:Linux vs. Windows

[Graymalkin]

Actually it means GCC is 30% faster than Visual Studio's compiler which is notoriously shitty. You're also basing your aguement on too few details. You don't know which compiler flags were used so you can't compare -O3 optimized code to VS optimized code. The VS compiler is not the world's greatest compiler and I think they should have gone with the Borland C++ compiler for Windows. Your compiler makes a big difference in the speed at which code is going to run.

Snide comments on "supercomputer" show bias

[jdb8167]

I know people are going to claim that the SPEC marks aren't susceptible to bias but the SPEC suite only test traditional architectures. As far as I know, they don't test for SIMD vector processing like the altivec.

No one ever claimed that the FP alone on the G4 was at supercomputer status, just that the G4 in conjunction with Altivec could crunch at FLOPs at "supercomputer" speeds.

Keep in mind that OS X is hardly optimized for this kind of test. OS X has just recently reached the point where it is useful as a general purpose platform. But Apple is making a big push in the scientific computing area so I expect that you will find vast improvements in the SPEC FP suite in the future.

Re:Snide comments on "supercomputer" show bias

[Matthias Wiesmann]

As far as I understand the problem, altivec is very performant but only handles single precision floats, not doubles.

While single precision floats are largely enough for multi-media processing (filters, compression, etc...), in general, number crunching is done in double precision and the floating point tests of SEPC reflect this. You don't always need double for scientific calculations, but this is altoghter another discussion.

Maybe one day we'll see a multimedia component of SPEC or Altivec will support double precision numbers (the author even mentions this at the end of the article) but until then Altivec is out and this has nothing to do with a bias of the author.

As for OS X being optimised for this kind of stuff, we are talking applications that nearly never call the OS for anything, so the impact of OS X is probably nil. The truth is, floating point calculation is not really important for most users and both Intel and PowerPC processors are optimised for integer calculations. There was a good article about this on Ars Technica.

One reason I could see to explain the large difference lies in the compiler: there has been much more work on gcc to optimise for the Intel instruction set than for the PPC instruction set. Like most RISC processors, the performance of a PPC processor is hugely influenced by the compiler.

Of notu to GNU and Linux Users

[gus goose]

All tests indicated that the GCC Compiler produced better results than MSoft's C. It is not clearly indicated whether the GCC Results were from Linux bachines, but I presume so.

gus

Processor only test

[gouldtj]

I guess, I am supprised by the results. In just USING the various machines I would say that there is about a 1:2 ratio between the MHz on a PPC to an x86 processor. But then I started to think about them. In reality the processors aren't what make the difference. It is everything else. Macs don't tend to use as much 'soft' stuff, and they make better use of the GPU, so the processor is left to do more 'processing'. I guess they're will never be an completely perfect 'user' test, I'll just have to go with feel. But in my experience those SPEC numbers don't tell the whole story.

It might be interesting to see a comparison with Linux running on both machines... Anyone have one of these?

FYI: Good articles on P4 v G4e architecture....

[jeblucas]

For those interested, arstechnica had some great articles a while ago on the processor families and the different ways they handle instructions.

Part I.

Part II.

my own experience

[jchristopher]

My own experience tells me that a 500mhz Mac (iBook, 640 MB RAM) runs OSX and common apps (browser, mail, newsreader, IM) at roughly the speed of a Pentium II 300 with Windows 2000. That's terrible.

Even a lowend PC these days ($700 or so) will run Windows FAST, whereas Apple's lowend end runs OS X slowly.

Most of the Mac's "speed problems" lie in the OS, not the hardware. Linux on the iBook described above flies.

Re:my own experience

[HypodermicEyes]

Sure, but that's not really relevant to the article. SPEC is a measure of the capability of a CPU+Memory Subsystem+compiler in performing Real Work, where Real Work is number/symbol crunching. You know - linear algebra, compiling, those kinds of things. SPEC couldn't care less about measuring how responsive a GUI is, and rightly so. Assuming the OS has decent process+priority management (as it should for Mach+BSD), the GUI bloat should not be relevant.

I was actually quite surprised by the poor performance of the G4. Although if the other posters are correct in stating that SPEC doesn't test very well for single-precision floats nor SIMD, it's probably not a very good test of the capabilities of the CPU. Of course, Apple has crippled their systems with PC133. That doesn't help either.

Re:my own experience

[HypodermicEyes]

If I may add something... there's a long thread on exactly this subject in comp.arch right now. Look for the thread called "SPEC2k results for G4". There's some very interesting comments from people that mostly seem know what they're talking about.

Re:my own experience

[Llywelyn]

Your experience, in essence, is not the norm for mac users.

My own experience is that a 300 MHz G3 will blow a 500 MHz Pentium out of the water, thats running MacOS 9.

System configurations matter, memory matters, &c.

Why a P3?

[george399]

Call me crazy, but why is there a benchmark between a PIII and a G4.
Wouldn't a P4 be a better test?

Re:Why a P3?

[doooras]

i don't think a 1GHz Pentium 4 exists.

In case you don't read the article...

[Johnny Mnemonic]

Buried in this article is this note: and switched off the second supporting processor of the dual machines. Which means that the Dual 1Gs were only run as single Gig machines--and would therefore be much faster in the real world, so cost comparisons should be made accordingly.