Slashdot Mirror
x86 vs PPC Linux benchmarks
Jay Carlson writes "We've all heard about how Apple's hardware is really fast compared to PCs. ("Supercomputer!" "Twice as fast as a Pentium!" "Most powerful laptop on the planet!") So, if you *aren't* going to use Photoshop and Final Cut Pro, how fast is it? I care more about integer apps like compilers, so I did some careful benchmarking of a few x86 and PPC Linux boxes. Submissions welcome."
From http://fampm201.tu-graz.ac.at/karl/timings40.html.
Numbers are relative to a G3-300MHz (higher are faster).
There are more numbers on the homepage.
Athlon 1.2 GHz, 512MB, Windows 2000 [73]: 4.78993
Athlon 1.2 GHz, 512MB, Linux [72]: 4.4734
Gateway Select 1000, AMD Athlon 1000 MHz (1GHz), 512KB L2, 192 MB, Linux [65]: 3.77305
Kryotech 1GHz AMD Athlon, 512k cache, 512MB, Linux [66]: 3.69674
Gateway Select 1000, AMD Athlon 1000 MHz (1GHz), 512KB L2, 192 MB, Linux [60]: 3.57748
Dell Dimension XPS B1000r, 512MB Ram, Win98 SE [64]: 3.38084
Dell 4100, 933MHz, 128MB, Linux [63]: 3.19988
COMPAQ AlphaStation XP1000, 2 GB RAM, 4MB L2, Digital Unix 4.0F [50]: 3.16987
AMD Athlon, 800 MHz, 512 KB L2, 256 MB, Linux [61]: 3.00154
Dual Xenon 866, 512 MB, Windows 2K [71]: 2.9618
PenguinComputing, dual 800Mhz PIII, 128Mb, Linux [67]: 2.76745
Athlon 700, asus k7m, 512 mb, win98 [43]: 2.70199
Dell 800 Mhz, Pentium III, 512 MB RDRAM, Win 98 SE [57]: 2.69821
Athlon 700 MHz, 128 MB, Linux [51]: 2.67027
Athlon 650 MHz, 256 MB, 100 MHz, Win NT 4.0 [42]: 2.60662
Compaq-Digital Alpha 8200, 625MHz, 1GB RAM, DEC-UNIX [11]: 2.48495
SONY VAIO F409 notebook, PIII 650 MHz, 128MB RAM, Red Hat LINUX 6.1 [55]: 2.21137
Athlon 650 Mhz, 32 Mb, Windows 98 [32]: 2.14558
Athlon 550MHz, 128MB, Red Hat Linux 6.1 [45]: 2.13797
Dell XPS T600r, P3 Copper wl 256kb, 128MB, Linux-2.0.36 [39]: 2.09939
Dell Precison 410, 2 PIII 550MHz, 256Mb RAM, Windows NT 4 [28]: 1.8561
Dell Precision 210, 550 MHz, 128 Mb, NT 4.0 [34]: 1.83606
Dell XPS T550, PIII-550, 128MB, RedHat-5.2, Linux [24]: 1.777
Gateway GP7-500, 500 MHz, 192 MB, WinNT 4 [6]: 1.70318
PowerMac 8500, 500 MHz MACh Carrier G3, 1 MB L2, 256 MB, MacOS 8.6 [35]: 1.68249
Instead of pitting his machines against one another, he should've assembled them into a BEOWULF CLUSTER so they could WORK TOGETHER.
The sole reason Photoshop is faster on Macs than PCs (recently), and the reason Steve always brings it out to show off, is altivec. We need benchmarks of vectorized vs non-vectorized versions of the same task, running on the same G4 or against an x86 box. Unfortunately I can count the programs that use altivec in the real world on one hand; hopefully this will change with OS X arriving (altivec in system libraries).
Yeah, well, the "Supercomputing" definition is based on whether you can process enough data to support the creation of nuclear weaponry. But it's not always relevant.
I really was not too impressed with the benchmarks presented here-- and not because they were anti-Apple. The fastest machine had poor disk performance-- but this is really very relevent to compilation and development. Because the benchmark was very specific-- cross compiling for MIPs machines-- (wtf?), I have no idea how far these benchmarks can be stretched.
I think he should have tried to optimize the binaries-- after all, he was compiling from source. And how do his compilers compare with the latest gcc releases (not snapshots, though)?
BTW, although I own both a Mac and a PC, my mac is old, and I mostly use my (faster) PC. I am not an advocate of either platform...
But it seems to me that GCC builds much slower code on powermacs than whatever it is Apple builds their binaries with. (If Apple's using a modified GCC, which wouldn't much surprise me, I sure wish they'd throw us their patches.)
Now, if I'm right, and GCC produces relatively unoptimized binaries, shouldn't you compile GCC itself (and maybe even the rest of the system) with another compiler before pitting machines against each other in a compiling race? It seems to me that you're using a slow, badly-compiled GCC binary, otherwise.
Granted, this seems an excellent real-world test, since nobody does that. But I can't help but feel that we're (by "we", I mean the Community[TM] )currently incapable of exploiting the PowerPC, and it seems unfair to blame the chip.
Maybe I'm wrong. I would love to see some discussion from the GCC team. I just thought since nobody else seemed to have brought this up...
Welcome to the wonderful world of vendor-supplied memory. All the big Wintel vendors rape you equally when you buy their RAM, except for the occasional promotion for "FREE 128MB RAM with system purchase, for a limited time!!!" that you see in the mags and in all the mailorder catalogs.
It's not just Apple.
However, to give you a bit of good news, all Macs sold in the last several years have user-installable RAM, and with the exception of the original (Rev. A through D) iMac, it's very easy to do - easier than in many PC's.
The new G4's use standard PC133 SDRAM, all other model desktops use PC100 (the Cube and iMac), and the TiBook uses PC100 SO-DIMMs, the iBook (old and new models) uses PC66 SO-DIMMs, though PC100 works fine, too.
PC133 SO-DIMMS seem a tad flaky so far - I just got a Gateway 9500 laptop at work and still haven't gotten a 3rd party 256MB SO-DIMM that will work with it (we've tried Samsung, Micron, and Hitachi, with Infineon on the way). Apple will probably start using them with a TiBook revision at some point, after the interoperability issues vanish.
- -Josh Turiel
-- Josh Turiel
"2. Do not eat iPod Shuffle."
According to an article in the linux journal
the Altivec unit handles only single precision
floats
-> only useful for special tasks or for calculations were you can take care for the
reduced precision,
-> not for general scientific numerical calculations
Thus any scores over .61 and .72 respectively, indicate that the PowerPC is doing more per clock cycle than then PIII. If Motorola can ever get their act together (and that is not a certain), normal code on the PowerPC will run every bit as fast and faster than the x86 processor
This illustrates a common fallacy - if chip A does more work per cycle than chip B, then chip A is "better" and as soon as those "idiots" who make it get the clockspeeds up there it will perform better. This neglects the fact that B may do less work per cycle precisely because it is designed for extreme clock speeds, and in fact there are plenty of instance where the "speed demon" cpus (high clock speeds, simple instructions) outdo the brainiac chips (lower clock, beefier instructions). The reason Pentium 4 trounces any PowerPC is that it is designed to scale to 2GHz. Of course it does less work per clock, but overall it does more per second, and that is the more important metric.
No, the Apple Store price I quoted in that table is for a single-processor G4/533. Go price it yourself.
(I'd give a URL but the Apple Store is too sessional.)
All this means is that compiling on a RISC architecture is bound to be a great deal slower.
I'm aware of that, and I talk about it on that page. See the "Choice of workloads" section.
The install-egcs test does measure native compilation performance. This is relevant for people who use the box for development.
The install-glibc and cross-gcc tests are both compiling to a single RISC architecture, little-endian MIPS. The amount of effort required to optimize for PPC or x86 doesn't factor into this.
If you don't care about development compile times, just look at the cross-only numbers.
Peak work done per clock or price per clock are meaningless. You should simply look at what you can get for your money at a particulat price and see which is faster. If a $1000 PC beats a $1000 Mac running your application, then who cares what clock speed they're running at - it's irrelevant.
It seems that the PC comes out on top when compared to apple. However did anyone really expect anything else? It always suprises me when people bring up apple's quotes (super computer, faster than a p3 etc.) What do you expect them to say? "Well our G4 is slower than the other new chips out there but costs more! Please place orders here!" I mean come on. The one thing that apple has going for them is their incredibly loyal userbase. My accounting professor uses Macs. I have gone over the benefits of PC's many many times with him. And on a lot of points he agrees that the PC is better.
He just bought a titanium powerbook.
'nuff said.
Although that cinema display is super slick (as it should be for its price tag...).
S.t.e.v.e.
Though it might more accurately be described as a GCC/Linux PPC vs. x86 benchmark, it's still interesting to those of us who are committed to GCC and Linux. Those two items, and their spawn, suck up almost all the (used) cycles on all of my machines.
Geeky modern art T-shirts
Hi. I'm a GCC maintainer. I don't work on this part of the compiler, but I can speak to this point:
Nearly all of the optimizations in GCC are not machine-specific. Those kinds of optimizations, ones which are specific to the processor, are called peephole optimizations, and while every little bit helps, they don't make that much of a difference. The big ones are done at an intermediate level, before the compiler "knows" what processor it's using and starts to chunk out the opcodes.
More specifically, unlike the Linux kernel, glibc, and other major projects, GCC is not designed for and targeted primarily at Intel chips. The x86 is just one more back-end like any other; sometimes it falls behind and sometimes it pulls ahead, development-wise.
Some have, some have but won't be in the upcoming 3.0 release in a few weeks, and some are yet to come.
The biggest problem is that many of the really cool optimizations -- the ones that make a big difference and aren't CPU-specific -- have been patented by IBM and other major players.
You cannot apply a technological solution to a sociological problem. (Edwards' Law)
I can give you over a thousand examples, with the help of our f[r]iendly patent office (my tax dollars at work). Just go to http://www.uspto.gov/, look under the green Patent Grants area, follow the Advanced Search link, and search on "compiler and optimization". Doing this today, I got 1,261 patents, but some of them don't apply here.
Er, that is to say, I got 1,261 search results each representing a patent. I don't have 1,261 patents myself. :-)
You cannot apply a technological solution to a sociological problem. (Edwards' Law)
Lies,
Damn Lies, and
Benchmarks
I read through these comments and get the impression that slowly a conclusion is being reached: PPC-type hardware is good for some things, x86 hardware is good for others. Nothing really new there, is there? For running Linux, it seems from this little (and far from in-depth) benchmarking session that PCs are a bit better, especially given costs. You can probably get a 1.2GHz Athlon box for the cost of a 533MHz G4, and it'll be better for Linux, so if you run Linux, why not?
MacOS X, stuff like Maya, Final Cut Pro, etc. etc. quite obviously runs better on PPCs, barring some strange circumstances. I imagine that with enough "brute force" (RAM, dual processors, etc.) one could get a PC to run this stuff faster than a Mac.. but what's the point? You might as well just keep it simple and buy a Mac that'll run it pretty well outta the box.
I agree though, that cost is an important consideration. With 760MP around the corner, if it ever does surface in quantities making it available, dual Athlons might give dual G4s a bit of a whippin', especially considering AMDs prices as of late. In general I find you can buy a PC with a much faster proc, more RAM, etc. for the same cost as a Mac from the Apple store.
Still, even a 1.2GHz Athlon would probably choke on OS X, and the G4 will at most hiccup...
"Caffeine is not an option. Caffeine is a way of life."
I wind up doing my own internal PPC vs X86 benchmarks almost every year.
I'll set up whatever current game I am working on to run with the graphics stubbed out so it is strictly a CPU load. We just did this recently while putting the DOOM demo together for MacWorld Tokyo.
I'll port long-run time off line utilities.
I'll sometimes write some synthetic benchmarks.
Now understand that I LIKE Apple hardware from a systems standpoint (every time I have to open up a stupid PC case, I think about the Apple G3/G4 cases) , and I generally support Apple, but every test I have ever done has had x86 hardware outperforming PPC hardware.
Not necessarily by huge margins, but pretty conclusively.
Yes, I have used the Mr. C compiler and tried all the optimization options.
Altivec is nice and easy to program for, but in most cases it is going to be held up because the memory subsystems on PPC systems aren't as good as on the PC.
Some operations in Premier or Photoshop are definitely a lot faster on macs, and I would be very curious to see the respective implementations on PPC and X86. They damn sure won't just be the same C code compiled on both platforms, and it may just be a case of lots of hand optimized code competing against poorer implementations. I would like to see a Michael Abrash or Terje Mathison take the x86 SSE implementation head to head with the AltiVec implementation. That would make a great magazine article.
I'll be right there trumpeting it when I get a Mac that runs my tests faster than any x86 hardware, but it hasn't happened yet. This is about measurements, not tribal identity, but some people always wind up being deeply offended by it...
John Carmack
I'm sure this person THINKS he's testing the same thing on each machine. Unfortunately, he's not.
Firstly, the action of compiling on different architectures is very different, even without considering optimisation strategies. To compile code into the CISC code of the x86 architecture is very different from that of a RISC chip such as the PowerPC. For a start, instruction ordering etc. for a RISC chip, even for not really optimised code can take far more processing time. Then, if you add optimisation, which in a RISC architecture is a FAR more complex task.
All this means is that compiling on a RISC architecture is bound to be a great deal slower.
Basically, this "benchmark" is measuring not only the intrinsic speed differences of the architectures and chips but also degree of optimisation the native compilers used can cope with and the extra processing power needed to generate the code during the comile stages.
Basically, using compilation as a benchmark is not at all useful, other than to test the difference in speed of two similar peices of equipment using identical software (ie. compilers & OS) or the difference between two versions of the same OS or two versions of the same compiler.. Basically, you can only change one variable to deliver a meaningful benchmark if using the method chosen in this "study."
The only way to get a half-way meaningful benchmark for the two systems used here would be to write a program which did lots on disk I/O and integer manipulation, worrying about whether it's being biased for or against certain types of architecture or use (eg. loops sitting in cache etc.). This would give you an idea of the real-world speed differences between the two systems. However, you won't be just measuring the intrinsic speed of the machine but also the different ways the kernels have to do things on the two architectures, the degree of optimisation the compilers building the kernel and the program could generate and the speed of the hard disk built into the machine.
As you can see, it's a tricky thing comparing two types of machine.
Agrajag: "Oh no, not again!"
----
Althea verified to run quickly on Mac and PC hardware.
Its opptimized for the x86. Sure OSX ships with it, but apple did do heavy optimization for it. Those chages may not have been rolled back in to the tree yet, so I would not trust these bench marks. Your running an os that is optimized for intel, and a compiler that is optimized for intel, and comparing them to a ported copy.
Most people know that gcc is slower on suns then the sun compiler. That is all about optimization. So why wouldn't it be the same for ppc?
Try doing this benchmark on darwin, and im sure the macs will do better against the intel boxes running darwin, then running linux. Im not saying that it will be faster, im just saying your comparing apples and oranges.
The comparable data are:
Everybody knows Linux is so fast it can execute an endless loop in 5 seconds flat.
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
I don't thnk that you will ever get an agument from anyone over how very, very, very badly Motorola has messed up with being unable to deliver faster versions of the PowerPC. However, you should look at a couple of issues with these benchmarks: 1) 450/733 = .61 and 533/733 = .72. Thus any scores over .61 and .72 respectively, indicate that the PowerPC is doing more per clock cycle than then PIII. If Motorola can ever get their act together (and that is not a certain), normal code on the PowerPC will run every bit as fast and faster than the x86 processor. Combined with the fact that the PowerPC has a nice quiet and fairly energy efficient air-cooled chip you might have some nice machines. Unfortunately, all benchmarks can have some rather un-intentional bias. My 1.2 Athlon would do 105 Scimarks in Windoze 98, 113 Sciemanrks in Wine under Redhat, and 119 under Windoze 2000 for Tim Wilkin's Science Mark benchmark.
Same machine, same memory, same disks, etc. the only difference was the os. Even given the same os, the tweaks that it goes through are also a function of the author's machine. Please pass the salt, I need a grain.
Sex is heriditary, if your parents didn't have it chances are good you won't either.
http://www.jdkoftinoff.com/eqtest.tar.gz
(gpl'd with source)
450 mhz g4:
1.7 gigaflops with altivec
410 megaflops without altivec
500 mhz pentium:
220 megaflops
--jeff
ipv6 is my vpn
Unsuprisingly, the Macatista wrote that meggy-hurtz don't matter, and besides, Mac's are 3 times more powerful than a Pentium 3 anyway! Pop.Sci. wrote back, saying breaking the Ghz IS a milestone thats important to note, and that tests have shown that yes, in some areas, Mac's are 3 times more powerful than a Pentium 3. However, these same tests show that in some areas, x86 platforms are 3 times more powerful than the Mac.
This argument has long bored me. The arcitechural differences between x86 and PPC have been vast until the last year or so. According to an article at Ars Technica, the Intel Pentium 3 chip is somewhat like the PPC, but the AMD Athlon is even more similar to the RISC found in the Mac. Even if that's the case, have you ever seen the price difference between the two platforms? Plus add in your options (and the price thereof) when buying a Mac. You take what Apple will give you. Apple's prices on memory are so laughable as to be a great stand-up routine.
Now don't get me wrong, I have nothing against the hardware, and the only problem I have with the OS as an intermidiate user is the file organisation system. I just think that Apple's managment sucks, and that I get more bang for my buck going out to a local computer store, in which I can support the mom&pop's of America. A one or two year parts, and three years labor warranty is good enough for me.
I've nearly completed a web serving benchmark with multiple PPC configurations running OSX Server, and Intel and AMD hardware running Linux.
The results look nothing like the compiling benchmark, and have convinced me to start a web hosting company using OSX Server on Macintosh hardware.
The benchmark utilities will be downloadable and you can run them on your own favorite hardware. Benchmark requires PHP and mySQL database support, but will run on more than just Apache. I'll also set up a site where you can upload your results - configuration and resulting data.
-t