Apple Hardware VP Defends Benchmarks

Greg Joswiak, vice president of hardware product marketing at Apple, in a phone interview today, defended Apple's performance claims for its upcoming Power Mac G5, after they came under fire in the wake of yesterday's announcement. Read on for the details. Joswiak went over the points in turn, but first said that they set out from the beginning to do a fair and even comparison, which is why they used an independent lab and provided full disclosure of the methods used in the tests, which would be "a silly way to do things" if Apple were intending to be deceptive.

He said Veritest used gcc for both platforms, instead of Intel's compiler, simply because the benchmarks measure two things at the same time: compiler, and hardware. To test the hardware alone, you must normalize the compiler out of the equation -- using the same version and similar settings -- and, if anything, Joswiak said, gcc has been available on the Intel platform for a lot longer and is more optimized for Intel than for PowerPC.

He conceded readily that the Dell numbers would be higher with the Intel compiler, but that the Apple numbers could be higher with a different compiler too.

Joswiak added that in the Intel modifications for the tests, they chose the option that provided higher scores for the Intel machine, not lower. The scores were higher under Linux than under Windows, and in the rate test, the scores were higher with hyperthreading disabled than enabled. He also said they would be happy to do the tests on Windows and with hyperthreading enabled, if people wanted it, as it would only make the G5 look better.

In the G5 modifications, they were made because shipping systems will have those options available. For example, memory read bypass was turned on, for even though it is not on by default in the tested prototypes, it will be on by default for the shipping systems. Software-based prefetching was turned off and a high-performance malloc was used because those options will be available on the shipping systems (Joswiak did not know whether this malloc, which is faster but less memory efficient, will be the default in the shipping systems).

As to not using SSE2, Joswiak said they enabled the correct flags for it, as documented on the gcc web site, so that SSE2 was enabled (the Veritest report lists the options used for each test, which appears to include the appropriate flags).

Comments

Who cares?

[ObviousGuy]

Really?

If you want OSX, you'll need to get the PPC.

If you want Windows, you'll get the x86.

If you want Linux, you can pick up 10 and build yourself a cluster for the price of one of these new machines.

Re:Who cares?

[Pootie Tang]

Personally I think the speed of the G5 and the validity of the benchmarks are both valid questions.

Does either of those questions alone determine whether you should get a G5 based system or not? No, but that doesn't mean the question isn't worth discussing.

I'm curious how fast the G5 is at certain kinds of tasks. Not because it helps me make a purchasing decision, but because I'm a geek and I'm interested in that kind of thing. This being slashdot, I'm sure I'm not the only one. Does superior floating point performance mean "better for photoshop"? Maybe not, but I'm more intersted in FP performance that PS performance.

I thought the original article was worth a read. I thought some of the comments are interesting. I thought this follow up was interesting. People like me are the ones who care. People who just want to know what kind of computer to buy, well yes, they are totally missing the point.

Re:Who cares?

[bursch-X]

>Apple's core competence is in making systems that are easy to set up and easy to administer and easy to use.

And that's not only thanks to the software, but also due to the great integration of software and hardware.

This integration ("it just works") is why people buy Apple. And therefore it's really hardware and software that attract customers (ey, and don't tell me I didn't buy my 17" PowerBook just for the software, I could have gotten an iBook if I only wanted to run OS X!)

Re:Who cares?

[drauh]

For some people, e.g. physicists who do numerical "experiments", the benchmarks are crucial, or, at least, a large factor when considering which machine to buy.

Sure, one could buy 10 Linare boxes and Beowulf them together, but if you're a lone physicist with relatively little funding--Beowulf clusters take lots of time, money, and space to feed and maintain--you might care about being able to run floating-point intensive jobs quickly while being able to use MS Word or PowerPoint or some such.

In fact, I already know one astrophysicist who will be getting a G5 in the fall when her new research grant begins. She also happens to be one of the 3 physicists I managed to convince to switch to Mac and get a PowerBook.

Honesty

[dioxn]

At least everything that they did seemed to be amply documented.
I found that to be refresing especially in light of all the recent benchmark tests that have not been so forthright with all their methods and procedures.

Re:Honesty

[switcha]

If I shit in a bucket and carefully label and document everything does that make it a bucket of gold?

No. It makes it a container containing homo sapien fecal matter, deposited on June 24 at 16:21 after a lunch of onion rings and a Rodeo cheesburger from the Burger King establishment.

And to top it off, you now have to deal with a shit in a bucket.

Re:Honesty

[pi radians]

What about the new 3.2's?

You mean the new chips from Intel that were announced the same day as the G5s?

Shit, some people you can never please.

Re:Honesty

[kherr]

Look at their numbers they are comparing the G5 to the P4 3.0 and Xeon 3.06. What about the new 3.2's?

How dare Apple! But then they only had the G5 2GHz. Maybe they should wait for the 3GHz, then the comparison will be fairer?

The fact that everyone is nitpicking these benchmarks shows how close the performance is. And with such a huge "megahertz" disparity between the Xeon and the G5 shows how much power the G5 has to offer.

Re:Honesty

[vought]

That's the dangers of making a comment like "fastest PC on earth"......it's a claim that one has to be VERY careful about making.

Considering that you couldn't get either system yesterday (G5 or 3.2GHz Xeon (I hate it when slashdotters write 'zeon')), I see the whole thing as moot.

It's marketing, folks, not the bible. Greg backed up the test parameters with his data. I think the world would be a less stressful place if Windows went away, but I'm not stressing over a minute saved over a week of Photoshop work.

We spend more time thinking about what to do next in Photoshop than could possibly be saved by a faster processor/architecture/whatever.

That being said, I'll continue to buy Macs because the extra up front cost is well worth the knowledge that one company (and a rather well-run one these days) is responsible and capable enough to develop and market botht he hardware and the OS. They do a rather good job of it for a small premium.

Put another way: If I lose an hour a month because of a hardware vendor who refers me to an OS vendor to resolve a problem, I've lost an hour. I can't get that time back. If my Mac emits smoke and kernel panics at the same time, I know I can get resolution to both problems by calling Apple.

Re:Honesty

[Stu Charlton]

Apple's response while prompt, was not frank. If it were frankly honest, Apple would have to admit that their brand new G5 would be slower than Dell's computer from last quarter under certain conditions.

This is unfortunately the problem with *all* benchmarks. Almost any two competitive systems can be measured to be faster than one another under certain conditions.

The point of Apple's benchmarks were to show that "in the general case", the G5 would be faster than an Intel workstation. The general case assumes either a) using photoshop, or b) using applications compiled with GCC.

Are either of these a large stretch? Well (b) might be, and a comparison with MSVC++ and/or ICC would be nice, but then Apple would probably just counter that with IBM's optimised G5 compiler.

If you want an honest benchmark, have Apple's system tweaked by Apple, Dell's system tweaked by Dell, and who gives a damn which complier is used? I'm a user, not a CPU. I don't care about the theoretical capacity of each computer, especially if the theory is tested using inefficient compliers.

Well then, frankly, you shouldn't be paying attention to SPEC benchmarks, you should be looking at the informal timed application benchmarks. These are more "user-centric" and "whole-system" measurements.

The point many on Slashdot are making is that SPEC benchmarks *are fundamentally* theoretical CPU capacity measurements, not intended for users.

Let Apple and Dell pick their own compliers, or even write their own compliers just for this test.

The amount of man hours that goes into an optimising compiler is arguably more than goes into an operating system kernel. It's a rough business, and usually why people tweak existing compilers to perform better on benchmarks such as SPEC.

Re:Honesty

[EelBait]

You've obviously never worked in a data center where cooling and power are a premium. In a room full of hundreds of Intel crap, you start to consider things like power consumption and heat dissipation. This is a fact: The higher the clock the more power and heat.

We started holding our data center manager accountable for his own electric bill. After that, efficiency (lower clocks to do the same work) started to take on a whole new meaning.

This particular criterion also got us to get the MS-weenies to shut up and we started to implement more Linux systems.

Everyone should benchmark with GCC

[RalphBNumbers]

If everyone benchmarked with open source compilers, there would be none of the shady benchmark-specific optimizations you'd expect to see in proprietary compilers. Everything would be above the table.

And that's not to mention the benefits for OSS compilers. Imagine the kind of resources and funding processor companies would dump into open source compiler projects if they were going to be the basis for their benchmark scores instead of their closed source proprietary compilers.

But..but..but...

[Anonymous Coward]

He's a Corporate Drone(tm) justifying Marketing Speak and Glossy Lit numbers.

Doesn't everyone realize that this is a black and white issue?

Corporate Drones == Lies
Populist Raving == Truth

Always always always. Doesn't matter what the numbers mean. They threw in that one graph with the single processor machine slower than the Intel just to throw off the hounds. But it didn't work.

G4 story icon

[Capital_Z]

/. had better get with it! We're talking about G5s now and the G4 chip icon is still up in the story post.

The G4 is so last month.

Curious

[igabe]

This is the first time I think _I_ have seen slashdot with an article they wrote compltely on their own.

Did you recieve a phone call directly or something(Apple calling Slashdot)? If so did they act really aggressive wanting to make sure people don't become anti-G5 before it is even shipped?

Not too important you might say, but interests me.

Re:Curious

[pudge]

Eh, we do this sometimes, when it is appropriate. In this case, I have a PR contact at Apple who asked me last week if I wanted to talk to someone about WWDC, and we set up a call last weekend, for this afternoon. It just happened to coincide with the benchmark discussion, which Greg was eager to set straight (he had read the arguments and already compiled his responses :-). We also talked a bit about some other topics, but nothing of interest that you haven't read elsewhere.

Re:Curious

[Graff]

we do this sometimes, when it is appropriate. In this case, I have a PR contact at Apple who asked me last week if I wanted to talk to someone about WWDC

You know, I always thought that this would be a good idea for Slashdot. I mean, you guys must have some pretty interesting contacts by now, use some of them to do a "news" article or two on your own. I'd still keep the old Slashdot question/answer interview around because they are interesting and good for the people who don't have time to do a traditional interview.

Other Benchmarks?

[WatertonMan]

Unfortunately the more egregious benchmark was the Quake benchmark. I'd have liked to have heard about that one. Th

Further I notice he didn't mention the problem of not doing comparisions to AMD.

While I can understand his reasoning, the fact is that most software on the PC runs under VC or Intel's compiler. It doesn't run under gcc. The benchmark might be a fair Linux/OSX comparison but implies something about Windows/OSX that is incorrect.

I'd also like to see the tests done under Mathematica and Photoshop discussed more. Apple's had a history with photoshop so there is prima facie reasons to distrust it. But the Mathematica test, which seemed the most exciting to me, is what I'd really like to see.

Realistically though the tools for Apple, including graphics drivers, are all very beta. So we should see improvements with time. And realistically benchmarks are typically kind of deceiving as an indicator of real world performance.

So any word on these other questions?

PS - I love OSX and would love to make a Mac my primary machine. If only Project Builder was up to the task so I could abandon Visual Studio. But I am excited about the G5, but I think Apple's "questionable" tactics have brought a lot of unfavorable press that more honesty would have avoided. Personally I think being within 10% - 15% of the top end PC would have been fine.

Re:Other Benchmarks?

[tbmaddux]

Unfortunately the more egregious benchmark was the Quake benchmark.

Are you talking about this one, where Apple posts 337fps at 1024x768/32bpp for the G5 and 275fps for a P4? I asked about that on another forum, noting that Tom's Hardware gets over 400fps from a P4/3GHz, and one respondent noted that

1. Tom's was using Q3A 1.16 instead of 1.32 (PunkBuster code is thought to be a little slower)
2. Tom's used set s_initsound 0 to disable sound while Apple noted default settings, which would imply sound was left on.
3. Tom's used demo_001 while Apple used demo_4

So I guess it's up to you to decide in the end if the benchmark was fair or no. I don't know enough about the details of Q3A to say whether the differences above are enough to justify a 30% decrease in framerate. It is worth pointing out that Apple's G5 matched Tom's reported framerates for the fastest Athlon XP.

Re:I love Apple

[sribe]

Fuck those $750 PCs, I'm getting me a $3000 Mac.

Q: You know what a $750 dual 3gHz Xeon PC is called?

A: "Stolen Goods"

Really, the point for Mac users is not so much whether or not the G5 wipes the floor with the Pentium, but whether or not the long period of performance stagnation is coming to an end, and whether or not top-end Mac performance will once again be reasonably comparable to top-end PC performance. And it looks like the answer to both questions is YES! (FINALLY!!!)

Amazing how people trust some no-named net person

[weave]

He made so many errors in his "debunking" yet so many people took it for gospel.

Like, the switch -mfpmath=sse when used in a P4 *does* use SSE2, but this guy thought just cause the switch flag says sse that it must be SSE only.

Then someone else (can't find the post, on usenet, under the mac advocacy group) pointed out that Dell's SPEC tests also disabled hyperthreading.

Then, based on this person's web page who no one even knows who he is, they start drawing conclusions that if Apple faked these (based on his flawed analysis), that they also must have faked those Adobe, Mathmatica, and other demos -- despite the execs for those companies being on stage also confirming the results.

Gotta love the net...

As for me, I don't know what to believe. I'm just going to patiently wait until some reputable sites spend a lot of time and do an in depth analysis and their own benchmarks, like Tom's Hardware for example. Then I may start drawing my own conclusions.

As for me, all I want is to be able to encode mpeg video at something greater than real time. Show me *that* benchmark please!

...but is gcc equal across architectures?

[Chad E Dirks]

Is there good reason to believe that the same compiler will produce relatively as well-performing of code for one chip it supports as it does for another? I don't think so.

In this case, performance will in part be a function of how mature and optimized the generation of code for the advantages of that particular chip is.

Because there is no guaruntee at all of fairness by using gcc for both processors, except of course if we had the expert opinion of someone intimately familiar with gcc's code generation for both processors, using gcc for both processors would seem to be little more than a marketing tactic to give the appearance of fairness and credibility.

It seems to me that a better test is to take the best compiler widely available for each chip, and then run your tests with the produced code. Now, this isn't necessarily real world application testing, but that isn't what we are necessarily looking for here.

How well the processor performs with code generated by the best generally available compiler, is, apart from extraordinary measures, the best prediction we have of how generally the processors will compare for any given well-written, production quality code.

Re:Benchmarks

[j3ffy]

But what was even more inpressive than the spec scores was watching the powermac squash the dual xeon in several applications from 3D video rendering, to photo editing, to audio processing, to mathematical calculations.

I'm a science guy, and for the calculations and simulations done here at the physics dept. where I work, the IBM power4 kills just about everything else. And when I saw the powermac calculate fractals with mathematica faster than the xeon box by more than a factor of 2, I was very excited (although a little cautiously) to see we will soon get power4 performance for well under $20,000

gcc a constant, that is naive

[AHumbleOpinion]

All this talk of gcc removing a variable is naive at best, misinformation if the speaker is knowledgable on the subject. Gcc is not a constant, the quality of it's code optimization varies from platform to platform. To be more specific, gcc is used by Apple to build MacOS X and Apple has been improving gcc PPC code generation. Apple provides gcc to Mac developers. Apple is also IBM's partner in the development of the PPC970. Gcc is the developer optimized compiler for the chip in many ways and is more comparable to Intel's compiler in this respect.

If everyone benchmarked with open source compilers, there would be none of the shady benchmark-specific optimizations you'd expect to see in proprietary compilers. Everything would be above the table.


No. Benchmarks would become less realistic. There is nothing wrong with proprietary compilers. If they use proprietary techniques not available to gcc, so what. The only consideration is whether the compiler is available to other developers. The Intel compiler is available under Windows and Linux so it would be completely fair to try it and gcc and pick the faster of the two.

Joswiak

[Jon Abbott]

Every time I hear the name "Joswiak", I keep thinking the guy is some hybrid between Steve Jobs and Steve Wozniak -- some kind of mutant creature straight out of the R&D Labs at Apple. :^)

But he didn't refute the most damning claim!

[faust2097]

Somehow this shifty Apple exec ignored the boldest claim of the bunch:

Misleading Prices

Both Apple and Dell are guilty of using misleading prices. For example, Apple gives the price of the low-end G5 as "$1999", and the high-end G5 as "$2999". In other words, they have subtracted $1 from a $3000 computer to make it seem cheaper, which is absolutely ridiculous. This demonstrates that both Apple and Dell are willing to mislead people when stating their prices.

What do you have to say now mister Joswiak if that is in fact your real name?

Re:But he didn't refute the most damning claim!

[doce]

this is the funniest claim i've seen in a while. not only does apple do this, so does dell, and so does virtually every consumer-oriented company on the planet. gas companies shave a TENTH of a PENNY off gas prices to make them seem cheaper.

a department store (was it macy's?) started this practice. the funny part is that the aim wasn't really to fool consumers into thinking it was less expensive. alas, the real purpose was to force cashiers to open the register, since the customer was almost always going to be due some change.

Re:Even if Apple is faster

[gerardrj]

Apple did.
Apple 2x2 G5: $3000
Dell (2x3.06 Xenon): $4000

Mac speed in "real world" application tests, about 2x as fast as the Dell.

Dell = $4000/work unit
Mac G5 = $1500/work unit

The Mac G5 is a much better value on cost on a price/performance basis.

Or were you thinking of something else?

Re:Even if Apple is faster

[pi radians]

Even if Apple is faster than Wintel, the Price:Performance ratio is horrible for Apple hardware and software.

I think this is the one time where Apple hardware hasn't been "horribly" priced. A 64bit dual-2Ghz workstation with SATA HDD, DVD-R, PCI-X, a 1 GHz FSB and a max of 8GB of DDR-RAM for under $3000.

As for software, that comment is just pure ignorance. 50% of the stuff they make is free, and the other software is all competitively priced. What software from Apple is overpriced?

It's money that matters.

[vought]

The "status quo" crowd that jumped all over Apple this morning for the "fake" benchmarks and "dishonest" wording will still find lots of reasons and ways to disparage the fruit company, simply because Apple isn't doing what they want - building the best, fastest, and most cutting-edge computers for $400.00.

Forget about Serial ATA - (Apple is the first top-tier manufacturer to make this interface stardard across their high-end machines.)
Forget about the new motherboard featuring HyperTransport, PCI-X, and the IBM-fabbed 1GHz northbridge chip. Oh, and 802.11G, USB 2.0, FireWire 400 and 800, and Bluetooth, too.
Forget about the imagination and creativity that goes into making a project like this go from concept to reality in eighteen months.

Why support a company like that? Bunch of dirty liars - there's no way a 2GHz chip could be faster than my Intel/AMD/whatever86!

Maybe it's not ultimately faster (although Greg's comments seem to indicate that the playing field was pretty equal). I don't buy "fast". I buy well-integrated tools that help me get work done, and in turn, bill clients. So I (still) use a Mac.

Jeez - to hear people around here, you'd think that innovation, style, performance, and the courage to move forward agressively and definitively with new technologies doesn't come at a price.

What other comapny would develop all these technologies to hardware and software maturity as part of a new hardware platform, then bring it all to market with system software already written (by the same vendor, I might add) to take advantage of new hardware features?

Those things DO come at a price. The price begins at $1999.00 for the 1.6GHz G5, or $799.00 for an eMac.

As long as there are people who just want to get work done on their computers without hiring an IT department or worrying about who is responsible for which component of the system, Apple will still be around.

I bill around eight hours a day with my Macintosh - the $400.00 price premium over PC hardware at the time I bought my G4/800 simply isn't an issue - over the lifetime of the machine, I'll probably bill at least two hundred times that amount for work made possible by its existence.

That $400.00 up-front cost means that I don't have to spend my time - my extremely expensive and finite time - having to deal with at least two vendors just to get a system with competitive hardware, a competitive OS, and support for them both. If your time isn't valuable, by all means cheap out and build your oft-touted (and perfectly capable) PC from parts you buy at Frys. $400.00 means nothing to professionals - it's cheap support insurance.

I hope Apple sells a TON of these machines - because they're practically the only personal computer company willing to take the initiative and responsibility for supporting hardware and operating system on equal terms.

Perhaps if Apple stressed the cost of ownership point to more people, they'd have higher sales. Our small business has nearly thirty Macs. I'm the lone IT person, spending an entire hour a week on supporting a bunch of artists and their Macs. What similarly-sized Windows-based business can make that claim?

Re:Removed one of the processors for the SPEC CPU

[dhovis]

Oh, yeah. Steve probably said "hey, vendors, come on over and do a little demo. Yeah, it'll be a duel, but don't worry about recompiling for the G5 (which is supposed to be trivial). We'll just see what happens."

Look -- they spent every last minute they could optimizing the builds they used for the demo - don't doubt it for a minute. On the other hand, every last minute probably wasn't all that long, and the demos did kick ass.

Actually, my favorite was the Mathematica guy who commented (IIRC) "We tried to come up with an example to show how being able to use more than 4GB of memory was helpful, but we couldn't come up with an example that didn't crash the Xeon"

Re:Even if Apple is faster

[vought]

Yeah, but you can roll your own dual 3.06GHz Xeon for WAY less than $4,000 (or $3000 for that matter).
2xCPUs would cost around $1400
Motherboard $300

'Cos everyone knows all you need is a motherboard and processors. Didn't you work in IT at a company I used to work for? You're the one who took the RAM out of my computer and said you'd be "right back", aren't you?

Excercise for you:

Add the cost of Bluetooth, PCI-X, 802.11G, Gigabit ethernet, SATA hard drives and controllers, DVD-R drive, power supply, all the other hardware stuff I've forgotten, plus iTunes, iDVD, iMovie, and the ten or so other bundled applications on the G5s, a Unix-based operating system with superior usability, and one year of free warranty and support for ALL of that stuff.

How much does your dual Xeon cost now?

Re:Apple: innovation or catch up?

[tuxedobob]

I don't think anyone's really arguing that most of Apple's improvements weren't catch up. Heck, when Steve Jobs was on stage he was saying how everything they added was "the latest PC standard". How could it not be catching up if it's already a standard on the PC?

The reason the Mac users are happy about all this is because we already knew we were way behind, and we've been begging Apple to catch up!

Even considering all the benchmarks, which may or may not be accurate, the simple fact remains that this Mac is much faster than the previous Mac. Which is good news for Mac users. And presumably the crowd at the keynote was full of Mac users.

Waiting for the ultimate benchmark

[TrekkieGod]

I'm gonna be modded as a troll on this one, but I gotta do it, because it will happen to me :)

::bangs head on keyboard for giving moderator ideas::

::Slashdot story: Tom's Hardware benchmarks the G5, and compares it to dual Xeons, dual Opterons, and (I guess), the P4::

Me: "Woohoo...I'll finally found out which is better" ::clicks link::

"Page 1: We have tested all these systems, and you will soon see our results." ::scroll down through ads, click next::

"Page 2: These tables show the systems we have tested on" ::scroll down, next::

"Page 3: Tables, cont.." ::yells out profanities, looks on table of contents, chooses "benchmark results"::

"Page 45: And now, let us take a look how the G5 stands against the current x86 and AMD64 processors" ::AAAAAAAAHHHHHH...can't stand it anymore, clicks on conclusion::

"Page 666: And thus, we conclude that the G5 is better in some ways and worse in others" ::NOOOOOOO...Now I'll never know!!!::

Compiler NOT constant

[Lulu of the Lotus-Ea]

Several other posters have noted that GCC/970 is really not the same compiler as GCC/Xeon. Sure there may be a bit of code in common between the versions, but the job of a compiler is to produce object code... and by definition, the object code for 970 is different from that for Xeon.

What matters to a purchaser is "How much performance can *I* get out of this machine". If I am performing CPU-intensive scientific calculation that require the fastest CPU I can find (at least for a given number of kilodollars), I'll almost certainly spring a few hundred extra for the compiler that produces the fasted object code on that platform (if needed, there's nothing ruling out GCC automatically because it's free).

It happens that for a Xeon or P4 (or Opteron, for that matter), the compiler that produces the fastest object code is ICC. Intel has done an amazingly good job with their compiler.

Now, sure, I *could* get a similarly optimized 970 compiler for comparison.... if one existed, that is. It looks like right now, GCC is the best you can get on a 970. It doesn't do a buyer any good to know that IN PRINCIPLE a more optimized compiler could be written.

All that said, the 970 looks like a very respectable chip. And Apple is selling their new machines at a very competitive price; and Macs have extremely friendly and stable OSs. All that means that it is probably well worth buying a PowerMac even if it will crunch big computations a few percent slower than a more expensive Xeon. But still... the "GCC is the common element stuff is pretty darn bogus."

Re:Who cares?... Geeks do!

[robvs68]

I'm becomming really dissapointed with what seems to be the majority of posts on /. (and especially in this thread). I was anticipating a lot of /.ers going on and on about how sweet the tech specs are on the PowerMac G5 hardware. It shouldn't matter what religion you are (M$, Sun, *NIX, Mac, IdogAppleToSoundSmart...), the hardware freeking rocks! Just like my attitude towards BeOS - I don't necessarily care whether the thing will gain 82% market share, its just cool shit.

Actually, gcc is not so bad

[Fefe]

It is quite difficult to produce better code than gcc, and my tests on powerpc (granted, those were a few years back using xlc on RS6000 with AIX 4) showed that xlc produced code of about the same quality -- sometimes worse, sometimes better.

The gcc "Haifa" scheduler was donated by IBM Haifa, by the way, so I think it's not surprising that gcc produces good code on powerpc.

On Intel it's quite the same, except that gcc does not vectorize code. From what I have seen, however, icc's vectorizer is not very useful either. I recently tested ogg-vorbis (which is a plain C floating point intensive benchmark) with icc 7 and gcc 3.3 and the gcc version was actually faster than the icc version (on my Athlon XP, target CPU pentium3) despite icc having vectorized several loops.

So all this "vendor-optimized C compiler" stuff is really besides the point. No C compiler will ever be able to match the quality of hand optimized assembler code, and the most important code (ffmpeg MPEG-2 decoder and MPEG-4 codec) has already been hand-optimized. You might be able to squeeze anoter 5 percent out of your code by using a vendor C compiler with insane optimizer settings, but what good is that if the end user is only going to use gcc anyway. I know I am, so I find the numbers for gcc actually more useful for comparison purposes than some vendor C compiler comparison.

Also, we don't want to encourage vendors to produce super vendor optimizing compilers, we want them to optimize gcc (so that everyone benefits, not just their users). So the more benchmarks are done using gcc, the better!

Photoshop WAS compiled with the AIX compiler!

[kriegsman]

I don't know if they still do, but for a while at the beginning of the PowerPC era Adobe was using the AIX compiler to generate its PowerPC binaries for Photoshop.

Admittedly, this was when the PowerPC was pretty new, and the choices were the IBM/AIX compiler which was robust and produced fast code but required an AIX box in addition to a Power Mac, or the nacent Metrowerks CodeWarrior compiler which run natively on the Power Mac, but generated poorly optimized code.

If I recall my history timeline correctly, after CodeWarrior came

• the Apple MPW "MrC" compiler (better code than CodeWarrior 1.0, but with a wacky command-line "IDE"), then
• gcc for PowerPC (cruddy code back then), then
• the Motorola PowerPC compiler (better code than Apple's compiler, with NO IDE - it plugged into the CodeWarrior or MPW IDE).
• Then Motorola inexplicably stopped selling their compiler.
• Later Motorola bought Metrowerks.
• Somewhere along the line, gcc learned to generate better PowerPC code.
• Eventually, Apple pretty much shelved their "MrC" compiler, and settled on using gcc for Mac OS X
• Monday, Apple released their "Xcode" environment -- still using gcc, I believe.

Apple's MPW tools are still available (free) here for Mac OS 7/8/9. The new Mac OS X tools including Xcode are available here.

As a side note, it's really nice to see Apple giving away a full development suite for free, and continuing to put development time and effort into improving it.

-Mark