IBM Releases XL compilers for Mac OS X

Visigothe writes "IBM released their XL Fortran Compiler and XL C/C++ Compiler for OS X. The compiler is binary compatible with GCC 3.3, and has multiple levels of optimization, creating binaries that are much faster than their GCC-compiled counterparts." No prices are noted, and the planned availability date is January 16.

Comments

Benchmarks

[melquiades]

Has anybody seen any useful benchmarks of compiler output comparing XL and GCC on PowerPC?

That would be interesting to see.

Re:Benchmarks

[integral-fellow]

for performance comparisons, see this page:

http://www.spscicomp.org/ScicomP7/Presentations/ Blainey-SciComp7_compiler_update.pdf

Re:Benchmarks

[York the Mysterious]

http://www.spscicomp.org/ScicomP7/Presentations/Bl ainey-SciComp7_compiler_update.pdf Here's the correct address. The space in there screwed it up.

Re:Benchmarks

[klui]

Slashdot mucks long lines. You need to use a link like this. Basically, on a POWER4 system (unknown re: G4/G5), specint2000 is around 30% improvement, specfp2000 is around 50% improvement. (Just eyeing the results.)

Supports Objective-C!

[jazuki]

Very cool! Looks like the C/C++ compiler also has support for Objective-C now. Even if it's in the form of a "technology preview" and probably preliminary.

This means that this could well be usable as a replacement for GCC in developing Cocoa-based apps. It's good to finally have some options. Can't wait to see how well it works!

Supports G4 and G5, but not G3

[Laplace]

It's great that IBMs compiler produces faster code that is compatible with gcc, however it appears that it won't generate code that runs on G3 machines. This means if you want to build apps with it you either need to write code that builds with two compilers or not support any G3 machines.

As a very happy G3 user I will be sad when I'm forced to upgrade.

Re:Supports G4 and G5, but not G3

[Gogo Dodo]

If I remember right (I'm not a Mac programmer), fat binaries are for mixing 68K and PPC code, not different PPC code.

Re:Supports G4 and G5, but not G3

[LordAlexander]

Actually, the G3 is a 603-based CPU and the G4 is a 604-based CPU.

It was rumored a while back that Apple might use IBM G3+Altivec chips in PowerBooks, further confusing the issue.

Re:Supports G4 and G5, but not G3

[Visigothe]

The original "Fat Binaries" on the Mac were for 68k/PPC version, however, the idea of the FAT binary is not limited by CPU architecture.

You could easily produce a "Fat Binary" that runs on G3/G4-G5/Alpha/68K/SPARC. That said, it would be one big binary before you stripped it =)

Re:Supports G4 and G5, but not G3

[Anonymous Coward]

NeXT did "Fat Binaries" before Apple did, and they are still possible in OS X's .app bundles. NeXT's added Fat Binary support on Black Tuesday in 1993. Apple's Fat binaries were introduced with the Power Macintosh line in 1994. NeXT's fat binaries could be built to run 68K, x86, SPARC, and PA-RISC.

Of course, right now the search algorithm isn't designed for a fallback mechanism. The system can consider itself either a "MacOS" or a "MacOSClassic". Both are assumed to be generic PPC code and one doesn't fall back to the other.

Re:Supports G4 and G5, but not G3

[Toraz Chryx]

Actually you're both wrong, kinda.
the original G4 is a G3 + Altivec + MERSI + the 604e's FPU hardware

the modern G4's bear very little resembalance to the 7400 however, having a pipeline three stages longer and a more elaborate altivec implementation.

Pricing

[Erect Horsecock]

According to an Ars thread the XLC compiler will be $499 for a single seat license. WAY below the cost for the AIX versions.

Linkage

Current compiler?

[Robowally]

What is MacOS X currently compiled with? Is it GCC? If so, the new IBM compiler would presumably speed up the entire OS somewhat if it were recompiled via IBM's compiler?

Re:Current compiler?

[Erect Horsecock]

What is MacOS X currently compiled with?

I'm not 100% sure but I seem to remember in the WWDC keynote Jobs saying it was built with gcc3.3 the version that ships with Xcode.

If so, the new IBM compiler would presumably speed up the entire OS somewhat if it were recompiled via IBM's compiler?

Ya probably. I was surprised that they implemented Vector support so quickly. XLC really shines on Floating Point code, but I'm really curious to see how well it handles Vector. Even if the whole operating system isn't compiled with xlc as long as the core libs and things like codecs for QT and other multimedia apps the speed up would be impressive.

Re:Current compiler?

[Selecter]

Yes, Panther was built with the GCC 3.3 "+" some poeple are calling it. The OS would speed up quite dramatically *if* Apple were to release G3/G4/G5 specific versions as the opts for the G5 if taken to the highest level would actually slow down or break things with a G4 or more likely a G3.

More likely they will break it down into chip specific libs and kibbles and bits and have the installer detect and choose. If they code for the middle ground I fear they will give up the best chance of having huge speed gains. They need those gains on the G5 to keep their momentum going.

Re:Current compiler?

[TALlama]

The only problem being that XLC doesn't support G3 Machines, so it would be impossible to create a binary that runs on any Mac, as is currently supported by GCC. In theory they could make separate binaries and install the 'correct' one, but that poses problems for systems booting off of external hard drives (which binary do you put on the drive?) and booting over the network. The ability to carry around a copy of OSX on your iPod is a powerful thing, and not one most people would give up lightly.

Of course, this could have been gotten around by using Bundles, which is a folder that acts like a double-clickable application. The structure is:
SomeApplication.app/ <-- The application
Contents/MacOS/SomeApplication <-- The OSX binary
Contents/MacOSClassic/SomeApplication <-- The OS9 binary
Contents/Resources/Blah.jpg
Contents/Resources/Foo.tiff

It chould be:
SomeApplication.app/
Contents/MacOS/SomeApplication <-- The generic binary
Contents/MacOS-G3/SomeApplication <-- The G3-optimized binary
Contents/MacOS-G4/SomeApplication <-- The G4-optimized binary
Contents/MacOS-G5/SomeApplication <-- The G5-optimized binary
Contents/Resources/Blah.jpg
Contents/Resources/Foo.tiff

When you double-click, it uses whatever binary is appropriate for the system. Unfortunately, this doesn't work for Frameworks, which lack the notion of platform.
A rant to beat the lameness filter: this bundle format should be adopted everywhere. It allows for a folder to be used as an application, and to contain all the resources it needs to be used and run. Moving the folder moves the application, and the folder doesn't use any vodoo to keep the data together, as pretty much any HD format understands folders and files.
In addition, the multiple-binaries trick (as shown above working with OS9/X and proposed for processors) would allow the same bundle to work on muluple platforms, so I could email you a zipped version of Office from my Mac that could work on your Wintel, no Java required.
The support is in the Finder/Explorer/Browser, which needs to understand that 'double click on bundle' == 'find correct binary and launch it'.

Re:Current compiler?

[Graff]

Of course, this could have been gotten around by using Bundles, which is a folder that acts like a double-clickable application...When you double-click, it uses whatever binary is appropriate for the system. Unfortunately, this doesn't work for Frameworks, which lack the notion of platform.

Ahh, but that is what fat binaries are for. A fat binary allows you to package up several different versions of a program optimized for different runtime architectures.

Macintoshes have been able to use fat binaries for some time, it was one of the things that allowed the fairly seamless transition from 68k processors to PowerPc processors.

Here is more information on optimizing for the G5, if you look towards the bottom there is notes on packaging an app to run on different processors.

Setting up a karma whore...

[Troy]

Pardon my ignorance, I was 31337 for only 37 seconds in 1997.

What do they mean when they say that two compilers are "binary compatible" Does it mean that XL produces identical machine code? Does it take identical switches so makefiles don't have to be rewritten? Does it simply mean that XL has the same foibles as gcc, so code written to gcc's foibles doesn't need tinkering? Use of the term doesn't quite fit with my current understanding of compilers.

-Troy

Re:Setting up a karma whore...

[dr2chase]

Binary compatible means same data layouts, same parameter-passing conventions, same conventions for shared libraries and position-independent code. However, between those interfaces, the generated code is probably different.

Think of it like nuts and bolts -- a nut and bolt are compatible if they have the same diameter and threads per inch, but they may be made of carbon steel, steel, bronze, nylon, titanium, whatever.

Re:Setting up a karma whore...

[davechen]

Looking at IBM's accouncement it looks like they use same headers and run-time libraries as gcc 3.3. They say you can combine xlc and gcc compiled files. I.e., you can link object files.

Re:Setting up a karma whore...

[MBCook]

I believe that it referes to the interfaces between code. What I mean is that, for example, it lists the functions in object files the same was as GCC and they are called with the same machine code sequence as GCC (they way arguments are put on the stack, etc). This is good for a few reasons. For one thing, it means that code written with this compiler can link to code written with GCC or vice-versa. Ordinarily you can't take an object file from VC++, one from OpenWatcom, one from GCC, and one from ICC and link them together. But if all the compilers were biniary compatablie, you would be able to. It has nothing to do with the internal code generated, as if both compilers generated identicle sequences of machine code, one couldn't be faster than the other. I think the main benefit is that, for example, you could use a static library that was compiled with this compiler with your code that uses GCC without a problem.

As for commandline switches and such, I would assume that they would be the same (or that there would be a simple option like --gcc that would turn on "gcc mode" so that it took the same command line stuff).

PS: If I'm wrong would someone please reply and correct me, and not just mod me wrong?

Re:Setting up a karma whore...

[sartin]

For C++, binary compatible goes the extra distance of requiring that the dynamic dispatch and run-time type information layouts be the same. When a C++ class gets created, there are hidden bits of data that are used to handle virtual functions and the likes of . Many binary interface standards don't specify how these work, but to be truly compatible, the two compilers have to match here. See Stroustup's Technical FAQ for a little more information.

Re:Setting up a karma whore...

[g_lightyear]

"binary' refers, indeed, to the binary compatibility of object files; in GCC terms, when there's an "ABI" change, you have to re-compile all applications, as new stuff compiled in the new Application Binary Interface can't access stuff compiled in the old ABI.

What it REALLY means:

1) You can compile the majority of your application in GCC, and selectively compile in IBM's XLC.

2) You can compile one library in XLC, and link it in to your GCC application.

3) You can compile a library in GCC, and link it in to your XLC application.

Etc. You get the point. Essentially, while the code they generate is very, very different in terms of optimization and performance, they are, in fact, completely interchangeable in terms of the things they produce as output.

XLC is, in fact, a very different beast than GCC. The number of optimizations it provides goes well beyond what GCC currently provides, and does include auto-vectorization and support for OpenMP - things which don't suck on parallel systems.

So XLC is a good thing for commercial software developers, and at minimum, the compatibility of the systems means that we as developers have no excuse not to be compiling, at bare minimum, the most *important* functions (and if we're doing it this way, it might as well be specific functions) in XLC, and link in that parallelized and optimized object file into our existing project.

As for commandline switches... nope. Almost never compatible. No hope. Basic stuff is mildly similar, but the guts you'd use once optimizing are very different.

But at a high level, yes, you just say xlc -O3 instead of gcc -O3, only you might say xlc -O5.

FASTER OS X?

[zulux]

What does Apple use to compile OS X - and if IBM get the Objective C sections woking properly, could Apple use the IBM comiper to get OS X to run faster?

Re:FASTER OS X?

[addaon]

OS X and all Apple software for OS X that I know of (maybe not including some high-end stuff, I just don't know for sure) is indeed built with GCC.

Porting an operating system to a different compiler is a pain in the neck, and most OS's use compiler-specific tricks to deal with low-level details. Also, most of Apple's higher-level software is in Objective C, and as of now only GCC really supports Obj C well on the mac.

Re:FASTER OS X?

[Visigothe]

Apple currently uses GCC to build Panther. As XL is much faster in 95% of the situations, I would imagine that Apple would transition to XL once the Obj-C portions of the compiler were a bit more mature. [The public beta of XLC didn't have any Obj-C support]

Re:FASTER OS X?

[DAldredge]

Have you thought about applying for a /. editors job?

Re:FASTER OS X?

[rmlane]

As mentioned by others, the majority of OS X is compiled by GCC.

The exception is Quicktime, which uses (and has used since well before OS X) a older, custom version of the IBM compilers. I believe, but am not 100% sure, that Quicktime has always used the IBM compilers on PowerPC CPUs.

This is very good news for Apple's science users, one of the real problems pushing Mac boxes into some markets has been the lack of a really good Fortran compiler. The performance boost for C/C++ code will also be appreciated.

As for a wholesale transition of OS X to the IBM compilers: next to no chance. QA of the transition would take far too long and absorb resources that could be better used on other improvements. It would also cause problems with the Open Source versions of Darwin, so expect the vast majority of OS X to remain GCC compiled.

That being said, I would expect that certain chunks will be transitioned, where it makes sense. The output of the IBM compilers is binary compatible with GCC, so you can recompile (and re-QA) chunks of the OS where you'll get a major improvment.

Quartz Extreme, CoreFoundation and AppKit spring to mind, but don't expect this to happen in 10.3 or 10.4, more like 10.5 or 11.0.

Re:FASTER OS X?

[g_lightyear]

As there's work going into XCode to ensure that any project can specify which compiler it uses on a target-by-target basis, I fully expect we'll see several core projects in the Darwin codebase switch over to using both compilers (where XLC will be used to compile specific branches) in 10.4.

OpenMP, at app-level, is pretty much guaranteed to get some use, and Apple will very likely spend some time in the vec/math libs fully OpenMPing that code to get parallel use of both CPUs.

CoreGraphics would probably get some small, critical sections built with it, but it's much more difficult to figure out how to get good benefit out of OpenMP code in those circumstances.

Anything else introduces problems; OpenMP will spawn off threads to do work, and if those threads start accessing code that isn't actually MT safe and/or fully reentrant at that level, it's going to get zero beneift.

Re:FASTER OS X?

[gerardrj]

You mean it would consume more resources than a complete re-write of the OS from Pascal to C as happened with System 6 to System 7 transition?

I expect that once IBM completes the ObjC compiler that most of the OS will be migrated to that compiler, as will much of the high performance commercial software out there. The developer tools will still come with the free GCC compiler, and Apple will still maintain it, but without changes to the core of GCC (which are being resisted by the maintainers), it will never make the G5 shine.

I have yet to see pricing from IBM on these compilers, so I don't know yet how viable they will be for smaller developers. It might be that developers offer two versions of their software: standard for the regular cost, and the 15%-40% better performance version from the IBM compiler for like 20% higher cost, just so they can recoup their added costs.

Re:FASTER OS X?

[gerardrj]

Apple has completed some very major improvements to the gcc code base, I don't recall, but I think Apple managed so improve compile times by something like 25%, and code performance of something like 15%. I don't remember where I read those numbers, or how close I am to what I read.

The reason (again from what I've read) that the gcc maintainers are resisting the changes that Apple would really like to make for the G5, is that the changes would fundamentally break many if not most of the other platforms compilers. I admit I don't understand compiler optimization, but apparently the logic that gcc uses for modeling the processor just can't keep up with the number of execution units, registers and/or the number of instructions in flight. I think the way the G5 "groups" instructions in to bundles of 5 for scheduling also throws things off a bit. I've read a few discussions that it's time for GCC to go through some sort of fundamental change that would allow for vastly different acitectures to be supported via plug-ins or modules or something.

Apple could indeed fork gcc, but given their commitment to "giving back" to the open source community, that 's probably not going to happen. Hopefully someone more "in the know" can either confirm or correct some of this. I personally think also that forking something as fundamental as the compiler that's used for almost all open-source software would be a "bad thing".

Autovectorisation ?

[Jesrad]

Weren't these compilers supposed to bring automatic conversion of multiple 32bits arithmetic operations into Altivec-accelerated code ?

Fortran Motives

[fm6]

From what I've heard, software companies hate selling Fortran compilers. You'll notice that Microsoft no longer has one. Not enough people use the language to make it worth the development and support costs.

So why are you still able to buy Fortran compilers? Because the people who use the language tend to be engineers (the physical kind) and scientists, and thus spend a lot of money on high-end computers. No Fortran compiler, not fat contracts for your Starfire and Origin boxes. Which is why Sun and SGI both sell Fortran. And whose the leading vendor of Fortran for the Itanium? Good guess.

So is IBM trying to help Apple sell more Macs? Probably not. They'd make a little money from the extra CPU sales, but not enough to justify something like this. More likely they have this compiler to help them sell more high-performance PPC systems. As long as they have it, not that much extra effort to port it to the Mac.

Re:Fortran Motives

[jabberjaw]

Yes indeed Fortran is still alive and kicking. Although I have heard that some of the physics libraries are being converted to C/C++.
As an aside, has anyone else noticed the lack of Fortran texts in brick & motor bookstores? I know Numerical Recepies in Fortran is online, anyone care to mention a good intro. text for a "n00b" like me?

Re:Fortran Motives

[fm6]

I used to work with a guy who had just finished a Physics PhD. He hated Fortran, and had insisted on using C++ all through school. But he admitted to being the only person in his program who did so. Proof that programming languages are as about the community they serve as the technology they encompass.

objective-c support

[sethdelackner]

So has anyone got better pointers towards the state of their objective-c support? I know they say it is there as a technical preview with no guarantees until they finish, but does it basically work and is slow, or is it unable to compile even modestly complex stuff?

Don't forget x86!

[MarcQuadra]

don't forget x86, there's nothing STOPPING the *.app files from holding code for any architecture. If Apple ever does have to jump ship to x86 I'm sure there'll be a lot of apps that are distributed with PPC and x86 (and probably x86-64) executables inside them.

One Application Icon to Rule Them All.

Re:0 is more than 500

[bash_jeremy]

I don't think IBM is trying to compete with the IDE. IBM made a somewhat superior compiler to gcc (what Apple currently uses to compile OSX and all their apps).