Comparing Linux C and C++ Compilers

ChaoticCoyote writes "I've posted a comparison of recent GCC versions (3.3, 3.4, and the coming 4.0) with Intel C++ 8.1, including several benchmarks and "state-of-the-product" reviews. The new article replaces an older piece I published in late 2002. This new comparison marks what I hope will be an ongoing series that tracks the quality of Linux compilers."

Comments

3.5 vs. 4.0

[slavemowgli]

Interesting. Something I don't understand, though, is why the article talks about both gcc version 4.0 and "the upcoming-3.5"; will there be a 3.5 version before 4.0, or is that simply a typo?

Re:3.5 vs. 4.0

[FullMetalAlchemist]

Tree-SSA is not new, it's new to GCC but the concept of Single Static Assignment itself is ancient and using it for optimizing trees in not new.
The problem is the GCC codebase is so badly designed it takes major changes to many parts to introduce a very old concept of SSA into it. So you can't take advantage of a generic SSA but have to introduce different patches for the system.

Back in my university days we did SSA in our own compilers, for a 5 week course! GCC needs a complete rewrite or a switch to something like TenDRA, which right now produces good solid code, but not extremely optimized machinecode. On the other hand TenDRA is really well designed so it would catch up really fast if more people worked on it.

Very informative.

[mind21_98]

Not bad. Not bad at all. Hopefully this will indeed spur development and add some competition to the otherwise small compiler market. :)

Binary size

[Guidlib]

Interesting that ICC seems to create consistently quite large binaries in comparison to GCC.

Re:Binary size

[psetzer]

Don't underestimate the importance of a full pipeline. If you can avoid a bubble in the pipeline at any cost, odds are you would want to do so. That's one of the beauties of predication in a processor architecture. Although I'm sure that there's some way to make an instruction a whole frigging cache line long, that's more an exercise in perversity than a practical limitation if you don't unroll your loops too much. I could see a series of instructions being like that, though. If you're just adding an array of four numbers to another array of four numbers, then unrolling is the only sane choice. If it's two arrays of 100 numbers, then you should do some testing based on how big the instruction cache on your processor is. If it's a combined cache, then you might see a really substantial drop. As always, profile, profile, profile.

I am dissapointed in Intel

[Billly Gates]

I hear people complain about gnu C++ alot because its not as good as commercial compilers.

However from what I see is the performance of the compiled code is the same with the exception that the resulting binaries are alot bigger.

Maybe gcc is good on x86 but sucks on other architectures? I know the Sun and SGI geeks have complained for years that gcc does not run well on their platforms compared to expensive alternatives by their vendors.

Has anyone run ICC and VC++.net on Windows? How does it compare to Borland and MS compilers under Windows2k?

Vectorization?

[r6144]

On some numerical tests icc seems to give twice faster code than gcc, and I think it is probably due to icc's automatic vectorization (the difference in almabench is said to be mainly due to the superior software SSE-based sin() and cos() functions used by ICC, correct me if this is wrong).

I think the author had better mark the tests whose inner loops have been vectorized, since while some algorithms are easily vectorizable, some are not, so the performance of both are interesting. After all, we care most about the features (such as automatic vectorization) of the compiler, while benchmarks only very roughly reflect the existence of such features, the applicability of them, and their effects.

Re:apples and oranges

[Jeff DeMaagd]

Are there any figures as to what proportion of Linux computers are x86?

Re:Performance isn't everything.

[fitten]

Different compilers also emit different warnings, helping one find bugs sooner.


You can say that again. We had a bug recently in some code that gcc compiled without any warnings or problems but the code didn't work right. Compiled it with another compiler and got a warning in the code, looked at it, and sure enough, that was a legitimate bug. Corrected that line of C, recompiled on that compiler and gcc and both executables worked.

Re:gcc!

[vadim_t]

Definitely a troll.

I googled a bit to take a look at this mudflap thing. While it looks interesting, the system looks considerably more complicated than -fstack-protector and should be noticeably slower too.

If I understood correctly, -fstack-protector simply adds a guard value and checks it hasn't been overwritten. The performance overhead is minimal. On the other hand, mudflap does some rather intensive checking that involves things like lookup tables, which obviously has to take more time than checking if a the guard value is still what it should be.

According to this PDF, mudflap can have a really high performance penalty in bad cases. The performance section is rather short though, so it's not very clear what gets slowed down the most.

Re:ProPolice

[DrSkwid]

The OpenBSD team use it, and I think they know better than you or I.

Not by technical choice, they want to use plan9's C compiler but have licence quibbles.

Quoting Theo : "in particular we were interested in the c compiler"

C/C++ vs. Fortran

[Aardpig]

I find it interesting that there are only two C/C++ compilers available for Linux, as compared with seven Fortran 90/95 compilers (soon to be eight with the release of GCC 4.0). This not only dispels the myth that Fortran is a dead language, it also suggests that there is much more of a competitive market in compiling numerical code, than in producing other types of software.

Re:C/C++ vs. Fortran

[ChaoticCoyote]

I'm working on a Fortran 95 comparison, which will include several products.

Most people doing serious numerical work are creating proprietary or in-house applications; few people have the clusters required to run the type of code written in Fortran. So there is a commercial market for Fortran; places like National Laboratories and universities have the money to buy software.

Re:C/C++ vs. Fortran

[Brandybuck]

the reason for the dearth of C++ compilers is because C++ is an ugly language that is nearly impossible to implement correctly.

I would rephrase that to say that C++ is a *big* language with a spec managed by committee.

Re:Not a lot of selection for Linux compilers, eh?

[Jollyeugene]

Other compilers for Linux (Try google before you troll):

TCC (tiny c compiler).
Compaq Alpha C compiler.
OpenWatcom C/C++ compiler
TenDra C/C++ compiler
egcs compiler (which merged into gcc- they saw that it had some advantages.)
ChEmbeddable
Cint c/c++ interpreter (not exactly a compiler)
CC65 Commodore C compiler
Absoft C/C++ Fortran compiler
lcc

These all run on linux. Some are open, some are not. GCC is used mainly because it is portable to just about anything-- so there goes your argument of GCC restricting choice. GCC exists to promote choice, and it does this.

Take your code written for Visual Studio and compile it on Sun. Can't do that with Visual Studio? No MFC on Sun? Your best bet will probably be good old GCC with WINE. So how are you restricted by gcc?

Now, if you want to make a cross compiler to compete with gcc- one with seperate front ends and back ends, so it can accept multiple languages, and can compile to just about every machine on earth-- then nothing is stopping you. Oh, what is that? Not interested. Neither is anyone else. Programers see no need to duplicate something that has been done well the first time. One good cross compiler is enough- GCC represents a NATURAL MONOPOLY.

If you must have optimized support for AMD...

[Ayanami Rei]

Portland Compiler Group has a compiler that's fairly decent and optimizes for both AMD64 and Intel 32-bit (although they're pushing the AMD64/clustering angle right now).

It's nice, but much less free. 15-day free trial. Prices are not bad, but it's still a little steep, even for an academic license.

This is not a plug... just in case anyone was wondering if there was something besides ICC you could try in to benchmark GCC on something that ICC can't cope with.

LLVM and C++

[pb]

Another thing to try--LLVM can use g++ as a front-end to compile C++, and can often do further optimizations to speed things up even more. LLVM also has a C back-end, so you can benchmark other C compilers against each other too!

Of course, the real fun will begin when it has completed Java and C# front-ends. :)

GCC usually whips ICC for me

[Baudelaire76]

I'm always perplexed by such comparisons. Specifically, everyone talks about how awesome ICC is for numerical stuff. Well, I have written a fair amount of C++ code for computational fluid dynamics, and, for my code, GCC has always beaten ICC by a fairly wide margin (typically, 50 percent or more). Perhaps it has something to do with the fact that my code makes heavy use of templates? I don't really know.

Also, in my experience, the C++ standard library that shipped with the previous versions of ICC just wasn't that good. Well, that may be a little harsh, but it was a source of considerable irritation to me that ICC's implementation of valarray did not even use expression templates (hello temporaries!). Yes, I know valarray isn't all that, but it is standard C++ (hence, in principle, code that uses it is portable). Once again, the GCC implementation was very much superior. This may be part of the reason why ICC now uses GCC headers.

(As an aside, I did roll my own implementation of valarray that uses expression templates. I tested it with both GCC and ICC, and GCC still won.)

I am now curious to test whether things are indeed better with ICC 8.1 (the last version I tested was 8.0). Maybe this time it will actually speed my code up rather than slow it down.

Compatibility issues

[captaineo]

Regardless of benchmarks, I consider the nasty backwards compatibility issues of GCC a serious problem. Virtually all of my Linux tech support time over the last two years has been spent messing with libstdc++ or libgcc_s versions to get various software packages to run. The problem is exacerbated because these libraries in turn are coupled tightly to glibc and pthreads, leading to more versioning headaches. My #1 wish for GCC is to get these libraries set in stone and make them as independent of glibc/pthread versioning as possible. I really want to be able to install a random C++ binary and have it work the first time.

Also, the ability to compile a C++ DSO without requiring the exact same compiler version as the program that will load it, is a major need for commercial Linux software vendors.

Silly gcc optimizations

[0x0d0a]

I was empirically analyzing the gcc 3.3.3 optimizer a couple of days ago. Some interesting points:

[argh...lameness filter refuses to allow source code]

I'm providing a link to my journal entry so that I can post these examples.

Re:gcc!

[adamwood]

Erm. If you look at the current top500 you'll see that there's a Xeon machine in the top 5.

Have you actually tried other platforms? The best compiler on a given architecture is usually the chip vendor compiler -- IBM's compiler beats GNU on PPC 970, HP's beats GNU on Alpha, etc.

Sure you can find the odd piece of code where Gnu will beat vendor compilers but overall, and particularly for large scientific workloads, vendor compilers win for raw speed.

What about documentation to write new front ends?

[master_p]

I am especially interested in GCC because I want to write a new front end (my own programming language). I don't want to mess with back ends, since I am not interested in that stuff; the linguistic part of the programming languages is far more interesting. But I can't find decent documentation anywhere on the internet for a GCC newbie. I tried looking at the source code, but it is an ugly mess of macros. The article of course mentions C/C++ only, but GCC is something more than that...I think it deserves a well written piece of how to extend it with new front ends.

Compile time

[Per Abrahamsen]

I always get confused about people claiming compile time is important for development. It seems to be people who make a habbit of compiling everything from scratch. Haven't they heard of make? To me, link time is the limiting factor. When I make a change, I have to compile just the file I changed, or maybe a handfull of files if I changed an interface. And then I have to link all 236 files, which is where the time is spend.

I can see it compile time being important to people who use the compiler as an install tool (like gentoo users), but not for any serious developers.

compiling linux with a different compiler?

[polyp2000]

Does anyone know if it is possible to use one of the alternative compilers ICC , OpenWatcom or Corneau ? to build a linux system from scratch?

I would be truely intrigued at what overall performance might be like.