A Distributed Front-end for GCC

format writes "distcc is a distributed front-end for GCC, meaning you can compile that big project across n number of machines and get it done almost n times as fast. The machines don't have to be identical or be running the exact same GCC version, but having the same OS is helpful." With the advent of faster hardware, I can't complain about kernel compile times anymore, but larger source trees could definitely benefit from this.

Comments

GCC version

[csnydermvpsoft]

The machines don't have to be identical or be running the exact same GCC version

Well, to some extent they probably do. If you're running GCC 3.2 on one, you wouldn't be able to run 3.0 on another because of binary incompatibility.

Re:Interesting approach

[csnydermvpsoft]

Definitelly would make beowulf clusters interesting for compilation as well as hard core numerics (no joke intendend).

Actually, you wouldn't need a Beowulf cluster at all - just a bunch of networked machines.

Apple Projectbuilder

[selderrr]

I sincerely hope Apple makes this feature into projectbuilder, which compiles insanely slow when compared to codewarrior. If it wasn't for the superior interface and integration with interface builder, I'd swap back to codewarrior right away.

Does anyone here know how good the speed increase is when compiling on dual G4s versus a single proc ?

Perfect for Gentoo

[waffle zero]

Whether you're looking to install Gentoo on a old pentium to use as a router or sacrificing your first born to compile KDE, it should make things go quite a bit faster.

Well unless every computer you own runs Gentoo you want to emerge world.

Re:hmmm

[Blkdeath]

In that case you might like to look at ccache

Isn't the default cache size somewhere to the tune of 2-4GB?

I recall that all of my lower powered machines were lucky to see a 6GB drive, letalone have 2-4GB to spare.

good software design

[ucblockhead]

If your system is well designed, compiling the entire thing should be a rare event. In a well designed project, most changes occur in c files or in headers only included in a few c files, so most changes only require compiling a very few files.

Compiling the whole source tree should be the sort of thing you do fairly rarely (for a big project), perhaps once a night, perhaps automated so no one has to watch it.

If compile time is something that is a significant problem for you, you really need to look at your code design.

Source code

[ucblockhead]

Running different versions could cause really nasty problems if different versions of gcc support different levels of C (like C99 or older C) or if one version has a compiler bug that another doesn't.

Can you imagine code compiling or failing to compile randomly depending on which machine happens to compile it? Yikes! Debugging nightmare...

Re:Source code

[wik]

I've had bad experiences with this in a Condor cluster of linux machines which had different versions of glibc. Seemingly randomly, my jobs would blow up into the netherworld without running and without an error message. Until the administrators matched all of the glibc's (but not the linux distributions, for some reason), I had to compile everything with -static on one machine and pray.

I wonder how much of a problem network bandwidth is in this system. With Condor, moving large datasets between machines is a problem. Object files can be pretty big and if you have a lot of them, you might risk pushing the compile bottleneck to the network. Even worse might be the link step, where all of the objects have to be visible to one machine (gcc doesn't have incremental linking yet, does it?).

Live boot CD?

[no_such_user]

I'd love a speedup, but the time I'd save compiling would be wasted on having to fully install another linux box. Being able to boot a CD with a live linux distro and this software, and then connect to these slave machines to help compile would be immensely helpful. My linux box is a Cyrix 200MHz PC. Being able to stick a CD into my Athlon 1800 to help the compile would be fabulous.

Re:Distcc compiled programs have problem with gdb

[truth_revealed]

Why so negative? They are aware of the gdb issue and are working to fix it. This does not make distcc 'worthless' by any stretch.
Even if you do have a crash gdb will still give you the stack trace of a distcc-compiled -g program complete with function names and line numbers - you just don't get the source code without setting a directory directive within a gdb session. Big deal. It's perfectly usuable.
Sometimes I'll launch production builds mid-day to correct - wait for it - mid-day production problems. This has to be done as quickly as possible, and distcc is very useful.
The gdb thing is by no means a showstopper.