A Distributed Front-end for GCC

format writes "distcc is a distributed front-end for GCC, meaning you can compile that big project across n number of machines and get it done almost n times as fast. The machines don't have to be identical or be running the exact same GCC version, but having the same OS is helpful." With the advent of faster hardware, I can't complain about kernel compile times anymore, but larger source trees could definitely benefit from this.

Comments

GCC version

[csnydermvpsoft]

The machines don't have to be identical or be running the exact same GCC version

Well, to some extent they probably do. If you're running GCC 3.2 on one, you wouldn't be able to run 3.0 on another because of binary incompatibility.

good software design

[ucblockhead]

If your system is well designed, compiling the entire thing should be a rare event. In a well designed project, most changes occur in c files or in headers only included in a few c files, so most changes only require compiling a very few files.

Compiling the whole source tree should be the sort of thing you do fairly rarely (for a big project), perhaps once a night, perhaps automated so no one has to watch it.

If compile time is something that is a significant problem for you, you really need to look at your code design.

Source code

[ucblockhead]

Running different versions could cause really nasty problems if different versions of gcc support different levels of C (like C99 or older C) or if one version has a compiler bug that another doesn't.

Can you imagine code compiling or failing to compile randomly depending on which machine happens to compile it? Yikes! Debugging nightmare...

Re:Source code

[wik]

I've had bad experiences with this in a Condor cluster of linux machines which had different versions of glibc. Seemingly randomly, my jobs would blow up into the netherworld without running and without an error message. Until the administrators matched all of the glibc's (but not the linux distributions, for some reason), I had to compile everything with -static on one machine and pray.

I wonder how much of a problem network bandwidth is in this system. With Condor, moving large datasets between machines is a problem. Object files can be pretty big and if you have a lot of them, you might risk pushing the compile bottleneck to the network. Even worse might be the link step, where all of the objects have to be visible to one machine (gcc doesn't have incremental linking yet, does it?).

Live boot CD?

[no_such_user]

I'd love a speedup, but the time I'd save compiling would be wasted on having to fully install another linux box. Being able to boot a CD with a live linux distro and this software, and then connect to these slave machines to help compile would be immensely helpful. My linux box is a Cyrix 200MHz PC. Being able to stick a CD into my Athlon 1800 to help the compile would be fabulous.