Reduce C/C++ Compile Time With distcc

An anonymous reader writes "Some people prefer the convenience of pre-compiled binaries in the form of RPMs or other such installer methods. But this can be a false economy, especially with programs that are used frequently: precompiled binaries will never run as quickly as those compiled with the right optimizations for your own machine. If you use a distributed compiler, you get the best of both worlds: fast compile and faster apps. This article shows you the benifits of using distcc, a distributed C compiler based on gcc, that gives you significant productivity gains."

Comments

DistCC is good, but there's some info missing

[PacketCollision]

While distCC is a great tool, there are a couple things to mention. First, the article blurb states that distCC is "a distributed compiler based on GCC." It is actually a method of passing files to GCC on a remote computer in such a way that the build scripts think it was done locally.

The article also says that other than distCC, the computers need not have anything in common; this is not strictly true. Different major versions of GCC can cause problems if you are trying to compile with optimization flags that are only on the newer version. I have run into this on my gentoo box, trying to use an outdated version of GCC on a redhat box.

Another thing is that some very large packages have trouble with distributed building of any sort (either multiple threads on the same machine, or over a network like with distCC). As far as I know, at least parts of xfree86, KDE and the kernel turn off distributed compiling during the build. Some of this might just be in the gentoo ebuilds, but I tink some of it is in the actual Makefiles. If a program has trouble compiling, it's always worth a shot to turn off distCC.

A good resource for setting up distCC on a gentoo system (since compiling is so large of gentoo, this is particularly important) is gentoo.org's own distCC guide

Re:DistCC is good, but there's some info missing

[XO]

Actually, you do NOT need to have the exact same version of GCC. However, there are certain points where the compile options given to one version will fail when given to another version. I used DISTCC quite happily between my two RedHat 7.2 boxes and my Debian box, until Debian upgraded beyond I think it was GCC 3.2.0 or so.. with the RedHat boxes having been on 2.96 i think.

Re:DistCC is good, but there's some info missing

[meowsqueak]

It's not a requirement, it's a recommendation. Mixing gcc versions will give you unpredictable results, some of which are perfectly acceptable, and some of which are fatal.

Re:DistCC is good, but there's some info missing

[sonicattack]

And I've compiled the Linux kernel with distcc splitting the load over two different architectures - one Intel box running Linux, and one UltraSPARC running Solaris (with a cross-compiling GCC).

No problems. That I noticed. Wouldn't compile a production system kernel this way, though. :)

I'm not sure how this is news

[Anonymous Coward]

It's also been been discussed here on Slashdot (two years ago!) in "A Distributed Front-end for GCC" and earlier this year in "Optimizing distcc."

Distcc is great for installing Gentoo on an older computer because you can have other (faster) computers help with the compile, and if you like distcc, you may also like ccache.

Re:nc: a better tool for distributed builds

[XO]

it appears that NC requires both computers having access to the same filesystem.. whereas with DISTCC it doesn't... loooks to be a fairly sizeable difference

My experiences with distcc

[meowsqueak]

I've spent the last week setting up a Gentoo cluster with distcc and I've noticed a few things:

1. when *recompiling*, the advantage due to ccache far outweighs the performance of distcc on the first compile. If you're testing distcc you need to be aware of this and disable ccache.

2. most large packages either disable distcc (e.g. xfree by limiting make -jX) or compile small sets of files in bursts and spend the majority of time performing non-compilation and linking. Distcc helps with the compilation but because it's only a small part of the total build time, the overall improvement isn't as great as you might have hoped.

3. distccmon-gnome is very cool.

4. using distcc with Gentoo transparently involves modifying your path and this can make non-root compilations troublesome (permissions on distcc lock files). I haven't figured this one out yet other than to specify the full path to the compiler: make CC=/usr/bin/gcc rather than CC=gcc.

5. the returns from adding an extra distcc server to the pool drop considerably after the first few machines. Even on a 1 gigabit LAN the costs of distcc catch up with the benefits after a while. This is more of a concern when compiling lots of small files.

6. it can handle cross-compilation with a bit of configuration.

So although distcc can often reduce build time, it's not quite as effective as you might assume or hope at first.

distcc causes kernel panic...

[kidlinux]

At the moment there's a bug in Linux kernel 2.4.26 that causes the remote compiling systems to encounter a kernel panic (and crash.)

It's a known bug and has been discussed on the lkml. The bug is also discussed on the gentoo bugzilla. A patch is also available, though the patch program didn't work for me so I had to apply it manually.

The patch seems to be holding up, too. If you're using distcc on systems with vanilla 2.4.26 kernels, I'd suggest patching them.

Incredibuild

[kingos]

For those using Visual Studio on Windows, I highly recommend a tool called Incredibuild to do the same job. It is not free like distcc, but is very effective and integrates nicely with Visual Studio. It cut my build time for a project at work from 15 minutes to 1 minute 20 seconds. Nice!

kingos

And by the way, GCC 3.4.1

[eddy]

... was just released.

Only available on mirrors, currently.