Reduce C/C++ Compile Time With distcc

An anonymous reader writes "Some people prefer the convenience of pre-compiled binaries in the form of RPMs or other such installer methods. But this can be a false economy, especially with programs that are used frequently: precompiled binaries will never run as quickly as those compiled with the right optimizations for your own machine. If you use a distributed compiler, you get the best of both worlds: fast compile and faster apps. This article shows you the benifits of using distcc, a distributed C compiler based on gcc, that gives you significant productivity gains."

Comments

nc: a better tool for distributed builds

[Eric Smith]

Instead of distcc, I use nc by Steven Ellis. It seems to be more flexible, though I'm not an expert on distcc, so I'm not certain.

I think nc can be used like distcc by redefining CC="nc gcc". However, more commonly it is done by putting $(NC) at the beginning of the build rules. Then you can use nc for any build rules, not just C compiles.

In addition to use with make, nc works well with SCons.

distcc and rendezvous

[brucehoult]

We use the distcc that Apple distributes with XCode even though we dont' use XCode itself. It really helps to get a few dual-CPU G5's working!

The cool thing about Apple's version is that by default it uses Rendezvous to determine which machines are available to distribute work to.

Excellent for software development

[inflex]

I came across distcc by chance about 4 months ago, and I must say, it has utterly improved things around here.

We reguarly develop/compile/debug a moderate-small sized software package, typically taking about 1 minute per compile. Now, while 1 minute doesn't sound like a long time, it starts adding up when you find yourself recompiling 100+ times a day.

With the inclusion of distcc into the whole situation, we're able to reduce that 1 minute compile down to a little less than 20 seconds; highly appreciated (although now we have less excuses to go get a coffee :-( ).

Distcc is a great package which can be extremely useful.

PLD.

Re:Productivity gains if and only if....

[evilviper]

I can't think of a single thing I do that's code bound.

The obvious and most popular answer is encoding video. I think a great many people do a lot of this. Since no processor is fast enough to encode DVD-res video at 16X, it isn't bound by IO speeds either. I can start videos encoding in far less time than it takes to complete the process as well. Pure CPU number-crunching.

Other applications are any form of crypto. Reduce the time you have to wait for PGP to encrypt. Reduce the delay on your SSH sessions.

Then there are databases. Sure, they're often IO bound, but it is commonly a CPU limitation.

Also any heavy-load service. If apache is serving lots of threads, especially PHP/Perl compiled pages, you are going to be maxing out your CPU.

Then there are the programs that are just bloated. Mozilla/Firefox is still quite slow, and I can open pages far, far faster than they can be rendered. Anything that makes it even 1% faster is very welcome, as those savings eventually add-up to large ammounts of time.

If you really never use any of those, hooray for you, but most people certainly do.

If you need perf, fine. But you lose testing

[Flexagon]

But this can be a false economy...

Every time something that is distributed in binary is rebuilt from source for local use, by definition it's to change some assumption that was inherent in the testing of the original binary (or else the binary distribution would suffice). And with that, some non-0 confidence that was built into the binary release by that testing is wiped out and must be recovered by local analysis and testing (i.e., time and effort) or reduced expectations. Otherwise, it's running on blind faith. This is particularly true with programs that are used frequently, i.e., one expects to depend on them repeatedly. So in my mind, "the best of both worlds" is more meaningful if it refers to fast and reliable apps. I don't care how fast the compiler is if I can't trust the results anymore. That is a different economy equation, and completely justifes the "convenience" of pre-compiled binaries in many applications.

Re:I have only one computer

[boots@work]

Absolutely, I use it at home all the time. It's great for sofa computing: sit on the sofa with a modest laptop, and send your compile jobs across a wireless network to a faster machine in the study.

If you use the LZO compression option then it's quite useful even on 5Mbps wireless.

You can also tell distcc to run as many jobs remotely as possible to keep the laptop from scorching your lap.

It's really nice to be able to build the kernel from source in a reasonable time on a 800MHz machine.