Should You Pre-Compile Binaries or Roll Your Own?

Jane Walker writes "The completion of pre-compiled packages and maximizing machine performance are two powerful incentives for Windows admins to use Linux and compile an OSS package." TechTarget has an article taking a look at some of the "why" behind rolling your own. What preferences have other Slashdot users developed, and why?

Comments

Re:Gentoo?

[stevey]

The story, and comment, is almost certain to generate a flamefest. So I'll get in early.

I'm a Debian user, and there are three things I know about gentoo:

• The distro is based around compiling from source, which many suggest gives a huge speedup.
• They have some neat tools for working with file merging conflicts in /etc, which sometimes happen when upgrading.
• They make use of "USE" flags which can disable parts of programs you don't want/need.

As for the first I think that compiling from source may well give you a speedup. But when my computer is setting with me at the desktop/ssh session very few processes are running and the network latency / my thinking time are most likely to be the biggest source of delays.

True for heavily loaded servers the compilation might give you a boost but I'd be suprised if it was significant.

Next we have USE flags. These do strike me as an insanely useful thing. But I have one niggling little doubt: I suspect they only work for code that supports it. e.g. project foo has optional support for libbar. If the upstream/original code doesn't have a feature marked as optional I don't imagine the Gentoo people would rework it to strip it out.

So the ability to remove things from the source must be neutered, right?

Finally the merging of configuration files in /etc seems useful. But I wonder if this is the correct approach. My distribution of choice, Debian, already does its utmost to preserve all configuration file changes automagically. I find it hard to understand what Gentoo does differently which makes it better.

Ultimately I guess there are pros and cons to source based distributions depending on your needs. But one thing is true: If you're building from source and making use of modified USE flags and compiler flags then changes are you're the only person in the planet with a particular setup - that means bug reports are hard to manage.

Theres a great deal to be said from having a thousand machines running identical binaries when it comes to tracking down bugs. (Sure diversity is good, especially for security, but there comes a point where maybe people take it a little bit too far).

ObDisclaimer: I'm happy to be educated about Gentoo, but be gentle with me, k?

Re:Gentoo?

[tota]

I'll try to be gentle;)

"The distro is based around compiling from source, which many suggest gives a huge speedup."
It probably does, especially when building for specific architectures
(like C3 or C3-2, etc..)
"... but I'd be suprised if it was significant."
Well, since you compile the compiler as well as everything else.
It does accumulate...
But point taken, in most cases it is not a reason in itself.

USE flags: "I suspect they only work for code that support"
"If the upstream/original code doesn't have a feature marked as optional I don't imagine the Gentoo people would rework it to strip it out."
Actually, that's not true: The Gentoo devs do apply some very useful patches, including some that make it possible to *remove* unused features like you described. Better yet, these patches do make it upstream eventually, albeit at a smaller pace (so the whole community benefits)

Re: configuration files: "Debian, already does its utmost to preserve all configuration file changes automagically. I find it hard to understand what Gentoo does differently which makes it better"
It is not that different, except maybe that Debian does not change as quickly as Gentoo.

"you're the only person in the planet with a particular setup - that means bug reports are hard to manage."
You would be surprised.... Check out the Gentoo ML, they are full of people ready to help, even you try to use that tweaked package XYZ and get into difficulty.

"thousand machines running identical binaries when it comes to tracking down bugs"
Well, if that's what you are looking for, you still can with Gentoo:
(as the parent posted noted) build binary packages on the build machine and deploy to all the others in binary form.

If you want to try it out, why not use UML to boot into it:
http://uml.nagafix.co.uk/
(images and kernels ready to use)

Re:Gentoo?

[kbielefe]

I use gentoo for a few different reasons, none of which have anything to do with ekeing out every last cycle from my machine:

• Nearly universal support for stack smash protection which must be enabled at compile time.
• Incremental updates. I update my system a little bit at a time instead of doing a major upgrade when the distro makes a new major release. I've used gentoo for a long time (5+ years), so I don't know how much of a problem this still is with other distros or not.
• Better dependency handling. No problems with different packages being compiled by different compilers against different development library versions.
• Not strongly married to either KDE or Gnome.
• Multimedia is easier to get working than any other distro I've tried. decss, win32 codecs, etc.
• Can stay bleeding edge where I want to, and extremely stable in other areas.
• Easy to make small changes to source. I occasionally add a minor feature, change a default, fix a bug, or apply a security patch from a mailing list instead of waiting for the next release.
• Easy to distribute to multiple machines. It's a snap to compile and test on one machine, then quickly install my custom binary package on many machines.
• USE flags. Almost everywhere you could use --disable-feature on a manual configure, there is a USE flag for that feature. This is very useful both for enabling features that most distro's wouldn't include and disabling features that most distros include by default. For example, when alsa was still pretty new and usually not enabled by default, the alsa USE flag made migration much easier.

Compiler more important than compiling?

[lawaetf1]

Fine the subject doesn't make complete sense.. BUT... doesn't compiling code with Intel's cc result in significantly better binaries than any flag you can throw at gcc?? From http://www.intel.com/cd/ids/developer/asmo-na/eng/ 219902.htm, MySQL claims 20 percent performance improvement over gcc.

I'm not saying we all have access to icc, but if someone wants to make a binary available, I'm more liable to use that than compiling from source. Call me crazy. And I know someone will.

Re:Gentoo?

[ComputerizedYoga]

Imagine what happens when libpng4 comes out - every program using libpng must be rebuilt to get the new features, so you've only sidestepped the problem.

Only if it breaks api compatibility with the previous version. Otherwise, that's what dynamic linking is for, isn't it?

Having multiple versions of libraries installed isn't a big deal either, unless you're tight on space. And if you're tight on space the idea of compiling large applications probably isn't something you'd appreciate anyway!

Right on ... openoffice 2 spooled out to fill 8 gigs of free space when I tried to compile it with "nostrip" and with debug symbols. The linux kernel source, unpacked, weighs in around 230 megs, where the binary .deb for 2.6.15 is somewhere closer to 16 megs. Sure, you don't get the ultra-lean heavily-tuned kernel that you'd get building from source, but it certainly works.

Personally, I think the big benefits of running gentoo over debian are things like ... the runlevel abstraction system, the ability to turn on and off features with more freedom, and the simple slickness of portage. Oh, and a very helpful and newbie-friendly community.

On the other hand, I'd say on a p4 3ghz desktop system with a very large software set, I'm probably averaging 2-3 hours a week of compiling for various updates, my debian and fc4 boxes spend more like 5-10 minutes a week downloading and unpacking them. But, if you're halfway decent at scheduling and don't have constant insanely-high demand everywhere, I'd say that update time isn't even a particularly big deal (after all, it's mostly non-interactive ... fire it, forget it, come back when it's done).

Re:Don't waste your time.

[Spoke]

Most packages I have seen on Linux distros are compiled with -O2 or -O3. It is highly unlikely that the various other switches provided by GCC will provide anything significant.

Besides the obvious -O parameters to gcc, specifying the arch of the platform (-march=i386 for example) can sometimes have a decent effect on performance. A lot of distributions compile for the most common platform which usually means specifying -march=i386 -mtune=i686. That gets you binaries that run on any i386 or better, while tuning the code for i686 machines. If you're running an older processor or something like a VIA or AMD cpu, often compiling with -march=c3-2 or -march=athlon64 or whatever specific CPU you're running can provide a noticable benefit, especially on newer versions of gcc.

FreeBSD

[vga_init]

For years I lived in the world of FreeBSD.

Not only had I built every package from source (using ports), I also took the trouble to rebuild the base system and kernel with a custom configuration and options.

The benefits to some of this were obvious; the FreeBSD GENERIC kernel at the time seemed (to my eyes) to suffer a massive performance loss from its configuration. Anyone running FreeBSD *must* build at least a custom kernel, even if they use the binary distribution of everything else.

It was a lot of effort. What did I get out of it? It was by the end one of the speediest systems I had ever used since the days of DOS. Most programs loaded faster than their binary equivalents (on older machines the differences were more glaringly obvious, such as the time it took to initialize X).

One time I clocked my old machine, running a custom built FreeBSD installation, against the other computers in the house from power-on to a full desktop (after login).

On my machine, the entire affair (BIOS, bootloader, bootstrapping, system loading, X, login, desktop environment (WindowMaker in this case)) cost a mere 45 seconds. My father's machine, which was in all respects a faster computer, loaded Windows 2000 in the course of perhaps two minutes. Also, I stopped timing after the desktop came up, but Windows does continue to load and fidget about for a good while after that. The extra time taken for it to settle down would have cost it another minute, but only because of all the crap my dad had set to load, which I don't blame Windows for.

The kitchen computer also ran Windows 2000, but had a slimmer configuration, so it loaded shortly over a minute. FreeBSD, however, still beat them both badly.

In light of my own experience, compiling from source can get you some rather wonderful results. However, I noticed that not all systems were created equal. While FreeBSD GENERIC was as slow as molasses, I find in linux that the binary kernels that come with my distributions seem to load and operate just as fast, if not faster than my custom build of FreeBSD. In linux I have used only binary packages, and the system overall "feels" just as fast, though some operations are a little slower (like loading emacs ;)).

I appreciate the arguments presented by both camps, but I feel the need to point out that some are too quick to downplay the possible performance gains offered by custom builds, because they certainly exist. Sometimes they can be noticeably significant.