Should You Pre-Compile Binaries or Roll Your Own?

Jane Walker writes "The completion of pre-compiled packages and maximizing machine performance are two powerful incentives for Windows admins to use Linux and compile an OSS package." TechTarget has an article taking a look at some of the "why" behind rolling your own. What preferences have other Slashdot users developed, and why?

Comments

Gentoo?

[corychristison]

I feel that Gentoo Linux offers the best of both worlds, with their ebuilds. :-)

Re:Gentoo?

[stevey]

The story, and comment, is almost certain to generate a flamefest. So I'll get in early.

I'm a Debian user, and there are three things I know about gentoo:

• The distro is based around compiling from source, which many suggest gives a huge speedup.
• They have some neat tools for working with file merging conflicts in /etc, which sometimes happen when upgrading.
• They make use of "USE" flags which can disable parts of programs you don't want/need.

As for the first I think that compiling from source may well give you a speedup. But when my computer is setting with me at the desktop/ssh session very few processes are running and the network latency / my thinking time are most likely to be the biggest source of delays.

True for heavily loaded servers the compilation might give you a boost but I'd be suprised if it was significant.

Next we have USE flags. These do strike me as an insanely useful thing. But I have one niggling little doubt: I suspect they only work for code that supports it. e.g. project foo has optional support for libbar. If the upstream/original code doesn't have a feature marked as optional I don't imagine the Gentoo people would rework it to strip it out.

So the ability to remove things from the source must be neutered, right?

Finally the merging of configuration files in /etc seems useful. But I wonder if this is the correct approach. My distribution of choice, Debian, already does its utmost to preserve all configuration file changes automagically. I find it hard to understand what Gentoo does differently which makes it better.

Ultimately I guess there are pros and cons to source based distributions depending on your needs. But one thing is true: If you're building from source and making use of modified USE flags and compiler flags then changes are you're the only person in the planet with a particular setup - that means bug reports are hard to manage.

Theres a great deal to be said from having a thousand machines running identical binaries when it comes to tracking down bugs. (Sure diversity is good, especially for security, but there comes a point where maybe people take it a little bit too far).

ObDisclaimer: I'm happy to be educated about Gentoo, but be gentle with me, k?

Re:Gentoo?

[EnigmaticSource]

Well, as a Gentoo user, I'll tell you my personal reasons for using portage (speed isn't one of them).

1.) Maintainability: I don't have to fiddle with 30+ binary dependencies when I upgrade a package, nor do I worry about having multiple library versions within the same major release

2.) Simplicity: Well, it's not particularly simple (in fact, until 2006.1 it was a nightmare) to setup, but once everything is in line I simply don't have to worry about be various `gotchas` of any given package, it's all been abstracted away

3.) USE Flags: An extension of the above, USE is like a homogeneous ./configure, no more silly --without-some-foo flags, or include paths that I forgot about 30 seconds after I installed a library. It's not so much about making things optional (at least in the real world) but more about keeping things simple (I specify all of my USE flags at install time, and simply add them to my list when new ones are created)

4.) Lack of Binary Packages: As an old slackware user, I got used to not finding package `foo` as a .tbz and having to deal with RPMs that are/were broken and took more time to install properly than to compile. By using a source based distribution, if I have a one-off or patched library I don't have to worry about will feature X work or why Sodipodi crashes, because whatever version I have is (within reason) now the native version to the application

Hope that helps

Re:Gentoo?

[RedWizzard]

As for the first I think that compiling from source may well give you a speedup.

Forget performance, it's a red herring. If you're considering looking at Gentoo for performance reasons you'll probably be disapointed. Increased performance is a minor side effect at best and more likely to be completely undetectable. Gentoo is about configurability and control, not performance.

Next we have USE flags. These do strike me as an insanely useful thing. But I have one niggling little doubt: I suspect they only work for code that supports it. e.g. project foo has optional support for libbar. If the upstream/original code doesn't have a feature marked as optional I don't imagine the Gentoo people would rework it to strip it out.

So the ability to remove things from the source must be neutered, right?

You are right - generally USE flags rely on the upstream source having optional support for various features. So in theory it might be that very little can be removed from a given package. But in practice most OSS software is highly configurable at the source level, particularly if it is portable rather than Linux specific. The number of USE flags recognised by each package is highly variable. For example, mplayer recognises over 70 USE flags, while Firefox recognises 11.

Finally the merging of configuration files in /etc seems useful. But I wonder if this is the correct approach. My distribution of choice, Debian, already does its utmost to preserve all configuration file changes automagically. I find it hard to understand what Gentoo does differently which makes it better.

It puts control back into the hands of the user (which is basically the fundamental point of Gentoo). Consider a package that is adding a new config option. The Debian way will be to ignore that option, set a default for the option, and/or at best log a message that is easily missed (I don't know Debian so any inaccuracies are unintentional). Gentoo provides a complete default config that can be compared to the existing config. It allows the user to decide what changes should be made to the config. This is particularly important for packages which have extremely complex configurations (e.g. apache). There is often no way to automatically translate configuration between versions. The user really needs to look at it.

Re:Gentoo?

[V. Mole]

The Debian way will be to ignore that option, set a default for the option, and/or at best log a message that is easily missed (I don't know Debian so any inaccuracies are unintentional). Gentoo provides a complete default config that can be compared to the existing config.

Nope, that's not the "Debian way" (at least as I, long-time Debian user/developer, see it). Debian provides a default config file (or files). When the package is upgraded, if the distributed config file is changed (new option, or new value for old option), then one of two things happens:

1. If the user has NOT changed their local version, just upgrade to the new distributed default. The assumption is if they were happy with the old default, they'll be happy with the new ones. This covers the vast majority of cases.
2. If the user has changed their local version, offer them the chance to look at the diff, and then either overwrite, don't overwrite, or shell it and deal with by hand. If they choose not to overwrite, then the distributed default is left alongside the real config file for later perusal/integration.

While there are a few obscure corners in the implementation, and individual developers can make mistakes, it mostly works pretty well. The net effect sounds pretty much like your description of the Gentoo way: don't overwrite local changes, and give them something to diff against.

Eh?

[kassemi]

Was I the only one who found that this article didn't really shed too much light on whether or not you should compile your software from source?

By the way, I know the benefits of compiling from source, but how this made slashdot, I don't know.

TEN PERCENT!

[LeonGeeste]

In my field, ten percent is everything. If I could increase performance by ten percent, I'd get a 100% bonus for the year. My servers need to handle so much data and are always backlogged (and adding more on is exensive!) Don't trivialize ten percent.

deployment?

[TheCarp]

Do these guys even know what deplpoyment means?

They are not only wrong in their conclusion, but the article barely scratches the surface of the question.

Put simply, compiling software on your own is fine for a one off, or your desktop, or your hobby machine.ou.. or if you either a) need the latest wizbang features (and maybe can put up with the latest gobang bugs) or b) need really specific version control or c) can't find a precompiled package with the right ompile time options set.

Other than that, you should pretty much always use pre-built.

Sure, you can build your entire system from scratch if you like, libc on up. Why? The performance increase will be so minor that you will have to run benchmarks if you even want to be able to tell there was a change. You will then have to recompile everything you ever decide to update.

This strategy is a nightmare as the systems get more diverse and complex.

it also has nothing to do with deployment. Deployment is what you do after you have decided what you want and made it work once. Deployment is when you go and put it on some number of machines to do real work.

I would love to see them talk more about the differences in deployment. With precompiled packages from the os vendor, ala debian or redhat, its easy. You use apt-get or rpm and off you go. Maybe you even have a redhat network satalite or a local apt repository to give yourself more control. Then you can easily inject local packages or control the stream (no, sorry, I am NOT ready to upgrade to the new release)

but should you compile "most of the time"? hell no!

It is, in fact, the vast minority of software where you really need the latest features and or special compile options. Its the vast minority of the time where the performance increase will even be perceptable.

Why waste the time and cpu cycles? Takes over a day to get a full gentoo desktop going, and for what? I can have ubuntu installed and ready to go with a full desktop in maybe 2 hours.

Lets take the common scenario.... new openssl remote root exploit comes out. The email you read just basically said, in no uncertain terms, that half your network is now just waiting to be rooted by the next script kiddie who notices. Thats lets say... 50 machines.

Now your job is to deploy a new openssl to all these machines.

You could notice that the vulnerability came out in such a time frame that they allowed the OS vendors like debian to release fixes (often happens, if not they are usually out within a very reasonable time frame)... so you hit apt-get update && apt-get upgrade

or maybe you just promote the packgae into your repository, and let the automated updates deploy it for you. You go home, have a coffee, and be ready to log in remotly if anything goes tits up.

Now if you are hand compiling what do you do? Compile, test. And then um.... ahh... scp the dir to 49 machines and ssh in and do a make install on each?

How is this better than using a package manager again? Now you ahve dirs sitting around, you have to hope that your compile box is sufficiently similar to the other boxes that you didn't just add a library requirement (say because configure found that yoru dev box has libfuckmyshit.so installed and this neat new bits of openssl can make use of it)

How about when a box crashes and burns and you now need to take your lovingly handcrafted box, and rebuild it, and put all that lovingly compiles software back on it.

Fuck all that... give me a good binary distro any day of the week. I will hand compile when I HAVE to... and not before.

-Steve