Build From Source vs. Packages?

mod_critical asks: "I am a student at the University of Minnesota and I work with a professor performing research and managing more than ten Linux based servers. When it comes to installing services on these machines I am a die-hard build-from-source fanatic, while the professor I work with prefers to install and maintain everything from packages. I want to know what Slashdot readers tend to think is the best way to do things. How you feel about the ease and simplicity of installing and maintaining packaged programs versus the optimization and control that can be achieved by building from source? What are your experiences?"

Comments

Personally

[jwthompson2]

I do a bit of both. I predominantly install items from packages, when available, for testing and review of something new that I am interested in. Once I establish whether what I have been playing with may be useful for some particular purpose I will research the source build options. If there are specific optimizations that can be made for my system's hardware or pre-installed software I will then look at installing from source in order to leverage those optimizations, but if there is no advantage to compiling the source due to lack of any worthy optimizations then I will install from packages any time I want that software.

That is my way of handling things, do what fits your needs best, that's why we have this option.

Re:Personally

[allyourbasebelongtou]

In short: I have to agree--I do a bit of both, too.

The main thing I encounter that keeps me from using them all the time is the need for specific add-ons that are available as part of packages but are available when rolling-my-own.

As an aside, there are certain bits that I just prefer to compile myself for any number of reasons

That said, there are other bits of software that are pretty generic items that the packages make *trivially* easy to work with, and where compiling those same things from scratch--particularly on older hardware--makes you get a bit long-in-the-tooth waiting for the compile to return.

To me, this is truly one of the ultimate beauties of open source: you're not stuck with pre-built, but you can leverage it when it makes sense.

Re:Personally

[Aneurysm9]

You don't have to wait days to get a working Gentoo system. With the GRP CDs you can have a working system up and running in a few hours. It's still going to take more time than Fedora or SuSE, but it will be optimized more for your platform with the option of recompiling for further optimization. That's how I setup Gentoo on my laptop as it's hideously slow. Over time it's had almost everything recompiled a piece at a time, but I didn't have to wait for it to do everything from glibc up at once.

Re:Personally

[jobsagoodun]

BUT

The really great thing is how well it wears. I've a RH8, RH9 installation that have lots of other bits & bobs installed, mainly from tgz's I've pulled down & built. Its an arseabout, and both boxes are cluttered with stuff - and as soon as you go off piste with an installed package, you're on your own.

OTOH I also have a couple of gentoo installations, and for nearly everything I want, I can just 'emerge xyz' and presto, its there. It was a pain getting it installed, but now its there it is really, really good. Also upgrading it was piss easy too.

If only I could get portage/emerge for redhat...

Whatever get the job done

[untermensch]

If you are working for someone else, maintaining servers that are intended for peforming specific tasks, then I think the best solution is to do whatever is most efficient at performing those tasks. If you really don't need the peformance gains brought by compiling from source (and you probably don't) and it's going to take you a long time to do the compiling, time that could be better spend actually doing the research, then it's not worth your effort. If however the compiling doesn't affect the user's ability to be productive and that is what you as sysadmin are most comfortable with, then it seems reasonable that you should be able to maintain the boxes however you like.

OSX

[artlu]

I used to be a huge debian fan because of apt-get and the direct install of packages, but I have migrated to OSX and find myself needing to build packages from scratch to work correctly. However, I will never hesitate to use Fink as much as possible. I think for 90% of what gets installed the packages should be fine, but if you know that there are certain optimizations that you can implement, why not build from scratch?

Re:Who are these people?

[Anonymous Coward]

Speaking of RedHat doing something weird... RedHat managed to _rename_ p_pptr to parent in task_struct in the kernel. How did they manage to get away with something like that? If there are custom kernel modules that happen to want to use p_pptr, then everything breaks!

Ports or Portage

[iiioxx]

As a FreeBSD user, I build almost everything from source using ports. I never install from packages. My reasons for this are many and varied, but basically, I prefer to build software myself, with the precise options I need. When you use packages, you are at the mercy of the packager and their preference for options and optimizations. Several years ago when I used Linux, I often encountered problems of pre-built packages lacking a particular build option, and sometimes installing to odd places, or other strangeness.

And once you've started using packages and package management, it gets harder to introduce source-built software into the same environment without screwing up your dependency databases, or worse - breaking things. So if a package lacks a required option, you really have to build your own package with the option included in order to keep things orderly. That's a lot more work than just installing from source.

I'm not a Linux user anymore (several reasons) but if I were I to go back to Linux, I would use Gentoo, specifically for its Portage system.

So, in my opinion, building from source may be a little more time and CPU consuming, but it is the better option for a controlled, tailored environment.

Re:Gentoo is something of a middle ground.

[whoever57]

I have one Gentoo machine that is my "Compile Server" On this, I build binary packages, which go into the portage tree. All other Gentoo machines on the network then sync against the compile server (instead of using "emerge sync"), and thus also get the binary packages.

Then, on the other machines, I install from the binaries.

This allows me to test the installs first, resolve any problems, etc.

Furthermore, to speed up the process, several machines run DISTCC and are used as clients of the compile server.

Re:One word for you...

[Stonent1]

But why would you ever use Gentoo in a production environment?

Security updates w/o waiting for them the be packaged?

Re:This is BSD vs Linux argument

[idontgno]

(And try installing the native Java from BSD ports - several hours of pure joy!)

I'm not sure whether to mod you -2 BSDTroll or +1 BSDFunny. However, I'll comment instead. (Commented earlier downthread, so it's already a foregone decision, but what the hey, you only offtopic once.)

The only joy I get watching compiler messages scroll by is laughing my butt off watching all the warnings. Don't these people use lint?

And that's funny only if I'm already in a good mood. Otherwise, I hate having to actually watch the unavoidable visible indicators of the quality of the software I'm about to start using. Just like most people don't like watching sausage being made...from live pigs...

Yeah, I know, if I know so much, why don't I fix it? Because I didn't sign up to indentured servitude, I just want to use the damn software. I realize that violates the canon of Open Source ethics in the minds of the extremists, but I have a job to do and it's not fixing your damn object cast mismatches.

OK, ok, cooling down now.

Thank you, in all sincerity, to the authors of those software packages. Please forgive me if watching 2423 warnings per compile cycle makes me a little crazy.

And that's why it was the best summer ever!

Not always the case

[Anonymous Coward]

Sometimes the exact opposite is true, especially in terms of "community support". For instance, mod_perl, which for some reason Red Hat decided to ship a very early version. The typical response on the mailing lists for mod_perl or any other alpha/beta package RH included usually goes "try it from source, then email us" (that's after someone submits a reasonably complete bug report).

Let's not forget the GCC fiasco and probably dozens of other examples where RH decided to "lead the pack" in terms of version numbers but not stability.

Of course, then there's Debian woody, living in circa-2001 land.

On building from source

[Kourino]

Optimization? Control?

Man, what is this, Gentoo?

Any sane distributor these days builds binary package with reasonable optimizations that won't break across architecture submodels, and occasionally releases binaries targetting submodels (e.g. PentiumPro-specific packages). On many machines, for many workloads, however, the model-specific optimizations just aren't that helpful. Obvious exceptions are floating point math on most platforms (especially x86, where x87 math code is a dog and should be replaced with SSE code if possible) and - I'm told - really slow hardware. (I'll be able to test that once I get these Indys running GNU/Linux.) In my experience, Debian hasn't really felt any slower than my LFS systems for personal use.

So, I'll say this: if you have enough time to build everything you're using, do some careful speed comparisons between your self-built packages and the vendor's binaries. If there's really a significant speed increase, and you need that increase, source is the only way to go for the packages that need the speed increase. Otherwise, it's probably not worth your time.

Unless whatever you're doing is extremely security critical, you can probably deal with the fact that server app foo has features bar and baz installed that you won't use. If you can't, you're probably auditing the source of everything you use anyway, and that doesn't sound like the case, so "control" probably isn't a real issue here either. Control can be found in config files as well as in the configure script.

People say, "but package dependencies suck!" Well, yes, rpm (the program) isn't built to deal with dependencies that gracefully. If it annoys you that much, go install apt-rpm or something, or even Debian (gods forbid). Package management isn't rocket science.

Re:Supprt: Naa, that's not true at all.

[cbreaker]

No way.

Usually when one builds from Source, they install it to wherever the original developer has it set to by default. Unless you did some heavy patching, the software will very likely be more "true" to the original software then many packages.

RPM's for distributions such as RedHat or Fedors often have to move configuration files all over the place to mesh with the OS properly.

You're more likely to be able to sit down at a strange Linux box and troubleshoot whatever program when it's compiled from source tarballs versus an RPM. Unless of course, you know the RPM, or the RPM doesn't do anything funky.

Considering the stuff is Open Source, and chances are the programs are not under a paid-for support contract, it's pretty safe to say that BOTH methods would have to be supported "In House." And if not, your support contract could very well support the source compiled versions anyways.

I choose the Gentoo way. Everything is compiled from source; it's just nice and automated. Almost never have I run into something where the program had to be modified to fit the distribution.

Source-based, binary-packaged Gentoo

[strider( corinth )]

My arguments on why to use a source-based distribution have been covered in other posts, so I won't repeat them here. I think Gentoo provides a solution that will satisfy both you and your professor: you can use a source-based, custom-built binary distribution.

As you probably know, Gentoo is a source-based distribution, but it also allows binary packages. Many (such as Mozilla Firefox) are distributed by Gentoo as source and binary; you can choose to install either. The ability to build a binary package from a source .ebuild (the file that describes to the system where to find the source and how to build it) requires adding only a single flag to the package compile command, ebuild.

Additionally, since (if I read you correctly) you're probably using similar hardware for each of your machines, it would be trivial to set up a compile box which would produce binary packages for your other boxen. Packages compiled for your architecture would be faster than most binary-only distributions (many are still compiled for the i386 architecture), and writing a new ebuild is trivial compared to writing a new spec file. (Trust me; I spent a quarter writing a paper on the topic while I was in school, not to mention having had to do it myself in the Real World.)

Finally, Gentoo integrates and tests its packages. Ebuilds come with Gentoo-specific patches, so you don't have to spend the time to make each source package work with the rest. This is probably one reason why your professor likes binary distributions: they all work together, and enough people rely on them that if something breaks, it gets fixed. A package-based Gentoo distribution would allow you to leverage that, while keeping your machines unified in their versioning (as much as you want them to be, at least) and also provide all of the benefits of a source-based distribution.