Slashdot Mirror


Should You Pre-Compile Binaries or Roll Your Own?

Jane Walker writes "The completion of pre-compiled packages and maximizing machine performance are two powerful incentives for Windows admins to use Linux and compile an OSS package." TechTarget has an article taking a look at some of the "why" behind rolling your own. What preferences have other Slashdot users developed, and why?

17 of 301 comments (clear)

  1. Gentoo? by corychristison · · Score: 5, Informative

    I feel that Gentoo Linux offers the best of both worlds, with their ebuilds. :-)

    1. Re:Gentoo? by MightyMartian · · Score: 4, Informative

      Actually, I'm becoming a very big fan of FreeBSD. Their ports and packing systems give me the best of both worlds. Where I just need a relatively generic version of a piece of software, I just install the binary, but where I need a few more options (common with stuff like Samba and FreeRadius) I can compile.

      --
      The world's burning. Moped Jesus spotted on I50. Details at 11.
    2. Re:Gentoo? by EnigmaticSource · · Score: 5, Informative

      Well, as a Gentoo user, I'll tell you my personal reasons for using portage (speed isn't one of them).


      1.) Maintainability: I don't have to fiddle with 30+ binary dependencies when I upgrade a package, nor do I worry about having multiple library versions within the same major release


      2.) Simplicity: Well, it's not particularly simple (in fact, until 2006.1 it was a nightmare) to setup, but once everything is in line I simply don't have to worry about be various `gotchas` of any given package, it's all been abstracted away


      3.) USE Flags: An extension of the above, USE is like a homogeneous ./configure, no more silly --without-some-foo flags, or include paths that I forgot about 30 seconds after I installed a library. It's not so much about making things optional (at least in the real world) but more about keeping things simple (I specify all of my USE flags at install time, and simply add them to my list when new ones are created)


      4.) Lack of Binary Packages: As an old slackware user, I got used to not finding package `foo` as a .tbz and having to deal with RPMs that are/were broken and took more time to install properly than to compile. By using a source based distribution, if I have a one-off or patched library I don't have to worry about will feature X work or why Sodipodi crashes, because whatever version I have is (within reason) now the native version to the application


      Hope that helps

      --
      The Geek in Black
      I know my BCD's (when I'm Sober)
    3. Re:Gentoo? by RedWizzard · · Score: 5, Informative
      As for the first I think that compiling from source may well give you a speedup.
      Forget performance, it's a red herring. If you're considering looking at Gentoo for performance reasons you'll probably be disapointed. Increased performance is a minor side effect at best and more likely to be completely undetectable. Gentoo is about configurability and control, not performance.
      Next we have USE flags. These do strike me as an insanely useful thing. But I have one niggling little doubt: I suspect they only work for code that supports it. e.g. project foo has optional support for libbar. If the upstream/original code doesn't have a feature marked as optional I don't imagine the Gentoo people would rework it to strip it out.

      So the ability to remove things from the source must be neutered, right?

      You are right - generally USE flags rely on the upstream source having optional support for various features. So in theory it might be that very little can be removed from a given package. But in practice most OSS software is highly configurable at the source level, particularly if it is portable rather than Linux specific. The number of USE flags recognised by each package is highly variable. For example, mplayer recognises over 70 USE flags, while Firefox recognises 11.
      Finally the merging of configuration files in /etc seems useful. But I wonder if this is the correct approach. My distribution of choice, Debian, already does its utmost to preserve all configuration file changes automagically. I find it hard to understand what Gentoo does differently which makes it better.
      It puts control back into the hands of the user (which is basically the fundamental point of Gentoo). Consider a package that is adding a new config option. The Debian way will be to ignore that option, set a default for the option, and/or at best log a message that is easily missed (I don't know Debian so any inaccuracies are unintentional). Gentoo provides a complete default config that can be compared to the existing config. It allows the user to decide what changes should be made to the config. This is particularly important for packages which have extremely complex configurations (e.g. apache). There is often no way to automatically translate configuration between versions. The user really needs to look at it.
    4. Re:Gentoo? by Theatetus · · Score: 3, Informative
      Next we have USE flags. These do strike me as an insanely useful thing. But I have one niggling little doubt: I suspect they only work for code that supports it. e.g. project foo has optional support for libbar. If the upstream/original code doesn't have a feature marked as optional I don't imagine the Gentoo people would rework it to strip it out.

      Yes and no. It's really more dependent on the Gentoo maintainer than on the upstream. Most "big" projects also include a patchset (generally small stuff like where config files go; sometimes big changes to the codebase). These will generally have fairly rich USE flags. And it's not simply disabling things; in some cases it's adding whole subsystems (like SASL for sendmail, or the postgres backend for named). But anyways, some maintainers will add a lot of USE flags to their ebuild and others won't.

      Finally the merging of configuration files in /etc seems useful. But I wonder if this is the correct approach. My distribution of choice, Debian, already does its utmost to preserve all configuration file changes automagically. I find it hard to understand what Gentoo does differently which makes it better.

      Part of it comes from the fact that /etc/foo.conf might be altered by both libfoo and gtk-foo. The utility dispatch-conf diffs the two packages' foo.confs to let you merge their conf files. I think the best thing Gentoo does in /etc is the utility rc-update. It's the most sane init/runlevel interface I know.

      --
      All's true that is mistrusted
    5. Re:Gentoo? by gorre · · Score: 4, Informative
      The problem is the USE flags are global.
      Not true, you can put USE flags for specific packages in the /etc/portage/package.use file. You might want to check out the relevant section of the Gentoo handbook for more information.
      --
      "Madness is something rare in individuals - but in groups, parties, peoples, ages it is the rule." -- Nietzsche
    6. Re:Gentoo? by V.+Mole · · Score: 5, Informative

      The Debian way will be to ignore that option, set a default for the option, and/or at best log a message that is easily missed (I don't know Debian so any inaccuracies are unintentional). Gentoo provides a complete default config that can be compared to the existing config.

      Nope, that's not the "Debian way" (at least as I, long-time Debian user/developer, see it). Debian provides a default config file (or files). When the package is upgraded, if the distributed config file is changed (new option, or new value for old option), then one of two things happens:

      1. If the user has NOT changed their local version, just upgrade to the new distributed default. The assumption is if they were happy with the old default, they'll be happy with the new ones. This covers the vast majority of cases.
      2. If the user has changed their local version, offer them the chance to look at the diff, and then either overwrite, don't overwrite, or shell it and deal with by hand. If they choose not to overwrite, then the distributed default is left alongside the real config file for later perusal/integration.
      While there are a few obscure corners in the implementation, and individual developers can make mistakes, it mostly works pretty well. The net effect sounds pretty much like your description of the Gentoo way: don't overwrite local changes, and give them something to diff against.
    7. Re:Gentoo? by greginnj · · Score: 2, Informative

      Lots of people have mentioned /etc/portage/package.use already ... if you've had enough of config files, you should check out the 'porthole' app, which is a GUI frontend to emerge. It makes it very easy for you to see all and only those USE flags which are appropriate to each app, and you can set them for each emerge distinctly. I'm also trying 'kuroo', which is a similar KDE-based app, but I'm liking porthole better.

      --
      Read the best of all of Slash: seenonslash.com
    8. Re:Gentoo? by rs79 · · Score: 1, Informative

      "Obviously, not everyone agrees with you..."

      I must be dense. I have no idea what point is being made here other than two closely related flavours of unix have some minor differences.

      In related news, water is wet, air is free and the sun will come up tomorrow.

      --
      Need Mercedes parts ?
    9. Re:Gentoo? by Anonymous Coward · · Score: 1, Informative

      Did you read the link? It's for Gentoo/FreeBSD, a 'Gentooized' FreeBSD with Portage. Many people find ports archaic.

  2. Other benefits by Rich0 · · Score: 4, Informative

    I have a feeling that this will turn into a general debian vs gentoo style debate, and we all have our preferences.

    The big benefits of precompiling are that you don't need to support 1500 different sets of libraries in your development environments, and that the package will generally work right with minimum fuss.

    The big benefits of source-based distros are the ability to tailor packages to each install (ie the ability to compile certain features in or out), to choose optimizations on each package (do you want -Os, -O2, -03, or are you really daring -> -ffast-math?).

    There are some things that cut both ways - often a given package can be compiled using one or more different dependencies and if you want this flexibility then source-based might work better. On the other hand, it also means that if you have 500 different users of your distor you have 495 different configurations and bugs that are hard to reproduce.

    As for me - I like source-based. However, if I had to build a linux email/browser box for a relative I'd probably use Debian stale...er...stable. The right tool for the right job.

  3. Don't waste your time. by jrockway · · Score: 4, Informative

    Recompiling software gets you almost nothing. Maybe 10% more performance, at the very maximum.

    There are special cases like when you want to use dynamic libraries instead of static (to save memory), or when there's a major architecture change (PPC -> x86 for Apple). In those cases you'll gain something.

    Another case is rewriting your program to use CPU-specific instructions, like Altivec or SSE3. That, in certain circumstances, will speed up your program.

    But if you're compiling OO.org or Mozilla because you think your 686 version will be 100% faster than the 386 version, you're wrong.

    --
    My other car is first.
  4. Re:"Absolute best performance" fallacy by Just+Some+Guy · · Score: 4, Informative
    I am personally not aware of a single mainstream linux app that does 5% better when optimized for P4 as opposed to generic 386

    Running "openssl speed" compiled with "-O3 -march=pentium4" gave about 3 times the performance of "-O" on my server. Being able to handle 3 times the number of SSL connections was certainly worth the 10 seconds required to put correct values in Gentoo's /etc/make.conf.

    --
    Dewey, what part of this looks like authorities should be involved?
  5. Gentoo vs. Debian by Noksagt · · Score: 3, Informative
    I usually find the most interesting criticisms of Gentoo (and the most insightful questions about it) come from Debian users. Yours are certainly no less relevant than others I've read.

    I've used both Debian and Gentoo. I am now (mostly) using Gentoo and not Debian. I hope you might find my perspective helpful. (But it should also be stated I also use ports on FreeBSD, and I have come to the conclusion that source-based distros are easier for me to use.)

    As for the first I think that compiling from source may well give you a speedup. But when my computer is setting with me at the desktop/ssh session very few processes are running and the network latency / my thinking time are most likely to be the biggest source of delays.

    How I'd love to have so much dead CPU time! If your computer's not doing anything for you, why bother having one? Truth be told, one can reap performance gains in more definitive ways than trying to have your compiler make different binaries. As you indicated, running few processes helps. As can swapping in a custom kernel and/or using a faster filesystem (both of which you can do on Deb fairly easily).

    True for heavily loaded servers the compilation might give you a boost but I'd be suprised if it was significant.

    I don't usually see a huge advantage, but it does depend on the app. For desktop users, app launchtime is often significant. I do think using '-Os' to make smaller binaries (which get into memory faster) does usually create a noticeable benefit. And for workstation/server apps, every few percent for "faster" apps could be helpful to some people (but I agree that it is typically only a few percent). But just leaving apps open is often "good enough" for load time & perhaps there aren't many who really need the extra few percent.

    These do strike me as an insanely useful thing.

    Yes. Absolutely. Particularly when you have relatively common needs across all apps. Perhaps you want to run an X-less server? Or perhaps you want to have apps with only KDE/QT or only Gnome/GTK+ or what not. USE comes to the rescue.

    But I have one niggling little doubt: I suspect they only work for code that supports it. e.g. project foo has optional support for libbar. If the upstream/original code doesn't have a feature marked as optional I don't imagine the Gentoo people would rework it to strip it out.

    You are typically correct. But the thing is that foo more often than not will have optional support for some feature. But some gentoo ebuilds do, indeed, have USE flags that aren't just ./configure flags for some applications. For example, you can install xpdf with the 'nodrm' use flag, which applies a patch to cause xpdf to ignore drm restrictions. Indeed, for making custom ebuilds, USE flags prove to be quite useful: you can use them to test multiple patches without the need to apply a given patch to all installations & can easily check which features a certain app has (by checking which flags it was emerged with).

    Finally the merging of configuration files in /etc seems useful. But I wonder if this is the correct approach. My distribution of choice, Debian, already does its utmost to preserve all configuration file changes automagically. I find it hard to understand what Gentoo does differently which makes it better.

    Gentoo's approach gives the user more choice. It preserves your old files by default. You can choose to replace the old config with a newer config or, more useful, merge (typically using sdiff) in changes between the old and new config. It doesn't choose what is best for you.

    Debian's defaults are normally sane. But not always.

    But one thing is true: If you're building from source and making use of modified USE flags and compiler flags then changes are you're the only person in the planet with a particular setup - that means

  6. Re:Compiler more important than compiling? by Pop69 · · Score: 2, Informative

    doesn't compiling code with Intel's cc result in significantly better binaries than any flag you can throw at gcc??

    Not if you're running on an AMD64 it doesn't.

    http://yro.slashdot.org/article.pl?sid=05/07/12/13 20202

  7. More to the point... by Pliny · · Score: 2, Informative

    A vast number of regularly updated packages. Put simply, I can emerge almost anything, whereas every other distribution I've used, sooner or later I come across a package I need which doesn't have an RPM or what have you, and I have to build my own complete with the dependency hell that can entail.

    More than that, almost half the time you can install an updated package not yet in Portage by renaming the most recent ebuild to match the new version number.

    If that fails, you can invest a few hours and learn how to write your own ebuilds... It's insane how easy it really can be to get any bleating edge bit of software running.

    --
    What does this button d$#%* NO CARRIER
  8. Right tool for the right job by tuna_boat_tony · · Score: 2, Informative

    Precompiled binaries suite most situations. IMO, it's silly (and time consuming) to compile everything from source all the time. However, if you need a newer version of a program or some specific settings comiling from source is the way to go. Arch linux approaches this well. They offer optimized, often completely up-to-date, precompiled binaries in pacman, but allow you to build from source with the Arch Build System if neccissary.