Slashdot Mirror
Build From Source vs. Packages?
mod_critical asks: "I am a student at the University of Minnesota and I work with a professor performing research and managing more than ten Linux based servers. When it comes to installing services on these machines I am a die-hard build-from-source fanatic, while the professor I work with prefers to install and maintain everything from packages. I want to know what Slashdot readers tend to think is the best way to do things. How you feel about the ease and simplicity of installing and maintaining packaged programs versus the optimization and control that can be achieved by building from source? What are your experiences?"
I do a bit of both. I predominantly install items from packages, when available, for testing and review of something new that I am interested in. Once I establish whether what I have been playing with may be useful for some particular purpose I will research the source build options. If there are specific optimizations that can be made for my system's hardware or pre-installed software I will then look at installing from source in order to leverage those optimizations, but if there is no advantage to compiling the source due to lack of any worthy optimizations then I will install from packages any time I want that software.
That is my way of handling things, do what fits your needs best, that's why we have this option.
Even if I knew that tomorrow the world would go to pieces, I would still plant my apple tree. -Martin Luther
As often as I've lamented how much employers spend on PC's, vs build them themselves from parts, they would rather not have to rely on someone in-house to support hardware.
A feeling of having made the same mistake before: Deja Foobar
Gentoo! (Combines the best of both worlds)
I actually make my own gentoo ebuilds and build everything emerging them... so, both.
Gentoo is a great OS as instead of having binary packaged systems, it builds everything from source but can build it effeciently and automatically. In addition it can allow you to just use it to manage the source and you compile it yourself. If you were dealing with many systems you could setup your own gentoo sync server and distribute custom copies of various packages exactly to your specs and compiling details. In addition it can easily determine dependencies, and even install them for you if needed. Gentoo is kind of like a bare bones OS that simply makes it easy to install whatever you want and rather helps shortcut the process of dealing with installing things by compiling things for you.
While building from source can be fun, and necessary sometimes, I don't think it makes sense. You spend far too much time tweaking minor issues, and lose sight of major problems.
One problem that I've noticed is the fact the build from source people tend to install things in a way that's completely different than anyone else. This means that anyone who tried to maintain the machine is hopelessly lost trying to figure out what the previous person did. OTOH, When (e.g.) RedHat does something weird, the explanation and fix is usually just a few google queries away.
Most (all?) package formats have source packages that can be modified and rebuild in case you need some really special feature.
Your installing a OS from a package, so why not applications? Old programmers moto "Don't re-invent the wheel".
Mod +5 Drunk
If you are working for someone else, maintaining servers that are intended for peforming specific tasks, then I think the best solution is to do whatever is most efficient at performing those tasks. If you really don't need the peformance gains brought by compiling from source (and you probably don't) and it's going to take you a long time to do the compiling, time that could be better spend actually doing the research, then it's not worth your effort. If however the compiling doesn't affect the user's ability to be productive and that is what you as sysadmin are most comfortable with, then it seems reasonable that you should be able to maintain the boxes however you like.
I personally try to use the packages when I can. It makes it a bit easier for myself to keep track of everything.
It's all in what you need to do. If you need those optimizations or special build options that aren't in the package, go ahead, it's what it's there for.
R.
Many sources include the SPEC file required to build the package.
Achille Talon
Hop!
I used to be a huge debian fan because of apt-get and the direct install of packages, but I have migrated to OSX and find myself needing to build packages from scratch to work correctly. However, I will never hesitate to use Fink as much as possible. I think for 90% of what gets installed the packages should be fine, but if you know that there are certain optimizations that you can implement, why not build from scratch?
-------
artlu.net
Anyways, I've found that by far the easiest and most simplistic and time-saving method is to use rpms or debs. But of any distro, Lindows has it down to one or two clicks...though, they're software database subscription is a serious money leech..
If it was up to me, source would always be an option to use, and the install process for rpms and debs would be one click and automatically update themselves into Menus and such..
Just a few thoughts..
___________________________________________
nothing.can.stop.me.now
My biggest grievance against packages is the dependacy fiasco. For instance, I have Red Hat at work. And the majority of the programs are .rpm's. Well there was a certain program that I could only get as source, so I compiled and installed it. It turns out that it was required as a basis for other packages I wanted to install. But when I tried to install those, it didn't recognize the prerequisite programs because they weren't installed via rpm.
I don't care for the dependancy model of packages, and I'd much rather install programs myself. That way I know I'm getting the program compiled most efficiently for my computer, and I don't have to worry about dependancy databases.
Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
is that compiling from source can sometimes even be slower executing depending on your compiler.
Also, better to install from packages because:
1. They WILL work
2. They install fast
3. They are easilly de-installed
4. They are painless
5. Dependencies are installed automatically sometimes, and other times packages are the only way to resolve a dependency loop
6. Most other OSes since the dawn of the home computer use pre-compiled binaries, and nobody has complained
7. It is surely the developers job to make sure it compiles properly and do all the compiler error headache solving
Packages are just so much nicer. A lot of the time, I can get pentium-optimised versions of the ones I want, and if I can't then 386 optimised versions are OK by me. The difference in speed one sees is pretty much only for the anally retentive, it is so minimal.
Run debian, if you absolutely must install from source you can use APT to get grab the source that you need, compile and then build a deb for it so you're still using the debian tracking system. It really is the best of both worlds.
For most packages though there really isn't a big need to compile from source.
I used to be a die-hard build from source person myself back when I ran slackware.
Since that time I have gained more experience with production Linux systems.
When it comes to managing production servers, I use Debian and typically only install programs that are in the stable tree.
Every once in a while I have to build a deb from source, but only in rare circumstances.
Now, when it comes to my development systems I am more likely to compile from source rather than rely on the packages to supply me with the latest and greatest.
It really all just depends on what kind of stability vs. "new" features you need as well as ease of managment. Installing a package takes 30 seconds vs. compiling/installing from source can take longer and requires more hands on.
~.Evanrude
I use OpenBSD, which like most of the BSDs has the ports tree, and also has packages. Most of the ports tree are built as packages and are available on the FTP sites, allowing you to either install 3rd party applications from source preprepared for the job, or install the package that has already been preproduced from that port. Best of both worlds, and indeed if you are after customisation and have a number of systems, you can make the changes on one system, and bingo - you have the package ready to roll out to the other systems.
As for what I use? I used to use solely ports, but now I usually grab all the packages when I do a fresh install, and only use ports for what isnt available as a package, as the packages give me no disadvantage.
Whenever a binary package for Debian is availible, I prefer it to hand-compiled source. First, it has all the Debian patches it needs. Second, it propably installs without a hassle. Third, it's easy to get rid of it, and last but not the least, apt resolves dependency problems without human intervention in 99.9% of cases.
In other words, binary packages work for me :)
I can tell you as a grad student with 3 years experience working in an engineering lab, packages are the way to go. Not just in software, but generally in most situations. As others have mentioned, you have the ease of use, tech support, and the time savings. While you may eke out a little bit of performance, your time is of significant cost to the lab, with which you can be doing many other more valuable services. Also, as a student, you will likely only be there for a couple of years. When you leave, and something goes wrong, someone else has to sort through what you did to try and fix it.
If you're responsible for the machines you run how can you abdicate that responsibility by using whatever some package maintainer decides to give you? At the University of Michigan we use Linux from Scratch to manage hundreds of machines that provide everything from web servers to IMAP servers to user Desktops & Laptops. The trick is leveraging the work used to administer one machine well out to hundreds of machines. The tool for this is radmind. Radmind doesn't require that you build your software from source, but it leverages the work you put into one machine to manage all of your machines. It also integrates a tripwire with your management software which means you can detect unwanted filesystem changes in addition to managing software.
It depends.
If you are advanced enough to compile source code in such a way that it performs better or in a tighter controlled manner, which suits the purposes you need better than off the shelf builds (packages), then by all means, build it from source.
If on the other hand, you don't have a compelling reason to compile the source, then use the packaged product.
I don't know about you, but for most of my servers, the extra configuration options needed to squeeze an extra few percentage points of performance isn't enough to bother running my own compile.
Those that say they review ALL code before compiling for security (backdoors, holes etc) problems are probably lying. I am sure there are a couple people who do.
Basically if you do it just so you can be 1337, you are just vain, as I doubt that most people would see/feel the difference.
Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
I would have to agree about using packages. One gripe I have about building from source is
that most packages do not have "make uninstall".
With packages, you have a much better chance of removing all the files that were installed with the packages when you need to.
"We can't solve problems by using the same kind of thinking we used when we created them." -- Albert Einstein
And any RedHat user won't really understand what the BSD user is talking about and will just keep on using binary rpms found from google or rpmfind. In a desparate moment one will use any rpm that seems to do the trick - nevermind security, PGP sigs, all that stuff...
Seriously speaking, building from source is the UNIX way in my opinion. There is just something very heart warming and satisfying about seeing all the compiler messages scroll every time you install a package. (And try installing the native Java from BSD ports - several hours of pure joy!)
I always do.
Most package systems allow you to "roll your own" packages from the software you build from source. I use Slackware myself, so I first install my apps into a "staging" directory and build my package from there using the makepkg command.
It takes an extra minute of your time when you're installing software but it really helps to keepi track of what software is installed on the system, what files belong to it, keeping track of versions etc.
There's one overriding detail that you should consider.
You're working for the professor. He's your customer. The customer makes the rules. The old saying "The customer is always right" is true. Most people don't know there's a second half to the saying. "The customer is always right, or he's no longer a customer", meaning if you don't do it his way, he could easily fire you, and rightfully so.
Regards,
bt
Personally, I use both binary packages and source. Basically, if my distribution has binary packages, and they fit my needs (recent enough version, etc, etc), I'll just use the packages. Why not? However, if I do decide I need to build something from source, I like to use GNU Stow to manage my software. Basically, Stow allows you to install your from-source packages in a nice, sane hierarchy (eg: /usr/local/packages/this-program-1.0, /usr/local/pacakges/other-program-2.4), and then Stow does the job of setting up symlinks into the traditional Unix filesystem (typically /usr/local). So, by using Stow, you get the easy management features of packages (minus dependency resolution) for your from-source build software. It's definitely saved my life... and it's especially useful in an NFS environment, as you can export your packages directory and then use stow on the workstations to install individual packages as you see fit. Quite handy. :)
I've often had a lot of trouble building programs from downloaded tarballs. Besides mysterious dependencies that I can't track down, sometimes things just don't compile, or they crash, or they produce errors of other sorts. But in many of those cases, I could download, say, an RPM of supposedly the same package, and it would install just fine.
On the other hand, I've never had any problems. Emerging new packages deals properly with all dependencies, and things always compile correctly. And there's like a review process where packages are first added to portage as "unstable" and then once they have passed everyone's criticism, they are added to "stable". So far, the only "unstable" package I've decided to emerge was Linux kernel 2.6.4, and that all worked out brilliantly.
Also, if you have a cluster of computers, you can do distributed compiles with, I think, distcc and/or some other package. Gentoo documents this VERY well. Plus, if your cluster is all identical machines, you can build binary packages once and then install them onto all other machines.
BTW, Gentoo isn't for everyone. The learning curve is STEEP. I had to start from scratch and do it all a second time before I got everything right. (Although I am a bit of a dolt.) Setting up is complex but VERY WELL documented. Only once you've finished building your base system does the extreme convenience of portage become evident.
Also, there are still a few minor unresolved issues that no one seems to have a clue about.
I agree. What the professor wants is a readily supportable, production environment, and tat's what you should supply. That means packages wherever possible. IFF there is a clear need, build from source- a 5% speed optimization may not be worth it (that's the prof's call). A 50% speed improvement (unlikely, but possible) would probably be worth it (prof's call). Otherwise, I'd only build from source when there was not a trustworthy package available, or to add features, fix bugs, etc.
I've been in both your and the prof's position, and this is generally the best bet. It'll make the prof's life a lot easier when you're gone, too.
build packages from source exactly how you want them , make a tarball of that, and then use ssh and key trusts to shoot them out everywhere (this coming from a person who maintains almost 1000 servers)
it works very well.
A year spent in artificial intelligence is enough to make one believe in God.
If you could demonstrate that installing/upgrading from the source results in a quantifiable improvent in maintenance or performance over a pure binary distribution, I would consider it. If there are no existing reliable benchmarks, but you'd make a good case, perhaps I'd let you turn your own workstation into a demonstration system.
Anything else. No way. If it works, don't mess with it.
I run Gentoo at home and, while updating with "emerge" is kind of nice, I've yet to find any compelling reasons why it'd be better than up2date or apt-get. There really are no measurable performance or reliability advantages.
The owls are not what they seem
I tend to do a lot of customization for daemons like a Apache, and some libraries like PHP. However, it's best to leave most libraries to your packaging system being as many other packages are likely to depend on them. It's a balance. Compile when you want/need to... but try to limit it to major apps like daemons that not many other packages depend on.
I used to run an ISP, built everything from source, but eventually it got to the point where it was un-manageable.
;-)
You end up with different versions, different compile options, upgrades are a mess, and it's hard to support.
Another problem is filesystem pollution. When you do your "make install", it's hard to track what files are installed, and when you upgrade to a new version, you can't be sure it's clean, since you might have configuration files or binaries anywhere on your system.
So, one day, I started to make RPM packages of stuff I needed, and modified existing RPMS, and sent all the patches to the community.
What happened is that Mandrake accepted all my packages, so all I had to do was to install the standard distro, and all I needed was there.
And eventually, I made so many packages that they hired me
But even if I wouldn't work for Mandrake, I'm still sold on RPMs. You have a clean SPEC file that contains the pristine source code, plus the patches, and basically all the instructions to build the stuff. You can specify the requirements, you can easily rebuild on another machine, uninstall the old stuff, or upgrade, with a single rpm command.
I don't use Linux; I use FreeBSD. I build applications from source (ie, from the ports tree rather than packages) on my work machines, and my home machines.
The biggest reason is that I can have macros in the global Makefiles that control how the application gets built (ie, globally build everything without LDAP support), and for things like PHP I can compile in exactly what I need, and not have to link against libraries I don't want.
(S(SKK)(SKK))(S(SKK)(SKK))
I've been working as a Solaris/Linux sysadmin since '99 and I can definatly say that over the long haul, packages are way better. However, I tend to custom build, or at least tweak, my own packages and create a local repository to store them in. Then create a local blog about what is installed where and anything special you had to do.
Best of both worlds with documentation for the next admins.
Oh yeah, -Os often gives better performance on modern processors than -O2 (IME) because more of the loop fits in cache.
Why not build from source on machine 1. Then have machine 1 build a package to use on machines 2->n? Yahoo! Best of both worlds.
Yes, I know, it is a great distro (it is mine too), it compiles everything from scratch, let you optimize the produced code for your machine, and does it automatically and nearly flawlessly. But I don't think enterprises having to manage dozens of linux servers will ever be really excited about this. Why ? Because compiling simply takes *time*, and that is exactly what most serious system administrators are trying not to loose. However, I agree Gentoo is an excellent distro for geeks and advanced users, especially because of its BSD-like+compiling powerful packaging system. But it is ridiculous to stand up to say gentoo combines "the best of both binary and sources packages". It doesn't.
As a FreeBSD user, I build almost everything from source using ports. I never install from packages. My reasons for this are many and varied, but basically, I prefer to build software myself, with the precise options I need. When you use packages, you are at the mercy of the packager and their preference for options and optimizations. Several years ago when I used Linux, I often encountered problems of pre-built packages lacking a particular build option, and sometimes installing to odd places, or other strangeness.
And once you've started using packages and package management, it gets harder to introduce source-built software into the same environment without screwing up your dependency databases, or worse - breaking things. So if a package lacks a required option, you really have to build your own package with the option included in order to keep things orderly. That's a lot more work than just installing from source.
I'm not a Linux user anymore (several reasons) but if I were I to go back to Linux, I would use Gentoo, specifically for its Portage system.
So, in my opinion, building from source may be a little more time and CPU consuming, but it is the better option for a controlled, tailored environment.
I have a dual processor Athlon MP machine; I use this machine for my Desktop at home every day. I use gentoo because I want the latest and greatest bleeding edge and I want it to runs as fast as possible on my set-up.
Some distro's (mentioning no names) still build for 386 and I've come across distros that only utilise one processor at kernel level let alone build individual packages for multiprocessor support. I prefer to know that im using my hardware to the best of its ability.
However if im installing a server; I'd probably choose a tried and tested distro Red-Hat for a colocated machine which i may never even get to see with my own eyes; Reason being a colo shop will have in house support staff able to fix any run of the mill problems that occur.
For an in house server I might choose Mandrake or SuSe (more likely Suse) and maintain packages that way (last thing you want is to spend several days at work getting a gentoo box up and running!);however, stuff like apache / php etc i often like to compile fresh and configure how i need them. plus it makes patching that little bit easier if you have a specific set up.
Generally speaking anything mission critical I'd try to use packages that have had a fair crack at being tested well after build.
Anything personal you might not care too much about uber-stability like a desktop / research/hacking machine its generally fun to hack about with stuff and compile your own from source.
Electronic Music Made Using Linux http://soundcloud.com/polyp
has a great ports system which allows you to build software from the source automatically. I find with ports, you can get the latest software in a more timely fashion as a package is not always available and it can be built to use your machines entire potential. The package system is integrated into the ports system, however, so you can build your own package from the newest port then distribute it onto several machines. The other good thing about packages is for older machines with small hard drives and slow processors-you would probably run out of drive space (and patience) trying to build something like openoffice from source. That's just my experience using source and packages for a particular system.
Build your software from source and then create a package. Distribute and install the packages. It is a trivial matter if you use Slackware. Other distributuions are not too difficult if you use the checkinstall utility. You get the best of both worlds.
Alex, I'll take keybindings not used by Emacs for $400....
I use fedora, and most often I get the *.src.rpm versions, then tweak the SPEC files as required, build my own binary rpms, and use those. Best of both worlds, IMO.
.nosig
I prefer to compile most important programs from source.
/usr/local with everyone elses crap. If you're compiling it from source, it's important enough that it should be separated.
For example, if I'm running a web server, I'll install and configure apache, php, some database, etc. from source. But I really won't care if I have vim or cron or jabber or whatever installed from a package. If you do a base OS install with all basic/necessary components regardless of the application of the server, and then install important software from source, this will generally be the case.
The key to managing this is creating a separate directory for your crap. I.e. don't just shove your stuff in
Then, if someone else comes in you can say, "Everything is packaged, except for important software which is in this directory. All the source and configure files for that software are located in this directory, in case you need to figure something out."
Just my two cents.
01100111 01100101 01110100 00100000 01101111 01110101 01110100 00100000 01101101 01101111 01110010 01100101 00101110
..of time.
It's like the programmer who spends six hours hand-optimizing the inside of a loop that gets called once a day and already executes in 10ms... but ignores the fact that the program takes 20 times longer to run than it should because of an inefficient algorithm. This programmer doesn't know *why* his program is slow, he's guessing, and he will almost always guess badly. This is why profiling was invented.
Look at it this way. Installing from the packages you get the following benefits:
- You save time compiling (multiply this by the number of patches you have to add over the box's life time)
- You save time tracking down dependencies
- You have a standard platform you can re-deploy at will
- You have something that another administrator can work on without asking where you shoved shit.
- You have a package database you can query for version information, dependencies, etc.
- You have an easily available source of "known good" binaries if you have a suspected intrusion problem.
- Depending on the package system you use, you might be able to stay on top of security vulnerabilities with very little (or no) work.
Now, installing from source, you get the following benefits:
- You can pick where the files go (whoopie)
- You tune the performance for your platform
- You can select specific features
- You can de-select specific features to save disk space
The only one which gains you a lot 99% of the time is where you can select specific features which are turned off in the standard package. If you need those options, you build it from source. If you're doing ten machines, though, you build it from source on *one* machine, package it up, burn it, and install it from YOUR package on all ten machines.
Saving a few CPU cycles is never worth saving a man-hour. You can use the man hour more productively on the macro-optimization level. Similarly, you can take the dollars that you would be pay the man and buy a new CPU with it.
The same argument goes for saving a kilobyte of disk space. If found out that any of my guys spent *any* significant time trying to cut less than a gigabyte out of our application footprint, I would give him a footprint of my own, right in the middle of his colon. Disk is cheap. People are not.
If you have an application is which is CPU-bound and running too slow, find out why (profile the system or binary), and build from sources only what you need to make your application conform to the target specification. Or, if that will take too long, just buy more CPU.
Long story short -- tuning of ANY kind should not be done at the micro-level across the board, that's just a waste of time. Tuning should be done by profiling the system as a whole, identifying the constrants, and relieving them. If that requires micro-tuning of a few things, that's fine... but squeezing every last little bit of performance out of absolutely everything is either impossible or incredibly time-prohibitive. And, of course, if you were going to spend that kind of time, you could either buy new hardware with the money (remember Moore's law), OR you example the system more closely at the macro level and come up with a better way to do things.
Do daemons dream of electric sleep()?
i worked at a university in virginia in the music technology lab, where we had two linux servers that did everything from serve web pages to run netatalk. my boss (also a professor) liked the RPMs too, simply because after i left there was no guarantee he'd get any help from the IT department, and he understood how to use RPM from the command line.
:) for a while i would recompile the kernel and he flipped out -- so i started using those crappy RPMs.
i guess in academia they are used to having funding for some things some of the time -- your professor probably wants to keep those machines running as long as he possibly can, because money has to be used for other things.
and besides, compiling programs is a hard thing for the "sorta unix geek" to get his head around
fortunatly, i think this will change when people realize there is an ample supply of knowledgeable folks out there who can do this stuff. it's easier to find a geek now than it was even 5 years ago!
If you have an application that you need performance out of, spend time compiling that once and then packaging it once and installing it on your 10 machines.
When looking from the prof's view, it will be easier to get someone else up to speed after you have graduated if your machines stick closely to standard packages.
Use the time that you'd spend compiling/installing doing more CS related activities.
Most people (including myself) that have gone through the phase of wanting to compile everything get out of it as soon as they have some real problems to solve.
As an experienced SysAdmin, I'm kinda on the side of your prof. Packages give ease of installation of over many machines and (perhaps most importantly) proper tracking of files that are installed to prevent files from being overwritten, and to allow for uninstalls too. OTOH, building from sources gives you fine tuned control over what gets installed and where, and specific build options.
So, why not have the best of both worlds? Build your own packages! I use EPM to do it and it's a breeze. You can get EPM at:
http://www.easysw.com/epm/
I'm not religious about building everything from scratch, but I like being able to include my own default config files, as well as have control over what gets installed where (I mostly manage Solaris machines, but often build Linux packages too).
As a shameless self-plug here, I recently wrote an article for SysAdmin magazine on packaging with EPM. It's especially handy in multi-platform environments. If you want to see my article check out the Dec. 2003 issue of sysadmin mag:
http://www.samag.com/articles/2003/0312/
Oh stop already. Unless you're building _every_ library from source then the optimization of later libraries is lost on the precompiled libraries they're dependent on.
It's a nifty feature of Gentoo but how many users really want to wait for glibc? If they don't wait for glibc then are they really gaining anything significant when they build Mozilla manually as opposed to using a nightly build?
Think Tetris. If you don't optimize from the very first row then optimization at row 15 isn't going to save your backside.
+++ATHZ 99:5:80
For servers, go with something like Debian: good clean integrated system with timely and automatic security updates. Not bleeding edge, but if it's at all a serious server you really don't want it to be.
Desktops, Ports based system all the way. Why? Because with something like Gentoo, it might take several days to compile but you can be assured you're not going to dependency hell anytime soon when you want to try the latest and greatest. Headers and such are installed by default, so you can usually compile something by hand and it will Just Work whereas if you're using three different unofficial package streams and you need to do some upgrade of a simple library somewhere which has an anal retentive versioning and dependency specification, attempting to apt-get that new version will cause your entire house of cards to come crashing down. I lived with Debian on a desktop like that for god knows how many years until I decided "No more". Yeah I have to wait a while with Gentoo but at least I only have to do it once.
This makes gobs of sense in some situation and very little in others.
I grant that when my coworker build the beowulf cluster, it made lots of sense to have everything optimised out the hilt. However when I watched him build his desktop, and it took him, on a very modern machine just a year ago, nearly 3 days to have a full working system with X etc... thats overkill.
I don't need an optimised ls and df... I can do just fine with them compiled for a 386. The vast overwhelming majority of binaries on my system will give no benefit whatsoever if compiled with all the optomisations to tweak it to the box.
I am a fan of this...
I use debian... I install everything from packages as a rule. However for those few, oh so very few, packages that really need customization, then I compile them myself.
At this point, I can't think of a single package that I do that with right now.
My basic view is this:
Compiling something that I don't NEED to compile is a waste of my time and CPU. I also like the idea of NOT having to have a compiler on every machine I run. A production server should never have a compiler on it, doesn't need it. Now I know the security argument is silly, any cracker worth his salt can put a compiler somewhere and use it... however....
Nobody should ever be compiling anything on a production server. If the machine is in production, then development should not ever be done on it. All that should be done elsewhere. SO why leave a compiler sitting there begging lazy admins to use it?
(OH yea and every rule can and should be broken at times, but its important to understand the rule so you know when those times are... thinking about where compilers are needed and where they arn't is a good exercise in developing those disciplines)
Overall, I think gentoo is neat... and on our solaris systems
we are considering stealing portage and hacking it a bit to build
solaris packages for us. Anyone else done this?
-Steve
"I opened my eyes, and everything went dark again"
Now if you'll excuse me, I have to go reboot 100 systems.
My beliefs do not require that you agree with them.
which is better, vi or emacs? ;-)
Schrodinger's cat is either dead or really pissed off...
I'm guessing it's a bit harder to rebuild and duplicate environments exactly. If I build 3 machines today, it's not easy to ensure I can rebuild the exact same machines 3 months from now, at least not with the standard 'gentoo' approach. At least, not as easy as saying 'pop this mdk10 in and install'. You at least know what base everything is starting from.
creation science book
Duck and cover, incoming Gentoo zealots :P
Personally, I install from packages (apt) wherever possible. If something is unpackaged and looks new and shiny, then I'll install from source. I really can't imagine managing a large number of applications without a package manger, even if it's something you've written yourself.
If installing everything from source is your thing, you're probably already using Gentoo with its package mangagment. So the question is moot.
"The number of Unix installations has grown to ten, with more expected." (Unix Programmer's Manual, 2nd ed.; june 1972)
apt-build provides automatic source based package installation in debian. Not every package offers a source package, however. This is something I'd like to see expanded in debian.
Also note the aptly named, though apparently dead project www.debtoo.org (google cache) which is based on apt-build. Don't let this stop you though, 'apt-get install apt-build' and give it a try.
Sometimes the exact opposite is true, especially in terms of "community support". For instance, mod_perl, which for some reason Red Hat decided to ship a very early version. The typical response on the mailing lists for mod_perl or any other alpha/beta package RH included usually goes "try it from source, then email us" (that's after someone submits a reasonably complete bug report).
Let's not forget the GCC fiasco and probably dozens of other examples where RH decided to "lead the pack" in terms of version numbers but not stability.
Of course, then there's Debian woody, living in circa-2001 land.
Optimization? Control?
Man, what is this, Gentoo?
Any sane distributor these days builds binary package with reasonable optimizations that won't break across architecture submodels, and occasionally releases binaries targetting submodels (e.g. PentiumPro-specific packages). On many machines, for many workloads, however, the model-specific optimizations just aren't that helpful. Obvious exceptions are floating point math on most platforms (especially x86, where x87 math code is a dog and should be replaced with SSE code if possible) and - I'm told - really slow hardware. (I'll be able to test that once I get these Indys running GNU/Linux.) In my experience, Debian hasn't really felt any slower than my LFS systems for personal use.
So, I'll say this: if you have enough time to build everything you're using, do some careful speed comparisons between your self-built packages and the vendor's binaries. If there's really a significant speed increase, and you need that increase, source is the only way to go for the packages that need the speed increase. Otherwise, it's probably not worth your time.
Unless whatever you're doing is extremely security critical, you can probably deal with the fact that server app foo has features bar and baz installed that you won't use. If you can't, you're probably auditing the source of everything you use anyway, and that doesn't sound like the case, so "control" probably isn't a real issue here either. Control can be found in config files as well as in the configure script.
People say, "but package dependencies suck!" Well, yes, rpm (the program) isn't built to deal with dependencies that gracefully. If it annoys you that much, go install apt-rpm or something, or even Debian (gods forbid). Package management isn't rocket science.
this is coming entirely from a *BSD perspective [especially FreeBSD], but the older and slower your hardware, the more you might depend upon packages, just because they take less time to install.
That said, I routinely build stuff from source on a Pentium Pro 200 MHz dual CPU machine at work. It's not our main server, so the performance hit is never noticed.
Portupgrade is a absolute must on this machine, as we have all kinds of software running on it. Without portupgrade, I'm sure it would be a nightmare.
In the end, it's whatever works best in your situation, and to have this as 'news' on slashdot seem really freakin' ridiculous.
For quite a while I used RedHat and did enjoy the ease that package management gave to a system. For a workstation equivilant, I still agree with this solution in general. However having run through Linux From Scratch (www.linuxfromscratch.org) I see that on a server-class machine, there is a TON of unnecessary bloat. Why should it take a GIG of space , or more, to host just a Web server with MySQL and FTP access? With LFS I can build a specific purpose system and get that footprint down to around 350 to 425mb and that's including the kernel sources being left for recompile and a full compile environment. I've been told that some people can get the same functions stripped down to less than 200mb (this is all of course NOT counting your SQL databases).
At this point there needs to be a big fork somewhere to divource the Linux Desktop from the Linux Server. Linux will do both, but one should not cause issues for the other. If a desktop user wants to run a FTP server, they should be able to. If the server admin wants to have a mail client (pine) or an IRC client (BitchX) installed for accessing information, he should be able to. But these features should be implemented with that specifically in mind. Not installing half a million libs because *maybe* the server admin wants to install addon XYZ for pine and it needs this lib while pine itself doesn't...
I've been a UNIX sys-admin for about a decade.
My advice is that for a workstation that is managed by an individual you can let the admin do whatever they want, but for any server that has to be stable and maintainable you want to stick with a well maintained package repository and try to avoid 3rd party packages and tarballs if possible.
You have to understand that there is a software stack in most services.
With the kernel and core libs (like glibc) and such at the bottom of the stack, and applications like Evolution at the top of the stack. In between you can have gdb and openssl and various perl modules (in AMAVIS for example) and you have sasl stuff which may be related to pam and openldap and cyrus or wu.... etc..
The thing is that even though all of those various pieces of the software stack may be linked against different libraries on the box, the maintainer of the library code may not have a QA group to co-ordinate regression testing and compatability testing before the latest CVS commit is enacted to fix a bug referenced in a CERT alert.
RedHat and Debian and SUSE and all the others have package repositories, the repository maintainers do an amazingly fantastic job of QA and testing to make sure that new patches don't break your software stack. As an individual you simply can't keep up with that.
For example the Development team that takes care of OpenSSL doesn't backport their bug fixes and security patches to old versions of the code. They just maintain the latest release version and the current CVS version. If you have an old server running IMAPs and HTTPs and SSH and SMTP/TLS and such, and CERT announces a bug in openssl vX.Y, then the OpenSSL development team will certainly release a patch for the latest version which may be version Z!
That might cause you to have to upgrade APACHE or wu-IMAP or OpenSSH or Postfix etc... Those things might then have divergent dependencies that would cause you to go and rebuild half a dozen other packages, and so on and so on. Also, do you remember all the magic flags you used for configure and make? Do you have the same environment variables set today that you did the last time you built PostFix? The possibilities for problems are endless. And if you do have a problem you are kind of on your own since your system will be a unique box. Whereas if there is a problem with a standard RedHat or Debian package, then you can always go to the general newsgroups and chances are there are a dozen other "me too" posts with answers already.
It is much easier to use apt or up2date.
So, unless you have a very good reason for using a tarball on a production server that requires reliability and security and high availability, then you should stick with packages.
If you want to build the packages from source, feel free! RedHat and Debian and SuSE make the SOURCE packages available so that you can dig in and read all about'em. I'm sure the Debian team could use a new package maintainer, if you are addicted to compiling and testing things, check them out.
It's still a smart move if you're building from source. Just package your source. Then you can build the sources under the control of a package manager (like RPM), and install the resulting packages. You get the full benefits of build-from-scratch and the full benefits of using packages.
This is exactly the approach I use. In fact, I'm a bit more strict about it: My policy is that I don't install any software that isn't packaged. If I need to install something that isn't packaged, I'll package it first. If I don't like the way a packager built an already existing package, I'll repackage it.
The bottom line is that creating your own packages (or fixing packages you don't like) is much easier than maintaining a from-scratch, unpackaged installation. Or ten of them.
To get you started, here a couple of RPM-building references:
Don't give up the benefits of source. Don't give up the benefits of packaging. Have them both.
Easy, automatic testing for Perl.
Stow lets you install each package in its own directory (i.e., /opt/pkg-x.y.z), then symlinks them into a unified /usr/local tree. Stow -D pkg-a.b.c removes the symlinks for just that package, letting you do a single package uninstall. You can manage the files on a per-package basis, while users can ignore all the details, as it looks like everything is installed in /usr/local/bin to them. Stow provides a simple solution for building packages from source on any UNIX.
I use SuSE (formerly RH), so I'm "into" using RPM. OTOH, I usually only like RPM's that have been built by the distro's creator. (Noteable exception: PackMan RPM's for Xine.) Anything else, I usually compile from source and stick in /usr/local. Checkinstall is what you need here. After configure and make, you ``checkinstall -R'', and it makes an RPM of whatever would be installed with ``make install''. That way you can take it back out very easily.
Acts 17:28, "For in Him we live, and move, and have our being."
You can have the best of both worlds with Gentoo. I began using it about a year ago, and I am sold.
Building from source using Portage is almost as easy as installing a Red Hat package. The community is extremely proactive. (I have only had problems installing or updating a couple of times in the last year, and the problems were remedied within a day or two and the portage trees updated after I submitted a bug report.) And you don't give up variety. The number of ebuilds available in the Portage tree is simply astounding.
I am even using it on my laptop these days and am extremely pleased that it seems to work well as both a server and desktop distribution.
Hope this helps
-- My choice of computing platform is a symbol of my individuality and belief in personal freedom.
If the professor has some sort of grant he may prefer a package because it is quicker to setup and save time so you can be more productive in other areas. If it is some sort of continuing income then you might as well try to incorage recompiling the source because you get more out of it educationally.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
Is that, like, 11 Linux-based servers?
Have fun: Join D.N.A. (National Dyslexics Association)
icc, btw, is free for non-commercial use on Linux.
The Raven
No way.
Usually when one builds from Source, they install it to wherever the original developer has it set to by default. Unless you did some heavy patching, the software will very likely be more "true" to the original software then many packages.
RPM's for distributions such as RedHat or Fedors often have to move configuration files all over the place to mesh with the OS properly.
You're more likely to be able to sit down at a strange Linux box and troubleshoot whatever program when it's compiled from source tarballs versus an RPM. Unless of course, you know the RPM, or the RPM doesn't do anything funky.
Considering the stuff is Open Source, and chances are the programs are not under a paid-for support contract, it's pretty safe to say that BOTH methods would have to be supported "In House." And if not, your support contract could very well support the source compiled versions anyways.
I choose the Gentoo way. Everything is compiled from source; it's just nice and automated. Almost never have I run into something where the program had to be modified to fit the distribution.
- It's not the Macs I hate. It's Digg users. -
That said - for a work machine, I prefer binary packages. I just want the damned thing to work, work well, and not futz with it.
For a hobby/play/research machine - I prefer source packages. I have found there are many compilers out there that will massively outperform GCC, especially when you turn on those crazy optimizations that most binary distributions won't (plus optimize for the EXACT processor I am running on, etc.)
I have mod points and I am not afraid to use them
Actually, you can 'emerge -buildpkg foo' and share packages between machines. if you are managing multiple machines, chances are that you will not have each one with an unique configuration, but only a few profiles.
My arguments on why to use a source-based distribution have been covered in other posts, so I won't repeat them here. I think Gentoo provides a solution that will satisfy both you and your professor: you can use a source-based, custom-built binary distribution.
.ebuild (the file that describes to the system where to find the source and how to build it) requires adding only a single flag to the package compile command, ebuild.
As you probably know, Gentoo is a source-based distribution, but it also allows binary packages. Many (such as Mozilla Firefox) are distributed by Gentoo as source and binary; you can choose to install either. The ability to build a binary package from a source
Additionally, since (if I read you correctly) you're probably using similar hardware for each of your machines, it would be trivial to set up a compile box which would produce binary packages for your other boxen. Packages compiled for your architecture would be faster than most binary-only distributions (many are still compiled for the i386 architecture), and writing a new ebuild is trivial compared to writing a new spec file. (Trust me; I spent a quarter writing a paper on the topic while I was in school, not to mention having had to do it myself in the Real World.)
Finally, Gentoo integrates and tests its packages. Ebuilds come with Gentoo-specific patches, so you don't have to spend the time to make each source package work with the rest. This is probably one reason why your professor likes binary distributions: they all work together, and enough people rely on them that if something breaks, it gets fixed. A package-based Gentoo distribution would allow you to leverage that, while keeping your machines unified in their versioning (as much as you want them to be, at least) and also provide all of the benefits of a source-based distribution.
Love justice; desire mercy.
How you feel about the ease and simplicity of installing and maintaining packaged programs versus the optimization and control that can be achieved by building from source? What are your experiences?
Humans do not scale well, they have very low bandwidth of information sharing, and have high latency (i.e. you can't get ahold of them). Humans are also expensive, wander off into different jobs, graduate or drop out of college, etc. So I tend to prefer the reducing human cost of the system administration complexity as a default position.
So my gut feeling is that unless there is a major time or dollar savings in the optimization by building from source (i.e. avoid buying 10+ new CPUs for the systems, or computation runs take a day less) go with the reducing administation complexity by using a package management systems so that you can concentrate on your actual goals (research, profit, or whatever).
My general idea is that if a pre-built binary is available, unless there's a good reason not to use it, I use it. The pre-built binaries are not always 100% cool, at least according to some people, but they tend to work for me in most of the cases.
I'm usually using prepackaged binaries if they're out there in a reasonably well-documented repository - that is, included in Debian, in some rare cases I might even consult apt-get.org.
For stuff that Debian doesn't yet have, or that absolutely insists that I build from CVS, there's always GNU Stow for easy management of stuff. I also build kernel from source using make-kpkg (because, once upon a time, it was a great Heresy to use the Pre-Packaged, Unoptimal Kernel, and building the kernel seemed to be everyone's baptism by fire so to speak).
The reason I'm often relying on pre-built binaries is that I'm a very patient person except when installing software (having had a share of installing proggies for friends and relatives tends to hurt one's very being), and I just prefer to have a quick and easy installation.
Building from source always seems to involve installing required development kits, and then million and one little bits and packages in semi-random order. There have been some pathological cases like mp1e / rte / whatever the hell it was that seemed so complex and convoluted that I needed a week's rest after that, or something like that.
Then there have been cases where I haven't been even able to build the things due to system constraints. Back in the early days of GNOME, it was hell to try to compile MICO on my Pentium 166MHz when I had meager 32 megs of physical memory, and trying to grab the last available bits of swap space from my 6 gigabyte disk... Oh, and this happens ocassionally even on recent times: I was unable to build Ardour on my current machine. Glad I found it from apt-get.org, and it's now in main Debian tree too.
I'm just secretly hoping that Debian goes i586 instead of i386 some time...
What some people don't seem to understand about Gentoo or the BSD's is that not everyone is hell bent on world domination and market share. Some people want something specific, and Gentoo and the BSD's are there for them. It's not like they are ever going anywhere. BSD "despite the rumors" has never done anything but grow in usership with the steady, yet slow trickle of new users and the fiercly dedicated long time users. Gentoo is growing rather fast, but will no doubt plateau off and settle in the same way the BSD's have. But by all means, continue to have your OS flame wars and make your comparisons and talk about market share or other things that aren't important or even remotely interesting to the majority of most Gentoo and BSD users. It's very humorous. :) HAVE FUN STORMING THE CASTLE!!!
For every annoying gentoo user, are three even more annoying anti-gentoo crybabies. Take Yosh from #Gimp for example.
#portupgrade -a
or, if you prefer packages
#portupgrade -aP
Even Jesus hates listening to Creed.
This isn't an either-or situation.
.src.rpm files will use the ./configure provided with the package, which will generally figure out good defaults for your system, and you can further tweak this with your RPM macros.
One of the main design goals of the RPM system (the most common package format, used in RedHat/Fedora, SuSE, Mandrake, Conectiva, and others) is that you can reliably rebuild packages in an automated way.
The meta-data provided by a package system is essential when maintaining production servers -- you need the ability to easily figure out which package owns a given file and what will break if you change something. This is not nearly as critical for a home machine, or a system that will not be running for years, gradually upgraded over time.
If the binary package works for you, cool. If you want to rebuild with the latest libraries and better optimizations, or for a newer architechture (amd64 vs. x86 for example), then go for it.
Most
If there's no RPM for something you want to install, then create one! It will take you half a day to make a really good RPM the first time, but just an hour the second time, and you'll have it down to 5-10 minutes + the compile time after your first dozen.
And to get out of dependency hell, use yum or apt/synaptic.
When you use binary packages you are generally at the mercy of how the maintainer decided to build the package. Want mysql with --with-lo-mem? You're stuck either without it or stuck trying to find someone else's build. I understand the need for binary packages. How many people would need coreutils with a specific build option? OTOH, how many people need Apache built a certain way. If you're using a binary package with something like apache that probably needs to be built for your needs, you done missed the boat.
If the program is available for my favorite distro (Debian) as a package I will use the package. But if there isn't a package available then I use will compile from source. But as most of the other posters have pointed out it also depends on the program and if I am testing it or if it is for a production system. If it is for testing then I take the package over the source if the package is available. But I, like many others here, will usually compile from source if I am going to us the program in a production environment so I can get the pest performance for my system.
"Really, I'm not out to destroy Microsoft. That will just be a completely unintentional side effect." Linus Torvalds
1) I am a newbie and have to use packages for *.
2) I know my way around. I like the level of control I get with compiling/know how to code/read far too much Slashdot. I compile by default.
3) I manage more than three boxes in my basement now. Having the ability to back out of system changes without a full OS reinstall is a necessity. I build my own packages from source that I've compiled.
4) I manage more than just three boxes in a department now. Now I have to deal with politics, ordering hardware, the freakin' network, and I generally have time for sysadmin. On top of all that I now have a family so spending two or three extra hours per day on my Unix hobby is no longer feasable. Precompiled packages work just fine.
However, I will never hesitate to use Fink as much as possible. I think for 90% of what gets installed the packages should be fine
90% of what gets installed when you use Fink has nothing to do with what you're installing.
I've given Fink a shot on a couple of occasions over the last two years, and every time I've invoked it, it's come up with false dependencies. X11 is not necessary to install, say, the Python interpreter, and there've been dependencies far more ridiculous than that.
I've had the same problem with the CPAN shell. RPMs, on the other hand, seemed to fail to install necessary things.
I build from source, then, simply because I don't trust the dependency handling from package managers. It's true that I have to pay more attention to such nonsense and would love to have it automated, but until I find a package manager that gets dependencies right, I'm going to have to do that anyway.
Tweet, tweet.
God intended man to compile from source. It's the 11th commandment.
I build the mission critical programs from source code, and just let the rest be installed as binary packages. I build from source even if I don't need to just to be sure I won't have extra unexpected issues should I ever need to actually make modifications to source and rebuild. I really don't have very many local modifications, but I'm prepared just in case.
Additionally, I do this all on one master machine (with a backup of it kept live on another machine), build binary packages of my own from my source builds, and install those packages on the actual servers. That way I have even more consistency, though at the cost of ultimate optimization. But I think it is better to be able to quickly reinstall a machine, as well as use checksum verifications that there are no trojans.
I use Slackware, but this could be done with most systems, including FreeBSD, Linux (most distributions, including Debian and the RPM based ones), NetBSD, OpenBSD, and even Solaris.
now we need to go OSS in diesel cars
People with jobs managing servers use binary packages. Real geeks work at Wendy's and live in their mom's basement.
>of "known good" binaries if you have a
>suspected intrusion problem.
A rather dangerous assumption to my mind, this one. I've heard of Red Hat releases in particular making it to the shelves while still having at least the odd security flaw. Of course you're not going to have time to go over it with a fine-toothed comb, but if you know how to read code I'd give at least really critical apps a cursory once over. It's better than your system going down or being invaded by some anarchistic 14 year old, anywayz IMHO.
As well as the security/stability issue, one of my main reasons for changing to Linux has been the level of customisability. I suppose we can let overworked corporate sysadmins off the hook for wanting to use predigested distros, particularly if they have to deploy to a lot of machines, (even the most broken distro release is likely to be infintely more secure than the IE+OE knock-out punch ;-)) but I'm not sure anyone else wanting to call themselves a respectable Linux user has an excuse.
To me, compiling from source is one of the main reasons for using Linux. The ability to compile exactly for your CPU and particular environment, coupled with the security of knowing that what you're getting is exactly what you think it is, and not something that's going to turn your system into a script kiddie gang's next 0-day ircd.
If you need something that can be deployed on a lot of machines, buy standard hardware that you know Linux supports, (avoid exotic Winmodems, onboard cards etc) prototype from source on one machine, and then mirror it to the rest. To me, a secure, stable, well-configured system is something that cannot and should not be attained in five minutes, and any corporate sysadmin who thinks it should be possible, ought to look for a career change. Just as it's true that in the rest of life there is no such thing as a free lunch, when it comes to security, the emphasis should NOT be on short cuts.
Do what your professor wants. Why you ask? Because its your damn professor. He will be happier with a package management system that he feels comfortable with. This will make him happier with you. Do not trifle with the grey beards, they have powers you do not yet comprehend.
I, myself, am working with a professor on a momentum problem generator in Perl (we're physics people) and I was given a nice equation solving library that he wrote for another issue. I've showed it to a number a people with years (1 or 2 near a decade) of experience with Perl and they said that it was some of the worst code they had ever seen. I thought the way I had to interact with it was stupid and klunky. One giant kludge. I fought it in my own head but tried not to let my emotions about it out in front of him. So I worked at it again and again and you know what a few months later he, a peer of mine and I will be doing a seminar for our deptartment on it this April. The code wasn't as awful to work with as I thought (though to this day I wish it wasn't so klunky) and it worked. I just had to suck up my pride and get it done.
Don't argue with them. Make their lives easier and you get to see the grey beards happy side. May you have many publications in your future.
I do packages when available and use whatever package management system available be it apt, fink, darwinports or whatever and I use whatever format for the packages that the management system needs if there is one. I have used rpm, darwinports,deb and llp(AIX). Packages with a management system allow you to easily install and uninstall items when you need to. They also ease upgrades.
Gorkman
The original poster has obviously never dealt with any number of machines. Building from source (with or without a package/ports system) is great fun for a single user systm. Once you get to multiple multi-user systems, it's just not worth the trouble to optimize one program by 5% when nobody ever cares about speed, just that they deleted an important email they've had sitting on the server for the last 18mo and never bothered reading.
For some things, building from source is unescapable, but with a large number of systems what you want is something that can easily be done itendically to any number of systems with little to no effort.
Right now, at work, we're trying to transition over to a system that uses Debian with FAI to do roll-outs/reimages and Cfengine to handle updates & other administrative changes (all the while, putting config files in CVS). About the only thing that's going to be custom compiled is going to be our kernel and we're only doing that 'cuz we like some custom patches applied to it.
my sig's at the bottom of the page.
Wow. There sure are a lot of posts about which is better, but I don't see any comments that deal with the underlying problem. And that is this: don't get into a pissing match with your professor. Seriously, what are you hoping to accomplish here?
If you were thinking that you'd get tons of pro-compiling comments, and then put that in front of the professor, stop right there. Coming to Slashdot for validation of your side of the argument is about as helpful as those wives who write to Dear Abby about their husbands. Because no husband on Earth is going to appreciate getting chastised by Dear Abby, and if Abby sides with him, he's going to gloat. It's lose-lose for the wife, just like it's lose-lose for you if you try to use Slashdot as leverage. Screw with the computers that the professor relies on, and he'll find a way to "thank" you for it. Don't sabotage yourself.
My Greasemonkey scripts for Digg &
Hey, you get the best of both worlds... easy install, maintenance, uninstall; plus everything is optimized and you still get to say that you build from source "just because you can".
We'll make a Debian package maintainer out of you yet!
That said, perhaps you two can come to a sort of compromise. You didn't say what distro you're using some I'm going to assume you're using Redhat. You could use RH's source RPM functionality to both compile packages the way you want them compiled and yet make it easy to distibute them to other machine and update them with little overhead. It's not too terribly hard to do. Frankly I won't ever do it this way but I can understand if someone does. I currently maintain an identical directory structure on all machines of tarballs (NFS shared of course) and host-specific source files (exploded tarballs in an organized fashion of course). I can quickly copy and paste the previous ./configure options from the older release (after reading the Changelog and docs) and get that package compiled and installed within minutes. A few minutes per host doesn't hurt me any.
Personally I'm looking into switching to Gentoo. It sounds like it matches my style of administration better than RH (anything's better than RH). You might consider trying it out as well. portage is supposed to be excellent.
I prefer to install everything from packages when I can. For stuff that I have to upgrade frequently -- usually server processes that need security patches -- I do it from source, partly because I prefer not to wait for a package to become available, but mostly because it saves me from the tangle of dependencies that come with packages. (The difference between RPM hell and DLL hell, as far as I'm concerned, is only that you don't have to pay for the privilege of RPM hell.)
In general, I haven't found that there is any real optimization benefit in compiling from source in most cases -- the kernel itself and Apache being the primary exceptions. I'm sure it's there, but it's small enough to be unnoticeable in most cases, and therefore not worth my hourly wage to futz with when I could be doing something that actually generates revenues.
Mind you, this is at work. At home, I tend to prefer compilation, but that's just because I like screwing around with the source.
Proud member of the Weirdo-American community.
the OP should be modded -5 flamebait?
Not necessarily true when your needs are custom.
Real geeks work at Wendy's and live in their mom's basement.
Oh, that's where I remember you from. You didn't give me my Biggie fries.
This is a test. This is a test of the emergency sig system. This has been only a test.
http://lunar-linux.org/
the installer is maybe not friendly enough for you, but maybe it is.
In short, Support? Who needs it? Not me. Do you?
I agree completely for vi, grep, etc.
However for glibc or other common libraries you gain much more than if you hacked sendmail or any other service.
If you have a backdoor in glibc, nearly ANY program will activate it. You just wait until a setuid root program accesses something in the library, and you have your exploit.
Or if you need something that stays aware, have this insert a kernel module that hides it's own existence and does whatever you need or launches and hides another process that does what you want.
In the end, putting a backdoor in a common library has many advantages to putting it in any program or service.
I'd rather use source all the time, but when you are using a really slow machine that could take forever... thats where packages make things a bit easier...
.02
However, the problem I've run into very recently, with mandrake 10, are that even though certain programs like apache, and php, can support a lot of things (i.e. can be compiled to support things), the binaries distributed with mdk 10 dont include all of things I needed...
Specifically, when installing Mambo on a webserver here I found that zlib support and xml support were both missing, even though apache and php were both installed. The only solution I could find is to install zlib (from source), and then recompile php and apache. I still havent quite figured out why xml support is missing, but I digress;)
Using packages and source together can also be extra work because a lot of the stuff you need to compile programs are in "devel" packages that arent installed by default (That doesn't happen at all with LFS.)
It's a tough call but the way I see it, try packages and if you don't run into problems, then fine... otherwise just don't forget that open souce is... well open source? =)
Sorry if all this was said already but I had to give my
Chaos is Divine *
As there are over 500 comments I'm assuming I'm being "-1 Redundant" but I'm also assuming moderators probably won't get this far ... come to think of it neither will readers! Oh well. Anywho I've always been a build-from-source kind of guy but that's due (at least in part) to my FreeBSD background. In FreeBSD I had the best of both worlds, the port list which made it very easy to install a software package, and the fact that the port list downloaded source and installed! Nowadays I use Suse and as such I can use RPM's, however I usually find myself building from source whenever possible. One of those, "just because I can doesn't mean I should" type of things. I think that until there is a universally accepted and implemented package type that simply works in all linuxes I'll stick to source, not packaged.
Kleedrac
Sure we wang, can.
I know there is temptation to make things a little bit better, but support after you're gone is the issue.
The genius who designs a system that only (s)he can maintain is a poor engineer.
Find out what your customer's (the prof sounds like the customer in this context) requirements truly are. Is good enough good enough for the prof? If you give him what he wants and he finds out next week that it could have all been optimized to perform
Meet those requirements with the minimum customization.
Document the system. This may be a nightmare if the system has already been "tweaked" by the previous maintainers. If that's the case, it's even MORE important to simplify and document.
Provide recovery tools--as simple as a set of drive backup images, or as complex as a set of scripts that rebuild the system from source. At a minimum, supply a system administrator's manual.
Building a system for a customer to use is a completely different endeavor from elaborately tweaking your own box so it is just exactly the way you like it.
"Reality is that which, when you stop believing in it, it doesn't go away." - Philip K. Dick
If you should need to recompile your Red Hat kernel do not try to install the raw kernel source from kernel.org. Red Hat and other distributions require that they use their versions of kernels specifically taylored to their distribution. For Red Hat, install the kernel-.src.rpm (note that this is different from the kernel-source package) and look at the file /usr/src/redhat/SPECS/kernel-2.4.spec. At the top of this file you will see something like the following:
Summary: The Linux kernel (the core of the Linux operating system)
# What parts do we want to build? We must build at least one kernel.
# These are the kernels that are built IF the architecture allows and
# no contrary --with/--without arguments are given on the command line.
%define buildup 1
%define buildsmp 0
%define buildBOOT 0
%define buildenterprise 0
%define builddebug 0
%define buildjensen 0
%define buildtape 0
%define buildBOOTtape 0
Change all of the 1's to 0's leaving a 1 for only the kernel version you wish to build. The above listing shows the modified spec file for building the uni-processor kernel. Now build with an appropriate target option like:
# rpm -bb --target i686 SPECS/kernel-2.4.spec
If you do not do this, RPM will attempt to build multiple kernel versions. Usually it is only necessary to build one kernel specifically targeted for your machine. Building all of the kernels defined in the default kernel.spec will take many hours even on a fast machine. When the build is complete there should be an RPM in the RPMS/i686 directory (or the directory for the specified target) that can be installed with:
# rpm -Uvh RPMS/i686/kernel-2.4.20-28.7.i686.rpm
Now can someone explain how to run make xconfig beforehand?
I've been building from source since the late 80s. What has happened is, I've gotten old, and tired of the same ol' repetition and screwups. These days, I always try the Deb package first. 95 times out of 100, that works fine. Even if it doesn't, the infrastructure to build is typically installable as Deb packages.
It's not even the compile time that's so significant. It's the pain of figuring out somebody's config/build system, and the even greater pain of configuring the thing once its installed. Deb packages make these problems mostly go away.
Go ahead and build from source if you like. Someday you'll get old too.
I am currently migrating to FreeBSD from Debian. The main reason is the easy of installing and maintaining software. With FreeBSD Ports system, installing is easy.
I get the latest stable software. I don't have to worry about crazy dependancies (I don't want MySQL dammit, I use Postgres). The software is in a standard place. It's easy to tweak things.
I also find that FreeBSD is much faster than my Linux system... Especially RedHat.
The above is not worth reading.
Usually when one builds from Source, they install it to wherever the original developer has it set to by default. Unless you did some heavy patching, the software will very likely be more "true" to the original software then many packages.
:-)
Correct me if I am wrong, but are you contridicting yourself here? Gentoo DOES use developer source, but they ALSO do what you call "heavy patching".
I interpret this "source vs package" debate to be something different: What is the NORM for your distribution, and are you using the OS in ways that were not tested by the vendor's SQA team
For example, ANY of these distros can get borked if you install Ximian on top of them and THEN go back to the vendor for updates. It wouldn't matter if you did it from source or packages.
Same with Alien packages on Debian, or "Redhat centric" rpms on Mandrake or SuSE.
Bottom line is don't mix oil and water.
I agree with your comments about what is good with Gentoo. I happen to like Gentoo and FreeBSD for the very reason that there's a BAZILLION source packages that all have cross-testing against each other. Same for Debian I suppose.
Best thing RedHat ever did for their desktop distro was set it free. They NEVER wanted to be in the business of supporting user-borked desktops when they install random stuff from the net, and they never wanted to manage and QA a large repository. Now it looks like there's a Fedora community (two actually) addressing the package distribution issue. Good for them.
personally I think you are silly if you use packages or ebuilds when it comes to Apache + modules. Your best bet is to just do it from source.
I run gentoo, redhat or FreeBSD, and I never use any of their packages/ports/portage for Apache or MySQL anymore, it just rarely works out right if you have complex needs.
1 Server, 1 Admin - Build from source
5 Servers, 1 Admin - Build Packages and install
1 Server, 5 Admins - Use Standard Packages
5 Servers, 5 Admins - Build Packages with custom names/versions and install
Seriously, I have 7 Admins managing a mix of 160 Servers.
The simplest way I've found to have the best of both worlds, is to D/L the source RPM (SRPM), customize to taste, modify name slightly, rebuild, and distribute.
For instance,
Needed customized apache to support a couple of things we're doing.
D/L apache SRPM
Modify config files with our own patch
modify configure line in SPEC file to suit
modify package name (!Important!)
rebuild
uninstall old packages
install our packages
WA-LA
Advantages
- still get to run up2date/autorpm/fav-update-package with no worries of breaking your own custom stuff
- Know which packages you've mod-ed by running rpm -q -a | grep "myinitials" or whatever.
Disadvantage
- Auto Update doesn't fix the stuff you're behind on...gotta keep up!
Exactly why developers shouldn't be systems admins. all too often source tar balls put things in the most f-ed up places. Atleast when I install a pre packaged debian supplyed .deb that it will fit to the system layout. conf files, documentation, binaries, and libs will all be in the expected places and not where the programmer thought about putting them. Some programmers would rather just run everything out of your home directory.
Paying taxes to buy civilization is like paying a hooker to buy love.
That's easy. Get the source again and build with the same target options as previously. Then "make uninstall" as root.
Isn't that the #1 reason to use packages instead?No, it is not.
There you are, staring at me again.
Packages in a distribution like Debian update and uninstall cleanly, you can build every one from source if you want to, and someone else has worried about (1) testing the binary and (2) getting all the dependencies right.
Build from source if you need the software and no package exists, or if you really, really need a processor-specific version. But for most applications, go with the pre-packaged version: as a system manager, there are a lot more useful things you can do than recompile "ls" on a dozen machines.
That's matter of methodology. I always keep a source tree I have built from, for 1. uninstall ;-) and 2. just for case something went wrong so I can look at the source.
I always uninstall old version only if new version build succeeds. If it fails at runtime, no problem to roll back the old one. This is not so easy with packaged dependencies. With critical ware, I keep even multiple versions build tree around. A single dedicated 100GB drive holds me stuff for all platforms I use (i486, i686, arm), and I still have a 80G in reserve. A CD/RW is my friend, too. And you are right, broadband *is* essential, but even some distros are about 1G of packages updates monthly (Mandrake cooker).
For example, I don't want to waste time compilingNo need to waste your time. Did you try "nice -n 10 make"? I always run builds on lowered priority. Just now, I am building globulation somewhere on the background here...
There you are, staring at me again.
When I build a machine for Linux, I will typically download and install the latest stable build of the Linux distribution which is going to land on the machine. From there, what I do varies. Anything that could affect security or efficient operation of the machine (starting with the OS here -- I always prefer to build my own kernel locally, you never *really* know what is included in a pre-built kernel) I build from source packages. Anything trivial included in default packages from Linux distribution (for example, games) I download as a package -- but with that comment, keep in mind that no package is ever *truly* trivial: if you are building a server that must be secure, you need to know what is in the package you are loading.
But I believe (and from a cursory view at the headlines to the responses here, I think most probably agree), that you have to draw the line somewhere. If you are running a top-secret government laboratory, you ought to compile from source, and not even start from a Linux distribution. If you are setting up a machine for your kids to play games and won't connect it to the internet, would it be worth it to compile from source?
Somewhere in between, there is a line you must draw.
The two cases I presented above are the two absolute extremes. Discounting the trivial game box, I think one should always compile their own kernel for that specific machine (and possibly for the kids' game box, possibly not); if you are building a server, or a machine which needs to have high performance or will see extreme usage, then there are obviously certain things which need to be compiled from source.
But do you need to build (trying to think of a fairly trivial example) more/less/most from source in most cases? Why?
Check the security sources -- are there any security holes reported for that particular package? Do you suspect any may be possible (it is possible to build a trivial game that includes a security hole, but how important is it to you to check the game's source for possible security holes?).
I will not say that it is wrong to compile *everything* from source, that is certainly admirable. But compile a to-do list with all the compiles and all the other stuff you have to do during the day/week/month, and then order all the tasks according to importance.
Now, think about the question about compiling a trivial package from source once again: how important is that to you?
Some instances of compiling from source are going to end up high on your list no matter how you order it, others near the bottom.
Fairfax
You probably won't ever see this because of how late I'm posting... However:
Building from source is great if you want to tweak a system and get it running exactly how you imagine. Be prepared for configuration and all the various issues associated with source builds. I'm assuming that even if you build from source that you are using some sort of package/file management system to alert you of dependencies and file modifications. This is easy to do with binary packages, not so easy managing sources. I regularly rebuild *on my test machines* all manner of software from source, including the kernel, KDE, glibc and a bunch of other libraries.
Now for the problems with source builds:
1) You need a development machine. I.e., you need the compiler tools and libraries. For a regular workstation this is no problem, but you DO NOT want these tools accessible on a server even if they're 'chmod 700' or otherwise locked away. This means you'll build on another machine and create a binary package and... well, you're back where you started except you lost some time.
2) There's no easy way to create snapshots of packages. Differences in libraries and config files can make or break software. The best errors are those that prevent the software from compiling. The worst are those that compile, but errors or weirdness doesn't show up until a month later. Now RPM is much maligned, but it does allow you to keep the build instructions, dependency information, etc.. inside the package. You get lots of control, once you've learned RPM, on where things get installed.
3) Backouts are not as easy. You can often do a 'make uninstall' but this requires the sources be kept around in some cases. Tools like checkinstall can ease the burden, however.
4) Duplication of effort. Source builds are good for customizing, as I mentioned. It's a myth, however, that rebuilding from source will dramatically improve performance except in a few, somewhat rare cases. E.g., rebuilding a 2.4 kernel with a pre-emptible patch can make your desktop faster. Rebuilding a stock 2.4 from kernel.org or your distro's sources will likely not be noticeable.
As the numbers of machines you manage increases, you will find the meaning of the word "control" changes. We only manage a couple of hundred, but the pressure to standardise, as far as is practicable, is a strong one.
Look at the people running clusters, and you can see where that gets to in the end.
The reason we (primarily) use Debian is that the potential architectures for distributing change, and for customisation-with-binary-releases seems to be much greater.
I do ./configure --prefix=/usr/local/pkg_name plus whatever other options, then make. When make finishes, I mkdir/usr/local/pkg_name-version, and ln -s /usr/local/pkg_name to that, then make install.
/usr/local (gnu utils like groff, less, stuff like that) only things like gimp, gcc, TeX, python etc get their own directory. This keeps the PATH sensible.
/var/log/app_name, and tablespaces. I always try to keep /usr/local as static as possible.
I get all my applications in their own directory, and it's only a matter of changing a link to roll back a version or two. It's also easy to copy an app to another host.
Some discretion is necessary here: I just dump a lot of small stuff into
My main OS is Solaris, but I employ this technique on HP-UX, Linux, BSD, whatever I'm working on at the time. Keeps things simple for me, and it's easy to tell someone else just where things are.
The only time I go outside the app dir is for things like logs, which always live in
As for maintining consistency across a network - NFS?