Gentoo On Server Considered Harmful

Siker writes in to point out his blog post — Why Gentoo Shouldn't Be On Your Server — which seems to have stirred up a lot of discussion, including a thread on the Gentoo forums. From the post: "I firmly believe in updating server software only when you need to. If you don't need new features, and things are working, why change anything? If you update anything you will undoubtedly need to update configuration files. You will need to fix things that break in the upgrade process... This is hard with Gentoo. Gentoo wants you to change a lot of stuff. It wants to be bleeding edge."

Comments

Re:This article makes good points.

[lordsilence]

Gentoo has proven troublesome in a production environment.
The problem isnt updating often... it's when you DONT update often.

We had one system which we didnt bother to update. (Dont fix what isnt broken)
Then one day we had to upgrade some of the services.. which in turn required lots of libraries to be upgraded.

In the end, we had to upgrade kernel.. cause libraries didnt support 2.4 kernel.
Stuff change too much in gentoo to put it simple.. It's easier to keep updating often

emerge sync && emerge -u world
Then iron out all config-changes. Find out which undocumented features were changed, which keys to add to startup script etc.

Lesson learnt: Dont use gentoo on production systems. Run it on your desktop computer you play around with...

My post to the gentoo forums

[RyogaHibiki]

(I posted this on the gentoo forums)

If someone is running a server room with many live production systems where downtime must be in seconds per year, they should ALWAYS have a test environment and a production environment. Gentoo makes it extremely easy to produce this setup. Imagine if you will, this setup:

1) Master rsync system (contains the portage sync used by all the systems)

2) Test boxes for each role needed (perhaps you have 3 different kinds of servers, WWW, Mail, DB)

3) Many production boxes

What you would end up doing is creating a fairly generic gentoo install (by generic, I mean hardware independent - like i686 or whatever you feel comfortable that will be supported for the lifecycle of the servers). All production servers are identical to the test boxes at the beginning of this example and have a simple backup of the whole test environments (perhaps a large tarball saved on a separate drive). A new update is necessary for apache so you do an emerge --sync on the master rsync system. Then you rsync all the test boxes so they have the same portage tree. You then run the necessary installs on the test systems to make sure that it works, if it doesn't, then you research why and figure out if its easier to fix after the update, or if the update needs to be done differently, if you need to, you can restore the test system from the backup and start over. After you have all the test boxes running well, you can then rsync the production boxes and reproduce the steps necessary to get them updated.

Once all this is said and done, the production boxes will all be updated successfully (and the updates were tested on the test boxes) and the test boxes will at this point have the same configuration as the production boxes. You would make a new backup of the test boxes and wait for the next time you have to do this cycle. As long as the boxes really are identical, you could even run konsole (or another xterm that allows you to send your input to multiple console windows) and perform the identical steps on all the same type of boxes (sending your update commands to 20 or even 50 servers at once).

I'm sorry, but in any real production environment, I see NO issues with this setup. It may be a bit time consuming if you have a lot of etc-updates to do, but still, the basic update should be painless to that point.

-Jason Pf.

Re:This article makes good points.

[lRem]

Gentoo has proven all right in my production enviroment - and that is an ISP.
First reason, is that you don't have to upgrade those production machines all that often. I sit down and read any security advisory that seems to affect me. And, not surprisngly, there are actually very few remote vulnerabilities that hit Gentoo-hardened. Furthermore, those tend to be in software right in a leaf of the dependency tree, or software I might consider disabling (or limiting to trusted hosts) to the next maintainance cycle.
And there comes it - once in 6 months a massive emerge -uDB world && emerge -uDk world && revdep-rebuild && perl-cleaner (better don't omit the latter two). The system is nicely trimmed down and the build runs on a few machines I have available, so it doesn't take any epic amounts of time. In fact, I even seen it done within half an hour. Still, back when it did take a better part of the day, I simply run the first command a day earlier and then used the packages, what of course is a breeze.
Finally comes the configuration updating. I haven't seen it easier anywhere. The first nice thing is that Gentoo developers don't toy around them - they usualy come as the original software developers intended. But what really makes a difference is the toolchain. By far, I have seen no other distro that automagicaly within the standard package system uses revision control for configs. And then, it gets the trivial updates done for me, and puts me into vimdiff anytime any decision is required.
At most times, this means no downtime at all, as everything runs smoothly. In case of a kernel upgrade, or anything going wrong (once till now), we still have redundancy. So there are no visible drawbacks of using Gentoo on those servers... Unless I, and my boss, am missing something.