Greg KH Favors Rolling Release Distros

jones_supa writes In an interesting Google+ post, the lieutenant Linux developer Greg Kroah-Hartman mentions him fully moving to rolling-release Linux distributions: 'Finally retired my last 'traditional' Linux distro box yesterday, it's all 'rolling-release' Linux systems for me. Feels good. And to preempt the ask, it's Arch Linux almost everywhere (laptop, workstation, cloud servers), CoreOS (cloud server), and Gentoo for the remaining few (laptop, server under my desk).' What's your experience? Would in the current situation a rolling-release operating system indeed be the optimal choice?

Comments

Uh

[Chess_the_cat]

Don't bother clicking the link. The *entire* post is contained in the summary.

Re:Uh

[AikonMGB]

Shit, I accidentally RTFA then =(

Re:Uh

[Jose]

hah, what a newb.

I really wish Greg would have told us which distro he is using though.

So much for stability and uptimes...

[TWX]

There was an era, probably inherited from the big-iron computing model, where we strived for stability and long uptimes. We didn't install things that we didn't need (with the exception of Fortune perhaps) and locked-down the box at the network stack. Granted, it required a lot of knowledge at the beginning to make sure that the box was indeed secure, but we were proud of setting up a good, usable box that didn't need a lot of maintenance after the fact.

I guess that era is now gone, with rapid-release and lots of little things constantly needing the system to restart.

Re:So much for stability and uptimes...

[gstoddart]

Some of us still work in environments where constant restarts are strictly not allowed, and software which expects to be on a constant release cycle is shunned.

We had a vendor once, who wrote a component for a large enterprise system ... they released builds pretty much weekly and thought that was grand.

We filed a bug once, and they said "we don't support that version because it's a month old, and therefore 4 versions out of date, you need to upgrade". We said "you'll be hearing from our lawyers because we don't take a prod outage every week just for you idiots". Needless to say, they quickly realized they were going to lose that fight.

Sorry, we need a lot more stability, and we don't care if you think you're on an agile cycle. It takes around two months to promote something through to Production ... we simply don't care that you want to build weekly.

Not all places (specifically most regulated industries) have the ability to have stuff constantly changing underneath them, and they certainly haven't got the patience for some company who thinks a product lifecycle is measured in weeks.

Continuous releases often have the effect of making your customers your beta testers. And we can't do that for you.

Re:So much for stability and uptimes...

[jythie]

While many consider it gouging, this is why I like support contracts. A nice signed piece of paper saying 'yes, we WILL support the version you are using even if our active development moves on'.

Re: So much for stability and uptimes...

[Anonymous Coward]

Using separate apps and libraries which have strict and unavoidable dependencies between them isn't "modularity".

Modularity requires those components to be very loosely coupled.

For example, GNOME consists of many separate libraries, apps, and scripts, but it isn't modular. Installing just one small GNOME app means you have to pull in tons of libraries and other apps, because they are tightly coupled.

Systemd is similar to GNOME. It's an all-or-nothing situation, which obviously isn't modular.

Traditional UNIX software generally is modular. I can easily change my shell, for example, without affecting the other software on the system. I can even install a different C compiler, and none of the other software on the system would even be aware of the change. That's true modularity.

Re:So much for stability and uptimes...

[radish]

You know it's interesting. I used to work in finance. We, like you it seems, had a very locked down production environment with huge amounts of testing - pushing builds through multiple stages, reviews and signoffs. Once every month or so we'd shut everything down for a few hours in the middle of the night and roll the world forward. Stability was everything. Downtime was OK if scheduled, a disaster if not.

Now I work at a web company. We push to prod multiple times per day. There's a process, there are reviews and approvals, but it all happens much more quickly and at a more granular level. Change is constant but small, as opposed to infrequent but total. What's more we're a 24/7 operation so no downtime (as visible to the user) is acceptable. We simply can't schedule a few hours to do our rollout - everything has to happen live.

You know what I've noticed? We're no less reliable, overall, than the bank was. Yes we have issues, but they tend to be noticed, and fixed, much much faster. When you change everything all at once you run the risk of not being able to figure out what broke when inevitably something does. Rollback is painful because you have so many interdependent changes - in the end you have to pull the whole release to avoid one small issue in a single module. When you roll frequently the scale of change is small so isolating the bug is trivial, and rolling it back the same. Now of course there are huge differences in risk when you're handling people's money vs their cat photos, but I think the view that people working on an agile schedule don't care about stability, and that the only way to achieve stability is through reducing the frequency of change, is demonstrably wrong.

G+

[fph il quozientatore]

The real news is, someone is still using Google Plus.

Situation Dependent

[jythie]

This really strikes me as something that is going to heavily depend on what the systems are actually doing, how tied to the distro-supplied software the usage is, and how often the releases are.

Even within 'rolling release' distros there is a huge variation in exactly what that means in terms of changes, updates, frequency, which parts are rolled vs versioned, user control over backdating. This combines with a bit of a matrix of use cases for one to find exactly how much manpower using such a distribution within an organization will eat up. So yeah, 'it depends' pretty much sums it up.

How critical is stability?

[davidwr]

For a machine that you would just blindly take updates for anyways, rolling releases are probably convenient.

For mission-critical systems where every change should be tested first, it's probably a bad idea unless rolling back is very easy, as it might be in a VM-with-easy-snapshots environment.

Re:How critical is stability?

[DigiShaman]

At what level? Rolling back a VM means not just rolling back the OS, but Apps as well. What happens if you already upgraded the SQL database? You can't exactly roll that back too. Well, at least not without a restore. When dealing with database driven application in a clustered environment, you simply can't pushed the VMWare "oops" button. Doesn't work that way.

I use Gentoo - but not for much longer

[QBasicer]

I've been using Gentoo for many years, and temporarily switched to Funtoo on my personal laptop. I've since graduated and don't spent nearly as much time on my laptop as I used to, which these days mainly runs MythTV.

I don't think I'd continue with Gentoo - it takes too much time to sort through updates, figure out which packages need to be masked, etc. I'd rather go to Arch next, although I was considering Debian unstable.

Recently, my video card stopped being supported by the newest nvidia graphics, and the newer versions of Xorg weren't compatible. My masked list is growing as more and more packages have deeper dependancies on newer versions of Xorg. I always figured Portage should honour my masked packages and keep everything at the latest version without stepping on my masked packages, but it wants me to do everything manually. If package 1.2.3 is incompatible with my Xorg, I'll mask 1.2.3 and newer. There is a slight chance, however, that 1.2.4 will be compatible, but it doesn't matter, since Portage made me masked out 1.2.3 and newer, I'll never even know.

Good for developers ...

[gstoddart]

I think rolling releases are good for developers, and gives you that whole agile thingy ...

But really what it instills is a culture of "almost got it" where you'll run the risk of breaking your user's systems and then just say "whoops, we'll fix that next time".

I think it leads to sloppy release engineering (because, after all, it's just a build), and will be fundamentally incompatible with how companies need to do IT.

And every time I see Firefox telling me "It is strongly recommended you upgrade to this version" what I really see is "holy crap, did we inject some garbage in that last one".

I think in general the "continuous release" says "we're not worried that people in the real world can't do this, and we don't care ... we'll fix it on the next release ... maybe".

So, for your personal desktop, or a sandbox, or a toy ... sure, have at it. But for a real machine, doing real work ... I think "continuous release" is a terrible idea.

Because in the real world, we're not prepared to patch Prod system just because you committed some new changes -- we have bigger issues to deal with than constantly updating software to keep you happy.

I should think nobody in a corporate environment is a fan of that. And if you're a small shop of 20 people who are risk takers ... you're not in what I'd call a corporate environment.

Debian SID

[raxx7]

I've been using Debian unstable in my personal computers for years. Occasionally, something breaks.

But I prefer the long term support of Debian stable and CentOS for internet facing servers and lab workstations.
Here, it's important to be able to get security fixes without fear of breaking anything for years.

Re:Debian SID

[jythie]

I think the 'for years' part is where the disconnect between the two professional use cases (or camps) tends to happen. The people really pushing rolling distributions are not really thinking about production systems that will be running for the next 5-10+ years, but instead on rapidly changing stuff that you do not have to plan more than a few months in advance.

Home USE !=Business Use

[bigdady92]

Congrats you can upgrade your latest hot shite box to the latest hotness. Fantatsic. Now what about those servers that have millions of people trying to contact your business through? Hell to the No.

You want your systems to be running stable, known working, and reliable code. Who cares if it's version 10 and not version 10.0.4134. Let the dev monkies play with the updates in the background and when a service release is out test it further.

Unless there is a positive gain (security, feature release, or annual patch) then the old code is just fine. It works, don't touch it, leave it the hell alone and go play with your crap in your lab.

Another reason I hate the DevOPS movement. Combines the worst of habits of a Dev Monkey and a System Admin.

Arch... Ugh.

[Anonymous Coward]

Arch breaks. Often. Breakage is the trouble with rolling release distributions, and an intolerable problem for anyone not wanting to spend the time un-breaking things.

Loyal but naive Arch users are always quick to defend it, "my system has never broken" "you must be doing something wrong" etc. but these discussions are always about semantics. Just because it's a one-liner to fix doesn't mean that it isn't broken. If it requires my attention to keep working, then it's broken. Just because it is fixable doesn't mean I want to spend time fixing it.

Arch is a great way to learn Linux, and the Arch wiki is a great resource not exclusive to just Arch. But you'd have to be out of your mind to use it for anything in production. The Arch FAQ makes it pretty clear: YOU, the user, is responsible for keeping your system updated, functional and stable; but the more packages you have installed, the more likely you are to get broken when upstream updates something.

Also from Arch docs:
Warning: Do not be tempted to perform partial updates, as they are not supported by Arch Linux and may cause instability: the whole system should be upgraded when upgrading a component. Also note that infrequent system updates can complicate the update process.

Translation: You want to update package foosicle-1.2 to foosicle-1.3 because it has a security problem. Oh, you don't want to update X, Firefox, KDE, and the kernel? I hope you do want instability then. BTW, stay on top of your updates unless you want to get really hosed.

No thanks.

I use Ubuntu LTS releases on my computers at work for three reasons:
1. Reading the Arch wiki to un-fuck Java after I updated my system to fix a security issue for a different package is not a good use of my time.
2. Not a good use of my time to compile from source because the distribution ships with something ancient or doesn't have it at all (I'm looking at you, RHEL).
3. Will keep getting updates for the lifetime of the hardware.