Debian Upgrade May Cause Serious Breakage

daria42 writes "Debian developer Bill Allombert has e-mailed the Debian community saying he estimates about 30% of users upgrading from Debian Woody to Sarge will suffer 'serious breakage'. Allombert says the upgrade process suffers from a number of bugs reported before the release went live several days ago. Chief among the problems, he said, were cyclic dependencies and the fact that software installation tool apt depended heavily on the changing C++ libraries. Allombert wants developers to test the upgrade cycle continuously during development and not just during the freeze period just before release."

Comments

Evidence of problems with packaging systems

[AKAImBatman]

Chief among the problems, he said, were cyclic dependencies and the fact that software installation tool apt depended heavily on the changing C++ libraries.

Let this be a lesson to those of you who claimed that "APT is unbreakable." There's no such thing as an unbreakable technology. There is however, such a thing as a robust technology that resists failure. As packaging systems go, APT is fairly good. However, my belief is that packaging systems are inherently flawed.

What you want in an OS, is a method for determining the precise core upon which you can base your applications on. Such a core would effectively be an immutable set of system APIs that cannot be changed. The upshot to this situation is that the given system is verifyable. i.e. I can have a script go through and ensure that everything that should exist does exist. From that information, I can then do a delta to find out what exists that shouldn't exist.

This is in direct opposition to a packaging system that builds an OS out of inter-dependent components. The problem with such a strategy is that using inter-dependent components only works if you're building from scratch. As anyone who has managed a version control system can tell you, things get extremely complicated (and tend to require manual intervention) as soon as files start branching. The same thing happens in packaging systems as soon as you start doing upgrades to individual components. Soon you find yourself with a mess of mismatched dependencies which require constant manual intervention to solve. Not a good situation.

In the case of a defined core, you can simply wipe out the old core and replace it with the new one. As long as testing has been done to ensure that the new components are still backward compatible with old software, everything should work fine after the upgrade.

Food for thought, anyway. To the Debian team: Thanks for the new release! Even if there are some growing pains, it's still nice to see you back in the game. :-)

Re:Evidence of problems with packaging systems

[Tharkban]

Give it a rest.

The Linux Standard Base is dead.

There is too much freedom for even the distributions to make cores effectively. Debian doesn't develop the software, they package it. They have no direct control over compatibility issues between versions in their software. This makes their job a whole lot harder than in commercial OS's where one entity controls both the core software and the packaging.

They also don't have the resources to making security patches for every package without upgrading to a newer version of said package (i.e. backporting). They really do a phenominal job given their constraints.

Re:Evidence of problems with packaging systems

[AKAImBatman]

They also don't have the resources to making security patches for every package without upgrading to a newer version of said package (i.e. backporting). They really do a phenominal job given their constraints.

I agree wholeheartedly. I'm not attempting to "diss" the Debian distro or its maintainers. I'm only pointing out that the packaging system is beginning to strain under the pressure of so many packages. The complexity of the package system is quickly becoming too difficult to maintain. Especially since the packaging system mixes the core system APIS with the user applications. (Always a recipe for trouble.) Thus it is time to start thinking about something new.

The Linux Standard Base is dead.

The LSB was always about the "least common denominator" and not about "the most usable configuration". For what it was, it wasn't too bad. But a real standard at this point would have to define a lot more libraries, although perhaps at more of a library version level than trying to force the individual APIs.

With that in mind, I don't think that such a standard should be attempted across all distros. For one, that would limit their ability to be different and provide new competitive services. For another, it tends to be better to allow a few different standards to compete before you attempt to pick one or two out of the fold. For example, there used to be many standards for Linus base distros. Now all distros tend to fork from either RedHat or Debian. Standards thus emerged.

The same thing should happen today. We should see different distros attempt differing solutions to the issue and see which ones work best. Symphony is certainly one of the most interesting, but mostly because it's the first attempt to break away from the current designs that Linux is stuck in. :-)

Re:Evidence of problems with packaging systems

[KiloByte]

The only issue is: if you don't read the freaking release notes, you will have problems. The apt in Woody is broken. The release notes say that you need to update it first, to let it handle circular dependencies.
The only fault of Debian is not putting this in a bold enough font.

Also, this breakage gives us a yet another reason to bash C++ as a poor excuse for a language :p

Re:Evidence of problems with packaging systems

[Qzukk]

I mean, come on, there were 10,000 pairs of eyes looking at the source code and fixing bugs before it was released, right?

Right. And they fixed the bug, and told everyone that apt was broken and to upgrade to the fixed apt before attempting to upgrade to sarge.

And nobody listened.

Re:Evidence of problems with packaging systems

[runswithd6s]

They also don't have the resources to making security patches for every package without upgrading to a newer version of said package (i.e. backporting). They really do a phenominal job given their constraints.

I'm not sure what weed you're smoking, but Debian backports ALL of their security fixes from upstream software to the version packaged in stable. Really, consult the Debian Security FAQ for more details.

To whom it may concern.

[ShaniaTwain]

Everything is falling apart. You may experience some discomfort. Just thought we would let you know. have a nice day.

Typical Debian!

[JimDabell]

Obviously this was a rushed job. Typical Debian, always cutting corners, never taking the time to do things properly :P.

I've upgraded 6 boxes without problems.

[khasim]

What, specifically, are the apps that will cause the problems and how does he determine that 30% of the boxes out there will have those apps?

I've upgrade 6 boxes and have not had a single problem on any of them. They run a combination of Apache, perl, python, mySQL, php, bind9, DHCP, etc.

If there is a circular dependency problem on an app, but no one uses that app, then there won't be any problem upgrading.

Re:Mixing Lilo and some kernel configs not nice ei

[SpiffyMarc]

My brain exploded trying to parse this sentence.

And we wonder why we aren't taken seriously by management. ;-)

so long and thanks for all the FUD

[costela]

This is FUD, even by Slashdot standards.

The problems do exist, but the "severe breakage" described does not implicate unbootable machines or unusable software. Cyclical dependencies mostly mean the algorithm used to select packages for upgrade or instalation will not run as expected and probably leave the problematic package on hold.

This is not a new problem and affects Debian mainly because of it's distributed and loosely coupled model of organization, where integration problems can go by unoticed for quite some time.

The original mail intended to push more developers into taking action about these integration errors and make sure the upgrade paths are always clear, which is a very big and important task.

I, for one, hope his message doesn't fall on deaf ears, but also hope it doesn't generate more FUD like this.

Apt Would be Unbreakable

[Greyfox]

If they statically linked it. Which they should really do for a base level of core utilities anyway. I've been burned by library upgrades and crippled recovery processes several times in the past because the correct libraries were no longer available. For something that might have a library pulled out from under it like apt, it really makes sense to incur the size penalty so that you never have to worry about it dying on you when you replace system libraries.

Re:Apt Would be Unbreakable

[Spazmania]

That's not quite true. For example, the staticly linked apt in a previous upgrade could run in to trouble looking up DNS entries. The problem? /etc/nsswitch.conf got upgraded and the staticly linked DNS library didn't understand some of the new options.

However, offering a staticly linked apt would probably have helped.

Re:Update took me two days ... grrr

[BeBoxer]

Suddenly apt-get dist-upgrade didnt do anything good, I had to do an apt-get -f install multiple times until the dependancy stuff was sorted out. In the process, some packages (notably apache and ftpd) were simple de-installed and I had to re-select them manually.

I can't say for sure that it would have helped, but the instructions specifically say to use aptitude because it handles dependencies better that apt. So while I feel your pain, I'm not sure it's a valid complaint.

Re:Mixing Lilo and some kernel configs not nice ei

[Qzukk]

SATA changed from IDE subsystem in 2.2 and early 2.4 to libata (and therefore part of the SCSI system) in 2.4 and 2.6

Re:I was waiting for Sarge but then came Ubuntu.

[varmittang]

Ubuntu is more of a desktop, latest updates type distro, while Debian is a strong, server type distro. So which do you need, depends on if you want a desktop or server, make your choice.

This is true.

[cdn-programmer]

I attempted an upgrade from woody to sarge about a month ago and it broke my system. I have 1000's of zombies running around. This shows up as a defunct process. Its not the end of the world mind you but you can't kill a zombie since it is already dead.

I have reported this and warned that there will be a lot of folks with broken systems. I was very surprised to hear that sarge went stable before this problem was sorted out.

A sarge install from scratch however is fine. Its just the upgrade that is broken and in more than one place.