Autopackage Universal Package Manager

nanday writes "I currently have an article on Linux.com about Autopackage. Autopackage is developing a universal package manager for the GNU/Linux desktop, separate from the package management for the system. It includes installation for individual users, a lot of concern for interface design and documentation, and some ideas about the future of package management that are sure to raise some debate." From the article: "Besides ... technical problems, the Autopackage team believes that managing system and desktop software together is a mistake. It requires developers to pay attention to desktop applications that are of secondary importance to them, and confuses end users with problems about dependencies and upgrades." Linux.com is a sister site to Slashdot. (say that three times fast)

Comments

Great Idea

[nbSouthPaw]

As someone who is new to linux this is the one area I struggle with. While not difficult installation of software on linux is quite different even among the different distributions. The autopackage software could make desktop solutions quite a bit easier for those of us that dont want to mess with system software very much but would like more control over desktop software.

Re:Great Idea

[Arandir]

"Native package managers' dependency detection depends on a database. Autopackage, on the other hand-detects dependencies by actually scanning for them."

Such a simple idea, such an absurdly simple idea. Yet it's one that 9 out of 10 distros just can't manage to get right. Building one library yourself should NOT break your entire package management system. A minor bugfix release to a library where no ABI has changed should NOT necessitate an update to every application that uses it.

Re:Great Idea

[/ASCII]

I don't know enough about the package databases to know if there is a simple way around your first problem, but the second one is only a trouble in cases of lazy packagers making bad dependancy specifications. Package dependancies should be to the relevant ABI version of the library, which is usually stable across minor version numbers.

Re:Great Idea

[Arandir]

but the second one is only a trouble in cases of lazy packagers making bad dependancy specifications.

But that's the whole point! 9 out of 10 package makers are lazy. While I don't agree that autopackage is the solution, they are correct in pointing out that there is a problem.

Re:Great Idea

[Anonymous Coward]

Usually it's not a problem. Packages should depend on MAJOR version numbers of libraries (named as *.so.xx.yy.zz). Least significant number is usually used for non-breaking bugfixes! There is ofcourse symbolic link *.so which is for apps that use generic name or any version. This way you can have multiple versions installed. Apps should look first for specific version number for which they were built, then for others. Windows is greater mess than this.

But system has many drawbacks.
What's wrong with with having all it's non-shareable data in specific subdirectory? E.g /usr/local/app/*appname*. Including custom versions of libraries which can be conditionally installed there if system is missing them after all.

Updating application is a pain (especially with rpm), detecting user changes and prompting to leave or remove/overwrite those files at uninstall or update is still just a dream. Undoing configuration changes can be tricky. Usable packager GUI isn't a big design issue though. Repair function is handful, so you can keep your config but repair accidentaly damaged installation. Those are few of ideas that just came on my mind.

Instaling and sharing apps without need to be root should also be addressed. Idea is installing it to a specific /home/shared or at least /usr/share, which specific or all user groups can access and write.

Kernel update and driver updates, well that's completely different problem, but without it you can't expect significant number of windozers to love linux (unix) ,because you can switch drivers with only a reboot there.
Problem here is you have to update to latest kernel to get new drivers, and it can mean breakage to essential system packages (e.g. hal, udev,hotplug daemon, cd-burning apps etc.). Distros often don't provide updates for major kernel numbers, so you are stuck with vanilla, which can differ significantly from distribution specific custom-patched kernel (and they often don't provide changelog)

Not exactly loved by the distro people...

[keesh]

Autopackage is not exactly loved by the distro people. See commentary from Gentoo, Debian, more Debian... Might be wise to keep those remarks in mind when considering using Autopackage packages on a distribution...

Re:Not exactly loved by the distro people...

[Stormy Henderson]

I agree. It's an interesting and potentially great idea to separate the system from the desktop, but poorly designed & implemented.

Re:Not exactly loved by the distro people...

[Nevyn]

The biggest problem IMO is that their claim that "the Autopackage team believes that managing system and desktop software together is a mistake", this is just obviously wrong. Every Linux distro. is essentially component based, it's just not possible to say ok "here" is the line in the sand.

I'm just as likely to want to upgrade postgresql, as get "muine" ... and if I upgrade muine, what about my installed plugins? ... what about if I have a distro. muine installed and want a plugin that requires a newer version of muine?

Also AFAICS their security errata story is basically non-existant atm. ... which is the final nail in the coffin.

Re:Not exactly loved by the distro people...

[FooBarWidget]

The plugins should be upgradable too. But what prevents you from doing that?

The problem is that, unless you make a distinction between system and desktop, the user will be presented with a huge list with thousands of packages. If my dad wants to upgrade mune he really isn't interested in seeing glibc and qt in that list. It's more a user interface problem.

Re:Not exactly loved by the distro people...

[abigor]

Phew, those are some fairly harsh commentaries. However, it would be nice if the bigger distributions would cooperate with the autopackage guys, simply because there is no way in hell any given distribution will have every piece of software everyone might want. What if Adobe wants to distribute Photoshop CS on Linux? I guess it's possible to do it like Crossover on Gentoo, which has an ebuild that installs the downloaded binary. But I don't think that solution scales well.

Re:Not exactly loved by the distro people...

[FooBarWidget]

They don't like us because they want to centralize software packaging. Don't just blindly assume we're evil just because they're critical to us. Read what they write. For example:

"To even unpack the package, you must run arbitrary code supplied by an untrusted source."
Untrusted source? Is the upstream developer an untrusted source? If he cannot be trusted, then why would one trust third party Gentoo packages?

"They install directly to the live filesystem."
We have this little feature called "installing to $HOME". It won't touch /usr if you don't want it to. How hard is that?

"They do not support configuration protection."
We backup stuff if they're being overwritten.

"They can clobber arbitrary files on uninstall."
No proof. Bug reports please, because we sure haven't received any from our users.

"The entire format is completely unportable and highly tied to x86 Linux systems."
The reason why that is so is because none of the developers have anything but x86 systems. It's a bit hard to support PPC if we don't have such a machine, right? We already have some stuff in place to make sure x86 packages aren't executed on other architectures.
But that aside: let us go back to the original problem: software installation isn't easy enough for the average user. And what architecture does 99% the average user use? x86! Now there's the main reason why we focus on x86: because that's where our target group are! The software installation problem is pretty much unique to x86 Linux.
Non-x86 architectures are usually servers, not desktops. They don't need autopackage, so why should we worry about them? PPC users most likely use MacOS instead of Linux, so there's not much point in supporting that either.

As for joey's blog: the only thing he's complaining about is that at the moment you cannot programmatically extract files from a .package. Average users don't even care about that! They just want the damn software to be installed and to work!

Yeah I haven't replied to everything but you get the point.

Re:Not exactly loved by the distro people...

[FooBarWidget]

Believe me, we have tried. Unfortunately the discussion often boils down to:
- We (= the distro guys) don't like proprietary software so its not our problem.
- We don't care about inter-distro compatibility. Not our problem.
- RPM/DEB rules all. Everything else is evil.
Really, we tried to negotiate. So far all attempts failed.

Re:Not exactly loved by the distro people...

[Nevyn]

They don't like us because they want to centralize software packaging.

Nobody wants that, but it's basically required. You can't solve the "I need to install mod_foobar, which requires Apache-httpd-2.2.0" problem without understanding what Apache-httpd is and what version you have, and how you get from here to there ... which requires a single way to query all software. This doesn't mean it can't be distributed, like DNS, just that you can't pretend you can have two roots and everything will be fine (or even blame those nasty root server operators for telling you otherwise when you try).

Untrusted source? Is the upstream developer an untrusted source? If he cannot be trusted, then why would one trust third party Gentoo packages?

Say you're installing a GNOME theme, it's basically just XML+pngs ... then assuming the installer is secure, there is no reason you can't just install this "package" from anywhere and try it out. The same is true with installing backgrounds, or documentation etc. (think CIA. world factbook).

With aribtrary binary applications this is somewhat muddier, but there is still a deliniation between trusting the package and trusting the package installer (the later of which likely needs more privilages).

For instance "conary" is one of the newer package management ideas, and to them packages are basically just collections of files which are tagged (Ie. doc/bin/shared-library). Installing/removing a package has basically zero security concerns.

Re:Not exactly loved by the distro people...

[Nevyn]

The problem is that, unless you make a distinction between system and desktop, the user will be presented with a huge list with thousands of packages. If my dad wants to upgrade mune he really isn't interested in seeing glibc and qt in that list. It's more a user interface problem.

And you still have to solve this problem when I distribute an epoll using application to a system using a glibc that doesn't have the epoll symbol. And this is such an obvious UI issue, I dearly hope you have a better example than this for why you screwed the pooch.

If X requires Y, you can't just pretend otherwise and say "not our problem we just do desktop apps".

The plugins should be upgradable too. But what prevents you from doing that?

It's the same problem, you have X provided by you and Y provided by the distribution ... you can either: 1) hope version of Y is compatible. 2) upgrade Y when required. 3) always provide your own version of Y.

Only option 2 works, and autopackage seems to think that not doing option 2 but a mixture of 1 and 3 is a feature. Back in the real world bugs aren't features.

Re:Not exactly loved by the distro people...

[FooBarWidget]

"With aribtrary binary applications this is somewhat muddier, but there is still a deliniation between trusting the package and trusting the package installer (the later of which likely needs more privilages)."

Not so with autopackage. You can run autopackages as user and install to ~/.local. You don't have to give it root access if you don't want to. And as autopackage is open source, you can check the source code for trojans. Autopackages are tarballs with a shell script header. Anyone can check the shell scripts for trojans.

"For instance "conary" is one of the newer package management ideas, and to them packages are basically just collections of files which are tagged (Ie. doc/bin/shared-library). Installing/removing a package has basically zero security concerns."

I'm not familiar with Conary, but do they take care of desktop integration stuff like menu items and icons? What about things like updating the icon cache? (If you don't do that some icons won't appear in the GNOME menu) Or the MIME cache?

And does Conary research inter-distribution binary compatibility? As far as I know we are the only group that have good knowledge about binary compatibility problems and actively try to solve them. Without a solution for binary compatibility problems, you cannot make inter-distribution packages no matter what packaging system you use.

Re:Not exactly loved by the distro people...

[FooBarWidget]

"And you still have to solve this problem when I distribute an epoll using application to a system using a glibc that doesn't have the epoll symbol. And this is such an obvious UI issue, I dearly hope you have a better example than this for why you screwed the pooch."

In this case, yes. But usually this doesn't happen. Remember that autopackage targets desktop applications: i.e. the stuff that average users cares about, not server applications. I don't know many desktop applications that use epoll or other OS-specific APIs - they usually use portable APIs like the GNOME/KDE stack.

But, even if you must use epoll, there's still a way to solve that. As autopackage installation scripts are script-based, you can just do this:
1. Compile two binaries: one with epoll support, and one without (falls back to regular poll).
2. Put both of them in your autopackage.
3. Autodetect at runtime whether glibc and the kernel supports epoll.
4. Install correct binary.
It's more work, but it's possible. And it sure makes things a lot easier for the average user. In fact we use a similar method to solve the C++ ABI breakage problem.

"you have X provided by you and Y provided by the distribution"

This situation is currently impossible, as the native package manager will think that X isn't even installed, simply because it isn't in its database. For example, if you install gtk from source, then install the Gaim RPM/DEB, then RPM/DEB will complain that gtk is not installed even though it is.

Re:Not exactly loved by the distro people...

[Anonymous Coward]

Huh, funny. First I've heard y'all have come sniffing around portage.

I'll remind y'all that you are trying to approach others to change their ways, onerous falls on your project to demonstrate the gain instead of assuming everyone is going to agree with you and jump in bed with y'all.

Re:Not exactly loved by the distro people...

[Anonymous Coward]

And does Conary research inter-distribution binary compatibility? As far as I know we are the only group that have good knowledge about binary compatibility problems and actively try to solve them. Without a solution for binary compatibility problems, you cannot make inter-distribution packages no matter what packaging system you use.

That seems like a rather large bit of hubris considering binary distros were already dealing with binary compatibility issues, long before autopackage existed.
Source distros hit the same thing anytime libstdc++ breaks abi also.

Re:Not exactly loved by the distro people...

[FooBarWidget]

"That seems like a rather large bit of hubris considering binary distros were already dealing with binary compatibility issues, long before autopackage existed."

I mean inter-distribution binary compatibility issues. Do distributions deal with that? So far I haven't seen any distribution who cares about being able to run their binaries on other distributions, or vice-versa. If you know one please let me know and I'll applaud them.

Re:Not exactly loved by the distro people...

[Anonymous Coward]

I mean inter-distribution binary compatibility issues. Do distributions deal with that? So far I haven't seen any distribution who cares about being able to run their binaries on other distributions, or vice-versa.

ABI compatibility issues, whether cross-distro or contained within a distro are the fundamental issue- it's disenguous trying to imply that autopackage is the only one working on the problem when distros have to do the same thing internally to keep their packages from not puking.
Hence the hubris comment.

Re:Not exactly loved by the distro people...

[petermgreen]

generally the problem is abi compatibility tends to be one-way, you can run apps built on an older system on a newer one without too many problems. the other way though you are most likely going to fail.

debians soloution to this is basically a combination of forced upgrades (when a package is built afaict it is automaticially made to depend on the versions of libs that were on the build system) and building stuff seperately on every version of the distro.

Re:Not exactly loved by the distro people...

[Anonymous Coward]

1. Compile two binaries: one with epoll support, and one without (falls back to regular poll). 2. Put both of them in your autopackage.

Granted, do what you can to make things work (even if kludges suck), but this is something of a shitty general solution- openoffice is large enough as is, without jamming abi compatibility crap into it's pkg. Nor is it the only package that's going to demonstrate it, it's just the easiest one to point at.

3. Autodetect at runtime whether glibc and the kernel supports epoll.

This solution seems rather questionable considering plugin frameworks are known for doing the dlopen route (kde libraries anyone?). Haven't looked into how autopackage tries to address this case, but rather doubtful this problem can be solved from on the fly detection unless autopackage becomes aware of all frameworks, and determines on it's own which plugins a package needs/would open come runtime.

Re:Not exactly loved by the distro people...

[IamTheRealMike]

Rephrased, "why think for myself when other people can tell me what to think instead".

Re:Not exactly loved by the distro people...

[IamTheRealMike]

Huh? Programs that use epoll can either use relaytool, direct dlsym or embed the syscalls directly (as Wine does). This is a non-issue, why is it even being debated? Programs don't grow dependencies by magic, developers add them, and it's up to the developers to decide what the minimum requirements are of their app. The lower they are, the larger their userbase can potentially be.

Re:Not exactly loved by the distro people...

[IamTheRealMike]

I think the rationale behind the project has been explained clearly enough time and time again. What's more, often it's end users asking for this distro support, not us. Shouldn't what end users want be a pretty big hint?

Re:Not exactly loved by the distro people...

[Anonymous Coward]

I think the rationale behind the project has been explained clearly enough time and time again. What's more, often it's end users asking for this distro support, not us. Shouldn't what end users want be a pretty big hint?

You're dodging the question of where the support goes- if a distro puts something out, the mess of it falls on their head. Autopackage's play at being primary package manager stomping all over the actual primary package manager- end result is the support for when stuff breaks falls on the distros head.

What the user wants is something the distro tries to address- just because autopackage comes along and states their desktop orientated, in no way makes them better, nor worse then the distro's focus of full integration and packaging of software.

Basically, cut the rhetoric (yes I know this is /.), and stick to the technical issues people keep pointing at for the implementation, not falling back to "it's what the users want" which is rather hard to back up. If users actually wanted this, you would see an actual push within distros to support this, rather then a vocal minority demanding/asking for it.

Stated it elsewhere, and restating it- the intention isn't to kick at autopackage, but the rhetoric thrown when technical issues are raised is tiresome.

Re:Not exactly loved by the distro people...

[Nevyn]

If a random guy from the street tells me my house is a fire accident waiting to happen, I might be curious and ask for reasons or just say "I don't think so" and ignore him. If a guy trying to sell me fire proofing comes up to me and says that I'm going to be very suspicious, and likely not believe him no matter my opinion. If the local fire marshal says it ... I'm likely going to believe him.

Thinking for yourself is all well and good, but when someone who knows what they are talking about says X then "I don't think so" is not a good response. In the same way when you say I need autopackage, given only a cursary assessment of said code ... I'm going to take my pinch of salt with that, and I don't need to think very hard to do so.

Re:Not exactly loved by the distro people...

[Nevyn]

Not so with autopackage.

Again, we know that and as we just said the fact that autopackage doesn't allow seperate privilages between the package and the package installer is a _bug_ not a _feature_.

You can run autopackages as user and install to ~/.local

Yes, well done ... in a few cases I might like to do that. However 99% of the time I'll want them outside my home dir. And that still doesn't excuse you from not seperating the package installer from the package. Again with a font/background/sound/text package there should be zero security concerns, the package installer will be trusted to put things in the correct place (in my home dir. if I wish) and the package itself can't do anything. Not so with your shar reimplementation.

And with SELinux it should be possible to limit the pacakge and the package installer even further, again this isn't possible with your broken installation methodology. You are incouraging people who don't know any better to run bash scripts from the 'net, if you think people aren't going to call you on such a stupid idea, you need to think again.

Re:Not exactly loved by the distro people...

[FooBarWidget]

"end result is the support for when stuff breaks falls on the distros head."

No. People are supposed to report bugs to us or to the upstream developers.

"What the user wants is something the distro tries to address- just because autopackage comes along and states their desktop orientated, in no way makes them better, nor worse then the distro's focus of full integration and packaging of software."

You are correct. But the fact is, distros can't package everything. If you look at Linux forums you'll see that people often complains about:
- My package is not in the repository.
- My package is in the repository but isn't the latest version.
- The package in the repository is broken. (*cough* Fedora Extras galeon 1.3 *cough*)
And this is where autopackage jumps in. They provide an extra choice for people who aren't satisfied with the native package management system.

Re:Not exactly loved by the distro people...

[FooBarWidget]

"Again, we know that and as we just said the fact that autopackage doesn't allow seperate privilages between the package and the package installer is a _bug_ not a _feature_."

Does RPM/DEB provide that seperation? Last time I checked (today), RPMs *must* be installed as root. As the preinstall and postinstall RPM scripts can do whatever they want, including rm -rf /. I also haven't found a way to install DEBs as non-root (on Debian and Ubuntu). Can dpkg prevent DEBs from wreaking havoc in pre- and postinstall scripts?

Autopackage is *open source*. If you don't trust us, you can inspect the source code. It's there, readable with vi, emacs, less or whatever your favorite text viewer is. You can easily search the header and scripts for trojans. Scripts aren't dangerous by themselves. Scripts are dangerous if they contain dangerous code. So if there is no dangerous code then what's the problem?

We're being used by high-profile projects like Gaim, AbiWord and Inkscape. Upstream trusts us. That should say something, right?

Re:Not exactly loved by the distro people...

[FooBarWidget]

The fire marshal is specialized in extinguishing fires. They are not specialized in other areas. Likewise, the distro guys are specialized in packaging their *own* distros, but not inter-distribution packaging. You still shouldn't let them do the thinking for you.

Autopackage is used by high-profile projects like AbiWord, Inkscape and Gaim. Upstream trusts us. What do you think that means?

Re:Not exactly loved by the distro people...

[IamTheRealMike]

Nobody wants that, but it's basically required.

It clearly isn't, as Windows and MacOS X - in fact, every commercial OS ever made - did not require the central repository scheme. They use things called "platforms" and we've been saying for a long time that Linux should have one.

Until then, autopackages - which are basically the binary equivalent of source tarballs with extra features - are the next best thing for end users who just want to click in a web page and get what they expect (as opposed to some randomly patched, out of date, potentially broken package uploaded by Mr. J Random)

Re:Not exactly loved by the distro people...

[Nevyn]

It clearly isn't, as Windows and MacOS X - in fact, every commercial OS ever made - did not require the central repository scheme.

Yes, they did it's just at the root of the repository was a large proprietary binary blob that couldn't be changed. And, yes I can see how that makes the incomplete solution easier ... but given that I can get complete solutions for less, I'll pass on your "next best thing".

Re:Not exactly loved by the distro people...

[EzInKy]

- We (= the distro guys) don't like proprietary software so its not our problem.


And you can't see why? Proprietary software is a security nightmare because if you can't see the sources then you can't be sure of what the binaries are doing. Closed source stuff belongs in one place and one place only, /opt, and it should never be given access to system resources that compromise the users machine.

Re:Not exactly loved by the distro people...

[IamTheRealMike]

You're assuming there's no complexity difference between 1 dependency and 100 - but this defies common sense and actual experience. If the "complete solution" were so great, how comes there are so many articles and posts from people wishing for MacOS type software management on Linux?

Re:Not exactly loved by the distro people...

[FooBarWidget]

"And you can't see why?"

Look at the post I replied to. "What if Adobe wants to distribute Photoshop CS on Linux?" And there currently is no alternative to Photoshop. If even mention Gimp, you'll be instantly flamed down by thousands of Slashdotters who say Gimp is nowhere near being able to take on Photoshop.
Or how about things like Flash? Or the NVidia drivers? No good open source alternatives.

But that aside, even assuming that there *are* good open source alternatives, you'll still have a problem because the distro guys say "We don't care about inter-distro compatibility. Not our problem.". And this affect open source software too!

"Proprietary software is a security nightmare because if you can't see the sources then you can't be sure of what the binaries are doing."

I'm becoming more and more critical to statements like that. Autopackage is fully open source. Anybody can check autopackages for trojans. Yet some people still say that autopackages contain "arbitrary untrusted code" (never mind the fact that upstream developers aren't untrusted sources). You are contradicting yourselves.

Re:Not exactly loved by the distro people...

[ScriptedReplay]

Does RPM/DEB provide that seperation? Last time I checked (today), RPMs *must* be installed as root.

Then you haven't checked hard enough. You can create a user-level rpm database and use that to install as a user to a user-defined location. See the --prefix option; combine with --nodeps if you want to skip duplicating the system db.

As the preinstall and postinstall RPM scripts can do whatever they want, including rm -rf /.

See the --noscripts option.

Re:Not exactly loved by the distro people...

[FooBarWidget]

"Then you haven't checked hard enough. You can create a user-level rpm database and use that to install as a user to a user-defined location. See the --prefix option combine with --nodeps if you want to skip duplicating the system db."

I know about that, but it isn't useful in practice. The fatal problems are:
1. The most important of all: programs packaged in RPMs are not designed to be relocatable. Path names for data file lookup are hardcoded into the binary. You can install RPMs to your home folder but the programs won't even work! So what's the point?
2. Since you have to specify --nodeps, it defeats the point of the package manager? No dependency checking, no dependency resolution, no yum/apt support.
3. There is no easy and friendly graphical way to do this. Typing commands is unacceptable for people like my dad. This is simply unacceptable for desktop end users.

Point 1 is still the most important though. RPM'ed programs installed to the home folder simply don't work! In order for RPMs to be useful *in practice*, they must be installed as root.

"See the --noscripts option."

1. And you've just broken all the packages that rely on the preinstall and postinstall scripts to setup things. What's that I hear? You can review the scripts before you install? Well, you can do that with autopackages too!
2. There is no easy and friendly graphical way to do this. Typing commands is unacceptable for people like my dad. Even if there is, people like my dad are not capable of reviewing scripts.

Haven't we debunked this before?

[dondelelcaro]

By default, programs installed with root privileges are placed in /usr/share.

I sure hope that's a typo for /usr/sbin... If not, well, that's so horribly broken that I don't even want to get strarted with it. FHS anyone?

When a package is removed, its dependencies remain. Hearn and Lai point out that dependencies are not always removed by some other package management systems, either.

Some package management systems may not do that, but at least they keep track of exactly which packages have been installed so you at least have a chance of removing the dependencies at some point in time. With this solution you end up with files on your system that have no clear correspondence to any package, which kind of defeats the purpose of having a package in the first place. (To just expose my bias... aptitude, synaptic or deborphan anyone?)

"A binary from one distribution doesn't always work on another, because of things like glibc symbols and C++ ABI." [...] "Native package managers' dependency detection depends on a database. Autopackage, on the other hand-detects dependencies by actually scanning for them."

Right, and what exactly is autopackage going to do about these dependencies once it has found that they don't match up? Use LD_PRELOAD and have multiple copies of system libraries in place instead? Oh wait, autopackage is for "desktop packages only".

Of course, all of that isn't to say that autopackage may not do something useful in the future, but it sure looks like some of the fundamental problems of developing and distributing packages which other packaging systems have already dealt with still remain to be solved.

In any case, if you don't believe me, see what Scott Remnant has had to say on the matter (he's currently the dpkg maintainer, so he at least is passingly familiar with the issues surrounding a packaging format.)

• http://www.netsplit.com/blog/tech/autopackage_II.h tml
• http://www.netsplit.com/blog/tech/autopackage.html

Re:Haven't we debunked this before?

[FooBarWidget]

"I sure hope that's a typo for /usr/sbin... If not, well, that's so horribly broken that I don't even want to get strarted with it. FHS anyone?"

It's either a typo, or the writer of the article wrote the wrong thing.

If you install an autopackage as root, it'll be installed to /usr. This can be customized by editing a configuration option. If you install an autopackage as user, it'll be installed to ~/.local.

"Some package management systems may not do that, but at least they keep track of exactly which packages have been installed so you at least have a chance of removing the dependencies at some point in time."

What do you mean? Autopackage, too, keeps a database of installed files. You can remove a package by typing 'package remove foo' or using the graphical user interface.

"Right, and what exactly is autopackage going to do about these dependencies once it has found that they don't match up? Use LD_PRELOAD and have multiple copies of system libraries in place instead?"

I don't really understand what you mean. But we detect dependencies in the same way like autoconf does: by directly scanning for their presence. But nobody seems to complain about that.

"Oh wait, autopackage is for "desktop packages only"."

You are correct. Is there something wrong with focusing on desktop applications? That is what average users care about.

As for Remnant's blog: we've replied to that before. And he may be the dpkg maintainer, but that doesn't mean he knows autopackage better than we do, or even understand why we designed it like this in the first place.

Re:Haven't we debunked this before?

[dondelelcaro]

If you install an autopackage as root, it'll be installed to /usr. This can be customized by editing a configuration option. If you install an autopackage as user, it'll be installed to ~/.local.

So autopackage will stomp all over the files installed by the distribution by default. Great to know. (What happens if it installs a version of a library that is ABI incompatible with a version that the distribution ships?)

I don't really understand what you mean. But we detect dependencies in the same way like autoconf does: by directly scanning for their presence. But nobody seems to complain about that.

Surprisingly, people in autopackage's target demographic probably would complain about autoconf... but regardless, the question was how will autopackage deal with an ABI conflict (lets say the gcc-3.3->gcc-3.4 one) between itself and a library it wants to dynamically link with at runtime. Seems to me that you're going to end up with the same sorts of package incompatibilities between distributions that autopackage was supposed to solve.

Is there something wrong with focusing on desktop applications?

What you focus on isn't the point; the issue was that dealing with the above will (most likely) require you to deal with a whole host of things which are decidedly not "desktop applications."

In any case, best of luck. Just don't expect distributions to be willing to support people who have installed packages using autopackage. [And don't be surprised either if distributions speak actively against the use of autopackage.]

Re:Haven't we debunked this before?

[FooBarWidget]

"So autopackage will stomp all over the files installed by the distribution by default."

No. It will only do that if you want it to do that. Install to $HOME if you're paranoid, or tweak the settings to not default to /usr.
- There's a good reason why we install to /usr: because distributions don't support /usr/local properly. All kinsd of things won't work. Menu items, icons, dbus servers, you name it. More info here.
- We don't overwrite glibc or gtk or any of your precious system components. Autopackages are desktop apps, remember? Worst that can happen is when your distribution's Gaim is overwritten.
- As of autopackage 1.2, autopackage even refuse to install if there's already a native version installed.
- The average user wants desktop integration (menus, icons, etc.) to work. And right now, installing to /usr is the only way to do that.

In other words: you *can* set it to /usr/local by default, but then we can't guarantee that things like menu items will work correctly simply because all (yes, you read it: all of them; unfortunately there isn't a single distribution that gets /usr/local right) distributions have incomplete support for /usr/local.

(Un?)fortunately, distributions even support menu items/icons/etc in $HOME better than /usr/local!

"Great to know. (What happens if it installs a version of a library that is ABI incompatible with a version that the distribution ships?)"

It won't.
- Dependencies are only installed if they're not already on the system.
- If a package overwrites a system library, then the package is broken and must be fixed. Shipped dependencies are to be installed to private folders, as recommended by our guidelines, exactly to prevent DLL hells like that.

"Surprisingly, people in autopackage's target demographic probably would complain about autoconf..."

Yes they would. But my point was to explain how autopackage detects dependencies.

"But regardless, the question was how will autopackage deal with an ABI conflict (lets say the gcc-3.3->gcc-3.4 one) between itself and a library it wants to dynamically link with at runtime."

You are talking about the C++ ABI problem. We have done extensive research on that area, and we have a solution planned for autopackage 1.2 (due very soon now).

"What you focus on isn't the point; the issue was that dealing with the above will (most likely) require you to deal with a whole host of things which are decidedly not "desktop applications.""

We know that. And that's why we have done, and are still doing, extensive research about binary compatibility problems. We aren't selling hot air. We have put real efford into solving practical, real-world problems related to this.

Re:Haven't we debunked this before?

[dondelelcaro]

We don't overwrite glibc or gtk or any of your precious system components. Autopackages are desktop apps, remember?

Perhaps that's your goal, but desktop apps are some of the applications with the most insane set of dependencies out there; it's definetly not unforseeable that your package will require a version of gtk that is slightly different from the version that is installed upon the system. [Worse, if it has the same soname but different API or ABI.]

You are talking about the C++ ABI problem. We have done extensive research on that area, and we have a solution [plan99.net] planned for autopackage 1.2 (due very soon now).

This "solution" is to double or trebble the size of the packages distributed by compiling them for every possible ABI incompatible version.

Re:Haven't we debunked this before?

[i_should_be_working]

My (possible) solution was to not give autopackage my root password, which forces it to install everything to /home/me/.local

It may result in multiple libraries and wasted space but at least it can't confuse apt. Also the security concerns are addressed this way.

I really don't see why this isn't the default way to do autopackage.

Re:Haven't we debunked this before?

[FooBarWidget]

"it's definetly not unforseeable that your package will require a version of gtk that is slightly different from the version that is installed upon the system."

You are correct. We don't claim to solve *all* problems, but we do solve a great portion of them. We have received a lot of positive user feedback.

"[Worse, if it has the same soname but different API or ABI.]"

Then that library is utterly broken and must be fixed. We actively encourage developers to adhere to libtool version.

"This "solution" is to double or trebble the size of the packages distributed by compiling them for every possible ABI incompatible version."

Right now, there are only two major ABIs: the GCC 3.2 and the GCC 3.4 ABI. Nobody (at least, nobody who cares about the desktop) uses the pre-GCC 3.2 ABI these days.
And it won't double the size. We use binary diffs, and this saves a lot of space (roughly 70%-90%, depending on the binary). And a major part of desktop apps usually consists of data files; those are not duplicated. Autopackage 1.2 also features LZMA compression, which offers better compression than even bzip2 (which is used by RPM AFAIK; not sure about DEB).

Do you know a better solution though? At least this one works. Making it work is more important than making it perfect.
Frankly, to truly solve the problems, the Linux community has to start caring more about binary compatibility. But from what I understand, you seem to have already given up on that, by claiming that it's not possible to create portable binaries. But we have to start *somewhere*, don't we? The solutions may not be optimal, but at least they work.

Re:Haven't we debunked this before?

[FooBarWidget]

"I really don't see why this isn't the default way to do autopackage."

I don't know what you mean, as every time you install an autopackage, it asks you to enter the root password. If you want it to install to ~/.local, then just click "No Password" in that dialog. What is the problem?

Re:Haven't we debunked this before?

[Anonymous Coward]

We don't overwrite glibc or gtk or any of your precious system components. Autopackages are desktop apps, remember? Worst that can happen is when your distribution's Gaim is overwritten.

Right there is one of the issues that makes autopackage such a hard sell. Package managers for the distro are responsible for managing all of the software installed (exempting /usr/local/*), both managing/merging dependencies, and removal of files during uninstalls.

Autopackage's goal is as a secondary package manager- which is fine. However, if that is the goal for autopackage, that means it needs to not screw with the primary package manager in anyway- your comments above about worst case overwriting gaim isn't exactly heartening.

Not after slapping autopackage here, but the assumptions here seem to be a bit daft. Primary package managers will be the ones who get the flak if/when autopackage stomps something, yet there is bitching about distro's disliking autopackage?

It's a two way street, if autopackage is after providing a secondary software repository that's cross distro, it cannot hork up the distro's base installation- it must not screw with the dependency graph for the primary pkg manager. If it does, it's not a good solution, mainly because autopackage, the 'desktop software repository' would be effectively a primary package manager.

Having two pkg managers screwing with files on disk doesn't seem wise, when both think they have total control (note also that the $HOME LD_PRELOAD tricks don't count here, we're talking about merging to /usr).

If /usr/local/ support is horked on a distro, suggestion would be to actually get it fixed rather then doing a kludge ('It works now' logic is dodging the fact that the pkg manager relationship here is non existant, thus it's a kludge).

Re:Haven't we debunked this before?

[i_should_be_working]

I mean since most of what people complain about (rightly or wrongly) with autopackage is security (binaries being installed system wide by a third party) and integration (files in filesystem that weren't put there by the package manager) it seems like a default way to do it would be to always install in ~/.local. It's not a problem for me to always have it do this. I just though it might be better to have this be the default behaviour.

Also, maybe autopackage has been updated since I last used it, but I was only prompted once for a root password. Subsequent package installations went straight to /.local without asking.

Re:Haven't we debunked this before?

[FooBarWidget]

The reason why we allow root installs is because there are people who want to install software system-wide. And some things just work better when you install to /usr as root (unfortunately not all desktop environments fully support locating resources in $HOME). And this is also why we *ask* the user whether he wants to enter the root password.

All the complaints seem to come from power users who don't like the idea of a new packaging format. The inexperienced Linux users seem to love us, as can be seen from the feedback we get from them.

"Also, maybe autopackage has been updated since I last used it, but I was only prompted once for a root password. Subsequent package installations went straight to /.local without asking."

Sounds like a bug. Are you using Ubuntu? It might be related to sudo.

Re:Haven't we debunked this before?

[FooBarWidget]

"Right there is one of the issues that makes autopackage such a hard sell."

Did you miss the bit in which I said "As of autopackage 1.2, autopackage even refuse to install if there's already a native version installed." ?

But let us go back to the problem. The user has two choices:
1. Install an autopackage of FooBar 2005 (latest version), which overwrites the existing copy.
-OR-
2. Not being able to install at all, and having to wait 3 months for a native package to appear.

Which one do you think is the better choice for the average user?
And again, "As of autopackage 1.2, autopackage even refuse to install if there's already a native version installed."

"If /usr/local/ support is horked on a distro, suggestion would be to actually get it fixed rather then doing a kludge ('It works now' logic is dodging the fact that the pkg manager relationship here is non existant, thus it's a kludge)."

You are correct. But the problem is, distributions refuse to fix /usr/local support! We've asked them. They all refused to do anything about it. Some of them even say that they've explicitly b0rked /usr/local support because they don't want people to install to /usr/local.

Re:Haven't we debunked this before?

[IamTheRealMike]

Right, and what exactly is autopackage going to do about these dependencies once it has found that they don't match up?

If a required dependency can't be found the install fails, same as with a source tarball. It's the developers job to ensure their dependencies are reasonable and widespread: projects like AbiWord or Inkscape haven't had an issue with doing this. There are various techniques you can use like static linking obscure/rare/unstable libraries or using relaytool/dlopen to access features which might not be present.

Of course, all of that isn't to say that autopackage may not do something useful in the future, but it sure looks like some of the fundamental problems of developing and distributing packages which other packaging systems have already dealt with still remain to be solved.

I would disagree that they've solved anything, all they did was add a massive amount of complexity to try and wallpaper over the underlying chaos. Systems like Debian don't solve the actual problem "there is no platform", they just bandaid the symptoms and you end up with weird artifacts of this approach like out of date packages, version freezes, broken packages, painful library transitions and so on. All of these things are stop-problems for end users.

Re:Haven't we debunked this before?

[i_should_be_working]

Are you using Ubuntu?

Yep

Re:Haven't we debunked this before?

[cortana]

How can you correctly map between the name of the autopackage, and the local package manager's name for a package?

Re:Haven't we debunked this before?

[cortana]

http://www.google.com/search?num=20&hl=en&lr=&safe =off&c2coff=1&q=site%3Abugs.debian.org+%2Fusr%2Flo cal+autopackage&btnG=Search gets no hits. Then again, perhaps I'm doing something wrong, as even http://www.google.com/search?num=20&hl=en&lr=&safe =off&c2coff=1&q=site%3Abugs.debian.org+%2Fusr%2Flo cal+&btnG=Search only yeilds one hit.

Re:Haven't we debunked this before?

[FooBarWidget]

We don't; we do a reverse lookup. Upon installing a file, we check whether that file is already owned by a native package.

Nor Slashdot...

[rincebrain]

...considering it's not posted to Index. :)

Yes, but...

[hackwrench]

...will it be ported to Windows?

Re:If it's anything like automake...

[FooBarWidget]

Come on. Autopackage has got as much to do with automake as Bill Clinton with Bill Gates.

Biting off more than one can chew?

[Kalzus]

If you don't mind keeping it separate, why not go with pkgsrc?

It mostly works and it's available today.

Old Autopackage story, bleeding edge installs

[bzipitidoo]

We read about Autopackage months ago in an old Slashdot story.

Sounded sort of cool, but they always do. tar + gzip or tar + bzip2 is still the ultimate in packaging. I use Firefox on Slackware. What I end up doing is not installing the Firefox package that came with Slackware. It's difficult to update. Instead, I grab the latest tarballs of Firefox as they appear and install them manually in /usr/local. Configured my window manager with a button that points there. Of course doing it that way, the Slackware package manager thinks I don't have Firefox. Now if this Autopackage could deal with a tarball so I don't have go manual or wait around for whatever distro I'm using to put out a new package, that really would be cool. RedHat managed to get themselves and their rpm app prominent enough that for a while a lot of software was offered in rpms as well as tarballs. I don't see rpms as often as I used to.

And how about that button in the Window manager? Doubt Autopackage can handle that detail, as I use fvwm2.

Re:Old Autopackage story, bleeding edge installs

[FooBarWidget]

The reason we don't deal with tarballs is because:
1. No desktop integration. Tarballs don't install menu items and icons for you to the right place. We do. We go into great length to deal with the myrads of different menu systems.
2. Binary compatibility. Tarballs are just that - tarballs. Part of the autopackage is research about binary compatibility. We figure out why binaries don't work on other distros, and try to fix that. Without a solution to binary compatibility problems, using any format is meaningless.

"And how about that button in the Window manager? Doubt Autopackage can handle that detail, as I use fvwm2."

You are correct, we don't support fvwm. But honestly: how many people use fvwm? More importantly: how many average users use fvwm? A simple cost/gain calculation reveals that we should focus on GNOME/KDE support instead. We intend to solve real-world problems with software installation on the desktop. We aren't trying to support every window manager out there, we aren't trying to replace RPM/DEB, we are trying to solve user-visible real-world desktop software installation problems.

Are people still using redhat?

[TheNarrator]

I think the "new package manager every month" people must all still be using redhat 9? I haven't thought about packaging since I switched to a Debian based distro (Ubuntu). The most intresting I've seen lately in packaging is Klik .

The MacOS way

[caseih]

There's no doubt about it that Linux really is in the stone age as far as managing software goes. Every time I help a newbie install Linux I dread the next question: "How do I install software." "Well it's easy. We just edit this source.list file here and add these repositories from dag, freshrpms, etc, and then you can just apt-get it. isn't that simple?" And they go running back to Windows. The point is that as developers and programmers and even tech-savvy people we can deal with this kind of thing, and in the end find it much faster than in the windows world.

Although there are times when it really is annoying. For example, I run Fedora Core, but I don't want to have upgrade the whole dang OS every 6 months. Once a year is almost too much. I tried to get beagle running, which required a patched kernel which is no big deal. Then I tried to install the latest mono libraries. They depended on a newer version of GTK than I had, so I upgraded that (used a SRPM). Then I found the gnome-sharp and other gnome-related bindings only worked with a newer version of these libraries. OF course to install those rpms would have required upgrading the whole OS to FC4 (dependency hell all the way back to glibc). So we've replaced DLL hell with dependency hell. Neither is good. If I could have somehow installed these newer libraries alongside my older ones, just for the newer apps to use, that would have been ideal. But package managers make that difficult (but not impossible).

Autopackage is a step in the right direction, but there are too many concerns about it (as mentioned in the previous posts) that either have not been addressed yet, or have been refused by the autopackage developers, including package signing, etc.

Watching my sister download and install software for her Mac really makes me ashamed to show anyone how stuff is done on linux. I wonder if the MacOS model might provide some insight into how to make things work. I'm not talking just about AppBundles, but also about Frameworks. Like linux, apple puts a lot of small libraries in /usr/lib. These libraries are really the domain of the system and so users should never mess with them (the old trust the distro idea, override them in the appbundle if necessary). However, large-scale application fromework libraries and dlls go into the /System/Library/Frameworks directory where you can install multiple versions. For example, I have Java 1.3, 1.4 and 1.5(5) all installed. Mono, GTK, Gnome libraries and similar things should also go in this place. If I need an app that needs the gnoem 2.12 libraries, I should be able to install them alongside the 2.10 libraries, but in a clean, modular way that can avoid dll hell. Then as far as app distribution goes, anything that is not a framework, but is a dependency of the app should go into the app bundle. There are some weaknesses to this approach, but it is worth thinking about anyway. Of particular note is that our build systems just don't want to deal with anything other than the standar /lib /usr /bin/ etc/ share/ directory structure. This fact does limit this endeavor greatly.

Judging by the comments on the blogs about autopackage by the ubuntu folks, they are pretty happy staying with the current state of things. I guess they don't really care if Linux goes beyond being a niche OS that only developers and techies use. Of course sometimes I feel the same way.

It's a tough problem and to date not a single OS or distribution of an OS (including MacOS, Linux, BSD, Windows) has yet to come up with a good solution.

Re:The MacOS way

[aconbere]

"... download and install software for her Mac really makes me ashamed to show anyone how stuff is done on linux."

That's the part that drives me nuts! I've been spoiled by package managers for too long now and I can't bare the thought ot having to prance around the internet trying to hunt down the latest builds of the software I want/need. I never want to have to go to gaim.sourceforge.net, mozilla.com, gimp.org, or hunt around for a new bittorrent client.

THAT! feels like the iceage to me. That is utterly avoidable with a decent package manager, apt-get, portage, freebsd's apt and ports. I can't deal with it, it makes me feel dirty and slow.

emerge -uav world vs. "scroung around like chicken with it's head cut off for an hour getting all my new software"

Now I'm not saying that there's a perfect package management tool. But hell if I would rather have one than watch people go back to the windows/mac installation model. Never again.

~Anders

Re:The MacOS way

[Akaihiryuu]

Portage (Gentoo) does something sort of similar. You can install multiple versions of some packages (example...kernels, GTK, GCC) in "slots". I always had the problems you mention with Linux (I used Redhat/RPM, bleh) until I switched to Gentoo. The package management in Gentoo is incredible. Even converting the whole system from GCC 3.3.5 to 3.4.4 (they aren't binary compatible with each other) is relatively painless. GCC 3.4.4 is installed in a new slot...switch to the new compiler, recompile system, remove 3.3.5. I remember trying to install something GNOME related under Redhat once...I used rpmfind.net and it took me hours to find all of the packages I needed...had to have the right versions, the right architecture, the right compilation options, etc. With Gentoo, it's as simple as emerge foo. I don't think I could ever use another distro. Portage has totally spoiled me. Sadly Gentoo isn't a very good distribution for those new to Linux or those without a significant amount of computer knowledge. I'm trying to find a distro that my gf can use, Gentoo is way beyond her. Thinking about having her try Ubuntu.

Re:The MacOS way

[Akaihiryuu]

No kidding...in my crontab, once a week, emerge --sync, updatedb, prelink -amR. Then once a week (I like to do the updates manually in case I have to adjust USE flags), emerge -uDNva world ; emerge depclean ; revdep-rebuild. I don't even think about updates anymore. Gentoo is freaking awesome...the only problem with Gentoo is that it's not a very good distro for those new to Linux or those without a decent amount of computer knowledge.

Re:The MacOS way

[caseih]

Thinking about having her try Ubuntu.

Somehow I don't think ubuntu can hold a candle to MacOS for her. Besides, MacOS runs so much nicer on her iBook than linux does. Which brings up another linux advocacy point. Sometimes we promote linux just to promote linux, even though it is not a good fit for some, like my sister.

I know this is crazy, but...

[T.Hobbes]

Why not either
- compile the libraries in with the binary, leaving one binary and one conf file for each application

or

- keep the libraries seperate from the binary, but store a copy of whatever libraries the binary expects in its own folder (/usr/bin/foo/lib)

Disk space and RAM are both cheap. This kind of thing would suck on systems where resources are limited, but otherwise it would simplify things dramatically

A Regression, Not a Solution

[chromatic]

PPC users most likely use MacOS instead of Linux, so there's not much point in supporting that either.

If you're really not interested in actually solving a problem, just say so. It's quite another thing to claim that real problems you have no intention of fixing Aren't Really Problems because your solution doesn't work for them.

Existing packaging systems Just Work for Linux PPC. Yours doesn't. How is that possibly an improvement?

Re:A Regression, Not a Solution

[FooBarWidget]

"If you're really not interested in actually solving a problem, just say so."

I don't say so, which means I AM interested in solving the problem.

"Existing packaging systems Just Work for Linux PPC. Yours doesn't. How is that possibly an improvement?"

Autopackage's goal is not to be "better" than RPM/DEB. Autopackage's goal is not to surpass current packaging systems in every way possible. Autopackage is not supposed to completely replace RPM/DEB.

Let us go back to defining the problem. The problem is: software installation on Linux is too hard for the average user. And what do those users use? x86 Linux! Really, look at the numbers. Browse some Linux forums, mailing lists, newsgroups, etc. Almost every time people complain about software installation on Linux, it's about x86 Linux. Have you ever heard of people complaining about software installation on PPC Linux? I don't. The people who want a friendly OS on PPC all use MacOS X. Have you ever seen people trying to convert Mac users to Linux? I don't. Have you ever seen people trying to convert Windows users to Linux? I have, many times!

In other words: the problem just doesn't exist (or is too small) on Linux PPC, or people would complain about it. The software installation problem is pretty much unique to x86 Linux, as that's where the majority of the users are! And there's the real reason. 99.9% of the problem is on x86 Linux, so logically it only makes sense to focus on x86 Linux.

MOD PARENT UP

[BinLadenMyHero]

Great post overall.

For example, I have Java 1.3, 1.4 and 1.5(5) all installed. Mono, GTK, Gnome libraries and similar things should also go in this place. If I need an app that needs the gnoem 2.12 libraries, I should be able to install them alongside the 2.10 libraries, but in a clean, modular way that can avoid dll hell.

Sounds good. The dinamic linker should be able to find and use the correct versioned files for the libraries, and gcc should be able to find the versioned include files/dirs too.

Just a quick note ...

[Anonymous Coward]

You seem to have a lot of vertical space around quoted text. You can reduce the amount by not having returns around the blockquote tags. For example:

My text.<blockquote><i>Quoted text.</i></blockquote>More of my text.

produces

My text.

"Quoted text."

More of my text.

. Hope this helps.