How to Make Easy-to-Package Linux Software

jmmv writes "The packages in any Linux distribution (or other operating systems) are a master piece to let the user install any program he wishes as painlessly as possible. However, the creation of these packages is not always free of problems. Usually, the packager finds that the program he wants to package suffers from a series of pitfalls - either conceptual or related to portability - that make the packaging task harder than usual. This is why I decided to write an article (published at ONLamp.com) summarizing these problems and proposing several solutions to each of them, aiming to make the life of the packager simpler. I hope it to be of interest to free software developers and also hope that they understand some of the issues and try to fix them in their creations."

Comments

Avoid modifying published distfiles.

[keesh]

"Avoid modifying published distfiles."

Oh heck yes. The number of times I've been bitten by a few projects (yes mplayer guys, I mean you) doing this and breaking Gentoo's digest system...

Re:Avoid modifying published distfiles.

[Papineau]

I think what OP meant was that Mplayer released a tarball, then found some bug in it and released a second one with the bug fixed but the same filename. Any MD5 hash of the first one will fail on the second one. Gentoo uses a MD5 hash to verify that the file is the right one.

How to properly package for linux

[ZephyrXero]

http://www.autopackage.org

Re:How to properly package for linux

[keesh]

Oh, brilliant idea. Force everyone to run arbitrary shell scripts before they can even unpack your package. At least with a tarball you can extract it and check the contents without having to run any untrusted package-supplied code.

Re:How to properly package for linux

[polyp2000]

who mentioned about forcing people to do anything? Most projects supply their files in several different formats so that people can make a choice as to what kind of install they want to do. I dont really care what you think, because if you are truly honest with yourself, you will admit that installing software on linux is a pain in the arse sometimes - even for seasoned users. Giving joe sixpack a foolproof click-through self-installer for linux is not just a good idea its a requirement. Especially as it continues to make headroads on the desktop.

I use Gentoo and Vidalinux by the way. The portage system is pretty good and dependancy issues are pretty much a thing of the past although annoyingly i still have problems. Im lucky enough to have a fairly beefy SMP computer so most of the time i dont have to wait too long for compiles.

Having said all that Autopackage is the best thing ive seen as an installation method yet. Have you actually tried to use it yet ?

Make sure you read the instructions first. You dont even need to go anywhere near a shell prompt!

4 steps to installing an autopackage...

Re:How to properly package for linux

[ZephyrXero]

Here's the problem... The open-source/free-software movement is similar to an organic evolution/adaptation system, while the proprietary software industry is just that, like industrial/factory systems. Because of the factory like mechanics, proprietary software is easy to package and pump out. Open source goes in all different directions and it's an electronic survival of the fittest. If someone comes out with a better piece of software, people will tend to use it instead. If someone comes out with a better distro or packaging system, it's the same thing. Only thing is... since it all goes in a million different directions, there are alot of "extinct species" along the way. This is our trade-off for the freedom vs. convenience. However, if we can find a way to get the best of both worlds (LSB, FreeDesktop.org, Autopackage, and other open standards)....things could really get exciting.

Third solution to Automatic Decisions

[Papineau]

The article is missing a third solution to the Automatic Decisions solutions: gracefully handle at runtime the absence of a (soft) dependancy.

What I mean is automatically enable all options which are available at build time, but don't hard link to them (use dlopen(3) or somesuch instead), so the same binary package would work in the presence or absence of such dependancies.

Of course, without the runtime dependancies, some options won't be available, but it's better to do it that way than to force everybody to download libfoo and libbar to satisfy an optional dependancy, or to arbitrarily disable some options (which will only be available to people building from source, not those using a package. You want your software to be useful, right?).

Re:Third solution to Automatic Decisions

[Anonymous Coward]

This is because amiga .library files were a bit like C++/COM objects: each library had a jumptable and base address. Jumptable entry offsets from the base address called LVOs (library vector offsets) were public and known. All library calls were indirected through the jumptable. So the compiler knew from the #pragma in the header to compile a call to, say, intuition.library/OpenWindow() or whatever (my amigaos zen is, uh, rusty, for some reason), to a jump to address IntuitionBase minus LVOOpenWindow, which was an entry in the jumptable (which grew downwards in memory, hence the minus), which then jumped to the actual library function (So every shared library call was a teeny bit slower than a direct function call (but even in those days of 7MHz computers, people seldom noticed)). And the application could pick and choose when to call OpenLibrary() to do the dlopen()-like operation, with very little hackery.

Actually, the WHOLE AmigaOS was like that. Address 4 in an Amiga process's memory map always pointed to AbsExecBase, the base address of the pre-opened exec.library (the AmigaOS pseudo-microkernel with interrupt handling and task switching stuff and so on). The process could then find exec.library/OpenLibrary() to open intuition.library (gui) and get IntuitionBase, dos.library (DOS) and get DOSBase, and so on.

This is VERY different to .so files, which are shared object files which are actually dynamically linked at load time by the ld.so program. The recent adoption of "prelinking" of linux .so files is a bit similar if you squint, though.

Another ramification of the LVO system was that library versioning was pretty easy. The .library file format specified a version number, higher numbers "win". But total binary backward compatibility could be maintained (with effort on the library author's part) even in the face of API modifications in the library, because binaries compiled against the earlier version would expect the old LVOs, so the the library author could just use new LVOs for his new API, and maintain the LVOs for the old API as wrapper functions for his new API (or made them bomb out with an error if he was too lazy to maintain backward compatibility, which is still more graceful than a typical .so or .dll "silently do the wrong thing").

Re:Third solution to Automatic Decisions

[FooBarWidget]

You may be interested in relaytool. See http://autopackage.org/ for info.

Re:There Should Be a Self-Installing Binary

[chromatic]

Is it simple or obvious?

64-bit? 32-bit? With debugging symbols? Without debugging symbols? Statically-linked? Dynamically linked? Which level of optimization? Which kernel? Which compiler? Which glibc? Which processor family? Which instruction set within a processor family? What options have I missed?

How big is this one self-installing binary you want someone else to create for you?

Simple...

[agraupe]

Appfolders, like in OS X, for the very simple method. This causes overlap in dependencies, but it will always work. This is the easiest system to deal with, IMO. There can be a simple tar-ball, or self-extracting archive.

Next, there's distro-based package systems like apt and portage. These work very well, as dependency control is automatic, and it is made to work with your distro specifically.

The third option, which is used by (I think, but am not sure) Loki Installer, and such programs, is to install the binaries, but leave you on your own to deal with dependencies. As long as you make sure the dependencies are easy to obtain, this is one of the best options for those without apt or a similar system. You could even, with GPL software, create your own installer for any dependancies, should you choose.

Why not Autopackage? Because it seems like a format that is, at best, a poor substitute for a package system like apt. The simplest solution, considering how much disk space is available on modern machines, is the folder concept like OS X. Very simple, and incredibly easy to remove programs.

This definately is a shortcomming in linux distros

[Stevyn]

Take a look around. You either get a good method for a few outdated packages, or you get a great system that takes forever to compile packages. Obviously, I haven't dealt much with debian based distros and their magical apt-get, but the major distros like Suse, mandrake, and fedora are still RPM based. I think RPMs have really slowed down linux adoption due to dependency hell. I hope systems like autopackage can help things out.