Slashdot Mirror
Building A Better Package Manager
SilentBob4 writes "Adam Doxtater of Mad Penguin has published a preliminary layout for his proposed cross-distribution package manager capable of adding/removing software from any locale. He is suggesting the interface will basically allow for installation of several major package formats including RPM, DEB, TGZ, as well as source code with the ability to pass build time options. All of this will come at the price of standards of course, including naming, documentation, and package structuring. If this idea were to catch on, it would signify a major leap in desktop Linux usability. This might be a project that UserLinux might benefit from. Read the full column here (complete with GUI mockups)."
I hope they will include Ebuilds as they are a great way to install software, and are becoming more and more popular as Gentoo does.
:) )
(No, this isn't a troll, I'd really like to see that.
libertarianswag.com
This topic comes up how often? Like twice a year?
And yet nothing ever changes.
This space intentionally left blank.
People get so hung up on the package format. It really isn't about the package format, it's about the people and organization behind the packages and whether they produce a consistent distribution. A "better" package format or a better installer isn't going to help you when a piece of software expects libraries to be there that just aren't available, or when an install script assumes functionality you don't have.
Lets do this from the beginning rather than slapping it on later.
APT already handles debs and rpms. tgzs should not be a far stretch. The problem is establishing standards and getting everyone to follow them. For example, all debs in the Debian archive follow the Debian packaging standard, else they would not be accepted into the archive.
Naturally, third parties are free to create their own non-conformant debs. This is just the same as someone creating an rpm for RH9, but it not conforming to the conventions used by Red Hat.
I assert that the tools already exist. I.e., we don't need a new one. The emphasis needs to be on getting people to follow the standards, and possibly creaitng a cross-dsitro standard fo everyone to follow.
All smoke and no fire... Don't talk about it just build it. Personally I think someone should sit down and hack together a install package builder program based on something like gdialog and python that outputs a executable compressed image into a single bin file.
A good installer is not hard to accomplish if the desire for it really exists. It is however one of the most overlooked things as open source programs are involved.
Don't make me go hunting down 20 dependency packages but offer to install them for me. A simple script based on wget can do that...
Got Code?
The problem is basically a manifestation of there being more than 1 distro of linux and having distro maintainers who have not agreed on a common standard for this stuff. It's Linux's major achilles heal IMHO.
I would argue this, actually. Just because it's s a problem that needs fixing, does not make it n achilles heal. Think about it: When we come up with an elegant solution that is cross-distro ( and possibly cross-platform ), it will make linux that much stronger.
Mod me down with all of your hatred and your journey towards the dark side will be complete!
Maybe Apple can do this because they have a standardized directory structure, but what can be easier than dragging an app package to the Applications folder? Poof, it's installed. Don't like it? Delete it. If it's more complex, there's an installer program. Playing with dependencies and makefiles is the reason I gave up on Linux.
"I am not a number! I am a free man!"-- The Prisoner
OK, I'll bite on the chance that you really mean it. Unlike Windows, Linux has multiple distributions, each with their own notion of what a complete GNU/Linux system should have and where everything should go. Each distribution caters to different audiences and that means people using Linux have a lot more choices as to what their systems should have and how they should work. Unfortunately, that also means that it's very hard to come up with a (clueless) user friendly way to install and remove programs. RedHat has its way of doing that, called RPM. Debian's is called DEB. Of course you can usually download the source, compile it and install it manually regardless of the distribution you run, but that means the user doing that can't afford to be _that_ clueless. Hope this helps.
The theory is fine. The problem is that package managers are, in many ways, incompatible. Debian packages, for instance, track dependencies based on the names of other Debian packages (libfoobaz-dev requires libfoobaz). I've seen package management systems that have dependencies based on files (libfoobaz-dev requires /usr/lib/libfoobaz.so). The former system won't recognize dependencies from packages installed in the latter format. Worse, the packages don't overlap. One distribution will have libgnome, whereas another will have 50 different packages, one for each of the Gnome libraries. The problem of dependencies there breaks almost completely.
.rpm packages on a .deb system, or vica-versa, without breaking something. It is possible to install packages of one sort on the other system, but eventually, things will break. Each package management system relies on some set of information about packages to work, and each system has a different set of information it provides and needs.
.deb, .rpm, .tgz, etc. packages, and give easy-to-read information about what systems it'll work on. I've heard of tools similar to this, but I haven't seen them used. Adding something like this to the standard autoconf/automake/... process would certainly be nice.
There's also a matter of versions and security updates. On Debian, I run 'apt-get update; apt-get upgrade' and have a new version. Since the packages are all maintained by the Debian project (and a few smaller projects that target Debian), this works. Versions aren't linear -- Debian back-ports security fixes. The package manager has no way of knowing whether kernel-2.4.24 is newer than kernel-2.4.19-with-patches.
Basically, there is no clean way to install
There is room for improvement in package management -- a really good GUI for finding and installing packages would be nice. I wouldn't mind having more information about the packages I'm about to install -- links to project web pages, ability to browse installed files (the packages.debian.org/freshmeat.net/etc. databases either installed locally or quickly accessable from the system), the ability to view screenshots of GUI programs, etc. There's a lot of metainformation that could be added, and better search functionality that could be implemented.
At the same time, on the package build side, it'd be pretty simple to have a system where you make a configuration file of information about the package, and it builds
The last solution is to have the groups work together to make sure all packages have the same set of metainformation (more than is needed for any given package system), so that cross-platform package installs become possible. In practice, I don't see this scaling across versions, as package management systems evolve.
One more thing to bear in mind is the perspective of the author of the article -- he says he runs Slackware, and builds most packages from source (something I've stopped doing maybe 3-5 years ago). Slackware's package management tools are very basic, manual, and crude. That gives a very different attitude towards package management than someone running a distribution like Red Hat, which has a much heavier-weight, more technologically-advanced, but somewhat fragile, somewhat inflexible package management system, or a user of Debian, which has a state-of-the-art ubermaintainable, uberupgradeable package management system, but that primarily relies on grabbing packages from one (or a small number) of sources. I apologize about the stereotypes in this paragraph -- they're not entirely true, but the package management systems differ a lot (more than most people realize, if you've ever tried to build the packages), and I'm just trying to make people aware that users of each of them will have a very different world view, and it's important to keep that in mind when reading these articles.
Please tell me that the BSDs use more than just checksumming for security. Checksumming is like security through obscurity, but without the security. It gives you a false sense of security only. There is no reason at all to think that someone smart enough to add a trojan to a package would be stupid enough to not update the corresponding MD5sum.
Dummy, the MD5 isn't distributed with the package!!! If you use public key encryption, you could also say that "There is no reason at all to think that someone smart enough to add a trojan to a package would be stupid enough to not update the corresponding public key."
When we come up with an elegant solution that is cross-distro...
.deb, .rpm, and others. The problem is that most geeks (at least the ones in charge) are stubborn idiots. If they said that debian packages are better than rpms five years ago because the only distro they ever used was debian, there's no way in hell that they will ever admit that another package management system could do the job, and agree to standardize.
There are already lots of them.
There's no technical reason why we can't get some people together to iron out the last differences and either create a standard package manager, or create well-defined interfaces that allow any front end to access any kind of package. However, if you did that, nobody would use it anyway.
How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
Yes, I know, this might be considered offtopic, but...
:)
If we get KDE and Gnome work together, then we might also, eventually get an installer too!!!
PLEASE!!!
Let KDE 4.0 and Gnome 3.0 be the same!!!
Why hasn't anyone developed a system that, from the End-user perspective, works similarly to MSI installations (which work very well). Point, click, next next next. In principal, DEBS/RPMs work similarly to MSIs, but the installation isn't as obvious a procedure to end-users.
And for that matter, why not make the installer intelligent about the distro? Use a single package/installer, but that includes all sorts of scripting information about installation in variosu circumstances. The installer checks to see if it's on RH9, and if so it puts files where RH9 expects them, editing any configurations and making RPM database entries as necessary. If it's on Debian, it takes the appropriate measures there. And so forth.
Why do we see such absurd dependencies that don't seem to happen in the windows and mac worlds? Install a new version of a KDE app, and you need the latest minor revision to the core KDE libs, which in turn requires a minor upgrade to the font server, etc. In the windows world, occasionally you need to update something big like DirectX to install a latest-and-greatest app, but even then the dependencies are often packaged with the app itself. Why isn't this practice more common in Linux/Unix (not counting Mac OS X)? I undestand that many of these apps are under CONSTANT quick-release development and are often tied to bleeding-edge versions of their libs, but why aren't major releases at least more dependency-friendly? Installing an app can be a real pain in the ass even with something like apt, if you don't have the dependencies in the repositories you've defined. And adding new respositories isn't exacly grandma-friendly.
Standards are not a price, they are an investment. I use standard XHTML, CSS, and SVG in my web design because I care about the future of the web. Besides, if a standard is well-designed (like W3C recommendations tend to be), it actually makes development and maintenance easier. Anyone who has migrated from HTML 3 (or some nonstandard IE/Netscape HackML) to HTML 4 or XHTML with CSS knows what a pleasure it is to work with modern hypertext (and probably also has an abiding and bitter hatred for IE). The same could be true of package installation in Linux if the standard is well-designed.
Because it's dependency hell, and you can't remove things. I would have thought that it was pretty obviously broken. It also doesn't help that it's a command line action, rather than just being able to double-click. Ooh, and users don't give a fuck about the source code, so compiling is really a waste of time.
No easy dependency tracking, no easy uninstall, no easy upgrade, no audit trail. On a server you don't usually want a compiler installed as it can be a security risk. It's really nice having a database of all the software installed, what versions of what other software it depends on, and reliable way to remove it without keeping the build tree around assuming the build system used has an uninstall method. The only way I would feel confident about not accumulating cruft due to upgrading big packages from source (gnome, kde, X) is if they are installed 100% into a single folder (like /opt/kde/3.2/(bin|lib|conf|man|...). Then I can safely uninstall by deleting that top version folder. Even then, I don't want to take the time downloading and compiling the source, I don't find it to be very recreational. I'd rather run `apt-get install kde` or `apt-get upgrade kde` or `apt-get remove kde`. With that remove command, it also removes packages kde depended on but nothing else does. You don't get that with source installations, you have to keep track of it yourself.
In the long run, unless you are meticulous about tracking which packages need which other packages, and where they were all installed, you are insuring you will have to rebuild your system from scratch at one point. Package managers like APT and Yum, and even up2date allow you to avoid this.
So this is a similar effort to Autopackage except that it plans on using the native package formats?
Except that this guy has just stated the idea and made a couple of mock-up screenshots, whereas the autopackage guys are coming up with a complete, sensible solution and are leaving the interface until the end.
The real problem isn't package formats (as your example with Suse and RedHat shows).
It's simply that distributions are different. Pretty much any non-trivial Linux program needs a lot of libraries that are non-standard (as in: not in the LSB). Different distributions make different choices, which is probably a good thing, but it breaks binary compatibility between distributions. This has (pretty much) nothing to do with package formats.
I'm not going to call you a troll. But I am going to assume you dislike the RPM format due to the dependency-hell problems of the past, if that's not why you dislike it, then feel free to ignore the rest of my post, except the PS. The problem is that many distro maintainers selected the RPM format (Red Hat, Mandrake, Suse, Ark, even the LSB chose RPM as the standard format), and then packaged software with conflicting package names and file system layouts. So you go looking for an RPM for Red Hat, and find one from Suse, and it says it needs xfree86 3.0.3, even though you have XFree86 3.0.3 installed. Or perhaps it needs some particular .so to be in /usr/lib but the Red Hat package owning that file put it in /usr/lib/ssl.
These aren't flaws in the RPM format, these are the problems this project aims to fix. You would see the same problem with dpkg if there were as many popular distributions which used dpkg but didn't base themselves off of debian's repositories.
The other complaint commonly made is that apt is better. Apt needs a tool like rpm or dpkg behind to actually be useful. Apt is purely that part of the system that locates dependencies, like the part of portage that knows to build and install X before building and installing GNOME. It doesn't actually install the package or maintain the database of installed software. Apt also runs very nicely on RedHat. Connectiva and Ark use it as the default system. Mandrake and Suse implement their own dependency tracking system.
PS - In addition to Red Hat and Mandrake, I've also tried FreeBSD (years ago, didn't support all the hardware needed to install), Debian, Suse, Ark, and Slackware.
It really isn't about the package format
/usr/local is a nominal installation point only. In the real world, site requirements can be much more complex. Over the years, I've noticed that autoconf handles this situation very cleanly. It's the people building binary packages that seem to overlook the need for relocation.
Of course it is! A common package notation allows dependencies between software components to be detected precisely and automatically. It's only a step from there to have dependencies resolved automatically as well.
If two software components use two different package notations, you're forced to do figure out and resolve their interdependencies by hand. Supposing you could do it at all, what do you think the chances are that you'll do it identically twice in a row on the same system, let alone on a server room full of systems which are to be configured identically?
Of course you're right that the problem is not entirely solved merely by committing to a universal package format. Developers still have to take care to use its capabilities correctly. Dependencies have to be expressed accurately, for example.
I've also noticed that relocation is often not done properly in open source packages. I suspect this may be because many open source developers are not used to installing software at complex sites where different administrative groups maintain different sets of software at different points on different servers. The point is that
Of all of the distributions I've used (Slackware (Walnut Creek '96 through 8.0), RedHat (5.1, 5.2, 6.0), SuSE (6.# through 7.0) and Debian (2.2 - Current Unstable)) I've found Debian's apt+dpkg combination to be the best built. It doesn't matter if I used Progeny, Storm, Knoppix, or any particular version of Debian Proper, I was always able to specify source servers and update the system. I was able to add third-party servers without much issue. The system just worked. RPM in the two distributions that I've experienced with it was a pain. It was hard to meet dependencies even with a particular package built for a particular distribution, and I ended up chasing my tail more than administering the box.
Of course, I liked Slackware due to the lack of general advanced packaging, since I didn't end up breaking a package management system with third party compiled-from-source software.
Do not look into laser with remaining eye.
In other words, there should not be any "pre-install" or "post-install" or "during-install" scripts inside packages.
Packages should contain data. That's it. Leave all the executable work to the package manager, like dependency checking, startup scripts, etc.
Letting packages run its own stuffs during installation is the root of all issues associated with uninstalls.
over 70% of the planet does not have boradband
[snip]
link your app statically, or include all your depedancies in your package. that is the only solution to this problem.
Um, you realize that both of those options basically mean "send all your dependencies with the package"...which doesn't decrease the download time at all? Particularly if more than one program does it? (you can end up installing 50 copies of part of libXYZ.a instead of one shared copy)
Daniel
Hurry up and jump on the individualist bandwagon!
-pipe is only good at saving compile time.
;)
-fomit-frame-pointer makes it a bitch to debug things - I don't like making debugging harder.
Yes, APT is a great piece of the puzzle, but it doesn't solve everything - for example, in Fedora, there was a package checked in for Bittorrent-GUI, which required a package wxPython, which exists NOWHERE on the 'Net. Smooth.
That is one thing I will give the Debian folks - you make your package work or you don't get in stable.
www.eFax.com are spammers
You've had XP installed for a year. Charming. That's one version of the OS. Wait for Longhorn to come out, and then whatever comes after that, and see if you can go through all the upgrades without reinstalling. Until then, you can't draw any conclusions.
One year is a lot of time by Windows 95 standards for not having to reinstall. In contrast, by Debian standards, you go through many, many major revisions of the core OS, and then don't have to upgrade.
I would also question how many major revisions of applications you could have gone through in one year. Under Debian, I have gone through hundreds or thousands of programs, from pre-alpha to beta to release to several major versions ahead, and never had a problem, in many long years of use. One year is nothing. Wait for the next version of XP, 3 more versions of Office, and a couple of Internet Explorer, keeping a couple of old applications incompatible with the new DLLs, and then tell me Windows doesn't have bit rot. Keeping the same version of an OS installed for a year says nothing about bit rot.
This is exactly what should be done, but people here are too brainwashed to admit that perhaps some features of Posix should be changed.
/proc/self/exe and use it to setenv LD_LIBRARY_PATH, and locate the actual executable. It then exec's the actual executable. In our case the wrapper also makes sure argv[0] is a clean full path name so the actual executable can use it to locate data files and plugins, you could also put this function in the main program if you wanted.
It is possible to fake this, we do it with our commercial Linux software, and it is obvious that other commercial software does this:
*All* files for your software are dumped into a single directory, so installation and removal is trivial (this is similar to OS/X App directories). In that directory is a single "wrapper" executable, usually with the name of your program, and only dependent on glibc. The main thing this program does is read
We then install Gnome and KDE shortcuts on their start menus directly pointing at the full pathname of the executable. Anybody who wants to run it from a shell has to type the full path or symbolically link it themselves into a directory on their path. Deleting our software leaves these start menu items and any symbolic links pointing at nothing, which as far as I can tell is not a real problem.
Intelligent users can easily peek into our directories and see that we ship libtcl and some others in an attempt to hide system differences. They can rename or remove these to better integrate the software into their system. But at least it works, first time, for any users.
I would like to see Linux altered to handle this cleanly, by adding direct support for "app directories". Unlike OS/X, I think they should be named with no extension. The rules are simple: if the user tries to run "foo" and it is a directory, it tries to run "foo/foo" after first adding foo to the LD_LIBRARY_PATH and after fixing up argv[0] to be the full expanded path to foo, and possibly other fixes for stupid Posix rules. This would get rid of 99% of the "install" headaches.
I suggest a pro/con comparison of each.
.msi installer may be uninstalled through the add/remove control panel.
*Windows*
Pros
+ Install is double-click, then run through the options for install (directory to be used, full or partial install (games and such).
+ Library files are either included with windows or are installed by the program, libraries are placed in the install directory, or are placed in windows/system32. Registry entries are entered by the program, as well as menu shorcuts and/or desktop icons.
+ All packaging is handled by the programmer, from the users standpoint, all installers function the same.
+ Anything, including drivers can be installed from a gui installer, installing video drivers does not require exiting to a CLI.
+ For most programs there is only one version, compatible across all Windows versions (occasionally, there will be a version for the 9x family and one for the NT family.) Drivers and such are for each version. Some will specify which service packs must be installed.
+ Program's files are contained in their directory in "Program files". There is one main location for installed programs (easy to find)
+ Even programs that do not use a
+ I never have to deal with dependencies, ever.
Cons
- No central repository for updated versions of software. User must download patches or new versions from the software author, or a third party. Some software provides it's own update mechanism.
- Poorly coded uninstallers may leave behind libraries or files/folders and registry entries.
- Binaries may be larger than needed since they must include all libraries not included with windows. (somehow, firefox is a smaller download for windows than it is for linux).
- Some programs may require a reboot after installation. Some windows updates require a reboot.
Neutral
= No CLI installation or uninstallation of programs.
= Multiple installers exist (installshield, cab self extractors, nullsoft installer, sfx archives, etc.). All of these are executables that function approximately the same from the users point of view.
= All updates to windows components are handled through windows update.
= Newer programs may save user files (profiles, saved games, etc.) in a folder in My Documents. Microsoft is said to be pushing centralization of saved files to the "My documents" folder. For example, image files are saved to "My Pictures" by default, and new saved games will be saved under "My Games".
I'd really be interested to hear what you think the pro's and cons are with the current linux methodology. I guess I'm having trouble understanding why the ease I experience with windows programs would somehow be unfit for Linux.