File Packaging Formats - What To Do?

Jeve Stobs writes: "It seems that nowadays, there are three ways of distributing a program. In a tarball (be it a .gz or .bz2), in a Debian package, or in a RPM. These are all fine methods of packaging a piece of software, but they each have their places, and they aren't as comprehensive as I would like. I really think that, as we move into a broader user base with the variety we have so far (not to mention the variety we are likely to have in the near future) that a new method of software distribution is needed. osOpinion has an excellent editorial piece which details some solutions to this growing problem."

Comments

the biggest problem with RPM...

[stevenj]

...is the lack of a central packaging organization, I think. This results in (at least) two problems.

• First, if you go to a site like rpmfind.net, you'll find a zillion different versions of a given package, all with different numbering schemes. Besides the obvious duplication of effort, it is often not clear which one to use, especially if you want the "latest" RPM (or SRPM so you can build on an atypical architecture).
• Second, suppose you find a program that is not packaged already, and the authors don't want to package it themselves, so you want to package it as a service to yourself and the Linux community. There is no guarantee that 10 other people aren't doing the same thing; you don't have any chance to become the "official" RPM maintainer for the program. Moreover, users will have no way of knowing whether you can be trusted. Finally, there is no definitive place to post the finished package (see above).

One of the strongest points about Debian, I think, more than any minor technical distinction between .deb and .rpm, is their strong centralized packaging organization that solves the above problems. (Based on this, they can also make nicer tools e.g. for automated network-based updates.)

Now, if only Debian stabilizes their new release so I can install it on my PPC...

Packages are nice, but.....

[ssd]

Packages are nice, but really, if we want to see Linux make it in the Real World (TM) we need to advance to the 21st century and recognize that we must have a simple and easy means of installing new software. I propose that we follow Apple's lead in this area and move to Self Extracting Executables, ELF binaries that you run and will extract the software and install it for you, without the hassle of remembering arcane flags and what program you're supposed to use.

Just look at it! The author of the article makes a very good point. How many of you really look over a binary before you install it? Do you just rpm a package and then run it? What do you have to lose from running this random binary? No more than if you downloaded the rpm and ran the binaries contained therein.

What we as the Open Source community need to do is create a standard install tool, like what windows programmers have with the install shield software. The software could be customised, and could be written to reconize the directory structure, and install the files in the correct locations, maybe through the use of certain files that would be kept in a standard location, like /opt/local/lib/install/.

So, whaddya think? Think we can get something whipped together? Even just a proof of concept done in a combination of perl and python? We certainly don't want to make the mistake OS/2 made and not at least keep up with what windows has had for years.

-ssd

Learn from Other Platforms

[goingware]

Learn from other platforms.

And I mean this not just in regard to installers and packages, but everything.

And no, I'm not proposing that what we need to do is make Linux look more like Windows or the MacOS.

But there are problems that others have solved and we can draw on their solutions, even if we can't use their source code.

(Even when I was working at Apple I would tell people about stuff from SunOS or Linux that I thought would go good in the Mac - they wouldn't hear of it).

I think an indicator of the problem we face in trying to bring Linux to the desktop was when I was corresponding with RMS about things I thought would be helpful to the users and I suggested an installer. He replied "What's an installer?"

The best installer I've ever come across on any platform, both to create packages with and for the user to install products with is Mindvision Vise.

It would be worthwhile to find a friend with a Mac and download it, and make a little toy installer that installs SimpleText and a readme file to try it out (you can download it for free - the installers created with it complain that you've lifted it until you get a valid serial number. It is possible to get a serial for free for installers for freeware).

It beats the living hell out of anything I've seen for Linux.

BTW - if you want to see an installer that really blows, check out PackageBuilder/Software Valet for the BeOS. The thing drove me to distraction. It wasn't just the way it would corrupt the data in my archives or crash while users were installing my software with it.

What really drove me nuts is that it had no concept of updating an installer when I had built new software to go in it.

With vise you just drop your new files in the folder next to your installer project and tell it to update. It gives you a list of files that have changed and you can approve or disapprove of updating them (or deleting the ones that are now missing).

PackageBuilder requires you to delete the old file from the installer project, which loses its settings, then you have to go and add your file back in and reset your settings. This is probably the number one reason for every time I've been reluctant to release a new version of my software on the BeOS - I enjoy programming it but I hate the damn installer.

Don't make me install as root.

[Tough Love]

A new package format that would combine the strengths of the current various is a good idea, but (and it's a big but) I hope the author isn't the guy to do it (he doesn't have a good grasp of security..) He mentions making the package self-extracting - "just do a chmod +x on it, and away you go" Sorry, but that's why "security" in Windows is so abysmal.. Think about it - to install, you're running this file as ROOT - would you run a binary from an unknown source as root? (I don't even untar source as root!) - this is inherently a bad idea.

The vast majority of RPM packages don't have any valid reason for being installed as root. The main reason you have to have root right now is to be able to write the RPM database. Fix that. For example, give me a local branch of the RPM database and give root the ability to merge that with the main, shared RPM database.

Miscellaneous other reasons for needing to install as root:

• Making the package available system-wide. Why can't I install a package in my home directory, then ask via a root-privileged command to link it from /usr/bin? Or, more permanently, to move it there. Such a privileged command would still require security checks of course, but these checks can be very well-defined, and it would be easy to ensure that no suid programs were being installed. With a little more work, it could satisfy itself that no common user commands were being overridden (a technique favored by script-kiddies for hiding their root-kits).
• Installing device drivers. Solution: user-space device drivers. We're some distance away from being able to do that just now. For now, elevate privilege to root just long enough to install the device driver, and require root's password to install such a package.
• Installing daemons and privileged programs. Obviously, only root should be installing them. In many cases a daemon doesn't really need root privilege, and in that case it should run in user space and be installable by a normal user for their own use, or be shareable by the mechanism mentioned above.
• Changing system config files. The author should try to write the program so it does this only as a last resort. We should have a way of overriding certain system config files locally, for example, resolv.conf, so that we have a per-user view of the system configuration. As a last resort, the installer can invoke a root-privileged utility just to change the config files, and require root's password.
• (Other reasons, please enumerate.)

This is all by way of saying that we can have mindless (read: stressless; user-friendly) auto-installing packages just like Windows, without breaking security.
--

Re:Directory Structure First

[ruud]

We need to get someone to provide a definitive explanation of what each part of the file system is for, and how they should be used, so that we'll be able to say "RTFM", and have a sounder understanding of our own operating system.

See www.pathname.com.

--

don't install at all

[jetson123]

I think the whole notion of exploding a piece of software and spewing the bits and pieces all over the file system is wrong. Applications should be self-contained collections of resources (somewhat like .jar files or the Apple format); let's call them "application bundles" like Apple/NeXT does.

For stuff that isn't packaged with the applications, the package format should contain version information and checksums for any other files it depends on. It should declare those dependencies. The operating system should then be able to identify what is needed, possibly fetch it over the Internet, and give the package access to those files under a symbolic path name.

Such application bundles with dependencies should not (normally) be allowed to access the raw file system for their installation at all. In fact, even most applications should probably not be allowed to reference absolute pathnames. All references should be through symbolic paths. That way, one actually has a prayer of figuring out what the thing depends on, and it would also help with security.

Furthermore, the notion of "install scripts" is broken because it is difficult for anything to figure out what went wrong in the bowels of some script, and because scripts may do things to the system that are difficult to undo. Information about how an application bundles and their needs should be entirely declarative.

Another way of looking at that is that applications and application installation needs to work similar to objects and software components. In the past, programs consisted of functions that looked for, and modified, global variables all over the place. These days, good software components have well defined interfaces, get access to their environment only through those interfaces, and rarely use global variables. That's the kind of change that also has to happen at the level of applications.

The system that is closest to that in many ways is Sun Java: with jar files and the JavaBeans standards, there are well-defined ways for software to get installed and interact with the rest of the system, yet the environment can limit what new software components can do. We need something similar for Linux packages. That would also require additional naming and name translation support for Linux, similar to what Plan 9 offers (however, Plan 9 never adopted a good way of packaging applications based on their naming model--a shame, they were pretty close).