Binary Package Formats Compared

jjaimon writes "There is a document on different package formats used in Linux/Unix systems. Worth reading." Another reader sends in this guide to creating Debian packages which seems apropos here.

Comments

Binary packages: Security suicide

[Anonymous Coward]

I don't understand why so many ostensibly clueful people are so enamored with the whole concept of binary distributions. One of the biggest selling points of open source software is that it is precisely that -- open. You can download the code, look at it, read through it, and then compile it yourself so that it is completely optimized for your individual CPU. You can also make sure that there are no virii, worms, Trojan Horses, etc. that you are about to introduce into your system by reading the code before you compile it.

Obviously, for large software packages, you probably don't have time to read every last line of code (I know I certainly don't.) But what I generally do is untar the source and then grep through it for suspicious things. For example, if it's not a networking application, then I'll usually grep for system calls like connect() or bind() .. a dead giveaway that somebody has tried to slip some kind of a backdoor into the code. If I find something like that, I rm -rf it and forget it. You can also search for calls like gets(), strtok(), fopen(), and other well-known security risks and fix them if you feel like it.

All of this goes away if you download and install binary packages. You may as well just hang your Linux box out on the net with 500 open ports and no firewalls. You may as well be using Windoze!! Because a well-hacked program will allow the hacker to get at your data, firewall or not. And with binary formats, you have no way of knowing if the software you install is safe. The only way you can be completely sure is to read the source. As a side benefit, you can pick up a lot of programming tips as well.

To paraphrase Bob Dole: Binary packages -- just DON'T do it.

Re:Binary packages: Security suicide

[dissy]

One of the main reasons I enjoy DEB binary packages which also do not have any of the bad issues you mention is this:

I'll download the source and compile it on my nice fast zippy P4 2.6ghz machine, then build a .deb from that, not just to install on that machine (Personally I like the fact i can remove it with a package manager) but i can also install that binary package I made myself on my slower systems that dont have a compiler installed.

My three router computers are all p133 or p166 machines. No way am I compiling anything there. Routers also dont need gcc installed.

I run my own private apt repository for this (Its just apache and some config files in text format, and one more line in my apt sources file.)
This way I can tell whichever machines to apt-get it, and later I can apt-get remove it as well.

I also dont have a huge server farm, I just have 8 machines at home for different purposes. Below the P4 mentioned above, my next fastest system is a p2 450. The others get way slower below that (p2 200 and the like, or worse.)

I also dont want a compiler on any system but my development system. The machines in my DMZ dont need compilers, as if they do happen to get rooted, that's one less tool for them to use aginst me.

Open source is also about choice. Please let us have ours, even if you have made your own.

Whats the point?

[ePhil_One]

I'm just not sure what the point of this is.

While at first glance it seems heavily slanted towards Debian's .deb packages (Its first and contains more "YES"'s than the competition), as a developer I'd be far more concerned with basics like "market penetration" than whether it allows me to assign my package a "priority" over other packages.

I suppose it might be of use to folks building their own distribution, but I expect thats a pretty short list.

But personally, I tend to grab the source when I'm adding something that RedHat didn't include (or seems woefully out of date)

Deb vs RPM

[GoRK]

I'm sure this will erupt into a huge flamewar like the last time it was posted when all it boils down to is that it doesn't really matter much to end users about the package format as long as the installation and upgrades are made easy. For me, aptitude/apt with .deb packages has proven easiest, but a lot of people like apt with RPM or RedCarpet or rpmfind or whatever else is out there. Lindows users use the 'one-click' install thing not even caring that there are .deb's behind the scene.

Part of the reason, I think that the deb format has always seemed to hold together really well is that most all of teh deb using distributions are so tightly integrated with the main debian distribution that packages are always totoally interchangeable (and are very good to notify you when they will not work.)

RPM on the other hand is adopted by many different and sligtly incompatible distributions that often finding the libraries and applications you want to install is difficult not because RPM's are hard to find but because RPM's that work in your current setup are hard to find.

This is simply why the management tool(s) on both ends (creating packages and maintaining installed packages) matter way more than the package formats themselves. Deb's are very compicated but sometimes easier to deal with because of all the good debhelper tools. RPM's are most often more 'hand-crafted', but they are a lot easier to create from scratch for many people.

The thing I really hate about deb's is the lack of signature verification. It's absolutely central to the development/upload/build process but until very recent efforts has been a total pain to use on the installation front. There is no good reason for this either.

~GoRK

Re:Why get binaries

[agrippa_cash]

I love Gentoo, but I can entirely understand someone not wanting to spend 18 hours compiling OO.org or KDE. Or even half an hour getting Moz-Frirebird. All I know about the Gentoo reference binaries is that they exist. But they weren't covered and since they seem to be the (distant) second choice for the Gentoo devs, I doubt it is as easy to use as portage.

LINUX needs to tell apps where they live!

[Arslan ibn Da'ud]

What Linux really needs is a dir-independent application running
system. Imagine a package of...oh, say, g++, where g++ runs properly
even if you move the whole g++ package to a different dir (say from /usr/bin to /usr/local/bin). Most packages, including g++, configure
themselves to run in one location, and they'll get confused if you
move 'em.

Some packages (eg Tomcat) let you move them and they'll still
work...but only if you set an environment variable (eg TOMCAT_HOME)
so that Tomcat now knows where it lives. In a proper environment, an
application could easily & consistently know where it currently
resides on the filesystem *cough* OSX *cough*.

What Linux needs is some standard 'run-app' script that would inform
a package of its location. For instance:

% run-app tomcat

Run-app would be simple, say, the following:

#!/bin/sh -f

$app = shift
$location = `which $app`
env {$app}_home = $location $location/bin/app

That would enable Linux to devise a package format (or better yet,
improve rpm, deb, etc) for more flexible package management.

A package would no longer need to place its binary, libraries,
manpages, etc. all in hardwired locations in the OS...it could just
leave them in its original dir. (or maybe create a 'obj' dir that you
can remove if you wish to clean up the package.)

Re:LINUX needs to tell apps where they live!

[baka_boy]

Check out GNU Stow for one simple implementation of this; also, see the appdir functionality in OS X, where all the resources an application needs (binaries, shared libs, pixmaps, etc.) are bundled into a single directory structure, which is made opaque at the Finder level. For Linux, the ROX Filer project is trying to do something similar, but also has the advantage of backwards-compatibility with traditional (i.e., '/usr/bin', '/usr/local/bin') installation paths.

Personally, I find the directory layout of most Linux systems to be painfully baroque, with the BSDs just a step behind. Both kick the crap out of Windows system layouts, esp. when it comes to quick configuration tweaks and the like, but the simple fact that you have to know how to do shell scripting to install applications for yourself only is rediculout IMHO. I can do it, but it'd be a lot easier for people I'm trying to get started on Linux to never have to worry about entering a password every time they want to install a new version of the Same Game.

Tradeoffs

[autechre]

Security is always a tradeoff between the effort required and the risk entailed. SSH is more secure than telnet. Even more secure is making all of your employees carry around OTP calculators, but most places don't do this because for them, it's not worth the effort. At a certain point, the "price/performance" tradeoff curve of security starts to get very steep in the direction of "price", and the only way to get 100% security is to have 0% usability (as often noted by the example of unplugging the computer and encasing it in concrete).

So, how much do you need to trust your packages? Do you have enough work and not enough top-secret data that you can trust the package maintainer, the upstream maintainer, and your copy of MD5? For most people, the answer is "yes". This does not apply to X Random Freshmeat App; if you're downloading a new program and installing it yourself, you should check it out first (if you have the means), since sometimes even good authors do things that are unintentionally destructive. But most people can afford to trust that a package which has been around for a while and comes from a reputable distributor is reasonably safe, especially if they're doing the work of 3 people, maintaining 5 platforms, and just trying to keep up.

Unless you're in a situation where many people want your very important data, you can usually afford at least a little well-placed trust. Otherwise, just keeping up with updates is going to consume an inordinate amount of your time, and the rest of your duties will suffer.

Ah yes, packaging

[Dark Lord Seth]

Definitely one of the features which makes Linux a powerful OS. A good and well-configured packaging system can be a blessing, automagically resolving dependancies or at least telling you where and how things will fuck up. The problem with package managers is that there are quite a few of them around. Normally, diversity would be a good thing but those package managers don't seem too willing to process eachother's packages...

For example, RPM packages are almost common these days; most open source software has a few packages ready to be implemented. Pretty much the same thing with Debian packages, because of the large userbase. Chances are that a few hours after the release of a major product someone has made a .deb somewhere, ready for you to install. However, if you'd look beyond these two packaging systems, you'd get a few nasty surprises...

The TGZ packaging scheme (also mentioned in the article, along with RPM and DEB) just... Well... Sucks. Or at least in Slackware, I don't know if any other distributions use it differently but lets use Slackware's TGZ system as an example for now. What's wrong with it, you ask? First of all (and possible the foremost reason) it's almost unused. Apart from the Slackware packages itself, I've never seen anyone distribute something in the TGZ format which worked. That excludes the few things which I found and simply refused to install. It doesn't do dependancy checking, conflict-checking, heck, it doesn't do anything or so it seems. I'd continue but ranting about the bad parts of Slackware isn't the issue at hand.

The issue at hand are the two remaining package systems, which might be technically sound and quite useable, but they still won't have allot of use. Who here has ever heard of SLP and PKG packages? And even then, who here knows of any major applications which distribute their software using those package systems? Sure, SLP and PKG might be a dream to use, but without any actual packages to install, they're (possibly sadly?) not really of any value.

Which brings us back to RPM and DEB, apparently two of the most common systems, courtesy of Red Hat and Debian. Looking at the list of summed up data, it's really not a miracle those two are more common: Both support mostly options listed, both are backed by a large amount of users/developers and both are relatively easy to use, yet still distinct. Perhaps a system which allows multiple systems to cooperate (regarding dependancies and conflicts and the like) would be a nice compliment to both RPM and DEB?

Crackpot security

[pv2b]

Um.

Yes, and it is completely impossible for users to (gasp) compile their trojans elsewhere and FTP them over?

Also, are you saying that you're compiling your stuff as root? Bad idea, since compiling software does not require root priviledges. A better idea is to compile your software as a user, and then "make install" as root through sudo.

There have been cases in the past in which open source software source code has been backdoored, so that running the ./configure script connects to a remote server giving it a shell. Running ./configure as root makes this potentially even more destructive. The ./configure script is the last you audit for security holes, no?

Sure, you could argue that "make install" is backdoored, but it's always a good idea to check exactly what scripts you're running through "make install" anyway, since that's done as root.

Run as little as possible as root, and don't fool yourself that chmodding user-space programs not requiring root privileges is somehow improving your security.

Ah, another linux-only discussion

[softweyr]

Gee, I would've loved to have seen this include the BSD package format, and perhaps the Mac OS X one too. Sigh. For what it's worth, they score fairly well on the feature comparison chart, similar to RPMs.

All of these formats could be done better. The OpenPackages project had a design project underway to consider the features of an ideal, multi- platform package format early last year but it seems to have died from lack of input. It'd be great to see it get a breath of new life. If nothing else, this article could serve as a starting point for what we do and don't like about current formats.

Its about how you can get them...

[Kegetys]

I cant read the article since it seems to be slashdotted, but in my opinion the RPM's and DEB's etc. seem all equally good, the difference comes from how easily you can get them and whatever the package depends on. I wouldn't care if my Debian box would use RPM's, as long as I could still use apt-get to grab them and the dependencies.

Re:APT (DEB) vs ??? (RPM) again.

[fo0bar]

While I agree with your statements, I just thought I'd butt in and fill in the "???" blank: up2date. Yes, it requires an RHN subscription (but if you're seriously using a redhat distro, $5/mo (1yr@$60) is not the end of the world), but in its basic function, it performs the way apt-get or emerge does. "up2date evolution" will grab evolution plus all of its dependencies, and will block you if you have any conflicts.

My biggest gripe is not the commercial aspect of it, but it would be nice if you could add secondary up2date servers, or if redhat released an officially-supported up2date server so you can manage a bunch of redhat machines without having to go directly to redhat's up2date server for each machine. (There is a program called "current" that mimics most of the up2date server functionality, but it's not perfect.)

What we need

[anshil]

is to distance ourselfs from the system V filesytem, and have each package installed in it's own dir, this would make things so much easier.

And instead of the old PATH environment variable idea, think of something new, how about a central file (with user modifyable sub-files) that contains a list of all binaries to be called by default.

Or about a package tree in the bash memory, that holds the information which binaries are callable... etc.

There are so much ways to get rid of PATH, and with PATH away, nothing speeks anymore against installing every package in it's very own directory, making administration and package management so much easier...

Zero Install

[tal197]

Zero Install

"The Zero Install system makes software installation not merely easy, but unnecessary. Users run their applications directly from the Internet from the software author's pages. Caching makes this as fast as running a normal application after the first time, and allows off-line use."

• No new commands to learn. All software in the world appears as part of your filesystem (/uri/...) and is cached on demand.
• Secure: nothing is run as root (only as the user who runs it), and GPG signatures can be checked automatically when upgrading
• Network efficient: only what is needed is fetched (no documentation or headers until you need them, then they get fetched automatically)
• Faster: no searching for resources -- everything is referenced by URI; only download package indexes per-site, not for the whole distribution.
• Easy to package for: just make your tree available via an HTTP server.
• No need for depends/recommends/suggests: whatever is needed is fetched when you access it.
• Easy to uninstall: remove anything you don't want from the cache at any time -- it will be fetched again later if needed.

Experimental, but give it a try. See especially the comparison with apt-get and the security model documents.

Re:Zero Install

[MntlChaos]

and you thought cross-site scripting was bad? now we have executables getting copied to your computer from god knows where (read 10 layers down in the dependency tree). now you have to worry about not only what program you are running, but every program it claims to depend on (etc, etc.). one word: ouch.

Re:Zero Install

[Anonymous Coward]

now we have executables getting copied to your computer from god knows where (read 10 layers down in the dependency tree). now you have to worry about not only what program you are running, but every program it claims to depend on (etc, etc.). one word: ouch.

And this is different to any other packaging system how, exactly? If you run 'gimp', then you have to trust its authors. Whether they have created a dynamic link to another site, or copied-and-pasted the code in directly makes no difference to your security.

Pkg format

[xenophrak]

It is important to note, that for the PKG format (Solaris, etc) that System V does not define certain functionality, but the system vendor may include it. I know that Solaris does support ghost files, editable config files, virtual packages and boolean dependancies, but the chart doesn't reflect this.

Not that this helps out the Linux community in the slightest... ;)

I still think Apple's (NeXT's) App format is still the best. A complete archive including all features and bits of an app that can be moved around at will.

My .02

wolfenstein ET installer

[Anonymous Coward]

I don't know what format it's considered (.run file), but did anyone else install Wolfenstein Enemy Territory on linux? It seriously couldn't have been easier. I executed the file on Mandrake 9.1, a nice little graphic popped up with a logo telling me that I was installing the program, it gave me options of where to install the files to, whether or not to add an entry to the Kicker, etc.

This was the closest thing to a simple Windows installer that I've ever seen on Linux. Absolutely painless, n00bie friendly... exactly what Linux needs to assist in penetrating the desktop.

OOOOPS

[Allnighterking]

Couple of points.

He states that rpm is not unpackable by standard tools.
Can an experienced user, when presented with a package in this format, extract its payload using only tools that will be on any linux system? They can remember a few facts to help them deal with the format, but remembering file offsets and stuff like that is too hard.

First problem I have is with the "any linux system" Ummmm I've a Linksys router running Linux that can't do jack with any of these. Next an RPM is actually a cpio archive rpm2cpio is actually just a tool to shortcut what is doable with cpio. This applies as well to all of the "standard tools" statements. I also would like to point out that standard depends on which standard you use. Posix, LSB etc. In that rpm is a standard of LSB but not of Posix.

His statement that binary programs are not allowed.
Must these programs be scripts, or can compiled binaries be used as well?

This is very, unclear. Can I execute a binary from within rpm. The answer is yes. I do it all the time. Can RPM be made directly from binaries (skiping all of the build etc.) Yes it can. Can I embed the binary in the RPM and not have it ever get installed... no. But I can run it then remove it before RPM finishes.

Suggestions ... he states that RPM doesn't have them.
A suggestion says a package may sometimes work better if another package is installed. The user can just be informed of this as a FYI

This is really the fault of the packager not of the product. There are two areas for comments which can give you this kind of data ... but it's up to the packager to use the tool. Second the author needed to get a little deeper into rpm's queryformat (info here) He would have found much of what he needed.

Statement that RPM can't do Boolean Relationships.
This means that a package can depend, conflict, etc on a package AND (another package OR a third package). Any boolean expression must be representable, no matter how complex.

RPM does have the conflicts and the depends paramaters that can be set. Once set you can't install a without b and c, plus removing y and x.
HOWEVER he is very right about the boolean "or" being missing... I've been championing this one for a while (I've talked with some of the developers.) but it seems it hasn't up till now been high enough on the horizon to have someone take a shot at it. (Sorry but it's beyond my ken to work on this personally) So I will keep politely advocating this until it does break the plane of need.

New Section

Sorry but this stament is just too nebulous. It's been coping with the unforseen for years. Just as debian has. That's why it get's upgraded. That's in fact a lot of the reason for the new version coming out now. To make the format more modular. and easier to mutate as times change.

All and all the article seems well done. However I'd say the chances are pretty strong that the author is a Debian fan. My personal recommendations would be. One lose the subjective nature of a number of statements. Next, when doing research be careful how you ask a question. Often times asking "Can this product do X" will yeild a no. But if you ask .. On a system that uses this product how do you do X" will yeild a completely different answer.

I'd give this article all in all a 6 on a 1 to 10 scale for research. a 3 for new info. and a 7 for layout and style.