Slashdot Mirror
Binary Package Formats Compared
jjaimon writes "There is a document on different package formats used in Linux/Unix systems. Worth reading." Another reader sends in this guide to creating Debian packages which seems apropos here.
← Back to Stories (view on slashdot.org)
Why don't you just ask what everyone's favourite distro is?
Apt-get, no emerge, no make world, ARRRGH!
It seems to me, if you want to cover everything, you would put a debian package inside a RPM.
Why? I have no clue, but you could if you wanted all the features. Kind of like putting a roll bar into a SUV, so that you can start amature racing.
=================
Unix is very user friendly, it's just picky about who its friends are.
It is silly for us to ignore such a large divide in our community though. It exists, therefore we should look at it in depth so we can improve it and better the community as a whole. We shouldn't set off a battle cry for the entire community to come together if that's not really what's going to happen, at least not in the reality that i see. but then again my personal reality distortion field is out of whack this week.
The distroes are quite divided. How 'bout we look at what is specific to each distro so people can chose which is best for them. That is a better way to garner support from the outside community, doncha think? Don't leave them in the dark, that's what scares people away from Linux.
First time I've ever seen guys compare packages and size doesn't seem to matter.
Not compared: my favorite package
* Please do not read my signature.
I'm sure this will erupt into a huge flamewar like the last time it was posted when all it boils down to is that it doesn't really matter much to end users about the package format as long as the installation and upgrades are made easy. For me, aptitude/apt with .deb packages has proven easiest, but a lot of people like apt with RPM or RedCarpet or rpmfind or whatever else is out there. Lindows users use the 'one-click' install thing not even caring that there are .deb's behind the scene.
Part of the reason, I think that the deb format has always seemed to hold together really well is that most all of teh deb using distributions are so tightly integrated with the main debian distribution that packages are always totoally interchangeable (and are very good to notify you when they will not work.)
RPM on the other hand is adopted by many different and sligtly incompatible distributions that often finding the libraries and applications you want to install is difficult not because RPM's are hard to find but because RPM's that work in your current setup are hard to find.
This is simply why the management tool(s) on both ends (creating packages and maintaining installed packages) matter way more than the package formats themselves. Deb's are very compicated but sometimes easier to deal with because of all the good debhelper tools. RPM's are most often more 'hand-crafted', but they are a lot easier to create from scratch for many people.
The thing I really hate about deb's is the lack of signature verification. It's absolutely central to the development/upload/build process but until very recent efforts has been a total pain to use on the installation front. There is no good reason for this either.
~GoRK
There are different levels of package management which often confuses the newcomer into believing (dogmatically) that one is better than the other.
The installable packages themselves have to have flexible dependency markings and coherent version markings. The low-level package tool has to be able to install and uninstall packages cleanly and repeatably. Seems like the dpkg/deb suite and the rpm suite are quite comparable here.
The package manager has to be able to build a requirements tree for a desired package, and then fetch all of the required packages to fulfill those dependencies on the local system. It should offer trust or signature verification to ensure only trusted repositories and trusted packages are used. The apt tool seems to be cross-platform, while non-Debian distros often spin their own service model here: up2date, Red Carpet, and whatever Mandrake and Lindows offer are each commercialized with some amount of sample access.
Lastly, the most important criteria, is the repository itself: it should contain packages which are clean and trustworthy. There have been cracking incidents, and there will be more. The quality of code between distro-produced packages and externally-produced packages can be as different as night and day. The package's meta-data and manifesting information can be crap, or it can be carefully constructed. The embedded installation scripts can be trivially exploitable or they can be carefully scrutinized against unexpected results.
Even if your package format is cool, and your package manager is cool, consider the repository. If the repository is not secure and offers poorly tested packages, many folks are going to unfairly blame it on the tools.
[
Are you smoking crack?
Having a compiler available on all of your systems to compile C code is far greater risk than the "threat" of getting trojaned builds from Red Hat.
Take your tinfoil hat off and breathe.
Conformity is the jailer of freedom and enemy of growth. -JFK
Joey Hess created that document (at least the first revision) around 1998 IIRC, so it's not so much new news (guessing it's been posted here before, but probably around then as well)
What Linux really needs is a dir-independent application running /usr/bin to /usr/local/bin). Most packages, including g++, configure
system. Imagine a package of...oh, say, g++, where g++ runs properly
even if you move the whole g++ package to a different dir (say from
themselves to run in one location, and they'll get confused if you
move 'em.
Some packages (eg Tomcat) let you move them and they'll still
work...but only if you set an environment variable (eg TOMCAT_HOME)
so that Tomcat now knows where it lives. In a proper environment, an
application could easily & consistently know where it currently
resides on the filesystem *cough* OSX *cough*.
What Linux needs is some standard 'run-app' script that would inform
a package of its location. For instance:
% run-app tomcat
Run-app would be simple, say, the following:
#!/bin/sh -f
$app = shift
$location = `which $app`
env {$app}_home = $location $location/bin/app
That would enable Linux to devise a package format (or better yet,
improve rpm, deb, etc) for more flexible package management.
A package would no longer need to place its binary, libraries,
manpages, etc. all in hardwired locations in the OS...it could just
leave them in its original dir. (or maybe create a 'obj' dir that you
can remove if you wish to clean up the package.)
Practice Kind Randomness and Beautiful Acts of Nonsense.
Believe it or not, like (essentially) all things, binary packages have a purpose, just as source packages have a purpose--platform agnostism.
Before giving me an explanation as to why you (read, the parent poster) in particular would not have a use for binary packages, allow me to explain why binary packages are useful. In the majority of instances, binary packages are useful when one is installing the userland on a system, or when installing a compiler, when you have no other systems to build the compiler on. Binary packages are also handy for systems where compiling from source would be inconvenient, resource-intensive, and time consuming.
Also, there are some proprietary applications that are not available as source, so a logical manner of packaging is with a standard binary packaging system such as RPM or dpkg.
Even NetBSD has its own binary package format (no, not the sets, those are for the base install and are just tarballs without package information).
All in all, binary packages are very convenient, despite the inconveniences caused by vendors who do a poor job of managing their package collection and dependencies. Binary packages are single files, smaller than source archives in most instances, and are installable in a uniform manner.
Let's not get into rogue "package vendors" who package trojans. They are the minority, and most reputable software developers release their own binary packages along with sources anyways.
I think I need a glass of water.
-
And the Angel said unto me, "These are the cries of the carrots! The cries of the carrots!"
One of the main reasons I enjoy DEB binary packages which also do not have any of the bad issues you mention is this:
.deb from that, not just to install on that machine (Personally I like the fact i can remove it with a package manager) but i can also install that binary package I made myself on my slower systems that dont have a compiler installed.
I'll download the source and compile it on my nice fast zippy P4 2.6ghz machine, then build a
My three router computers are all p133 or p166 machines. No way am I compiling anything there. Routers also dont need gcc installed.
I run my own private apt repository for this (Its just apache and some config files in text format, and one more line in my apt sources file.)
This way I can tell whichever machines to apt-get it, and later I can apt-get remove it as well.
I also dont have a huge server farm, I just have 8 machines at home for different purposes. Below the P4 mentioned above, my next fastest system is a p2 450. The others get way slower below that (p2 200 and the like, or worse.)
I also dont want a compiler on any system but my development system. The machines in my DMZ dont need compilers, as if they do happen to get rooted, that's one less tool for them to use aginst me.
Open source is also about choice. Please let us have ours, even if you have made your own.
Security is always a tradeoff between the effort required and the risk entailed. SSH is more secure than telnet. Even more secure is making all of your employees carry around OTP calculators, but most places don't do this because for them, it's not worth the effort. At a certain point, the "price/performance" tradeoff curve of security starts to get very steep in the direction of "price", and the only way to get 100% security is to have 0% usability (as often noted by the example of unplugging the computer and encasing it in concrete).
So, how much do you need to trust your packages? Do you have enough work and not enough top-secret data that you can trust the package maintainer, the upstream maintainer, and your copy of MD5? For most people, the answer is "yes". This does not apply to X Random Freshmeat App; if you're downloading a new program and installing it yourself, you should check it out first (if you have the means), since sometimes even good authors do things that are unintentionally destructive. But most people can afford to trust that a package which has been around for a while and comes from a reputable distributor is reasonably safe, especially if they're doing the work of 3 people, maintaining 5 platforms, and just trying to keep up.
Unless you're in a situation where many people want your very important data, you can usually afford at least a little well-placed trust. Otherwise, just keeping up with updates is going to consume an inordinate amount of your time, and the rest of your duties will suffer.
WMBC freeform/independent online radio.
Best thing about RPMs? GPG signatures built in. Try rpm -K whatever-x.x.x.rpm next time. Second best thing? rpm -Va.
Get your own free personal location tracker
Because some people have older machines that take several days to compile large subsystems such as Gnome and KDE?
Because some people run dozens or hundreds of machine with identical configurations, so compiling the same package on every single machine is pointless?
Because companies prefer working against a known build of a piece of software for support reasons?
Source distributions are far from a panacea.
Definitely one of the features which makes Linux a powerful OS. A good and well-configured packaging system can be a blessing, automagically resolving dependancies or at least telling you where and how things will fuck up. The problem with package managers is that there are quite a few of them around. Normally, diversity would be a good thing but those package managers don't seem too willing to process eachother's packages...
For example, RPM packages are almost common these days; most open source software has a few packages ready to be implemented. Pretty much the same thing with Debian packages, because of the large userbase. Chances are that a few hours after the release of a major product someone has made a .deb somewhere, ready for you to install. However, if you'd look beyond these two packaging systems, you'd get a few nasty surprises...
The TGZ packaging scheme (also mentioned in the article, along with RPM and DEB) just... Well... Sucks. Or at least in Slackware, I don't know if any other distributions use it differently but lets use Slackware's TGZ system as an example for now. What's wrong with it, you ask? First of all (and possible the foremost reason) it's almost unused. Apart from the Slackware packages itself, I've never seen anyone distribute something in the TGZ format which worked. That excludes the few things which I found and simply refused to install. It doesn't do dependancy checking, conflict-checking, heck, it doesn't do anything or so it seems. I'd continue but ranting about the bad parts of Slackware isn't the issue at hand.
The issue at hand are the two remaining package systems, which might be technically sound and quite useable, but they still won't have allot of use. Who here has ever heard of SLP and PKG packages? And even then, who here knows of any major applications which distribute their software using those package systems? Sure, SLP and PKG might be a dream to use, but without any actual packages to install, they're (possibly sadly?) not really of any value.
Which brings us back to RPM and DEB, apparently two of the most common systems, courtesy of Red Hat and Debian. Looking at the list of summed up data, it's really not a miracle those two are more common: Both support mostly options listed, both are backed by a large amount of users/developers and both are relatively easy to use, yet still distinct. Perhaps a system which allows multiple systems to cooperate (regarding dependancies and conflicts and the like) would be a nice compliment to both RPM and DEB?
Hate me!
The binary distributions usually come with a way to compile your own binary packages if you wish to inspect the source code. Just download the source RPMs or use "apt-get source". The "binary distributions" are generally binary and source distributions where you can choose whether you use prepackaged binaries or compile your own.
All of these formats could be done better. The OpenPackages project had a design project underway to consider the features of an ideal, multi- platform package format early last year but it seems to have died from lack of input. It'd be great to see it get a breath of new life. If nothing else, this article could serve as a starting point for what we do and don't like about current formats.
are better than packages because you get more control over what is installed onto your file system. If you really want to the slight advantage that package formats provide over ports, use SRPMS. You will have some of the customizability of using ports and all the features that RPMs provide.
Every time I've bought a car I bring a socket wrench set with me and tear apart the engine right there on the dealership floor. If I didn't how could I know that Al Queda hasn't set me up to be a pawn!?! Not checking your equipment like this is tantamount to supporting terrorists.
Similarly, I could install a Debian binary package if that were all that existed for my particular environment, with a simple
(I digress, simple may be relative)
On the other hand, since RPMs have a special binary header, the lazy would be forced to install RPM and Berkeley DB on non-Red Hat-machines in order to build an RPM package. Though it is possible to extract the gzip'ed+cpio'ed data in an RPM without using rpm.
So, in my view, Debian has a bit of an upper hand in simplicity, from a technical standpoint, but not by much.
-
And the Angel said unto me, "These are the cries of the carrots! The cries of the carrots!"
Is that the rpmlib API is almost completely undocumented. As GoRK pointed out, management tools such as apt, rpmfind, up2date, etc. are far more important than the underlying package format.
But it's very difficult to create those management tools for RPM when the API is a "black art" known only to a few. Questions on the RPM mailing list/newsgroup will generally be met with the advice to "use the source, Luke"--all several hundred thousand lines of it!
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety."
Obviously, for large software packages, you probably don't have time to read every last line of code.
That is the understatement of the year. I would dare say that in order to read and understand a program that is on the order of five million lines, it could take you a year or two. For a non-expert programmer (or even an expert with no operating systems experience) it could take forever to just begin to understand something like the Linux kernel source.
But what I generally do is untar the source and then grep through it for suspicious things.
That's great if you know what you are an expert programmer (and if you think that a simple grep will help you that much.) But what of the small business that doesn't employ you? Do they need to perform that same review? Of course, they could skip it and just compile the thing, but that is the same as just using the binary packages!
You may as well just hang your Linux box out on the net with 500 open ports and no firewalls.
Baloney.
Because a well-hacked program will allow the hacker to get at your data, firewall or not.
A well-hacked program will be completely invisible to you, as well. Your grep methodology is too simplistic to catch any sort of sophisticated trojan. Even if you were to laboriously go through the code, line by line, you still wouldn't catch anything but the most obvious of hacks/problems.
The only way you can be completely sure is to read the source.
No, the only way is to not run it. Software is not a mathematical formula that you can "prove". Large programs are horribly complex, as you most likely already know. Binary packages serve a very useful purpose for many people. If you choose not to use them and to perform some limited form of code review, then that is great for you, but don't try to demean anyone who doesn't do the same.
Here is the link to googles cache of the site:
CLICK HERE
"If ignorance is bliss, why aren't there more happy people in the world?"
You cannot get your binary into the main site without going through a social screening process.
.rpm based installations.
This allows more standardization than amongst the various
Standardization means fewer problems for the rest of the users.
But it means more work for the people developing the packages.
All of this goes away if you download and install binary packages. You may as well just hang your Linux box out on the net with 500 open ports and no firewalls....The only way you can be completely sure is to read the source.
You haven't read the entire source of the GNU/Linux system you're running, so you have no business telling us that we ought to do it!
Why am I so confident you haven't read the source? Because it's not possible. Even a relatively basic Linux install will correspond to over 2GB of source code. That would be about 800,000 pages of a typical book (2500 to 2800 characters per page). Source code is typically much less dense in terms of characters per page, so it would be millions of pages. It would take you several years to read it, by which time the bits you'd read first would be long obsolete.
Why? A compiler is only a translator. How is the ability to translate an arbitrary C program into assembly going to aid an attacker? The attacker could just as easily precompile the program themselves.
Zero Install
"The Zero Install system makes software installation not merely easy, but unnecessary. Users run their applications directly from the Internet from the software author's pages. Caching makes this as fast as running a normal application after the first time, and allows off-line use."
Experimental, but give it a try. See especially the comparison with apt-get and the security model documents.
I (author) am currently enroute to Norway, only found out I was slashdotted in the airport. I don't really understand why they posted it today, and not some time in the past 5+ years.. Anyway, I will respond to anything worth responding to sometime later.
see shy jo
I've used many systems, and many package systems.. from old machines where there really was no concept of a package, to debian, with it's superb package management, and everything in between.
The only conclusion I've come to is this: The package format itself isnt' so important.. what matters is the whole system approach to packaging and distribution.
Take Debian. Everyone agrees, I think, that the debian package format, and apt, together make for a great system.. but that's because of the method of package distribution and tracking, not the packaging system itself.. that and the fact that it's fairly universal in the debian world. Several apt repositories make up basically all software available for debian... and it's a lot. SO the overall experience is "Great package management". It's not just about the format, but the people.. people know what's in the standard packages, and can refer back and forth to them, checking for compatability and whatnot. The overall appraoch to package management is what rocks.. not the binary format.
Look at OSX.. they have fink. Fink, if you don't know, is basically apt-get for OSX. Works fine, no problem... except, it puts stuff in it's own folder (/sw) and it doesn't necessairly know about apple stuff already installed.. it only tracks stuff that is in the fink repositories.
In other words.. it's useful, but it doesn't have the feel of a really great package system.. because the system itself isn't based on it.
People say "ports rocks" in bsd land... but why? Becasue it's superior? No.. just because it's a big collection of useful stuff that handles dependencies well. The actual package management system is extremely basic. But the system is more or less based on it, so it works very, very well.
Redhat.. is kind of a mess. Is it because rpm sucks? Heck no.. it's just because, well, the overall approach wasn't right.
OSX.. (yeah okay I'm a mac fiend now.. I admit it.). What package management? Apps tend to be one single file, which is a package containing all the bits and pieces. No real package management system to see what's installed or not.. and who needs it.. you can just go to the Apps folder and toss stuff in the trash to get rid of it. The system was designed to work that way. so it works really well. You don't say "Gee I wish the system tracked apps" because it's so very simple to get rid of them, and to ferret out any pieces they may have left behind, which is rare..
So overall... the complexity of the package management isn't as important as everyone sticking together on how things are going to be installed and removed. If everything works the same way, it doesn't really matter how sophisticated it is.