Slashdot Mirror
Binary Package Formats Compared
jjaimon writes "There is a document on different package formats used in Linux/Unix systems. Worth reading." Another reader sends in this guide to creating Debian packages which seems apropos here.
← Back to Stories (view on slashdot.org)
No matter how many "packages" I see. Still love slack-packs the best.
Come with everything you need, esp created to make you learn what the hell your doing before you do it.
argan0n
It is silly for us to ignore such a large divide in our community though. It exists, therefore we should look at it in depth so we can improve it and better the community as a whole. We shouldn't set off a battle cry for the entire community to come together if that's not really what's going to happen, at least not in the reality that i see. but then again my personal reality distortion field is out of whack this week.
The distroes are quite divided. How 'bout we look at what is specific to each distro so people can chose which is best for them. That is a better way to garner support from the outside community, doncha think? Don't leave them in the dark, that's what scares people away from Linux.
There are different levels of package management which often confuses the newcomer into believing (dogmatically) that one is better than the other.
The installable packages themselves have to have flexible dependency markings and coherent version markings. The low-level package tool has to be able to install and uninstall packages cleanly and repeatably. Seems like the dpkg/deb suite and the rpm suite are quite comparable here.
The package manager has to be able to build a requirements tree for a desired package, and then fetch all of the required packages to fulfill those dependencies on the local system. It should offer trust or signature verification to ensure only trusted repositories and trusted packages are used. The apt tool seems to be cross-platform, while non-Debian distros often spin their own service model here: up2date, Red Carpet, and whatever Mandrake and Lindows offer are each commercialized with some amount of sample access.
Lastly, the most important criteria, is the repository itself: it should contain packages which are clean and trustworthy. There have been cracking incidents, and there will be more. The quality of code between distro-produced packages and externally-produced packages can be as different as night and day. The package's meta-data and manifesting information can be crap, or it can be carefully constructed. The embedded installation scripts can be trivially exploitable or they can be carefully scrutinized against unexpected results.
Even if your package format is cool, and your package manager is cool, consider the repository. If the repository is not secure and offers poorly tested packages, many folks are going to unfairly blame it on the tools.
[
Are you smoking crack?
Having a compiler available on all of your systems to compile C code is far greater risk than the "threat" of getting trojaned builds from Red Hat.
Take your tinfoil hat off and breathe.
Conformity is the jailer of freedom and enemy of growth. -JFK
Because all those optimizations don't mean squat when most computers are not maxing the CPU on a regular basis anyway. Also, portage is not a silver bullet. While it's got a very good system for handling a whole lot of things (especially cool stuff like being able to peg versions or inject versions if you want to roll something by hand), it's not nearly complete. Look at the lame way it handles config issues by just post-poning that to etc-update, which isn't remotely user friendly, imho.
I do not have a signature
Believe it or not, like (essentially) all things, binary packages have a purpose, just as source packages have a purpose--platform agnostism.
Before giving me an explanation as to why you (read, the parent poster) in particular would not have a use for binary packages, allow me to explain why binary packages are useful. In the majority of instances, binary packages are useful when one is installing the userland on a system, or when installing a compiler, when you have no other systems to build the compiler on. Binary packages are also handy for systems where compiling from source would be inconvenient, resource-intensive, and time consuming.
Also, there are some proprietary applications that are not available as source, so a logical manner of packaging is with a standard binary packaging system such as RPM or dpkg.
Even NetBSD has its own binary package format (no, not the sets, those are for the base install and are just tarballs without package information).
All in all, binary packages are very convenient, despite the inconveniences caused by vendors who do a poor job of managing their package collection and dependencies. Binary packages are single files, smaller than source archives in most instances, and are installable in a uniform manner.
Let's not get into rogue "package vendors" who package trojans. They are the minority, and most reputable software developers release their own binary packages along with sources anyways.
I think I need a glass of water.
-
And the Angel said unto me, "These are the cries of the carrots! The cries of the carrots!"
Because there are folks that I do trust. I don't trust the latest nightly build of Mozilla, but I do trust the most recent stable release after it's been out a week or so.
You see, I know there are folks out there like you... so I don't have to be like you too. Enough hobiests and security folks will bang on popular newly released code to pacify my concerns. For specialty apps coming from unknown sources, care is taken and sourcecode reviewed. But, for code from the likeness of a major OS distributor (RedHat, Debian, etc) or a major code project (Moz, Apache, etc), I don't have to bother.
I want it fast and pain - free. Binaries please.
Support a few technologists in Washington.
If your paranoid enough not to trust the RPM Builders, the checksums, the download process, etc, then download the src.rpm; unpack the contents, and do your vodoo tricks. Then run rpmbuild to build your own damned binary package. I've done it (Ok, I'm a RHCE) and its cake.
Basically, this isn't a reason to dis binary packages unless your paranoia is well into the tinfoil hat level.
You are in a maze of twisted little posts, all alike.
Because some people have older machines that take several days to compile large subsystems such as Gnome and KDE?
Because some people run dozens or hundreds of machine with identical configurations, so compiling the same package on every single machine is pointless?
Because companies prefer working against a known build of a piece of software for support reasons?
Source distributions are far from a panacea.
are better than packages because you get more control over what is installed onto your file system. If you really want to the slight advantage that package formats provide over ports, use SRPMS. You will have some of the customizability of using ports and all the features that RPMs provide.
You have got to be kidding. Do you think everyone who uses Linux has the time to do that for every package they install? When I installed Debian, I wanted X. I typed 'apt-get install x-window-system'. The following download and install of all the required packages took about a half hour (over dsl). Your way would have taken me about 2 hours to get all the right packages, and another couple days to compile them from source (after of course checking for security vulnerabilities and fixing them myself).
Not everyone who uses a computer is a programmer. Not everyone who is a programmer is paranoid enough to think that a binary package from RedHat contains a secret connect() to send out their passwd files. Not everyone who IS a programmer, and who IS that paranoid, has the time to audit the code of every package they install.
For these people, binary packages are quite convenient and useful.
While I understand your logic, there's a HUGE hole in your reasoning (besides the ones already commented upon.)
:o) and binary packages save me a TON of time and effort - the result of which makes my boxes more secure than compiling from source.
Binary packages are a great boon to any sysadmin that manages more than one box..
I admin a few dozen Linux boxes (all slackware
For example, not too long ago I updated Squid (due to a security hole), which was running on 20 or so of my servers.. can you imagine HOW MUCH TIME it would take to compile from source for EACH one (considering they are Pentium 1's)? And that I'd have to have a working dev environment on each one!
Instead, I compile the software on an offline server, and check to make sure it works (another imporant step - Squid changed it's config file between versions, so I had to modify the config file).. then turn it into a package, which gets scp'ed to each of the servers, and installed.
Package management allows me to keep track of each file, so upgrades are painless.. and if one of the servers goes down, I can have a new one up and running in 20 minutes, using the same packages I used to upgrade the other systems.
By using binary packages in this way, my systems are MORE secure, because I'm able to minimize the window of vulnerability, and because I don't have to maintain extra software on the boxes (from a security standpoint, anything you don't need should be removed - on a production server, this would include a compiler.)
Remember - not every binary package is untrusted. The ones you make yourself are as trustworthy as the source you used to create them.
Is that the rpmlib API is almost completely undocumented. As GoRK pointed out, management tools such as apt, rpmfind, up2date, etc. are far more important than the underlying package format.
But it's very difficult to create those management tools for RPM when the API is a "black art" known only to a few. Questions on the RPM mailing list/newsgroup will generally be met with the advice to "use the source, Luke"--all several hundred thousand lines of it!
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety."
So just use gentoo then.
Obviously, for large software packages, you probably don't have time to read every last line of code.
That is the understatement of the year. I would dare say that in order to read and understand a program that is on the order of five million lines, it could take you a year or two. For a non-expert programmer (or even an expert with no operating systems experience) it could take forever to just begin to understand something like the Linux kernel source.
But what I generally do is untar the source and then grep through it for suspicious things.
That's great if you know what you are an expert programmer (and if you think that a simple grep will help you that much.) But what of the small business that doesn't employ you? Do they need to perform that same review? Of course, they could skip it and just compile the thing, but that is the same as just using the binary packages!
You may as well just hang your Linux box out on the net with 500 open ports and no firewalls.
Baloney.
Because a well-hacked program will allow the hacker to get at your data, firewall or not.
A well-hacked program will be completely invisible to you, as well. Your grep methodology is too simplistic to catch any sort of sophisticated trojan. Even if you were to laboriously go through the code, line by line, you still wouldn't catch anything but the most obvious of hacks/problems.
The only way you can be completely sure is to read the source.
No, the only way is to not run it. Software is not a mathematical formula that you can "prove". Large programs are horribly complex, as you most likely already know. Binary packages serve a very useful purpose for many people. If you choose not to use them and to perform some limited form of code review, then that is great for you, but don't try to demean anyone who doesn't do the same.
Have you ever heard of signed binary packages ?
Mandrake RPMs are security-signed, and if they don't have a valid Mandrake GPG signature, you get prompted as to whether you really want to install them. I'm sure most other distributions which use binary packages do the same.
C'mon, people. This is obviously a joke. It might not be deserving of a 5 (I think it is, but that's not important), but it certainly isn't flamebait.
Each of these does what InstallShield et al. do: install a program, keep track of how & where it was installed, and give the user the the ability to uninstall it.
I've had problems with un-installing things on my Windows XP box at work: too many software makers try to get around it, and there's no way to figure out where things were put. At least with Unix, I can do a find(1) before installing, then after, do a diff(1) and figure things out from there if I don't trust the developer.
What's not simple by installing (say) Apache with an `apt-get apache`? Under FreeBSD it's `cd /usr/ports/www/apache2; make install clean`. Out of all the systems I (personally) think that FreeBSD's and Debian's are the best (prefer FreeBSD in general as an OS). I haven't used Gentoo though.
I'd put Windows just a little ahead of Linux (because of the multiple formats of Linux packages), but OS X (and all Macs) are way out in front in this arena, IMHO.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
You cannot get your binary into the main site without going through a social screening process.
.rpm based installations.
This allows more standardization than amongst the various
Standardization means fewer problems for the rest of the users.
But it means more work for the people developing the packages.
All of this goes away if you download and install binary packages. You may as well just hang your Linux box out on the net with 500 open ports and no firewalls....The only way you can be completely sure is to read the source.
You haven't read the entire source of the GNU/Linux system you're running, so you have no business telling us that we ought to do it!
Why am I so confident you haven't read the source? Because it's not possible. Even a relatively basic Linux install will correspond to over 2GB of source code. That would be about 800,000 pages of a typical book (2500 to 2800 characters per page). Source code is typically much less dense in terms of characters per page, so it would be millions of pages. It would take you several years to read it, by which time the bits you'd read first would be long obsolete.
Actually, the point goes much further than this. If you are on a RPM system and you lose control of RPM, either through library problems or dependencies, or what have you, you suddenly are robbed entirely of your ability to control the packages on your system, usually including the ability to fix the system itself.
.deb seems to be a far superior package format to recover from catastrophic failures in system utilities such as the package manager. Of course, as you may have ascertained from my comment on cpio, I have experiences precluding bias. ;)
This is not something anything but highly technical users, or even faint of heart, will encounter. However, it is something that has undermined my ability to recover from catastrophic failures on machines with RPM that do not have CD or network access. I have even been reduced to binary manipulation of RPM files to extract the cpio compatible archive (not a task I would undertake lightly).
In contrast, with Debian packages, I have been able to rebuild a machine from scratch with ar, tar, and gzip, which are extraordinarily unlikely to break. Even in the event that they are unavailable, one can copy them to lightweight media, statically compiled, and then they have no real dependencies. Even if dpkg or apt fails (the latter more likely than the prior, in my experience), it is almost always possible to recover from catastrophic mistakes.
In summary,
Why? A compiler is only a translator. How is the ability to translate an arbitrary C program into assembly going to aid an attacker? The attacker could just as easily precompile the program themselves.
Todo.
* relocatable packages
* support for arch name in metadata, arch indep packages
* multiple version of the same package can be installed simultaneously (is this really a package format issue?)
Sigh. The guy has an entire section on how well "standard" tools can manipulate these file formats, as if the typical user has any desire to do home surgery on their software.
(Well, why shouldn't he? The typical linux user does want this level of control...)
But there, at the end, in the neglected "ToDo" section, are the real issues. Features that put the user in control of their software instead of the other way around. Is anyone ever going to write a package management system that addresses the needs of the user, instead of the sysadmin?
-pmb
apt is not tied to the deb package format. There is apt for rpm, but the lack of apt-gettable rpm sites is a problem. On the other hand, Mandrake has urpmi which is similar in functionality.
Escher was the first MC and Giger invented the HR department.
Now this whole episode took me a matter of hours. Whereas Windows would take me a whole 30 minutes to choose what I did/didn't want to install and then download my fixes.
.figure out the best way to install your packages.
Why didn't you just drop them all in a directory and execute rpm -Uvh * instead of manually installing one at a time? That let the package manager do what it does best . .
As far as the 30 minutes on windows deal, on my Debian box I type apt-get update; apt-get upgrade and walk away. On my Gentoo box I type emerge sync; emerge -u world and walk away. Mandrake has urpmi that does the same thing, Connectiva has apt for rpm. If you would have looked, Redhat8 has an automatic updater that will do it for you. Shit, even Slackware has an automatic updater in the works.
Get your information straight before you bitch. Just because you can't figure it out doesn't mean it doesn't have that capability.
I've used many systems, and many package systems.. from old machines where there really was no concept of a package, to debian, with it's superb package management, and everything in between.
The only conclusion I've come to is this: The package format itself isnt' so important.. what matters is the whole system approach to packaging and distribution.
Take Debian. Everyone agrees, I think, that the debian package format, and apt, together make for a great system.. but that's because of the method of package distribution and tracking, not the packaging system itself.. that and the fact that it's fairly universal in the debian world. Several apt repositories make up basically all software available for debian... and it's a lot. SO the overall experience is "Great package management". It's not just about the format, but the people.. people know what's in the standard packages, and can refer back and forth to them, checking for compatability and whatnot. The overall appraoch to package management is what rocks.. not the binary format.
Look at OSX.. they have fink. Fink, if you don't know, is basically apt-get for OSX. Works fine, no problem... except, it puts stuff in it's own folder (/sw) and it doesn't necessairly know about apple stuff already installed.. it only tracks stuff that is in the fink repositories.
In other words.. it's useful, but it doesn't have the feel of a really great package system.. because the system itself isn't based on it.
People say "ports rocks" in bsd land... but why? Becasue it's superior? No.. just because it's a big collection of useful stuff that handles dependencies well. The actual package management system is extremely basic. But the system is more or less based on it, so it works very, very well.
Redhat.. is kind of a mess. Is it because rpm sucks? Heck no.. it's just because, well, the overall approach wasn't right.
OSX.. (yeah okay I'm a mac fiend now.. I admit it.). What package management? Apps tend to be one single file, which is a package containing all the bits and pieces. No real package management system to see what's installed or not.. and who needs it.. you can just go to the Apps folder and toss stuff in the trash to get rid of it. The system was designed to work that way. so it works really well. You don't say "Gee I wish the system tracked apps" because it's so very simple to get rid of them, and to ferret out any pieces they may have left behind, which is rare..
So overall... the complexity of the package management isn't as important as everyone sticking together on how things are going to be installed and removed. If everything works the same way, it doesn't really matter how sophisticated it is.