Slashdot Mirror
Unified BSD packaging system?
Chris Coleman is putting his money where his mouth is, after his recent suggestion that the BSDs need a unified package collection. The creation of www.openpackages.org was the next logical step, and Chris discusses this in his latest Daemon News editorial. With representatives from the three free BSD projects, as well as Apple (MacOS X) on board, this certainly has the potential to bring about closer ties between the BSD distributions at a level that will affect a lot of users.
Slashdot posters need to check their links. The working link is here
Disregard the above. It's a troll.
"If I have seen further than other men, it is by stepping on their glasses." - Michael Swaine
How will all the BSDs choose an appropriate lisence for the packaging programs though? Seems like since each BSD has a different license they will all want the package system to be in their own native license, or have I been using Debian too long?
.deb with a simpler style?
Still this is an interesting concept, especially since not everyone can write portable code, and porting can then be done by one person and distributed as packages (I ran into this problem trying to get Mozilla to run on Irix, just reading the process turned me off)
I also wonder what packaging systems it wil be based off of; will it be like RPM with lots of functionality but confusing or absent categorization (my RPM databases always turned into one package per category because I'd install mandrake or SuSE packages on top of RedHat), or will it be like
# debian/rules
It looks like there will be a single ports tree from which binaries for any of the BSDs can be built easily. (For those of you who don't know, ports are an automated way of downloading, compiling, and installing software).
I imagine that this will be a huge benefit to users of things like OpenBSD, which has 400-500 ports (I think), in comparison to FreeBSD's several thousand. Each of the free BSD's already have their own binary package system (which is basically the same), but their port systems are in very different states. This will be a huge boost to BSD, and make it even easier. I hope it succeeds.
-- Floyd
-- Floyd
If this could become a reality (meaning that a package is a package is a package, on all BSDs) it would also make life much easier for those admins who need to work heterogenous environments.
The concept of bringing OpenBSD's security scrutiny to the BSD community at large appeals to me. In fact, this could concievably be a rather powerful tool in the battle for Open Source:
Basically the idea is to create some sort of standards body / code review group, operating independently of any one (free) OS but ideally with input (at least a person or two) from each OS community. This group would check code against portability criteria, stylistic standards, security checks, etc. Heck, you could even have submitted code ranked in these areas.
Once examined, the (formally, legally) Open Sourced code could be put into this central repository (hint: here's the connection to the UniPort thing the post is talking about) and packaged in such a way for standards bodies (maybe an interoperability group, for example) to examine them, ports groups to package them, yada, yada.
This is a rather rough-hewn idea; got any suggestions for it?
Oh yah, to mail me.... sed -e 's/sp.*am\.//g'
cat email | sed -e 's/sp.*am\.//g'
.tgz's are used to package the various OpenBSD programs in binary form (which are only a few big generalized packages) but tarballs can't do everything in binary packaging.
/usr/share/doc/<package-name>/README but this isn't much functionality here.
.tar.gz is great especially because of the excellent functionality of GNU tar, and I agree that it's the most cross-platform solution (what unix doesn't have at least some kind of tar??) but this was not designed for such complex packaging as .rpm, .deb, and the others.
The advantages to package management I've seen are:
- pre/post-install/removal scripts (tex stuff needs to update the hash, sshd needs to get started after install, etc.)
- file verification (rpm can do this with rpm -V, does checksumming, etc on all files)
- upgrading; if you install a new version of a program it should be smart enough to know whether or not to overwrite your config files which may be important, i.e. for linux upgrading LILO shouldn't overwrite lilo.conf
- dependencies/conflicts, which sometimes get annoying
- provides (in Debian, for example, packages can recommend www-browser, which is provided by lynx, netscape, chimera, etc.)
- previewing; before installing a package you can get version info, descriptions, etc. and yes you could make a simple script to make tar display the file
I do agree that
# debian/rules
A few posters here don't get it. The ports-collection (package for NetBSD) has nothing (or at least almost nothing) to do with .deb .tar or .rar. It is a system to install programs on your system. So before you shout "Yet another incompatible packging-system" try to understand, that it's the opposite case here. It tries to bring three (four with Darwin) Systems together. And I hope that this project will suceed.
I understand the general idea is with binary packages, which is fine and dandy but my question is WHY? Binary packages kind of suck, IMHO -- so many things have build-time configuration options that I've never seen documented worth a damn in other binary package formats (like RPM, for example). Then there's the library go-round, especially on Linux, that makes Windows
Binary packages? No thanks, ports is just fine for me.
Maybe, with enough attempts, someone will eventually come up with a simple packaging system that is fully compatible. Hey, at least they keep trying, though it would be nice if they tried to make it broader than just BSD ... like maybe all of UNIX.
now we need to go OSS in diesel cars
But is it the very same ports tree for all 3 BSDs? And what would it take to integrate Linux, and even commercial Unices like Solaris? I would really like to see such a collection that would (or could when the platform specific elements are included) put things on all different Unix class systems.
now we need to go OSS in diesel cars
How come the only OS's that seem to have ports are all BSD-based? What's keeping somebody from creating a ports system for Linux? It could do wonders for cleaning up the mess that RPMs, DEBs, et al have left behind in terms of piles of incompatible package management systems.
I've only played around with FreeBSD a little bit, but the ports system was one of the coolest things I've ever seen, and a Linux version would be sweet.
--
Why limit it to just BSD? Even within BSD some differences exist. A standardized mechanism to detect the platform (and architecture where appropriate) which is modular should allow even Linux and commercial UNIX systems to have these ports (where someone is willing to do the work to add the platform specific component, which in most cases should be a few simple options).
now we need to go OSS in diesel cars
A tgz package can include an install script, and even an uninstall script.
A tgz package can include an md5 checksum file which have the sums for each individual file in the package except for the checksum file itself.
Upgrading and changing config files is something that should be specific to the program being installed. A packaging system shouldn't try to handle it, just accomodate it by having a standard place to identify the version so it knows it is upgrading. Sometimes config files do need to have newly required items added, or even whole reformatting (and uninstall has to format backwards).
RPM's way of handling dependencies was a failure because RPM itself could not accomodate every instance of needing to install something. I needed to install stuff before any RPM was available. RPMs are complicated to build compared to a TGZ, so software often doesn't originate in RPM.
Generic dependencies would be cool, but the whole system needs to be able to handle establishing dependent relations even if the program is installed from source (i.e. go look and see if the friggin library is actually there, and let me install the required library afterwards if I want and still make the dependency come out right).
A chroot option on package install mught help.
I do believe the KISS principle should apply to ports and packaging, while still getting great functionality out of it.
now we need to go OSS in diesel cars
Whatever packing is used, it needs to be kept simple. RPM isn't. I don't know the inner details of .DEB yet (since the Debian system can't be installed without either a floppy drive in your machine or downloading/buying a fullblow ISO/CD).
My experience with RPM tells me it is a failure. The reason is that RPM was not simple enough for it to be the ORIGINATING format. Someone has to take a source package and convert it to RPM (or apparently to DEB). THIS is the failure because that step introduces a distribution latency, as well as a filtering effect (not all programs get converted).
The proper format for packaging would be one which is JUST AS EASY to build as building a TAR file and compressing it. Right now, the TGZ format meets that requirement. So maybe it should be the starting point to add the additional needed features.
now we need to go OSS in diesel cars
- It should list WHAT CHANGES will be applied to the configuration files AND allow me to install w/o them being changed (leave behind the document that explains what needs to be changed and let me do it).
- How about configuration files that are version specific in their names with symlinks pointing to the currently active version (likewise with the executeable).
- I don't want my C code to be treated like Java.
now we need to go OSS in diesel cars
A PATH of package/port site locations, and the ability to redirect to another site when a package isn't at a site, could be a useful means to find the needed dependency. Then an interactive install can bring up a menu/choice of where to install from (if desired). A non-interactive install can have a site preference list.
/opt/*/bin.
I do agree that having each package in its own tree would be useful. Piling all the executeables into a "bin" directory has always made things difficult (though it certainly makes commands go faster). Some ideal solution to that would be great. Possibilities include piling in symlinks, or PATH entries like
And I also do agree WRT to those annoying info files. That system was always a pain, and the usual culprit was dependencies or not being able to find stuff. HTML should be written to be at least functional in lynx.
now we need to go OSS in diesel cars
What library go around? Windows .DLLs are part of the problem of why stuff crashes in Win95/98/NT/2K systems where the wrong DLL is present. In Linux you can reference libraries by name, name and version, or however you like. Then you can have multiple versions there ready to use at run time.
now we need to go OSS in diesel cars
A unified BSD ports/packaging system is definitely a good and logical step forward. Despite the differences in focus of each BSD flavor, I believe that a unified ports tree will stop a lot of duplication of work. If this goes down, we BSD people should hope to see more ported apps with less failed builds.
One thing I must mention, though -- and since all I know about is Free BSD, this might not be true for the other BSD distros. There need to be both source and binary packages available, and there needs to be only a MINIMUM amount of disparity between the two. Currenly, there is too big of a difference in what you can get with the ports tree (source) and the packages collection (binaries). The way applications are named and organized is different in either place, and the selection differs, too. Any unified ports tree needs to make it simple to choose whether you want to build from sources or just install binaries -- and offer both choices for each app unless licensing issues would prevent it.
Being a programmer, I prefer to build most of my third-party apps from source. However, I might not want to wait for some of the bigger ports (kde2, for instance) to build. Installing binaries should be equally accessible.
Washington, DC: It's like Hollywood for ugly people.
Just thought I'd point this out. ;)
--
--
We have fought the AC's, and they have won.
Why binary... slow machines!
For many years my only box was a 75Mhz pentium. Compiling anything took forever on this box so... binary packages were good.
Also... with something as intelligent as the Debian packaging system, I don't need to worry about dependencies and compiling every little thing that package foo needs. One good example is GNOME or KDE. I actually have things that I need to get done other than watch my machine compile a desktop environment all day.
I have nothing against compiling from source, and I do really like the FreeBSD ports. I also like typing apt-get source foo then running dpkg-buildpackge and having a .deb with all the dependencies in tact.
> Whatever packing is used, it needs to be kept simple. RPM isn't. I don't know the inner details of .DEB yet
Makes RPM look simple, that's what. It requires a whole *directory* full of miscellaneous spec type files, and about 4 different programs to build. Nightmarish, really.
I've finally had it: until slashdot gets article moderation, I am not coming back.
This is no big deal, as this is very likely the case already for the differing versions for the differing BSD variations.
Note that there are Ports for all sorts of GPLed software and such.
The point here is that what licensing is crucial is that of the package tool, and not anything more than that.
By the way, it is eminently likely, and should be blatantly obvious to anyone that does a modicum of research that the system proposed is to be based on the BSD Ports system, which is neither particularly like RPM nor is it greatly like dpkg. (Although the differences betwixt Ports and dpkg are a bit less dramatic, as the Debian project has a similar interest in having "somewhat centralized public access to the source code tree."
Ports involves a "central" Makefile that knows how to look up source code for individual packages. It is essentially based on the philosophy of pulling sources for software, wrapping around it a Makefile and any patches needed to resolve differences between pristine sources and the code that is needed to make it compile and run on the system in question.
If you're not part of the solution, you're part of the precipitate.
Hooking up Ports to Slackware would have the merits that:
Note that this doesn't contribute to "cleaning up" any of the "mess of incompatible package management systems;" that is something that is not going to be changed quickly, if ever.
By the way, for "flip side," there was a proposal at one point to build a FreeBSD kernel with a Debian user space... Definitely different strokes!
If you're not part of the solution, you're part of the precipitate.
OpenBSD may have the same apps, but
they've been thouroughly audited for security.
This is generally only true for those ports in the ports/security tree. Most of the others haven't been havily audited. Certainly no more so than for FreeSBD's system. There are exceptions on both sides.
There are some issues to be worked out as far as whether dependencies need to built for the host or target architecture, but I think it would be a very useful addition since all of the BSDs support more than one architecture (well Darwin can build for x86 even if it can't actually be booted on x86 machines yet...) Speaking of Darwin support for fat binaries would be interesting, too.
I started my journey into the world of Unix OS's with RH 6.0, then upgrading to 6.1. The first couple of rpm's I dealt with really impressed me with how it handled stuff. Then, as many have mentioned, I started running into the enevitable dependency problems. Especially true with Gnome based apps. As time wore on I also noticed that it was becoming increasingly difficult to locate the latest version of an app as an rpm. The GUI rpm tools that were supposed to go hunt down the latest stuff off the web consistantly pulled back far older versions, and often pointed to dead web and FTP resources. I always loved running into the RedHat FTP server in a 24/7 overloaded state. So here I was manually compiling and install stuff outside the rpm database. Completely defeating the purpose of having package management.
One hard drive crash later I decided I'd try out a different distro. After a couple of days mulling it over I decided instead to try out FreeBSD. After figuring out how to pull down the latest ports package I tried running a couple of them. I can't begin to express how impressed I was with this. It appears to consistantly have the very latest stuff, it deals with the compilation and installation for me, and enters the app into the package management system. That, and as I watched it poll a list of ftp sites looking for an active one, then also went about pulling in required libraries my jaw just dropped.
As impressed as I am with how FreeBSD handles apps, there is one aspect I fealt wasn't quite as strong as rpm. rpm's provide a fairly elegant method of udgrading applications that appears to deal with the issues of cleaning out the old app and getting the new one in there. FBSD on the other hand has you go with a process of deinstalling then reinstalling which seems to be prone to all kinds of nasties.
For example, when trying to deinstall a library that other apps depend on the system will stop ya in your tracks. Just trying to install over an older app puts both versions into the database. I'm still fighting a wrong turn I made when upgrading XFree86. I'm certain that a more experienced user would have danced right around the probs I had, but it seems evident that there is room for improvement here.
As it is, I'm far happier with the dealing with apps on FreeBSD than I ever was with RedHat. I genuinely hope that this new project can help better streamline this process. Furthermore, it seems that the various Linux distros could greatly benefit from something like this as well.
The line must be drawn here. This far. No further.
But is it the very same ports tree for all 3 BSDs?
.deb packages. Too large of an installed base for both to just change gears now, and there's also a point of pride that kicks in as well. If anything, I expect the various Linux distros will most likely just hunker down further in however they handle package management.
Nope. That's the reason for starting up the Open Packages project in the first place.
And what would it take to integrate Linux
Hopefully I don't screw up this description here. On FreeBSD there are two different methods for adding packages. The first is from the "ports" collection. These ports are a collection of fancy makefile scripts that have a series of expected locations of the source files, and some instructions on what to do with them, ie. configure, compile, install. These scripts also include a list of dependant packages which are used to either verify they exist, or go and install them.
The second method of installing an app is doing a pkg_add, which is a method for dealing with already compiled packages.
The only stumbling block I would think that a Linux distro would have in implementing this is the final stage of a port install: entering the package into the database. For instance, on RedHat it's expected that all apps will get an entry into the RPM database. FreeBSD sort of expects the same thing, only to it's package db instead. The concept of package management breaks down real quick like when more than one db is in use, since that is what is referred to for tracking dependencies and such.
The really sad part here is that even if this Open Packages project ends up making folks wonder what was so great about sliced bread anyway, it still won't get used on Linux. RedHat isn't showing any signs of giving up on rpm's, and the same can be said for Debian's
The line must be drawn here. This far. No further.
In FreeBSD, this is the difference between a port and a package-- packages are what ports build. From the FreeBSD Ports page:
pkg_add will download and install dependencies, too.One thing I don't like about the FreeBSD ports system is that you can install a package that conflicts with another package, sometimes without even knowing about it. For example, I installed kde 1.2 from ports a while back. Now I recently installed kde 2.0beta4 from ports. I had trouble getting kde2 to compile in the first place, so I didn't want to remove kde 1.2 . But now that both are installed, they both claim ownership to certain files. So now if I were to uninstall kde 1.2, it would delete kde2 files as well.
I see this happening a lot with library packages. You might have a previous version of gtk installed, for example. But then when you compile a new GNOME application, it'll trigger the compile and install of a new version of gtk. But the old one doesn't get uninstalled. It just installs on top of it. And then there's no way to clean out the old one after the fact. I end up deinstalling both, and reinstalling the new one. I think that at the very least, packages should all attempt to uninstall earlier versions before installing.
With FreeBSD ports, you can find out what files are part of an installed package. But you can't go the reverse direction. You can't ask what package a file belongs to. RPM can do this. I suppose that's how RPM prevents packages from clobbering each others files. And somehow RPM can do this without making the install process slow.
Redhat Linux's RPMization of everything does make things rigid. But there are some advantages. It's mostly easy to upgrade to newer versions of the distribution. Since the Redhat installation just consists of bunches of installed rpm's, you just need to upgrade every package. Once I had a Redhat 6.1 upgrade installer break on me. But all I had to do was do an "rpm -Uvh" on each of the rpms to upgrade from RH6.0 to RH6.1 .
But in FreeBSD, an upgrade install (or make world) just dumps stuff on top of whatever you already have. So shared libraries and programs from the earlier install get left around.
Yes, you're missing something. Although BSD packages are distributed in files ending with ".tgz", they do have internal structure -- there are certain files in certain formats that must be present in the tarball, or it is not a package.
--
Ben "You have your mind on computers, it seems."
[Debian's] package management system is simply amazing. No other Linux distribution (well, except the ones based on Debian ;-) come even close. Heck, no other OS has anything like it, period!
This statement is put into doubt by the ignorance you show below about BSDs.
Imagine a system, similar to FreeBSD's ports, that:
1. Works with binaries. You can download and install binaries instead of having to wait for it to compile.
Why, FreeBSD's pkg_add does this. Downloads dependencies, too.
You can also request to get source though, and compile it (just like in FreeBSD).
Not just like FreeBSD. The Debian support for auto-managing dependencies does not extend to source. If you don't have the packages required to build the source, it will simply fail.
2. Figures out dependencies auto-magically and downloads all the required libraries.
FreeBSD already does this.
3. *Handles upgrades*. "apt-get upgrade" will upgrade the packages you have installed.
Take a look here. Essentially, pkg_version -cv' is the FreeBSD equivalent to apt-get upgrade.
The point is simple-- the philosphies behind apt and the ports trees are different, and there are many intricacies behind each. To make a fair comparison one needs to know each in depth.
The ports tree's emphasis is on building from source, and it simply whoops apt/dpkg's ass at this. In my particular environment, I'm finding that this is a feature I could use-- I build a few Debian packages by hand frequently to enable features, which I see could more easily done in FreeBSD.
I've yet to try FreeBSD, but I suspect Debian's automatic configuration of packages is superior, and will get even better as more and more packages are made to use debconf. But once your system is configured appropriately, this is no longer an issue.
Also, I think FreeBSDs development model (e.g. the base OS/ports split) lends itself to more frequent, high quality stable releases for the basic system, and fairly up to date additional packages. You can't get this combination in Debian-- you either run stable, where you get a well-tested system with out of date software, or unstable, where you are on the bleeding edge. (I run both versions in two different machines).
In short, FreeBSD, from what I've researched, is a very good system, which I suspect I will much prefer to Debian (and this comes from a 2.5 year Debian user).
pkg_add -r xyz? It's smaller. :-)
(8-DCS)
Yeah, Not Invented Here syndronme is a drag. I mean, why did they have to invent rpm and dpkg instead of using FreeBSD ports/package system?
Well, the OTHER BSDs did not make this mistake, and adopted the ports/package (though ports is called packages on NetBSD, since they already used the word "ports" for something else). But since they are different, the _content_ of the ports tree is different. You can't use the FreeBSD one on OpenBSD. That's what they are unifying.
(8-DCS)
>Linux as one kernel with lots of userlands
If it is ONE kernel, then things like PICK should work on *ANY* Linux distro. PICK only works on a few.
The linux kernel as shipped on CD's from the 150+ linux vendors is not ONE. It is one+small changes that break compatibility.
If it was said on slashdot, it MUST be true!
Correct. I was referring to the byzantine build system of debian.
I've finally had it: until slashdot gets article moderation, I am not coming back.
FreeBSD's ports/pkg system:
1) works with binaries. you can use pkg_add binary_package to install a binary, and to build a binary package, just 'make package' from the appropriate ports directory.
2) figures out dependencies auto-magically and downloads all the required libraries.
3) doesn't handle upgrades nicely, you have to do a pkg_delete foo ; pkg_add foo combination to upgrade.
lastly, apt stands for 'advanced package tool', something that 'man 8 apt' would've told you.
----------------------------
handles upgrades quite nicely.
pkg_version -l '<' -c > update
produces a handy upgrade script.
----------------------------
I've found that often (but apparently not always) make tries to make all the dependencies for which there is currently no build directories.
Roughtly, if I build X (which takes forever), then delee the source (for the space), and then build xload, I'll end up with the machine making X again. I'm sure there's a simple solution, but I haven't found it yet . . .