Manage Packages Using Stow

dW writes "This article is about Stow, a software installation management utility for Linux that offers a number of advantages over the tried-and-true Red Hat and Debian package management systems. With Stow, you can package applications in standard tar files and keep application binaries logically arranged for easy access."

Comments

Stow isn't Perfect, alas...

[dkf]

We (the Tcl core developers) have had problems in the past with Stow, mainly because it relies on being able to specify the installation process at 'make-install' time instead of normal 'make' time, leading to messed up baked-in paths... :^/

Re:Stow isn't Perfect, alas...

[Anonymous Coward]

try encap. predates stow.

http://encap.cso.uiuc.edu

Re:Stow isn't Perfect, alas...

[virtual_mps]

We (the Tcl core developers) have had problems in the past with Stow, mainly because it relies on being able to specify the installation process at 'make-install' time instead of normal 'make' time, leading to messed up baked-in paths...

I'm not exactly sure what you're unhappy with, but it sounds like a build problem rather than a stow problem, IMHO. With stow you can build the program to expect either the "real" path, or the "stowed" path, depending on your purposes. (Using the "real" path means that multiple versions can be installed in the stow tree and used simultaneously with an explicit path, while using the "stowed" path means that things like config files are in an expected location like "/usr/local/etc".)

Re:Stow isn't Perfect, alas...

[StormCrow]

A quote from http://cr.yp.to/slashpackage/finding.html

``Software should never ever assume it knows where to get files from,'' someone once wrote. (He says I'm taking his quote out of context, so I won't identify him here.)
Here was my sarcastic response:

Yes, that's a very important principle!

Let's take, for example, csh, which uses /etc/csh.cshrc and /dev/log and /bin/sh and many other files. The reason that all those filenames are listed in /etc/csh.conf is so that they can be changed.

Now, some people want to move /etc/csh.conf itself. That's why csh looks for the /etc/csh.conf filename in a hashed /etc/registry.db file.

Of course, on some machines, we need to move /etc/registry.db. That's why the registry filename is listed in a COMPILEDFREGISTRY environment variable.

There's still the possibility of conflict with previous uses of the COMPILEDFREGISTRY variable. That's why the name of that variable is listed in /etc/fregistry_variable_name.txt.

You say you want to move /etc/fregistry_variable_name.txt? You fool! We have billions of programs that /etc/fregistry_variable_name.txt at the top of main(). Everything _else_ has to be configurable, obviously, but /etc/fregistry_variable_name.txt isn't going anywhere.

well...

[virtual_mps]

stow is not at all a "package management utility for linux". It's a perl script and runs on almost anything. I've used it to manage local packages on IRIX, Solaris, and various flavors of Linux. IMO, the great strength of stow is exactly local packages--it's a great way to manage a shared /usr/local or such. I suggest thinking of stow as a powerful complement to your native package management scheme.

Have you tried Gentoo's Emerge

[salimfadhley]

One of the reasons I switched from Redhat 8.1 / 7.3 to Gentoo Linux (Beta) was the amazing Emerge package management tool. It combines simple Tar based package files with cool scripts called eBuilds, that automagically fetch and compile all the components I need.

Of course Gentoo is not for everybody... it takes longer to install than Debian (and that is before you have compiled the entire OS from scratch), but for those who are interested in that sort of thing it can be a refreshing alternative.

Re:Have you tried Gentoo's Emerge

[Imran]

Flamebait disclaimer: I have been running Gentoo on all of my machines for over a year now, so don't take this as an anti-Gentoo comment.

stow and ebuilds aren't really operating in the same space.

rpm,deb,portage = full blown package managers, controlling everything under /usr. These can start with source (or pre-compiled binaries), and handle everything from installation to dependency-handling, etc (with varying degrees of efficiency).

stow = simple symlink manager, providing an easy way to maintain order within /usr/local, for those apps I compile and install manually (and, for whatever reason, don't want to repackage as an ebuild/rpm/whatever)

There are times when one does create one's own ebuilds (v simple) or rpms (slightly more involved). For all other occasions, stow is a helpful tool :)

What about dependencies?

[jointm1k]

The article does not mention anything about dependencies. In my opinion dependencies are almost as important as keeping track of which file belongs to what application. Maybe they should do some more homework and take a look at Gentoo's Portagepackaging system. This system not only compiles a tar/tar.gz/tar.bz2 package, but also retrieves the needed packages (including the dependencies) from their homepages.

Wow?

[j1mmy]

This has about as much flexibility as distributing binaries in a tarball. You can't include installation/uninstallation scripts (what if my application needs to install a cron job?). Everything is forced into /usr/local/stow/PACKAGEDIR and a mess of symbolic links are used to bump everything into the corresponding bin, lib, include, whatever directories. While it may be easier for the software to manage, it creates countless unnecessary files on your drive.

I don't see the benefit.

Why can't it be more like Windows?

[esanbock]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes. Why can't someone make something like this for Linux? It would greatly improve the user experience in Linux. Instead of having to edit 8 configuration files, the user just starts setup.sh or something and the setup asks questions. This is why I like apt-get - one line setup. But every time I download something that's not part of Debian it turn into a horrible experience I wish I would have never had.

Re:Why can't it be more like Windows?

[IamTheRealMike]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes. Why can't someone make something like this for Linux?

A few reasons. Firstly, these programs are tremendously complex under the hood. Almost all generic ones (even light ones like NSIS) include their own scripting language. InstallShield 6 and up has used DCOM to provide remote procedure calls between the install script and the engine (ikernel.exe if you've ever wondered what that is). They do a lot of messing around under the hood in order to make things just work.

Even then, they are too primitive for Linux. For instance, they have only basic concepts of dependancies. The lack of proper dependancy management almost brought Windows to its knees in the mid-nineties. Simply packaging every dependancy inside one self-extracting archive is simply not possible on Linux in any scalable fashion, so we have to build dependancy resolvers like apt. Windows installers tend to be GUI only. And so on.

Now, systems like apt are pretty cool. When they work, they work really well. The problem is, that they tend to be built by distro projects, and then they are relatively tied to that distro. Apt as used on Debian for instance, is not the same as apt4rpm. URPMI is Mandrake, and emerge is basically tied to Gentoo, though I'm sure it could be generalised.

So, the real solution is not to build Windows style setup.exe files. The real solution is to make something like apt, but that can be easily used by everybody, so you rarely if ever come across software that doesn't use it.

There are two approaches to solving that problem. We're trying both at once. The first is to invent a new system, independant of the existing ones. See my sig. The second is to try and standardise key interfaces in a standards body, so that apt/urpmi/emerge and others can interoperate, and so you can plug distro-neutral packages into that framework. See here. Note - most of the activity so far related to that group has been off-list, hopefully there will be action starting in a few days.

Re:Why can't it be more like Windows?

[IamTheRealMike]

I've lost track of the number of times I've had to do the "upgrade tango" and install a dozen different packages just to satisfy the dependencies for a program I needed

That's why we have/need dep resolvers like apt. I rarely, if ever, hear Debian users complaining that dependancies are too complex. They don't need to care.

stow is broken

[Ender Ryan]

Don't attempt to use stow for things such as Gnome or KDE. If you attempt this, things will get horribly broken for a number of reasons.

1. Stow requires you configure all the packages into their own directory, which will cause problems with Gnome and KDE. Some packages are easy to configure into one directory, eg. /usr/local, and then install into another, eg. /usr/local/stow/packagename. Others, not so much...

2. Stow has a serious bug in the way it handles directories. If only one package touches a certain directory, it simply creates a symlink to the directory. And then if another package puts something there, it then removes the symlink and does the normal thing. This is a good idea, however, Stow borks this up sometimes, which is bad.

If you're interested, there's a program similar to stow, called srcpkg, at tempestgames.com/ryan. Yes, I wrote it, sorry for the blatant plug. I thought it relevent because I wrote it after experiencing said problems with Stow. FYI, I use srcpkg to manage all the non-system software on my machine, including Gnome, KDE, mplayer, ogg, SDL, and a very large number of other libs and programs.

There's also a number of similar programs on freshmeat. They're all tailored to slightly different needs, but they're all generally better than Stow.

Stow and problems with "make install"

[Anonymous Coward]

As you may or may not know, Stow relies on "make install". And as you may or may no know, "make install" has many weaknesses, maintainability and readability being two of its most glaring problems.

The reason is readily apparent. There is no clean, high-level way to specify installation details to make. Almost always "make install" invokes a messy ad hoc jumble of Bourne shell commands. Make has its own variables. Bourne shell has its variables. You end up double escaping all kinds of items and end up with $$Variables and a plethora of back ticks. The consequence is that install details in a make file tend to look like Perl's uglier cousin. Throw in line extension by escaping line ends with a reverse solidus, and you have the makings of a maintenance nightmare. Try previewing "make install" with "make -n". Not too helpful is it?

How to fix it? I don't know. Perhaps if all Unix vendors could agree on an "installation specification language" -- ISL. Then each vendor's "make" program could incorporate an interperetter for ISL. Other programs like Linux RPM could benefit for this too and incorporate an ISL interpreter because RPM installation specifications are only slightly better than plain Bourne shell (although definitely a step in the right direction).

Encap is better

[tskirvin]

encap is a better and more established system that works on the same general idea - put everything in /usr/local/encap/PACKAGE-VERSION, and symlink into place. It's mostly just used at UIUC, but good Gods it works well. I use it for absolutely everything, and essentially refuse to install anything on our systems that won't support it. And I have yet to encounter a workplace where it doesn't win over absolutely everyone with its simplicity within six months.

Also, cpanencap is the perfect tool for perfecting Perl's module system. All it needed was versioning.

Re:How does Stow handle dependencies?

[Anonymous Coward]

Stow does not handle dependencies. All it does is use symbolic links so that your packages may install in one directory (completely) and then have symbolic links to a shared directory tree. This was once a standard technique that was manually performed by several system administrators. More recently packaging systems have gained widespread acceptance by people so tools like stow have not been as amazingly handy.

Stow still has importance, though. For example, some people would prefer to build their own application distribution area. This is of particular utility when you have a network of machines and want the same applications available everywhere. Pick a machine and have it NFS share the applications. In these situations Stow still is important. Maybe the stock packaged Perl is not good enough, maybe you want the multithreaded options and a few extra modules from CPAN. Then creating a new Perl directory and stowing it somewhere else is handy.

Stow is not perfect. I have found that it is a bit buggy with its delete operation. I usually erase the directory with the given software and then look for symbolic links that are broken:

ls -lL > /dev/null 2> /tmp/T
rm `sed s/:.*$//g`

remember this when deciding to try out stow

[aagren]

I've used stow on different unix platforms during the last couple of years, and I think it is a great tool to maintain software packages which aren't supported by the platforms own packaging system (deb, rpm, pkg, etc..)

But remember one thing. If you are starting with a new stow system in f.x. /usr/local, then be sure to make the directory structure:

/usr/local/bin
/usr/local/lib
/usr/local/include
etc

if it doesn't exit before stowing anything. Otherwise the following will happen. let's asume that you have the software package in: /usr/local/packages/app-1.4

with it's own strucure like: /usr/local/packages/app-1.4/bin /usr/local/packages/app-1.4/lib
etc.

stow'ing this packages without the /usr/local-structure will result in:

ls -l /usr/local

bin -> packages/app-1.4/bin
lib -> packages/app-1.4/lib
etc.

Then the nect package (let's call it app2-1.5) you will be stow'ing to /usr/local wille see that f.x. /usr/local/bin allready exits at then link the files from it's own bin-drectory to /usr/local/bin, which result in files from app2-1.5 will be linked to the /usr/local/packages/app-1.4 structure, which will mess up things.

because that would be bad

[g4dget]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes.

That works fine for a few applications. Linux has thousands of applications, and people tend to install hundreds of them (they are free, after all, so why not). Do you want to go through hundreds of GUI installers, and then hundreds of GUI updaters? I don't.

Why can't someone make something like this for Linux?

There are interactive installers for Linux packages, but they are usually a nuisance compared to a normal package.

But every time I download something that's not part of Debian it turn into a horrible experience I wish I would have never had.

Well, then don't install non-Debian packages. After all, there are plenty of Windows programs that come with horrible installers. As a Debian user, think of non-Debian packages as "programs that come with horrible installers", and then decide whether they are worth the trouble. (Note that you can usually import packages reasonably well via "alien".)

The package system you get with Debian (or RedHat, for that matter) is already so much better than anything you get for Windows that it isn't funny. If Linux developers adopted the equivalent of setup.exe more widely, that be a real blow to Linux.

Use checkinstall instead

[wct]

Checkinstall automatically produces native packages (rpm, deb, slackware tgz) from a standard make install. I've found this gives the best of both worlds - easy, consistent package management coupled with flexible/optimized source configuration.

You don't want InstallShield

[Wee]

how long is it going to take for something on linux to do what installshield does on windows?

I was a build engineer for a large telecom company who had a commercial Windows product which used InstallShield. I wrote the installer for that product (twice). I've been using Linux since 1994 in various ways. I know InstallShield and I know Linux, and trust me, you don't want IS for Linux, no matter how much you think you do. People are better of sticking to whatever package management system comes with their distribution than pining away for something like InstallShield.

Having a single point of installation management is far superior (even with dependancy checking issues, like sometimes happens with RPM) than leaving it up to the software maintainers. Windows should have orginally shipped with such a centralized system IMO. Now they force software developers to jump through hoops (which cost money, incidentally) in order to get a sticker on their box saying their software was "Designed for Windows". Barring that "certification" a person can do whatever the hell they want in an installer, and your system can become an unorganized mess in very short order.

Even now on this Win2K machine I've got more than a dozen apps (most of which are commercially released) that aren't listed in the Add/Remove Programs applet. If I install something via RPM or apt, that can't happen. I just have to hope that one of those Win32 apps doesn't conflict with something else down the road, since there's no way to remove them programmatically. That's a critical flaw of Windows' installation setup, IMO. Just one that happens to come to mind, even. I could go on all day...

I admit not knowing about linux, and am doing little to change that.

Your parents must be terribly proud at having imbued in their progeny such an insatiable thirst for knowledge.

-B