Manage Packages Using Stow

dW writes "This article is about Stow, a software installation management utility for Linux that offers a number of advantages over the tried-and-true Red Hat and Debian package management systems. With Stow, you can package applications in standard tar files and keep application binaries logically arranged for easy access."

Comments

Stow isn't Perfect, alas...

[dkf]

We (the Tcl core developers) have had problems in the past with Stow, mainly because it relies on being able to specify the installation process at 'make-install' time instead of normal 'make' time, leading to messed up baked-in paths... :^/

Re:Stow isn't Perfect, alas...

[Anonymous Coward]

try encap. predates stow.

http://encap.cso.uiuc.edu

Re:Stow isn't Perfect, alas...

[virtual_mps]

We (the Tcl core developers) have had problems in the past with Stow, mainly because it relies on being able to specify the installation process at 'make-install' time instead of normal 'make' time, leading to messed up baked-in paths...

I'm not exactly sure what you're unhappy with, but it sounds like a build problem rather than a stow problem, IMHO. With stow you can build the program to expect either the "real" path, or the "stowed" path, depending on your purposes. (Using the "real" path means that multiple versions can be installed in the stow tree and used simultaneously with an explicit path, while using the "stowed" path means that things like config files are in an expected location like "/usr/local/etc".)

Re:Stow isn't Perfect, alas...

[Uerige]

We (the Tcl core developers) have had problems in the past with Stow, mainly because it relies on being able to specify the installation process at 'make-install' time instead of normal 'make' time, leading to messed up baked-in paths....

Why is this a problem with stow? You probably shouldn't hardcode paths into the binary, as that may lead to problems like this.

Re:Stow isn't Perfect, alas...

[StormCrow]

A quote from http://cr.yp.to/slashpackage/finding.html

``Software should never ever assume it knows where to get files from,'' someone once wrote. (He says I'm taking his quote out of context, so I won't identify him here.)
Here was my sarcastic response:

Yes, that's a very important principle!

Let's take, for example, csh, which uses /etc/csh.cshrc and /dev/log and /bin/sh and many other files. The reason that all those filenames are listed in /etc/csh.conf is so that they can be changed.

Now, some people want to move /etc/csh.conf itself. That's why csh looks for the /etc/csh.conf filename in a hashed /etc/registry.db file.

Of course, on some machines, we need to move /etc/registry.db. That's why the registry filename is listed in a COMPILEDFREGISTRY environment variable.

There's still the possibility of conflict with previous uses of the COMPILEDFREGISTRY variable. That's why the name of that variable is listed in /etc/fregistry_variable_name.txt.

You say you want to move /etc/fregistry_variable_name.txt? You fool! We have billions of programs that /etc/fregistry_variable_name.txt at the top of main(). Everything _else_ has to be configurable, obviously, but /etc/fregistry_variable_name.txt isn't going anywhere.

well...

[virtual_mps]

stow is not at all a "package management utility for linux". It's a perl script and runs on almost anything. I've used it to manage local packages on IRIX, Solaris, and various flavors of Linux. IMO, the great strength of stow is exactly local packages--it's a great way to manage a shared /usr/local or such. I suggest thinking of stow as a powerful complement to your native package management scheme.

Unix Directory Structures

[Boss, Pointy Haired]

Whilst this is some way towards making Linux more user friendly (and ultimately gaining acceptance on the corporate desktop), what are the chances of anything being done about the crazy directory layout of a *nix system?

If the answer is nothing, then a suitable GUI for Linux that has the objective to gain corporate desktop acceptance really has to isolate the [l]user from this - i.e. with something like "My Documents".

I like using Linux, but even as a seasoned IT pro, the directory structure and "what goes where" of a *nix system still bugs me.

Re:Unix Directory Structures

[42forty-two42]

"My Documents"? Ever heard of /home/$username?

Re:Unix Directory Structures

[Sh0t]

You have it backwards friends. It's not a crazy scheme it's a STANDARD and LOGICAL scheme. Programs installed to certain restrictive locations so you KNOW without trying to guess, where certain binaries go, where config files go and so on. It's a standardization and it works very well YOu just aren't used to it. IF you aren't installl from a system package you can put whatever you want, wherever you want, but to keep things orderly you should follow the scheme windows does the same things actually when you use installshield or msi, some files go to program files/program name, some go to /windows some go to a temp dir for settings, some entries go into the registry etc. I think the standard unix method is much tidyer overall, but it may be a bit confusing at first to those who are migrating.

Re:Unix Directory Structures

[cyb97]

Or as everybody else refers to it... ~

Re:Unix Directory Structures

[Permission Denied]

You must be trolling.

Take any file on a modern Linux system. Any file at all. Explain to me its purpose and I'll tell you where you'll find it in the filesystem. A keyboard map? /usr/share. A subprogram to be executed by a program, not by the user? /usr/libexec.

Take any file on a modern Linux system and tell me its full path. I'll tell you what it does, and I'll probably be able to tell you what package it comes from. I can also tell you if you need it or if you can get rid of it. /usr/X11R6/lib/X11/rgb.txt? Rgb color database to define textual names for colors, from XFree86. /usr/local/share/automake/elisp-comp? Support file for automake to integrate with emacs, installed by user after system installation time, from GNU automake. /usr/local/lib/libjpeg.so? Jpeg image library, installed after system installation time, from the JPEG group.

Take any file on a modern Windows system, and you won't be able to do anything with it. C:\winnt\keyacc32.exe? Does that come from MS or from a third party and what is it used for? C:\winnt\system32\getstart.gif? An image that says "Get started with Beta 2" in the Microsoft Arial font. Is this a remnant from a win2k beta and if so, why is still here in a production post-sp3 win2k pro system? Or does it come from some third party? Try figuring out what files are necessary on a Windows system and what's cruft. You certainly won't be able to get a Windows install to fit in less than ten megs because all these files are spread out and undocumented - however, you know it's possible to get a Windows image in only a few megs because Microsoft does it (miniwindows during Windows installation). Getting a FreeBSD install to fit in only a few megs is not a problem - just did it for a compact-flash-based system and it's not hard.

Anyway, this scheme that stow uses is very useful. Djb also has something like this in mind, but his way of doing it is not very elegant IMHO. I've been using something like stow for all my machines for the past three years or so, but it was just a 50-line shell script. Keep all software installed in /opt. Like /opt/emacs/emacs-21.1 with a symlink /opt/emacs/default which points to "current" version (that way, upgrades can be done with just changing a symlink, and downgrades are also just changing a symlink - you don't have to learn some tool's syntax, and, more importantly, the admin that replaces me won't have to learn some esoteric system because you can figure out my system in 30 seconds). All that my shell script does is make symlinks from the /opt/cvs/default/bin to /usr/local/bin, from /opt/python/default/lib to /usr/local/lib, etc. This way, when you run a poorly-written autoconf script, it will always find the requisite packages (because people always assume the required package is in /usr/local) and my users don't have to deal with $PATH. In addition, I can run this:

find /usr/local -type f -print

And this tells shows me all the stuff that I've written locally for the particular system (anything in /usr/local that's not a symlink must have been written by me).

Never run into any dependency problems, upgrades are "ln -s," uninstalls are "rm -rf."

Re:Unix Directory Structures

[TheOrquithVagrant]

I'd say it's definitely just familiarity.

For anyone confused about the "what goes where" in a Linux system, I warmly recommend taking a look at:

http://www.pathname.com/fhs/

which describes the Filesystem Hierarchy Standard, part of the Linux Standards Base. It should clear things up.

Have you tried Gentoo's Emerge

[salimfadhley]

One of the reasons I switched from Redhat 8.1 / 7.3 to Gentoo Linux (Beta) was the amazing Emerge package management tool. It combines simple Tar based package files with cool scripts called eBuilds, that automagically fetch and compile all the components I need.

Of course Gentoo is not for everybody... it takes longer to install than Debian (and that is before you have compiled the entire OS from scratch), but for those who are interested in that sort of thing it can be a refreshing alternative.

Re:Have you tried Gentoo's Emerge

[Imran]

Flamebait disclaimer: I have been running Gentoo on all of my machines for over a year now, so don't take this as an anti-Gentoo comment.

stow and ebuilds aren't really operating in the same space.

rpm,deb,portage = full blown package managers, controlling everything under /usr. These can start with source (or pre-compiled binaries), and handle everything from installation to dependency-handling, etc (with varying degrees of efficiency).

stow = simple symlink manager, providing an easy way to maintain order within /usr/local, for those apps I compile and install manually (and, for whatever reason, don't want to repackage as an ebuild/rpm/whatever)

There are times when one does create one's own ebuilds (v simple) or rpms (slightly more involved). For all other occasions, stow is a helpful tool :)

Re:Have you tried Gentoo's Emerge

[nautical9]

I agree, Gentoo's portage is the best package management I've come across. Not only doe it make ANY package a one-line command that will automatically "download, untar, [patch], configure, make, make test, make install", but it uses system global optimizations for compiles, takes care of all dependencies, and places binaries, libraries, config files and startup scripts all in standard locations. And their "gentoo-sources" version of the kernel has over 70 high-performance patches it includes to the vanilla kernel.org tree (but of course, you don't have to use them if you don't want to, but why not?)

It even has a great /etc/env system for managing environment variables (both bash and csh flavors), so if it needs to install binaries in a non-standard location, you "PATH" is automatically updated to include it.

I don't use Gentoo as a desktop platform, so I can't comment on its X/KDE/Gnome setups, but I'm sure they're just as complete and easy. And although Gentoo may be rather intimidating for a n00b initially, it does have excellent documentation and a great support community at their site.

Keeping a system up-to-date with the latest and greatest has never been this easy!

Re:Have you tried Gentoo's Emerge

[Ed Avis]

I don't understand why many people seem to assume 'tar good, rpm/dpkg bad'. For example, the article says:

Although some Linux flavors such as Red Hat and Debian come with their own package management utilities (rpm and apt-get, respectively) that are as efficient as Stow, they work only on specific packaging formats (.rpm and .deb, respectively). When it comes to managing applications simply packaged in .tar files, Stow is the best bet.

Why is a tar file any more 'simple' than an RPM or Debian package? If you are just storing a bunch of files, then yes. But what about metadata? That is, the information on dependencies (what libraries the package needs), where to get the source code, who the packager was, which version of the software these files represent, whether this packge conflicts with any other packages that might be installed, and all the other things that a decent package manager keeps track of automatically so you don't have to check them by hand, or get nasty surprises when you've installed a package and only later find a necessary library is the wrong version.

If you use tar files then you'd need to have an extra metadata file inside storing this information: then you need to decide a format for that file and write a tool to parse it. And you've then reinvented rpm or dpkg, only with the spurious 'improvement' that people on other systems can unpack the archive if they have tar installed. As if anyone running a different system would need to unpack someone else's binary packages.

Perhaps you could argue it would have been better for rpm to be based around tar.gz format, with the package details stored in the tarball as a script of some kind. But then it would become much slower to query a large directory of packages. Maybe that is important, maybe not. Also you have digital signatures to worry about, it's not clear how to do those with a tar archive (unless you have one tarball containing another tarball plus its PGP signature). Perhaps things could have been done differently, but the mere fact that the rpm and dpkg developers chose to make their own format rather than use tar is not an excuse for committing much more serious wheel-reinvention in making a kewl Yet Another packaging format.

I'm not meaning to knock Gentoo here, that distribution really does do something new (building everything from source), and they perhaps had good reason to make a new packaging format. (Although it might have been worthwhile to investigate whether building from source could fit in with rpm or Debian source packages.) And Slackware has used tgz packages since the beginning, and doesn't seem that bothered about automatically tracking dependencies.

But for most uses, I just don't see why 'simple packaging with tar' is particularly simple in the long run or much of an advantage. It sounds like those Freshmeat projects which say 'this is yet another MP3 player but it has the advantage that it doesn't use GTK or Qt but implements its own user interface code instead'.

Re:Have you tried Gentoo's Emerge

[Ari Rahikkala]

Because we're everywhere. You can't escape the horde of the Gentoo users. Just give up and in - one day you, too, shall love Gentoo... why not start today?

-revision-

We are Locutus of Gentoo. Binary distribution is futile. Your sources will be assimilated.

-revision-

In A.D. 2003
Distro-War was beginning.
Red Hat: What happen ?
Mandrake: Somebody set up us the shiznit
Debian: We get signal
Red Hat: What !
Debian: Main irssi screen turn on
Red Hat: It's You !!
Gentoo: How are you gentlemen !!
Gentoo: All your sources are belong to Portage
Gentoo: You are on the way to irrelevancy
Red Hat: What you say !!
Gentoo: You have no chance to survive make your time
Gentoo: HA HA HA HA ....
Red Hat: Take off every 'rpm'
Red Hat: You know what you doing
Red Hat: mv 'rpm'
Red Hat: For great justice

Re:Have you tried Gentoo's Emerge

[dattaway]

The perceived problems of customized building of an entire may be a strong advantage. It requires all software to be of highest quality or the bugs will show. A community that encounters problems encourages debugging or switching to higher quality packages and not sweeping problems under the rug.

Re:Have you tried Gentoo's Emerge

[supergiovane]

To be more general, an ebuild is a file containing the description of the operations needed to have an app installed.

1) It checks dependancies. If dependancies are satisfied it goes to step 2, else it launches the installation of needed packages.

2) It retrieves the app (either sources or binary package, in any format). Some times the portage system simply cannot automatically retrieve the app through the network (apps with required registration or license agreement acceptance requirement). In this case it stops asking the user to manually retrieve the package files, then it continues with the installation.

3) It prepares the app for installation (compiling sources or simply extracting in a temp dir the precompiled binaries (obviously it can deal with every package format)).

4) It installs the app, updating config files.

This way of working simply separates the actual app (source or any other package) from the metadata (contained in the ebuild script). So Gentoo can handle a lot of packets written for other distros (acting as a wrapper), simply trashing the original metadata and substituting it with its own. As a matter of fact writing an ebuild is easier than packaging an rpm.

Re:Have you tried Gentoo's Emerge

[pyros]

my wife installed Woody and it was nothing but errors and issues.

i'll say. I installed Woody for my wife and in the end it just core dumped.

What about dependencies?

[jointm1k]

The article does not mention anything about dependencies. In my opinion dependencies are almost as important as keeping track of which file belongs to what application. Maybe they should do some more homework and take a look at Gentoo's Portagepackaging system. This system not only compiles a tar/tar.gz/tar.bz2 package, but also retrieves the needed packages (including the dependencies) from their homepages.

Re:What about dependencies?

[virtual_mps]

The article does not mention anything about dependencies. In my opinion dependencies are almost as important as keeping track of which file belongs to what application.

Stow doesn't do dependencies. IMO, that's fine for local package management, where dependency handling is often more trouble than it's worth. (A local package will typically be run on a know configuration, and can target that configuration.)

Maybe they should do some more homework and take a look at Gentoo's Portagepackaging system.

Do some homework? You are aware that stow has been around since 1996, right? It's amazing how many gentoo fanboys don't know what else is out there.

How does Stow handle dependencies?

[Max Romantschuk]

I've got some experience with Debian's package management system, and while hard to use for a novice and somewhat complex there is one great benefit: conflict and dependency handling.

Based on the article I didn't quite understand if Stow provides similar services. There were some hints on this, but could someone with experience shed some light on the subject?

Re:How does Stow handle dependencies?

[Anonymous Coward]

Stow does not handle dependencies. All it does is use symbolic links so that your packages may install in one directory (completely) and then have symbolic links to a shared directory tree. This was once a standard technique that was manually performed by several system administrators. More recently packaging systems have gained widespread acceptance by people so tools like stow have not been as amazingly handy.

Stow still has importance, though. For example, some people would prefer to build their own application distribution area. This is of particular utility when you have a network of machines and want the same applications available everywhere. Pick a machine and have it NFS share the applications. In these situations Stow still is important. Maybe the stock packaged Perl is not good enough, maybe you want the multithreaded options and a few extra modules from CPAN. Then creating a new Perl directory and stowing it somewhere else is handy.

Stow is not perfect. I have found that it is a bit buggy with its delete operation. I usually erase the directory with the given software and then look for symbolic links that are broken:

ls -lL > /dev/null 2> /tmp/T
rm `sed s/:.*$//g`

Wow?

[j1mmy]

This has about as much flexibility as distributing binaries in a tarball. You can't include installation/uninstallation scripts (what if my application needs to install a cron job?). Everything is forced into /usr/local/stow/PACKAGEDIR and a mess of symbolic links are used to bump everything into the corresponding bin, lib, include, whatever directories. While it may be easier for the software to manage, it creates countless unnecessary files on your drive.

I don't see the benefit.

Why can't it be more like Windows?

[esanbock]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes. Why can't someone make something like this for Linux? It would greatly improve the user experience in Linux. Instead of having to edit 8 configuration files, the user just starts setup.sh or something and the setup asks questions. This is why I like apt-get - one line setup. But every time I download something that's not part of Debian it turn into a horrible experience I wish I would have never had.

Re:Why can't it be more like Windows?

[IamTheRealMike]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes. Why can't someone make something like this for Linux?

A few reasons. Firstly, these programs are tremendously complex under the hood. Almost all generic ones (even light ones like NSIS) include their own scripting language. InstallShield 6 and up has used DCOM to provide remote procedure calls between the install script and the engine (ikernel.exe if you've ever wondered what that is). They do a lot of messing around under the hood in order to make things just work.

Even then, they are too primitive for Linux. For instance, they have only basic concepts of dependancies. The lack of proper dependancy management almost brought Windows to its knees in the mid-nineties. Simply packaging every dependancy inside one self-extracting archive is simply not possible on Linux in any scalable fashion, so we have to build dependancy resolvers like apt. Windows installers tend to be GUI only. And so on.

Now, systems like apt are pretty cool. When they work, they work really well. The problem is, that they tend to be built by distro projects, and then they are relatively tied to that distro. Apt as used on Debian for instance, is not the same as apt4rpm. URPMI is Mandrake, and emerge is basically tied to Gentoo, though I'm sure it could be generalised.

So, the real solution is not to build Windows style setup.exe files. The real solution is to make something like apt, but that can be easily used by everybody, so you rarely if ever come across software that doesn't use it.

There are two approaches to solving that problem. We're trying both at once. The first is to invent a new system, independant of the existing ones. See my sig. The second is to try and standardise key interfaces in a standards body, so that apt/urpmi/emerge and others can interoperate, and so you can plug distro-neutral packages into that framework. See here. Note - most of the activity so far related to that group has been off-list, hopefully there will be action starting in a few days.

Re:Why can't it be more like Windows?

[Darth Yoshi]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes. Why can't someone make something like this for Linux?

Didn't Loki write a graphical installer for Linux? I can't access the Loki site from work to check because it's blocked by websense (ha).

Re:Why can't it be more like Windows?

[Niggle]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes.

In Mandrake, I single-click an RPM and the package manager start installing it. Is that easy enough?

Some notes on Windows and Linux package/program installation:

1) Windows setups handle dependancies by basically not handling them. They almost always include a bunch of system DLLs and OCXs that might not be on a user's system or which might be outdated. This obviously leads to much larger packages which for a large part contain stuff that is already on the system. It would be relatively easy on Linux to make every package include every package it depends on. These don't have to be statically linked, you could include the packages for the shared libraries within the main package and have these install automatically. I think the bloat problem would be worse on Linux than Windows, because my feeling is that open source programs tend to use a much wider variety of shared libraries than their proprietary equivalents (where everybody re-invents the wheel on a daily basis because they can't use somebody else's design).

2) Different languages are handled in many cases on Windows by having several setup programs. The main setup.exe in these cases is just a shell that selects which one to run. This adds to package bloat. Linux fares slightly better on this, because (IMHO) i18n is easier on the programmer here.

3) Windows only needs to consider one architecture. If it had several to worry about, we'd probably see a situation much like we have with languages.

4) Configuration at install time on Windows is mostly just choosing which optional extras to install. Most configuration is done within the program itself. This is more-or-less true on Linux as well (for desktop programs at least).

To get close to the Windows installation experience under Linux, what we need to do would be to make every package include every sub-package it depends on and sub-packages for every architecture, disro and langauge. Then you could just download the single file, click it and get everything installed. That package would be enormous however.

Tools like apt-get and urpmi give a very similar experience without the overhead of downloading a bunch of stuff you already have. So long as you stick with stuff that is packaged for you distro, they are painless.

Re:Why can't it be more like Windows?

[Chacham]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes.

There are differences

• Windows gives more choices.
  
• Windows does not have a central installer.
  
• Most Windows programmers have no clue as to how to install.
  
• The Installer is a third party program.
  

Windows gives more choices.

On a general install, Windows asks where to install it. Linux follows the Un*x scheme, and gives less choices here. Also, Windows programs are larger, and thus there is an issue with how much to install. Linux binaries tend to be small, and so they don't bother asking what type of install you'd like. Finally, Windows programs are usually closed source, so the package lives in it's own little world. Linux packages are generally open source, so when you install a package it is an implicit choice. For example, a front-end, or a data file. With Windows it all comes in one closed package. In Linux they are separate packages, so choosing the package is like choosing an option.

Windows does not have a central installer.

This has changed recently with the Windows Installer, but it is not yet that popular. And things such as the uninstaller rely on the person programming the installer to put the appropriate entry in the registry. If they don't (and many don't) Windows has no record of it. So, each program needs it's own installation program. Linux distributions have a general installer that keeps track of everything. You can always query the rpm database, or the dpkg cache.

Most Windows programmers have no clue as to how to install.

You'll have to trust me on this one. I worked for WISE, and dealt with emails during my 20 or so months there. I proably answered over 20,000 emails (At least 30 emails a day), so I have a general idea. I also dealt with the newsgroups, but those people were vastly more intelligent.

For example. One person has a CD with tens of thousands of images on it and wanted to know how to make a link to *each* image in the start menu. I warned the person how this will use up much space due to the cluster size, and they agreed to only make links to the folders. Then there was that guy who after making temp files (instead of letting the installer handle them with its own feature) would delete *everything* in the temp folder. I warned him a well. Oh, there were people who just assumed the Windows directory was "C:\Windows", and people who hadn't the slightest idea as to what the registry was. And, these people wrote programs to run on your computer!

Thus, luckily, there are install programs for Windows. Linux does not seem to have these issues.

The Installer is a third party program.

Usually InstallShield, WISE, or InstallVise. So, they need frills to sell. In Linux all people care about is a packager, so it just isn't needed.

To sum that all up, Windows is a more complicated install with space issues, that relies on programs to register themselves, with programs written by the cluless, and has third part programs charge for their use to install. Thus, there are installers, with choices, and frills.

Linux has no need for all of that. So, the GUI just was never required.

Why can't someone make something like this for Linux? It would greatly improve the user experience in Linux. Instead of having to edit 8 configuration files, the user just starts setup.sh or something and the setup asks questions.

Those programs that need editing, give you a great many more choices then the Windows programs. Though, many Windows programs have it in their "options" or "preferences", the draw back being that you *require* a GUI to get to those, which makes editing harder, slower, more limited, and not easily distributable to other computers.

This is why I like apt-get - one line setup. But every time I download something that's not part of Debian it turn into a horrible experience I wish I would have never had.

That's what unoffical sources are for, and what /usr/local is for.

Re:Why can't it be more like Windows?

[IamTheRealMike]

I've lost track of the number of times I've had to do the "upgrade tango" and install a dozen different packages just to satisfy the dependencies for a program I needed

That's why we have/need dep resolvers like apt. I rarely, if ever, hear Debian users complaining that dependancies are too complex. They don't need to care.

stow is broken

[Ender Ryan]

Don't attempt to use stow for things such as Gnome or KDE. If you attempt this, things will get horribly broken for a number of reasons.

1. Stow requires you configure all the packages into their own directory, which will cause problems with Gnome and KDE. Some packages are easy to configure into one directory, eg. /usr/local, and then install into another, eg. /usr/local/stow/packagename. Others, not so much...

2. Stow has a serious bug in the way it handles directories. If only one package touches a certain directory, it simply creates a symlink to the directory. And then if another package puts something there, it then removes the symlink and does the normal thing. This is a good idea, however, Stow borks this up sometimes, which is bad.

If you're interested, there's a program similar to stow, called srcpkg, at tempestgames.com/ryan. Yes, I wrote it, sorry for the blatant plug. I thought it relevent because I wrote it after experiencing said problems with Stow. FYI, I use srcpkg to manage all the non-system software on my machine, including Gnome, KDE, mplayer, ogg, SDL, and a very large number of other libs and programs.

There's also a number of similar programs on freshmeat. They're all tailored to slightly different needs, but they're all generally better than Stow.

Stow and problems with "make install"

[Anonymous Coward]

As you may or may not know, Stow relies on "make install". And as you may or may no know, "make install" has many weaknesses, maintainability and readability being two of its most glaring problems.

The reason is readily apparent. There is no clean, high-level way to specify installation details to make. Almost always "make install" invokes a messy ad hoc jumble of Bourne shell commands. Make has its own variables. Bourne shell has its variables. You end up double escaping all kinds of items and end up with $$Variables and a plethora of back ticks. The consequence is that install details in a make file tend to look like Perl's uglier cousin. Throw in line extension by escaping line ends with a reverse solidus, and you have the makings of a maintenance nightmare. Try previewing "make install" with "make -n". Not too helpful is it?

How to fix it? I don't know. Perhaps if all Unix vendors could agree on an "installation specification language" -- ISL. Then each vendor's "make" program could incorporate an interperetter for ISL. Other programs like Linux RPM could benefit for this too and incorporate an ISL interpreter because RPM installation specifications are only slightly better than plain Bourne shell (although definitely a step in the right direction).

Re:Stow and problems with "make install"

[IamTheRealMike]

A better solution would be to replace automake with a totally new build system. We've been hacking around the deficiences of make for years, and the time when compatability with lame commercial unices form of make was an issue is long gone.

Something like SCONs perhaps, although I'm not sure python is the best language for this. Although it's possible, easy even, to write really ugly bash, it's a very good language for filing system manipulations, which is a large part of build management. There was another build system based on bash that was a LOT easier than autotools, but I can't remember what it was called! :(

Encap is better

[tskirvin]

encap is a better and more established system that works on the same general idea - put everything in /usr/local/encap/PACKAGE-VERSION, and symlink into place. It's mostly just used at UIUC, but good Gods it works well. I use it for absolutely everything, and essentially refuse to install anything on our systems that won't support it. And I have yet to encounter a workplace where it doesn't win over absolutely everyone with its simplicity within six months.

Also, cpanencap is the perfect tool for perfecting Perl's module system. All it needed was versioning.

It doesn't

[ggeens]

Stow has no concept of dependencies. There is no way you could use it to build a distribution on top of it.

I use stow on my (Debian) Linux PC at home, to manage the software I build from source. If I want to upgrade a program, I can just delete the directory and install the new version in the same location. If a Debian packages becomes available, I remove the directory and have stow remove the links in /usr/local/*.

Until now, I have been able to get all the libraries from Debian, so I never needed to work with dependencies.

remember this when deciding to try out stow

[aagren]

I've used stow on different unix platforms during the last couple of years, and I think it is a great tool to maintain software packages which aren't supported by the platforms own packaging system (deb, rpm, pkg, etc..)

But remember one thing. If you are starting with a new stow system in f.x. /usr/local, then be sure to make the directory structure:

/usr/local/bin
/usr/local/lib
/usr/local/include
etc

if it doesn't exit before stowing anything. Otherwise the following will happen. let's asume that you have the software package in: /usr/local/packages/app-1.4

with it's own strucure like: /usr/local/packages/app-1.4/bin /usr/local/packages/app-1.4/lib
etc.

stow'ing this packages without the /usr/local-structure will result in:

ls -l /usr/local

bin -> packages/app-1.4/bin
lib -> packages/app-1.4/lib
etc.

Then the nect package (let's call it app2-1.5) you will be stow'ing to /usr/local wille see that f.x. /usr/local/bin allready exits at then link the files from it's own bin-drectory to /usr/local/bin, which result in files from app2-1.5 will be linked to the /usr/local/packages/app-1.4 structure, which will mess up things.

Re:remember this when deciding to try out stow

[virtual_mps]

Then the nect package (let's call it app2-1.5) you will be stow'ing to /usr/local wille see that f.x. /usr/local/bin allready exits at then link the files from it's own bin-drectory to /usr/local/bin, which result in files from app2-1.5 will be linked to the /usr/local/packages/app-1.4 structure, which will mess up things.

Odd. On my system it will notice that bin is a symlink to a bin dir in a different stow package, remove the symlink, create a dir, then link the contents of *both* packages bin dirs into the new /usr/local/bin.

Excuse me?

[arvindn]

Could somebody please enlighten me? I don't get the point at all.

... a number of advantages over the tried-and-true Red Hat and Debian package management systems. With Stow, you can package applications in standard tar files

"A standard tar file" is just a bunch of files. The reason rpm and other packaging formats are used is to do dependency tracking and management. There is no way you can figure out the dependencies from just the tar file. So comparing stow with rpm is like comparing apples and oranges. Stow is not an alternative to rpm. (Of course I agree that if we had a single universal packaging format it would be great. But the answer is not to throw all the features overboard.)

... and keep application binaries logically arranged for easy access.

Wtf? What do you mean, access a binary? When is the last time you did "vim /bin/ls"? The only thing you do with binaries is to execute them, and putting them in /bin/ or /usr/bin/ etc. is perfectly adequate.

gives users the freedom to store or install the software package at any desired location

Excuse me, but "configure --prefix=dir" already does that?

Imagine installing an application that accidentally overwrites a file belonging to another application, and then you have to replace the file.

Has anyone ever encountered this? It seems somewhat contrived to me.

Or imagine, before uninstalling and deleting an application, trying to determine which files belong to that application.

Any half-decent package manager allows you to list all the files belonging to an application.

The UNIX way of putting applications is well thought out, matured and perfectly fine. Needlessly playing around with it is likely to cause more problems than it solves.

Yes, the package management scene on Linux sucks right now. But it is because of dependency management, and has nothing to do with all the files of an application in a nice folder.

Re:Excuse me?

[Urban Garlic]

> Could somebody please enlighten me? I don't get the point at all.

I'll try to give it a go. I have several Debian boxes, and mostly use apt, but every now and then I need to install something for which there is no .deb, and stow is perfect for that. I have an m68k cross-compiler version of gcc (for Palm(tm) development), a locally modified version of the GTK canvas, and a few other obscure, specialized bits of software, all of which are "stow"-ed. I have realized many of the advantages described in the article -- I can uninstall these things cleanly, and move between versions just by unstowing the old one and stowing the new one.

I do think the article (and much of the commentary here) overstates the role of stow. It's not a substitute for a package manager, and the way it works makes it unsuitable for system-level software that, for instance, might need to set up cron jobs, require scripts in /etc/init.d, or be configured from a file in the /etc directory. But it *is* very useful for those occasional, obscure bits of software which primarily consist of libraries and include-headers, or non-system executables.

Stow itself is not new, and interestingly is packaged by debian -- I got it by "apt-get install stow"...

Another way

[tallniel]

I think more people should take a look at tclkit (http://www.equi4.com/tclkit) and the concept of starkits (http://www.equi4.com/starkit). This is a great concept where an application is delivered as one self-contained file (compressed, with an internal virtual file-system). This gets rid of the problem of "installation" all together.

Very cool stuff.

See also Linux Mag France

[dolmen.fr]

For french guys, there is a Stow tutorial in GNU Linux Magazine France of this month (http://www.linuxmag-france.org/). The article is not available online.

Here is the author web site : http://hocwp.free.fr/ln_local/index.html
However I don't recommend his ln_local tool (a simple stow replacement) as it is seriously flawed: this shell script doesn't escape spaces (and other more dangerous shell chars) in filenames when handling them.

Stow is here : http://www.gnu.org/software/stow/stow.html
See also XStow : http://xstow.sourceforge.net/

Dolmen.

Re:Try Using Gentoo Instead

[Skater]

So, I took my Chevy Cavalier to the dealer this morning to get brake work done. He strongly recommended that instead of fixing the Cavalier, I should convert to Gentoo. It apparently solves all of my problems.

Sorry, but I get tired of people always saying this type of thing about ...well, anything. Examples:

Q: "I need help with my sendmail configuration." A: "Use postfix/qmail/etc."

Q: "How do I get <some device> working under Linux?" A: "Use FreeBSD."

Q: "Take a look at this package management system." A: "Switch your distribution!"

All of these have better, usable answers that actually answer the question.

I don't mean to flame or troll, but responses like this are counterproductive and annoying.

--RJ

because that would be bad

[g4dget]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes.

That works fine for a few applications. Linux has thousands of applications, and people tend to install hundreds of them (they are free, after all, so why not). Do you want to go through hundreds of GUI installers, and then hundreds of GUI updaters? I don't.

Why can't someone make something like this for Linux?

There are interactive installers for Linux packages, but they are usually a nuisance compared to a normal package.

But every time I download something that's not part of Debian it turn into a horrible experience I wish I would have never had.

Well, then don't install non-Debian packages. After all, there are plenty of Windows programs that come with horrible installers. As a Debian user, think of non-Debian packages as "programs that come with horrible installers", and then decide whether they are worth the trouble. (Note that you can usually import packages reasonably well via "alien".)

The package system you get with Debian (or RedHat, for that matter) is already so much better than anything you get for Windows that it isn't funny. If Linux developers adopted the equivalent of setup.exe more widely, that be a real blow to Linux.

Re:because that would be bad

[sir99]

The package system you get with Debian (or RedHat, for that matter) is already so much better than anything you get for Windows that it isn't funny. If Linux developers adopted the equivalent of setup.exe more widely, that be a real blow to Linux.

You are painfully correct. Just look at the commercial programs available for Linux that use installer programs. I have several examples:

1. Macromedia Flash plugin 6: It comes with a ridiculously long script that checks all kinds of special conditions, which you can tell was written by a newbie. What does it ultimately do? Install two (2) files! I simply unpacked the tarball and symlinked them into my plugins directory.
2. RealONE (Real media player): It doesn't appear possible to install this program globally, only in your home directory. It uses a GUI installer that's heavy on flashiness, but low on usability. I didn't feel like messing with it to try and make it world-usable.
3. Intel C Compiler: Haven't looked at this in a long time, but I heard that it's very difficult to install it on some systems; you basically have to fix the installer's broken assumptions. The old version I have had a hacked-up RPM to allow it to be installed on Debian.
4. Sun JRE 1.4: This comes as a "self-extracting" executable. In reality, it's a shell script with a tarball tacked on the end. I don't remember why anymore, but I had to futz around with the script to make it work. In addition, it depended on an old version of libstdc++ that I had to find.
5. Oracle: Never used this, but I hear it's a bitch to get working on some systems.

Now granted, they all use a custom installer rather than something like InstallShield, but I see parallels to Windows setup programs here.

best of both worlds?

[Ender Ryan]

How about `/home/$user/My Documents' ?

I don't see what's wrong with that... Personally, I keep a lot of miscellaneous documents in `/home/$user/Docs'.

I can't stand the way in Windows everything is "My" this and "My" that... It's like it's made by Fisher Price for 3rd graders... I guess that would explain the new look in XP :)

Use checkinstall instead

[wct]

Checkinstall automatically produces native packages (rpm, deb, slackware tgz) from a standard make install. I've found this gives the best of both worlds - easy, consistent package management coupled with flexible/optimized source configuration.

Re:Use checkinstall instead

[volkerdi]

Because checkinstall uses tar+gz instead of Slackware's package building tool (makepkg), it produces broken Slackware packages. It should be fixed to use makepkg -- then I'd actually recommend it.

Slackware packages are not simply tar+gz. It's important that the files are stored into the tar archive in a certain way, the correct version of tar is used, and the symbolic links are moved into the installation script properly, otherwise the package can't be effectively managed. You wouldn't try to make an rpm or deb with tar/cpio/bzip2/gzip/etc, so why people think they can tar up some files and call it a Slackware package is beyond me.

Dependencies should use open format ...

[vinod]

It is good to be able to use independent directories for applications that are installed at the site (i.e. not part of the distribution.) And RPM can accomodate such independent directories as well. Within the independent tree, the applications should standardize the directory structure just like in unix: ./etc etc.

Then, putting symbolic links in various directories is bad idea. Instead, users could explicitly 'subscribe' to the directories. A special, user specific ./bin directory can be used to keep the subscriptions to bin directories of subscribed packages.

Bad thing about RPM is that it uses a centralized DB for tracking dependencies, which can't be manupulated by hand. Instead, it could evolve to use 1. open format based on XML 2. Put the dependencies as part of independent directory tree of the package.

In most cases, it is sufficient that dependencies be evaluated dynamically. After all, sysads know what they are doing.

-vgk

SEPP ideal package management

[schweikert]

SEPP is a package management system that allows to separate packages in directories like Stow and similar, but in addition:

• solves the distribution problem by allowing to mount packages with NFS and using the automounter to make the applications available under a standard path (/usr/pack/PACKAGE)
• provides for each application a wrapper script that takes care of all the necessary environment setup so that users don't need to edit their bashrc
• supports installation of multiple versions of the same application by installing version-tagged binaries in addition to the normal binaries. I can for example run mozilla-1.1 or just mozilla, in which case I get the "default" version. This is very important for example for a Ph.D. student that wants to finish his thesis with Matlab 5.3.
• automatic generation of web documentation (have a look here)
• usage logging with syslog
• dependencies

Re:How long

[virtual_mps]

how long is it going to take for something on linux to do what installshield does on windows?

And what would that be? Guess what things are installed and try to remove them properly? This is a tool for a system manager to keep track of what's installed on a system--windows has no advantage whatsoever for that. (Multiple programs overwriting the same dll, registry entries spreading everywhere, lack of seperation of users in many programs, etc., mean that there's a lot of work to do in windows.)

I admit not knowing about linux, and am doing little to change that.

If you don't know and don't care, wouldn't it be easier to just not post?

bah

[Illserve]

I don't need help managing my package, thank you very much.

Stow is great!

[__david__]

I've been using stow for years. I use it on RedHat, Debian, Solaris, and Mac OS X. It's just a perl script so it is very cross-platform.

One thing I noticed is that the author of this aritcle installed stow into /usr/local/stow/stow but then didn't stow it. He says:

2. After installing Stow, add the absolute path to the stow directory to the $PATH variable.

I always stow stow itself so that I don't have to mess with paths. Most every OS already points to /usr/local/bin. This line will bootstrap stow:

/usr/local/stow/stow/bin/stow -d /usr/local/stow stow

Despite the authors implication, stow isn't meant to be compared to rpms or .debs. It's a tool for managing your /usr/local heirarchy. And it's very good at that.

The one thing to look out for with stow is "make install" on various packages. I learned to ALWAYS "make -n install" because a lot of packages are broken and don't install into the "prefix". That is, even thought you "configure --prefix=/usr/local/stow/xxxx" the dumb package will still try to put things in /usr/bin or /usr/local/bin or some other hardwired path. I've had to clean up numerous braindead makefiles.

The greatest part of stow is not the installation, it's the deletion. Before I knew about stow, I didn't ever attempt to delete anything in /usr/local. When you look at a random file, it's impossible to tell where it came from! With stow it's a symbolic link so you can tell who "owns" it, and with stow -D, you can remove whole packages at a time. I've used it for installing new compilers: I have 2 gccs in /usr/local/stow/gcc-arm-elf-2.95 and /usr/local/stow/gcc-arm-elf-3.0.

This:


stow -d /usr/local/stow -D gcc-arm-elf-2.95
stow -d /usr/local/stow gcc-arm-elf-3.0


Will switch from the old compiler to the new compiler. It's just as easy to switch back to the old compiler if you need to.

Ah. I love stow!

-David

You don't want InstallShield

[Wee]

how long is it going to take for something on linux to do what installshield does on windows?

I was a build engineer for a large telecom company who had a commercial Windows product which used InstallShield. I wrote the installer for that product (twice). I've been using Linux since 1994 in various ways. I know InstallShield and I know Linux, and trust me, you don't want IS for Linux, no matter how much you think you do. People are better of sticking to whatever package management system comes with their distribution than pining away for something like InstallShield.

Having a single point of installation management is far superior (even with dependancy checking issues, like sometimes happens with RPM) than leaving it up to the software maintainers. Windows should have orginally shipped with such a centralized system IMO. Now they force software developers to jump through hoops (which cost money, incidentally) in order to get a sticker on their box saying their software was "Designed for Windows". Barring that "certification" a person can do whatever the hell they want in an installer, and your system can become an unorganized mess in very short order.

Even now on this Win2K machine I've got more than a dozen apps (most of which are commercially released) that aren't listed in the Add/Remove Programs applet. If I install something via RPM or apt, that can't happen. I just have to hope that one of those Win32 apps doesn't conflict with something else down the road, since there's no way to remove them programmatically. That's a critical flaw of Windows' installation setup, IMO. Just one that happens to come to mind, even. I could go on all day...

I admit not knowing about linux, and am doing little to change that.

Your parents must be terribly proud at having imbued in their progeny such an insatiable thirst for knowledge.

-B

Checkinstall DOES use makepkg

[izto]

And has used it from the very beginning. So you could start recommending it right now ;-).

Checkinstall was initially developed in the time when makepkg had no command line options to answer the questions about symlinks and permissions, so Checkinstall uses by default a makepkg from Slackware 7.1 which was hacked to not ask those questions and instead assume specific answers to them. And it has been updated with the changes and patches introduced in Slackware 8.0 and 8.1 (i.e. the "slack-desc" file with handy ruler and all)

Aditinally, you can still use Slackware's native makepkg if you set the MAKEPKG variable in checkinstall.

Re:Checkinstall DOES use makepkg

[volkerdi]

The fork of makepkg used in checkinstall produces broken packages, because it uses the wrong version of tar. This packs files differently, and it DOES make a difference.

As soon as checkinstall uses Slackware's native makepkg by default on Slackware I'll be inclined to recommend it. Still, it's misleading to users of other versions of Linux who might think they are building correct Slackware packages and aren't. I did commit one patch to installpkg to detect and correct for the most common problem with non-standard packages, but can't correct for them all.

Pat

Re:Checkinstall DOES use makepkg

[izto]

Yes, I found out about this recently, and already patched checkinstall to use the right tar version if available. The issue was kind of confusing, though: having two different tar binaries in a Slackware system, with "tar" being the wrong one to use when building a Slackware package.

Feel free to contact me with any other issues about the way checkinstall creates the Slackware packages. Building proper packages for the system it is running on (Be it Slackware, Debian or RPM based) is very important for me. The Debian folks are particularly picky about this and we've managed to get checkinstall to build mostly Debian-compatible packages already, for example :-)

No ... don't use make-install, use configure!!!

[shellbeach]

We (the Tcl core developers) have had problems in the past with Stow, mainly because it relies on being able to specify the installation process at 'make-install' time instead of normal 'make' time, leading to messed up baked-in paths... :^/

Yes, but that's only a problem with the stow documentation. Use "--prefix=" during configure and you'll have no worries at all (except that the "baked-in" package names will be '/usr/local/stow/yourpackage/etc ...', but that has never mattered to me and I have about fifty stow packages installed on my system, with everything from gtk-2.2 to lyx to rxvt).

I have used stow for the past year and absolutely love it. It allows me to have complete control over all the software I compile by had, as opposed to the base system installed by my distro. And since I have a bash alias to ./configure to include an automatic prefix assignment based on the directory name I'm configuring from (which is almost always based on the name and version number of the software), I can compile a new version and

stow -D /usr/local/stow/foo-1.2

stow /usr/local/stow/foo-1.3

... without losing my old version that I know works. (so, if my new version segfaults, I can "install" the old one simply by reversing the above process)

And ... I can do a simple "du" command in the /usr/local/stow directory to see exactly how much disk-space each package is using and I can easily find, modify or delete a part of a package I compiled months ago!

Stow is one of the most fantastic pieces of software, and it's simplicity itself as well. It reports conflicts and only installs sym-links. The scary thing is that this is the first and only time I have ever seen it reach "mainstream" coverage - like most of the best linux software, it seems to be unheard of and unused.