Rage Against the File System Standard

pwagland submitted a rant by Mosfet on file system standards. I think he's sort of over simplified the whole issue, and definitely wrongly assigned blame, but it definitely warrants discussion. Why does my /usr/bin need 1500 files in it? Is it the fault of lazy distribution package management? Or is it irrelevant?

Comments

Re:The Alternative?

[Anonymous Coward]

I'd much rather have 2000 binaries in /usr/bin than 2000 path entries in my $PATH

Here's what every unix administrator I know (including myself) does:

1. everything is installed in /opt, in its own directory:
   
   example$ ls /opt
   apache emacs krb5 lsof mysql openssl pico ucspi-tcp
   cvs iptables lprng make-links openssh php qmail
   
   (pico is for the PHBs, by the way)
   
2. Every version of every program gets its own directory
   
   example$ ls /opt/emacs
   default emacs-21.1
   
   
3. Each directory in /opt has a 'default' symlink to the version we're currently using
   
   example$ ls -ld /opt/emacs/default
   lrwxrwxrwx 1 root root 10 Oct 23 16:33 /opt/emacs/default -> emacs-21.1
   
   
4. You write a small shell script that links everything in /opt/*/default/bin to /usr/local/bin, /opt/*/default/lib to /usr/local/lib, etc.
   

Uninstalling software is 'rm -rf' and a find command to delete broken links. Upgrading software is making one link and running the script to make links again. No need to update anyone's PATH on a multi-user system and no need to mess with ld.so.conf. You can split /opt across multiple disks if you want. NO NEED FOR A PACKAGE MANAGER. This makes life much easier, trust me.

Keeping one applications files in one place

[tjwhaynes]

The unix system doesn't really dump all the files in /usr/bin. These are, almost without exception, executable files. For each executable, support files are usually installed into one or more directory trees, such as /usr/share/executable_name/. The main convenience gained by having all the main binaries in one place (or two - I usually try to leave system binaries in /usr/bin and my own installations in /usr/local/bin) is convenience for searching paths when looking for the binaries.

However, this paradigm is pretty ugly if you are browsing through your files graphically. It would be nice if each application/package installed into one directory tree, so you could reorganise the system simply by moving applications around. For example,

/usr/applications/

/usr/applications/games/

/usr/applications/games/quake3/

.. this dir holds all quake 3 files ...

...etc..

/usr/applications/graphics/

/usr/applications/graphics/gimp/

... this dir hold all gimp files

...etc...

If this appeals to you, you might like to check out the ROX project. This sort of directory tree layout was the standard on the Acorn Risc OS and made life extremely easy for GUI organisation. It makes a lot of sense to use the directory tree to categorise the apps and files.

Cheers,

Toby Haynes

Re:I think it is better...

[Daniel Serodio]

No need to do the dirty work by hand, that's what GNU Stow is for. Quoting from the Debian package's description:

GNU Stow helps the system administrator organise files under /usr/local/ by allowing each piece of software to be installed in its own tree under /usr/local/stow/, and then using symlinks to create the illusion that all the software is installed in the same place.

sounds like Encap

[_|()|\|]

I think it is better to install all your programs binaries under a subdirectory, then symlink the executables

You want the Encap package management system. From the FAQ:

When you install an Encap package, the files are placed in their own subdirectory, usually under /usr/local/encap. For example, if you install GNU sed version 3.02, the following files will be included:

• /usr/local/encap/sed-3.02/bin/sed
• /usr/local/encap/sed-3.02/man/man1/sed.1

Once these files have been installed, the Encap package manager will create the following symlinks:

• /usr/local/bin/sed -> ../encap/sed-3.02/bin/sed
• /usr/local/man/man1/sed.1 -> ../../encap/sed-3.02/man/man1/sed.1

The normal user will have /usr/local/bin in his PATH and /usr/local/man in his MANPATH, so he will not even know that the Encap system is being used.

The technique is essentially compatible with RPM, but Encap goes so far as to define a package format, which probably is not. If you like RPM, you might do better to simply follow the same convention.

Look at opt_depot

[jonabbey]

Many years ago, we wrote a set of Perl utilities for automating symlink maintenance called opt_depot.

It's similar to the original CMU Depot program, but has built in support for linking to a set of NFS package volumes, and can cleanly interoperate with non-depot-managed files in the same file tree.

Re:The Alternative? No Alternative!

[rnturn]

We do the same thing on our Tru64 boxen. All 3rd party software goes in /opt or /usr/opt. 3rd party executables go in /usr/local/bin. Some executables live in an app-specific subdirectory under /opt and the symlink in /usr/local/bin points to the physical location. It makes OS upgrade time tons simpler. And the first step of our DR plan is to backup OS-related stuff and backup software on special tapes. Those get restored first so that we get a bootable system in a hurry. Then the rest of the software and data can be restored using the 3rd party backup software. None of this would be as easy to do if we had 2000 programs all living under /usr/bin. If Mosfet has a point it's that some distribution vendors make a mess out of the directory structure by dumping way, way too much stuff under, say, /usr/bin.

\begin{rant}
RedHat, are you listening? I like your distribution but the layout of the files you install sucks big time. Anyone who updates their applications (Apache, PostgreSQL, PHP, etc.) from the developer's sites has to undo the mess you guys create. Either that or don't install the versions on your CDs at all and just go with the source tars.
\end{rant}

(OK, I feel better now...)

Re:he's pretty far off base

[Marasmus]

You're right - Slackware, Debian, and SuSE (relatively older players in the Linux game than RedHat) did do this heavily in older versions. However, there has been some work in each of these distributions to remedy this. For example, in Slackware 8, all GNOME default-install stuff is in /opt/gnome (which is sensible and clean), all KDE default-install stuff is in /opt/kde (likewise), and contrib packages normally get installed in /usr/local (the semi-official place for things you compile yourself) or /opt (more sensible, since these are still distro packages).

As far as commercial UNIXes go, they really *are* better organized than the average Linux distribution. I'm speaking mainly from Solaris experience, but BSD/OS and HP/UX also keep a pretty good level of modularity to the filesystem structure.

RedHat certainly didn't start this fiasco, but then again they haven't been very proactive in fixing these problems either. I can't speak for GNOME or KDE on RedHat (since I only use RedHat for servers without X), but the contrib packages practically all get thrown into /usr and make things a real nightmare to manage. Added atop that dependency conflicts where Program A needs library 2.3.4 while Program B needs library 2.4.5, and the system approaches unmanageable at a very high rate of speed.

A little more modularity in the file organization department wouldn't hurt us. It could also help the dependency problems if the package maintainers use a more modular file structure to their advantage.

Re:The Alternative?

[psamuels]

The system does not go through all of the directories in the path every time you type a command. No shell that I know of is stupid enough to do that.

True, but it doesn't help in the situation where you have a short shell script running - the shell that runs the script has to hash all those directories.

It just adds to overhead of running a shell script, and that is something I am opposed to on principle. (It's also why I use ash for /bin/sh rather than bash.)

Now, I believe the truly intelligent shells do not pre-emptively cache your whole path, they just add entries to the cache as needed. Either way, though, having a long path is harmful to performance - and a short-running shell (running a short script, say) is penalised more than a long-running shell due to less use of cache.

As an aside, I believe the only things that belong in bin dirs are binaries a user or administrator might ever actually want to run. In this regard, I think Debian packages sometimes go overboard - daemons, in my opinion, should go in /usr/lib/{subdir} or something rather than /usr/sbin, since you should really be invoking them via /etc/init.d/* scripts.