Monitoring What Files Your Applications Leave Behind?

GoRK asks: "I have to install a commercial application on one of my servers. The application refuses to locate itself anywhere other than under the /usr tree, and I am concerned with it munging with my configuration under /etc as it automatically configures its daemons and whatnot. I am also a bit concerned with the method it goes about to verify its license on install. Is there any way I can run the installer in some sort of wrapper that allows me to monitor what files and network sockets get read/written to during the change so that I can monitor what data on my machine is getting moved around and also build a catalogue of every last little bit of the app in case I ever have to remove it?" In my opinion, a sensible software installer should have some form of user accessible package manifest included. Why should consumers trust third party software to "do the right thing" in the right locations, especially when installing software you don't have the source to.

"I am a big fan of MD5 sums and package management as a very reliable integrity check and I value the fact that every file on my system save user documents, etc. belongs to a package and I can verify its authenticity. I need to make sure that I know about each and every system file that gets modified during the installation. It would also be nice to see and control if it accesses anything under /dev/ just out of curiosity.

I've never been a diehard security freak before, but I just feel like it's 'time to do things right' so to speak. Is there a tool that will assist me here?"

Comments

chroot

[Anonymous Coward]

Run the application in a chroot'd sandbox.

Re:chroot

[BMazurek]

FreeBSD extends the ideas behind chroot with it's jail functionality.

Using this, you could set up a virtual machine, similar to the way the Openroot Project does.

Simple. Geesh.

[Amphigory]

bash# locate /usr/ > /tmp/usrfiles.old

do the install

bash# locate /usr/ > /tmp/usrfiles.new

bash# diff /tmp/usrfiles.old /tmp/usrfiles.new

Or, if you wanted to get fancier and check for changed files:

for i in `locate /usr/`
do
echo $i:`sum $i` >> /tmp/usrfiles.old
done

install

for i in `locate /usr/`
do
if grep -v $i:`sum $i` /tmp/usrfiles.old; then
echo $i >> /tmp/usrfiles.changed
fi
done

###

Shell scripting is your friend. Learn it well.

--

strace is your friend

[Convergence]

strace, which uses posix tracing, can trace every system call made by an application.

Actually, I frequently use it when debugging. Program doesn't start up correctly, I run it under strace to see if there's a file it can't find.

It'll show you every subprocess, every kernel call, every file access, network access, etc.

'strace netscape'

or

'strace -eopen netscape'

Re:Simple. Geesh.

[err666]

locate only finds files that are already in its database. You'll need to use find instead for this to work.

Simple way of seeing what files were create/modded

[dburr]

$ touch /tmp/instdate.<package-name>
$ <run the appropriate install procedure>
$ find <tree where it installed itself, or if unknown, /> -newer /tmp/instdate.<package-name> -print

Works for me... :)

--

Tripwire?

[tve]

Well, perhaps tripwire would be an option.

Re:This is where NT admins have it good :)

[crucini]

Yes, the Windows crowd are like children or savages, easily dazzled by shiny objects (shrink-wrapped software) and perpetually dependent on parents/witch doctors (boxware vendors/MS) for the talismans to ward off a scary, complex world.
But there are drawbacks to being an adult - we are burdened with the knowledge that the world is held together with duct tape and dried dung. For us there are no happy surprises in shiny packages, only the unerring certainty that software sucks and will continue to suck.
With regard to shells, I agree - it's utterly amazing that Microsoft hasn't managed to "innovate" a decent shell yet. But I have used Bash on Windows, and it wasn't pretty. Windows is sluggish in many ways that come annoyingly to the fore in shell interaction.

This makes a good point...

[TotallyUseless]

Many software installers will leave an installer log when they are done, but by then it can be too late. It seems rare that a software installer actually tells you what it is going to do before it does it. It is things like this that lead to unneeded tedium for the end user, such as backing up your configs before an install, 'just in case.' I just don't think we should be forced to go through this hassle, and I will make sure to clue the user in during/before the install in any software I write in the future. It had previously never occurred to me to do this, and I think it is the same way with most developers, be they commercial, open source, shareware, or whatever..

Re:You're taking this too seriously

[Grishnakh]

Oh please. You don't need to know how everything works. Microsoft made Windows easy to use and reliable, so all you have to do is use the install wizard and everything works great. If you're really curious about the inner workings, though, part of the way they achieved this unparalleled reliability and ease-of-use is through architectural decisions.
1. The Registry. This piece of utter genious is crucial in an easily-maintained system. Instead of having a bunch of separate configuration files to keep track of, you just have one big binary file which all the install wizards access and change. Sure, sometimes one application screws it up for everything else, but that's no problem. Just re-install everything, no big deal. In this day and age, it's simply ridiculous to consider going back to ASCII text configuration files; it's just too difficult for certified system administrators to read these and edit them. Besides, we can always pass the blame to one of the third-party software vendors. I know that every time I have a computer problem at work and I need to meet a deadline, I can just tell my boss how some non-Microsoft program screwed up my whole system and he'll say "no problem, take as long you need. As long as we're using reliable MS software, we don't need to worry about deadlines."
2. A central location for DLL's. By keeping all the DLL's in one or two directories, and allowing all applications to modify or overwrite them, we achieve breathtaking gains in system efficiency and reliability. Sure, sometimes you'll have minor problems with incompatible versions of the same DLL, but we can just blame that on the non-MS vendor and reinstall everything.

Honestly, you talk like your time is precious or something. What's the big deal if you spend days setting up a system and some big-$$$ application screws it up? You just call their tech support (for $300+ per call) and tell them about your problem, then spend the next week or two redoing everything. And if it messes up that time, do it again. Using the intuitive and attractive Microsoft Windows user interface is such a pleasure, those hours (days, weeks...) will pass in no time. Plus, you can have Clippy (tm) help you! What other software company has such a fantastic, useful innovation?

installwatch - checkinstall

[nsrbrake]

Checkinstall is a script that uses installwatch and rpm to build rpms from a source install. Do a search on freshmeat.net for checkinstall, the author also maintains installwatch. You only need to run installwatch as a wrapper to the install program and it will give you a listing of the files that it creates.

Change your mount points

[ConsumedByTV]

And then install it. Make a second /usr for the install. Or try using a small shell script that will traverse the file system and tell you all the recently modified files. Or just use a virtual machine.


The Lottery:

Test it first!

[JediTrainer]

I agree that it's probably a good idea to have a look at what it does (by running in some sort of sandbox), but here's my take on the whole situation in general...

First off, you shouldn't be installing anything untested onto a production server. What your company should have is a box identical in configuration to your production box (or at least a development server. You DO have those, right?).

Install the package on this server first. See what it touches. See if other packages misbehave after this is installed. Above all else, do not touch a production box period unless you've already seen what this program does.

If your staging box blows up, so what? It's happened to me. Nobody really relies on it, and that's what the box is for anyway. No big deal. Document everything that happens when you install on this machine. Since it's exactly the same as your production box, if everything works, then all you have to do is follow your documentation to install onto your production server, and all will work fine.

Trying to install stuff (even in sandboxes or wrappers) onto a server without testing it in a closed environment or staging area first is asking for trouble.

One Technique I have used...

[IBitOBear]

I used to keep a mock-up of a minimal system (or you can use a disposable separate computer) in a sub-directory and I would do test installs to that sub-directory using a chroot(ed) shell. This gave the opportunity to deconstruct things nicely. It was only used after other suspicous nonsense happened with a package or source.

If this is an RPM on a linux box use "--root somedir" to prefix things like /usr with "somdir". Other install methods like SVR4 "pkgadd" have similar relocate facilities.

If it is just a tar/cpio archive and an associated script to install, read the scripts.

In short, there is no one tool to do this stuff, but it *is* doable by a number of means (on a *nix box) where it is otherwize impossible to do it on a windoze install-wizzard bundle.

Hope this helps some.

Rob.

PS I have also had some luck by looking at the errors generated after trying to run the install as a normal, non priveliged, user.

--