Monitoring Your Unix Boxen?

Griim asks: "I've been using Linux for years and loving it, and have also worked a bit on a few Sun stations and BSD boxes as well. My question is this: what is the easiest way to keep tabs on all of the activity?"

"I know a few people who 'tail -f' the main log files, or who run 'top' every so-often. These require constant monitoring though, and you could miss essential error messages if you step away for too long. Are there any projects that do this successfully? I've seen a couple out there that started to do this, but appear to be abandoned.

Ideally, I would like some type of all-in-one, that possibly generates a daily (email/web) report of network statistics, user logins, and (web)server traffic/hits, as well as anything 'suspicious' that might be happening, perhaps what apps have been taking most of the processor time, or if any of the daemons have been busier than they normally would be. I know there probably isn't one single app out there that does all of this, so what's the best configuration , for keeping tabs on multiple machines, something I can skim for a minute or two each day, to make sure things are the way they should be? I want to know what works best, and just as importantly, what *doesn't* work (I do realize that relying on a single solution would be bad here too, so if you have more than one suggestion, that would be appreciated)."

Comments

monitoring, no one size fits all

[perlchild]

owing to the fact almost no product will fit everyone's needs

here are aspects where you can compare what you will find

aspects of monitoring:
-availability
-uptime(subtly different from availability)
-performance
-security
-capacity
-log or otherwise event-based monitoring

nature of tools:
-web based
-daemon with web based front end
-daemon without web based front end
-other

language tool is written in, license and source
-closed source, nuff said, available in licensed per cpu, licensed per target/service, etc...
-open source, but with paid-for license that includes support(shameless plug... I do support for this kinda thing)
-open source, roll your own support
-perl
-php
-java
-python
-c/c++

integration with other products
-by snmp traps
-by snmp agent extensibility(smux/agentx/proxysnmp,etc...)
-by proprietary methods
-by sharing a RDBMS with another monitoring tool(usually used for things like remedy ARS)

measure of performance/capacity/throughput/usage
-by the exec family of functions
-by the language of choice's own internal library conventions
-by snmp
-by proprietary methods to a Manager of Manager or NMS system
-by ciscoflow/other hardware vendor's protocol
-by parsing logs
-by exec-over-ssh-connexion

examples that don't fit neatly into any category that comes to mind is monitoring of backups(were they performed, how much, which files were skipped, etc, location in jukebox of which tape for which file...

Hope this helps you even draw the lines towards evaluating the product that meets YOUR needs

Re:Big Brother

[schon]

*please* make sure you read the article on it in issue #60 of Phrack

I've run BB for a number of years, and I got a good laugh from that article.. thanks..

the security concerns I had were enough to cause me to move along to the next product

The thing is, that if you've got security concerns, then you souldn't have a problem with using BB, because you're already aware of what needs to be done to prevent this information leakage.

The article you linked to didn't provide me with anything I didn't know before I originally installed BB - I run BB as an untrusted user without a valid login (why in the world would a daemon process require a password? - just set it to '*', and be done with it.).. and my status page is password protected, and encrypted.

BB's "security" is only a problem for people who don't understand security in the first place. If you know how to adequately secure a box, BB is no different than any other application.