Monitor Linux Performance With The Tools At Hand

Jan Stafford writes "Need to monitor Linux performance without purchasing a pricey diagnostic package? Try these simple, built-in command line tools. This article was written by site expert and author (Rapid Application Development with Mozilla) Nigel McFarlane."

Comments

Historic..

[noselasd]

Most utilities mentioned are really great, but mostly realtime stats
sometimes it's nice to see historic view on the machine as well.
sysstat does just that.
Now if only I can remember the thing that also use that statistics do
draw graphs (with gnuplot iirc.) Anyone ?

Re:Historic..

[Monster Zero]

Ganglia Distributed Monitoring System is a nice tool for monitoring lots of systems, I use it to keep an eye on 66 servers. My current template has plots for Current Cluster CPU/MEM/Beowulf Queue, Cabinet Temperature, Free Disk Space for both RAID units, then shows the utilization of each of the 64 nodes. Its fairly customizable and works great. Shows hour/day/week/month/year plots selected by a drop down menu.

It uses RRDTool to create the plots - very nice.

Re:Historic..

[sydb]

Is there anything Free around to give per-process historical information a la BMC Patrol ("Draw me a graph of memory utilisation over the last 4 hours for PID 3325, and show me how much of it was in real memory and how much was in swap"). OK the graph is a nice to have, the data would suffice.

Yeah, while true; do; ps -Fp3325 >> data.txt;sleep 1;done but I'm looking for something... Enterprise... I hate that word.

innacurate

[Danny Rathjens]

No offense, but this story is a joke. Did he even read the man pages for those utilities? The 'stat' in those tools names stand for 'statistics' not 'status'. And don't even get me started on stupid statements like "linux never swaps" and "If paging activity happens all the time, buy more RAM"

These utilities are explained better in the man pages themselves or the various system administration guides and howtos at the linux documentaion project.

Oh yeah, and he is missing one of the best tools for this type of thing: namely 'sar', the system activity reporter, which is enabled by default on all redhat distros. (I have an xpostit note dedicated to all the flags to sar for various things)

As for the graphing/monitoring questions people are asking in other posts; look for tools like nagios and mrtg and sysmon and mon or just search freshmeat.net. It's quite a common task which has been done many ways. My personal monitoring/graphs are perl scripts I wrote to fetch stats via ssh which I plug into mrtg.

Re:Remote realtime monitoring?

[ajayrockrock]

I use gkrellm2 (www.gkrellm.net) to monitor a handful of machines. It has a ton of stats that you can use and it's very helpful...

--Ajay

Re:disk performance?

[digitalunity]

If it is an option for your situation, you should at least investigate a migration to kernel 2.6.x. The IO system is MUCH faster, as is thread allocation. You should see a fairly significant increase in RAID performance, and least for striping. You should also make sure DMA/32 bit mode is enabled on the drives, not all drivers will set this automatically. I'm sure you've checked these, just a mention though.

As far as the network, it sounds like the driver is setting it to 100Mb mode instead of 1000Mb mode. You might try to find updated drivers for the cards. You might also check the documentation to see if the eth drivers support debug-info dumping. That might be a lot to read through, but you should find your answer in there somewhere.

If you post more hardware info, I'll post more specific answers.

Test Tools

[frobisch]

This is a good summary of available performance/test tools for Linux:
Linux Test Tool Matrix