Slashdot Mirror
Top 10 Items in the Linux Admin Toolkit
Joe Barr writes "Joe 'Zonker' Brockmeier has listed his favorite top ten tools for Linux system administration in a story on Linux.com, one of Slashdot's sister sites." From the site: " Since I spend a lot of my time working with text files, either when I'm writing and editing or when I'm mucking with configuration files and shell scripts, I've become very attached to my editor of choice -- Vim. Over the years, I've tried a lot of other editors, but none of them has been sufficient to coax me away from Vim. Part of the reason for that is the fact that I no longer have to think about using Vi-style keybindings, and adjusting to anything else would seriously hinder my productivity."
by far the most useful tool in troubleshooting.
.
You're right, with this, Emacs rocks.
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
Zonker is a great, and extremely smart guy. I had the opportunity to work with him for about 8 months at a data center. Anything I would have trouble with... and I mean ANYTHING, Zonker would always have the answer. He has lots of great articles, I recommend reading them!
Fundamentalism stops a thinking mind.
One of the most useful remote admin packages out there. Especially useful for those just getting into Linux/Unix. Want to install a Perl module? just select or enter the name. Wanna change a cron job for a user, it's in the Cron screen.
http://www.webmin.com/
Shame on those that only provide info, text, or worst, html documentation (or none)
For reading HTML docs over ssh or on an X11-less machine, might I recommend w3m? It's a very comfortable text-mode web browser. Unlike Lynx, it renders tables nicely, and even makes a brave attempt at frames.
-Stephen
I agree with most of your stuff and you're probably better at *NIX than I am but I might suggest replacing telnet with nc which does both connecting and listening.
-- Alper
telnet is the program that lets you test SMTP by connecting to 25/tcp, unlike ssh.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
I'm actually suprised this guy used telnet. I find netcat (man nc) as a much better alternative. I can script it, I can do udp, and I can do port "testing".
While rsync is very nice for syncing files, I've found another, somewhat related tool much nicer for my purposes. I have a desktop and a laptop computer, and to a great extent I work on the same files on both. Syncing my home directories could be done using rsync to some degree, but it's rather painful if you have modified files on both sides.
This is where unison comes in handy. It keeps two directories synchronized nicely, adjusting changes in both directions automatically and querying the user in case of collisions. One of my favorite tools, and I'm sure I'm far from being the only one with a need for something like it.
There's no easier way to lay the blame squarely in the Windows camp (or to eat your hat)...
A few diagnoses I've performed:
I would say tcpdump is the #1 program for serious problem-solving -- at least with the work I did...
rsync - although its binary diff algorithm is not very good, rsync is close to heavenly
Except it's a piece of junk! We used to use it to maintain a set of files between two servers. As long as the number of files was trivial, it worked. When copying 22M files with gigabit ethernet between two Sun servers each with 8 gigabytes of RAM, it took almost a week to copy over the single file that had changed. rsync took almost a gigabyte of RAM and almost a week of time just to copy one file! Now we just run a simple script that runs find then scp to copy complete files that changed. It take about two hours. The simple shell script is over 75 times faster than rsync.
These problems with rsync have been known for years. Do a search on deja.com to see all of the complaints.
Now, some of those are "well-known", but there are plenty there that few people (even on Slashdot) are likely to be overly familiar with.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
when I have to change a word ([^ ]+) up to the next space, my screen ends up full of yellow blocks.
:syn off)
:nohls (to disable HighLight Search results until the next search) or :set nohls (to disable for the session)?
Worst vi enhancement ever!
(yes I know about
But do you know about
I quite often benefit from highlighting my search results. But when they annoy me, I simply turn off the highlighting.
Definitely a good vi enhancement.
sdb
All the best tools are the great networking tools which allow me to track down and eradicate owned windows machines on the network.
nmap - hmmm what trojan do we have on the network today
tcpdump - great at finding worms scanners
iptraf - for finding weird clients p2p movie downloaders etc.
ethereal - great for grabbing pop and telnet passwords
aimsniff - just for amusement
With just those couple of tools I can keep the helpdesk busy for weeks by finding owned machines that need cleaning.
Got Code?
Better thank lynx is links. Handles tables, frames, etc. no problem. Also, it can be compiled with various options to make it a graphical browser too, which is handy for X exported over SSH type things.
Don't blame me, I voted for Kodos
ethtool and mii-tool. The Cisco 2970 switch we use has the knack of initializing the interface in half-duplex mode if the port is set to "Auto". Easily fixed by setting the port to Full, but useful nonetheless.
/etc/hosts.allow and /etc/hosts.deny, or where pop3/imap are looking for PEM certificates to configure pop3s/imaps, etc...
/proc filesystem. Favorites include /proc/net/dev, and /proc/uptime, /proc/cpuinfo, /proc/loadavg. Good for aggregating individual server load data in a cluster.
... permissive. Cut down on access to your logfiles and to config files in /etc.
.. might take a while. When you forgot to & your command, use CTRL+Z then bg your job.
strings. Good to check if executables are using
vmstat. Think your system is paging, or a card is generating too many interrupts?
awk and sed. Mentioned elsewhere, but priceless.
chmod. I think the Linux filesystem permissions are too
*quota*. A must for restricting disk space use.
umask. When you need root, set your default umask fairly tight. I use 0077, but when you need to cpan some common perl modules, switch to the more common 0022.
jobs, fg and bg. Old-school unix commands to play with jobs that
There may be more, but I can't think of any others right now...
Screen is very useful. I think of it as providing virtual terminals (you know, Ctrl-F1 to Ctrl-F6) but over remote login (usually ssh in my case)
My normal usage is like so:
ssh into the box
screen -r (resume session, that was created first time I went in after bootup, with screen -S)
Ctrl-A Ctrl-N (next virtual terminal) to flick through things i have running, usually centericq (text multi protocol chat client) and btdownloadcurses, or the results of a compile I left going.
Ctrl-A Ctrl-C (open new virtual terminal) which gives you another command prompt. Here I'll wget a tarball, extract it, start the compile, then Ctrl-A Ctrl-N back to centericq
I'll disconnect from the box (Ctrl-A Ctrl-D to disconnect from screen, then Ctrl-D to close the login shell), go home, reconnect from one of my other machines, go through the same process.
Also, I can stay connected, go to another machine, and use screen -x to have multiple connections to the same screen. In the case of a dropped connection, and screen thinks my old login is still valid, screen -rD will disconnect the old login.
Oh, and Ctrl-A ? will show all the in-screen escape sequences. Some of them I haven't used (there is one to do horiz split display, but it crashes btdownloadcurses), but look like they could be even more useful!
But really, the best part is not having programs killed on me because the internet connection dropped (as happens all too often around these parts).
I like vim for command line.....
You can't do a whole lot of typing on the command line then either. I've never bothered to learn to use the Emacs editor (tried about 15 years ago and hated it) so I always use the vi editor
The only time I've had to use vi mode for command line editing is on HP-UX, where the lack of a modern shell by default and crap emacs mode support have forced me into it. The usability difference is horribly noticeable.
Step 1. Begin compiling some huge thing (tm) during your screen session.
Step 2. Spend all day doing other things as compilation is off in some hidden window in screen.
Step 3. Check on compiling every now and then, you will be able to scroll back and forth as needed as if you'd been sitting with an xterm open the whole time.
Step 4. Leave work for home.
Step 5. Reattach to you screen session and check on your compiling some more.
Step 6. Realize inner peace knowing you need not worry about where you started some long process ever again by using screen.
we speak the way we breathe --Fugazi
(I didn't know this until I learned emacs many years ago and realized that C-a and C-e worked in bash too. Then I noticed, hey, so does C-t, C-p, C-n, etc. C-r [isearch-reverse] is now my favorite bash feature, thanks to emacs. I always like it when learning one thing [emacs] makes another easier [bash].)
:q!
As for vi, I know how to do basic stuff in it, but it basically upsets me. I have a wrt54g that doesn't have emacs on it and editing config files is a PITA. Why doesn't vi let me go to the end of the line by pressing, say, end or going to the end of a shorter line and then pressing the up arrow? Irritating. I know a vi guru is going to explain how to do this now, but I don't really care about the answer. What am I going to do the next time I'm stuck in vi and have a problem? (Hint: ESC
Disclaimer, I don't care what you or my neighbors use for a text editor, nor do I care what kind of toilet you prefer to take a dump in. Its not that big of a deal.
I will say, that I find it a little strange that shells come with emacs bindings, when vi is kinda the default. I'm saying this because vi (or for some historical reason, ex) is the default editor and bindings in things like less and more. Aside from ^e and ^a, I don't know any other emacs commands. I do know that most shells will take vi commandline editing commands. Honestly, I forgot how to do this in my shell, zsh. Or maybe its the default. One of the things I like about zsh is that it is the only shell that correctly does multiline editing in vi mode, or at least it was. bash was unusable and broken at least in the past, there is no reason to change now.
Oh, and going to the end of the line and beginning of the line in vi uses the carat ^ to the beginning and $ to go to the end. I've never tried keys that are on the dark side of the keyboard (where my hands aren't), so I don't know about that. The ^ and $ are not that foreign, because they are used to denote the beginning and end of lines in regular expressions. Intuitive? No. Easy to remember and associate with other stuff that I do every day, yup.
As for vi, I know how to do basic stuff in it, but it basically upsets me. I have a wrt54g that doesn't have emacs on it and editing config files is a PITA.
Again, I don't care. But I will say that emacs is not a standard UNIX thing, vi is. vim, my favorite editor, is not. I too have to deal with using "plain vi" sometimes. Its close, but certainly no cigar. I will say, that it is frustrating to work with someone that does not know vi at all. I'm not saying guru, but at least the basics. I would say that ^ and $ are a little beyond the basics, but it is something that someone should be able to pick up and remember after the first time of hearing about it.
Again, vi, vim, or emacs are not better. vim and emacs have been in active development and used extensively over the years. They are both powerful editors. But everybody needs to learn the basics of vi if they are going to be using UNIX systems. Its one of those things. You do not have to like it.
Besides from being shorter, mmv doesn't stop there. Your example doesn't scale to multiple replacements within one string, i.e. 'mmv "*_*.htm" "#1-#2.html"'. But above all, your example isn't safe. You could avoid filename collisions and accidental overwriting by aliasing "mv" to "mv -i"; however, that still wouldn't give you a complete check in advance, but make the command exit half-finished and non-reversable. mmv on the other hand cancels the operation before doing anything if there are any name collisions. - If you try to put all these features and safety measure into your shell script, you will probably end up with something that is as complex as the the C sourcecode of mmv...
gopher://cramer.plaintext.cc http://cramer.plaintext.cc:70
Vi plugin for the Eclipse editor: http://www.satokar.com/viplugin/
Anyway, if you use a GNU system like Linux, you should learn emacs. Bash (readline), info, etc. all work just like emacs. It will really save you time and effort when every app uses the same keystrokes. C-t is a great timesaver in both bash and emacs.
Glad you realized you can use emacs keystrokes on bash. This doesn't mean much for emacs though. Bash allows for either emacs or vi keystrokes to be used. Type "set -o vi" to get it it use vi keystrokes, complete with the two modes. "set -o emacs" (which happens to be the default) to use emacs.
Also note, that while info might use keybindings similar to emacs, 'less', which is one of the most often used commands, uses vi keybindings.
Why doesn't vi let me go to the end of the line by pressing, say, end or going to the end of a shorter line and then pressing the up arrow? Irritating. I know a vi guru is going to explain how to do this now, but I don't really care about the answer.
Next time you don't really care about the answer, please do us a favor and quit blaming it on vi.
fgrep = grep -F
I speak England very best
You type "mk" (as in "mark") and "rt" (as in "return") to mark a directory and later go back to it.
Or you can give it a name: do "mk foo", and later on "rt foo" will move you back there.
But the Big Win? With the above, it gets set as a shell var: $foo is also set to the directory, so you can do things like "cp $foo/*.baz ." to good benefit. In addition, setting up
this system is just a trivial matter of setting environment variables in your .profile.
PS: Trivia: the "mk" and "rt" names were inspired by troff, where those commands were used to keep mark and go back to vertical positions on a page... yeah I'm an oldie.
If you need to automate something like telnet or ftp (or anything interactive on the command line), "expect" allows for some very powerful scripting.
It "expects" certain things and provides an appropriate output.
Concur. But even more importantly, screen by default doesn't come with any sort of on-screen window list. This .screenrc line makes a screen session tremendously more comprehensible to me:
backtick 1 0 0 whoami
caption always "%>%{kw}%1`@%H %{bw}%?%-Lw%?%{bW}%n*%f %t%?(%u)%?%{bw}%?%+Lw%?%"
This will put a caption at the bottom of the screen listing your open shells and highlighting the current one.
[an error occurred while processing this directive]
Also, put this in your ~/.bash_profile:
if [[ `who -m` ]] ; then
[ -z "$WINDOW" ] && screen -xdR
fi
to start screen automatically on terminal logins and grab the previous screen session.
[an error occurred while processing this directive]
I was actually going to say something very similar, and then I realized that I use telnet over netcat for certain purposes, because telnet notifies me when it actually connects. However, a couple minutes ago I realized I was being an idiot, because 'nc -v' does the same thing.
Also, if you haven't already, check out nmap-ncat.
Better to light a candle than to curse the darkness.
Elinks has javascript support. I've deployed elinks to many systems as a last resort for the tech support crew when the ppp connect is hosed.
It is somewhat distressing that there are so many people in the Linux community who have missed out on a whole era of Unix and it's history. I'm being perhaps a little pedantic here but for clarity, /bin/sh is a link to /bin/bash on all Linux systems I've been on. So what
the author really meant to say was, I like /bin/bash
(me too!). /bin/sh is there to satisfy backwards compatibility with the Bourne and later POSIX shells. Strictly speaking, when bash is invoked as /bin/sh it should defer to the historical behaviour of Bourne or POSIX shells. Likewise it is telling (and painful) that
no mention is made of 'ksh' (korn shell), IMHO the best
shell of all (from a programming perspective). Much of
bash is taken from ksh. I suppose that if bash becomes
fully compatible with ksh I will have no longer of have
a need ksh but until then, ksh still sits as king of
the shell hill-top. And yes, please spend some time
writting shell programs. Shell is a most underated
programming tool. It is much more capable than many
people realize.
Nice post, but...
/bin/sh Only because it is always there by definition on UNIX systems, and a good shell programming language. tcsh, csh, and zsh are not as good as /bin/sh, and its always available, but a little boring to write about.
/bin/bash is there by definition. Functions, arrays, and other useful stuff.
> * [rs]sh - enables me to go places w/o passwords, copy files, and remotely execute commands. I can't live without it
rsh has lots of known security vulernabilities and shouldd not be used at all.
> * telnet - no, I never telnet to login to a machine, but I do it to test if a port is open, what's listening there, etc. very handy.
You should use nc for this. Telnet will fail with lots of data, only does TCP, and can't listen on ports. nc handles lots of data, can do TCP or UDP, and can listen as well as send.
> * Honorable mention -
True, but also, if the only Unix-like OSs you use are Linux,
Your /proc stuff if great, here's a handy accompaniment: ever logged onto a system and had no idea what it is? Maybe a crappy Dell Poweredge or whiteboxx kit, maybe a nice solid IBM/HP box? Wanted to get the system's asset tag for support reasons?
dmidecode. Part of the kernel-utilss package on most Linux distros.
Sorry for plugging my own project but I think MultiTail can be very usefull for a lot of admin tasks. MultiTail lets you view one or multiple files like the original tail program. The difference is that it creates multiple windows on your console (with ncurses). It can also monitor wildcards: if another file matching the wildcard has a more recent modification date, it will automatically switch to that file. That way you can, for example, monitor a complete directory of files. Merging of 2 or even more logfiles is possible. It can also use colors while displaying the logfiles (through regular expressions), for faster recognition of what is important and what not. It can also filter lines (again with regular expressions). It has interactive menus for editing given regular expressions and deleting and adding windows. One can also have windows with the output of shell scripts and other software. When viewing the output of external software, MultiTail can mimic the functionality of tools like 'watch' and such. For a complete list of features, look here. Multitail can be found here: http://www.vanheusden.com/multitail/.
www.vanheusden.com - home of Multitail, HTTPing, CoffeeSaint, EntropyBroker, rsstail, bsod, listener, nagcon, nagi
As this table shows, curl does not have "Recursive Downloads", which wget does. Which is the most useful feature of wget - you can point it at a pr0n link or gallery site like so:
:)
wget -r -k -H --level=3
and it will follow all the pr0n links to three levels down, and retrieve all the pix/movies it links to. Saves you tens of hours of frustrated clicking and saving manually. Not that I personally use it for such vile things, of course