Slashdot Mirror
Top 10 Items in the Linux Admin Toolkit
Joe Barr writes "Joe 'Zonker' Brockmeier has listed his favorite top ten tools for Linux system administration in a story on Linux.com, one of Slashdot's sister sites." From the site: " Since I spend a lot of my time working with text files, either when I'm writing and editing or when I'm mucking with configuration files and shell scripts, I've become very attached to my editor of choice -- Vim. Over the years, I've tried a lot of other editors, but none of them has been sufficient to coax me away from Vim. Part of the reason for that is the fact that I no longer have to think about using Vi-style keybindings, and adjusting to anything else would seriously hinder my productivity."
Condoms! With all the groupies chasing Linux system admins, you can never have too many condoms!
by far the most useful tool in troubleshooting.
.
You're right, with this, Emacs rocks.
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
the article said "toolkit" not "tool shed, tool warehouse, and tool factory on wheels".
MORTAR COMBAT!
Yes, let's dig one of the oldest flamewars back up in the summary rather than discuss some of the article's other excellent information.
TFA mentions WGet, one of the most wonderful, most needed applications that most users and admins ON EVERY PLATFORM don't know they need. Why not focus on this rather than ressurect the text editor wars?
The next Slashdot story will be ready soon, but subscribers can beat the rush and slashdot the links early!
adjusting to anything else would seriously hinder my productivity, for a couple of weeks after which it may improve significantly.
I like vim for command line and something like textpad for GUI text editing. I've never tried Emacs, but then again I don't do a huge amount of text editing.
thank God the internet isn't a human right.
nothing quite like a vi/emacs flame fight to cap off the afternoon, eh? thanks y'all!
The only 3 commands any Emacs user needs to know:
Ctrl-x
Ctrl-c
vi
Don't forget:
- 1 bag of Cheerios
- 3 lettuces
- 1 lbs of carrots
- Detergent
- 6 Dr. Pepper bottles
- 1 box of tampons
otherwise don't bother to come home.
-- Your wife Linda
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
d that bash has made some progress over the years, but zsh is my friend.
The autor makes a quick mention of vim, but what I'd really like to see is a full-blown developer session with vim, covering everything--common activities dealing with source, navigation, most used keystrokes, popular .vimrc mods, everything a "dyed-in-the-wool" GNU/Linux developer does when they are coding.
To take it another step further, a peer review and commentary about the session from other hardcore GNU/Linux developers at the end, so I can see the critiques, favorites, etc.
If anyone knows of such a article, or book, or could get such a thing started, I'd like to see it. The vim site is very helpful, but it is a lot more fragmented than what I'd like to see, something like "How the pros develop under GNU/Linux" with a full commentary/bull session attached...this would be very helpful to me.
Wget sucks, curl rules!
Media that can be recorded and distributed can be recorded and distributed.
-kfg
It is suprising the author chose "telnet" as one of the programs in his list.
Sure it is useful for diagnosing random problems, and troubleshooting things - for example connecting straight to a webserver, or simulating a POP3 login request, but I've always preferred netcat.
netcat is much more useful, it allows you to bind to sockets and handling incoming requests as well as make outgoing ones this introduction is a good read.
Missing tools from the list? curl, links/lynx, rsync, sudo, nmap, lsof, and less.
They usually give me stuff in Excel format...gotta export it to csv file...then, clean out imbedded CRLF's, headings and such...especially with free form txt fields. Anyway, a little scripting and scrubbing with awk and sed...and I can generate some good insert scripts for the databases.
Light travels faster than sound. This is why some people appear bright until you hear them speak.........
rm
preferably with the -Rf options.
Sony ha
But then, I've been using most of those tools for quite some time.
The one that interested me is cdb, which I hadn't heard of before. I'll have to try that out.
One of the most useful remote admin packages out there. Especially useful for those just getting into Linux/Unix. Want to install a Perl module? just select or enter the name. Wanna change a cron job for a user, it's in the Cron screen.
http://www.webmin.com/
cat .bash_history | awk '{print $1}' | sort
Mine: (flawed commands removed)
cal
cat
cd
cdd
random text to get past lameness filter - isn't this why we have mods? size does matter!
chmod
clear
cp
crontab
curl
du
echo
grep
random text to get past lameness filter - isn't this why we have mods? size does matter!
gunzip
head
kill
killall
links
locate
ls
lynx
mkdir
mv
random text to get past lameness filter - isn't this why we have mods? size does matter!
open
pbpaste
pico
pine
ps
rm
rmdir
scp
screencapture
random text to get past lameness filter - isn't this why we have mods? size does matter!
ssh
su
sudo
tail
tar
telnet
top
touch
unzip
uptime
which
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
I'll plug my own project here: Cream is Vim tricked out in single mode with all the development tools pre-configured with all useful shortcut keys self-documented in the pull-down menus. You won't need to go searching through the help ever again.
There is no need to use a SlashDot sig for SEO...
Sorry, I was going to have an example of a for loop with sed, but I got busted by the lame filter. Seems like it gets harder to get past the stupid lameness filter all the time, yet I *STILL* constantly see goatse ascii. Why even fucking bother with the stupid thing if it doesn't work? You assholes.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
When did a CD audio ripper become an essential part of an admin's toolkit?
my sig's at the bottom of the page.
Just kidding.
http://curl.haxx.se/docs/faq.html
1.3 What is curl not?
Curl is *not* a wget clone. That is a common misconception. Never, during
curl's development, have we intended curl to replace wget or compete on its
market. Curl is targeted at single-shot file transfers.
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
telnet is the program that lets you test SMTP by connecting to 25/tcp, unlike ssh.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Sorry, I was going to have an example of a for loop with sed, but I got busted by the lame filter. Seems like it gets harder to get past the stupid lameness filter all the time, yet I *STILL* constantly see goatse ascii. Why even fucking bother with the stupid thing if it doesn't work? You assholes.
Obviously and sadly, the gay niggers are intellectually superior to you.
apt-get update ; apt-get dist-upgrade
Oh. Not everyone uses Debian, right, sorry. They should, though. *mumbles*
"Eddies," said Ford, "in the space-time continuum." "Ah," nodded Arthur, "is he? Is he?"
I'm actually suprised this guy used telnet. I find netcat (man nc) as a much better alternative. I can script it, I can do udp, and I can do port "testing".
There's no easier way to lay the blame squarely in the Windows camp (or to eat your hat)...
A few diagnoses I've performed:
I would say tcpdump is the #1 program for serious problem-solving -- at least with the work I did...
Now, some of those are "well-known", but there are plenty there that few people (even on Slashdot) are likely to be overly familiar with.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
when I have to change a word ([^ ]+) up to the next space, my screen ends up full of yellow blocks.
:syn off)
:nohls (to disable HighLight Search results until the next search) or :set nohls (to disable for the session)?
Worst vi enhancement ever!
(yes I know about
But do you know about
I quite often benefit from highlighting my search results. But when they annoy me, I simply turn off the highlighting.
Definitely a good vi enhancement.
sdb
All the best tools are the great networking tools which allow me to track down and eradicate owned windows machines on the network.
nmap - hmmm what trojan do we have on the network today
tcpdump - great at finding worms scanners
iptraf - for finding weird clients p2p movie downloaders etc.
ethereal - great for grabbing pop and telnet passwords
aimsniff - just for amusement
With just those couple of tools I can keep the helpdesk busy for weeks by finding owned machines that need cleaning.
Got Code?
cat .bash_history | awk '{print $1}' | sort | uniq -c | sort -nr | head -20
ethtool and mii-tool. The Cisco 2970 switch we use has the knack of initializing the interface in half-duplex mode if the port is set to "Auto". Easily fixed by setting the port to Full, but useful nonetheless.
/etc/hosts.allow and /etc/hosts.deny, or where pop3/imap are looking for PEM certificates to configure pop3s/imaps, etc...
/proc filesystem. Favorites include /proc/net/dev, and /proc/uptime, /proc/cpuinfo, /proc/loadavg. Good for aggregating individual server load data in a cluster.
... permissive. Cut down on access to your logfiles and to config files in /etc.
.. might take a while. When you forgot to & your command, use CTRL+Z then bg your job.
strings. Good to check if executables are using
vmstat. Think your system is paging, or a card is generating too many interrupts?
awk and sed. Mentioned elsewhere, but priceless.
chmod. I think the Linux filesystem permissions are too
*quota*. A must for restricting disk space use.
umask. When you need root, set your default umask fairly tight. I use 0077, but when you need to cpan some common perl modules, switch to the more common 0022.
jobs, fg and bg. Old-school unix commands to play with jobs that
There may be more, but I can't think of any others right now...
I'm surprised at how few serious *NIX Administrators I know are using GNU Screen. For some reason, it seems that the majority of them have not yet discovered the sheer power of the console window manager. It allows me to manage dozens of virtual windows all within the same terminal. In addition to eliminating the need to window-switching on my local machine, it also allows me to perform complex select, copy, and paste operations using only my keyboard. WIthout using a mouse, I can select and copy text in one window, advance into another window, and paste the text. The best part is that if my DSL drops or I decide to reboot my computer, all of my screened sessions stay on the server, leaving my work in exactly the same place as it always was, and with a nice scrollback history. I couldn't work without it.
# wrote sig.txt, 23 lines, 31337 chars
Screen is very useful. I think of it as providing virtual terminals (you know, Ctrl-F1 to Ctrl-F6) but over remote login (usually ssh in my case)
My normal usage is like so:
ssh into the box
screen -r (resume session, that was created first time I went in after bootup, with screen -S)
Ctrl-A Ctrl-N (next virtual terminal) to flick through things i have running, usually centericq (text multi protocol chat client) and btdownloadcurses, or the results of a compile I left going.
Ctrl-A Ctrl-C (open new virtual terminal) which gives you another command prompt. Here I'll wget a tarball, extract it, start the compile, then Ctrl-A Ctrl-N back to centericq
I'll disconnect from the box (Ctrl-A Ctrl-D to disconnect from screen, then Ctrl-D to close the login shell), go home, reconnect from one of my other machines, go through the same process.
Also, I can stay connected, go to another machine, and use screen -x to have multiple connections to the same screen. In the case of a dropped connection, and screen thinks my old login is still valid, screen -rD will disconnect the old login.
Oh, and Ctrl-A ? will show all the in-screen escape sequences. Some of them I haven't used (there is one to do horiz split display, but it crashes btdownloadcurses), but look like they could be even more useful!
But really, the best part is not having programs killed on me because the internet connection dropped (as happens all too often around these parts).
Step 1. Begin compiling some huge thing (tm) during your screen session.
Step 2. Spend all day doing other things as compilation is off in some hidden window in screen.
Step 3. Check on compiling every now and then, you will be able to scroll back and forth as needed as if you'd been sitting with an xterm open the whole time.
Step 4. Leave work for home.
Step 5. Reattach to you screen session and check on your compiling some more.
Step 6. Realize inner peace knowing you need not worry about where you started some long process ever again by using screen.
we speak the way we breathe --Fugazi
There are other useful tools, but I pretty much use those on every single box I touch for any reason.
... (as a sysadmin)
grep - Simple, Powerful, Elegant
sed - Powerful and versitile
awk - For what sed doesn't do easily
perl - For what awk doesn't do easily
vim - In the end there can be only one... it's vim.
cut - is your friend
make - automation is good and make makes it easy
lynx - great for testing web stuff
slrn - news, it keeps you sane after being locked in the closet for a month
mtr - just flatout helpful
Shadus
How about that? Run it and your social life will immediately improve. It is the first step toward a better life. Won't you take it today?
1) Your analysis is based on bad assumptions so your result is way off. 2) You're a sick bastard for fucking a horse.
Knoppix.
Join the Slashcott! Feb 10 thru Feb 17!
I don't have any problem with that, I'm secure in my race and my sexuality. What I find amusing is that the gay niggers are intellectually superior to the people writing the lameness filter.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Plain and simple, zonk points out that you can not work all the time. Having that available allows you to rip to your local machine and then listen to them as you work. Made total sense to me.
I prefer the "u" in honour as it seems to be missing these days.
(I didn't know this until I learned emacs many years ago and realized that C-a and C-e worked in bash too. Then I noticed, hey, so does C-t, C-p, C-n, etc. C-r [isearch-reverse] is now my favorite bash feature, thanks to emacs. I always like it when learning one thing [emacs] makes another easier [bash].)
:q!
As for vi, I know how to do basic stuff in it, but it basically upsets me. I have a wrt54g that doesn't have emacs on it and editing config files is a PITA. Why doesn't vi let me go to the end of the line by pressing, say, end or going to the end of a shorter line and then pressing the up arrow? Irritating. I know a vi guru is going to explain how to do this now, but I don't really care about the answer. What am I going to do the next time I'm stuck in vi and have a problem? (Hint: ESC
Disclaimer, I don't care what you or my neighbors use for a text editor, nor do I care what kind of toilet you prefer to take a dump in. Its not that big of a deal.
I will say, that I find it a little strange that shells come with emacs bindings, when vi is kinda the default. I'm saying this because vi (or for some historical reason, ex) is the default editor and bindings in things like less and more. Aside from ^e and ^a, I don't know any other emacs commands. I do know that most shells will take vi commandline editing commands. Honestly, I forgot how to do this in my shell, zsh. Or maybe its the default. One of the things I like about zsh is that it is the only shell that correctly does multiline editing in vi mode, or at least it was. bash was unusable and broken at least in the past, there is no reason to change now.
Oh, and going to the end of the line and beginning of the line in vi uses the carat ^ to the beginning and $ to go to the end. I've never tried keys that are on the dark side of the keyboard (where my hands aren't), so I don't know about that. The ^ and $ are not that foreign, because they are used to denote the beginning and end of lines in regular expressions. Intuitive? No. Easy to remember and associate with other stuff that I do every day, yup.
As for vi, I know how to do basic stuff in it, but it basically upsets me. I have a wrt54g that doesn't have emacs on it and editing config files is a PITA.
Again, I don't care. But I will say that emacs is not a standard UNIX thing, vi is. vim, my favorite editor, is not. I too have to deal with using "plain vi" sometimes. Its close, but certainly no cigar. I will say, that it is frustrating to work with someone that does not know vi at all. I'm not saying guru, but at least the basics. I would say that ^ and $ are a little beyond the basics, but it is something that someone should be able to pick up and remember after the first time of hearing about it.
Again, vi, vim, or emacs are not better. vim and emacs have been in active development and used extensively over the years. They are both powerful editors. But everybody needs to learn the basics of vi if they are going to be using UNIX systems. Its one of those things. You do not have to like it.
A rubber hammer - you lack ambition. A 4lb bricklayers hammer works far better. Learned this one 25 years ago when I worked in a camera shop.
Repairs to fiddly bits of cameras are best not attempted while severely hungover - especially the morning after someone has tried to break in through the window in the basement. Anyway customer comes in with jammed camera - this model (Russian Leica 3 clone) easy to fix once the top plate removed so wandered through the back to get small screwdrivers. Meanwhile boss in basement trying to fix broken window and yells up for me to bring down hammer.
Customer sees me emerging from the back of the shop wielding a hammer. His expression truly terrified at the thought of my attempting percussive maintenance to his camera with it. He had not been reassured by the aroma of stale whisky either and I've never seen anyone leave the shop so fast before or since.
Much relief when I wandered down to the basement with the hammer. Ever since then a 4lb hammer has been in the toolkit as the customers tend to shut up and let you get on with it once you have produced that. Only time it ever got used was when trying to make an antique colour laser printer fit in a skip. Now that WAS fun, if a little sad - that beast earned me something like its purchase price over the years when it played up.
Besides from being shorter, mmv doesn't stop there. Your example doesn't scale to multiple replacements within one string, i.e. 'mmv "*_*.htm" "#1-#2.html"'. But above all, your example isn't safe. You could avoid filename collisions and accidental overwriting by aliasing "mv" to "mv -i"; however, that still wouldn't give you a complete check in advance, but make the command exit half-finished and non-reversable. mmv on the other hand cancels the operation before doing anything if there are any name collisions. - If you try to put all these features and safety measure into your shell script, you will probably end up with something that is as complex as the the C sourcecode of mmv...
gopher://cramer.plaintext.cc http://cramer.plaintext.cc:70
Vi plugin for the Eclipse editor: http://www.satokar.com/viplugin/
Anyway, if you use a GNU system like Linux, you should learn emacs. Bash (readline), info, etc. all work just like emacs. It will really save you time and effort when every app uses the same keystrokes. C-t is a great timesaver in both bash and emacs.
Glad you realized you can use emacs keystrokes on bash. This doesn't mean much for emacs though. Bash allows for either emacs or vi keystrokes to be used. Type "set -o vi" to get it it use vi keystrokes, complete with the two modes. "set -o emacs" (which happens to be the default) to use emacs.
Also note, that while info might use keybindings similar to emacs, 'less', which is one of the most often used commands, uses vi keybindings.
Why doesn't vi let me go to the end of the line by pressing, say, end or going to the end of a shorter line and then pressing the up arrow? Irritating. I know a vi guru is going to explain how to do this now, but I don't really care about the answer.
Next time you don't really care about the answer, please do us a favor and quit blaming it on vi.
You type "mk" (as in "mark") and "rt" (as in "return") to mark a directory and later go back to it.
Or you can give it a name: do "mk foo", and later on "rt foo" will move you back there.
But the Big Win? With the above, it gets set as a shell var: $foo is also set to the directory, so you can do things like "cp $foo/*.baz ." to good benefit. In addition, setting up
this system is just a trivial matter of setting environment variables in your .profile.
PS: Trivia: the "mk" and "rt" names were inspired by troff, where those commands were used to keep mark and go back to vertical positions on a page... yeah I'm an oldie.
If you need to automate something like telnet or ftp (or anything interactive on the command line), "expect" allows for some very powerful scripting.
It "expects" certain things and provides an appropriate output.
I found sed to be usful when I was writing shell scripts. Find for shure. And to amuse myself wall, for example, fortune -o | wall (add escape codes to kill your banner). Also ps can be important when things go wrong.
Many a long talk since then I have had with the man in the moon; he had my confidence on the voyage. Joshua Slocum
I was actually going to say something very similar, and then I realized that I use telnet over netcat for certain purposes, because telnet notifies me when it actually connects. However, a couple minutes ago I realized I was being an idiot, because 'nc -v' does the same thing.
Also, if you haven't already, check out nmap-ncat.
Better to light a candle than to curse the darkness.
Elinks has javascript support. I've deployed elinks to many systems as a last resort for the tech support crew when the ppp connect is hosed.
You may want to check out Tsync, one of the recent Google "Summer of Code" winners: "Tsync is a user-level daemon that provides transparent synchronization amongst a set of computers. Tsync uses a peer-to-peer architecture for scalability, efficiency, and robustness." Unlike rsync, Unison, etc., Tsync is a locally installed daemon which automatically and transparently syncs two or more hosts.
Your /proc stuff if great, here's a handy accompaniment: ever logged onto a system and had no idea what it is? Maybe a crappy Dell Poweredge or whiteboxx kit, maybe a nice solid IBM/HP box? Wanted to get the system's asset tag for support reasons?
dmidecode. Part of the kernel-utilss package on most Linux distros.
Sorry for plugging my own project but I think MultiTail can be very usefull for a lot of admin tasks. MultiTail lets you view one or multiple files like the original tail program. The difference is that it creates multiple windows on your console (with ncurses). It can also monitor wildcards: if another file matching the wildcard has a more recent modification date, it will automatically switch to that file. That way you can, for example, monitor a complete directory of files. Merging of 2 or even more logfiles is possible. It can also use colors while displaying the logfiles (through regular expressions), for faster recognition of what is important and what not. It can also filter lines (again with regular expressions). It has interactive menus for editing given regular expressions and deleting and adding windows. One can also have windows with the output of shell scripts and other software. When viewing the output of external software, MultiTail can mimic the functionality of tools like 'watch' and such. For a complete list of features, look here. Multitail can be found here: http://www.vanheusden.com/multitail/.
www.vanheusden.com - home of Multitail, HTTPing, CoffeeSaint, EntropyBroker, rsstail, bsod, listener, nagcon, nagi
As this table shows, curl does not have "Recursive Downloads", which wget does. Which is the most useful feature of wget - you can point it at a pr0n link or gallery site like so:
:)
wget -r -k -H --level=3
and it will follow all the pr0n links to three levels down, and retrieve all the pix/movies it links to. Saves you tens of hours of frustrated clicking and saving manually. Not that I personally use it for such vile things, of course
Here is my list of tools I can't live with in my admin life:
;))
#1: vim (My editor of choice for anything)
#2: netcat (beats telnet for the network testing forte)
#3: ssh-agent/ssh (for secure remote logins and other nice tricks (like tarring over ssh)
#4: nmap (for a quick network scan to see what hosts are up in an unknown network or an in depth portscan of a particular machine)
#5: perl (who needs shell script when you have perl
#6: mtr (cause traceroute is sooooo 1980's)
#7: screen (for those long taking operations on a bad network connection)
#8: grep (the all purpose filter/text finder with -r)
#9: find (flexible find/recursive operations tool)
#10: host (quicker and easier name resolves than dig)
The way to corrupt a youth is to teach him to hold in higher value them who think alike than those who think differently
- ssh (file transfers, port forwarding, encryption and remote login in one tiny tool. I even use it in place of WEP or WPA) /proc/cpuinfo, free etc. (Invaluable for hardware discovery. Boot a knoppix CD, run those commands and instantly you know everything about the hardware that you need to know.)
- pico (can't stand vi but pico is small and has enough of a help that I don't have to memorise keystrokes)
- grep, sed (with grep and sed, you can pretty much manipulate any file/program output into whatever you want, strip IP's out of errors/logs, etc.)
- x11vnc (like any other VNC program but supports Tight encoding and also lets me see what an EXISTING X session is doing. Combined with a script that seds/greps the auth code from the process list and you have automated remote desktop)
- screen (if for no other reason than it lets you start a job at work (like a kernel compile) and watch it's progress throughout the day even if you have to log off in between. And when you get home, you can still check on it)
- tinyproxy (wonderful small, easy to use web-proxy that I tunnel into from work to bypass the far-too-restrictive filters in the schools that I work in)
- slocate (worth it's weight in gold when you have it auto-indexing overnight across all filesystems. Where's that file I used ten years ago that had Xen in the name? a simple command, 2 seconds wait and you get the full path).
- dnsmasq (tiny util, bung it a massive list of public DNS servers and point your DNS requests to 127.0.0.1 and it will loop through them all until it gets a response. Failover to other servers, built-in full DHCP server, invaluable behind a NAT, simple config. Saved my life I-don't-know-how-many-times when my ISP DNS servers were feeling flaky. No one even noticed that half the time our ISP's weren't responding to DNS at all.)
- lsusb, lspci,
- dd, cat, more, sh, etc.(where would we be without them?)