Essential UNIX Tricks and Tools?

Chris Lesner asks: "What handy UNIX tricks/tools do you use everyday? I'm asking for stuff that amazes your friends and makes you wonder how they use UNIX w/o them. Some simple examples include: job control (with fg, bg/&, jobs, Ctrl-Z); moving login sessions between machines with Screen for vt100 and VNC for X11 and using screen and VNC to share login session b/w users for demos etc.; using find, xargs -i and echo to build command strings which after inspection can be piped back though bash e.g. `find . -type f | xargs -i{} echo "cp {} {}.bak" | bash` I'm asking b/c my source for this kind of information has dried up as my UNIX skills have matured. I'm guessing other Slashdot readers have the same problem. By the way, if you think the examples I give are lame I challenge you to better them!"

Comments

Unix Power Tools

[Bouncings]

All the tricks you can cram into one of those big books, and more: Unix Power Tools -- O'Reilly's best book, IMHO.

Stupid shell trick

[Bouncings]

This is pretty simple, but on one of my machines I like to have the same directory location when I log in and out. If I'm in /home/foobar/monkey/dance, and I close my ssh connection, then I open a new one, I want to start in /home/foobar/monkey/dance. Here's how:

In my .bashrc:

cd "`cat ~/.lwd`"

In my .bash_logout:

pwd > ~/.lwd

Pretty basic, but it does the job. One problem you have to watch for: scp'd files appear in your last working directory.

How about

[Hard_Code]

find . -type f | xargs -i{} echo "cp {} {}.bak" | bash

find . -type f -exec cp '{}' '{}'.bak ';'

Re:How about

[Zapman]

The slight problem with this (not in this case though) is you have the expense of forking and killing a lot of processes (equal to the number of files +1 for the find). On the mv or cp case, you can't (easily) get around it, but if you were to do:

find . -type f -exec chmod 644 {} \;

vs

find . -type f | xargs chmod 644

you'll find that the second runs amazingly faster, since it will group a lot of the commands together into 1.

I did 'cd /usr/bin ; time find . -type f -exec ls {} \;' and got:

find . -type f -exec ls {} \; 0.98s user 3.28s system 88% cpu 4.831 total

I did 'cd /usr/bin ; time find . -type f | xargs ls -1 ' and got:

find . -type f 0.00s user 0.02s system 31% cpu 0.063 total
xargs ls -1 0.03s user 0.04s system 74% cpu 0.094 total

That's a BIG difference, especially when you have a LOT of files.

(NOTE: if I did a bare 'ls' with the xargs, the output would be different due to the way xargs works (man xargs for more details). It may or may not make a difference depending on how you're using the output. If you're shleping the whole line at once, it could make a huge difference... if you're spliting the lines by $IFS or something, then it's probably alright.)

As for my contribution, I find these 3 find | xargs commands, wraped together in a script I call "makereadable" help me a LOT (for example, if you install from source, and the permissions get borked due to forgetting to set root's umask to 022):

find . -type d -print | xargs chmod 755
find . -type f -perm +0100 -print | xargs chmod go+x
find . -type f -perm +0200 -print | xargs chmod go+r

The first makes all directories 755. 99.99% of the time, that's what you want... just don't do it with /tmp (sticky bit will get wacked). The second finds all files that are executable by the user owner, and makes them executable to others too. The third finds what is readable to the user, and makes it readable too.

If you want to do this with write permissions, you're probably doing something stupid you will regret later. Figuring out the command to do this is then left as an exercise for the reader. :-)

"watch" command

[Anonymous Coward]

A lot of people don't know about the "watch" command

watch -n1 df -h

I always use hdparm to speed up IDE drives too.

hdparm -c1 -d1 -A1 -a16 -m16 -u1 -S240 -W1 -k1 -K1 -X66 /dev/hda

Of course there's aliases you can set to reduce keystrokes.

alias ll='ls -l'

pgrep and killall can be pretty useful.

killall java
renice 19 `pgrep java`

I also have a button on my GNOME panel that runs xkill so I can click on a locked up GUI app and terminate it.

If you have a multiprocessor box, you can do "make -j3" to run parallel builds.

And updatedb and locate are really useful for finding files quickly.

If you're trying to remember a command you've run before you can run this to find it quickly.

history | grep 'command'

They you can run !850 to run command #850 from your history for example.

And you can run this to find GNOME packages.

rpm -qa | grep gnome

That's all I can think of for now. Other posters are welcome to point out faster ways to do any of the tips I've given. :p

Re:"watch" command

[Permission Denied]

If you're trying to remember a command you've run before you can run this to find it quickly.
history | grep 'command'

If you're using bash or zsh with the default emacs-like bindings, you can use ctrl-r to do an interactive backwards search in your history. Just keep hitting ctrl-r to search deeper back into the history.

Also, the one thing I've noted a lot of intermediate Linux users don't know about, but find very useful upon investigation is job control. Even with good window management and multiple xterms, job control is very useful: I often have three or four jobs on each xterm. Also, a neat trick: with zsh or bash, "fg %-" goes back to the second-to-last job. You can use this to quickly suspend and switch between two jobs continuously (eg, after you've done it once, "ctrl-z, ctrl-p, enter" switches to the "other" job).

Also, if there's a long command I recently executed, I usually won't search through history, but rather use the "!cmd" thing. Eg, if you've recently run "gcc -g -Wall -O3 -blahblah ..." a few commands ago, you can do it again by just typing "!gc" without searching. I guess it's really a question of preference, but I picked this up from the older gurus who were weaned on the monstrosity which is the C shell.

Re:"watch" command

[phraktyl]

If you do a set -o vi in bash or ksh you get the vi mode. Then you can hit <Esc>/^scp<Enter> to get the last command that started with `scp', and then hit n for next, as in vi. You can also do most of the rest of your vi commands.

Real shell timesaver...

[stevey]

The biggest single command which saves me time is 'cd -' which changes to your previous directory under bash.

It doesn't sound terribly useful, but it is... Take my word for it.

Re:poor man's ftp

[tchuladdiass]

An easier way (and not needing xterms) would have been to use script:
$ script
Script started, output file is typescript
$ telnet remotehost
...
remotehost $ cat filename
...
remotehost $ exit
$ exit
Script done, output file is typescript
$
Of course, instead of cat, use uuencode for a binary file...

best grep trick ever

[novarese]

Instead of
$ ps aux | grep foo | grep -v grep
use
$ ps aux | grep [f]oo
The brackets will show up in the ps output but don't match your pattern, so your grep is automaticly excluded from your final output.

sort | uniq -c | sort -rn

[coyote-san]

I consider this pretty basic, but I remember some junior coworkers blown away by

<tt>something | sort | uniq -c | sort -rn</tt>

On the coding level, I constantly use the regex library (with extended regular expressions) and profiling. There are a lot of places where a single well-designed RE can eliminate many lines of code, and profiling can help you ensure that you close all files you open, free all memory you malloc, etc.

debugging

[Chacham]

Two biggies for me are 2>a to send errors to a file named "a". And strace, when you really need to know what a prgram is doing.

I couldn't live without strace. I had a font problem with X, strace told me where the problem was. xine wasn't playing sound, strace showed be that it was hitting the wrong entry in /dev/. The list goes on.

For Debian users, apt-spy (requires installation of the package) to find fast sites for package downloads, and apt-listchanges to list what all the changes are.

To wow silly Windows users, eject and aafire (part of aalib).

Re:Most important unix trick

[toast0]

your python script may crap out due to the stdin stream disappearing (i know nohup redirects stdout and stderr, but i don't think it redirects stdin)

you might try nohup python myscript.py /dev/null &

xtail is essential

[RevDigger]

One of the first things I install on a new system is always xtail.

It has nothing to do with X Windows.

It works just like 'tail -f' but it can watch multiple files. You can even watch whole directories and it will notice when new files are created.

http://www.unicom.com/sw/xtail/

Very handy tool.

Here's the useful one

[devphil]

After hacking around on *nix for years, everything in that book was old news to me. Well, almost everything. Out of 1000+ pages, here's the only trick I had never seen before:

If you have a directory full of files that are precious to you, run "touch ./-i" in that directory. This creates an empty file named "-i". Why do this? Because:

If you accidentally "rm *" there, the shell's default collating order while expanding the * will put the "-i" file first in the list, which will then be interpreted by rm as the interactive option, rather than a file.

(I've never used this trick, but I can see how it would be much less hassle than trying to remember to chmod u-w a precious file every time you get done working with it. Refinements on the technique include hardlinking the "-i" file to different directories instead of touch'ing it more than once, to save on inodes.)

(This also assumes that the user is already familiar with any of the half-billion methods of actually deleting a file whose name starts with '-'. This is /the/ most frequently asked FAQ in comp.unix.*.)

There. Now you know the most interesting trick in the whole book, and you don't need to spend the sixty bucks. :-)

Unix Guru's Univers

[GeorgeH]

UGU offers a Unix tip of the day at http://www.ugu.com/sui/ugu/show?tip.today - that should keep you in tips for a while.