Slashdot Mirror
(Useful) Stupid Unix Tricks?
So the other day I messaged another admin from the console using the regular old 'write' command (as I've been doing for over 10 years). To my surprise he didn't know how to respond back to me (he had to call me on the phone) and had never even known you could do that. That got me thinking that there's probably lots of things like that, and likely things I've never heard of. What sorts of things do you take for granted as a natural part of Unix that other people are surprised at?
I'm sure everyone at some point is surprised of tabbed completion.
I discovered if you give the echo shell command an asterisk as a parameter, it dumps out the file names of the current directory. (The sad thing is I had a practical use for this when a less-than-clueful-collegue deleted the /bin directory, leaving the system without an ls program).
--
Luck is just skill you didn't know you had.
I've seen Windows people go slack-jawed in astonishment as I ssh to the other side of the world and run X programs over forwarding.
Some refuse to believe it, others shake their heads and walk away.
On the older systems, talk was a great utility.
dd, device duplicator / disk destroyer
mount, what I can't have a desktop icon?
also managing disk volumes and the old conventions of /opt, /u, /usr, /usr/local
This new fangled Linux craze with all of the UI tools is feeding it. Redhat is training admins that are dependent on a given release of their enterprise software (which I am a huge fan of) but not teaching them how it works under the hood.
How about slirp? scp?
The one ray of hope seems to be a new generation hacking their bsd and linux based (iPhone/Android) phones and having fun in a somewhat embedded (but full blown) *nix environment.
In terms of navigation directories efficiently, I find that "cd -" is often forgotten (changes directory to your previous directory). I personally find it very useful, and couldn't live without it!
fg, bg, kill, Ctrl-Z, &. Learn it. Know it. Live it.
Even if they do know about job control, I've seen people look for a background job with ps, and then kill it using the PID. In most shells you can just do kill %, e.g. kill %1
Hands in my pocket
You're not giving echo an asterisk as a paratemer. You're giving the shell an asterisk, which it dutifully expands. echo (which in this case is a shell builtin, but it doesn't have to be then just echoes them back.
This isn't some echo peculiarity. It works for anything, even commands that don't normally take files, or even with files that look like switches (conversely, if you want to treat all subsequent arguments as files, not switches, most programs have a '--' switch):
$ ls
a -l b c
$ ls *
-rw-r--r-- 1 marcansoft users 0 2008-11-05 21:58 a
-rw-r--r-- 1 marcansoft users 0 2008-11-05 21:58 b
-rw-r--r-- 1 marcansoft users 0 2008-11-05 21:58 c
$ ls -- *
a -l b c
In the second example, ls sees "ls a -l b c" and takes -l as a switch instead of a filename.
One great feature of find that many people are unaware of is that you can use -exec as a test, not just as an action. For example, this is equivalent to your command above:
find . -exec grep -q {} \; -print
The "-print" action is only executed if the -exec command returns success.
You can do a lot of handy things with this. Here's a real-world example from earlier today. I wanted to change the mime-type of all the xml files in my svn repository from "application/xml" to "text/xml":
find . -name \*.xml -exec sh -c "svn propget svn:mime-type {} | grep -q application/xml" \; -exec svn propset svn:mime-type text/xml {} \;
Some more common ones I've thought of:
screen - too useful, run apps in a virtual console which you can attach, deattach and share
cd `pwd -P` - Jump into the real directory (from a linked directory).
history - use it with grep if you forgot what you did
strings - just show the printable strings from a file
tail and head - tail -f is a lifesaver
sftp - i really shouldn't need to explain this.
file - do magic stuff
Hope that's some help.
Assuming you already know the simple stuff like how to use shell quotes correctly, what you can do with ps and top, ...
Need to know the serial number of a server or bios version or many other things dmidecode is your friend.
When I pop up with my laptop to discuss with a colleague, after a while I might do on their computer:
xhost +mylaptopname
and on my laptop I do:
x2x thecomputername:0 -west
Then suddenly my mouse can go over the two computers, my keyboard works on both as well, and I can even copy-paste between the two computers. It looks like the two computers got united. In a flash, newbies get a new idea of what means unix and X ;-)
The advantage of the directory stack over "cd -" is that the directory stack always remembers where you last were. "cd -" only remembers until you change directories again.
In tcsh (I don't know other shells), you can do directory stack substitution. =0 is current directory, =1 is one up, =2 is two up, and so on.
I also use bindkeys to bind Control-G to 'dirs -v' so I can look at the directory stack with ease, even in the middle of a command.
Personally, I think directory stack commands are the least-known, but most useful feature in tcsh.
The ultimate goal of science is to unify all forces of nature to a single law that can be silk-screened onto a T-shirt.
!! in bash uses the last command you entered. $ make_me_a_sandwich What? No. $ sudo !! okay. $
ls -l | sort -n +4 -- sorts files in size order, good for finding big files in a directory
du -s * | sort -n -- similar to above, find the biggest files & subdirectories of the current dir
du | xdu -- only when you're in X, obviously. Better grain than above, with the ability to drill down into subdirectories
This shows all attached block devices (it also errors like crazy, hence the | more)
blockdev --report /dev/* | more
Redirect stderr much?
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
Xargs is much more fun with complex data processing. e.g.
Convert all PSDs to PNGs:
Parse out and sort column 2 from a semicolon delimited file:
Oh, I almost forgot about one of my favorite tricks. Count the number of items:
It even works when the list of items has oddities. e.g. I had a list where every other line was blank. So I needed to count n/2 the value. Except that one of the blank lines wouldn't copy, so I actually needed (n+1)/2.
Want to make sure your sig is under 120 characters? Type "wc -c" in, paste it into your terminal, then press CTRL-D. Instant character count.
Ah, all the fun stuff you can do with Unix tools.
Javascript + Nintendo DSi = DSiCade
DISCLAIMER: Don't run this!
I didn't think I needed to say this, but I just showed someone this and they thought it was a legitemately helpful command...
cd -
Change to your previous directory. It's great for going from dev to test environments from time to time.
My previous place of employment was a Mac shop, where I discovered the wonderful pbcopy and pbpaste commands. Why they aren't a standard part of every X windows distribution, I'll never know, but they are damned handy.
What they do is allow you to read and write from the cut-and-paste buffer from the command line. "pbpaste" will print the currently copied text to stdout, while "pbcopy" will replace the buffer with stdin.
Fortunately, there are some third-party X equivalents for this, such as xsel or xclip, which can be adapted to work in the same way.
Rougly equivalent:
pbcopy
xsel -i --clipboard
xclip -in -selection clipboard
pbpaste
xsel -o --clipboard
xclip -o -selection clipboard
grep --color
For some reason, many people are greatly surprised when they figure out that grep will highlight matches for them.
lsof is a LIFE SAVER for trying to find what's still using something in a mounted resource when trying to unmount something. For example:
/mnt/myMount
/mnt/myMount open
/dev/snd
lsof
That will list which processes have anything under
It's also useful to find who's accessing what device. For example, say you're trying to listen to an mp3 and Amarok bitches about the sound device not being available. In that case, you could do something like this (assuming you're using ALSA):
lsof
That will list what processes are accessing any of your ALSA sound devices.
ctrl+r (in bash?): reverse incremental search through history.
pushd/popd , change directory saving the old one on a stack.
sftp - i really shouldn't need to explain this.
I much prefer sshfs. Diff doesn't work so well over ftp ;)
Your
is fine for non-GNU UNIX grep.
If you have GNU grep, then
Most systems will GNU grep will also have an rgrep command which is the same as 'grep -r'.
But the find approach allows much more sophisticated searches. Such as:
find . -name \*.xml -not -user root -exec grep -l keyword {} \;
(search all .xml files that aren't owned by root)
tail and head - tail -f is a lifesaver
I use tail -F, which is the same as tail -f, but works on non-existent files. Useful when tailing log files from programs that start a new log file every time it runs. Using tail -F in this case, you can just leave tail running while you start and restart the program overwriting the log file.
There are a whole bunch of "history" tricks, to recall old commands without using the mouse.
When I started college, I studied the shell's man page until I knew them all. Some are so obscure I have forgotten them.
Generally, these involve an '!' character in some way.
Here are a few I use:
!! # run again the last command that was run
!9 # run again the command with history number 9
!v # run again the last command that started with a 'v'
!vi # run the last command that started with "vi"
!?foo? # run the last command that had the string "foo" anywhere in it
diff oldfile newfile
mv !$ !^ # same as "mv newfile oldfile"
# !$ is last arg of previous command, !^ is first arg
ls foo bar baz
rm -f !!* # same as "rm -f foo bar baz"
# !!* repeats all arguments from previous command
There are actually some baroque tricks that recall a previous command and perform a search-and-replace on it, but for anything that complicated I just recall the line and edit it. The baroque tricks would have been pretty darn cool back in the paper teletype days, though.
By the way, the Bash shell can be configured to edit command lines using vi or Emacs commands. I described how to do it in an article I wrote for Linux Journal magazine. It's the last section, "vi or Emacs Mode in the Shell".
http://www.linuxjournal.com/article/8361
Oh, not exactly a history trick, but here's something I use all the time: /tmp/files /tmp/files # edit list to include just the files I want /tmp/files`
ls -1 >
vi
rm `cat
# `cmd` inserts the standard output from cmd into the command line as if you typed it
ls -1 > /tmp/files /tmp/files :%s+.*+mv & /some/directory/path/&+ /tmp/files
vi
# edit list to include just the files I want
# now run this command:
# save file and quit vi
source
This moves the chosen files to "/some/directory/path". The breakdown of the vi command is as so:
: # invoke "ex mode" for search and replace command
% # run the following command on every line of the file
s # do a search and replace
+ # use a '+' for the command delimiter, so I won't have to backslash escape '/' chars in the path
.* # all characters on the line /some/directory/path/& # replace "foo" with "mv foo /some/directory/path/foo"
+ # end the match pattern, begin replace pattern
# & refers to the match pattern, thus all chars on the line
mv &
Takes less time to do it than to explain it!
The above is perhaps overkill if all the files are going to the same place. It's great if you want to send some files one place, some to another, because you can just edit the destinations until it looks right.
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
You only need a tiny bit of entropy to seed the pseudorandom number generator in /dev/urandom. Once it's seeded you get all the pseudorandom numbers you want. /dev/random gives you truly random numbers, and is highly dependent on the amount of entropy the system has. It will block if you run out of entropy, urandom will not.
Give me Classic Slashdot or give me death!
Along that line are pushd and popd. pushd <dir> changes to the specified directory and pushes it onto a stack of directories; popd changes to the directory at the top of the stack and removes it. There are commands for manipulating the directory stack but I don't know or use them.
With zsh, and I think with Bash as well, you can setopt AUTO_PUSHD and setopt PUSHD_SILENT and then cd behaves like pushd.
(Both of these commands, along with cd -, work in the Windows command interpreter too.)
I'd say most people don't know how to use SSH very well...
Stop typing passwords for every system: ssh-keygen, ssh-auth and ssh-add.
Transferring files both with scp/sftp and ssh user@host "cat file" > file, and the like.
Changing encryption algo for significantly improved speed, eg. -c arcfour
Enabling/disabling compression for internet/intranet. -C
An $HOME/.ssh/config file to map names to IP addresses, specify the default user names for each host, toggle compression per host, enable/disable port forwarding, keepalive, etc.:
host webserver
ForwardX11 no
ForwardAgent yes
Compression yes
hostname slashdot.org
port 2100
user cmdrtaco
And parenthesis and backticks seem to be going out of fashion in short order... Too bad, since they're quite time-saving: mkdir `date +%Y`
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
sshfs is truly pimp, but both deserve mention; AFAIK sshfs can't be used on a system where you don't have root that doesn't have FUSE installed.
For example, I did these to some of my favorite people:
banner "PORN HERE" > /dev/pts/4
echo "All files deleted." > /dev/pts/3
cat dictionary.list > /dev/pts2
--Chag
I can't live without svn. Svn is a revision control repository, usually used for source code. What makes it really powerful is that you can _easily_ have a history of everything that has changed in a file and when. On my systems, I keep /etc in svn, plus bind's zone files, plus all the non-image web content, and the "Network Documentation" folder.
Second trick, rsync. I use it to backup my home directory to another box. Very nice when you go through a hard drive/year.
Screen -x was my next pick, but somebody already mentioned it.
"echo ProtocolKeepAlives 120 >> /.ssh/config" No more dropped ssh sessions because of stupid nat boxes.
su -u Username -s .. become Username, but keep the current shell. Good for diagnosing permissions problems when the user has a /bin/false shell. (named/www-user/backup/etc).
A little awk goes a long way. Not the big-bad-I-am-a-programming-language-awk, but the smaller-friendlier extract one or two columns of text from something awk. ex. awk '{print $2}' prints the thing in the second column. Add -F the field separator tool and it gets really useful.
Better example. Here is a postfix log line.
Nov 5 16:27:19 pdc postfix/smtpd[13601]: 92B3F499C25F: client=exprod5mx254.postini.com[64.18.0.49]
Here is the awk to extract just the message id. awk -F': ' '{print $2}'
And here is the "I didn't get this message your mailserver must have eaten it" disprover. It searches the maillog for every message from or to a given address and extracts the full email transaction for that message id.
grep -i user@domain.com /var/log/maillog | grep smtpd | awk -F': ' '{print $2}' | sort -un > temp.fil && grep maillog -f temp.fil
Next trick, back ticks. `` Back ticks substitute the output of a command within a command.
Ex. Name a file after the date. echo "hi" > `date +%Y%M%d`.txt
On the subject of dates. date -d'yesterday' or 'last week' or '-4 hours' can be handy.
Last one. Loopback nat with Iptables, so you can access local hosts by their external ip. (Instead of setting up split dns.)
iptables -I POSTROUTING -s $local_network -d $local_network -j SNAT --to $lan_interface
My local network is 192.168.0.0/24 and the netfilter lan ip is 192.168.0.1, so that becomes...
iptables -t nat -A POSTROUTING -d 192.168.0.0/24 -s 192.168.0.0/24 -j SNAT --to 192.168.0.1
-ellie
I actually like less better than tail -f. If you less a file, and then hit SHIFT-F it will tail the file, but you can break out of it and scroll around and search for terms. Very handy while looking at log files.
Don't count your messages before they ACK.
or if you're a vi freak like me:
set -o vi
escape (enters command mode)
k to scroll up
/ to search
William of Ockham had no beard. The most likely explanation is that it was chewed off by squirrels every morning.
cal 9 1752
We must repeat.
Organizing them all is left as an exercise for the reader.
Cakewalk. Put the following in ~/.bash_profile or ~/.profile
For ubuntu:
SHELLID=(echo `tty` | sed 's!/!.!g')
HISTFILE=$HISTFILE$SHELLID
Logout. Log back in. Bada-bing.
Your data is corrupt: only the current sig is ever shown.
Shop as usual. And avoid panic buying.
More awesomely, if you have found something in your history with ^R or up arrow or whatever, then you can press ^O to "execute this line and put the next line in the history onto the command line". Thus:
./foo
vi foo.c
make
^Rvi^O^O^O^O^O^O^O^O^O^O^O^O^O^O
and so on.
I think you missed a '$' before the open parenthesis, it should be:
SHELLID=$(echo `tty` | sed 's!/!.!g')
Also, you could replace "echo `tty`" with simply "tty":
SHELLID=$(tty | sed 's!/!.!g')
I don't get it. You told him to give rm an absolute path, it doesn't matter where he was.
Diff works fine with ssh.
ssh $remote_system cat remotefile | diff - localfile
cat localfile | ssh $remote_system diff - remotefile
Pshaw! All 1337 sysadmins just live as root!
Yes, but all the really stupid ones do as well.
eject is a useful tool, If you have a rack of servers all alike and you need to identify one of them . Some servers have blinking lights etc. but mine had no audio nor lights. But it had a CD tray.
I simply put eject and "eject -t" in a loop and go look in the server room -- the hyperactive server is the one I was looking for.
http://slashdot.org/submission/1062723/Cheap-mobile-data-plan?art_pos=2