Slashdot Mirror
Essential UNIX Tricks and Tools?
Chris Lesner asks: "What handy UNIX tricks/tools do you use everyday? I'm asking for stuff that amazes your friends and makes you wonder how they use UNIX w/o them.
Some simple examples include: job control (with fg, bg/&, jobs, Ctrl-Z); moving login sessions between machines with Screen for vt100 and VNC for X11 and using screen and VNC to share login session b/w users for demos etc.; using find, xargs -i and
echo to build command strings which after
inspection can be piped back though bash e.g.
`find . -type f | xargs -i{} echo "cp {} {}.bak" | bash` I'm asking b/c my source for this kind of information has dried up as my UNIX skills have matured. I'm guessing other Slashdot readers have the same problem. By the way, if you think the examples I give are lame I challenge you to better them!"
There's more fun too. It completes tons of crazy stuff. I'd check it out.
-- Ken Kinder ken@_nospam_kenkinder.com http://kenkinder.com/
It's 10 PM. Do you know if you're un-American?
su root cd /; rm -rf *
:-)
This one works really nice when your getting ready to do a fresh install, since it allows to vent all that pent up frustation with your now obsolete system
-- The morphemes of your disquisition are ascertainable, but they have eschewed an ambit of transpicuous exposition.
Here are some stupid shell tricks I've found moderately useful:
- bash: HISTCONTROL=ignoreboth, zsh: setopt HIST_IGNORE_SPACE;
This means when you type a command with a space in front of it, it doesn't go in your history. Useful if you do something that you don't want others to see (eg, xv ~/.pr0n/*.jpg).
- The zsh FAQ has an entry which describes how to get your xterm title to describe all sorts of fun stuff. Here's a bash equivalent:
- PATH management:
- This next one is absolutely necessary on any Linux machine:
- Recursively do something for each subdirectory:
Here's a dumb shell script that I call "randsort" - it prints out lines from stdin in a random order:This is useful 'cause you can do something like this:
xv `ls -t ~/.pr0n/*.jpg | head -200 | randsort`
and this randomization, for some reason, "feels" better than xv's -random option. It also has a number of "legitimate" uses. For instance:
Anyway, I got lots of stupid little crap like this. I try not to get too far into the customization stuff, as it's definitely a timesink and it's very important to be able to use a "standard" unix setup. For example, my .emacs is circa 80 KB (I'm big into Lisp), but I'm extremely proficient with vi, because there are lots of situations where emacs is not available, or vi is better-suited to the job.
The lameness filter is such a piece of horseshit. I mean, are the editors deliberately trying to prevent us from discussing code and technical matters? You'll need to re-indent all the snippets above because of this.
Lameness filter fodder:
C este un limbaj de programare cu scop general ale carui caracteristici sint economia de expresie, structuri moderne de control al fluxului si de date, precum si un set bogat de operatori. C nu este un limbaj de nivel "foarte inalt", nici "mare", si nu este specializat vreunei arii particulare de aplicatii. Dar absenta in restrictii si generalitatea sa il fac mai convenabil si mai de efect pentru mai multe scopuri decit limbaje presupuse mai puternice. C a fost la inceput proiectat si implementat pe sistemul de operare UNIX pe DEC PDP11 de catre Dennis Ritchie. Sistemul de operare, compilatorul C si in mod esential, toate programele de aplicatii ale lui UNIX (inclusiv software-ul folosit pentru a pregati cartea aceasta) sint scrise in C. Compilatoare de C exista deasemenea si pe mai multe alte calculatoare, intre care IBM System/370 Honeywell 6000 si Interdata 8/32. C nu este legat de nici un hardware sau calculator anumit si e simplu de scris programe care se pot executa fara nici o modificare pe diferite calculatoare care au limbajul C implementat. Aceasta carte are drept scop sa-l ajute pe cititor sa invete sa programeze in C. Ea contine o initiere, pentru ca noii utilizatori sa poata incepe cit mai repede posibil, capitole separate pentru fiecare caracteristica majora, si un manual de referinta. Marea parte a textului nu se bazeaza atit pe expunerea de reguli si propozitii cit pe citirea, scrierea si revizuirea de exemple. In cea mai mare parte exemplele sint programe reale si sint complete si nu fragmente izolate. Toate exemplele au fost testate direct din text, care este intr-o forma citibila pe calculator. Pe linga faptul ca am aratat cum se utilizeaza efectiv limbajul, am incercat in plus, acolo unde era posibil, sa-l ilustram cu algoritmi utili si cu principii de bun stil in programare si proiectare sanatoasa. Aceasta carte nu este un manual introductiv de programare. Ea presupune anumite familiaritati cu conceptele. de baza din programare, ca variabile, instructiuni de asignare, bucle, functii. Cu toate acestea, un programator novice va fi in stare sa citeasca cartea si sa-si insuseasca limbajul, chiar daca ajutorul unui coleg cu experienta mai mare i-ar usura munca foarte mult. In experienta noastra, C s-a dovedit un limbaj placut, expresiv si adaptabil pentru o mare varietate de programe. Este usor de invatat si "se poarta bine" pe masura ce experienta in programare cu el creste. Speram ca aceasta carte va va ajuta sa-l folositi bine.
C este un limbaj de programare cu scop general ale carui caracteristici sint economia de expresie, structuri moderne de control al fluxului si de date, precum si un set bogat de operatori. C nu este un limbaj de nivel "foarte inalt", nici "mare", si nu este specializat vreunei arii particulare de aplicatii. Dar absenta in restrictii si generalitatea sa il fac mai convenabil si mai de efect pentru mai multe scopuri decit limbaje presupuse mai puternice. C a fost la inceput proiectat si implementat pe sistemul de operare UNIX pe DEC PDP11 de catre Dennis Ritchie. Sistemul de operare, compilatorul C si in mod esential, toate programele de aplicatii ale lui UNIX (inclusiv software-ul folosit pentru a pregati cartea aceasta) sint scrise in C. Compilatoare de C exista deasemenea si pe mai multe alte calculatoare, intre care IBM System/370 Honeywell 6000 si Interdata 8/32. C nu este legat de nici un hardware sau calculator anumit si e simplu de scris programe care se pot executa fara nici o modificare pe diferite calculatoare care au limbajul C implementat. Aceasta carte are drept scop sa-l ajute pe cititor sa invete sa programeze in C. Ea contine o initiere, pentru ca noii utilizatori sa poata incepe cit mai repede posibil, capitole separate pentru fiecare caracteristica majora, si un manual de referinta. Marea parte a textului nu se bazeaza atit pe expunerea de reguli si propozitii cit pe citirea, scrierea si revizuirea de exemple. In cea mai mare parte exemplele sint programe reale si sint complete si nu fragmente izolate. Toate exemplele au fost testate direct din text, care este intr-o forma citibila pe calculator. Pe linga faptul ca am aratat cum se utilizeaza efectiv limbajul, am incercat in plus, acolo unde era posibil, sa-l ilustram cu algoritmi utili si cu principii de bun stil in programare si proiectare sanatoasa. Aceasta carte nu este un manual introductiv de programare. Ea presupune anumite familiaritati cu conceptele. de baza din programare, ca variabile, instructiuni de asignare, bucle, functii. Cu toate acestea, un programator novice va fi in stare sa citeasca cartea si sa-si insuseasca limbajul, chiar daca ajutorul unui coleg cu experienta mai mare i-ar usura munca foarte mult. In experienta noastra, C s-a dovedit un limbaj placut, expresiv si adaptabil pentru o mare varietate de programe. Este usor de invatat si "se poarta bine" pe masura ce experienta in programare cu el creste. Speram ca aceasta carte va va ajuta sa-l folositi bine.
to get the top ten hogs in
Also, sometimes there are some big files, and you are only interested in the directories full of crap:
While we're on the subject, you can use this handy-dandy snippet to find the disk usage of one user in any part of the filesystem:
I believe nothing is more impressive than knowing what you're doing with the world of UNIX command line tools and standard in/out on the pipe.
I often show up Perl studs at work, creating similar functionality to their spaghetti code with a few lines of korn shell code in much less time.
Shell programming with the pipe is the most powerful form of elegant simplicity I've ever seen in any computing.
history | grep 'command'
If you're using bash or zsh with the default emacs-like bindings, you can use ctrl-r to do an interactive backwards search in your history. Just keep hitting ctrl-r to search deeper back into the history.
Also, the one thing I've noted a lot of intermediate Linux users don't know about, but find very useful upon investigation is job control. Even with good window management and multiple xterms, job control is very useful: I often have three or four jobs on each xterm. Also, a neat trick: with zsh or bash, "fg %-" goes back to the second-to-last job. You can use this to quickly suspend and switch between two jobs continuously (eg, after you've done it once, "ctrl-z, ctrl-p, enter" switches to the "other" job).
Also, if there's a long command I recently executed, I usually won't search through history, but rather use the "!cmd" thing. Eg, if you've recently run "gcc -g -Wall -O3 -blahblah ..." a few commands ago, you can do it again by just typing "!gc" without searching. I guess it's really a question of preference, but I picked this up from the older gurus who were weaned on the monstrosity which is the C shell.
The biggest single command which saves me time is 'cd -' which changes to your previous directory under bash.
It doesn't sound terribly useful, but it is... Take my word for it.
Instead of
$ ps aux | grep foo | grep -v grep
use
$ ps aux | grep [f]oo
The brackets will show up in the ps output but don't match your pattern, so your grep is automaticly excluded from your final output.
Regular expressions aren't so much either a trick or a tool exactly, but you can use them with all the "good" tools.
Get the book "Mastering Regular Expressions," by Jeffrey E. F. Friedl. http://www.oreilly.com/catalog/regex2/
Read it slowly, a couple of pages every day. I didn't understand much of what he was trying to say until I read the book the second time.
But why make up my own clever things to say... From http://www.oreilly.com/catalog/regex/desc.html "There can be certain subtle, but valuable, ways to think when you're using regular expressions, and these can be taught."
I find that books that teach you how to think about problems and solutions are few and far between, and books that do it well are almost impossible to find. This books is one of those.
Once learned regular expressions are one of those things that can profoundly effect the way you work. And once your there "you wonder how they [other people] use UNIX w/o them".
I consider this pretty basic, but I remember some junior coworkers blown away by
<tt>something | sort | uniq -c | sort -rn</tt>
On the coding level, I constantly use the regex library (with extended regular expressions) and profiling. There are a lot of places where a single well-designed RE can eliminate many lines of code, and profiling can help you ensure that you close all files you open, free all memory you malloc, etc.
For every complex problem there is an answer that is clear, simple, and wrong. -- H L Mencken
If you do a set -o vi in bash or ksh you get the vi mode. Then you can hit <Esc>/^scp<Enter> to get the last command that started with `scp', and then hit n for next, as in vi. You can also do most of the rest of your vi commands.
Karma: Marginal (mostly due to the border around the website)
your python script may crap out due to the stdin stream disappearing (i know nohup redirects stdout and stderr, but i don't think it redirects stdin)
/dev/null &
you might try nohup python myscript.py
Need a Catering Connection
After hacking around on *nix for years, everything in that book was old news to me. Well, almost everything. Out of 1000+ pages, here's the only trick I had never seen before:
If you have a directory full of files that are precious to you, run "touch ./-i" in that directory. This creates an empty file named "-i". Why do this? Because:
If you accidentally "rm *" there, the shell's default collating order while expanding the * will put the "-i" file first in the list, which will then be interpreted by rm as the interactive option, rather than a file.
(I've never used this trick, but I can see how it would be much less hassle than trying to remember to chmod u-w a precious file every time you get done working with it. Refinements on the technique include hardlinking the "-i" file to different directories instead of touch'ing it more than once, to save on inodes.)
(This also assumes that the user is already familiar with any of the half-billion methods of actually deleting a file whose name starts with '-'. This is /the/ most frequently asked FAQ in comp.unix.*.)
There. Now you know the most interesting trick in the whole book, and you don't need to spend the sixty bucks. :-)
You cannot apply a technological solution to a sociological problem. (Edwards' Law)
UGU offers a Unix tip of the day at http://www.ugu.com/sui/ugu/show?tip.today - that should keep you in tips for a while.
Why can't I moderate something "Wrong" or at least "Grossly Misinformed"?
xrdb -merge
To make all of your files and directory world-readable with one command:
.
$ chmod -R go+rwX
(the X is the key, you don't have to worry about executables vs. directories)
Need a list of numbers of letters or anything? Check out jot (on FreeBSD, maybe on others):
$ jot 3
1
2
3
$ jot -w %c 3 65
A
B
C
(This is especially useful with for loops - "for i in `jot 20`;do touch foo.$i;done" will generate foo.1 through foo.20. Extra hint: -w %02d will give you a leading zero on 0-9)
grep's -A, -B, and -C flags can be very useful. Using them, you can have grep display the lines immediately before and/or after a match.
If you have a lot of files in a directory, so many that "*" complains of the argument length, bash (and probably other shells) can let you get around this by changing your command to one using a for loop. It won't be as fast, but it won't churn for several minutes only to tell you it won't work. ;)
instead of:
$ rm *
do
$ for i in *;do rm $i;done
You can get a list of all files in your directory excluding . and .. with 'ls -A'.
One of the most annoying things about 'less', at least in Debian and probably other dists, is that it clears the screen when you exit. Ugh! You can fix this problem by setting PAGER='less -X' in your environment.