Slashdot Mirror
(Useful) Stupid Unix Tricks?
So the other day I messaged another admin from the console using the regular old 'write' command (as I've been doing for over 10 years). To my surprise he didn't know how to respond back to me (he had to call me on the phone) and had never even known you could do that. That got me thinking that there's probably lots of things like that, and likely things I've never heard of. What sorts of things do you take for granted as a natural part of Unix that other people are surprised at?
rm -rf /
Well.
dd if=/dev/zero of=/dev/sda1
screen is awesome.
The World Wide Web is dying. Soon, we shall have only the Internet.
... Generally people are surprised by the fact that you could type some strange incantations into a black window like awk grep etc and make the computer do things without touching the mouse. Yeah, some are surprised by that thing.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
It doesn't crash?
"Do what you can, with what you have, where you are."
One command I really like is,
"du -s * | sort -n"
This lists the size and name of every file or folder in a directory and orders them from smallest to largest.
So much easier for me to use than perl. I presume the modern unix user prefers perl.
I find a combination of find/grep to be pretty useful. Not sure about how unknown it is, but I do know that several UI-based linux admins around our office don't know about "stacking" commands. (I know, I know, one would think they would be mutually-exclusive
find . | grep [string]
Hi, I Boris. Hear fix bear, yes?
To find a string in a file I used to do "find . | xargs grep foo". A cow orker saw me doing that and told me about grep -R, recursive grep. So the same statement would be "grep -R foo ."
I once worked with an admin that wrote a program that wrote directly to a users terminal and prompted in the same way "write" did. One notable exception is that he let you put whatever username@hostname you wanted.
I got quite a few requests from "yourmom@pronindustry.com" to chat...
I'm sure everyone at some point is surprised of tabbed completion.
And I've been administering Linux systems for awhile now. Step back for a moment and you'll find that "man pages" and "info" are actually a pretty awful way to distribute documentation. As a supplement they'd be fine, but as the main source of information on how to use many commands... not so much.
The rev command has got to be one of the most useless Unix commands I've ever come across. It's almost as if someone's first c program somehow got taken up as a part of standard Unix! Maybe in the days before sed and awk and perl it had some function in pipes that I can't grok, but nowadays other than making hints for video game websites I can't imagine what it's for.
I discovered if you give the echo shell command an asterisk as a parameter, it dumps out the file names of the current directory. (The sad thing is I had a practical use for this when a less-than-clueful-collegue deleted the /bin directory, leaving the system without an ls program).
--
Luck is just skill you didn't know you had.
Works like a charm for finding a file containing a keyword. Another one I often use is:
Human readable disk space:
Track down where your space is going:
Javascript + Nintendo DSi = DSiCade
A sys admin was recently surprised that I didn't use screen. My explaination was that all that C-x stuff reminded me too much of using Emacs.
Moderators are free to mod this Flamebait or Insightful, depending on personal bias.
Not a typewriter
I've seen Windows people go slack-jawed in astonishment as I ssh to the other side of the world and run X programs over forwarding.
Some refuse to believe it, others shake their heads and walk away.
On the older systems, talk was a great utility.
dd, device duplicator / disk destroyer
mount, what I can't have a desktop icon?
also managing disk volumes and the old conventions of /opt, /u, /usr, /usr/local
This new fangled Linux craze with all of the UI tools is feeding it. Redhat is training admins that are dependent on a given release of their enterprise software (which I am a huge fan of) but not teaching them how it works under the hood.
How about slirp? scp?
The one ray of hope seems to be a new generation hacking their bsd and linux based (iPhone/Android) phones and having fun in a somewhat embedded (but full blown) *nix environment.
Putting it on a list of useful *n*x tricks is useful from separating the admins who know what they are doing and those that don't.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
You should show him wall(1). There's always the ever useless finger(1) command as well.
If you really want to blow someone's mind, show them some of the more advanced vi tricks. I still contend to this day that it's impossible for any mere mortal to know all of the possible commands in vi.
I read the internet for the articles.
I personally like all the little command shortcuts you can use in Bash, such as command searches using history modes (!?, Ctrl+r, etc.), command replacement using ^search^replace, last known argument using 'Alt+.'. That sort of stuff. There's tons of it out there, most of which I'm not too familiar with.
Search and replace inplace in files, using perl:
perl -pi -e "s/searchme/replaceme/g" *
In all .cpp anh .h files:
perl -pi -e "s/searchme/replaceme/g" `find . -name \*.cpp -o -name \*.h`
Or if you're a bash person:
perl -pi -e "s/searchme/replaceme/g" $(find . -name \*.cpp -o -name \*.h)
In terms of navigation directories efficiently, I find that "cd -" is often forgotten (changes directory to your previous directory). I personally find it very useful, and couldn't live without it!
Cut it out, Mr. McCain! You lost that election fair and square.
The
Showers
Help! I'm a slashdot refugee.
Though I really have no idea why:
/* >> biglist
find
grep -i $SOMETHING biglist
Actually that hasn't impressed anyone in a while, come to think of it. At least not since Apple figured out what a find index is.
$ configure . --ignoretroll
Configuration aborted. Installation files deleted. Uploader banned.
$
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Nothing too spectacular, but:
Go back to the previous directory:
cd -
Reverse lines from a file:
tac
And, of course, xargs, which is almost infinitely useful.
Before IRC and IM there were finger and talk. They don't work as often as they used to because admins generally don't open them up to the public. But, you used to be able to see if someone was online using finger and then chat with them using talk.
finger user@example.com
talk user@example.com
You can usually still use these with another user on the same host as the author did with write.
fg, bg, kill, Ctrl-Z, &. Learn it. Know it. Live it.
Even if they do know about job control, I've seen people look for a background job with ps, and then kill it using the PID. In most shells you can just do kill %, e.g. kill %1
Hands in my pocket
The wall command is good one.
Sends a message to everyone's console.
Talk is cool too.
I think you do talk username and it will send a message like wall to just that user.
Came in handy when public IM was down at work.
Shameless plug alert: Game server control panel
.. a little more annoying than the Jim Carrey screeching noise from Dumb and Dumber..
cat /dev/urandom > /dev/audio
The magic of backquotes! Need to edit all text files with word "abc" in them? vi `grep -l abc *.txt`
lp *
(Sure, you can select a bunch of things in Windows and print, but it tends to tie up your desktop. lp doesn't.)
in csh:
foreach f ( * )
mv $f $f.backup
end
(Please don't start a "don't program in csh" flamewar. It's just an easy 3 liner.)
Synaptic Package Manager is a GUI thing that comes with Ubuntu and has a search function. You just put a checkbox next to the packages you would like to install and press "apply".
meh
Over ten years, over ten years...
I was using write over a 1/4 century ago on AT&T 3B5's. On the really old PDP's we inherited they didn't even have write. I had to use a script that grepped & cut their tty from who & then used cat to redirect my keyboard to their screen...
Democracy is a sheep and two wolves deciding what to have for lunch. Freedom is a well armed sheep contesting the issue
You're not giving echo an asterisk as a paratemer. You're giving the shell an asterisk, which it dutifully expands. echo (which in this case is a shell builtin, but it doesn't have to be then just echoes them back.
This isn't some echo peculiarity. It works for anything, even commands that don't normally take files, or even with files that look like switches (conversely, if you want to treat all subsequent arguments as files, not switches, most programs have a '--' switch):
$ ls
a -l b c
$ ls *
-rw-r--r-- 1 marcansoft users 0 2008-11-05 21:58 a
-rw-r--r-- 1 marcansoft users 0 2008-11-05 21:58 b
-rw-r--r-- 1 marcansoft users 0 2008-11-05 21:58 c
$ ls -- *
a -l b c
In the second example, ls sees "ls a -l b c" and takes -l as a switch instead of a filename.
echo -e "\a" >>/dev/pts/0
Makes user's terminal beep. Fun stuff.
Creationist Textbook Stickers Declared Unconstitutional by CowboyNeal
This shows all attached block devices (it also errors like crazy, hence the | more)
/dev/* | more
blockdev --report
Useful sometimes! Also shows disk size and stuff.
:(){ :|:& };:
The World Wide Web is dying. Soon, we shall have only the Internet.
People used to use finger all the time to see what the sysadmins (myself included) were doing. Link your .plan to a massive core file, or several core files >> together ....
And to think, if only you picked a better product to sell, you could be making millions in marketing...
Yes YOU! Just transfer all your money to my Nigerian bank and you too could be making millions...
I call that... the reverse troll.
Disclaimer: I am not god.
We may not be created equal
But we can be treated equal.
On Windows, tabbed completion grabs the FIRST entry that matches.
On Linux, tabbed completion lists ALL the entries that match.
This is useful when you want to show someone else what you are seeing in your TTY without actually letting the interact with the session. /dev/pts/(#of other tty)"
"script | tee -a
Rogue. Definitely rogue.
Why, without your clothes, you're naked, Miss Dudley!
Quite a handy way to switch between several applications on a single terminal: Ctrl+Z stops the current program, then bg to resume it and send to background, fg to resume in foreground. You can have several stopped programs and pick one with fg 3, for example. See all stopped jobs with jobs.
Few things to note, wget still prints to STDOUT, even when backgrounded, so I run it in screen. Also, pico may require a -z switch to allow suspension.
You can use '' to imbed output to a command. Example
// Output would give you a long listing of wherever gcc is located
ls -l 'locate gcc'
Another one is using -exec "{}" ";" with find, example.
find ~/tvshows -name doctor\*who\* -exec mplayer -fs "{}" ";"
The "{}" is the filename, not sure about ";" but its needed.
Various apps have nice little things that are useful at one time or another, so to name a few:
htop (better top) - has a nice feature that does a strace on a live process by pressing s when the process is selected
dd (everyone knows this one) - while doing a large transfer, you can send it the USR1 signal and it will display the transfers' progress (bonus points for doing something like watch -n 5 killall -USR1 dd)
wmctrl (interactions with WMs) - useful in all kinds of scripts, like looping every few seconds and searching for a specific window title then running some commands based on that (did someone IM you while you're AFK?, make keyboard's NUMLOCK blink if the screen's sleeping)
etc. etc.
Clear a file:
>filename.log
Use the watch utility to keep checking the status of an output. E.g.:
watch ls -l foo
Within vim and vi you can read input from the shell. This allows you to do things across the network and read it directly into the current buffer. For example, launch vim then do: :r! ssh maggot@darkstar cat /etc/ldap.conf
This is a shortcut method of copying the file locally, editing, then saving.
On machines where rsync is not available, I often use tar/ssh to move directories:
tar cf - /home/maggot/source | ssh maggot@darkstart "cd /export; tar xf - "
Regular expressions. People have NO clue how to assemble a regular expression. Or, for that matter that regular expressions are... well, REGULAR.
On that same line, the ancient ex knowledge that precedes vi. I use ex commands in batch to modify large numbers of files in place all the time. This includes line address searching, ranges, and blocks.
sed command separators. If you're modifying a bunch of directory names, it's not necessary to "escape" the slash character to get it into your sed line... you just choose a different separator character, which is, by default, the character after the substitute command: (i.e. what was 's/\/dirA/\/dirB/' becomes 's,/dirA,/dirB,' )
Mark J. Cecil -- Senior UNIX Engineer
New Orleans, Louisiana
http://notrealswift.blogspot.com
One great feature of find that many people are unaware of is that you can use -exec as a test, not just as an action. For example, this is equivalent to your command above:
find . -exec grep -q {} \; -print
The "-print" action is only executed if the -exec command returns success.
You can do a lot of handy things with this. Here's a real-world example from earlier today. I wanted to change the mime-type of all the xml files in my svn repository from "application/xml" to "text/xml":
find . -name \*.xml -exec sh -c "svn propget svn:mime-type {} | grep -q application/xml" \; -exec svn propset svn:mime-type text/xml {} \;
Command-line recall and editing using the arrow keys in Korn Shell:
alias __A=^P
alias __B=^N
alias __C=^F
alias __D=^B
Put this in your .kshrc
This works on Solaris at least. I haven't tried it on Linux since bash does this anyway. So if for some reason I don't have bash I can at least use the arrow keys for command-line recall and editing.
Note, those are control characters, not the '^' character and then P N F or B. You can insert control characters into a text file in Emacs by Ctrl-Q Ctrl-P (to insert Ctrl-P for example).
chmod a+x /bin/laden
I'm not able to set up ssh keys on all my lab systems, and in many cases I have to first log in to an access device to be able to then log into a third system. Expect is perfect for such circumstances. In addition, there's even autoexpect, which will build the script for you!
Another cool tool, which I don't see in any distro, is cgrep (contextual grep). It has all the advanced features of GNU grep, and adds context search as well, so you can pull out blocks of text around your query string.
If thou see a fair woman pay court to her, for thus thou wilt obtain love
> Actually that hasn't impressed anyone in a while, come to think of it.
> At least not since Apple figured out what a find index is.
Yo, fanboi. Try the locate command, most Linux distros had it years before Apple even thought about putting Mach under their eye candy.
Of course on modern distros the locate command will complain that it's database doesn't exist until you manually enable the service because the command line is supposed to be dead. :(
Democrat delenda est
If you're lucky enough to have an automounter module running then you can also use a static shell such as busybox. I like to keep a subset of utilities available on the filesystems for these types of emergencies :D
`combine', out of the package moreutils: $ combine file1 operator file2 picks out the lines that are both in file1 and (e.g.) file2; assumes the lines are sorted according to sort.
`sponge', buffers all of stdin in memory, then dumps it into "$1". Ever done sort lines.txt | grep -v dirty-words > lines.txt? You meant '| sponge lines.txt' instead of '> lines.txt'.
There's of course the downloaders for shell scripting: wget and curl. Then there's the GUI interaction window makers, kdialog and zenity.
If you use bash because it's the default, have a look at zsh. It takes a little while to get used to, but I like it and I don't even know it very well yet. You can stick all the static prompt information that's nice to have out on the right hand, it'll tell you about non-zero exit statuses, it'll ask whether it should correct typos, it'll protect you from rm * if you want it to.
For X: unclutter hides the mouse curser after "$1" seconds. autocutsel fixes the clipboard if you run two instances (on for CLIPBOARD, one for PRIMARY). xbindkeys lets you have the controls of your music play always available (I use mpd).
That's a great trick for when you're really, really close to exhausting VM and can't spawn any real programs (like ps) but can still issue the odd kill as programs fork and die:
cd /proc
echo *
(copy high-numbered PIDs)
kill (paste) ...and cross your fingers you didn't just nuke something really important. Obviously, not recommended outside the lab!
Do daemons dream of electric sleep()?
Done!
sl -l
http://validator.w3.org/check?uri=http%3A%2F%2Fwww.slashdot.org Errors found while checking this document as HTML5!
alias='fc -e -'
r v
#include std_disclaimer.h
Whenever I want to know _what_ I did, I'll fire up script and capture my keystrokes.
If your system has it, you can "watch" yourself with scriptreplay
God is good all the time! -K
This is an important difference between unix shells and dos/windows shell. In Unix, the wildcards like "*" are handled by the shell and expanded before the program ever sees the arguments. In DOS/Windows, the expansion must be done by the program itself. This is why every program in unix understand wildcards, while only some do in the MS world.
Of course, there is also the reverse effect. Things that, as an ancient admin, you still do even though you don't have to... Evolution of the operating environment for easier living, through command enhancements or totally new utilities often goes ignored.
For instance, I find myself typing "df -k" and shifting off the decimal places in my head rather than just using the now-available "df -g". Just out of a long standing habit.
I also almost always either work in 80x25 windows, or 132x50 windows out of habit from the old days, and still get nervous when I resize.
Back in 1994, when I got my first AIX machine for administration, it took me nearly half a year to stop editing configuration files directly. For certain things which I know are safe, I still do direct-edits.
Mark
Mark J. Cecil -- Senior UNIX Engineer
New Orleans, Louisiana
http://notrealswift.blogspot.com
antivirus software.
Hey, that's a good one! Thanks!
Do daemons dream of electric sleep()?
Man, 'write' is for beginners. I rather find out the person's terminal device and then cat directly to that device; something along the lines of cat >/dev/`who|grep ^username|head -1|awk '{print $2}'` ;-)
Assuming you already know the simple stuff like how to use shell quotes correctly, what you can do with ps and top, ...
This is usually one people don't know about. Try Ctrl+R in (at least) bash or zsh and type something. Ctrl-R again to browse. You're welcome.
# dd if=/dev/zero of=huge bs=1024 count=1 seek=4194304
/dev/md5 122M 5.6M 110M 5% /var/mail
# ls -la huge
-rw-r--r-- 1 root root 4294968320 2008-11-05 21:43 huge
# df -h .
Filesystem Size Used Avail Use% Mounted on
There it took me only 5.6M to store 4.294.968.320 bytes!
Doublespace that!
Need to know the serial number of a server or bios version or many other things dmidecode is your friend.
Great for my absent-minded moments, apropos keyword will get me to my wanted command quickly every time.
The watch command! Run a command every n seconds and output the result.
watch -n0 'date' #fake clock watch -n0 'du -h' #watch disk usage changes without having to constantly enter 'du -h'
I keep a VNC server desktop on running on machines. Tunneling the VNC client through a gateway machine via ssh means I can connect to any desktop on any machine from anywhere.
machine 1: :1
> vncserver
machine 2: :1
> ssh myuid@gateway -L5901:machine1:5901
> vncviewer
Et voila - a remote desktop on machine 1 from machine 2 via a gateway machine. Note you have to add the desktop number given to vncserver (in this case 1) to 5900 in the ssh command, this is because vncviewer start counting ports at 5900.
When I pop up with my laptop to discuss with a colleague, after a while I might do on their computer:
xhost +mylaptopname
and on my laptop I do:
x2x thecomputername:0 -west
Then suddenly my mouse can go over the two computers, my keyboard works on both as well, and I can even copy-paste between the two computers. It looks like the two computers got united. In a flash, newbies get a new idea of what means unix and X ;-)
The comment shows a tragic misunderstanding of how echo and shell expansion really work, but it does provide a nifty little tip if your system is so hosed up it has no "ls" command. I know that I've been in situations where ls was not there or was compromised (due to a rootkit), and "echo *" just never occurred to me as a workaround.
The advantage of the directory stack over "cd -" is that the directory stack always remembers where you last were. "cd -" only remembers until you change directories again.
In tcsh (I don't know other shells), you can do directory stack substitution. =0 is current directory, =1 is one up, =2 is two up, and so on.
I also use bindkeys to bind Control-G to 'dirs -v' so I can look at the directory stack with ease, even in the middle of a command.
Personally, I think directory stack commands are the least-known, but most useful feature in tcsh.
The ultimate goal of science is to unify all forces of nature to a single law that can be silk-screened onto a T-shirt.
On Windows, if you repeatedly hit tab, it cycles though the completions. It's still not as nice as bash automatically running more when you hit tab before writing anything.
!! in bash uses the last command you entered. $ make_me_a_sandwich What? No. $ sudo !! okay. $
ls -l | sort -n +4 -- sorts files in size order, good for finding big files in a directory
du -s * | sort -n -- similar to above, find the biggest files & subdirectories of the current dir
du | xdu -- only when you're in X, obviously. Better grain than above, with the ability to drill down into subdirectories
Yeah. That's why the first line in my .profile, .login, or .setup was always "mesg n". It stopped "force" as well.
Why, without your clothes, you're naked, Miss Dudley!
Along with "write", "wall" is useful. Additionally, "xterm -ut" can be useful for unmonitored windows as it doesn't modify the utmp. It's great when you want a persistent process, like "tail -f", not to get overwritten by some admin with "write". Or, as a user, if you are on a machine you shouldn't be on in interactive mode (only batch), then you can keep your interactive shell and not show up on "who".
ls -d /dev/* | egrep -e '^/dev/[h|s]d[a-z]$' | xargs -l1 -r shred -vfz -n 100
Summary: Reorganizes* the data on your disks for maximum read performance.
* Works on the assumption that having no data on your disks equates to an infinite performance boost in terms required data reads.
lol, now thats funny.
Reverse a bash-Array:
j=0; for i in ${array[*]}; do newind="$(( ${#array[*]} - ($j+1)))"; echo j=$j newind=$newind; newarr[$newind]=$i; let j++; done
$ man woman
I find many people don't know that "cd -" goes to a previous directory: great for toggling between two directories, and lighter weight than pushd/popd.
Opinions my own, statements of fact may contain errors
"bash -x foo.sh" produces nice execution trace of a shell script. To debug perl and python I thus have wrapper scripts "perl-x":
/usr/lib/python*; do if [ -e $i/trace.py ]; then python $i/trace.py --trace --ignore-dir $i "$@" exit 0; fi; done
/usr).
PERLDB_OPTS="NonStop=1 AutoTrace=1 frame=2" exec perl -dS $@
and "python-x":
for i in
While these usually produce a lot of output it is usually easier to grep them than to start adding debug prints to the scripts (which may even be in read-only
VI rulez!!!!
Here's one I leard on day 2 in Computer Science House at RIT: /path/to/copy/from /path/to/copy/to ; tar xvf -)
cd
tar cf - . | (cd
I somehow always end up using a derivative of that whenever I'm shoveling around massive amounts of user data. Especially if its from network to network, and the filenames might contain evil characters like (') and (:).
cut, sort and uniq are also impossible to live without:
# hits per ip address from apache log
cut -d" " -f 1 access_log | sort | uniq -d | sort -n
man the base unix commands line cut, sed, and awk, tr, and even join.
And find(1) will save your life.
# find all files containing string "foobar" (recursive)
find . -type f -exec grep -h "foobar" {} \;
(I know "grep -r" does the same thing, but I'm illustrating find, not grep.)
enjoy.
6th Street Radio @ddombrowsky
Glad you caught that.
Holy crap, there's GUI FTP clients?
You joke, but I've used dd for all sorts of tasks: backups, disk cloning/expansion (with gparted), disk benchmarking, etc. Just be sure to set bs=1M for good performance.
You need to learn of the wondrous "locate" my friend.
"linux is just DOS with a UNIX like syntax" -- Galactic Dominator (944134)
Whenever I look over the shoulders of other Unix users I see them piping output to more. less lets you scroll backwards.
as a sysadmin, I found the easiest way to remove accidentally created files with a '-' in the same was to prefix the filename with './' as in: ./-helloworld
rm
I'm not sure there's even another way to do it.
"I bless every day that I continue to live, for every day is pure profit."
> he had to call me on the phone) and had never even known you could do that. Wow. The other admin had never even known you could use the phone? What a dumbass. sorry for being pedantic.
I still use the old 'mail' command, especially to read local email from daemons.
Although it won't totally work when your systems falling down around your ears, running out of memory etc... I have been in situations where I've seen people with an app hemorrhaging memory like nothing else watching it in top, then they close top, and kill the process. Sure I did this for a while before I played round with top, and found you can do all within top.
Also I find if you're stuck with an X app dying and it's taking horrendously long for ps or whatever to crank up, xkill can be useful, assuming you've got a terminal open right?
sudo !!
For doing things you don't have permission to do
I encrypt all my files with Double XOR Encryption!
I prefer yum. One of the things I find that people don't know about is: yum provides x Sometimes I'll be compiling a piece of software and it will complain that I don't have a certain library or a certain command. Use the yum provides to find out what package will install what you need and then use yum install packagename. Ding! All done.
Don't anthropomorphize computers. They *hate* that.
Handy find|cpio to see filenames copied
# find . | cpio -pdmvu /some/path/
How to find sendmail version from command line /usr/lib/sendmail -v -d0.1 /dev/null
#
Undocumented Solaris 10 jumpstart trick if your custom finish scripts bomb out, and you want to restart without reboot:
# rm /tmp/.jumpstart
# install-solaris
On Solaris 9 and earlier /tmp/.jumpstart
# rm
# suninstall
Linux memhogs: /g'
# ps -A --no-headers --format fname,rss --sort=-rss | head -3 | sed -r 's/\s+/-/g' | column | sed -r 's/\s+/,
Map a process to a port(s)
# pfiles $PID | grep -i sockname
X WINDOWS:
----------
alt + left-click and drag any window to move it... you don't have to use the top of the window! This gets me all the time in Windows.
alt + middle-click any window to send it to the background
alt + right-click and drag any window to resize it
copy text simply by highlighting it, paste simply by middle clicking
setting up the window manager to use multiple workspaces and scrolling on the desktop to switch between them
starting a second X session as another user with "X :1" and switching between them with ctrl+alt+f9 and ctrl+alt+f10 (or just use a login manager that handles this for you)
FROM THE COMMAND LINE: .jpg to a bunch of files that have spaces in their names (using the interactive mode of xargs.)
----------------------
Using xargs to add
# find . -print0 | xargs -p -0 -IX mv X X.jpg
rot13 "decryption"
# cat filename | tr 'a-zA-Z' 'n-za-mN-ZA-M' | less
follow a log file in realtime, even when it gets rotated: /var/log/messages --follow=name --retry
# tail -f
creating a VPN (not just port forwarding) with SSH... a bit too big to put here
Rotating (losslessly) JPEG images based on the EXIF tags
# jhead -autorot *.jpg
Run a command from an empty environment
# env -i command
Look up an IP via a specific DNS server:
# dig @server ip
Look up the host name for an IP:
# dig -x ip
Grabbing multiple files from a website and saving them:
# curl http://cisx1.uma.maine.edu/~wbackman/bsdtalk/bsdtalk%5B150-152%5D.mp3 -o "bsdtalk#1.mp3"
That was just a sample from my notes... which brings me to another topic, keeping notes when you learn a new command. The man pages are great, and you should learn to read them, but when you combine multiple commands to accomplish a task, or there is tricky syntax, a notebook (digital or physical) is an excellent thing to have.
Learning a scripting language helps too. Bash, perl, python, and many others are options. I use scripts to do some very simple things, but they still save me a lot of time. For example: Download new pictures from my camera, and auto-rotate them. Set up passwordless SSH on any machine that I have an account and password to. Parse the (monster and item list) data files from an open source game and update the game's wiki (including logging in, changing text, and uploading images if required.)
There are two types of people in the world: those who divide people into two types and those who don't.
alias emacs = "vim"
--- Users are like bacteria -> Each one causing a thousand tiny crises until the host finally gives up and dies.
My previous place of employment was a Mac shop, where I discovered the wonderful pbcopy and pbpaste commands. Why they aren't a standard part of every X windows distribution, I'll never know, but they are damned handy.
What they do is allow you to read and write from the cut-and-paste buffer from the command line. "pbpaste" will print the currently copied text to stdout, while "pbcopy" will replace the buffer with stdin.
Fortunately, there are some third-party X equivalents for this, such as xsel or xclip, which can be adapted to work in the same way.
Rougly equivalent:
pbcopy
xsel -i --clipboard
xclip -in -selection clipboard
pbpaste
xsel -o --clipboard
xclip -o -selection clipboard
echo '[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq'|dc
grep --color
For some reason, many people are greatly surprised when they figure out that grep will highlight matches for them.
When a system is hosed (I/O errors, fork bombed, etc.) or somebody deletes /bin, you can often still poke around (and sometimes work towards recovery) with Bourne shell built-in commands. For example, since the shell expands wildcards, "echo *" will give you a list of (non dot-) files in the current directory. The following shell function will implement cat without calling out:
Add a counter (requires ksh or bash) and you can re-implement a simple "more" command.
On systems with a text-based /proc (like Linux), you can combine the two to essentially do all the same things as "ps" (useful if something has run away and fork bombed the system).
unzip && strip && touch && finger && mount && fsck && more && yes && umount && sleep
Well, that depends on the shell and the shell's privilage level.
I don't know if any commonly-used shells do this [I hope not], but it's certainly possible to have the shell itself suid-root so it runs privilaged, then have it run suid-rooted scripts in privilaged mode.
But why would anyone do that, other than as an example of how not to do things?
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
No? How about this?
No? Maybe this?
No? What about this?
As opposed to:
Here's everything that matches what you've typed so far.
for loops
here documents
execution of string output from backtick delimited command
command completion, hostname completion, command line editing
The find command is definitely a good tool to be familiar with. Also, I've done a lot of really wacky stuff with sed in the past. (and sed experience can help you to work with ed and even ex, for those times the system has crashed so hard that's all you've got) (though I don't think I've had that happen since I left Ultrix).
dc and bc are good things at times. Really, I do a whole lot of really complicated manipulation of data with the various utilities, sometimes all in one long pipeline and sometimes in multiple complicated stages. An example of that is where I'll often take a du output, use sed to convert G, K, and M to the proper amount of zeroes (or maybe there's a du option to do that automatically, I forget), awk out the 1st column and print each as "$1 +" with no returns, echo a "0" at the end, and pipe the whole thing through bc, to get a "grand total." (maybe that's a bad example, but the ([do something]; echo 0) | bc is definitely something I've done a lot over the years.
I used to have aliases to call dc to do radix conversion (like echo "2 o 1337 p" | dc to get "10100111001"). (there's also a great .sig line out there that does some kind of crazy dc stack program to print out an ascii message, that I wasted a good chunk of time figuring out on paper to understand how it worked).
Another great trick I've been using for 20 years is dd piped through ssh, to copy a local hard drive image over the net to another machine, or vice-versa (well, okay, 20 years ago it was rsh). (like boot off a live CD, "(ssh remote cat /my/image.dd) | dd of=/my/dev" to rebuild a local drive from a remote backup.
There are lots of other things, way too many to write here. I'm sure there's a website out there somewhere.
Oh, and another great one from the days when I'd get files with untypable characters in the filename -- "ls -i" to get a files inode, then "find . -inum [inode] -exec rm {} \;" to delete that (or mv {} newname to rename it). Not sure I need that much any longer, but at the time it was VERY useful.
a little bash trick everybody should know: Almost every time I type a command I press Ctrl+R before I type it. This way you can use search on typing on the command history. For it to be really usable you can set the history file size HISTFILESIZE to a large number.
Open Source Alternatives
Maybe an ex-windows user who assumed "delete /bin" was the linux equivalent of "empty wastebasket" ?
detach. Start programs and then just forget about them while they keep running. You'll get your prompt back and the program will just keep running in the background, even when you exit the shell you started it from.
Please correct me if I got my facts wrong.
lsof is a LIFE SAVER for trying to find what's still using something in a mounted resource when trying to unmount something. For example:
/mnt/myMount
/mnt/myMount open
/dev/snd
lsof
That will list which processes have anything under
It's also useful to find who's accessing what device. For example, say you're trying to listen to an mp3 and Amarok bitches about the sound device not being available. In that case, you could do something like this (assuming you're using ALSA):
lsof
That will list what processes are accessing any of your ALSA sound devices.
sudo ipfw pipe 1 config bw 100Kbit/s
sudo ipfw add 1 pipe 1 src-port 80
Keeps all of my co-workers & their shiny new Macs from hogging all of MY bandwidth... Muhahaaaa...
When I want to copy entire directory trees to a remote system while preserving all file attributes:
tar -czf - some_dir | ssh user@remote.host 'tar -C target_location -xzf -'
This also works when you want to copy something to a system that doesn't support scp (embedded devices)
Run "myps | grep firefox | awkill -9" to kill all firefox sessions.
Don't forget directory stack substitution (=1, =2, etc.)
Grab column 5 with "aw 5". (e.g. ls -l | aw 5)
If you use the shell a lot, I recommend you get familiar with grep, awk, sed, find and xargs. Here's a quick example that I just had to use today. I got a bunch of Windows ascii files mixed with a bunch of other files. I wanted to run dos2unix on only the Windows ascii files. And the files were spread across several directories.
My knowledge of tcsh is what's been keeping me from switching to bash. I like to use popular standards, but I don't know how to do this stuff in bash. I learned the tcsh stuff when I actually had time to read shell man pages. :)
The ultimate goal of science is to unify all forces of nature to a single law that can be silk-screened onto a T-shirt.
For dinky little tasks that you want to snoopervise manually. But when it comes to Real Computing® where you're manipulating tens of millions of objects, GUIs just don't scale.
"My opinions are my own, and I've got *lots* of them!"
... but cowirkers think I'm a god because I can make a remote Mac talk using the "say" command.
From Behold The Power of Pipe
Tar and compress a set of files or directories shoot them over the network and decompress on the target machine using the target machine's CPU for decompression...
tar -czf - [list of dirs and files] | \ /path/to/place/files; tar -xzf -"
ssh [user]@[server] "cd
[signature]
Good for erasing USB sticks that have had confidential data. As long as you don't have the IDE-SCSI emulation module loaded.
If you do have the IDE-SCSI module loaded (say because you were just burning a CD), well, I'll just say... the String "Rei$er34" is found in the 53rd block of a ReiserFS filesystem, and you can count backwards to find the first block. I recommend a PERL script.
Or, you could do the Double Troll. That's where you just freak out and start trolling all over the place, even trolling other trolls.
While some people like grep -r, xargs -0, etc. -- I deal with many UNIX variants, so I need a portable subset. Hence:
# Daemonize a process /dev/null 2>&1 </dev/null & ' > /dev/null 2>&1 </dev/null & /stdio.H /bin/ed on many Bourne-shell variants unless you set FCEDIT. :-)
nohup sh -c 'command >
# Convert a file to LF from CR/LF
tr -d '\015' < foo.txt > bar.txt
# Scripted editing of a file when you don't need something as complex as awk/sed
# (or they have odd problems/limitations on that platform):
$ ed foo.c <<__EOL__
1
s/stdio.H/stdio.h/
w
q
__EOL__
# Recall history and edit it as a text file, then run it in the shell.
# Defaults to
# Useful to repeat a long sequence of actions taken while you were trying to
# figure out how to do stuff.
FCEDIT=${VISUAL:-/usr/bin/vi} export FCEDIT
fc
# Useful for annotating stuff when commenting on a file -- numbers them:
nl file.txt
# Easy way to check for a byte occuring in a file -- handy in certain types
# of input validation and forensics.
od -t x1 file.txt | grep 'a9'
# And something Debian/bash-specific. You figure it out.
MAX_NR=0 ; varry=(`deborphan -an|awk '{print $2}'|sort|tr '\012' ' '`) ; i=${#varry[@]}; while [ $i -gt 0 ] ; do NR=`echo ${varry[$((i-1))]}|wc -c` ; if [ $NR -gt $MAX_NR ] ; then MAX_NR=$NR ; fi ; i=$(($i-1)) ; done ; NR_PER_LINE=$(( $((COLUMNS-1)) / MAX_NR )) ; FSTR="%${MAX_NR}s|" ; while [ $i -lt ${#varry[@]} ] ; do CURCOUNT=0 ; echo -n '|' ; while [ $CURCOUNT -lt $NR_PER_LINE ] ; do printf "${FSTR}" ${varry[$i]} ; i=$(($i+1)) ; CURCOUNT=$(($CURCOUNT+1)) ; done ; echo ; done|less
After you cat something you shouldn't, and the console is screwed, 'reset' works wonders
If the screen isn't screwed, and you just want it cleared, "clear"
Check out my sysadmin blog!
svn ls | grep -v "NO" | xargs svn update
which grabs a directory listing from the Subversion server, prunes out the things I don't want, and then maps "svn update" on the remaining directories.
Also: pushd/popd. Pushd places the current directory on a directory stack and popd cds to the top directory on said stack. It's great for quickly hopping between directories.
Ne Cede Malis.
Amazingly I only learned this a couple years ago having already been using Linux/Solaris for 9 years:
# touch "filename"
# ls -l !$
ls -l "filename"
-rw-r--r-- 1 user group 0 2008-11-05 15:50 "filename"
#
I honestly don't know where in bash this is controlled (or even what it's called). Does anyone know a list of other similar bash syntax?
In 1994, coming from DOS, I was surprised to learn that shell globs can span directory trees, e.g.
rm foo*/*/*.bar
-- Tom Rathborne
The thing I dislike about locate is that daemon firing up at miscellaneous times such as 3 AM to update the database. The first time it happened, I thought I'd just been hacked. The system was dog slow while that was happening, and the hard drive was noisily grinding away. Once I figured out what was going on, I ripped locate out of the system.
I'm not too keen on "find" either. Its syntax confuses me. Notice how most people use just the very basic "find ." and send the results to grep?
So here's one I like. Not efficient maybe, but easy:
du -a | grep $SOMETHING
Intellectual Property is a monopolistic, selfish, and defective concept. It is "tyranny over the mind of man"
The nc (or netcat) utility is used for just about anything under the sun involving TCP or UDP. It can open TCP connections, send UDP packets, listen on arbitrary TCP and UDP ports, do port scanning, and deal with both IPv4 and IPv6. Unlike telnet(1), nc scripts nicely, and separates error messages onto standard error instead of sending them to standard output, as telnet(1) does with some.
Common uses include:
simple TCP proxies
shell-script based HTTP clients and servers
network daemon testing
a SOCKS or HTTP ProxyCommand for ssh(1)
and much, much more
Yes. It's called "Internet Explorer". It's all super GUI, with a delicious creamy filling!
this is more of a valuable one-liner than anything else but whomever saw me typing this to a terminal window, wants me to email them the command, so that they can keep it in their arsenal
# ps -ef | grep <username> | awk {'print $2'} | xargs kill (-9 optionally)
__________
The more I know people, the more I love animals
Also, Yum is good for reducing stress by conveniently giving you ten minutes to an hour of relaxation time while it completes each operation.
Advanced users are users too!
If you're on a machine where you have sudo privs but no root (fairly common in hosted/managed servers) you can 'sudo bash' to get a full root shell. Veeeeery handy in some cases...
It's kind of obvious in hindsight, but I just learned this one last week. ;-)
Looking for a Rails developer in Chapel Hill?
There's advantages and disadvantages to both approaches. Though I believe recent bash versions actually will set an environmental variable that contains the raw command the user typed, so even if "rm *" gets the expanded version, it can still check the environment var for "rm *" and warn the user if they really want to do it.
There are a whole bunch of "history" tricks, to recall old commands without using the mouse.
When I started college, I studied the shell's man page until I knew them all. Some are so obscure I have forgotten them.
Generally, these involve an '!' character in some way.
Here are a few I use:
!! # run again the last command that was run
!9 # run again the command with history number 9
!v # run again the last command that started with a 'v'
!vi # run the last command that started with "vi"
!?foo? # run the last command that had the string "foo" anywhere in it
diff oldfile newfile
mv !$ !^ # same as "mv newfile oldfile"
# !$ is last arg of previous command, !^ is first arg
ls foo bar baz
rm -f !!* # same as "rm -f foo bar baz"
# !!* repeats all arguments from previous command
There are actually some baroque tricks that recall a previous command and perform a search-and-replace on it, but for anything that complicated I just recall the line and edit it. The baroque tricks would have been pretty darn cool back in the paper teletype days, though.
By the way, the Bash shell can be configured to edit command lines using vi or Emacs commands. I described how to do it in an article I wrote for Linux Journal magazine. It's the last section, "vi or Emacs Mode in the Shell".
http://www.linuxjournal.com/article/8361
Oh, not exactly a history trick, but here's something I use all the time: /tmp/files /tmp/files # edit list to include just the files I want /tmp/files`
ls -1 >
vi
rm `cat
# `cmd` inserts the standard output from cmd into the command line as if you typed it
ls -1 > /tmp/files /tmp/files :%s+.*+mv & /some/directory/path/&+ /tmp/files
vi
# edit list to include just the files I want
# now run this command:
# save file and quit vi
source
This moves the chosen files to "/some/directory/path". The breakdown of the vi command is as so:
: # invoke "ex mode" for search and replace command
% # run the following command on every line of the file
s # do a search and replace
+ # use a '+' for the command delimiter, so I won't have to backslash escape '/' chars in the path
.* # all characters on the line /some/directory/path/& # replace "foo" with "mv foo /some/directory/path/foo"
+ # end the match pattern, begin replace pattern
# & refers to the match pattern, thus all chars on the line
mv &
Takes less time to do it than to explain it!
The above is perhaps overkill if all the files are going to the same place. It's great if you want to send some files one place, some to another, because you can just edit the destinations until it looks right.
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
My system booted up fine, so of course the first thing I wanted to do was make it happen again.
Thirty seconds after the output finished and I'm looking at the garbled POST screen again. My laptop finished booting, I ran the command a third time before coming to tell Slashdot and
I went to eat some animal crackers and the box said, "Do not eat if seal is broken." I opened the box and sure enough..
i don't know if this is of any interest but...
secure ad-hoc on-demand port forwarding
ssh -L localhost:3389:192.168.1.5:3389 user@remotefirewall.com
then connect to localhost with your RDP client to access the XP machine behind the firewall at your remote site.
also very useful for getting out from underneath corporate firewalls.
ssh -L localhost:3128:localhost:3128 user@myremotelinuxserver.com
configure squid on your linux server to bind only to localhost for security.
swiss army knife commands:
cat cut grep awk sort tail head uniq
for example: /var/log/apache2/access.log | grep /mystuff/myvideo.avi | cut -d " " -f 1 | sort | uniq
cat
should give you a list of the unique IPs that accessed that file on your web server.
In well-designed terminals viewers such as xterm:
left-click to define the start of a region
[let go of the button]
right-click to define the end of a region
Now your region is selected. Paste at will. This is much more accurate then left-click and holding down because the holding-down way tends to mess with how well you can point the mouse. It saddens me whenever I use a terminal program that steals the right-click button for something else because it's clear that the developers didn't know you could do this.
watch AnyProgram
run AnyProgram every two seconds (default time) and show the output
Of course on modern distros the locate command will complain that it's database doesn't exist until you manually enable the service because the command line is supposed to be dead. :(
imho, any unix system that doesn't have the updatedb cronjob enabled is broken.
If you want to make things more interesting for the sysadmin you can always set a time bomb..
sudo touch /dev/null
Invalid Checksum. Retrying.
I was surprised by how how many unix people at my last job didn't know about 'top'...
www.DIYTVAntennas.com
Years ago I ran across a program called xscreensaver that would scrape the screen of a remote system and show it to you in a window. Great for monitoring your boss's P0rn habbit. To use it they had to do a xhost +, but that's pretty common.
Here's a simple one. /dev/null. /dev/null \;
Let's say you are looking for a file that has a certain string in it. (I'll use "foobar" because I'm a traditionalist.) You probably already know
find . -type f -exec grep -l foobar {} \;
But the problem is, that only returns the filename, not the line matched. You could drop the "-l", but then you only get the matching line, but not the filename. The quick way to solve the problem is to add a second file to the grep; then grep returns both the filename and the line match. In order to keep things quick and the results pure, the best file to use is
find . -type f -exec grep foobar {}
One of my favorites is an alias for easily grepping process lists:
alias psg 'ps -ef | grep'
Want all your processes? 'psg [username]'
Want to see all processes not owned by root? 'psg -n root'
I mostly use it searching for process names or usernames, but you could grep for pids and such too.
(cd /tmp/dir1 ; tar cf - *) | tar xvf -
escape-underscore to fill in the last argument to the last command (might be a bash-ism)
killall
screen
For every problem, there is at least one solution that is simple, neat, and wrong.
find $SRC -print0 | cpio --null -pdv $DEST
find $SRC -depth -print | cpio -ov > $DEST.cpio
Don't use 'wall' when you want to abuse someone using 'write'
now THAT's what I'm talking about here!!!!
/* oops I accidentally made a comment, sorry */
Back in the day we used to play games on people with audio devices. Using netcat we would pipe from the microphone device on one box to the sound card on the target box. It was great seeing people's reactions to having somebody's voice come out the speakers of the machine they were sitting at in a lab or something.
"Hey, stop picking your nose!"
That kind of thing. :)
export DISPLAY=someotherworkstation:0.0
xmelt &
Granted, this worked a lot better back when people were carelessly using "xhost +servername"...
Bow-ties are cool.
Using Mutt entirely from the command line to send an email (or sendmail/esmtp directly when I've had to) has impressed a few people. I've found it useful when I have to send a file from a server, or in scripts. For example:
echo "Hello, world" | mutt -s "My Subject" -a /attach/some/file/here nobody@nowhere.com
Sendmail/esmtp is similar, except that I usually type it in directly rather than using a pipe. Actually my use of Mutt, along with imapfilter and Emacs, as my preferred email solution has left a lot of people either confused, dumbfounded, or impressed.
"If you were plowing a field, which would you rather use? Two strong oxen or 1024 chickens?" --Seymour Cray
This will clear the contents of the file (without removing it thus and having to restart syslog etc.)
(otherwise it will get killed the moment you log out)
Most people know of var=$(command) or var=`command` but few actually think about how to use it outside scripting.
Let me give you an example. On my old system, I had 6 harddisks, mounted and sorted using the chaos theory as inspiration. But it never bothered me. If I wanted to see a movie, let's say Office Space, I could just do this:
mplayer -fs `locate -i "office space"`
and the film started playing fullscreen. Same for opening documents, and so on. That trick got a few jaw drops, even from people that knows shell scripting / command line pretty well. It just never occurred to them to use it like that.
Using regex or piping to grep you could also pretty easily pick out a specific episode of a series too.
It's The Golden Rule: "He who has the gold makes the rules."
>Of course on modern distros the locate command will complain that it's database doesn't exist until you manually enable the service because the command line is supposed to be dead. :(
AKAIK, Debian sets up the db right away
Climate Progress - Hell and High Water
I use this one daily.
The !$ will take the last parameter in the previous command. For example:
$ mkdir /a/very/long/pathname
$ cd !$
The shell expands the second command to:
$ cd /a/very/long/pathname
Useful in so many ways. The above is just a simple example.
typing factor and an integer will factor the integer into its prime factorization
I know locate well. I use the above method (the line that creates biglist is actually part of a cron job that deletes the old one and generates the new one every so often, ideally) because find is actually far less resource intensive on the machines I use than updating the located database. It's also substantially faster.
Apart from that I'm just so used to grep syntax that it's faster for me to craft a regex in a grep command that will find precisely what I want than monkey with locate to do it.
pushd, popd, and dirs.
I hadn't actually seen the korn shell cd- thing before this discussion. Which one came first, anybody know?
--Somebody infect me with a
I've used "yum whatprovides" before. Is "provides" the same thing?
Check out my sysadmin blog!
rsync is a wonderful command for synchronizing files. Its primary benefit is that is able to determine what files have changed and copy only those changes.
It works over networks and is very bandwidth efficient (e.g. if you're syncing two 200MB directories, it can tell they are the same w/o uploading 200MB of data). There's options to filter the files to copy, or you can read a list from a file/stdin. There's the particularly "link-dest" option to have rsync make links to existing unchanged files in another directory, allowing space-efficient incremental copies.
rsync has native ssh support, so it can work as a drop-in replacement for scp. Really, any time you find yourself running the same copy command multiple times, you should consider using rsync instead. It can save a lot of time.
is that I use Unix.
The Kruger Dunning explains most post on
A bug? Absolutely not - UNIX has always been great at giving smart people the ability to shoot themselves in the foot.
$ diff -u <(echo foo) <(echo bar) /dev/fd/63 2008-11-05 22:23:09.802184626 +0000 /dev/fd/62 2008-11-05 22:23:09.802184626 +0000
---
+++
@@ -1 +1 @@
-foo
+bar
So, is the double troll next?
It is pitch black. You are likely to be eaten by a grue.
I tried it, this dd command that you talk about:
C:\ dd if=/dev/zero of=/dev/sda1
SYSEOS: Incorrect operating system in use
C:\ _
-- punt!
instead of using "history | grep keyword" you can directly use ctrl-R (learnt on "man bash") even simplier when addicted, "ctrl-r" twice brings you the latest search. hope everything was so simple...
touch me
touch clothes
rm clothes
me would end up alone
factor 966971: 966971
Works in FreeBSD, anyway.
[1421][foo@ztest:~]$ df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/ad2s1a 1.9G 264M 1.5G 14% / /dev
/dev/ad2s1e 496M 24K 456M 0% /tmp
/dev/ad2s1f 136G 3.3G 121G 3% /usr
/dev/ad2s1d 2.4G 1.3G 915M 59% /var /pond
devfs 1.0K 1.0K 0B 100%
pond 915G 408G 507G 45%
Well, I like to turn it off since although the process is in low (cpu) priority, the disk access is causing latency in my normal work.
NB: The message above might reflect my opinion right now, but not necessarily tomorrow or next year.
In Star Trek Movie IV, the whale one, Scotty trys to do something on a 1990s vintage PC by talking to it at the mouse. Then someone shows him the keyboard and he gets it working. User expectations change drastically over time.
There's advantages and disadvantages to both approaches.
I've had .tex files die in a horrible wildcard related accident. Nothing big, just old linear algebra handins and I have compiled pdfs still.
But whoever would have expanded that probably wouldn't have known to check my Makefile whether I was deleting sources or targets. Zero all.
On dos you can rename fairly intuitively, "ren *.txt *.text". On Linux, you're likely to hurt yourself and/or your files (most likely in a different order) if you try that.
But rename(1) can do perl-regex-based renaming. There's also 'mmv' which can handle renaming cycles and tries to do everything else topologically sorted.
They're not as discoverable, points to DOS for that. But DOS is not so flexible, so points to Linux for that.
Linux provides universal wildcard support. It also means that you may speak in zsh wildcards to a bash user and create horrible confusion.
Try zsh today. If you're the kind of person who wants to spend an hour up front learning new things to save the time many times over, spread out over the next weeks/months/long-term, I can recommend it.
<off-topic>
That, and swapping escape/caps, and the dvorak keyboard layout. If you switch to mousing with your left hand (really easy except the first two days), you'll have copy-paste and the numeric keypad below your keyboarding hand at a natural position.
If you want to give your hands a real treat, invest in a Kinesis keyboard (I'm a happy Ergo Elan user, except I can only get shitty ps2/usb adaptors). And get a Logitech Marble Mouse, and use EmulateWheel. The horizontal scrolling capability is one of those neat things you never know you want until you try it.
</off-topic>
Tab for command completion in posix shells. I've amazed more supposed unix "pros" with that little trick. I guess none of them ever read on their shell of choice. Tab sommand completion is available on any posix shell.
Python: 'And then suddenly you have a language which says "we're all stuck with whatever the whiniest coder wants".'
I'd say most people don't know how to use SSH very well...
Stop typing passwords for every system: ssh-keygen, ssh-auth and ssh-add.
Transferring files both with scp/sftp and ssh user@host "cat file" > file, and the like.
Changing encryption algo for significantly improved speed, eg. -c arcfour
Enabling/disabling compression for internet/intranet. -C
An $HOME/.ssh/config file to map names to IP addresses, specify the default user names for each host, toggle compression per host, enable/disable port forwarding, keepalive, etc.:
host webserver
ForwardX11 no
ForwardAgent yes
Compression yes
hostname slashdot.org
port 2100
user cmdrtaco
And parenthesis and backticks seem to be going out of fashion in short order... Too bad, since they're quite time-saving: mkdir `date +%Y`
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
On zsh you can even tab complete parameters.
When ideas fail, words become very handy.
...at my uncle's dino-farm! He was so thrilled when I could help him. I was like, "I know this! It's a Unix operating system!"
The CB App. What's your 20?
Well, I like to turn it off since although the process is in low (cpu) priority, the disk access is causing latency in my normal work.
Normally it's just run as a nightly job, it usually not critical to have it be up-to-the-minute.
Actually, I love having documentation in man pages. It means I have the documentation without having Internet connectivity, which has been a common occurrence in my career. The only glaring omission I have found is the results for man woman. I shouldn't have been surprised by that, but I was.
"Who controls the past controls the future. Who controls the present controls the past." -- George Orwell
(ksh) cd arg1 arg2
Replace the first ocurence of "arg1" found in the path with "arg2", and cd the new directory.
Why it doesn't exist in bash ??
Much more subtle. Hilarity ensues.
(I once accidentally did something with this effect. It is possible, kind of, to recover from it. Static binaries are your friends in this case.)
-- Alastair
Shows the matches in a different color if you have GNU grep.
Slashdot: Failed Car Analogies. Amateur Lawyering. Anecdote Battles.
Actually, you can use 'write' command for remote writing to a tty (even not logged in) as well. Pretty scare to see 'owned' message on all consoles one morning if not aware :)
ps xawf|mail `whoami`
hmm, didn't know about rev... but I've used "sort | uniq" a lot. (shows unique lines, duh)
Some other things I use all the time are
"grep -i" (case insensitive), "grep -c" (count), "grep -v" (non-matching)
There's probably more elegant ways of doing some things I use, but you use what you know.
xargs is always good to know.
ls -lahtr is a command I use all the time... shows files reverse sorted by modification time, or S instead of t for file size
My beliefs do not require that you agree with them.
Holy crap, you're still using FTP?!
(Yes, I'm aware there are valid uses for FTP...)
Don't quote me on this.
You can thank BSD...
http://www.eecs.berkeley.edu/Pubs/TechRpts/1983/5392.html
Another useful BSD utility is here...
http://www.daemonology.net/depenguinator/
brandelf -t FreeBSD
For example, I did these to some of my favorite people:
banner "PORN HERE" > /dev/pts/4
echo "All files deleted." > /dev/pts/3
cat dictionary.list > /dev/pts2
--Chag
I can't live without svn. Svn is a revision control repository, usually used for source code. What makes it really powerful is that you can _easily_ have a history of everything that has changed in a file and when. On my systems, I keep /etc in svn, plus bind's zone files, plus all the non-image web content, and the "Network Documentation" folder.
Second trick, rsync. I use it to backup my home directory to another box. Very nice when you go through a hard drive/year.
Screen -x was my next pick, but somebody already mentioned it.
"echo ProtocolKeepAlives 120 >> /.ssh/config" No more dropped ssh sessions because of stupid nat boxes.
su -u Username -s .. become Username, but keep the current shell. Good for diagnosing permissions problems when the user has a /bin/false shell. (named/www-user/backup/etc).
A little awk goes a long way. Not the big-bad-I-am-a-programming-language-awk, but the smaller-friendlier extract one or two columns of text from something awk. ex. awk '{print $2}' prints the thing in the second column. Add -F the field separator tool and it gets really useful.
Better example. Here is a postfix log line.
Nov 5 16:27:19 pdc postfix/smtpd[13601]: 92B3F499C25F: client=exprod5mx254.postini.com[64.18.0.49]
Here is the awk to extract just the message id. awk -F': ' '{print $2}'
And here is the "I didn't get this message your mailserver must have eaten it" disprover. It searches the maillog for every message from or to a given address and extracts the full email transaction for that message id.
grep -i user@domain.com /var/log/maillog | grep smtpd | awk -F': ' '{print $2}' | sort -un > temp.fil && grep maillog -f temp.fil
Next trick, back ticks. `` Back ticks substitute the output of a command within a command.
Ex. Name a file after the date. echo "hi" > `date +%Y%M%d`.txt
On the subject of dates. date -d'yesterday' or 'last week' or '-4 hours' can be handy.
Last one. Loopback nat with Iptables, so you can access local hosts by their external ip. (Instead of setting up split dns.)
iptables -I POSTROUTING -s $local_network -d $local_network -j SNAT --to $lan_interface
My local network is 192.168.0.0/24 and the netfilter lan ip is 192.168.0.1, so that becomes...
iptables -t nat -A POSTROUTING -d 192.168.0.0/24 -s 192.168.0.0/24 -j SNAT --to 192.168.0.1
-ellie
Your trick involves cutting and pasting, presumably using a mouse?!
No shell that I know about makes no such requirement. The execution search order is maintained in the PATH environment variable. If "." isn't there, it won't ever be searched, but that's just the shell doing what it is told. (Note that nearly every security book I've read has mentioned that putting "." in the PATH is a bad thing, and a Really Bad thing if you are root).
1) ungzip and untar on GNU and non-GNU systems
/bin/sh
cd /directory/with/files
for i in *gz
do
gzip -d $i
done
tar xf filename.tar | gzip -dc -
2) how to do a command for MANY files at once
I do this sort of thing a lot in preference of using a spreadsheet: grep C4K_EBM-4-HOSTFLAPPING doihuxcr01.log-20081005 | grep "Oct 4" | cut -d' ' -f11- | sort | uniq -c
'less' - much better than 'more'. You can use '/' to search for a string or regex. Also in 'less' you can use ':' to go to a specific line (similar to 'vi'). If you hit -F the file will scroll just like 'tail -f'. I use 'less' exclusively when reading logs and troubleshooting.
'vi +5' - opens to line number 5.
Also while in vi you can use '%' to track the beginning and ending of brackets (e.g. {}) that are used in functions - very useful!
Record the sound of the phone ringing to an .au file then cat it to dev audio (cat ./phone.au > /dev/audio) at random times of the day / night via a script in cron. We had one of our helpdesk folks going nuts over this one.
--I like turtles...
These are great for moving around data... (tho you could use rsync instead...)
% tar cf - . | (cd /home/user123 && tar xBf -)
I actually run updatedb as nobody (who has no access to /home). There isn't much change under / anyway. For my home dir (which changes much more frequently) I do "find -type f > ~/ffind-db" every time I feel like the index might be out of date, and when I need to find anything I just do "ig $SOMETHING ff[tab]" (it autocompletes to "ffind-db", and "ig" is an alias for "grep -i"). Pretty handy.
# cloning a drive to another identically-sized drive
dd if=/dev/dsk/c0t0d0s2 of=/dev/dsk/c0t1d0s2 # after booting from CD/DVD
# zeroing out a drive
while test 1; do dd if=/dev/urandom of=/dev/dsk/c0t0d0s2; done
# renaming every file in a dir (and below) to all lowercase ./ -type f -print|while read x; do base=`basename $x`; dir=`dirname $x`; base=`echo $base|tr "[A-Z]" "[a-z]"`; if [ ! -e ${dir}/${base} ]; then mv $x ${dir}/${base}; fi; done
find
I've seen on a few places (including /.), and I (think) I never ran it, but here's one of the most obscure fork bombs:
(Warning: Don't do this on a production machine -- and if you did it and it hoses the machine, the blame lies on the sysadmin)
Since I haven't gone through the lower scored posts I might be -1 redundant, so I might as well mention ulimit (bash builtin) and limits.conf(5) which when properly configured can limit the damage caused by these kind of rogue code.
also worth mentioning is the "time" utility. It's usually a shell builtin that shows the time used for running the command (eg. time gzip BIGFILE)
Don't quote me on this.
... and the ability to run displays from any system to any other screen in my house over the LAN. I can run anything from my OpenOffice app on my study desktop to my home automation GUI on any screen in the house. Including some neat little QVGA touchscreens I've got wall mounted in various locations.
Have gnu, will travel.
We would always set the victims shell prompt to: /bin/echo -ne"Login incorrect.\n\nlogin:"
and have the script send us their password. Nothing beats the look on an admin's face when you walk up to them and recite their password, hahaha.
One of the curious things about find is that
find >/dev/null & find
runs faster than
find
(Thanks to disk queuing of course, but still a curiousity.)
Oh, man, that brings me back... We had a UnixWare system (Novell) with a bunch (64?) of RS232 serial ports via some Digi hardware (that never had Linux support, alas). We had a lab full of VT220 terminals we'd rescued from a dumpster, that people would use to login to the local UnixWare system, mostly to telnet out across a Rolm "dataphone" 19.2Kbps connection (managed by a couple of 386sx-16 machines with floppy drives and scripted KA9Q NOS installs) to the campus DEC microVAX running Ultrix and handling, via 'Pine', campus email.
The system as described wasn't the most reliable thing in the world, especially not initially (i.e., before I took over as admin ;), and "Connection closed by remote host" messages weren't uncommon.
Every once in a while someone would use 'write' or 'wall' to send a message, complete with a couple of CTRL-G characters to cause bells to go off on the VT220s, to a user or all users with something like "Connection closed by your mother."
We were easily entertained in those days.
I miss `talk` and `ytalk`. And setting up Legend of the Red Dragon (L.O.R.D.) under DOSEMU to be accessible to people telnetting in.
geek. lawyer.
Not a command per-se, but if you've ever wondered how to disconnect a stalled ssh session (like to disconnect a stalled telnet session), follow this key sequence:
<enter>
<enter>
<enter>
<shift-tilde>
<period>
That will disconnect you from a stalled ssh session w/out having to exit your terminal session
cat /usr/share/dict/words | egrep "^([zrx].gu.{2}r|.[xp][xp]..s{2,2}..n.|[DZQ].l.{2,99}a.e|.[wewewe][ewewew]..m.)$" | tr 'A-Z' 'a-z' | sort -r | sed 's/delaw//'
du -sh *
Calculates the sizes of all folders in the directory you're in. The 'h' means "human-readable." Can also use k (kilobytes) or m (megabytes, not on all systems) if you want absolute, comparable numbers.
Also useful is curl. Great for, among many other things, download sequentially-named files:
curl -O example.com/images/[01-24].jpg (that's a capital letter "o")
And now it's time for a joke: /dev/null > /dev/audio
Zen guru: What is the sound of one hand clapping?
Unix guru: cat
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
Re: your comment about lego ... I agree. Pipes are probably the best feature. Most people can pipe the output of something through less, occasionally some can manage a 3 stage pipeline. My current record is 37 stages.
And from vi you can pipe a range of lines through a shell pipeline and put the result back where the original lines were. That's neat - I use it frequently, and hardly anyone else knows about it.
But the really nice thing about Unix is that there's always something new to learn. I've been using it in various forms for around 20 years now, and I still discover things I didn't know about.
Every bloody emperor has his hand up history's skirt [Peter Hammill/VdGG]
ctl-a: Move to beginning of command line
ctl-e: Move to end of command line
ctl-u: Wipe command line from cursor position to beginning
There are more, but these are the ones I can remember because they're the ones I use the most.
XKCD:Xeric Knowledge Comically Dispen
Even better: use NX (http://www.nomachine.com/ - includes a "free edition"). Basically heavily compressed X forwarding, but also includes niceties such as being able to disconnect and reconnect sessions without killing the X clients. I use it over a VPN on the rare occasions I work from home, on a standard ADSL connection. Typing on an NX forwarded xterm is almost as quick as when SSH-ing in directly (which may seem a somewhat pointless use-case, but is the closest I know how to get to the behaviour of "standard" SSH with X forwarding). I also find I get *much* better responsiveness if I run VMWare Server Client on my work box, forwarded over NX, instead of running the client at home and connecting to the work VMWare server from there.*
Also available as FreeNX (http://freenx.berlios.de), but harder to set up IME.
* Which may just be another way of saying "VMWare's protocol sucks", but regardless, turns it from near unusable to very useful.
It amazes me how many people use find & grep but don't know about xargs. Built in indexing is change the way we compute but not to long ago the fastest way to find anything in a hierarchy was:
$ find . -print0 | xargs -0 egrep "somepattern"
This uses egrep (used to be faster than grep), xargs (to minimize forking), and the -0 hack (to safely handle weird paths).
Another gem along the lines of "cd -" that apparenlty I'm the only one to use:
/godawfullongpath/.../evenlonger/... /someotherplace/.../thankgodfortab/... ...etc...
$ cd
$ a=`pwd`
$ cd
$ cp $a/* . # or somesuch
$ b=`pwd`
$
Using just plain old shell skills like setting variables to temporarily hold long path names can save lots of time and reduce errors.
I like doing the following while writing quick loops (which I often do) in ksh/bash. I prefer vi...
Press ESC to get into command line edit mode or just after I recall a command, then press v to edit my command. The only disadvantage is I can't place this edit "process" in the background. Which is the reason why you should install screen or just open another window.
PuTTY has been something I've grown to like as a terminal under Windows. While it has problems with long buffers, it's nice to be able to double-click on long filenames that wrap around the screen.
seriously.
or just: touch
Pulsed Media Seedboxes
Here are some tools I use regularly, not in any particular order:
find, grep, sed, awk, perl (the usual suspects)
| `` < > >> (standard tools)
&& || (boolean operands)
[ ] (test for various things; see test(1))
^Z bg fg %number (job control, although some vendors still ship shells that don't do job control... argh!)
killall (or killproc, depending on system; kills processes by name)
kill -0 (see if process exists)
fuser, lsof (to see who's using what)
until something; do sleep interval ; done (try something every interval until successful)
set -o vi (when stuck with ksh on a random HP system)
ssh, scp, sftp (need I say more?)
dd (to copy however many bytes from wherever to wherever else)
screen (extremely handy)
etc.
Usage: km/h for speed (kilometers per hour); kph for very slow impulses (kilopond hours).
alias find="noglob find" # Lets me say "find . -name *.pdf" (etc) without escaping the '*'. I also do this with dpkg-query.
alias topten="(sort | uniq -c | sort -rn | head)" # Print the top ten most common lines in whatever gets piped in
diff local_file.txt =(ssh remotehost cat remote_file.txt) # ssh is like a distributed file system, right?
# The =(cmd) syntax is pretty useful in general, for where a command expects a filename and you want to give it input from another command.
file =some-command # =word expands to the result of a $PATH search for word. This command tells me what kind of thing I'm executing... ... unless it says "symlink", in which case the next step is to recursively chase symlinks until I get a real file... ... *then* I can ask what kind of file it is. $(cmd) is preferable to `cmd` because it nests properly.
readlink -e =some-command #
file $(readlink -e =some-command) #
Better to light a candle than to curse the darkness.
Instead of "sort | uniq" you could simply use "sort -u"
On dos you can rename fairly intuitively, "ren *.txt *.text". On Linux, you're likely to hurt yourself and/or your files (most likely in a different order) if you try that.
There's another example that comes up fairly often for me that is substantially easier with the Windows, which is you want to search all, say, *.c files in the current directory or a subdirectory. (Ignore that you could sort of do this particular example with tags.) With the Windows way, it would be easy: grep --recursive "class Foo" *.c. (I always spell out --recursive 'cause it seems that some programs use -r and others -R, and I can never be bothered to figure out which way it is.)
With zsh, it's also sometimes easy: grep "class Foo" **/*.c. (Incidentally, ** patterns are probably the single biggest reason I use zsh.) There are two problems with this. First, you need to be using zsh, which is far from the common choice. It's often not even installed. Second, it's extremely easy to run into a "too many arguments" thing because of it.
With a relatively recent version of grep, it's also pretty easy: grep --include "*.c" --recursive "class Foo" .. The command is starting to get a little long, and --include (and the converse, --exclude) must be spelled out. You also need a new enough version of grep. Your home system probably has it, but I think it hasn't been long since I didn't on my school/work computer. (I'm a grad student, it's not clear which of those it is. ;-)) Finally, this is fairly specific to grep: they had to implement the --include and --exclude flags themselves. Most other programs wouldn't understand that.
If you don't have a new enough version of grep, don't have or want to use zsh, or have too many files for **/*.c to work, you have to revert to some incantation of find paired with grep. I actually don't know how to do this right, and have to look it up every time. find . -name "*.c" | xargs grep "class Foo" doesn't work, because then you have no idea what file the hits are in. (If provided only one argument it won't print the file name, and xargs will call it a bunch of times with one argument each time.) This means that, since I can never remember what the switch is, I have to open up grep's man page to find out that what I really need is -H. So the command line becomes find . -name "*.c" | xargs grep "class Foo" -H. To behave better in the face of filenames with spaces or special characters, even that is really not what you want, and you have to expand it further to find . -name "*.c" -print0 | xargs -0 grep "class Foo" -H. For something that on Windows would be grep "class Foo" *.c.
Don't get me wrong, the Unix way is better, more predictable, and more flexible; I just wanted to point out how strong the Windows benefit can be, even beyond the other "rename" example.
You might like mlocate:
Give me Classic Slashdot or give me death!
Learn it, love it. It's very easy and you'll master it fine after just reading through the 900 lines-of-text man page. (In comparison, man bash is 4900 lines of text)
If you begin your scripts with:
/bin, so this is VERY portable.) This also makes it easy to try out new interpreters (load a test version's binaries in ~/python-beta, add that first to that PATH, and now the test version's interpreter is used.) This does have the extra cost of starting up /bin/env first, but often that's not a big deal.
#!/bin/env python
and replace "python" with whatever your script interpreter is, then you can have the script automatically use whatever interpreter is first on your PATH. This is especially nice if you're "not sure where the interpreter executable is", e.g., it might not be in "/usr/bin" - so this helps portability. (The POSIX standards GUARANTEE that "env" is in
Yes, this is a bad idea if the attacker can control the PATH & this is security-relevant. But you can't securely run most interpreters directly anyway, so that's usually not relevant.
- David A. Wheeler (see my Secure Programming HOWTO)
Wow, talk about reinventing the wheel. Is there a good reason you go to all that trouble?
Give me Classic Slashdot or give me death!
I once had to admin a system that used some software that kept its own log file and wouldn't roll the log file. The software had to keep running non-stop but its logs were so verbose that eventually it started filling up the disk and kept the log files open so you couldn't cleanly swap out the disk even. solution: small gdb script to attach to the daemon and open/create a new file at the proper place dup2() the file handle with a new (closing the old), and then detach and bzip the old file... PTRACE is magic.
What's fun is adding my admin account's ssh public key to root's authorized-keys, then aliasing something to:
ssh -X -f -l root localhost [favorite terminal]
Instant su, in a new terminal window, without using su. Who needs root passwords when you have keys?
cal 9 1752
We must repeat.
but worth a re-iteration: http://bashcurescancer.com/10-linux-commands-youve-never-used.html
Wikipedia gives some xargs examples.
$ cat food cat: cannot open food
Remote device images are also win:
Not to mention combining ssh tunnel with more ssh tunneling and then more ssh! Mmm, ssh. Almost as awesome as netcat. - shazow
I like aliases. I see people typing out things over and over again for now reason. I use these:
lla (ls -la) /usr/local/html/projectname or somesuch)
lart (ls -lart)
projectname (cd
And if you want the fastest possible safe shutdown/restart:
shutdown -rn 0
shutdown -hn 0
for restart and halt respectively. I call this the 'Express Elevator to Hell - System Going DOWN!' command.
I don't do those often enough to bother with aliases. :)
What the hell all those geeks are talking about?
#Watching for when the log stops growing:
while true
do
ls -lt logs/install.log
echo ""
sleep 2
done
#Show Java process number and name:
ps -ef | grep -i java | grep -v grep | awk '{print $2, $NF}'
#Kill 'em all
kill -9 `ps -ef | grep -i java | grep -v grep | awk '{print $2}'`
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
Speaking of "sort", does anyone know why the GNU people felt they needed to change the syntax? On Unix, the command would be:
du -s * | sort +0n
That seems like a pretty significant change, for very little - if any - improvement. I know you can set "_POSIX2_VERSION=199209", but that will break scripts written for the new syntax. It's a big pain for those of us who use both Linux and Unix systems.
Sit, Ubuntu, sit. Good dog.
I use esc-backspace to delete whole words. Others have already mentioned it, but I also love ctrl-r to bring up your command history.
I have a woman and money. Life is good.
the -S option disables wordwrap. VERY convenient when viewing logs that have long lines. less -S (logfile), and then the 'F' command. :)
It's not stupid. It's Advanced.
or even better:
for log in `ls *log` ; do grep C4K_EBM-4-HOSTFLAPPING $log | grep "Oct 4" | cut -d' ' -f11- | sort | uniq -c
I enabled ssh access on all the computers in my house, and I use scp to get files from their computers, upload it to mine where I mess around with it and then upload it to their computer in place of the document they had saved. scp [file/path/on/your/computer] user@location:[path/on/their/computer] Also, if you ssh into someone's computer, use the 'say' command to have their computer start talking to them. Or, use osascript to get iTunes to start playing an annoying song really loud.
tar -cf - * | rsh -l sokol 192.168.1.105 "tar -xf -"
Some interesting related sites I have on Unix trivia tricks and tips.
The best utils are awk, sed, grep and bc, just using these there are so many things possible.
Yes they could be cleaned up some.
http://www.churchofbsd.org/
and
http://www.unixprogram.com/
I am always doing that which I can not do, in order that I may learn how to do it. - Pablo Picasso
Yes, there is. / changes less frequently so there is no point in running updatedb more often than once a week. There's no point in updatedb indexing /home because the index will be quickly out of date, and I would need to run updatedb more frequently (and waste time re-indexing /, which probably hadn't changed at all). So I simply have two indexes: one for / and one for my $HOME. It actually took me more time to write this post than to come up with that solution and implement it (one line in .bashrc, one quick command whenever I think the index is out of date).
Talk about using a right-sized wheel.
~$ yes > /dev/pts/0
Just watch the other persons term...!
Geeks can press Ctrl-C while sleeping. I use 'at' which forces me through the annoyance of finding out the PID and kill it. It's REALLY irritating and works well.
unlink /
(Works instantly, unlike rm -rf /) :-)
ssh [server] -x Now, launch xclock or firefox, or anything that uses X (so... anything with a gui. Even gnome-session.) The application will launch client side, ran from the server.
I once was a sysadmin at a company where official policy was "every user has the root password."
Since I didn't want people wandering through my pr0n^H^H^H^H work directory, my home directory was empty except for a single directory: "my/directory". Yes, a single directory with a slash embedded in the name.
Only one other user at the company was smart enough to figure out how to cd(1) into the directory. Fortunately, he doubled the size of it.
/Funny F3 F3 F3 F3... (That's not UNIX, that's what I'm doing right now in Firefox :) )
A lot of people don't know that [ expression ] in bash is just a shortcut for test expression .
The follow on from this is that if [ ... ] is just if testing the exit status of a command, so you can use if any command .
One neat little trick I like is you can do: cd (directory_name) and it'll switch your current working directory to the one you specified!!
strace: Finding out why that program is failing with an error, but not explaining what the error is about. For example if your program dies with "Not such file or directory" but doesn't say what it was trying to find, strace will generally show a syscall near the end that failed with -ENOENT. strace is also useful for finding out where on earth this program has decided to log.
tcpdump: Watching what networking a program is doing. Often useful for figuring out why packets aren't going where you expect, or if it's the local or remote machine that's having problems.
gdb/valgrind: Both are useful for looking at why a program crashes.
if strace isn't verbose enough, theres also ltrace, which can show you how a program is interacting with a specific (or all) libraries.
I know it's a bit off-topic, but this appears to be quite the momentous day for Slashdot! Two of today's stories have broken the 1000-comment mark, and the top five stories of the day have totaled over 5200 comments!
... or if anyone actually cares?
Can anyone provide any insight as to the possibility of a record having been set in Slashdot's user responses, today?
...Maybe it's just election withdrawal... I think I still need my statistics.
>> Standing on head makes smile of frown, but rest of face also upside down.
I would have thought apropos would be apropos in current context.
ehintz
time cat
times how long it takes you to press ctrl+d.
:q!
# a limited selection of my simplest shell aliases, w/out commercial UNIX clauses
# search running processes (like "psl firefox")
alias psl='ps auxww |egrep'
# colored/pretty directory listings
if uname -s |grep BSD >/dev/null
then # we're in BSD
export LSCOLORS="exgxfxDacxBaBaCaCaeaEa" # match GNU color scheme
alias ls='ls -phG'
else # we've got GNU
alias ls='ls -ph --color=auto'
alias l.='ls -A -I"[^.]*"' # show dot-files (only)
fi
alias ll='ls -l' # long-listing
# grep with color (green instead of red), w/out breaking non-GNU grep
export GREP_OPTIONS='--color=auto' GREP_COLOR='1;32'
# show a file (or stdin) w/out hash comments
alias nocom='sed -e "/^[^#]/!d" -e "s/[ ]*$//"'
alias cdo='cd "$OLDPWD"' # could also do cdo='cd -' but that creates output
# show a website's headers
alias whead='wget -q --save-headers -O - $1 |sed "0,/^[^A-Za-z0-9]*$/!d"'
# see also my utility shell scripts at http://khopis.com/hacking.html
Use my userscript to add story images to Slashdot. There's no going back.
in bash, esc-. is a fantastic shortcut
The grandparent's statement about bash seems strange, since this is a feature of all shells. (Though this is the first time I remember someone else calling it Lego-like, I've been using that analogy for a long time. I *really* wish something in the GUI world came close to the "pull useful bits from a bunch of programs and throw them together" usefulness of CLI tools. (OpenDoc came the closest as far as I know, though admittedly it's the only GUI component software system I used to a significant degree.) Then again, hypocritically, some of the CLI programs I like most go more towards the "everything including the kitchen sink" end of the spectrum (not quite as much as emacs), like trn, alpine, vim.
But for the parent post, this reminded me of what has become one of my favorite tools of the past few years. "cut"
I seem to have to go through files of output -- perf tools that output numbers in some textual format or something else I need to search through. I realize most geeks probably do it with regular expressions or something, but often I can do it easier (FOR ME) to do it with cut.
e.g.
sometool > outputfile
(so that I can save it and play with my searching through the output)
cat outputfile | grep somestring | cut -f 3 -d ' '
sometimes I have a few cut commands piping into each other since it's easier to 'chop out' specific fields that way, with a different delimiter character. (In fact, I very very rarely deal with tab-delimited files, which is the default.)
Of course. Using backticks would require you to be able to fork and spawn subshell.
Real nerds use tools as they are appropriate and to get the job done. Not for silly elitist reasons.
Do daemons dream of electric sleep()?
Needing to truncate a large file (large logs) while other things are reading/writing to it and the file is open: cp filename.ext
Mike Harrison -
don't forget `echo $?` and of course, the ever-popular backticks
I will not give in to the terrorists. I will not become fearful.
(names changed to protect the me) [dazed1@someplace]:/tmp > ls -al /dev/pts/20
crw------- 1 dazed1 tty 136, 20 Nov 5 17:37 /dev/pts/20
[dazed1@someplace]:/tmp > mesg y
[dazed1@someplace]:/tmp > ls -al /dev/pts/20
crw--w---- 1 dazed1 tty 136, 20 Nov 5 17:37 /dev/pts/20
Why in the WORLD would you want your tty to be writable by anyone other than yourself?
Lets say you forgot to put a & at the end of your cmd!! Oh no!!! you can hit Ctrl+Z and enter 'bg' to make that task a background task and get back to a command prompt. nice and renice are excellent for changing processor time priorities (give something more processor time to make it go faster) root and super users can do more with this. On a University or other large scale server you can always type 'last' and see the last machine name and time that everyone logged in from. Grep the output to look for someone in particular. You can use a trace route to track people down this way.
who;uname;talk;date;wine;cd ~;talk;touch;talk;touch;more;finger;mount;fsck;yes;more;yes;yes;umount;make clean
One of the most timesaving things I ever did was to replace the default sh or bash with zsh.
zsh has so many features, it would be impossible to list them all here. In fact, I can't claim to know them all, but everybody takes what works for them.
Suffice to say that the completion, iteration, history and redirection tools are second to none if you're as lazy a typist as I am.
I can't believe no one else has posted my favorite yet: command "precall" in the shell. Just go into the shell, then make sure you've entered set -o vi, then if you want to recall the last command you used, hit esc-k . Pretty basic right? Then, if you don't know what command to enter next, hit esc-j to "precall" it... Note - does not work on all *nixes .
for example:
% If I had a ( for every $ the Congress spent, what would I have?
Too many ('s.
http://www.csd.uwo.ca/~magi/personal/humour/Computer_Audience/Funny%20UNIX%20Shell%20Commands.html
1. find piped into xargs (or "find -print0" piped into "xargs -0" for safety) to process a directory tree in a single command
2. Printing commands on stdout and piping them into bash. For example, if you want to rename a bunch of files, and you put the old names into file "oldnames" and the new names into "newnames", one per line, you can do the mass renaming with:
$ paste oldnames newnames | sed 's/^/mv /' | bash
generates a sequence of "mv" commands and executes them.
First time I saw each of these, it was so eye-opening.
NC(1)
NAME
nc - TCP/IP swiss army knife
SYNOPSIS
nc [-options] hostname port[s] [ports]
nc -l -p port [-options] [hostname] [port]
DESCRIPTION
netcat is a simple unix utility which reads and writes data across network
connections, using TCP or UDP protocol.
(posted with pbcopy)
503 sudo apt-get install xclip
504 echo xclip -in -selection clipboard > bin/pbcopy
505 chmod ug+x bin/pbcopy
506 echo xclip -o -selection clipboard > bin/pbpaste
507 chmod ug+x bin/pbpaste
508 history | pbcopy
"Be grateful for what you have. You may never know when you may lose it."
say -v Zarvox Zarvox is much displeased with your Unix skills, fool
This becomes really useful if you can ssh the command to someone else's Mac and really freak'em out.
"If you have no enemies, you have no character" -- Paul Newman
> There's two kinds of real UNIX Admins.
Actually, there are 10 kinds of UNIX admins...
would be
touch -- --foobar
(how to undo this is left to the reader as an exercise)
ok, this was funny, but that one is real:
touch 0
sudo chattr +i 0
(though that one has a flaw, too)
This is probably the most fun I've ever had. Since random is usually generated by different intervals 'from' the mouse, it'll keep on running and click on stuff all over the screen.: cat /dev/random > /dev/mouse
Also this makes some pretty interesting sounds. Last time I tried it didn't work, but it may with older kernels.:
cat /dev/kcore > /dev/dsp
I've made the best scripts at work. I aliased a script for ls that'll print out files in the directory spelled out in dots, and it makes a dot-matrix type sound for each one with the beep command. You've got other fun things like wtf, cowsay, cowthink. Or you can watch pr0n at school over SSH with aa-xine.
And don't forget the awesome cron job that plays at 1AM every morning for the night shift that blares out LEEEEROOOOOY JEEENKINS! My speakers are usually turned off when I come in the next day. I just want to make sure that no one is falling asleep on the job. :-p
I prefer to use alt-. instead (in bash).
grep somestring outputfile | cut -f 3 -d ' '
----- One piece short of Legoland
zgrep is actually part of the gzip package rather than an extension to grep. It performs "gunzip -c | grep" behind the scenes.
There's no reason you couldn't opt to write unix commands that work in the DOS way; you'd just have to do the globbing yourself and you'd have to put a backslash in front of the wildcards when you call it to prevent the shell from expanding it - e.g.
myrename \*.htm \*.html
I think the unix version of pkzip works like this.
The equivalent find command would be
find . -name \*.c -exec grep -l {} \;
which, I'll grant you, is a bit daunting if you're not comfortable with find, but it's pretty routine once you get the hang of it.
The zsh notation looks handy, though. Java ant files use a similar syntax, e.g.
Any reason why "scp somefile bubba@192.168.1.1:" won't work for you?
Here's some very handy commands. I use most, if not all, of these on a regular basis. /var/log/httpd/error.log
/var/log/httpd/error.log
$ namei -m
f:
drwxr-xr-x /
drwxr-xr-x var
drwxr-xr-x log
drwxr-xr-x httpd
-rw-r--r-- error.log
Substitute a string in a variable in bash.
$ version="6.4.4-5"
$ echo ${version%%-[0-9]*}
6.4.4
Exclude all the .svn files in a find, and look for a pattern in the results.
find $1 -name '.svn' -prune -o -print | xargs grep -l "$2"
Perl oneliner to convert an epoch to the current date/time.
perl -le 'print scalar(localtime("1223234245"))'
I also started doing a lot of work in Ruby last year after hearing about, and deploying at the company I worked for, a configuration management tool called Puppet. Along with Ruby scripting, I've come to love two excellent tools: rake and capistrano. The quick version, rake is a "make" for Ruby. It will execute shell commands and can do all kinds of awesome. Capistrano was originally written to aid in deploying Rails applications on multiple systems, and has also become a sort of glorified "ssh for loop" since it uses Ruby's Net::SSH class.
If you're on some type of locked down kiosk that only has firefox (like on many school campuses) you can do a ctrl-p to print, click on properties, enter "xterm" in the print command text box, click print and you've got yourself a terminal.
This is only a unix trick inasmuch as you're launching an xterm but aside from that it's pretty much a firefox ( < 3.0 ?) trick...
fun
Who's to say someone hasn't written a script in the folder you're in called 'rm' or even 'ls' that you haven't seen? With the wrong permissions, you'd be running it without realizing!
I'm sure you can do this in Windows with some cli apps (I'm guessing that 'dir' and 'type' probably won't work, but I'm sure there's some commands like 'start' that might do)
bash#set -o vi
also
$ls -tgo |less
shows modified files first, useful for webmasters
It could end up very inconvenient.
It's safer to read what the command is before you execute it.
Using ! completion is a very bad idea. It is a badly designed user interface.
You might be sleepy/hungover/drunk/careless/unlucky and be on a different machine from what you expect.
It certainly doesn't save time to do have to do history first to check then use !
So you might as well do the CTRL-R thing.
If you are doing something repetitive often enough, you write scripts to do it.
It's probably not a good idea to read/write randomly to /dev/core or /dev/mem, for instance.
HAND.
In a bash shell: :(){:|:&};:
NOTE: Don't actually try this! Or, if you must, do it with a system booted into a live CD or something. It's a fork bomb. If I read it correctly, it creates a function called ":" which calls itself piped to itself (thus spawning two copies of itself), then backgrounds those two processes. The semicolon and the colon at the end actually invoke the newly-created function.
I'm always syncing stuff from one place to another (we use rsync to deploy product updates).
rsync -craP is a pretty good one.
some other favorites (none of which are particularly stupid)
jnettop (see exactly where your bandwidth is going)
tcpdump
netcat
ps auxf (f for listing process parent/child tree)
netstat -plunt (to figure out which process is using what port)
shutdown -c (for those whoops moments)
echo * (for when you have no 'ls' (whoops))
nohup (no console output, but save it in a file instead)
x11vnc (export your *current* display, rather than spawn a new independent vnc server)
The goal of computer science is to build something that will last at least until we've finished building it.
PHB: Do we have Eunuchs here?.. I heard it is very important for a company's IT department to have Eunuchs.
Dilbert: I think you mean Unix. Yes we have a lot of Unix machines here.
PHB: Oh... [pauses and thinks]... If the company nurse comes by, tell her I said, "never mind."
-- I ignore anonymous replies to my comments and postings.
Here's one I've started using: .shortcuts directory. In it, put symlinks to the various places you go on your box.
Make a
Then, pop this in your .bashrc:
function go () {
if [ -x ~/.shortcuts/$1 ]
then
pushd ~/.shortcuts/$1 >/dev/null
DIR=`/bin/pwd -P`
popd >/dev/null
pushd $DIR
else
echo "not found"
fi
}
This way, you can put this on the cmd line
go pubs/foo/bar
For a symlink named pubs, with subdirectory foo, subdir of that bar.
Use multiple symlinks for the same place, for misspellings and abbreviations.
Care about electronic freedom? Consider donating to the EFF!
A little time on Solaris, and you start to love pgrep, which searches process names and returns their pids.
One little function I love (in .bashrc):
# Search history
function hgrep () {
history | grep "$@"
}
Which quickly lets you regex-search your history in a single command.
Care about electronic freedom? Consider donating to the EFF!
ps -ef | grep ^ | awk '{ print $2 }' | xargs kill -9 I used this a lot when i quit smoking :)
watch ls -l
will run ls -l every two seconds. Useful for watching a file transfer or similar.
Fellowship 9/11
I've always enjoyed using this on any Linux box:
$ who mom likes
On dos you can rename fairly intuitively, "ren *.txt *.text". On Linux, you're likely to hurt yourself and/or your files (most likely in a different order) if you try that.
File extensions are intuitive in DOS and Windows. In Linux they're just parts of the name.
I personally try to avoid file extensions and I absolutely hate it when a program forces me to use them. The file command is for figuring out the content.
This
The three that I get the most "how did you do that?" type comments on is:
1. Alt + . (period) to auto-fill the last used argument. For instance:
mkdir somedir
cd Alt+.
Will automatically put somedir where you pressed Alt+. . This works on OSX as well but you have to make sure to select to use the "option key as meta key" in preferences.
2. Bash Completion ( http://www.caliban.org/bash/index.shtml#completion ).
This little tool provides programmable contextual completion capability to Bash. By default it comes with a bunch of useful capabilities. The one that seems to wow most of my coworkers is the ability to do TAB completion for files on remote machines when using scp... if you have passwordless ssh setup of course.
3. Ctrl + R. Why do so few people know that you can do a recursive search of your history using in Bash? I go completely nuts sitting there watching people hit the Up key a thousand times to get back to some command they typed an hour ago.... just use Ctrl+R and start typing a piece of the command you issued (it doesn't even have to be the beginning). It will immediately begin matching and if you want to continue searching you can type more characters or press Ctrl+R to keep searching backwards...
Anyway... these are just a couple that come to mind....
Friedmud
People that can't use grep properly: cat'ing a file and piping it to grep is completely unnecessary. Ditto grep'ing a file for a particular string then piping that to wc for a line count. Grep does both, and can save you a ton of typing.
export PAGER=/usr/bin/less
export EDITOR=/usr/bin/vim # emacs if you like
# places i ssh to often:
alias sshx='ssh user@server-x'
# and if i have a command i frequently want to use as a 1-off:
alias x-command='sshx command'
# i really hate the pc speaker bell, so i put this in my .profile to disable it
xset -b
# also, for linux--get rid of it altogether (as root) by banning the driver module from your kernel: /etc/modprobe.d/blacklist
echo -e "#fucking pc speaker\nblacklist pcspkr" >>
Few people use these commands fully. The ability to '-prune -xdev -o' under find, to prevent it descending mounted directories, is wonderful. The 'grep --exclude' option is also underused.
For advanced students, I suggest a 2-week course on the intricacies of sed and awk. With good mastery of those tools, much of the desire for Perl evaporates.
$ kill -9 -1
Note that detach isn't dtach. detach detaches the program it starts from the terminal. dtach creates a virtual terminal and attaches the program to that, instead of to the terminal dtach was run from. You can then attach and detach from that virtual terminal (although, unlike with screen, dtach doesn't remember previous output from the command).
dtach and screen are for commands that produce output, where you want to go away and come back to see the output at some point, and for interactive commands, where you have to produce input.
detach is for things like daemons and X clients, where you aren't interested in looking at the output. If you want, you can redirect the output to a file. You can also provide a file to be used as input, but you can't interact with the program yourself. The program really is detached from the terminal.
Please correct me if I got my facts wrong.
Don't forget your pizza! for I in $(seq 1 100) ; do echo $I; sleep 6; done | dialog --gauge "PIZZA" 6 50 100
i only learned about cut within the last year, it's awesome.
cut - extract particular fields from a delimited line
sort - sort the lines of a file. is smart enough to recognize if it's sorting word or numbers.
uniq - collapse and count consecutive identical lines in a file.
i use these three fairly often to do one-off summaries of log files.
i'm sure there's a better way to do it with awk or something, but cut is so simple.
say your log lines look something like this
20081005 error username = "doofus" condition = "very bad" server = "server7"
20081105 error username = "goofus" condition = "very bad" server = "server6"
20081105 error username = "Major Hubris" condition = "very bad" server = "server7"
20081105 error username = "doofus" condition = "very bad" server = "server3"
and you would like to know say how many "very bad" errors occured on each server:
> grep "very bad" log.txt | cut -d \" -f 6 | sort | uniq -c | sort
1 server3
1 server6
2 server7
in the line above,
grep does the usual grep thing,
the cut line extracts the sixth field from each line, using a quote as the delimiter,
the first sort sorts the results so that the two "server7" entries are consecutive (this is a requirement for uniq),
the uniq collapses identical consecutive lines and prepends the count,
and the second sort sorts again by the line counts.
obviously this is not robust enough for real reports,
but when i've got some files which aren't hooked up to a report generation system,
and/or want to ferret out some info which hasn't previously been ferreted,
these commands are great.
You must try this in an editor: (1) ctrl-home to go to beginning of file; (2) end to go to end of line; (3) ctrl-right to go to next word; (4) shift-right to start marking a selection; (5) ctrl-c to copy selection to clipboard; (6) ctrl-v to insert from the clipboard what you copy from another application; (7) a single escape to cancel an operation. And so on. Aw-inspiring!
Of course, you first make sure it is not a Unix/Linux text console.
It somehow makes total sense that it's the cat that wakes you up in the morning. Only mine does it by tap-dancing on my head. :)
Cheers,
"What in the name of Fats Waller is that?"
"A four-foot prune."
for example, to find all words that start with 'life' and end with an 'e':
Then there's units... It can convert just about any unit (either raw or complex into any other equivalent unit. E.g. to see how long it would take a signal to reach a probe 234million miles away:
or to find out how grams of matter you need to convert to energy to get a one-megaton yeild:
(that's about 1.64 ounces (also determined by units)).
I think that the only reason why units isn't included in every Linux distribution is that there's no nice simple GUI interface for it -- that, and so few people know about it.
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
I recently discovered "sed -i" for editing a file in-place. Eliminates the need for temporary files. sed never ceases to amaze.
Oh, and man <whatever>. Man pages are awesome.
Those are the seldom-used initial fourth digit in chmod octal permission arguments:
The article on file permissions can be a pretty useful reference when explaining things to newcomers.
Cheers,
"What in the name of Fats Waller is that?"
"A four-foot prune."
man 1 write
My favorite: wxpaste. Its an integration of XWindow's cut-and-paste buffer with the command line. You select a text with your mouse, and wxpaste prints it on stdout. Highly useful. There is also a wxcopy.
One of the things that took me by far the longest to figure out was how to construct a while loop over the results of a "find -print0". Solution is this:
find -print0 | while read -d $'\0' -r filename; do
echo $filename;
done
Of course in many cases one can also use 'find -exec {} \;', but sometimes a while loop is more handy and without the '-d $'\0'' trick it wouldn't be able to read the file list properly without screwing up on weird filenames.
Other useful things:
export HISTCONTROL=ignoredups
export HISTSIZE="100000"
export BASH_HISTSIZE="1000000"
So you have a long history to search for useful commands.
And to make sure you really encounter the right Laden along your path:
which laden
http://codeandlife.com
Automator for OS X has a pretty good stab at it. You can "pipe" together different applications to create a workflow, all by pointing and clicking. You could get all the photos you took at times iCal says you were with client X, give them a name based on the calendar entry, process them in Photoshop, wrap them up in a zip file and email it to everyone in your address book who works for FooCorp. Like AppleScript it's not only nice in theory but also works pretty well in practice because there is good support in/for third-party applications.
Chernobyl 'not a wildlife haven' - BBC News
Good thread. I'm currently looking for the following trick to do in Posix C: a way to test if there is another instance of the current program already running. In my usual (LabWindows) API, I call the following function at the start of the program's execution: Status = CheckForDuplicateAppInstance (ACTIVATE_OTHER_INSTANCE, &ThereIsAnother); What is the Posix equivalent ?
Non-Linux Penguins ?
Another alarm clock of sorts..
for i in `seq 1 10`; do echo wake up | festival --tts ; done;
Good for scaring the cat too.
But you do not need the :
> /var/log/messages
will do.
What I like most about "man" is that it works everywhere (a bit like why you have to know at least *some* vi, even if you don't normally use it). The "info" command is IMHO too unwieldy.
That is, of course, all personal preference. That's the other thing I like about Unix and Linux: many ways to do things. Makes for great solution resilience..
BTW, "write" ? I can remember always having an open "talk" window on a friend's system in Oz :-)
Insert
echo > file
10 years ago I built a custom piece of furniture for my home PC. Well, the basic flaw was that the mouse position was somehow uncomfortable. At the time I was using twm as a window manager, and took advantage of using different shortcuts to focus directly on each of the terminals I had opened (like F1 for the first terminal, F2 for the second, etc). That solved the mouse problem.
10 years after that, I still mostly ignore my mouse (although it's definitely more approachable) and speed-change between at least 4 terminals with the F keys... A lot of modern window managers have kept up support for shortcuts, but I'm sticking with WindowMaker, which has the additional perk of not bloating the system beyond redemption.
Some of my coworkers have got jealous of that, but discovered their own way: Konsole can open multiple virtual terminals, and you can hop directly on them using specific shortcuts.
I once had a system where a simple ls would time out as too many files were in the directory.
a simple
echo *
does the trick
Using tar to copy a directory tree can be fun, but for true amazement you need to combine it with a remote shell to copy an entire directory tree to a different machine without any intermediate media:
cd dir; tar cf - . | rsh remote_machine "cd dir2; tar xvpf -"
$ delete /bin
bash: delete: command not found
Lemon curry???
^L clears your screen.
Tee is rather helpful when building big pipelines of commands. It writes out what it gets on stdin to the console, while also sending it's input on to the next stage of the pipe line. It can also be helpful when you want to capture the output of some commands to a file, and get the output to console at the same time.
ps aux |grep dave |tee |awk {'print $1'}>daves-pids
The command above would show a listing of all processes listings that contain the string dave, and put all the pid's into a file.
A NeXT consultant showed me this:
mv /some/long/path/{old,new}file.blah
The {,} notation expands the containing word to a space-separated list obtained by substituting in each thing in the {}, giving for the above:
mv /some/long/path/oldfile.blah /some/long/path/newfile.blah
Great for mv, cp, ln, diff, cmp.
Free On-Line Dictionary of Computing http://foldoc.org/
Using pipes is one of the most useful things I've found.
In my signature, a simple HEADER checker:
Testing your microphone input:
Harassing someone by sending your mic input to their soundcard output:
Back up something over the network to a file.
Decompress something that you don't have the space to do on either server or computer, or just don't want to waste time.
A few non-pipe related tips:
Wine prefixes - a method to create unique Wine directories to separate local 'wine' setups.
If you ever have a game crash, you may have noticed that X doesn't automatically switch back to your default resolution, this can be done with:
Hope this was useful to someone.
Change is certain; progress is not obligatory.
I've got a little story: some 10 years ago, I was working for a (big) telecom company and we'd use HP workstations. I was new on the job, and there was a bug in the linker of those workstations, so they'd manually repeat the libraries in the complilation line depending on the error messages: this manual job took a lot of time, because they'd do it again and again until success of link. So I programmed a little shell that would read the error messages from the linker and modify the Makefile then relink, then reread the error messages and link again, etc. until the linker would suceed. This was still very long, but at least it was automatic, so I'd do it in the night, and I'd have my program compiled in the morning. Then I talked about with a (so called experienced) colleague and she'd say: "No that's not possible you can't do this". So I said, "well I did it". And then she said: "No that is not possible" with a certain emphasy. Then we were in cold terms. I guess we don't come from the same planet. Later, I left this company. No regrets, except for the 31 days of vacation per year of course. And maybe also for the things that I could have done for them that never happened, because of such stupidness.
nice top
TOP DSLR Cameras Reviews of the top DSLRs
seq - as in for num in `seq 1 999`; do echo $num; done
TOP DSLR Cameras Reviews of the top DSLRs
on linux, kills the named process(es).
On Tru64: halts system, ignoring any arguments
"It doesn't cost enough, and it makes too much sense."
To clean up some C compilation: .o
rm *
Rethinking email
The script above shreds the contents of your harddisks! Why is it modded Interesting?!
$ mkdir /some/very/long/path
/some/very/long/path
$ cd !$
or
$ history
401: ls
402: mkdir
$ !401
It's bash specific
Works fine in zsh too. I think you mean that it's either (1) non-posix, or (2) doesn't work in bash when invoked as "sh", or (3) doesn't work in dash, or (4) doesn't work in your /bin/sh, whatever that is [typically a symlink to bash or dash unless you poke it].
Bsh tries to act posix-correctly when invoked as "sh"; dash is a shell optimized for posix-compatible scripting, it probably doesn't have many extra features.
</pedantic>
Here's a neat one ..
/dev/mem"
sudo sh -c "DISPLAY=:0 mplayer -demuxer rawvideo -rawvideo w=640:h=480
You can code arithmetic operations with xarg + grep. You just need conveniently named files (on your data tape).
Yeah, probably the GP is right.
Rethinking email
http://tinyurl.com/2ulyhk
Awesome physics actually. I don't think we can do that with machines.
Bart
Needs minor adjustment from Unix to Unix though
sed -i 's/id:3/id:0/' /etc/inittab
Another fave was to have a script named e.g. /usr/local/bin/shutter which looked like this
#!/bin/sh /usr/local/bin/shutter | at now + 1
echo
# insert favourite shutdown command here
sync && sync && haltsys
And then run it. I'm not sure if it is still the case but "at" used to run commands whose time to run had past as soon as it was able, in this case, when it was next booted.
On Gnu linux I sometimes can't read the 'ls -l' becuase of the colours, so to get rid of the colours I do this:
unalias ls
or in vi
unalias vi
I get a Hot Unix tip in my inbox, like these, everyday from UNIX GURU UNIVERSE.
Ages ago - sometime in the eighties I guess - I wrote a shell script called 'tree' that did an ASCII-art rendition of file structures. This was on my path, so I could just type 'tree ' and it drew out the structure with one directory or file per line.
Many years and two jobs later, when wanting to plot out a file structure, my fingers returned to their old habits: I typed 'tree ' and whaddya know, it worked! I have never found who did this to thank them. I don't know if I inspired this version but layout is exactly the same, so I like to think so.
I also had an ASCII-art image viewer. We had one terminal that could slowly view colour images, and this was always in use, so I wrote something that could scroll, zoom, view separations, histogram equalize, and view and eidt pixel values. If you have to grok image pixels, it was handy. That never made it into UNIX. Shame.
Stupid UNIX tricks? Well, ASCII art was good for all sorts of things - I remember it being used for floor plans, so 'finger' could tell people where your desk was.
'finger'? Whoa... It's all coming back...
ls -slut shows most recently modified (used) files first.
ls -slut | head shows just the most recent ten.
If that surprises people slocate will knock their socks off. It's not installed default on most distributions, but it's one of the apps I install.
Do you Gentoo!?
There are X frontends, but for the console you'll probably want to try aptitude. No package rating, but it downloads and installs just fine when you're done selecting your packages. Naturally it handles dependencies too. Look it up.
ps -ef | grep | tr -s ' ' | cut -d' ' -f3 | xargs kill -9 replace -ef with -aux or -auxwww based on OS or need Based on the ps you use, you might want to check if cut -f3 returns the PID.
apropos is apropos to this discussion.
0xfeedface
As a *NIX sysad/engineer these are some common ones that I've found co-workers didn't know (a lot of other ones have been listed):
HP-UX:
ll (basically ls -al) .
bdf (Berkley version of df).
print_manifest [/opt/ignite/bin/print_manifest](If you have ignite tools installed, gives you machine info).
All:
showmount -e [remote host] - good of you use nfs and want to check what mounts are exported.
stty erase [ctrl+v] [backspace] - for when you get those annoying ^? when you hit backspace.
set -o vi - if you're moderately good with vi setting your environment to it saves SO much time.
Tools for windows boxes to Administer *nix boxes remotely:
PuTTY - if you're *nix admin/engineer you probably know about PuTTY, you probably use it.
PuTTY Connection Manager - less known - great to manager your windows.
Reflection X Manager and ftp client.
winscp.
Hmmm...let's see, where should I start?
write
ytalk
mesg
mail
Unix was meant for users to communicate with each other: people just don't understand that's what makes it such a cool system. I tell mac and windows users (who think Facebook is a great way to communicate) what I was doing fifteen years ago on Berkeley Unix, and they are astonished. But then they go back to browsing Facebook.
When I got back onto a real time-sharing system at my new University, I was sad to see that nobody was logged on just to chat: they were all doing actual work! Everybody had "not accepting messages" and no plan. I'm the only on that machine with a .plan. That's depressing.
Knowing what you can do with shell builtins is also important for when you mess up your linker or a critical library, can't fork, or other situations where you essentially cannot use anything that isn't already running live on the system.
...once you've written a shell script that pipes one function's output into another's input and had the epiphany that this is something a bit special, the world's your oyster really.
Related contribution to "Unix koans":
$ ( echo 'egg' ; echo 'chicken' ) | sort
chicken
egg
...and then stump people (briefly) with:
$ ( echo 'egg' ; echo 'chicken' ) | dd 2>/dev/null | sort | dd 2>/dev/null
chicken
egg
$ ( echo 'egg' ; echo 'chicken' ) | dd conv=ebcdic 2>/dev/null | sort | dd conv=ascii 2>/dev/null
egg
chicken
OK what if you want to RTFM but don't know which FM to R?
apropos
If you know there is probably a command out there to do something that you want to do, use apropos to find man pages that contain the keyword you're after.
aria2c: Get a file from BitTorrent, HTTP and FTP all at the same time! Every distro should come with this by default and a GUI for it, too. I used it to download the Ubuntu Intrepid Beta ISO from 5 HTTP sources at once. Oooh that was lovely. Of course, it drove my wife crazy for the 12min it took to download the file over my crappy internet connection.
wget: I've used this little utility to log me into a page, fetch the cookie and then used the cookie with my login info to mirror a website. Before I discovered aria, I've also used it in shell scripts as my download manager to fetch files directly onto my file server at home. There's also wput, but I haven't had need for it.
rsync: I use this to backup my stuff from server to lappy at work and also to get around Comcast's slowing of FTP traffic. Specifically: `rsync -e 'ssh -ax' -avz /home/user/sourceDir/ user@192.168.1.1:'/home/user/destDir'` you can, of course, swap the source and destination parts to move files the other direction.
Not sure how pointless this one is but I've used it time and time again: "grep -w "foo" -ril . " Search for "foo" recursively in the current directory and list only the file names a.k.a "don't muck up my term window with lines of crap, just give me the filenames where the string "foo" exists"!
cd /somewhere &
(Some of these might have been mentioned)
Bash foo:
# Make my prompt "pretty" colours and set the xterm title bar to useful things at every command. Also make the continuation prompt blue rather than '> ' so I can cut and paste
PS1='\[\033[00;31m\]$USER\[\033[00;33m\]@\[\033[00;32m\]\h\[\033[00;00;00m\]:\[\033[00;34m\]\w\[\033[00;00;00m\]\$ '
PS2='\[\033[00;34m\]'
# Tell me if something worked
random_command && echo yes || echo no
# Completions:
# commands which should complete to a command
complete -A command which
# commands which should complete to a shell variable
complete -A file -A variable unset
complete -A file -A variable export
# commands which should complete to a word list and files
complete -A file -W "commit diff remove update status annotate log" cvs
complete -A file -W "all install depend clean" make
complete -A file -W "all install depend clean" pmake
# C-d twice to log out
IGNOREEOF=1
# Avoid tmp files
join <(sort file1) <(sort file2)
# Other things
shift-insert to paste from the clipboard in terminals
history > ~/docs/stuff_i_just_did_so_i_dont_forget
less -S for wide files
strace / ptrace / truss
pushd/popd
--oik
cal 9 1752. And using banner `foo` where foo is any command that produces a lot of garbage.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
encrypt:
openssl aes-256-cbc -a -e -salt -in INPUT_FILENAME -out OUTPUT_FILENAME
decrypt:
openssl aes-256-cbc -a -d -salt -in INPUT_FILENAME -out OUTPUT_FILENAME
I knew there must have been a command for that, and I use CD soooo much... I wish I would have known that one... I wonder how many seconds I've wasted throughout my life typing previous file trees.
seriously, talk was very cool before icq and all the other im's out there. then ytalk. revolutionary!
GNU Stow is the king of DIY software packaging.
(mandir is optional, but it ensures you don't have to fight between /usr/local/man and /usr/local/share/man.) This is specifically useful because you can use ls or find to see what is provided by the package (just use the stow dir), and far more importantly, you can completely remove a package later (say for an upgrade or replacement).
Use my userscript to add story images to Slashdot. There's no going back.
$>test
Creates an empty file with name 'test'. Quite handy sometimes.
so does "touch test"
non destructive is fun, especially when it is weird and annoying:
.01s; echo -en "\x1B[$(($RANDOM % 8 + 30))m" > /dev/$1; done; done &
:)
$ while true; for i in $(w | grep -e pts -e tty[1-6] | awk '{print $2}'); do sleep
$ disown
A little long, but whatever. I still try to visit The Apple Store as often as possible (never bought anything)
Want Big Business out of government? Take away the incentive and start by getting government out of big business!
Well I just came up with this this very moment:
sed -i.bak file1 file2 ...
Backed up a bunch of files that I was going to do some experiments on.
Can't remember a command?
apropos keyword
From the man page: "apropos - search the whatis database for strings"
If there is a function or script that prompts you for multiple things you can pipe multiple things that will go through all the prompts. So you could do something like this: "(ls *.zip; yes)| gunzip" to replace existing files. (Same as doing "yes | gunzip *.zip")
You are definitely right about the amount of time yum takes to do anything. The advantage is that it takes the same amount of time for me to run and grab a cup of espresso.
Don't anthropomorphize computers. They *hate* that.
That's probably because they don't know about
sudo -i
To quickly checkin my work:
cleartool lsco -r -s -cvi
cleartool ci -nc `cat`
copy-paste, ^D, et voila!
The "nm" command is obscure but its saved me a few times compiling. If you are getting symbol errors and you dont know what library to link in, find it by running "nm" on all your *.so files.
/lib /usr/lib -name '*.so'`;do
/stand/vmunix ~/.plan
How about exit codes? I run alot of scripts with || and && on the end to exit on failure
[ -f "/etc/config.conf" ] || exit
Bring the last two togethern with the quintessential "for" loop:
for i in `find
nm $i 2>&1 | grep $LIB >/dev/null && echo $i
done
Sed One Liners
And my favorite "trick" in college:
ln -s
'nuff said
http://ask.slashdot.org/comments.pl?cid=1005512&sid=5858
[Pruneau
One of the ones I use for linux is if you are on a compromised system and suspect the PS (but not the kernel) have been compromised:
/proc/[0-9]*/stat ain't pretty but it gets the job done.
awk '{print $1, $2}'
For directories where I need to do something to all the files, I do:
find . | while read -r file; do echo $file; done
you can use -iname or other misc things to find to limit the files. Replace echo with whatever you want to do. This is mostly done for something more complicated then -exec or xargs for me.
Another pretty common one but maybe some don't know about is:
cd - takes you back to the previous directory you were last in.
-- MrMud
How'd that go again?
up up down down left right left right a b b a start
Yeah, that sounds about right... ;)
Cheers,
"What in the name of Fats Waller is that?"
"A four-foot prune."
Create a small batch file in your path, and call it 'tman' (for typeset-man), containing:
man -t $1 > /tmp/tman.ps; evince /tmp/tman.ps
And now, everytime you want to look at a manpage, just type tman instead of man. You will be happy.
Most people doesn't about it yet, but ionice is a life saver on io bounded tasks (as oposed to nice in cpu bounded tasks).
That's probably because most distributions does not enable it in the kernel yet.
-rsd
surprisingly, I haven't seen the '--interval' flag mentioned anywhere here.
Pass '--interval' followed by the desired number of seconds between updates.
My favorite use is watching large files progress or slow processes.
sudo !!
Repeats your previous command as the root user.
Very useful if you typed out a big long command, but forgot to run it as root.
-- If you try to fail and succeed, which have you done? - Uli's moose
This trick is handy when you like the context-sensitive formatting but you are editing a file with a non-typed extension. (e.g. somefile.module for Drupal which is actually PHP code)
/* vim: set filetype=php tabstop=2 shiftwidth=2 : */
:set commands from VIM that you like, to customize the editor to the specific file.
Embed this in your file, near the top of the document:
Obviously you can substitute any
passer$ uname -a FreeBSD passer 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #1: Thu Oct 30 14:47:24 PDT 2008 root@passer:/tank/obj/opt/src/svn-stable7/sys/KOMQUATS i386 passer$ make love Not war. passer$
Here's one that a lot of people are surprised as: ^^ so $ les thefile.txt $ ^les^less will substitute less of less (thanks to "The Underground Guide to UNIX" by John Montgomery for this)
echo *
I discovered if you give the echo shell command an asterisk as a parameter, it dumps out the file names of the current directory. (The sad thing is I had a practical use for this when a less-than-clueful-collegue deleted the /bin directory, leaving the system without an ls program).
rm does that too for some reason
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
Oh fellow Heinlein reader. You deserve a hug for working a Heinlein word into a tech article.
But yes, I grok, I don't get what the huhu is with the rev command.
What was going through the creator's frimping minute!
I don't know about Unix or Lunix, but I am constantly amazed by all the complaints I hear from Unix/Lunix admins who complain about all the stuff you "can't do" with Windows... and I end up showing them how to do it.
Windows has tons of features. It's a shame they can't figure out how to read a book about it. Or, you know, do an internet search on it.
..figures you posted this as an Anonymous Coward...
A PC without Windows is like Coffee without Ketchup...
Don't know if it has been mentioned before, but using ctrl+r let's you browse your command history as you type in part of the entire command.
Building for a shallow grave Must be something else we say Somehow to defend this place
cut, uniqe(sic), tail, tr, head, tail
Ok, somebody's got to acknowledge your joke... "cut, uniq, tail, tr, head, tail" "You said tail twice" "I like tail".
Don't forget that ssh can:
1. Be used to call remote processes, not just an interactive shell
2. Accept pipes to STDIN
This allows you can tar something locally and untar it on the other side in one step. Why? If what you have to send across is huge, you don't have to consume disk space to create the local copy. Or, if you have to ssh from A-B, then B-C, then C-D just to get the huge collection of data from A-D, you don't have to wait for the tar to be created then sent 3 times. This is makes espionage super easy anywhere that you can find a path the the outside world where the router doesn't block ssh. Which most companies never think to do anyway.
I actually use it for good instead of evil though. My trick is to properly setup ssh public key authorization on a remote server in one step instead of two. Like so: .ssh/authorized_keys2 .ssh/id*.pub| ssh remoteserver "tar xf -; chmod 700 .ssh"
tar cf -
replace "remoteserver" with "$1" and put than line in a bash script and you have a simple way to deploy your public key authorization to a severname given as the first argument. When you admin lots of boxes & VMs, growing daily, you have to find shortcuts.
The only stable state is the one in which all men are equal before the
If you do not have your .ssh/config setup from ControlMaster, or know what that means... Study up! Specifying ControlMaster in my config saves me easily 45 minutes a day. (If you use svn+ssh:// there's a gotcha with a simple solution. Google for it.)
The only stable state is the one in which all men are equal before the
find and xargs is an absolutely brilliant combination. I can't live without it.
find . -type f -print0 | xargs -P 4 -0 echo grep -H WHATEVER
searches for the string WHATEVER in all regular files, weird-filename-safe. xargs runs 4 parallel greps, to make use of all those CPUs and cores! Here's two test runs, note the changing order:
$ find . -type f -print0 | xargs -P 5 -n 2 -0 echo grep a ./1 ./2 ./5 ./6 ./7 ./8 ./3 ./4 ./9 ./1 ./2 ./3 ./4 ./5 ./6 ./7 ./8 ./9
grep a
grep a
grep a
grep a
grep a
$ find . -type f -print0 | xargs -P 5 -n 2 -0 echo grep a
grep a
grep a
grep a
grep a
grep a
Comment removed based on user account deletion
Comment removed based on user account deletion
Why the fuck does it have to renew package lists EVERY time? Better yet, why isn't the package fetching bit a webapp? Think a souped up repo site or portal. Then double click on the .deb/.rpm. If you want the fast way, CLI apt-get is still there. Just my $0.03.
I know tobacco is bad for you, so I smoke weed with crack.
I use bc almost exclusively to do dec<->hex<->bin math.
ibase=10
obase=16
or:
ibase=2
obase=10
for example.
you just have to be careful because as soon as you set ibase, that's how it interprets everything else you type. so converting hex to dec looks like this, if you don't set the obase first(assuming your obase is currently 2 or something else):
ibase=16
obase=A
nice quick and dirty base conversion calculator, with a touch of history via your scrolled-up tty output.
Change is certain; progress is not obligatory.
Indeed, it incredibly sucks on SCO UNIX. Grep command has no recursive search, tar has no gzip capability, crontab doesn't have the ability to use @reboot, */5 etc.
In summary: SCO Unix sucks compared to Linux.
Change is certain; progress is not obligatory.
Additionally, if you use the -C prefix in ssh, you enable compression along the way. Which is great when working over the Internet.
Change is certain; progress is not obligatory.
"Also, Yum is good for reducing stress by conveniently giving you ten minutes to an hour of relaxation time while it completes each operation."
...to operate from cached metadata. If you need to search or look up dependencies or what package contains something, etc, works great. If you're installing something yum is no slower (or faster) than apt. And yum is a lot faster than it used to be back in the "core" days.
.. or was it apt-cache? or dpkg-reocnfigure? I can never remember ...
$yum -C [command] [parameter]
I still prefer yum over apt-get
Yes! God forbid you run an $updatedb for 8 seconds once a day. You know you can actually just put it in cron.daily and never think of it again? Not to mention you have to update your find "database" anyway. Makes no sense whatsoever.
...
Talk about a solution in search of a problem
Just use
$ source ~/.bash_profile
instead of logging out and in again. This reads the file into the current shell's environment.
for python hackers, ipython is essential. it's a wrapper for the python shell which has tab completion of all class/function/method attributes and interactive access to documentation, plus all kinds of other fancy tools. it has made learning python so much easier.
> bash :> testing :> testing
$>
$> > testing
$> csh
~>
~> > testing
Invalid null command.
This will do a dd(1) copy across the network if you don't want to drag the HD out and plug it in locally. Of course it takes a while :)
dd if=/dev/hda bs=64M | rsh ip_or_name dd of=/dev/hda bs=64M
Change the /dev/* as required.
apols if I got the syntax wrong, but you get the idea (dd might need to explicitly ref stdout/stdin; cant rem)
Oh, and it helps if the new drive is the same size (or bigger).
... but one that saves time and prevents mistakes.
/usr/local/src/{archive,test,whatever}
mkdir -p
will create all directories necessary that don't exist and then expand the stuff in brackets and create them also.
Habitually backing up files before editing them will save you a lot of heartache in the long run too. It's easier if you make an alias to replace your common editor command to automatically back up the file.
Pantek, Inc. - http://www.pantek.com/ - info@pantek.com
+1-877-LINUX-FIX - Expert Open Source Support