Slashdot Mirror
How To Adopt 10 'Good' Unix Habits
An anonymous reader writes to mention an article at the IBM site from earlier this week, which purports to offer good Unix 'habits' to learn. The ten simple suggestions may be common sense to the seasoned admin, but users with less experience may find some helpful hints here. From the article: "Quote variables with caution - Always be careful with shell expansion and variable names. It is generally a good idea to enclose variable calls in double quotation marks, unless you have a good reason not to. Similarly, if you are directly following a variable name with alphanumeric text, be sure also to enclose the variable name in square brackets ([]) to distinguish it from the surrounding text. Otherwise, the shell interprets the trailing text as part of your variable name -- and most likely returns a null value."
Join moola.com, play games to earn money.
The quoted paragraph from the article is incorrect -- and it is in the article too -- but the example immediately following it correctly shows the use of braces ("curly brackets"), not square brackets, for variable names in shell.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
...is so littered with basic errors that it really shouldn't be recommended to anybody. How is 'tar xvf -C tmp/a/b/c newarc.tar.gz' expected to work, for example? Quote variables with square brackets? Running subshell commands using ; instead of && ? No mention of 'xargs -0' ? Don't pipe from cat to grep? Does anybody actually care that people do this (primarily so that the syntax is consistent between a munged- and unmunged-grep, and also such that the order of the command-line is logical from a human point of view)? Plus, of course, it's possible that cat | grep could yield better performance than grep alone: if cat uses mmap() to efficiently read the input files, and the kernel's pipe implementation is good, then it could do better than a grep implementation alone that simply read()s the files.
Comment removed based on user account deletion
*nix is a highly modular component-based software system with a standard interface (flat byte streams) between components, and a basic set of standard components (given in the POSIX standard) that can be relied upon to always be present.
This code is not pure shellscript : it uses awk and wget to get the job done...
A Python equivalent might be :
It's not that much longer, it's much easier to read and less error-prone (especially the awk part), and it uses fewer external utilities.
To me, the *only* advantage of shellscript is that it's the only language that you are sure to find on any Unix system.
"10 good habits that improve your UNIX command line efficiency" would probably have been a better title.
The title did however bring back fond memories of Eric Raymond's The Art of Unix Programming. The book is available online, and if you were hoping for something a bit more substantial as well, then the section Basics of the Unix Philosophy might be worth a read.
Um, whats wrong with
wget -i filename
Or have I missed something?
Monkeyboi
In their example with tar they did
tar xvf
without the dash. (E.g., tar -xvf)
While that does work, I prefer to add
the dash as it makes it more consistent
with the other commands. So I consider
that a bad example. tar is one of the
older commands like dd that have weird
command line syntax.
As has been pointed out, this article is riddled with errors. It's also not very interesting. So in the interest of perhaps actually providing some interesting tips:
/).
/etc/modules.autoload.d/kernel-2.6 /etc/m*a*d/*6
/etc/m* -> /etc/mail /etc/m*d -> /etc/modules.d /*/m*/*6 -> /etc/modules.autoload.d/kernel-2.6, and /etc/modules.d/i386 (not quite!)
In scripts, prefix dangersous commands with an 'echo' for a test run (So you can catch all those rm -rf
Single quotes are the best quotes for plain strings. The only reasion to use double quotes is if you need to quote a variable or a single quote.
Completion is fun, but using wildcards is more flexible (though you'll only want to use benign commans like cd, less, etc):
nano
nano
Note that the use of subpaths reduces the amount of flexibility.
cd
cd
nano
Finally, as a comment for the article, using:
test -e $DIR || mkdir -p $DIR
is much better than their suggestion and probaly faster anyway. Though I'd just do "mkdir -p $DIR" and maybe "&>/dev/null" under most circumstances anyway.
That's all I can think of at this point. Anyone else have tips?
When timing commands, it's best to repeat the command several times and see if the times change significantly.
for i in *.JPG ; do mv $i `basename $i .JPG`.jpg ; done
cat-ing a file and then piping it to grep. surely that is a good point he is making, because grep already takes filenames as an argument?
. or tar cf foo.tar *. You will piss yourself and others by doing that.
That list was fairly arbitrary, but the piping cat thing is something that basically only annoys the most anal of anal, and they probably do it sometimes too.
Its common for me to do cat foo and then hit the up arrow and append a pipe to another command instead of editing the whole command line. Computers are pretty fast, and real anal people would use fgrep instead of grep, but again I always use egrep, because I never know when a regular expression will be edited into a more complex one, and to me all of the speeds are the same.
My #1 habit to tell people, although it is not a habit, but just where to start it to learn your shell. No science guys, csh is not a worthy shell in 2006. If you have to suffer with the wacky behavior of a csh variant, at least use tcsh.
My #2 thing to learn is a text editor.
As far as habits go. First and foremost, unalias cp, mv, rm to have the -i flag. In my opinion, that is a BAD habit to start. You WILL lose files sooner or later, and the more painful the better so that you will think so you will stop doing it. the -i flag will NOT stop you from redirecting into a file, and the most dangerous is the -rf flag with rm will override that -i. Remote copies via rcp or scp will not honor the -i flag. Unarchiving an archive will not honor the -i flag. There are tons of ways to lose files, and you will lose them. Its a much better habit to universally save yourself from yourself to not lose them by testing with -i, working off of a copy, and thinking before you hit return, creating new directories to eliminate clobbering a file, NEVER, EVER, do tar cf foo.tar
Actually, this top 10 list is pretty lame, and should be ignored.
I don't think it ever makes sense to use cat with one file - something I have seen far too many people do. To do so, logically, is to tell the commands to run through the file twice.
First you are telling cat to output the entire file, and then you are telling grep to go through the entire output of cat. If you're working with gigabytes of data here, that can quickly be a frustrating exercise! Folks who are in the mentality of using cut | grep and even a visual editor like vi instead of sed are up the creek when they find themselves needing to manipulate and get portions of very large data sets.
-bugg
Why is the perl script so hard? The command line would have a bunch of sed going on, whereas the perl script only requires running perl.
I'm guessing... perl -e 'for(`ls`) { chomp; $n = lc $_; system("mv $_ $n"); }'
-- The act of censorship is always worse than whatever is being censored. Always.
a.txt: foo
b.txt: foo
c.txt: foo
whereas cat ?.txt|grep foo produces
foo
foo
foo
I've also seen Unixes where their shells are linked against spectacularly broken libc's. Under Tru64's Bourne and Korn shells, for example, a multithreaded program foo fork bombs when run as foo < z.txt, but works fine as cat z.txt|foo (foo < z.txt under Bash works, though, because Bash is linked against the GNU libc).
This is not my sandwich.
Oh, for the love of God, stop bitching about how hard this is to do under UNIX without package x.y.z installed and imagine doing it under Mac OS9 or Windows.
... should do the trick. If you don't have tr, you can use sed with the y command. But I can't think of the last time I saw a box w/o tr.
Break it down into its constituent parts: Iterate, rename. Whoo! Simple now, eh?
# cd cameraDir
# find . -type f -prune | while read file
> do
> mv "$file" "`echo \"$filename\" | tr '[A-Z]' '[a-z]'`"
> done
#
Do daemons dream of electric sleep()?
oh, I ran into this problem too many times so I set up the following function:
function pmv ()
{
local src dest oifs
oifs=$IFS
export IFS=$'\n'
src="${1:? 'Error: input pattern not specified'}"
dest="${2:? 'Error: destination pattern not specified'}"
for i in $src ; do
mv "$i" $(sed -e "s/$dest/" <<< "$i")
done
export IFS=$oifs
}
Now I can move files by regexps by typing pmv <file-match-pattern> <sed-matcher>/<replacement>
very very convenient.
xargs can split up the args to execute bar multiple times.
bar `foo` can fail if `foo` happens to make the argument list too large.
Also use find -print0 | xargs -0 instead of find -exec if you prefer not forking a jillion times.
Actually it will work with Korn shell as well, and probably zsh too. Not to mention that many systems have a /bin/sh which is Bourne-compatible but enhanced. Many systems have a /bin/sh implementation that supports this, not just bash-based Linux systems.
-Lasse
$ for i in *JPG ; do mv $i ${i/JPG/jpg} ; done
.jpg at the end, but ${i%.JPG}.jpg would be best.
/bin/sh in mind, and should as a rule be as simple, clean and efficient as possible.
This isn't the first time I've seen this, but it will result in a file MYJPG.JPG being called MYjpg.JPG, ${i//JPG/jpg} would be better as at least the it would end up with the
Again, there's lots of ways to do it. To use the trivial JPG -> jpg, example, yes, you're correct in that using the shortest match at the end would be a better approach (excluding other issues). I just wanted to illustrate the redundant (and typically overused) use of basename with a simple example, and remind the folks that using parameter expansion is preferrable both in interactive form, and in scripts.
Me, I've always relied on Larry Wall's script exclusively to rename files interactively. Scripts, on the hand, are often best written with
Proprietary? From the man page:
Program Intellivision!
Great, IBM, way to ignore the dreaded "xargs" security bug! Seriously, IBM notices some kind of obscure danger about underscores, but completely ignores the fact that xargs separates arguments by newlines??
/tmp directory by a certain user for some reason: /tmp -user 1001 | xargs rm
/tmp called "haxor\n". Inside there he puts another directory "etc" and inside there he puts a file called "passwd."
/tmp/tmp43cc91 /tmp/haxor /tmp/haxor /etc/passwd
e tc/passwd"]
/tmp -user 1001 -print0 | xargs -0 rm
Let's say I'm a sysadmin and I'm running as root, trying to remove all the files in the
find
User 1001 has a directory in
Can you guess what happens?
find prints:
xargs sees: ["/tmp/tmp43cc91","/tmp/haxor","","/tmp/haxor","/
Oops!! You just hosed your system!
The correct way to use xargs is to use the -0 switch, which will separate the input by null characters, which cannot appear in filenames. find has a handy -print0 option which will output the correct output:
find
And your system is safe.
Did you ever notice that *nix doesn't even cover Linux?