Unix Shell Programming, Third Edition

honestpuck writes "Back when dinosaurs roamed the earth and NCR made Unix computers I first started to program for a living. Back then when someone said 'script' they meant a shell script, generally for a Bourne shell." Even if the definition of "scripting" has grown somewhat, honestpuck argues, the old meaning still has merit and use. Read on for his review of the latest edition of Unix Shell Programming. Unix Shell Programming, Third Edition author Stephen G. Kochan and Patrick Wood pages 406 publisher SAMS rating 8 - Well written, good topic coverage, some small flaws reviewer Tony Williams ISBN 0672324903 summary Good introduction to shell programming and using the shell

Now that we have languages such as Perl and Python, much of shell scripting has been forgotten. The need still arises for the times and places where running Perl would be just that little bit too much overhead; cron jobs, process start and stop scripts, even machine start and stop scripts. For these we could best go back to the old ways. Combining the power of the common Unix tools, pipes and scripts in a fairly obscure and slightly arcane syntax is not easy to pick up, though the language's simplicity does, in some ways, make it easier than more complex ones such as Perl. This book does a good job at introducing shell programming and I found it an excellent book when I needed a refresher.

I don't want to sell this volume short: you won't just learn about shell programming. The first ninety or so pages provide an excellent guide to getting the best out of the shell, and the last chapter is devoted to the features specific to an interactive shell such as command-line editing and using the history.

The authors have chosen to use the POSIX standard Bourne shell ('bash', available on many *nix systems, is a superset of the POSIX standard). That seems the right decision, given that it is so universally available and usually the default shell.

The book is well structured, starting out with a brief look at *nix operating systems before introducing the shell followed by some basic tools; cut, paste, sed, tr, grep, sort and uniq. One minor quibble, the book explains how to redirect STDOUT to a file and STDERR to a file, but not how to redirect both to the same file. That aside, these few chapters provide a good introduction to the shell.

The text goes on to systematically explore shell programming starting with variables and arithmetic. The chapters are kept short, in a good order and have a number of exercises at the end of each. The structure of the book and the order each new concept is introduced is well thought out; at each stage small examples are given that only use material already introduced and are complete in performing a task. In early chapters they are fairly trivial but by the end there is a fairly complete rolodex program written in shell script that would be a good model for anything you wished to do.

There is also a good summary of the shell syntax and common commands in Appendix A and good 'Further Information' in Appendix B. Kudos must go to the authors for a list of books for further reading that is not ashamed of mentioning other publishers, indeed they say "One of the best sources of books on Unix-related topics is O'Reilly and Associates" and list volumes from them before mentioning their own publishers.

There are some small typographic errors in the text but I did not find any in the script examples I tried. I found it to be well written and readable throughout, perhaps an advantage of a third edition in a slow moving technology.

You can visit the Sams web page devoted to the book which has the Table of Contents and the third chapter available for download. It has no errata or source code, I looked to see if the authors maintained a site for the book but could not find one.

I would recommend everyone read this book once or twice, it provides a comprehensive, well written tutorial on one of the most basic (and often overlooked) tools at your disposal. Even Windows users could install Cygwin and gain the benefit of a good POSIX compliant shell and this book. It also has the advantage that once purchased it will be useful for many, many years to come - the language has not changed noticeably in twenty five years and should not change in another twenty five.

You can purchase Unix Shell Programming, Third Edition from bn.com. Slashdot welcomes readers' book reviews -- to submit a review for consideration, read the book review guidelines, then visit the submission page.

Comments

Shell scripting is a Lost Art

[Saeed al-Sahaf]

Shell scripting is a Lost Art. You know the True Gurus in the IT department by how comfortable they are with shell scripting. And of course, in the world of Microsoft, now one knows what you're talking about when you talk about scripting, they assume VBScript (the language of virii).

Re:Shell scripting is a Lost Art

[MoonFog]

I wouldn't call it a lost art, but with all the advancements in GUI, it certainly doesn't get used as much as it should.
I find it to be a very nice tool to do simple jobs, without having to write a program to do it.

If I'm not mistaken Longhorn (next Windows) will even have a shell like interface (something other than the command prompt), so perhaps shell scripting will have some sort of a ressurection in the following years.

Re:Shell scripting is a Lost Art

[rmdyer]

In my mind shell scripting sits at the highest level of process control. You are basically working above all other methods to control your machine with a thin and light interface. In shell scripts, you typically start and stop processes, check for active processes, log information about processes, control the flow of or filter process data. You also work directly with files on the file system by checking for them, moving them, renaming them, deleting them, appending to them, etc. In short, shell scripting is exactly what it says it is...scripting what you would otherwise do by hand at the shell prompt.

Shell scripting usually trumps all other programatic languages in that the engine is so small. That allows it to start up and shutdown really quick, not like the bloated Perls, Pythons, Java's, VBs, etc. A typical shell script engine might use about a Meg or less. Shell scripting is not programming in the strict sense of the word. You are not working in a well structured and cohesive language to be used for a variety of application development. No, shell scripting was made primarily for process and file system management. Making shell scripting work much beyond those means is an art, and not something that should ordinarily be done because other better languages should be used to fulfill broader purposes.

In our Windows environment we use CMD.EXE, or the command shell engine for just about all of our network control scripting. It comes with Windows, and is the only standardized default shell console. The command shell of NT/2k/XP is NOT your fathers BATCH language. Microsoft made some extensive professional changes to the command shell engine starting way back with Windows NT 3.51. The command shell is now very functional and has many Bourne shell syntax features.

Microsoft hasn't promoted the use of the command shell until very recently. This is because early on, they wanted to push system programmers and admins into using their GUI tools and VB scripting languages so that you would have to buy other products. Microsoft really pushed people away from using command shell scripting for the purpose of their own gain. Recently, with the gowing popularity of Linux and Unix again, they are now pushing to get people to use the command shell. They've finally documented most of the many changes they've made to the shell over the years, those changes that were kept in secret to all but the most diehard shell users.

The most popular changes come with the following commands...

CALL, GOTO, EXIT, FOR, and SET.

These commands have been expanded for the development of subroutines, string manipulation, math functions, tokenizing, file system recursion and enumeration. The command processor is fairly powerful in its own right because you don't have to exit very much for externals like FINDSTR, or SED/AWK type executables.

Under NT/2k/XP if you are using the ".BAT" extension for your scripts, or the "COMMAND.COM" processor, then don't. These are now outdated. If you fire up command.com, the the 2k kernel must startup the 16 bit virtual dos machine (VDM). This is S-L-O-W, and the BATCH language is indeed very poor. You should now use the ".CMD" extension for your command shell scripting.

See the following for more command shell (cmd.exe) documentation...

Command Shell Docs

Default shell?

[christurkel]

As the risk of being flamebait here, Bash is not the default shell on *BSDs (Is this the reason its dying?), its tsch. Neither is it the deault shell on QNX (ksh is). Perhaps the statement that bash is the default shell on most system seems overly broad.

Re:Default shell?

[morelife]

The csh/tcsh default in freebsd is a mystery.

When bash is ultimately so much more feature ridden than tcsh/ksh. Ok, just my observation/opinion.

But since bash is the default shell in Linux, and there are probably more linux systems than any other unix-like platform on planet earth, I vote that bash become earth's default shell.

Windows' default shell can probably be considered either Symantec or McAfee Anti Virus.

Re:Default shell?

[Meowing]

Well, csh is the Berkeley shell, and with FreeBSD being based on BSD and all, it's just one of those things you kind of expect to see....

Shell scripting is great, even in Windows!

[simetra]

I use shell scripts for many things under Windows using the free version of many Unix utilities found HERE. Scripts are fast, and don't break (short of being over-written). Many times, scripts can be written that perform the functionality of expensive Windows programs, with the advantages of being free, not easily broken, and not subject to dll version issues, continual upgrades, etc, etc, etc.

Shell Scripts are really The Cat's Pajamas.

Oldie, but goodie differences ...

[pgrote]

When I first started learning about the concept of shells, I was moving from the MSDOS world.

One of the first things I remember reading was the FAQ describing the shell differences.

Imagine my surprise that the command line could hold so much control and there were other aspects of controlling a PC. I haven't looked back since, but that page really brings back some memories.

It's funny how something you read can cause one of those lightbulb moments.

Or, for the free speech *and* beer people

[Zocalo]

The LDP includes the "Advanced Bash-Scripting Guide" which is available in both HTML and PDF (1MB!) formats, and there is even a version for PalmOS! Since I found it, it's been the only reference I've *ever* needed when I've stumbled over some obscure nuance of shell scripting.

I did some work for him on this book

[Jack Wegnar]

Patrick's research staff contacted me two years ago and I did some minor work for him, via email (I never met him so don't go all crazy and ask for details) and was paid very well, considering it was research for a book.

Patrick's a pretty sharp guy but he outsources a lot of his research to proffesionals (makes sense) and has several staff people help him put the pieces together, as it were.

I offered my services as part of the FTEST (final tech editing service team) but Patrick didn't want a publishing pundit as much as he was looking for computer pundits. Ah well, at least now I'm in his rolodex and hopefully I'll get more chances to work with him.

Warmest regards,
--Jack

Download scripts and errata here

[Helevius]

You can download their scripts and fixes for the text at http://www.kochan-wood.com/.

The one Amazon.com review gave this book 4 stars.

Helevius

Shell scripts

[RealProgrammer]

I make my living as a Unix sysadmin, to support my hobby as a Real Programmer :-p.

Without shell scripts, I'd be lost.

Shell scripts provide the ability to leverage the work of every well-behaved perl (etc.) script, binary in /bin, and other shell script ever written. While the same can be said for perl scripts (which I happily use or write when needed), with shell scripting this is the intended paradigm.

Scripts are intrinsically open source, even if copyrighted and under a closed license. The techniques are visible, and thus instructive.

Given Unix's design attribute of easy process creation, shell scripting is often the best way for me to handle a task.

See some examples here.

Really the better one is this one:

[funwithBSD]

UNIX(r) Shell Programming, 4th Edition


Of the same name, not related. Go figure. I got both and the one above is the better of the two, IMHO. Chapter 5 of this one is golden, lots of simple tables covering built in operaters, varibles, and control structures.

I really did not GET programming until the book above, it made it click for me. Perl came right after, as this got the concepts in my thick head.

A shell-less admin is a useless as a mermaid, and not as pretty to look at.

Re:Shell scripting is deprecated

[ChristTrekker]

The shell is universally available. On some low-end systems (yes they still exist), Python and even Perl might not be installed.