Slashdot Mirror
Book Review: Linux Shell Scripting Cookbook
vellorean writes "I have been reading Linux
Shell Scripting Cookbook by Sarath Lakshman, published by Packt,
for a while. While most people I know learn shell scripts themselves,
I was looking to refresh my concepts a little as well as have a
reference lying around on the table for fast access." Read below for the rest of vellorean's review.
Linux Shell Scripting Cookbook
author
Sarath Lakshman
pages
360
publisher
Packt Publishing
rating
9
reviewer
Kumar Appaiah
ISBN
1849513767
summary
A book for beginners and intermediates, which introduces shell scripting and proceeds to provide several practical real-world recipes of useful shell scripts
First of all, let me remark by saying that shell scripting is
something learned more on a need basis than as a tool to solve the
main problem. People would seldom write shell scripts as standalone
programs (exceptions exist). However, what makes shell scripting
invaluable to know is the fact that knowing some tricks can save
several minutes, or hours, of work by automating and simplifying
certain tasks, generally (but not restricted to) file management and
data processing. Linux
Shell Scripting Cookbook does go quite far in pursuing this goal,
and is appropriate for both beginners who are looking to gain
dexterity in shell scripting, as well as intermediate users who wish
to polish their skills. The book also can double up as a quick
reference, though I would argue that the "Advanced Bash Scripting
Guide" would suit that more.
At the outset, the author clarifies that the focus will be on Bash. This, people may or may not like, but the fact that bash has become ubiquitous in terms of the available shells on Unix-like systems today, starting out with bash is not a bad thing to do. Besides, learning other shell scripting languages while knowing bash isn't too hard, since the paradigm remains the same.
The book is organized into chapters based more on utility than scripting concepts themselves, although the language aspects are brought onto the reader gradually. For instance, the examples in the first chapter focus more on the basic data elements (variables, arrays, functions etc.) as well as operators (for numbers, files etc.), and all the examples demonstrate simple usage of these concepts, and he further chapters build upon these in a gradual manner.
At the same time, if he reader has some familiarity with shell scripting and needs to only refresh or learn a certain concept, he/she needs to just read the relevant chapter. It is not too difficult to grasp the examples of the later chapters, provided some basic shell knowledge is assumed.
A positive trait in the presentation of this book is that it is all based on practical everyday examples which, with minor adaptation, can be used by many for their own daily tasks. For instance, there are several examples which describe searching for and processing files, which, I'd imagine, many users would want to do on a regular basis. Thus, providing realistic examples allows the book to double its utility. The language and approach used is simple and conversational, and the presentation is very clear, with each idea being described as a problem statement followed by a "How to do it" section with the actual code, and ending with a discussion of the nitty-gritties of the code. It is easy to go for a quick scan for those in a hurry, while those who with to read in more detail will not be disappointed either.
The book also covers a wide array of applications. For instance, there are examples on automating fetching web pages and processing them, demonstrations of parsing and simplifying and even some queries around databases wrapped around in shell. It also spans to utilities and tasks connected to statistics, backups, compression, version control and many more.
The book goes into a fair amount of detail in terms of describing the shell scripting concept under consideration. The examples used go into a fair amount of detail in order to describe to the user all the aspects involved in the method or command being used. The concepts described are fairly complete, and would be sufficient for the reader to use immediately or with just a little bit of fine tuning. In terms of breadth, the book covers most of the features of shell scripting while also describing the various facilities the shell provides access to in a Unix-like environment. Thus, the book does not disappoint in this front either.
In summary, probably the only thing I'd have liked to see more of is some emphasis on how to write more efficient shell scripts. Granted, most of the shell scripts described in the book are very simple and succinct, but a some words on how loops can be made better, or how to spot situations where pipes are not needed to solve a problem etc. might have been a nice addition. Some explanation of differences with dash, tcsh, zsh etc. might also have been nice, since a lot of users have different default shells. But all this isn't going to prevent me from giving this book a high rating, since it delivers quite well on the promises it makes at the beginning.
This is definitely a good book to have near your desk, and kudos to the author for having taken the effort to put it together. I would highly recommend it to the beginner and occasional shell user for a thorough read, and to an intermediate to have on his/her desk for borrowing the cool scripting ideas and applications the author has written in this book.
You can purchase Linux Shell Scripting Cookbook from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
At the outset, the author clarifies that the focus will be on Bash. This, people may or may not like, but the fact that bash has become ubiquitous in terms of the available shells on Unix-like systems today, starting out with bash is not a bad thing to do. Besides, learning other shell scripting languages while knowing bash isn't too hard, since the paradigm remains the same.
The book is organized into chapters based more on utility than scripting concepts themselves, although the language aspects are brought onto the reader gradually. For instance, the examples in the first chapter focus more on the basic data elements (variables, arrays, functions etc.) as well as operators (for numbers, files etc.), and all the examples demonstrate simple usage of these concepts, and he further chapters build upon these in a gradual manner.
At the same time, if he reader has some familiarity with shell scripting and needs to only refresh or learn a certain concept, he/she needs to just read the relevant chapter. It is not too difficult to grasp the examples of the later chapters, provided some basic shell knowledge is assumed.
A positive trait in the presentation of this book is that it is all based on practical everyday examples which, with minor adaptation, can be used by many for their own daily tasks. For instance, there are several examples which describe searching for and processing files, which, I'd imagine, many users would want to do on a regular basis. Thus, providing realistic examples allows the book to double its utility. The language and approach used is simple and conversational, and the presentation is very clear, with each idea being described as a problem statement followed by a "How to do it" section with the actual code, and ending with a discussion of the nitty-gritties of the code. It is easy to go for a quick scan for those in a hurry, while those who with to read in more detail will not be disappointed either.
The book also covers a wide array of applications. For instance, there are examples on automating fetching web pages and processing them, demonstrations of parsing and simplifying and even some queries around databases wrapped around in shell. It also spans to utilities and tasks connected to statistics, backups, compression, version control and many more.
The book goes into a fair amount of detail in terms of describing the shell scripting concept under consideration. The examples used go into a fair amount of detail in order to describe to the user all the aspects involved in the method or command being used. The concepts described are fairly complete, and would be sufficient for the reader to use immediately or with just a little bit of fine tuning. In terms of breadth, the book covers most of the features of shell scripting while also describing the various facilities the shell provides access to in a Unix-like environment. Thus, the book does not disappoint in this front either.
In summary, probably the only thing I'd have liked to see more of is some emphasis on how to write more efficient shell scripts. Granted, most of the shell scripts described in the book are very simple and succinct, but a some words on how loops can be made better, or how to spot situations where pipes are not needed to solve a problem etc. might have been a nice addition. Some explanation of differences with dash, tcsh, zsh etc. might also have been nice, since a lot of users have different default shells. But all this isn't going to prevent me from giving this book a high rating, since it delivers quite well on the promises it makes at the beginning.
This is definitely a good book to have near your desk, and kudos to the author for having taken the effort to put it together. I would highly recommend it to the beginner and occasional shell user for a thorough read, and to an intermediate to have on his/her desk for borrowing the cool scripting ideas and applications the author has written in this book.
You can purchase Linux Shell Scripting Cookbook from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
wow. how many ways can i answer that?
- there is no one true way
- people learn a skill and want to continue using it
- concise syntax / small footprint / fast load times
- high confidence shell scripts will run on unknown systems
- can be layered on years of useful scripts
- distrust of your options for doing work
Every topic is littered with them...
ELOI, ELOI, LAMA SABACHTHANI!?
I find it useful to do shell scripting when I've been doing something by hand, and want to automate it. It's pretty easy to just echo the command I've been doing into a file and touch it up from there. Then I can take advantage of the organic growth of the command as it usually happens, and can also use it to run that same command on lots of systems if needed.
Sysadmins who can't work in plain shell scripts are dangerous. What will you do if you lose your /usr filesystem ? Be stuck with no automation tools at all as you fix by hand ? Shell scripts have little or not dependencies, and can function where other languages aren't available.
More-over they are ubiquitious. Perl, python and ruby are common in Linux, but what will you do if you're on a different Unix altogether ? A sysadmin who can use shells efficiently can easily transfer his skills to solaris, hp-ux or several others (I did just that, and now I'm paid quite a lot more than a Linux sysadmin earns as a Unix sysadmin because the skillset is rarer). Of course there are other skills you'll need to learn (package management for example is vastly different between various unixes and most are frankly primitive if you are used to Linux) but the core Unix fundamentals remain the same, and if you don't know powerful and flexible shell scripting, in my book, you're no sysadmin at all.
OT: What is it with url shortening services? At best, the damn service works and it's a blind link to what the author claimed (which is inferior to just LINKING). At worst it's goatse. Somewhere in between is someone who f-ed up using it (like parent). Can Slashdot please ban links to url shortening services?
http://www.tldp.org/LDP/abs/ is helpful -- I printed and older version of it 2-up and duplexed it, then comb-bound it. It's been very handy.
#!/bin/sh
echo "Lot of the time it's just quick and easy"
First of all, let me remark by saying that shell scripting is something learned more on a need basis than as a tool to solve the main problem. People would seldom write shell scripts as standalone programs
/etc/init.d scripts in something other than bash (or csh if on solaris, or ksh if on aix)? Granted not everyone would want, let alone try, to write 500+ line bash scripts like I occasionally do; but, there are a ridiculous number of 100+ line perl scripts that could have been done in bash or ksh in fewer lines, and with more clarity, and without the overheard of loading dozens of perl modules due to interdependencies among them.
Seriously? So you write your application's
Maybe because bash is installed by default on most Linux/UNIX systems and is portable across all yoru installed base.
perhaps they exist but in 20 years I've not see a Linux distro without Perl.
Or maybe your security requirements disallow python, or ruby or whatever to be installed.
if they disallow perl then shell needs to be disallowed too
Or that running shell scripts using many built-ins run faster than the interpreted languages?
this is patently false. Perl is vastly faster than shell even taking into account load times when you do anything significant. Bash scripts that do anthing important generally are calling things like awk and grep and sed to parse things and these invocations are dramatically slower in a loop than a single invocation of perl. Additionally, perl I/O is in many case faster than standard unix operations that move large blocks of data.
finally invoking and disposing a command like awk many times in a row makes for very poor memory management and system resource utilization compared to keeping one program like perl resident.
So don't try to argue this on speed or system resource utilization.
Bash scripting features seem to stay more stable over time vs. other scripting languages that are constantly changing.
huh? perl is very static and backward compatible.
moreover, it's the helped commands that bash needs like grep and awk and so many other system resource introspection calls that one cannot count on being present or taking exactly the ame arguments.
even a braindead command like "cp" is not only different on Linux and BSD but it's command arguments have changed over time. Whereas is perl the commands needed to copy files have not changed in 20 years.
The app teams within your company may want to constantly upgrade the installed version that might break you scripts/programs.
the only time i've seen broken perl was the 5.8 to 5.10 switch changed the timing of a few things.
And bash is dirt simple to learn and implement for non-programmers.
Perl is very very very close to bash in syntax
Some drink at the fountain of knowledge. Others just gargle.
So the bash fork bomb got popular back while I was at UNH. If you're not familiar with it, it's similar to this: ;(){ :|:& };: (I changed one character so that you dont paste it on recommendation) The way it works is trivial, and it's mystique is only in that it uses punctuation marks in lieu of letters for its own name: make a new function called :, run it and force the output into a new fork of itself in the background; then a final call to the new function. h4rdc0r3 1337.
So, we had this UNH policy that if you hung a shared unix dev box, you got evicted from CEPS (the college for the engineering/cs kids). They could give you a warning or go straight to evict. Their call.
I wrote to their head unix admin once, and with some humility, cause I knew they knew this, pointed out that they could just change the system's ulimit values to disallow casual fork bombing. The default system ulimits on the distro back then (still?) allowed a single user to consume all memory. So we saw a couple kids run the fork bomb and they never actually got evicted, cause maybe 40 grand tuition outvotes a cranky unix admin's wanton lust for cruelty.
Anyways, as a student, my issue was this - set the damned ulimit so that my editor session doesn't get wiped with my homework -- you can't argue for frequent backups when more than 0 seconds of hard work are arbitrarily on the line solved by a simple config change.
But the UNH unix admin guy replied to me and told me that yeah, he knew about ulimit back when I was in diapers, but he wasn't going to change it up. And a few more times that semester, I lost a couple lines of uncommitted code due to some clown pressing enter on a dare.
Fork ya later,
slashdot: where everyone yells sarcastic metaphors to themselves to understand the issue
You phrase the question differently than I would. I would ask why is perl not the default shell language.
Because Perl is huge and sh is tiny. Would you be able to port Perl to BusyBox without dramatically increasing the size of the executable?
Give me Classic Slashdot or give me death!
If I stick it in root's cron on, say, an AIX system and it pukes I'm not interested in rewriting somebody's bash script to handle running under sh, csh or ksh. I want it written by its original author in sh. And I don't want to hear about a shebang line (#! /bin/bash) - if I haven't installed bash, it'll still puke.
I take option 3, not working there.
Python could be nice, if it ignored whitespace/indentation. Also if it had its own version of CPAN.
The script receiving stdout from telnet would watch for input lines, keeping track of what response was expected, and send either a USR1 or a USR2 back to the script outputting commands into telnet's stdin, which would then either bail or send the next command. It was rudimentary pass/fail error checking, but much better and faster than echo a; sleep 1; echo b; sleep 1; echo c...
I often prefer #!/bin/sh as I know it's a common denominator on all *nix systems, and often my scripts work even if the shell is busybox. Yes, perl or python may handle the task more efficiently and with less code, but keeping fresh on Bourne shell makes it easier to fiddle with initramfs images and embedded systems with limited space and no advanced shells. In fact, I'll often stick with it just for the challenge. I like making the original Bourne shell truly sing...
Open your eyes. There's much more to the world than just "every major (and most minor) distros." Even more than just Linux and the BSDs.
Crikey. If nobody learned assembly, we wouldn't even have operating systems.
In particular Bourne shell makes gluing other programs together far easier, cleaner, and more reliably then can be done with any of the languages you list...and I say that even given that I'm extremely fluent in at least two of them.
Correctly and reliably handling even a simple "foo | bar | baz" construct in the languages you list can be done, but you're talking about an LOT of non-trivial systems level programming to do it, effectively "coding C inside perl/python/ruby". No, really. Sure, it's very easy to do it wrong and fragile (just call system(), what's so hard right?), but if you actually want to handle it as correctly and reliably as Bourne does out of the box...it's going to take a hell of a lot more code and detailed systems programming knowledge then even most "Sr" Unix sysadmins tend to have. Almost without fail ever attempted I've ever seen in the industry gets it wrong...typically very, very wrong.
Bourne is built to run other programs, to manage the interactions between them, programming logic being the exceptional use. All of the languages you list are built to be self-contained, programming logic being the primary task and communicating with external programs a very secondary use.
Now of course...if your script/program is self-contained and not just wrapping a bunch of exec()s of other programs, then sure Bourne shell is one of the least favorable options.
There's also nothing saying you can't freely mix and match. Most anyone worth their salt does so all the time. To do otherwise would be like saying a web coder could only work in HTML or only work in JavaScript or only work in PHP. We're talking about complimentary languages here, with some overlap.
Use the right tool for the job.
My
I've just had a look through the book via Amazon's Look Inside.
I simply cannot fathom how so many authors manage to get a book published on a subject they don't even master properly. "Shell scripting" is by far the worst in this category. So much that I recommend anyone to run screaming in the other direction when you see mention of "shell scripting" in any title. If it mentions "Linux", it's usually also an excellent indicator of junk. Not to take away from the beauty of the Linux kernel (which obviously has nothing at all to do with shell scripting, but let's ignore that fact).
I obviously haven't read the book, but when I skim through it and I see failure to quote parameters, failure to recognize the difference between executing a script with /bin/bash in the hashbang and running it by passing it as an argument to sh, and more, I know for a fact that I'm going to be sitting in #bash trying to re-educate poor misguided souls for as long as I can suffer it.
To those interested in the subject: Do not read this book. Do not read this "Advanced Bash Scripting Guide" mentioned in this review. You will end up writing junk.
Go see Greg Wooledge's wiki, it's got an extensive FAQ of actually useful issues (http://mywiki.wooledge.org/BashFAQ), it's got a great newbie guide (http://mywiki.wooledge.org/BashGuide), it's got a great cheat sheet (http://mywiki.wooledge.org/BashSheet), and much more. Best of all: Unlike that broken ABS, they're all community supported and unlike this book, the knowledge is free.
``OK, so ten out of ten for style, but minus several million for good thinking, yeah?''
For me its usually;
- too lazy to learn another language or...
- not enough time to learn another language
Perhaps because perl's about an order of magnitude larger:
# quickpkg bash perl /usr/portage/packages/app-shells/bash-4.1_p9.tbz2 /usr/portage/packages/dev-lang/perl-5.12.2-r6.tbz2 /usr/portage/packages/app-shells/bash-4.1_p9.tbz2 /usr/portage/packages/dev-lang/perl-5.12.2-r6.tbz2
# du
1120
12372
20 January 2017: the End of an Error.
OK, how much RAM does your router have? Your cable modem? A small NAS device? Your concept of embedded devices may be only a small fraction of what's out there.
Even my phone has 256MB of ram, and its replacement will have at least 1GB. Remind me again why this is a problem?
What is up with embedded devices having less than 64MBs of ram?
What is up with simply using the correct tools instead of just throwing more memory at the problem? I mean, embedded devices are often bought in large quantities and thus the less they cost the better. So, if you can drop the cost with more intelligent software choices then that should be more than enough of an answer to you.
Besides, I feel the whole "lets just throw in more memory so we don't have to think about our choices!" a very poor approach to computing, embedded or otherwise.