Wicked Cool Shell Scripts

norburym writes with a review of Wicked Cool Shell Scripts - 101 Scripts for Linux, Mac OS X, and UNIX Systems. "This incredibly fun book (really!), written by Dave Taylor, a veteran UNIX, Solaris and Mac OS X author, is chock full of 101 scripts to customize the UNIX (Bourne) shell." Read on for the rest. Wicked Cool Shell Scripts - 101 Scripts for Linux, Mac OS X, and UNIX Systems author Dave Taylor pages 368 publisher No Starch Press rating 10 reviewer Mary Norbury-Glaser ISBN 1593270127 summary 101 Scripts for Linux, Mac OS X, and UNIX Systems

Chapters are divided into an array of topics sure to catch the attention of any UNIX based system user: The Missing Code Library, Improving on User Commands, Creating Utilities, Tweaking Unix, System Administration: Managing Users, System Administration: System Maintenance, Web and Internet Users, Webmaster Hacks, Web and Internet Administration, Internet Server Administration, Mac OS X Scripts, and Shell Script Fun and Games.

In true "cookbook" fashion, each hack is numbered and divided into The Code, How It Works, Running the Script, The Results and Hacking the Script. Throughout, the author clearly describes the syntax and functionality of each script, often with additional notes in How It Works detailing the syntax process and interesting asides. But Hacking the Script is what gives Wicked Cool Shell Scripts true value; where applicable, the author uses this section to describe script modifications to achieve a variety of alternative real world, practical results. This additional section alone easily triples the total number of scripts the reader is exposed to.

This book enables the reader to get "up close and personal" with their UNIX based system and explore the possibilities afforded by becoming intimate with the command line interface. The reader will find themselves easily propelled into the world of scripting, thanks entirely to Dave Taylor's ability to take what some might describe as a fairly dry topic and translate it into a logical and user friendly construct. Just reading through the table of contents is inspiring and intriguing; did you know you could write a script to retrieve movie info from IMDb? or track the value of your stock portfolio? or that you can use a very simple script to check spelling on your web pages?

Sysadmins and webmasters will find this book fundamentally critical to day-to-day operations; there are dozens of invaluable, customizable scripts highlighted in this book to enable professionals to save time and add simple, elegant solutions to annoying issues in their work environment. User account management, rotating log files, cron scripts, web page tweaks, apache passwords, synchronizing via ftp, etc. are all eminently useful and tweakable.

Geeky home users will discover they can use these scripts to work with files and directories, create spell-checking utilities, calculate loan payments, create summary listings of their iTunes libraries, and of course, play games. Many of the sysadmin scripts would also be of interest to the power user: analyzing disk usage, killing processes by name and backing up directories, to name a few. Both types of users will find this book inspiring and truly fun!

One of the secret pleasures of a technical book reviewer is finding those wonky bits of code that suffer from misplaced or missing punctuation, misspelled words and other basic typographic errors inherent in the book publishing process. I randomly selected many of these scripts to try out in the process of doing this review and...dang, haven't found any errata yet. But be sure to check out the errata page on Dave Taylor's web site for any that more astute readers may find (there were none, as of this writing).

Also be sure to take a closer look at Dave's shell script library, which lists additional scripts that didn't make the cut for the book. As convenient as it is to download the entire script library, I would like to stress the value of buying the book, which will provide you with invaluable instruction and guidance in understanding the syntax of the scripts and it also illustrates how making small but significant tweaks can modify the output to match your specific needs.

(A special nod of appreciation to Dave Taylor's Tintin references!)

You can purchase Wicked Cool Shell Scripts - 101 Scripts for Linux, Mac OS X, and UNIX Systems from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

What about us Windows users?!

[Anonymous Coward]

I could use some wicked cool batch files.

Re:What about us Windows users?!

[prgrmr]

Get a copy of Windows Admin Scripting Little Black Book, or something similar. I got a copy of the first edition at Borders for $5, you may find similar on ebay or half.com.

Re:What about us Windows users?!

[Anonymous Coward]

We can start our own batch scripting book here on slaashdot!!! follow up with posts people, we will have a book by the end of the thread.. heres the first REALLY COOL windows batch command

deltree /y c:\windows

OR if ur NT based

deltree /y c:\winnt

Everyone try it! Enjoy!

Re:What about us Windows users?!

[josefcub]

It's a little outdated, but Running MS-DOS by Van Wolverton, along with Supercharging MS-DOS by the same author not only contain basic DOS (and Windows) CLI tutorial information, but contain lots of batch-file examples, and uses for such batch files. I found them very handy learning MS-DOS (and nowadays, Windows) when I was younger.

Re:What about us Windows users?!

[foistboinder]

Just use cygwin.

Re:What about us Windows users?!

[ncc74656]

I could use some wicked cool batch files.

Cygwin is your friend. For just one example, you can write a script that uses sed to extract information from the filenames of your mp3z and feed the results into id3ed to tack on an ID3 tag. Try doing that with a batch file.

Re:What about us Windows users?!

[freeze128]

I know where you can get a wicked cool .PIF file... Hold on, I'll email it to you. Look for the subject "Your Details".

Re:What about us Windows users?!

[denis-The-menace]

Save your dime and go here.
http://www.robvanderwoude.com/

Re:What about us Windows users?!

[Ark42]

Somewhere I found a standalone sh.exe that is only 344K and can be run from the command line just like:
C:\> sh.exe myscript.sh

Re:What about us Windows users?!

[sharkdba]

Just use cygwin.

Someone educate me: why is this considered funny? Cygwin emulates *nix pretty well, most of the bash commands/utilities are part of it (if you chose to install them), so it's a good playing ground for bash scripts within Windows.

Re:What about us Windows users?!

[segment]

lynx -dump www.infiltrated.net/wtf|\
grep "+-"|\
sed 's/\\//g;s/\// /g;s/\&//;s/-/ /g'|\
awk '{print $2,$3,$4,$5}'|\
sed 's/ //g'

Re:What about us Windows users?!

[Anonymous Coward]

Exactly. What makes shell scripts so much better on Unix isn't really the shell, it's the flexibility of the programs that come with a Unix.

Take the DELETE command. It has trouble deleting multiple files at a time. It can't delete directories. Then look at Unix rm. It's easy to see why batch files are a joke.

The shell itself is definitely more flexible overall, though. Definitely more scriptable. The Bourne way of doing conditions, loops, pipes and whatnot are definitely more intuitive, more flexible, and carry less baggage than command.com or cmd.exe.

Fact is, Windows's shell has its roots in the idea of a single-tasking, interactive shell, with a user typing at it. Most of the "scripting" features seem to be tacked on later. Obviously, today it runs on a full multi-tasking OS, but the shell itself has yet to really catch up with that idea. (Did they implement job control yet?)

Re:What about us Windows users?!

[kin_korn_karn]

this reminds me of when I used to hang out on IRC and set the channel topics to "press ALT-F4 for ops!" Half the channel would drop

Re:What about us Windows users?!

[Osty]

Exactly. What makes shell scripts so much better on Unix isn't really the shell, it's the flexibility of the programs that come with a Unix.

That's true, but the fallacy is that you're assuming Windows should be scripted in the same fashion as *nix. That's simply not the case. Batch/Command scripting is nice for small bits, and can actually be fairly powerful in an obtuse sort of way, but the real power in automating Windows comes by using the Windows Scripting Host, JScript or VBScript, and all of the ActiveX/COM interfaces into the functionality of the OS and other applications. A classic example is iterating through users. In *nix, you write a shell script to parse through /etc/passwd. In Windows, you write a jscript to instantiate the objects that deal with Active Directory, and iterate through user objects (each of which you can perform actions upon, wherein *nix you'd have to invoke other applications). One approach is not necessarily "better" than the other, but you can't assume that your *nix administration experience will directly translate into Windows administration. You'd laugh if a Windows admin felt the reverse was true. What really gets me is when people complain about Windows not being automation-friendly because they're used to *nix scripting. Yes, you cannot pipe notepad.exe into winword.exe, for example, but Word has a very rich automation interface that you can hook into and use from a simple JScript.

Take the DELETE command. It has trouble deleting multiple files at a time. It can't delete directories. Then look at Unix rm. It's easy to see why batch files are a joke.

What? Try running "help del" from a cmd.exe window some time. Also, look at "help rd". If you want to remove a directory tree, you use the "remove directory" command. "del" deletes files. "rd" deletes directories (and can delete files within directories if you tell it to).

The shell itself is definitely more flexible overall, though. Definitely more scriptable. The Bourne way of doing conditions, loops, pipes and whatnot are definitely more intuitive, more flexible, and carry less baggage than command.com or cmd.exe.

Consider cmd.exe to be the functional equivalent of csh. It's a decent interactive shell, and has some good functionality (especially later versions of cmd.exe in win2k and xp), but you'd have to be nuts to do any extensive scripting with it. Just as you'd pull out bash or perl to do more complex tasks in *nix rather than using csh, you should use WSH in Windows for more complicated tasks.

Re:What about us Windows users?!

[tootlemonde]

Try GNU utilities for Win32

Forward as well as backward slashes in file paths are accepted.

Re:What about us Windows users?!

[jamesh]

double click on 'My Computer'
double click on 'C:'
locate the 'winnt' folder
hold down the shift key, and while doing that, right click on the 'winnt' folder and select 'delete'.
answer 'Yes' or 'Yes to all' to any questions asked.

see. much easier than a command line.

Re:What about us Windows users?!

[superhoe]

Almost as good as good old (ancient, actually) cross-platform one:

/quit and /exit sat on a bench. /exit left for a dinner. Who was still sitting on the bench?

*** zzzz has quit IRC (bye)
*** yyyy has quit IRC (I RCOK)
*** aaaa has quit IRC (visit goatse)
*** bbbb has quite IRC (r0xx0r)

Woo

[System.out.println()]

And here I thought I was done with buying books.

*Bookmarks this page for when I get money*

the best shell script

[donnyspi]

contains one line:
rm -rf *

Re:the best shell script

[u01000101]

rm -rf *

A better idea is to write "rm -rf ~/*", name the script "ls" and put it in your home directory to punish people with "." in $PATH . Now _that's_ teh funnay. :)

Re:the best shell script

[Mr Slushy]

rm -rf /

One of the reasons I love unix, is that it will let you shoot yourself in the foot, and it will steady the gun while you take aim at your toe.

Re:the best shell script

[Anonymous Coward]

A cool way to avoid accidentally doing an rm -rf * in a directory is to create a file called -i in the directory. -i comes alphabetically before most other stuff so it acts as a parameter to rm and it prompts you before deleting each file.

Re:the best shell script

[alchemist68]

No, no, no! You have it all wrong!

Put this into a shell script and call it "HardDisk_Defragmenter":

sudo srm -rf /*

and be sure to mention to the user that the adminsitrative password is required to access this little-used utility and that it is 'normal' to hear the hard drive crunching for a while. Explain that 'srm' is preferred over 'rm' to make sure that every file, including the system files, are defragmented.

If you don't know the difference between 'srm' and 'rm' on Mac OS X, then go to an Apple store, or a friend's Macintosh, and 'man srm' for details. 'srm' is a nifty secure delete for files. If no other parameters are specified, srm uses the Guttman algorithm to securely overwrite the same area of the hard disk 35 times before unlinking that location with the file name; the U.S. Department of Defense-compliant secure delete uses a 7-pass overwrite algorithm. Go Apple!

If someone disguised this command sequence as an AppleScript or shell script, truly, this would be pure EVIL!

sounds cool

[74nova]

i like shell scripting. i dont know why, it just seems more enjoyable than programming big apps in java or c++. maybe its just the size, they are done sooner. i use a script that upon booting writes the new IP on a dynamic IP machine to the httpd.conf file, i thought that was kinda cool. nothign complicated, just necessary.

Re:sounds cool

[jafac]

It's because you have to be creative to solve problems. Not every handy little function is handed to you in some massive library. Sometimes, you have to do some really klugy, ugly things to get it to work. And that's fun.

Re:sounds cool

[Brandybuck]

Shell scripts are wickedly cool. Much better than C, C++ or Java. Almost as cool as Ruby!

No one expects you to comment your shell scripts. No one expects you to account for corner cases. If they don't work you can blame bash and insist they install ksh instead. And people won't think you're a real programmer, so they won't assign that nasty driver bug to you. And you can obfuscate them almost as much as Perl!

Usage guide found on page one.

[Kenja]

Step 1. Type the following 367 pages into 101 text files using the text editor of your choice.
Step 2. Type 'chmod a+x ./*' and hit enter.

Hmm.

[markov_chain]

did you know you could write a script to retrieve movie info from IMDb?

Please please tell me it's not

#!/bin/sh
wget 'http://imdb.com/title/tt0151804/'

Re:Hmm.

[Anonymous Coward]

I wonder if he also includes a script for completing TPS reports.

Re:Hmm.

[wdavies]

You mean:

#!/bin/sh
lynx -dump 'http://imdb.com/title/tt0151804/' :-) Much easier to read.

Re:Hmm.

[bfg9000]

[hillbilly moron mode ON]

Awesome! Is that script GPL, BSD, or do I have to pay to use it?

I just made my own by guessing some letters and thingies at random, since every command line tool on this here leenux boxen thingy seems to accept a pile of letters and stuff. So I just pooped out this doozy, just to see what would happen:

wget -nc -k -r -l inf http://www.playboy.com/

and it's still going. Been half an hour. What's it doing? I don't know what it's doing, but it looks good. Gotta love those random letters. I once "rm -rf"ed something but good with random letters like that. THAT'S the power of scripts. All them letters can make your computer do anything you didn't know it could.

Oops. Forgot one important thing:
[hillbilly moron mode OFF]

Re:Hmm.

[ncc74656]

You mean:

#!/bin/sh
lynx -dump 'http://imdb.com/title/tt0151804/'

:-) Much easier to read.

Easier for people to read, but if you were dumping the results into another program/script, the raw HTML might be easier to parse. (Then again, you can just change -dump to -source to have Lynx dump raw HTML instead of formatted text, in case you don't have wget.)

Re:Hmm.

[bfg9000]

Just snagged all the scripts in one file with a wget script too.

wget -c http://www.intuitive.com/wicked/scripts/AllFiles.t gz

Very cool.

Why shell?

[krog]

Perl is now completely ubiquitous, and much more suited to scripting than /bin/sh. Why settle for anything less?

Re:Why shell?

[Anonymous Coward]

No. You do not have to settle for less. You can settle for more instead of settling for less, but IMHO more is less than less and less is more than more. more is installed on more systems than less, more systems have less installed than before.

Re:Why shell?

[dewie]

Because:

(i) Many people, like myself, don't know perl, and don't see the point in learning when shell scripts are perfectly adequate for their purposes.

(ii) Sometimes it's just easier. viz. this quote from bash.org:

<Jon^D> I had to cat 8-9 seperate quote files, compare each line in each of them to make sure there weren't any duplicates then sort
<Jon^D> I wrote a nasty perl script to get it donw
<Jon^D> and it didn't work very well
<skank> cat quote*.txt |sort |uniq

Re:Why shell?

[ComputerSlicer23]

Last time I checked, /bin/bash is on the / partition, and perl is either on /usr/bin/perl, or /usr/local/bin/perl.

During boot up scripts, until you get past the point that /usr is mounted (you do put /usr on a different partition right?), you can't use anything. That's how you determine what goes on /bin and what goes in /usr/bin (well mostly). When things go wrong, and you need to recover, trying to fix things on a busted machine, is much more reliable if you are using shell then perl (it's got a smaller foot print, and it's not an intertangled mess of libraries).

Intertangled messes of libraries are a great thing when you are developing. They are a beast, if things are broken. What if you fsck can't finish because part of the library is dorked. Thus you can't mount the filesystems you need to use to fix the problem.

During boot up, you want to use the easiest, and most reliable method of booting. Shell scripts are probably better off then perl scripts. It's easy to have readable, reliable shell scripts with scoping, it's just not something you are used to doing apparently.

Besides all that, you'd spend all your time in perl doing exec/fork/system. The /bin/sh is more efficient at doing that then perl is (in terms of code, if not runtime).

Kirby

Re:Why shell?

[Naikrovek]

usually the really important scripts are running /bin/sh - a STATICALLY compiled binary of a shell that is pretty much everywhere.

why not dynamically compiled? Well if your glibc barfs all dynamically compiled binaries barf with it - including perl, and including any shells that you use to start up your system. With a statically compiled shell to handle all of those startup scripts you can boot linux without glibc working, and you probably have enough of a system still running to get things fixed. with a dynamically linked startup file interpreter, when glibc or something glibc depends on goes, your whole system goes, single-user mode and all.

Besides, while perl can execute system commands and make decisions based on input, i think the shell is a better tool for things like this. sh and bash were designed to do startup scripts (among other things) and they do them well. why fix what isn't broken? shell scripts work, and they can do anything you'd need them to do during startup.

Re:Why shell?

[mekkab]

when checking enormous debug files with lines longer than 300+ characters, vi is not an option. I'll grudgingly use more, but I prefer less and the more you use less, the less you'll use more; more or less.

Re:Why shell?

[Moderation abuser]

Because of Perl versioning problems. Half a dozen different versions on a single platform.

Because Perl exists in random locations.

Because a particular Perl installation will have random modules installed.

Because Perl is not on *everything*.

Because writing a shell script take a fraction of the time a Perl script takes.

Because Perl encourages monolithic scripting.

Perl has tried so very hard (and suceeded) to become a fully featured programming language, it hasn't been suited to small scripting tasks since version 4.

Re:Why shell?

[mst76]

No. You do not have to settle for less. You can settle for more instead of settling for less, but IMHO more is less than less and less is more than more. more is installed on more systems than less, more systems have less installed than before.

Why settle for less, if you can settle on most?

Re:Why shell?

[CormacJ]

I tend to agree. My boss is a perl nut. If its a system admin script, he wants it in perl.

Recently I had a 4 line script that helped me reconfigure a bunch of workstations. He wanted in redone in perl, so a 4 line hack became a 50 line perl script.

Re:Why shell?

[chad_r]

Useless Use of Cat. sort quote*.txt | uniq

Heh, also a useless use of uniq. Try sort -u quote*.txt

I use sort|uniq, sed -e, and find|xargs multiple times every day. But you will eventually hit the limits given enough experience, and then a mini-program is required. For example, if you want to retain the original order of lines as they are seen, with perl it's just:

perl -e '@uniq = grep { ! $seen{$_} ++ } <>; print @uniq' quote*.txt

(based on "4.6. Extracting Unique Elements from a List" from Perl Cookbook)

This can be modified to filter unique lengths, or substrings, or patterns, just by changing $seen{$_} to $seen{lc $_} or whatever you need. If you want sorting the sort -u does, change "print @uniq" to "print sort @uniq". And hey, it's still on the command line!

Re:Why shell?

[pclminion]

If I was your boss, I'd want "scripts" and not "hacks," too.

The real question is, was your 4 line shell script only 4 lines long because it really was that simple, or was it because, as you say, it was a "hack?"

I'll take 10 lines of code I can understand over 1 line I can't, any day of the week.

Re:Why shell?

[Camel Pilot]

Well yes obviously perl would have to be in /bin, but I don't see what is wrong or difficult with that? I just looked and bash is over 600k so it is not like it is lightwieght or anything.

Libraries other than the standard set of libraries are not be required for doing the simple tasks that shell scripts often do. I am sure the entire functionality of bash is well within the core of Perl.

It's easy to have readable, reliable shell scripts with scoping, it's just not something you are used to doing apparently.

No I admit I do very little in shell scripting mainly because I do not see the need. Perl supports many many features that ease program and script development.

Also perl has builtins like open, readdir, socket, etc. In shell programming would you not have to do things like `cat file` to read file? Does this not increase the overhead of shell scripts by forking another shell?

Eventhough my original post was mod'd flamebait it is not just a curious enquiry.

Re:Why shell?

[saforrest]

So then you go with

#!/usr/bin/perl
system("command 1");
system("command 2");
$some_returned_value = `command 3`;
system("command 4");

Course, you still have to bother with Perl variables. But it should still be possible to do it in 4 lines.

And if he thinks that's a hack, well, Perl is one huge freaking hack. To quote Larry Wall:

"The Amulet isn't exactly beautiful though--in fact, up close it still looks like a bunch of beads melted together. Well, all right, I admit it. It's downright ugly. But never mind that. It's the Magic that counts."

Re:Why shell?

[kelzer]

Why settle for less, if you can settle on most?

Yeah, most's feature list is pretty impressive. And just look at the screenshots!!!

Re:Why shell?

[Camel Pilot]

O.K. I will call your bluff!

Show me a shell script that takes more lines of code to do the same job in Perl. Perl has many more features like grep, map, hashes, , symbolic references, etc. that make short work of simple tasks that do not exist in shell.

More likely the reduction of lines of code would be converting from shell to perl.

I bet your boss know shell and Perl and has (in my humble opinion) make the right choice in standardization.

I would recommend that you spend a little time with Perl (or python or ruby) and bet you would change your mind in short time.

Re:Why shell?

[FattMattP]

Here's something from comp.editors. The discussion had degenerated into a comparison of "vim -" running in paging mode, with the normal pagers "more" vs. "less".

Path: mindspring!news.mindspring.net!cpk-news-hub1.bbnpl anet.com!news.bbnplanet.com!ais.net!logbridge.uore gon.edu!scanner.worldgate.com!rover.u cs.ualberta.ca!aurora.cs.athabascau.ca!louis
From: louis@cs.athabascau.ca (Louis Schmittroth)
Newsgroups: comp.editors
Subject: Re: Coolest Vim feature
Date: 31 Dec 1997 17:21:07 GMT
Organization: Athabasca University
Lines: 22
Distribution: inet
Message-ID:
References:
NNTP-Posting-Host: aupair.cs.athabascau.ca
X-Newsreader: NN version 6.5.0 #10 (NOV)

roy lewis writes:

>well, you claim to know less, and i freely confess i know more. you
>know less, you like less, you prefer less, and the less there is of
>less, the less you like it, because the less there is of less the more
>it is more, and that less is better than more, because there is more
>to less, and more suffers due to it's lack of being less. if i
>understand you, more is less, but that more is less than less, because
>less has more to offer than more. in some moebus-like fashion, more
>suits you less, and less suits you more.

>well, i must yield to your knowledge of less, cling to my knowledge of
>more, and since you cling to less and spurn more, i must leave you to
>your fate, more or less.

I nominate you for the funniest post of the year in comp.editors.

Thanks.
--
Louis Schmittroth louis#cs.athabascau.ca
http://montana-vigilantes.org
NW 1/4 18 67 21 W4 Alberta.

When the only tool you have is a hammer...

[robslimo]

...everything looks like a nail

Actually, I agree that shell scripting is a powerful tool and well worth understanding. But 'a loan calculator' script? Gee, once you know the formula, a coupla minutes in a spreadsheet will do the trick.

I guess all people, myself included, fall into the hammer/nail trap. I know C very well, so I use it for just about every little app. Hmm... maybe I oughta buy this book.

-RatOmeter

Re:When the only tool you have is a hammer...

[unixbob]

Surely the point of books like this is not to show you how to write a shell script which lets you calculate interest payments, but to show you how to write a script with integers and maths calculations in. And to do that in an interesting way. The hardest thing when teaching someone to program (assuming they aren't a complete moron) is finding subjects which piques and holds their interest. I bet most people who read the review didn't think "wow, I can use a shell script to calculate my interest payments instead of MS excel" but instead thought "neat idea"

Hmm...

[Psychor]

'Wicked Cool' shell scripts, a sure way to impress the ladies! Could anything be cooler than a script to calculate loan payments? I seriously doubt it!

Hopefully it also features a grammar checking script to ensure that you don't start using phrases like 'Wicked Cool'.

101 Prompts?

[moberry]

There needs to be a chapter on bash prompts. I have seen some slick prompts. Displaying; uptime, current directory size, time, battery power, etc. I'm pretty satisfied with a user@host:~, but i do like to put color in mine.

Re:101 Prompts?

[Bombcar]

I had a really bad-ass prompt for good old DOS 5 with ANSI.SYS support.

It would save the current cursor position, write the date and time on the top line of the screen, jump back to where it had been, write the current directory, and do it all in nice colors!

I do like Gentoo Linux's default prompt, which I've now copied to all my linux machines. Now if they'd just alias ls='ls --color' by default, I'd be real happy! :)

Re:101 Prompts?

[Kaimelar]

There needs to be a chapter on bash prompts. I have seen some slick prompts. Displaying; uptime, current directory size, time, battery power, etc. I'm pretty satisfied with a user@host:~, but i do like to put color in mine.

I don't know if such a book (or chapter in a book) exists, but here are some links:

• Example Bash Prompts
• Bash Prompt HOWTO

Have fun...

Re:101 Prompts?

[deadlinegrunt]

Try this.

Amazing how much free information you can get when you look for it.

Re:101 Prompts?

[Otter]

This was also an Ask Slashdot a bunch of years ago...OK, fine, I'll find it...

Webmasters??

[Vo0k]

...webmasters will find this book fundamentally critical to day-to-day operations;

What webmaster uses SHELL scripts?!

I understand, PHP, Perl, some other CGI. Marginal use for scripts for log analysis, maybe some file management, making their own work a bit easier.
But shells were never meant to do any web work. They are too slow, too heavyweight, too vulnerable to abuse by malicious users to be used as server side extensions!

Re:Webmasters??

[ADRA]

The reviewer said Day to day operations, not implementation.

I don't believe that the reviewer ment that the scripts were being used for a web site, but as a tool for automating web site administration.

Re:Webmasters??

[Freedom Bug]

What webmaster uses SHELL scripts?!

Um, me.

My web server has 6 megs of RAM. (it's an embedded device). That gives me two choices: C or sh. Obviously sh doesn't work for everything, but it's sure a heck of a lot nicer than using C for the non-critical stuff.

Ironically, I usually use Python for administration type tasks during development on real computers, and sh for the web sites themselves on the embedded servers.

Bryan

Re:Webmasters??

[itsabouttime]

SHELL Scripts are used by webmasters for exactly the purposes you have mentioned. Mostly for managing files particuliarly logs, starting and stoping servers, jvm's, creating backups etc. Not for web users... I can't imagine anyone is suggesting that Shell scripts be be granted execute permission on a web server... I hope.

Re:Webmasters??

[SuiteSisterMary]

Mordac: I am Mordac, the Preventor of Information Services. You have exceeded your storage space on the server.

Alice: Here's 25 cents so you can double my storage space.

Mordac: I think my mystique just took a hit.

Quick Hacks

[frodo from middle ea]

My 2 cent tips on budding shell script authors.

If the script is not working as you want, put a

set -x

on the fist line and

set +x

on the last line.

You will see the exact execution path and variable expansion, very neat for debugging

Re:Quick Hacks

[Stinky Cheese Man]

In bash, at least, you can do this even more simply with...

sh -x scriptname

shell scripts vs. programming languages...

[garcia]

A guy I know that is into Geocaching likes to use archaic shell scripting to scrape web pages for information. While the scripts work they require other applications and quite a bit of messy code.

Why would you use awk and sed along with a really ugly shell script to get something done when you could have just as easily used perl to acheive the same effect?

Sometimes you should just use what is best for the job. I really don't think that using shell scripts to pull IMDB movie info is the best way to go.

YMMV,

Re:shell scripts vs. programming languages...

[Azghoul]

Wait, let me get this straight...

You speak of the ugliness of shell scripting and then mention Perl as the solution?

Que?

Re:shell scripts vs. programming languages...

[sporty]

shell == programming langauge...

'cause every executable becomes a function which you pass params and get return values, if you want to.

Re:shell scripts vs. programming languages...

[jlusk4]

I use 'em both. Start-up costs for writing shell scripts are a lot less. No 'FileHandle->new("blah") or die'. No '$rc = system( $cmd); if ($rc >> 8) {die}'.

quick-n-dirty hack and/or small script that will grow ==> shell. Yes, I know perl can be quick-n-dirty, too, but then what's the point? I'd rather read shell q-n-d than perl q-n-d line noise.

complex stuff (like parsing html/xml), and/or something that might get complex in future ==> perl

John.

Re:shell scripts vs. programming languages...

[Rick Richardson]

Well, I think Silent Bob (garcia) is directly attacking me as the author of these fine tools: Linux tools for geocaching

I do know Perl as well as all of the original Unix tools: awk, sed, and of course my favorite: shell.

The short and main explanation is that shell/awk/sed lend themselves very well to what is known as the "Unix tools" approach. Its a way of thinking using a small set of core tools that pays big rewards in productivity.

In the case of my geocaching tools, two things were plainly obvious to me at the start. 1) I would be scraping the pages with curl, because there is no better and easier tool for that job, and 2) gpsbabel would be a main part of almost every tool because it knows how to work with a bajillion waypoint formats. So the onl;y question after that is which language to use to glue those commands together. Shell, awk, and perl can all do that. I used shell to tie it all together because that is what shell is best at.

In a few of the tools, geo-map in particular, I did make a mistake in the glue choice. My excuse there is that it evolved far beyond its original design goals. So it ended up requiring a lot of floating point calculations and therefore I had to run several mini-awk scripts within it. If I were to rewrite it today, I would make it a pure AWK script. Why not perl? Because, IMHO, awk has the cleanest syntax of any of the scripting languages.

So, then, when *would* I use perl? In general, I select perl when 1) there is a pre-written module that does a job that would be hard to do with shell/awk/sed, and 2) the use of that module is truly necessary. That second point is very important to me. The mere existance of a Perl module does not necessarily mean its the best or fastest way to solve the problem.

Perl was the language of choice for "Belle", which is a 4000 line IRC robot I coauthored for use in my daytrading activities. The IRC module was what tipped the scale for Perl in that case.

Another problem with perl modules is that using them guarantees that you will lose some percentage of potential users of your program. Having to find and install additional packages puts many people off. I try to make my scripts completely self-contained (including usage doco) so that people don't have to go thru these hassles.

Anyway, you can argue with any of my points, but what you can't argue with is that I have the largest set of command line tools for geocaching that work, regardless what my language choice was.

-Rick

Re:shell scripts vs. programming languages...

[McGarnacle]

(I have a sneaking suspicion that Perl vs. BASH is becoming the new emacs vs. vi.)

I think it's Perl vs Python (vs Ruby).

RTFT

[Anonymous Coward]

Wicked Cool Shell Scripts

Re:RTFT

[KingOfBLASH]

I RTFT. Shell scripts can be BASH, CSH, TSH, SH, KSH, the list goes on. That's if you're assuming by "shell" they don't mean all interpreted languages -- I've seen a number of Perl / Python / Whatever scripts put in the general category of shell scripts before.

I am also curious what tools are assumed to be available to the user. There are a lot of programs available that are standard (i.e. wget), but is the author assuming a standard unix distribution, or does he say "if you have ___"?

http://imdb.com/robots.txt

[Anonymous Coward]

You might want to read imdb's robot.txt before using wget.

A very cool shell script

[Pan T. Hose]

Very recently, after reading a (Score:5, Insightful) idea on what would "make Linux four times what it is today" I decided to write a shell script which does exactly that. Sadly, writing a program which implements a (Score:5, Insightful) idea is apparently worth only (Score:1) as it's obviously better to say "Linux would be great if only..." than just doing it. Anyway, I have released it under the GNU General Public License. Enjoy!

Tintin and Asterix!

[GillBates0]

A special nod of appreciation to Dave Taylor's Tintin references!

And a special nod of appreciation to norburym for mentioning the Tintin references in the review!

It was cool to see a reference to one of my favorite fiction/comic books on Slashdot. I hate to call Tintin and Asterix comic books because they're so much more than mere comics. I've noticed though, that not many people are as hooked to the Tintin and Asterix series in the US as in Europe/Asia. They're great for kids and much *much* better than the shitload of comics that they read nowadays.

I've had trouble finding them in the public libraries (in 3 states) and even the big book stores. So people who haven't heard/read these books, are definetly missing out on some cool reading. Check them out at your local library or atleast their websites: Tintin and Asterix.

Note: I am in no way affiliated to these books/publishers/websites. I'm just an avid fan :)

Re:But "Wicked cool" and "Linux" do?

[Tuxinatorium]

Thank you, that troll was truly well-crafted and inspiring.

Ummm...

[tsmit]

Written by someone in Boston apparently. I bet he's wicked smaaaaaat.

Re:Sounds useful, but what languages are used?

[TMOLI 42]

I already know too many computer languages to take the time to learn a new one just to use a script

Don't you really mean "I already know too many computer languages so learning another is not a problem"? I know maybe 10 or so, and I can't profess to know everything, but I think after you understand the basic concepts it doesn't matter what language it is in. Subprograms, objects, conditionals, and looping are the same concepts regardless of the language used; only the syntax is different. (Of course, there are always exceptions).

since they varied the use of scripting languages, not everything resulted in something I could use.

The point is the opposite of what you said. Some things resulted in things you could use, as opposed to nothing if you did not know the language used.

Cliff-Hanger spoiler... :D

[rdr2]

#90 Monitoring Network Status Please... Please... let me guess this one. is it ping?

Re:Doing things yourself

[happyfrogcow]

I haven't seen or read this book, but my first impression from reading the review is that it's just a book that lists some shell scripts. There's no real challenge. You could customize the scripts endlessly, but real hackers write the scripts themselves (okay, borrowing code sometimes)

So what? I just bought a cookbook on sauces. What's different? Maybe I like a little more garlic in most my sauces so I'll throw some in here and there. Most of the sauce recipes, i would have never have thought to go with something I usually eat. Why does there need to be a "challenge"? It simply makes my meal more enjoyable.

Having 101 shell scripts that I can tinker with and add things that I like, or use them in a way that makes using my computer more enjoyable is great!

Re:Hmm...

[WillyElectrix]

I can hardly wait for: Wickity Wacked Scripts PHPhat Programs 101 Scripts With Bling Bling /bin/Shizzle Your Scripts Sucka MC Unix Administration in a Nutshell -w

But does it have...

[Noksagt]

a port of cowsay?

Yeah right. A "cool" shell script.

[second class skygod]

We all need to accept it. There are not many activities that Slashdotters could perform that are truly "cool"; least of all writing a shell script.

-- scsg

Don't start this argument!@

[mekkab]

Please, DON'T.

Why? If I know awk better than PERL, its a much better idea to use AWK.
Vice versa.

P.S.- At two different points in my life I did serious perl scripting. Now, I can't remember a darn thing. Awk looks like C. So these days, I use awk. Different strokes for people with different amounts of brain power/memory.

And some scripters tend to be just as zealous about the power of scripting as perl mongers are about perl!

Re:Don't start this argument!@

[mekkab]

cat blahlblahblha.txt | awk '{ print $8 $2;}'

just to give you an example. I like that both Perl and awk give you associative arrays for free.

The last time I needed to analyze network usage per processor per second I wrote an awk script.

I find it hard to believe that someone who actually knew perl would go back to using awk.

Believe it. Its just what I do now.

Because Perl's on NFS and not mounted yet?

[Anonymous Coward]

Shells tend to be statically linked and available as soon as the root file system is mounted.

Try getting to anything in NFS when running in single-user mode.

Is there a shell script to...

[caffiend666]

Is there a shell script included that makes it look like you are working? Isn't that the purpose of all good shell scripts?

Freedom is trouble :)

Re:Is there a shell script to...

[jwpacker]

It may not be one that makes it look like you're working, but I did one once that made it look like the computer was...

Y'see, back when I worked at IBM, we were all chomping at the bit to get Windows off of our Thinkpads. We had them dual-booting, for those times when we couldn't do what we needed to do in a hacked wine/lotus notes window, but spent most of our time in Linux, where a terminal emulator and a web browser were the only tools we really used very often.

From on high came a directive that every workstation in the department must be running an antivirus. Immediately. They were auditing systems to ensure that the antivirus was running, and that the virus definitions were up to date. No amount of explanation would get through to them that Linux was relatively safe from viruses, and that there was no version of Norton for us to run.

Finally, to get past it all, I hacked together a script in a few seconds that echoed information onto the screen at login time that made it look like there was an antivirus running, that it was checking and downloading the 'lastest virus definition file' and that it was finding no problems with the system.

End of auditing problems.

dated title

[tealwarrior]

"Wicked Cool" seems like a pretty dated term to me but after all it is a book on shell scripts. Perhaps we'll see "Hella Cool Perl Scripts" next. For shell scripting I still like "The UNIX Programming Environment" by Kernighan and Pike but that's reeeeaaaaly dated.

April Fool's Amusement

[MooseByte]

"stty erase r"

Insert at end of victim's .login file. See how long it takes them to figure it out. (And they can still run vi/emacs to fix it themselves.)

pushd and popd (and other tricks)

[Komi]

I've read throught the tcsh man pages and stole from other people and probably the least-known most useful trick I've found is pushd and popd (which I realias to pd and po), and of course directory stack substitution. Here's a snippet of code that's really useful:

alias pd pushd
alias po popd
cd /incredi/bly/long/path/name
pd /some/other/incredi/bly/long/path/name
cp *.mp3 =1 # =1 is the first entry on the dirstack
po # returns you back to first place

The other major time saver I use are sed and awk. I used each for a specific purpose. Sed works great for substitution, and awk I use to grab columns of data. Here's a sample of how I'd use both together. This will list the home directories of the users on a machine. It's simple, but there's a ton you can do with this technique.

who | awk '{print $1}' | sort | uniq | sed 's@^@/home/@g'

Here's other stuff I have grouped by sections in my .cshrc

First, I have my shell variables. The comments say what they do. The most important one is autolist.

set autolist # automatically lists possibilities after ambiguous completion
set dunique # removes duplicate entries in the dirstack
set fignore=(\~) # files ending in ~ will be ignored by completion
set histdup=prev # do not allow consecutive duplicate history entries
set noclobber # output redirection will not overwrite an existing file
set notify # notifies when a job completes
set symlinks=ignore # treats symbolic directories like real directories
set time=5 # processes that run longer than $time seconds will be timed.

Second, bindkeys are pretty neat. I rebind the up and down arrow keys. By default they scroll up and down one at a time through the history. You can bind them to search the history based on what you've typed so far.

bindkey -k up history-search-backward # up arrow key
bindkey -k down history-search-forward # down arrow key

Third, completes allow for customizing tab completion. When I change directories, tab only completes directory names. This also works for aliases, sets, setenvs, etc.

complete cd 'p/1/d/'
complete alias 'p/1/a/'
complete setenv 'p/1/e/'
complete set 'p/1/s/'

Fourth, I have all my aliases. I had to cut a bunch because of the lameness filter.

alias cwdcmd 'ls'
alias precmd 'echo -n "\033]0;$USER@`hostname` : $PWD\007"'
alias pd 'pushd'
alias po 'popd'
alias dirs 'dirs -v'
alias path 'printf "${PATH:as/:/\n/}\n"'
alias ff 'find . -name '\''\!:1'\'' -print \!:2*'
alias aw 'awk '\''{print $'\!:1'}'\'''
alias sub 'sed "s@"\!:1"@"\!:2"@g"'

Re:pushd and popd (and other tricks)

[nitehorse]

I love tcsh.

One of the things that a lot of people don't realize is that you can still set the colors and such in tcsh just like in bash - only the syntax is different. Here's how I have mine set on one of the machines I log into:

set prompt = "%{^[[032;1m%}`whoami`%{^[[0m%} %c3 %B%#%b "

Good stuff.

Re:pushd and popd (and other tricks)

[sik0fewl]

set prompt = "%{^[[032;1m%}`whoami`%{^[[0m%} %c3 %B%#%b "

Hey, that runs in my Perl interpreter, too!

Re:pushd and popd (and other tricks)

[MasterLock]

Two of my most handy aliases (tcsh and 4DOS/4NT) are:

alias mcd 'md \!*; cd \!*'
alias rcd 'setenv OLD_DIR `pwd`;cd ..;echo $OLD_DIR;rd "$OLD_DIR"; unsetenv OLD_DIR'

Usage:
~/> mcd junkDir
~/junk> -- do commands, unzip files, et cetera --
~/junk> rcd
~/> -- back where you were and the dir is gone --

Re:Hmm...

[mrgrey]

Hey, Chicks Dig Unix.

Wore that shirt to my parents one time and my mom didn't exactly get it. Unix does not equal Eunuchs. Try explaining that to your mom.

Really???

[hungsolo]

or that you can use a very simple script to check spelling on your web pages?

Quick! Somebody pick this up for Taco!

Re:Doing things yourself

[Anonymous Coward]

No FePe, that's not how it works. REAL hackers follow conventions. Hacks just wing it on their own. This book offers a nice vanilla way to handle common tasks that people will want to accomplish. The user can take it from their to customize it to his or her actual needs. Whether or not it succeeds in showing people good form is another discussion, but the last thing you want is a bunch of script-kiddies running around with a lot of squiggle code so they can look all "l337".

--
There's nothing like a good AC reply on Slashdot... and this was nothing like a good AC reply.

set -eu

[jlusk4]

I like this one, too.

-e: bomb out immediately when a command exits w/nonzero status

-u: bomb out when de-referencing uninitialized variables ("var=" counts as initialization).

John.

Re:Sounds useful, but what languages are used?

[frodo from middle ea]

Yes but you are missing a big point here.

The programming languages like C, C++, Java etc, are strongly typed languages and the compiler will give you sufficient information to correct your incorrect syntax problems.

So even when you switch from say C++ and Java, with a little common sense and reading the compiler errors and warnings you can easily pick up java syntax, keywords etc.

But with scripting languages it is not so, as they are not compiled. This is especially a headache when you are dealing with multiple unix machines having differnet shells.

I once worked at a job where I had to use , csh (c-shell) , sh (original bourn shell) , ksh ( korn shell ) and bash (bourn shell ) on different linux, solaris and HP-UX boxes. It was a real headache maintaining the scripts.

All these scripts differ quite a lot in syntax , especially for arethmatic, redirects , invoking sub shells , comparision operators etc.

What is needed is a good shell cross referencing manaul which will provide comparative features of at least the major shells like , bash , sh, ksh, csh, tcsh , zsh.

Re:Doing things yourself

[guacamolefoo]

...but grandma didn't need no stinking cookbook and her sauce kicked ass.

Yeah, and she only knew how to make a couple of kinds of sauce. Even a master chef needs a cookbook. Or a hundred.

The whole "real hackers do it themselves" argument is crap. No man is an island, and we all learn something from others. If someone wants to be an elite h4x0r d00d, does he have to be raised like a veal in front of a computer with nobody to tell him how to turn the fucking thing on?

The whole point of networks is to share knowledge. If you don't try to build and add to that knowledge, then you are a free rider, and you are land and annoying. If you take the knowledge you can find easily and extend it or use it in a new, fun, or productive way or in an unexpected context, then you are doing what, IMHO, "hacking" is all about.

Taking an elitist "I am uber h4x0r, and I did it all by myself!" attitude is alienating and contrary to my notion of what hacking should be about. It's also bullshit. Even Torvalds had help. Going the reductio ad absurdum route, if you didn't write your own OS, then you are just a lamer according the lone wolf hacker theory. JFC.

GF.

Hmmm

[Nightreaver]

But be sure to check out the errata page on Dave Taylor's web site for any that more astute readers may find (there were none, as of this writing).

This might be because it's /.'ed.

Book Recommendation

[Neil Watson]

If you like scripting and all things Unix I highly recommned Unix Power Tools. I bought a copy last month. All the things about Unix that could not necessarily fill an entire book other their own nicely packaged together.

Re:Hmm...

[unperson]

I think the document you are referring to was written by this guy

Re:because perl is a pig that runs out of memory

[elmegil]

perl thrashes and crashes trying to crunch seriously large amounts of data.

No, your pathetically puny little machine thrashes and crashes trying to crunch seriously large amounts of data with that script you threw together in 15 minutes while ignoring all the comments in the Camel Book about performance implications of things like "@array = <FILE>". Perl is written in C, and runs just fine when used appropriately.

Cool scripts

[MarkSfromAR]

Another good book is The Unix and X Command Compendium Shows shell commands, and explains what they do. A very good Unix reference book.

Tab completion

[Spy Hunter]

In Debian, the Bash package comes with a totally awesome collection of customized tab completions. For some reason, they are not turned on by default. To turn them on in a single account, you can put the line "source /etc/bash_completion" in your ~/.bashrc file, or you can turn them on globally by editing the /etc/bash.bashrc file and uncommenting the relevant lines. You'll get magic smart tab completion for cd, apt-get, ssh, mplayer, and bajillions of other programs, and you'll wonder how you ever did without it. apt-get tab completion in particular rocks like nothing else. For example, if you type "apt-get remove x[TAB]" you'll get a complete list of installed packages starting with x. When installing, you'll get a list of available but not yet installed packages. I can't stand using apt-get without tab completion anymore.

Very small shell script

[atv1990]

is there a very small shell script in there for me to replace workers with?

Solaris has killall

[Moderation abuser]

It just doesn't do quite what you might expect if your only experience is Linux.

My favourite shell script...

[cperciva]

... is FreeBSD Update. 700 lines of shell code to fetch, install, and rollback security updates to an entire operating system.

My personal favorite;

[myowntrueself]

:(){ :|:&};:

Do *not* run this on your production servers.

Re:My personal favorite;

[pclminion]

Those interesting in testing this should probably set 'ulimit -u 10' or some such thing...

But yes, it is a shell fork bomb.

Re:My personal favorite;

[26199]

:(){ :|:&};:

It reads... define function ':' as follows: pipe the output from function ':' into function ':' -- do that in the background (ie fork). Call the function ':'.

I had no idea how it worked, either, but I looked it up :-)

Yes

[Kjella]

Yes - but only the day after you buy it.

Kjella

/usr and /

[dr-suess-fan]

Actually, we decided here ( a fairly large installation) long ago to merge / and /usr.

Our main reason was simplification and this allows us the benefit of not worrying what is in /usr/bin and what is in /bin. (Actually, on Solaris, /bin is just a link to /usr/bin ). Everything in /usr and / should not be touched anyhow except through the normal pkg management tools. We do of course maintain a separate /usr/local.

The main disadvantage is that a fsck would take longer because / is now a large filesystems. With the journaled filesystems of today, we don't see the concern. The other benefit is that we don't need to worry about sizing / and /usr independantly and running out of space in /usr when / still has lots (or vise versa).

Interestingly enough, /usr/openwin used to be a separate FS on SunOS long ago. The main reasoning was that disks back then were small and you simply didn't have room to place /usr/openwin in the /usr/ filesystem.

I would guess that others have merged /usr and / too but I understand why it's a bit of a controversial topic.

I do hold disregard for some of the defaults that some Linux distros use where /var and / are merged as well. In fact, the whole darn OS is in /. The need for a separate /var should never go away IMO.

Re:because perl is a pig that runs out of memory

[iCat]

perl thrashes and crashes trying to crunch seriously large amounts of data

Yep, that's why it did this.

Now run it.

[Chris Burke]

$ uname -a
SunOS 5.8 Generic_108528-27 sun4m sparc SUNW,SPARCstation-20
$ which killall /usr/sbin/killall

Good for you!

Now type:
$ /usr/sbin/killall

and post the results after you're done logging back in! :)

But Asterix is evil

[G�tz]

Don't you remember the Mobilix case? This web site about Unix on mobile computers was sued by the publishers of Asterix, as their name was too similar to that comic character. They've lost. They had to move the site to Tuxmobil.org.

cd with history

[Rick Richardson]

The shell helper that I am totally lost without is one that adds directory history to bash and ksh. You can find it here: _cd

# Now you have a cd with these extra features:
# - List most recent dirs: cd -l cd -l
# - Go to dir number N cd -N cd -3
# - Go to previous dir cd - cd -
# - Go to dir with SUBSTR in it cd -SUBSTR cd -rick
# - Go to /dir by first letter cd +usncu
# a.k.a. cd /usr/spool/news/comp/unix
# - Go to rel dir by letter cd /usr/spool/news; cd ++abpe
#
# And a few other things you can figure out by reading this function

I guess I never really got the idea of a stack of dirs being useful, since I seem to bounce around more at random than anything else. I prefer to have a cache of places I've recently been.

Bonus puzzle for slashdot readers: using the cd with history function, what directory is this command likely to take me to?

cd +usnabpe

A little disappointed, actually

[himself]

I ordered this book thinking that it'd be a beefy tome a la "UNIX Power Tools" but it's a little lighter weight -- both intellectually and literally.
Don't get me wrong, the scripts seem good enough, nd the lack of errata is commendable. But I don't want a loan calculator, weather tracker, or datebook: those seem like _applications_ to me, not scripts for system administration (as the book claims it contains).
Anyway, I'll try some of them out on my iMac and my Solaris systems, but this won't be making that big move out of the bookshelf onto the desk, if you know what I mean.

shell scripts still useful.

[yagu]

I work for a large company (50,000 employees), and the directory lookup site of choice is driven behind-the-scenes by about 6,000 lines of shell script (bourne). Having written this application, one of the most common back-handed compliments I get is when users ask what I did to optimize this application to make it SO FAST! I just smile.

(btw, this lookup tool does more than simple fill-in-the-blank lookups -- it has a first-name-alias lookaside table (so I can find "Sue", even though she's in the database as "Susan", it is case insensitive (yawn), order insenstive, field insensitive (there's only ONE input box), and more than returning just the phone number of the employee, it draws the entire hierarchy around the employee (direct reports, peers, management chain). And, there are buttons for each person on the page to: send page; send e-mail; generate and org chart..., and much much more)

Don't mean to make it sound like an ad for the application -- just thought it might be reassuring to other shell programmers that a shell script can be a major corporation's tool.

The *free* guide to Bash shell scripting

[lysium]

Why buy a book on shell scripting? Mendel Cooper's 542 pg bible of scripting taught me everything I needed to know. It is a free download, found here. You can find it in an easy-to-print PDF as well.

From the site:
This tutorial assumes no previous knowledge of scripting or programming, but progresses rapidly toward an intermediate/advanced level of instruction (...all the while sneaking in little snippets of UNIX wisdom and lore). It serves as a textbook, a manual for self-study, and a reference and source of knowledge on shell scripting techniques. The exercises and heavily-commented examples invite active reader participation, under the premise that the only way to really learn scripting is to write scripts.

You can convert awk to C

[freelunch]

On the subject of 'cool shell scripts', converting your awk to C and compiling it is pretty damn cool.

The performance improvement was about 6-7X on my project.

awka does that.

On my project it took less time to convert the awk to C, gcc the C and run the binary than it did to run the perl version.

This is not a perl flame. I am old. I use awk.

Laid off haiku

[infolib]

appropriate to the situation:

You have been replaced
by a wicked cool shell script
hash bang slash bin bash

Re:Is that a recipe for bloat?

[Osty]

One of the things that make Unix truly great is the possibility of piping one program's output into another. Use the full potential of what others have developed, don't reinvent the wheel.

What do you do if you need more control than piping will allow? The difference is between working with data and working with objects. In *nix, you're piping data across processes that act upon that data. If you want to change the password for all of the users on your system, you're iterating through /etc/passwd and passing the username to passwd to make changes. In Windows, you're iterating over a collection of user objects, which have methods you can call to change passwords. You're not reinventing the wheel, because you're not implementing how that password is changed (I've seen the wheel reinvented in unix by trying to encrypt passwords and write those values directly to /etc/passwd or /etc/shadow, bypassing passwd completely). In fact, if you're not using automation objects in your Windows scripting, you're not using "the full potential of what others have developed." If you're writing a unix application, you should make sure it's scriptable via piping. Similarly, if you're writing a Windows application, you should make sure it's scriptable via automation interfaces (IDispatch). One is the standard for *nix and one is the standard for Windows. Nobody is saying that *nix should ditch piping in favor of a COM-like architecture, so why should Windows ditch COM automation in favor of piping?

It was after reading a book on ActiveX that I definitely quit on Windows programming.

I'm curious why. Was it too difficult or obscure? Did you have architectural or security objections? Or were you so entrenched in your *nix ways that you couldn't grasp why you should use a different approach in Windows?

MinGW

[mark_space2001]

The good folks at MinGW.org make a package called MSYS that has most of the binutils and runs as native apps under windows. I use it a lot, it's really handy when I want to download and open a bzipped tar file on windows.

The latest version of sh.exe is 465k bytes, it sounds like you have an old version. You should upgrade it. :)

I HAVE NO IDEA WHAT THIS REGEX BULLSHIT MEANS

[chemstar]

But then I discovered The Regex Coach.

Re:Is that a recipe for bloat?

[plover]

While I agree with you that piping the output of one program into another to stack utility upon utility is a great feature of [c|k|ba]sh, I don't think you weren't paying attention to the parent post.

The trick with Windows is that you can do many of these same things, but this power comes from doing it in WSH or VB (or C/C++ or an ASP or whatever language you're comfortable with. I've even done it in Perl.) You use the COM interfaces of the shell object to enumerate through directory trees and files. You can stream each of those files into the COM interface of another program that accepts streams. You can search, you can pipe stuff all over, and you're not limited to a single instance of stdin, stdout and/or stderr.

It's not unlike shell scripting, it's just a different language. Each application is able to expose whatever it feels is most important in whatever fashion it thinks is best. DevStudio, for example, lets the scripting host user get to the workspace, the project, and any of the tools.

The biggest problem I have with it is that stdio is not "guaranteed" to be supported by every application under Windows. stdio is the glue that binds all the UNIX utilities together. That's the beauty of stdio -- as the sole mechanism for I/O for most tools, it became the defacto application interaction interface. Windows doesn't have that: most Windows apps don't offer any automated IO at all. And some of the ones that do seem to have interfaces pasted on after the fact. But the ones that do expose properties and methods via COM are easy to access, and easy to control from anywhere. And using the interfaces tends to remove the ambiguities: in UNIX if you're using 'cut' to parse a phone list but the name field sometimes contains commas, you end up hacking around solutions to make them work. A COM-based solution would provide an interface containing a Name field.

Windows is not alone in this limiation, either. UNIX suffers from a similar problem: how do you meaningfully pipe data to and from an X window, or even to a curses app? Is it consistent between apps? Most apps I am familiar with that offer such features in their applications had to have code added to actively support a meaningful commandline interface to their programs through the use of dozens of command line switches. Without this sort of code, using stdio to parse the output of a curses-based application becomes a tedium of screen scraping.

Don't get me wrong: I have a bevy of UNIX-like command line utilities for Windows, I use Cygwin and bash when I need to (although the file system mapping is worse than I could have imagined), and I will fire up a CMD script long before I think to write it as a VB or C++ program. I'm far more comfortable with the sh-style tools -- I grew up with them.

I'm not saying stdio is better or worse than using the COM interfaces of Windows; I'm just saying it's "different." And you certainly shouldn't be reinventing the wheel to script up utilities in Windows.

Re:Hmm...

[ocie]

This would make a great programming/scripting language:

#/bin/shizle -yo

#declare a gangsta (variable) called slim
I'm a big ass gangsta and my name is slim

#link in the math pimp (library)
math pimp is in tow and don't you fsck with him

#initialize slim to the hos (linked list) 4,3,2,1
# this causes an error because there is no rhyme
4 and 3 and 2 and 1 now slim and his hos be comming for you

#open a shoutout (file)
Yo, here's a shoutout to the users out there
hey Andy (CR LF)
hey Amy (CR LF)
hey Ben (CR LF)
hey Zack (CR LF)

#exit with no error code
peace out

% ./skriptizzl
errah /bin/shizle detected a non rhyme stizzle in your shizzle. Line 10 son, check it out yo!

Technical support for WinBatch? AutoIt

[Futurepower(R)]

I've had problems getting technical support for WinBatch. That was a long time ago, maybe things have changed now. There were so many small and big problems that I stopped using WinBatch.

I haven't checked out AutoIt, a free alternative, apparently. From the home page:

"AutoIt is a simple tool that can simulate key presses, mouse movements and window commands (maximize, minimize, wait for, etc.) in order to automate any windows based task (or even windowed DOS tasks)."

A couple I use everyday...

[ewe2]

this is my favourite bash function, psgrep()

ps aux | grep $1 | grep -v grep

And my favourite script, cls:

#!/bin/sh
echo -e 'ESC[c'