Adding Some Spice To *nix Shell Scripts

An anonymous reader writes "Developing GUI script-based applications is time-consuming and expensive. Most Unix-based scripts run in a CLI mode or over a secure ssh session. The Unix shells are quite sophisticated programming languages in their own right: they are easy to design and quick to build, but they are not user-friendly in the same way the Unix commands aren't (see the Unix haters books). Both Unix and bash provide features for writing user friendly scripts using various tools to build powerful, interactive, user-friendly scripts that run under the bash shell on Linux or Unix. What tools do you use that spice up your scripts on the Linux or Unix platforms?"

Comments

None!

[Anrego]

I know this is troll-ish, but the way I view it a script is just that.. a script. A series of commands to be executed in a specific order designed to automate a repetative task. Basic logic, control, and input are generally ok.. but interaction is in my opinion an indicator that your task is out of scope for a "script" and should become a full fledged application.

(you may now freely argue amongst yourselves on the difference between a script and an application)

There are a metric ass-tonne of dialog-type apps out there .. just google for your favorite toolkits prefix and "dialog" and you'll probably find something..

gdialog
kdialog
xdialog
etc..

Off the top of my head...

[hkz]

I work as a Linux netadmin and system developer, so I do a lot of shell programming in (ba)sh. Here's some of the niftier things you can do to a shell script:

- Make colored output with shell escape sequences. Works especially well with red type for error messages, makes them really stand out.
- Use inline functions a lot. The great thing about them is that you can pipe to them and use them in all kinds of places. For instance, to do mysql queries:

mysql_query() { /usr/bin/mysql --user=root --pass=topsecret database
}

echo 'SELECT * FROM accounts' | mysql_query

- "Here documents". For long MySQL sequences, something like the following (reusing the mysql_query function from above):

cat - EOF | mysql_query
      SELECT bar
      FROM foo
      WHERE baz ...
EOF

This lets you easily format stdin for scripts and other programs. Also really useful for outputting HTML and stuff like that. Best thing is that variables are expanded inside the block.

- The || and && operators. Check if a file exists, remove if so, else complain:
[ -f /tmp/somefile.txt ] && rm /tmp/somefile.txt || echo "Does not exist!"

Also common in this form:
[ -x /usr/bin/necessaryprogram ] || { echo "aaargh"; exit 1; }

- Making a "multithreaded" shellscript is also one of my favourites. Say, you want to start five virtual machines at the same time. Write a function that starts a vm, and call it a few times in a loop, backgrounding each instance with &, and saving their PIDs. Then have a "wait loop" that waits for the PIDs to exit the system (or for a timeout to occur).

- Proper argument handling with getopt. Have your script take "real" arguments in any order, just like real binaries.

This just scrapes the surface of the surface, of course. I learn new stuff every day.

Stop using the Shell

[Bruce Perens]

The shell is a poor clone of 1950's algol. Today, scripting in Ruby or Python yields scripts that can handle errors with advanced facilities like exceptions, and is more maintainable, and can connect to a number of different GUIs or the web.

Re:Stop using the Shell

[Bruce Perens]

There are times when bash/ash/dash... are all that is available or can be made available

Because you're running Busybox. Which means it's my fault :-)

The evolutionary successor of busybox, which I've been thinking about for a while, will not use the shell language. We can raise the bar significantly.

Re:Stop using the Shell

[martin-boundary]

You're comparing end results, not interfaces.

Sure you can get the same result, but the syntactic sugar in your example is much more verbose, and conceptually more complex.

For each of the three components, there's a mental context switch (File object on the left, reader object in the middle, and substitution method on the right).

The shell language does the right thing by handling components more uniformly (ie they all have STDIN/STDOUT regardless of the nature of the command). The user needs to know what each command will do, but he does not need to know if the result is an array object, or a stream ojbect, or a file object etc.

The shell also has less redundancy. Compare cat foo.txt with File("foo.txt"), there should be no need for both parentheses and quotes. Now in the wider scheme of Ruby this redundancy makes perfect sense, but users don't need all this, only programmers do.

Users need the bare minimum to communicate with the machine in a language that takes 30 seconds or less to type (or speak in a microphone...), but still lets them do as much as possible.

It's an interface issue, it's got little to do with the range of things that can be done in the language. Ruby is much more powerful than bash, but bash is still better at starting and stopping programs (and rc is better than bash...).

Re:None, I have given up bash scripting

[Hatta]

Does `find . -print0 | xargs -0` really qualify as "serious hack magic"?

Why

[silas_moeckel]

The CLI is powerful because it's a CLI, you do not need or want pretty dialog boxes. Help is whats available with man --help usefull errors messages and the contents of /var/log. It works over 9600 baud serial and works pretty well so you can ssh from your smartphone with 1 bar and fix something at 3am before the GUI would have time to come up to a login screen. A good CLI expects things to be piped into and out of it and can get any required information via the command line. The power of the CLI is that you can chain bits together run to do things or wrap scripts around other scripts and do useful work.

You point to a 20 year old book that mostly bitches about how slow/ugly X is, guess what things have come a long way, I run one laptop with native X and it looks good is responsive I export X all the time over ssh to my primary desktops. Take a step back and think why your trying to shoehorn GUI functions onto a CLI if you really need to do it look at some of the toolkits that can detect if there is a X server present and use that fallback to text gui and run entirely headless by pure command line but think long and hard about why you would want to do this.

tools I found useful

[tpwch]

Here are some random things I find useful, related to user interaction (mostly becuase it notifies the user):

Oven timer:
sleep $((20*60)); xmessage "Dinner is done"

Quick macro for automating some repetitive task in a program:
xdotool type "something"; xdotool key Return; xdotool mousemove $x $y; xdotool click 1; (and so on)

Copying a file to/from the clipboard (can also copy from /topipe, so the output of any command). Faster than opening a text editor:
xclip -in file

Notifying me when some specific thing changed on a website:
CHECKLINE="$(curl -s http://somewebsite.org/somepage.html | grep "currently undergoing maintenence")"
while true; do
    sleep 120
    [ -z "$CHECKLINE" ] && xmessage "somewebsite is open again" && exit
done

Or just checking for changes in general (I use this for notifying me when something changed when tracking something I ordered, so I know the minute the package is ready to get picked up at the post office):
while true; do
    OLD_MD5=${MD5}
    CONTENT=$(elinks -dump 1 -dump-charset iso-8859-1 "http://someurl.com/track?id=someid")
    MD5=$(echo -n $CONTENT | md5sum -)

    [ "${MD5}" != "${OLD_MD5}" ] && {
        xmessage "$(printf "New action: :\n\n${CONTENT}")"
    }
    sleep 120
done

If you don't want to interrupt what you're doing with a pop-up you can pipe it to osd_cat instead to have the text appear over whatever program you're currently working with. Adding a few beep; beep; beep; beep; is also a good way to get your attention if you're not paying 100% attention to your computer all the time.

Best book for this -- hands down.

[CFD339]

Linux Shell Scripting with Bash
by Ken O. Burtch
Sams Publishing

One of only two "computer" books I've ever been able to just sit down and read rather than just using as reference (the other being Kathy Sierra's "Head First Java" -- which is amazing).

Ken does a fantastic job at putting "just the right" level of background, detail, context, and and depth for someone new to shell scripting to get started, then to use the book as a reference for all the traditional tools (sed, awk, etc..).

I've bought two copies, one for me and one I gave to someone else who wanted to learn how to do this stuff.

Re:Best book for this -- hands down.

[actionbastard]

The only bash scripting guide you will ever need:

http://tldp.org/guides.html

free as in beer.

Re:None, I have given up bash scripting

[DaleGlass]

Use:

tar tf file.tar | xargs -d "\n" rm

That will work unless the filenames contain newlines in them.

Re:None, I have given up bash scripting

[vux984]

Simple solution: don't use filenames with spaces in them. They're an incredibly stupid idea. If you need something that looks like a space, use an underscore. The same practice has been done in C since the early 70s, since having spaces inside C tokens would be stupid.

Simpler solution. Don't use computers.

Seriously now. You expect all the end users out in the world to stop using spaces... just so your script works?

Re:None, I have given up bash scripting

[CAIMLAS]

You mean something like perl? Or maybe python?

My vote is for perl. It's more common in a "base install" than any other shell (in the BSDs and most Linux distros) and has a non-trivial amount of power. It's good at dealing with path and input permutations and you can interface it with pretty much anything. Hell, pcre came from perl, and that's used almost everywhere these days: it's got a lot of things right for the little that's wrong, at least in terms of being a good scripting language.

I avoid "shell" scripting (csh, sh, bash) if at all possible, too. The contortions necessary to do the frequently-necessary evaluations takes quite a bit longer, even with a chain of awk/sed/grep and the like. Unlike those languages, perl is entirely self-contained and does not have any system-specific oddities (eg. with a shell script, many system binaries are different and an option/parameter pair on one system might do something entirely different on another - or not work at all).

I realize perl can often (usually) be difficult to read. But for my purposes, it's good enough, because I'm a bit of a prolific comment writer as a matter of process.

"Interaction"

[betterunixthanunix]

The way I look at it is this: the "interaction" may actually be with another script. The whole abstraction that Unix-like OSes enforce, at least with file based IO, is that it is irrelevant what is on the other side of a file descriptor -- a disk, a pipe, a user, a socket, or something else entirely.

Of course, this all starts to break down with GUIs.

Consistency is the only spice ...

[bhepple]

As said previously, scripts are scripts and don't often need a GUI. But for grep's sake, make them consistent!!! The only spicing up _really_ needed are some standards:

o output errors to STDERR; normal output to STDOUT
o include (-h, --help) processing - and send it to STDOUT so the help can be piped to 'less'
o use getopt(1) or process-getopt(1) so that options on the CLI parse in a predictable and standard way
o keep it terse except for errors so that the user can easily see if it worked or not without scanning vast output
o provide a --verbose option to help with tracking down those errors

... and the most annoying thing of all - make sure --help _always_ works, even if the script body itself can't - at least the user can then be told about what the prerequisites are.
Head over to http://mywiki.wooledge.org/BashFAQ for much wisdom on how to write better bash scripts.

Re:None, I have given up bash scripting

[camperdave]

You still need to work with the user's files, which will inevitably have spaces in them. If a space is a valid character in the filesystem then your scripts need to reflect that. Erroring out is not a valid solution.