Slashdot Mirror
Beginning Portable Shell Scripting
Joe MacDonald writes "The earliest UNIX shell I encountered was the Bourne shell on a SPARCStation
2 at my university. As with many students of my generation, prior to that
nearly all of my exposure to command line interfaces was some variant of DOS.
I was quite proficient with the primitive scripting language that was available
on such platforms but I immediately felt far out of my depth in this new
environment. The commands seemed arcane, possibly dangerous, and almost
immediately I regretted stepping into this unfamiliar wilderness without some
sort of guide." Read below for the rest of Joe's thoughts.
Beginning Portable Shell Scripting: From Novice to Professional
author
Peter Seebach
pages
376
publisher
Apress
rating
4/5
reviewer
Joe MacDonald
ISBN
1-4302-1043-5
summary
A guide on how to write portable shell scripts.
It was probably a few weeks after that first, rough introduction that I returned for another round with this strange but somehow seductive tool, armed with a book I'd found and a determination to learn it's secrets. I had no idea then that seventeen years later I'd still be learning new tricks, discovering new features and taking so much pleasure from sharing what I've learned with others. In fact, in those early forays into the realm of shells and scripting, I didn't even really have a strong concept of the separation between the shell and the operating system, so at the time I couldn't have conceived of how much fun I would have in later years discussing and debating the relative strengths and weakness of shells with friends and colleagues, but it is probably my favorite touchstone of computer geek conversation. Discussion of shell features, scripting tricks and semantics almost always result in my learning something new and interesting and having a new tool to add to my collection.
Peter's book, Beginning Portable Shell Scripting, therefore may sound like something intended as a gentle introduction, aimed at the initiate — the sort of text I'd been seeking to carry with me when I first attempted to write what I thought of as "batch files" on that now-ancient UNIX machine — but there's more truth in the subtitle, From Novice to Professional, than one might expect. He writes in an accessible, at times conversational, style and presents detailed technical information alongside a mixture of anecdotes and historical detail that does more than simply serve as a technical reference, it helps the reader understand a great deal about why things are the way they are. It was such an entertaining read that I frequently found myself skipping ahead, reading a section I knew was coming up, then resisting the urge to just keep going from that point. The first of these I encountered on page 18 in which he discusses the relative portability of printf in shell scripts. I knew what he knew, it's clearly non-portable and should be avoided, and thoroughly enjoyed the explanation of how he determined his (and by extension my) assumption was in error. Another on page 108 is the sort of good advice all UNIX users, not just those aiming to write good scripts, should take to heart. Many times, though, I've related precisely the same advice to colleagues to be met with confused stares, so it certainly bears repeating.
This book is a desktop reference in the truest sense of the term for me, it is an interesting, at times laugh-out-loud amusing, discussion of how to write shell scripts that will work on the widest possible range of Bourne-derived and POSIXly correct shells and why this is a desirable goal. In true UNIX tradition, the author doesn't provide simply a set of rules, but guidelines that will help you find your own way through the task of creating portable, maintainable shell scripts.
The real meat of the book begins in Chapter 3 (more on Chapter 2 in a moment) with a discussion of control structures and redirection, the latter being perhaps the defining characteristic of UNIX command line interfaces. I struggled somewhat with trying to decide if redirection would be better discussed after the material on how the shell parses tokens, presented in the first part of Chapter 4, but it does seem that the correct logical grouping is the one presented. It would be easy to get lost, for example, in the semantics of why the same streams of redirection tokens behave differently on different shells, but the key concept in the early chapters is that of many tools, each doing a specific task, working in concert. That objective is achieved quite effectively.
Chapters 5 and 6 go into detail (possibly too much for some, just right in my opinion) on how UNIX executes shells and how shells can spawn other shells, the costs and the benefits and the available alternatives for one to make an informed decision. Frequently there isn't one right answer whether some activity is better done in a script, in a shell function or in a subshell, but the material here will certainly aid in making those determinations. My personal bias being almost always toward writing a shell function — perhaps an indication I've had too much exposure to C programming, perhaps more due to a frugal upbringing and my own sense that spawning a whole new shell to do something is overkill — had me wishing for a larger section on the value of such constructs, but there should be enough there for me to win some converts to my cause.
By far the sections I learned the most from, however, would be Chapter 7: Shell Language Portability and Chapter 8: Utility Portability since I actively avoid exposure to other shells. I have my two preferred options and a third that I will use when presented with no alternative. While this does mean I know "my own" shells very well, it also means that I often bump into the furniture, so to speak, when I find myself using a new shell. These chapters haven't been immediately useful to me, but I know they're the ones that I'll be turning to in the future, I've needed something like them in the not-too-distant past, after all.
The final three chapters assemble the information presented in the earlier sections and suggest a sort of "best practices" approach to writing scripts. Concepts like "degrade gracefully" seem like pretty fundamental ideas when you hear them but I frequently find myself writing functions or scripts that don't do that at all when intended for a limited, usually singular, audience. It may seem like an okay idea when you're doing something for your own use, but when you write a complex function that works then discover a bug in it two or three years late and you have to return to fix it, it can be just as helpful for it to simply fail in an informative way as it would be to have detailed comments explaining the intent and the mechanics.
Truly, there's something here for everyone. In my office I'm considered something of an expert when it comes to complex regular expressions and the subtleties of using them in different editors and tools, but Chapter 2 and Appendix C both had enough new material in them that I found myself frequently making notes in the margins.
I have many, many books in my bookshelf in my office but nearly none on my desk. Beginning Portable Shell Scripting is going to be one of the very few that will be spending a great deal of time lying flat on my desk, in easy arm-reach.
You can purchase Beginning Portable Shell Scripting from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
It was probably a few weeks after that first, rough introduction that I returned for another round with this strange but somehow seductive tool, armed with a book I'd found and a determination to learn it's secrets. I had no idea then that seventeen years later I'd still be learning new tricks, discovering new features and taking so much pleasure from sharing what I've learned with others. In fact, in those early forays into the realm of shells and scripting, I didn't even really have a strong concept of the separation between the shell and the operating system, so at the time I couldn't have conceived of how much fun I would have in later years discussing and debating the relative strengths and weakness of shells with friends and colleagues, but it is probably my favorite touchstone of computer geek conversation. Discussion of shell features, scripting tricks and semantics almost always result in my learning something new and interesting and having a new tool to add to my collection.
Peter's book, Beginning Portable Shell Scripting, therefore may sound like something intended as a gentle introduction, aimed at the initiate — the sort of text I'd been seeking to carry with me when I first attempted to write what I thought of as "batch files" on that now-ancient UNIX machine — but there's more truth in the subtitle, From Novice to Professional, than one might expect. He writes in an accessible, at times conversational, style and presents detailed technical information alongside a mixture of anecdotes and historical detail that does more than simply serve as a technical reference, it helps the reader understand a great deal about why things are the way they are. It was such an entertaining read that I frequently found myself skipping ahead, reading a section I knew was coming up, then resisting the urge to just keep going from that point. The first of these I encountered on page 18 in which he discusses the relative portability of printf in shell scripts. I knew what he knew, it's clearly non-portable and should be avoided, and thoroughly enjoyed the explanation of how he determined his (and by extension my) assumption was in error. Another on page 108 is the sort of good advice all UNIX users, not just those aiming to write good scripts, should take to heart. Many times, though, I've related precisely the same advice to colleagues to be met with confused stares, so it certainly bears repeating.
This book is a desktop reference in the truest sense of the term for me, it is an interesting, at times laugh-out-loud amusing, discussion of how to write shell scripts that will work on the widest possible range of Bourne-derived and POSIXly correct shells and why this is a desirable goal. In true UNIX tradition, the author doesn't provide simply a set of rules, but guidelines that will help you find your own way through the task of creating portable, maintainable shell scripts.
The real meat of the book begins in Chapter 3 (more on Chapter 2 in a moment) with a discussion of control structures and redirection, the latter being perhaps the defining characteristic of UNIX command line interfaces. I struggled somewhat with trying to decide if redirection would be better discussed after the material on how the shell parses tokens, presented in the first part of Chapter 4, but it does seem that the correct logical grouping is the one presented. It would be easy to get lost, for example, in the semantics of why the same streams of redirection tokens behave differently on different shells, but the key concept in the early chapters is that of many tools, each doing a specific task, working in concert. That objective is achieved quite effectively.
Chapters 5 and 6 go into detail (possibly too much for some, just right in my opinion) on how UNIX executes shells and how shells can spawn other shells, the costs and the benefits and the available alternatives for one to make an informed decision. Frequently there isn't one right answer whether some activity is better done in a script, in a shell function or in a subshell, but the material here will certainly aid in making those determinations. My personal bias being almost always toward writing a shell function — perhaps an indication I've had too much exposure to C programming, perhaps more due to a frugal upbringing and my own sense that spawning a whole new shell to do something is overkill — had me wishing for a larger section on the value of such constructs, but there should be enough there for me to win some converts to my cause.
By far the sections I learned the most from, however, would be Chapter 7: Shell Language Portability and Chapter 8: Utility Portability since I actively avoid exposure to other shells. I have my two preferred options and a third that I will use when presented with no alternative. While this does mean I know "my own" shells very well, it also means that I often bump into the furniture, so to speak, when I find myself using a new shell. These chapters haven't been immediately useful to me, but I know they're the ones that I'll be turning to in the future, I've needed something like them in the not-too-distant past, after all.
The final three chapters assemble the information presented in the earlier sections and suggest a sort of "best practices" approach to writing scripts. Concepts like "degrade gracefully" seem like pretty fundamental ideas when you hear them but I frequently find myself writing functions or scripts that don't do that at all when intended for a limited, usually singular, audience. It may seem like an okay idea when you're doing something for your own use, but when you write a complex function that works then discover a bug in it two or three years late and you have to return to fix it, it can be just as helpful for it to simply fail in an informative way as it would be to have detailed comments explaining the intent and the mechanics.
Truly, there's something here for everyone. In my office I'm considered something of an expert when it comes to complex regular expressions and the subtleties of using them in different editors and tools, but Chapter 2 and Appendix C both had enough new material in them that I found myself frequently making notes in the margins.
I have many, many books in my bookshelf in my office but nearly none on my desk. Beginning Portable Shell Scripting is going to be one of the very few that will be spending a great deal of time lying flat on my desk, in easy arm-reach.
You can purchase Beginning Portable Shell Scripting from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Even if a shell script is portable, the underlying OS will always fail to be for any task significantly complex to do useful work.
does it say more than this? http://www.dreamsyssoft.com/unix-shell-scripting/intro-tutorial.php
I was behind the comfortable wall of a multi user system even with disk quotas. We had access (read only) to our backups and pretty much coudn't screw up. Bourne was standard but I used zsh and later switched to bash. Man pages was all we had to study and we competed in learning as much as possible, writing cool scripts, exploiting the system in various ways. The system was rooted and sabotaged a couple of times but nothing in it was critical in any way. A playground really. I remember the tingling feeling of the 'su' command, the only really dangerous one. Nice that they write books about this but nothing beats the good old feeling of having worked HARD to get the knowledge. The cry of success is so much louder.
Don't do it.
Shell scripts have horrible error handling, and quickly become a maintenance nightmare. These days, e.g. Python is installed everywhere you need to go.
Just do this:
def c(s): os.system(c)
and you have mostly covered the area where shell scripts excel. You can still write minimal "shell scripts" inside c().
Unluckily, you still *need* to grok shell scripting to some extent, or at least be able to read them. Just don't write them if you can help it.
Save your wrists today - switch to Dvorak
I find shell scripting have a nasty habit of not working quiet right when moved between Linux, the BSDs and Mac to be safe, and it's always a pain to write scripts that work correctly with spaces in file names.
Why isn't there (or is there?) a simple python cheat guide, or library, that do the same things as grep, awk, find, mv and xargs?
Combination - fun iPhone puzzling
One does not write a web server in Bash, one wraps a webserver in it, pipes its output to a log analyzer, restarts it automatically if it crashes, and so on.
The most important part of any UNIX-derived shell langauge is not its syntax or power but the fact it lets you construct large ad-hoc applications out of a toolbox of tens of thousands of pieces.
This is where all other operating systems (that I've ever used, and that's 30-40) have failed.
Any serious developer should know several glue languages, Unix shells being the most flexible and accessible.
My blog
I'm sure all of us at one time or another have had a shell script we've relied on for years fail miserably when bringing it to a new environment. The sad fact is, shell scripts were never meant to be programming languages in and of themselves, and I wonder if, knowing what we know now, it isn't overly ego-driven and masochistic to try to take this feature -- tied to a shell which is tied to an operating system -- and promote it beyond its competency.
So, let's say we take the PSS principles seriously, and abstract away any non-platform-agnostic features you can think of. A few years down the road, you've got PSS all over the shop and you want to upgrade to a different platform nominally supporting your shell of choice. Even if you shake off PSS features you thought could create incompatibilities, you discover the new system buffers differently. Or added a parameter somewhere. Point is, if you went with something like Perl which is designed for cross-compatibility you would have been fine, but now you're all wet.
Shell scripting is good for what it's meant for, but at the risk of oversimplifying with a tool analogy, I'm concerned that this falls into the trap of "If all you have is a hammer, all your problems look like nails".
It's true. I work with a guy (rather old himself) that writes on the Korn shell because it's the only shell that is included on pretty much all Unix based OSs, including Linux. (and Solaris, HP-UX, and AIX, which we also use).
Not so much a guide, but "man bash"* or "info coreutils" are incredible resources for those common bits you routinely forget. If you stick to what's in coreutils, I think you've got a pretty good chance of portability.
*for systems that use bash as the shell, of course. Frankly, I think there might be too much in there, though and some of the builtins should have their own man pages.
Can you be Even More Awesome?!
Actually, that's useful in bash too sometimes. :)
Anyway, the reason I bother is this:
I wrote a bunch of scripts in 1992 or so. I'm still using them. I haven't touched any of them in years, except for one update to deal with Linux differing from BSD in where the actual errno definitions are.
I don't have to worry about what shells are installed, I don't have to guess whether bash is "/bin/bash" or "/usr/pkg/bin/bash", I don't have to wonder whether the sysadmin bothered to install the "GNU utilities". I just copy my scripts into ~/bin and go.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
Not everybody has bash and/or zsh installed.
Do you even lift?
These aren't the 'roids you're looking for.
When it's free -as in beer: 'Advanced bash Scripting Guide - Mendel Cooper'.
I've got your sig, right here.
First off, in the interests of full disclosure, Joe MacDonald is one of my coworkers.
Anyway... The big surprise to me was the word "Beginning", which somehow showed up in the publisher's cover pages, but which I didn't know about during the writing process. My tech reviewer was Gary V. Vaughan (yes, the autoconf/libtool guy). I bounced material off a number of seasoned expert scripters during the process. Basically, my goal was to write a book that I could use as a reference, and which would teach me something.
I succeeded beyond my wildest dreams. The discovery that printf(1) is essentially universal these days was a complete shock to me; I had no idea it was portable. During my first pass on the regular expressions section, I started by writing down what I believed I knew about sed, awk, etcetera. Then I tested it... and had to revise most of it. A number of things I was used to were GNU or BSD extensions. When Gary sent the chapter back for tech review, he'd flagged most of these things, because he "knew" the same things I did.
So everything there should be pretty thoroughly checked out now -- I realized very early on that this field was full of things "everyone knows". Many of them wrong. We tested things on a field of around 30 different versions of Unix and Linux. We tested them on unusual installs, we tested them on special cases.
Why?
Because portable shell is an incredibly portable language, and sometimes that matters. Because shell is a very powerful language, too. Because sometimes shell is all you have -- and because sometimes shell is more expressive for a task than your other choices. I love me some C, I program in C by preference much of the time -- but there are a lot of tasks I'll do in shell rather than in C. There are similarly many tasks I'd rather write in shell than in perl. Shell is what make uses to run commands, and sometimes you need to write something clever in shell because make doesn't have quite the right feature.
In short, it's something I have found consistently useful, day in and day out, for about twenty years now. I just wish I'd realized how much more there was to learn years ago, I coulda saved a lot of time... :)
And, to answer a question hinted at earlier: Yes, now that this book exists, I keep a copy on my desk. I look stuff up in it about once a week.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
why deal with limited shell scripting when you have something so batteries included and universal as Python
Sounds great.
Now, off the top of your head, what happens to variables set in the last component of a pipeline in ksh? Do you know whether it's the same on systems where "/bin/ksh" is actually pdksh? ... Oh, and just for reference, about half the Linux systems our IT department installs don't have ksh. No, I don't know why. (I only know because I can't log into them because my default login shell is /bin/ksh...)
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
But you gotta love that mic stand H.R. Giger created for Jonathan Davis.
I don't care why you're posting AC
Wherever you want to do an awful lot of things with the input and output of) system utilities and./or bash builtins. Look at the gargantuan effort that is the Knoppix boot scripts - I seriously doubt it would make sense to rewrite those in Python or Perl, since nearly every line is a pipe between utilities or redirection. And it works well.
Digital Research threatened legal action, claiming PC/MS-DOS to be too similar to CP/M. IBM settled by agreeing to sell their x86 version of CP/M, CP/M-86, alongside PC-DOS.
So Digital did threaten, but there was a settlement. DR-DOS on Wikipeida
Mod points: Guaranteed to remove your sense of humor.
Side effects may include gullibility and temporary retardation
Want your shell to be portable? Write it in Korn Shell. You will find this shell on 15 year old *NIX boxes and the script will still work with bash on Linux.
Except when it doesn't.
There are differences between bash and ksh, and between different versions of ksh. In some cases the difference means syntax errors, but sometimes it can be more subtle and a script written for ksh runs just fine but doesn't do in bash what it did in ksh, or doesn't do in pdksh what it does in ksh88.
It is also not the case that ksh exists everywhere. It wasn't on MacOS until 10.4. It isn't on FreeBSD by default. It isn't on many Linux boxes, particularly the small network devices that use stripped-down systems. If you don't work in a broad variety of systems you may not need to understand the issues of shell portability, but if you don't understand them you aren't really prepared to work on different sorts of systems.
I've heard that said for 15 years, that might have been true 15 years ago, but now ... ?
Saying "I love C but there are things that are better in shell" is completely anachronistic.
Seriously. The question's been settled for over 20 years.
And there are other languages, you know. The question is more whether to use Python (for example) instead of shell in some cases, and when.
Why bother with portable shell scripts, seriously? Everybody has bash installed, and/or zsh that is mostly compatible, and even then you have bash anyway. I understand retro-nostalgia and all that, but necrophilia is overrated
False.
The majority of systems I work on these days and the majority of systems I have worked on since the mid 90's have not had bash installed. That includes systems running FreeBSD, NetBSD, OpenBSD, AIX, Tru64, Solaris, MacOS, and even Linux. Current versions of some of those will usually have bash in a default installation, but some still do not. Companies running stable systems as important parts of their business do not generally upgrade their OS's just for the sake of novelty. Running older systems isn't usually about nostalgia or necrophilia, it's more often about not having any compelling reason to upgrade. There is also a system hygiene practice common on the BSD's of keeping the base system minimal and only adding on what is needed, a practice that helps in keeping systems secure and stable because they are easier to fully understand. This is also common in many virtualization environments, where a running OS instance is likely to exist for a very narrow purpose and intentionally have a stripped-down set of utilities fit to that narrow purpose.
(And no, trying to enforce a "no newlines in filenames" policy doesn't work. Been there, tried that.)
Recent versions of MacOSX have bash by default. By recent I mean 10.4 had bash, and probably 10.3 but I'm not sure.
All Linux distribs have had bash installed by default for ever. And by all I mean 99.999% of the installed base, I'm sure you can find a silly exception.
Recent versions of Tru64 ... do not exist.
As for the BSDs, Netcraft confirms it, .. err. I don't know, what's their default shell?
And as for Solaris, its default shell -- a Soviet-era knock off of the original Unics v1.0 -- is so fucktarded that nobody in their right mind keeps it as their default shell, and I've always seen bash or zsh instead.
There is also a system hygiene practice common on the BSD's of keeping the base system minimal and only adding on what is needed, a practice that helps in keeping systems secure and stable because they are easier to fully understand.
Yeah, you're right, installing such an experimental, little used and unmaintained software package as bash is irresponsible. /facepalm
Linux is obliterating commercial Unices.
It has in part something to do with the backspace key working out of the box without typing "stty erase ^H" every time. (Have they fixed that on Solaris, yet?)
I know paleounices suck. Does it matter to 99% of shell writers, is the question.
ksh isn't part of Linux normally. Sometime's it's pdksh and sometimes it's genuine AT&T ksh. Same with cygwin. You have to explicitly add it too.
I had to port some shell scripts from Ultrix to SunOS, Solaris, HP-UX, Irix and OSF/1. ksh wasn't part of SunOS. However, we had bought a license for ksh and put it in /bin/ksh everywhere.
I needed functions and Ultrix /bin/sh didn't have them. IIRC /bin/sh5 (?) did.
Anyways, ksh had to be licensed back then. This was Linux 1.09 era. pdksh wasn't even close to being a substitute. bash was 1.x.
Also, sometimes /bin/sh is ash.....
AIX uses ksh.
Do you even lift?
These aren't the 'roids you're looking for.
All the DBA's I have worked with (as well as C programmers worth their salt) have tended to use ksh by default.
I think Oracle's documentation always uses korn, and maybe I have just worked with a bunch of old IBM'ers..
Recent versions of MacOSX have bash by default. By recent I mean 10.4 had bash, and probably 10.3 but I'm not sure.
10.3 had it. Prior versions did not.
All Linux distribs have had bash installed by default for ever. And by all I mean 99.999% of the installed base, I'm sure you can find a silly exception.
A large fraction of the Linux systems I work with are embedded versions which use things which seem to be descended from the BSD (Almquist) sh. Talking about installed base numbers is silly, because it is probably also true that 90%+ of those systems have never been seen by a competent sysadmin who has any intention of ever using anything Unixy that isn't a major Linux distro. They might as well be Windows for all of the relevance of portability....
Recent versions of Tru64 ... do not exist.
Obviously you are not paying attention. There was a 5.1bsomething maintenance release by HP in the last few months, and they have said that they will continue support through 2012. Whether the current version has bash I don't know, but in the real world there are still major companies running 4.0f, which I am absolutely sure does not have bash. I would not suggest the project of installing bash on Tru64 4.0f to anyone I liked.
As for the BSDs, Netcraft confirms it, .. err. I don't know, what's their default shell?
FreeBSD uses the Almquist sh and I expect the others do as well. The product of Bourne sh and ksh procreating while watching kinky csh porn...
And as for Solaris, its default shell -- a Soviet-era knock off of the original Unics v1.0 --
Hardly a knockoff. That's the real Bourne Shell, built with genuine AT&T code.
is so fucktarded that nobody in their right mind keeps it as their default shell, and I've always seen bash or zsh instead.
Which is a career-limiting choice on Solaris 9 or earlier if done to root. People who fancy themselves sysadmins have wisely been fired for it. I don't recall if Sol10 includes bash or zsh, but I've never seen an experienced Solaris admin picking either of them for personal use and it is still prudent to stick with sh for scripts that may have to run under odd circumstances and ksh for anything that might get put on a Sol9 box. The selection of a shell for non-root interactive use is far more a matter of personal taste than the selection of what you put after #! on the first line of scripts, particularly ones that need to be functional at bad times.
There is also a system hygiene practice common on the BSD's of keeping the base system minimal and only adding on what is needed, a practice that helps in keeping systems secure and stable because they are easier to fully understand.
Yeah, you're right, installing such an experimental, little used and unmaintained software package as bash is irresponsible. /facepalm
it's not necessarily irresponsible, but why bother? It's one more thing to track, because even if bash hasn't had a real vulnerability identified in a long time, it still could have one tomorrow. It's a little more backup space. It's one more thing to explain to the security auditors who treat every variance from a stock installation as something that needs explaining, on an annual basis. If it's the last Tru64 box with a lot of interactive logins in a largely-linux environment, installing bash makes a lot of sense. For a FreeBSD jail that handles one website, it probably doesn't make sense.
My first Unix shell was the Mashey shell that preceded Bourne shell, on a PDP-11 running v6. :-) Before that I'd used RSTS-11, HPUX Basic, the IBM System 34 shell (OCL was fairly powerful, though less than sh), CMS, Plato, and various things with punch cards. Unix shell seemed powerful, flexible, and really really appropriate. I later used other shells like TSO and occasionally CP/M; I forget if I used VMS and ddt before DOS or only after.
Eventually MS-DOS came around, and it was painfully clumsy. It was like they'd seen Unix on a viewgraph, but hadn't gotten the concepts and had left most of the interesting commands out - they had hardware that was about as powerful as a small PDP-11, dedicated to one user!, and an integrated graphics system as well, and yet the environment was hopelessly lame. There were directories, but they didn't have the same syntax for each level of the file system, there wasn't the equivalent of "cd ..", you could use variables but they were really limited, you couldn't pipe commands together, and of course regular expressions were missing. MS-DOS 2.11 and 3 were better than MS-DOS 1. People claimed it was friendly, but it was almost as bad as being on a punch-card system like JES-3 or HASP except that you had a backspace key.
A few years later the Macintosh people came around and claimed that _they_ were more friendly than Unix, and they were correct - the system was less powerful, but it really was helpful. VMS was ugly and evil, but it was reasonably powerful, and had a very good help system.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
VMS looked far more like RSTS-11 or RSX-11 than like Unix. I don't know if CP/M was inspired by VMS or by its predecessors. DOS certainly wasn't inspired by Unix, at least in the original versions; they clearly didn't have any of the concepts, even though they had a machine that was almost as powerful as a small PDP-11 and more powerful than an IBM System/34 (which had a quite nice shell), and later versions of DOS (around 3ish?) started emulating some Unix shell syntax while still not having the underpinnings to do the job.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
I recall one script that broke because ls on one system put an extra leading space in its output. (I think it was an old sgi box.)
Yes, there is posix. So if you must write scripts, try to stick to that. But I have some systems here that are pretty old, and I can't be sure that they are posix compliant.
When I have to use unix tools, I try to arrange for the gnu utilities to be built and installed, so I only have to consider one set of man pages, and not a set for each platform the script has to run on. Similarly, makefile portability is difficult because make behaviour varies between platforms e.g. vpath, so I try to arrange for gnu make to be installed.
In response to those people who just shell out from python: that's a method that I have seen a few times among programmers transitioning from shell to python scripting, including myself. You end up with a bash script where every line is wrapped with os.popen calls. No, the way to write a python script is to look in the library doco for the modules that have the functionality you want. As someone pointed out, find -> os.walk for example. Plus os.popen or os.subprocess from python is still vulnerable to the non-portability that results from behaviour variation of unix tools across platforms.
My preference these days is to write scripts in python, but I agree that there will be systems without python, where shell is required. In those cases, it is great to know how write scripts with better portability, so the book in the post is likely a good addition to the library.
Exactly -- you can't just "write for ksh", and even if you do, it's not universal.
You can do pretty well writing for a common subset of ksh88/pdksh, but I'd rather do the extra few minutes' work and write for plain old POSIX shell by default.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
So a lot of scripts tend to make the same mistake. They set up the environment on the first line, then use that throughout the script. Another script using the same stuff will have to do the same thing.
So you put this at the top of your script:
FOO="path/to/some/stuff"
And in lots of other scripts. WRONG!
Create "/opt/yourcompany/etc/env.d/foo" and put it in there.
Create "/opt/yourcompany/etc/profile" which executes env.d/*
Put "source /opt/yourcompany/etc/profile" at the top of all your scripts.
Wow.. now your scripts are relocatable!
HTH HAND burp.
I wrote my first program at the age of six, and I still can't work out how this website works.