A Real Bourne Shell for Linux?

the_code_poet asks: "I'm a lead developer for a software development company, and one of my responsibilities has been writing an installer for our product (of which Linux is one of the platforms). In keeping with UNIX tradition, the installer is written in shell (thats /bin/sh), but as many of you know there is no Bourne shell for Linux - only bash. This has caused inconsistencies (mostly bugs in bash) when writing a generic UNIX sh script that works fine on commerical *NIX's." For a semi-complete list of differences between bash and sh, you will want to check out section C1 of the Bourne Again Shell FAQ. To be honest, I have yet to run into much trouble with a script starting with #!/bin/sh with /bin/bash, and I've been using the latter for years. If any of you have had problems related to this, please tell us what the problem was and how you solved it. Also: would anyone out there be interested in writing a real Bourne Shell for Linux?

"On every distro I've ever seen /bin/sh is just a soft link to /bin/bash. If bash is invoked with sh as its name (argv[0]) then its supposed to act like Bourne - but that just doesnt happen (for example: export FOO=bar is *not* valid Bourne shell syntax, you must say FOO=bar; export FOO)

Do you think that the startup scripts for most distributions would break because, even though they say #!/bin/sh at the top, they REALLY mean #!/bin/bash?

Given that there is no real Bourne shell for Linux, and that bash has an exhorbitant file size. Quoting bash's man page, here: '...it's too big and too slow' for something that is to be used as the defacto-standard shell for scripting, do you think its a worthy venture to set out to write a small, tight, pure Bourne shell?

*asbestos disclaimer*: This has nothing to with Bash as an interactive user shell and has nothing to do with a holy war over who's favorite shell is better than whomever's."

While doing a small bit of research on this question, I noted there was another Bourn-compatible shell out there called "ash", yet it's billed as doing "some things better and some things worse than bash". Does anyone use it, and find it better than bash for their shell scripting needs?

Comments

Easy.

[Flarners]

1. Download FreeBSD source.
2. cd /usr/src/bin/sh; make; make install

AFAIK the BSD Bourne shell is more or less the same as the real Bourne shell.

/bin/bash

[Z4rd0Z]

Starting your scripts with #!/bin/bash shows that you live in a Linux-centric world. No other OS puts bash under /bin. Since there's normally a link from bash to sh, it really makes sense for compatibility reasons to use #!/bin/sh. Of course, if you use bash-specific features in your scripts, your scripts won't be portable anyway and it doesn't really matter, I suppose.

It's a non-issue.

[mindstrm]

You are looking for a solution to a problem that does not exist.

bash is backward compatable to sh. Period.

Yes, you can do things in bash you can't do in sh, but not vice-versa.

If you write your scripts for stock Bourne, they will run fine under bash.

Re:It's a non-issue.

[pete-classic]

What you are saying is valid, but it misses the point.

Even if every valid bourne script runs perfectly under bash there is still a problem.

That problem is that an interpreter enforces valid syntax. It would be nice if everyone who wanted to write a portable (read bourne) script spent a month studying and was totally zen-ed out on the subject. But that 'aint the real world.

I personally could benefit from a real sh on Linux, since it is the only *NIX I have, and I can't test scripts for sh compatibility without it.

To be more specific, I wrote a bash script that was included in an OSS project. The maintainer wanted me to change the first line from "#!/bin/bash" to "#!/bin/sh" "for compatibility." I told him that I wasn't comfortable with this since the script wasn't tested on sh. He said he tested it and it worked fine. I asked him if he really tested it with sh or with "/bin/sh -> /bin/bash". Turned out it was the latter. Apparently he made the change anyway.

A while later we got a bug report on the list to the effect "your script is not a valid sh script." Luckily the guy sent a diff, and I tested it with bash. I guess it works with sh now too, but I still don't know for sure.

See how it isn't a non-issue, at least for me?

-Peter

Re:It's a non-issue.

[seebs]

Embrace and extend. There is no way to write a script in bash and have bash warn you if you're using an extension, so they tend to creep into scripts. Embrace and extend is evil; the compatability mode should be *pure* sh, with no extensions.

Re:/bin/bash - It all sounds so brutal

[Anonymous Coward]

hash bang slash bin slash bash

Slackware uses ash for toolset

[dangermouse]

Slackware uses ash because (a) it's tiny and (b) it's restrictively Bourne-compatible, which means you won't write some tool that is accidentally dependant on bash.

Basically, Slackware uses ash for development for the exact reasons you're looking for a Bourne-compatible shell.

Have fun.

bash as /bin/sh

[zdzichu]

"On every distro I've ever seen /bin/sh is just a soft link to /bin/bash."
You haven't seen Polished Linux Distribution. PLD's /bin/sh is not bash, and all 'bashisms' are removed from distribution's scripts - it's all plain /bin/sh.

(Plus, PLD is fully IPV6 ready :)

Bourne shell or POSIX shell?

[psamuels]

Do you think that the startup scripts for most distributions would break because, even though they say #!/bin/sh at the top, they REALLY mean #!/bin/bash?

Many would. However, note that POSIX requires /bin/sh to be a POSIX shell, whose specs are derived mostly from the Korn Shell - so ksh is a valid POSIX shell, as is Bash, modulo a few features. Plain-vanilla Bourne shell is not.

AIX has the same dilemma: to be POSIX-compliant they have /bin/sh -> ksh and /bin/bsh is the real Bourne. HP-UX 11 has /bin/sh distinct from ksh, but it too allows things like 'export FOO=BAR'. I don't know if a real Bourne shell exists on HP-UX.

For a POSIX shell without the bash overhead, use ash, which is distributed with many (most?) Linux distributions. ash is the NetBSD /bin/sh, and at least on Debian the installation gives you the /bin/sh symlink option as well. Let me tell you, ash is much faster than bash in the autoconf-generated-configure "benchmark".

Since many Debian people use ash for /bin/sh, packages regularly have bugs filed -- and fixed -- vis-a-vis #!/bin/sh vs. #!/bin/bash. I don't know how careful other distros are about this sort of thing. Note that even many Debian scripts would fail if you found a real Bourne shell for /bin/sh rather than a POSIX-compliant shell.

Did you even read the complaint?

[Wakko Warner]

The guy writes installer scripts for a living. Every other unix has a Bourne shell implementation except Linux, and Linux's /bin/bash has incompatibilities. Telling the guy to fuck off and use something else has got to be the most ignorant thing I've ever heard. I'm sure he can tell his boss the same thing, right? "Fuck this, I'm gonna use ksh so it works on... AIX only!"

Ask Slashdot becomes less helpful by the nanosecond.

- A.P.

Re:Did you even read the complaint?

[aheitner]

Yes. I did.

And it was nonsensical.

If he's going to complain about bash being not exactly POSIX, he's got to show an example of an incompatibility (as a decent sometime bash and ksh hacker, I can't think of one -- and I can think of one subtlety that ksh and zsh do differently than POSIX: word splitting).

Ok. "export FOO=bar" doesn't work on a strictly-POSIX shell. But why is it bad for that to work in bash (and ksh and zsh)? How does that hurt you?

Instead, why doesn't the article's poster come up with an example of a valid POSIX statement that breaks in bash -- that way he actually has a complaint: he did something POSIX (i.e., the standard), and bash did the wrong thing. As it stands, he's just griping.

I mean, I agree with him -- bash is bloated and not a good choice for scripting. But that just means that scripts should be written in a shell optimized for scripting -- ksh. Or at least a minimalist straightforward POSIX shell (ash).

But if a system wants to provide just one POSIX shell (to save space, or whatever), and if bash is going to be used by 95% of users anyhow (so is likely to be loaded anyhow), making the one POSIX shell available on the system be bash isn't a bad call. Now that doesn't excuse scripts using non-POSIX features under the name of /bin/sh. But I'd hazard to guess that 99% of the time those scripts are distro-specific in many other ways, so it doesn't really matter.

I still have yet to see how (from anything but an efficiency perspective), this guy is finding that bash somehow makes his life harder.

(side-point: it is annoying that you can't solve the efficiency problem by relinking /bin/sh to ksh or ash, since all the system scripts will do slightly-non-POSIX things and therefore should be specifying /bin/bash).

...

I apologize for my confrontational tone. It really irks me when people propose writing things without bothering to check if the work has been done for them. I mean, come on, guys, it's Free Software. Avoiding duplication is the name of the game.

Ash

[runswithd6s]

Ever try ash? Here's a size comparison just to intrigue you a bit.

(82780) /bin/ash*
(407356) /bin/bash*

As to distro startup scripts...

[Anomie-ous Cow-ard]

Do you think that the startup scripts for most distributions would break because, even though they say #!/bin/sh at the top, they REALLY mean #!/bin/bash?

I couldn't say for other distros, but Debian policy says that /bin/sh can be any POSIX-compatible shell, with the one extension that 'echo -n' must not generate a newline. If any script uses /bin/sh assuming bash features, it's considered a serious-level bug.

And which "real Bourne shell" would that be?

[mj6798]

There have been lots of different versions of the "real Bourne shell" over the years. From a practical point of view, you should have no problem writing scripts that run in bash and in whatever shell you consider "real". Stick to the POSIX specs where possible.

However, why are you writing "installer scripts" anyway? If you want to deliver on Linux, please use the Linux packaging system. If you deliver on Solaris, please use the Solaris packaging system. Etc.

Which direction are you worried about?

[MarkusQ]

I think that there is some confusion here. The_code_poet clearly asked about compatibility going from sh to bash (i.e., he wants to write standard sh scripts and have them work on linux, and therefore bash). But the incompatibilities everyone is discussing (e.g. initialized export syntax) are all going the other way; things that you can write under bash that won't work under sh. So what?

The things that matter are first, the things sh has that bash does not (from the FAQ):

* uses variable SHACCT to do shell accounting
* includes `stop' builtin (bash can use alias stop='kill -s STOP')
* `newgrp' builtin
* turns on job control if called as `jsh'
* $TIMEOUT (like bash $TMOUT)
* `^' is a synonym for `|' * new SVR4.2 sh builtins: mldmode, priv

...and the implementation differences:

* redirection to/from compound commands causes sh to create a subshell
* bash does not allow unbalanced quotes; sh silently inserts them at EOF
* bash does not mess with signal 11
* sh sets (euid, egid) to (uid, gid) if -p not supplied and uid < 100
* bash splits only the results of expansions on IFS, using POSIX.2 field splitting rules; sh splits all words on IFS
* sh does not allow MAILCHECK to be unset (?)
* sh does not allow traps on SIGALRM or SIGCHLD
* bash allows multiple option arguments when invoked (e.g. -x -v); sh allows only a single option argument (`sh -x -v' attempts to open a file named `-v', and, on SunOS 4.1.4, dumps core. On Solaris 2.4 and earlier versions, sh goes into an infinite loop.)
* sh exits a script if any builtin fails; bash exits only if one of the POSIX.2 `special' builtins fails

None of these seem to me to be show-stoppers if you are writing the script from scratch (or even porting a reasonably written one)--I mean really, are you counting on it to dump core if you use multiple option arguments? Is there some reason you can't ballane your quotes? So my question to the_code_poet is, what exactly are you trying to do in sh that won't work in bash?

--MarkusQ

(Sorry, a little clarification)

[aheitner]

I apologize if my tone comes off confrontatively :-).

I'm reacting negatively to the idiotic suggestion that people need to waste time "writing a real Bourne shell for Linux".

If you don't like bash, there are several other shells which attempt to be POSIX. Some of them are quite strict (ash). Some are optimized for scripting (ksh).

If you want to argue that bash sucks and should be replaced as the default system scripting shell, I totally agree. If you want to say that it sucks when scripts request /bin/sh but use bash-specific features, I agree.

But for goodness sake, give some respect to the work that's been done.

...

I did see one good post noting the bash specifics that are missing, as a POSIX shell. They really are quite minor...

Re: (Sorry, a little clarification)

[psamuels]

In bash you cannot do this:
test -d "$dir" || ( echo "$dir doesn't exists"; exit 1)

That's buggy, isn't it? The () force the 'exit 1' to occur in a subshell, so the main script doesn't exit anyway. Use {...;} instead of (...) for what you want.

Thus, autoconf creates this kind of exit:

What you are seeing in autoconf macros is an attempt to support every single Unix vendor shell ever released, including old versions of bash that had weird bugs since bash had not yet matured ... but also weird bugs in old versions of other shells. Are you sure the above workaround is for bash, specifically?

[As an aside, note that the autoconf macro also directs output to two separate files, adding to its complexity. Apples to apples, etc.]

If you are trying to do that, certainly you will have to jump through hoops.

If you can assume bash 2.0 or above, you shouldn't really need any bash workarounds.

Depends on what your requirements are, I guess.

Re:No other OS?

[green pizza]

1. There is no such thing as Solaris 2.8, and 2.7 only exists in some places. I assume you meant 7 and 8.

Bah, "Solaris" is the marketing name. It's SunOS 5.8. Do a 'uname -a' if you don't belive me. Besides, it fits perfectly with SunOS history... versions prior to 5.0 were BSD based, version 5.0 and newer are heavily SysV based. The whole "Solaris" name just confused everyone.

Re:No other OS?

[Doktor Memory]

There is no such thing as Solaris 2.8, and 2.7 only exists in some places. I assume you meant 7 and 8

You're just going to have to imagine me dressed up as Moon Unit Zappa circa 1985 as I intone from on high here:

Whatever.

Sun's breaking of a perfectly functional numbering convention for the sake of their marketeers is, well, their own lookout. Every other reader knew exactly what I was talking about, and only you were dumb and/or desperate enough to think that you could score some sort of lame debating point by pointing it out.

bash was only introduce into Solaris as of Solaris 8

Hm, I'm pretty sure I recall it showing up in one of the MU packs for 7, but I could be wrong on that count -- I was mercifully spared the experience of 7 for the most part.

/bin is actually a symlink to /usr/bin, so technically it doesn't put it under /bin

From Moon Zappa straight to William Shatner: GET. A. LIFE.

Linux Standard Base

[kingdon]

Others have addressed the various issues about what is a "real" Bourne shell and bash extensions and all that. Anyway, the Linux Standard Base has a section on shells. In a nutshell, bash 2.x was the most POSIX-compliant of the shells that the LSB tested (and no, I don't know exactly what versions or which shells or the like), with pdksh getting an honorable mention. And there were two ways in which bash was not POSIX-compliant and concerning which the LSB therefore diverges from POSIX (whether $0 is the full pathname or just the basename, and what happens if you try to use "." to run a script without the read bit set). I don't know whether a future version of POSIX is planning to change the specification, or whether this is likely to remain a divergence for the foreseeable future or what. In any event, these two issues shouldn't be hard to deal with in writing scripts.

Example bash bug

[kevinank]

To be honest, I have yet to run into much trouble with a script starting with #!/bin/sh with /bin/bash, and I've been using the latter for years. If any of you have had problems related to this, please tell us what the problem was and how you solved it.

The following works in UNIX sh, but not in bash:

/sbin/ls -l | while read bits links user group size junk ; do
echo $user $group
done

To work around the bug you can either put the output of ls into a file and redirect input to read from the file, or you can use an immediate mode file (EOF) in the script, but pipes are broken with respect to read.