Slashdot Mirror


Book Review: Linux Shell Scripting Cookbook

vellorean writes "I have been reading Linux Shell Scripting Cookbook by Sarath Lakshman, published by Packt, for a while. While most people I know learn shell scripts themselves, I was looking to refresh my concepts a little as well as have a reference lying around on the table for fast access." Read below for the rest of vellorean's review. Linux Shell Scripting Cookbook author Sarath Lakshman pages 360 publisher Packt Publishing rating 9 reviewer Kumar Appaiah ISBN 1849513767 summary A book for beginners and intermediates, which introduces shell scripting and proceeds to provide several practical real-world recipes of useful shell scripts First of all, let me remark by saying that shell scripting is something learned more on a need basis than as a tool to solve the main problem. People would seldom write shell scripts as standalone programs (exceptions exist). However, what makes shell scripting invaluable to know is the fact that knowing some tricks can save several minutes, or hours, of work by automating and simplifying certain tasks, generally (but not restricted to) file management and data processing. Linux Shell Scripting Cookbook does go quite far in pursuing this goal, and is appropriate for both beginners who are looking to gain dexterity in shell scripting, as well as intermediate users who wish to polish their skills. The book also can double up as a quick reference, though I would argue that the "Advanced Bash Scripting Guide" would suit that more.

At the outset, the author clarifies that the focus will be on Bash. This, people may or may not like, but the fact that bash has become ubiquitous in terms of the available shells on Unix-like systems today, starting out with bash is not a bad thing to do. Besides, learning other shell scripting languages while knowing bash isn't too hard, since the paradigm remains the same.

The book is organized into chapters based more on utility than scripting concepts themselves, although the language aspects are brought onto the reader gradually. For instance, the examples in the first chapter focus more on the basic data elements (variables, arrays, functions etc.) as well as operators (for numbers, files etc.), and all the examples demonstrate simple usage of these concepts, and he further chapters build upon these in a gradual manner.

At the same time, if he reader has some familiarity with shell scripting and needs to only refresh or learn a certain concept, he/she needs to just read the relevant chapter. It is not too difficult to grasp the examples of the later chapters, provided some basic shell knowledge is assumed.

A positive trait in the presentation of this book is that it is all based on practical everyday examples which, with minor adaptation, can be used by many for their own daily tasks. For instance, there are several examples which describe searching for and processing files, which, I'd imagine, many users would want to do on a regular basis. Thus, providing realistic examples allows the book to double its utility. The language and approach used is simple and conversational, and the presentation is very clear, with each idea being described as a problem statement followed by a "How to do it" section with the actual code, and ending with a discussion of the nitty-gritties of the code. It is easy to go for a quick scan for those in a hurry, while those who with to read in more detail will not be disappointed either.

The book also covers a wide array of applications. For instance, there are examples on automating fetching web pages and processing them, demonstrations of parsing and simplifying and even some queries around databases wrapped around in shell. It also spans to utilities and tasks connected to statistics, backups, compression, version control and many more.

The book goes into a fair amount of detail in terms of describing the shell scripting concept under consideration. The examples used go into a fair amount of detail in order to describe to the user all the aspects involved in the method or command being used. The concepts described are fairly complete, and would be sufficient for the reader to use immediately or with just a little bit of fine tuning. In terms of breadth, the book covers most of the features of shell scripting while also describing the various facilities the shell provides access to in a Unix-like environment. Thus, the book does not disappoint in this front either.

In summary, probably the only thing I'd have liked to see more of is some emphasis on how to write more efficient shell scripts. Granted, most of the shell scripts described in the book are very simple and succinct, but a some words on how loops can be made better, or how to spot situations where pipes are not needed to solve a problem etc. might have been a nice addition. Some explanation of differences with dash, tcsh, zsh etc. might also have been nice, since a lot of users have different default shells. But all this isn't going to prevent me from giving this book a high rating, since it delivers quite well on the promises it makes at the beginning.

This is definitely a good book to have near your desk, and kudos to the author for having taken the effort to put it together. I would highly recommend it to the beginner and occasional shell user for a thorough read, and to an intermediate to have on his/her desk for borrowing the cool scripting ideas and applications the author has written in this book.

You can purchase Linux Shell Scripting Cookbook from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

173 comments

  1. 2 Packt reviews in 2 days? by Anonymous Coward · · Score: 1

    Is Packt desperate for books sales that they have 2 shill reviews in just the last couple of days? Or is it that Slashdot is low on funds and needs another shill check from Packt?

    1. Re:2 Packt reviews in 2 days? by Anonymous Coward · · Score: 1

      Or maybe they just happened to release a couple of books relevant to the interests of the slashdot demographic close to each other?

    2. Re:2 Packt reviews in 2 days? by Anonymous Coward · · Score: 0

      Yeah, except for the fact that this book was released on Jan 25th and the other December 11th of last year. So they are both at least 5 months old now. Looks like Packt is trying to bump up shitty book sales with another shill review.

  2. Why are there still shell scripts anyways? by i+kan+reed · · Score: 1, Informative

    Why does anyone still use shell scripts anymore? Every major(and most minor) distro ships with python, ruby, and perl either built in or a trivial statement away. These languages give you access to basic OS functions and much more sane syntaxes. Shell scripting is something I haven't felt the need to do in 5 years.

    1. Re:Why are there still shell scripts anyways? by mugnyte · · Score: 5, Insightful

      wow. how many ways can i answer that?

        - there is no one true way
        - people learn a skill and want to continue using it
        - concise syntax / small footprint / fast load times
        - high confidence shell scripts will run on unknown systems
        - can be layered on years of useful scripts
        - distrust of your options for doing work

    2. Re:Why are there still shell scripts anyways? by beatle42 · · Score: 5, Insightful

      I find it useful to do shell scripting when I've been doing something by hand, and want to automate it. It's pretty easy to just echo the command I've been doing into a file and touch it up from there. Then I can take advantage of the organic growth of the command as it usually happens, and can also use it to run that same command on lots of systems if needed.

    3. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 5, Insightful

      Sysadmins who can't work in plain shell scripts are dangerous. What will you do if you lose your /usr filesystem ? Be stuck with no automation tools at all as you fix by hand ? Shell scripts have little or not dependencies, and can function where other languages aren't available.
      More-over they are ubiquitious. Perl, python and ruby are common in Linux, but what will you do if you're on a different Unix altogether ? A sysadmin who can use shells efficiently can easily transfer his skills to solaris, hp-ux or several others (I did just that, and now I'm paid quite a lot more than a Linux sysadmin earns as a Unix sysadmin because the skillset is rarer). Of course there are other skills you'll need to learn (package management for example is vastly different between various unixes and most are frankly primitive if you are used to Linux) but the core Unix fundamentals remain the same, and if you don't know powerful and flexible shell scripting, in my book, you're no sysadmin at all.

    4. Re:Why are there still shell scripts anyways? by goombah99 · · Score: 1

      You phrase the question differently than I would. I would ask why is perl not the default shell language.

      I have a hard time understand the need for bash when perl is around. I can't quite make the same argument for Python and Ruby. THose are arguably better programming languages but not better administrative script languages. Perl on the otherhand is still very close to the common shell languages while being much more powerful. It's also faster than shell for almost every purpose where time matters. But this is achieved without making simple things difficult.

      Indeed that was Larry's mantra: simple things should be simple, and hard things possible. So perl is the glue language of choice to supplant shell.

      Perl's name was derived from the acronym Practical Extraction and Report Language. That's what Linux administrative scripting is all about. If you want to write a GUI word processor, use python.

      --
      Some drink at the fountain of knowledge. Others just gargle.
    5. Re:Why are there still shell scripts anyways? by SuseLover · · Score: 1

      Maybe because bash is installed by default on most Linux/UNIX systems and is portable across all yoru installed base. Or maybe your security requirements disallow python, or ruby or whatever to be installed. Or that running shell scripts using many built-ins run faster than the interpreted languages? Bash scripting features seem to stay more stable over time vs. other scripting languages that are constantly changing. The app teams within your company may want to constantly upgrade the installed version that might break you scripts/programs. And bash is dirt simple to learn and implement for non-programmers.

    6. Re:Why are there still shell scripts anyways? by jandrese · · Score: 1

      Because there are lots of things that are more awkward with Ruby/Python/Perl than they are with shell scripts?

      If you're running a bunch of commands on a system and just verifying return codes, a shell script is the right tool for the job. It's also the right tool for those times when you're working on a very stripped down machine that may or may not have your favorite interpreter installed.

      --

      I read the internet for the articles.
    7. Re:Why are there still shell scripts anyways? by Hatta · · Score: 1

      Because I've always got two or three shells open anyway. If I'm using Bash all day anyway, why not write my scripts in it?

      There are definitely better scripting languages, but I get a lot more Bash practice than I do Perl or Python practice. That's all there is to it.

      --
      Give me Classic Slashdot or give me death!
    8. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 1

      There are TONS of very valid uses for shell (bash) scripts on modern machines. Take a look at NVidia's proprietary driver for instance, the file you download from Nvidia for linux is "self extracting", this was accomplished using a BASH script...don't believe me, take a look at the file yourself using vim (or emacs if you must!), pretty nifty and an excellent example of a valid use of an advanced bash script!

    9. Re:Why are there still shell scripts anyways? by kale77in · · Score: 1

      Since I use the shell as my preferred work environment, rolling up commands into scripts simply multiplies my existing effectiveness with very little extra effort.

      Correspondingly, what I learn from scripting improves my ability to act quickly in a 'live' shell.

      Since I work in PHP (mostly), I use it for anything that BASH can't do, since then I have access to my application libraries. If my app was in Ruby I'd use that.

    10. Re:Why are there still shell scripts anyways? by smcdow · · Score: 1

      initscripts. If you're not writing them, you aren't a software developer.

      --
      In the course of every project, it will become necessary to shoot the scientists and begin production.
    11. Re:Why are there still shell scripts anyways? by Bork · · Score: 1

      Because from a shell script I can run anything else. From a shell script I can tie the other together into one script. Because it does what I want it to do.

    12. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      PERL is dead, get over it. It had uses over a decade ago, now almost nothing serious is developed using it, just admins' glue for when shell scripting is too much of a PITA.

    13. Re:Why are there still shell scripts anyways? by Desler · · Score: 1

      Which are all things you can also do in perl and python...

    14. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      Sure you can use a wrench to hammer in a nail but a hammer is much more appropriate. Modern scripting languages have concise syntax, small footprints and fast load times. There is a high confidence that such languages will be on unknown systems and they have been around for years sporting vast libraries cough... CPAN.. cough that rival and exceed most anybody's bag of shell tools.

      The real point is that modern scripting languages are languages and as such offer sophisticated and proper design for engineering eloquent and useful solutions to real world problems. There's a reason why languages are designed and there are reasons for each and every bit of functionality and concepts that they put in them - even if you fail to grasp them. As such there is good reason to move to them instead of arcane shell incantations that do not promote sound software engineering principals.

      That said I will whip out a quick bash script to do some research or investigative type stuff but if I ever find it being run frequently, growing bigger or being put into production I usually take the time to re-write it properly in Perl.

    15. Re:Why are there still shell scripts anyways? by Desler · · Score: 1

      but what will you do if you're on a different Unix altogether ?

      You do realize that Perl and Python exist on AIX, HP-UX, Solaris etc as well, right? If you are on a different Unix... you just use them as well.

    16. Re:Why are there still shell scripts anyways? by rallymatte · · Score: 2

      #!/bin/sh
      echo "Lot of the time it's just quick and easy"

    17. Re:Why are there still shell scripts anyways? by goombah99 · · Score: 2

      Maybe because bash is installed by default on most Linux/UNIX systems and is portable across all yoru installed base.

      perhaps they exist but in 20 years I've not see a Linux distro without Perl.

      Or maybe your security requirements disallow python, or ruby or whatever to be installed.

      if they disallow perl then shell needs to be disallowed too

      Or that running shell scripts using many built-ins run faster than the interpreted languages?

      this is patently false. Perl is vastly faster than shell even taking into account load times when you do anything significant. Bash scripts that do anthing important generally are calling things like awk and grep and sed to parse things and these invocations are dramatically slower in a loop than a single invocation of perl. Additionally, perl I/O is in many case faster than standard unix operations that move large blocks of data.

      finally invoking and disposing a command like awk many times in a row makes for very poor memory management and system resource utilization compared to keeping one program like perl resident.

      So don't try to argue this on speed or system resource utilization.

      Bash scripting features seem to stay more stable over time vs. other scripting languages that are constantly changing.

      huh? perl is very static and backward compatible.
      moreover, it's the helped commands that bash needs like grep and awk and so many other system resource introspection calls that one cannot count on being present or taking exactly the ame arguments.

      even a braindead command like "cp" is not only different on Linux and BSD but it's command arguments have changed over time. Whereas is perl the commands needed to copy files have not changed in 20 years.

      The app teams within your company may want to constantly upgrade the installed version that might break you scripts/programs.

      the only time i've seen broken perl was the 5.8 to 5.10 switch changed the timing of a few things.

      And bash is dirt simple to learn and implement for non-programmers.

      Perl is very very very close to bash in syntax

      --
      Some drink at the fountain of knowledge. Others just gargle.
    18. Re:Why are there still shell scripts anyways? by cptdondo · · Score: 1

      Try working in an embedded environment. When you have 4 MB of flash and 8 MB of RAM, and python (even a barely usable stripped python) takes 3 MB, you use shell. Most of my programming is on embedded platforms; if I have 32 MB of flash I don't know what to do with it. I use a lot of shell and awk.

      Don't even get me started on PERL; it works for some and for others it is the most obtuse incomoprehensible glop you can imagine. Instant cruft. And it takes megabytes of storage.

      Just because you can't see it from your house doesn't mean it has ceased to exist.

    19. Re:Why are there still shell scripts anyways? by Hatta · · Score: 2

      You phrase the question differently than I would. I would ask why is perl not the default shell language.

      Because Perl is huge and sh is tiny. Would you be able to port Perl to BusyBox without dramatically increasing the size of the executable?

      --
      Give me Classic Slashdot or give me death!
    20. Re:Why are there still shell scripts anyways? by h4rr4r · · Score: 1

      So nothing serious, just the stuff no business can live without holding everything together. Ok got it.

    21. Re:Why are there still shell scripts anyways? by codepunk · · Score: 1

      In my shop it is either bash or python take your pick, perl is not to be used on company systems.

      --


      Got Code?
    22. Re:Why are there still shell scripts anyways? by sjaskow · · Score: 1

      Don't forget that there's still a lot of commercial unix in the work place. Most of those OS's don't come with python or ruby and the perl that does come with them tends to be old and not have very many useful modules already. Learn ksh/bash and you can write scripts on any modern Unix-like OS; only learn perl/ruby/python/current-flavor-of-the-day-language and you won't.

    23. Re:Why are there still shell scripts anyways? by SuseLover · · Score: 0

      I see. So when I am doing disaster recovery of a system where nothing but /usr/sbin and bash is available, I can use perl how? All of the DR scripts I have seen are written in C shell or Bash.

    24. Re:Why are there still shell scripts anyways? by h4rr4r · · Score: 1

      Even my phone has 256MB of ram, and its replacement will have at least 1GB. Remind me again why this is a problem?

      What is up with embedded devices having less than 64MBs of ram?

    25. Re:Why are there still shell scripts anyways? by h4rr4r · · Score: 2

      I take option 3, not working there.

      Python could be nice, if it ignored whitespace/indentation. Also if it had its own version of CPAN.

    26. Re:Why are there still shell scripts anyways? by maxwell+demon · · Score: 1

      You phrase the question differently than I would. I would ask why is perl not the default shell language.

      Because a language which compiles the full file before execution simply isn't usable for an interactive shell. When doing interaction you can't wait whether a function called now but not yet defined will be defined later. You need to execute a command as soon as it is issued, or give an error if you can't. And it has to be possible to change the definition of a function later.

      Also, the primary use of the shell is to start other programs from it. Therefore something which needs an extra command to do that simply isn't acceptable as shell. A shell has to take any command it doesn't know as call of a program.

      --
      The Tao of math: The numbers you can count are not the real numbers.
    27. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 1

      Yeah, how can you possibly work when I invent a contrived example that is set up so I can win!

    28. Re:Why are there still shell scripts anyways? by h4rr4r · · Score: 1

      Or you could use perl and `` to run whatever you want.
      I use bash all the time, but sometimes perl is the way to go.

    29. Re:Why are there still shell scripts anyways? by maxwell+demon · · Score: 1

      initscripts. If you're not writing them, you aren't a software developer.

      And I always thought it's the sysadmins who write the init scripts ...

      I guess anyone who writes software on Windows or OS X, or writes application software which is started by the user, not by the system at startup, as well as anyone doing embedded development on specialized devices which don't run a traditional operating system at all, isn't a software developer, then :-)

      --
      The Tao of math: The numbers you can count are not the real numbers.
    30. Re:Why are there still shell scripts anyways? by erice · · Score: 1

      You phrase the question differently than I would. I would ask why is perl not the default shell language.

      I have a hard time understand the need for bash when perl is around.

      [snip] Larry's mantra: simple things should be simple, and hard things possible.

      Perl isn't simple enough for basic scripting. Most shell scripts are little more that lists of commands with minimal control flow. For that, Perl is too verbose. The extra syntax means more places for bugs and it gets extra messy if you are generating scripts programatically. Now, I agree that more complex scripting should be done is something like Perl. "Advanced Bash" makes about as much sense to me as "Advanced duck tape application"

      It's too bad that Unix shell scripting languages are so dreadful. They didn't have to be. REXX was/is an awesome scripting language for VM/CMS. Negligible syntax overhead for simple operations and you could still do complex things with a clean syntax. Perl is better for bigger things but REXX effortlessly spanned the zone from simple list to substantial program. REXX is available for Unix but it doesn't really have the same effect because the close coupling between language syntax and the default shell isn't there.

    31. Re:Why are there still shell scripts anyways? by Desler · · Score: 1

      So the best you can do to counter him smashing all your arguments to bits is something as lame as that?

    32. Re:Why are there still shell scripts anyways? by turgid · · Score: 1

      I have a hard time understand the need for bash when perl is around.

      Conversely, I have a hard time understanding why anyone would use perl when bash, Ruby and Python are around.

      A shell should be (needs to be) light-weight. Perl is not. Perl's syntax is horrendously complicated. Bash (and sh) syntax is not.

    33. Re:Why are there still shell scripts anyways? by Zenin · · Score: 4, Insightful

      In particular Bourne shell makes gluing other programs together far easier, cleaner, and more reliably then can be done with any of the languages you list...and I say that even given that I'm extremely fluent in at least two of them.

      Correctly and reliably handling even a simple "foo | bar | baz" construct in the languages you list can be done, but you're talking about an LOT of non-trivial systems level programming to do it, effectively "coding C inside perl/python/ruby". No, really. Sure, it's very easy to do it wrong and fragile (just call system(), what's so hard right?), but if you actually want to handle it as correctly and reliably as Bourne does out of the box...it's going to take a hell of a lot more code and detailed systems programming knowledge then even most "Sr" Unix sysadmins tend to have. Almost without fail ever attempted I've ever seen in the industry gets it wrong...typically very, very wrong.

      Bourne is built to run other programs, to manage the interactions between them, programming logic being the exceptional use. All of the languages you list are built to be self-contained, programming logic being the primary task and communicating with external programs a very secondary use.

      Now of course...if your script/program is self-contained and not just wrapping a bunch of exec()s of other programs, then sure Bourne shell is one of the least favorable options.

      There's also nothing saying you can't freely mix and match. Most anyone worth their salt does so all the time. To do otherwise would be like saying a web coder could only work in HTML or only work in JavaScript or only work in PHP. We're talking about complimentary languages here, with some overlap.

      Use the right tool for the job.

      --
      My /. uid is better then your /. uid
    34. Re:Why are there still shell scripts anyways? by sdguero · · Score: 2

      For me its usually;

      - too lazy to learn another language or...
      - not enough time to learn another language

    35. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      True, but they come out of doing them at a command prompt, so why would I write it twice? I could do it all in assembly too, but that doesn't mean its a good choice.

    36. Re:Why are there still shell scripts anyways? by Waffle+Iron · · Score: 1

      You phrase the question differently than I would. I would ask why is perl not the default shell language.

      For one reason, you wouldn't want to have to type in something like

      system("ls -l");

      instead of

      ls -l

      for everything you do.

      However, the exact same syntax features that make sh good for interactive prompts make it a horrible programming language. IMO, it's tied with COBOL for the worst language still in general use. I'm familiar with a couple of dozen languages, and sh is the one I find hardest to use without constantly referring back to the documentation. I can never remember its crazy variable expansion rules and escapes, its weird logic and test operators, or its cryptic built-in variable names.

      Lately, if a script looks like it's going to be longer than a single line, I use Python and import the "subprocess" module. This greatly helps my sanity.

    37. Re:Why are there still shell scripts anyways? by Paracelcus · · Score: 1

      Rah! A really good comment, saves time, works with all shells.

      --
      I killed da wabbit -Elmer Fudd
    38. Re:Why are there still shell scripts anyways? by Ruke · · Score: 1

      #!/usr/local/bin/perl
      print 'I see your point. Perl is just too unwieldy in this case.';

    39. Re:Why are there still shell scripts anyways? by blair1q · · Score: 1

      Shell scripting lets you turn things you do on a shell command-line into scripts, without changing them.

      That you can then cruft them up with reams of flow-of-control syntax, variables out the wazoo, optargs for days, and trap statements, is just a bonus.

      Sure you would have done it better in Python. But if you don't know Python you take what you type into the shell more than once, and make a script out of it.

    40. Re:Why are there still shell scripts anyways? by hawguy · · Score: 1

      I still use Bash for most automation tasks because there's no guarantee that the next person in my job is going to be familiar with Perl, Ruby, and Python, but every unix admin is going to know Bash.

      If all I need to do is start up a job, make sure it created a non-zero length output file, and log a message if it ends with a non-normal return code, bash is simple and easy and has no real drawbacks.

      For anything non-trivial (i.e. a monitoring plugin to do a transaction on our website) than I would use Perl (because it's what I'm most comfortable with, someone else might choose Python or Ruby), but 90% of the time when I need a script to do something, Bash is just fine and I know that the next guy in my job is going to be able to maintain the script.

    41. Re:Why are there still shell scripts anyways? by mangu · · Score: 1

      Why does anyone still use shell scripts anymore?

      Because they are the simplest and quickest way to do a lot of things one needs to do when using a computer. Try this in python, ruby, or perl:

      the disk is nearly full, find who is the hog

      in bash:

      du -x / | sort -n

      Now do it in another language

    42. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      I would buy the book "Advanced duck tape application".

    43. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      % ./slashdot.pl
      ./slashdot.pl: bad interpreter: /usr/local/bin/perl: no such file or directory

    44. Re:Why are there still shell scripts anyways? by tnk1 · · Score: 1

      If you are talking about actual commercial enterprise software that is sold for money, perl probably never had anything "serious" developed using it. It's not really a language for that.

      Perl is the scripting language you use when you want more than you get with shell, but don't really care if it is optimized to the point it can be used for an enterprise app. Just like bash, perl is on every UNIX-like host out there and if you write your scripts using only perl and perl modules, the code is entirely portable to anything you can get perl to run on, including Windows. With things like PAR, you don't even need to load the modules on each host, it will create a self-extracting archive with the necessary modules and shared libraries once you archive up your script.

      The other nice thing about perl is that it works quite a bit like shell already. You can run command line apps directly from perl just like you can from shell. All you have to do is use system() or the dreaded backticks. I avoid that, because it can affect portability, but if you need to upgrade an existing shell script quickly, it's easy to do. You can upgrade a shell script, whose data source is a text file, to a script that does the same thing to data stored in a database or an XML file, or on the web with trivial effort in perl and not even have to re-write much of the portion that you used in the original bash script.

      Perl is always there, has modules for everything, and has been used for so long that you can find documentation for best practices to do anything with it. Ruby and Python have their uses, but they aren't solving any problems for me that haven't already been solved by perl long ago. There's simply no way to go wrong with perl if you understand it's limitations, have already taken the time to learn it and have become proficient in it. It is also possible to write readable, well-structured code with perl, they just don't force that on you.

      I use:

      bash = startup scripts, quick scripts where portability is a non-issue.
      perl = creation of in-house tools, advanced scripts
      java = business enterprise apps for customers that can be developed fairly quickly
      c++ = applications that need less portability and a lot more optimization.
      ruby = we had great success using ruby for making great looking reports for customers

      Perl may not be exciting or the next thing, but its far from dead. It has a niche that it fills very well.

    45. Re:Why are there still shell scripts anyways? by cloudmaster · · Score: 1

      You listed the options backwards. Most UNIXes (you might be shocked to find that Linux isn't the only Unix variant people use) do not ship with Python, and I do everything I can to avoid having Ruby installed on even my Linux systems. And while I love perl, you are the first person I've heard describe its syntax as sane. :)

    46. Re:Why are there still shell scripts anyways? by cloudmaster · · Score: 1

      Because I don't want to pay for more RAM in my device just because you're too lazy to use the right tool for the job? :)

    47. Re:Why are there still shell scripts anyways? by hawguy · · Score: 1

      finally invoking and disposing a command like awk many times in a row makes for very poor memory management and system resource utilization compared to keeping one program like perl resident.

      Poor memory management? The awk binary and its libraries will stay resident in the pagecache between invocations, and awk+bash has a lower memory footprint than perl (on my system, running perl uses around 19MB of Virtual memory, awk+bash is around 14MB).

      So, it could be argued that running awk+bash is better for memory overhead.

      fork+exec overhead is so low that unless the script is calling awk thousands of times, it's not worth the time to even think about it. It takes around 2ms for me to call awk from a script.

    48. Re:Why are there still shell scripts anyways? by Bigbutt · · Score: 1

      Why does anyone still use shell scripts anymore?

      Because I'm a Unix Sysadmin?

      Shell scripting is something I haven't felt the need to do in 5 years.

      I'm guessing you're not a Unix Sysadmin any more?

      If we were interviewing someone who said they hadn't written a shell script in 5 years, I'd have to spend some extra time querying the reasons and then think twice about hiring him. See, we use shell scripts. For init scripts, for simple tasks, for data gathering; loop through 300 systems getting the uptime for instance. Someone we hire would need to be able to perform maintenance on the scripts and hopefully not break an existing script or start using something like Python that the other sysadmins may not know and have to either learn in order to maintain your scripts or convert to regular shell scripts again.

      See I've dealt with people who have their own little way of doing things. We had one sysadmin who felt Rexx was a great scripting language. Which was fine for him but when he left, we had to convert his scripts into a shell script in order to continue to maintain things.

      So we have a rule that system scripts are done using a specific template, which is editable of course in case of need, which manages several aspects of system scripts including checking for multiple copies of the script and killing sessions if scp to a system hangs for some reason.

      [John]

      --
      Shit better not happen!
    49. Re:Why are there still shell scripts anyways? by Bigbutt · · Score: 1

      I'm curious as to the reasons behind this. Is there some inherent problem with perl that it's banned or is it just a requirement to maintain consistency?

      [John]

      --
      Shit better not happen!
    50. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      If my embedded device is a sensor on a pipeline then I will want the smallest size and footprint possible. If I can get them at $5/unit with 8 MB of RAM and no local storage or $20/unit with 256 MB RAM and a little storage I will, without a doubt, use the 8 MB unit since I can stick 3x as many sensors on the pipeline for the same cost at the 20/unit (once you actualize labour costs).

      For a device that only collects temperature, pressure, solids PPM every 180 ms having fast running application regardless of the hardware is essential (I'd probably use assembler in this case, but YMMV). You wouldn't use Perl (or Python or Ruby) in development in these cases.

      You need to expand your mind as to what an embedded device actually is.

    51. Re:Why are there still shell scripts anyways? by h4rr4r · · Score: 1

      I mean consumer embedded devices. I would probably just stick with actual compiled code on devices like that.

    52. Re:Why are there still shell scripts anyways? by Bigbutt · · Score: 1

      /usr/local/bin/perl: bad interpreter: No such file or directory

      Shoot, perl is in /usr/bin and not in /usr/local/bin. So I'll have to ln -s /usr/bin/perl /usr/local/bin/perl and hope it works.

      [John]

      --
      Shit better not happen!
    53. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      Wait... you think perl has a sane syntax?

      Emperor Zombie: Pardon me if I say "Poppycock".

      Screw-on Head: Then say it!

      Emperor Zombie: POPPYCOCK!

    54. Re:Why are there still shell scripts anyways? by grumbel · · Score: 1

      Shell code is much easier to write then any other language, thats why we type it into our xterms after all instead of Python code. So naturally shell one-liners grow into shell scripts. There is also a mismatch between what Python/Ruby provides and what your shell provides, thus converting shell scripts into Python/Ruby isn't exactly straight forward and its easier to keep things as shell instead of doing a complete rewrite.

    55. Re:Why are there still shell scripts anyways? by hawguy · · Score: 1

      It's trivial to do it in perl!


      #!/usr/bin/perl

      system("du -x / | sort -n");

    56. Re:Why are there still shell scripts anyways? by ncc74656 · · Score: 2

      You phrase the question differently than I would. I would ask why is perl not the default shell language.

      Perhaps because perl's about an order of magnitude larger:

      # quickpkg bash perl
      # du /usr/portage/packages/app-shells/bash-4.1_p9.tbz2 /usr/portage/packages/dev-lang/perl-5.12.2-r6.tbz2
      1120 /usr/portage/packages/app-shells/bash-4.1_p9.tbz2
      12372 /usr/portage/packages/dev-lang/perl-5.12.2-r6.tbz2

      --
      20 January 2017: the End of an Error.
    57. Re:Why are there still shell scripts anyways? by hawguy · · Score: 1

      When I need to rescue a server, I boot with a recovery USB flashdrive, and then I have a full linux environment at my disposal Including Perl (and probably Python and Ruby, but I've never checked).

      Gone are the days when you only get what's in /sbin and if you can't fix it with cat and sed then you're out of luck.

    58. Re:Why are there still shell scripts anyways? by fnj · · Score: 3, Insightful

      OK, how much RAM does your router have? Your cable modem? A small NAS device? Your concept of embedded devices may be only a small fraction of what's out there.

    59. Re:Why are there still shell scripts anyways? by Hultis · · Score: 0

      I definitely agree that all sysadmins should know their way around shell scripting for situations like these, but I really don't see why they can't write the majority of their scripts in scripting languages which will be written much faster. Doing everything with shell scripts is kind of like developing all applications in plain C or even assembler, rather than whatever higher level language (where you can develop the same application some orders of magnitude faster) strikes your fancy.

      Also, as others have pointed out, many of these scripting languages are very widespread nowadays.

    60. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      I'm not a very good programmer, but my boss needed me to find a way to characterize our internet connection over a couple weeks, and send him the data in a spreadsheet. He wanted routes, ping times, download/upload speeds, etc... He wanted me to do this in addition to my normal workload as a data analyst. I'm not even a programmer! But I had played with shell scripting a bit, and was able to build a suitable script and spreadsheet in just a couple hours. Boss happy, and most important, I'm happy.

      That's why there are still shell scripts.

    61. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      Python could be nice, if it ignored whitespace/indentation.

      I keep hearing this as the #1 complaint about Python, but isn't Python open source? Why doesn't someone just modify it to ignore whitespace and use braces instead of indents? I don't get it.

    62. Re:Why are there still shell scripts anyways? by jgrahn · · Score: 1

      Why does anyone still use shell scripts anymore? Every major(and most minor) distro ships with python, ruby, and perl either built in or a trivial statement away. These languages give you access to basic OS functions and much more sane syntaxes. Shell scripting is something I haven't felt the need to do in 5 years.

      Then your tasks are different from mine. There are things that can be done much better in a shell script than in Perl or Python. Anything that executes a lot of other programs for example. Anything that pipes them together. I use Perl and/or Python a lot for many things, but shell scripts still have their place. (And if you choose to use bash rather than /bin/sh the language isn't *that* primitive.)

    63. Re:Why are there still shell scripts anyways? by rallymatte · · Score: 1

      Shut up, you're uid is higher than mine!

    64. Re:Why are there still shell scripts anyways? by grcumb · · Score: 1

      It's trivial to do it in perl!

      #!/usr/bin/perl

      system("du -x / | sort -n");

      As a career Perl hacker, I sympathise, but you're enthusiasm is misplaced in this situation.

      First, system() only gives you the return value of the command you executed, so you haven't got anything useful. Backticks would work, but they're horribly insecure if you're doing variable interpolation.

      Second, a more realistic example would look like this:

      for DIR in `find /home/ -maxdepth 1 -mtime -7 -type d`; do du -sh ${DIR} | grep '\d+G'; done;

      This, too, is easily done in Perl, but why bother when you're just dropping to bash and running various command-line utilities anyway?

      For anything that requires data collection and reorganisation, or which implies non-trivial logic (e.g. finding duplicate files across multiple file systems/machines), I agree that Perl is a tremendously powerful tool. Inline POD documentation and input validation are also extremely useful for the long-term health of the system. Most of the glue logic on my production systems is written in it. But as always, it's horses for courses, and for short, simple laps, bash rules.

      --
      Crumb's Corollary: Never bring a knife to a bun fight.
    65. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      Well then your shop is populated by numbskulls. Perl was good enough for JPL (where Larry worked when he developed it, in response to real needs as a sysadmin there), so if it's not good enough for you, well, you're a dope.

    66. Re:Why are there still shell scripts anyways? by hawguy · · Score: 1

      It's trivial to do it in perl!


      #!/usr/bin/perl

      system("du -x / | sort -n");

      As a career Perl hacker, I sympathise, but you're enthusiasm is misplaced in this situation.

      First, system() only gives you the return value of the command you executed, so you haven't got anything useful. Backticks would work, but they're horribly insecure if you're doing variable interpolation.

      That's like saying that the original poster's shell command didn't give him anything useful since it just wrote the output to stdout (i.e. his terminal).

      My perl script did the exact same thing since all system does is fork() and exec(), so the stdout/stdin are inherited by the child process(es). In this case, since there were shell metacharacters in the command line, my system() argument was actually passed onto the shell for execution, so in that respect you're right that I haven't gotten anything useful - I started a Perl interpreter to call the shell to run my shell command.

      The previous poster's suggestion of using exec() was better than using system() since it avoids the overhead of doing a fork first since I don't really care to return to the Perl session.

    67. Re:Why are there still shell scripts anyways? by Gaygirlie · · Score: 1

      Which are all things you can also do in perl and python...

      Not all systems have perl or python installed, but all of them have a shell. Lowest common denominator is a powerful thing sometimes.

    68. Re:Why are there still shell scripts anyways? by Gaygirlie · · Score: 2

      Even my phone has 256MB of ram, and its replacement will have at least 1GB. Remind me again why this is a problem?

      What is up with embedded devices having less than 64MBs of ram?

      What is up with simply using the correct tools instead of just throwing more memory at the problem? I mean, embedded devices are often bought in large quantities and thus the less they cost the better. So, if you can drop the cost with more intelligent software choices then that should be more than enough of an answer to you.

      Besides, I feel the whole "lets just throw in more memory so we don't have to think about our choices!" a very poor approach to computing, embedded or otherwise.

    69. Re:Why are there still shell scripts anyways? by walterbyrd · · Score: 1

      Not all systems have perl or python installed, but all of them have a shell. Lowest common denominator is a powerful thing sometimes.

      Not all systems have any particular shell installed (UNIX/Linux systems have bourne shell, but nobody uses that). Lots of Solaris systems do not have bash (I assume the book is about bash).

      I certainly cannot count on any UNIX/Linux shell to be running on Windows, but perl runs fine on Windows.

    70. Re:Why are there still shell scripts anyways? by walterbyrd · · Score: 1

      I keep hearing this as the #1 complaint about Python, but isn't Python open source? Why doesn't someone just modify it to ignore whitespace and use braces instead of indents? I don't get it.

      The core developers do not want that. They had an opportunity - a mile wide - with python 3.x; but instead decided to implement a lot of nearly useless features.

      There is also the issue of backward compatibility.

    71. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      Oh come on. this is lame

    72. Re:Why are there still shell scripts anyways? by goombah99 · · Score: 1

      Try running bash without grep and awk some time. bash is smaller because it calls other programs. that makes it fragile if those other programs change.

      --
      Some drink at the fountain of knowledge. Others just gargle.
    73. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      #!/bin/sh
      echo "No, really, it is easier." | openssl enc -base64 | mail ruke@slashdot.org

    74. Re:Why are there still shell scripts anyways? by Tetsujin · · Score: 1

      - high confidence shell scripts will run on unknown systems

      I guess it depends what you consider a "high confidence shell script". Avoiding GNU extensions to programs you run and sticking as strictly as possible to POSIX options and behavior would probably get you most of the way there, but that's a pretty big limitation.

      The way I see it, the strengths of a shell for scripting are that it's an environment truly made for running other programs and interfacing them together, and it's an automation (scripting) environment that matches your interactive session (and thus it's easy to experiment, and translate your experience in day-to-day usage of the system to scripting and vice versa). But on the flip side, the fact that the shell and programs running in it don't share any common conventions for exchanging data is a crippling limitation...

      --
      Bow-ties are cool.
    75. Re:Why are there still shell scripts anyways? by Tetsujin · · Score: 1

      Sure you can use a wrench to hammer in a nail but a hammer is much more appropriate. Modern scripting languages have concise syntax, small footprints and fast load times. There is a high confidence that such languages will be on unknown systems and they have been around for years sporting vast libraries cough... CPAN.. cough that rival and exceed most anybody's bag of shell tools.

      The real point is that modern scripting languages are languages and as such offer sophisticated and proper design for engineering eloquent and useful solutions to real world problems. There's a reason why languages are designed and there are reasons for each and every bit of functionality and concepts that they put in them - even if you fail to grasp them. As such there is good reason to move to them instead of arcane shell incantations that do not promote sound software engineering principals.

      That said I will whip out a quick bash script to do some research or investigative type stuff but if I ever find it being run frequently, growing bigger or being put into production I usually take the time to re-write it properly in Perl.

      It amazes me that you can write all that about proper design in the same message as you're advocating Perl... :)

      --
      Bow-ties are cool.
    76. Re:Why are there still shell scripts anyways? by Tetsujin · · Score: 1

      Python could be nice, if it ignored whitespace/indentation.

      I keep hearing this as the #1 complaint about Python, but isn't Python open source? Why doesn't someone just modify it to ignore whitespace and use braces instead of indents? I don't get it.

      If you create your own in-house branch of Python that behaves differently from the mainline version, then you've just given yourself more work to do - maintaining this piece of software. Presumably you'd also want to keep it up-to-date with all the changes in mainline Python, so you don't find yourself in an awkward situation years later where you're still using that in-house branch and maybe you've lost the source code with your modifications but you have tons of stuff that relies on it and not a whole lot of free time to remake your altered version or migrate your scripts or whatever....

      And personally I see nothing wrong with Python's approach to whitespace.

      --
      Bow-ties are cool.
    77. Re:Why are there still shell scripts anyways? by Tetsujin · · Score: 1

      It's trivial to do it in perl!


      #!/usr/bin/perl

      system("du -x / | sort -n");

      As a career Perl hacker, I sympathise, but you're enthusiasm is misplaced in this situation.

      First, system() only gives you the return value of the command you executed, so you haven't got anything useful. Backticks would work, but they're horribly insecure if you're doing variable interpolation.

      That's like saying that the original poster's shell command didn't give him anything useful since it just wrote the output to stdout (i.e. his terminal).

      But in shell that's exactly what you need for the next step of processing. In Perl you'd want to capture this in a variable, or split it on newlines to produce an array value, or something. That's... what, backticks? I am no Perl master.

      --
      Bow-ties are cool.
    78. Re:Why are there still shell scripts anyways? by aaron552 · · Score: 1

      And me having just spent my last mod point. Mod parent up people.

      --
      I had a sig once. It was lost in the great storm of '09.
    79. Re:Why are there still shell scripts anyways? by hawguy · · Score: 1

      But in shell that's exactly what you need for the next step of processing. In Perl you'd want to capture this in a variable, or split it on newlines to produce an array value, or something. That's... what, backticks? I am no Perl master.

      He was debugging a disk space problem and wanted to see what was using the disk space. No need to put it into variables, no need to build an array, he wanted to see where his disk space was going. Sometimes even in perl, you don't want to do any complicated data manipulations, you just want to see the output.

      Though admittedly, there'd be no reason to run his bash commands in Perl, but hey, he asked, so I provided.

      He said this:

      in bash:

      du -x / | sort -n

      Now do it in another language

      Nothing about storing it in variables, nothing about building an in-memory data structure to calculate average disk usage, all he wanted to do was see the top folders in disk usage.

    80. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      Actually someone did just that.

      It's called "Whython" - Whitespace Haters' Python. :)

    81. Re:Why are there still shell scripts anyways? by evilviper · · Score: 1

      Try again... I'm not taking a flight to the data center to plug in a usb flash drive.

      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
    82. Re:Why are there still shell scripts anyways? by evilviper · · Score: 1

      I agree with you, right up until you claim perl is remotely close to bourne syntax...

      Sure, some of the latest bash' isms look perl' ish, but that's about it. Perl gets -1 trillion points for the very existence of the "unless" function... You could read through an entire program, line by line, looking for any possible issues, only to find an unless statement that is always false... Meaning none of that is ever executed anyhow.

      Perl is notorious for how horrendously unreadable and un maintainable it is. Bourne has nothing of the sort to contend with.

      Nothing's perfect, but if it's not massive, and not performance critical, I'm writing it in bourne.

      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
    83. Re:Why are there still shell scripts anyways? by psmears · · Score: 1

      Maybe because bash is installed by default on most Linux/UNIX systems and is portable across all yoru installed base.

      perhaps they exist but in 20 years I've not see a Linux distro without Perl.

      I've seen many. Usually they are embedded systems, where memory is at a premium.

      Or maybe your security requirements disallow python, or ruby or whatever to be installed.

      if they disallow perl then shell needs to be disallowed too

      Nonsense. From a security point of view there's a lot of sense in minimising the amount of code on a system. If you have bash and perl running, and someone discovers a vulnerability in perl, you're wide open. If you just have bash, you're not. Of course, the same applies the other way round too - but there is a strong security argument for having "less stuff" installed and running wherever possible.

      Or that running shell scripts using many built-ins run faster than the interpreted languages?

      this is patently false. Perl is vastly faster than shell even taking into account load times when you do anything significant. Bash scripts that do anthing important generally are calling things like awk and grep and sed to parse things and these invocations are dramatically slower in a loop than a single invocation of perl. Additionally, perl I/O is in many case faster than standard unix operations that move large blocks of data.

      finally invoking and disposing a command like awk many times in a row makes for very poor memory management and system resource utilization compared to keeping one program like perl resident.

      So don't try to argue this on speed or system resource utilization.

      You're right that it's not a hands-down win for bash, not by a long way. But there are tradeoffs - for instance, in embedded systems (once again), the footprint of bash alone is much smaller than that of bash+perl.

      Bash scripting features seem to stay more stable over time vs. other scripting languages that are constantly changing.

      huh? perl is very static and backward compatible. moreover, it's the helped commands that bash needs like grep and awk and so many other system resource introspection calls that one cannot count on being present or taking exactly the ame arguments.

      even a braindead command like "cp" is not only different on Linux and BSD but it's command arguments have changed over time. Whereas is perl the commands needed to copy files have not changed in 20 years.

      You do have a point here - but then, copying a file in perl is rather more work than it is in bash (especially once you take into account error checking, recursive copying etc...)

      The app teams within your company may want to constantly upgrade the installed version that might break you scripts/programs.

      the only time i've seen broken perl was the 5.8 to 5.10 switch changed the timing of a few things.

      Were you not around for the perl4->5 change? That broke scripts... though to be fair I don't really see this as an argument either way, because bash (and the utilities called from it) change from time to time too...

      And bash is dirt simple to learn and implement for non-programmers.

      Perl is very very very close to bash in syntax

      I actually disagree that bash is very simple to learn for non-programmers. It's very easy to write simple scripts that appear to work, but break if they're passed a parameter containing a space or an asterisk. In this respect perl is better - it is (by design) much less vulnerable to special characters in input. But I also disagree that the syntax is very close to bash - superficially there are a lot of dollar signs in both, and the quoting *looks* similar, but that's about as far as it goes...

    84. Re:Why are there still shell scripts anyways? by tusam · · Score: 1

      Really, echo? That means the command can't include any shell expandable parts, no pipes, subshells etc (5 insightful??)
      Why not just ^xe (or the relevant counterpart in non-bash shells) and save the commandline from the editor?

      I hope the book also includes some basic readline instructions and reasons for the need of proper quoting, it's been too many times that I've seen people with supposedly years of linux experience use cursor keys and backspace to work with long commands, "oops there's a typo in this 500 char URL wait a minute while I get to it.." and "what do you mean it's wrong, find * -name bla*bla seems to work most of the time", that really grinds my gears.

    85. Re:Why are there still shell scripts anyways? by tusam · · Score: 1

      Or actually, if you quote the command line (by single quotes naturally) before echo'ing, which I guess you do, then I suppose it's ok.. that just caught my eye as it's not something I usually do. :)

    86. Re:Why are there still shell scripts anyways? by spiralx · · Score: 1

      Use IPython, and not only do you get an interactive Python console with tab completion, code highlighting, simple introspection, logging, macros, profiling and more, but it also acts as a shell where you can do things like ls -l, and treat the output as a Python type that you can pipe to variables and functions if you so desire. Awesome software.

    87. Re:Why are there still shell scripts anyways? by mvdwege · · Score: 1

      Try doing anything useful in Perl without loading modules.

      And I'm saying this as a Perl fan. I use it for everything that can't be done in a 10-line shell script, but for some simple automation tasks it is just overkill.

      Mart

      --
      "I know I will be modded down for this": where's the option '-1, Asking for it'?
    88. Re:Why are there still shell scripts anyways? by hawguy · · Score: 1

      Try again... I'm not taking a flight to the data center to plug in a usb flash drive.

      Sounds like you need a different KVM, just mount your virtual USB drive and boot from it remotely.

    89. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      It's a shame you are clueless. One of the marvels of perl is that very useful things can be done without any modules. Contrast this with python where match or regex or system io all need modulres.

    90. Re:Why are there still shell scripts anyways? by Gorshkov · · Score: 1

      I believe it's been published by Mythbusters Press

    91. Re:Why are there still shell scripts anyways? by RaymondKurzweil · · Score: 1

      While I value "sane" syntax for certain types of large scale program, there is a reason why shell and make are extremely useful. The sane syntax that makes Python and C# so great, just gets in the fucking way for some shell fu.

      Even if you could use the parsing utilities of these languages, you'd end up with something quite close to shell anyway.

    92. Re:Why are there still shell scripts anyways? by Anonymous Coward · · Score: 0

      Ruby's nice for some situations too.

    93. Re:Why are there still shell scripts anyways? by cloudmaster · · Score: 1

      People always hold Bash up as the pinnacle of shell scripting capabilities. Those people should try ksh93 sometime. The ksh coprocesses are really nice, and with ksh93 you get things like variable namespaces, keystroke bindings, good performance (it's actually faster running shell scripts than most other Bourne shells), and the ability to load in C-compiled modules (and shell functions) at runtime. So you can do stuff like writing a simple LDAP module in C which exposes functionality as native shell functions; most of the slow in shell scripts comes from having to fork and exec external programs repeatedly; if you learn how to use the string manipulation stuff, control $IFS and use read, and learn arrays, you can do amazing stuff with pure shell. Bash has cool stuff, too, but it's all (aside from some near-worthless things like that really annoying >& redirect Bash has) in ksh93 as well. I've personally written some multi-thousand line programs with the Korn shell which can get performance on-par with well-written Perl programs.

      Anyway, AT&T put ksh93 under a decent license, so most distros actually have a good korn shell available now (pdksh is *not* a good Korn shell). It's worth trying out for shell scripts. Bash maybe has the upper hand in interactive use due to programmable completion, but then zsh comes along and beats Bash for that too. Besides, Bash's vi mode is awful. :)

      Regarding perl's complexity and tendency for errors, every perl script under any kind of development should have "use strict; use warnings;" at the top, and should have "perl -c" run repeatedly until it doesn't complain at all. ;)

    94. Re:Why are there still shell scripts anyways? by mvdwege · · Score: 1

      Actually, it is you who are clueless.

      While regexes are great, and Perl's treatment of them as first-class parts of the language is a great boon, there are plenty of tasks where using a regex is the wrong thing to do, such as parsing (X|HT)ML, parsing a spreadsheet, or reading CSV files. For that you need a true parser, which is a loadable module, often with a lot of dependencies.

      I also submit that if your task requires regex support or heavy system i/o, by definition Perl is not overkill anymore. However, for most of the things I use shell scripting for, I don't need Perl's advanced features. And if I only need Perl for its control flow structures, I might as well use shell script with less overhead.

      Mart

      --
      "I know I will be modded down for this": where's the option '-1, Asking for it'?
    95. Re:Why are there still shell scripts anyways? by ncc74656 · · Score: 1

      Try running bash without grep and awk some time. bash is smaller because it calls other programs.

      What are the odds those packages will be missing? They're small enough (236K for sed, 264K for grep, 924K for gawk, and 3504K for coreutils, which contains things like head, tail, sort, and uniq) that the only reason you'd leave them out is if you're really crunched for space and intend to use something like BusyBox instead...oh, wait, looks like even BusyBox includes cut-down versions of grep, sed, awk, etc. BusyBox doesn't include a cut-down version of Perl, though.

      --
      20 January 2017: the End of an Error.
    96. Re:Why are there still shell scripts anyways? by WorBlux · · Score: 1

      You probably wouldn't even have to branch python, just create a script to parse brackets into the proper whitespace and pipe the output into the Python interpreter. If you wanted this feature, the script could probably be done in less than 50 lines of python.

  3. More efficient scripts by Anonymous Coward · · Score: 0

    In summary, probably the only thing I'd have liked to see more of is some emphasis on how to write more efficient shell scripts.

    For that, you'd want a book on Perl, Python or Ruby.

    Really, as soon as you've got more than a screenful of shell code, you're better off rewriting it in a regular scripting language. You can do amazing things with Unix utilities, but a huge amount of it winds up requiring launching 6 processes to, e.g., read the greatest value of the third field of the output of 'ls'. That's nuts.

    1. Re:More efficient scripts by Anrego · · Score: 1

      Agreed!

      This book is probably interesting, but in my opinion if you are doing anything beyond a basic set of commands run in order (with _maybe_ some very primitive control statements) .. you are far better off using a more powerful scripting language... or even a real programming language.

      Personally I don't like shell scripts at all for anything more complex than a temporary convinience. From what I've seen they are usually poorly written (lacking proper error handling and logging), are usually riddled with dependancies, and fragile. Oh, they added a few spaces in the output of some command you call? And your script didn't notice and just kept right on doing? Well now you're screwed!

      Obviously you can have the same kind of problems with a real programming language.. but shell scripts just seem to encourage bad things to happen.

    2. Re:More efficient scripts by bigredradio · · Score: 1

      Is this what you were thinking of?

      ls -l | awk {'print $5'} | sort | tail -n1

    3. Re:More efficient scripts by prgrmr · · Score: 1

      nope, it only takes three:

      ls -l |sort -k3|tail -1

    4. Re:More efficient scripts by Tsiangkun · · Score: 1

      ls -l | perl -ane 'print "$F[2]\n" '
      Two processes.

  4. Boy Goatsex is out in force today... by Lead+Butthead · · Score: 2

    Every topic is littered with them...

    --
    ELOI, ELOI, LAMA SABACHTHANI!?
  5. Re:Remember the torture by Anonymous Coward · · Score: 0

    my eyes are burning... argh! Damn you!

  6. Useful by MonsterTrimble · · Score: 1

    This would actually be pretty useful for total noobs (to Bash) like me. I haven't played with programming since 1997 and Fortran. Before that it was C64 Basic. I find that lots of the resources online about Bash scripts are not very good at explaining what the heck is what, syntax and how it all goes together. I can muddle my way through it, but not at the level I like.

    --
    I call it 'The Aristocrats'
    1. Re:Useful by StuartHankins · · Score: 4, Informative

      http://www.tldp.org/LDP/abs/ is helpful -- I printed and older version of it 2-up and duplexed it, then comb-bound it. It's been very handy.

    2. Re:Useful by MonsterTrimble · · Score: 1

      You Sir, I owe a coke.

      --
      I call it 'The Aristocrats'
    3. Re:Useful by CrashNBrn · · Score: 1

      It shouldn't be necessary though. Unfortunately, documentation in Linux is extremely poor - if you don't know what you are looking for already. I don't think any of the line-command tools have examples in their appName* --help, nor their respective man pages.

      Comparatively, when I need to do a quick batch cmd.exe script, almost every single command has the syntax detailed and examples of usage for most of the syntax. If I hadn't taken linux scripting in school back in the day, I doubt I would of ever had the time to parse through "tech-specs" to figure it out on my own. Whereas batch cmd.exe and AutoHotkey are fairly easy to self-teach.

  7. Re:Remember the torture by Anonymous Coward · · Score: 0

    MY EYES... dude i am at work here "S

  8. Re:Who uses shell scripts today? by Anonymous Coward · · Score: 2, Interesting

    OT: What is it with url shortening services? At best, the damn service works and it's a blind link to what the author claimed (which is inferior to just LINKING). At worst it's goatse. Somewhere in between is someone who f-ed up using it (like parent). Can Slashdot please ban links to url shortening services?

  9. MOD DOWN by Anonymous Coward · · Score: 0

    mod down, link is gotse

  10. Perl server pages. by bobs666 · · Score: 1

    Yes, I write web applications with little to no programing. Yes I use scripts, and even some unix commands called from my HTML+scripts.

    <<date +%d/%m/%Y>>

    works on my pages. ya it prints the date... Powered by PERL.
    PS. I can read an write SQL on my web pages. no programing needed.

  11. BAN "backslashdotperiod (2042346)". by mmell · · Score: 1
    =====> You must be this intelligent to ride the internet.

    Apparently, backslashdotperiod is a model and wants us all to see his picture.

  12. reviewer isn't a sysadmin by prgrmr · · Score: 4, Insightful

    First of all, let me remark by saying that shell scripting is something learned more on a need basis than as a tool to solve the main problem. People would seldom write shell scripts as standalone programs

    Seriously? So you write your application's /etc/init.d scripts in something other than bash (or csh if on solaris, or ksh if on aix)? Granted not everyone would want, let alone try, to write 500+ line bash scripts like I occasionally do; but, there are a ridiculous number of 100+ line perl scripts that could have been done in bash or ksh in fewer lines, and with more clarity, and without the overheard of loading dozens of perl modules due to interdependencies among them.

    1. Re:reviewer isn't a sysadmin by Anonymous Coward · · Score: 1

      Seriously? So you write your application's /etc/init.d scripts in something other than bash (or csh if on solaris, or ksh if on aix)? Granted not everyone would want, let alone try, to write 500+ line bash scripts like I occasionally do; but, there are a ridiculous number of 100+ line perl scripts that could have been done in bash or ksh in fewer lines, and with more clarity, and without the overheard of loading dozens of perl modules due to interdependencies among them.

      You're right. Sorry to have said that; I had myself in mind when I wrote it.

      I stand corrected. :-)

      Kumar

    2. Re:reviewer isn't a sysadmin by Culture20 · · Score: 1

      csh if on solaris

      Speaking of csh on solaris, I had a devil of a time figuring out why my csh scripts never worked until I realized that I needed to always put a newline character on the last line (the system wouldn't parse the last line without it). I slowly learned to hate csh's oddities as I learned tcsh (and later, [ba]sh).

    3. Re:reviewer isn't a sysadmin by Bigbutt · · Score: 2

      csh on Solaris? All scripts are generally in /bin/sh on my Solaris systems.

      [John]

      --
      Shit better not happen!
  13. i can forkgive if you can forkget by digitalsushi · · Score: 2

    So the bash fork bomb got popular back while I was at UNH. If you're not familiar with it, it's similar to this: ;(){ :|:& };: (I changed one character so that you dont paste it on recommendation) The way it works is trivial, and it's mystique is only in that it uses punctuation marks in lieu of letters for its own name: make a new function called :, run it and force the output into a new fork of itself in the background; then a final call to the new function. h4rdc0r3 1337.

    So, we had this UNH policy that if you hung a shared unix dev box, you got evicted from CEPS (the college for the engineering/cs kids). They could give you a warning or go straight to evict. Their call.

    I wrote to their head unix admin once, and with some humility, cause I knew they knew this, pointed out that they could just change the system's ulimit values to disallow casual fork bombing. The default system ulimits on the distro back then (still?) allowed a single user to consume all memory. So we saw a couple kids run the fork bomb and they never actually got evicted, cause maybe 40 grand tuition outvotes a cranky unix admin's wanton lust for cruelty.

    Anyways, as a student, my issue was this - set the damned ulimit so that my editor session doesn't get wiped with my homework -- you can't argue for frequent backups when more than 0 seconds of hard work are arbitrarily on the line solved by a simple config change.

    But the UNH unix admin guy replied to me and told me that yeah, he knew about ulimit back when I was in diapers, but he wasn't going to change it up. And a few more times that semester, I lost a couple lines of uncommitted code due to some clown pressing enter on a dare.

    Fork ya later,

    --
    slashdot: where everyone yells sarcastic metaphors to themselves to understand the issue
  14. Obviously, these guys haven't used UNIX. by mmell · · Score: 4, Insightful
    There's a reason we use /bin/sh (Bourne) to write scripts, and it's the same reason we use vi to edit them - mainly it's available on ALL UNIX systems. Despite widespread adoption, bash is not universally available - sorta like EMACS.

    If I stick it in root's cron on, say, an AIX system and it pukes I'm not interested in rewriting somebody's bash script to handle running under sh, csh or ksh. I want it written by its original author in sh. And I don't want to hear about a shebang line (#! /bin/bash) - if I haven't installed bash, it'll still puke.

    1. Re:Obviously, these guys haven't used UNIX. by grub · · Score: 1

      A longstanding pet peeve of mine are systems that symlink /bin/bash to /bin/sh

      Pretty much any Linux system, for example.

      --
      Trolling is a art,
    2. Re:Obviously, these guys haven't used UNIX. by Anonymous Coward · · Score: 0

      Here is how new I am to LINUX / UNIX. What is the difference?
      Please explain.

    3. Re:Obviously, these guys haven't used UNIX. by Anonymous Coward · · Score: 0

      Pretty much any Linux system, for example.

      On Debian (and Ubuntu?) /bin/sh is a link to dash, not bash.

    4. Re:Obviously, these guys haven't used UNIX. by Anonymous Coward · · Score: 0

      Oh my god, bash is not available in a fantasy situation!
      What are we gonna do?

    5. Re:Obviously, these guys haven't used UNIX. by tnk1 · · Score: 1

      There's a reason we use /bin/sh (Bourne) to write scripts, and it's the same reason we use vi to edit them - mainly it's available on ALL UNIX systems. Despite widespread adoption, bash is not universally available - sorta like EMACS.

      If I stick it in root's cron on, say, an AIX system and it pukes I'm not interested in rewriting somebody's bash script to handle running under sh, csh or ksh. I want it written by its original author in sh. And I don't want to hear about a shebang line (#! /bin/bash) - if I haven't installed bash, it'll still puke.

      That makes good sense if you are writing scripts that you will have the need to run from maintenance mode, but that's a pretty small amount of administrative scripting when it comes down to that.

      However, if you are writing all of your shell scripts with the assumption that you will not be able to access something as widely available on all UNIX platforms as bash, even when those scripts are not needed in maintenance mode and/or make reference to things that wouldn't be available in maintenance mode anyway, that's just masochistic.

    6. Re:Obviously, these guys haven't used UNIX. by Culture20 · · Score: 1

      A longstanding pet peeve of mine are systems that symlink /bin/bash to /bin/sh

      Doesn't bash emulate sh if it's called as sh?

    7. Re:Obviously, these guys haven't used UNIX. by DiegoBravo · · Score: 1

      > mainly it's available on ALL UNIX systems

      Mainly it doesn't matter since this book is about LINUX shell scripting.

      Besides Bash, a lot of tricks and utilities are only available if you assume a Linux environment.

    8. Re:Obviously, these guys haven't used UNIX. by Culture20 · · Score: 1

      There's a reason we use /bin/sh (Bourne) to write scripts, and it's the same reason we use vi to edit them - mainly it's available on ALL UNIX systems. Despite widespread adoption, bash is not universally available - sorta like EMACS.

      But not all GNU/Linux systems. Gentoo is a prime example. Nano instead of vi. And as sibling stated: almost no GNU/Linux system has true sh, they symlink sh to bash.
      Oh, and don't stick things in a crontab on Mac OS X systems. Mac OS X prefers XML plist files for scheduling and startup.

    9. Re:Obviously, these guys haven't used UNIX. by Bigbutt · · Score: 2

      bash uses shared libraries and sh doesn't. If you lose the /lib directory (or /usr/lib, or whatever), you lose the ability to log in if a system crashes to single user mode and the library file system didn't mount. Then it's boot to CD to recover which is great fun on a remote system (but not always impossible with virtual mounts, etc).

      [John]

      --
      Shit better not happen!
    10. Re:Obviously, these guys haven't used UNIX. by Bigbutt · · Score: 1

      Perhaps but if you're in single user mode and can't log in because the shared library isn't available, what do you do now?

      At least that's where my pet peeve would come from. It doesn't happen often but when it does, it's a pain to get the system working.

      [John]

      --
      Shit better not happen!
    11. Re:Obviously, these guys haven't used UNIX. by VoidCrow · · Score: 1

      This link has a tabular description of the differences between sh & a small raft of other shells, including bash. Bash syntax is very similar similar to sh, but there are a few gotchas, which I'm too long out of the game to remember.

    12. Re:Obviously, these guys haven't used UNIX. by grub · · Score: 1

      Thanks, I was going to reply and found you pretty much summed up my reasons. That and the link another fellow pasted.

      Don't call it sh if it ain't! :)

      --
      Trolling is a art,
    13. Re:Obviously, these guys haven't used UNIX. by fnj · · Score: 1

      You, sir, understand the Unix philosophy.

    14. Re:Obviously, these guys haven't used UNIX. by loufoque · · Score: 1

      Not all scripts are meant to be re-usable.

      I often find myself writing a script directly in my terminal, without going through a text editor. A simple for i in `find the_files_i_want_to_do_something_with | preprocess_the_input`; do what_I_want_to_do_on_those_files $i; done.
      In this particular, I'm just using the shell I happen to be running (bash). I don't have to care about compatibility. Whatever works to get my job done is enough.

      Actually, whenever I want to write a re-usable script, I write it in Python.

    15. Re:Obviously, these guys haven't used UNIX. by Anonymous Coward · · Score: 0

      It's not even a valid assumption in the Linux worldsome major distros have been moving away from using bash as /bin/sh

    16. Re:Obviously, these guys haven't used UNIX. by Culture20 · · Score: 1

      I boot up with a rescue CD or thumb drive. If I'm in single user mode, I assume I have physical access.

    17. Re:Obviously, these guys haven't used UNIX. by Anonymous Coward · · Score: 0

      This sort of garbage is endemic to Linux programming mentality in general. A lot of people writing Linux software don't write portable code, and don't much care about writing portable code. More often than not, it's crap code too.

    18. Re:Obviously, these guys haven't used UNIX. by Bigbutt · · Score: 1

      You shouldn't assume :)

      I have servers all over the country with various methods of access, generally through a Lights Out Manager of some sort. Fortunately the newer ones have the ability to attach a virtual CD so I can get by if necessary. It leaves me stuck on the older ones if some nimrod decides to change root's shell. Fortunately I changed the servers when I started the job and sent out a nice note advising folks not to do that and why.

      [John]

      --
      Shit better not happen!
  15. In Soviet Russia by Revek · · Score: 0

    Script replaces you.

  16. Thanks by Anonymous Coward · · Score: 0

    I have a quad core phenom-II. I had no idea how fast it was until I clicked on that link. Well done.

    BTW it's a while since I've seen goatse.cx. Very funny.

    1. Re:Thanks by underqualified · · Score: 1

      you make it sound like you view it regularly

  17. Why do people thing there's only Linux distros? by PetiePooo · · Score: 4, Interesting
    One of my fondest hacks was a shell script. It was for an embedded Solaris box where we didn't have the ability (through written policy) to add any other programs or packages, but scripting was OK. Ultimately, it was a job for expect, as I was communicating via telnet to another platform to upload firmware. But I only had sh, csh, or ksh. I chose vanilla Bourne shell, and wrote a script that accepted as inputs the IP address to upload the file, and the name of the file to upload. It set some variables relaunched itself in the format

    exec $0 | telnet $DEST_IP | $0

    The script receiving stdout from telnet would watch for input lines, keeping track of what response was expected, and send either a USR1 or a USR2 back to the script outputting commands into telnet's stdin, which would then either bail or send the next command. It was rudimentary pass/fail error checking, but much better and faster than echo a; sleep 1; echo b; sleep 1; echo c...

    I often prefer #!/bin/sh as I know it's a common denominator on all *nix systems, and often my scripts work even if the shell is busybox. Yes, perl or python may handle the task more efficiently and with less code, but keeping fresh on Bourne shell makes it easier to fiddle with initramfs images and embedded systems with limited space and no advanced shells. In fact, I'll often stick with it just for the challenge. I like making the original Bourne shell truly sing...

    Open your eyes. There's much more to the world than just "every major (and most minor) distros." Even more than just Linux and the BSDs.

    Crikey. If nobody learned assembly, we wouldn't even have operating systems.

    1. Re:Why do people thing there's only Linux distros? by turgid · · Score: 1

      You rule.

  18. I dare you to diagram this sentence! by Anonymous Coward · · Score: 0

    This, people may or may not like, but the fact that bash has become ubiquitous in terms of the available shells on Unix-like systems today, starting out with bash is not a bad thing to do.

    Go on, I dare you to diagram that sentence!

  19. Re:Remember the torture by Zontar+The+Mindless · · Score: 1

    That's somewhat clever, but some of us do know what base-64 encoding is.

    --
    Il n'y a pas de Planet B.
  20. This book gets the details critically wrong. by lhunath · · Score: 4, Informative

    I've just had a look through the book via Amazon's Look Inside.

    I simply cannot fathom how so many authors manage to get a book published on a subject they don't even master properly. "Shell scripting" is by far the worst in this category. So much that I recommend anyone to run screaming in the other direction when you see mention of "shell scripting" in any title. If it mentions "Linux", it's usually also an excellent indicator of junk. Not to take away from the beauty of the Linux kernel (which obviously has nothing at all to do with shell scripting, but let's ignore that fact).

    I obviously haven't read the book, but when I skim through it and I see failure to quote parameters, failure to recognize the difference between executing a script with /bin/bash in the hashbang and running it by passing it as an argument to sh, and more, I know for a fact that I'm going to be sitting in #bash trying to re-educate poor misguided souls for as long as I can suffer it.

    To those interested in the subject: Do not read this book. Do not read this "Advanced Bash Scripting Guide" mentioned in this review. You will end up writing junk.
    Go see Greg Wooledge's wiki, it's got an extensive FAQ of actually useful issues (http://mywiki.wooledge.org/BashFAQ), it's got a great newbie guide (http://mywiki.wooledge.org/BashGuide), it's got a great cheat sheet (http://mywiki.wooledge.org/BashSheet), and much more. Best of all: Unlike that broken ABS, they're all community supported and unlike this book, the knowledge is free.

    --
    ``OK, so ten out of ten for style, but minus several million for good thinking, yeah?''
    1. Re:This book gets the details critically wrong. by turgid · · Score: 1

      What's wrong with the Advanced Bash Scripting Guide at tldp.org?

      It's not perfect, but it will get you along way.

    2. Re:This book gets the details critically wrong. by Greg+W. · · Score: 4, Funny

      Go see Greg Wooledge's wiki

      Just not all at once, please.

    3. Re:This book gets the details critically wrong. by lhunath · · Score: 1

      "It's not perfect" is an understatement. Firstly, it is a key learning resource for a lot of beginners and more knowledgeable users. That means it has a certain responsibility: To be correct.

      Bash is a very lax interpreter, most sh's more so. As a result, they allow you to do a lot of stuff you really, *really* shouldn't be doing, and your test cases will make you think that doing it that way actually "works". Combine that with teaching sloppy practices and you naturally get bad results: People don't know what quoting really is and how crucially necessary it is. People don't pay attention to separating data from code which means most scripts are very vulnerable to arbitrary code execution, but more importantly, bugs such as receiving unexpected data can have devastatingly destructive results, and you might not even notice. Most people don't even understand what the difference is between running "bash" and running "sh" (while I'm sure more understand the similar difference between C and C++).

      The ABS is very popular. A lot of people read it and learn from it. The problem is that the code in its examples have terrible practices, important key details are not explained, let alone in sufficient detail, and beginners learning from this will not notice until they shoot themselves in the foot (and will end up blaming bash for it). The resources I enumerated are actively monitored by very knowledgeable people. They are extensive, detailed, clear and useful. But above all, they are correct.

      --
      ``OK, so ten out of ten for style, but minus several million for good thinking, yeah?''
    4. Re:This book gets the details critically wrong. by turgid · · Score: 1

      I'm of the opinion (just an opinion, mind you) that shell scripting is a bit of a black art and very quirky and unpredictable. I do my shell scripting by "hack and test, repeat."

      I think it's very dangerous when people claim that shell scripts can be portable (some people abhor bash and say that you should stick with sh) especially since they rely on the commands and utilities installed on any particular system. Are they GNU, BSD, SysV, POSIX? What versions are they? What directories are they in etc.

      I think that shell scripting to any extent beyond 20 lines, say, is way too complicated for beginners. People should be introduced to programming through a sensible and simple high-level language before they are condemned to shell scripting.

      Shell scripting can be fun in a perverse kind of way if you enjoy all the "clever" and dirty hacks that can be done, but, as you say, there are may pitfalls for the beginner.

    5. Re:This book gets the details critically wrong. by greg1104 · · Score: 1

      The main issue I have is that the "Advanced Bash Scripting Guide" seems to be fascinated with listing every possible way to do the things it covers in bash. But missing from that tutorial is any notion of which of those approaches is the best one in any context. Certain shell constructs are best avoided, in favor of ones that are either more portable to other shells, more resilient in the face of bad data, or easier to maintain. But you get no sense of this from the guide; just a set of ways to hack the low-level code into place.

    6. Re:This book gets the details critically wrong. by Compaqt · · Score: 1

      This might be the appropriate place to mention the venerable Unix Power Tools.

      It goes through most of the common Unix commands (including Bash, but also sort, uniq, awk, sed, etc.). It teaches you about redirection and piping, customizing your shell prompt, xterms, X, remote X, Unix file times and permissions, find/exec, diff, head/tail, grep, vi, emacs, jobs, cron, time, regexes, sed, bash scripting, tar, rcs/cvs, perl/python, /dev, symlinks, tty's, ssh, users/groups.

      In short, everything you ought to know before opening a root shell.

      It's written in a funny/witty (for geeks) manner.

      --
      I'm not a lawyer, but I play one on the Internet. Blog
    7. Re:This book gets the details critically wrong. by mvdwege · · Score: 1

      How about some examples? So far you have wasted two posts and lots of verbiage on mere assertions.

      I happen to think the Advanced Bash Scripting guide is just fine. Those who enter into it with no prior knowledge of shell and mess up their scripts because they don't understand what they are doing cannot blame this on the ABS. It is there in the bloody title after all: "Advanced Bash Scripting".

      Then again, cargo-cult programmers who rely on copy/pasting code without understanding, and then blame the resulting breakage on the language or on their reading material exist in any language.

      Mart

      --
      "I know I will be modded down for this": where's the option '-1, Asking for it'?
    8. Re:This book gets the details critically wrong. by evilviper · · Score: 1

      The thing that strikes me about that wiki is the absolute obsession. bash' isms are pushed, HARD, on the unsuspecting, even when there is damn near always a standard equivalent that works well. Also, the constant hammer of recommending the most complex of all possible options, the one that proves impossibly complex when stuck in a pipeline, all because it may behave nominally better in a given border case on a single release of a single less popular OS. It's extremely pedantic, and even then, in ways no two pendants would agree on...

      It's sad that "bash" shell scripting guides are so prevalent. Doubly so when it's even bash 4' isms! Even if you wish to have nice features at your finger tips, ksh93 has been gpl for some time, and oksh/mksh are quite superior to bash in several ways (line wrapping in bash has never worked right) and being far tinier and more portable is quite nice. Preferences not withstanding, you close off a big world to yourself, which you will regret later, when you depend on proprietary features in your coding/scripting in general.

      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
    9. Re:This book gets the details critically wrong. by lhunath · · Score: 1

      The Bash FAQ/Guide/Sheet are indeed, and perhaps unsurprisingly, bash-specific. They do not deal as much with "Shell scripting" as they deal with "Bash scripting". I find that a good thing. You hear people talk about "shell scripting" all the time, as though all shells behave in the same way when scripted. There is a unifying feature-set among POSIX shells, but this feature-set is so limited that it can suffice for nobody in writing decent shell scripts (no, if you need to rely on word-splitting to do your lists, you can't write a decent script). Most people really don't even have a clue about any of this and put their bashisms in .sh scripts which they commonly invoke with the sh interpreter because "it works" on their system that has /bin/sh symlinked to /bin/bash. And they think they know "shell scripting". Let's not get into (t)csh, zsh et al.

      There are many shells that one can choose from and use, many with less and many with more features than bash (not to say worse and better). In the end, a great many systems come with a recent version of Bash which makes it ideally suited for general use (just because sh really isn't a reliable and decent alternative). If you use bash in your hashbangs, at least you know that you can use arrays, glob pattern matching in [[ and more. With sh you don't even know if you're talking to a POSIX shell or an old Bourne-like one. And while ksh (and dare I say, zsh) may be equally good if not better alternatives at times, the fact that they're not as widely available leaves them interesting but not my first choice, and not the choice I would recommend people invest time to learn.

      On the matter of "standard equivalent that works as well", when your hashbang is /bin/bash, you'd be silly to break your script's consistency and go for a "portable" solution (whose portability is broken by your hashbang) whose bash-equivalent has nothing but advantages.

      On the matter of "recommending the most complex of all possible options", sometimes doing things correctly and supporting all the edge cases is indeed more complex than supporting only filenames that contain no spaces or glob characters. But you know what, a huge amount of people that read this information every single day doesn't know or think about pathname expansion or wordsplitting during unquoted expansions and all those pitfalls. You really shouldn't recommend the "simpler" solution that falls victim to bugs caused by expected input data, just because it'll make your inside voice cry when your pipeline doesn't fit on your 80 column terminal (you probably shouldn't be pipelining it then anyway). Your code should do what it's supposed to do *regardless* of what the input data is or might become (and that's the prime principle that tends to make bash code only moderately more complex, but usually not at all).

      --
      ``OK, so ten out of ten for style, but minus several million for good thinking, yeah?''
    10. Re:This book gets the details critically wrong. by evilviper · · Score: 1

      On the matter of "standard equivalent that works as well", when your hashbang is /bin/bash, you'd be silly to break your script's consistency and go for a "portable" solution (whose portability is broken by your hashbang) whose bash-equivalent has nothing but advantages.

      You'd be a bad script writer for using #!/bin/bash. Most OSes (outside of Linux) won't have bash under /bin/. All the *BSDs will be /usr/local/bin/bash. Sadly, I haven't seen a consistent location for env (/bin or /usr/bin ?) either, so you're at best 50/50 there.

      My point in short, changing the hash bang in a script is trivial. Replacing all bash'isms is NOT. "Porting" to a system with only ksh, perhaps where bash won't compile, can be trivial, or a nightmare, all for the sake of a few features that are just as easy to use in a more compatible way, rather than the bash way...

      sometimes doing things correctly and supporting all the edge cases is indeed more complex than supporting only filenames that contain no spaces or glob characters.

      No. File name handling is a major issue. The trivial issues I'm talking about are more along the lines of inserting a single line in several files... Trivial, right? Not if you follow the recommendation of piping printf to ed, nested inside a find, awkwardly escaped, incidentally with poor performance (one file at a time). This twisted usage suggested because (whoever) likes that it will transparently handle symlinks to files...

      Or perhaps see the recommendation to "never, ever use seq". Even though "sed -w" will automatically give you the right number of leading zeros you need, every time, and the bash'ism equivalent requires KNOWING how many leading zeros you will need, and/or writing a complex function to replace this single simple usage. Why? See above...

      Frankly, if you think you need bash'isms to write reliable shell scripts, you need to expand your horizons greatly, because it's really more the opposite. You're far more likely to run into bash'ism behavior changing, than to find a case where you NEED bash'isms to write a resilient script.

      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
    11. Re:This book gets the details critically wrong. by lhunath · · Score: 1

      You'd be a bad script writer for using #!/bin/bash. Most OSes (outside of Linux) won't have bash under /bin/. All the *BSDs will be /usr/local/bin/bash. Sadly, I haven't seen a consistent location for env (/bin or /usr/bin ?) either, so you're at best 50/50 there.

      I've found env is most commonly found in /usr/bin/env and as such #!/usr/bin/env bash is a fair attempt at a constant hashbang, but as you've indicated yourself; if you really need to write a script and deploy it to random target OS'es, you'll probably want to customize the hashbang for each to point at wherever you've got or decided to put bash in it. It's not such a major undertaking.

      My point in short, changing the hash bang in a script is trivial. Replacing all bash'isms is NOT. "Porting" to a system with only ksh, perhaps where bash won't compile, can be trivial, or a nightmare, all for the sake of a few features that are just as easy to use in a more compatible way, rather than the bash way...

      Writing bash code by using often broken pure-POSIX equivalent solutions just because it'll mean you'll have it easier in a few years from now if you were silly enough to consider changing the language of your script to POSIX without rewriting it is going to give you bad bash code now and bad POSIX code later. If your needs require you deploying to targets where bash is really unavailable (it rarely happens that you can't even install it), then either write specific code for those targets in ksh (or heck, perl?) or suffer restricting yourself to POSIX. In the latter case, use /bin/sh as your hashbang. Changing a hashbang from bash to something else is always silly: Either you made the wrong choice of language initially and you should rewrite your code for the new language or you made the wrong choice of hashbang initially and you should never have put a bash hashbang in your POSIX-code. Writing C++ code in a C way because maybe you'll decide to switch back to C at some point is equally dim.

      printf to ed, nested inside a find, awkwardly escaped, incidentally with poor performance (one file at a time)

      The FAQ does its best to offer all the alternatives. This is indeed a convoluted method, but there is no portable alternative. GNU sed comes with a -i and BSD sed comes with a -i, both work differently, and neither is "standard" (and indeed, neither handle symlinked files well), ed is. I'm not sure what you're implying here, but if you were trying to make a case for sed's -i, it would utterly defeat your case for portable shell code in bash scripts. Non-interactively editing files is a difficult thing to do right, even more so if you want it to be "standard" or portable. You can't get a simple one-liner for everything. In this particular case, it would be better to iterate find's results with a while read loop and do non-escaped ed'ing inside, IMHO.

      Or perhaps see the recommendation to "never, ever use seq". Even though "sed -w" will automatically give you the right number of leading zeros

      Your shell can do math just fine, no, you really shouldn't be forking for it. And you really shouldn't be putting leading zeros in your variables: your data should always be pure and normal. You can format it appropriately when you use it (eg. with printf).

      Frankly, if you think you need bash'isms to write reliable shell scripts, you need to expand your horizons greatly

      What would you recommend? Your own arguments defeat ksh for the same reason they defeat bash, which leaves only POSIX sh, and no, you cannot write resilient code in POSIX sh, unless it involves no more than simple delegation. To be resilient in POSIX sh you need to abuse positional parameters for arrays or do nasty and convoluted eval hacks to basically write your own shell features. Thanks, but no thanks.

      --
      ``OK, so ten out of ten for style, but minus several million for good thinking, yeah?''
    12. Re:This book gets the details critically wrong. by evilviper · · Score: 1

      Your shell can do math just fine, no, you really shouldn't be forking for it. And you really shouldn't be putting leading zeros in your variables: your data should always be pure and normal. You can format it appropriately when you use it (eg. with printf).

      Thank you for the universal truths. You certainly wouldn't need to back that up with reasoning or anything. You should be doing things the much more complex way, because it is POSSIBLE to do so. Wonderful reasoning.

      Your own arguments defeat ksh for the same reason they defeat bash

      Quite the opposite. Stick with ksh88 syntax, and avoid just a handful that bash doesn't understand/agree with, and you can pretty well guarantee your script will work damn near anywhere.

      Of course, you've again jumped to the answer without any reasoning, so it must be true...

      and no, you cannot write resilient code in POSIX

      Thanks for the straw man! Nice.

      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
  21. Re:Who uses shell scripts today? by h4rr4r · · Score: 1

    It really should just expand them when the comment is submitted.

  22. Re:Who uses shell scripts today? by Anonymous Coward · · Score: 0

    I move they ban any user that clicks them without first expanding them.

  23. yahoo by Hazel+Bergeron · · Score: 1

    The only interesting thing about this book was to find the author's web site and see how many hoops there are nowadays to get through the Yahoo interview process. I've a friend who managed to get a senior sysadmin role at Yahoo a few years ago who can barely program and has no computer science knowledge whatever - his only previous experience was some ISP admin work, although he was a tremendous suck-up to^W^W^Wgood friends with a few core FreeBSD developers. Anyway, I had the impression that the company was a has-been which runs on inertia.

  24. Alternative book by Anonymous Coward · · Score: 0

    This text shows how to write portable Bourne shell, a virtue IMO:

    http://www.amazon.com/Portable-Shell-Programming-Extensive-Collection/dp/0134514947

  25. Re:Who uses shell scripts today? by Anonymous Coward · · Score: 0

    I would be happy with this. BUT it is way more work than just forcing people to NOT link to URL shortening services. Expanding links can be made difficult by people "shortening" already shortened URLs. A quick wget does the job, of course, but it would be better to just offload task to the poster (who probably made the damn "shortened link" in the first place).

  26. Re:Who uses shell scripts today? by h4rr4r · · Score: 1

    The problem is then these folks will link to blog pages that they setup to host this material. Some have already started doing that.

  27. OK, 5 seconds and here it is in perl by fnj · · Score: 1

    You left yourself wide open. Here's my answer:

    #!/usr/bin/perl
    exec("du -x | sort -n");

    Would I use it in place of

    #!/bin/sh
    du -x | sort -n

    No. But it's not something I have to "try" very hard to do in the same number of lines and not many more characters.

  28. Thank you! by DrJimbo · · Score: 1

    I had been limping long with the Advanced Bash Scripting Guide and Google searches for a while. The Wooledge guide looks like a vast improvement. Thanks.

    --
    We don't see the world as it is, we see it as we are.
    -- Anais Nin
  29. spellening by Tetsujin · · Score: 1

    It's trivial to do it in perl!


    #!/usr/bin/perl

    system("du -x / | sort -n");

    As a career Perl hacker, I sympathise, but you're enthusiasm is misplaced in this situation.

    "you're" is a contraction. What do you suppose it's a contraction of?

    --
    Bow-ties are cool.
  30. Cosplay Wigs is an attractive option for people now adays. you can find your friends buy wholesale Cosplay Wigs ,your colleagues buy Costumes and wholesale Costumes .You can see the wholesale Costumes , wholesale Cosplay Costumes ,is the trend of fashion. Christmas Suits , wholesale Christmas Suits and all kinds of Maid's Suits are appear in our daily life.