Randal Schwartz's Perls of Wisdom

r3lody (Raymond Lodato) writes "Anyone who has been working on the *nix platform has had a brush with Perl, the scripting language whose acronym (depending on who you ask) could mean Practical Extraction and Report Language, or Pathologically Eclectic Rubbish Lister. In either case, there is a distinct difference between learning to use Perl, and learning to use it well. In my opinion, the best way to learn any language well is to see how others have used it to solve problems. One of the foremost experts in the use of Perl, Randal L. Schwartz, has been writing columns since March of 1995 on the use of Perl in the real world, and has provided us with 6 books and over 200 columns with many examples on how Perl is used." Read on for the rest of Lodato's review. Randal Schwartz's Perls of Wisdom author Randal L. Schwartz pages 350 publisher Apress rating 7/10 reviewer Raymond Lodato (rlodato AT yahoo DOT com) ISBN 1590593235 summary A dated compendium of the best of Randal's Perl columns

Perls of Wisdom is a collection of 65 selected columns from Linux Magazine, Unix Review, and the now-defunct Web Techniques magazines, written between May 1995 and July 2004. In each column, Randal discusses some problem that he had to solve, or that someone else needed help in solving. He carefully discusses the problem, and then shows the Perl code needed to resolve it. Many of the columns are complete applications that can be run (with minor modifications) by the reader. (The listings are also available from the apress.com web site.) Each column has been reproduced as it was written in the original magazine, with "Randal's Note" prepended. Therein lies this book's best feature and greatest flaw. Allow me to explain.

When I first picked up this book, I had only read a couple of Randal's columns (from Web Techniques), and saw that he wrote tutorials of proper Perl usage. He also relies on the wealth of modules submitted to CPAN to leverage a solution. After all, why reinvent the wheel? I expected to see more commentary on the reasons behind choosing one solution over another, or insights into the inner workings of Perl itself. I more or less got what I expected. For example, the first column reproduced in the book (It's All About Context) explains why, when someone used my ($f) = 'fortune'; instead of my $f = 'fortune'; he got in trouble with the law (see the book to understand the legal issue). The first form only retrieves the first line of the output of the fortune program, while the second form retrieves the entire output. Little items such as scalar versus list context can trip many Perl coders.

The first chapter (Advanced Perl Techniques) does give you many tips and insights like the example I just gave. All but two of the twenty columns are little tutorials on the ins and outs of handling the commonplace day-to-day issues that Perl can address with ease. Some delve into more obscure topics, such as the difference between shallow and deep copies of structures, and Perl's Taint mode. Two columns contain complete programs. One extracts the text from the man pages and determines their "fog" index (a measure of readability). The other creates a mirror of files needed by CPAN.pm to install new modules. For each program, Randal gives the entire listing as well as an almost line-by-line description of how it works. Each column is written in a conversational style that is easy to read, yet doesn't talk down to you.

The following chapter is comprised of seven tutorials on the various aspects of searching, sorting, and formatting text. In addition to describing the creation and compilation of regular expressions, Randal also discusses formatting and the nifty "Schwartzian Transform" (Perl's map-sort-map idiom for sorting on almost anything) which was named for him, but not by him. While some of the information is a little long-in-the-tooth (the column on Text-Processing was written shortly after Perl 5 was released), it's all interesting and educational nonetheless.

Chapter 3 starts refocusing the use of Perl to web sites. This chapter discusses HTML and XML processing in six little columns. He shows how to generate a web page index, producing a web page calendar from a file of events, and parsing XML to retrieve the data within. He also includes a lesson on how to use Perl to compare two arrays to create an HTML-formatted difference table.

The next chapter demonstrates that Randal has spent a lot of time working out ways to update and improve his web site. It covers the intricacies of CGI programming in Perl. All but one of the fifteen columns are complete programs (again, available from apress.com) with line-by-line commentary. The programs do implement mostly worthwhile functionality, but each column was pretty much "I had this problem, so I wrote this program. Lines 1-3 do this; lines 4-5 do that, etc." Granted, some of the programs are pretty nifty (check out how he automatically keeps track of the "What's New?" pages), but the reading of one program after another started to become stale.

The final chapter is titled "The Webmaster's Toolkit," consisting of fourteen programs and three tutorials. Randal covers diverse web server background topics such as creating a light-weight load balancer, random links, and forcing users to enter through the "front door." There are also instructive techniques for throttling your web server's usage of the processor (a necessity at the time for Randal, as his web site was co-resident on a server with others), and calculating download times.

In its entirety, Perls of Wisdom contains 65 columns, split roughly half-and-half between tutorials and fully commented programs. More than half of the columns show that Randal uses Perl for web processing more than for general scripting, data reduction and reporting. His tutorial articles are top-notch, but I have a quibble over his program articles, which are somewhat dated. There were a number of prefaced notes to the effect that today he'd do it differently with some new feature or CPAN module. I really wish he had actually updated the column to show the new coding techniques. The original code is interesting in the historical sense, but I wanted to see nuggets of Perl wisdom for me to use in my daily job. The writing style is fine; the bits of insight are useful, but many of the programs are too specific to problems you or I may never see, and were solved in code that's showing its age. I'm glad I got to read the book, but I think it only rates a 7 out of 10.

You can purchase Randal Schwartz's Perls of Wisdom from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

Reading Perl code?

[treerex]

Because of Perl's TIMTOWTDI mantra reading other people's code can be an exercise in frustration, IMHO, because to grok what they have written you may end up delving into some of the more baroque syntax in the language. I've found that many people like writing incredibly terse code which makes its readibility incredibly difficult. While Schwartz may write comprehensible Perl, I expect that the bulk of Perl code in the wild is anything but. This is my biggest peave with the language, and how it can pride itself on this "feature" is beyond me.

Re:Reading Perl code?

[eln]

A good programmer will write readable code in any language he's writing in. Yes, there is a considerable faction who delight in making Perl code as obfuscated as possible, but it certainly doesn't have to be that way. C allows you to write some pretty ugly code too, but that doesn't make it a good idea to do so.

The TIMTOWTDI ethos is ultimately what makes the language as useful as it is. The more ways there are to do things, the more things you can do with a little creativity. The Perl language is abstracted, as all higher level languages are, but it's forgiving enough to allow you to do things that may not be possible in other languages without a great deal of pain.

The myth that all Perl code is unreadable is preposterous. I've written several very complex pieces of code in Perl that are not any more difficult to comprehend than any piece of C code designed to do the same thing, and often much less so. The phenomenon of obfuscated code is a result of poor programmers (and a pervasive subculture), not of the language.

Re:Reading Perl code?

[Phillup]

A language can encourage readability or discourage it. What does Perl do?

Perl encourages writing code.

It leaves readability as a lesson to be learned by the programmer when he has to fix his own bugs.

If I added a "+++" operator to C that increments by two then I now have at least three (obvious) ways to increment by two. Does that mean that I can now be more creative in C?

That depends, if you don't know the other two "obvious" ways... but you can easily remember the "new" way... then, yes. You are now more creative. Because now you have a tool you know how to use that will let you increment by two when you need to. The other two ways weren't much use to you, because you could not remember how to use them. (I'll leave discussion as to whether you should be programming if this is the case to another thread... that isn't the point)

I think that is the part that people don't get about Perl. It isn't necessarily about the many, many ways to do a thing. It is about doing things in ways that are easy for the programmer to understand.

By having many ways to do a thing, chances are, one will fit your thought process better than the others... and you'll have a tool you are comfortable with. Working with tools you are comfortable with... will make you more creative. Because you will spend your energy focusing on the problem, not on how to use your tool.

Re:Reading Perl code?

[jdavidb]

If you think the Perl community describes the Obfuscation contests as an argument for the use of Perl, you are confused about who the community is.

To me, the Perl community is that group of people that does promote readable Perl. Randal Schwartz is a part of that. There is a community out there of very good programmers who love Perl, and plugging into that community has been one of the best moves of my career. The people who actually invented and maintain Perl (Larry Wall, et al) are the real community, and they are very good programmers. The fact that there are a lot of other people out there knocking off quick scripts, running obfuscated contests, and writing ignorant books without reference to the accumulated wisdom of the real Perl community is not in any way a poor reflection on the Perl community and the Perl language.

Yes, the good programmers of the Perl community do occasionally play a round of "Perl golf" or have an obfuscation contest. But noone worth his salt has ever put those forth as if it were a positive reason to use the language.

Good programmers who use Perl use Perl because it is possible to write very good, high quality, modular, readable code in Perl. The people who actually create Perl exemplify and encourage this.

Re:Reading Perl code?

[rs79]

Warning: I'm grouchy. A CPU fan failed and let the smoke out of a new chip. I also discovered by accident Windows has a (320x240) television mode; it took me hours to unfuck somebody's PC.

"And I would not include C in a list of languages for writing readable code in."

I can understand how you can feel that way after looking at some of the C code that's out there. But you have to go out of your way to obfuscate C. There's a reason there's no obfuscated Perl contest: "everybody's a winner!".

It's possible to write readable code in C. Not many people do it, I'll agree but this does not mean it cannot be done and you learn for your own beenfit, in time, to write very clear, lots of whitespace, one idea per line code. DMR's "one true brace style" is as much to blame for the obfuscation and unreadability IMO, as anything, and while he and I disagree on this, our understanding we've agreed to is as long as you can read and write both, that's ok. Keep in mind his justification is based on physical limitations of 24 line 80 column crts.

if (thing == that){
pritnf ("This sucks, Dennis.");
}

or

if (thing == that)
{
printf ("My kids can understand this.");
}

C is a simple langauge with not that many operators, but it requires discilpine to write code you can go back to in 6 months or a year and still tell at a glance what it's doing.

I also do not buy the "perl has more features" argument. If you need to do something often enough you write yourself a set of routines and use them as a black box library. My "web" one is 10 yrs old. Prior to that I carried around the same one for decades and with slight modes it worked from (real) Unix to CP/M to MS-DOS. Things got a bit fuzzy around the edges with Amiga and Windows GUI code, but they did not last long in my world. My one Windows gig was my last, it was too revoting to take seriously; I'm not into masochism.

I would encourage aybody whose written a quick Perl hack to write it again in C as much for the learning experince as for the benefit of crusty farts like me who are sick of 7 levels of dependancies. Grabbing "the needed library" seems to be a recursive loop without end and I've given up many times trying to run some perl thing and found in most cases it's easier just to write it in C, which, once you're used to it you can do very very quickly. I will grant that it take a fair amount of time to overcome the learning curve but IMO it's absolutely worth it.

To address the original point I have no problem with APL, Forth or Poscript. While they take a day to get used to, they're simple, fun and elegant. Parl just makes me hurt all over.

At the risk of being permanently banned fom slashhdot forever I believe 1) Linux is more popular than BSD not because it's better in any sense, but because it's easier to install, and 2) Perl, similarly so, provides an instant gratification at a penalty of readability and to some extent, performance. I prefer "short term pain for long term gain" and shortcuts are just that. My grandmother said it best: "quality is the only economy".

Re:Reading Perl code?

[Anonymous Coward]

By having many ways to do a thing, chances are, one will fit your thought process better than the others... and you'll have a tool you are comfortable with.

This speaks to the developer's experience, not the code maintainer's. The code maintainer is stuck maintaining code idioms not of his choosing: and the more idioms there are, the more idioms the maintainer needs to understand to maintain them all.

Working with tools you are comfortable with... will make you more creative. Because you will spend your energy focusing on the problem, not on how to use your tool.

Again, this balances developer comfort ("I get to choose the best fit of ten expressions! Woot! I'm so comfortable!") against maintainer comfort ( "I have to know ten different ways to maintain ten different developers code. Why can't they all just pick one! *grumble*").

Maintaining someone else's code is usualy harder than writing your own, for the comfort factor you cited. A development culture that accepts esoteric ways to write code can be a nightmare, because the coder's defense is always There's More Than One Way To Do It.

If there's One Way To Do It, and that's what the coder did, then I know exactly what it does. If there's One Way To Do It, and the coder didn't quite do that, then I've found a bug, or the coder wrote something subtle, and didn't comment it clearly. In either case, the fact that the code needs fixing stands out clearly.

If There's More Than One Way To Do It, and what the coder did isn't clear, then I have to think hard to decipher what's going on, and whether it is what it appears to be, or whether it's some subtle little Perl trick.

In short, TMTOWTDI helps developers, but at the expense of maintainers, and QA staff charged with proving code correctness.
--
AC

Re:Reading Perl code?

[Chrax]

I more or less learned perl and C cotemporaneously, so I see where you're coming from. But really, once you get over the idea of no real data types, and learn a couple of the tricks perl does in order to drop extra variables, it's not a big deal, and I think it's time well spent learning.

And while yes, Java is easy to understand, it's just another step in the C line that, in the end, doesn't allow for the flexibility of perl, or rather can't be both as flexible and compact as perl.

Re:Reading Perl code?

[SimHacker]

"Don't get me wrong. Common LISP is on my list of cool languages to learn more about right below Python and Ruby."

Please, work your way a little bit further down that "list of cool languages to learn more about", before telling people they should use Perl.

learning languages

[bungley]

In my opinion, the best way to learn any language well is to see how others have used it to solve problems.

Strange, everyone else I've spoken to EVER has said the best way to learn a language is through practice. Then again, I suck at programming.

Perl doesn't kill readability...

[jqh1]

People kill readability (with Perl).

Seriously, choose the right tool for the job. When we've got a sys-admin level scripting task, and someone can go in and knock it out in a half hour (or less) with a few lines of Perl, who can say that's bad? I'm currently wading through a bunch of heavily patternized java that pulls checkin logs from a scm system and updates an issue tracking system as part of the build process. It's taken me *days* to begin to "grok" what's going on in the many associated xml config files and bizarre string handling approaches that were used in this undocumented hack. I'll replace it with probably less than 150 lines of Perl, and someone else will happily (and much more easily) maintain it. So there!

At the same time, we've got > 25 developers distributed around the world working on a big commodities trading app -- java works pretty well for that.

Re:Perl doesn't kill readability...

[m50d]

Yes, but perl is a hair-trigger for you. Obfustucating C or Java takes a bit of effort. Obfustucating Python is pretty hard, your best bet is to lambda over everything and hope the reader isn't a primarily lisp programmer. Obfustucating perl can be done without thinking. Seriously, perl is a tool for quick scripts that will be rewritten rather than edited. It's designed for things which take up less than one page of code to write. It's fast to write short programs, but you should think again before writing a big one in perl, maybe try and chain some small ones together instead.

Re:Perl doesn't kill readability...

[moof1138]

I have had to deal with a lot of Perl, Java, and PHP. The nastiest ugliest code I ever inherited was PHP. I agree that Python tends to be more readable since it has a more limited... expressiveness... than Perl, but PHP with it's endless heap of little functions that do nearly the same thing and its oddly inconsistent namespace can mutate as readily as Perl, especially when wielded by an inexperienced programmer.

Re:Perl doesn't kill readability...

[Fahrenheit 450]

Regular expressions - This is a common feature in almost all languages these days, but the ease with which they are integrated into Perl syntax makes them more common in Perl programs.

And this is exacerbated when people pull out all sorts of jiu-jitsu to apply regexes to problems that are better and more cleanly handled with a general purpose lexer. But I can see where pulling out a lex-alike doesn't work too well when you're dashing off a script.

Perl is weakly typed, and that frustrates many who are used to strong typing.

Actually this isn't quite right. [confession_mode=on]A couple of weeks back I ranted a bit about this and the whole "a number is a string is a boolean..." nature of Perl and how this implied weak typing. Well after thinking about it for a while, I realized that Perl is actually strongly typed with respect to its (fairly unconventional) type system. The fact that the type system is so unusual is what troubles most people (myself included).[confession_mode=off]

Re:Perl doesn't kill readability...

[merlyn]

There are messy desk people, and clean desk people. If you force someone who normally works with a messy desk to have a clean desk, their productivity goes down. Likewise in reverse.

Similarly, there are Perl people, and there are Python people. One is not better than the other. It just matters on a manner of fitting.

Please stop with the "one is better than the other" stuff. Better for you perhaps.

Re:Perl and work

[gurps_npc]

You made an error that I think a lot of people make.

It is not how long it takes you to understand a perl script of X length that is relevant.

What SHOULD be the important data is: Whether it would take you MORE time to understand a C or Java program that does the same thing.

I have seen cases where people complain that it takes them a week to understand a Perl Script of 500 lines. So they write a new program in C, that does the same thing in 5,000 lines. Which then takes me 8 days to understand.

One of Perl's image problems is that because it does so much with so little, people underestimate what the tiny program does and therefore get frustrated when it takes them more time to understand than a C program of the same size, even though the C program does 1/10 what the Perl one does.

Re:Perl and work

[bani]

that's part of the problem with perl -- the code is often incredibly terse and dense.

you can do incredibly complex things in a few characters in perl, which can be far more difficult to grok than the same thing described in 20 lines of C code.

it's both a blessing and a curse. a blessing because you can beat out very complex things quickly, and a curse because it is a major pita to maintain.

perl also stubbornly avoids some useful language constructs in the name of "language purity". eg case statements. yes i know about Case.pm, but its not stock perl. and yes i know perl6 will have it. only took them ~20 years to get there :-P

Re:Perl and work

[m50d]

It still takes more effort for me compared to python. Not compared to C, but C takes lots of instructions to do simple things - use only when necessary. And I think the problem is that the syntax is so different. You throw a c/java/python/etc. program at a programmer from another language and they'll get the basic idea. They'll be able to see where the functions are, how the control flow goes, with a little bit of effort what the variables are. You throw a perl script at someone who hasn't seen perl, and a few are alright, but most look like so much line noise. You have to relearn a lot of your programming knowledge, and if perl isn't your only language then you have to switch modes when you see it. If I could program with it all the time, it would be good, but the fact is I have to use C and Java a lot of the time, so I need a language which isn't too different from them.

Its not the language...

[psicode]

its the developer that writes unmaintainable code.
TMTOWTDI and the Perl syntax have hardly anything to do with badly written code in Perl or any other language. Sure understanding an expressive language like Perl takes a bit more than learning a language like Java. But that does not mean that Java or [insert any language here] prevents you from writing ugly code. I am primarily working as a Java developer on mostly large projects and the amount of badly written code I have to deal with is simply frustrating. And most of it is written by "Senior" developers and design pattern aficionados!!!

Re:Perl and work

[John Bokma]

Well written, and agreed. instead of: s/foo/bar/i; people somehow prefer:

private RegExpThingy re = new RegExpThingy;
try {
re.compileASearchAndReplaceCaseInsensitiveThingy( "foo", "bar" );
re.doDaSmartThingyOnAStringy( sumvar);

} catch RegExpCompilingFUBARException {
/* ignore it */
}

I wonder why... the days one got 0.10 USD a line are over, or not?

It's easy!

[fprog26]

It's not difficult to read at all; however, you wrote it in backward perl syntax with implicit variables. It's easy. It prints a concatenated list of prime numbers between 2 and 100 with no delimiter.

So, let's rewrite this program without implicit $_ variable, without && comparision and with a temporary variable.

for my $number (2..100)
{
my $unary_string = (1 x $number);

if ( $unary_string !~ m/^(11+)\1+$/ )
{
print $number;
}
}

Which is quite readable now.

For all numbers between 2 and 100. Take that number convert it into UNARY format. Now, if it doesn't match recursively the first set of ones (at least 2) with the trailling set of ones having the same number of ones (at least once).

So basically the matching algorithm is like this:

2 11 insufficient
3 111 insufficient
4 11 11 matched 4
5 11 1 11 mismatch
6 111 111 matched 6
7 111 1 111 mismatch
8 1111 1111 matched 8
9 111 111 111 matched 9
10 11111 11111 matched 10
11 11111 1 11111 mismatch

It seems to test for prime numbers...

If it's not a non-prime number since no match is found it prints the number without any delimiter on stdout.

Conclusion:

If I was to write this script with this "algorithm", I would write it like this:

#!/usr/bin/perl -w

# Print prime numbers between 2 and 100 by converting it into unary format and by testing for matching multiple pairs of double ones or more.

for (2..100)
{
my $unary_string = (1 x $_);

print if ( $unary_string !~ m/^(11+)\1+$/ );
}

BTW, writing a program in C to test for prime number would be quite more complicated!
Especially if you use this algorithm without a regexp library!

Enjoy!

Fred.

- Long life to Perl!

Re:Perl and work

[kneeless]

perl also stubbornly avoids some useful language constructs in the name of "language purity". eg case statements. yes i know about Case.pm, but its not stock perl. and yes i know perl6 will have it. only took them ~20 years to get there :-P

From the Camel:

Unlike some other programming languages, Perl has no offical switch or case statement. That's because Perl doesn't need one, having many ways to the same thing. A bare block is particularly convenient for doing case structures. Here's one

SWITCH: {
(!$whatever) && do { &go("yes"); last SWITCH; }
($something) && do { &go("no"); last SWITCH; }
($nothing) && do { &go("???"); last SWITCH; }
$somethingelse = 1;
}

After this, there are four other examples showing this in more ways. Go get the Camel, it's invaluable.

I like Perl

[Cyno]

I wrote a Perl script yesterday with a bad algorithm using 2 lists. Ran all night and was still running when I got into the office this morning, and it didn't spit out any relevant data.

Rewrote the script using 2 hashes in about 15 minutes. Completed in less than a minute with all the info I was looking for.

So, um, I think an evaluation of most programming languages is more dependant on the programmers than the language itself. Sure some languages lack the proper data types or features, but any modern/complete language is capable of being readable and resource efficient if the programmer knows what they are doing. IMO, of course. I'm not a developer.