Pro Perl Debugging

Michael J. Ross writes "The typical computer program has more bugs than there are ants at a picnic -- except ants are usually easier to find. Programs written in Perl are no exception, because the compactness of the language does not make any existent bugs easier to spot; they can simply be packed into fewer lines of code. To help remedy this problem, Richard Foley and Andy Lester, two seasoned Perl programmers, offer a new book, Pro Perl Debugging: From Professional to Expert." Read the rest of Michael's review. Pro Perl Debugging: From Professional to Expert author Richard Foley with Andy Lester pages 269 publisher Apress rating 8 reviewer Michael J. Ross ISBN 1590594541 summary A comprehensive tutorial and reference for the Perl debugger

This title was published in hardcover in March 2005 by Apress, a relatively new member of the technical publishing world. The publisher has a Web page for the book that includes links to all of the source code in a Zip file, the table of contents in PDF format, and a form for submitting errata. The book comprises 269 pages, the majority of which are organized into 16 chapters: Introduction (not to be confused with the true Introduction immediately preceding it), Inspecting Variables and Getting Help, Controlling Program Execution, Debugging a Simple Command Line Program, Tracing Execution, Debugging Modules, Debugging Object-Oriented Perl, Using the Debugger As a Shell, Debugging a CGI Program, Perl Threads and Forked Processes, Debugging Regular Expressions, Debugger Customization, Optimization and Performance Hints and Tips, Command Line and GUI Debuggers, Comprehensive Command Reference, Book References and URLs.

For programmers who wish to learn how to fully utilize Perl's debugger, what options are open to them? A terse summary of the debugger's commands are always close by, within the debugger itself. Those Perl coders who have yet to try the built-in Perl debugger, really owe it to themselves to give it a whirl. In most cases, it is superior to embedding lots of "print" statements in your scripts, and then wading through the results. Simply include perl.exe's -d flag on the system command line, and you should be put right into the debugger, and see the debugger's "DB<1>" command prompt -- the "1" meaning that it is ready for your first command. To display the aforementioned command summary, simply enter "h", or "|h" to see the output one screen-ful at a time, which you will probably want to do unless your system window can show all of the dozens of lines at once. The command summary is best used as a quick reference, and naturally cannot be expected to serve as any sort of tutorial. Yet it has its use, and for that, it's fine.

Most Perl books devote at least some space to explaining the basics of firing up and using Perl's debugger. The (in)famous "camel book," Larry Wall's Programming Perl, has a chapter on the debugger. It covers breakpoints, running, stepping, tracing, displaying code, commands, debugger customization, debugger options, unattended execution, creating your own debugger, and performance profiling. Aside from that last topic, the chapter is mostly an expansion of the command summary mentioned earlier. It is sparse on examples, and does not cover any advanced topics, such as using the debugger in the context of forking, threads, and POE, as well as the debugger's special capabilities for regular expressions, CGI programs, and shelling out.

The advanced topics are where Pro Perl Debugging really shines in relation to the coverage that I have seen in any other book, partly because the authors have the space to thoroughly explore those topics in depth, and to provide much more meaty examples, with adequately illustrative sample code. Even for the more complex topics, the writing is clear, and the examples are worthwhile.

The authors clearly intend for the book to serve as both a comprehensive tutorial and a reference for the Perl debugger. In both respects, they succeed admirably. But the practical value of their accomplishment could be called into question by any programmer who has grown tired of the limitations of the Perl debugger, and has switched over to any Perl-capable standalone GUI debugger or integrated development environment (IDE). More specifically, watching a variable change value, while stepping through the lines of a Perl script using the debugger, requires that the programmer manually or programmatically echo that variable's value, by issuing a print command ("p") followed by the variable name, one way or another. This process quickly becomes tedious when multiple variables need to be watched, because each individual variable must be printed, one at a time. Admittedly, previously entered print statements can be recalled by using the up-arrow key, but only if the particular command has not been pushed out of the debugger's limited storage. This usually becomes even more frustrating when trying to print the values of indexed arrays, hashes, and nested arrays and other structures. There are workarounds, but none are pretty, and even the most promising techniques still seem to require excessive focusing on the debugger commands themselves, drawing attention away from the code being debugged.

As a result, some disheartened Perl coders eventually switch back to embedding "print" statements in their code. Fortunately, there is a better alternative, in the form of IDEs, which can automatically report the changing values of a large set of variables, none of which need to be typed in, owing to the drag-and-drop capabilities of most IDEs. There are many IDEs available, including freeware and open source offerings. Most if not all of them support advanced editing, syntax highlighting and verification, visual breakpoints, and other much-appreciated capabilities. Even if they were to lack all of these features, and only have the advantage of easily and dynamically displaying the current values of variables, then they would be much more pleasant to use than the built-in Perl debugger. This is especially true in the case of nested structures, which can be expanded with a mouse click within most IDEs. All of this being said, it should be noted that the authors include a chapter that briefly touches upon the most well-known Perl GUI debuggers -- but at only seven pages in length, the chosen applications get only a cursory treatment, highlighting their major features.

Nonetheless, given the intended purpose of Pro Perl Debugging, and its target audience, the book cannot be faulted for its contents nor its approach to presenting the material. Anyone looking for a detailed and competent explication of the native Perl debugger, would likely not be able to find a more thorough treatment anywhere else.

Michael J. Ross is a freelance writer, computer consultant, and the editor of PristinePlanet.com's free newsletter."

You can purchase Pro Perl Debugging from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

In defense of print statements

[winkydink]

Properly placed and well-thought out, I can leave:

print "a bunch o' stuff\n" if $Debugging;

in my code forever and never have to go through the trouble of firing up an IDE.

Re:In defense of print statements

[Billosaur]

print "a bunch o' stuff\n" if $Debugging;

Alternatively, I convert print statements into logging facilities:

open LOG, ">> $log_file" or die "Can't open $log_file: $!\n";
# Magical happenings
print LOG "And this happens here...\n" if $debug_flag;

I can then match things up to my pseudocode and determine if I'm pushing values around the way I should. Of course it's made a a lot easier with "use strict;" and the -w switch enabled.

Re:In defense of print statements

[hattmoward]

How many times is that conditional checked at runtime? They can add up. In perl, you could have it optimized away at compile time...

sub DEBUG() { return 1; }

...

DEBUG and print "value of blah:", $blah, $/;

but... TIMTOWTDI ;)

Re:In defense of print statements

[Mark_Uplanguage]

When debugging I emphasize the use of "warn" over "print". It's the same syntax, but the warn statements don't get spooled and therefore their timing is quicker.

This is vital when you code just plain blows up. Using "print" means that a statement which got executed before the disaster may not make it to console, thus leading you to believe that it never got executed. "warn" avoids this problem and thus leads you to the problem more accurately. It also makes it easy to globally comment out the warn statements before going releasing the code.

Re:In defense of print statements

[Baron von Leezard]

The issue of whether to use a debugger or not is a matter of speed and ease of debugging. There's few things you can't do with print statements that you can do with the debugger. [There are some: try using printf to get a backtrace of a segfault in C.] Lacing your program with print statements to check values is a time consuming, annoying, and certainly uglifies your code. Worse still, if you didn't anticipate wanting to see the value of some variable in advance, you will have to modify your program, (compile it), restart it, and get back to the point at which the bug is occuring--in some cases this can be very time consuming. With a debugger you don't have to anticipate anything. And you can modify values, which is often quite useful. [In Perl, as opposed to C, you can basically do anything you want in the debugger, including replace subroutines with new versions.] In some debuggers you can even try some stuff and then restart the current subroutine and step through it again from the beginning by popping the stack! [I'm not sure if you can do this in Perl, but gdb lets you do this for C programs].

Admittedly, there is a time and place for everything. I especially favor print statements when you need to consider non-localized program behavior: compare values at many different points of the program or across many iterations of some loop. In such cases, using the debugger is usually annoying and far too slow. A really useful technique for this kind of debugging is to format the printf output so that it can be piped into a database like postgres or mysql. Once you have the debugging output in a database table, you have a really powerful way to pore through the data trying to figure out what's going on.

[B/v/L]

Re:In defense of print statements

[DrVomact]

I programmed Perl for years and used nothing but print statements to diagnose my code. Then my boss took mercy on me and bought me a copy of the Active State Perl Developer Kit. This thing comes with a debugger that is actually easy to use--and it works! Wow. The program executes inside a window that allows me to step through the code, shows me variables I selected to be "watched", lets me set breakpoints by clicking on a line number...I haven't written a print statement since. Wait...maybe that's why I'm not getting any output...

Disclaimer: I don't work for Active State, I don't own any piece of them and I'm not getting paid for writing this.

Re:In defense of print statements

[davorg]

You could print STDERR though, since that's not buffered like STDOUT...

Exactly. And that's what warn does. It prints to STDERR.

Re:In defense of print statements

[hattmoward]

I didn't see that anyone gave you the full explanation here. What happens is during the compile pass (perl first does a parse/compile pass, then executes), it encounters the definition of sub DEBUG(). The closed parenthesis indicate that this sub will never accept any arguments -- not like a C prototype, more of a compiler hint. It effectively becomes a constant within this package that never needs to be written as DEBUG() because there is no ambiguity as to whether there are parameters. During the compile pass, it also sees that the return value of DEBUG will never change, and pretty much does an s/DEBUG/1/g or s/DEBUG/0/g to your code.

More explanation here: http://www.xav.com/perl/lib/Pod/perlsub.html#const ant%20functions

Combine that with perl's handling of logical operators, and it allows you to "short-circuit" the print statements when DEBUG evaluates to 0.

Re:I luv Perl, but...

[winkydink]

IMHO, that says more about the programmer than the language.

Obsolete before printing?

[Jerry Coffin]

Hmm...let's see: the basic idea here is that the book is devoted to the (text mode) debugger built into Perl, but even the review comes barely short of flat-out stating that nearly nobody is ever likely to be happy with this, and anybody doing any real work will want to use an IDE with a GUI-style debugger instead.

I suppose this might be handy a bit the same way knowing something like ed or vi is handy for anybody using UNIX (or similar) systems -- even the absolute strangest distribution (or installation) will almost inevitably include at least that much and it's pretty consistent. OTOH, debugging Perl on a machine that hasn't really been equipped for it doesn't sound to me like a lot better idea than using ed to write most of my code...

Re:Obsolete before printing?

[Baron von Leezard]

I've been a serious Perl programmer for about eight years, and I never use an IDE. I always do my debugging at the terminal. [I do use a syntax highlighting editor.] I understand that a lot of people like IDEs, but I find them annoying and hampering. I'd prefer to have a text mode debugger pipe a large data structure to less and let me use / and ? to find various things in the pager. I haven't found an IDE yet that lets you do that easily. This is a very useful technique--especially in the perl debugger, where you can run arbitrary perl code at any point in the program and pipe the output to a pager.

Moveover, you're assuming that you can run an IDE in your development environment. I often have to work on deployed server systems that have neither a windowing system not an IDE installed. However, I can always run the perl debugger and interact with it in an SSH session.

[B/v/L]

Perl makes bugs very hard to spot

[Anonymous Coward]

The only thing harder to spot in Perl is the good code. My rule of thumb is: If I can understand it easy in Perl, it's probably a bug.

Debugging?!

[ObsessiveMathsFreak]

Debugging perl? Doesn't that defeat the whole point?

If it's Tuesday . . .

[rodentia]

this symbol must be scalar.

Unless the moon is gibbous.

use strict and Data::Dumper!

[licamell]

#! /usr/local/bin/perl
#
# Two things that make debugging perl easy:
#

use strict;
use Data::Dumper;



The first one will solve most peoples perl problems, and the second one is helpfull sometimes when dealing with complex datastructures.

Re:use strict and Data::Dumper!

[bahwi]

But what about this?

use Coy;

Debug this...

[Cherita Chen]

#!/usr/bin/perl $_='A=15; B=30; select(stdin); $|=1; select(stdout);$|=1; system "stty -echo -icanon eol \001"; for C(split(/\s/,"010.010.010.010 77.77 022.020.020 330.030.030 440.044.000 055.550.000 666.060.". "000")){D=0;for E(split(/\./,C)){F=0;for G(split("",E)){C[P][F++ ][D]=G} D++}J[P]=F; I[P++] =D}%L=split(/ /,"m _".chr(72)." c 2". chr(74)." a _m");sub a{for K(split(/ /,shift)){(K,L)=split(/=/,K );K=L{K};K=~s/_/L/; printf "%c[K",27}}sub u{a("a=40");for D(0..B -1){for F(0..A-1){M=G[F][D];if(R[F][D]!=M) {R[F][D]=M;a("m"."=". (5+D).";".(F*2+5)); a("a=".(40+M).";" .(30+M));print " "x2}}}a( "m=0;0 a=37;40")}sub r{(N)=@_;while(N--) {Q=W;W=O=H;H=Q;for F( 0 ..Q-1){for D(0..O-1) {Q[F][D]=K[F][D]}}for F(0..O-1){for D(0..Q- 1){K[F][D]= Q[Q-D-1][F]}}}}sub l{for F(0..W-1){for D(0..H-1){(K[ F][D]&& ((G[X+F][Y+D])|| (X+F=A)|| (Y+D>=B)))&& return 0}}1}sub p{for F(0..W-1){for D(0..H-1){(K[F][D]>0)&&(G[X+F][Y+D] =K[F][D]) }}1}sub o{for F(0..W-1){for D(0..H-1){(K[F][D]>0)&&(G[ X+F][ Y+D]=0)}}}sub n{C=int(rand(P)) ;W=J[C];H=I[C];X=int(A/2)-1 ;Y=0;for F(0..W-1){for D(0..H-1){K[F][D]= C[C][F][D]}}r(int(rand (4)));l&&p}sub c{d:for(D=B;D>=0;D--){for F(0..A-1){G[F][D]||next d}for(D2=D;D2>=0; D2--){for F(0..A-1){G[F][D2]= (D2>1)?G[F][D2-1 ]:0; }}u;}}a ("m=0;0 a=0;37;40 c");print "\n\n".4x" "." "x(A-4). "perltris\n".(" "x4)."--"xA."\n".((" "x3)."|"." "x(A*2)."|\n")xB .(" "x4). "--"xA."\n";n;for(;;) {u;R=chr(1); (S,T)=select(R,U,V, 0.01);if(S) {Z=getc;}else {if($e++>20){Z=" ";$e=0;}else{next;} } if(Z eq "k"){o;r(1);l||r(3);p}; if(Z eq "j"){o;X--;l||X++;p}; if (Z eq "l"){o;X++;l||X--;p};if(Z eq " "){o;Y++;(E=l)||Y--;p;E|| c |c|c|c|c|n||goto g;};if(Z eq "q"){last;}}g: a("a=0 m=".(B+8).";0 " ); system "stty sane"; '; s/([A-Z])/\$$1/g; s/\%\$/\%/g; eval;

Re:Debug this...

[Phroggy]

You accidentally used = instead of == in a conditional.

Re:Debug this...

[Phroggy]

I dont know which is scarier?

The code or the fact that you read it and understood it at plain site???

The code is obfuscated. It basically takes a really big string, and runs it through a couple of regex substitutions to turn it back into perl code, then eval's it. I replaced "eval" with "print", which made it output the code instead of executing it, then I used "perl -MO=Deparse" to clean it up. This gave a compilation error at ($X+$F=$A). Changing that to ($X+$F==$A) makes it compile... but Deparse optimized several whole subs away entirely, and I really don't care enough to figure out exactly what it's supposed to do.

And no, I'm not going to run it.

Re:Who Uses Perl Anymore?

[outZider]

Nah, you're just going to be marked as someone who doesn't know what's going on. Perl is very relevant, used in a lot of high profile areas. I don't know what you think 'replaced' perl, but I know that a lot of languages have 'joined' it in many tasks. Web development, system administration, you name it.

Print statements

[Anonymous Coward]

As a result, some disheartened Perl coders eventually switch back to embedding "print" statements in their code. Fortunately, there is a better alternative

Actually, after ~25 years of writing code, I've found that 'print' statements are the best alternative. I've used every kind of debugger and IDE in the past, but in the last couple of years I've found that I rarely use them.

When you're in a debugger, you're down in the trees and it's hard to see the forest. With skillfully placed print statements that pick out the relevant data, stack traces for exceptions and a decent language, you can get a commanding overview of what's going on in your app. With a good editor, you can manipulate print statements just about as fast as breakpoints in an IDE, too. Just map a function key in your editor to "save file; make go".

older books

[Anonymous Coward]

Thanks, I'd like to remind everybody that the 3 older books are still very worthwhile:

Effective Perl, Hall and Schwartz
Debugging Perl, Martin Brown
Perl Debugged, Scott / Wright

debugging loops

[abiessu]

I've found that debugging loops can be tedious in most of the 'normal' ways of going about it. I finally got sick of lots of 'print' output and discovered that using 'warn' statements and trapping the '__WARN__' signal (using a BEGIN) provided an especially effective method of debugging loops: store the warning text as the key of a hash where the value is incremented each time. Inside an END block, write the contents of the hash to your favorite stream.

This method will debug loops of all kinds, and if each 'warn' is labeled well, it's very easy to see what happened.

(Note: the above may or may not depend on the use of '#!/usr/bin/perl -w' (warnings) as your interpreter.)

Watched variables

[lewiscr]

> More specifically, watching a variable change value, while stepping
> through the lines of a Perl script using the debugger, requires that
> the programmer manually or programmatically echo that variable's value,
> by issuing a print command ("p") followed by the variable name, one
> way or another.

$ perl -d -e 1
    DB h ...
    h [db_cmd] Get help on command w expr Add a watch expression
    h h Complete help page W expr|* Delete a/all watch exprs ...

Admittedly, the watch expressions don't detect changes in deep data structures. But it works just find on dependant expressions. I'll regularrly set watches on a loop index and a data structure being indexed by said loop var. The watch expressions will display both results.

In addition, the 'x' command works much better than 'p'. 'x' dumps the data structure, so it will display arbitrarily complex and deep data.

Re:Who Uses Perl Anymore?

[wonko_el_sano]

Tell me, what major sites have been build using Ruby?

I can list several very high volume sites/applications that use Perl.

Perl debugger is fine, Log::Log4perl way better

[talexb]

I love the debugger -- my background is BASIC, assembler, C and finally Perl, so it fits right in with all those "programmning down to the bare metal" tools.

However, a way cooler method is to use logging. The one I use these days is Log::Log4perl, a terrific logger that you can turn on and off at will. Thus, if you have a bit of disk space, you can turn the knob up to 11 (DEBUG) and let the application go until you get to the Weird Bug. Then it's just a matter of pawing through the log files to find all of your excellent messages. Print statements sprinkled through the code? Please.

And the bit about having to print the variable value every step in the debugger is a bit of a red herring .. you can set up commands that get executed before and after each step. Yes, it's a geeky thing to do .. that's why they call it Engineering.

PRO perl debugging?

[infinite9]

Personally, I'm anti perl debugging.

linking to C libraries

[bcrowell]

For me, the real problems come when I'm debugging a Perl program that's linked to someone else's C library. I have an app that had been stable for a long time, until I found that with recent releases of Perl/Tk it suddenly started crashing once a week or so. A stack trace shows that it's crashing inside a certain C function in Perl/Tk, but my lack of C skills have stopped me from getting to the point where I could file a useful bug report. This is what I really need a book on. AFAICT, I would have to compile Perl/Tk myself using certain options, and the options that work for most C software don't actually seem to be the right ones for Perl/Tk. Then I would have to dust off my decades-old C and C debugging skills and figure out what was wrong. Ugh!

This guy is making money off this post

[Billly Gates]

Barnes & Noble pays for references aka clicks just like amazon and the click links to his account.

Also work4hire refers to his website to make his google page rank go up.

Re:You must not maintain servers

[gregarican]

If you delve into Ruby for such tasks and then take a look back at Perl perhaps you'll understand where I am coming from. For admin scripting, text parsing, etc. it's just as powerful. But easier to read, more logical, and more concise. I know I misspoke when I said Perl is irrelevant. It still has a big place in a lot of projects out there. But with languages like Ruby out there hopefully its days are numbered...

Re:I luv Perl, but...

[el-spectre]

And perl's greediness in matching strings sucks (because someone was too lazy to put proper string functions into the language?)

Hey, just cuz you can't write a decent regex...

first half of the deparsed code

[Phroggy]

#!/usr/bin/perl

$A = 15;
$B = 30;
select stdin;
$| = 1;
select stdout;
$| = 1;
system "stty -echo -icanon eol \cA";
foreach $C (split(/\s/, '010.010.010.010 77.77 022.020.020 330.030.030 440.044.000 055.550.000 666.060.000', 0)) {
    $D = 0;
    foreach $E (split(/\./, $C, 0)) {
        $F = 0;
        foreach $G (split(//, $E, 0)) {
            $C[$P][$F++][$D] = $G;
        }
        ++$D;
    }
    $J[$P] = $F;
    $I[$P++] = $D;
}
(%L) = split(/ /, 'm _H c 2J a _m', 0);
sub a {
    foreach $K (split(/ /, shift @_, 0)) {
        ($K, $L) = split(/=/, $K, 3);
        $K = $L{$K};
        $K =~ s/_/$L/;
        printf "%c[$K", 27;
    }
}
sub u {
    a 'a=40';
    foreach $D (0 .. $B - 1) {
        foreach $F (0 .. $A - 1) {
            $M = $G[$F][$D];
            if ($R[$F][$D] != $M) {
                $R[$F][$D] = $M;
                a 'm=' . (5 + $D) . ';' . ($F * 2 + 5);
                a 'a=' . (40 + $M) . ';' . (30 + $M);
                print ' ' x 2;
            }
        }
    }
    a 'm=0;0 a=37;40';
}
sub r {
    ($N) = @_;
    while ($N--) {
        $Q = $W;
        $W = $O = $H;
        $H = $Q;
        foreach $F (0 .. $Q - 1) {
            foreach $D (0 .. $O - 1) {
                $Q[$F][$D] = $K[$F][$D];
            }
        }
        foreach $F (0 .. $O - 1) {
            foreach $D (0 .. $Q - 1) {
                $K[$F][$D] = $Q[$Q - $D - 1][$F];
            }
        }
    }
}

Re:I luv Perl, but...

[jonadab]

> It took me less time to learn python and write my first utility
> (a page generator for 195 content categories in2 languages from
> source data files) than it would have to write it in perl and
> debug it.

*shrug*. Maybe you just think Guido's Way. Some of us don't. I tried to learn Python, but every time I tried to do anything in it, I ran into frustrating scenarios wherein I'd need to write eighty lines of code just to do something really simple. Maybe it's because I didn't know Python well enough yet to write it compactly, but whatever the reason, I just couldn't make myself finish learning the language.

In any case, a page generator for 195 content categories in two langauges from source data files sounds more like a round of golf; if it were an actual programming task, 80% of the time would be spent deciding which three modules to use (for parsing the source data, for generating the content's form, and for the language issue), and writing the code to put it all together would be 5%. (Allow 2-3% for debugging, and the rest of the time would go into writing the documentation.) Sounds *much* easier than learning a new language. (The *fastest* I ever learned a programming language to my satisfaction (discounting data languages such as SQL) was two weeks, and I spent twice that long attempting to learn Python... there's absolutely no way writing the page generator you describe could take that long.)

> The whole "TMTOWTDI" is just wrong, from a maintainability point of view.

Why? Why does it make the code less maintainable to solve each problem in the way that is natural to solve that problem, rather than trying to coerce every problem into a particular paradigm that may or may not be a natural fit for the problem? The best feature of Perl is its multiparadigmatic nature, that allows the programmer to select the best type of solution for any given problem. (Well, okay, maybe the *best* feature is the CPAN, but the multiparadigmaticity is a close second.)

I find that well-written Perl code is much more maintainable than code in most other languages. (Poorly-written Perl code is, of course, completely unmaintainable, but it's possible to write unmaintainable code in any language.)

> And perl's greediness in matching strings

Umm, did you read the documentation? If you want non-greedy matching, that's easy enough to do. The really handy things, though, are lookahead and lookbehind assertions. I just wish we could get variable-width lookbehind; presumably Perl6 will make this possible.

> Perl - because when it all looks like line noise, nobody can
> criticize the "quality" of your code.

If it all looks like line noise, the code doesn't *have* any quality.