Extending and Embedding Perl

ggoebel writes "Extending and Embedding Perl is, as it boldly states on the cover, 'The definitive guide to XS, embedding, and the Perl internals.' This book is well organized and information dense. One could spend days sifting through the available perlapi, perlcall, perlembed, perlguts perlxs, perlxstut, and h2xs documentation. After which you'll probably understand very well references to nethack's 'You are in a maze of twisty little passages all alike.' Or you could get yourself a copy of this book and find your way out of the maze." Read on for the rest of ggoebel's review. Extending and Embedding Perl author Tim Jenness and Simon Cozens pages 384 publisher Manning (August 2002) rating 9 of 10 reviewer ggoebel ISBN 1930110820 summary The definitive guide to XS, embedding, and the Perl internals

Most of the available documentation on extending and embedding perl is written from the prospective of the core perl developers for core perl developers. This book is written for advanced Perl programmers who for whatever reason need or wish to peer into that netherworld between Perl, C, and the glue that interfaces Perl with other languages. It is a deliberate thorough guide led by authors that are both extremely knowledgeable and also capable of communicating that knowledge.

While it would greatly reduce the learning curve, no prior knowledge of C is required to read this book. This is a surprising claim and while it won't be easy, this reader is proof that someone with little true knowledge of C can in fact read and for the most part comprehend what the authors wish to convey.

There are clearly areas for improvement. Things like NULL being used throughout chapter 3, only to finally be defined later in a footnote in chapter 4. And other cases of terms being used before they are explained. Things that leave the reader juggling unnecessarily until the information is provided that lets understanding fall into place. But for the most part, if you are a competent juggler and are patient your questions will eventually be answered. You won't walk away a C programmer, but you will learn enough to solve the problems which led you to consider reading this book in the first place.

One thing I liked very much about the layout of the book is how it switches back between presenting sections on C programming and Perl. The authors revisit C each time it is necessary to understand the next Perl internals topic. Those that are learning C or need the review receive the relevant information just before it is required.

Over the course of the book, you'll learn about interfacing from Perl to C and C back to Perl. For those that must plug references to Tolkien in things Perl... you can go back and rephrase that into an appropriate reference to Bilbo's book "There and Back Again". You'll also learn the perl api, data structures for core variable types, and how to work with scalars, arrays, hashes, strings, regular expressions, file handles, typeglobs, typemaps, objects, callbacks and PDL with C and C++. And there is even mention of working with Fortran, Java, and more esoteric alternatives.

The book finishes with an in depth look at Perl internals: the parser, tokenizer, op code trees, execution, and compiler. And closes with a discussion of the Perl development process: How it may be monitored and participated in.

What's missing? Detailed coverage of the I/O subsystem and the regular expression engine. I.e., topics which might themselves make for a good book. There was also light coverage on things like scratchpads. There were times while reading when I didn't know whether the issue being discussed was fully covered or curtailed. But you will certainly find better coverage of the issues in this book than elsewhere. This is an impressive book. I hope it will greatly influence the way Perl6 internals are documented.

You can purchase Extending and Embedding Perl from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

You are likely to be eaten by a grue

[Dynedain]

After which you'll probably understand very well references to nethack's 'You are in a maze of twisty little passages all alike.'

Funny, I remember that exact phrase from Zork.

Re:You are likely to be eaten by a grue

[hoggoth]

It's not from Zork. It's from the Original Adventure (Colossal Cave). If it's in Zork, it's as a homage to the Original Adventure.

misread that title :)

[Photon01]

misread that title at first glance, was dreading to see what microsoft had done to perl :)

This is a dead end

[Ars-Fartsica]

One of the key reasons the Perl 6 project is taking such radical steps in remaking the language is precisely because extending and embedding Perl 5 is HELL.

Investing in this book and this knowledge at this point is practically a dead-end, as most of the annoying kludges will end with Perl 5.

Only invest in this book and this knowledge if there is a project you are working on that requires embedding or exteninding perl now. Otherwise wait for the sane cleaned-up world of Perl 6.

Re:This is a dead end

[Rick the Red]

Otherwise wait for the sane cleaned-up world of Perl 6.

Coming when, exactly? This book could be useful for years.

Re:This is a dead end

[Koschei]

Of course, Mono is just working to implement a preexisting VM. Parrot has grander aims.

Here, why not learn something.

NULL?

[Anonymous Coward]

>Things like NULL being used throughout chapter 3, only to finally be defined later in a footnote in chapter 4.

You said yourself that this book is intended for experienced programmers. If you don't know what a NULL is as well as the implications, and are a programmer, you are in deep doo doo. You definitely cannot be considered to be an advanced programmer. I am suprised this book even defines NULL.

However, I have seen a lot of programmers that have trouble and bugs related to NULL values ... of course I tend to immediately not listen to anything they say anymore, but some have produced code that does work. I hate to imagine what else they don't know... especially about other important stuff like security and database efficiency.

l8,
AC

Re:Hmm, let's see ...

[cmburns69]

You're dissing perl for being quick and dirty, but just because you can use a shortcut, doesn't mean you have to. I can code with perl and have it come out looking as clean as Pascal (or even *gulp* Java).

Things can be done in other "structured" languages that would be as unreadable as the most obfuscated perl code. Being able to read your code 9 months from now is more a function of programmer discipline than the language used.

This game was written in perl, when I was learning, 5 years ago, and because I used good design and comments, I can still read and update the code.

An online Starcraft RPG? Only at

Re:Hmm, let's see ...

[B3ryllium]

use Perl or die();

I use PHP for web interfaces, Perl for unix scripts, and C++ for ... well ... Windows crap. Gawd I hate MFC. But that's another story.

PHP's function handling has a certain ellegance that Perl seems to totally and completely lack.

function MyFunction($line1, $line2)
{
print $line1 . "\n" . $line2;
}

vs.

sub MyFunction {
my ($line1, $line2) = @_;
print $line1 . "\n" . $line2;
}

People argue with me endlessly that perl's method "isn't a hassle". I use Eclipse to edit PHP, and thanks to an add-on, whenever I'm working on a PHP page, I have instant access to every function and object in that file, complete with the function prototype - at a glance, I can see what any given function needs as input. And, given the fact that all of those variables in the prototype are named, one can indirectly provide information on what the whole function is intended to do, if the function name and variable names are written properly.

I suppose there's always the alternative.

sub MyPerlFunctionWhatTakesTwoScalarVariablesAndPrints ThemOnSeparateLines {}

But like I said, I prefer PHP's method.

Guide to doing it the hard way?

[Andy Dodd]

Or does the book cover easier ways to embed C into Perl, such as SWIG.

SWIG rocks. SWIG is your friend. I'll agree that extending Perl by embedding C is hell and the documentation sucks, but SWIG makes it all (relatively) easy. With SWIG all you have to do is be careful about data types. (Mainly, you can't directly pass a Perl array to C code, you have to convert it into a C array first. How to handle situations like this with SWIG is well documented.)

I spent five days trying to figure out how to embed some C functions into Perl. Then I discovered SWIG and was up and running in 3-4 hours.

Re:Guide to doing it the hard way?

[cquark]

SWIG is much easier to use than XS, but today I prefer Inline, for embedding C, C++, java, python, or whatever other language in which you might be interested in your perl code.

Re:Guide to doing it the hard way?

[doop]

Or does the book cover easier ways to embed C into Perl, such as SWIG

There's a whole chapter on "Alternatives to XS", covering SWIG and Inline::*.

Perl IDE

[cquark]

Your arguments remind me of what my professors told me about C when I went to university. It was too easy to write unreadable code so no one should ever use it.

Anyway, ActiveState produces Komodo, a perl IDE, and they also sell a perl environment for Visual Studio .NET.

I still write perl in vim, but I do use ddd for debugging my code.

twisty passages

[syle]

As an experienced C coder, I can happily testify about the need for this book. The man pages to perlembed, perlapi, perlguts are obviously written without regard to the reader actually understanding the content.

I consider myself a competant programmer, but years of experience have never taught me what it means to 'assign magic to a variable' or deterime the 'taintedness' of a string I found to be magical. Problems of global scope of the loaded programs alone merit a book, or at least a chapter.

The documention behind it is some of the most befuddled I've ever had this displeasure of witnessing, so nearly all my learning came from studying other programs who managed to do it, along with a brief and ill-advised stint into the garbage collection routines of the interpreter source.

Merely by existing and being written in English, this is already the best reference on the planet. Hell, it could be written in Klingon and still be more understandable than half the API documents.

Now extend your example....

[smcdow]

... to handle an arbitrary number of arguments

PHP:

function MyFunctionWith2Args($line1, $line2)
{
print $line1 . "\n" . $line2;
}

function MyFunctionWith3Args($line1, $line2, $line3)
{
print $line1 . "\n" . $line2 . "\n" . $line3;
}

function MyFunctionWith4Args($line1, $line2, $line3, $line4)
{
print $line1 . "\n" . $line2 . "\n" . $line3 . "\n" . $line4;
}

You get the picture...

Perl:

sub MyFunction {
print join( "\n", @_ );
}

this works for any number of arguments. Perl's argument passing semantics are inherently varardic. I prefer this rather than having to define n number of functions for n possibilities of argument passing.

Re:Tcl

[cymen]

"...without using any wierd half perl, half c languages."

This is somewhat ironic considering Tcl's own syntax. Maybe it's just me...

Re:Actually it is from Zork...

[Cyberfox]

Greetings,
Well... To get this right would take a while, and is massively off-topic, but IIRC, the original Colossal Caves (Adventure, by Crowther and Woods) was written in Fortran, and had a twisty maze of passages, which was also used in Dungeon/Zork, which was very heavily influenced by Adventure.

The commercial (Infocom) Zork series is a splitting up of the Dungeon/Zork program, which was not originally written in Fortran, it was in fact originally written in MDL by Marc Blank and Dave Lebling, and translated to Fortran by a 'somewhat paranoid DEC engineer who prefers to remain anonymous'.

Zork took the Dungeon world, split it up into three worlds, and then added a bunch more stuff to each part of it, although most additions were to Zork II and III, IIRC. Zork I was mostly identical to the early part of Dungeon.

The Adventure versions are are also known by their point values (330, 551, etc.). There are modified versions of Adventure which add large amounts of other areas, and up the points to as many as 1000 points. I've played Adventure on machines ranging from my PalmPilot to PC's of all shades, to Vaxen and even a Prime mini/mainframe which had the largest and highest point version I'd ever seen. (>1000 points, iirc).

The names Zork and Dungeon have been completely intermingled. I was under the impression it was originally named Dungeon, and then later named Zork, but many of the history pages have it the other way around.

The original Dungeon/Zork had 'GDT', a 'Grand (Game?) Debugging Tool', that let you examine objects, and rooms, in the world. Getting to it required knowing some magic way of translating a key that was printed when you typed in 'GDT'. In my case, it meant teaching myself VAX assembly language, so I could debug and patch the binary, so whatever I typed was accepted...

Some more information is available here (Colossal Caves), and here (Dungeon/Zork), and you can find out more about Interactive Fiction's history as well.

I'm a terribly long-standing fan of IF, even before it had that moniker, having learned a lot about programming by writing text adventure games, parsers, and all the database-like coding needed to make a good text adventure game.

I can still lose myself in the games, just like I can lose myself in a really good book. It's a different world, and a lot of fun as long as you're okay letting your imagination provide all the graphics.

-- CyberFOX!

Re:Hmm, let's see ...

[Koschei]

(a) use pod (q.v. perldoc perlpod )

(b) get a better editor. sane editors (vim, emacs, presumably others) are able to comment regions, or automatically insert comment leaders as you type, or both.

(c) Acme::Comment