Extending and Embedding Perl

habit forming writes "Enjoy using Perl? Ever marvel at how Perl can "do the right thing" but still be written in C? Extending and Embedding Perl aims to take the black magic out of understanding our favorite language. In fact, the authors flat out admit they think it is unfair that only so few of us get to have one foot in Perl and one in C. Tim Jenness and Simon Cozens attempt to break down that barrier with lots of annotated code examples, direct analogies from the structures in Perl to those in C, a fine-grain look at XS and what it takes to robustly use a Perl interpreter in C." Extending and Embedding Perl author Tim Jenness and Simon Cozens pages 375 publisher Manning (http://www.manning.com/) rating 8.7 out of 10 reviewer habit forming ISBN 1930110820 summary Get in touch with the inner Perl. What's that up your sleeve?

It is my experience that many situations require us to "look under the hood" of (thoroughly examine) a solution to understand how to best use it effectively. Perl is no exception. The ability to bring such a force as Perl to a project at the proper time is a valuable skill to possess. However, wading chest-deep into XS and the Perl internals is not for the faint of heart. Jenness and Cozens ease this process by stepping in lightly at first.

What's in it?

The book begins with simple C examples that are then related back to the readers' knowledge of Perl. Then the text seems to throw us a curve by leaping off into building Perl modules. But there is method to the madness: building Perl modules correctly is inextricably linked to XS. Light introductions to XS are performed and the reader is well on his/her way to building .so extensions to any .pm.

After building a very specific foundation of simple C examples, module building, and some XS, the text returns to C to introduce pointers, arrays, file I/O and memory management. With these new skills, we begin to explore the structure and implementation of Perl variable types. Chapter 4 provides many useful diagrams of how Perl variables "look" and what C structures they translate into.

Still following a logical and constant order, we explore the Perl 5 API, learning how to post and retrieve information to the variable types explored in the previous chapter. As much as it might seem, this is not a rehash of the perlapi doc. It is consistent with the perlapi doc, but Jenness and Cozens provide extensively annotated C code examples.

Casting deeper still, we add the advanced C of pointers, arrays, file I/O and memory management to our knowledge of XS. At this point we have everything we need to effectively extend Perl, but the text continues deeper still by exploring how XSUB interfaces to Perl's internals. It is only the clearly documented, step-by-step explanations of this chapter that make it manageable for an average user like myself. Chapter 7 ends our stint with XS by discussing some alternative XS (or equivalent code) generation suites.

Switching gears entirely, we grab libperl.a and stuff into a C program. Chapter 8 begins the task of embedding Perl into a C program. Jenness and Cozens continue the embedded discussion through a Case Study in Chapter 9 and end with a look through the Perl internals in Chapter 10.

The final chapter (Chapter 11) details some of Perl's history, its development process, how we could become involved and what the future of Perl and Perl 6 may entail.

Final Thought

This book was indispensable in gaining a good foothold on using Perl in, from, and around C. I found it a good reference guide as well as an easy ,linear read. It is not a replacement for the perlguts, perlapi and perlxs documentation, but then again, it doesn't try to be. The annotated code examples with every line explained make following the book with development of your own solution a lot easier than in some other books. However, the in-depth explanations can be a bit frustrating for the impatient.

You can purchase Extending and Embedding Perl from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

interpreter for other applications

[Sludge]

I'm interested in using Perl as an interpreter for other applications. Or, more speficially, game scripting logic. I am confident the time has come for a scripting language to be used for NPC interactions, even in high-performance environments. However, I've never worked with Perl in such a place. I would like to. Has anyone got any performance comments about embedded Perl?

Re:interpreter for other applications

[Anonymous Coward]

Perl is not the fastest embeddable interpreted language, but it ain't the slowest (TCL) either. The problem with most embedded interpreters is that you have to constantly convert to/from the native C types in your code. This can have a large performance overhead. If you do decide to embed with Perl (or other interpreted languages for that matter), keep your API as coarse-grained as possible, i.e., do most of your work in C functions and only do high-level stuff in perl.

Perl is Perl, C is C

[signine]

While I can think of numerous times embedding perl in my C would have made life easier, I can't think of any real reason to actually do so, it seems like a real waste of time to me. If you want to complete a project in Perl, then just do it in Perl. If you want to use C for a project, just use C. It's bad enough when people start using buggy bloated libraries, but it's worse when they give their compiler an identity crisis.

It's much like the great masters of Funk, The Parliament once said: "I've been down to the south, saw some great Funk...The Doobie Brothers...but do you want white guys all up in your Funk?"

In the same way, "I've been down through the source, I saw some great code...Perl...but do you want scripts all up in your C?"

Re:Perl is Perl, C is C

[hondo77]

In my case, I wanted to use a third-party's binary-only C API from Perl. Gotta use XS.

Re:Perl is Perl, C is C

[NerdSlayer]

I agree. Scripting seems best for a quick and dirty approach, while actually writing some C or C++ takes more time and consideration. I can't see why you want to mix the two, unless of course you need some low level access to hardware that Perl doesn't offer directly...

Spoken like a dumb college student. I work for a small company started by my friend from college. We do everything almost exclusivly in perl (with some PHP). Our application is mostly a CGI interface to a obscenely diesel MySQL server (80 GB+, some tables have over 300 million rows, you dirty mysql nay-sayers). It has some low level functions that I wrote in C. It's extrememly modular. Perl has allowed us to write small daemon programs quickly. As our system grows large and larger, we're easily able to replace to slow performing sections with a optimized C version of the same daemon.

We took our particular space (I'd rather not get into it) by storm. Me and one other developer worked day and night for 6 months and we now have a product that we license for between $50,000 and $250,000 depending on how the client wants to scale. It is easily in the top 3 of similar programs. And it was written by me and another dude. In Perl. With some C. In 6 months.

As much as I love C and have mixed emotions about perl, if it was written in C we'd still be core dumping and our company would have probably gone out of business, as opposed to quadrupling in size and increasing our gross income 10 fold. Again, I hate perl, even, but there's no replacement for it. I, like you, was once a dumb college student who swore I would never write anything but C. But, then I graduated and realized I needed 22" dubs on my escalade. (okay, I have neither 22" dubs, or an escalade, but you can imagine if I did, right?).

C and C++ do take more time and consideration, and when you're part of a lean and mean company you do what you need to get ahead. If it wasn't for perl I'm 100% sure that we would not be where we are today.

Re:Perl 5 API???

[gorilla]

Perl 6 is still being designed, and is expected to be at least 18 to 24 months away from the first version. It would then take a period of at least a year before it was production ready.

Re:I think Perl5/XS will be with us for long time.

[truth_revealed]

Dude, by simply saying that something is not true does not make it true.
The Parrot calling convention POD was pulled a while back. I don't call saveall/restoreall a true calling convention. Prove me wrong - show me a doc with the new calling convention.
Where's Parrot's 64 bit integer type? If you hope to support Java and C# you will need it.
You *do* need some form of distributable bytecode library format - bytecodes alone do not cut it - all projects cannot be monolithicly compiled in a single bytecode file.
Threading issues *must* be considered from the very start - it affects every aspect of your design - from garbage collection to memory layout. Look at all the trouble Perl 5 had grafting threads to its design - they only "got it right" in Perl 5.8.
You're living in a dream world of denial. The project has to address these basic issues if it is to suceed.
It's history repeating itself all over again. Remember what happened to Topaz: Perl for the 22nd Century?

Re:I think Perl5/XS will be with us for long time.

[Elian]

Nope, but being right makes it true.

1. The Parrot calling convention POD was pulled a while back. No, it wasn't. PDD 3 is in, and has been for a long time. There have been minor revisions, but that's it--minor revisions.
2. Where's Parrot's 64 bit integer type? Inside a PMC, at the recommendation of the dotGNU folks. We'll stick the 128 bit, 256 bit, and 512 bit integers in one too, if we need them.
3. You *do* need some form of distributable bytecode library format No, we don't, not for perl, python, and ruby support. (Who was it whining about lack of focus earlier?) Bytecode files do work just fine for it anyway, however. Nobody said they needed to be structureless.
4. Threading issues *must* be considered from the very start Yes, I know. If you'd have bothered to do any research, you'd have seen the direction we're going, and you'd have realized that what we've done so far has no ramifications for our chosen threading model.

The only part of history repeating itself is the naysaying from people who've not done their research. If you're going to criticize, at least get it right.

Re:Perl 5 API???

[hondo77]

...every good programmer has a half dozen Perl 5 books anyways.

I do have a half dozen Perl books and I barely survived my first foray into XS. Maybe this book can help me go back and get it right.

Why do we need comparisons?

[DEBEDb]

People who know C probably will not have
problems learning Perl, but I always thought
it's more fun to learn new way of thinking
that comes with a new language. Thinking
of a new language in terms of the one you
know may result in taking the old-language
style of programming to the new one - so
what's the point of learning?

You can probably explain LISP in terms of Basic
concepts, but that's a waste... :)

Re:Perl: Fitting into the Big Picture

[Anonymous Coward]

Here's my view of the big picture.

System-level languages: C, C++. If you want to write an OS, or a native object library or executable, you'll probably use one of these.

Rapid prototyping and non-expert programming: a weakly-typed script language, such as Python. Here, bindings to any native object (as in code, not oriented) library can be provided for things such as OS interfaces, GUI class libraries, application specific APIs, etc. If you want to provide a command-oriented interface (as opposed to function-call-like syntax), you might choose Tcl instead.

Random data hacking: it really doesn't matter. Perl seems popular, but also sh/grep/sed/awk still works (and it can be done in Python as well). If you're not on an "open-source platform" you'll probably find no native tools for this and will wind up installing Perl or something similar, although you'll find the command-line painfully hard to use.

So then the question is: where to Java, C#, Basic and Lisp fit in. Nowhere, really. Java and C# were invented by Sun and Microsoft to take market share from one another. If you have a choice, it isn't clear why you'd use them because they offer all of the complexity of C++ with only the minimal benefit of taking care of some memory management issues. However, you may be forced to use one of these languages because the bindings to the libraries you want are only available there. There is likely no technical reason for this, only a marketing reason. As for Basic and Lisp, they are just old solutions to the rapid prototyping problem, and should be replaced with modern languages.

IMO, that's the way it is.