Slashdot Mirror
Extending and Embedding Perl
It is my experience that many situations require us to "look under the hood" of (thoroughly examine) a solution to understand how to best use it effectively. Perl is no exception. The ability to bring such a force as Perl to a project at the proper time is a valuable skill to possess. However, wading chest-deep into XS and the Perl internals is not for the faint of heart. Jenness and Cozens ease this process by stepping in lightly at first.
What's in it?The book begins with simple C examples that are then related back to the readers' knowledge of Perl. Then the text seems to throw us a curve by leaping off into building Perl modules. But there is method to the madness: building Perl modules correctly is inextricably linked to XS. Light introductions to XS are performed and the reader is well on his/her way to building .so extensions to any .pm.
After building a very specific foundation of simple C examples, module building, and some XS, the text returns to C to introduce pointers, arrays, file I/O and memory management. With these new skills, we begin to explore the structure and implementation of Perl variable types. Chapter 4 provides many useful diagrams of how Perl variables "look" and what C structures they translate into.
Still following a logical and constant order, we explore the Perl 5 API, learning how to post and retrieve information to the variable types explored in the previous chapter. As much as it might seem, this is not a rehash of the perlapi doc. It is consistent with the perlapi doc, but Jenness and Cozens provide extensively annotated C code examples.
Casting deeper still, we add the advanced C of pointers, arrays, file I/O and memory management to our knowledge of XS. At this point we have everything we need to effectively extend Perl, but the text continues deeper still by exploring how XSUB interfaces to Perl's internals. It is only the clearly documented, step-by-step explanations of this chapter that make it manageable for an average user like myself. Chapter 7 ends our stint with XS by discussing some alternative XS (or equivalent code) generation suites.
Switching gears entirely, we grab libperl.a and stuff into a C program. Chapter 8 begins the task of embedding Perl into a C program. Jenness and Cozens continue the embedded discussion through a Case Study in Chapter 9 and end with a look through the Perl internals in Chapter 10.
The final chapter (Chapter 11) details some of Perl's history, its development process, how we could become involved and what the future of Perl and Perl 6 may entail.
Final Thought
This book was indispensable in gaining a good foothold on using Perl in, from, and around C. I found it a good reference guide as well as an easy ,linear read. It is not a replacement for the perlguts, perlapi and perlxs documentation, but then again, it doesn't try to be. The annotated code examples with every line explained make following the book with development of your own solution a lot easier than in some other books. However, the in-depth explanations can be a bit frustrating for the impatient.
You can purchase Extending and Embedding Perl from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
For many applications, you might find it easier to use Inline::C or SWIG. Neither gives you the total power that XS does, but they're much easier to get into.
gperf
So now we can combine the readability of both C and Perl with the elegant memory protection of C?
Isn't Perl 6 coming out soon?
Maybe the author should have held off releasing this book
HallmarkOrnaments.Com
I'm interested in using Perl as an interpreter for other applications. Or, more speficially, game scripting logic. I am confident the time has come for a scripting language to be used for NPC interactions, even in high-performance environments. However, I've never worked with Perl in such a place. I would like to. Has anyone got any performance comments about embedded Perl?
While I can think of numerous times embedding perl in my C would have made life easier, I can't think of any real reason to actually do so, it seems like a real waste of time to me. If you want to complete a project in Perl, then just do it in Perl. If you want to use C for a project, just use C. It's bad enough when people start using buggy bloated libraries, but it's worse when they give their compiler an identity crisis.
It's much like the great masters of Funk, The Parliament once said: "I've been down to the south, saw some great Funk...The Doobie Brothers...but do you want white guys all up in your Funk?"
In the same way, "I've been down through the source, I saw some great code...Perl...but do you want scripts all up in your C?"
If there is a God, you are an authorized representative. - Kurt Vonnegut Jr.
XS Mechanics
Well, I learned it so I could write scripts on ;-)
open platforms.
Ben "You have your mind on computers, it seems."
It's a bug. I'm fixing it.
Look for a companion volume to be published by Microsoft Press entitled, "Embracing and Extending Perl."
When that happens it will be just about time for the second coming!
the above is my personal opinion and does not necessarily reflect that of the little voices in my head
This book explains how to expand the functionality and usefulness of the Perl programming language. This guide delves into the complex issues of using real code examples from the Perl source. Detailed is how to use Perl from C programs, such as writing interfaces to C libraries, implementing Perl callbacks for C libraries, and passing Perl hashes and arrays between Perl and C. Additionally, developers are provided with an API reference for the internal C interface to Perl and a reference on the typemap system.
It's amazing how much this book covers: Not only does Sam Tregar show how object-oriented Perl modules are architected, how to write regression test suites, how to extend Perl modules with C code, but he gets also the community aspects right -- how does your module get really popular? You can tell that Sam is a successful Perl module author himself.
-dk
Java does everything that C++ does
Uh, no. Thanks for playing. There are things that C++ does that Java does not -- some of which I'm thankful do not exist in Java (preprocessor) and some of which I miss (generics). But despite its C-like syntax and superficial resemblances (finalizers seem like destructors but aren't) Java is more like Smalltalk than C++.
Take a quick gander the section For C, C++ Fans in Peter van der Linden's Java Programmers FAQ
But then, why am I arguing over the relative merits of Perl, Java, C++, and C# with a user having the handle "Microsoft Research" who posts pure FUD?
An "8.7". Not as bad as the totally garbage "8" books, but well below the standard "9".
Recently, the DotGNU have made an overture to try to use the Parrot runtime for their C# compiler but found that Parrot needs a lot of work to get to the point where they could use it.
Some Parrot VM problems:
no calling conventions yet for subroutines. There is no hope of offering mixed language support unless they do this.
no conversion opcodes for various builtin types (float, char, short, int)
non-perl languages expected to provide additional support in the form of C code libraries for their opcodes. This would nix any hope of having a single standard universal virtual machine.
no way to call out to C code
no equivalent of Java's jar file or CLR's assemblies for parrot library distribution
way too many registers: their register based VM (32 int registers, 32 double registers, 32 string registers, 32 PMC registers plus various stacks) requires a sophisticated compiler to do proper register allocation and needlessly complicates their VM.
no consideration of threads in their design. How will they handle synchronization, for example?
The points above are not coding issues, but issues of design. It seems that Parrot is too hung up on making the VM efficient and are not seeing the bigger picture - to get the features in place first so that high-level languages can work. Or perhaps they should simply concentrate on getting Perl6 to work first. They need more focus. The project tries to be all things to all people, but ends up satisfiying no one.
bn.com has the book for $35.96 Amazon has it for $31.47.
----
Associates Link
As for C in Perl... Perl is a scripting language, it's simply not fast enough for everything, and you're going to need C to access different things, like joysticks, video, graphics libraries, etc...
Sticking feathers up your butt does not make you a chicken - Tyler Durden
Why would you want to embed perl into a C application? Probably to access perl's powerful regular expressions. If that's the case, the Perl Compatible Regular Expressions library is a more ideal solution. PHP, Python, and Ruby all use it to support perl regexes.
Do you even lift?
These aren't the 'roids you're looking for.
In my case, I'm part of a large scale C++ project. I have the ownership of a module with clearly defined interfaces with the other modules written in this project.
Since my module relies heavily on XML and strings, I have always wanted to pair it with the power of Perl for testing purpose.
Among various possibles solutions (XS, SWIG, etc.), I settled on SWIG because it could handle 'shallow' classes. (allowing to expose my module as a perl object)
This has been the best decision I have made over the last year: when I get a bug case, I simply write a perl script to try to reproduce the problem, add some loops to get some combinatory, then check the result. This drastically cuts down on the time spent on debugging my module (or the modules used by it, for that matter :)
Pros:
- SWIG: Relatively easy generation of stub code (by using interface files)
- SWIG: It is possible to use the same interface files to generate stub code for Java, Python (though, I didn't test this feature)
- SWIG: Excellent doc.
- Perl: you can leverage the CPAN/PPM modules to do some truly magical hackings
Cons:Summary: If you are a C/C++ developper and your code can use XML/text files/strings, consider using SWIG or XS for testing purpose.
PS: if you want to Quantify/Purify your module/Perl script, using ActiveState Perl, you need to recompile Perl with the -DPURIFY option toggled on.
- no calling conventions yet for subroutines. Alas not true. They're in place and have been for quite some time.
- no conversion opcodes for various builtin types Also incorrect--we've got them and have since the very beginning. What we don't have is low-level support for specific size integer and floats, since our target languages (perl, python, ruby) don't have them. Adding them adds no overhead, though, so they're going in since it'll make base
.NET and JVM compatibility simpler.
- non-perl languages expected to provide additional support in the form of C code libraries for their opcodes. Once again, incorrect. No external libraries are or will be required. We're turing complete and generally have a richer set of base semantics than
.NET or the JVM does. This doesn't mean that someone can't choose to require an alternate set of opcodes if they want, but the engine doesn't require it.
- no way to call out to C code This one's actually true. We've not gotten to that part yet, though it's on the list.
- no equivalent of Java's jar file or CLR's assemblies for parrot library distribution Incorrect, bytecode files work just fine for this. That part of the design is somewhat incomplete, though.
- way too many registers: their register based VM
... requires a sophisticated compiler to do proper register allocation and needlessly complicates their VM Wrong yet again, sorry. Doesn't requore a sophisticated compiler at all. At best it requires some sophisticated register allocation algorithms. Luckily for us, those algorithms are old, known territory, which is why we've got them implemented already. So what if it makes the VM slightly more complex? (And it is only slightly more complex because of it) We're not writing something for CS101 here.
- no consideration of threads in their design. Once again, incorrect, if you'd bothered to read any of the design documents. Threading isn't, at the moment, implemented because other things have been more important, but it has been thought about in the design.
1.5 for 7. Not too good there. Oh, and this: What drugs are you on? If we piss away efficiency in the design, no amount of clever coding will ever get it back. Maybe you're willing to sacrifice a factor of two in speed for "clarity" but so are nine of your friends. (To misquote the author of make) I, on the other hand, am not.Finally
If you'll look, you'll notice that perl 6 isn't fully designed yet, but the bits that are have been implemented.Just because you can't (or won't, or don't want to) see the focus doesn't mean it's not there. It is, thanks very much, and we're well on track to do what we need and do it well. The design's flexible enough to pick up things like JVM or .NET compatibility without a loss of focus or efficiency, so there's no reason not to.
Dude, by simply saying that something is not true does not make it true.
The Parrot calling convention POD was pulled a while back. I don't call saveall/restoreall a true calling convention. Prove me wrong - show me a doc with the new calling convention.
Where's Parrot's 64 bit integer type? If you hope to support Java and C# you will need it.
You *do* need some form of distributable bytecode library format - bytecodes alone do not cut it - all projects cannot be monolithicly compiled in a single bytecode file.
Threading issues *must* be considered from the very start - it affects every aspect of your design - from garbage collection to memory layout. Look at all the trouble Perl 5 had grafting threads to its design - they only "got it right" in Perl 5.8.
You're living in a dream world of denial. The project has to address these basic issues if it is to suceed.
It's history repeating itself all over again. Remember what happened to Topaz: Perl for the 22nd Century?
The result, of course, is undebuggable random crashes in the high-level part of the system. Here's are some typical bug reports from mixed Perl/C work:
-
#1
Okay, it seems to be some kind of conflict between mod_perl/Embperl and PHP and
perhaps Apache::DBI. My Embperl stuff works if there's no database access. It
also works if I don't load libphp4.so. I guess the best solution is to either
build everything statically or run seperate servers for PHP and mod_perl.
-
#2: Following a large number of updates to our database, slapd is prone to crashing
when reading values back. We load a database of about 3800 users with slapadd,
then modify a single attribute of every 'person'. Then slapd is likely to
crash
on reading values back. Restarting slapd seems to make it work again. Just
prior
to the crash, slapd will give incorrect query results.
... We have a large client site limping along due to this kind of problem ... so any
help would be welcome.
After this, you begin to understand the logic behind Microsoft's C# mixed-language run-time environment. That's ugly, too, but more maintainable, because the toolset has some support for mixed-language work.I'd like to see safe inter-language calls across a protection boundary. CORBA is about as good as it gets, but it's slow, because it marshalls the data into a stream and pumps it through a socket to the other side. There are faster approaches (look at Multics protection rings) but they need some hardware support, which we don't have today.
The only part of history repeating itself is the naysaying from people who've not done their research. If you're going to criticize, at least get it right.
Perl, IMO, is the worst of the scripting languages to combine with C... the interface is not pretty. Other languages like Python aren't bad. Lua is good if you want something really small and fast.
http://www.welton.it/davidw/
People who know C probably will not have
:)
problems learning Perl, but I always thought
it's more fun to learn new way of thinking
that comes with a new language. Thinking
of a new language in terms of the one you
know may result in taking the old-language
style of programming to the new one - so
what's the point of learning?
You can probably explain LISP in terms of Basic
concepts, but that's a waste...
Considered harmful.
If I remember the talks Dan has given to Boston.pm correctly, you've got this almost completely backwards. A more accurate reading is that Parrot is designed to act as kind of a superset of the target languages (Perl, Python, and Ruby) and that, in the handful of cases where an abstracted general feature of Parrot doesn't map perfectly back to the languages, any of those languages -- *including Perl* -- may have to adapt to Parrot's needs. In other words, Parrot has certain features because the Python & Ruby folks needed them, and these will require Perl to adapt if it wants to take advantage. (If we're lucky, Dan may clarify &/or add to this -- it's been a few months now and I forget most of the details of the presentation...).
So let's not have this kind of trolling, please. If you think Parrot is making such fundamental mistakes, help out or fork off your own project to prove yourself. Antagonistically spouting off half-correct technical gibberish on Slashdot isn't helping anyone.
DO NOT LEAVE IT IS NOT REAL
You seem to be missing the point with some regularity. PMCs don't have to be written in C--they will be doable in parrot code if need be. And just because 64 bit integers will be done with PMCs doesn't mean that they won't be part of the core distribution, or a recommended library.
.NET library to get full .NET functionality will undoubtedly be needed. (We're certainly not going to ship the full .NET core library with Parrot any more than we're going to ship the full Java core library) If we don't ship 64 bit ints as part of the core, they'll either be in the .NET library, or in an extended data type library.
There's nothing particularly wrong with saying "You must have the X library/module/kit to do Y". Requiring the install of the
What's next, will you start complaining next that we're going to require installing Postgres to access Postgres databases? (Or will the next complaint be about the bloated size of the distribution to provide the features that match your expectations?)
But for code that calculates my taxes or bank balance, or drives the displays at the operating theatre if I'm brought in for emergency heart surgery, I want implementation languages where there is no question of ambiguity in the interpretation of the semantics of the code.
--
Perl, C, C++ (urgh) hacker. Don't tell anyone I can read COBOL.
...and Buy.com has it for $28.31 plus free shipping.
Chapter 7 is alternatives to XS. He talks about SWIG and Inline::C as well as some other lesser known things like PDL::PP.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
It's a good thing, according to Larry, because Perl more closely maps to spoken language patterns. His theory is that the computer should do extra work to make the life of a programmer easier.
I don't understand your comment about interpreting that code as a boolean construct. That's exactly how Perl does evaluate it. See B::Deparse for clarification.
how to invest, a novice's guide