Perl & XML

dooling writes: "Perl & XML is a well-written book that accomplishes what it sets out to do. It states in the preface that it is written for Perl programmers who want to learn about XML and what is available in Perl for XML processing. It achieves this goal, but little else. When you are done reading this book you will have been given an overview of Perl and XML, know where to begin to attack an XML document, and know where to look to find more information." For dooling's more complete review, read on below. Perl & XML author Erik T. Ray & Jason McIntosh pages 202 publisher O'Reilly and Associates rating 6 reviewer dooling ISBN 059600205X summary Good introduction to XML for Perl programmers.

The book starts out with a brief explanation of why XML and Perl are well-suited for each other. It then provides a teaser of things to come: an explanation of how to use the XML::Simple module. The first chapter concludes with some warnings and gotchas that seem a little premature since they have not really explained XML. Fortunately, most of these gotchas are covered in context later in the book.

The second chapter provides a whirlwind overview of XML -- covering its structure, DTDs, schemas, and XSLT (transformation). The discussion of XML in general, its history, and parts of an XML document are well done. They give someone who is familiar with static HTML the needed background to understand the structure of an XML document and the vocabulary used to describe it. Unfortunately, the discussion of where XML begins to distinguish itself from HTML, namely with DTDs, the new replacement for DTDs called schemas, and the transformation language XSLT, is too brief. They gloss over these topics with little explanation and few examples. That said, there are other books that do provide more in-depth coverage of XML (this book only promises an introduction).

The next five chapters cover Perl modules designed to process XML, starting with simple parsers and writers. Only methods and syntax relating to XML processing are explained. Therefore, if you are considering reading this book, you should be fairly comfortable with Perl and object-oriented (OO) interfaces to CPAN modules (nearly all the modules discussed provide OO APIs). Again, there are other books and perldoc documentation that cover Perl and it's OO features; so read them first if you are not familiar with OO Perl. If you are familiar with OO Perl, these chapters provide a good overview of the different ways XML can be processed (stream- and tree-based approaches), the advantages and disadvantages of each, and the Perl modules best suited for each approach. These chapters are the biggest strength of this book. The modules discussed in these chapters are by no means an exhaustive list of XML-related modules available from CPAN nor do the explanations of each module cover everything the module does. These chapters do, however, provide the reader with enough information that she can begin to process XML documents intelligently and know where to turn when she needs more information.

The next chapter, Chapter 8, covers XML tree iterators, XPath, XSLT, and XML::Twig. All of these topics are covered in a span of 16 pages (with only slightly over two pages dedicated to XSLT). Indeed, after reading the chapter, you may get the feeling that it was only included so the authors could cram more trite colloquialisms into the book. The short shrift given to these topics creates the impression, which is strengthened in the chapters that follow, that this book was rushed a bit to press.

Chapter 9 discusses applications of XML, including RSS and SOAP, and Chapter 10 is mostly example code. These chapters are intended to give you a feeling for what is possible without really giving you enough information to make it happen. The main problem with these chapters are the examples: the examples are long and the explanations are short. Thus, they are more useful as templates or a quick reference than for learning these topics in detail. Of course, the authors never promised you would be programming SOAP applications when you were done reading this book. And again, there are other books out there which discuss these topics in more detail. So the authors stay true to their promise throughout the book: they will introduce you to XML and tell you how to interact with XML using Perl, no more.

Personally, I found this book did, in general, give me enough information to get started using XML and pointed me where I needed to go to get more information. I am an experienced Perl programmer who is new to XML and comfortable with on-line documentation. This book seems to be written for people who fit this profile and who want to learn by doing (finding the answers to the "hard" questions as they arise). It does introduce a wide variety of XML-related topics and the Perl modules used to interact with them, which is what the authors promised to do in the preface. While it is by no means an authoritative text on Perl and XML, there is something to be said for keeping promises ...

Index As with most first-edition books, the index was adequate but not complete. For example, XML::Twig, which has an entire section covering it, does not appear in the index at all.

Contents
Preface

1. Perl and XML
2. • Why Use Perl with XML?
   • XML Is Simple with XML::Simple
   • XML Processors
   • A Myriad of Modules
   • Keep in Mind ...
   • XML Gotchas
3. An XML Recap
4. • A Brief History of XML
   • Markup, Elements, and Structure
   • Namespaces
   • Spacing
   • Entities
   • Unicode, Character Sets, and Encodings
   • The XML Declaration
   • Processing Instructions and Other Markup
   • Free-Form XML and Well-Formed Documents
   • Declaring Elements and Attributes
   • Schemas
   • Transformations
5. XML Basics: Reading and Writing
6. • XML Parsers
   • XML::Parser
   • Stream-Based Versus Tree-Based Processing
   • Putting Parsers to Work
   • XML::LibXML
   • XML::XPath
   • Document Validation
   • XML::Writer
   • Character Sets and Encodings
7. Event Streams
8. • Working with Streams
   • Events and Handlers
   • The Parser as Commodity
   • Stream Applications
   • XML::PYX
   • XML::Parser
9. SAX
10. • SAX Event Handlers
    • DTD Handlers
    • External Entity Resolution
    • Drivers for Non-XML Sources
    • A Handler Base Class
    • XML::Handler::YAWriter as a Base Handler Class
    • XML::SAX: The Second Generation
11. Tree Processing
12. • XML Trees
    • XML::Simple
    • XML::Parser's Tree Mode
    • XML::SimpleObject
    • XML::TreeBuilder
    • XML::Grove
13. DOM
14. • DOM and Perl
    • DOM Class Interface Reference
    • XML::DOM
    • XML::LibXML
15. Beyond Trees: XPath, XSLT, and More
16. • Tree Climbers
    • XPath
    • XSLT
    • Optimized Tree Processing
17. RSS, SOAP, and Other XML Applications
18. • XML Modules
    • XML::RSS
    • XML Programming Tools
    • SOAP::Lite
19. Coding Strategies
20. • Perl and XML Namespaces
    • Subclassing
    • Converting XML to HTML with XSLT
    • A Comics Index

Index

You may also want to check out Erik T. Ray's home page, Jason McIntosh's home page, or O'Reilly's page for the book. You can purchase Perl &amp XML from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

Right...

[alexhmit01]

You realize that if I get an XML file, I can figure out what it is saying and decide what to do with it. With your ideal (binary) files, I need to reverse engineer the format.

With binary, I need permission to interoperate. With XML, I need a text editor (or print-out) and some common sense.

You worry all you want about the computer's efficiency. I use my machines to make my life easier. I don't jump through hoops to make the computer's life easier...

Taking troll bait,
Alex

Re:Right...

[jkroll]

With XML, I need a text editor (or print-out) and some common sense.

Actually you still need the same thing you would need with the binary format. You need the documentation as to what the tags mean. Try looking at XML produced by commercial software sometimes. Just because SAP calls the tag "ITEM" doesn't mean it refers to a part number or something you can pick up at the store. And looking at the schema or DTD probably won't help you figure out the tags mean either.

Finally parsing XML is easy when the document produced by an external system exactly mirrors your internal representation, unfortunately if you are interfacing XML documents with external systems this will almost never be the case.

Don't get me wrong, XML has its uses - just the exchange of data between disparate systems isn't one of them.

Why parse XML in the first place?

[SystemFork]

Parsing XML indeed. I mean seriously, have any of you ever actually tried to impliment XML parsing? It's an order of magnitude slower than accessing a database, ten zillion times slower than reading a flat file ASCII database, and a trillion times more expensive (well, I'm exaggerating a bit) than reading in a text file with nested variable=value pairs.

Interoperability is great and all, but I think XML is nothing but hype.

Programmers, hear my cry! Spend your precious hours working on your program interface, your error-checking, your overall design and modularity, don't spend time worrying about a scheme with a fancy name that saves data like this: value.

Don't mod me up or down, I just want to foster a discussion about this. I mean, as a standalone programmer using Perl for a majority of their web application products, what benefit does XML give you other than buzzword compliance?

----

Re:Why parse XML in the first place?

[eaeolian]

Having done some Java/XML work last year, then stumbling back into a homogenous Windows environment, I can honestly say I think XML has strengths and weaknesses.

Speed, however, is a primary problem, or at least it was when we were using XML to store/parse tens of thousands of elements for a Financial Services app in Java. It had the advantage of not being tied to a particular platform, and needing no database of any form distributed with it, and being read-compatible with some existing applications. So it was worth it to the client.

In my current environment, it makes no sense, other than "buzzword compliance".

So it's got it's uses, but it's not a magic bullet. Then again, I think Client/Server is a better solution than going web-based for a lot of things, so what do I know? :)

Re:Why parse XML in the first place?

[scott1853]

We just dumped XML for our primary storage format for an application we're about to release. We're still using it as a backup/restore format though.

The problem was that while our program will generate probably only a few K of data per day in binary format, once you translate that binary data to XML with descriptive tag names and attribute names, it would get to be 100K per day due to the number of attributes per record. After a couple months you may be looking at 10 megs or more. Load/save times were several minutes. We switched to the BDE (I know it sucks) and load times are instant due to being able to load records on demand instead of everything at once.

Re:Why parse XML in the first place?

[JohnFluxx]

The trouble is that people use it wrongly. (IMHO)

If it is done right, most of the time you should never have any actual XML text!

Store the data you are using in DOM, and manipulate it there. Store the data in DOM format on the disk, with a way to dump to XML if you have to (which you rarely will). Compile your Schemas, etc.

I'm also not sure why you imply it is costing time using XML - one of the ideas is that it is supposed to save time - and it does.
I can write schemas to validate the output and input, write XSL's to transform the data to anything, and so on.

Re:Why parse XML in the first place?

[The Pim]

Interoperability is great and all, but I think XML is nothing but hype.

Heck, let me give this my crack... :-)

Ok, obviously the biggest reason for XML's popularity is hype. That's just the way the industry works; it doesn't make XML good or bad.

There are a several legitimate technical benefits to XML, that might be persuasive in one context or another.

• It looks like HTML, so everyone intuitively "gets" it.
• It's textual (not binary)--but of course, many formats are textual.
• It's reasonably easy for humans to understand without a spec, provided the tag and attribute names are not obfuscated, and the relationships are relatively simple. Note this does not make it easy for programs to understand!
• You don't have to write your own parser. You don't even have to write a grammar--just throw in a tag and the corresponding code to read and write it. This advantage is not as big as some make it out to be: many languages have easy-to-use features for parsing, and those that don't can make use of easy-to-use parser-generator tools.
• There are lots of libraries and tools. Of course, this is self-reinforcing (tools -> popular -> more tools -> more popular -> ...).

Many XML proponents, including some in this thread, would add to this list that XML is a good data storage and/or interchange format. Some "insightfully" note that it is better for data interchange than data storage. This is the biggest delusion over XML: XML is a rotten format for data.

Remember what XML was back before the hype machine was in overdrive? It was a better HTML, and a simpler SGML. HTML and SGML have always been formats for documents, and XML was intended to be the same. XML is indeed a pretty good match for documents. (This is debated of course: documents are complex things, and modeling them is non-trivial. Embedded Markup Considered Harmful, by Ted Nelson, is a good introduction.)

But XML is a poor match for data. This is because an XML document is a tree, and most data are not hierarchical. Consider that the database industry abandoned hierarchical databases many years ago (ok, abandoned is a little strong: we still use LDAP). Hierarchical data formats force you to pick which relationships will define the hierarchy, and any other relationships have to be kluged in.

Take a simple example of the sort of thing people use XML for: address book entries. Say you start out with a person element (I'm not going to write out the examples in XML syntax because it's too painful on slashdot) containing a name element and an address element. Now, you realize that multiple people may live at the same address, and you don't want to duplicate the address (data formats should be normalized). You either have to turn things inside out, putting the person element inside the address element, or make person and address both top-level elements, and link them somehow. In the former case, you have chosen an awkward hierarchy, and have "used-up" your ability to group people. What if you want a different grouping in the future? In the latter case, you have given up a lot of simplicity and read/writability (since now names and their corresponding addresses are in different places) by forcing non-hierarchical data into a hierarchical format.

What is the solution? Well, I won't assert that it is the best data model that will ever exist, but the database industry has settled (roughly) on the relational model. So I think we should create a format describing relations, combined with the other advantages of XML: extensible, textual, readable, and most of all, standards-based. Yes, this would mean we would have to learn two technologies, one for documents and one for data. But the technology for data would be so much simpler--and as a bonus, integrate easily into our databases--that it would be a huge win overall. I don't have time to defend this model in depth. But think about it.

By the way, another example of the bad match between XML and data is the great debate over when you should use elements, and when you should use attributes. The fact that there is an arbitrary decision to be made shows that XML has degrees of complexity that only get in the way when you use it as a data format. (If you're going to use XML for data, at least have the decency to eschew attributes except for an id attribute.)

C sucks (parody)

[FortKnox]

C is a giant leap backwards. It makes us convert data from an verbose ASCII format, translate and compile it into a binary format. This leads to:

• Conversion/translation complexity
• Syntax Errors
• Conversion errors
• Storage requirements (object files)

The only benefit AFAIK is that people can read the code better. However, the applications still have to understand the standard coding syntax, which comprises of a hideous amount of keywords and styles. Said applications would have been better off using Assembly (read: efficient) code in the first place.

Ban C!


Please note the extremely sarcastic tone of this post.

Your complaints are old fashioned. Maintainability is a major overlooked flaw in Computer Science.

Re:XML sucks

[sporty]

Ok, i'll reply to this. I'm proly being trolled :P

Conversion complexity, granted. It does take a bit of work. But would you recomend describing each record with individual lines? That's a bigger pain than ever. What XML gives isn't just a structure for your data, but a language to describe it. It also allows for non-2d data. By this, we can have people with subsets of data, with subsets of different types. This is great, as now we can have a language that describes data in a logical manner and be completely portable.

Conversion errors, please be more spefic. If you convert to a comma delimited format, you are screwd if you do it wrong. If you do it straight to binary, you have to worry about how many bits represent any given data. Why do you think that pack has so many different switches for converting data?

Just because you use XML doesn't mean you must store data in XML form. Hell, it's stupid if you have gigs of data to use XML to relate it all. DB's dont' use xml except for expression of data back to the user/software it talks to (if asked for).

If you are worried about bandwidth, on a simplistic level, gzip it. Yes, compress it. Hell, do a gzip stream which is supported by many browsers.

If your program plans on sharing data, you'll want to use XML. If you never want to share your data, fine use binary. It's not terrible. But once you wanna share it between two machines or processes, now you have to worry about deciphering the binary format. THat is.. unless you work by yourself and have documentation on everything.

www.xmlsucks.org

[alispguru]

For a more detailed, and more depressing, take on the above, see http://www.xmlsucks.org/but_you_have_to_use_it_any way/.

Yes, it's a PDF. Unroll it - it's worth the effort.

XML should not be a database replacement.

[jjn1056]

I use XML as the interchange format for a web publishing system which publishes our internet web site (http://www.bms.com), but the data is actually stored in an oracle database. I have a perl object which handles all the fuss of getting/putting xml into the database.

As an interchange format XML is ideal; think of it comma separated files on steroids. When all your data can be serialized to XML you get the following benefits:

1. XML has rich data structures for complex info.
2. XML can be self describing.
3. XML is 100% portable.

Like HTML, people will discover uses for your XML files that you never thought of. Also, if you lose all the docs, you can read the XML in a standard text or unicode editor and figure it out. This is even better than comma separated, since most CSV files don't bother to include a first row field discription.

Like CVS, you can parse XML files with standard command line tools like grep. And in 100 years, all those Oracle tablespaces will require a lot of reverse engineering to get the data off it, while your text based xml files will still be parsable.

I agree though, with the general notion of the parent. Definitely don't do XML because it sounds cool. Use the best process for the job, and for many data related jobs, relational tables and SQL are best.

One thing you can do to improve speed; serialize your DOM objects using the Perl Storable module, and save along with your plain text versions. Then when you need to access the data, all you need to do is unserialize the object, which is a lot faster than reparsing.