Does the World Need Binary XML?

sebFlyte writes "One of XML's founders says 'If I were world dictator, I'd put a kibosh on binary XML' in this interesting look at what can be done to make XML better, faster and stronger."

Comments

KISS

[stratjakt]

On the face of it, compressing XML documents by using a different file format may seem like a reasonable way to address sluggish performance. But the very idea has many people -- including an XML pioneer within Sun -- worried that incompatible versions of XML will result.

I agree with his point.

What's wrong with just compressing the XML as it is with an open and easy-to-implement algorithm like gzip or bzip2?

Re:KISS

[ZakMcCracken]

What's wrong with just compressing the XML as it is with an open and easy-to-implement algorithm like gzip or bzip2? I'll tell you one thing that's wrong: these compression algorithms might run fine on your desktop or server; but on an embedded system with restricted memory and CPU power, that's another matter...

Re:KISS

[e2d2]

What you said is right on target. I've worked with XML in a few applications (specifically web services) and everytime we saw a performance drop it was not because of a network bandwidth issue but instead it was because the documents were so large that the parser became the bottleneck. And then when you throw in style sheets for manipulation.. well you get the point.

So if the need is for compression over networks, well thats only half of XML performance problems. And if the end result becomes a binary format, then how is it, and why would it need to be related to XML in the first place? Data compression over networks is not a valid reason for another standard IMO.

gzip ?

[JonyEpsilon]

Am I missing something, or would just gzip'ing xml when it goes over the network not solve the problem ? And isn't this sort of solution already widely implemented for web content ?

Somebody fill me in ...

there are already standards for this...

[ophix]

... its called zipping, most webservers have it as an option to zip the data up as it streams to the client browser

i fail to see the need to have a "binary xml" file format when there are already facilities in place to compress text streams

Re:For Starters

[Omega1045]

Why? Microsoft has done a fairly good job promoting XML and SOAP XML Web Services. As long as they stick to the standards (yes, I know) I see no reason to keep them out.

IBM has actually tried to introduce some goofy stuff into the XML standards, like line breaks, etc, that should not be in a pure node-based system like XML. Why are not you picking on them in your comment?

As far as SOAP and XML Web Services (standardized protocols for XML RPC transactions) Microsoft was way ahead of the pack. And I rather enjoy using their rich set of .NET XML classes to talk to our Unix servers. It helps my company interop.

But ASCII is binary after all...

[MarkWPiper]

The fact is, ASCII is a binary format. It just happens to be a format that has become universally accepted. As the article says, there are certainly benefits to having ASCII-based XML: "The fact that XML is ordinary plain text that you can pull into Notepad... has turned out to be a boon, in practice," he said. "Any time you depart from that straight-and-narrow path, you risk loss of interoperability."

However, if anything, XML has shown us the power of well-structured information. XML has given the possibility of universal interoperability. Developments in XML-based technologies have led us to the point where we know enough now to create a standard for structured information that will last for several decades.

It's time that we had a new ASCII. That standard should be binary XML.

When I think of the time that has been wasted by every developer in the history of Computer Science, writing and rewriting basic parsing code, I shudder. Binary XML would produce a standard such that an efficient, universal data structure language would allow significant advances in what is technically possible with our data. For example: why is what we put on disk any different from what's in memory? Binary XML could erase this distinction.

A binary XML standard needs to become ubiquitous, so that just as Notepad can open any ASCII file today, SuperNotepad could open any file in existance, or look at any portion of your computer's memory, in an informative, structured manner. What's more, we have the technology to do this now.

Re:But ASCII is binary after all...

[MattRog]

Jesus Christ, no. The solution is simple:
(1) Have every PC OS contain a DBMS (this is not as difficult as you would think)
(2) Always keep your data in a DBMS
(3) Have said DBMS transfer the data via whatever method it would like. Chances are this would be some sort of compact, efficient binary method.

Re:But ASCII is binary after all...

[FangVT]

The fact is, ASCII is a binary format. It just happens to be a format that has become universally accepted. As the article says, there are certainly benefits to having ASCII-based XML: "The fact that XML is ordinary plain text that you can pull into Notepad... has turned out to be a boon, in practice," he said. "Any time you depart from that straight-and-narrow path, you risk loss of interoperability."

Not that anybody will care but...

XML is not ASCII. XML is Unicode. That's why Tim Bray said "plain text" not ASCII.

Because it was such a long hard road for ASCII to become the universal data format that it is for English text the creators of Unicode wisely made sure that there was backwards compatibility such that any valid ASCII texts (ones that do not include OS-specific, proprietary extensions in the range above 0x7F) are also valid Unicode texts when the encoding is UTF-8.

Re:Step 1 to getting binary XML

[Tsiangkun]

In my hands, bzip compresses better, but is somewhere between somewhat slower and orders of magnitude slower on my system, depending on the options used to invoke the command and the size of the file being compressed. gzip is fast, works on streams instead of blocks, and is available on nearly every system.

Why not re-examine http?

[digitalgimpus]

I think that's where the true problem lies. HTTP.

We need to look towards http 2.0. What I would want:

- pipelining that works, so that it could be enabled for use on any server that supports http 2.0
- gzip and 7zip support.
- All data is compressed by default (a few excludes such as .gz files, .zip files etc. since that would be pointless).
- Option to initiate persistant connection (remove the stateless protocol concept), via a http header on connect. This would allow for a whole new level for web applications via SOAP/XML.

There are tons of other things that could be enhanced for today's uses.

HTTP is the problem. Not XML

Ask Erik Naggum!

[notany]

Erik Naggum (SGML/XML-guru) who first proposed empty elements

<foo/>

form Re: Lisp syntax, what about resynchronization?

... so it had to come up, and one of the least
productive solutions, XML, won the day. I was there, at the conference
table where the first thoughts that became XML surfaced. a few months
earlier, I had proposed the need for a special marker for empty elements
-- and then retracted that proposal because it led to new problems -- but
guess what survived in XML!...

Attributes in XML are inherited from SGML and they were thingking markup for textual documents. When you want to represent data it being attribute or not is completely irrelevant.

Whether something is an attribute or element is _completely_ arbitrary.
It is based on some arbitrary choices in the design process that reveal
absolutely no inherent qualities. For purely pragmatic reasons, SGML
folks will use attributes for some things and elements for others because
their tools can deal with some things in attributes and some things in
elements. The faulty idea that attributes say something "about" the
element and sub-elements somehow constitute be their contents is the same
premature structuring that premature optimization of code suffers from.
The whole language is incredibly misdesigned in making that distinction.

Deep explanation: From:The horror that is XML

... XML, being the single suckiest syntactic invention in the history of
mankind, offers you several layers at which you can do exactly the same
thing very differently, in fact so differently that it takes effort to
see that they are even related.

<foo type="bar">zot</foo> actually defines three different views on the
same thing: Whather what you are really after is foo, bar, or zot,
depends on your application. XML is only a overly complex and otherwise
meaningless exercise in syntactic noise around the message you want to
send. Its notion of "structure" must be regarded as the same kind of
useless baggage that come with language that have been designed by people
who have completely failed to understand what syntax is all about. It is
therefore a mistake to try to shoe-horn things into the "structure" that
XML allows you to define.

In the abaove example, foo can be the application-level element, or it
can be the syntax-level element and bar the application-level element.
It is important to realize that SGML and XML offer a means to control
only the generic identifier (foo) and their nesting, but that it is often
important to use another attribute for the application. This was part of
the reason for #FIXED in the attribute default specification and the
purpose of omitting attributes from the actual tags. In my view, this is
probably the only actually useful role that attributes can play, but
there are other, much more elegant, ways to accomplish the same goal, but
not within the SGML framework. Now, whether you use one of the parts of
the markup, or use the contents of an element for your application is
another design choice. The markup may only be useful for validation
purposes, anyway.

Let me illustrate:

<if><condition>...</condition>
<then>...</then>
<else>...</else>
</if>

The XML now contains all the syntax information of the "host" language.
Many people think this is the _only_ granularity at which XML should be
used, and they try to enforce as much structure as possible, which

Anecdotal example

[plopez]

Had data to be delivered to client, dumped from a database. As flat files they were ~20mb in size as flat files. That bloated ~120mb after conversion to XML.

Client attempted to open in a DOM based application which I suspect used recursion to parse the data (easy to code, recursion). Needless to say it brought their server to its knees.

We switched to flat files shortly there after.

In my problem domain, where 20MB is a small data set, XML is useless. XML seems does not scale well at all (though using a SAX parser helps at times).

YMMV.

Re:You don't need to change XML itself

[TheRaven64]

Actually, you could compress XML by a significant amount by making one simple change to the language. Picture the following piece of XML:

<SomeTagName>some character data</SomeTagName>

According to the XML spec, the closing tag must close the nearest opening tag. So why does it have to include the opening tag's name? This is 100% redundant information, and is included in every XML tag with children or cdata. An obvious compression would be to replace this with:

<SomeTagName>some character data</>

I really don't know why this wasn't done from the outset (backwards compatibility with HTML, where tags often overlap - although they're not meant to - I suppose). Either allow tags to overlap (which allows some more interesting data structures to be easily encoded in XML) or make the name optional in the closing tag.

Microsoft XML

[Spy der Mann]

take an example on microsoft XML formats. Word, or the MSN messages format... they're _NOT_ xml. They're proprietary formats DISGUISED as XML.

If Microsoft doesn't respect text-only XML, what do you think will happen when^H^H^H^Hif binary XML is out?

DNS is binary; does that make it proprietary?

[Skapare]

DNS is binary; does that make it proprietary? Not at all. It is a published open standard in RFC 883 and later documents. Other examples include ASN.1/BER as used in SNMP. It's not whether it is binary or text that matters; it's whether it is openly documented and unencumbered by intellectual property claims (a separate issue some of XML has).

The decision of binary vs. text for a format should be the result of specific needs. XML is verbose. XML can be compressed for transmission purposes, but it still has to be uncompressed to its verbose form for parsing. If speed in parsing is necessary (it might be as I have noticed quite many XML based progams are rather slow), a binary format can have things like length prefixes and continuation tags, instead of having to detect and verify collection of characters whose position is unknown. A parser that does not recognize a given tag, or does not need to process it, in a binary format can simply skip it by jumping the specified number of bytes. Binary format is very optimal for machine processing.

The usual argument for a text format spans the range of permitting humans to create the content for most things directly in an editor like vi or emacs (no wars here, I listed my favorite last), or reading that content directly, such as to diagnose the real cause of misunderstood errors. XML is too utterly complex for human creation or interpretation to be effective on a direct basis. There may be some argument that it can still be effective for diagnostic purposes (I have in fact needed to do so many times). Given that it is the powerful tools of XML that are used as the basis for the benefit of XML and promoting it, then what does it really matter what format is underneath as long as it is open and unencumbered?.

A binary format for XML will absolutely not kill XML. DNS is obviously not dead (and you'll love it even more when IPv6 rolls into your network). What a binary format might do is weed out some of the weaker programmers who are sticking their fingers a bit too deep into the inner workings of some applications and tools.