Open Source XML Databases?

tarun asks: "I am creating the next version of my open source UDDI registry and decided to use an XML database backend - if I can find any good ones. The reason to make this choice was that I was impressed by oracle and db2's xml capabilities in my past lives. However, when I tried looking for an open-source alternative it seems there is nothing around except perhaps xindice -which clearly is less then perfect. I am looking for something that can work with more than one existing databases (I will ship my software with MySQL but if a large organization wants to deploy it, it should be able to do it using Oracle, DB2 or whatever they want to use) and xindice currently only works with Berkley-DB. Also, I am looking for something that can create database tables for me given an XML schema (I can tweak it later to create indexes, stored procedures etc) and given an XML document - write it to these tables. If it supports something standard like Xupdate or XQuery, that is even better."

"There are some other XML Database projects but either they have too few features or are not open-source. What is the XML-aware portion of the Slashdot community using? Have you ever run across such problems? Do you guys create your database schemas by pain-stackingly copying every element in every XML schema you have to handle to database tables and write huge amount of parsing/deparsing code both ways?"

Comments

Right here, baby

[PhysicsGenius]

What features doesn't eXist have that you need?

Re:Right here, baby

[tarun]

From eXist homepage,

XPath support XPath support is still preliminary. Some functions and numerical operators are missing and only abbreviated XPath syntax is supported. The parser has also some problems to recognize the full range of unicode characters. I have started to write a new XPath parser (using JavaCC instead of ANTLR) to overcome these limitations.

XUpdate The basic model has been designed to provide efficient, index based retrieval. As a drawback, eXist does currently not support direct manipulations of the DOM tree like node insertions or removals. A document always has to be deleted or updated as a whole.

This is clearly a major restriction for applications which need to directly manipulate the DOM tree. Such applications have to create a new document (as XSLT does) and insert this into the DB after all transformations are done. Documents should be kept small to easily reinsert them whenever they change.

DOM manipulation methods and XUpdate are planned for one of the next releases

Use an XMLJavaDB toolkit

[jefflinwood]

Try using an XML to Java toolkit with a Java object-relational mapper. XML Databases aren't really all they are cracked up to be. Really quickly, projects you should look at are:

• Castor - http://castor.exolab.org/
• Hibernate - http://sourceforge.net/projects/hibernate/
• Torque - http://jakarta.apache.org/turbine/torque/
• ObjectRelationalBridge - http://jakarta.apache.org/ojb/

Also you might want to check out Brett Mclaughlin's O'Reilly book on Java XML binding's, or the Wrox Professional Java XML book.

Umm..no

[quinto2000]

Do you guys create your database schemas by pain-stackingly copying every element in every XML schema you have to handle to database tables and write huge amount of parsing/deparsing code both ways?"

I use Dia, Agata, and Dia2SQL. (There are several variants of Dia2SQL). What exactly are the benefits of XML? I'm new to databases, but this way seems to me more efficient. At what level of complexity does the XML schema become so useful?

Re:Why XML?

[DevilM]

Sure! To use XML with an RDBMS you have to do one of two things. First, map the XML to a relational schema. It is well understood that doing this has two main problems. The first problem is the resulting schema. To create a schema to support heirarchical data results in a complex and ugly schema. The second problem is a spatial one. To retrive a given XML document, the database must pull data from a variety of pages. This results in poor performance as the database has no context to store the different bits of an XML document together on the disk. FYI, this one of the main reasons for the creation of OO databases.

The second way of handling XML in an RDBMS is to store the document as a CLOB. Storing it as a CLOB has the advantage of solving the two above issues, but introduces one of its own; You can't query the data that is represented by the CLOB because it is all stored in a single column. This means you have to extract the document from the CLOB and parse it before being able to use any of the data. Some databases now have built in XML parsers so you can do this from stored procedures and combine the XML document with tabular data, but the performance sucks.

I do cover why you would want to use an XML database and how to use Xindice in an article I wrote for DevX that can be found here.

From the sound of it...

[broody]

Your looking in the wrong place.

It sounds like what your looking for is really an object-relational wrapper rather than an XML based RDBMS. Take a look at OJB and see if it is not really what you are seeking.