Slashdot Mirror
Expanding the use of XML in Linux?
elemur asks: "I was wondering if there are any projects to expand the use of XML in Linux? There are alot of areas where XML could be more easily and consistently used than continuing making more and stranger configuration files. Many configs could probably fall under a generalized standard application config DTD, and applications that needed something more targeted could supply their own. Some sort of DTD repository could be setup on the machine to handle this. Then, apps just need to use libxml (or whatever it would be called) to handle the reading and parsing. It would seem to make things much more consistent. Has anybody looked into this sort of thing?" It's a good thought. And a standardized configuration file format might be the thing to reduce some of the complexity most folks find in Linux. What do you all think about the capabilities of XML?
Update: 09/29 04:03 by C : Screwtape submitted this tidbit "I just saw this on MozillaZine and I'm quite impressed. Somebody has taken the XML parser from Mozilla, and written software that makes it work like an xterm - but with extra features. For example, you can write a replacement for ls where all the filenames are hyperlinks to the actual files. The site is here. "
XML means an infinite set of yet undefined HTML like languages with no semantics whatsoever. With a proper DTD you have one language from this XML set of languages. You still have no semanctics, except what you write down to the DTD as human language comments.
With an a style sheet, either XSL or CSS, you can define a semantic transformation where a syntactic element of any XML family language, like a tag , is given a semantically meaningful visual presentation. Instead of using XML you could as well use plain HTML 4.0 with CSSS and tags like or to get the same result. XML gives you no benefit at this level here.
There exists no mechanism for specifying semantics for a program which intends to use your DTD-defined language as the form of its own configuration files, or whatever purpose. Implementing those semantics as as hard manual works as with any other non-XML syntax.
XML is a hoax. It is actually nothing but a free licence to invent an infinite number of your own "" and just pray that somebody will agree with you on their meaning.
Of course the world can negotiate standard XML DTDs with standard semantics, and standard software to interpret them. But agreeing on them and getting the semantics implemented is the exactly same problem as would be regardless of their syntactic family ties. The computing world is full of different formal languages, and the their syntactic variance is not a big deal. Finding a good syntax for an application domain is important, and implementing the semantics is the challenge. Just _parsing_ simple specialiced syntaxes for configuration files or application specific data is trivial with modern tools anyway.
I could not care less, if the data I have to parse is comma separated fields, name=value pairs or XML compliant value.
XML is vapor, a hoax, but XSL is the most horrible thing I have ever seen on my career. I won't go to details here.
Anssi Porttikivi
app@iki.fi
currently mostly a html/http/Tcl/Oval/database programmer
While I agree this can definitely be another good way to use XML; I'm not sure everyone will be willing to abandon the ASCII config file formats they have been using for a very long time, and move to an XML-based configuration registry. But something like this has to be done sooner or later...
Why? Using XML for configuration files doesn't actually buy you anything. All XML is is a way of concisely describing the format of a file -- but the data (and more importantly the data's semantics) in each configuration file will vary between programs just as much as they do now.
If you proposed that all configuration files be written as Lisp s-expressions, people would look at you funny because they could easily see that that doesn't magically win -- but mumbling the phrase "XML" seems to escape those filters, even though it's just s-exps with typed parentheses.
[I'm not kidding about the s-exp thing, either. I started to write a little system using XML and XSL to transform some of my docs into HTML, flat text, and LaTeX forms. But when I realized (thanks to Erik Naggum) that the XML document was just a way of serializing a tree structure I changed tacks. Instead, I stored my docs as Lisp s-exps and then was able to use Common Lisp instead of XSL to write the tree-walker and was done in a quarter of the time the other way would have taken.]
In any event, the thought that might bring real benefit is the idea that you can build a hierarchical configuration structure where apps can acquire config data from the containing configuration classes. (It shouldn't be a registry tree, though, because an application might rightfully belong to several disjoint classes -- you'd want a registration directed acyclic graph.) Defining XML DTDs would be a convenient (but not essential) way of creating a lingua franca.
In essence, configuration classes would correspond to OO classes, and the configuration acquisition tree would be an inheritance graph. This could be huge win if the graph were well-designed, because each piece of config data would be kept in one place, and program configurations would automatically adjust when that unique datum was changed. However, if the tree were poorly-designed then you would lose big in the same ways that bad OO designs suck hard -- configurations would be very brittle with lots of interapplication dependencies and needed information would be scattered all over the place.
My personal belief is that the registry approach would tend towards the "huge lose" case -- incrementally designing a good OO architecture requires aggressive refactoring, and refactoring configuration information among dozens of software projects that aren't even necessarily aware of each other would be an interesting problem in change management.
So I see having to manually adjust individual configuration files is the price we pay for letting each project develop independently of the others.
Are there some that do reliable write, e.g. with file locking, backups, and automated backout if it encounters errors?
If you're not part of the solution, you're part of the precipitate.
However. It is not all fine and dandy.
The "configuration problem" has not one issue, but several:
XML represents Yet Another Format; it is of value if it pushes out some of the existing formats. If it merely augments the population with another, there is no win here.
Result: Ambiguous. XML might provide value.
The issue here is that you need to ensure that the configuration is written out correctly.
This may require writing out the new config to a new file, validating that it is readable and correct. (Oops, made a mistake updating /etc/inet.d. Now the system won't reboot...)
There is merit to having a "database form" ala IronDoc where the physical representation is a database system, which provides a somewhat different persistence model than the typical text file.
(Before people start proposing that I be shot, I tend to favor the notion of, if using a binary format, synchronizing it carefully with a text format.)
The merit of a "databased" scheme, which should provide a separate database for each facility, is that updates can be implemented "instantly" without needing to rewrite a whole file, and without a need to parse the file. Note that even in a situation where XML is used as an interchange format, there is still merit to storing the "tree" in database form. David McCusker, author of IronDoc and architect of the (regrettably failed) "Bento" database system that was part of OpenDoc, suggests this very use for IronDoc.
For those that feel religious about using text files, a system like libPropList still has merit over the "let's do something with XML" idea since it has, already debugged, the locking, parsing, and config-file-rewriting code that let's use XML, it's k001 doesn't inherently provide.
In short, deciding to use XML merely establishes a format; it does not resolve that:
Michael Stonebraker (of fame with such developments as Ingres and Postgres) has most recently founded a company called Cohera based on the Mariposa Distributed Database Management System. This tool allows many databases to work together to process queries.
The "obvious" implication of this with this thread is that a valuable thing to be able to do is to join together many "databases" that are configuration repositories, and provide a central way of getting at the data.
The critical thing that is necessary is for configuration repositories to provide some sort of "metadata" so that they, in effect, publicize their existence.
A "federation" tool like Linuxconf, Ganymede, or such, can then be used to join together the metadata and manage it all together.
Unlike the situation with the infamous Windows Registry, this doesn't force all the configuration data into one fragile binary DB; it allows the data to stay wherever it was concluded that it should reside.
The critical factor here is not that data files all have a common format; it is that there be some way of translating their data into a common format.
XML has a lot to offer here in terms of providing a central "presentation" format. It could offer more if tools were available to make this a two-way street, where updates done to the central XML could be pushed back to the individual configuration data repositories.
However. If someone writes some integration code to (say) connect Linuxconf to libPropList so that it could directly manipulate libPropList files, that would also represent a movement in the right direction.
Conclusion: XML may have value to offer in confederating config information.
That has to come along with a whole lot of coding effort to build robust configuration data repositories that may or may not use XML.
If you're not part of the solution, you're part of the precipitate.
Like another poster said, so long as they have a basic grasp of the XML libraries that exist, it's just a matter of calling the library's parse and write functions to read and create your own application-specific XML files.
Though learning XML would most certainly be a huge advantage, but it's hardly necessary. It's pretty easy for a non-XML-savvy person to edit an XML configuration file. It's not too difficult to figure out.
A libxml already exists. In fact, most languages already have some sort of library support for XML.
The support is there, the libraries are there. It's just that there's no "config file standard" yet and developers haven't really looked into it.
One project that will eventually come out of the Jabber IM project is JNX, which would be expanding certain *nix functionality with XML data storage/routing. This actually goes one step further, and allows any program to transfer data to any other, by sending XML 'streams' of data. With the right transports, one could extend it further by having a 'Configuration Repository' for the system, which could actually store configuration data in something like an XML flat file, or even a MySQL database. Imagine 100 *nix desktop systems, all controlling their configuration via XML and a centralized database. This is some of the things that we will be looking at within the next year or so. www.jabber.org
-- I'm the root of all that's evil, but you can call me cookie..
The game project I'm working on uses XML for its network protocol, its database stuff, and (some) configuration scripts. We've explored a lot of the available parsers and run into some issues but no major sticking points. The two key issues:
Streaming - To use XML across the web it is nice to be able to stream XML packets (e.g., object definitions) and collect them client side and make use of them in real time. None of the current parsers provide this adequately, although several are working on it. We had to develop our own library for streaming this stuff (libAtlasWF). It's focused mostly on real time 3D information transferral, customizable by receiver to filter out unneeded information. It's generic enough to be useful for a wide range of applications, though we're using it for game systems.
Binary - A major requirement for games (and other applications) is binary formats for performance reasons. This was a major argument against XML until we realized that the XML tags (and lots of the data) could be rendered in binary simply by replacing tags with particular bytecodes and such. Probably not as compactly efficient as a custom binary code, but extraordinarily flexible (e.g., develop in ASCII XML, then just flip a switch to go to performance-oriented binary, and redefine binary tags as needed).
We're calling this real-time, 3D, binary-ready protocol "Atlas". We'd love your input (and help) in bringing it to any application that could use it. We invision client applications that understand XML-Atlas and can communicate with any server talking in this language, and a variety of specialized servers doing the same.
Here's some links to information about Atlas: Atlas Version 1.0, and WF Protocols
Bryce
It seems the poster wants XML introduced into all applications - not really the Kernel. I don't know many places where the Kernel would benefit from XML - except for the one configuration file the code itself wouldn't need XML.
I think what is being addressed is more an application issue. That would need to be addressed to many different vendors simultaneously. For instance: Apache config files, WuFtpd/ProFtd/etc files, Gnome/KDE config files...
I could see a lot of reusability for a config file parser for application development. But it would seem like the development tools would need this XML ability and not just everyone using XML. But I imagine that C/C++, Perl, Python, Java, etc already have XML parsers/creators. So really people are waiting for developers to embrace XML.
Does anyone now embrace XML? What are the advantages of XML over other config file parsers? Are there other standardized config file parsers? I know I've written my share of wheels in different languages for parsing config files.
Joseph Elwell.
IMHO, the *real* win of XML is not in replacing plain-text configuration files, but rather in replacing binary file formats and simple databases. One example would be word-processor file formats, which are usually a poorly documented, poorly structured binary mess. XML makes sense there. Another example is the Windows registry.
But XML is a significant *disadvantage* for the plain-text configuration files that dominate the Unix world. Generally, those plain-text files have a comment mechanism to clearly explain what needs to be edited. Adding XML will just add a bunch of unnecessary tags that will make it difficult to hand-edit configuration files - and the ability to easily hand-edit the human-readable configuration files is one of the most powerful advantages of the Unix Philosophy.
What i would like to see in the Unix/Linux world is a GUI that is capable of managing the entire OS via simple graphical tools, but doesn't *prevent* hand-editing of configurations. I don't see XML as a significant boon in that regard.
---
Maybe that's just the price you pay for the chains that you refuse.
Hand me that airplane glue and I'll tell you another story.
Glade does something like this. It will produce C, C++, Ada source code or an XML file describing the GTK application. With python you parse the XML and then use a build tool to build the GTK objects as they are laid out in the XML by Glade. You just have to attach the events to code that you write.
:-).
While I see a great possibility here of having a way in which the author gives the user GUI elements and a set layout that the end user can change, the implementation suffers from the same problems that almost all XMl suffers from.
First, very few people understand the full scope of XML and all the adjoining technologies. The only way I understand it is putting it into the same perspective as SGML, but that doesn't cover many of the other parts. I guess most of us are waiting for an O'Reilly book on it.
Second, part of XMLs power is the way it can be used for almost anything. Part of the reason of it not being universally excepted is the the way it can be used for almost anything. Has anyone seen XML used the same way twice? From what I can see this has resulted in a lack of tools to deal with XML from beginning to end. Yes, you can merge a lot of tools to do some really amazing things, but it takes a lot of time and mental effort to come up with these chains of tools (much more so then other chains of tools).
Third, the resource consumption for most XML based projects is high. High enough that it will limit the viable uses of XML in the short term. Building a large GUI from XML may increase the start time of memory consumption beyond where it is usable for some applications. Netscape already loads slowly, if that time were doubled I'd be looking to use lynx for most everything (half
Yes indeed a great idea.
.blablabla files in your homedirectory.
Using XML would allow us to separate the configuration files from the configuration applications and those could be separated from the applications that are being configured.
If some effort is made, a generic configuration program (or more than one think expert and novice user targeted software here) could be made that would work with any application that needs to be configurated. I imagine application creators could write an xml file describing all the properties in their application, with documentation included as meta tags. A generic configuration application would read the resource file and present a nice GUI for it. The entered values could be stored in another xml file. The administration possibilities are also way cool. Imagine having permissions on certain properties, default properties and user specific properties managed automatically.
The advantages are clear:
- platform independent, possibly even program independent (imagine having a set of default parameters that any app can use) configuration
- fine grained control on configuration for administrators
- user friendlier
- no more obscure textfiles in obscure places on your hardrive, no more
- advanced tools for configuring any application
Plain ASCII files with fixed syntax are obsolete. The revolutionary thing about XML is not it's syntax, not the way it stores stuff but the fact that you can define extensible syntax structures with it and that you can use generic software to read and write to it.
In fact I have an idea that may be shocking for hardcore UNIX guys (stop reading if you want to sleep tonight): why not replace the syntax of programming languages like C and Java with XML equivalents. The reason I would want to this is that XML is extensible so you could add meta tags to it. Imagine having author tags, documentation tags (in many flavors), etc in your code. The possibilities are endless. There may be some problems with the grammar, for instance I know that c++ syntax is more complex than Java syntax (one of the reason why you don't find so much tools like javadoc for c++).
I'd love to see your reactions on these ideas.
Jilles
A common dtd is good if you want to exchange the xml data that it describes among different applications. In the case of a document or spreadsheet, this is clearly a good thing. However, I don't see people developing a pressing need to exchange their configuration files between different applications. What other application could possibly have a use for a sendmail configuration other than sendmail?
If each application has its own configuration dtd, then editors can use that dtd to help the user write a valid config file. It can specify required tags, optional tags, and describe the structure of the file. Rather than using generic tags like <item id="username">Ethan</item> you can have <username>Ethan</username>, and this way the dtd can require the username tag, so you don't forget it. A common configuration dtd would be far too generic to be of much use in this.
XML in place of the current config files would have some advantages. For one thing, it would allow use of a single high-performance parser to parse the files. The days of writing/copying a config file parser for every application would be over. Perhaps we could create a shared library to do this. (If there is any interest in a libxml.so, please let me know. It sounds like a cool project.)
It might reduce version control issues in some cases since new/unknown tags in XML can be ignored (much like unknown HTML tags are ignored). However, a well-written config file parser would do this already.
It would probably speed up the process of creating GUI front-end configurators, since the parser/generators could be reused. An advanced user without a GUI configurator is like a fish without a bicycle, but it would be helpful to newbies and regular desktop users. The "Linux is hard to use" argument would start to go away.
There are some big drawbacks, though. The first is that tons of applications would have to be revised in order to read XML config files. In an open-source world, this means a long painful process where some developers switch to XML immediately and others wait a while. Then there is the pain of converting your customized http.conf/fstab/.profile/etc (bad geek pun intentional) files to XML.
Also, there is the fact that most of the cool tools in Linux are really designed for all of the Unix world. Realistically, the Linux environment can't just switch over to XML config files unless the entire Unix community does.
Maybe future apps should use XML as their config file format. I don't see our well-worn existing tools making a switch anytime soon, however.
Just my 2 cents.
Save the whales. Feed the hungry. Free the mallocs.
But the biggest win will come from minimising the proliferation of DTDs. If the community can co-operate on the development of common DTDs then the exchange of data between software agents developed by different projects will be hugely easier. By all means have diffrent projects - both KDE and Gnome have, in my opinion, benefited from the competition between the two - but if that competition develops the sort of bitterness which reduces communiction and co-operation we all lose.
I would strongly urge anyone who is developing a new software agent - whether it's a user-level application or a new daemon - which either stores data or exchanges data with other agents to seriously consider XML as a format, but more importantly should look at the DTDs that already exist to see if any will fit, and should communicate with anyone else working on related tools.
If anyone wants to look at the XML tutorial I gave at INET99 it's here
I'm old enough to remember when discussions on Slashdot were well informed.
The more you stray from line-based text files that you can easily call things like awk and grep on, the more you'll alienate some people. I don't believe there's a version of awk or grep that handle XML-based records. This is pretty important to tool-minded people.
While I agree this can definitely be another good way to use XML; I'm not sure everyone will be willing to abandon the ASCII config file formats they have been using for a very long time, and move to an XML-based configuration registry. But something like this has to be done sooner or later.
People are moving and creating DTDs on their own which has the potential to cause a huge fragmentation on the XML arena; so before someone tries to design a configuration DTD, there must be some concerted effort to start a group within the Linux community that will work on this and other relevant XML DTDs.
Just my 2 cents.
Zigbee Central: A Zigbee weblog
I started out using XML for simple configuration files on a Java software project. Once we started to use it, we realised that it's extremely powerful and soon started finding many uses for it.
Expect to see XML cropping up everywhere soon. Microsoft (boo, hiss) is going to be using it for document exchange (XML is very good at this) in their Office products. There are rumours that M$ is already bastardising XML, rather than stick to the standards (now where have we heard that before?).
XML initially looks daunting, but really isn't too difficult to learn. There are some standard API's being developed (SAX and DOM), at least in Java. XML and Java work very well together. I haven't used XML with Linux, so can't comment on available libraries, if indeed there are any yet.
I certainly would encourage developers to look into using XML. It certainly beats writing your own parsers and you'll soon appreciate its flexibility. HH
Yellow tigers crouched in jungles in her dark eyes.
She's just dressing, goodbye windows, tired starlings.
The LinuXML Project is "devoted to changing the UNIX de facto standard for inter-process communication (IPC) and storage from line-based ASCII records to XML."
Check out the
There is indeed a version of grep for XML-based records. It's called sgrep (structured grep) and "is a tool for searching and indexing text, SGML, XML and HTML files and filtering text streams using structural criteria." It is based on the concept of regions, i.e. nonempty text substrings that are typically occurrences of constant stsrings, SGML tags, or meaningful text elements recognizable via delimiting strings or the built-in SGML, XML and SGML parser.
Check out the