Slashdot Mirror
ESR On XML-RPC
daveuserland writes "Eric S. Raymond has discovered our XML-RPC project, going strong since early 1998. I don't think XML-RPC has ever been discussed on Slashdot. It's one of the most ambitious and successful open source projects, but not from the usual suspects. Raymond says: "XML-RPC is very much in the Unix spirit. It's deliberately minimalist but nevertheless quite powerful, offering a way for the vast majority of RPC applications that can get by on passing around boolean/integer/float/string datatypes to do their thing in a way that is lightweight and easy to to understand and monitor. This simple type ontology acts as a valuable check on interface complexity.""
There are several XML compression formats out there already; it's too bad people haven't decided on one, though.
I heard him talk a year and a half ago or so. He basically gave the same line he's used since 1997, with examples of free software projects that have succeeded and theoretical assertions that open development provides a better way to run a business.
There was absolutely no acknowledgement that he's been pushing this line for years without a single prominent success, and that the project he was so eager to associate himself with, Mozilla, has nailed Netscape's coffin shut. People flame CmdrTaco all the time but at least his site addresses the failures of the open source model. Has Eric Raymond been seen anywhere near Mozilla since people started wondering when it was going to produce?
Naw, he'd rather just keep talking about Linux and Doom and his *gag* "tribe"....
What the heck are you talking about?
Eric's stuff is everywhere. Take a look at the Acknowledgements file for Emacs, for example. Every time you fire up a debugger in Emacs, or edit a Makefile, or shoot off a cvs command you have Eric to thank. In fact, he apparently has more Lisp code in Emacs repositories than anyone besides RMS. He also wrote the keeper software for FTP supersite sunsite. He is currently working on a new build structure for the Linux kernel, and apparently some stuff for the PNG picture format. He is also active on the Python development lists (where he is pushing for wxPython over Tkinter).
He also is a major contributor to Nethack. Which means he has been responsible for hours of my lost productivity.
And that's just the stuff that I can think of off the top of my head, and it doesn't include stuff like the fact that he is maintainer of the Terminfo files, the Jargon file, and a whole pile of other stuff (if you haven't read the documentation for Intercal then you really are missing out). Oh, and let's not forget fetchmail. It seems he wrote an entire book on that particular experiment. If you are really interested in reading about the code he has contributed go here. Most of the interesting bits are at the bottom.
I don't agree with all of ESR's views either, but saying that ESR doesn't deliver is completely untrue. What's more, the software that he has written is quite well done. The documentation, in particular, is usually quite astoundingly good.
The fact of the matter is that ESR is probably twice as talented a hacker as both you and I put together.
Eric Raymond stopped being relevant for me when he stole the Jargon File and published it for his own profit without compensating either those who contributed or redirecting a nickel to any open-source organizations.
After that, anything he has to say about communal efforts is the height of hypocrisy.
--Blair
People interested in XML-RPC might be interested in checking it out using KDE. Since KDE 2.0 a DCOP XMLRPC bridge has been included allowing easy access to a wide range of the desktops APIs.
Rich.
How about this: /etc/termcap
$less
######## TERMINAL TYPE DESCRIPTIONS SOURCE FILE
#
# Version 11.0.1
# $Date: 2000/03/02 15:51:11 $
# termcap syntax
#
# Eric S. Raymond (current maintainer)
# John Kunze, Berkeley
# Craig Leres, Berkeley
#
# Please e-mail changes to terminfo@thyrsus.com; the old termcap@berkeley.edu
# address is no longer valid. The latest version can always be found at
# .
You'll see it from time to time, Eric did this little bit, Eric did this little bit. He shows up on linux-kernel from time to time. It adds up. Have you looked at termcap recently? JHC, I wouldn't touch that nonsense with a 10 foot pole. But he does. Same goes for his rewrite of the kernel configuration file system. Nutso stuff that is way beyond either of our abilities (admit it, you lamer), and he's working on it right now. I may not be the biggest ESR fan on the planet, but he has made many positive contributions, and his heart is in the right place.
I'm a software architect for a major bank, where I design and implement the primary app used by all bankers to open new accounts. It's a web-based app written almost entirely in Java and Perl. A pretty monstrous app, under heavy load and very high profile. We decided to use XML-RPC for interprocess communication between our JVMs and Perl CGI's. We found the Java-based XML-RPC server (at least the one we looked at) to be in need of serious help. Exception handling was poor, and there were some significant performance issues. The perl client had similar problems. However, after a couple of months of working with it, we were able to tune it enough to support our demands, and our app is demanding (20 to 100 heavy-weight hits per second, with at least 1 XML-RPC transaction per hit). XML-RPC has greatly simplified RPC between our JVM and Perl CGIs, and has made complex interactions a breeze to code. Once the exception-handling and performance issues were addressed, we realized that we had an unusually effective piece of RPC software on our hands. We no longer fear Java-to-Perl RPC. I highly recommend it. We hope to release our fixes out to the public sometime soon.
We use XMLRPC in one of our enterprise-(ish) applications for webbased API calls. Works very well. As for SOAP, SOAP more or less does the same thing; except it's more of an "enterprise" solution. Microsoft is one of the big bangers behind SOAP, and numerous other large companies such as IBM (and sun, sorta).
The problem with SOAP, is it's not really done, and it doesn't really work. So we decided to go with XMLRPC (because it works), and eventually replace it with SOAP once the standard stabilizes and there's actual functional implementations.
One example of XMLRPC not being quite ready for prime-time, is it's lack of Null support. As the article described XMLRPC allows the passing of primitives and some complex datastructures nativly (strings, lists, hashes, ints), but it does not support passing nulls. Anybody whose tried to use XMLRPC will spend about an hour scratching their head when they program an XMLRPC call which (in java or C) has a return type of void (such as a setter for a variable). XMLRPC will choke because it can't describe a Null. So you end up having to return blank strings to work around the problem.
You know some XMLRPC specifications guy just slapped his head when he read this saying, "dammit, I knew I forgot something!".
This is my experience with XML-RPC:
I work for a company (Technisys) which have created several years ago an RPC tool called "tmgen". This tool is built as a layer on top of rpcgen, adding session cookie handling, SSL support, a stateless server, handling of enumerated values with associated long and short descriptions, and many other thing. It's in fact, an application server built on top of RPC.
This baby have been running for many years in the most important banks and credit cards companies here in Argentina (yes, you know the brands, but I'm not sure I can tell you which ones =) ).
The "tmgen" tool reads a ".def" file that defines the datatyes, and ".trn" files which have the code of the different "transactions". Having read those files, it automatically generates the server source (including the rpcgen input source).
I was asked to make it possible for the clients to be programmed in the Java language. I evaluated several possibilities, one of them using a Java client for RPC. This required us to go for a proprietary solution, besides, being in control of both sides it looked silly to be tied to a protocol. Another possibility would have been to modify tmgen to create an RMI server. But the best was to create an XML server (IMO). I then evaluated SOAP and XML-RPC. SOAP seemed very nice, but XML-RPC was *direct* mapping of the semantics and types of our existing solution. The benefits of SOAP were a drawback in this case, we just wanted to have strings, structs, ints and floats.
So, now it's working. It takes a definition of the structs, the method and which parameters they get, and it creates code (using the gnome-xml library (which I recommend). The automatically generated code works as a standalone inetd webserver which reads an XML-RPC query from the net, parses the query, loads it in the generated C structures, run the "transaction", and creates a response from the reply structures. The final result was that all those old C, RPC-only, programs started to have an XML interface.
I added the Helma RPC-XML client and voila, we had a Java client. So I must say that my experience in this legacy system with XML-RPC was great.
Talking about new systems, I think that XML-RPC does the wrong thing, by defining markup for the types instead of marking the semantic of the data.
I hope I haven't bored anyone with my experience.
I am a big fan of distributed computing, heck I wrote an article about it on K5, and have always wondered what the XML-RPC payoff is.
From what I can tell, XML-RPC is a way to replace the binary protocols that current distributed systems use (e.g. Corba's IIOP or DCOM's ORPC) with an XML based one. So when an object needs perform a remote method call, instead of just sending it's arguments in a compact efficient binary packet, it builds an XML string which has to be parsed on the other end. So with XML RPC, remote method calls now need the addition of an XML parser to their current bag of tricks.
On the surface it seems that this makes it easier to perform distributed computing since any shmuck can use an XML parser and call a few functions. But it means that an extra layer of abstraction has been added to operations that should be performed rather quickly for the dubious benefit of compatibility across platforms (which is yet to be realized) which seems to be more jumping on the XML hype bandwagon than reality. My biggest issue is that for XML-RPC to support things that are the biggest issues of distributed computing (e.g. keeping track of state) would add so much bloat to the XML parsing, string building, etc process for making a remote call as to make it unfeasible.
Anyone see any errors in this thinking?
Finagle's First Law
So, despite the community's protest that people are needed to do things other than write code, the one criterion that you express is that what ESR wrote ISN'T HARD ENOUGH. And then you get modded up in a reply, suggesting that the community agrees with you. You know, the most damning writers at ZD have figured it out. The open source community ensures that end users are systematically locked out. The community has a genuine contempt, not only for "users", but also for the simple. And so it produces sendmail -- and a whole set of similar interfaces that are so complex to configure that you need more 3" thick O'Reilly books. ESR is not given some respect for having coded a utility that you don't care about. ESR is given respect because he helped a gigantic and very powerful set of users to understand the revolutionary and important nature of open source. If open source fails at its alleged "mission", we will be running Microsoft (TM) TCP-IP.NET by 2005 and the state of freedom across the world will be endangered. (OK, allow me a little melodrama.) But ESR will be responsible for holding off that state of affairs for a few years. Do you believe me? Do you care?
It is good to hear ESR's views on this subject too - he is an articulate and interesting writer.
--
Clarity does not require the absence of impurities,
/* And you'll never guess what the dog had */
/* in its mouth... */
--Larry Wall in stab.c from perl
In any event, it seems to me that his role as a community mouthpiece has been largely taken over by the CEOs of the Linux companies, who I see on TV all the time (TechTV and the various financial networks).
I know this is marginally off-topic, but I need to know if I'm alone amongst the serious /. inhabitants in not wanting to hear from this guy again (I already know what the trolls think), or am I just being too hard on a guy who still has something worthwhile to say despite letting the dollar-signs go to his head at one point?
----
Every year during my review, I just pray the words "slashdot.org" aren't mentioned.
As I understand it, SOAP evolved from XML-RPC. See: http://www.develop.com/soap/soapfaq.htm#22 for a hint at history and ancestry.
I'm of a mixed mind about both SOAP and XML-RPC. XML-RPC includes explict typing of passed values, and SOAP does not. Point to XML-RPC. XML-RPC is substantially simpler than SOAP. Point to XML-RPC.
I bet SOAP wins, because it's heinously complicated. Big companies and big devel shops love complicated: it raises the cost of entry into a market where the complicated standard prevails.
At first glance, it seems like CORBA is the better technology. IIOP is at least a little bit lighter on the network. An application built on a capable ORB can be switched to different local and remote transports with little or no modification. That can get useful.
But, I think that some bridge should exist between GIOP and XML RPC and it's http relatives. GIOP over XML RPC? An IIOP to (XML)IOP bridge daemon?
I've had situations where I had to contact MS SQL Server 7 from Linux. (Way to many) Easysoft gets the job done, but not in the way I want. I'd be happier if some CORBA/XMLRPC bridge existed.
When I need to grab and process data I could write a simple cgi on a webserver running on NT. It would parse the XMLRPC REQUEST and grab a bunch of data for me and spit it out into XMLRPC. As needed I could rewrite these things as ISAPI dlls, NT Apache/mod_perl programs or even ASP scripts, depending on performance needs.
Ultimately I could have written an NT network service that was dedicated to just my data, and used plain IIOP. My app on linux would need no modifications.
Anothe way to look at the benefits is that you have CORBA when you want network transparency and XML RPC when you are willing to fiddle with the bits but are too lazy to implement IIOP and friends yourself.
Now I will say SOAP. There I said it. SOAP SOAP SOAP. I got chewed out one time on the ORBit mailing list. The guy accused me of being obsessed with buzzwords but I think it's really because I said SOAP.
Of course I realise this is slashdot, and pandering to the marketroids is tantamount to heresy, but it needs to be said.
For this XML-RPC effort to succeed, it needs to be implemented as a kernel-loadable module, and be included in the default Linux kernal.
Then and only then will Linux be fully riding the XML bandwagon
XML-RPC predates SOAP by a few years, and has more implementations, and has demostrated interop. SOAP, which I am also a co-author and supporter of, has not yet reached substantial interop, and this is going to be a bit difficult, but we'll get there. You're right that SOAP has more support from large companies, and that's great, but as an independent developer who doesn't want to be controlled by the large companies, I think XML-RPC provides a valuable insurance policy. I'm very happy to get ESR's endorsement of XML-RPC.
Saw this over on LinuxToday.
Zoom over to Google and put in Eric Raymond (or just check this link out). Notice the paid hotlink...
Linux - Because Mommy taught me to Share.
XML was designed to transfer data between programs and across platforms. The cross-platform clarity and reliability is great, but an XML file is really bloated relative to a specific file format, which is hardly conducive to transfering files.
.xmlc or something. Then include the xml parser and compressor/decompressor code into standard libraries for all platforms.
Those folks ought to incorporate a universal, cross-platform standard for compressing XML files into something a little smaller and more portable. Maybe call them
cryptochrome
---If you can't trust a nerd, who can you trust?
The biggest technical security risk in most environments is that Bad Guys can hand programs input that doesn't have the properties that the programs' authors expected, and in networked environments (such as an RPC) there are Bad Guys all over the Solar System who have the means of handing input to your programs (both client and server), and RPC mechanisms have been a popular target for crackers, not that networking in general isn't a target-rich environment. Anybody who provides a new general RPC mechanism and doesn't make the default behavior to be "secure" at least to the extent of safety-checking input is a danger to the community. I don't see any evidence that XML-RPC addresses this problem.
The great thing about XML, besides flexibility and popularity, is that it provides a mechanism for defining valid input, through the use of Document Type Definitions, so parsers can determine whether input is both well-formed and valid. That doesn't mean that everybody writing a program that uses your XML DTD will write it safely, or even if they use a parser that they'll refrain from mismanaging memory, but at least they'll all have a consistent spec for what they need to support, and if they've got an xml parser that they're using, it'll have something to work from.
The XML-RPC Spec doesn't provide a DTD, and provides examples that don't use it. They don't say what resources a conforming call might use, and is pretty sketchy about responses to invalid calls, as well as about what to do with non-existent or invalid responses. Of course, it does help to use an XML parser that itself responds safely to maliciously designed inputs, but that's a job for open-source commonly used code, just as any XML-RPC toolkit built using it is. But with no DTD, it's harder to decide if requests are malformed or invalid, so the author's program needs to do that work itself, which means that 25% of the popular packages won't even bother, because the deadline for the next release is coming, or the overhead of the XML parser makes the dancing pigs GUI skin too slow. And it only takes one widespread dangerous application to carry the next internet worm - you don't want QuakeSter OpenMusic&WeaponsAuction to start DDOSing metallica.com because a buffer overflow in the PayLars feature lets bad guys in (and the distributed database that replaces Napster makes it easy to find people running that client.)
Please, there are so many loose ends on this stuff - give it a full-scale spec, and build enough documentation and API tools that encourage safe use and strongly resist unsafe use. (Of course, that's harder than it sounds - the obvious way to protect against buffer overflows in an environment where all the input is character strings is for the API to (safely, with checking) malloc (or language-appropriately-create-objects) any input objects, so the malicious client can probably find some good resource starvation attack, but that's at least a Denial Of Service rather than an attack that lets the bad guy hand the server some code to run.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
We use XML-RPC on our Python-powered intranet to grab headlines from O'Reilly's free Meerkat wire service via Secret Lab's free xmlrpclib module. Dave Warner wrote an excellent article on XML-RPC and Python here, using Meerkat as an example of a real XML-RPC server.
If Python is not your thing, there are XML-RPC libraries for PHP and Perl. Rael Dornfest wrote an excellent article on Meerkat's XML-RPC interface here, with sample PHP and Perl code (as well as Python).
-M
You're smart; why haven't you learned Python yet? diveintopython.org