Programming Jabber

Reader cpfeifer contributes the review below of O'Reilly's Programming Jabber: if your job (or hobby) includes instant messaging in all its glory, Jabber is a free-beer, free-speech framework for setting up instant messaging systems not bound to a single server in the middle. As cpfeifer points out, instant messaging can mean a lot more than popping an on-screen note to your friend in Des Moines -- machines and programs can use a general purpose communication system like this, with no human middleman required. Programming Jabber author D.J. Adams pages 4555 publisher O'Reilly rating 9 reviewer http://cpfeifer.blogspot.com ISBN 0596002025 summary A detailed guide for developers to understanding and extending the Jabber messaging framework. Examples in Perl, Python and Java. The Scenario

Jabber was first conceived by Jeremie Miller (pic) in early 1998 in an effort to unify the disparate instant messaging networks. Instant Messaging networks rely on the network effect to gain and retain marketshare. The concept is the same when applied to any sort of participatory network whether it's a junk exchange, or content exchange, the value of the network increases with the square of the number of participants.

If this is true, then doesn't it follow that it is in the best interests of the IM networks to establish peering agreements with each other so that their users can directly contact users on other networks without having to install each client?

Hello, Jabber.

When I first picked up this book, I expected to understand the Jabber protocol in sufficient depth to implement my own IM client. Instead, the approach this book takes is that Jabber isn't just an XML-based protocol strictly for IM, rather it is a general purpose event notification protocol that has some very nice message routing and user management features built into it. While i was reading about the messages that Jabber has defined as part of the protocol, I could easily see other applications/devices generating Jabber messages to notify subscribers (either other systems, or people) of events.

Part 1 of the book focuses on getting you up to speed on the basics of Jabber technology: motivation, major features, XML protocol sample and compiling/configuring your own Jabber server. Chapter 2 presents the "10,000 foot view" of Jabber technology. In here you will find a sample client-query request/response flow with full HTTP headers, discussed step by step. The next two chapters are a very in-depth discussion of installing and configuring your own Jabber server. When you dive into a custom configuration of a fleet of Jabber servers (a "constellation" in Jabber terminology), it really starts to hit home that the real problem Jabber solves is far deeper than just IM.

From there, part 2 kicks off with a detailed discussion of the most basic building blocks of Jabber technology: resource identifiers, XML handling mechanism and the set of XML elements/attributes that make up the vocabulary of the Jabber protocol. Each element/attribute is presented with an annotated example and sample client/server interactions where appropriate. Examples can make or break a technical book, and these examples do a good job of illustrating how the element/attribute is used.

The following chapters take you through using standard Jabber features, user registration/authorization, messages, presence, groupchat, components and the event model to enable new applications. One very interesting application presented is enabling developers to receive CVS commit notifications via Jabber.

What's Bad?

I know the /. community is suspicious of glowing book reviews where everything is wonderful and nothing could be done to improve the book, so I'll nitpick. My major problem with this book is that the overwhelming majority of the sample applications are written in PERL/TK. This isn't a problem in and of itself, but I'm not a PERL/TK developer. If I build a Jabber solution, it will be in java, so PERL/TK samples don't do me a lot of good. I think equal time should be given to implementing Jabber using the two most-used languages, as defined by the number and activity of open source projects using Jabber technology.

What's Good?

This book covers everything relevant to Jabber technology, from lowest level inner workings and extensibility examples for developers to configuration and deployment for admins. Most of the book is spent looking directly at the Jabber XML protocol, instead of a specific API implementation. This way, the book covers the technology and doesn't get lost in how one particular API models the protocol.

So What's In It For Me?

If you want to implement an inside-the-firewall IM solution for your company/group/tribe or investigate integrating event notification into an application, this is a great starting point. If you're just curious about Jabber and want to know how it works, then this will give you enough information to get you hooked.

Table of Contents

PART 1: Getting Started with Jabber

• Chapter 1. Introducing Jabber
• Chapter 2. Inside Jabber
• Chapter 3. Installing the Jabber Server
• Chapter 4. Server Architecture and Configuration

PART 2: Putting Jabber's Concepts to Work

• Chapter 5. Jabber Technology Basics
• Chapter 6. Jabber Namespaces
• Chapter 7. User Registration and Authorization
• Chapter 8. Using Messages and Presence
• Chapter 9. Groupchat, Components, and Event Models
• Chapter 10. Pointers for Further Development

Appendix A. The Jabber.xml Contents

Appendix B. The IQRPC Classes for JabberRPCResponder

Index

O'Reilly has posted other reviews of the book on their site. You can purchase Programming Jabber from bn.com. Want to see your own review here? Just read the book review guidelines, then use Slashdot's handy submission form.

Comments

Jabber + SSL

[cygnusx]

I've set up a Jabber box (an early 1.x release) and played about with it, and it was a *very* good experience. Everything worked as advertised. On the other hand, setting up Jabber with SSL was a confusing process without too much documentation and I eventually gave up. Since SSL is a must for `serious' Jabber use, has there been some progress made on making secure Jabber installations easy to achieve?

My big problem with Jabber...

[mo]

I read this book looking to use jabber for automated XML messaging and I'll have to say, it has a lot of nifty features that I'd love to use. Unfortunately, it's never getting deployed in my network. Why?
You can't cluster jabber servers. If the main jabber server goes down, you're hosed. In any application that's worth the effort to deploy, having such a single point of failure is a big problem. Additionally, I was kinda annoyed at how jabber leans so much towards instant messaging. I know, I know, that's what it was built for, but this book is trying to pass it off as an "XML messaging" tool, but it's properties often sway back to IM.

In conclusion, if you wanna fool around with a nifty IM robot that doesn't need to be relied on, jabber is a nifty tool. If you wanna do real XML messaging, try something like xmlblaster.

Re:My big problem with Jabber...

[jeremie]

What your talking about here is a particular implementation of a Jabber server, jabberd, not Jabber in general (people often confuse this point). You can do some minimal clustering with the jabberd-1.4 series, but probably not the kind of reliability that your looking for or that jabber.com has built into their server.

Jabber is an open system/protocol, anyone can build new servers/clients/etc with whatever features and extensions they want, including building it on/with xmlblaster. Jabberd is also an open source project that your welcome to help with (farming/clustering is a frequent need and I suspect that it will be a large part of the jabberd-1.5 development series).

Hey, remember SMTP?

[hqm]

One thing that confuses me about Jabber is that
people seem to forget that good old SMTP solves many of the same problems, and in fact solves them better.

For example, many years of work have gone into making sure that email never gets lost. SMTP mailers just don't lose email anymore. Jabber messages, on the other hand, are not really reliable. If the user to whom you are targeting a message is not online, the server may queue the messages, but the policy is not clear as to how long they will be stored, or if the server is rquired to store them at all.

This makes me worry about the idea of using Jabber to build infrastructure where you
rely on messages to always be delivered.

It seems to me that many of the issues that Jabber
solves have been solved using existing
technology such as SMTP, and mailer and mailing list services built on top of it, like qmail, mailman, etc.

Jabber is a hack

[ProfessorPuke]

As are all "Instant Message" programs. They are a poorly-designed, short-sighted solution to a problem that should've been addressed elsewhere in the internet architecture.

Part of the problem stems from the fact that IM software addresses 2 applications at the same time, unnecessarily coupling the implementations. These problems could really be approached separately:

• Learn the IP address associated with a globally-unique username
• Send a text message to the interactive operator of a machine with an IP address

The first problem is the much more interesting one- Jabber & AIM already somewhat solve it, but in an unsatisfactory and poorly extensible way. Better solutions would be based on an extension to the normal DNS system- essentially, you want each human to have a resolvable domain name associated with her. With that in place, InstantMessaging is an easy problem.

A person could try to implement "TCP over IM", but it would've been nicer if the systems had been designed for this from the start. Actually, there is a 3rd general-purpose facility that might be needed, for reasons of privacy. There should be a way to send a packet to a "resolvable human name", without knowing the IP address it currently maps through. The (trusted) central server will have to forward packets in both directions. (I think that's how AIM normally operates, except that it doesn't accept generic packets, only AIM-formatted messages).

However, that method doesn't uniformly improve privacy. While it does prevent other users from learning your IP address, it makes it much easier for AOL (or other central server operator) to spy on the contents of your discussions. (You should be using encryption, anyway).

Re:Jabber is a hack

[devnullkac]

Learn the IP address associated with a globally-unique username

Unfortunately, the user identification problem is complicated by the fact that there may be more than one person using a given IP address. Firewalls which implement NAT and servers which have multiple simultaneous logins are quite common and give this complication real teeth.

You could perhaps claim that the task is really to associate an IP address and TCP/UDP port with a person, but you can only realistically use one service per port (port 80 overloading notwithstanding), so you'd have to say that the identification is solely for IM, and so the solution of the problem isn't really useful outside IM applications.

Or you could instead say that the task is to associate an IP address and TCP/UDP port with a person and a service, but now you've got the problem of identifying all services and handling the dynamic nature of port assignment as users become available/unavailable and declare themselves as participating/not participating in the various services. Not impossible, but hard enough that nobody seems to have solved it yet.

Re:Jabber is a hack

[Temas]

I think you're failing to grasp some of the points of Jabber messaging, especially as something more than basic chat. The idea (at least from the Jabber point of view) is to _NOT_ learn the IP address of the other party. You only reference them from their "username". Username is the wrong term though, we user the term JID (Jabber id). The JID format is: username@host/resource. So it is built upon a DNS like system. Once you know another users JID you can interact with it using messaging, presence, or other methods. The power of not trying to interact with an IP directly is it's ability to more cleanly go around firewalls. The client makes a connection outside of the firewall to their Jabber server (potentially through some proxy), and they are then on the entire Jabber network. Applications that are exposed on the network then have the ability to interactively use presence and a clear path to the user for more complete interaction. So perhaps you are looking at this from the wrong viewpoint?

Re:Jabber

[JabberWokky]

FWIW, when I upgraded to KDE 3, I took a look around and decided to play with new software. I had had much the same experience with Jabber, where it just didn't really work well with other IM systems. For the past three days or so, I've been using Psi, which seems to work quite nicely with Yahoo Messenger and AIM (the latter of which I use quite heavily in both work and socially). It's Qt based, and so it runs on Windows, Linux, OSX and embedded systems, and since it's under the GPL, you can use the Qt free edition. I think there were binaries there for all the platforms.

YMMV, but it's working for me, plus the cross platform nature means that I'll start recommending it to people who have been using Trillian in the past. It's at the "almost there, but not quite finished" level, with two major bits missing - a total lack of documentation (which can get gotten around), and lack of support for group chat - which means the IRC service won't connect (not to mention AIM group chats). I just discovered it, so I can't say how fast work progresses on the project, but it's very much usable for my needs right now. Sounds like it might work for you, too.

My Jabber ID is JabberWokky@charente.de, and that server supports AIM, ICQ, MSN, YIM, Jabber and IRC.

--
Evan

Jabber shortcomings - not in the book

[mgkimsal2]

The book, from what I read of it (not 100% - maybe 60%) is handy, but didn't tell me much beyond the jabber documentation already out there.

What seems to be a huge issue for Jabber is user profile integration with databases. There seems to be an unsupported mysql hack, but the key is 'unsupported'. If you look in the Jabber mail list archives, every month there's people asking how to do it, but NEVER any answers.

Another great one that doesn't get answered - which the book doesn't address either - is the format of the user XML files. Each user by default has an XML file, and many people would like to create them programatically. There is no definitive resource which explains what's in a file and what isn't, and how to put one together. I've hacked something, and it works, but only after several attemps, and it doesn't *feel* good. I'm hesitant to try to add anything else lest I break what's working.

Jabber.com has a huge vested interest in keeping some of this stuff not in the public knowledgebase, because they charge (comparitively) a LOT of money for their stuff.

Last time I spoke with them the minimum to get started was $16,000. Their package offers a completely rewritten jabber server (better thread handling), Oracle and LDAP connectors, and a good Java applet client.

NO ONE in the open source community has even come close to having a Java applet client that is workable in a practical sense.

So yes, the protocol is open, and free, but there doesn't seem to be much consensus on tools, except from Jabber.com and they cost.

What I think Jabber as an open source project needs to focus on:

* XML user file definition and/or database support for user profiles
* Good applet client

:)

Re:Jabber shortcomings - not in the book

[Temas]

I like to make sure that all questions asked on our (jabber.org) jdev and jadmin mailing lists get answered. Sometimes the questions will get answered in our groupchat, or in other ways, but an answer is usually given.

As far as I know xdb_sql currently lacks a maintainer, but it's open source so maybe someone will pick it up.

The lack of good docs on the user file definition is valid, but at the same time it's not suggested you edit it by hand due to the aggressive cacheing used on it. We're starting a new docs effort right now and I'll make sure this is on the list.

The basic applet that is on sourceforge (here) is focussed on simplicity, but it does work, and I know people have built more off of it.

As to your concerns with Jabber, Inc., I don't know why they would want to keep any of that stuff secret. It's mostly useless to them since their server ships with a different XDB backend and their web client has a different focus than a pure applet approach.

Re:Jabber shortcomings - not in the book

[mgkimsal2]

Thank you for your response.

The basic applet is just far *too* basic to be of much use to our situation (and I guess many others). For starters, it assumes you want to allow people to create an account on your server - there seems to be no way to shut that off in the client.

The person I spoke with at Jabber.com told me they'd completely rewritten the jabber server to be high-volume capable, but that they wouldn't be releasing that code. Possibly ever, or possibly just much later. It's an investment for them, and they have every reason *to* keep is secret. If it's open, and people could implement it themselves, why would they pay Jabber.com?

I wasn't wanting to edit the user XML files by hand, but create them programmatically for users.

I've see the jdev lists and it looks like most questions get answered, but I'd gone looking and never found an answers on the xml file structure for user files, but many questions about it.

Again, thanks for answering. :)

wrong

[Milkman Ken]

You can very easily cluster jabber servers. In fact, I have five running:

one for the main server

one specifically for AIM

one for ICQ

one for MSN

one for yahoo! IM

the four IM trasport servers have their own jabberd process. If a transport server dies (as they occasionally do), you can bring that server back up without affecting any other servers.

But you don't have to break up the servers this way. You could run multiple jabber servers, and place bandwidth restrictions on them so that when a jabber server got "full", it would stop receiving connections, so the jabber server above it in the chain would then forward it on to the next jabber server in the chain, or back up if it's out of children servers.

it's a relatively simple matter to setup an init.d script to monitor the health of all the processes, and restart them when and if they fail. I've been running a jabber server on one of our linux boxes for weeks now, and I haven't had to touch it once. I highly recommend jabber for intranets.

Java Oriented Jabber Book

[Temas]

I've seen a lot of comments disliking the abundance of Perl/Tk usage in DJ's book. Recently Manning Publications released Instant Messaging in JAVA: The Jabber Protocols in print and ebook. It was written by Iain Shigeoka, and is ISBN 1-930110-46-4. It's a good read and goes over the creation of both a client and a basic server in Java, plus a good deal more.

Re:AIM

[Temas]

I beg to differ. I develop the AIM Transport. I've also worked on libfaim, and was around for the initial introduction of TOC. TOC looked promising despite it's odd ASCII protocol, but we continued work on OSCAR because TOC was considered a side project and not 100% supported by AOL. As luck would have it AOL has proved that decision to be wise many times. They have stopped work on TOC numerous times and have even removed features from it. OSCAR has continued to grow. When AOL started to try and block us (Jabber) we grew fairly confident that their changes were directed solely at Jabber. The blocks always happened after minute changes I made in the aim source specifically, and we were told so in an indirect way. Some ended up affecting other libfaim based projects such as Gaim. Until everything was figured out I was heavily considering a TOC implementation. The problem was that would have caused problems for other programs using TOC if they continued to actively target Jabber. I decided this type of behaivour would be unfair to the other projects, and it continues to allow them to always have a "pure" channel. In the end it has all worked out. We have fully figured out their attempted blocks and everything seems to be moving forward. There are specific IP blocks on some of the larger Jabber servers, but that's life.

Currently I'm actively working on the AIM-Transport (more information). and expect to put out a version 0.10 in not too long.