Programming Jabber

Reader cpfeifer contributes the review below of O'Reilly's Programming Jabber: if your job (or hobby) includes instant messaging in all its glory, Jabber is a free-beer, free-speech framework for setting up instant messaging systems not bound to a single server in the middle. As cpfeifer points out, instant messaging can mean a lot more than popping an on-screen note to your friend in Des Moines -- machines and programs can use a general purpose communication system like this, with no human middleman required. Programming Jabber author D.J. Adams pages 4555 publisher O'Reilly rating 9 reviewer http://cpfeifer.blogspot.com ISBN 0596002025 summary A detailed guide for developers to understanding and extending the Jabber messaging framework. Examples in Perl, Python and Java. The Scenario

Jabber was first conceived by Jeremie Miller (pic) in early 1998 in an effort to unify the disparate instant messaging networks. Instant Messaging networks rely on the network effect to gain and retain marketshare. The concept is the same when applied to any sort of participatory network whether it's a junk exchange, or content exchange, the value of the network increases with the square of the number of participants.

If this is true, then doesn't it follow that it is in the best interests of the IM networks to establish peering agreements with each other so that their users can directly contact users on other networks without having to install each client?

Hello, Jabber.

When I first picked up this book, I expected to understand the Jabber protocol in sufficient depth to implement my own IM client. Instead, the approach this book takes is that Jabber isn't just an XML-based protocol strictly for IM, rather it is a general purpose event notification protocol that has some very nice message routing and user management features built into it. While i was reading about the messages that Jabber has defined as part of the protocol, I could easily see other applications/devices generating Jabber messages to notify subscribers (either other systems, or people) of events.

Part 1 of the book focuses on getting you up to speed on the basics of Jabber technology: motivation, major features, XML protocol sample and compiling/configuring your own Jabber server. Chapter 2 presents the "10,000 foot view" of Jabber technology. In here you will find a sample client-query request/response flow with full HTTP headers, discussed step by step. The next two chapters are a very in-depth discussion of installing and configuring your own Jabber server. When you dive into a custom configuration of a fleet of Jabber servers (a "constellation" in Jabber terminology), it really starts to hit home that the real problem Jabber solves is far deeper than just IM.

From there, part 2 kicks off with a detailed discussion of the most basic building blocks of Jabber technology: resource identifiers, XML handling mechanism and the set of XML elements/attributes that make up the vocabulary of the Jabber protocol. Each element/attribute is presented with an annotated example and sample client/server interactions where appropriate. Examples can make or break a technical book, and these examples do a good job of illustrating how the element/attribute is used.

The following chapters take you through using standard Jabber features, user registration/authorization, messages, presence, groupchat, components and the event model to enable new applications. One very interesting application presented is enabling developers to receive CVS commit notifications via Jabber.

What's Bad?

I know the /. community is suspicious of glowing book reviews where everything is wonderful and nothing could be done to improve the book, so I'll nitpick. My major problem with this book is that the overwhelming majority of the sample applications are written in PERL/TK. This isn't a problem in and of itself, but I'm not a PERL/TK developer. If I build a Jabber solution, it will be in java, so PERL/TK samples don't do me a lot of good. I think equal time should be given to implementing Jabber using the two most-used languages, as defined by the number and activity of open source projects using Jabber technology.

What's Good?

This book covers everything relevant to Jabber technology, from lowest level inner workings and extensibility examples for developers to configuration and deployment for admins. Most of the book is spent looking directly at the Jabber XML protocol, instead of a specific API implementation. This way, the book covers the technology and doesn't get lost in how one particular API models the protocol.

So What's In It For Me?

If you want to implement an inside-the-firewall IM solution for your company/group/tribe or investigate integrating event notification into an application, this is a great starting point. If you're just curious about Jabber and want to know how it works, then this will give you enough information to get you hooked.

Table of Contents

PART 1: Getting Started with Jabber

• Chapter 1. Introducing Jabber
• Chapter 2. Inside Jabber
• Chapter 3. Installing the Jabber Server
• Chapter 4. Server Architecture and Configuration

PART 2: Putting Jabber's Concepts to Work

• Chapter 5. Jabber Technology Basics
• Chapter 6. Jabber Namespaces
• Chapter 7. User Registration and Authorization
• Chapter 8. Using Messages and Presence
• Chapter 9. Groupchat, Components, and Event Models
• Chapter 10. Pointers for Further Development

Appendix A. The Jabber.xml Contents

Appendix B. The IQRPC Classes for JabberRPCResponder

Index

O'Reilly has posted other reviews of the book on their site. You can purchase Programming Jabber from bn.com. Want to see your own review here? Just read the book review guidelines, then use Slashdot's handy submission form.

Comments

Jabber

[Sabby]

I got really excited about Jabber for the longest time. I'm sort of disappointed in it now, since it seems like they're still having problems connecting to AIM and ICQ. The AIM connection is the most vital for me, since our department uses AIM to send short quick messages to each other. Most of the people here are using AIM's own client, but I started to use Jabber so that I could talk to my friends on ICQ. (And promptly signed up for MSN and Yahoo, so I could catch everyone from everywhere.) Now I use Trillian, which only disappoints me by neither providing source code (which I only want for the principle of it) nor supporting Jabber itself (which does kind of bug me).

Re:Jabber

[garcia]

I couldn't see using AIM to send "quick messages" in the sense you seem to be trying to convey.

I use a wireless laptop in the living room to play MP3s from the wired computers in the house. I had to delete IM from the computer b/c stupid college students are fucking addicted to it.

The addiction isn't so much the problem. The poor CPU is only a p133. It can't handle MP3s and IMs. Then the bastards complain that it takes too much time to send messages and to top it off they fight over who gets to send an IM next or see which profile has been updated in the last 4 minutes.

The people that left college and are now working in the real world sit on AIM all day and chat. My father, 55, sits on AIM all day and chats. My mother, working at a funeral home and a church, doesn't chat only b/c there is no Internet connection? there.

Jabber (GAIM, Trillian, etc) would complicate this problem furthur by allowing crazy fools who use IRC, MSN, Yahoo, AIM, etc to talk even more and claim it was for good use.

I say down w/AIM. ;-)

Just a little half-off-topic humor for Thursday.

Re:Jabber

[JabberWokky]

FWIW, when I upgraded to KDE 3, I took a look around and decided to play with new software. I had had much the same experience with Jabber, where it just didn't really work well with other IM systems. For the past three days or so, I've been using Psi, which seems to work quite nicely with Yahoo Messenger and AIM (the latter of which I use quite heavily in both work and socially). It's Qt based, and so it runs on Windows, Linux, OSX and embedded systems, and since it's under the GPL, you can use the Qt free edition. I think there were binaries there for all the platforms.

YMMV, but it's working for me, plus the cross platform nature means that I'll start recommending it to people who have been using Trillian in the past. It's at the "almost there, but not quite finished" level, with two major bits missing - a total lack of documentation (which can get gotten around), and lack of support for group chat - which means the IRC service won't connect (not to mention AIM group chats). I just discovered it, so I can't say how fast work progresses on the project, but it's very much usable for my needs right now. Sounds like it might work for you, too.

My Jabber ID is JabberWokky@charente.de, and that server supports AIM, ICQ, MSN, YIM, Jabber and IRC.

--
Evan

Jabber and Sendmail

[Anonymous Coward]

While on the topic of Jabber. Why not have the sendmail folks and the jabber folks get togethor and unite their work into a single project. Complete with admin tools so that once someone has a sendmail account on a Unix, they by default have a jabber IM account. It would go a long ways towards taking down AIM, MSN, and ICQ.

Re:Jabber and Sendmail

[Raleel]

Check out the smtp transport for jabber...this might provide what your looking for.

I've thought this would be an excellent idea as well. So much in fact, that I've been looking at encorporating it into my esmith box. It would be great...add an account, they get domain access, windows shares, email, webmail, groups, jabber...a real single point of service (ya, I know...take it out and your screwed...there are ways around that).

Jabber + SSL

[cygnusx]

I've set up a Jabber box (an early 1.x release) and played about with it, and it was a *very* good experience. Everything worked as advertised. On the other hand, setting up Jabber with SSL was a confusing process without too much documentation and I eventually gave up. Since SSL is a must for `serious' Jabber use, has there been some progress made on making secure Jabber installations easy to achieve?

Re:Jabber + SSL

[cowmix]

SSL on the server side is a no brainer now.. Most clients also implement SSL too.

More important, IMHO, many clients support end to end security via PGP/GPG...

My big problem with Jabber...

[mo]

I read this book looking to use jabber for automated XML messaging and I'll have to say, it has a lot of nifty features that I'd love to use. Unfortunately, it's never getting deployed in my network. Why?
You can't cluster jabber servers. If the main jabber server goes down, you're hosed. In any application that's worth the effort to deploy, having such a single point of failure is a big problem. Additionally, I was kinda annoyed at how jabber leans so much towards instant messaging. I know, I know, that's what it was built for, but this book is trying to pass it off as an "XML messaging" tool, but it's properties often sway back to IM.

In conclusion, if you wanna fool around with a nifty IM robot that doesn't need to be relied on, jabber is a nifty tool. If you wanna do real XML messaging, try something like xmlblaster.

Re:My big problem with Jabber...

[jeremie]

What your talking about here is a particular implementation of a Jabber server, jabberd, not Jabber in general (people often confuse this point). You can do some minimal clustering with the jabberd-1.4 series, but probably not the kind of reliability that your looking for or that jabber.com has built into their server.

Jabber is an open system/protocol, anyone can build new servers/clients/etc with whatever features and extensions they want, including building it on/with xmlblaster. Jabberd is also an open source project that your welcome to help with (farming/clustering is a frequent need and I suspect that it will be a large part of the jabberd-1.5 development series).

Hey, remember SMTP?

[hqm]

One thing that confuses me about Jabber is that
people seem to forget that good old SMTP solves many of the same problems, and in fact solves them better.

For example, many years of work have gone into making sure that email never gets lost. SMTP mailers just don't lose email anymore. Jabber messages, on the other hand, are not really reliable. If the user to whom you are targeting a message is not online, the server may queue the messages, but the policy is not clear as to how long they will be stored, or if the server is rquired to store them at all.

This makes me worry about the idea of using Jabber to build infrastructure where you
rely on messages to always be delivered.

It seems to me that many of the issues that Jabber
solves have been solved using existing
technology such as SMTP, and mailer and mailing list services built on top of it, like qmail, mailman, etc.

Re:Hey, remember SMTP?

[Raleel]

Actually, you should check out the smtp transport for jabber....it might provide some of the functionality that you require.

Re:Hey, remember SMTP?

[MeNeXT]

Forget POP3 just extend SMTP.

I'll call it and SMTP client. When it connects to the SMTP server it identifies the user and the users IP. The server forwards all messages to this IP. If it's unable to forward it places messages into a mailbox for POP3.

SMTP client first opens POP3 receives mail. Sends ID and IP to SMTP server. Client displays messages and relays messages through SMTP server.

No central server. Your email is your ID. nice and easy. Some detaisl to be worked out.

Jabber

[Sabby]

I've played around with the jabber module in Perl, which was pretty easy to use.

Jabber started to disappoint when they stopped supporting AIM/ICQ. I don't know if it's permanent, I don't actually know if it's still not supported. But, since AIM is what I have to use for work (otherwise, I would still just be using ICQ to talk to my friends), I needed something that could stay connected.

I use Trillian now. It still does ICQ/AIM as well as IRC/MSN/Y!, which is why I need something like this, but it doesn't provide source code (which I only really want for the principle of it) and it doesn't support Jabber's protocol. (They're talking about releasing an API for writing plugins. At least it's free (as in beer). (I've got a few of my coworkers switched from AIM to Trillian...) Hopefully Jabber will fix up the connectivity issues (or have ALREADY fixed them up.) gosh, I should download WinJab again and check.

Could Jabber replace IBM's MQ-Series?

[s390]

I'm serious. IBM spent a ton of money building MQ-Series, which is a hideously complex messaging protocol for inter-and-intra systems communications in and between mainframe subsystems/LPARs and Unix systems (AIX mostly, since this is IBM, after all).

MQ-Series really is complicated, maybe over-complicated, to the point that IBM and customers even have "MQ-Series Specialists" on staff.

I'm not flaming IBM here (h*ll, I used to work for them, and they're a great company to work with), but they do have an unfortunate tendency to build overly complex systems where simpler ones might be a lot easier to use.

Re:Pretty good stuff.

[CaseyB]

This is one place I'd focus on. You know, perhaps an avatar sort of thing; in your programming (work) avatar, you are online to only a certain people

More complex event handling in general, yeah. It'd be nice to have a generalized, maybe scripted, event handlers.

"When (user in group "family") (logs in) (after 5:00 pm), (play ring.wav)".

"When (Joe) (changes status to (idle) (between 9am and 5pm)) send msg to joe: 'Get to work!!'".

Jabber is a hack

[ProfessorPuke]

As are all "Instant Message" programs. They are a poorly-designed, short-sighted solution to a problem that should've been addressed elsewhere in the internet architecture.

Part of the problem stems from the fact that IM software addresses 2 applications at the same time, unnecessarily coupling the implementations. These problems could really be approached separately:

• Learn the IP address associated with a globally-unique username
• Send a text message to the interactive operator of a machine with an IP address

The first problem is the much more interesting one- Jabber & AIM already somewhat solve it, but in an unsatisfactory and poorly extensible way. Better solutions would be based on an extension to the normal DNS system- essentially, you want each human to have a resolvable domain name associated with her. With that in place, InstantMessaging is an easy problem.

A person could try to implement "TCP over IM", but it would've been nicer if the systems had been designed for this from the start. Actually, there is a 3rd general-purpose facility that might be needed, for reasons of privacy. There should be a way to send a packet to a "resolvable human name", without knowing the IP address it currently maps through. The (trusted) central server will have to forward packets in both directions. (I think that's how AIM normally operates, except that it doesn't accept generic packets, only AIM-formatted messages).

However, that method doesn't uniformly improve privacy. While it does prevent other users from learning your IP address, it makes it much easier for AOL (or other central server operator) to spy on the contents of your discussions. (You should be using encryption, anyway).

Re:Jabber is a hack

[devnullkac]

Learn the IP address associated with a globally-unique username

Unfortunately, the user identification problem is complicated by the fact that there may be more than one person using a given IP address. Firewalls which implement NAT and servers which have multiple simultaneous logins are quite common and give this complication real teeth.

You could perhaps claim that the task is really to associate an IP address and TCP/UDP port with a person, but you can only realistically use one service per port (port 80 overloading notwithstanding), so you'd have to say that the identification is solely for IM, and so the solution of the problem isn't really useful outside IM applications.

Or you could instead say that the task is to associate an IP address and TCP/UDP port with a person and a service, but now you've got the problem of identifying all services and handling the dynamic nature of port assignment as users become available/unavailable and declare themselves as participating/not participating in the various services. Not impossible, but hard enough that nobody seems to have solved it yet.

Re:Jabber is a hack

[Temas]

I think you're failing to grasp some of the points of Jabber messaging, especially as something more than basic chat. The idea (at least from the Jabber point of view) is to _NOT_ learn the IP address of the other party. You only reference them from their "username". Username is the wrong term though, we user the term JID (Jabber id). The JID format is: username@host/resource. So it is built upon a DNS like system. Once you know another users JID you can interact with it using messaging, presence, or other methods. The power of not trying to interact with an IP directly is it's ability to more cleanly go around firewalls. The client makes a connection outside of the firewall to their Jabber server (potentially through some proxy), and they are then on the entire Jabber network. Applications that are exposed on the network then have the ability to interactively use presence and a clear path to the user for more complete interaction. So perhaps you are looking at this from the wrong viewpoint?

Re:Jabber is a hack

[ProfessorPuke]

Maybe so. I didn't mean to be so negative, I've been wanting to add Jabber support to my own software. I was unfairly lumping it together with AIM, ICQ, and MSN (to the general public, all instant messengers are the same program with different colored icons).

One problem I have is that, as an external optimist, I assume that "IPv6 is right around the corner". That would mean that no one ever needs to use NAT again, and that a single computer can have multiple IP address for all of its users (or other purposes). And we already have a DNS system to provide mappings from human-readable strings to software-usable network addresses. I feel a little bad (and dubious) about seeing someone try to reimplement that, even if it is the surest way to ensure that the existing DNS features don't get broken.

The final concern I have with this Jabber approach is that its a complete layer above TCP- applications are written to the Jabber API and don't even know that TCP is involved. That's a good thing from the perspective of modularity and OSI-style layering, but bad in terms of evolutionary adoption. Existing software is written for the "static web", and that's where corporate money is going to focus future developement. Without a way to gradually shoehorn into popular internet applications, Jabber support may remain a hobby for open-source outcasts, and not benefit the majority of users.

As a transitionary step, someone could write (maybe someone already did?) a Jabber utility that behaves like a combination of DNS lookup and RPC portmapping- providing a ip address/portnumber in response to a JID string. Many applications could utilize something like that by adding just a few lines after their "hostname()" calls.

Two use cases where evolutionary Jabber ID support could be valuable:

• I'd love to read someone a "JID/filename" URI over a telephone, and have him type it into Mozilla/Konqueror/Internet Explorer and pop up a file I've selected to share from my workstation, without having to play around with "Dynamic DNS" services (which are unreliable, expensive, and an even worse hack).
• Groups of video-game players ("Clans", they were called, back when Quake came out) should be able to create a server for their own use, and get into it by supplying the server admin's JID into their client software. The gameplay can't afford the overhead of Jabber messages (or even TCP) slowing up the running & shooting, so there should only be a brief burst of Jabber traffic at startup to bootstrap UDP communications. (That's an automation of what today's players do over AIM messages).

  John Carmack is friendly to free software- when the inevitable Quake4 developement starts up, someone should offer him a simple Jabber interface as an optional way for players to connect to servers, instead of the corporate Gamespy / MSG Gaming Zone options that you see today.

(Now, if only I knew a way to mix freenet into this equation...)

Jabber shortcomings - not in the book

[mgkimsal2]

The book, from what I read of it (not 100% - maybe 60%) is handy, but didn't tell me much beyond the jabber documentation already out there.

What seems to be a huge issue for Jabber is user profile integration with databases. There seems to be an unsupported mysql hack, but the key is 'unsupported'. If you look in the Jabber mail list archives, every month there's people asking how to do it, but NEVER any answers.

Another great one that doesn't get answered - which the book doesn't address either - is the format of the user XML files. Each user by default has an XML file, and many people would like to create them programatically. There is no definitive resource which explains what's in a file and what isn't, and how to put one together. I've hacked something, and it works, but only after several attemps, and it doesn't *feel* good. I'm hesitant to try to add anything else lest I break what's working.

Jabber.com has a huge vested interest in keeping some of this stuff not in the public knowledgebase, because they charge (comparitively) a LOT of money for their stuff.

Last time I spoke with them the minimum to get started was $16,000. Their package offers a completely rewritten jabber server (better thread handling), Oracle and LDAP connectors, and a good Java applet client.

NO ONE in the open source community has even come close to having a Java applet client that is workable in a practical sense.

So yes, the protocol is open, and free, but there doesn't seem to be much consensus on tools, except from Jabber.com and they cost.

What I think Jabber as an open source project needs to focus on:

* XML user file definition and/or database support for user profiles
* Good applet client

:)

Re:Jabber shortcomings - not in the book

[Temas]

I like to make sure that all questions asked on our (jabber.org) jdev and jadmin mailing lists get answered. Sometimes the questions will get answered in our groupchat, or in other ways, but an answer is usually given.

As far as I know xdb_sql currently lacks a maintainer, but it's open source so maybe someone will pick it up.

The lack of good docs on the user file definition is valid, but at the same time it's not suggested you edit it by hand due to the aggressive cacheing used on it. We're starting a new docs effort right now and I'll make sure this is on the list.

The basic applet that is on sourceforge (here) is focussed on simplicity, but it does work, and I know people have built more off of it.

As to your concerns with Jabber, Inc., I don't know why they would want to keep any of that stuff secret. It's mostly useless to them since their server ships with a different XDB backend and their web client has a different focus than a pure applet approach.

Re:Jabber shortcomings - not in the book

[mgkimsal2]

Thank you for your response.

The basic applet is just far *too* basic to be of much use to our situation (and I guess many others). For starters, it assumes you want to allow people to create an account on your server - there seems to be no way to shut that off in the client.

The person I spoke with at Jabber.com told me they'd completely rewritten the jabber server to be high-volume capable, but that they wouldn't be releasing that code. Possibly ever, or possibly just much later. It's an investment for them, and they have every reason *to* keep is secret. If it's open, and people could implement it themselves, why would they pay Jabber.com?

I wasn't wanting to edit the user XML files by hand, but create them programmatically for users.

I've see the jdev lists and it looks like most questions get answered, but I'd gone looking and never found an answers on the xml file structure for user files, but many questions about it.

Again, thanks for answering. :)

wrong

[Milkman Ken]

You can very easily cluster jabber servers. In fact, I have five running:

one for the main server

one specifically for AIM

one for ICQ

one for MSN

one for yahoo! IM

the four IM trasport servers have their own jabberd process. If a transport server dies (as they occasionally do), you can bring that server back up without affecting any other servers.

But you don't have to break up the servers this way. You could run multiple jabber servers, and place bandwidth restrictions on them so that when a jabber server got "full", it would stop receiving connections, so the jabber server above it in the chain would then forward it on to the next jabber server in the chain, or back up if it's out of children servers.

it's a relatively simple matter to setup an init.d script to monitor the health of all the processes, and restart them when and if they fail. I've been running a jabber server on one of our linux boxes for weeks now, and I haven't had to touch it once. I highly recommend jabber for intranets.

Re:wrong

[Milkman Ken]

either you don't understand how jabberd linking works, or I don't understand what you're trying to do. You can link different jabberd servers in any way you want. If you want a HA, failsafe system, put your jabberd's on different switched subnets in a hypercube pattern, and you can lose several servers or subnets without affecting the network as a whole.

You mainly seem to be concerned that since there in a single access point to the system, the whole thing can fail with a single attack on the main server. To a certain extent that's true. The user login data is kept on a jabber server, somewhere, and if that machine fails you lose the ability for certain users to login. I'm not sure if you can replicate user data across several jabberd's (with proper delegation and syncing), but it's probably not hard to implement.

Java Oriented Jabber Book

[Temas]

I've seen a lot of comments disliking the abundance of Perl/Tk usage in DJ's book. Recently Manning Publications released Instant Messaging in JAVA: The Jabber Protocols in print and ebook. It was written by Iain Shigeoka, and is ISBN 1-930110-46-4. It's a good read and goes over the creation of both a client and a basic server in Java, plus a good deal more.

Re:AIM

[Temas]

I beg to differ. I develop the AIM Transport. I've also worked on libfaim, and was around for the initial introduction of TOC. TOC looked promising despite it's odd ASCII protocol, but we continued work on OSCAR because TOC was considered a side project and not 100% supported by AOL. As luck would have it AOL has proved that decision to be wise many times. They have stopped work on TOC numerous times and have even removed features from it. OSCAR has continued to grow. When AOL started to try and block us (Jabber) we grew fairly confident that their changes were directed solely at Jabber. The blocks always happened after minute changes I made in the aim source specifically, and we were told so in an indirect way. Some ended up affecting other libfaim based projects such as Gaim. Until everything was figured out I was heavily considering a TOC implementation. The problem was that would have caused problems for other programs using TOC if they continued to actively target Jabber. I decided this type of behaivour would be unfair to the other projects, and it continues to allow them to always have a "pure" channel. In the end it has all worked out. We have fully figured out their attempted blocks and everything seems to be moving forward. There are specific IP blocks on some of the larger Jabber servers, but that's life.

Currently I'm actively working on the AIM-Transport (more information). and expect to put out a version 0.10 in not too long.

Re:Jabber : great concept, awful reality.

[tzanger]

I'd mod you as flamebait, but it looks like someone else already did. Quit spewing FUD.

I've set up a Jabber server over 6 months ago and I'm using a client called Psi. I regularly connect to the MSN and ICQ networks through my server. I have not experienced one problem, much less the disaster you predict.

I prefer Jabber to the mess of carrying a cellphone, pager, checking email and the office phone. Yes I have them all but I only carry the cel/pager when necessary. I tell people to use Jabber or email if the need to get in touch with me, since telco charges are expensive and I'm not likely to be at the office anyway. My email client isn't always open but my IM is. Jabber is excellent for tying things together.

In a similar vein, if someone were to suggest to me firing anyone who suggested Jabber I'd end up firing them for being so small-minded. I've far less use for a person who won't consider new technologies than someone who is constantly on the lookout for the next best thing. Then again I'm the network admin for this company, so what do I know?