Will BEEP Simplify Network Programming?

hensley writes "There is a (not quite) new effort by the IETF to standardize a framework for network applications, called BEEP, the Blocks Extensible Exchange Protocol. Standardized in RFC3080, it takes care of all lower level tasks an application level protocol has to like framing, authentication and capabilities negotiation in a modular and lightweight way. In the current issue of Internet Packet Journal (a quite nice and free-as-in-beer technical publication by Cisco) is a well written Introduction to this framework. Why isn't anyone adopting this protocol besides some Java libraries like beep4j and PermaBEEP and a C library called RoadRunner. I couldn't find any applications based on this protocol, regardless of it's promised capabilities. Is everybody still inventing his own application layer protocol?"

Comments

REST, Jabber, SOAP

[tburke]

I looked at BEEP early on, when it was called BXXP, for doing RPC calls. At the time we decided that SOAP over HTTP was the better option for our application. I think that the world has moved on since then, today I would design using REST principles and HTTP, and take advantage of Apache. For applications requiring something more 'stateful' I would probably choose Jabber, which is reasonably well established, over BEEP. BEEP will have a hard time competeing with established extensible protocols which do, if not an excellent job, a good enough job for most applications.

to clarify

[adam_megacz]

Technically, you can argue that UDP is not "connectionless", because when I send a UDP packet from my laptop, it causes it to dial up my ISP, making a PPP "connection" to the ISP.

The idea is to introduce as little channel state (as few nested channels) as possible. BEEP is essentially "TCP over TCP".

Not so impressed

[Zeinfeld]

I was one of the authors of the HTTP protocol and did some work on HTTP-NG so I am broadly sympathetic to BEEP. However there are a number of reasons why it is not having a huge impact, some political, some technical.

BEEP's main proponent, Marshall Rose was one of the main wheels in the OSI project. So much of the initial buzz came from his name alone. People were talking about the protocol before they read the drafts (oh yes that is normal for the IETF).

I do have a bunch of quibbles technically. First using XML is a good idea, Using the obsolete SGML DTD mechanism to describe the protocol sucks. I think Marshall started to suplement the DTD with schema fragments but that makes things worse, not better, we now have two specs in one document, the schema version which is what people will implement and the DTD version which is normative.

The other problem is that SGML is a real baaad choice for an encoding at that level. The main complaint about http is that the encoding is too verbose leading protocol exchanges to require multiple round packets instead of one. BEEP does not address that problem.

Politically BEEP has bigger problems, first being that IETF does not have as much influence as it might appear when it comes to promoting new protocols. There haven't been very many IETF protocols that started in IETF process and took off like wildfire in the past ten years. HTTP took off and was brought into IETF process, same with TLS (SSL). Most successful IETF protocols had a userbase before the working group was formed.

The problem with BEEP is that Marshall did not start with a constituency who had a problem that BEEP was the solution for. Instead he wrote the protocol and then went off looking for consumers. So we start to see Marshall popping up at random in working groups like SACRED peddling the BEEP Kool-Aid. The problem being that if you are doing a researchy protocol like SACRED the last thing you should be doing is layering it on someone else's research.

After this happened a few times Marshall started to alienate folk like myself who might be interested in BEEP as an option but certainly were not going to allow him to insert himself onto our critical path.

The other problem is the nature of the IETF these days. The problem is that they talk a good talk about being open and such, but it is really an old-boys club. The old-fart faction is strong on the IESG and IAB, they have known each other for 20 years and they don't want anyone messing with their turf.

In theory the IETF process is open. In practice there are a bunch of shadowy cliques who make the real decisions in private. BEEP got to RFC status in record time because it was proposed by an IETF insider. Problem is that the IESG does not have much influence with the people in the Web Services world which is where all the interest in XML based protocols is at the moment.

Most of the people I see at W3C and OASIS Web Services meetings have no IETF experience at all. Of those who do, none are IETF insiders and so an endorsement by the IESG does not have much force.

For BEEP to take off it really needs an endorsement by one of the heavy hitters of the Web Services world which basically means IBM. Microsoft or Sun. I don't think that is very likely because everyone knows that there is a lot of work to be done to make Web Services work and there is simply nothing to be gained by putting BEEP on the critical path. People are more interested nailing down WS-Security, SAML, XKMS, geting WSDL to work and such.

Re:Not so impressed

[robla]

Hmmmm...."Zeinfeld". Pardon me for asking, but which HTTP author are you? I've met most of you, and like all of you, but quite frankly, at least one of you has a conflict of interest when it comes to opining on BEEP. You're entitled to an opinion, but I think your identity is important data in this case.

As for the IETF being an old boys/girls club...well, yeah. Welcome to politics. I think the IETF has generally been a reasonably equitable old boys club.

As for Marshall Rose, he kinda earned his way into the old boys club. He was co-author on many RFCs, including POP3.

But enough about the politics, let's talk about BEEP on its own merits. Maybe the reference implementations are crappy (I can't speak one way or another on that), but the spec looks pretty solid to me. As I said before, it fits the requirements that we had when we were shopping around for a base protocol for RTSP.

At any rate, I encourage people to do some digging themselves on this. I can't say that my experience is overly deep in this area, and I certainly haven't tried to design a protocol on top of BEEP, but based on what I know, BEEP looks like a pretty good idea.

Re:Not so impressed

[Zeinfeld]

second, i have to praise this post for the way it seemlessly blends a little bit of truth, a little bit of wisdom, along with a healthy amount of ignorance and untruth. (my favorite untruth: marshall as the OSI guy. the actual truth: marshall killed OSI in the IETF on November 4th, 1993 with the "roadkill in motion" speech.)

Ah so you are claiming that the Marshall Rose who wrote The Open Book : A Practical Perspective on Osi is a different Marshall T. Rose. No sorry, Marshall you made a major technical contribution to OSI, or at least he claimed to have done so on the jacket cover of the copy I read. You did not 'kill it' at the IETF, OSI was killed in the marketplace long before 1993. The speech had the impact it did precisely because you knew the OSI stack.

on the technical front: the beep specs use DTD not schema -- there aren't any schema fragments in either rfc3080 or rfc3081

Well Duuuhhh read what I wrote. I checked the RFC just before posting and saw that. Schema may be 'ugly' as you put it, but none of the major XML programming platforms are based on DTDs, they are all based on schema. And BEEP is dead at Microsoft, Sun and IBM without schema, cold stone Deeeeeeaaaaadddd.

with beep, no one gets to argue those things any more (e.g., how to frame packets), instead they get to go off and presumably argue things specific to their application domain. (if folks want to understand the reasoning behind this, check out rfc3117.)

No, we have a five minute argument on whether to use BEEP or not, reject it and carry on.

the sacred working group, that you're so pissed off about me cutting into, is a perfect example. a bunch of guys focused on security issues, trying to write an application protocol. sorry, wrong skill set.

I wrote a fair bit of HTTP, I don't think that you have the right to go arround saying who is and who is not competent to write application protocols. If you want to get into a reputaion war you are going to loose this one. I think that BEEP is very naive when it comes to the problems that arose when it came to layering application protocols on it. I suspect that like LDAP and X500, by the time BEEP has been extended enough to be useful it will look like HTTP.

if you're unhappy that i stuck my nose in your business, then all i can suggest is you get more clueful in the application design space, so "the management" doesn't feel they have to go out and get you help. particularly help that you don't like, and especially help that would rather be doing other things with other people

I only attended the one SACRED meeting and your comment on 'the management' is quite illustrative of the George W. Bush style crony-standards process the IETF is becomming notorious for.

finally, as far as web services go, well, let's just say that those guys could learn a lot from what xerox did back in the early 80's. in a few years, they may actually have something that works half as well as what xerox did... /mtr

Standard old fart response 'we did it all twenty years ago sonny', 'and we did it better'. Yeah and you should have seen the anti-gravity machines we made twenty years ago.

I don't much care for the arrogance of the IETF 'management' as you call them. I certainly don't appreciate folk who think that they have the right to make the type of off-hand blanket pronouncements on other people's work that you and they make habittually without backing it up. Your Xerox comment is absolutely typical of IETF old fartism, you want to have the right to be dismissive, you don't have the technical arguments on your side. So instead of detailing a real technical issue you allude to an earlier system, the more obscure the better. The message: 'I am too important to have to justify my comments but I believe that you are not competent to work on this problem'.

Your comment on Web services only illustrates that you really don't understand what is going on, what people are trying to achieve or why previous efforts such as CORBA failled. I am not going to explain why or how Web Services are different because I am faaaar toooo important. I may not be old enough to have achieve old fart status but I can certainly play the part on the net.

Still you are right on one battle, if the IETF is to regain some relevance at the upper end of the protocol stack adopting XML as the way to author RFCs is the only way forward. However the IETF is going to have to do a lot more than produce its documents in a format that does not look like utter crap from a teletype before I am going to take any standards there. I want a genuinely open and geuinely transparent process. I want standards groups to complete in 18 months, not 10 years - and yes it is possible, the SAML group I was a member of developed the basic specs which have been adopted as the basis for liberty in 18 months.

Re:Not so impressed

[Zeinfeld]

Hmmmm...."Zeinfeld". Pardon me for asking, but which HTTP author [ietf.org] are you?

I am not one of the Editors, I am listed as a principal contributor before the group members.

I've met most of you, and like all of you, but quite frankly, at least one of you has a conflict of interest when it comes to opining on BEEP.

Not so in my case, I was not big into HTTP-NG, I was not involved in DIME or Multiplex. I am not HFN which should be bloody obvious if you know him. However I am a friend and my company does a lot of work with his (Microsoft), but I also do a lot of work with Sun.

As for the IETF being an old boys/girls club...well, yeah. Welcome to politics. I think the IETF has generally been a reasonably equitable old boys club.

It is reasonably equitable until you actualy try to get things done. Look at DNSSEC, they have been trying to deploy the thing for ten years. In the meantime the .com zone has grown so large that the scaling bug in the design of the NXT record means that the protocol is not going to be deployed by the registrar for those zones without modification. A fix has been proposed for two years and has been agreed on the list. But the fix is currently blocked by secret discussions between the 'DNS Directorate' which is a closed and entirely unaccountable group. The not so obvious strategy being to delay OPT-IN until the DS flag day has gone through and then reject OPT-IN as requiring a flag day. Just about the only reason that strategy makes sense is if the IESG wants to delay DNSSEC for at least a year and risk a lawsuit.

On IPSEC the group is currently facing two problems that have been known for five years. First IPSEC was deliberately designed to make it impossible to use through NAT devices. I remember the comments in the WG at the time, people took it as a badge of honor to sabotage NAT - pretty lame when one of the major reasons NAT is needed is a design blunder by the IETF itself when it choose the IP address space to be smaller than the human population of the world - and on reliable authority this was done to allow an address to fit into a single register.

The other problem with IPSEC is that they ignored the problem of PKI and so we now have a wildly successful deployment of IPSEC for VPNs, a function which it is particularly ill suited for, and almost no extra-net or genuine internet deployment.

At any rate, I encourage people to do some digging themselves on this. I can't say that my experience is overly deep in this area, and I certainly haven't tried to design a protocol on top of BEEP, but based on what I know, BEEP looks like a pretty good idea.

I haven't tried to implement a protocol on top of BEEP either, and I am even less likely to do so given Marshall's outburst earlier in the thread.

My point about old boys network is that I have no confidence that BEEP is ready for prime time. I will not be confident until someone actually does build a protocol on top and demonstrates the bugs have been ironed out. HTTP looked pretty straightforward until people tried to use it.

The IETF is in severe danger of becomming irrelevant, as attendence at recent meetings is deonstrating. The much bigger problem with the IETF is that they really don't feel any responsibility to anyone but themselves. They take their role as supreme Internet standards body for granted.

The average IETFer has pretty much a slashdot mentality. They have plenty of contempt for technology have nots while curiously tollerating computing environments from the stone age. MIME is an IETF standard but send an attachment to an IETF mailing list and the old-fart faction complain that their antiquated mailer can't handle it. Send HTML mail and they will have appoplexy. (And if you think HTML is a bad idea I presume you read Slashdot in the no-graphics, plaintext mode).

The average IETFer does not think that they have an obligation to the community of Internet users or the vendors that support them. So holding up a protocol for a year while an AD who does not have a clue about security 'gets comfortable' with a security modification advocated by the Security ADs, well that is not a problem. Designing a protocol that breaks through NAT, well that is not a problem, and so on.

I am certainly not antithetical to the aims of the IETF. However there is a reason why I helped take the Web standards out of the IETF and set up W3C, there is also a reason why I am currently helping take standards from W3C to OASIS. There is a free market in standards bodies and the IETF is in dire need of a severe kick up the pants.

Re:Not so impressed

[Snorbert Xangox]

Ah so you are claiming that the Marshall Rose who wrote The Open Book : A Practical Perspective on Osi [amazon.com] is a different Marshall T. Rose. No sorry, Marshall you made a major technical contribution to OSI, or at least he claimed to have done so on the jacket cover of the copy I read. You did not 'kill it' at the IETF, OSI was killed in the marketplace long before 1993. The speech had the impact it did precisely because you knew the OSI stack.

Sheesh, how on earth is knowing about OSI before publicly trashing it a crime? For people trying to make sense of the burgeoning pile of manure that was the OSI movement in 1990, MTR's books were a way to at least work out WTF OSI was on about, because it wasn't a picnic extracting this information from either: (1) the standards (unless you had a whole lotta time and masochism on your hands); or (2) the vendors (who just said "this is what you have to buy to be futureproof, shut up and buy it".)

MTR may have attempted to inject some sanity into the OSI standards process at some stage, but given that many governments were STILL treating OSI as manifest destiny in the early 90's ("it's definitely the future, we just don't know when it's turning up in a usable form"), this could be seen as an effort make the best of a bad lot on behalf of the people who were apparently going to get it inflicted on them, failure in the marketplace or not, rather than wholehearted support for everything OSI stood for. Jump in my time machine and take a job as an IT Officer in the Australian Public Service in the early 90's, then you might see this point a bit clearer. Back then, we honestly thought we were condemned to live with this crap forever, courtesy of the Government OSI Profile (remember that?). MTR's books were very very useful for arguing with our management in an informed manner about why OSI would not actually solve our problems. And it was pretty obvious if you actually read his books that MTR was not a big cheerleader for OSI.

Annoyed

[KidSock]

Well thanks. Now I'm ticked. The ./ post made this sound very interesting taking about framing and low level down to the metal "network programming". So I followed the links. I get 5 minutes into the spec and what do I find? It's another telnet based protocol that is largely XML. I'm going to puke if I see another stupid XML networking protocol. I can't believe I'm the only one who sees how stupid it is to format all network traffic in a verbose text format like XML. Here's 448 byte "hello" example.


C: MSG 0 1 . 52 158
C: Content-Type: application/beep+xml
C:
C: <start number='1'>
C: <profile uri='http://iana.org/beep/TLS'>
C: <![CDATA[<ready />]]>
C: </profile>
C: </start>
C: END
S: RPY 0 1 . 110 121
S: Content-Type: application/beep+xml
S:
S: <profile uri='http://iana.org/beep/TLS'>
S: <![CDATA[<proceed />]]>
S: </profile>
S: END

Re:The BEEP community is strong and gettng stronge

[binkley]

I wrote a BEEP implementation for Perl when I was working at Enron Broadband Services, but my group was never able to get the lawyers to let give it GPL licensing and release it publically.

At the time, it seemed like our choices for letting partners have some say in network routing through our backbone were SOAP or BEEP, and we favored BEEP because of partner pressure from Sun against all things favored by Microsoft (go figure). Eventually, we used simple XML messages rather than an entire application layer.

Early on, BEEP wasn't a difficult protocol to implement; however as time passed, it grew more and more complex, until it maintaining BEEP in a closed-source environment outweighed any benefits. At that point we switched to simple XML messages.

BEEP isn't a bad protocol at all. It is a little over-designed: as a fan of eXtreme Programming, I'd have preferred that smaller versions of the protocol get wider use and more feedback before being expanded.