Slashdot Mirror
Will BEEP Simplify Network Programming?
hensley writes "There is a (not quite) new effort by the IETF to standardize
a framework for network applications, called BEEP, the Blocks Extensible Exchange Protocol. Standardized in RFC3080, it takes care of all lower level tasks an application level protocol has to like framing, authentication and capabilities negotiation in a modular and lightweight way. In the current issue of Internet Packet Journal (a quite nice and free-as-in-beer technical publication by Cisco) is a well written Introduction to this framework. Why isn't anyone adopting this protocol besides some Java libraries like beep4j and PermaBEEP and a C library called RoadRunner. I couldn't find any applications based on this protocol, regardless of it's promised capabilities. Is everybody still inventing his own application layer protocol?"
I used beep in a programming course in High School. Unfortunately, every time I used it people shouted "nosound!".
I've also been disappointed with the lack of uptake on BEEP. It's a very cool concept.
BEEP is Blocks Extensible Exchange Protocol (RFC 3080). More details can be found at here.
When we were designing RTSP, we looked for something like this, and at the time, the options weren't very appealing. We ended up using HTTP as a quasi-base protocol. I think it was the best solution at the time, but had BEEP been available, we'd have used it in a heartbeat.
This way RoadRunner goes really fast with
MEEPMEEP!(thp-thp-thp-thpppp) this was all just a funny lie
Mordor...a magical, mythical land where women are more rare than dragons--but where every man would rather find a dragon
The most interesting use for BEEP that I've found so far is iCalendar's Calendar Access Protocol (CAP).
It strikes me that this protocol doesn't handle multiple connections to different servers/peers aka Kazaa/winMX...
There's nothing stopping an app from having multiple BEEP connections open at once.
no authentication that the data being transferred is accurate aka Freenet
It's impossible to do that in a generic way, so BEEP doesn't do it.
By and large, yes... it's a symptom of the needs of applications being so varied.
(warning: blatant plug follows) For what it's worth, however, I've developed mine over the course of three years and a dozen or so projects, to the point where I think it's pretty mature and useful; it's open source, and portable to most environments, although the IETF has of course never heard of it... ;^)
I don't care if it's 90,000 hectares. That lake was not my doing.
Why do you assume we should have ONLY ONE application level protocol? There is a REASON that the Internet is based on IP, and not, say, TCP only. or UDP.
Or GRE.
Or anything else we have yet to invent.
Because we don't yet know the best way to use the network.
Maybe we haven't adopted BEEP because you don't just 'create' a standard by declaring your stuff is better. Maybe it's because peopel ALREADY know how to do regular socket programming.
Who knows.
BEEP came to life as BXXP at Invisible Worlds a few years back, where I was an executive. Our goal for a while was to use it to federate heterogenous search engines. Invisible is no more, but the protocol lives. A lot of thought by people with a great deal of Internet architecture went into it... but I'm not about to pass judgment on whether or not that's a good thing!
Ironically, the layer at which my application specific needs are met? Yes, I am still writing that particular layer.
If we don't need stateful connections, why are most of the protocols you mention using a stateful connection?
Hanging my head in shame, I'm one of those "still inventing his own application layer protocols". ASN.1 and RPC were also supposed to save me from doing this. Lately, I've found I've been implementing my own protocols using the concept of netstrings to suit my admittedly low-level needs better. Sadly, as XML and its derivatives mushroom in complexity, I find them less appealing.
I think it's pretty funny that the only message in the July mailing list archive for Road Runner is from that spam-bitch, "Christine Hall."
2 002-July/thread.html
List:
http://lists.codefactory.se/pipermail/roadrunner/
(codefactory is the company working on GtkHTML and Mr. Project, btw).
Napster-to-go says "Fill and refill your compatible MP3 player", which is a lie. It's not MP3. It's WMA with DRM.
I looked at BEEP early on, when it was called BXXP, for doing RPC calls. At the time we decided that SOAP over HTTP was the better option for our application. I think that the world has moved on since then, today I would design using REST principles and HTTP, and take advantage of Apache. For applications requiring something more 'stateful' I would probably choose Jabber, which is reasonably well established, over BEEP. BEEP will have a hard time competeing with established extensible protocols which do, if not an excellent job, a good enough job for most applications.
Damn, for a minute there I thought it was a post about the Beeb (the most wonderfull piece of computer hardware ever to come out of the UK, also known as the BBC Micro to the un-initiated).
Oh how it takes me back. The days spend on the old Beeb, learning to program (in basic!) and playing those wonderful old games. I feel quite nostalgic about the old Beeb.
Ah shit! Did I just admit that I actually learned to program on a BBC Micro? Dammit. There goes all my credibility. Nobody on Slashdot will take me seriously anymore! I can just hear the comments: Shutup Grandpa... go play with your 8 bit calculator Granpa... I have a question about Basic Granpa... what does "goto" mean Granpa...
People couldn't type. We realized: Death would eventually take care of this.
BEEP's main proponent, Marshall Rose was one of the main wheels in the OSI project. So much of the initial buzz came from his name alone. People were talking about the protocol before they read the drafts (oh yes that is normal for the IETF).
I do have a bunch of quibbles technically. First using XML is a good idea, Using the obsolete SGML DTD mechanism to describe the protocol sucks. I think Marshall started to suplement the DTD with schema fragments but that makes things worse, not better, we now have two specs in one document, the schema version which is what people will implement and the DTD version which is normative.
The other problem is that SGML is a real baaad choice for an encoding at that level. The main complaint about http is that the encoding is too verbose leading protocol exchanges to require multiple round packets instead of one. BEEP does not address that problem.
Politically BEEP has bigger problems, first being that IETF does not have as much influence as it might appear when it comes to promoting new protocols. There haven't been very many IETF protocols that started in IETF process and took off like wildfire in the past ten years. HTTP took off and was brought into IETF process, same with TLS (SSL). Most successful IETF protocols had a userbase before the working group was formed.
The problem with BEEP is that Marshall did not start with a constituency who had a problem that BEEP was the solution for. Instead he wrote the protocol and then went off looking for consumers. So we start to see Marshall popping up at random in working groups like SACRED peddling the BEEP Kool-Aid. The problem being that if you are doing a researchy protocol like SACRED the last thing you should be doing is layering it on someone else's research.
After this happened a few times Marshall started to alienate folk like myself who might be interested in BEEP as an option but certainly were not going to allow him to insert himself onto our critical path.
The other problem is the nature of the IETF these days. The problem is that they talk a good talk about being open and such, but it is really an old-boys club. The old-fart faction is strong on the IESG and IAB, they have known each other for 20 years and they don't want anyone messing with their turf.
In theory the IETF process is open. In practice there are a bunch of shadowy cliques who make the real decisions in private. BEEP got to RFC status in record time because it was proposed by an IETF insider. Problem is that the IESG does not have much influence with the people in the Web Services world which is where all the interest in XML based protocols is at the moment.
Most of the people I see at W3C and OASIS Web Services meetings have no IETF experience at all. Of those who do, none are IETF insiders and so an endorsement by the IESG does not have much force.
For BEEP to take off it really needs an endorsement by one of the heavy hitters of the Web Services world which basically means IBM. Microsoft or Sun. I don't think that is very likely because everyone knows that there is a lot of work to be done to make Web Services work and there is simply nothing to be gained by putting BEEP on the critical path. People are more interested nailing down WS-Security, SAML, XKMS, geting WSDL to work and such.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@dbc.mtview.ca.us and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Apache/1.3.20 Server at beepcore.dbc.mtview.ca.us Port 80
Way to go, BEEP! I won't fuck with you anymore, because I'm no pedophile, OK?
BEEP could be the best new acronym since SCSI
"Do I have a scuzzy hard drive?! What kind of person do you think I am?"
Free cell phone tracking
No by definition the aplication layer protocol provides the application with access to the network stack through an API.
Um... You remember what API stands for right? That'd be "Advanced Programming Interface" or some variation. If you'd ever had to code using one of these, particularly a "new" one, you'd realize that "Interface" is a very relative term.
So, is it easier to just use sockets as your application's API to the network? In many cases yes... Interestingly enough, TCP already provides multiple streams of cummincation, and port numbers are supposed to tell you how to behave when listening or connecting. Adding another layer on top of this to do essentailly the same thing seems redundant. The only benefit I can see to this might be effects on performance(for wireless) or the ability to "sneak through" firewalls. But at what cost complexity and efficiency?
That said, if anyone ever comes up with a useful presentation layer, I'd be happy to give it a whirl. (XML has been kind of fun that way...)
for complex, two-way, stream-based communications.
I believe TCP already solves this problem, perhaps herein lies the lack of excitement...
There are a couple of (significant) differences between BXXP and BEEP, but no fundamental changes. And the name change came about as a result of commentary from folks at the IETF session on the topic. It appears they were tired of the eXcessive use of the letter X, that and BEEP is easier to say.
The BEEP activity I noticed most recently was on a proposal for syslog-reliable, and I believe there are some intrusion detection things using it in the academic space.
There is also an excellent white paper (on beepcore.org I think) on the transport of SOAP using BEEP rather than HTTP, and it really is a pretty sweet fit, though early on they were in competition for mindshare at the IETF and neither wanted to be absorbed conceptually into the other.
O'Reilly has published a book on BEEP by Dr. Marshall Rose, the principal BEEP designer. It's good.
Sweetums
------------------------
Jack not name, jack job!
to erase the TCP connection negotiation overhead.
Isn't this why UDP and other protocols exist on top of IP (or other routing layers). So you can forgo the overhead that a fancy session layer (like TCP) incurs.
It seems to me that most of these new protocols are just trying to get a free ride through existing firewalls... Why can't we all just use IP the way it was meant to be used? (and move to v6 to alleviate addressing pinches)
I know that. That was the point.
The point was, we don't NEED a single protocol, we already have one, it's called IP.
No state is maintained from one ftp connection to the next.
No state is maintained from one telephone call to the next.
No state is maintained from one quake session to the next.
By that definition, all protocols are stateless.
The ICAP protocol is based on the IMAP4 protocol. As such, it has the assumption of a client/server relationship. It also has state assumptions that are not necessarily valid over time, and BEEP appears to not support the concept of nested transactions (the IMAP4 and ICAP protocols are biased toward set encapsulation -- that's why there are all the irritating nested parenthesis -- which lends a strong bias toward stack based interpretation of requests: transactions).
Since BEEP is intended as a peer-to-peer connection oriented protocol, it's not ever going to really be practical as an ICAP transport.
The main issue that a server deals with is a shared resource that belongs to the system, rather than participates as a peer. This becomes an issue when you want to schedule use of a confernece room or interview office, where these resources, unlike human beings, don't have a machine which participates in the process as a peer. It's not possible to totally distribute such a system, since the peers are not the only mutually contended resources (if "Bob" drops out, then you can have a request outstanding, but not accepted; no matter who else attends your meeting, though, the conference room *must* "attend").
As far as a peer-to-peer workgroup calendaring service, it *may* be possible to use MIME encapsulated calendar update notifications, similar to the way Microsoft Outlook can be a transport for Microsoft Schedule information (the message type involved lack standardization).
However, such an approach would have the drawback of "netsplits" (in the IRC sense) redulting in shared resources ending up with conflicting scheduling (just like "operator" status in an IRC network following recovery from a "netsplit").
In general, calendaring is a much more complicated problem than people tend to give it credit for being.
Right now, the best protocol bet for Calendaring is LDAPv3 with the "persistant search" or a similar notification mechanism (e.g. LTAP), except not as a proxy, and mandatorily integrated into the LDAP server itself (due to replication requirements).
One place BEEP *might* be useful is data replication between peered servers. The primary reason it could be useful here is that the entity relationship is *peer*, not *client/server*.
-- Terry
Then why do all these long distance calls show up on my bill to places like Texas, Tennessee, Florida, etc?
--
Evan
"$30 for the One True Ring. $10 each additional ring!" -- JRR "Bob" Tolkien
so, um, does the part of the program implementing UDP communication also responsible for routing and network interface management? If that's the case, not only are you using a really funny UDP implementation you've written half an OS around it. that's probably not the most efficient way to do things.
BEEP is NOT "TCP over TCP". Using BEEP instead of, say, sockets for implementing network application layer protocols is more like using Perl for your CGI scripts instead of C.
My point: Do you like mucking about with character arrays and pointers and making up your own regex implementations? Do you like having to worry about packet ordering algorithms when writing, for example, a client authentication protocol? Then use a standard TCP API. But if you like being able to specify your packet length with a funcion call then ship out your data without having to mung chop slice and dice all the while worrying about buffer overflows and packet framing, use BEEP.
It's pretty much a higher-level framework to create TCP-based application protocols. (higher-level than using the OS's standard socket API's)
If that sounds like "TCP over TCP" maybe you think Perl is "C on C"???
OK now that I've ranted I might add that I've never written a network application using C, nor have I designed an ALP using BEEP, so I could be full of shit. At least I read the article.
I'm done with sigs. Sigs are lame.
UDP is great for DNS since queries are small and the overhead of using TCP is large compared to the data exchanged. UDP is also great for things like cache servers using ICP, since then you only need one socket descriptor that can serve however many sibling/child caches you have.
From RFC 768, entitled "User Datagram Protocol": The RFC is two pages long (as opposed to RFC 793 -- TCP -- which has 84 pages) and explains that it is simply a packet of data that does nothing but carry data. It does no handshaking in the protocol, and does not establish a connection, and is thus connectionless.
BEEP's an impressive protocol framework with even more impressive implementations.
I'm using it in a burgeoning open source project because of it's ability to multiplex bidirectional communication channels in a transparent fashion. Other features such as dynamic client/server roles, authentication, and channel encryption are just icing on the cake. The less I have to muck around with protocol state details the better!
When I first started looking at BEEP I was impressed by the spec but I was suspicious as to the quality and breadth of implementations. After looking through the high level Java abstractions and more specifically BeepCore-Java I was able to throw together a workable protocol that's proven to be extensible and quite robust in just a few days.
The BEEP community is alive and well!
There are a number of opensource BEEP implementations:There's an excellent IRC channel at OpenProjects with the kitschy name #beepnik.
There's the obligatory O'Reilly book, BEEP: The Definitive Guide.
But the best source for information is the Beepcore.org site which has, among other things, an excellent whitepaper on the justification and design of the BEEP protocol.
Good to see that this protocol is as bandwidth efficient as most of the rest of the IETF protocols. Ahem.
When I read the headlines, I thought it said "Will BEER simplify network programming"? I know the answer to that one, and I can prove it over and over that BEER simplifies any kind of programming.
It's the debugging that demands sobriety....
Because -- for English speakers, anyway -- "missing" vowel sounds are only inserted between letters that cannot be pronounced together. Thus the natural pronounciation of "SCSI" puts the vowel between the C and the second S. The "cs" combo is rare in English (we use "x" or "cks" for that sound) but "sc" is common (scare, scarf, scam, score, scum ...).
If the coiners of the acronym really wanted it pronounced "sexy", they should have come up with some "E" word between "Small" and "Computer".
BEEP is pretty clearly pronounced how it sounds. Just so long as nobody confuses it with ping (or PNG, for that matter).
-- Alastair
Hmm.. Perhaps that was the real question. None of this "how many roads" idiocy! :-)
Well thanks. Now I'm ticked. The ./ post made this sound very interesting taking about framing and low level down to the metal "network programming". So I followed the links. I get 5 minutes into the spec and what do I find? It's another telnet based protocol that is largely XML. I'm going to puke if I see another stupid XML networking protocol. I can't believe I'm the only one who sees how stupid it is to format all network traffic in a verbose text format like XML. Here's 448 byte "hello" example.
/>]]> />]]>
C: MSG 0 1 . 52 158
C: Content-Type: application/beep+xml
C:
C: <start number='1'>
C: <profile uri='http://iana.org/beep/TLS'>
C: <![CDATA[<ready
C: </profile>
C: </start>
C: END
S: RPY 0 1 . 110 121
S: Content-Type: application/beep+xml
S:
S: <profile uri='http://iana.org/beep/TLS'>
S: <![CDATA[<proceed
S: </profile>
S: END
We've already got PING, now we've got BEEP. So when do we get SHOOT and EXPLODE?
If anyone knows what I'm talking about, I'll be pleasantly surprised!!!
"Information wants to be paid"
I was working in the IDWG (intrustion detection working group) and was on team that started implementation of the BEEP version of the intrusion detection exchange protocol. It was certainly less work getting the BEEP version functional than the hacked-HTTP protocal that IAP (the first version of the protocol) was using. It also gave us some benefits vis a vis pluggable autentication/encryption. Most of the people involved got kinda busy, so I'm not sure how far the project has gone. I think we still have a webpage up under java-idxp on Sourceforge.
Of course, it used to be calles "SASI" for "Shugart Associates Computer Interface" (IIRC). And Atari had a variant called "ACSI", for ... Oh, you work it out. It had a funny D connector that you couldn't get when you wanted to make up cables, 19 pins or something.
I wrote a BEEP implementation for Perl when I was working at Enron Broadband Services, but my group was never able to get the lawyers to let give it GPL licensing and release it publically.
At the time, it seemed like our choices for letting partners have some say in network routing through our backbone were SOAP or BEEP, and we favored BEEP because of partner pressure from Sun against all things favored by Microsoft (go figure). Eventually, we used simple XML messages rather than an entire application layer.
Early on, BEEP wasn't a difficult protocol to implement; however as time passed, it grew more and more complex, until it maintaining BEEP in a closed-source environment outweighed any benefits. At that point we switched to simple XML messages.
BEEP isn't a bad protocol at all. It is a little over-designed: as a fan of eXtreme Programming, I'd have preferred that smaller versions of the protocol get wider use and more feedback before being expanded.
--binkley
I see that you are talking about the Web Browser as an application, as looking at a series of HTML pages.
I suppose it all depends on your point of view. When I transfer a file via HTTP, it's just as stateless as, say, FTP.
Auto-reconnect logic, as you call it, would work just as well with HTTP as it does with FTP. It's a function of the client, not the protocol.
Thanks, this is very interesting! I have a page where I've been messing around with some thoughts about XML and how it should have been - minXML gives me some more material to think about.
Personally I prefer having the structure represented by tags, and data only in attributes. It makes it a little easier to represent certain categories of information (IMHO).
i-name =twylite [http://public.xdi.org/=twylite], see idcommons.net
XML is to the presentation layer as TCP is to the session layer. It strikes me that XML isn't the end all be all (just as TCP isn't) of its layer. However, it is a very compelling implementation against which all other implemenations can be measured. I expect great strides in the presentation layer in the next few years primarily because of XML.
... (think of RPC, for example)
This automatically allowed for sessions
Which is what I thought TCP was all about... though there is overhead of course.
it supported asynchronious request-reply paradigm
I didn't manage to get through the whole specification, but this did seem to be one of the useful things in there... Of course, how useful is this really? It might be somewhat nice to have a tool I can use in my application to create an RPC mechanisms or the like, but what I'd much rather have is an RPC mechanim already built for me. (Like RMI in Java for example)
Don't get me wrong, BEEP will probably be useful in some applications, but I don't expect it to become a "canonical" implementation like XML or TCP.
Basically, I see a need for some level of tools to help in creating generalized TCP application protocols. In fact every TCP programmer I've met that does this for any length of time has created (or modified) their own toolset for this end. The problem with BEEP seems to be that it is a little too heavy and complex for many simpler applications. Also, the application protocol programmer, usually spends a lot more time writing code to communicate with legacy systems then coding new application protocols. I don't see a lot of functionality in BEEP for dealing with legacy issues.
Why can't we all just use IP the way it was meant to be used?
According to the beepcore FAQ, it's because of NAT and server pooling - you're not guaranteed that multiple requests for connections between two machines actually will be between the same two machines.
As you mentioned, IPv6 would have prevented the problem. It's just a decade too late.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)