Slashdot Mirror
Do We Need Another OO RPC Mechanism?
Paul68 queries: "I am looking for an RPC mechanism for a project. Granted, there are many to choose from, yet there seem none that meet my requirements! When one has toyed with the requirements the solution generaly becomes obvious. So, yeah sure, I can set out and create the next RPC mechanism, but it is a lot of hassle. But does the world need yet another OORPC, or have I simply not looked in the right corners?"
Does anyone have any options that I may have missed?"
"My requirements are:
- object oriented
- extensible
- platform independent
- supports signatures for integrity and sender checking
- supports privacy of the message contents (i.e. encryption)
- time sensitive: I should be able to detect a dead server and do failover while the user is waiting for the response
- bandwidth efficient, as I am looking to deploy it in wireless environments
Does anyone have any options that I may have missed?"
And ask yourself, "What am I really trying to do?"
How much overhead do you expect XML to incur? Or more importantly, what do you expect your data/(data + XML) ratio to be? With large amounts of data, the relative amount of XML is relatively small.
Of course, if you are sending many packets, each with small amounts of data, perhaps strewn across many XML tag sets, then the XML "noise" ratio may be high. How about gzipped XML-RPC ?
10b||~10b -- aah, what a question!
...that bugs me is that there's no way to make a call that will return multiple pieces of data over a long period. Like, I want to connect to a server and say "tell me every time the price of IBM stock changes". I can use SOAP for most of a trader type UI but for streaming the data back to the client, you have to use another protocol (unless I'm missing something).
The dude asking the quesiton here is, I think, a little too worried about overhead - the overhead of XML is fairly tiny, and the overhead of TCP/IP isn't that great either - if both of those really are too much then you're probably on your own as there isn't likely to be a "general purpose" solution that will work for you.
- Steve
It's easily capable of representing objects, platform independent, encryptable (via SSL), compressable (via gzip [and probably SSL as well]), and textual.
The advantages of being textual in your protocols is well laid out in Eric Raymond's book The Art of Unix Programming. He even treats it as a case study.
Definition from dictionary.com: 1. Capable of being extended or protruded: an extensible tongue; extensible tables. 2. Computer Science. Of or relating to a programming language or a system that can be modified by changing or adding features.
You might want to take a look at the Neutral Message Language, NML. Developed at the Intelligent Systems Division of the National Instute of Standards and Technology is was intended from the start for use in real-time/time critical situations. I know that it currently has support cor C, C++ and Java.
I think you'll find that in practice a UDP-based solution isn't going to make your life any easier than TCP. You may want to use TCP in conjunction with an external timer to fail the connection well before TCP gives up though. If you tend to do multiple RPCs to the same host the overhead of TCP should be minimal, and you get a lot of functionality (retries, backoff, etc.) for free. There's RFCs for transactional TCP which solves some of the overhead problems for small exchanges but I don't think they're widely implemented.
If you do end up writing your own UDP-based protocol, take some time to study TCP as well as other UDP based protocols and make sure you write something network friendly. There are a lot of naive protocols which fail spectacularly in less-than-perfect conditions.
Just because TCP provides a sequenced, reliable byte stream with no need to transfer data past connection setup does not mean you can't use it to implement your application level requirements. Simply implement a null request with the client demanding a response within X seconds or you declare the peer dead (or effectively dead) and try another (yes this is a little simplistic and can result in livelock under heavy load but adaption, making X depend upon the peer's load and possibly dynamically measured response time, can help here).
RPC causes untold security/authentication headaches and is often hard to program with besides.
See ESR
char a[]="lbiitgt l e \n\n\0";main(){for(char*c=a; *(short*)c;c+=2){putchar(*(short*)c);}}
If you arent scared of 'scripting' languages,/ twisted
http://www.twistedmatrix.com/products
Haven't used it myself yet, but from the specs it seems to cover everything you could need.
If I had a DeLorean... I would probably only drive it from time to time.
The problem with RPC is that, to be useful at all, it has to be used where the function-call abstraction fails. In inter-process communication and (more so) in network communications, there are too many failure modes that just don't fit that abstraction, but that a reliable application needs to handle anyway.
The whole point of RPC is supposed to be that the code invoking them looks just like regular function calls. To be reliable, though, they need to be decorated with so much error handling junk that any such benefit is usually lost. You're better off with explicit message passing and a documented wire protocol. You need the latter anyway to have a debuggable application.
It has been a long time for me, but CORBA might fit the bill.
+++ UGUCAUCGUAUUUCU
Well I reccomend the good old time SunRPC. ;) There are other ways of doing OO..
It doesnt have 1. though that depends on what you mean by
"object oriented". I sense you want do export your C++ objects
It is somewhat lacking in 4 and 5 , though GSS-RPC will give you both. It is also not that difficult to implement authentication providers for sunrpc.. And for those that hate the portmapper, you don't have to use it. Another + is that it's small and fast. And you can do rpc over unix domain sockets, easing the pain to create custom protocols for interproces communication on the local box.
Effective dead connection detection and failover is a Hard Problem in general. Asking for a general-purpose RPC mechanism that solves this problem is asking quite a lot. Perhaps it would be worth re-examining the requirements for the system that lead to asking for failover, and considering what other solutions to those requirement there might be other than robust failover.
RPCs are inherently insecure and introduces a balancing act of keeping the algorithms secure while keeping an eye on inputs and making sure nothing gets memory-leaked.
All this overhead also will take resources. So whats is it exactly youre working on that needs RPCs? Most networked applications Ive seen doesnt need RPCs, especially ones that are secure and efficient.
"Give orange me give eat orange me eat orange give me eat orange give me you." -Nim Chimpsky
You could probably implement what you want using
some type of message queuing server, like IBM Websphere MQ, or equivelant.
As a mental exercise, and to make sure I'm not talking out my ass, let's run down your list of requirements:
equirements are:
1. object oriented
Well, in the case of Websphere MQ, I'd say "yes" to this one, at least partially. There is a C++ based client library, IIRC, a (IBM specific) Java client library, and a JMS client. And even if the client libraries weren't OO, you could write OO wrappers for procedural calls more easily than writing your own RPC mechanism from scratch, I think.
2. extensible
Check. If you use pub/sub messaging instead of point to point messaging, even more so. And if you use a message type (MapMessage in JMS) that is based on name / value pairs, it's easy to extend your messages without breaking backwards compatibility.
3. platform independent
Not completely, but with both Java and C++ clients available, you should be able to support most everything.
4. supports signatures for integrity and sender checking
Not sure if it has native support for signatures, but a signature can always be added to a message as a property.
5. supports privacy of the message contents (i.e. encryption)
Again, using Websphere MQ as an example, it does support the use of SSL for communications, if that helps meet your requirements.
6. time sensitive: I should be able to detect a dead server and do failover while the user is waiting for the response
You're right, that's the tricky one. I think there are ways to achieve this goal using message passing servers, but it might take some work.
7. bandwidth efficient, as I am looking to deploy it in wireless environments
I think this requirement is met as well. Most messaging products support a message type that is nothing more than a stream of bytes that you interpret as you will... and even if you use a slightly move involved message type (MapMessage or StreamMessage to use JMS types as examples) you're still not carrying a lot of overhead.
Now #1-3 are no problem, #4 and #5 can be found, and #7 rules out anything XML-based. #6 seems to be the killer, as this rules out anything over TCP, and at that point the list gets pretty short.
Depending on just how "time sensitive" #6 really is, I do think you could come up with a solution using a message passing server. If you're able to use Java for the clients, JMS makes doing an RPC style messaging very easy, using the QueueRequestor and TopicRequestor interfaces. And even if you can't use Java, it shouldn't be to hard to cook up your own request / reply mechanism, built on top of the messaging system.
// TODO: Insert Cool Sig
I think that messaging is nearly always a better solution than RPC, especially for wireless where the network might not be very reliable. I've been playing with Spread recently. It looks pretty good.
It's true. There has been very little research done in RPC mechanisms. In November I attended three different conferences (two on distributed computing) to see what work was being done. Most research is in either building on top of CORBA or XML and web services. Infact, I was disapointed at the lack of research in the low level guts of RPC mechanisms.
.Net on the todo list.
The reason I attended the conferences is that I have spent a lot of time developing a binary RPC mechanism. The method I use can be used in messages (eg MQSeries) or over TCP or UDP or whatever other connection mechanism you feel like.
It fulfills:
1. object oriented
Its currently implemented in java, and allows basic data types, objects and streams to be sent between client and server.
2. extensible
Any object can be sent between client and server, and the programming paradigm is simple.
3. platform independent
Currently written in java, with implementation for
C++ and
4. supports signatures..
Not yet. This is an additional layer which could be quickly added into the communication stack. I haven't had a need for it yet, so haven't been interested in implementing anything like it.
5. privacy
as above. Simple solution is to use SSL as the connection layer.
6. time sensitive
simple timeouts are easy to implement, however this hints at other issues which is a big can of worms. ie Was the task completed on the server?
7. bandwidth efficency
pure binary with very small overhead
The negatives are that its not available yet. We are currently working out licensing issues. Sorry, this isn't open source. More information will be made available over the next month or two.
A small hint at what Colony can do is at www.livemedia.com.au
CORBA fulfills essentially what you describe (not sure about #4). However, the CORBA spec is fairly large and baroque, so you are going to eat some overhead in complexity (not necessarily performance!) if you choose CORBA.
RMI is also a nice spec. Contrary to popular belief RMI is not a wire protocol but simply a spec for an RPC interface. The standard implementation uses a specific wire protocol and is tied to Java. However, RMI/IIOP also comes standard and is transparently interoperable with CORBA. I'm sure if you wanted to you could write your own transport implementation.
XML-RPC and SOAP are not really OO RPC mechanisms (despite SOAP containing the word "object" in it). SOAP is a bloated compromise spec created by committee by a few large players in the industry to satisfy all their requirements, and hence does not really enforce any sort of object or typing system. Already I see people addled by XML-think outright proposing this protocols. If it fits your project go for it, but XML is not the panacea people think it is.
It's 10 PM. Do you know if you're un-American?
Keep in mind that if you are truly cross platform and your communications are pure binary, then endian-ness will be an issue. I had to deal with this once when passing messages from a MIPS SGI to an x86 PC.
It can be a real pain when the underlying communication protocol blindly assumes endian-ness is the same on every machine. I prefer text-based protocols because they alleviate that problem.
I am a big fan of CORBA, though I admit it has some issues.
:)
> 1. object oriented
CORBA is definitely object-oriented. Much better than XML and SOAP (SOAP is -NOT- OO at all). I love the fact that once you have a reference to an object, it does not matter where that object is.
> 2. extensible
Yes. There are many useful (and optional) services available for CORBA. You don't pay for what you don't use with CORBA, so if you don't require a Naming Service or a Transaction Service, you don't have to include it.
> 3. platform independent
This is where CORBA wins. It is platform, language and network agnostic. I don't pay attention to new technologies unless they have a way to interface with CORBA.
> 4. supports signatures for integrity and
> sender checking
Some ORBs can do CORBA over SSL for security, which can include certificate-checking if you wish.
> 5. supports privacy of the message contents
> (i.e. encryption)
See previous comment.
> 6. time sensitive: I should be able to detect
> a dead server and do failover while the
> user is waiting for the response
Yes, CORBA calls will fail with a COMMUNICATIONS_EXCEPTION, which you can catch and take action. Some ORBs let you configure the timeout.
> 7. bandwidth efficient, as I am looking to
> deploy it in wireless environments
CORBA is binary and fairly BW-efficient. Again, you don't pay for what you don't need.
On the downside of CORBA, there are issues with:
- complexity. It is definitely not for the beginner and has a large initial learning curve. If you write client-side stuff, it is heaven. If you write server-side stuff, prepare yourself. If you want to do CORBA-compliant fault tolerance or security, don't call me
- penetration. Not all ORB providers implement all the nifty services. Finding an ORB for your environment that provides what you need can be tricky (e.g. POA, Portable Interceptors)
- mindshare. So many people pushing alternate technologies with a few useful features and a promise of equalling CORBA, if only they get enough interest.
- Openness. The OMG process is lengthy and can only be crafted by consortium members. Maybe a W3C-style process would make it evolve more rapidly and get implemented quicker ?!?