Slashdot Mirror
Do We Need Another OO RPC Mechanism?
Paul68 queries: "I am looking for an RPC mechanism for a project. Granted, there are many to choose from, yet there seem none that meet my requirements! When one has toyed with the requirements the solution generaly becomes obvious. So, yeah sure, I can set out and create the next RPC mechanism, but it is a lot of hassle. But does the world need yet another OORPC, or have I simply not looked in the right corners?"
Does anyone have any options that I may have missed?"
"My requirements are:
- object oriented
- extensible
- platform independent
- supports signatures for integrity and sender checking
- supports privacy of the message contents (i.e. encryption)
- time sensitive: I should be able to detect a dead server and do failover while the user is waiting for the response
- bandwidth efficient, as I am looking to deploy it in wireless environments
Does anyone have any options that I may have missed?"
And ask yourself, "What am I really trying to do?"
How much overhead do you expect XML to incur? Or more importantly, what do you expect your data/(data + XML) ratio to be? With large amounts of data, the relative amount of XML is relatively small.
Of course, if you are sending many packets, each with small amounts of data, perhaps strewn across many XML tag sets, then the XML "noise" ratio may be high. How about gzipped XML-RPC ?
10b||~10b -- aah, what a question!
...that bugs me is that there's no way to make a call that will return multiple pieces of data over a long period. Like, I want to connect to a server and say "tell me every time the price of IBM stock changes". I can use SOAP for most of a trader type UI but for streaming the data back to the client, you have to use another protocol (unless I'm missing something).
The dude asking the quesiton here is, I think, a little too worried about overhead - the overhead of XML is fairly tiny, and the overhead of TCP/IP isn't that great either - if both of those really are too much then you're probably on your own as there isn't likely to be a "general purpose" solution that will work for you.
- Steve
It's easily capable of representing objects, platform independent, encryptable (via SSL), compressable (via gzip [and probably SSL as well]), and textual.
The advantages of being textual in your protocols is well laid out in Eric Raymond's book The Art of Unix Programming. He even treats it as a case study.
"Extensible"? What an obscure word to use instead of "extendible" which fits better. After all, have you ever extensed something? Then, have you ever extended something?
You might want to take a look at the Neutral Message Language, NML. Developed at the Intelligent Systems Division of the National Instute of Standards and Technology is was intended from the start for use in real-time/time critical situations. I know that it currently has support cor C, C++ and Java.
I think you'll find that in practice a UDP-based solution isn't going to make your life any easier than TCP. You may want to use TCP in conjunction with an external timer to fail the connection well before TCP gives up though. If you tend to do multiple RPCs to the same host the overhead of TCP should be minimal, and you get a lot of functionality (retries, backoff, etc.) for free. There's RFCs for transactional TCP which solves some of the overhead problems for small exchanges but I don't think they're widely implemented.
If you do end up writing your own UDP-based protocol, take some time to study TCP as well as other UDP based protocols and make sure you write something network friendly. There are a lot of naive protocols which fail spectacularly in less-than-perfect conditions.
Just because TCP provides a sequenced, reliable byte stream with no need to transfer data past connection setup does not mean you can't use it to implement your application level requirements. Simply implement a null request with the client demanding a response within X seconds or you declare the peer dead (or effectively dead) and try another (yes this is a little simplistic and can result in livelock under heavy load but adaption, making X depend upon the peer's load and possibly dynamically measured response time, can help here).
It is in early stages of desing and development, but is fully fucntional and comes with a complete toolkit and devloping framework, including a C++ interpreter (ROOTCINT), visualisation, object collection introspection and streaming interfaces from ROOT, autogenerated GUI for system and user C++ classes, object-collection monitor, cluster infrastructure using a hierarchical OO RPC system and visualisation with a GL renderer.
It is a complex design and still not a stable target, but it is already very usable. A a development environment, it provides for a very fast development cycle with advanced introspection and monitoring abilities. Autogenerated GUI also saves a lot of time.
-Kvorg
RPC causes untold security/authentication headaches and is often hard to program with besides.
See ESR
char a[]="lbiitgt l e \n\n\0";main(){for(char*c=a; *(short*)c;c+=2){putchar(*(short*)c);}}
If you arent scared of 'scripting' languages,/ twisted
http://www.twistedmatrix.com/products
Haven't used it myself yet, but from the specs it seems to cover everything you could need.
If I had a DeLorean... I would probably only drive it from time to time.
The problem with RPC is that, to be useful at all, it has to be used where the function-call abstraction fails. In inter-process communication and (more so) in network communications, there are too many failure modes that just don't fit that abstraction, but that a reliable application needs to handle anyway.
The whole point of RPC is supposed to be that the code invoking them looks just like regular function calls. To be reliable, though, they need to be decorated with so much error handling junk that any such benefit is usually lost. You're better off with explicit message passing and a documented wire protocol. You need the latter anyway to have a debuggable application.
It has been a long time for me, but CORBA might fit the bill.
+++ UGUCAUCGUAUUUCU
Well I reccomend the good old time SunRPC. ;) There are other ways of doing OO..
It doesnt have 1. though that depends on what you mean by
"object oriented". I sense you want do export your C++ objects
It is somewhat lacking in 4 and 5 , though GSS-RPC will give you both. It is also not that difficult to implement authentication providers for sunrpc.. And for those that hate the portmapper, you don't have to use it. Another + is that it's small and fast. And you can do rpc over unix domain sockets, easing the pain to create custom protocols for interproces communication on the local box.
#8. Performs sexual favors at a discount price.
Have you looked at Java RMI or Jini?
Do We Really Need Another Program? We have so many already... thousands... millions!
I do not think it is a good idea to try to create one "mechanism" providing options along the dimensions of security, performance, reliability, simplicity, portability, expandability, elegance, extensibility, etc, etc. If "separation of concerns" and "aspect oriented design" is still trendy, then I use that as proof by reference.
Larry
Effective dead connection detection and failover is a Hard Problem in general. Asking for a general-purpose RPC mechanism that solves this problem is asking quite a lot. Perhaps it would be worth re-examining the requirements for the system that lead to asking for failover, and considering what other solutions to those requirement there might be other than robust failover.
RPCs are inherently insecure and introduces a balancing act of keeping the algorithms secure while keeping an eye on inputs and making sure nothing gets memory-leaked.
All this overhead also will take resources. So whats is it exactly youre working on that needs RPCs? Most networked applications Ive seen doesnt need RPCs, especially ones that are secure and efficient.
"Give orange me give eat orange me eat orange give me eat orange give me you." -Nim Chimpsky
You should have a look at zeroC. I haven't used it but have had very good reports about it. Plus the licensing is purely volume based or free if your product is open source.
Something like:
COMMAND: getStockQuote
PARAM COUNT: 1 PARAM 1: MSFT END COMMAND
This is the basic principal of POP3 / SMTP and could be extended to allow for RPC. Each side would have to know how to execute the 'command' and extract parameters and then you'd have to deal with return types. Not difficult to come up with the idea, but more difficult to implement. Could be a fun project,....
Only 'flamers' flame!
Does slashdot hate my posts?
Why does it have to be OO? You could put OO wrappers around it on your own, no? It is hard for inter-language communications to be OO anyhow because OO is mostly about behavior wrapping data; but you will generally need "naked" data between two potentially different languages and/or systems for them to communicate. Making language-specific protocols is generally tougher because it reduces the number of users/testers/buyers, etc. than a multi-language multi-paradigm-friendly approach.
Table-ized A.I.
Have you looked at ICE? The developers are long time CORBA gurus who set out to fix some of CORBA's problems. It looks quite promising but then I haven't used it in anger.
You could probably implement what you want using
some type of message queuing server, like IBM Websphere MQ, or equivelant.
As a mental exercise, and to make sure I'm not talking out my ass, let's run down your list of requirements:
equirements are:
1. object oriented
Well, in the case of Websphere MQ, I'd say "yes" to this one, at least partially. There is a C++ based client library, IIRC, a (IBM specific) Java client library, and a JMS client. And even if the client libraries weren't OO, you could write OO wrappers for procedural calls more easily than writing your own RPC mechanism from scratch, I think.
2. extensible
Check. If you use pub/sub messaging instead of point to point messaging, even more so. And if you use a message type (MapMessage in JMS) that is based on name / value pairs, it's easy to extend your messages without breaking backwards compatibility.
3. platform independent
Not completely, but with both Java and C++ clients available, you should be able to support most everything.
4. supports signatures for integrity and sender checking
Not sure if it has native support for signatures, but a signature can always be added to a message as a property.
5. supports privacy of the message contents (i.e. encryption)
Again, using Websphere MQ as an example, it does support the use of SSL for communications, if that helps meet your requirements.
6. time sensitive: I should be able to detect a dead server and do failover while the user is waiting for the response
You're right, that's the tricky one. I think there are ways to achieve this goal using message passing servers, but it might take some work.
7. bandwidth efficient, as I am looking to deploy it in wireless environments
I think this requirement is met as well. Most messaging products support a message type that is nothing more than a stream of bytes that you interpret as you will... and even if you use a slightly move involved message type (MapMessage or StreamMessage to use JMS types as examples) you're still not carrying a lot of overhead.
Now #1-3 are no problem, #4 and #5 can be found, and #7 rules out anything XML-based. #6 seems to be the killer, as this rules out anything over TCP, and at that point the list gets pretty short.
Depending on just how "time sensitive" #6 really is, I do think you could come up with a solution using a message passing server. If you're able to use Java for the clients, JMS makes doing an RPC style messaging very easy, using the QueueRequestor and TopicRequestor interfaces. And even if you can't use Java, it shouldn't be to hard to cook up your own request / reply mechanism, built on top of the messaging system.
// TODO: Insert Cool Sig
Twisted is Python-oriented. If you need it language-independent then try Jabber. It has all XML-RPC good things, and it's more reliable on a transport layer. Someone will suggest you IBM MQ - functionally it's same as Jabber, just costs lots more dollars.
Less is more !
I think that messaging is nearly always a better solution than RPC, especially for wireless where the network might not be very reliable. I've been playing with Spread recently. It looks pretty good.
It's true. There has been very little research done in RPC mechanisms. In November I attended three different conferences (two on distributed computing) to see what work was being done. Most research is in either building on top of CORBA or XML and web services. Infact, I was disapointed at the lack of research in the low level guts of RPC mechanisms.
.Net on the todo list.
The reason I attended the conferences is that I have spent a lot of time developing a binary RPC mechanism. The method I use can be used in messages (eg MQSeries) or over TCP or UDP or whatever other connection mechanism you feel like.
It fulfills:
1. object oriented
Its currently implemented in java, and allows basic data types, objects and streams to be sent between client and server.
2. extensible
Any object can be sent between client and server, and the programming paradigm is simple.
3. platform independent
Currently written in java, with implementation for
C++ and
4. supports signatures..
Not yet. This is an additional layer which could be quickly added into the communication stack. I haven't had a need for it yet, so haven't been interested in implementing anything like it.
5. privacy
as above. Simple solution is to use SSL as the connection layer.
6. time sensitive
simple timeouts are easy to implement, however this hints at other issues which is a big can of worms. ie Was the task completed on the server?
7. bandwidth efficency
pure binary with very small overhead
The negatives are that its not available yet. We are currently working out licensing issues. Sorry, this isn't open source. More information will be made available over the next month or two.
A small hint at what Colony can do is at www.livemedia.com.au
Hi,
... you get the picture!
You're looking for an Object Oriented RPC language but you forget to tell us what language you're actually coding for. Objects in C++ are very different from Objects in Java, Python
Also, to get better input it'd by nice to know why exactly you need an RPC! Perhaps you have chosen to have one to solve your problems?
Regards!
CORBA fulfills essentially what you describe (not sure about #4). However, the CORBA spec is fairly large and baroque, so you are going to eat some overhead in complexity (not necessarily performance!) if you choose CORBA.
RMI is also a nice spec. Contrary to popular belief RMI is not a wire protocol but simply a spec for an RPC interface. The standard implementation uses a specific wire protocol and is tied to Java. However, RMI/IIOP also comes standard and is transparently interoperable with CORBA. I'm sure if you wanted to you could write your own transport implementation.
XML-RPC and SOAP are not really OO RPC mechanisms (despite SOAP containing the word "object" in it). SOAP is a bloated compromise spec created by committee by a few large players in the industry to satisfy all their requirements, and hence does not really enforce any sort of object or typing system. Already I see people addled by XML-think outright proposing this protocols. If it fits your project go for it, but XML is not the panacea people think it is.
It's 10 PM. Do you know if you're un-American?
You can find Ice at this page. From what I've seen, it's everything CORBA should have been, and easy to use to boot!
A deep unwavering belief is a sure sign you're missing something...
Here are some links. See esp. the REST Wiki: :Message 1314 of 1646
Adam Bosworth's Weblog: Learning to REST
Bitworking - The Well-Formed Web - REST
Debate foams over SOAP 1.2 - REST versus SOAP
How To Convert Rpc To Rest
http://www.xfront.com/ - REST Tutorial, XML et al - Roger Costello's site
ITworld.com - XML IN PRACTICE - XML, Web Services, and the REST Architecture
Mark Baker, Tech Curmudgeon - REST - Transport, transfer and coordination in HTTP
O'Reilly Network: REST vs. SOAP at Amazon [June 24, 2003]
Paul Prescod's REST Resources
Reliable delivery in HTTP - REST
REST A Web-Centric Approach to State Transition - Paul Prescod
REST could burst SOAP's bubble - Hoobler
REST Faq - Alternative to SOAP XML
REST SlideShow: Representational State Transfer: An Architectural Style for Distributed Hypermedia Interaction
REST wiki - Representational State Transfer - alternative to SOAP XML
rest-discuss Message 2330 - ROP vs RPC vs OOP pt 1
Roots of REST - SOAP Debate - Paul Prescod Yahoo! Groups : rest-discuss Messages
Roy T. Fielding - REST Architect
Sean McGrath BLOG - REST proponent
W3C mailing-list search service on REST
Why you should not use RPC for GET
xml-dev - Re: [xml-dev] SOAP-RPC and REST and security
XML.com: In a Lather About Security - SOAP security vs REST security
Yahoo! Groups : rest-discuss Messages : 2371-2428 of 2428
Keep in mind that if you are truly cross platform and your communications are pure binary, then endian-ness will be an issue. I had to deal with this once when passing messages from a MIPS SGI to an x86 PC.
It can be a real pain when the underlying communication protocol blindly assumes endian-ness is the same on every machine. I prefer text-based protocols because they alleviate that problem.
Disclaimer: I'm a Team RO member.
Check out RemObjects. It's really a brilliant piece of software and is the first RPC development system I've used that's been a real joy to work with.
LOAD "SIG",8,1
LOADING...
READY.
RUN
If you want to have two completely anonymous, very large systems talk to each other, SOAP isn't a bad way to go. But how many of us are working for giant firms that require building hooks into our system for a completely anonymous client?
Do note that you can also substitute "anonymous client" with "corporation with a heck of a lot of red tape".
As another post in this thread has already well put, SOAP is the compromise product of a number of these giant corporations. It doesn't do nearly so well when you've got two reasonably-sized *partners* (much less something in-house) working on a very specific product.
Because so many practical implementations don't need all the overhead of SOAP, you see tons of much easier implementations, like XML-RPC, that don't get the same press. This lack of press (so lack of manager buy-in) combined with the one-shot nature taken by so many coding projects, with a bit of the "Not Invented Here complex" thrown in, and I think we begin to see why so many people are scratching the same itch.
It's all 0s and 1s. Or it's not.
and go as far as to say JMS would probably achieve exactly what you want to do, and using Java is a good idea. JMSExpiration, JMSPriority and JMSTimestamp are some of the header fields that might help with the time-sensitivity. In addition, JMS can provide some delivery guarantees, and persistence.
Check out openjms:
http://openjms.sourceforge.net/
or the JBoss impl. for starts. Plus it's more fun that CORBA/DCOM/RMI.
UDP is not connection based so you have no idea a connection was made on the other end. Its kind of a send a forget protocol, unless you want to write your own handshaking in the application.
Also udps message size limit of 64k might be limiting..
QNX messaging reliably passes an arbitrary-length string of bytes from a client to a server, waits for a response, and returns another arbitrary-length string of bytes from the server. This is the basic primitive of the QNX operating system. Everything, including I/O, uses it. It's integrated with scheduling, so if you do a MsgSend to another process that's waiting in a MsgReceive, it takes only one process switch, and that process switch happens immediately.
Messaging works across the network, but local messaging isn't held down by running it through some elaborate "object broker". So local performance is so good that messaging can be used freely.
Other mechanisms can be built on top of this. But this is what you need at the bottom: a discrite message system, not a stream system like sockets. There's an inherent inefficiency with running messages through a stream system.
For most other operating systems, messaging is an afterthought and sluggish. In the Windows world, it's bad enough that objects are often placed in the same address space to avoid overhead. In the Linux world, there are about five different versions of messaging, all incompatible. And then there's SOAP, a great way to use up CPU time.
Do failover in the network. A load-balancing machine that can detect failures will work wonders, and simplify your software a great deal. Your app can set a short timeout, and if it doesn't get a response, close (assuming a connection) and retry.
What you're looking at is a client/server set accompanied by a custom protocol. If you're wanting to work with keeping the client synched up with a server, you could look at SyncML.
Alternatively, you could use HTTP and set the refresh time-out to automatically poll the server for refreshes. Send your data in HTML/WML/xHTML/cHTML/whatever and let the client deal with interpreting it. Mod your client to provide a default error page if the refresh times out.
Done.
I am a big fan of CORBA, though I admit it has some issues.
:)
> 1. object oriented
CORBA is definitely object-oriented. Much better than XML and SOAP (SOAP is -NOT- OO at all). I love the fact that once you have a reference to an object, it does not matter where that object is.
> 2. extensible
Yes. There are many useful (and optional) services available for CORBA. You don't pay for what you don't use with CORBA, so if you don't require a Naming Service or a Transaction Service, you don't have to include it.
> 3. platform independent
This is where CORBA wins. It is platform, language and network agnostic. I don't pay attention to new technologies unless they have a way to interface with CORBA.
> 4. supports signatures for integrity and
> sender checking
Some ORBs can do CORBA over SSL for security, which can include certificate-checking if you wish.
> 5. supports privacy of the message contents
> (i.e. encryption)
See previous comment.
> 6. time sensitive: I should be able to detect
> a dead server and do failover while the
> user is waiting for the response
Yes, CORBA calls will fail with a COMMUNICATIONS_EXCEPTION, which you can catch and take action. Some ORBs let you configure the timeout.
> 7. bandwidth efficient, as I am looking to
> deploy it in wireless environments
CORBA is binary and fairly BW-efficient. Again, you don't pay for what you don't need.
On the downside of CORBA, there are issues with:
- complexity. It is definitely not for the beginner and has a large initial learning curve. If you write client-side stuff, it is heaven. If you write server-side stuff, prepare yourself. If you want to do CORBA-compliant fault tolerance or security, don't call me
- penetration. Not all ORB providers implement all the nifty services. Finding an ORB for your environment that provides what you need can be tricky (e.g. POA, Portable Interceptors)
- mindshare. So many people pushing alternate technologies with a few useful features and a promise of equalling CORBA, if only they get enough interest.
- Openness. The OMG process is lengthy and can only be crafted by consortium members. Maybe a W3C-style process would make it evolve more rapidly and get implemented quicker ?!?
I can also pull a list of features for unfinished software out of my ass. I will remember to avoid your product when I am searching for real middleware.
Its robust, full featured and does all the below.
>platform independence:
It is language independent as well.
>supports signatures for integrity and sender checking
>supports privacy of the message contents (i.e. encryption)
IIOP the on the wire protocal can run over SSL/TLS.
>time sensitive: I should be able to detect a dead server and do failover while the user is waiting for the response
Has a robust support for these cases. The latest specification defines numerous timeouts and cases. Generally, vendor products provide more. For eg. Orbix has support for tens if not hundreds of such cases.
>bandwidth efficient, as I am looking to deploy it in wireless environments
- You can compress payload
- You can fragment the payload. i.e Marshaling and unmarshaling can happen almost simultaneouly. Great for large payloads. And ofcourse takes care of collisions.
Besides CDR - the data represenation is very iffecient.
You can also use RMI/IIOP if going for an all-java implementation. But it is NOT feature rich in the case of failover, quality of service, granular timeouts etc.
Obviously it depends on what you want to do, but you're requirements point to Distributed Objects.
1) object oriented
Objective-C certainly is
2) extensible
Objective-C certainly is
3) platform independent
Well, yes, if you use GNUstep on other UNICES or Windows...
4) supports signatures for integrity and sender checking
Would be possible, depending on your implementation
5) supports privacy of the message contents (i.e. encryption)
Again, would be possible, depending on your implementation
6) time sensitive: I should be able to detect a dead server and do failover while the user is waiting for the response
DO can do that for you
7) bandwidth efficient, as I am looking to deploy it in wireless environments
Not sure about this one
could probably gzip the xml to bring it back in the list of options
Go to www.zeroc.com
Fast binary protocol, OpenSource, supports C++ and Java, Plugin for OpenSSL.
The company employs a number of CORBA experts, and the employees have implemented a popular CORBA ORB before.
Perhaps you find this article interesting: "A New Approach to Object-Oriented Middleware". This article appeared in the IEEE Internet Computing magazine, and compares Ice with CORBA.