Slashdot Mirror
Open Source Message Queuing System
psicode writes "John Davies has announced AMQ, an effort at JPMorgan Chase & Co. to create an open-source message queuing system that can compete with proprietary message systems like IBM MQSeries and Tibco/RV. The announcement was made at the
annual conference Web
Services on Wall Street during Davies' presentation on February 1. eWeek has an article today with more details and some funny statements about Red Hat, SuSE and Sun possibly integrating AMQ into their "kernel". If JPMorgan Chase & Co. follows through with their announcement and they come up with a suitable open-source license, AMQ could become the Apache of messaging systems."
I think web services are going to be the next big thing.
yay
I thought that's what CodeHaus's ActiveMQ was for.
CodeHaus rocks, even if they are overly java biased for my tastes.
Why was kernel in quotes?
I like suggestions, but I don't like contributing towards them.
*clicks*
"Move along, nothing to see here."
Looks like there's already an open-source message queuing system in place here.
Do they mean an IMing system or a system for sending data threw a database or the like?
~ Mooga
The trick is in Web Services, I was lead developer on a LARGE project during the DotCom Era (2000 ish) that used Vitria, a great system and its interop capablities blew my mind at first but if we could have utilized web services we would have halved developemnt time, not to mentio forgoing a 1 Million $ Vitria Liscence.
It was cool though, we took about 6 different Apps on TOTALLY Different Platforms, 1 only HP UX, 1 , Two on Solaris ( on on 7 one on 2.51) One on DOS (Yes DOS it only ran there) and a couple on Windows, tied them all together through Vitria then hooked it up to a Web Front end, It was sooo slick the first VC that came to see our Proof of Concept (which was totally functional )gave us 15 Mil in VC
But alas another DotCom fatality, we went from 12 employees to 165 in a year......can you say BAM
Buzzword buzzword buzzword, incorrectly used terminology, buzzword buzzword.
wdd
How about Joram which is supposed to be pretty good.
--Stupidity is Self Curing!
[ ] You know, what message queing is?
?
Bankers may not love proprietary systems, but they do have some nice advantages.
A system like AIM is not going to go offline for days at a time, and leaves the organization without the hassle of managing an extra set of servers.
And AIM might not have message queuing built in, but systems like DoorManBot allow the function to be added.
Then again, Open Source is always nice, but I'm for a stable system any day.
Send offline messages on AIM with DoorManBot
"Message queuing is a communication tool that allows applications to reliably interconnect in a distributed environment where one of the applications may or may not be available at any given time. The queue acts as a holding container for messages as they are sent between applications. The applications send messages to and read messages from queues to communicate back and forth. An application writes a message to a queue, which will then be received and processed by another application at some point determined by the receiving application. This type of communication is designed for asynchronous use where the applications involved are not waiting for an immediate response from the other end."
9 11
From http://www.developer.com/net/asp/article.php/3297
Since everyone else is laughing at you, I'll tell you why.
Message Queueing != Instant Messaging. RTFM luser.
I am probably wrong, perhaps someone can fill me in as to how a these types of services meet a need that can't be built on top of HTTP (or are they in fact sitting on top of HTTP?) ??
http://shit.slashdot.org/article.pl?sid=05/02/08/1 97251
We have jabber supporting business services at work, but this turned to not be too great of a solution for us. Many of the MOM (Message Oriented Middleware) features that we need were never implemented in Jabber, due to that the early usage of jabber was in the instant message world. We have been looking primarily at XmlBlaster that seems to be somewhat feature complete, but we have yet to find a really good alternative to a commercial MOM.
As an MQ user/programmer: It's (MQ) transactional, can failover, has all the really expensive reassurances that big business craves.
As a Jabber enthusiast: They're exactly the same.
You mean they can't just fork POPFile::MQ and use it :-)
John.
I think more importantly, how does this compare to MSN messenger's message queue?
Wall Street is also looking for a new Open Source Ethics and Honesty implementation as current proprietary implementations seem to be faulty, buggy, and expensive to maintain.
We have had MQSeries at each IBM implementation that I have worked at (SAP software).
My perception of MQ is that we would be better off throwing it away and FTPing files back and forth while using SAP's job control to kick things off. I cannot imagine how many weeks we have lost in a production environment because of some fiddly piece of junk involved in the MQ process decided to hiccup.
"they come up with a suitable open-source license, AMQ could become the Apache of messaging systems."
If they want to be the Apache, then they'll have to use the Apache license too. Effects have causes.
-russ
Don't piss off The Angry Economist
... and I will never do business with them nor with anyone affiliated with them. I won't get into all the details, but it was a very shady deal. I highly recommend http://chasebanksucks.com/ as it has comments from ex-customers and ex-employees. Chase has tried to shut them down numerous times in the past, but they can't touch them since they're hosted in France. If I have detered even one person from ever working for or doing business with this shady organization, then I have done my deed for the day. I could care less about how this is modded.
Maybe these services layer on an extra confirmation mechanism...but isn't this what TCP performs in any case?
Message queueing is application-to-application messaging, which once-only guaranteed delivery. In addition, both the reading and writing of messages can be done transactionally, with a transaction spanning multiple queue operations.
Think of it like a database where an application can write a record that can be read and committed by exactly one reader; however, if that reader crashes before committing, another reader can pick up that record and try to commit it, etc.
Financial services firms use this tie their many apps together.
As a voting libertarian, I whole-heartedly encourage companies to actively develop whatever technical solution that best fits their needs. This is one example of how open source software really fits in with the libertarian world view. It's not the "free software versus only commercial software," but a mix and match that lets both coexist. If someone wants to develop their own software, then let them and don't even begrudge them the right to give it away.
This is what really pisses me off about the copyright expansionists. They don't want people to be able to easily develop software for free. Copyright expansionism, as pushed by groups like the "Progress and Freedom Foundation" is not about freedom, it's about protecting business against hobbyists and other "asymmetric competition." I bet it makes such groups' heads spin to think that one of the biggest corporations on Wall Street is now about to dive head first into open source development and that the result of this development could send shockwaves through a segment of the commercial software industry.
The freedom to write open source software is the same freedom to write closed source software. If you erode the foundation for the former, then you have no right to demand respect for the latter. That is why when the big copyright cartels lobby Congress, I see nothing wrong in doing things like buying academic licenses for software and taking them out into the real world. The copyright holder makes no profit on the academic license, only the seller does.
Click here or a puppy gets stomped!
an open-source message queuing system that can compete with proprietary message systems like IBM MQSeries
Somewhere in IBM's headquarters there is a camel. A straw has just been placed on its back.
Direct away from face when opening.
They should check out tipc for message queueing. TIPC is a reliable messaging protocol designed for clusters. TIPC is fairly transparent whether a task is local or somewhere else on the network.
-Aaron
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
n/c
And they'll have to match the quality we expect from Apache products... and the tolerance for bugs in message queueing is a lot less than that for web servers. The Apache web server is excellent... AMQ will have to be better to be acceptable.
Some years ago a company I worked for was quoted one hundred thousand dollars as the cost to license IBM MQ Series messaging. There could be some serious savings in using an OSS product like this.
By the technical ignorance of some of the posters here... Before you post, please make sure you have at least an inkling about what you're talking about! A Message-Oriented Middleware system has absolutely nothing to do with IM, or Web Services (except in a very indirect way), or even REST systems. /. seems to be in dire need of real geeks, it seems...
Not that more options are not a good thing, but the Java Messaging Service (JMS) built into Jboss rocks. It is LGPL as well, which is one of the better open source licenses out there in my book.
+++ UGUCAUCGUAUUUCU
I think they're talking about OS queue stack for transaction process (i.e Stratus VOS queues).
If a company was going to contribute that type of technology to the kernel, that would be awesome.
Yes Francis, the world has gone crazy.
I work for a major airline (and the only one that has always been profitable =)) Because of the nature of our business, we have to use both Tibco and MQSeries for our communications. We must use both because of the transactional nature of running an airline (communication to / from the aircraft) and the non-transactional (Sabre / Ticketing). If there was a product that gave us the ability to do guaranteed / clustered messaging in both a synchronous and synchronous environments we would be all over it. In the end we end up with MQSeries butting messages on a Tibco bus that everyone uses as a pub / sub system. This system is used for everything from liqueur kits to engine maintenance to baggage and flight plans. If there was a combination of point to point (MQSeries) and Tibco (pub/sub) I would be in heaven!
Stranded.org
I hate to respond to my own post, but I forgot to include the link. TIPC is available under a dual license, both BSD and GPL. It can be found at http://tipc.so urceforge.net/</a>. We're planning on using it in an embedded project. It has good performance and reliability capabilities.
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
Horus and Ensemble allow for process groups. Doesn't that do a superset of this stuff? It is open source.
Horus is/was used to run the NYSE, and a prototype radar for the Aegis battlecruisers.
Why is it not good enough?
http://www.thebricktestament.com/the_law/when_to_
For me, the best solution for a large enterprise is something like JMS, this way we can change the backend (MQSeries/Tibco) and not impact the developers because they still connect the same way.
Stranded.org
For a message queing project to be sucessfull, it will need to be cross platform and therefore should NOT be built into a kernel. It should be build as a daemon/service to run in userland on as many OS flavors as possible.
"/. seems to be in dire need of real geeks, it seems..."
They all left in the Great Fark Rush of 2000.
Does anybody besides me wonder why anybody can sell expensive message queue software when Sendmail + Procmail can do the same thing? I've set up numerous apps that use this approach for asynchronous messaging.
JDR
Bingo, sir.
General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
I think I'm going to be the next big thing - you should probably start sucking up now, in order to get a good position in my grand army of world domination!
T.J. Schmitz - the man, the myth, the legend - o
>This is what really pisses me off about the copyright expansionists.
I really appreciate source code licenses that actually let you use the source code permanently such as BSD.
Copyright expansionist vehicles, such as GPL, end up costing too much in the long run.
Our turn to laugh at you...
Jabber > Instant Messaging
IM seems to be the only significant application on the Jabber architechture though, unless I'm missing something.
Oh yeah, and to respond in kind: "try google next time, you insensitive clod!"
"I bet this really pisses off the copyright expansionist"
Well I just gotten through reading the story. Sorry! Didn't see any "copyright expansionist" upset over the news. Any particular reason you set your soapbox down here?
(I hate it when people use totally generic terms like "message queuing" when they are actually referring to some much more specific functionality. I especially hate it when articles written about such systems make no attempt to explain what that functionality might be.)
"Wall Street is also looking for a new Open Source Ethics and Honesty implementation as current proprietary implementations seem to be faulty, buggy, and expensive to maintain."
Unfortunately 1567 Slashdot posts later. No one can agree on who's ethics should be used, and whether it should be under the GPL or the BSD license.
Mod this parent up as most slashdotters don't seem to know what MQ'ing is.
You can also have a publish/subscribe method, where one application can "publish" a message onto a queue, and any other app can "subscribe" to this queue and get the message. Useful for something like stock market prices (that constantly change) that lots of apps might rely on.
What is a message queue, why should I care, what does this have to do with anything, what are they talking about? In short: what kinds of messages are we talking about, what are sending them to what, and why do we need to queue them?
"A message queue is a queue onto which messages can be placed."
thanks a fucking lot google
-- 'The' Lord and Master Bitman On High, Master Of All
Ummm... am I wrong here or did not IBM's REXX integrate this as a direct application add-on? As I remember correctly, you could add a REXX message pipe to any program/app/daemon/whathaveyou, and any other program could pass a command or data to the application through this pipe and either get a response or not depending on what the message was. IIRC, these pipes had built-in queueing through a REXX master daemon waiting for the destination app to become avaialable. Am I dreaming, or didn't this already happen?
It is the best way to get data from distributive systems to/from the mainframe. So... are they going to write a COBOL interface from 3270? Without IBM's help, I don't see that happening.
For distributive->distributive, web services is all you need.
Thank you people, today I got my first "-1 Flamebait" moderation. For no special reason :-P
:-)
Please mod the TIPC reference further down to page too, it's as much off-topic and flamebait as my post. It's even a plug for his project.
Have a nice day
GPL = something for something.
Changa hates change.
Um, isn't that pretty much what D-Bus is?
Keep in mind that Wall Street was probably *the* major engine driving Web applications into the mainstream in the 1990s. One reason Wall Street went crazy touting the Web as an unlimited business panacea was that it was actually like that for its own industry. By the time they promoted it in 1995, Wall Street shops had been churning out httpd patches, CGI apps, Perl revisions and code, DB connection software and techniques, TCL patterns, and all kinds of R&D that underpinned much of the development momentum. Message Queueing has a similar arc, because Wall Street both inhabits the bloodiest edge on which they can survive, and has the least tolerance for failure. The flakiness of network messaging is the root of some of the worst inhibition on network growth among users and deployers - Wall Street has been there, and has answers. Let's use them.
--
make install -not war
When I use free software, 100% free for family and friends.
$60.00 per hour for MicroSoft support.
Bill_Gates.m4a
For the clueless among us (eg me) WTF is a 'message queueing system' and what would one use it for? Something to do with email? Kernel API calls? Tech support ticket system?
We had a similar debate on c2 wiki.
It seems that FTP, email, web services, and/or databases can be used instead. I am not sure we need Yet Another data-sharing mechanism/protocol unless it offers something truely unique.
Table-ized A.I.
My company built and sold a message-oriented messaging product that merged the pub/sub and transactional models several years back, and it included guaranteed delivery. And yes, there was a lot of discussion about open sourcing the code base. The system is still in daily production in a bunch of Wall St. firms.
What these firms all wanted as a key requirement was clean integration with C/C++, and extreme performance. We had to handle thousands of messages (ranging into megabytes each) per second, with full guaranteed delivery, over IP networks and through firewalls. Oh, and none of the flakiness and admin problems of MQSeries.
None of the available JMS products could handle all these requirements.
I have always thought that Discovery is the most interesting area of research on the Internet. The idea of applications seeking out each other, learning about their resources and interfaces, and hooking up and communicating, without any human intervention is fascinating.
The prices and importance of messaging middleware applications to many companies are such that they will be precisely the LAST application that would be allowed to go F/OSS/no vendor support world.
XML blaster is an open source message oriented middleware that has bindings for multiple languages (C,C++, Java, PHP, Python, Perl).
I have never used it, but it's been available since a long time.
My heart is pure, but make no mistake, it's pure evil
I wrote my own AMQ implementation about 4 years ago for a project that I was working on. The project was done in Java on Weblogic 5.1 (flashback - I am working on W5.1 on my current contract today,) and message beans were just introduced but we did not have them in our implementation.
The project was for life insurance industry (for a little known company named Worldinsure that spent over 2.5*10^7 dollars on Aeron chairs and plasma monitors and - surprise, surprise - went belly up.) The app. would ask hundreds of questions that needed to be populated page by page. On some page the app would have to generate a PDF file, and since the Adobe pdf generator would crash the JVM after a couple of uses, we desynchronized it.
I wrote a message queue that was generic enough to start whatever action that was appropriate for the message type. It was not too difficult, a relational db (Oracle) was used as the message storage mechanism and I had 2 different options for message retrieval - message polling from the java code and Oracle triggers. The tricky part was the multithreaded message queue processor that could start any action - a java class in a seperate safe thread. Every message had a status - New, InProgress, Failed. Messages could be setup to be retried with a retry-decrimenting counter, so a message would become 'Failed', once the retry counter reached 0. Message types had priorities assigned to them, so some messages would take priority over others. To make sure that all messages eventually were drained, there was also a priority timeout. Messages could be 'peeked at' and retreived. Once a message queue processor decides to retrieve a message, this message is not deleted, but marked with status 'InProgress', so that other processors would not pick it up as a new message.
The interesting side-effect of this design was that it was possible to run as many message queue processors as one wished, thus an application could spread the load onto multiple computers (this adds redundancy and scalability.)
All of this can be implemented quite easily to follow JMS standard, but JMS standard is even simpler (in terms of functionality) than what this designed allowed.
You can't handle the truth.
The article does not mention a web page for this project and googling did not help me. So, is this a "secret" open source project, or what?
http://www.spread.org/. By the way: How complex is a message queue? That sounds like kindergarden technology to me.
... the glue that holds the worlds together.
Seriously, I would have to think a while to count the projects that I have used ISIS, JMS, etc. for. Asynchronous messaging makes architectures simpler.
ISIS supported "virtual synchrony": guaranteed order of deliver also.
FTA: "We now need something that's not proprietary. Banks don't like proprietary things,"
Maybe Microsoft should write a message queuing library, in order to guarantee "interoperability"
Don't blame me, I voted for Baltar.
you are an idiot
So this is quite interesting; I'll be very interested to see their implementation. With IBM's MQ, their big thing was that there was something like 8 function calls, tops. I can still see the IBM guy talking about how simple it was, listing out the functions.
What he neglected to mention was that each of those function calls takes *structures* as arguments...some of those structures had a dozen+ members.
To be fair, MQ is pretty slick; I wrote a connector between the webserver and an ES/9000 mainframe using MQ. No EBCIDIC tranlation, no worrying about going from TCP/IP to SNA...all was handled by MQ.
The other big thing (and I'm sure this is where all this work is coming from), is that, on a Windows platform, I think MQ cost around $5000 (circa 1999). On the mainframe, they charge *by the cycle*. I don't remember the exact figures, but the head of the mainframe group said that it cost something in the high five figures just to get the cartridge, and then there was both a monthly support cost, and then the per-cycle cost.
When testing my program for load, I would send a couple of big books I had gotten from the Gutenberg project on a round trip from the NT box to the mainframe, and do it all weekend. My manager was presented with a "bill" for around $10,000 in MQ time on the mainframe. Yowza.
And in addition to all that, there were the Candle consultants who helped set the topography up. For an end-to-end system, it sure had a lot of people involved, and a lot of moving pieces.
The upshot was that it did what it was supposed to do, and we handled millions of requests through it every single day for years. Still one of the pieces of code I'm most proud of.
What JP Morgan and other Wall Street firms use message queuing software for is trading in the financial markets. So if the software bugs out, they stand to lose millions of dollars per minute or more depending on the size of the trades they are executing. So I see this story as a positive for open source software because if Wall Street people think they can get a usable mission critical piece of software out of the open source community then no other potential target for OSS is invulnerable.
"Lack of technical competence coupled with the arrogance of power, as usual, leads to no good end."
This is incredibly exciting!
Java zeolots propose tons of various java queuing apps, forgetting that some folks can't afford a giant Java runtime. I keep on reading about Java having caught up and beating other languages speed wise, but have yet to come across the real world application where it actually does so.
Comparing JMS to a bunch of other messaging toolkits (explore some cluster message passing) and I'm blown away again with how slow the whole thing is.
MQ, probably be rocking if it received a cleaner fresh, less cruft and finicky setup. I like IBM.
Tibco, prepare to bleed money, weird tools around the system.
Jabber? They have largely abandoned the open source server it seem to me for high volume messaging apps.
pubsub.com A neat ASP that seems to have some ripping performance.
amazon.com Super basic. When will you setup the commercial stuff. No fun building anything until one knows how it'll end up.
In the end, this is super cool. This should help plug a MUCH needed gap in the mono space.
The company I work for uses FioranoMQ and it's such a pile of garbage. Their client-side RTL will not even reconnect to the message broker if it's down unless the connection is marked as durable, in which case it mindlessly queues all the messages on disk. The client application can't control any of this.
As for their C++ API, it's hard to believe their developers even know how to code in C++. It leaks like a sieve and doesn't even do durable connections. If the connection to the broker is ever lost, you literally have to restart the entire C++ app.
I guess this is what you get for software developed in India. The cost of other providers like Tibco/RV is grossly prohibitive.
If JPMorgan Chase & Co's experiences are anything like this, it's no wonder they want to build their own system.
The key to this MQ project is getting the interoperability btn languages to actually work. I see no mention of Perl, for example. Or of any scripting language. Certainly if they wish to "walk the walk" as well as "talk the talk" there should be reference implementations for each *client* language they wish to integrate, ditto for the server-framework, although the latter would be a smaller pool.
If they can't do this, then the best thing is to stick to sockets for inter-language interoperability:-)
BTW, there are rumours of C++ interop with JMS - Code Mesh and MantaRay I suppose you could use jython for python so I suppose you could say that JMS is spreading outside the compound but I'm not convinced.
ALso, having it in the kernel is a Good Thing. Well, it is if you're windows - I think most windows system comes with a MQ, although I suspect that these are stripped-down versions of a more expensive "Enterprise System". If this system came with the kernel, then I'd've had one less thing to worry about :-)
There's no mention either of a proper Java implementation - including Message Driven Beans. This is the one part of JMS which is worth keeping, I think.
in short, good on the big picture but a lot of niggling worries. I shan't be attempting to use this system until rev 2.
Patriotism is a virtue of the vicious
If JPMorgan Chase & Co. follows through with their announcement and they come up with a suitable open-source license, AMQ could become the Apache of messaging systems.
Why does that scan like "If [company] comes up with an anti-gravity machine and they fit into boots then people will be able to leap tall buildings in a single bound."
In other words, 'here is a press release promising manna from heaven and that damn flying car I was promised, COOL! I'LL POST TO slashdot! They'll approve anything!'
Can anyone elaborate?
Now imagine that type of scenario on steriods. With atomic transactions, xml serialization of objects, custom routing, security, and guaranteed delivery.
Have you ever been to a turkish prison?
Does anyone else get the feeling this "Message Queue System" is about 85% marketing buzz? It sounds like a standard component that might be found in any basic networking software. It's probably ~500 lines of code in any language that has builtin support for condition variables and RPCs (e.g. Java).
I *HOPE* they choose the BSD or GPL. I'm really tired of every group inventing yet another license. Half the time they are poorly worded, they have trap doors, they aren't GPL-compatible, or they are just the same as some other existing license except they replaced the name and move some paragraphs around.
I also hope "Apache" means "popularity" and not "code base size", "feature set", "security", or "performance."...
We utilize Data Queues on our AS/400's for this.
Basically, we write output to a data queue on the AS/400.
When a PC comes along to process that data, it does a destructive read on the data, processes it, and sends the reply back to another data queue on the AS/400.
We do this with satellite communications, we do this with AS/400 to/from PC communications, and we do this with AS/400 to/from *nix communications.
The overhead and complexity of using a file is negated by this. We handle many millions of transactions an hour, so it is important to tune this out the wazzo.
And if any one system is down, the other systems just wait until it comes back online and start talking again.
V5R1 we are limited to 2GB of queued up data per each queue. They plan on taking this up to TB level, I hear.
There are already several very good GPL'd Message Queue systems available - they've been around for years - there is even a POSIX standard for message queueing !
Oh wait, I know, somebody want's a good performance review ! What was I thinking ?
Mule (http://www.muleumo.org) is an opensource messaging framework that abstracts the messaging technologies from components so that transports/proptocols such as Jms, Http, tcp, smtp, pop3, xmpp, soap, multicast, vm, file, etc can all be used transparently to communicate between Mule nodes or external applications. Mule also handles all transaction, routing, delivery and transformation of events across the network. Also, it uses a SEDA processing model that allows for hugely scalable deployments.
Users of Mule are using MQ Series (Jms), Oracle AQ, JBossMQ, ActiveMQ, Joram, See Beyond and Open Jms and getting the benefit of the other services Mule provides.
-Ross
The idea of applications seeking out each other, learning about their resources and interfaces, and hooking up and communicating, without any human intervention is fascinating.
Is it not disturbing , rather?
OSMQS can be used by applications, and appears useful (if it is reliable, not over-designed, efficient, and portable to multiple platforms), however it's something you or your grandmother will invoke directly.
Whether it belongs in the kernel or in userland is another question.
It's not like there aren't several open source message queueing systems around already. But then again, you can't expect Slashdot to notice such things.
Karma: It's all a bunch of tree-huggin' hippy crap!
Everyone's asking "Why?" and debating the technical bits. I'll tell you why. So they can get a deep discount on MQSeries from IBM.
It's been out for five years on freshmeat, and supports CORBA, RMI, XML-RPC, nativ socket, or persistent HTTP.
See the link at http://freshmeat.net/projects/xmlblaster/.
- jon
Ganymede, a GPL'ed metadirectory for UNIX
[....] Ensemble is actively under development and we will do series of releases over the fall of 1996 and spring of 1997. By early in 1997, Ensemble will be an outstanding environment for building Java-based groupware applications that do multimedia conferencing on the Web.
Sound like it'll be great Real Soon Now!
Get in line now!
There's been a opensource, cross-platform solution for a number of years (shameless plug), http://www.osmq.org/index.html
~~~ They call me Little John, but don't let the name fool you...in real life I'm very big.
because they clearly do not make enough money as is. They are just using this "product announcement" as pressure to lower their licensing/support contract costs with IBM and Tibco.
http://www.omg.org/technology/documents/formal/dat a_distribution.htm
Data Distribution Service.
Like OMG or not, it is open, and you can build it (there is an open source project started on source forge with no activity yet), and it is an adopted standard. It supports "queueing" via a number of different QoS parameters.
I haven't read the spec on AMQ, but I'll make a wager that DDS is more powerful, and has more features..plus it can be built for embedded systems, as well as the Enterprise.
There are a number of commercial implementations of the spec to be released very soon.
>I really get sick of people whining about how they want free code so they can sell it to other people.
I really get sick of people parroting that the GPL licensed code is free software.
Freely licensed code is avaiable for unencumbered redistribution in non-commercial and commericial format without requriing source code distribution.
Indeed! And if they want to become the Linux instead of the BeOS, they'll have to use the GPL.
Hmm.. most banks run their core systems on Minis and Mainframes -- some of the most proprietary things around. Most banks run Microsoft Windows, or MS-DOS (yep!). Most banking software (critical systems) is *extrememly* proprietary and runs on Windows, using Microsoft extensions to standards where applicable.
I've worked in banking for about 10 years now and that's one of the silliest things I've ever heard.
The bank I work for wouldn't know what Linux was if you dropped a brick on their head. (Ok, the IT staff actually would know because I've told them, but the rest of the folks are clueless -- they think that Windows *IS* a PC).
If you don't know what MQSeries can do, don't even think about this. MQSeries ( and on limited functionality MSQ ) are very powerfull systems.
Now - getting some subsystem that has even a small part of those would be great - it would be a winner (IMHO)! I could use it now for my current project - assuming that our company gets out of this NIH ( not invented here ) mode. I'm good but the time is limited....
If they plan on offering reliable delivery, they'll need a place to store messages until the recepient is able to receive them. I wonder what they'll use for this -- At the reliability levels that Wall Street demands, I don't think something like MySQL is up to it.
Chip H.
Seriously, MS has had message queueing built in to the OS for years. When will OS produce something innovative?
What the heck is a Messaging System?
Just about everything you can do over a network is a messaging system.
Another bunch of greedy companies trying to get a free product. Why can't the proponents of OSS understand this simple fact, or is it that they just dont care as long as the leader gets a fat check?
Lets see... all these financial firms pay a lot for licencing commercial software. They want to drive down the cost, so they band together and pay 2 or 3 times the salery of an engineer to the team lead and average pay for a few other engineers. Another bunch of fools rush in to work for free thinking that they can get some consulting deals later on and continue to work on improve the OSS product. Now they can leverage the commercial software firms saying they are going to switch to the OSS product. They get tremendous leverage from this, at the cost of the jobs of those engineers working for the commercial products. Once these greedy firms get their demands met, they quietly brush aside the OSS product and continue to use the commercial product at a big discount.
Dont you OSS guys get it? These are the same bullys that called you geeks in high school and college and the same guys that you laughed at for being dumb and going to biz school. Well, look who is the dumb ass now?
FTP, email and databases can be used to implement low-volume, low-speed message queueing. There's nothing wrong with that - many web-based systems use that.
Where MQ plays a role is in the high-speed, low-latency version of this. I work with an MQ system that does 1500+ messages per second, transactionally, for prolonged periods of time, with a latency 0.1 second. And this runs on a 4-CPU Linux box.
MQ solves a different problem than a database. A database allows data updates and multiple readers. In MQ, each message has one reader and there are no updates. These different semantics allow vastly different performance.
are we talking about IM systems (AIM, MSN Messenger, Yahoo, ICQ, IRC)
:)
:)
Or are we talking about message QUEUING systems like... erm... MQ
Used them before - they're way cool. Used to have a bunch of iPlanet & Apache servers talking to weblogic boxes which used MQ to talk to NT boxes running Universe and some thing or other in C somewhere else. There were big drawings and maps on the wall and it all made sense to whoever was in charge but the rest of us just looked on, shrugged shoulders and got on with it.
Rebooting was the funnest though - having to take down everything so the NT boxes could be rebooted then all the Queues brought back up
MQ - a way to tie multiple tiers of dysfunctional software on inompatible hardware into a big ball of system fluff.
Yay
is jabber XA-compliant?
Should'a used Vitria's BAM!
from http://www.vitria.com/products/platform/bam.html
Business Analysis and Monitoring capabilities give line of business and IT users unprecedented real-time and historical visibility and analysis of strategic processes and data.
I wonder if this software is a descendant of that developed at TransactPlus, a message-queueing-services company that JP Morgan invested in heavily. (I'm a former NOC employee.)
Message backing store? It's called a "file" - see also "transaction log". Extra reliability comes from writing it to 2 different disks. What do you think databases use for transactions, anyway?
This is really stupid stand to take. If you leave your financial and personal data out on the street, you take the risk of some one taking a look at it, and using it for something other than what you intended, even if you intend ONLY those who do the same. Grow up.
0.1 second latency? Surely you mean 0.01 second latency or less. You'd never hit the bid/ask with 0.1.
It is now apparent to me that /. has redefined "Insightful" to mean "Talking out of one's ass."
Here's a clue for you - many of us understand the idea behind messaging. You do not, yet you criticize based on superficial characteristics. You are wrong. You do not understand the purpose, the usage, or the concept at all.
At my work, we use SonicMQ, which works pretty well. Its Java based, but has a C and COM API as well as a HTTP Server so you can post messages to it via HTTP. Anybody can write a messaging system, but getting one that fails over, clusters, routes, etc. is a bit more effort.
I've found messaging based systems to be really enjoyable to work with. They take a lot of thought, but you can massively scale (e.g., the Aggregator pattern). Also, they encourage loosely coupled systems.
Also, when you use XML as your message payload you can get very good interop (provided you pick a messaging system that provides APIs for the clients you need).
We tried to make "Web Services" work, but you really need messaging infrastructure (e.g., security, routing, guaranteed delivery). You can get a fairly cheap / robust integration architecture with a decent messaging system and an XML canonical message format.
It looks like the messaging layer will get commoditized. Looks like trouble for MQSeries, SonicMQ, Tibco, etc. Not sure where AMQ will go, but ActiveMQ is bound to go places since it is a part of Geronimo.
There is also (well at least on the Java side) an effort to drive standardization up the stack with JSR 208 - Java Business Integration. Its basically a spec for Enterprise Service Bus (ESB) / Service Oriented Architecture.
CORBA has been simplifying while SOAP has been complexifying. SOAP is now more complicated to develop for than CORBA.
CORBA is actually remarkably simple to grok if you are not using C++. CORBA bindings for Python are a snap (and esp. omniORB is very very fast), and it never ceases to astound me that CORBA hasn't "made it" in the mainstream. Instead, people think they need web services to "simplify" matters.
Ah, the folly of humanity...
Save your wrists today - switch to Dvorak
I have seen other people's hacks at moving large quantities of data around. Generally speaking, they didn't sequence the data and they managed to drop data between systems. Bit of a shame when you are at a bank and that data represents money.
See my journal, I write things there
Actually, the IM model is just a special case application of message queuing.
See my journal, I write things there
How about XMPP/Jabber?
The J-EAI project is already providing such tool. It has been published in version 1.0 beta 3. Maybe it does not offer yet all the planed feature of AMQ, but we will improve it over the time.
It is already multi-platform and multi-languages and based on the XMPP IETF messaging standard (XML based). The architecture is already clustered and can handled a lot of simultaneous connection.
More details on: J-EAI web page
I strongly believe that using an XML standard like XMPP is the only way to go to ensure generalized interoperability of application around the integration platform.
--
Mickaël Rémond
Erlang-projects
> Since everyone else is laughing at you, I'll tell you why.
XMPP is an IETF standard for messaging. Jabber with instant messaging is only on possible use, but Messaging systems are other examples (Such as J-EAI).
--
Mickaël Rémond
Erlang-projects
I bet it'll take at least 5 years before this new thing makes any inroads in financial customers, but this must be really scary for IBM's MQSeries folks.
Things like MQ Series are bread-and-butter of many sales reps as there's no serious competition to their solution - I haven't talked to many banks, but those to which I did all use MQSeries.
OSS is moving from the edge to the data center, it's already half-way there. Five more years and banks will be able to get 80% of their apps under GPL license.
Why ZZ when you can C-x C-c? :P
Why there is nowhere a link to the source (or even the binary).
and do they meet the performance requirements? reliability requirements? Nay!
-Stu
I doubt JP Morgan has been using sourceforge to stash their code, but does anyone have the source code or a website with more information for AMQ. It is open source isn't it?