Yes I concur.
What I'd like to see are possibly some standards by which these new "databases" are measured both in terms of capabilities and performance thereof.
www.vhayu.com
www.kx.com
www.streambase.com...
Thanks for the pointer! I will research that.
I already found an interesting tidbit that discusses the relationship to tuple spaces--something I've used in the past as a basis for some high-performance asynchronous messaging.
http://www.artima.com/weblogs/viewpost.jsp?thread= 72527
Persistence is only necessary for as long as something is useful to persist.
When trading futures one might devise a trading system that only cares about data that has been collected over the past 20 minutes.
Yet the amount of data might surpass several gig over a moving 20 minute window. And the kinds of queries executed over the data may indeed be determined after the data has been persisted.
Mechanical automated "Black Box" trading systems are doing this already today.
Perhaps.
I once had the shocking idea of inverting the system. Persist a mapping from queries to interested parties.
Then as data streams through the system it pattern-matches and sends the hits to the recipients.
The trick here is to build an optimal tree that can quickly find matches.
The answer is NOT to simply store queries and walk through them on every update/stream message.
My first reaction is: He is late in the game.
Check out www.kx.com. They've already done this. And this kind of thing has been used for years to analyze real-time stock and commodities trading data as the trades occur in real-time.
I've deployed several systems that are essentially streaming databases like this.
Or did I miss something here?
Slashdot Mirror
Yes I concur. What I'd like to see are possibly some standards by which these new "databases" are measured both in terms of capabilities and performance thereof. www.vhayu.com www.kx.com www.streambase.com ...
Thanks for the pointer! I will research that. I already found an interesting tidbit that discusses the relationship to tuple spaces--something I've used in the past as a basis for some high-performance asynchronous messaging. http://www.artima.com/weblogs/viewpost.jsp?thread= 72527
Persistence is only necessary for as long as something is useful to persist. When trading futures one might devise a trading system that only cares about data that has been collected over the past 20 minutes. Yet the amount of data might surpass several gig over a moving 20 minute window. And the kinds of queries executed over the data may indeed be determined after the data has been persisted. Mechanical automated "Black Box" trading systems are doing this already today.
Perhaps. I once had the shocking idea of inverting the system. Persist a mapping from queries to interested parties. Then as data streams through the system it pattern-matches and sends the hits to the recipients. The trick here is to build an optimal tree that can quickly find matches. The answer is NOT to simply store queries and walk through them on every update/stream message.
My first reaction is: He is late in the game. Check out www.kx.com. They've already done this. And this kind of thing has been used for years to analyze real-time stock and commodities trading data as the trades occur in real-time. I've deployed several systems that are essentially streaming databases like this. Or did I miss something here?