Slashdot Mirror
Why Reactive Programming For Databases Is Awesome
Nerval's Lobster writes "'Building on many lessons learned from spreadsheets, functional languages and model-driven application development environments, Reactive Programming (RP) is a recent approach to creating applications,' Val Huber, CTO of Espresso Logic, writes in a new column. 'In the RP paradigm, you can create complex applications from a series of simple declarative statements. Many of the details of implementation and work of gluing together various sorts of application constructs can be hidden because of the declarative approach.' He goes on to argue that RP makes maintenance easier and applications more adaptable, because RP offers automatic dependency management and reuse; at the same time, he also acknowledges that RP has some shortcomings. What say you?"
I think excel is one of the greatest programs ever and reactive programming sounds great. This article isn't very specific as to how to get started though. Is it a concept, a product or something I have to implement myself?
How does this differ from what the rule engines Drools or Clips provide? They are reactive systems that update state by monitoring changes in the underlying data.
Well, yeah, after a fashion, since web servers themselves are reactive entities. ;-)
Ezekiel 23:20
Interesting.
Code modeled in RP sounds like VHDL, Verilog et all - the languages used to model modern Programmable Logic Arrays, Programmable Gate Arrays and Programmable Logic Devices - aka, hardware.
No doubt this approach has its merits, but this kind or "programming" has its gotchas - a lot of gotchas.
I stand aside for now, looking with great interest.
Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
Did this guy just reinvent spreadsheets? There's something to be said for this, but having written in Prolog, which works that way, the 'reactive programming' people have to make a better case than the article does.
On the other hand, one of the big problems in databases is change notification. Microsoft at one point had a research project on this. The concept was that you could make a request to be notified when something of interest changed in the database. This was expressed as an SQL SELECT statement. When some change was made that would affect the output of your pending SELECT, you'd be notified. The problem was to figure out some efficient way to do this with thousands or millions of pending SELECT statements.
Finance people use notifications like that all the time. Limits on stocks, limits on combinations of stocks, changes in key ratios, that sort of thing. But the systems for that are specialized, a special case of a more general problem. The most general form of the problem is that B wants to know when something has changed at A. Most of the solutions for this have problems. Polling is either inefficient or slow. B relying on A to message them on a change has a failure problem. Both approaches can result in overloading A with checks for events that aren't happening.
From the article description I couldn't figure out a) what is actually reactive programming; b) how will that help me build better applications. Complex applications are already built from a series of simple declarative statements. In fact, we call the most simple declarative statements language available "assembly". We call the machine representation of those very complex applications built with simple declarative statements "binary". On the other hand, Brainfuck also allows you to build complex applications with very simple statements, and I'm not migrating to it anytime soon.
Not cool, Slashdot, I'm going to have to read TFA!
Complex applications from simple declarative statements?
I have a button you can push and it will create an app for you.
George Jetson, have you been playing with the time machine at Spacely Sprockets again?
A feeling of having made the same mistake before: Deja Foobar
I had never heard of Reactive Programming before, but after reading the article, it seems that Reactive Programming is the same thing as using views, which have been a part of mature RDBMS's for several decades now. Am I missing something obvious?
I don't respond to AC's.
Usually database manuals calls them triggers.
I can see this being useful for problems that are extremely linear and require extreme parallelization on large quantities of data, but that's about it.
I've done this 'methodology' many times using Excel. I did it because I I needed to give the spreadsheet to other people, and wanted to avoid having Office nag about 'potentially dangerous VBA'.
It makes it very easy to see *exactly* how your data is flowing, which is a bonus. It also uses a *lot* more ram because you are now maintaining a permanent block of memory for every single operation, for every different piece of data you are coding against. Of course, the second you get a cyclic dependency, the whole thing blows up.
But this method of 'programming' was a natural and convenient extension based on how Excel (or any spreadsheet) operates. Nothing more.
Is it just me, or does it seem like everyone now-a-days is trying very hard to come up with new methodologies and paradigms and web 6.5isms, so they can get their 5 minutes in the lime light?
I remember experimenting with a C++ framework that would work somewhat similar. Lazy evaluation and such. http://sourceforge.net/projects/ditto, it's long since been abandoned because, well... the only thing this way of programming solves is a small part of the "R" in "CRUD". It worked brilliantly for showing information on screen and automatically updating all kinds of stuff whenever some data changed, but in the end it only solved the least difficult part of creating an application, and doing so at a relatively high CPU% cost.
It was also supposed to have lazy/dynamic/late-binding expressions (i.e. lazy_int a = b + c;), but it turns out you only very rarely have any use for that type of expressions. At the time, there were also far superiour C++ frameworks being developed which could do these things as a side effect of supporting more useful programming models.
This RP thing doesn't really seem to do anything more than that.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
This won't create an application. It just automates calculations that populate form fields. That's the easy part of most applications. The hard parts are the business logic that isn't just figuring out the values for form fields. In addition, you have nasty cascade effects. What happens in this system when the price for an item changes at the same time the customer's changing the number of that item they want to order, the customer's credit limit is being updated and payments are being applied to his 26 orders each of which causes an update to his balance due? You end up with a massive cascade of updates propagating up the chain, with each change causing changes to values that depend on it which in turn cause yet more changes. Multiply that by thousands of customers and it can kill the systems dead. Meanwhile a non-RP system will handle it cleanly and without dying by simply not updating values until everything's finalized. So as the accounts payable system applies payments to his 26 invoices his balance doesn't change, then when payments are done it updates the balance once to reflect the payments.
I'd use RP to help with things like forms where I want live updates and RP will help automate what I need done anyway. Lots of other areas... not so much. It'd be just another tool in the toolbox, and it's not quite as useful in the real world as it appears in simple academic examples.
Part of it's that the people describing these things often haven't dealt with truly large, complex applications before. Simple example: we have 5 fields in a request, each data file has a match value for each of those fields, and we want to grab the file that matches any one of the fields in the current request. "The" file? Um, what happens when I have 3 files each of which matches a different field of the request? Which one should be grabbed? Nobody thought of that. They say "It doesn't matter, just grab one.". And then later "We wanted file 17 that matched field 3 being X, why aren't we getting that file?", answer "Because file 9 matches when field 1 is A, which it is in your request, and we find file 9 first when checking.". Simple request, just grab an arbitrary file. Translated into the real world, complications pile up that were always there but nobody wanted to talk about because they were... well, complicated.
I don't understand this new slashdot scheme where the link to the story in the summary, leads to another story under the slashdot domain. Slashdot Business Intelligence? It's still slashdot... The summary on slashdot.org proper and the story under Slashdot Business Intelligence both offer a comments section. Am I supposed to double post or what? I'm posting this here out of habit but in the midst of confusion. The road to hell is paved by websites that suffered unrecoverable identity crisis.
Brought to you by Carl's Junior.
I started using reactive extensions for .Net for over a year ago and am constantly learning new ways to use it. UI, message busses, async calls, business objects, the list keeps growing. It is by far the most useful programming concept I've learned in the last year. There is one for javascript if you are not into .Net
This article is just marketing bullshit double speak without a concrete implementation.
I like the idea of having variables auto-recalculate based on a dependency tree built automatically by a tool. But recalculating vars sounds kinda slow, esp when resolving the dependency calculations involves time-consuming operations like database fetches. What control does the tool provide in limiting the timing and scope of the recalcs?
I hope this silver bullet is better than the last one.
"I'm not too worried about that" said every vampire on the planet.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
which I think fits into this model. It was neat, but kind of a mess. As soon as I wanted to do something the creators of OL didn't think of I ended up writing a lot of JavaScript and stuffing it in weird places.
I like the HTML/Android approach: Use Declarations for the UI layout and let old fashion code control the logic.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
Vampires do not fear silver bullets. They fear stakes and sunlight. Werewolves fear silver bullets.
Write failed: Broken pipe
to /active/ programming.
Anyways, seriously, please, please please my fellow programmers. Don't jump
on this just because it looks or sounds esoteric.
Tl;DR [yet], but initially this sounds a lot like HDL's like Verilog and VHDL. The two fundamental constructs there are things that happen continuously (collections of simple logic gates) and things that happen based on a clock (registers, built around flip-flops). At a high level, this sounds like A = B+C is a continuous adder, and A is changed instantaneously with B and C. In the HDL world, this gets "compiled" down to silicon, but in a software world this would be radically harder to do, because you have to notify anybody listening to A that B and/or C has changed. Yes, it's like a spreadsheet in that sense.
GStreamer - The only way to stream!
is in its usual place.
OK, B+C can be evaluated once when you assign A, or it can be evaluated every time you evaluate A. Sometimes you know that A will not change after you calculate it, or that a fresh value of A is required, and sometimes you don't.
Delaying the evaluation of A is usually safe; but it can sometimes be very slow. Like it or not, performance still matters. If you evaluated A unnecessarily inside a tight loop in a game, you'd be dead. Even less CPU intensive thinks like web apps could be killed by this kind of thing, if A is evaluated in JavaScript every time you move your mouse pointer.
Evaluating A every time is not only slow, it might not even be correct. Let's say you pull in some weather data and then render a map. The frontal zone should be a smooth line. If you pull in new frontal boundary data while rendering the map... you'll render a broken front.
Sorry. It looks like the programmer still has to think. Re-calculating A every time the symbol A occurs may or may not be required.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
Vampires do not fear silver bullets. They fear stakes and sunlight. Werewolves fear silver bullets.
That's why they said, "I'm not too worried about that"
That is "recent" as in "about half a century old"?
The idea of embedding a calculation into the system that is automatically updated by underlying data changes -- is that not just a database view?
We use this sort of technique quite widely in a Ruby on Rails app I work on -- complex calculations such as for profitability and cash flow are defined as views in Postgres, and referenced by the app as read-only models. Thus we can: Profitability.where(product_id: 27).group(:month).sum(:value)
Performs monstrously fast, as is extremely flexible. It breaks the whole "for the love of gods don't put business logic in the database" separation of concerns idea, but we have a system to ship right now and we can't wait for RoR performance and flexibility to catch-up that much.
Yeah, and in client side systems they're known as call-backs or event driven, etc. etc.
I don't know about you, but if it's old let's call it new!
Blade wants a word with you.
It is pitch black. You are likely to be eaten by a grue.
Event-driven programming is HELL, except for interfaces. Every once in a while a DB trigger is justified, but event-driven languages have failed time and time again because... it's impossible to predict what will happen when and avalanche of eventual complexity causes the system to implode under its own weight.
For a programming language to make the cut, it must be utterly predictable down to the last side-effect.
Sent from my ENIAC
I'm just seeing an observer pattern with publishers and subscribers. It's been done time and again in probably all the major languages. In the .NET space, using aspect-oriented decoration and an expression store, both of which are declarative, well understood, and promote reuse, this concept is already covered. Regardless of the platform, any implementation is going to be constructed in the context of the domain being addressed, and I believe any one-size-fits-all approach would lead to a confusing mess.
'In the RP paradigm, you can create complex applications from a series of simple declarative statements. Many of the details of implementation and work of gluing together various sorts of application constructs can be hidden because of the declarative approach.'
Sounds like another wishy-washy piece of shit language designed to be "easy", "fast", and "human readable" that will end up being kludgey, slow, and a pain in the ass.
Are there any examples of real-world reactive databases in production or development?
No, I will not work for your startup
'In the RP paradigm, you can create complex applications from a series of simple declarative statements. ..."; or, even , visually:
http://hypercard.org/
It's too bad the Newton wasn't just a hypercard engine, preloaded with a couple of removable apps.
Everyone knows the last vampire was killed off by a teenage film series.
In Coursera's Reactive Programming MOOC, the difference between reactive programming event-handling and traditional event-handling is described in two slides from the introductory lecture:
http://subbot.org/coursera/reactive/callbacks.png
http://subbot.org/coursera/reactive/howtodobetter.png
A traditional Java event-handler is first presented, and the problems enumerated: it relies on a side-effect (the variable "count" in the example), which involves shared mutable state; events and their handlers are not first class. Reactive programming tries to do better so that complex handlers can be composed from primitive ones.
Good god (or in this case bad god)
Every damn company I have ever worked in has some (albeit smart) idiot who creates a smart excel/access application which does wonderful things.
Who then distributes it to xyz people
Then the datasource (ip address) changes and the thing breaks.
Or the datasource changes (database table changes) and the thing breaks
Or his hardcoded status codes changes or is expanded and the thing breaks
Then they call you out of the blue and say, please fix it!
Access is an evil which should die. Advocates of it are welcome to come debate the matter with me. I need fresh meat.
Excel is extremely powerful and a handy tool, but as soon as anyone adds a datasource they should come and debate the matter with me.
There are three kinds of falsehood: the first is a 'fib,' the second is a downright lie, and the third is statistics.
Yes. After the site stops processing once it gets about 20-50 users, you react by displaying a 'failwhale' graphic, firing your development department and hiring new developers that don't code in 'hip new language that just entered alpha last week'.
Sleep your way to a whiter smile...date a dentist!
The rules engine I've seen (BRMS) utilizes the Rete algoritm (http://en.wikipedia.org/wiki/Rete_algorithm)
actual cascading data modification based on reverse trigger like syntax would seem to have the level of intelligent design choice on par with Intercal's comefrom statement.
Right, I don't think current DB technology, based as it is on 1970s timesharing mainframes, is up to the task. I suspect to make RP work nicely with distributed updates you would need to abstract out the idea of 'database' a bit; make it more like the Git/Mercurial model, where there's a big tree of data that anyone can hook into and receive updates, but can't affect anyone else directly.
This is the direction I think the Net will eventually have to evolve; things like Content Centric Networking are a step in this direction, if we added computation on top of that, we'd be getting very close to something much like the original 1970s 'hypertext' concept. Ted Nelson's Xanadu -- confusing, patent-ridden and badly-specified as it was -- I think had a concept of 'applitudes', applications as spreadsheet-like cells that can import data from anywhere and use it to update themselves. Imagine what you could do with a Web of live data like that.
You are not a brain: http://books.google.com/books?id=2oV61CeDx-YC
I remember experimenting with a C++ framework that would work somewhat similar. Lazy evaluation and such. http://sourceforge.net/projects/ditto
MUCHO KUDOS FOR THE LINK, my man !
It was also supposed to have lazy/dynamic/late-binding expressions (i.e. lazy_int a = b + c;), but it turns out you only very rarely have any use for that type of expressions. At the time, there were also far superiour C++ frameworks being developed which could do these things as a side effect of supporting more useful programming models.
This RP thing doesn't really seem to do anything more than that.
Looks like this weekend I'll be fully occupied
I need to know more about all this !
Thanks again, man !!
Muchas Gracias, Señor Edward Snowden !
Yeah but if we can put it in the Web 2.0 then its for sure something new.
In the RP paradigm, you can...
I'm sorry, but as soon as I read the word "paradigm" I knew you were management and therefor had no idea what you were talking about. I could be wrong, but I can' risk reading any more of the post for fear of running into that word again. I don't want to get contaminated.
Also I think the Observer pattern is all about this.
The Tao of math: The numbers you can count are not the real numbers.
No magic smoke. Just an infinite loop. Unless they have made precautions to prevent recursive re-evaluation. In that case it's either a simple update or an error. Or if they use lazy evaluation, it's even no action at all (but any subsequent read of A or B may result in any of the previous options).
The Tao of math: The numbers you can count are not the real numbers.
There is a nice book about composing complex systems from more primitive ones: Structure and interpretation of Computer Programs
shhh. don't mention this. it's a secret weapon. well, it has limitations, but it's very useful under the covers.
Just instinct, but I'm guessing you can buy something that does it from Espresso Logic, of which Val Huber is the CTO.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
I guess you're talking about http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-22.html#%25_sec_3.3.4 "A Simulator for Digital Circuits":
I suppose the modern version uses events more for user interaction, and also gets rid of lots of idiotic, stupid parentheses.
Yes...sorry I have no mod points for you Bugs...
Boy, they really hate grammar Nazis around here, don't they?
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
Yes, there are products and libraries that do this for you. It is probably not something you'd want to implement yourself. Espresso Logic has a commercial offering that provides a combination of declarative logic, declarative security, and actions extensible with JavaScript to build complex SQL based REST systems.
Performance is absolutely a key concern. But in the same way a database query optimizer can (usually!) select an access plan, it should be possible to do the same for reactive update logic. At the systems level, reactive servers can be stateless rest servers that cluster. Within the rest server, a transaction can be analyzed to determine what *actually* changed, and prune the dependency graph and save (you are right - tons of) SQLs.
It is a product. A service actually, so you can fiddle without download or install, at espressologic.com. Earlier generations were Wang Pace and Versata. We'd love to hear your reaction. On the client side, EL is a very close approximation, and Microsoft has done some work: http://msdn.microsoft.com/en-us/data/gg577611.aspx
I sincerely apologize for the harsh tone. That was uncalled for, and I shot off without reading what I wrote and how it comes across. Again, I apologize.
Like you I have some "triggers". One of my triggers is blanket dismissal. I think what I perceived as your blanket dismissal was what triggered. You are correct about Ruby-on-Rails. It turned out that it was not up to the job. But at the same time it *did* popularize the "web-MVC" pattern in a way that Java frameworks (e.g. Struts) never achieved.
When someone praises a new technology as the new "best thing since sliced bread", we should approach it with a healthy portion of skepticism. However, we must also keep our minds open to new advances.
Reactive Programming is not an alternative to OO or FP. It complements both, and at the same time it promises to make certain aspects much more manageable, like e.g. asynchronous and parallel processing.
Reading slashdot one-liner: (irm http://rss.slashdot.org/Slashdot/slashdot).rdf.item | fl title,desc*
http://deduced.sourceforge.net/
I liked the amount of functionality I could get in just a few lines of code. Even the user interface was generated by reactive rules.
The thing that puzzled me the most with this tool is the lukewarm reaction of most developers who looked at it even when I proved that it could do things in about 10 times less code than a typical OOP solution.