Slashdot Mirror
Has Flow-Based Programming's Time Arrived?
An anonymous reader writes "Flow-based programming keeps resurfacing lately. FBP claims to make it easier for non-programmers to build applications by stringing together transformations built by expert programmers. Many projects have already been using similar approaches for a long time, with less (or different?) hype. Is it time to take a closer look at flow-based programming? 'Clean functions – functions without side effects – are effectively pure transformations. Something comes in, something goes out, and the results should be predictable. Functions that create side effects or rely on additional inputs (say, from a database) are more complicated to model, but it’s easier to train programmers to notice that complexity when it’s considered unusual. The difficulty, of course, is that decomposing programs into genuinely independent components in a fine-grained way is difficult. Many programmers have to re-orient themselves from orthodox object-oriented development, and shift to a world in which data structures are transparent but the behavior – the transformation – is not.'"
old is new is old is new is old is...new?
Coming from a fairly basic understanding of programming (I'm happy pushing bits around microcontrollers, but Object Oriented design makes my brain dribble out of my ears), isn't this how programs are supposed to be written? What's the big deal?
Gee, never heard that one before.
What the people pushing these ideas don't seem to know is that it's not the tools, it's the way of thinking about a problem. I once worked at a place where we made a manager a tool that would let him create his own reports, and he immediately started adding up all flavors of apples and oranges (e.g., dollars of this and pounds of that). Then he wanted the small IT staff to help him make sense out of his reports.
Sheesh, evil *and* a jerk. -- Jade
When you mix the two and don't tell the application programmer then you get Symbian.
I use flow-based programming every morning, usually the first thing I do when I wake up.
No.
Flow as in flow chart programming. Think UML diagrams with prebuilt libraries. Known as visual programming if you're old enough to remember the hype the last time... or was that the last last time this came up? It hasn't taken off in 30 plus years... probably a reason for that.
it wasn't a good idea to start with and it hasn't gotten any better since then.
Anons need not reply. Questions end with a question mark.
scripting, in sh.
See Betteridge's law.
NI LabView has had flow based visual programming for more than a decade. The nice thing is make parallel flows to your data, and it's multi-threaded.. It makes an easy way to visualize that something is run in parallel.
I'll go against the push button responses on here and say that it definitely might be the future. As systems become complex enough that we need UML diagrams to visualize what's happening, this is like taking the UML and actually turning it into code. It gets really interesting when you're capable of making your own flows out of existing flows to simplify your task. Why can't it complement code and not replace it?
Because it's not a novel concept, and real programmers have been doing this long before some kid thought it wasn't a thing and started calling it FBP.
I have been tryign to programming in Labview, in which you literally string components together. If you want to want to make anything even a bit more advanced than the stuff in any tutorial, it becomes a mess. And more important: you will still need the same thought patterns to get your stuff to work. It would be just like learning to type in Japanese. Everything looks completely different, but it still does the same.
Your fancy FBC compiler will be written in C[++], with its primary purpose being that folks who suck at programming also get to do some programming...
Help! I am a self-aware entity trapped in an abstract function!
And before long, the whole company descends into chaos. Can we please keep non-programmers AWAY from trying to cobble together automated systems?
While we're at it, can management please be made to bother to consult the development department BEFORE making technical decisions on things such as data structures, key formats etc?
so what is the point?
I don't know if this is the same concept as the new Streams API in Java 1.8, but if you are a Java guy, I think they are *really* cool: http://download.java.net/jdk8/docs/api/java/util/stream/Stream.html
The basic problem is that, while it sounds great in theory, in practice the transformations you want don't exist. If they did, you'd have software doing the job already and you wouldn't need to create it. Your business isn't going to go very far just doing the same thing everybody else is doing, is it? You need to be doing something they aren't. Which means, in this context, you need transformations that don't already exist (either because they haven't been written yet (ideally) or because the people who wrote them are keeping them confidential to keep you from copying what their business is doing (less than ideal)). So on top of your FBP team stringing components together, you're still going to need that expensive conventional development team to write the components for the FBP team to string together. You haven't saved much, in fact it's probably costing you more than just having the conventional dev team.
Plus, stringing components together isn't quite as simple as it sounds. Real-world systems usually depend on interaction and complex relationships between otherwise simple components. Keeping track of the synchronization between all those parts and keeping everything coordinated is usually the hard part. For instance, when creating an application to set up appointments the part where you take a set of existing appointment slots (some occupied, some free) and a set of one or more slots the customer would like ordered by preference and determine the best requested slot to put them in is easy. Picking up the current set of slots, putting them up on a Web page in a sensible way, letting the user select what ones work for them and set the order of preference, sending that information back to the server and, across this whole process, making sure that either nobody could alter the assigned appointments between the time you picked them up to display and the time the customer hit Submit and you started to sort out the assignment, that's nowhere near as simple. Doing this in a modern system with multiple servers where the submission may not go back to the server that handled the initial request, when you've got thousands of customers and hundreds of your users making, changing and cancelling appointments at the same time, ... can we say "highly non-trivial"? And it really doesn't fit the FBP model at all.
Even where things fit the model, it's rarely as simple as "just string them together". I work with software that fits that model. Well, it did. Once. Long ago. And then the business people said "Oh, but customer A wants it do to some other thing if it's a transaction for them.". Followed by "Well, we want to do X, unless C applies in which case we want to do Y.". "Oh, unless M was specified for the account, in which case do X even when C applies except where X set V to 7 we need it set to 14.". Lather rinse repeat for 10 years and the quick one-line summary version ran to 5 pages single-spaced. Until that is we tried to print it out and found things were nested so deep some lines were starting past the right-hand edge of standard letter paper, so it's more like 10 pages, legal-sized, in landscape mode.
I think where this would be interesting is using Behavior Engineering (not UML!) to debug the design (and requirements) and then have automatically generated skeleton loaded into some sort of Flow-based programming system. If you're unfamiliar with BE (it's not really taught outside Griffith University) then you can have a look at http://en.wikipedia.org/wiki/Behavior_Trees
It may or may not be a good idea, but it's not new. The article is not very good, either. It's all over the place, from punched cards to XML.
Data flow programming has been tried many times. It's used heavily in GPUs, where pipelines are the only way to get work done. So there's reasonably good understanding today of what can and can't be done by data flow methods.
If you like "flow-based programming", one easy way to do it is to write programs in single-assignment style. Assign to each variable only once, and make it "const" or read-only if your language supports it. This makes your program a directed acyclic graph. Single assignment is a lot like functional programming, but values have names and the nesting depth of calls doesn't get so deep. It's also possible to use the same result as input to more than one function, which is hard to do in pure functional programming (That's the difference between a directed acyclic graph and a tree.)
The use of pure functions makes for cleaner programs, but more data copying. Data copying isn't necessarily bad today. It's cheap to copy data you just used or created, because it's in the cache. Modern CPUs have wide buses, are good at copying, and can probably do the copy in parallel with something else. Don't avoid copying data to "improve performance" unless the data item is large.
The place where this all comes unglued is when the program's goal is to affect something, not grind data down to a "result". Trying to write a GUI in a functional style is tough. (I once wrote a GUI in pure Prolog. Very bad idea.)
We can all go back to programming in C/Pascal/Fortran and forget all this OO crap now, can we?
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
"Many programmers have to re-orient themselves from orthodox object-oriented development, and shift to a world in which data structures are transparent but the behavior â" the transformation â" is not."
No, they most absolutely do not. Ideas come and go, and if programmers would 're-orient' themselves each time, they would be spinning endlessly and not get any work done.
As always, choose the proper approach, the proper tools, the proper environment, the proper language, and do the job. Let the idiots 're-orient' themselves however often they want to.
I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
grep EVENT log*.txt | sort | uniq | awk '{print $2}' | ssh reportserver "gzip > results.txt.gz"
Flow based, side effect free, distributed computing on one line. There is a reason shell scripts refuse to die in the face of python, perl or anything else.
“There’s two roles: There’s the person building componentry, who has to have experience in a particular program area, and there’s the person who puts them together,” explains Morrison. “And it’s two different skills.”
Nope. There's the person who can do both, and there's the person who's only smart enough to draw lines.
I still code in PET Basic.
Does this for mathematicians. That's why I always hated working with it, I din't know most of the functions in it and I wanted control over my program.
Talend.com offers Talend Open Studio - a great free software product for flow-based visual programming, based on Eclipse.
I used it for projects for 5 years and was amazed, how easy to make complex data transformation by dragging and dropping components and links.
A result is a java (or Perl) program, which can run standalone.
The program is a visual data flow, easy to modify and understand even for non-programmers.
I really appreciate Talend team effort and recommend this product for all, who need data transformations.
That sounds like a Structure Query Language that was developed some years ago to allow business users to query databases. It would free programmers to focus on programming instead of running reports. Looks like it worked pretty well.
-- Reality checks don't bounce.
We have proven implementations of "flow-based programming" floating around since the 80s.
Can it flow in the cloud?
All hail 4GL
-- I ignore anonymous replies to my comments and postings.
I scanned the summary as "data flow programming" and had flashbacks to university research projects around data flow computing... in the mid-1980's (probably before many of the 'hip' programmers were born). Alas, it was not to be the resurgence of programming in SISAL... just a another programming-by-drawing rehash. Imagine my disappointment.
Patent litigation: A doctrine of Mutually Assured Destruction... in which everyone seems willing to push the button
Can anyone tell me what October 18, 8 pm to Saturday October 19, 1 am (Eastern Time). is supposed to be in UTC so people might actually be able to calculate their local downtime-frame?
Since I've progressed beyond 1979, instead of mentioning UNIX and pipes I'll point out that Powershell is especially powerful in this facet of a pipeline/flow.
Also, Workflow Foundation (WF).
The code is the model. You can't make it much simpler. You can simplify the common things, but this will make the language less general, more domain specific. For programming synthesizers it makes a lot of sense.
If a function call is represented with two boxes with a line between them, the simplest of applications will be a confusing mess.
A witty
"Everything Flows"
-Cyberneticists
As some other people already remarked, on the face of it this looks like the venerable Unix approach of small tools in a script. My point is that the real world outside, that you are trying to capture in a programming language, can be very complicated. For some domains, e.g. logic or arithmetics, the language can be pretty complicated too - see APL, LISP or Prolog.
But in thirty years of programming (computational linguistiscs), I have found that Unix scripts, awk and plain C covered pretty much 90% of (my) programming needs. If and when necessary I tacked on a larger database system. Of course I tried the new (well, in the nineties they were new) OO systems, but I rapidly got lost in a jungle of libraries and methods and even more documentation. Compare that to the almost ascetic (and aesthetic) clarity of the Unix environment.
Yes, I feel that Unix still has a lot of mileage in it and intentionally or not, this item and the reactions on it, confirm me in this view
Paai
Flow-based computation has always been what occurs under the hood, and it's only blind programmers who haven't realized that that's what they're actually doing, and hence they do it poorly.
This sounds awfully like the Last One, a British 'language' from 1981 that used lines of code to pull together pre written blocks then produce the executable:
http://en.wikipedia.org/wiki/The_Last_One_(software)
I want a list of atrocities done in your name - Recoil
This is what BPEL and executable BPMN promise. In practice for the business its "string these together and get a programmer top write a transformation or show you were to obtain a key data element that is missing from the stream" at best, or "try to string these together in an illogical way and complain to the development team that the components don't know what you mean".
I have no experience with programming so I could definitely use something like this. Reminds me of an old game my son used to play.
Am I the only one who read "Flaw based programming"?
This has been tried over, and over again many times over the last 30 years.
It just plain-old doesn't work. You get inefficient, bloated code at the best of times, and 99% of the time you need some sort of custom function that still requires conventional software development.
So, no, flow-based programming's time has _NOT_ arrived.
I was thinking of that too. LabView looks utterly cool on some sort of brochure or in a movie scene but was a slow way to put together scripts (just like "it's a Unix system, I know this" - with the similarly cool looking but ultimately useless file viewer).
This idea was obsolete when I did a subject in doing this sort of programming for analog computers (patch cables to amplifiers) in the 1980s. Oddly enough a good way to design analog computers was to optimise the model which started with discrete components into something like a script and then convert that back into a simpler system of discrete components.
A complex problem won't be simpler just because the tool isn't as powerful. All these ideas comes from a fundamental misunderstanding about what it is that actually makes system development hard.
So you mean like unix pipe. Where you have those wonderful transformations like sed, grep, cut, sort, etc ? Youngsters.
If programs would be read like poetry, most programmers would be Vogons.
Er, yes ... the way of thinking about the problem in dataflow programming languages is the whole point. The tools are an aside (and in fact you don't want to be writing a complex vector calculation using black boxes of add and multiply anyway).
If you're wondering what flow-based programming feels like, I work on an open-source framework called FlowVR.
It is advertised towards high-performance computing but suits small applications just fine.
Actually that's a poor example; reporting is an area where this way of programming has worked well, at least in my own experience. In most places I've worked, managers knew bugger all about programming or SQL, but they understood their own data well enough to produce reports themselves in tools like BusinessObjects, even fairly complex reports. In this case, the tool is an enabler for people who already know how to think about the problem.
If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
I have |
What else do you need?
People are not able to describe their problems and requirements in a conflict free way. You may reduce the amount of technical experience, like how to use J2EE or network communication etc., however, the complexity of software solutions cannot be hidden by a non creative system. Research has tried this, they even tried to use AI, it all cannot help you with the first step. Understand what you want. The second step, transforming requirements to architecture is also a creative task and cannot be mechanized. And the general population is not able to conceptualize problems properly. Look at BPNM models drawn by business people. They mostly do not what the customer wants.
Lambda calculus predates the invention of the compiler. It's always funny when programmers rediscover stuff and think they've invented something new. :P
And before long, the whole company descends into chaos. Can we please keep non-programmers AWAY from trying to cobble together automated systems? While we're at it, can management please be made to bother to consult the development department BEFORE making technical decisions on things such as data structures, key formats etc?
The ugly trick is that if you want to keep non-programmers away from something, some programmer has to take one for the team and gingerly approach whatever horrible little ad-hoc business logic problem or whatever the non-programmer was trying to solve. If dual-class programmers were in ready supply, it's not like the non-techies would be clamoring for more chances to get their hands dirty.
People who think that anyone should be allowed to program should be forced to read books written by anyone. Writing takes skill. Sure there are languages which might seam easier to write in but that doesn't mean any idiot can even write with a decent book.
Gee, never heard that one before. What the people pushing these ideas don't seem to know is that it's not the tools, it's the way of thinking about a problem. I once worked at a place where we made a manager a tool that would let him create his own reports, and he immediately started adding up all flavors of apples and oranges (e.g., dollars of this and pounds of that). Then he wanted the small IT staff to help him make sense out of his reports.
Yep.
And even if they do get good at using it, now you've got a {whatever} doing "programming" (in a limited clunky "environment/language") instead of {whatever it was they were supposed to be doing}.
There's a marketing manager in my last job's company doing exactly that (not entirely unrelated to why it is my last job). I'm glad he's having fun playing programmer, but he does even less actual marketing than he was doing before ...
The flow what you are describing is a business flow. That is something you must not use this model for. In a business flow you use all kind of side effects you store in a database. don't mix them up because the business process have a flow, that is different from a program flow.
Using the wrong technique to transform a business flow into a program will get big drama. Because you have so many side effects, you get a lot of states/ global variables you have to test for in each of the sub-processes, soon all hell breaks loose over the number of possible states.
Beside that, a business is 90% generic, only a small part of it unique. SO only customize that part of your software, the other part might be better of with generic building blocks. But that is a different discusssion after all.
http://en.wikipedia.org/wiki/Betteridge's_law_of_headlines
Spreadsheet software such as Excel or Calc are very good examples of flow-based programming, and it seems non-programmers can cope with it. However, we all know how hard it is to get larger-scale projects in Excel to work :)
You should all do some homework before throwing out all this prejudice. Flow based programming has been successfully used in many implementations ever since it's invention in the 70's at IBM. One Canadian bank has been running (and maintaining) an FBP program continuously for over 40 years!
You are right that FBP (or something very close) can be found in successful systems such as Unix pipes, LabView and Apple quartz composer. If you read FBP inventor J P Morrisons canonical book on the topic (linked below) though, you will notice that there are some details to the FBP specifications that should be observed, in order to gain all the full benefits of a full-featured FBP implementation.
Some homework suggestions for you all:
1. Why not start with NoFlo creator, Henri Bergius' excellent post, "NoFlo Kickstarter, the hacker's perspective":
http://bergie.iki.fi/blog/noflo-kickstarter-launch/
2. Watch the intro video to FBP and NoFlo:
http://vimeo.com/72065207
3. Read up/watch some more on the NoFlo (finished) kickstarter page:
http://www.kickstarter.com/projects/noflo/noflo-development-environment
4. Read up a bit on FBP inventor J P Morrisons website:
http://jpaulmorrison.com/
5. Join the discussion in the FBP group:
https://groups.google.com/forum/#!forum/flow-based-programming
6. Join the FBP Google+ community:
http://gplus.to/flowbased
7. Last but not least, read the book!
http://tinyurl.com/fbpbook
Ok, not everybody is prejudical of course, but that's very recommended links in anyway! :)
http://www.youtube.com/watch?v=u7KIZQzYSls
A modern library implementing dataflow programming is: https://rx.codeplex.com/
The relationship between the actor model and flow based programming is discussed here: http://c2.com/cgi/wiki?ActorsAndFlowBasedProgrammingDiscussion
Personally i like using f# with rx using this wrapper: https://github.com/fsharp/FSharp.Reactive
I see FBP as a generalized form of functional programming where you can, and have to, specify how the expressions are put together temporally.
It gives you the advantage of automatic (in rx you have to specify the scheduler) concurrency but is cumbersome for stuff that is easily expressed in a synchron way (List processing...).
I guess you would want the most coarse grained granularity.
A nice book on FBP is: http://www.jpaulmorrison.com/fbp/ (the first edition is freely available)
As for the idea that FBP is coupled to visual programming and will enable non programmers to write complex programs, well:
Visual programming makes writing programs for easy problems even easier and for complex problems intractable.
And guess what: The easy problems were already easy to begin with!!!
Indeed LabView is quite close to the FBP specification. Have a look at this thread in the flow based programming forum where this is discussed in quite some detail:
* https://groups.google.com/forum/#!msg/flow-based-programming/zhkfHlunu5Y/kYDGVsqtQaoJ
Indeed, unix pipes, especially if usning named FIFOs as well, is a subset of the Flow based programming paradigm fullfilling quite a bit of it's specifications. A full FBP implementation adds a few things like option ports (could be done in shell with parameters though), custom information packet design (is only text strings in unix), and so called "initiation information packets", and surely a few more.
There is actually an attempt at implementing a full FBP system in shell scripts only, which is kind of cool. It was discussed in the flow based programming forum,here: https://groups.google.com/forum/#!searchin/flow-based-programming/shell/flow-based-programming/PC96WYOAwAU/ICRZg_5K1XMJ
it's means "it is", even in the '70s. It always amazes me that programmers, who basically learn 500 different languages to say the same things can't handle the simple apostrophe.
I take the "hell" of Labview everytime over the HELL that is html+css+js.
If you have $45 to burn, grab a free copy of the Unity Game Engine and the $45 PlayMaker add-on.
There are some things it works great for, others not so much. We use a neat fbp environment for creating media installations called vvvv (vvvv.org). It's scope & focus are a bit specific, but for prototyping it's just ridiculously fast to develop with.
Real programmers program in assembly. Actually in any language but if you cant program in assembly, you are no programmer. ...
In many companies I see "programmers" with their masters degrees that cannot write a sort. Not even a bubble sort.
They are not programmers. They are so used in using Libraries that miss the target. Actually i have seen projects run twice as long because the library is not parsing something right. In 1982 wé wrote our own math coprocessor since the "built in" was actually producing a wrong answer in a specific calculation. Oh those glorious DOS days
Thanks, yea, I looked up this recently but I don't seem to ever learn this properly. Always tend remember it the other way around :/
After my boss reads that I can see the look in my bosses eye thinking that. Especially when I hint around for a raise.
All hail Visual BASIC.
This is something that has been available for quite a while in industrial control systems used for oil+gas, process industry and whatnot. The language is called Functional Block Diagram, and is a simple dataflow language for building control applications. I have used it for the better part of a decade, it works great and can be a nice replacement for the older ways of programming PLC/PACs, which has traditionally been ladder logic.
Q.E.D.
Can you be Even More Awesome?!
-- Bald Eagle [n]: an animal you paint on the side of something hideous, to prove you're more patriotic than your neighbor.
So you've seen my neighbor's wife's tattoo.
This notion of allowing non-programmers to do programming is flawed on its face. The challenges of programming can not be overcome by drawing some cute pictures. And if you have expert programmers making the building blocks it will only take them a few minutes more to connect them which eliminates the need for the "non-programmer" entirely. Then there is this notion of "pure functions". Dude, functional programming is a fad. Real programs manipulate data.
I've been "programming" with Simulink for 5 years now and it's great for control systems but shit for most other kinds of programming. So let your controls engineer use it to design and test and even generate C code. Then drop that code into your app and call it as a function. Never let that guy convince your organization that this is the way to go and ALL of the software should be created this way.
This sounds like the event system of VRML. I really like that. There are "EventIns" and "EventOuts", and they can be transformed. So you can define a "TimeSensor" that generates a scalar signal to be fed into a RotationTransformer which is linked to a piece of geometry to make it spin.
Or you could define a ClickSensor that feeds a boolean signal into the TimeSensor that ...
It workes really nice.
Nae king! Nae laird! Nae yurrupiean pressedent! We willna be fooled again!
/r/DataflowProgramming subreddit for what's going on in modern dataflow and flow-based programming.
http://www.reddit.com/r/DataFlowProgramming
I've really only seen people getting serious about writing libraries for C++ in the last decade or so. Boost has come a fantastically long way in that time and adds most of the stuff that the newfangled C++ STL was lacking. In fact, they added a lot of the Boost stuff I've been using to the C++11 spec, pretty much verbatim. I can pretty much change a header include from to the new C standard thing, change the namespace from boost:: to std:: and not change any other code and it'll work. I was actually stunned they kept the thread syntax the way boost did it.
No matter what you're trying to do or the programming paradigm you're trying to do it in, you will never be able to just crap out some code without thinking about its design and what you're trying to accomplish. When people think of the difficulty of programming, that's where the hard part is. Learning a language is easy. Learning to express your instructions to the computer in a way that's maintainable, efficient and reasonably bug-free... that's not so easy. I've seen many companies through the years constrained by the design of their in-house software. Once the business starts changing to meet the demands of its software, you start having problems. "We can't do that because our software won't allow it," is a huge drag on a company. You start losing opportunities not because something is not possible but because something is not possible with the tools you have.
A lot of the programming fads (OO, Functional, Ruby) imply that you can just start crapping out code without thinking much on what you're trying to accomplish. Buying in to that hype is foolish, and potentially damaging to your business. Some of the fads actually do make it easier to write reusable and maintainable code (OO and test-first come to mind, though I haven't drunk enough of the test-first kool-aid to actually write the unit test first. I usually don't wait much longer than the object I'm testing being "mostly done" to write a test for it, though.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
I'm a hardware/software engineer (i.e. I do both). By analogy to logic design (ASIC's, FPGA's), this is a big step backwards. Once upon a time logic was designed using schematics. Everybody gave that up about 20 years ago, and switched to HDL's (VHDL, Verilog). For those unfamiliar with them, HDL's are text based hardware description languages which get compiled and then synthesized into gates, flip-flops, etc.
Despite this change 20 years ago, there are those who cling to a love of schematics. You can draw a schematic and have it automatically transformed into HDL, and from then on it follows the same design flow. It's a mistake to do things that way. As recently as 5 years ago (and perhaps still) I had a customer who insisted that the top level of the design be a schematic (with the boxes in the top-level schematic implemented however you wanted). It sounds nice, with a pictorial representation (which is especially well suited to hardware), but it turns into a mess for any non-trivial project. Top level diagrams are a great form of documentation, but when you try to use it for actual implementation, it gets so cluttered with detail that it becomes unreadable. Better to look at the text. I can't see how this would be any different for a non-trivial program.
I've been doing Oracle SOA development for the past six months. The BPEL and XSLT transformation tools are supposedly business user friendly however I have to drop into the source code several times on every project to perform simple tasks and to make corrections to the assumptions that JDeveloper makes. Even things as simple as defining you want the value from the Nth occurrence of a repeating element have to be done in the source code. God forbid you don't want to use oracle as your namespace I have to open up several source files in order to uniformly use our URL as the namespace for everything that we've defined.
Gee, never heard that one before.
What the people pushing these ideas don't seem to know is that it's not the tools, it's the way of thinking about a problem. I once worked at a place where we made a manager a tool that would let him create his own reports, and he immediately started adding up all flavors of apples and oranges (e.g., dollars of this and pounds of that). Then he wanted the small IT staff to help him make sense out of his reports.
While I understand what you are saying, it is hard to argue that Microsoft's Visual Basic and Access effectively allowed millions of non-programmers to build applications. Were they good, robust applications? Probably not, at least most of them. But at the time, those products were first released and centralized IT shops ruled, most departments would never have had their requests get high enough up the priority list to every be programmed by a "real" programmer.
Unfortunately, VB's own success killed it for the non-programmer because more and more features were added to the point that by the time it was VB.net, you needed to be a programmer to really use it and if you were, well C# provided much more functionality.
End users will find a way to get their needs met, one way or another. And, an IT shop that is viewed as thwarting a production department from being able to function won't last long in today's economic situation.
Listen here sonny, when I was just starting out "object oriented development" was anything but orthodox!
Mark my words, when you get to be as old as I am now people will be talking about newfangled things like "disks that hold data instead of putting it in The Cloud" and "getting that computer out of your brain and wearing it instead."
Mark my words.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
This isn't new. Back in the early 90s, there were various end user tools to design simple systems that used a card metaphor. Basically, you strung a bunch of cards together to build your program and for each card you told it what that card did (accepted input, made a decision, generated a report, etc.). In reality, it was building a flow chart (but not using the traditional systems). When it was all done, you hit the button and your program ran. If it needed changed, you edited the individual card or inserted new cards or removed old ones. You also had the option to open each card and tinker with the code (pascal, I think), but for the most part, each card was a black box just accepting inputs and having specified outputs.
Obviously, the programs weren't terribly sophisticated, but they were surprisingly functional and maintainable and for end users it was a lot easier than programming dBase or Lotus.
"Functions without side effects"...funny that they don't mention Haskell. Haskell is the *only* pure language (meaning without side effects) out there today with significant adoption. A recent presentation at CUFP[1] talked about how one commercial user saw *vastly* better reusability with Haskell than other languages because of purity. Haskell has been all about this idea (using a different name) for roughly 20 years. They've learned an incredible amount and now have a surprisingly powerful language that is used by a number of companies of all sizes. The community has a huge ecosystem of open source code available for reuse. For those drawn to the "flow" ideas, there's also Functional Reactive Programming (FRP). There are at least 5 Haskell FRP frameworks available today.
The ugly trick is that if you want to keep non-programmers away from something, some programmer has to take one for the team and gingerly approach whatever horrible little ad-hoc business logic problem or whatever the non-programmer was trying to solve. If dual-class programmers were in ready supply, it's not like the non-techies would be clamoring for more chances to get their hands dirty.
I don't think that is true. End users have been doing their own thing for decades. They usually just do it with spreadsheets and databases. In the old days it was Lotus and dBase, today Access and Excel. In the 90s there was a lot of VB programs written by end users that while not of professional quality, surely got the job done for their purpose.
The reason that end users get their hands dirty has nothing to do with the availability of programmers overall. It is the priority of their projects compared to the rest of the corporation's projects. In most organizations, increasing the programming staff another 10% is not going to get those smaller end user projects accomplished any sooner. Those programmers will be added to the big projects to get them finished sooner so the next big project can begin.
In many organizations, there has been a major push to decentralize IT precisely because of this. If management is going to be held accountable for timelines for projects that require programming, but that programming isn't high enough on the corporate priority list, then those business units want their own programming staff. Simply put, the manager needs to be responsible for all the resources required to complete the project. If they don't get their own staff, they will simply come up with creative internal ways to do it with end user tools.
As for IT and business logic, almost always, IT is the one that doesn't understand business logic. The end user usually has a good grasp of the business logic and how it fits into current operations. The professional programmer, on the other hand, usually wants to use existing patterns to speed up their development time, but causes the end user department to change the way it processes. In short, they forget that the user is the customer. That is why, in many organizations, IT has about the same approval rating as congress.
As per usual, the answer to the title is "No", but flow-based programming has its uses. I've recently come across GNU Radio, in particular its Companion (GRC) and the flow model is a perfect match for signal processing. Take a look at these GRC screenshots. With the advent of RTLSDR, more people are discovering this programming model.
A complex problem won't be simpler just because the tool isn't as powerful. All these ideas comes from a fundamental misunderstanding about what it is that actually makes system development hard.
But, if you break a complex problem down into it's simpler components and solve each of those components, then a simpler tool can be used on the components and the components strung together to solve the complex problem. That is how you solve a business problem in the real world. That is also how you solve a programming problem.
the transformations you want don't exist.
True for whatever your company competes on, but not for everything else (the so called "hygiene" factors, that don't need to be great, just basically work. e.g. Apple doesn't compete on their internal payroll software).
Your point on complexity is spot on. The abstractions and symbolic references that make programming so difficult handle complexity far better than the blocks and connectors of FBP and visual programming.
OTOH the two roles of component-maker and assembler is realized in the massive standard libraries and open source libraries of all mainstream languages.
Gee, never heard that one before.
What the people pushing these ideas don't seem to know is that it's not the tools, it's the way of thinking about a problem. I once worked at a place where we made a manager a tool that would let him create his own reports, and he immediately started adding up all flavors of apples and oranges (e.g., dollars of this and pounds of that). Then he wanted the small IT staff to help him make sense out of his reports.
Yep.
And even if they do get good at using it, now you've got a {whatever} doing "programming" (in a limited clunky "environment/language") instead of {whatever it was they were supposed to be doing}.
There's a marketing manager in my last job's company doing exactly that (not entirely unrelated to why it is my last job). I'm glad he's having fun playing programmer, but he does even less actual marketing than he was doing before ...
If their job involves the collecting of data and assimilating it and compiling it into various formats, etc., as in marketing reports, and they can't get the data unless they do it them self, I'm pretty sure the organization would fault the IT shop and not the manager. Managers are usually evaluated on results and if the result is the reports, then he/she is successful.
Look, money is made by companies like Facebook which is why they have the natural language parsing graph engine (waves hands) which should be used for what the OP is talking about but instead is used to stalk each others' friends' drunk photos.
If you are talking about "real programming" that is another ball of wax, but if you are talking about 98% of web apps, android apps, etc. there is already a full corpus and standard way of representing whatever is to be done. There only so many patterns.
These patterns all have to be created line by line. The problem is there are too many patterns for anyone with the attention span of a non-programmer to use it. And also they haven't trained themselves to think comprehensively and in depth about an application. Something like salesforce.com is a big step toward locking in^h^h^h^h^h^h^h^h^h^hempowering businesses to build their own software, however if has lots of cracks and places it falls down. For one thing, a lot of it drives the user to developing in a certain way, whereas if there was an easy to use synchronous real time messaging system that did not require you to write apex (kinda like java scripting) it would be more useful to more people.
The real solution is to have some kind of (hand waving) artificial intelligence built in, which would let you describe the kind of system you want built with prose and back of napkin drawings. Imagine an interative process in which a businessperson is describing to a system engineer what he wants, and "take care of all the obvious stuff as appropriate". Anyway, the salesforce and facebook approach has a managed web service / application engine / full stack infrastructure that self-augments itself over time and a community-driven open source version of this based on open repositories for example would be cool. This would release developers from having to reimplement the same kind of thing over and over. Some of this idea was visible in the beginning with catalyst perl app framework and perhaps in Moose object framework (not AI but resembling English
Unfortunately the development priorities of a facebook or a salesforce tend to target high school kids and marketers respectively, and profit-making above all, so you get something really powerful that nevertheless is not necessarily aligned with your goals. Anyway, the term "dataflow programming" is a bit old. I think the new term that will encompass it and much more will be NLP (natural language processing) programming or automated design perhaps autoflow for short.
it's a good model for audio synthesis, for example: PureData and MaxMSP are essentially this...
it's possible to code in a FlowBasedProgramming style but using a procedural or object oriented language.
actually, it's a subset of Object Oriented programming, in a way...
think about it...
there's a classic text on this by J. Paul Morrison. read it.
FBP proposes some very interesting ideas and could solve problems we have with 8+ cores and concurrency.
I'm looking forward at noflo project http://noflojs.org./
My users will love this. They are getting bored with overly-complex parametric CAD models, driven by somebody's undocumented Excel macro, reaching into the Access-database-from-hell, that pulls the wrong tables from SQL Server and our "who-changed-this-field?" ERP system, fed from a LabView rats-nest-of-code system.
Should be fun when they call me, because "computers are YOUR job, not ours".
Place nail here >+
As data flow seems to be a more well known concept for some, watch this excellent 6 minute exposition of the differences between:
1. Data flow vs. Flow based programs
2. Reactive vs Non-reactive
3. Event based vs Query or Call/Return based
4. Eager vs Lazy evaluation
Some of the most clarifying bits of speech I have ever listened to.
Seems like the parts are there.
when you learn all the tricks, it becomes a relatively simple way of doing a lot of programs, especially those involving hardware(as that is what it is made for). moving hardware with labview code is infinitely easier than with written code.
labview also makes it easier for someone to pick up your code and finish from where you left off, or modify it to suit his needs. written code is awful for this, even when you comment.
Oops, forgot the link: http://youtu.be/iFlT93wakVo
Every field seems to have this issue to one degree or another. There are "music made easy" books, languages made easy, everything under the Sun "for dummies". It seems like it's particularly strong with programming. Maybe this is because programming is relatively new. Music and language are ancient. They're ancient enough for people to realize that for some people they will always be hard. Certainly a good teacher helps; but you have to have some innate ability.
Programming is too young for us to figure out how far teachers can take the typical student. I suspect it's similar to how far they can take the typical math, music, or language student. That's just my pet theory though. Until we reach consensus, I welcome your efforts to make programming easy, with a huge dose of skepticism.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
ROTFLMAO!
Lessee, so, is flow-based programming like /programming/ ) { print length($0) " " $0;}} myfile | sort -u > myreport_on_the_use_of_the_word_programming
awk '{$0 ~
?
mark "*Nix, the object-oriented operating system"
Can't Named Unix Named Pipes Be Used To Do This. Seems like the parts are there.
Yep, partly done alreday. Have a look here: https://groups.google.com/forum/#!searchin/flow-based-programming/shell/flow-based-programming/PC96WYOAwAU/ICRZg_5K1XMJ
No.
Oh, I don't think that we are in disagreement here. My intended point is that, while railing against the (often genuinely ghastly) shambling horrors that people cobble together out of spreadsheets, duct tape, and optimism is entertaining, There Is No programming tool(barring strong AI-level expert systems, at which point humanity can pretty much pack up and call it a day) that will stop that from happening unless people who are both programmers and understand and care about/are allocated to the problem are available.
As much as it might offend the purists, almost everyone would rather have an imperfect, or even lousy, implementation of what they want, now, rather than a technically correct implementation of what IT thinks they ought to have 6 months from now. In fairness to IT, of course, when they can be properly integrated, things are somewhat less likely to turn into an unmaintainable and horribly brittle clusterfuck. (I once had the... pleasure... of observing the period where a small group of economic consultants, experts in that area, ran very hard indeed into the memory-space and compute-speed limitations of macro-laden spreadsheets. Luckily, they still had most of the human capital in-house, so the task of breaking out the algorithms and logical structures for implementation by the dedicated software developers they eventually had to hire was an expensive inconvenience rather than futile archaeology. Even so, if executing a spreadsheet in a 64 bit memory space across a modest sized compute cluster had been a supported option, they'd probably be using MS Excel for HPC to this day...)
Betteridge finally proven wrong?
It is better to think of FBP as a language / methodology for software architecture. FBP ideas are indeed old, yet tend to be ignored. FBP keeps resurfacing and keeps being rediscovered ("steam engine time"). This is due to the fact that these ideas do solve real problems encountered in software design. Software practioners who care about software design are the ones who rediscover FBP, or immediately "get it" when they find out about it. An excerpt from my slides (intro.ppt) here http://www.meetup.com/Toronto-GTA-Flow-Based-Programming-Meetup/files/ ...
FBP is Often Compared to...
- Erlang, CSP, Actors. FBP is hierarchical and sibling components are invisible to one another.
- UML. FBP is meant for compiling designs, not modelling them.
- LabView. FBP presevers component separation at runtime.
- Scratch, et al. FBP is primarily meant to simplify software engineering / architecture. FBP is not targeted at non-programmers, per se.
[If you're in the GTA, feel free to come out to the next meetup, Oct. 23. Join the meetup.com group, the google group, the Bento Miso mailing list https://www.bentomiso.com/ , or email me paul.tarvydas@gmail.com]
Stop it already! Non-programmers will never build applications, because on a programmer CAN build an application. Stop tooting your flavor-of-the-week 4GL solution and focus on providing more astute tools to programmers. The myth of a business type obtaining a proper result out of a computer without any a programmer being somewhere in the picture is NOT going to happen before we get Turing test worth IA. Period.
If only all these companies selling those products would learn a bit of computer science history before embarking into yet another flawed implementation of a 50 years old idea, and start enhancing testing, debugging, profiling, and - $Deity forbig - deployment and environment dependency manager tools, we'd be all for the better.
Damn. Do I sound as bitter as I feel?
> "easier for non-programmers to build applications"
Yeah. Excuse me while I go punch the person who built this SharePoint list and made a column called 'group' and set its type to 'text' instead of 'number' so I can't sort it correctly. And set 'owner name' to 'long text' so it can't be sorted at all. :-|
Yes, moar like this, please.
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
People who can't program in current languages will not be able to do so in any language, no matter how simplistic you make it. These are the same people who can't use the remote control for their VCR / PVR and who struggle every time Facebook makes a small change to it's UI.
Programmers can make it happen in any language. Non-programmers will be incompetent no matter how easy you make it.
Yes. But tools don't do that. Person does it (using and not abusing tools)
I don't think that we have to worry about non-programmers.
We don't need to make programming accessible to non-programmers. We don't make brain surgery accessible to non-doctors, we don't need to invent ways to write apps and software by people unwilling to learn how to code.
While there is always room for better tools to write software, focus on making software development more efficient, secure, powerful and error proof for people that want to commit to a career in writing software and when a Luddite has a passing idle thought for an app they can either contract a real developer or learn how to code properly.
Besides, they tried to do this already and the earth is still plagued by Visual Basic.
I haven't thought of anything clever to put here, but then again most of you haven't either.
CanDo and Amiga Vision were cool. Yes, you could so a lot more in C on the platform, but Amiga Vision excelled at developing multimedia programs quickly and easily, CanDo could create fairly complex tools (I made a decent Opus-like file manager in it) and the AREXX hooks into these tools (and everything else, including the OS) really extended their capabilities. Ah, fond memories.
I did use LabView only for one small project in university, we used it to control and read out several different sensors plot the readings and export them to files for further analysis and plotting. We had three days, two of which were taken up running the experiment. So effectively we had to pick up how to use labView in one day and design an application in it. I don't think a text based programming language would have been nearly as painless and easy to use for this application.
The premise for this sort of stuff is really that you don't have a clue what you are doing and string some stuff together on the fly. That doesn't scale. A large labview project looks like spaghetti unless it is organised with as much care (and more time) as you would take with a scripted method.
Many programmers have to re-orient themselves from orthodox object-oriented development, and shift to a world in which data structures are transparent but the behavior – the transformation – is not.
Then use objects as state machines to model behavior, a piece of common knowledge from the 80's and before.
Flow-based Programming reminds me very much of current data mining suites like IBM SPSS Modeler or RapidMiner. The prefered way to model an analysis is to build a graphical flow chart using various building blocks connected to one another. Those high-level blocks can be as diverse as data transformations (filtering, aggregation) or data mining algorithms (decision tree, regression analysis).
The input for the analysis usually comes from one or many database systems. The output is usually also written to databases. FBP makes it very easy even for non-experts to build data mining analyses using these tools.
Here's a neat trick: Near the bottom left and right sides of your keyboard are two keys named "Shift". Pressing either of them while typing allows you to enter captial letters, which lets people know that you aren't a complete moron.
You are making it sound easier than it is.
If its an actual complicated problem, the parts interacts. The complexity is in the interaction.
It is also a matter of "leaky" abstractions. There is always a corner case where the abstractions isn't enough.
Business problems, like all real life problems, are hard- but in a toally different way.
You can't rollback death and not even a spoken word, and you may spend days thinking about unsolvable problems,
but the reality is normally not as rigid and fragile as the abstractions and rules of a computer system.
In real life you can bend the rules.
All tools that tries to remove complexity are fighting against the law of thermodynamics.
Or put in another way, they will make simpler problems simpler, and the rest of the problems harder.
There is probably a sweet spot somewhere around where most popular programming languages are today,
somewhere between assembler code and bpel.
It is possible that the expressability can be improved for some types of problem. It is for instance rather strange that no popular language natively can describe relations between entities, and are forcing us to work with data structures from the 60ies as a workaround for the lack of relational constructs.
I think you are overcomplicating it. Look at how many millions, if not billions of spreadsheets that are in use to solve various problems. Each of those hard coded formulas in Excel are basically a black box with predefined inputs and outputs. What is lacking are the black boxes to control logic flow. But really, that would not be difficult at all.
The biggest challenge is that such a model, when you get past the terminology, creates structured code versus today's paradigm of OOP. Obviously the predefined components are objects, but they don't have inheritance, and all of the other properties we take for granted now a days. Code generated this way could actually be fairly robust, but couldn't be inherited either so like in the old days, a lot of duplicate work would be involved for similar systems.
From an end user perspective, though, that probably isn't an issue. Again, look at all of those spreadsheets, many of them quite sophisticated and used in ways that were never intended by the developers of Excel. Would a professional programmer use such a tool? Probably not, but then such a tool isn't actually designed for them.