True Visual Programming

eberta writes "We are still stuck with text programming for the most part. I can think of only a few truly visual programming environments like LabView and none are really mainstream for application development. Being recently unemployed and having ample spare time, I have started a pet project to work on my own version, GIPSpin (Graphical Interface Programming System). With multithreading becoming an increasing issue in software development, I'm wondering why hasn't there been more focus on visual programming. I see so much possibility of making coding easier and handling threading issues semi-automatically by allowing the user to graphically guide the auto-threading AI. Right now it seems the industry is focused on figuring out how to get just small chunks of code auto-threaded either through hardware or compiler technology, with longer term solutions like OpenMP still text based environments."

Comments

Don't reinvent the wheel ...

[bitsformoney]

... check what is/has been out there before. I.e. something called ProGraph has been mentioned on /. a couple of times before.

It's inefficient

[0x461FAB0BD7D2]

There are ways of visual programming right now, such as WYSIWYG HTML editors (Dreamweaver, Nvu, Frontpage) and Visual Basic.

However, I can't see this happening for Perl, PHP, C, Java, etc. Everyone has their own style of coding with their own ideas, many of which are abstract and cannot be effectively visualized. To make an IDE which effectively deals with all the quirks programming has would be quite a feat, but would be so bulky that text-based programming would be the most efficient.

There might be a place for visual programming in rapid application development or some simple programs/scripts like HTML pages and the like. Beyond that, I'd doubt it.

Another implementation.

[saintlupus]

Here's a visual programming environment called SIVIL that one of the professors here at Canisius has been working on for a while.

http://www-cs.canisius.edu/~meyer/VP/home.html

--saint

Because...

[Anonymous Coward]

I'm wondering why hasn't there been more focus on visual programming

Because in the 1980's people realized that visual programming was a dead-end idea. Trust me, one of the products I work with lets you program "workflows" using a visio-like toolkit. It's the most unproductive thing I've ever had to use.

Seriously, researchers have beaten this one to death already. Even as far back as The Mythical Man Month it was already recognized that graphical/visual programming would not give us any improvements on efficiency.

I guess the old addage of "reinventing lisp, badly" still holds true. "Computer Science" sure seems to ignore a lot of its past research.

Problems with 'Visual Programming'

[drspliff]

Visual Programming has always been a sort of industry goal, every product you see out there is trying to make it easier on the developer, be it UML or another modelling software, iMatix's Libero or another code generation framework.

One of the reasons I think Visual Programming won't catch on for a long time, or will take serious innovational leap is because with existing solutions the developer looses too much control over the path of execution, optimizations, memory management and all the other lower-level stuff we developers have to tinker with.

There have been numerous frameworks which have tried to bridge the gap, one that sticks in my mind is SilverStream's/Novell's exteNd Composer/Director. They follow the basic roles of a point-and-click programming environment, flow layout, assisted statement creation etc.

But there is only so much you can do before you end up just writing solid code again. I don't want to sound like an elite-ist, but personally I think all these high-level visual programming environments will lead to 'Joe Blogs' developing your [name critical financial or business application here].

And not to mention the thousands of zealots out there who you'll have to bring kicking and screaming into the new 'visual' era.

Rant or rave, new developments in this area can be a great aid to experienced developers, but in the wrong hands they can cause more harm than good (Visual Basic anybody?)

Moderate: -5 Zealot bait

Re:Problems with 'Visual Programming'

[kentquirk]

Back in '93 or so I was the UI designer on a visual programming version of VB that Lotus developed (and later sold to Revelation software). We had "links" between UI elements and code snippets so that you could build basic VB apps without actually breaking out the text editor. It worked pretty well as an accelerator for basic application development, but eventually you ended up writing text-based code. It just saved you time getting started.

I liked the idea so much I went out and built a game around it -- MindRover (http://www.mindrover.com/) where players constructed robots using purely visual programming techniques (no text-based code at all). It worked, and it was great for a game, but it made for a cumbersome development process. It's hard to look at an image and see everything you need to see.

Visual programming is cool and fun. It makes the learning curve for programming a little flatter, and it can be a nice way to prototype stuff and get the early versions of an application running. But using it to build apps that are more than toys is a much bigger challenge, and I haven't seen anyone yet who has done it effectively. The original poster's research looks an awful lot like a demo I saw at MIT Media Lab about 5 years ago. IMO, it exposes a bit too much of the syntax of conditionals and things in the visual UI. It's not far enough from textual programming to be helpful.

INPUT DEVICES

[ericandrade]

Computer interfaces suck.

Programming languages are visual and depend on the computer screen or whatever the computer outputs. Changing the way it is disposed on the screen is just trying to make it cute.

The major restraint right now on programming is INPUT.

The mouse dates back 50 years,
the keyboard's even older than that, and it's designed to slow down users! (both cause +RSI and other crap, and are very slow and innacurate)

For example, where are dual cursors?
There needs to be more OS implementations and design to have superinterfaces. Make the long, tedious, well planned out programming that will accelerate future programming. And think out of the box: Future programming will not be done with a keyboard and a mouse.

http://sloan.stanford.edu/MouseSite/1968Demo.htm l

Why can Stephen Hawkings write speeches, scientific texts and do tons of complex things with a single thumb clic?

Where are the standards for new interface developpement?
The OS developpements to support the hardware?

Screens are getting bigger, why do I have to rely on a menu in the top left of the screen if i'm working in the bottom right of a 2nd screen?

Design and manufacture of new technology is slowed down by the limited ways we have to transfer, collect, manage and create the complex data we have to deal with. F*ck Moore's law! With the keyboard and mouse, the bottleneck is between us and it. The idle time prooves it.

Computer's are an instrument. Instruments are allways hard to master. They evolve. They're not supposed to only get cuter.

Imagine a computer using 50% of it's processing power to know what you want /to say/it to do... /rant

Re:INPUT DEVICES

[pavon]

The mouse dates back 50 years
The steering wheel dates back 1000s of years, yet we still use it because it is an effective interface.

the keyboard's even older than that, and it's designed to slow down users!
No it wasn't - that's a myth. Besides the best text entry devices which are designed to be as efficient as possible, such as Dvorak (use it myself) and chording, are only incrementally better than QWERTY - not the revolutionary jump you are looking for.

For example, where are dual cursors?
Dead on the research table where they belong. They never provided any significant improvement in performance, and even slowed users down due to having to stop and think about what they were doing - even after spending a significant time learning the system.

Why can Stephen Hawkings write speeches, scientific texts and do tons of complex things with a single thumb clic?
You can too - on a cell phone. They are smart ways to make the most of limited input capabilities; however, they are incredibly slow compared to the keyboard and mouse, and no one would choose to use one unless it was the only option available.

With the keyboard and mouse, the bottleneck is between us and it. The idle time prooves it.
Yet, much of the idle time is due to user thinking, not slow user input - you don't notice yourself being slow, because you are preoccupied, but you do notice when the computer slows you down even slightly. Honestly, ever since I learned to to touch-type, I have been able to type as fast as I can think, and there isn't really any reason for me to want to input text faster. Once I get a wacom tablet, I'll likely be able to say the same about the 2d/3d graphics work (err play) that I do.

Most of the instances where I notice the computer slowing me down - making me do more work then I ought to - is due to poor design and integration of the software, not the keyboard and mouse. For example, I had to start up and entirely separate program just to spell-check this post.

Where are the standards for new interface developpement?
I don't know - why haven't you come up with one?

But we *do* think visually

[TuringTest]

Humans think of the world using language, but we also think of the world using visual, spatial, temporal, sensorial... reasoning.

Do ask a real psychologist, she will say that there are different kinds of think. Textual is best suited for abstract, logical reasoning. But associative thinking is often better done visually. In the Programmer's Guide to the Mind you have an interesting classification of all these.

A programming environment should take care of all these kinds of thought, not just support the logic abstractions as they do now. A promising field of research is Programming By Example. This programming style tries to build the final program by using concrete reasoning over samples of data, instead of forcing you to think of the general, abstract procedure.

Graphical Programming is Inefficient

[ActionAL]

Believe me we use an old Honeywell system at work called MeasureX which uses graphical programming. EVERYTHING IS BLOCKS AND LINES CONNECTING EACH OTHER. It is the most inefficent way to read code possible. The human brain can read text code insanely faster than trying to decipher a huge picture with tons of blocks and lines strewn everywhere with complex connections and pipes from page to page. It is the most horrible thing I have ever seen. And if you have one missing connection or pipe, goodness good luck trying to debug it.

Unreal 3 engine uses visual programming

[Junks Jerzey]

The text-based UnrealScript has gone away, to be replaced by a fully-visual language, Kismet.

Visual Languages are Lacking

[fenris_23]

A programming language is a language used to communicate both to a machine and to other humans. Language features help us encapsulate, hide, and organize complexities so that communicating very complex ideas to a human or machine is more efficient and maintainable.

Non-written languages do not provide the same depth and strength. For example, a CD of James Joyce's Ulysses is not as accessible and understandable as the same book in written form.
Furthermore, how would you express those concepts visually? In my opinion, we developed our forms of written communication over the years because it is the most efficient and expressive.

Take hieroglyphics for example. Everybody knows that the Egyptians used a written language of symbols referring to entire concepts rather than words. However, many people do not know that in every day practice, the Egyptians developed a linear form of the same language. Similarly, Asian cultures have adapted their languages to a linear form to use with computers because it is easier than adapting a computer to work with more complex symbols.

Also consider that amount of complexity that can be expressed in a written (text) programming language. When you begin thinking about designing a visual language, you begin thinking about logic flow and control structures. However, you should begin at the most basic level. A programming language's lexicon has both closed and open classes. The keywords are closed but the open class of identifiers is infinite. Furthermore, the idioms used to express these identifiers in various statements are also practically infinite with respect to designing a visual language. Statements can be combined into idioms that vary between languages, programmers, development teams, and application domains.

Worse, is the problem of side-effects. Many programmers using languages such as C and C++ use side effects all of the time. How do you adequately express that in a visual language?

UML is a visual language that has seen a massive amount of research and development. Much progress has been made but even the most die-hard UML designer still has to go down to the code-level to fix the various idioms they wish to express in the programming language that the UML cannot express.

Ultimately, I think the biggest problem is the lexical and syntactic constraints. A programming language allows one to easily expand the lexicon of a programming language as well as various syntactical forms. To do this visually, you will have to create a symbol set to handle each form. If you tried to implement this dynamically like how a written language works, then you are really just developing a written language that uses pictures for words. In that case, you are wasting your time and may as well stick with text.

Different tools for different purposes

[booch]

I think the parent poster has a lot of important points. But one thing I haven't seen mentioned in this entire thread is that each type of programming/thinking/communicating has its place. While visual layout of complex program logic is not a good way to program, a written procedural language is probably not the best way to create a GUI. We should be using descriptive languages (whether visual or text) for GUIs and such.

For example, a typical how-to book is mostly text, but it also includes diagrams of how to put things together, and pictures of the completed project. All of these are essential to the book. Another good example is teaching. Teachers employ multiple methods of teaching, as different people learn differently, and different topics lend themselves to different styles. Some people learn best by doing; others by watching; and others by reading.

I think the right thing to do regarding visual programming is to relegate it to where it works best. I suspect that this would be mostly related to things that are already visual -- i.e. graphical (GUI) layouts, etc. I think it's also useful to be able to visually represent program code "in the large" in a graphical manner, to see the inter-relations, but I don't think manipulating that type of graph is useful.

I remember the old NeXT Interface Builder. You would take components, and connect them together. You'd have to write code if you were doing anything complex, but you could set up the GUI, including callback functions, visually. I think it's still around in Mac OS X. Things like GLADE work similarly. I'd like to see these types of tools used more.

Re:I like it!

[Jerf]

Keep going, and don't let the nattering neybobs of negativism here at /. get you down.

You know, I understand what you're getting at here, but you need to understand where the "nattering naybobs" are coming from. We're not saying "We've never seen this before and we're sure it won't work, don't try, fingers stuck in my ear, blah blah blah!" We're saying "We've seen several implementations of this idea, they all suck in the same fundamental way, and we've never seen anybody with a functional solution to the suckage problem."

You want to try your hand at it, be my guest. If you have taken the time to understand history, what has failed in the past, the problems encountered by others, and you still want to try, well, go for it. It's your time.

But if you think, and I get this vibe from the original poster, "Hey, there's nobody who's tried this great idea, I think I'll start a project on it!"... you're fucked, plain and simple, and throwing a bit of water on that ethusiasm is a favor, right up there with "Your first programming project should not be a real-time strategy RPG." Encouragement is great and all, but you should not encourage someone who's just been mountaineering for a week to take on Everest. Discouragement can be a positive thing.

At the very least, educate yourself before starting. There's a lot of failure in this field.

Two of my "general principles" come into play here:

1. "When a lot of other very smart people have tried to solve the problem, but you think you have the answer, you almost certainly don't, and you certainly don't if it seems 'simple' or 'obvious' to you." This fits that in spades; a lot of very smart people have tried this and failed.
2. "There exists a lot of solutions that only work in demos, or worse yet, when you just 'imagine' it in your head, but crash and burn when faced with the real world. Anything works in a demo, or in your head, even the absolutely impossible. If you've never seen it outside of a demo, there's a reason." And that fits this whole "graphical programming" to a T; it's trivial to create an "ooo, ahh" demo with the execution flying around the screen, two or three 'if's and a "for". But there aren't very many functions, let alone programs, that work with two 'if's and a for loop. (Another thing that fits here is many interesting AI solutions in academia that are claimed to be generally powerful, with varying justification, but in five years of working with the solution they've only managed one very particular demo.)

Go for the gusto, but if you just close your eyes and hum when people try to explain why it's a hard problem, you're dooming yourself from the getgo and wasting your time.