RSI, WIMPs and Pipes; What Next?

Tetard asks: "Long live the pipe! Since the `|' was invented by Doug McIlroy in 1973, has there ever been a more effective way of reusing tools and connecting data ? The mouse is a device of the Beatles era; Rather than try and provoke nostalgia in the older ones among us, I'm asking myself, as are others: when we don't try to reinvent the wheel, or at least improve it, why must we try and copy it every time ? Xerox PARC exposed us to WIMPs and we haven't done better: some innovation, some plastic surgery -- but no "paradigm shift" -- where's the creative destruction that will take us further ? Graphical component programming is turning us into click-happy bonobos^H^H^Hchimpanzees, as we fail to find new ways to manage and connect richer data streams. My web designer friends are damaged for life because of mice, and yet we persist... Where do we go from here ? If we ever invent the graphical pipe, let if have keyboard shortcuts." Yes, you've probably seen a similar question to this run by Ask Slashdot before, but this time I'm wondering if maybe we need new input devices before the WIMP paradigm is replaced with something better. Might any of you have ideas on what form these input devices might take?

For those interested, here are the previous stories that have handled this type of question:

• GUI Research - Is it Still Being Done?
• Pipes In GUI's?
• Pipes in GUI's? (Redux)

So what it will take to break us out of the WIMP box (or prison, depending on your bias), maybe new input devices would do it, but quite frankly, I wouldn't be surprised if a 3D interface might be another route (it would possibly spark interest in designing a new input device that would work better with 3D interfaces, or maybe data-gloves could serve this purpose?). Going on a limb, maybe this guy might just be the ticket.

Comments

You know what else?

[aiken_d]

We're still using *wheels* on our vehicles! I've got a nearly brand new car, and it's using 5,000 year old technology to get around? From Detroit to Stuttgart, automakers are clearly devoid of imagination!

Get over it -- sometimes it makes sense to stick with what works. A love of change for the sake of change is the one thing that unites geeks and fresh-out-of-school MBA's.

-b

Re:NLP

[JanneM]

Nope. It will have a role in niche applications, but I don't see it ever being the dominant method of interaction. A couple of reasons:

Imagine a large room full of office workers. Now, imagine the same room with every worker talking to his wordprocessor or spreadsheet, trying to make him or herself heard over all the others, getting irritated and fatigued because of the constant noise of everybody else talking to their computers.

Imagine trying to do some work in an airport or on an airplane. Now, imagine trying to do the work using your voice _without_ other people hearing the budget details for your company or hearing the steamy endearments you will be mailing to your spouse.

Imagine talking. Now, imagine constantly talking all day, every day. Some actors and singers get permanent damage to their vocal cords - and they've had professional coaching and access to medical services. It could become RSI for your throat.

/Janne

3D Environments will lead to change

[Bonker]

The real problem with computer software advancement is that its all very firmly based in 2D land. WIMP environments are about as eficient as you can get. Even assuming that you can wrap your work habits around something as 'next-gen' as 'The Brain', you're still stuck in 2D land.

The real advance that will open innovation (real innovation and not some corporation's twisted idea of it) is the beginning of a 3D workstation environment.

We already have the primatives for this kind of environment in games like Unreal, Q3A, and Black and White. Assuming that we implement a 'graphical pipe' that will work for a truly 3D application system, ie: Allow 3D applications to pass information back and forth between each other semi-effortlessly, this will ignite a new 'interface revolution' similiar to what we experienced as a result of Xerox's early WIMP system and the first versions of Apple's MacOS.

Once programs and applications can truly be represented as 'objects' in a 3D environment, we'll end up with something like the 'God' interface in Black and White, where processes are represented by animated people and files are represented as other objects. Tasks best handled in 2-D such as composition, coding, or painting will still be easy to handle, but tasks best performed in 3-D such as file management, database management, and even some advanced programming tasks like linking and compiling files, will take place in a representational environment. Imagine opening up your HDD and pouring objects into it, then sorting them into containers based on type, as you would sort files into directories.

Eventually, I see us moving into something like Stephenson's 'Street' metaphor for shared environments.

Along with these advances, will come new interfaces. I think that eye-tracking cameras have the biggest potential, but we keep coming back to the data-glove in one form or the other. I know CADesigners who still have an old Nintendo powerglove hacked for basic 3D manipution tasks. We're also probably going to see a renaissance of 'body tracker' devices that will track human motion via sonar or laser. Any one of these has the potential to vastly reduce RSI injuries.

The real trick in jumping from 2D to 3D is reverse compatibility. All the shells I've seen that attempt 3D interaction do it badly. Even then, they fail completely when faced with most of the tasks we do on a daily basis, like write or paint. I think we're going to have to use 'easels' or something similiar. In Cowboy Bebop, Radical Edward's computing environment is shown as a multitude of 2D windows hovering around her in 3D space. This wouldn't be that hard to do, really.

Navigation will be the true challenge for any 3D application designer. It will be that itch to scratch that will spawn new, inventive input and coding ideas.

Re:Face Recognition

[sporty]

Until you sneeze and reboot your computer :)

Re:We need Star Trek interfaces

[slamb]

Artifical intelligence should be able to translate speech from any language to any language and add new patterns without reprogramming.

Oh, okay. I'll have that for you tomorrow.

I'm really tired of people making these dramatic statements that are all but impossible to realize. There are people who have spent their careers trying to do what you mentioned in that paragraph, and they've gotten nowhere, because the task is so enormously complex. Translating from one language involves taking one set of nonsensical rules and ambiguities and replacing them with another. It can not be done without a complete understand of what is being translated. That's...hard, to make the understatement of the year.

What if we could build layers of software on top of layers of software, which are then again used to build greater layers of software? This would require that we accept standard functionality at each level of the layering process and then allow people to write ever simpler code due to this great deal of layering. Why should anyone be required to rewrite a quicksort once it has been written?

There are already so many levels of complexity in any piece of software that it's completely insane. Your idea is nothing new. Here's a partial list of the layers involved in a Java Virtual Machine I'm sure I am leaving many things out, putting them in a bad order, etc.:

1. Gates. Operations on ones and zeros.
2. Hardware components. I'm sure there are layers I am leaving out here and much more descriptive ways to say this...but I have only a token knowledge of these layers.
3. The instruction set architecture of the machine. Already a huge step up, this is a layer at which I can (slowly and painfully) produce useful software.
4. Assemblers. They take somewhat human-readable code and generate machine code from it. (Ever had to do this by hand? Not fun.)
5. High-level programming languages. They take relatively abstract concepts like loops, functions, etc. and create assembler from them.
6. Operating systems. (At this point, the "layers" metaphor shows its imperfections...high-level programming languages and operating system functionality are orthogonal, meaning they abstract different things. My order is arbitrary.) They make it possible to access devices in abstract ways, plus a million other useful things for software.
7. The core library. (libc on C-based machines.) Makes system calls to the operating system easier. Implements stuff common to lots and lots of programs.
8. Other libraries. There's actually many layers hidden here. For example, on UNIX, libxpm depends on libXt which depends on libX11 which depends on libm which depends on libc.
9. Various other processes, like the X11 server. These interact with client-side libraries through communication primitives provided by the operating system layer and made more accessible by the libc layer.
10. The Brahm GC. (For Kaffe. I don't know what GC Sun's VM uses or where exactly it fits in.) This makes memory allocation easier.
11. C-side Java support stuff. Part of the String class implementation, for example.
12. The virtual machine. Implements a completely different instruction set.
13. Java-side support libraries. A lot of the java.lang stuff, for example.
14. More Java-side support libraries. GUI stuff. This depends on stuff below. Extend to arbitrary depth.
15. Your code.

My point? Layering software is nothing new. If you want to add a layer, say so, but don't pretend it's a new concept.

So here is my challenge to every programmer/newbie/manager/hardware designer in existance. Stop trying to focus on how we can optimize or tweak each individual computer function so that you gain temporary marketshare, outperform your competitor, or push your political agenda. Instead, begin pushing for hard standards in the industry.

You know, I really hate it when people push for other people to make dramatic changes I suspect they don't understand (evidence: above) and use the word "we". It's entirely inappropriate.

Stop worrying about making languages that allow you to produce specific functionality in fewer lines of code, and worry about producing the highest level of quality

The language is one of those layers you advocated. The language itself is a piece of software to be reused. You suggested people add layers to make things possible with simpler code, and this is one way to do it. Don't knock it.