RSI, WIMPs and Pipes; What Next?

Tetard asks: "Long live the pipe! Since the `|' was invented by Doug McIlroy in 1973, has there ever been a more effective way of reusing tools and connecting data ? The mouse is a device of the Beatles era; Rather than try and provoke nostalgia in the older ones among us, I'm asking myself, as are others: when we don't try to reinvent the wheel, or at least improve it, why must we try and copy it every time ? Xerox PARC exposed us to WIMPs and we haven't done better: some innovation, some plastic surgery -- but no "paradigm shift" -- where's the creative destruction that will take us further ? Graphical component programming is turning us into click-happy bonobos^H^H^Hchimpanzees, as we fail to find new ways to manage and connect richer data streams. My web designer friends are damaged for life because of mice, and yet we persist... Where do we go from here ? If we ever invent the graphical pipe, let if have keyboard shortcuts." Yes, you've probably seen a similar question to this run by Ask Slashdot before, but this time I'm wondering if maybe we need new input devices before the WIMP paradigm is replaced with something better. Might any of you have ideas on what form these input devices might take?

For those interested, here are the previous stories that have handled this type of question:

• GUI Research - Is it Still Being Done?
• Pipes In GUI's?
• Pipes in GUI's? (Redux)

So what it will take to break us out of the WIMP box (or prison, depending on your bias), maybe new input devices would do it, but quite frankly, I wouldn't be surprised if a 3D interface might be another route (it would possibly spark interest in designing a new input device that would work better with 3D interfaces, or maybe data-gloves could serve this purpose?). Going on a limb, maybe this guy might just be the ticket.

Comments

Face Recognition

[Sludge]

With a sub-$100 webcam watching you, look at the point of the screen where you would click, and blink.

Are there lots of problems to doing this? Yes. Should that stop me from throwing out the idea? No.

Re:Face Recognition

[JanneM]

Except that the eye isn't all that accurate. You can generally get a resolution of about 1 degree, no more. It's because the fovea (the sharp area of the eye) is about 1.5 degrees, and the eye will skip around slightly all the time within that area (the visual system needs a bit of movement at all times).

You can probably do a decent interface using that kind of accuracy, but you won't be doing any kind of precision work.

/Janne

Why replace the current GUI paradigm?

[kurisuto]

I'm wondering if maybe we need new input devices before the WIMP paradigm is replaced with something better

This seems a bit like asking what it would take to replace the current way of driving a car (steering wheel, gas and pedal brakes, etc.) with something better. But the interface between humans and automobiles is pretty much a solved problem, and nobody seems to spend much time speculating on what a paradigm change in automobile control would be like.

There's a curious assumption which I've seen repeatedly-- namely, that a paradigm shift in human/computer interaction would be a good thing. Why, exactly? I see no reason to pursue a paradigm change for its own sake; I view it as a problem which has basically been solved for now, much as the problem of steering cars is a solved problem.

NLP != Voice Recognition

[Greyfox]

A lot of people think NLP implies voice recognition and to a large extent that's true, but it also means being able to communicate with your computer using plain english sentences. Difficult because English is so damn ambiguous (Note to self: Perhaps Klingon would be a better place to start...) Still, if I want to tell my computer, "Book the least expensive flight from Denver to Miami on the 23rd of this month." I should be able to say that to the computer or type it and have the computer understand what I'm telling it to do. I should also be able to modify my command to specify times or airline or both and my computer should be smart enough to say "There's a flight for $100 less 5 minutes earlier than the timeframe you specified. Would that be acceptable?"

That sort of thing will be the wave of the future, and it will mean that apps will have to be smarter and communicate a lot more than they do today. My personal agent should reside on my local machine, not the network, and should watch out for my personal privacy. It should divulge only what is necessary to others in order to perform the commands that I give it. It should be flexible and configurable, but I should never have to configure it; it should learn what I like by how I interact with it.

Several large companies have been working toward this holy grail for years, but thus far not even common voice recognition much less NLP has emerged from their research. Sure there are some voice recognition packages out there, but there's very little integration, and AFAIK nothing at all in the NLP arena. We could start working toward the level of integration that would be a necessary foundation for a lot of this stuff, but I don't know that you could get the necessary level of cooperation in ANY software development community.

Re:3D Environments will lead to change

[Mr. Sketch]

This would be good, but it would require a 3d interface. I think the only way to truly do a 3d environment is for it to exist in the physical world. Any 3d interface on a 2d screen will become kludgy pretty quick. The best way to do a fully 3d interface is to put in in the real world. Imagine if your desk *WAS* the computer. The desktop *WAS* your actual desktop. You open your draw to see 'real' manilla folders with names on the tabs for your documents and thumbing through them to find the financial report you were working on, 'grabbing' it and pulling it out and it appears on your desktop for you to work with. You open up another draw and see pens, pencils, markers highlighters, etc that you then 'grab' to select what you want to start writing with. You could just slide your hand across the desktop to move documents out of the way and tab or 'grab' a document that was 'under' the one you were working on and it comes to the top and you can begin working on that one.

This would require a lot of holography and motion tracking, touch sensors, etc, but it would be the ultimate in 3d interfaces. It would avoid klunky HUDs and gloves, etc that just detract from the actual work. You could even bring up a keyboard on the desktop and use that instead of a virtual pen or pencil.

3d interfaces would be nice, but on a 2d display I think it's best to stick with a 2d interface.