Slashdot Mirror
10/GUI — an Interface For Multi-Touch Input
Naznarreb writes "R. Clayton Miller has an extremely impressive GUI concept he's calling 10/GUI (video; written description here). Essentially, it combines the high-bandwidth input possibilities of multi-touch interfaces with the ease and immediacy of a mouse. The video is quite interesting, and, for me at least, pretty jaw dropping. This is a dramatic re-imagining of the current mouse/screen schema, one that I think has significant potential."
Theres still a few problems though. For one, mouse is an incredibly precise input device - you can pretty easily move it along same pixel axis, or get it precisely to a specific pixel. It's hard to do that with your fingers because the area they touch is a large one, it's not easy to just move your finger by one pixel and your hand tend to shake a little bit too. If you look at the video, you see everything in the interface is quite big and even a few small windows take lots of place.
Other problem is that now your both hands lay on the wide touch area and you dont have a keyboard. If you put them side to side, you'll only have one hand on the touch area and dont get the full power of it. Moving hands between them all the time is inefficient. Typing on the touch area gives no feedback and again takes your hands of the "mouse".
It would also be quite impossible to play FPS or other kinds of games with this type of setup.
So no, I still dont see touch interfaces replacing the usual keyboard+mouse combo anytime soon. However, I would love to have this kind of system in my living room (either just for the tv, or the computer thats connected to tv screen). It's clumsy to have keyboard or mouse in living (at the moment I have MX Air -mouse, which is okayish), but this would be perfect for such job. Not for a desktop pc replacement though.
...play an FPS on it?
But how would you...play an FPS on it?
That'd be like asking how one plays a Wii-style game on an Xbox 360. The games specifically made for this sort of device would have a different design. Imagine the kind of control in an RTS that this would enable.
Besides, from what perspective are Duck Hunt and Time Crisis played?
I can see why the demo didn't show any typing in the text editor...
How can you over-hype a one-paragraph summary?
Five minutes into the video and I'm still none the wiser as to how this is supposed to be an improvement in the use of my computer, or more comfortable, or easier. The "real-world" demo towards the end doesn't seem at all impressive and leaves out an awful lot of computer uses (we'll start with gaming, because it's easier to pick on multitouch for that).
Why is everyone determined to sell me multitouch but can't actually show a decent use that justifies the price/hassle/upheaval/software development costs?
I love the concept, but I imagine myself stretching over the touchpad area to type, which wouldn't be very ergonomical. I can also imagine that the base of my palms would rest on the touchpad area occasionally as I type.
But I'm not sure how many people will be wanting to drag their fingers across a surface for 8 hours a day. One of the benefits of a mouse over a touch service is that there's less friction for the hands- all of the rubbing of objects is between the mouse and the surface.
Can I see this replacing the mouse? No.
Can I see this supplementing the mouse? Yes
Can I see this being placed with a mouse and keyboard? No- the combined three objects would take up too much space (who really has that much desk space?)
It doesn't really seem like an improvement in window-management for me either. Sure, window overlays are a bit cluttered, but then again there's only so much information one can process at a given instant.
I tend to have a *lot* of items running as I multitask. A web-browser, document, several terminals, perhaps a coding window, and others. Having windows aligned horizontally it going to be a PITA if I have to zoom out every time I need to jump from #1 to #15. In that event, a taskbar really is quite a nice thing and "just works". Perhaps rather than having a left (contextual) and right (global) menu, they could also have a bottom/top (taskbar) menu.
The flat surface is not ideal from an ergonomic point of view. The touch-sensitive surface should be curved so that the user's hands can be held at a more natural angle. Preferably two domes of soft, touch-sensitive material, and two small raised dots on the top for tactile positioning. Hm? what? sorry, I drifted off there for a moment... what were we talking about?
Put this functionality ON the keyboard and you've got yourself a deal.
Practicing this sort of movement on my E5500 keyboard, I think I would have no problem typing and using the KB surface for manipulation, assuming it could be done.
Should a desktop GUI paradigm like this take hold, how would those with disabilities use it? Managing multiple pointers is physically impossible for some people. The GUI has to be usable with only one such pointer. Sure it can be better with more, but it must be usable with only one.
... it's too limited. For pure window manipulation it looks fine, but I expect it to break down horribly as soon as you get to move on to real work. 99.lotsofnines% of the time doing actual work I still spend typing. And I'm getting endless grief from the touchpad that suddenly decides I touched it when my thumbs were merely in the vicinity, like, oh, when I'm touch typing, which I just noted I do most of the time. Now imagine that touchpad expanded to be posively unavoidable.
I don't want a mouse. I want my trackpoint back.
The name is too similar to "Tengu." Google that for a quick reason why.
When I saw this video, I immediately thought of this article: http://www.appleinsider.com/articles/09/10/01/evidence_of_apples_tablet_like_input_interface_reappears.html
I've GOT multitouch.
It's called a keyboard in conjunction with a tiling window manager.
I like the touch surface but that "simple" window manager is just that. Simple. Too bad that there is a difference between simple and better. Just like a skateboard is simpler than a car doesn't make it more suitable to go to the grocery store with.... After all that talk of "lets increase interactivity because you can't reduce 10 fingers to one x/y coordinate" I think its a little strange that they then go to "lets reduce x/y window layouts to just x"...
In the end it just looks like an effort in changing things just for the sake of change.
Peter.
I like my GUI text-only, 80 characters wide by 25 lines long. The way Ghod intended.
oh yeah and 7-bit ASCII only... none of this fancy schmancy 8-bit extended code-page goop.
and GET OFF MY LAWN! Damn kids with their game boys.
Karma: Excellent. 15 moderator points expire sometime.
- iphone
- window button position
- dashboard
- expose
- dock
- cinema display
wonder what platform they're going to market to first eh?
I work for the Department of Redundancy Department.
Right now this comes across as an interesting thought exercise. I think that FPS style games should be possible (where there's a will...) but I think that things like typing will be more less efficient than on a keyboard, which provides tactile feedback. And I spend more time typing than I do mousing and reorganizing windows. This isn't optimizing the 80% case. Since all the tech is available, they should be able to provide a real demo as opposed to animation mockups; then we'll see how typing and other real world problems fit in. Until proven wrong about slowing down the things that I do on a PC, I agree that this isn't for a desktop PC. The other question is, where else could this be used?
I disagree with an earlier poster who says this can't replace the desktop interface we know.
He points out that it is inefficient because one has to move hands between keyboard and pad.
It seems to me that this interface can be manipulated with just one hand. that's how the mouse works now. you take your hand off the mouse to work with the full functionality of the keyboard, why couldn't 10/GUI replace that?
Better still, why not have both! The pad can sense a mouse and act as a mouse pad. If you need to use it as a pad move the mouse off, or perhaps use both the mouse AND the pad at the same time (one hand each). I can see a lot of possibilities there. It could clutter a desk, sure, but I'm sure we can solve that problem.
Second, but probably more importantly, I can easily see this for the work I do, which I imagine is similar to many other people.
I use my system (two monitors) for business. I have data on one side of the screen and sometimes excel or word on the other screen. In addition (here's where 10/GUI could be useful) I'll have a pdf open in the background which i need to quickly scroll through ("in adobe, quickly!?" you ask?). With one hand on the mouse I can quickly zip through Morningstar data, and use my free hand (on the pad) to scroll through a document, then quickly zip back and forth, scrolling and zooming as necessary. Right now that's just using a mouse and it can be tiresome to move around with just that little pointer (especially morningstar! oh it would be nice to have a touch interface for that...)
Finally, I need to have a "document scroller" or whatever it is that I can resize and move around, as I find myself with documents that sometimes need to show different parts of their data on screen. Basically I need to be able to "undock" documents so i can use them effectively (like papers on an actual desktop) and then redock them when I'm done.
That would be a beautiful interface that I do feel would save me some time and frustration. I would buy that for my business.
-
I am a proponent of change, when it makes sense. I love the interactions here and I would greatly enjoy this, for about two days. Then the luster would wear away and this would become an inconvenience.
I don't understand the typing reference as the video clearly shows the pad in the lower section of a traditional keyboard. It seems to be based entirely on replacing the mouse.
Against cluttered windows, just use frames (ion). Couldn't live without.
IMHO, an interface's success is directly proportionate to the amount of effort required to use it.
I really like the idea, but not so much the implementation. Two observations, and one theoretical quibble:
1) On the hardware side, turning the multitouch interface into a second touch screen that could work as a mouse-like input device (like the video shows), or bring up a keyboard (like the lower half of the iphone, only taking up the whole space) would be preferable. A keyboard without physical feedback would be awkward at first, but after getting used to it, you could do away with a physical keyboard entirely. You could also stick meta-materials from the UI down there. You could have something like a system dock with quick links to open programs and switch windows. And you could have, say, the clock, wireless indicator, battery indicator, etc. down there. It would cut down on clutter on the main screen.
2) On the software side, I'm not sure that I see the advantage of their version of a linear window manager the way they have it set up. Instead, it seems more useful to fan applications in and out, turning the name bars on the side into tabs. Window one opens and slips into place with its name bar on the left side of the screen. Window two opens from the right, partially obscuring window one. Window three moves window two all the way to the left, leaving window one entirely obscured, except for a tab. And so on. That way, you have a visual representation of every window on the screen at all times. Much simpler to track than having to zoom out.
And my theoretical quibble: I know, it's odd coming from a longtime mac user, but I dislike the concept of a physical UI so closely tied to a particular software system that you have no alternative but to use them together. I look at this, and I fear "The Windows Computer of the Future." You buy a multi-touch computer that has Microsoft's future OS on it, and the hardware interface is so specific that you couldn't, if you wanted, chuck the software and install Linux. I can imagine a Linux-like alternative being written for this interface. But I can also imagine a set of patents that would encumber alternate OSes, such that you end up with a One Computer One OS system. Which is far, far too restrictive, and invites vertical monopolistic practices.
In the end it looks like Apple is not far with the buttonless large touchpad featured on the latest MacBook pro, which permit gestures with up to 4 fingers that trigger expose, switch from applications, scroll documents and more.
We've basically already solved the issues shown in the video. Problems with too many windows: get a second monitor. It's not very expensive, and doesn't require me to learn a different desktop paradigm. You can also take advantage of various taskbar/Expose/Spaces type features, which (depending on your preferences) make the problem of window management a lot easier. Need more/better input "bandwidth"? Get a Mac laptop with multi-touch trackpad (if you're not a Mac person, ok, I can't imagine Windows is very far behind with this feature). But even this is a little dicey as it requires you to learn how to use the multi-touch interface - I've had an iPhone for about a year now, and I'm still not familiar with all the multi-touch gestures the system can do (it turns out that they're not really very "discoverable", at least for me).
It turns out that there's a reason we've stuck with our standard WIMP metaphor for interacting with our computers... it works really, really well. This effort strikes me as change for the sake of change.
It seems that every time an "improved" interface to your computer is invented, they get more complicated whilst simultaneously looking simpler.
This particular interface perhaps has potential for the expert user (like most slashdot folks), but I don't see your average housewife or Grandpa wanting to remember how many fingers to use for what.
And just what are you supposed to be doing with the fingers not touching the screen? Hold them in the air?
How about your left hand? Keep it in your pocket until you need a context menu?
I'm guessing you will still need a keyboard; where is that supposed to fit on your desk?
Apple got it right with the iPhone - by restricting the buttons and losing the stylus they have greatly simplified the interface.
*I* personally miss the hardware buttons from my Windows phone and taskbar - but those things were sacrificed for the greater good.
Humor from a Genetically Molested Mind
On linux, I use a windows manager that lets me bind keys for "goto firefox" or "goto emacs". Or I use virtual desktops basically for the same thing. On my mac I can use Quicksilver for the same task. Some pre-defined, determined key combination that brings me directly where I want to go. Any task not in my day to day works with expose, or alt tab, etc. Often Quicksilver solves this task quicker than these methods anyway. No zooming over a list, or stack, or grid, or anything. The keyboard alone can already solve these tasks with the mouse providing extra support when needed. While the video was cool, I don't think it solves any issues that can't already be solved in a much easier fashion.
Yes, this is slick. Yes, it's an improvement. Yes, this will happen. But...
Having seen people have trouble with pressing control and clicking at the same time (to deselect a single item), I foresee a chilly reception, user frustration and a training issue. 10GUI is like playing Mozart among people only able to manage Chop Sticks.
I see this as stratifying feature... the have's and have not, the able and the un-able. I would request this for my workflow, but the run of the mill admins would be stuck with the keyboard. Aside from the social aspect, there is the difficult task of convincing the boss that "you need this, even if the others don't". Good luck with that.
I have grown to hate the windowing paradigm for all the reasons cited. I'm not convinced that the linear arrangement is an improvement. I'm more in favor of multiple monitors, the main screen for the primary task and satellites with multiple windows for ancillary tasks. 10GUI doesn't address this.
Apple got its multitouch technology by buying a company called FingerWorks. FingerWorks' primary product was just like what 10/GUI describes: a multitouch surface that could either replace they keyboard or the mouse(pad). It largely failed in the market.
People use the keyboard and mouse because they really work well. If people did want more DOF, it would be easy to add more sensors to a standard mouse, for example to record twisting, pushing, and other pressures, but even that isn't catching on.
Another idea that keeps bubbling up is the idea of pressure sensitive keyboards; they also keep failing because the resulting interactions just become too complex and add little benefit.
The real flaw in all these devices is the assumption that the limiting factor in communicating with machines is they "bandwidth" of they keyboard and mouse. It really isn't. Generally, people can think no faster than they can type and mouse, and speeding up the keyboard or mouse any further is pointless.
little behind these guys: http://www.perceptivepixel.com/ Doncha think?
Next they will introduce a non-Cartesian grid of ridges with hybrid haptic and aural feedback, featuring standardized cartographic symbols.
Unfortunately IBM may have some prior-art with their model-M.
Application centric user interfaces are already a problem. On both Windows and Mac these days there's an increasing level of application-centric organization, and that breaks the task-centric workflow badly. I normally have a separate workspace for each task, with windows from each application all visible simultaneously. I can surround each primary document with windows of all sizes, to the sides, above, and below. The 10/GUI control model looks very very good, and would work well for a multi-desktop window-oriented workflow, but the Con10uum user interface would be a huge obstacle multi-document workflow.
This thing has RSI and carpal tunnel written all over it, even though it's a creative idea.
And one of the most obvious choices in a slashdot poll...how can you go wrong?
Is it just my observation, or are there way too many stupid people in the world?
Like tiling WMs the ideas here are good for most people, most of the time, however there are too many weaknesses in this WM. This WM seams to have no provisions for people that keep a window(s) open at the edge (e.g IM window(s) on LHS) and use the remaining space for work. While i think this/a tiling WM would suit me 95% of the time there are those odd occasions when overlapping windows are useful (especially with compiz+transparency).As an example of the strength of conventional WM vs tiled/this, the other day i wanted to watch peep show fullscreen on a 2nd display, flash couldn't handle this, so i just stretched the non full screen version and used some overlapped konsoles to cover over the remaining space. It is rare that I need to do something so convoluted, but I refuse to switch to a tiling WM because it would prevent me doing that, as this is appears to be a more strict WM there is NO chance. Perhaps what we need to do to "fix" the windowing problem is have windowmanagers that tile/horizontally stack by default, but still let you overlap when you need to.
IranAir Flight 655 never forget!
I see a few issues with this approach, some of which I'll offer solutions for:
1) They can invoke an application menu, but not a context menu. You need both (even from the start the Mac has supported context menus with only one button). They could divide the left bar into a top and bottom half, where you could hold a finger on something and then use your pinky to activate a context menu (or perhaps just hold a finger down on anything while pressing the single bar, but that may be too easy to accidentally trigger).
2) Try pinching with four fingers. It's not very comfortable, at all (the only practical way is to lift up the pinky). The solution is to make that a five-finger pinch (to enlarge or shrink the bar).
3) With a lot of apps open (or a lot of app windows) the bar approach is simply not feasible - you have to zoom out multiple times to see everything and then the windows are too tiny. Instead double tap with all five fingers to bring up the bar, but wrapped around (so it ends up kind of like expose filling the screen as much as possible). Then click any screen to jump to it.
4) They don't show any typing going on. Obviously since the clicks have effect, they have to have a typing "mode" you enter. Hello VI for desktops! People always complained that Emacs was really an OS, I guess VI got jealous. My solution to this is that you keep the keyboard (which people want anyway for extended typing) and make the 10 surface basically a large touchpad off to the side of the keyboard (or even just below, though to be large enough to use all five fingers I don't think there's enough space below a keyboard).
Although 10Gui is pretty interesting, I think what we'll see in practical use is an evolution of the trackpad to control more and more things using gestures and multitouch. Awesome video though.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Looking through the examples it looks as though most of this interface has been accomplished with the Palm Pre already. Sliding windows multitasking, check. Multitouch input... check. Gesture area for system menus outside of the screen... check.
Yes Yes! But does it run on Linux?
Seems like all this could be accomplished with just a keyboard. Tap a key to activate "touchpad controls" and then just use certain keys to manipulate the environment -- A and D scrolls left and right. W and S zoom in and out. And many other combinations to mimic what this thing does, and my hands never leave the keyboard.
I like it. I want it. I just wish the keyboard was part of the touch interface, rather than a separate peripheral.
1) tiring to move hands from/to keyboard/touch surface all the time.
2) the easy selection of another window from the task bar becomes a very time consuming operation of zooming out, select window, zoom in.
3) redundant visual effects like scrolling and zooming.
4) where is the equivalent of the task bar icon tray? where is the current language and time, for example?
5) since I can't stack one window below the other, how do I put a media player, chat program, web browser and text editor on the same screen?
6) how do I know the widget my fingers are on? given a menu bar, with 3 items next to each other (File, Edit, Help), how do I know which of my fingers is on File, on Edit and on Help?
7) what about information conveyed by mouse over? with ten cursors instead of one, the screen will literally burst with information at each movement of my hand.
8) how do I work with maximized windows?
I don't think this is ever going to go somewhere. There are lots of usability issues.
While the demo was very cool and full of eye-candy-tastic animations, there is one thing that stands in the way of this idea. Tactile response. You have this mat in front of you that can track all 10 fingers when you place them on the surface, and then to click with a finger, you press it into the mat.
Well first off, you're going to need to feel that as a click, if this is to be the Next Big Thing. The other issue is how hard do you have to press to get it to register a click? If you think about your mouse, the "click" motion is very very tiny and light and yet there is a definite clicking sound and feel. On a trackpad, you're moving your finger around and then (assuming you don't use the button) you have to lift your finger, tap it, and lift it again to register a click. That won't do either. Its going to be a challenge to "tune" this to not be clicking all the time or take too much pressure before it "clicks." If its too sensitive, you'll have people unconsciously straining their arms slightly to reduce the weight of their hands on the mat.
Another thing thing I'm concerned about is how a mouse feels in your hand. I rather like the feel of griping a mouse in my hand, and I'm not sure I can get the same feeling from resting my hands on the desk all the time.
We shall see. I doubt Cindy from accounting will be very gung-ho about this, unfortunately.
This looks really great. But for me, a mistake is that the touchpad is in front of the keyboard. You have to move your arms forward and back when you want to switch; using one hand on each is tricky if the touchpad registers your dangling sleeve or your wrist.
I think it would be much better if the total area of the touchpad was split between the two side of the keyboard. Then you could have both hands on the touchpad or one on the touchpad and one on the keyboard, and you wouldn't have to do the forward-back thing with your arms.
Let's keep our fingers crossed!
nobody proposes replacing mice with light guns on the PC.
True. But a light gun is just a light pen with different optics. As touch sensing technology matured, light pens fell obsolete in favor of touch screens like Wacom's Cintiq and the multitouch surface on which the GUI described in the article runs.
He was still single clicking from a single point of interest, though he had several points he could choose from, he just had 5 mouse pointers on the screen slid around using the basic inaccurate touch pad finger method. But then again, I for one hate touch pads in general (prefer intellipoint like devices on business class laptops), so I guess maybe I am biased. Seemed to me to be more chaotic than graceful.
The mouse and keyboard aren't going anywhere until our computers have a direct link with our brains. The mouse is probably one of the simplest tools humans have ever created. How is the 10/GUI system going to work for people with dexterity problems? Watching the demo cartoon character move windows around by sliding with the right hand and grabbing an application with the left hand using three or four fingers requires too much from the user. Reminds me of the string game kids play.
There's no reason a virtual keyboard could not be integrated into the surface being used as the interface. Add a third 'edge' surface to activate the keyboard, perhaps the edge closest to the user, so a palm touch would convert the surface to act as the keyboard; with a kb displayed as the surface, there would be little difficulty in using a virtual; the system is already sensitized to pressure response so typing a key vs resting on it would be an easy transition. The touch surface could even be slightly textured to represent the divisions between keys, which is one of the common issues with virtual keyboards now. The laser-projected keyboard that is available as a blue-tooth peripheral is another approach; display a kb over the surface with the projection; activated by the 'bottom edge' touch perhaps. This looks like a first pass concept at a real interface, and will take refining, but it covers a significant portion of the needs of the interactions with a quite simple but thorough approach. Where can I get one to start trying this out :)
On a touch sensitive pad you could take an object, like a Hemisphere or Starfish-shape, and the software could interpret that as "The Mouse". By using it on the touch surface, you have a cordless mouse that requires no batteries and can be taken off, or moved to a specific location when not in use.
I have in my workplace a perspex stand that sits between my keyboard and screen for resting documents. I don't see why you can't mount a touch pad on it so that you can change the angle it rests at for a more ergonomic typing position.
With these modifications, or others, you can easily replace keyboard and mouse with a large touch sensitive pad. Without making any more modifications you could use it as is to play games. But, you are not taking the idea and using it's full potential. You are still only considering the hardware. It would be easy to create a mod for , or design a game around, the use of a touch pad for interaction with abilities.
Create an area where movement is controlled by sliding finger from middle to edge to signal movement in that direction (holding it at the edge to keep moving) while having another area designed for triggering abilities, changing view, selecting chat options. ATM we use a keyboard to control characters in game because that is the input device we have to other applications in the computer. I consider it to be a woefully inadequate input device for almost all games I play, and a hand-held console controller to be much better for most. In WoW for example the keyboard acts as a control and command input device for 90+% of the time and only become a text input device when I hit the / key. There is no reason for the same to happen to a dynamic input device like a touch pad. The advantage is that when it is not being a keyboard, it can be optimized to be a far greater command input device.
If you imagine where the future (and much science fiction) is going (has gone), the idea of VR goggles + gloves is going to be a much more common reality. When this happens you can expect that game input will be in the form of a virtual device, an orb, a disc or any other shape you wish. It is very unlikely that game input in a VR setting will be a keyboard + mouse.
Real example: I recently edited a photograph portrait to fix some glitches around the subject's eyes. I had a 1:1 view of the photo in a window at the top of my screen, and a zoomed-in version of just the eyes in another window at the bottom. This enabled me to make fine edits to the eyes and see what they looked like in the original scale. The zoomed window needed to span the entire width of the screen because ... well, eyes are wider than they are tall. Had the windows been organized side-by-side, I'd have had to scale down the view of the eyes to fit the narrower window. The window manager would thus be dictating the scale at which I can work in a way that a more flexible manager would not.
This would be a problem in 3D apps, too, in which users often need multiple views of the same object simultaneously. If I want to see a model of a head, let's say, from front, back, top, and side views, I need to arrange the windows accordingly. I'd go insane trying to look at them all in tall, side-by-side windows.
And what about apps with tool palettes or timelines? Where do those go? Spacial arrangement is key for these UI elements. Does each window get its own?
The interface as proposed is fine for regular desktop use, but it just doesn't work for windows that need different (x,y) sizing while appearing on the same screen.
Now, if we could have a window mosaic instead of a strip, we'd be getting somewhere.
You know what other interface uses all 10 fingers...a piano.
Virtuoso pianists can make magic with the interface.
Most people never really get it.
This is the fundamental limitation of all these schemes. The mouse is easy enough that anybody can use it. The more manual dexterity that's required, the more it becomes like learning to play the piano.
Workspaces are an amazing new way to manage your windows an applications.
With conventional window managers, your windows are just grouped by application, or even not at all. Suppose you're writing an article, and you open up a web browser with some reference material. You open up your mail client and open a PDF attachment which a colleague thought might be helpful to you. Then you put on some music and start writing. But now you're switching back and forth through all the webpages you have open until you reach the one you want, past the mail client and the music player and the PDF back to the document you're writing. Things like this are a big problem, but thankfully, that's the old way.
With Workspaces, we offer you an awesome and new way to organize yourself. So you're ready to start writing that article again - you open the word processor. We're all familar with that, but what happens next is where the incredible new power of Workspaces is shown: press ctrl-alt-2 to switch to a second Workspace where you can open the browser of reference material. That's just amazing!
But that's not all. Next, you can press ctrl-alt-3 and open your mail client, and it opens in yet another workspace. We can open the colleague's email attachment and press ctrl-alt-shift-left to move it to the previous workspace, where we can arrange it side by side with the web browser window. Now what's awesome and incredible about that is you can flip between your word processor and the reference material with the keyboard shortcuts, without having to sort through windows ever. Amazing!
Oh, but before you start, you want your tunes again. Ctrl-alt-4, open your music player, ctrl-alt-1 back to the word processor. Awesome!
Workspaces. It really is just that simple.
Guys, you are missing one incredibly important aspect of this -> Porn browsing.
Everyone knows it's impossible to browse various websites while you're favourite hand is in use. This changes everything.
It seems pretty conventional to me. Nothing here that hasn't been talked about since the 80's.
Yes, it was a terrible idea to make cars in different colors. I mean, blind people can't even see them, how useless!
Signed,
Henry Ford
I've said it before and I'll say it again - get your greasy stinky corn chip covered fingers off my screen. Now get out of here and do not cross my lawn!
Computing got revolutionized by using base 2, not base 10. This is be cause on/off (click/no click) is easy to understand. Imagine explaining to your parents or grand parents how to pan between applications. Touch/Pinch is doable, but once you get into 2/3/4 finger pinch, things get confusing. What really blows my mind is that there is no hand-mash predefined function.
if i need two hands to navigate, surfing porn will get much tougher
Yeah, everybody thinks they want this interface. But if you want to go with "familiar" and "supporting the user's already well-developed skills", I think you'd have to go with the semi-rigid cylindrical form-factor.
The good news is that it could be very compact.
The idea is interesting, the execution is intriguing, but the problem it solves isn't necessarily a problem. The video goes to great lengths to say that current window management systems are arbitrary and messy, but this is the problem. Alot of people arrange those windows intentionally to specific positions to better enable multi-tasking. Additionally, the concept seems to completely forget about the existence of task bars and minimizing in current windowed GUIs. Furthermore this solution isn't even all that superior anyway what with all open programs being oriented horizontally. It sounds like a good idea at first as it allows you to simply flip through your programs, but this is an example of an interface being built around an intended feature instead of the other way around. Any more than a few programs open at once and I can see it becoming very easy to lose track of where the program you were just using is, at which point you have to shrink the entire application space until you bring up their own version of alt-tab, at which point you've pretty much used a more cumbersome method to perform the exact same window management technique that the the video initially comdemns. The final issue that I have with it is that it nearly eliminates the desktop and all it's utility, opting instead to hide it most of the time behind every program you are working with. You can't use it as a quick workspace for file management, or a temporary destination for works in progress, all it's good for is widgets that you can't even see unless you hide everything you are doing at the moment.
OK the video is fine, the device and model are interesting but the soundtrack is awful. Also the narrators cheesy voice acting and brutal pronunciation of words like obstacle, proxy and opposite were hard to listen to.
I know that's all petty and nothing do with the point of this discussion and the video but hey, this is Slashdot.
The first thing that popped into my head as an alternative to the standard window paradigm was a tiling window manager, like XMonad. Seems an awful lot more efficient than a linear window manager. And why not multiple desktops too, while we're at it?
I don't see why we can't have something like this very soon, and the pad below keyboard arrangement doesn't bother me at all. After all, that's how modern laptops are setup, albeit with a much smaller touch pad.
So it's basically rat poison, but with continuous-buttons, but less powerful, and less useful. http://www.youtube.com/watch?v=Rs0GMeL2zA0
You know, if thing is already a capacitive screen, you might as well make it an LCD as well. Yes I know he demonstrated that you can't see it without your hands being in the way but I'm imagining a iPod-esque scroll wheel that can be moved around and let you roll through open windows. Or how about audio/video controls of cover art flow.
Heck, for the people concerned about precision, make it a graphics tablet as well so you can use a stylus to draw. With the LCD multi-touch you could drop controls and a color wheel on there. Even setting/controls that change every time you switch to a different application. When you put more than one finger on it, it could revert back to the basic multi-touch pad.
This could actually work if you used a Touchco multi-touch resistive sensor (http://www.touchco.com/tech.html) because they have precise tracking resolution plus true pressure sensitivity.
Granted, keyboards aren't exactly multi-touch (except for modifier keys like SHIFT)
But that's four possible keys right there - Shift-Ctrl-Meta plus a normal key In Emacs land Meta was also Alt for a while because of PC keyboards, but I seem to recall some original keyboards (probably on Lisp machines) also having a real Meta key just as we have Windows/Apple keys today, so we really have five possible (though on keyboards really three modifiers are the most practical to use at once).
I totally agree with your assessment of Emacs being a keyboard based variant of this gui, after all there's even buffer rings to traverse... in a way the 10/Gui video has presented a new way to explain why Emacs is so productive.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
It would also be quite impossible to play FPS or other kinds of games with this type of setup.
Have you ever played Metroid Prime: Hunters on the DS? It could work very well. I don't see any problems with gaming on a touchscreen. The keyboard placement issue is the only real problem I can see.
"When information is power, privacy is freedom" - Jah-Wren Ryel
In fact, they are so good that I wish they would sell a stand alone trackpad to add onto a desktop keyboard.
The way this could really work, is to have the standalone trackpad work as the Apple trackpads do today... but then also support use of a stylus with pressure sensitivity, like Waccom tablets today.
The only people it would not satisfy are FPS gamers, but they buy specialized mice already anyway so they can keep doing so. For all other uses I would prefer the dual trackpad/stylus pad.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
George Orwell teaches 10GUI to his mother:
"Four fingers good, two fingers bad!"
No but seriously, if my mom ever asked me about this, I'd have to tell her to stay away from it. If people can't remember the commands for copy and paste, there is no way they can keep track of 8-10 fingers all at once.
-b
No offense, but I've stopped responding to AC's.
What if there was a place to touch on the pad to turn the pad into a keyboard? I'm not sure how easily we'd be able to type without feedback but I think there are keyboards today that project your keyboard and track your fingers. Seems like you could do the same sort of thing here.
Why settle for just 10 finger tips? Go for the whole hand, arm or body like in deaf-language. There are people working on this.
While a lot of pros and especially cons where underlined, they can all be compensated at the user level. You're missing one finger? Set the preferences accordingly. FPS? bring it on, I can't wait in fact; Developers should be able to adapt it, the same way they did for the Wii. Why don't make a virtual keyboard that you can fetch with a bottom pane? Need precision drawing? make a pen that works on this. Btw, you really think you are making straight line with a mouse? Thankfully you selected the straight line option in photoshop...
Seriously... something like that would probably work fine for say, my grandmother. Who tended to only have her e-mail app open, or her card game of the day, or whatever.
But god, I can't begin to imagine the suffering of trying to write code on an interface like that. I'm all for multi-touch, but sheesh. Why is it every 2-bit designer who comes along thinks that to progress UI's they have to toss out everything that's managed to stick over the past 30 years.
I have a lovely 30" screen... plus another smaller screen. When I'm coding it's not uncommon to have the project open, multiple source and header files, plus other windows open for reference, testing, source control, server connections, and others. This is hardly cluttered either, I -want- absolute control over where each of these windows is on the screen and what size they are. I don't want some OS trying to organize them for me off the screen. Because especially with client/server coding, when I say, run the app on one screen, I don't -want- it hiding all the windows that are tracking log files on the server too. I'm watching those for immediate feedback.
So while I think multi-touch is great and certainly has a place on the desktop of the future, please won't someone come up with a way of using it effectively amongst the UI's we already have!
From watching the video, it looks like they presuppose quite a high degree of dexterity on the part of the user. To resize an app window in the video, they touched three fingers to the pad, held two of them still while pinch-zooming with the third. Now, you probably don't have to hold two of the fingers still relative to each other, but this still sets the bar quite high in its requirements for independent control of your digits.
This would exclude a lot of users, and not just those who are missing a few fingers. About 10 years ago, I helped set up some web-browsing PC's in a community-college library. One of the things I hadn't anticipated was that some of the older users were lacking in coordination to the extent that it made it difficult for them to use the PC effectively. Even the task of trying to double-click fast enough to get it to register as a double-click would cause these users to twitch some of their arm muscles, causing the mouse to scoot a couple of inches in the middle of the double-click. They'd try several times to get it right, but the mouse always ended up moving off of the targeted icon before the double-click was complete.
What we eventually had to do was to provide trackballs, so that the user could move the mouse pointer to where they wanted it and then, taking their hand off of the ball part of the trackball, double-click the buttons. So, that is the level of dexterity that some users are at. So, this three-finger-pinch-zooming idea... I could see this being extremely frustrating for them.
Metaphors are held to the laws of physics, aging and death. Witness crop of new gestures creeping up on desktop files and apps.
Transitional schemes bridge us to the next heuristic interface which hasn't evolved yet to mass market technology.
Voice .vs. touch will define the future as technology obviates the necessity to command .vs. control.
Before GUI's there was the command line. .csv extention in your current location, and no files in other places. You have instructed to change only the last 3 letters of these filename, no matter how long they are. You dont have this kind of single-command power with a GUI.
The command line is great in that you can initiate an unlimited variety of tasks. These tasks are powerful because the interface is based on your imagination.
For example: \rename *.csv *.txt
You cannot see the files in the directory you are in, but you know in your mind that you are selecting all files with the
The downside of a command-line of course is that you have learn the commands before you can use them.
That leads us to the GUI. All of the controls of the objects are presented on the screen. The idea is that how to manipulate the objects is self-evident (somewhat). Users learn by interacting in real-time.
i've never liked gesture-style screen commands because they have all of the drawbacks of a command-line but very limited power. perhaps some general commands like on the ipod interface, but it will never be a replacement for menus or text commands.
Regarding the "problem" of organizing windows as shown in the video..I contend that that is not a real issue. The only problem is the current window managers drop stuff 'wherever'. I work perfectly fine on a small monitor with all my apps maximized. My coworkers have twice the screen real-estate I do but still only use one app at a time becasue the foreground application is hogging most of the space and the others are splattered around behind it.
At home I have a nicely organized side toolbar with all my frequently used stuff, and I have ZERO desktop icons.
oldhack: "Security is a waste of money until shit hits the fan. 5 minutes later, it becomes waste of money again. "
Used a multitouch pad a few years back replacing a mouse. Had multiple gestures and macros. After a while, though, my fingers became oversensitive to the surface and the touch became very uncomfortable.
Regarding pointer precision, it's easy to imagine a setting where long movements of one's finger result in short movements of the pointer on screen. Most mouse current mouse driver's have such a "speed" setting. Now consider implementing this in a multi-touch context. A pinching gesture with one hand, for example, could dynamically adjust the pointer "speed" as it's being positioned by the other hand. You could have almost arbitrary precision on demand.
Regarding the need for a keyboard, it's also easy to imagine a virtual keyboard overlaid on the touch area. The surface already detects presses of all ten fingers. It's simply a matter of defining which areas you want to correspond to which keys. This would enable several advantages. You could choose different keyboard layouts, remap keys and even scale the entire keyboard to suit the size of your hands.
You may wonder, how would you align your hands correctly on an invisible keyboard? Easy. Place all ten fingers in their starting positions and have the virtual keyboard align itself to YOU.
What I've long wanted for MMORPGs is a screen in front at the normal position, and a second touch screen in the position 10GUI has the touch panel. Note I want the touch thingy to be a touch screen, not just a touch panel.
The screen in normal position would show me my view of the world, with minimal overlays (probably just health bars, damage and heal numbers, and such, and minimap, and maybe the last couple lines of chat).
The bottom screen would also show the world, but would have action bars, full chat, and so on.
It would allow casting spells and taking other actions by touching the target with a designated finger, and invoking the action by either touching the icon for that action on an action bar, or by gesture. There would be gestures that combine targeting and action.
For instance, lets say we get hit by AoE, and I need to get heal spells on everyone. I'd touch and hold on the icon for the heal spell I want to use, and then with the targeting finger tap everyone I want to heal. Similar for attacking, defending, and buffing.
I don't know about your mouse, but mine does make use of more than one finger. It has multiple buttons, a wheel and such. I can zoom, rotate, move, etc, etc, with the mouse. In fact some of it is easier. I'd say a scroll wheel is way better than using two fingers for zoom.
I'm not opposed to a new input paradigm, but I am very unconvinced this is generally useful. To me this seems like a flight stick or something. May have limited use, but in general isn't your all in one input device. I do not see this replacing my mouse.
That's a very good point. And even if a user cannot use a mouse, they can still find hardware that emulates it more easily, since, by 10/GUI's own definition, their system requires more "bandwidth" (input channels).
Still, while I'm as concerned about accessibility as all but its most serious champions, I'm not sure that holding back a good new GUI model for accessibility reasons would be a wise move. Then again, I've had a few ideas myself for a new GUI model, and it, being higher-level for the machine, would be easier to translate to a audio interface, for instance.
So yeah... basically, I think it could be trouble, input-accessibility wise, but I still think it looks good enough to (rightly) gain some traction in the IT/user community.
My main concern though, is more fundamental, both from a normal user's point of view, and accessibility-wise. That is... how many people are goint to understand an interface with 5+ "meta-levels"? Can the average user really get that three fingers to zoom around windows is linked to using two fingers to zoom around in an app? Or that five fingers is logically the thing to do to control all apps? Maybe, but I don't know that I'd like to count on it just yet.
I blogged this last night. Short version: fail.
Problems:
1. You just doubled the amount of space I need between myself and the monitor.
2. Multitouch allows for more kinds of interaction: true! However, this interface steals ALL of them away from use by the applications.
3. Left and right sides of the screen aren't discoverable. Might as well be top and bottom -- i.e. bottom of the screen for application launching (call it a "dock") and top of the screen for context-specific options (a sort of "bar" of "menus").
4. Linear spatial overload of windows is no better than two-dimensional spatial overload of windows. Labelled zoom-all-the-way-out cheat no better than Expose and application switcher.
5. Where does file management fit into this scheme?
Lukas Mathis calls 10/GUI "one of the most dramatic reimaginations of the desktop user interface I've seen in a long time" but on examination it's an incremental hardware update with no real interface breakthroughs. Keyboard + mouse has gone on for far too long, as has the W.I.M.P. interface. A better direction would be a tactile multitouch surface which can be anything it needs to be, including a keyboard (for any language), coupled with a GUI that represents tasks and actors rather than objects in a space. 10/GUI does nothing about window and document clutter, squinting, scanning large lists, or making the computer's workings and status an organic part of its presentation. The video may be a slick investors' reel, but shows no real progress.
as opposed to still having movable keys that also happened to be capacitative...
Unless it actually said that, in which case, I was on the phone at the time :-D
1. I hate the window manager; it is an interesting idea but a horrible implementation.
2. Two hands is a problem. Think Twister (the game).
In my opinion, it would be better to keep a normal window manager and have one touch pad on the right (or left) for one hand.
Steven
I don't agree at all with their conception that the control surface and the display should be separated. I'd argue that the display should continue on to the touch surface, allowing both to be used. After all, you don't see many people using keyboards without key caps, or light boards without captioned buttons, or audio mixers without labels, even if they are as simple as colors on the knobs.
Removing any sense of context whatsoever, and therefore having to devote more of the vertical display surface to controls such as a virtual keyboard (sans physical feedback), or a virtual palette of actions, is a waste of valuable space and would also serve to dramatically reduce input accuracy.
Give me a keyboard and a screen on the desk with a context sensitive multitouch interface and I think it would be significantly more usable than this video concept.
A flat plane of glass sitting on a desk has all the same ergonomics issues a flat keyboard has. The 10GUI concept uses 5th fingers way too much too, just like a standard key board uses 5th fingers too much.
There need to be an input device with both hands resting on the desk with thumbs up and palms facing each. Until they figure out a way for hands to rest in a natural position while inputting, there's going to be big problems with repet. motion injuries. You have to spend too much time in whatever positions manufacturers put you in.
The only one that gonna work without injuries is the natural position hands return to at rest.
This Perceptive Pixel product doesn't solve repetitive motion injury issues.
Different parts of the body will get injured, but there will still be injuries.
For your input device, think hands at rest. Thumbs up. Palms together. Finger curved.
Anything else will still be causing rep. motion injuries.
Window management is a solved problem.
I see their video with this contorted user interface for accessing what are just standard applications, and I know they've missed the boat.
If they had spent all the time they put into the window manager, into researching powerful uses of multi-touch to carry out the functions of common business APPS, then they might have something.
As it stands this is just another window manager to toss in the dustbin of failed window manager history. I mean seriously, the window manager reverts back to Alt+Tab or Expose style operation for more than 5 windows anyhow. I would never consider using this over Expose.
They wow you with eye candy and showing how easy it is to move windows around, and draw random lines, and zoom in on photographs. Who is spending all their time doing these tasks? Show me the same interface with someone doing real work, like coding/writing documents, or touching up artwork, or working with spreadsheets - the stuff we spend 99.9% of our time doing. Methinks the new GUI would show many many weaknesses.
Fortunately half the population is already naturally equipped with this interface.
I actually watched the video yesterday before the /. post came up and sat through my thoughts for a day and this is what i feel.
Multi touch interfaces are overhyped.
Point being, 1) I think one thing that the mouse works really well is that it basically is a amplification and translation of human motion. This is my main criticism of this interface design. I can understand why touch works for the iPhone, but say for the computer, simply put, for zooming in and out, most photo managing programs (hello picasa) uses the mouse scroll button to zoom. And it really translates and require far less effort on my part to perform this task as to if i were to keep pinching my fingers. And reset and pinch again. Basically the mouse wheel allows me to zoom to theoretically infinity. Thus that is a reason why I don't think I would want something like this. The effort to use this device is basically presented in the concept itself - It has a surface that is wide, and it needs ten fingers to operate. To what benefit does it bring? Interestingly, I'm more of a fan of the new Wacom Bamboo Touch. I think while the concept is similar, the execution is different and the device itself would have more potential in a still windows based environment.
2) I don't see the point of tiling windows in a long strip. It may be good for some applications but not for others. What about applications that have inherently small windows? Like Instant messengers. Are all my conversation windows going to be tiled full screen over my big monitor? The other thing that I'm thinking is what if I'm referencing something. I've done this a few times, say if I'm writing code or essay and I keep a webpage or something maximised in the background while my current window occupies a smaller window on the front? And basically the zoom all the way out fees like expose, or compiz on linux where you can line all windows up. And scrolling the windows tile is simply.. alt tabbing? But with touch gestures? Imagine if you have a capacitative touch pad, and you can say use a 2 finger gesture to alt-tab (+ auto maximise if you so desire). Does it work the same way?
I love the idea of reimagining interfaces. And I really laud the video poster for his good work. But I don't think it works for me and I'd like to post this as some constructive feedback.
I like that they used Linux for the demo. That means they probably have a window manager ready to go (i.e., I won't have to wait for Microsoft to adopt it). When it comes out, all I'll have to do is "emerge 10-gui" (oh, and buy the hardware) to try it out.
Speaking of which, the hardware had better not be too expensive. That would kill it.
I don't think I like their linear window management idea. I like that I can arrange windows to overlap on a 2D space, and access them with a single click, or Alt+Tab in 1-D fashion. It seems better to me than manually scrolling through all my open windows.
Oh, and attaching the touch surface to the place where I rest my wrists to type? No. Just no. That's the dumbest idea ever.
...because "hacker" sounds way sexier than "code drone."
I guess I needed to watch the last 10 seconds or so... until then, the video showed fingers touching a surface which I assumed to be about the size of a keyboard, and I just assumed that it was the keyboard. I went back and watched the last bit and see what I was missing (did you feel the earth come to a momentary stop... someone on /. just WTFA).
Now I'm curious how they will manage to prevent me from accidentally brushing the pad with my palms while I'm typing. I already have this problem with my mac track pad, as I'm typing the cursor goes flying in weird directions. They have pressure sensitive abilities that may be able to help, I'm now more interested...
Now, instead of just moving a piece of plastic around and clicking one or two buttons, we can use this much more simple solution.
You just have to remember what a one finger command does, what a two fingers command does, what a three fingers does, what a four finger command does, and what a five finger command does and you're on your way to input SIMPLICITY.
Oh, and make sure you don't touch the pad with an extra finger or you'll end up resizing your desktop instead of scrolling through windows.
I found the most interesting part of the video to be the SIMPLE new way of organizing windows. Now, instead of all those cluttered windows all over the screen, you can have the simple and easy to use solution of having them in a line with most of them not visible off of the screen!
And you can bring up a view that will show you the names of currently running programs! That seems so innovative, but I'm sure I've seen something a little bit similar to it before. Oh, that's right, I've been looking at something that does that everyday I've used a computer since Windows 95.
I'd still want a mouse too in addition to the touchpad and keyboard. Even if you can use the touchpad as mouse and keyboard I'd still want them for heavy use.
I'd also want a task-based system added to this. Instead of one monolithic group of apps, windows, and files let them be broken up by task so they fit how users think. Maybe a vertical scroll for switching between tasks and when zoomed out it's easy to move/copy between tasks.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
From a psychological perspective, the "attentional spotlight" does not lend itself to having 10 points on the screen to focus on. Imagine if you wanted to sort out coins on a table. It's more natural to look at coins individually and sort them sequantially than look at them all together, place multiple fingers on different coins etc. The gesture side of multitouch is also unconvincing. The video talks about tasks that are already extremely easy with a mouse and keyboard, and take up very little time. I'm sure it's easier to scroll a mouse wheel than move two fingers apart on a pad, and why would I want 10 fingers on the screen to swap a window when I can do it with one simple drag and drop with the mouse?
Why would we need to place both hands on such a pad? I can easily imagine myself using a single hand on such a pad while still having my other hand on the mouse for finer control. I think the idea is to have the choice between the keyboard and the pad in order to better adapt to various situations.
In order to prevent having to switch from keyboard to pad as we already do with the mouse, we could even make a keyboard that has touch-sensitive keys. When you just drag your hand over the keys without pressing them, you could use the keyboard just as one would use the pad, and when you press the keys you get normal keyboard usage.
I have a question about the feasibility of the design. As I understand the video a user rests their hands on the 10gui input device and the capacitive sensor detects the positions of those ten fingers. Pretty normal multitouch so far. Where the 10gui design is innovative is the combination of the resistive layer to detect 'clicks'. If you watch the video at 5 minutes you'll see what I'm talking about and where I'd like to understand how this is being done. The problem is, you cannot detect multiple presses with a resistive layer - that's a basic limitation of that tech and no way around it afaik. My initial reaction to this was to think 'hey, this is just a concept for now, lets look in to the boring details later' so I went back and did some research on the other multiple touch sensor technologies and found a very interesting problem. A very fundamental problem too! Take a look at this video http://cs.nyu.edu/~jhan/ftirsense/ So, all the other multitouch sensors are optical - That I've found, and I would dearly love to be corrected! - are optical. So the fundamental problem is how do you 'see' the difference between a user resting their fingers on a surface and the user pressing down on the surface. You need to use two very different technologies - as 10gui proposes. But, all the other technologies are single touch only. I use a multitouch sensor for my desktop (wacom bamboo 8x5) and it is tiring as hell to hover your hands above the input device. The only way these device will work is as demoed in the video. However, I dont think its technically possible.
This is a nice revisit to the technology but even now I am using a multi-touch interface to type this comment. I was luck enough to be an early adopter and have a few of the now defunct FingerWorks LLP products. Notably my two TouchStream LPs and an iGesture pad. Unlike the displayed product my keyboard/mouse has two pads and is arranged in an ergonomic arrangement (see image: http://pcworld.in/uploads/images/pcworld/4915355_15.jpg).
There are some finger markers to let you find the home row by touch. I move the mouse with two fingers on the right side of the screen and control the cursor in applications with two fingers on the right. There are a number of programmed gestures that allow for a great deal of task automation in stead of multiple click and drags.
I love the device but it does have limitations. Response time is not so great for gaming as you have the Gesture interpreter layer to contend with. There is a game mode for the device but I've rarely used it. I still mis-type a lot but then the driver also helps and makes a lot of corrections automatically based on what word I am typing. In my case it's usually about 90% correct when it fixes my typos.
I loved the GUI interaction metaphors that the video describes. I hope that older versions of devices like the one I have will get grandfathered in to design considerations or even further development!