Slashdot Mirror
Making Fingers Work With Touch Screens
An anonymous reader writes "A paper was recently published about Shift at the Computer Human Interaction Conference earlier this month. The authors (Daniel Vogel, a Ph.D. candidate at the University of Toronto and Patrick Baudisch, a research scientist at Microsoft Research) developed the technology to solve several problems with mobile-phone touch screens. Most such screens are designed to be operated with a stylus; when touched with a finger the UI doesn't work so well. They also created a short video with a demonstration of how Shift works. Shift builds on an existing technology known as Offset Cursor, which displays a cursor just above the spot a user touches on the screen. That allows a user to place their finger below the item they wish to choose so that they can see the item, rather than hiding it with their finger."
Then you'll be able to use your stylus-like fingernails, thus solving any such problems! :)
"He who can destroy a thing, controls a thing." --Paul Atreides, Dune
The prof already has this one sorted.
Do not try to read the dupe, thats impossible. Instead, only try to realize the truth
What truth?
There is no dupe
So if I see a button the screen, I don't press the button; I press below the button. That seems rather counterintuitive, no? And how do I push stuff at the bottom of the screen?
-dave
http://millionnumbers.com/ - own the number of your dreams
"The fingers you have used to dial are too fat. To obtain a special dialing wand, please mash the keypad with your palm now."
Ever since it was announced, I've been wondering how well its touch screen is actually going to perform in everyday use? Anyone had an opportunity to play with one of the demos for an extended period of time?
u-bend
So using an offset is going to help people who can't seem to hit the target in the first place?
This guy's the limit!
That allows a user to place their finger below the item they wish to choose so that they can see the item, rather than hiding it with their finger
Just like when I use a telephone, I hit the buttons next to the number I am looking to dial and when I park my car, I park next to the spot I want.
Yeah, yeah, grooming experts tell you to trim your fingernails but why is it I always run into a problem where I need them right after I trim them? Yeah, yeah, that's what pocket knives are for. Anyways, with a small touchscreen like a Palm pilot, if you have just a wee bit of fingernail, you can poke the screen with that rather than pull out the stylus. The trick is to have just enough nail that you can do this but not so much that you look like Freddy Krueger. I suppose another way around it is to put a poking device on your pointer finger but that might get mistaken for a gom jabbar. Try getting THAT past the TSA at the airport!
Kwisatz Haderach
Sell the spice to CHOAM
This Mahdi took Shaddam's Throne
I've had a touchscreen in my car for years now for my PC, and I quickly learned to adjust some of my habits to it. Specifically, I:
1. Use my fingernails. No fancy glue on stylus or anything, but finger nails don't leave oily traces unless I've just finished gutting a whale by hand or something.
2. I do it palm facing towards me, pointed up. This keeps the contact area visible the whole time. If the computer were british, it might look like I was flipping it off, sure, but it works well.
When you have limited screen real-estate (like on MP3 players and phones), there isn't a lot of room for fancy hacks like what the article suggests. I'm pretty sure customers will meet the technology half way by necessity if nothing else.
My mom operates her mobile-phone by throwing it into the corner until someone calls her.
e -crap.
She doesn't need such lets-bloat-our-product-to-become-even-less-useabl
I don't like mobile-phones because some people use them as excuse to open and shut their vocal-a**hole even more.
Even in church!
'The tyrant will always find pretext for his tyranny.' - Aesop's Fables
Humans didn't grow up pressing below what they want. When I go to dial my desk phone I don't press just below the buttons. It's assumed in my brain that I will be covering up what I want. The problem I see is the size of a stylus vs size of a finger. If you cram buttons close together my finger won't be able to resolve which one I'm actually trying to press. But the same is true of real buttons (Simpsons and the dialing wand).
Not that we can't learn. Just as spear fishers learned to take into account the refractive index of water when fishing. I'm sure it took a while, but after the learning period I'm sure it's second nature to aim X below what I want to kill.
I'm interested in seeing how Apple solved this problem with the iPhone
tip the screen around all willy-nilly like. it'd be fun for emulating those old tilt-the-thingy-and-get-the-ball-in-the-whole games.
If so, keep your existing touch-screen-enabled gadgets calibrated, else...
This is interesting, because I've been working on finger touch based UIs of late. I've come to a few conclusions:
The touchscreen for many devices is physically designed for use with a stylus. They require quite a bit of force to register, and it is difficult to apply that much pressure with a finger because of the amount of surface area contacted. The DELL Axim touchscreens work particularly well with finger touch, while others, like the Asus a716, do not.
GUI Design is critical. Microsoft's history with mobile devices has been to make them as much like Windows 95 (and up) as possible. Windows CE 1.0 was exactly like Windows 95. Although with Pocket PC (CE 3.0) they tried to follow Palm's dominant (at that time) lead, and simplify the GUI, it is still most conducive to mouse / stylus input. The iPhone is a perfect example of how to design a GUI for finger based input. The multi-touch hardware capability is not even an issue at this point - pure software design is responsible for the bulk of the usability.
Along those lines, Microsoft prefers static dialogs that show as much information at once as possible, requiring small, desktop-like controls that demand precision stylus input. The iPhone is dynamic, scrolling in new options as the user make selections. Thus they have room for large, finger-sized buttons, because the display changes constantly. Many controls, like scrollbars, are unnecessary because entire display areas (like lists) can simply be dragged and tossed, which is the most natural behavior in the first place. The scrollbar then becomes only a visual indicator, which can even be hidden when the user is not interacting with the screen.
I've put together some code that behaves like the iPhone's drag interface, both in 2D for rectangular regions, and 1D for lists. It works really well on the Axim, again, because its touchscreen is nice and sensitive, even when retrofitted to existing Windows List controls. So it obviously is not a matter of hardware, but GUI design, that Windows Mobile isn't conducive to touch input.
So basically, this article is not stating the real problem, which is that MS is completely missing the mark with the fundamentals of their mobile GUI. But instead it offers a clumsy hack to work around an improperly designed UI. The ironic thing is Shift's Offset Cursor doesn't work at the bottom of the screen. That area is the most important for user interaction, because controls are strategically placed their so the user's fingers (hand / stylus) conceal as little extraneous area of the display as possible. That is why onscreen keyboards are always at the bottom, which makes them inaccessible to this Shift hack. The article fails to mention that little detail too.
Dan East
Better known as 318230.
my windows start button won't work!!
FOXTROT UNIFORM CHARLIE KILO
... with Windows Mobile 6:
User: (Touches 4 key)
Phone: You have touched the 4 key. Allow?
User: (presses yes button)
Phone: You have touched the yes button. Allow?
User: (presses yes button)
Phone: You originally pressed the 4 key. Were you pressing the 4 key, or using our new counter-intuitive software that would allow you to really be pressing the 1 key?
User: (touches the 4 key in response)
Phone: You have touched the 4 key again. Allow?
User: (presses yes button)
Phone: Please stop picking on the 4 key. It doesn't like being touched like that.
... please mash the keyboard with your hand.
I am a tech writer and deal with GUIs every day.
I agree on your points about not working at the bottom of the screen being a big issue. Not only are controls placed so you can see the screen when using them, on a cell phone your thumbs/fingers are easier to use with controls placed closer to the keypad as your hand is holding the center or center-bottom of the phone. I'm surprised that the target can't (at least not yet) flip its targeted zone based on the area of the screen - too close to the left edge? flip it to the right!
That said, this is a move in the right direction. I'm curious if it can change the size of the circle based on the pressure area - ie. if my kid brother is using a pinky finger, or if my fat cousin is using his thumb. The first is near stylus in size, where the latter might cover a quarter of the screen.
Not all digits are created equal!
You want what? by when? Sorry we haven't finished the time travel project yet... that's next week.
Shouldn't it be "Making Touch Screens Work With Fingers"?
The headline makes it sound like they've figured out how to make those pesky humans more compatible with the touch screen technology. Granted, most handheld operating systems involve the computer and the user meeting each other halfway, but this headline made me envision plastic surgery to make fingers more pointy...
Sam! If you will let me be,
I will try them.
You will see.
mousePos.setY(mousePos.y()-20);
If you are planning to use this technology in your own software, please contact the Microsoft Research to purchase the appropriate licenses.
I don't care if it's 90,000 hectares. That lake was not my doing.
This was an article on finding a way to work with convention show style touch screens. My fingers, for whatever reason, don't seem to always register as a touch on touch screens. I'm not sure if my skin is too thin, or not conductive enough (or too much?). A few of my coworkers have the same problem. Does anyone else have this problem, and have they found some sort of device to counter it? Pen tops, and Stylus' don't react with this particular sort of screen, unfortunately.
Between calibration errors and finger size I've just learnt what size objects are worth fingering on my Toughbook screen. Only when dragging does the hand remain in the way - I typically use my nail with my palm facing me for precise placement in Visio or selection of text. If I have to take my hand off the keyboard then a quick tap on the screen can get the pointer within a few pixels. Precision on a small screen is no different - just less useful in general. Adaptive software seems a better avenue - think of how easy text selection is because of auto-word boundaries. Does anyone else find it difficult to use a touchpad one handed? How about for a drag operation?
Anti-Smudging, or rather quick smudge removal technology has already been patented.
I read a paper on LTWS (Lick Tongue Wipe Shirt)
I believe that the title is correct and appropriate because
Microsoft's generic approach is to leave no options to customers other than to do it the Microsoft
way with Microsoft proprietary solutions without regard for the
relative quality of the Microsoft solution
The fully expanded version of the title would be Making Fingers Work With Microsft-Touch-Screen (TM).
Since the likely presumption is that all are most touch screens will be driven by Microsoft
software the original title works well.
The word combination "touch screen" or something even more generic will shortly be a
Microsoft proprietary name/phrase/word combination.
I thought they were talking about chocolate fingers :(
Backup _all_ your data, then do a hard reset. When the screen calibration drill appears, make sure you tap some distance _below_ the markers. Next you have time to practice the new input method with the cut'n'paste drill and while you install . Viola!
The Hacker's Guide To The Kernel: Don't panic()!
Its been a while I dont have a PDA, but a few years ago when I used to work with one all day I always used it with my finger and it worked perfectly, so good I rarely bothered to pull the Stylus.
Im not sure if Im missing something here...
In most touch screens, the best way to use it is to press with your fingernails. Seriously. Lightly press with your fingernails, and the response will be at least 90% dead on. I'm not talking "Lee Press On Nails" length nails, just enough to protrude at least a half millimeter beyond the end of your finger. The keratin is an excellent conductor, and works as well as a PDA stylus in many cases.
Ironic that the most common biological factor that could aid in an electronic interface is the one most people cut to the quick out of sociologically enforced hygiene, despite its minimal worth in a hygienic setting for most people.
Just because you can mod me down, doesn't mean you're right. Shoes for industry!
"Making Touchscreens Work With Fingers"?
I'd hate to think what kinds of surgery our digits would require to make them as effective as a regular stylus...
It depends on what the touchscreen technology is, to some extent. I would recommend against using fingernails. What I would recommend is that when you put your finger on the display, make your touch something that is not so much of a 'poke' as a 'touch'. Seriously, don't poke and get your finger off as fast as you can - make a gentle but slightly longer lasting touch to the display and it will work much better. A touch is actually an average of multiple readings taken when you touch the screen. The slightly longer lasting touch will give the controller more information to work with and result in a more reliable result.
A finger-driven user interface should require, as a core feature, that the user interface did not present to the user buttons or other selectable entities that are too small to mash with a finger.
Once you've designed with that requirement in mind, the need for this software becomes rather moot.
Now maybe for something like an on-screen keyboard you have an issue, because you can't fit many finger-pressable keys in that. Apple's iPhone however enlarges the key as you press it, and this solution would slow you down on each keypress as you rock in a direction to select the correct key.
Other people have already pointed out that most people use a fingernail after a while for accuracy anyway, rather than hitting with the fleshy bit of the finger tip.
I'm always amazed that the people who design GUI user interfaces don't study anatomy or even have a basic understanding of the relationship between the screen elements and the body activities needed to interact with them.
The most glaring example of this is ever-present and manifestly stupid Windows message box that appears right in the center of the screen at the wrong time and demands that you reposition the mouse pointer on it and click before continuing with your work. Inevitably the movement to reposition the cursor is outside the 15-20 lateral movement range of the wrist that is guiding the mouse which forces an upper arm movement that completely disrupts the mental concentration on the user's current task.
Every time I bring up this issue to a programmer they give this look that resembles that of a monkey encountering his first music synthesizer. Windows makes it so 'easy' to deal with program exceptions by just slapping an error box in the center of the screen. But from the perspective of a user seriously engaged with using the software inactively, it is just so wrong and boneheaded. And, of course, since it was introduced in the Windows GUI, it tends to migrate into the Open Source GUIs as well.
People who design interactive weaponry for a living spend a lot of time getting the interface between the solder's body movement and weapon response exactly right. In these situations, 100 millisecond intervals are critical.
Would be that programmers started thinking in the same way. For more discussion in this area, I suggest the AskTog website.
The application for a patent on this saddens me. I have had thousands of conversations in the past 25 years during which I have freely discussed all kinds of touchscreen GUI issues. Some of the conversations were about touchscreen GUI effects that I had created and some of them were about effects that would obviously make the operation of the touchscreen easier.
I had many conversations over the years dealing with this specific issue, of using the magnifying glass effect on the GUI to display the area occluded by the finger. I didn't implement this effect because I have not been doing much work on displays with a diagonal measurement of 2 to 3 inches, but it is an effect that was often the subject of conversations I've had with many people and even in some lectures I've given.
I'm sad to see that somebody has now decided to patent something that has been a common topic of touchscreen GUI conversations for many years. The patent can hardly be considered non-obvious. It could well be that the two people involved here, one a student, one a microsoft employee, are simply ignorant of the basic design issues of graphical touchscreen GUI's.
I would go so far as to say that this patent application is morally reprehensible, right up there in league with patents on seeds that have been around since the dawn of time.
to dip the end of the spear under water to cancel that refractive difference.
But then what do I know? I've only watched a few seasons of survivor.
You want what? by when? Sorry we haven't finished the time travel project yet... that's next week.
Sure maybe you might miss a small target the first time with a standard touch screen, but it's not rocket science to try again.
I can see this being useful where targets are very small, and very densely packed on the screen.
A little off-topic, but has anyone used Home Depot's self-checkout lanes? Just bought a house so I'm there a lot :( It's probably the same everywhere, but the item scanner is a good 2 feet away from the card reader, which isn't well integrated. You find yourself going back and forth sometimes and inappropriate selections such as payment type are actually on the item scanner. It's a very hacked-together system, probably since they wanted to recycle existing card readers.
The classic card-readers need their UI's redesigned as well. Do I really need to be funnelled through the "cash back" options every time I buy something (which I use about 1% of the time)? Especially when there's a long delay with no feedback and I've already pocketed my card. So annoying!
I swear to God...I swear to God! That is NOT how you treat your human!
There's practical issues as well. For instance, if you let your nails go, typing gets weird. You have to move your wrists back and down, putting stress on the wrists to compensate for the nails, or you have to accept that you won't have soft fingertips hitting the keys. This gets old really quick.
Plus, if you do anything useful with your hands, you're gonna get gunk under there. Which you have to clean out unless you're a gross slob. And the longer the nails are, the harder that is.
Can you be Even More Awesome?!
Another option is rethinking the UI in general to start with. I think the article details a good option for exising UI's, backwards compatible and all. But designing for a touch screen really takes a bit of refactoring, in general. For example, most programs on my Pocket PC phone are unusable with fingers on the touch screen; but Tom Tom Navigator isn't; it is a dream to use with just a meat stylus, in almost all the modes/screens it has.
Love many, trust a few, do harm to none.
I was playing with an LG Prada touch screen phone and when you press the numbers to dial or text, the on screen buttons re-appear offset vertically above their original position. It works a treat. (This was in London, England.)
O'WONDERWe're working on it.