The Ultimate Dual-Hand Touchscreen

LithiumX writes "This morning I saw a video demonstration of the most interesting input technology I've seen for a long time. This is a touch-screen that accepts inputs from multiple (I saw at least 8) points at once. It seems very responsive, the display is large and of decent resolution, and they actually wrote software to take advantage of it. It appears to be entirely research at the moment. I'd offer up organs for one of these things."

Comments

Touchscreen keyboards

[Ekarderif]

There have been touchscreen keyboards for quite some time now... So what's so special about this?

Re:Touchscreen keyboards

[Anonymous Coward]

touchscreen keyboards? huh? did you even bother to watch the video?

Re:Touchscreen keyboards

[88NoSoup4U88]

It's 'special' because, as a first, it looks very cool, but it also serves as more than a keyboard (actually watching the video might help, you know).

While applications like this have been around before, most of the time they still had to be controlled with a special hardware-device: And it's very cool to see they now succeeded in bringing it to only be controlled by the fingers.

Re:Touchscreen keyboards

[canning]

So will applications have to be written (or rewritten) to except inputs from this screen?

Re:Touchscreen keyboards

[SimHacker]

Just as keyboard driven applications had to be rewritten to accept input from mice. Horribly traumatic, wasn't it?

-Don

Re:Touchscreen keyboards

[murphyslawyer]

The difference is you could *totally* take down a Gibson with one of these puppies.

Re:Touchscreen keyboards

[Qzukk]

I'd go with "written" though there might be a few that exist that can break through the concept of having a single pointer, which is what this kind of touchscreen amounts to. Someone might make a mouse driver that uses the multiple points of contact to emulate a mouse (might even be able to make some operations easier than a normal touchpad, like drag and drop) but that wouldn't really use the full power of something like this.

It might be interesting to try and get some OS/windowing system to accept multiple pointers, and then write a driver for this that emulated N mice at once, where N = number of fingers currently on the screen. Moving a finger would attempt to figure out which mouse was released, and moving that mouse to the new location.

I can see plenty of uses for something like this in games or 3D visualization/manipulation software, but less use for it in word processors or the like (hmm... maybe hold down bold with one finger, while tapping the words you want bold with another?)

Re:Touchscreen keyboards

[fosterNutrition]

The fact that it accepts multiple input points is the key here. I'm really looking forward to the future when PCs are more like desks, with everything you need sitting right there on a giant touchscreen monitor (cf. the desk of that guy in the movie The Island). It would also be nifty to use this for digital music. No need to connect keyboards or anything like that, the monitor itself could replicate the functionality exactly, even touch-sensitiveness.

Re:Touchscreen keyboards

[Anonymous Coward]

Must.. not.. laugh.. at work.

Damn you!

Re:Touchscreen keyboards

[Bloke down the pub]

I guess that if there had been several mice, each with a separate, independant cursor, then it might well have been necessary to do so.

Re:Touchscreen keyboards

[Anonymous Coward]

Yep. How cute.

Re:Touchscreen keyboards

[rm69990]

So you're trying to tell me DOS applications ran with full mouse abilities on Windows 10 years ago? Drag+Drop and everything? Linux console applications in an Xterm?

Re:Touchscreen keyboards

[SimHacker]

No, I'm not making any such claim. I'm pointing out that maybe it's not such a bad idea to rewrite applications and redesign user interfaces to account for new input technologies at least every 10 years or so. I won't make any excuses for the fact that the Unix/Linux community is still using the same old command line shells after all these years, and still trying to play catch-up with Windows 3.1, nor will I attempt to claim that real men use the keyboard for everything so it's a good thing that Linux is so awkward to use with a mouse (the unspoken implication being that Linux isn't for women, which my mom would disagree with).

-Don

Re:Touchscreen keyboards

[StarkRG]

Well, yes, I suppose you could write a program to except multiple inputs, but then that would kind of defeat the purpose. It'd brobably be better to write them to accept the inputs...

Re:Touchscreen keyboards

[GuyWithLag]

I've been only using the command line for the last twelve years, and in that time it has evolved more than you credit it. The problem with the commandline is that it isn't self-documenting, while a GUI usually is, and that is what gives the initial impression...

The CLI, while linear, is a very expressive interface and it is quite mature, and Vista's commandline is playing catch-up with Unix in that regard.

Re:Touchscreen keyboards

[rm69990]

Wasn't replying to you. Go to my original post and click "Parent". He was modded down, which is why you aren't seeing him.

Re:Touchscreen keyboards

[jacksonj04]

I want the Minority Report style wall. I did a user interface experiment using the virtual whiteboards at my college: Which was more intuitive for arranging data. Up/Down buttons in a list, drag and drop with mouse, or drag and drop on a 'real' surface. Guess which won?

Re:Touchscreen keyboards

[SimHacker]

The TOPS-20 command line was certainly self-documenting, with full auto completion. When the Arpanet only had 8 bits of address space, it even had auto-complation over all host names, so you could go:

@tel[esc]NET ?[lists out all telnet options and parameters] mit-?[lists out all matching hosts]mul[esc]TICS

I knew somebody on a TOPS-20 system with a long hard to spell Polish last name, whose nick name in real life was pyz[escape]. You could just type "finger pyz[escape]" and the finger command completed with user names, telnet completed with host names, del completed with file names, etc. All commands had built in documentation and completion, integrated with the shell. Many years ago.

The problem is that Unix was such a huge leap backwards in terms of usability, that these incremental hacks and kludges you call improvements still haven't gotten us back to where other operating systems were several decades ago.

-Don

Escape Recognition, Noise Words, Help

One of the most favored features among TOPS-20 users, and one most identified with TOPS-20 itself, is "escape recognition". With this, the user can often get the system to, in effect, type most of a command or symbolic name. The feature is more easily used than described; nonetheless, a brief description follows to aid in understanding the development of it.

A Brief Description of Recognition and Help

Typing the escape key says to the system, "if you know what I mean from what I've typed up to this point, type whatever comes next just as if I had typed it". What is displayed on the screen or typescript looks just as if the user typed it, but of course, the system types it much faster. For example, if the user types DIR and escape, the system will continue the line to make it read DIRECTORY.

TOPS-20 also accepts just the abbreviation DIR (without escape), and the expert user who wants to enter the command in abbreviated form can do so without delay. For the novice user, typing escape serves several purposes:

* Confirms that the input entered up to that point is legal. Conversely, if the user had made an error, he finds out about it immediately rather than after investing the additional and ultimately wasted effort to type the rest of the command.

* Confirms for the user that what the system now understands is (or isn't) what the user means. For example, if the user types DEL, the system completes the word DELETE. If the user had been thinking of a command DELAY, he would know immediately that the system had not understood what he meant.

* Typing escape also makes the system respond with any "noise" words that may be part of the command. A noise word is not syntactically or semantically necessary for the command but serves to make it more readable for the user and to suggest what follows. Typing DIR and escape actually causes the display to show:

DIRECTORY (OF FILE)

This prompts the user that files are being dealt with in this command, and that a file may be given as the next input. In a command with several parameters, this kind of interaction may take place several times. It has been clearly shown in this and other environments that frequent interaction and feedback such as this is of great benefit in giving the user confidence that he is going down the right path and that the computer is not waiting to spring some terrible trap if he says something wrong. While it may take somewhat longer to enter a command this way than if it were entered by an expert using the shortest abbreviations, that cost is small compared to the penalty of entering a wrong command. A wrong command means at least that the time spent typing the command line has been wasted. If it results in some erroneous action (as opposed to no action) being taken, the cost may be much greater.

This is a key underlying reason that the TOPS-20 interface is perceived as friendly: it significantly reduces the number o

Re:Touchscreen keyboards

[emmaussmith]

The only reason it looks cool is because we saw it on ST:TNG.

Re:Touchscreen keyboards

[sabernet]

LOL

Re:Touchscreen keyboards

[The Mad Debugger]

No, it was on TNG because it looked cool, not the other way around.

Re:Touchscreen keyboards

[KermitJunior]

That's funny. No excuses need to be made since a lot of linux users today don't touch the command line.

But you're totally right... I hate having an OS that doesn't crash on a regular basis. I mean, Linux has a long way to go to catch up to Windows. Next thing you know, it'll up and give me choices on what to install for different hardware. I hate having the same programs on my 300MHz, my 1GHz and my 4GHz machines. Such consistency is painful, I tell you... pain.full.

THat's cool

[suso]

But can you transfer warp power through the nacels with it.

It reminds me of the IOBrush.

Re:THat's cool

[Anonymous Coward]

Bah. Screw L-Cars. This brings me one step closer to getting Dillinger's desk computer from Tron.

Re:THat's cool

[rolfwind]

It reminds me very much of Fingerworks technology with multi-touch technology with gestures of various sorts. More gestures here. On the surface, it looks like the exact same technology as the touchpads/keyboards but on the screen.

This company is now out of business (actually sold out VERY QUIETLY by Apple for the patents) and supposedly being used in the new iPod (I don't own one so I can't verify). It's really too bad, I always wanted their keyboard because it acts as a mouse too (on either side, plus has editable gestures plus a built in Emacs set plus a programming pad without moving your hand....)

Plus it has been suspected that Apple are using those patents now and applying for more patents for a Tablet that will have similiar capabilities.

Plus there's another company that has something for the music market that I find cool (but expensive).

Re:THat's cool

[iamlucky13]

My thoughts, as well. This looks just like something out of star trek, or minority report, or star wars, or a ton of other cool sci-fi show. Despite the relatively poor lighting in the video, that's one of the most fascinating technical demonstrations I've seen.

I was most impressed by the fingertip image manipulation, but the Worldwind navigation was pretty darn neat, too.

Looks similar to Apple's recent patents.

[Orrin Bloquy]

Which in turn look a lot like Apple recycling their iPod scrollwheel and Synaptics double-finger gestures.

Re:Looks similar to Apple's recent patents.

[quakeroatz]

They are "Apple's recent patents"!

RTFA

Apple

[Anonymous Coward]

Isn't this basically what Apple just tried to patent (multipoint touch displays)? Are they affiliated with Apple or any tech they may have bought? (No, I didn't RTFA).

Re:Apple

[giffnyc]

I was going to answer your question but noticed it could be answered in more depth by reading the second sentence of TFA. Btw, the article only contains two sentences. Shouldn't take long.

You also might consider reading the posts one or two inches above, should clicking the mouse prove overtaxing.

Thanks.

Sheesh.

Apple Patent

[MankyD]

Didn't we see Apple patent this sort of thing recently? Can anyone describe how this patent may or may not apply to the above demo?

organs

[Anonymous Coward]

I'd offer up organs for one of these things.

Ok, let's say two original Mini-Moogs in mint condition for one touch screen display?

Re:organs

[Anonymous Coward]

minimoogs aren't organs. they're not even polyphonic.

Re:organs

[tomhudson]

I'd offer up organs for one of these things.

I'm sure a lot of us would be happy to offer up CmdrTaco's organs for one of these things, too. Now where's that bathtub full of ice cubes :-)

Re:organs

[Philip K Dickhead]

I think I left them in my London hotel room - with the Turkish girl.

I need a kidney....

[MajorDick]

I bet you I could get a hold of one, whats your bloodtype ?

Just kidding but that is seriously cool, and I dont say that often.

I'd pay 2,500 for that Way before I would shell it out for a plasma TV....

Uber-cool

[TheBeansprout]

The satellite imagery & topographic maps are the user navigating NASA World Wind. Way cool.

Re:Uber-cool

[slavemowgli]

The funny thing about this is that SGI had things like this ten years ago already; I myself saw demos at the CeBIT fair several times. It's nice to see that this kind of thing is available for the average PC now, but it's also sad that SGI seems to be in its death throes at the same time...

The Exploratorium had an exhibit like that

[SimHacker]

The Exploratorium in San Francisco had a multi-point touch screen paint system like this in the early 90's, which anyone could play with. It was really great, and quite elegant! It was running a fun program that let you paint with your fingertips, real paintbrushes dipped in water, as well as textured objects like a sponge and play-dough. It used an oblique video camera behind a plate of glass, and your fingers or the wet brush changed the index of refraction in a way that would show up brightly on the camera, and thus paint on the screen. There was no limit to the number of points you could paint at once, and what you could use as a brush was only limited by your imagination and what you could get away with in public: you could paint with brushes, sponges, clay, your fingertips, the palms of both hands, your face, your tongue, your boobs, greasy french fries and hamburger patties, or vomit on the screen to make interesting textures. (It's a good thing the Exploratorium makes everything robust, kid-proof, and easy to clean! I've been to some great parties at that place...)

-Don

Re:The Exploratorium had an exhibit like that

[Anonymous Coward]

Imaging trying to hook that thing up to your 360. Oh, the possibilities!

Re:The Exploratorium had an exhibit like that

[Council]

robust, kid-proof, and easy to clean

Here's the thought that blew me away: someday the walls of all our homes will be that stuff, once it's really cheap. Or at least whole walls, somewhere, will be screens like that.

Imagine thinking "I want a poster of this picture I have, and I want it on THIS part of the wall" and reaching up and sliding your other posters to the side, right-clicking, and pulling it from the background, resizing it, and arranging it with your hand.

And then imagine thinking "I want my room to look like a Parisian coffeshop!" and just hitting a button. All your posters vanish and your room has windows overlooking the Eiffel tower (as all windows in Paris do).

And then you invite friends over, clear the back wall, and start watching TV. Though come to think of it, none of this really requires a touch-screen (though it's cool). So I guess this, though prompted by the touch-screen, would work for wall-sized LCD/plasma displays. So someone get me one of those plzthx.

Re:The Exploratorium had an exhibit like that

[deetsay]

I think you need good-looking 3D screens for that to work really well...

Re:The Exploratorium had an exhibit like that

[deetsay]

It used an oblique video camera behind a plate of glass, and your fingers or the wet brush changed the index of refraction in a way that would show up brightly on the camera, and thus paint on the screen.

Wooow, sounds like there's a cool DIY project in there for someone. Take one of those overlay-LCD's for overhead projectors, put a webcam behind it, put the whole thing inside an empty monitor case, then come up with a way to filter the touches somehow... Don't know how exactly, but maybe something like using only blue and green for the screen, and using red to detect touching...

Re:The Exploratorium had an exhibit like that

[Kazoo the Clown]

I saw the Exploratorium thing myself, and it was REALLY cool. I still want one. In fact, I've since built a rearscreen device that uses infrared reflection to accomplish multi-point interaction using a modified version of some of the techniques used in the Holowall.

Also working on a new version due to make an appearance in an art show in March. -- Sync

fingerworks

[Gyorg_Lavode]

It looks like a continuation of the technology employed by fingerworks which used some type of capacitance array to track points. It looks like they finally have it on a visual screen. Hopefully this will increase the addoption of gesture-based controls.

Re:fingerworks

[Ford Prefect]

Entirely different - it's based on something called 'frustrated internal reflection'. Simple version: you have a thin slab of transparent perspex with LEDs around the sides. The light from the LEDs is kept inside thanks to total internal reflection - it's a bit like a big flat piece of fibre optic cable in a sense.

When you place a finger or other appendage on the upper surface of the perspex, the total internal reflection breaks down and the fingertip (or whatever) gets illuminated - you track this with a camera pointing upwards at the perspex. To get the computer display gubbins, you also have a video projector pointing at the perspex.

I'm not sure how amenable it is to miniaturisation, but since it's used in fingerprint readers (without the video display) it's probably not too bad - presumably you'd have to change the projector and camera to flat equivalents, of course...

(Something I noticed on the page last week - a reference to work on identifying which finger is touching the display. He's updated that sentence to "Wouldn't it also be nice to identify which finger (e.g. thumb, index, etc.) is associated with each contact?" - but I'd had a sudden vision of this thing using fingerprints as, well, unique finger identification tags. The guy behind it seems pretty big on computer vision, and is also working on stuff like a "new generation of CMOS imaging sensors that feature on-board signal processing functionality, we are experimenting with creating a 1000fps non-invasive eye-tracker for under $100" - maybe some custom hardware for tracking and zooming in on the glowing fingerprints and identifying the fingers from there?)

Re:fingerworks

[hasdikarlsam]

One of these selling on Ebay right now.
I wonder if I should get it... though it /is/ rather expensive. Well, what's a kidney between friends?

Re:fingerworks

[Gyorg_Lavode]

They keyboards themselves aren't very realistic. I had one. You spend as much time finding the home row keys after using it as a mouse or for a gesture as you would. Also, the loss of tactile feedback really matters.

On the other hand I would recommend the number pad/mouse pad fingerworks product. They aren't precise like "select 1 specific pixel" precise, but they are great for using gestures instead of digging around in a menu or doign some key combination repetitively.

Not all the Software

[fourtyfive]

They didnt write all their own software, they used NASA World Wind (http://worldwind.arc.nasa.gov/ as well (satellite / aerial imagery viewer).

One problem

[Anonymous Coward]

"I'd offer up organs for one of these things."

But then how would you touch it?

very cool indeed.

[canning]

Hmmmmmm, though I wonder what power consumption is like.

Damnit!!!

[CatsupBoy]

Now my favorite touch screen gag is obsoleted:

User: This touch screen is awesome, but how do I right click on items?
Me: Use your right hand

*user stares blankly*

Re:Damnit!!!

[Anonymous Coward]

Use a modifier key on the keyboard. This is ultimately less confusing and more convenient to new and experienced users alike.

Re:Damnit!!!

[Anonymous Coward]

What was that *whooosh* sound I just heard?

Oh, must have been the joke going over your head...

Re:Damnit!!!

[Anonymous Coward]

I just finally got good at chatting with one hand. :/

Re:Damnit!!!

[hotdiggitydawg]

...or ask a Mac fanboy to help you.

</runs away quickly>

Re:Damnit!!!

[Anonymous Coward]

Hahaha. Joke's on you, and (apparently) you don't even realize how stupid you are. Good job.

Vastly different than Touchscreen keyboards

[SimHacker]

What's special is that it can sense more than one point of contact at once. In fact not just "more than one" but "any number of" points of contact in parallel. It's a totally different ball game than standard touch screens. A typical touch screen only reports one X,Y position at a time (like a mouse), which is typically the average of the points of contact (depending on the pressure, and the type of touch screen of course).

-Don

Re:Vastly different than Touchscreen keyboards

[DigitlDud]

It's not really that special. I think just about every CS research lab in a University has one of these projects.

Re:Vastly different than Touchscreen keyboards

[onemorehour]

What's special is that it can sense more than one point of contact at once.

That's not actually special when you're talking about some keyboards. I am typing this right now on a Fingerworks Touchstream LP, which is based on this technology. To type a single letter, you make one contact on the touchpad. To move your mouse, you put down two fingers simultaneously and move them. To click and drag, you use three fingers. To scroll, four. It also understands five-finger combinations and tracks movements, processing them as interactive "gestures" that can be mapped to functionality like opening, closing, saving, zooming, etc. This company was sadly bought out by some third party (rumored to be Apple or Wacom), who took the technology but has not kept up the line of keyboards. Apple's recent announcement makes me believe that they may have been the buyer.

Re:Vastly different than Touchscreen keyboards

[rolfwind]

I've researched Fingerworks for a while (look at my other post on this thread) - I have to ask on the keyboard itself - after a while is it as fast and accurate as a regular keyboard or do your fingers drift and you get accidental stroke input?

Re:Vastly different than Touchscreen keyboards

[onemorehour]

I've owned mine for almost a year now, and I've found that while my typing speed is almost as good as it is on a normal keyboard, there are certain typos I make quite often. To some extent, I think you can't use this keyboard if your primary concern is speed. My primary concern was carpal tunnel, and it helps with that tremendously. I barely have to move my hands around at all, and I certainly never have to do any strange, hand-stretching movements in order to reach common keys like backspace. Modifiers are also easier--you just make a pattern with four fingers on one hand while typing the modified key with the other. It's surprising how quick this is to learn instinctively. Also, for what it's worth, I can go back and forth between this keyboard and a normal one without thinking about it.

It's a damn shame about the line going under though... I just hope this one I have lasts for a long long time.

Re:Vastly different than Touchscreen keyboards

[Gyorg_Lavode]

You can learn to type just as easily on the fingerworks as you can on a normal keyboard. When it comes down to it though, you will severely miss the tactile reaction. And the moving of your hands to/from the home row keys will be as disruptive as moving to a mouse. I would much more recommend one of their mouse/number pads to replace a mouse. They are very good for day-to-day computing, (working with windows, browsing etc). They are not good for playing games, (at least the keyboard portion wasn't), and for precision graphics, (just not accurate enough to click 1 specific pixel).

Re:Vastly different than Touchscreen keyboards

[SimHacker]

I mean special in the sense of comparing multi-point input to normal single-point touch screens. Some people who haven't read the article are asking questions like "There have been touchscreen keyboards for quite some time now... So what's so special about this?" I am certainly aware of previous multi-point input systems. (See the excerpt about Myron Krueger's Videpplace I posted from Jakok Nielson's CHI'88 trip report, and the description of the Exploratorium exhibit from the early 90's).

-Don

Re:Vastly different than Touchscreen keyboards

[Anonymous Coward]

Why is this one special?
I've owned a multiple input touchscreen for some months now called the Lemur.
http://www.cycling74.com/products/lemur

The Lemur *is* special, as not only do you get multiple inputs, you also get them fast enough to perform with, and loads of presets for music apps.

Re:Vastly different than Touchscreen keyboards

[SimHacker]

Wow, I just saw one of thoes for the first time last week. They're really cool. A friend of mine was folding one of them up and taking it between work and home with him because he liked it so much, and he only had one because you can't get them any more. He said they were going for about $500 on eBay. I would say that's pretty special.

Re:Vastly different than Touchscreen keyboards

[mallardtheduck]

Nothing special.
I saw a video of the diamondtouch (http://www.merl.com/projects/DiamondTouch/) system in my HCI lecture today.
Diamondtouch has been around since 2001.

Re:Vastly different than Touchscreen keyboards

[SimHacker]

Multi-point gesture interfaces go back a whole lot further than 2001. What's so special and original about Diamond Touch? Other than the obvious advantage s of being built out of modern technology, how does it compare with Myron Krueger's work, which goes back to 1969?

-Don

Re:Vastly different than Touchscreen keyboards

[Anonymous Coward]

...at least every 'CSI' lab, from the looks of the TV show...

Re:Vastly different than Touchscreen keyboards

[JourneyExpertApe]

Just having a list of all the points that are being touched at a given time isn't all that useful. To be useful, it would have to track them as they move. To see an example of this problem, tap two places on opposite sides of a laptop's touchpad with two different fingers in close succession. If you do it right, you will see the mouse cursor jump across the screen. That's because it thinks your finger is moving across the pad really fast. Now imagine having ten fingers on the touchpad at once and ten cursors tracking them. The cursors would jump all over the place.

Re:Vastly different than Touchscreen keyboards

[jzeejunk]

If five of your fingers are on a touchscreen, how could one use EMACS?

Re:Vastly different than Touchscreen keyboards

[David Rolfe]

and for precision graphics, (just not accurate enough to click 1 specific pixel

But it's so fast/easy to zoom in and out with gestures, all the pixel pushers would never have a problem. Maybe what you meant is that the DPI and pressure sensitivity doesn't surpass a dedicated tablet.

People who draw with a mouse are missing the point. Is that a pun? They surely aren't interested in "precision graphics" (if we agree on what that means) anyhow.

Re:Vastly different than Touchscreen keyboards

[VaderPi]

I am also typing on a TouchStream. This technology was really not around long enough for really good applications to be developed with it, but all the things that are in the video would be possible with the API that FingerWorks shipped with the keyboard.

An example of the power of the API is a sample application, XWinder, that extends the gesture language of the keyboard to allow direct window manipulation. While running XWinder, I can put down three fingers on each surface, and then move my hands around to resize a window. There is another gesture that allows you to move windows around using only one hand.

I remember being impressed with the window manipulation in the movie Minority Report, but now I can actually do that with my Touch Stream.

Oh, and the TouchStream is able to tell each finger from one another as well. That is something that the researchers mention that they wish they could do.

Re:Vastly different than Touchscreen keyboards

[Damien Conlon]

I'm wondering how 'any number of points' are sent to the software, and what it does with them.

Did anyone else get an idea for a DoS attack on one of these? :>

Re:Vastly different than Touchscreen keyboards

[SimHacker]

The way "any number of points" are sent to the software is through a video image. I'm sorry, "any number of points" is an exageration: let's just say 640x480=312,000 points per frame. It's not rocket science (any more).

-Don

May I suggest?

[Anti Frozt]

"I'd offer up organs for one of these things."

This being /., we all know which organ should be first to go, seeing as how it's the least used.

Re:May I suggest?

[Anonymous Coward]

This being /., we all know which organ should be first to go, seeing as how it's the least used.

Brain?

Re:May I suggest?

[LithiumX]

[insert insanely witty riposte regarding the awesome might of my hardware here]

Re:May I suggest?

[Keebler71]

Offer up your hands... that monitor ain't so great now is it?

Re:May I suggest?

[AhtirTano]

This being /., we all know which organ should be first to go, seeing as how it's the least used.

You're right, the appendix should be the first to go.

Re:May I suggest?

[noidentity]

Just don't offer up both your hands and/or arms!

Re:May I suggest?

[Anonymous Coward]

The brain?

Re:May I suggest?

[mojotoad]

This being /., we all know which organ should be first to go, seeing as how it's the least used.

You mean his left hand?

;)
Matt

Re:May I suggest?

[slavemowgli]

This being /., we all know which organ should be first to go, seeing as how it's the least used.

Considering this *is* Slashdot, it'd probably have to be the brain. :)

Re:May I suggest?

[Queer Boy]

You mean this one?

Re:May I suggest?

[utexaspunk]

That nobody ELSE uses it does not imply that it does not get used. I think it's safe to say that the organ which you imply gets used quite often by most /.ers...

Re:May I suggest?

[ConceptJunkie]

This being /., we all know which organ should be first to go, seeing as how it's the least used.

The brain?

Re:May I suggest?

[Anonymous Coward]

Not the least used - but the least touched by a "member" of the the opposite sex.

Re:May I suggest?

[Anonymous Coward]

> Brain?

The brain isn't an organ. It's a brain. Kidneys, and livers, those are organs. Bonehead.

Re:May I suggest?

[blues_shuffle]

Don't talk about my liver that way!

Re:May I suggest?

[Belseth]

This being /., we all know which organ should be first to go, seeing as how it's the least used.

Define used?

Re:May I suggest?

[Anonymous Coward]

Wait, we're talking about the brain, right?

Who wants one?

[Anonymous Coward]

Really? Organs eh? I'm in the process of building two of these for myself. The restrictions on FTIR are that you need lots of space behind the screen, for the cameras and the projector.

Benefits vs cost

[zebadee]

I appreciate that touch screens are faster to use in some situations compared to a mouse, and in some situations (public access terminals in a cinema etc.) they are better but for the average consumer are touch screens necessary. Most people out there have been brought up with the mouse and are very adapt at using it. Other than the coolness factor (akin to having the fastest graphics card available to play doom3 at 200fps) is there a real market/need for touch screens for general consumers? Especially comparing the price of a mouse/LCD monitor vs touch screen?

Re:Benefits vs cost

[LithiumX]

While the cost of this sort of hardware will be prohibitive for large-scale use, I fully believe that it's just this sort of interface that will someday replace the mouse. Keyboards are likely to remain in use for a very long time, but mice are simply a pointing device... and we all come with a natural built-in pointer (our fingers).

In the comming decades, I'd expect people's "monitors" to be replaced with hardware similar to this drafting-table design.

Re:Benefits vs cost

[SimHacker]

What's so hard about using touch screens for the average consumer? They had to learn how to use mice, and it didn't kill them. I don't know of many ATMs that use mice, but a whole lot of them use touch screens, and they seem to be pretty popular with consumers.

But the point of this article that some people seem to be missing, is that the device is much more advanced than a typical touch screen, because it can sense multiple points of contact at once. Which is an advantage for people who have more than one finger. Why only use 1/10th of the fingers you were born with?

-Don

Re:Benefits vs cost

[heinousjay]

The number of ludditic comments on this technology website never cease to amaze me. To hear it from you guys, I would think we should all be happy with punch cards and blinkenlights.

Re:Benefits vs cost

[Anonymous Coward]

I have always had the opinion that the way we interact with information systems, by means of a physical pointer device and/or single point of interaction is counter-intuitive and cumbersome. Also, the confines of a single scale of visual representation makes utilization of display real-estate highly infefficient. By getting multiple points of interaction direcly onto the display, and having the possibillity to effortlessly move and scale information on the workspace, the user interaction reaches a higher level.

This demo does not show a standard touch screen. This may form the foundation for a more efficient and natural way of interacting with informations systems. This represents progress, not reuse of technology.

Regarding the Apple patent relevance, I cannot see much of a connection. Apple has patented gestures, not multiple points of interaction. Multiple points of interaction, and the user interface demo performed in the demo has huge amounts of prior art associated (ref. Minority Report, the Exploratorium installation, Wacom tablets and so on..)!

Re:Benefits vs cost

[Bloke down the pub]

What's so hard about using touch screens for the average consumer?

The design of their arms is all wrong.

Re:Benefits vs cost

[eltonito]

The initial market for the average consumer might be fairly small, but there are immediate applications for this product in the retail sector, particulaly quick service restaurants.

A major yardstick of QSR's is speed of service - a multi-point touch screen might save seconds on the time it takes to enter and complete an order, which is worth gold to most restaurants.

Re:Benefits vs cost

[jacksonj04]

The keyboard concept will stay the same, but it will likely become integrated into a screen like this. I don't mind having one normal monitor in front of me to view work on, and one of these touch video screen things on my desk (Or even in my desk) to interface with. If I could drag work from one to the other, so much the better. Move my keyboard out of the way, get to a graphics tablet, then a piano keyboard etc. all on one screen?

Re:Benefits vs cost

[LithiumX]

I don't know about virtual keyboards. There have been many touch-screen style keyboards, and I have used membrane keyboards that were sensitive enough to practically be touch screen. I have never liked using a single one. I like the tactile feel of a full-sized keyboard. I actually even miss the Click you got from the older ones. I don't need anything heavy, but I like the curved spring-loaded keys, and it makes it far easier to stay aligned when you have tactile feedback.

Virtual keyboards might see use, but I think physical keyboards will remain the primary preference - one reason they haven't really changed much over the decades.

Re:Benefits vs cost

[feepness]

Keyboards are likely to remain in use for a very long time, but mice are simply a pointing device... and we all come with a natural built-in pointer (our fingers).

Yes, but pointing with this device requires the arm and shoulder.

I recently switched to a trackball. Now I truly only use my finger (and love it).

I don't think it is ultimately entirely practical. Though it would be neat if you simply had access to that as an additional possibility.

Re:Benefits vs cost

[dr.badass]

Most people out there have been brought up with the mouse and are very adapt at using it.

Why would you assume that a touch screen would replace the mouse? We've had touch screens, touch pads, pen tablets, mouses, scroll-wheels, joysticks, keyboards, accelerometers, and plenty of other input devices for years, and they all coexist. Some are just more useful for certain kinds of work.

Re:Benefits vs cost

[jafuser]

This is neat, but why can't we just have two mouse pointers *right now* using two mice?

Imagine what possibilities that alone would open up.

Man this sucks...

[Afecks]

...thanks for ruining the mouse and keyboard for me!

Re:Man this sucks...

[Anonymous Coward]

haha offtopic?!

oh well at least i get the joke

Go to the NYU page

[Hoffy97]

The NYU guy put the video up on Coral Cache, so just go to the main project page to get better response. The author should have listed THAT as the primary link anyway. That tech blog has nothing to do with the project, and that site is Slashdotted now anyway.

http://mrl.nyu.edu/~jhan/ftirtouch/

This has nothing to do with Apple specifically, but after watching this demo, it has a lot of people thinking that way.

A deal...

[Call Me Black Cloud]

What if someone offered you one of these for one of your hands?

Me too

[thefirelane]

I'd offer up organs for one of these things

Me too, just not mine.

Ba-Bing!

Wow

[Tom]

That's an incredible technology. If it works as demonstrated, I can see it replacing the mouse. If we can get useful keyboards in there (sorry, software-based on-screen keyboards suck, they lack tactile feedback) as well, this could open up a whole new way in which to interact.

See, a lot of buttons on the mouse and on the screen are merely to differentiate between different actions, e.g. resize, fullscreen or close a window. More logical and intuitive options are possible with multitouch technology, e.g. as shown in the demos.

Re:Wow

[amliebsch]

Good point about the tactile feedback. I think the ultimate iteration of this device would incorporate some kind of magnetic or piezoelectric layer in a pixel grid, so that arbitrary pixels can be made to pulse. Pulse rapidly to create vibrations. This would enable tactile keyboard response, button clicking response, "dragging" response - all kinds of interesting tactile feedback.

Re:Wow

[nschubach]

Silly, me...
I was thinking along the lines of fine wires with alternate charges set to shock you when you hit the wrong (or right) spot on the screen.

Re:Wow

[macsox]

actually -- a company is introducing haptic touchscreens that, IIRC, use side-to-side motion to fool your fingers into thinking you're pushing into the screen.

Re:Wow

[Tom]

That's an interesting technology, I hadn't heard of that before.

Yes, absolutely. Combine those and you have a winner. I want my Apple TouchBook Pro! :-)

Re:Wow

[admactanium]

That's an incredible technology. If it works as demonstrated, I can see it replacing the mouse. If we can get useful keyboards in there (sorry, software-based on-screen keyboards suck, they lack tactile feedback) as well, this could open up a whole new way in which to interact.

personally i can't see it completely replacing a mouse for general computing. but for certain functions i can see it being a great addition. the ergonomics of using one of those things is actually not so great when you think about using it all day like we use our mouse(s). in order for the touchscreen to be convenient and useable by our hands our head would have to be pointing down the whole time to see what we're doing. but if we arrange it so that it's in a comfortable viewing position like a monitor, it's inconvenient and tiring to use with our hands. the only way i could see it working over the long term is if it mirrors what's happening on your main screen and you only look down at it occassionally. for example, i get neck pains when i have to type and view things on my laptop for a long time. it probably doesn't help that i'm fairly tall.

i'm looking at my computer set up right now. i have two lcd's mounted on vesa mounts to raise them up closer to my eyeline. i rarely, if ever, look down at my input devices. there's no need and it's not particularly comfortable to keep my head in that position anyway.

i can see it being really good for a digital device. mostly because we're already accustomed to staring down, up, sideways at it. i use a tablet regularly for photo retouching. but i didn't buy one of them that has and lcd behind the touchpad. while it may be more intuitive and exact in some situations, i just didn't want to have to stare down into the tablet surface all the time. plus it's way too expensive.

Re:Wow

[queazocotal]

On-screen keyboards could work well.

I'd imagine room temperature superconductor plates on the fingertips of gloves.

These are repelled by an array of electromagnets in the screen, to give tactility.

Not tomorrow.

Re:Wow

[rudolfel]

That's an incredible technology

Too bad it doesn't work in real world

Lemur!

[MaestroSartori]

It's like a bigger fancier version of Jazzmutant's Lemur device, used for controlling virtual synth plugins and the like. It even uses the same OSC protocol, I wonder if they're based on similar multi-touch tech...

OK, Show of Hands..

[ReidMaynard]

Who *likes* oily finger marks on their screen...

Re:OK, Show of Hands..

[gcranston]

Seeing as I just had to wipe the pizza oil off my fingers to type this reply... you've got quite a good point there. On antoher note, I don't think organs will cut it for this thing. I think we're talking first born son here. I mean come on.. it's 2 steps away from the display in minority report: 1. Immersive 3d 2. Wicked awesome 3 finger glove deallies. And I totally want one.

Re:OK, Show of Hands..

[Basehart]

No problem. Just pick up a pack of Apple iGlove's.

Re:OK, Show of Hands..

[Anonymous Coward]

Nobody. The logical progression of this device is a touchpad (about the size of your mousepad), separate from the main "viewing" monitor. Like those horrible old ones with the stylus, except it actually works.

Re:OK, Show of Hands..

[Hurricane78]

as if there weren't already tons of coating techniques against this...
they are just not boult in every cheap product. but so is this touchsceen. ;)

btw: I NEEED this thing RIGHT NOW!

YouTube Mirror

[rasjani]

http://www.youtube.com/?v=iVI6xw9Zph8

One word...

[Ostien]

Energize

Give your organs? How about 2190.00 EUR?

[dockthepod]

There's already a device like this on the market called the lemur. It's not as big and grand, but you can buy it now and there's no need for surgery. http://www.jazzmutant.com/lemur_overview.php

Offer up organs you say?

[agent_blue]

Would you give your your hands?

Reminds me of an idea...

[eno2001]

...I had back in the late 80s. I wanted to set up a dual layer LCD that would show you 3D images as generated by a computer. But it would be a transparent LCD panel mounted on top of a box that had IR grids scanning the X, Y and Z axes. As you would move your hands, you could shape and mould "virtual clay" with my software. Sadly I never got too far with it. This is kind of like a 2D version. The main point being that you should NEVER have to use an input device where interaction with the displayed object is too far abstracted from your hands interacting with normal objects. To be honest, this kind of technology SHOULD have happened around 1990. WE were already "there" so to speak at that point. Well those of us in the Amiga, Mac and even the Atari computer circles...

Lemur++?

[Jeremi]

FWIW, you can buy something like this right now. The Lemur is a touch screen that supports multiple touch-points at once, and communicates over Ethernet via OpenSoundControl. I have one on my desk at work, and it works well -- e.g. I can use 5 fingers to drag 5 different balls around the Lemur's touch-screen simultaneously, and see my actions mirrored instantaneously on the software on my PC.

Re:Lemur++?

[crovira]

Tell us more.

I am fascinated by the use of this. The resolution sesme to be limited to 600x800. It might seem like enough, but it isn't though unless I could scroll a reactive 'landscape' under it.

The project at http://mrl.nyu.edu/~jhan/ftirtouch/ is just a reactive screen with .1" resolution. It uses any image that can be 'back projected' onto its surface.

Quite apart from the "Minority Report" flat-screen object/relationship presentation level aspect, it ccould be combined with another project at the Media Research Lab of New York University which traces 'social networks,' and labelling the 'rays' between objects could be used to support multiple Relationships/connections between object instances.

That would make navigation of a complex database a great deal easier, either at the schema level, with the addition/deletion of Objects and Relationships, or at the instance level with the creation/deletion;connection/disconnection of instances.

Re:Lemur++?

[Jeremi]

Tell us more.

There isn't too much more to say... the Lemur is marketed primarily as a customizable controller for audio applications, and so it's designed for very low latency (i.e. you could use it as a virtual drum pad, etc). It comes with a program called JazzEditor that lets you drag-and-drop any combination of six or seven various widgets (1D and 2D sliders, text displays, etc) into a work area in the editor window, and matching widgets show up on the touch-screen. You can then set various properties of the "physics" of the widgets, to give them more or less friction, bounce, etc. It also lets you type in mathematical equations to further control things, but I haven't played around with that very much. Finally, it lets you set up a few dozen different "virtual screens" (analagous to virtual desktops under Linux) and then use the buttons at the top of the Lemur unit to switch quickly back and forth between them, or you can press the menu button (also at the top of the unit) to bring up a screen showing all the programmed screens at once (in miniature) and you can touch the one you want to bring up.

The built-in OpenSoundControl support is quite good; it gives you full bi-directional control over the unit (i.e. your app can receive UDP packets whenever the user touches a widget, and your app can also send UDP packets to the unit to tell it to move a widget automatically)

I agree it would be a little small for general computer use, but as a customizable controller / input device, it works well. I think the price is around $2600US.

Re:Lemur++?

[Anonymous Coward]

I was ready to jump ship to the mac and nab one of their (hopefully) future table macs, just to escape DRM issues and to get that awesome multipoint touch screen. The Lemur is how ever an awsome alternative, and a simple nework based KVM replacement all in one. If they come out with a Wi-Fi version of the Lemur, it will be my ultimate remote for a usable Media PC.

Now if there was just a Windows compatable anti-drm hardware device (eye-eye maybe), all my problems would be solved.

Oddly enough I as well am

-Lemur

Re:Lemur++?

[Yvanhoe]

But I wonder : does it work well as a screen ? From there website it seems that it is okay to draw GUIs but does it have more than 16 colors and a 10 Hz frequency ? I am really interested ...

Re:Lemur++?

[Kancept]

Nice Jeremy, so then you know what I use in BeShare with my Touchstream, since the Lenur was designed with Fingerworks liscensed technology. :-) -Kancept

Re:Lemur++?

[Fear the Clam]

If only I got paid to finger my balls all day!

Re:Lemur++?

[Jeremi]

From there website it seems that it is okay to draw GUIs but does it have more than 16 colors and a 10 Hz frequency ? I am really interested ...

I'm not sure about the number of colors it supports, but the update frequency is definitely better than 10Hz. Probably around 30Hz at least (that's by eye, I don't have any way to really measure it)

I think the main obstacle to using it for arbitrary GUIs would be figuring out how to upload your own firmware to it, since AFAIK that isn't documented anywhere. But I'm sure it's possible and that someone will have Linux running on it eventually :^)

on digg last week - and not an actual display

[xirtam_work]

Sorry to say this was on Digg last week which is why i thought it was either a dupe or old news. The display is a rear projection - not part of the input device. All looks very nice and the recent Apple patent s do appear to be based on this work, at least in part.

Re:on digg last week - and not an actual display

[Anonymous Coward]

Slashdot digs a dug digg article a week later. I dug it then, dig it?

I'd offer up organs for one of these things.

[AviLazar]

I'd offer up organs for a lot of things, this just happens to NOT be one of them. Maybe for a Lamborghini, a mansion, some super-model to love me and hold me and squeeze me - but yea, not a touch screen.
Now if we were talking true VR (think matrix) then yea they could have my organs.

multitouchreel.mpg

[NowakPL]

http://madmappers.freeearthfoundation.com/multitou chreel.mpg

For all you mad slashdot clickers :D

Oh no...

[Anonymous Coward]

What happens when someone watches porn on this thing....

Star Trek

[vertinox]

I'm surprised no one has mentioned this looks like Star Trek: TNG consoles.

Either way... I could really use something like this, but I bet it gets dirty really quick. My Nintendo DS is kind of greasy as it is.

Software more impressive than hardware...

[Anonymous Coward]

The demo is simply amazing and the future seems like a very cool place. But, in many ways, hardware like this really highlights the need for more inuitive SOFTWARE. The demo is not cool because you can wave your hands around on the screen. It is impressive because the computer in the demo seems to understand what the user wants to do. Whether you're using a single-button mouse or eight fingers, software that can anticipate a user's needs is what is really impressive. Programmers should understand this.

How it works

[Gogogoch]

So by following the actual links it seems to work by capturing an image of the plexiglass screen from behind. The screen is lit from the side, and each contact creates some scattering (via frustrated total internal reflectance) that can be picked up by the camera. Software then sorts out each contact (from background) and tracks them. So it is multi-touch up to the capability of the software, and the ability of the system to resolve one touch from another. I notice in the video they show in their website that these multiple touches dont get too close to each other. I wonder what the limitation is?

The display is then created by rear-projection, ie probably a regular video projector. I wonder how they separate the scattered touch image from the projected display? Perhaps one can be subtracted from the other?

Anyway, this means they have done a lot with a little. I imagine that the practical limitation is the physical dimension used to implement all these projectors and cameras.

Very nice demo though, particularly the software that their contributors wrote to show it off.

Re:How it works

[NowakPL]

I think the touch camera works in infrared

Re:How it works

[Gogogoch]

Brilliant - that would do it. Separation by wavelength.

Not practical

[DoofusOfDeath]

I had an orgasm just watching the video. Can you imagine how embarassing that would be if I used the thing 8 hours a day in an office???

Re:Not practical

[Gogogoch]

Using this for 8 hours a day? Imagine how tired your arms, your wrists would be! I hope it gets you off because you won't be up to doing anything yourself!! :-)

Re:Not practical

[Foerstner]

Using this for 8 hours a day? Imagine how tired your arms, your wrists would be!

Yeah, it's much healthier to hold your hands rigid in the same position, and use your fingers to perform the same short motions repeatedly for eight hours a day.

Luddites, go hide!

[TheSkepticalOptimist]

Anybody that can't see the benefits and cool factor of this need to go back to their caves and pull out some charcoal.

Someone said they can't see the average user wanting this? Did you see the video? I could see about a dozen areas that the average end user would wan this display for:

Multimedia organization( group photos quickly and in a more native concept)
Multimedia editing.
More robust UI interaction and quicker access. Believe it or not, the computer mouse is not intuitive compared to point and touch.
Video games, more interaction and unique game play possibilities.

Did you watch the video?

As a UI interface designer, I could easily see how some fairly complex interaction is handled quickly by being able to use multiple points of contact. Trying to duplicate the same interaction with mouse and keyboard is ancient and slow by comparison.

In the end, this is an interactive display that the average end users WANT. Get rid of the keyboard and mouse! This will allow computers to be setup as interactive displays on walls, or the coffee table or counter or desktop top allowing quick and easy access without cumbersome external interfaces forced on us.

Re:Luddites, go hide!

[Ford Prefect]

As a UI interface designer, I could easily see how some fairly complex interaction is handled quickly by being able to use multiple points of contact. Trying to duplicate the same interaction with mouse and keyboard is ancient and slow by comparison.

I'm not sure how far the physical technology used in the demonstration can be developed, but I do think it was aimed more at investigating new concepts in the software for user interfaces - things like the two-fingered scroll/zoom look so incredibly obvious when compared with scroll-bars, cursor keys, scroll-wheels and other abstractions...

Anyway. Do hurry up with this stuff. My typewriter-with-a-television computer is looking increasingly anachronistic. :-)

Re:Luddites, go hide!

[feepness]

You're missing the difficulty mentioned above... ie... our arms will tire quickly using something like this.

I agree it would be highly useful in ADDITION to current methods... but recently my arm/shoulder was killing me until I switched from a mouse to a trackball. I can't IMAGINE trying to use this thing for a 6 hour Battlefield marathon.

The other solution is not to play games for hours on end... naaaah...

I would love to give it to my wife to help her sort pictures though. Another major component most people have missed though is the fact that a relatively large screen was required for the "canvas". These are getting cheaper, but still expensive.

I didn't think this was for me

[sjonke]

until I read that it was based on Frustrated Total Internal Reflection. This baby *is* for me. Plus, butt mousing.

Why isn't Google?

[JeepingNET]

What? Why isn't google working on this!!

Minority Report

[tamnir]

After the initial "Oooooh, shiny! I'll give a kidney for one!" impulse, this reminds me quite a bit of the spiffy user interface in Minority Report, probably because of the intense arms-waving involved. So, makes me think the same too: very cool to see, but highly impractical. Your arms and shoulders would get painfully tired after just a few minutes using this...

So, I'll be keeping my kidney this time, thank you very much. I'll just go grab a box of tissues and watch the video again... ;-)

Re:Minority Report

[Queer Boy]

I'm sure when the mouse was first demonstrated they were very erratic with its movements. In practice, I'm sure most of the screens going to be used will be smaller that the one demonstrated as well.

Re:Minority Report

[Lumpy]

No way.

This could enalbe such things like the doctor's desk as seen in "The Island" that was out last summer.

I would kill for my entire desktop to be a LCD with the multiple touch capabilitiy like that input device.

Couple both of those together and that would be the biggest revolution in Computer -human interfacing that has ever happened in history. most of those demos did things that were how using computers should be right now.

unfortunately this will be patented and locked up so that we will not see widespread use to improve computing for another 30 years because some ass will want insane amounts of money to "license" the concept.

Re:Minority Report

[EnglishDude]

What about sign language? :) I can sign for hours on end and I don't get tired arms/shoulders/whatevers, though a few interpreters gets RSI from signing for too much.

Re:Minority Report

[Gulthek]

Painfully tired? From what essentially looks like drawing board arrangement and layout on an LCD? After just a few minutes?

I think you shoud like, work in a supply house or dock or something before you start talking about "painfully tired". Or heck, just wash a load of dishes by hand; does that make your arms painfully tired? (It shouldn't.)

Re:Minority Report

[everythingeverything]

The thing that causes me discomfort when using a computer for long periods is shoulder tension from suspending my arms and hands in place over my keyboard. This screen will free up your shoulders to move. Besides I think that if it takes a little more physical effort to use it's a good thing.

As long as you could freely adjust the angle and position of the screen I bet you wouldn't need as many micro-breaks.

Re:Minority Report

[tamnir]

IANAErgonomist, but I did suffer briefly from RSI symptoms and did my homework. Been paying close attention to ergonomics since then, and never had any more problem

Thank you for the sarcasm. But please, do try this test yourself: while using your mouse as usual, extend the other arm and point a finger to the screen, following the cursor around, and tapping on the screen every time you click. Now go ahead a browse the web like that for a while (it actually feels pretty cool at first, like in the movies, wooohooo :-) ). Note that when your hand rests on the mouse, it is ready for the next cursor movement, so that means you keep your finger up there, unless you do take your hand completely off the mouse. Please do report your findings here.

My guess is, unless you're an alien, you will find that the pointing arm becomes uncomfortable very soon. That's Ergonomics 101: it is not a matter of strength but a matter of of posture and leverage.

To counter your points directly: unless you're a masochist, you don't carry crates with your arms extended. And that counts also for the other comment in this thread about sign language.

Now, you may think you could reduce arm/shoulder strain by keeping your arms along your body, bent at the elbow. Then that puts the screen much closer to your face, way too close actually, leading to eye strain. And if you are thinking abou t putting this uber-touchscreen where your keyboard is, that is too low: you'd have to look down all the time, a source of neck pain. These eye and neck strains will take longer to set in than the arm/shoulder ones, but they will eventually.

Or, like in the video, you could be standing. Care to spend your whole day of work standing? I wonder why we bother with chairs in the office ;-)

So again, cool tech to look at and play with at a Computer show, but still, not practical.

Re:Minority Report

[tamnir]

The thing that causes me discomfort when using a computer for long periods is shoulder tension from suspending my arms and hands in place over my keyboard.

Ouch, sorry to hear that. Sounds like you need to lower your keyboard: in the rest position, there should be no strain in your arms or shoulders. If you feel you're raising your shoulders in the rest position, your keyboard/desk surface is too high.

Unfortunately, desk surface height is rarely adjustable. The trick then is to get a higher chair. Note that your legs shouldn't be hanging, so you may also need something under your feet to rest them on (for the correct height: your thighs should be horizontal).

Re:Minority Report

[TheNoxx]

Or, just maybe, it would help you to not only work faster, but get in shape for once.

Re:Minority Report

[Anonymous Coward]

How about at an angle, like a drafting desk? That would seem to make sense, and I don't think there are any problems with it ergonomically.

Re:Minority Report

[Anonymous Coward]

very cool to see, but highly impractical. Your arms and shoulders would get painfully tired after just a few minutes using this...

That's assuming you'd have your arms up in the air the whole time you're using it.

I think it's no great stretch to imagine that, with a sufficiently rich, high-level interface, we'll be spending more time communicating, reading, thinking, and generally doing things that don't correspond directly to "hands {in the air, on the mouse}".

At least, I like to think that when we've got systems as badass as in Minority Report, we're not using them to type "for (int i=0; i<n; i++)" 73 times a day still.

Professors (even wimpy ones) don't seem to get "painfully tired" after using a whiteboard for a few hours, and they *do* spend a lot of that time with their arms in the air. I don't see why a computer would be any worse, and if its interface was well-designed, it could be a lot better.

Painters don't seem to need especially strong arms to paint at easels, either. They seem to find that the benefit of working in front of your eyes outweighs having to lift your arms a little. (True, they have additional limitations, but they still lift their arms.)

I'll reserve judgement until I've had a chance to use a real one. After all, if you'd seen the original mouse, you probably would have declared that "impractical", too (because it was).

Re:Minority Report

[tamnir]

That's assuming you'd have your arms up in the air the whole time you're using it.

Indeed, you are correct, that was the assumption, in order to match the current average computer usage, where you hands, even in the rest position, are ready for input.

And I pretty much agree with the rest of your post too: with a different man-machine interface, such a device may be more practical. However, I don't share your optimism regarding such an interface. The windows/icons/menus/cursor metaphore was developed in the 70's. What has changed since then? Not much, really. Check out this Xerox Star Desktop picture. Not that different from the desktop I am looking at right now as I type these lines. And the date on that picture is 1980!

Re:Minority Report

[Anonymous Coward]

However, I don't share your optimism regarding such an interface. The windows/icons/menus/cursor metaphore was developed in the 70's. What has changed since then? Not much, really. Check out this Xerox Star Desktop picture.

I think quite a lot has changed. What used to be the domain of research or super-expensive computers can now be had for a song. So if you want to know what a typical cheap computer will look like a few years down the road, check out the current research and super-expensive stuff.

In 1980, this interface was only at Xerox. In 1985, it was on the Mac -- cheaper and available to anybody, but still fairly expensive. In 1995, it was on the PC -- available to pretty much everybody. Starting around 2000, we've seen some new ideas crop up to try to make the interface scale better (Expose, Spotlight, etc.). And that's a good sign that this interface design could well be nearing the end of its reign.

There are a lot of projects on the horizon. The most prominent is that of people trying to replicate all "desktop" functionality on the web, which arguably has a different interaction model than desktop apps. (The network will be important to whatever is to come, but I'm not sure the web is the best way to do it.) Apple is hiring engineers to work on tablet interfaces. Prices on tablets, touch-screens, and big LCDs have fallen (would your 1980 Xerox users have had a 30" flatscreen, for the price of a cheap used car?).

And again, if we look back to 1980, things didn't seem so hot then, either. GUIs like that were only available on super-expensive / research machines. The world was being taken over by 8086's with character graphics, and obscure keyboard commands. If you're pessimistic today, you must have been *really* pessimistic in 1980. :-)

Re:Minority Report

[Gulthek]

Points:

1) Why keep my finger on the screen unless I'm going to click or drag something?

2) Why keep my monitor perpendicular to my desk when I could just shove off the keyboard and mouse and put this device on it at an angle?

3) You say the keyboard is too low to look at, how did I do all that drafting (by hand) then?

4) If you really and truly want to keep your monitor in its place, then have a blank touchscreen at your keyboard with cursors that display on the monitor when you use it.

5) I worked retail. I *did* spend eight hours a day standing (well walking and talking mostly), on a carpet covered concrete floor no less. I did that for 1.5 years, no issues. My co-workers ranged from teenagers to mid-forties and all worked similarly (and longer than I had) with no problems.

I think this will replace the mouse

[gurps_npc]

After watching the video, I think this new touch screen seems to be a major improvement. I can see it replacing the mouse for laptops, then later desktops, unlike the current stuff that is nowhere near that good. Which really surprised me. Just the simple things like zooming that you can do with two touch points (move fingers together to zoom out, move fingers away to zoom in), really impressed me.

Re:I think this will replace the mouse

[Anonymous Coward]

Yeah, the sorting of pictures was quite impressive in it's improvement of speed.

And it is the little things that do wonders. I use Linux pretty much exclusively, and in KDE if you hold down alt when you click-drag the mouse, you move the window no matter where you've clicked in it. Similarly alt + right click resizes. I don't even thing about it, except when I go and use a Windows machine... missing those 2 little features, makes the whole experience of trying to organise windows very frustrating.

The future is here

[eebra82]

Soon, new software will allow nerds to graduate in nipple handling. It's very delicate and requires some hardcore training.

Me, too.

[butterwise]

Very cool. I'd offer up your organs for one of these, too.

Won't sell to one market

[Anonymous Coward]

This will have zero appeal to all the people who surf with one hand...

Ultimate? Er, I doubt it

[dlefavor]

Since "ultimate" means "last".

Sorry...pet peeve. Mod me down if you must.

Least Used Organ - Appendix?

[SeanDuggan]

Appendix?

Myron Krueger's Videodesk System

[SimHacker]

Here's a description of Myron Krueger's classic Videodesk system, from Jakob Nielson's CHI'88 Trip Report (in which he also described our presentation of pie menus).

-Don

Videodesk: Computing on the Desktop

Current marketing trends in the personal computer business emphasize "desktop this" and "desktop that" - desktop publishing, desktop presentations, desktop video, desktop CAD... as a catch phrase for doing things on small, desktop computers. It is also possible, however, to actually do computing on the desktop itself. This was demonstrated by Myron Krueger from the Artificial Reality Corporation in the Videodesk system: Videodesk consists of a large surface over which you move your arms, hands, and fingers. A video camera mounted over the desk picks up these movements and use them as input to the computer which then shows then as an outline on the display. This display is currently separate from the desktop surface but one might imagine that a future system would feel even more natural to the user by having the output display projected directly onto the input surface.

Several applications were shown. One of the most immediately understandable was a finger painting system where the color used was determined by the number of fingers shown. I asked Krueger why the system deposited the paint over the user's finger rather than under it which might have seemed more natural. His answer was that sometimes one would not want the hand to obscure the work being drawn.

The painting was cleared by spreading all fingers. Some of these gestures seemed very natural, including the clearing gesture. Gestures in other applications were not that obvious but still frequently very nice, such as having a straight line appear between two fingertips in a CAD-system. One problem they had in developing their gestural language was in parsing hand movements to determine when you just want to move your hand to another part of the screen and when you want to issue a command. In general, there seemed not to be much consistency in the interaction techniques used in the different parts of the system with the exception of the technique of reaching to the upper right corner of the screen to pull out the main menu.

Videodesk is really a special version of the older Videoplace system where the computer is an entire room which you enter to use your body as input device. As such, Videodesk was yet another example of the evolutionary trend at this CHI. The full Videoplace system was not available for the conference as it was installed as part of a large exhibit on Computers and Art at the IBM Building in New York. This was a very interesting exhibition which I had seen by accident before coming to Washington: I had originally jumped on the M2 bus to go uptown to the Metropolitan Museum when I looked out the window and saw a poster at the IBM Building for their special exhibition. Yet another advantage of not using a constrained "transport interface" like the subway: You can change your mind.

Finally!

[1 (smarterThanYou)]

A way to eliminate those annoying dual-touch errors in Erotic Photo Hunt.

It'll never fly.

[doublem]

Let me spell it out:

Major technological innovations in computers and the Internet have been driven by porn. Adoption rates are, among most early adopters, driven by that technology's ability to deliver porn. This is true of Broadband, the early graphics card races, DVD drives and the Internet itself.

This interface requires two hands.

Need I say more?

Don't make me to spell it out in anatomical detail.

Re:It'll never fly.

[glsunder]

Major technological innovations in computers and the Internet have been driven by porn

In soviet russia, touch screen touches you?

Re:It'll never fly.

[Anonymous Coward]

It does not REQUIRE two hands, it is TWO HAND CAPABLE. Watch the vid.

Re:It'll never fly.

[Anonymous Coward]

Cool, interactive dual nipple tweaking.
Nice.

Re:It'll never fly.

[happyslayer]

Yeah...but imagine a pr0n site designed around this one!!!

(Where can I get a third hand grafted on....?

One Question

[Anonymous Coward]

Where's my minority report!?!?!?

Fantastic, bar none...

[ursabear]

The demonstration of the technology is without a doubt the nicest technology demo I've seen in a decade (on top of the fact that the technology is great).

I don't like to predict, but feel like I must: My children's children will see this type of thing on a daily basis.

I've always wanted to be able to brainstorm in a free-form and extremely editable way, with both hands and all fingers - this technology would be intuitive to my design process. This beats even a touch-tablet by a mile.

I'd give my left hand

[mattnuzum]

I'd give my left hand for a two-handed touch screen. ;-)

Re:I'd give my left hand

[sumanjay]

Sure you would... you're not left-handed like some of us, you insensitive clod!

In other news

[Anonymous Coward]

Windex shares are up 15%

GUI Patents

[viewtouch]

I've said it many times before and I'll continue to say it - Patents on user interfaces and user input techniques are insidious and odious. The reason that this demo and the user interface techniques it shows even exists is because for a very long time you could not get a patent on how something is touched and what happens when you do touch it. Now that the USPTO has decided to begin patenting how you touch something and what happens when you do we can all expect the damage of user interface patents to be forthcoming and unlimited. I can only hope that all the prior art is used to overturn any recent patents and any attempts in the future to patent how we interact with technology.

One problem...

[corellon13]

The problem is that the consumers who will actually be interested AND willing to spend the money are the same people (including self) that sit at the computer all day eating cheetos, chips, , and drinking pop. I can only imagine how short of a life span these things will have given that kind of use. Hell, look how fast people in this profession burn through keyboards...

Two handed input!

[umask077]

What a useless invention. When im surfing my porn I only have one hand free.

Problems with touchscreens

[SeanDuggan]

Yeah, but then you'd have to deal with gorilla arm. Could you imagine spending 8 hours with your arms raised and moving within the area of your monitor? Besides which, I have my doubts that this kind of method would be any more beneficial than the current setup with mice and keyboards. The reason it works so well in movies is that they use CinemaOS, where hitting the space bar repeatedly lets you zoom in to a single pixel on a digital image, all passwords are "password", and "self destruct" is a command line option. Biggest problem with CinemaOS is that by default it renders in ugly neon-green 72-point font.

Re:Problems with touchscreens

[fosterNutrition]

Good points. It always did amaze me how their computers had such advanced AI, but I suppose it was just mere telepathy with the user that made them so prescient.

Minor clarification of little note: The computer I mentioned from The Island would actually not give gorilla arm. It was essentially a table, with the user's hands resting horizontally on the surface or downwards if used while standing. Basically it was like a PC-less standard office desk, only nothing on it was physical but rather images on the desk surface.

Don't be silly...

[ch424]

nobody wants to buy your appendix. ;)

new application

[demon4]

i can see people using this to play twister

    Right foot blue! (Right foot blue!)

Accenture's Interactive Wall

[timfoolery]

This isn't entirely unique as the other commenters ahve said. Another example is something called Interactive Wall by Accenture.

Accenture has developed and has been showcasing their Interactive Wall for a few years now. A detailed article entitled "Supporting Collaborative Touch Interaction with High Resolution Wall Displays" on how it works can be found here.

If you're really interested Accenture hosts "road shows" where you can take a peek at this an a lot of other things the Accenture Labs have developed.

new appication

[demon411]

i can see people using this to play twister

      Right foot blue! (Right foot green!)

Pay to see the paper? stupid..

[HeavyMS]

I love to be able to read the paper that they have done but that costs cash? I find that strange..

how the touches are measured

[dr_leviathan]

Here is a link to the technology used to detect the touches (I scarfed this link from a forum post on fingerfans.dreamhosters.com, thanks to eelfinnTy for posting it):

http://mrl.nyu.edu/~jhan/ftirsense/index.html

I think the touchscreen is backlit with a projector. The scattered light from the touches is probably infra-red and could be measured by an IR camera located back with the projector, which is how the device could be 'scalable to very large installations'.

Very cool, but the slimline version for a tablet PC is still a few years away.

Re:how the touches are measured

[dr_leviathan]

The multi-touch tablet is only a matter of time. Below is a link to a pdf on Cypress's capacitive switch array technology (think iPod wheel or other MP3 player sliders).

Capacitive Key Scan

All they need to do is make such array's transparent and they can turn a touchscreen into a keyboard or other interactive device.

You'd offer up organs?

[Anonymous Coward]

I'll keep my mouse, and my organs, thanks. This looks like way too much effort/work for me.
 
//typed as I am lying on the couch with my notebook.

Re:Ultimate? Er, I doubt it

[heinousjay]

There's this concept in the English language where one word can have multiple definitions. You'll have to expand your horizons a bit to understand that, but ultimately it will help you.

whats wrong with ye olde mouse wheel for zooming?

[__aapspi39]

Looks like it could be great fun after a bong hit or two, but in terms of interaction its nothing new, and probably of little use.

Star Trek, Here We Come!

[JohnG307]

One step closer to LCARS! ;)

Re:whats wrong with ye olde mouse wheel for zoomin

[allanw]

You can't really select what part of something you want to zoom in with the mouse wheel, just the whole thing.

Re:whats wrong with ye olde mouse wheel for zoomin

[__aapspi39]

in applications where i need to zoom in/out the position of the mouse on the screen determines which part of the screen is magnified

It seems to me...

[hummassa]

That you never had a conversation (especially a bar-talk) with Italian descendents. In my family, we move continuously (both) our hands to maintain a conversation, for twelve hours or more.

Offer up organs, eh?

[piotrr]

Would that include sacrificing both hands?

I was marveling about this some 2 monthes ago!

[Anonymous Coward]

I was thinking about cursor-less, off-screen touch screen some time ago.

Finally, i've got to think that somebody has already invented it ;)

My idea: [http://spaces.msn.com/esteewhy/blog/cns!6B8EE681D 2C79B18!407.entry?_c11_blogpart_blogpart=blogview& _c=blogpart#permalink%5D

"Entirely Different"

[RealGrouchy]

At first I thought this "Entirely different" comment was in reference to the Exploratorium comment above, and I was going to reply saying "actually, your description sounds like it's pretty much the same technology." Then I saw the collapsed "fingerworks" parent thread.

And now for my terrible joke:

It looks like this technology will require users who are either tall, have long arms, or both. They will have to develop midget widgets for the software.

Drivers

[dammy]

For Lemur (and other simuliar products), are there any open source support for them?

Dammy

Re:Drivers

[Anonymous Coward]

The editor app works in mac, windows, linux. (Not open source)
All the parameter communication is OSC (open sound control) which is well supported in the open source world.

The thing probably runs linux itself.

LCARS

[extremescholar]

So basically, this is LCARS?
http://en.wikipedia.org/wiki/LCARS

Interactive Whiteboards...

[mikael]

You should read up on
Interactive Whiteboards. It completely blew me away to see that many classrooms no longer had chalk and blackboards, but instead presented everything using either an Powerpoint presentation or an interactive application. The advantage of the whiteboard is that is eliminates time being wasted on preparing/cleaning the whiteboard, and the mess created by chalk.

Primary school teachers seem to have developed around 100 applications already.

Ooooh... no membrane!

[Mad-Bassist]

After working with a regular (and expensive) touch-screen monitor in a convenience store and seeing it slowly die after one year of constant use, it's nice to see a technology using a plain surface that could handle wear and tear in the real world. And then there's the multi-touch aspect. I hope Mr. Han is successful with this and we start seeing *nix drivers for those bad boys!

Re:Ooooh... no membrane!

[Anonymous Coward]

Actually, this prototype already works with *nix: it uses a dedicated machine that puts out all the information on a socket through UDP. The apps generally run on a different machine that talks to the socket.

Re:Apple / yes // more links

[Lord Satri]

This Vic tech blog do not share where they first got the video, but it's been out for at least a few days. And yes, it's related to Apple patents. Read more here: http://www.macrumors.com/pages/2006/02/20060212165 558.shtml and http://www.macrumors.com/pages/2006/02/20060211144 712.shtml

This link offers Apple patent application pictures:
http://guides.macrumors.com/Gallery_of_Gesture_Use r_Interface_Patent
Very interesting indeed...

Music?

[xtieburn]

Off topic. what was the music? Im hoping it wasnt made for the video seems like some decent ambience.

On topic. Puts my DS to shame.

Re:Music?

[Anonymous Coward]

it sounded like juno reactor to me but i'm not 100% sure. i could swear i have it on cd at home. i think it is also in one of the animatrix shorts. it definately sounds familiar.

Re:Music?

[Anonymous Coward]

According to someone from metafilter, it is "Who Am I?" by Peace Orchestra, from the Animatrix soundtrack.

Don't know if that's right, but hopefully it'll help :)

Re:Music?

[xtieburn]

Thats the one. I just checked the 30 second sample of it on Amazon.

Thanks for the responses.

It's a scam

[icandodat]

I saw a similar video years ago done as a magic trick. You simply record something like a screen saver or an animation then you memorize the sequence of movement and pretend like you're doing it with your fingers. Watch the video and you can see that this is not one long demo but several short ones spliced together. Each one is short enough for someone to memorize and practice. Very clever, but not science.

Re:It's a scam

[deglr6328]

you're stupid. HAND

Never Mind Productive Stuff

[jacksonj04]

Can you imagine gaming with this thing? Strategy where you can *really* manipulate units quickly. Solitare would become an elegant affair. Move those keys *exactly* where you want them, and use gestures for rapid weapon switches.

Re:Never Mind Productive Stuff

[feepness]

Solitare would become an elegant affair.

I hear they are even working on another prototype that actually uses solid pieces of pressed plant matter painted with ink... with get this... one "screen" representing each card!

And on top of that they are paper thin and capable of being moved and stacked in three complete dimensions!

Truly amazing.

Do not touch (Fingerprints)

[GreenPlastikMan]

The problem with most touchscreens is that the best material for a display panel is usually also quite reactive to the oils found on the skin of our fingertips. This leads to unsightly smudges and fingerprints on the screen. The way this could be avoided would be something along the lines in Minority Report. They would have to invent, for lack of better words, a cyber thimble.

That way, one could do the manipulation without touching the screen, but rather a few inches in front of the screen. That would also alleviate the problem of the "arm waving" that was mentioned in an earlier post. No-touch technology would allow the user to scale his motions by how close or how far they are from the screen. A motion further away would have a bigger impact, one that was closer would be more precise.

I wonder what the projected time for release to consumers would be...

WTFV

[hobbit]

...and if after watching the fine video you still don't get it... shoot yourself.

Star Trek control panels

[mknewman]

Reminds me a lot of the touch screens used on the bridge of Star Trek, especially Tne Next Generation. Multiple hand motions controlling a single function.

The Philips Entertaible was shown recently at CES

[golemite]

Check it out:
http://www.research.philips.com/initiatives/entert aible/index.html

Organ donation?

[Anonymous Coward]

I'd offer up organs for one of these things

Whose organs?

I think that is awesome

[nroose]

Of the demos, I thought the photo organization one was the best, with ability to enlarge using 2 fingers on one hand. Who knows what other great stuff could be done. Most of the demo was more like performance art than business productivity demo. I am guessing that there could be some great games, and that things like page layout will benefit from this.

It might

[coolnicks]

Two male hands, two breasts, two (or more) points .. imagen what you could do with porn on that! 3D interactive porn!

Class ;-)

[cheros]

That made me laugh, thnx ;-)

I'll never need more than 5 pointers at a time...

[GodfatherofSoul]

...since I'll only be working the touchscreen with one hand. Oops, make that six.

WTFV

[p3d0]

Go watch the video. You can't do that stuff with a mouse.

Two trackpoints on one keyboard

[SimHacker]

Ted Selker invented the "joy button" red keyboard cursor control thingie, and developed the "Trackpoint" at IBM's User Ergonomics Research Lab. (Anybody remember the "So Hot We Had to Make it Red" two page Thinkpad ad?)

At one of the New Paridigms for Using Computers conferences, he demonstrated a custom Thinkpad he'd modified to support two Trackpoints at once! It was an inexplicably attractive and approachable interface: operating the computer by tweaking two red nipples! Unfortunately the keyboard was not drool-proof.

He demonstrated another cool custom keyboard job with a piezoelectric buzzer under the Trackpoint, that gave tactile feedback as the cursor moved across textured surfaces and over edges.

He also made conference badges that clip onto a Trackpoint to measure physical motion and position -- it's so sensitive it can be used as a postage scale, wind speed sensor, seismograph and accelerometer.

Unfortunately none of those cool weird technologies made it into production. They require special APIs and deep modification of the desktop user interface and applications, in order to meaningfully exploit the special hardware. Take a look at the DirectX force feedback API that eventually came along, for example -- it's quite complex, and not many applications support it. Applications would have to know about the special hardware and go out of their way to support it, which just won't happen soon, because current user interfaces and applications are extremely inflexible, and have brittle, device-dependent user interfaces.

-Don

Cool ...(OT)

[smoker2]

It does look cool, and what's even more surprising, no one has complained about the movie being Flash !

I hope this is a "non-evil" use for flash, coz I payed for Swish Video2 recently and I don't want to alienate users.

I'd offer up organs for one of these things

[Anonymous Coward]

You could offer up your hands...

Application to Audio (Synth Apps)

[Gice]

Those of us that use Reason or similar Synth-type audio composition apps could conceivably be able to tweak multiple mixers / settings etc. at the same time without having to replay and re-record the song if doing it singly threaded (record,adjust,repeat) by mouse. It'll open up new effects we couldn't experiment with previously when recording in that way. *drool*... *more drool*. I'd pay thousands $ for one.

And the OS is...

[greginnj]

Drool... Somebody tell the Raskin Center about this -- there's finally a screen that will make Archy work. If only they dropped the 'infinite undo' requirement, they might actually manage to use this screen to release something more than a CLI proof-of-concept plus a flash demo.

Where no man has....

[GmAz]

Ok, combine that with http://slashdot.org/science/99/05/27/1215204.shtml Warp Engines and now you have something!

Does not require two hands

[Foerstner]

Don't make me to spell it out in anatomical detail.

I don't know about you, but I have five fingers on each of my hands. That gives me up to five points of contact for the screen (one of which is opposable to the other four) while leaving the other hand free.

LCARS at last!

[fbg111]

Sweet, for the next phase, let's hope they program a dynamically reconfiguring LCARS interface.

Re:LCARS at last!

[sinewalker]

hope so. But it also looks a lot like the MCP interface in Tron. I want one of those....

Make your own.

[Anonymous Coward]

If you'd like to keep your kidney (and have an ACM subscription) you can just Build one yourself.

Video Background Music

[savorymedia]

This is cool...but I'll be REALLY excited when I see a dual touchscreen laptop or tablet computer with this tech that I can buy.

What's killing my brain is the background music for that demo video. I *know* it's David Bowie...but I can't remember which song. It might have been bg music for Omikron (video game that DB was closely tied with) or something else.

Anyone have a clue?

The pancreas!

[MS-06FZ]

Definitely the pancreas. That thing never did anybody a bit of good.

Then again, I may be vastly underestimating the importance of the pancreas.

What artist?

[Pivot]

Does anyone know what music is accompanying that video?

Re:What artist?

[sinewalker]

Haven't heard it yet (slashdotted:-(). But, from TFA:

Music:
"Who am I?", by Peter Kruder, Peace Orchestra

Assistive AI

[hackwrench]

Assistive AI has certainly taken a leap beckwards in the Windows era. Computers should be autocompleting like mad. It boggles the mind how much productivity we've lost due to lack of automation that should be obvious implementations.

What do you mean I actually have to drag the mouse to the next control element!

Re:Assistive AI

[SimHacker]

Assistive AI was perfected years ago, at Roanoke AI Laboratories.

-Don

Revelation

[Yvanhoe]

I think I just saw the future...

Interesting URL

[pattokun]

I hope no-one who happens to check my browser history is too alarmed that I visited a site ending in the_world_of_sm.html.

Frustrated Internal Reflection?

[Tavor]

And here I thought "Frustrated Internal Reflection" was something different entirely... something that a lot of womanless Slashdotters do.

It won't replace the mouse.

[plover]

Even if the price were to drop to below that of a decent optical mouse, there are good non-technological reasons it won't replace them.

Fatigue and precision are the two heavy hitters.

We've done extensive ergonomic testing of touch screens vs. keyboards. The biggest problem with a touch screen is that it requires the user to raise their hand and hold it up without support. Resting your palm on the screen will cause false-positive touches. If you are physically standing above the screen, it requires you to keep your wrist bent backwards a bit. With a mouse, you can use a wrist-rest to keep the muscles in the arm and hand relaxed. With a touch screen, the user must keep them under tension. Not good for long durations, and it leads to repetitive strain injuries.

Precision is another big problem. Your fingertip covers a fairly sizeable area, and is not necessarily center-weighted by the detection technology. Where do you put the aim point? More importantly, where do you put the aimpoint so that the user can see it while your finger is obscuring the display? When the user keeps their fingernails longer, they can also interfere with aiming (although they can be used in lieu of a stylus!) But we found with testing that most people have a hard time hitting a "default sized" OK box with their fingertips. The standard we settled on for a touch panel was a minimum of half an inch on each side. But we found the larger we made the buttons, the faster they were able to make their desired selection without error. On a nine-inch screen we would put no more than about eight buttons high. It also helped greatly to give them a graphical target inside the touch area, even though we accepted a touch anywhere in the containing box.

Another precision issue we had (that this technology is not affected by) was calibration. Capacitive and resistive touch technologies both drift over time for environmental reasons such as temperature, humidity, and dirt. They require periodic recalibration, which is tough to get an employee-end-user to do and a periodic maintenance point for interactive kiosks.

Surprisingly to us, a dirty environment was a positive in favor of touch screens. With a touch screen, dirt can obscure the displayed data, so we initially thought it would be a bad thing. But you can just give the user a bottle of Windex and they'll figure out what to do. With a traditional ball-and-roller mouse, dirt means it will jam up quickly. The cords are also fairly fragile, and the mice themselves are susceptible to being knocked to the floor. Mice are extremely high-maintenance when compared to touch screens (thus far more expensive.) With a mouse, they need to call the help desk every time it doesn't work, and that means a costly service call.

Now, we made our hardware decision before the advent of cheap optical mice, so we never repeated the tests with an optical mouse. That may have changed completely.

So, we decided touch screens are great for occasional use. But they're not good for long-term data entry, and they really slow down the users for precision input.

Re:It won't replace the mouse.

[Bloke down the pub]

The biggest problem with a touch screen is that it requires the user to raise their hand and hold it up without support.

Seconded, I mentioned gorilla arm syndrome elsewhere.

With a traditional ball-and-roller mouse, dirt means it will jam up quickly. The cords are also fairly fragile, and the mice themselves are susceptible to being knocked to the floor.

Turn the mouse over and attach it flush to the front of the device so just the ball sticks out. Nah, silly idea. Never catch on.

While this certainly looks cool, I don't see it catching on for general applications. Maybe a few specialist uses such as music or graphics. File under "solutions looking for problems".

Re:It won't replace the mouse.

[plover]

Well, I didn't go into details of every device we tested, but trackballs are problematic as well. They're as big a dirt magnet as a mouse (perhaps bigger, being face up,) and take substantially longer for the average person to get used to. In our environment, that steep learning curve means a not insignificant amount of cost.

Besides, the purchase price for a trackball was higher than for a mouse, and the ongoing maintenance costs were highest of all input devices. They lost!

star trek

[xpyr]

This looks like it could be used as one of those touch panels like the kind that was used in star trek tng.

The perfect gaming input device...

[Twisted64]

...for Black & White. I can't believe nobody has mentioned this yet, after seeing the map demo.

Prior Art

[sandover]

For another example of a multi-touch technology (one not based on total internal reflection, however), see http://wall.accenture.com/ . We've been working with multi-touch screens 10-12 feet wide for a couple years now.

One of the challenges with multi-touch technology is the question of where to put the cameras (or other detectors). If the cameras are in-plane -- if they look directly across the field of interaction -- then there are ambiguity problems as you try to interpret multiple occlusions (fingers) as definite x,y points.

If the cameras are behind the screen plane (as seems to be the case with the NYU work, and also with Microsoft's multi-touch system exhibited at SIGGRAPH last year), then ambiguity is easier to deal with, but you can *only* work with projected or freestanding screens. This is a serious limitation in practical applications. Most of the durable, day-to-day screen technologies are enclosed in some way.

"The Next 1984"

[thegattaca]

If Apple computer releases this new human interface technology as these patents describe ( http://blog.wired.com/cultofmac/ ), this seriously could mean a major change in the direction in the market.

Jobs is scheduled to make a "huge" announcement on April 1st of this year, Apple Computer's 30'th anniversary.

We shall see . . .

Drawing board

[Pseudonymus Bosch]

There are rules linking working area and angle. You want a horizontal DIN A-4, a tilted drawing board and a vertical canvas. Painters have been working for centuries on big vertical surfaces.

Unless you have to work like Michelangelo, upwards on the Sistine Chapel what was extenuating for him.

Kungfu typewriter

[Pseudonymus Bosch]

Your arms and shoulders would get painfully tired after just a few minutes using this...

Myron Krueger proposed already in the 1980s the Kungfu Typewriter, where you type by kicking and punching. A healthier alternative to sitting all day developing RSI.

You're just asking for this

[TheFlamingoKing]

1. Take one of those overlay-LCD's for overhead projectors
2. Put a webcam behind it
3. ???
4. Profit!

Other locations

[MickLinux]

There's also one in the Childrens' museum in Portsmouth Virginia, USA. Anyone else know of other locations where they have this?

Mmm... nice kit, but not really comparable

[Anonymous Coward]

Point. However, it doesn't display realtime video, is single-user, restricted in its configurability, only comes with music-synth software (which cannot be easily modified), and doesn't run under Linux natively :)

Not really comparable, unfortunately. It's pretty much a single-purpose software-hardware bundle, not a flexible general-purpose interface.

Insensitive clods!

[Anonymous Coward]

I have no hands!

Voting machines..

[Myself]

Figures I wouldn't see this story 'til it was off the front page...

I use a touchscreen-enabled laptop and have first-hand (ha!) experience with what happens when you touch it in multiple places: It averages the touches, and the cursor ends up somewhere in between them. This makes for some fun line-drawing in a paint program, but mostly it's just confusing when it happens accidentally.

I have a feeling that a lot of the problems with touch-screen voting machines happen when people rest a palm or other fingers on the screen before pushing with their finger to indicate their choice. The screen ignores the light touches until a firm one comes in, at which time all the touches are averaged and Pat Buchanan gets a vote.

Seems to me, screens capable of sanely handling multiple touches would be able to avoid this problem.

has anyone tried building one yet?

[Bartholomew007]

From the description it does not seem to hard to build. i was just wondering if anyone has tried and would like to hear your experiences.