Slashdot Mirror
Video Chat Via Transparent Desktop Overlay
Jason0x21 writes "Wired News has an article about UNC Comp. Sci. researchers developing a transparent desktop overlay for video conferencing, allowing remote coworkers to literally point and interact with things on your screen. The researchers say that Apple's Quartz graphics engine let them go from idea to prototype in 'about 45 minutes'. Windows versions predicted in the future."
One of my favorite pieces of technology I've ever gotten a chance to play with is the SmartBoard Interactive Whiteboard It's a whiteboard that's touch-sensative. Basically, combine it with your favorite projection monitor and you've got a 60 inch touchscreen monitor. Just like any other touch screen, anywhere you tap the board is treated like a mouseclick in whatever application you're using. As an added bonus, "magic crayons" (really nothing more than plastic styluses) are at the bottom of the board. When the board detects one of the pens removed from its holder, it treats all touches as requests to draw on the screen.
:)
It's a great presentation tool to liven up a powerpoint and avoid the need to have to walk accross the room to get the next frame. Furthermore, playing solitare with foot-high cards is quite fun.
Windows has had the ability to draw transparent windows since 2000. However, there's a limit to how far they can go.
Particularlly, you can't do any blending against windows that are being drawn with DirectX/DirectDraw which is the way that any program that wants to approach full-motion video or 3D graphics has to do things. And that's what prevents Windows from handling this application.
Mac's OSX is a lot cleaner in this department because in their universe there are no exceptions to the rules... everything passes through Quartz, so there's a chance to capture and play with anything on the screen. DirectX and DirectDraw are painted onto the screen after all mortal windows are drawn in Windows, and that's why there's no chance to add an overlay to them.
ollaborating with co-workers in the same office is painful enough, but it's nigh impossible over a network.
For a couple of decades, researchers have tried to blend shared workspaces -- systems that allow two or more people to work on the same document -- with Internet video-conferencing systems, with little success.
Now researchers at the University of North Carolina at Chapel Hill have designed a new system that cleverly blends a video-conference feed with a transparent image of a computer desktop into one full-screen window.
Called Facetop, the system simultaneously transmits a video feed of users along with a shared, transparent image of the desktop. It allows two colleagues to work on the same document, Web page or graphic, while communicating face to face.
The system also tracks the position of the users' fingertips, which can control a cursor. As well as operating the shared desktop -- opening and closing files or selecting text, for instance -- the collaborators can use natural pointing gestures to communicate ideas about the document.
Developed by David Stotts, an associate professor of computer science, and graduate student Jason Smith, Facetop was conceived for collaborative tasks like programming or editing text. But the researchers say it has obvious uses in other areas such as medical imaging or remote teaching.
"So far, from the feedback we've received, it works fantastically," said Smith. "It's a very natural interaction. You can see the facial expressions and all the nuances of face-to-face communication."
"It's spectacular technology," said Robert Gotwals, associate director of Chapel Hill's Morehead Planetarium and Science Center, who saw a demonstration of an early version. "I've done lots of video-conferencing work. This is pretty cutting edge. It's a fast-moving field and the stuff David (Stotts) is doing is pretty cool."
The system can also be used for delivering lectures or PowerPoint presentations: The speaker is projected in the background of the document allowing her to point out bullet points or important passages. According to Smith, users easily switch attention between the subject and the desktop.
"The brain is really good at picking out what part of the screen the person is interested in," said Smith. "It's like being in a room full of conversations but having no trouble paying attention to only one.... People adapt to the system really naturally."
Facetop may also be used to as an alternative to the mouse, for controlling a machine simply by pointing with a finger.
The system is implemented in Mac OS X and is made possible largely by the system's Quartz rendering engine, which can make any part of the interface transparent. Thanks to Quartz, a quick prototype was whipped up in about 45 minutes, Smith said.
A PC version will likely be delayed until the release of Longhorn, the next major version of Windows, due in 2006, which will include a similar graphics subsystem.
The system is fairly inexpensive; it has been implemented on a pair of Apple PowerBooks and two $100 FireWire cameras. So far it has been tested only on Ethernet networks and not the Internet, though the researchers say there's no reason it shouldn't work just fine. They are also trying to hook it to Apple's iChat instant-message/video-conferencing software and other similar systems.
Facetop was initially developed for "pair programming," an increasingly popular form of collaborative coding that pairs programmers in teams of two: one to program, the other to suggest and correct. Stotts said programmers normally sit next to each other, and he has been interested for some time to see whether they could collaborate over the Internet.
According to Stotts, pair programming -- sometimes called extreme programming -- is fast and effective and is becoming increasingly popular for small projects.
The idea for Facetop occurred to Stotts and Smith accidentally. Instead of a computer monitor, Stotts projects his
I had an internship at DoE lab outside of Chicago, Argonne Natl. Laboratory, at which we worked on a project similar to this. The system allowed multiple users (of various geographic, or digital distances) to connect to a Desktop Server, on which all users could interact with icons, windows and programs in tandem as if they interfacing with a local deskptop in windows. Althou, we used BeOS as our platform because it had a small footprint. Interesting that three years later private companies have out-done the DoE's work. Sad.
There are photos on the article
w w.wired.com/news/images/full/cimg0401_f.jpg
http://a1112.g.akamai.net/7/1112/492/2002091464/ww w.wired.com/news/images/full/cimg0407_f.jpg
http://a1112.g.akamai.net/7/1112/492/2002091464/ww w.wired.com/news/images/full/imgp0173_f.jpg
http://a1112.g.akamai.net/7/1112/492/2002091464/w
(\(\
(^.^)
(")")
*This is the cute bunny virus, please copy this into your sig so it can spread
here's an Endeavors article about the project at UNC
FaceTop
!siht ekil skool gnihtyreve weiv fo tniop rieht morf ,yletanutrofnU
Show me on the doll where his noodly appendage touched you.
So, let's say those clever folks over at whatever-Gator-calls-themselves-now gets the brilliant idea that they could download one of them thar transparent-overlay-thingies whenever you browse to random-evil-webpage. Then, whoosh! They can sell remote access to your desktop so that advertisers can move all the annoying icons out of the way so that you can see the advertising more clearly. Or whatever. An since the overlay is transparent, the user can't figure out what is happening and simply thinks their system is posessed by the devil.
Heaven forbid that people should actually have to talk to each other face to face!
This concept was extensively researched by Hiroshi Ishii and his team between 1991 and 1994 while he was at NTT.
I saw the concept videos in my HCI class at the time. They went through all the various issues of pointing alignment, video flipping and the like.
Fine, i'll do Slashdot and Wired's jobs for them:
Screenshots
--
Mod up a post Rob doesn't like and you'll never mod again
-Ian
Business to business relationships have already become so depersonalised. This is just the next logical step - advancing technology that allows people to sit on their chairs to help other people. Heaven forbid that you would have to get up from your desk to help somebody!
Perhaps this kind of überchatting software is THE place where they can use those 3D desktop environments / window managers.
I don't really know if it would be useful, but perhaps it is cool to lean a window so you can see your partner while keeping an eye on the app content.
Anyway, beeing so far from the world as *I* am (yep, there are places on the south of the globe), where the bandwidth is kinda expensive, i can tell that i'll not be using this kind of technology for a while...
--krahd
mod me up scottie!
Looks like a reflection, rather than a transparent window.
If I point out that you are incorrect, making me a foe does not make you any more correct.
One possible feature for them to implement: one party can "flip the bird" to restart the whole session (as opposed to ALT-F4 or CTRL-C or whatever), thus giving new meaning to "Giving your co-worker the finger" for bad suggestions.
The Wknd Sessions - Malaysian and South East Asia independent music
Comment removed based on user account deletion
Seeing some misconceptions, tossed up a quick FAQ at http://www.cs.unc.edu/~smithja/facetop/index.html for your perusal.
I'll be adding material to it through the morning as issues pop up, but these are the ones we've seen the most of this weekend.
Actually, the camera can be anywhere, as long as you're in the field of view.
As for ease of use, it literally takes people about two seconds to calibrate their hand motions to the cursor movement, and they're off and running. It's exactly like you're standing in front of a mirror (assuming the camera is in front of you), and gesturing... the visual feedback you get from your own image is the key. The transparency lets you see both your 'reflection' and the document content simultaneously.
Don't worry, we're seeing a lot of people confusing the single-user mode (one head on screen) with the video-conferencing mode (two heads on screen), simply because they're not used to video conferencing including themselves.
Actually, our experiments have found that it really doesn't matter.
First off, the translucency is adjustable. Looks too cluttered? Make it more faint. Secondly, it's much like being in a room full of conversations at a party - you select particular conversations to pay attention to, and the rest just 'fade away'. In this case, when the user turns their attention to the document content, they don't notice the video, and when they concentrate on the video (either for hand motions or interaction with a remote user), the document content is ignored. The brain is much better at this sort of thing than most people realize.
Sadly, no. Forty-five minutes was about the time it took me to write the initial proof of concept, not the full application. (That included reading the documentation on various APIs.)
But yes, Cocoa made it much easier to do so.
I'll take that as a compliment on our making it look like you're standing in front of a mirror.
;)
That's rather the *point*.
Trust me, it would have been much easier to take a picture of a reflection on the screen surface than develop the bloody thing.