Slashdot Mirror
CMU Video Conference System Gets 3D From Cheap Webcams
Hesham writes "Carnegie Mellon University's HCI Institute just released details on
their "why-didn't-I-think-of-that-style" 3D video conferencing application. Considering how stale development has been in this field, this research seems like a nice solid step towards immersive telepresence. I was really disappointed with the "state-of-the-art" systems demoed at CES this year — they are all still just a flat, square, video stream. Hardly anything new. What is really cool about this project, is that researchers avoided building custom hardware no one is going to ever buy, and explored what could be done with just the generic webcams everyone already has. The result is a software-only solution, meaning all the big players (AIM, Skype, MSN, etc.) can release this as a simple software update. 'Enable 3D' checkbox anyone? YouTube video here. Behind the scenes, it relies on a clever illusory trick (motion
parallax) and head-tracking (a la Johnny Lee's Wiimote stuff — same
lab, HCII). It was just presented at IEEE International
Symposium on Multimedia in December."
Dundle Linux has been supporting something like this since its first version, just a few months ago. Skype is already on board.
The post title/summary is misleading -- this is actually 2.5D and not 3D at all. (It works on the premise that the background is static, and obtains a matte of the background, and using subtraction to dynamically key/mask the participant from the image, and then add the user as a second foreground layer; on the viewer side, headtracking is used to gently shift the user layer to reveal background hidden behind it)
For what it's worth, I really don't care for this effect at all. I am not denigrating its inventors in the slightest; this is a novel (read: low cost) approach, and I am sure some people would enjoy having this in their iChat/AIM/skype. To me, it's the equivalent of Apple's Photobooth filters (fisheye, inverted colors, etc) -- a cheap parlor trick that seems nifty for about 5 seconds, and then becomes precipitously distracting. True 3D has its own issues with distraction and visual anomalies (leading to headaches, etc). Even the best 3D cinematographers around have to be very careful to avoid these issues (for instance, Vince Pace, who shoots 3D for James Cameron (Titanic, Terminator, etc) has plenty of headache-inducing scenes in his demoreel, and this is a guy with state-of-the-art facilities who has as much knowledge as anyone about how to do stereoscopic cinematography). Frankly, I think video conferencing is best left 2D, and any efforts toward improving it should be spent increasing framerate/resolution (and reducing lag + dropped frames).
I am Jack's complete lack of surprise.
...but that sample conversation at the end of the video may have well been between two drunken epilepsy sufferers on boats in the North Atlantic. Who moves around like that while they are talking?
CGNU Video Conference System Gets 3D From Cheap Webcams
One of our competitors trademarked the term "hypothesis". From now on, we will call them "boneheaded ideas".
This does tons for immersion! It has to be implemented wherever there is a stationary camera (it obviously doesn't work with a camera phone). IIRC, Johnny Lee's work was free to use, so get to it and add that "Enable 3D" checkbox, developers! If only they'd cropped the resulting image to get rid of the black-ground, but that was probably just to show how it works.
Back in 1988 at an IBM convention in Vegas a similar technology was demonstrated running on a OS/2 server. The premature termination of OS/2 caused of course the end of the project: I am happy to see that somebody revived again the idea.
old tech: http://www.youtube.com/watch?v=Jd3-eiid-Uw
This would be really cool for controlling a videogame, like looking around a corner or something. Just add the webcam to the controls, and use lean left/right in real life to do the similar things in the game...
Just watched the video. I'm sure this will go over real well for those drunken late night videochats.
That makes a girls boobs pop out 3x larger.
I wonder if a more practical use would be to use the technique for video bandwidth reduction. If you know where the person is, you could concentrate video bandwidth on the face region, while keeping the rest of the "video" relatively static. No point in continuously compressing and sending boring background. Of course many codecs already do temporal compression that gives a similar effect, but this might increase the efficiency for video chat.
W-w-wow! I'd n-n-never guess they'd thought of t-t-this in the 80's!
Now I can see everyone's zits in 3d.
There's no -1 for "I don't get it."
The reason we'd be moving left to right would be to see something which isn't in the frame.
An idea I have been thinking about for awhile is to have the remote camera move when the user on the other side does. This would be much more convenient instead of having to ask the person to keep adjusting the camera angle to see outside the frame.
Inspired by Johnny Lee's stuff, I pulled some old code out over a year ago and turned it into a decent engine that handles multiple screens and head tracking (TrackIR) to achieve the motion parallax effect. Like with all 3D effects, it needs to be seen but the following videos give you a good idea.
Have a look at these demo videos and you can even download a demo:
My first test
http://nz.youtube.com/watch?v=X8PevTuEWlg
More accurate tracking
http://nz.youtube.com/watch?v=yf1hu6GLmf0
Multi screen study
http://nz.youtube.com/watch?v=ZBdtPz2V_vY
Engine complete
http://nz.youtube.com/watch?v=ku76aHq3pps
Download Demo
http://vandinther.googlepages.com/virtualwindow
It looks like the application for this is chatting when you are drunk, standing up, and swaying about. I don't know anybody who constantly moves their head around when videochatting. They tend to look straight into the camera. And wouldn't you be rather concerned if the person on the other end of your chat did start moving around and looking at you from weird angles?
... and then they built the supercollider.
The floating square of background with a floating talking bust reminds me of Max Headroom.
Wow...that looks better than what Star Trek people get when they say "on screen"!
and I can't believe no one else has mentioned it
http://www.minoru3d.com/
it comes with red-blue glasses for the purchaser to send to people that they intend to use it with.
every day http://en.wikipedia.org/wiki/Special:Random
Wouldn't it make more sense if the image moved as it tracked the other persons movement instead of yours? Otherwise you look sorta like a jagof swaying back and forth.
This is not even 2.5D. 2.5D was the term used for games like Doom, where you had a flat floor plan, plus floor and ceiling height (but couldn't have things above each other).
This is more like the side scrolling games from the Amiga days, except they usually had four layers moving at different speeds, rather than the two layers presented here.
In the Amiga days, nobody called those games 3D or even 2.5D. It was simply the way 2D side scrollers were made back then.
Some cartoons use the same technique, and have done so since long ago.
While this is pretty neat, I'm not sure it 'enhances interpersonal communication' since everyone using it will be bobbing back and forth like a Stevie Wonder impersonator convention.
Not to mention some schmuck in the US will soon sue because it made them puke from motion sickness.
USB webcams are pretty cheap these days. Why not use two, one on each side of the monitor?
In fact I've seen web cam kits with 2 in the package.
The would let you have true parallax, AND would have the benefit of making it appear that you are looking at the viewer.
Solves the two main problems I see being discussed here for an extra $29.95 or so.
Plus, it would make cool things like 3D position tracking possible (think Minority Report).