Slashdot Mirror
Intel Releases Open-Source Stereoscopic Software
Eslyjah writes "Intel has released a software library that allows computers to "see" in 3D. The library is available for Windows and Linux, under a BSDish license. Possible early applications include lipreading input. Check out the CNN Story, Intel Press Release, and project home page."
Nobody needs it. Nobody. Those who do - will write their own stuff..
<^>_<(ô ô)>_<^>
John
John
I'm sure I'll be the first of many to say please fix the HREF in the story. How did that get past the editors?
I can just see the next toy recalls: "Buffy the Barfing Doll is being recalled due to misalignment of Buffy's two camera 'eyes'. It seems Buffy is crosseyed."
Looks like only two uses for this:
1. 3D Research
2. Games
I honestly cannot think of any other logical uses for this library. And the places that do either category probably already have this, but it's nice to see some free stuff though. Now if only they'd GPL it...
Everything is mainstream now.
Being able to see all those lusty ladies in 3d, it might use too much bandwidth downloading pr0n.
When it is missing the leading "<"! Try this
oops, it is GPLed, or something pretty close. ignore line in previous comment. New thought: Hey! free game companies can use it too. Maybe a Bot-Building game...? Are you listening, SourceForge community?
Everything is mainstream now.
Try the CNN story here.
Possible early applications include lipreading input.
Didn't we learn anything from 2001? You would think that people wouldn't be so eager to teach computers to read lips.
While AMD processors may be slightly cheaper and run legacy x86 programs more quickly, SSE-optimised code compiled with Intel's compiler completely creams even the new Athlon XP "1900". Intel doesn't need to make up marketing numbers in order to make their processors look faster than they actually are. Our web server used to run on a 1.4GHz Thunderbird, which was cheap but notoriously unstable. And when the fan blew out a month ago, the whole computer was taken with it! Meanwhile, the Itanium system we used to replace it has been humming along without a hitch since it arrived. I do keep an Athlon box around for the occasional Quake 3 match, but for serious business, I support Intel and their strong open-source backing.
Loneliness is a power that we possess to give or take away forever
"Check out the A HREF='http://www.cnn.com/2001/TECH/ptech/12/18/com puter.lip.read.idg/index.html">CNN Story, Intel Press Release, and project home page.'"
/. staff for posting such things? ;)
Dear editor, please check this out first...
I'm sure chrisd wanted to have the l33t first (story)post..why can't we mod down
I can never see the pictures, except sometimes when there's glass in front.
-
Let's not stir that bag of worms...
A HREF="http://wwwwwwwwwwww.cnnnnnnnnnnnnnn.com/2001 /TECH/ptech/12/18/computer.lip.read.idg/index.html ">
This is not for rendering in 3d, but for allowing a machine to build a 3d model (internally) of the environment it uses. I assume it is based on the same sort of binocular mechanism as animal eyes, but the algorithms to build the internal structure are probably pretty advanced.
A cool application (I haven't seen if they've done this yet) is rendering in Open GL the internal view of what the robot eyes see. It would allow you to walk through a building, and then have a 3D model for various other uses. Reverse engineering blueprints.
THis would be great technology to have on any mars lander, or even just to analyze the data sent back.
Open Source Identity Management: FreeIPA.org
Why would we want or need computers to 'see' in 3d?
Sure, if might be of an advantage to people who need their computer to read their lips. All in all, though, it is of very little use.
I can see it now, webcams that have stereoscopic input. A whole new breed of porn/camwhoring'll rear it's useless, ugly head.
Our computers don't need to see us. We don't need THEM telling us we're all ugly.
"I wasn't sniffing your spicy brains."
...will be a terminator device. Hook this library up to stepper controlled aiming device and your burglar alarm can now be lethal. Imagine the possibilities -- your nosey relatives walk into your workshop and four .30 cal machine gun replicas suddenly swing into position, targeting said relative... As they walk, they're tracked. Perhaps mount a camera and superimpose terminator-like vision crosshairs on them...
What's the use? Well, besides the obvious uses in architecture, etc., how about being able to play (insert favorite 1st person shooter game here) in your house?
Geeze, now that I think about it, maybe this isn't such a good idea. What would JonKatz think?
If all this should have a reason, we would be the last to know.
Too bad /.'ers here (at least the first 18 posts) don't see the benefits of this.
How about a way to have a PC recognize the position of your fingers and hands. You could use this to manipulate shapes in 3D in a 3D rendering and animation program WITHOUT SPECIAL GLOVES. You'd simply gesture into something like 3DStudioMax, Lightwave, or Caligari TrueSpace and create shapes by molding them with your fingers.
Or wouldn't it be cool to develop a "hand gesture API" which you could use to say play a karate game??? Think about a 3D Bruce Lee in front of you kicking, and you moving your OWN hands in front of the monitor to block it (and if you wear those cool 3D shutter glasses now common on graphics cards you will essentially have a low-budget VR system).
Or how about a driving game where you use no driving wheel but rather simply move your hands IN THE AIR. The game could be smart enough to recognize when you shift your hands away from an invissible steering wheel to grab an invissible gear stick on your side.
Or how about a tool to allow people like Stephen Hawkins gesture expressions and small movements in the air and have the computer react to these actions (like moving a wheelchair around, turning lights on and off, calling on the phone, changing TV channels, etc).
Think about the possibilities!!!
Sounds like a project being worked on by Carnegie Mellon University researchers. I know CMU has a close relationship with Intel. Anyone know any more about the connection to this research?
Ceci n'est pas un post
Uh... what happened to the slashdot policy of "check them dar.. links" before posting, and then what happen to the editorial checking, and reading, links before passing a post to the mass?
It isn't a lie if you belive it.
So now when I flip my computer the bird, it can delete my porn, right?
.cig - what you do after winning a good flame war
This is pretty neat, but reminds me of something from a few years ago.
I interviewed with Intel and during the interview they said in no uncertain terms that they were actively trying to keep people upgrading their systems, and hence keep the dollars rolling in. At the time, the interviewer said that the technique was largely by helping Micro$oft to keep new OSs coming that required more and more horsepower to run properly.
This is very cool in its own right (or could be, I haven't looked at it completely), but strikes me as another way they can push that curve...
1. 2.
This is great!
Now the field of Stereoscopic software (I know, I never even thought it existed) could open up thanks to the pushing of a sorta-powerful company.
Hey, at least we'll see an open standard file format.
whoever thinks this is useless is an absolute idiot. you could improve security by having computers recognize the users' face, or how about making a model airplane that can fly itself (or at least be smart enough to know when it's about to hit the ground), or perhaps combine this with a servo-controlled gun? or perhaps create instant 3D models of whatever you want (substitute lara croft with your girlfriend? have her running around your apartment complex?) or combine this with some robotic arms and have it make sculptures out of something you modeled on the computer? the possibilities are endless, and this kind of technology -interfacing computers with the real world- will be the driving force of our next big economic boom. -william
In my windowmanager I want to switch from click to focus to 'focus follows eyeballs'. I don't even think you need stereoscopic code to handle it. It seems possible, I just don't know how to do it.
Now my computer that I'm sending on a manned mission to mars will be able to lip read! That may come in handy if the humans start to become a risk to the mission. That'll makes Hal complete.
OpenCV relies on IPP (also from Intel) for it speed. IPP is free for Windows and Linux. However, the source to IPP is not publicly available.
OpenCV is still a very complete vision processing library. Having it be completely open would make it 100x more interesting to me.
-Dan
WARNING: When using this software, do NOT talk about disconnecting your computer in front of it, if your plan to operate heavy machinery in outer space afterwards.
on intel's part. There aren't many applications these days that require a lot of cpu power, but I bet this library does. So intel just gives app writers a little push so that the demand for faster intel processors does not slow down. Heck, it's a win-win for the consumer (rare these days).
how about using this to improve video compression quality? make a codec that correctly identifies the more important objects in a scene (ones closer to the camera, ones that are moving) and use more of the bandwidth for those objects, less on things like the background, or plants, or whatever. or create 3D animations from reality that rival the quality (but not the size) of raster-based video.
Voice recognition is the next major advancement in computer user interfaces. Lip reading will increase the accuracy of voice recognition software. It is exciting that Intel is furthering the field of cybernetics.
I look forward to the day when I can dictate to my PC by just mouthing the words. Voice recognition and touchscreens will save the office worker from Repetetive Stress Injury and Carpal Tunnel Syndrome. Lipreading will make voice recognition practical for large offices and many other areas.
-Mike_L
Given that Intel wants this to be availible at a consumer level in the near future, why couldn't people create robots with the ability to map terrain in front of them.. hmm...
You see, I'm a mumbler. That's right, I talk to my machines all day long...coaxing, cajoling, cursing, crowing...I do it all.
I'd hate for my corporate desktop to be recording everything I'd say for the posterity of HR - like my emails are today.
Even worse, imagine if my machine decided to take all of that personally...*shudder*
Combine this with Magic Lantern and the FBI can "read" your conversations too. :)
Mozilla's a nice operating system, but it needs a better browser.
While i have to say that Intel's OpenCV library rocks (for a number of reasons), stereoscopic vision is nothing new. The cnn article is more or less crap ("Until today, computer vision applications has been restricted to two dimensions
;-) The inclusion of stereo vision will be cool for people trying to write vision applications, but it's not advancing the state of the art.
"? nice try...) It's mishmash of reporter hype and stock text which describes computer vision in general ("Over the next 5 to 10 years, Intel Corp. expects computer vision to play a significant role in simplifying the interaction between users and computers"). The Sussex Computer Vision Teach Files page has a reasonable description of stereoscopic vision from 1994. Lip reading is not really a 3D problem, so stereoscopic capabilites aren't going to help much. Many of the other uses- 3D environment modeling, object modeling and recognition, etc, are being worked on (again, the algorithms aren't new, this is just a new open source implentation) but they're not easy.
I don't mean to sound pessimistic, though. OpenCV is really cool, both as a corporate contribution to open source, and as a programming library even if you never look at the code. And the Matlab interface means fewer MSVC++ sessions which end with me feeling homicidal
Here's how you do make stereoscopic images with a digital camera:
Take a picture like you normally would, but be mindful of the position and angle of your camera.
2) Snap a picture.
3) If the subject you're photographing is close to you, take a small step to the right. If the subject is far away, take a large step to the right.
4) Aim your camera at the subject and photograph it again.
5) Pull up both images in the photo editor of your choice.
6) Arrange the photos side by side. The first image you took should be on the left, the second image you took should be on the right.
7) Sit directly infront of your monitor, and blur your eyes. If you cant blur them, try crossing them slightly. Try to focus on "the picture in the middle". If you still cant do it, hold up a pencil (eraser-side up) exactly halfway between your eyeball and the screen. Focus on the eraser. The image on the screen should pop out at you in stereoscopic 3D.
For some good examples of a stereoscopic images I took, go here. Try the picture of the steering wheel first...Its really easy. You'll also see a number of stereo photos of Tumacoccori, an 18th century Spanish mission that got the shit beat out of it by native americans. You'll also find another picture thats rather interesting---It's a downward view of a deactivated nuclear missile still in the silo at the Titan Missile Museum outside of Tucson. The view extends about 20 floors below ground. If I were to have taken this photo in 1981 versus 2001, I would have been shot on sight.
Cheers,
Bowie J. Poag
Point Grey likes to use three-camera systems, with the cameras arranged in a triangle. This eliminates most ambiguities found with two-camera systems.
Algorithms to do this have been around for years, but only in the last few years has it become possible to do it in real time on commodity processors. Hans Moravec was the first, almost 30 years ago, back when it took him 20 minutes of mainframe time to process a stereo image. Point Grey was selling a DSP-based solution a few years ago. Now you can do it on consumer hardware.
Mobile robots should be getting much better shortly. Systems based on Polaroid sonars have the resolution of probing the world with the big end of a broom. Laser rangefinders cost way too much and have moving parts. Millimeter wave radar is complicated to use as an imager (although it opens most supermarket doors in the developed world.) Affordable, fast vision is finally here.
It sounds like it could have quite a few different uses. Can you imagine using this for a security system, perhaps when combined with face-recognition software? They would make a good combo.
Yeah, a couple of web cams for target acquisition and tracking, two torquey servoes to aim the Nerf blaster, one more to pull the trigger, this software to track the perps, and whammo! I could even calculate trajectories for the nerf darts (within their uncertanty, which is large), and give the target a few seconds to yell out the password in case they don't want to get shot. Very cool. Automatic cubicle defense systems - I could make a fortune here!
Actually, IMHO, pure monocular vision is a more interesting (and challenging) problem-- it's pretty clear that human stereo vision is an exercise in redundancy, since we can do pretty well with one eye closed, not to mention the fact that we perceive all kinds of 3d structure in 2d contexts (like your favourite pr0n- umm, quake screenshot ;-). The fundamental question is how do we interpret 2d images into 3d models (or whatever representation we use in our heads)? This a distinctly different (and more difficult) problem from building a 3d model from a motion sequence or stereo pair.
So long, and thanks for all the Phish
> One word: mensuration.
Sorry,
No matter how useful people say this
technology is, it's still no replacement
for tampons.
How was that trolling? It was a waste of post sure, it should be -1, but, trolling?!
I have been to San Xavier outside of Tuscon, and your pics brought back memories - they are excellent. Now I need to try some myself (not that I didn't know about the technique, but I have never used it before)...
Reason is the Path to God - Anon
...if only my Intel webcam was supported. Intel isn't supporting it. *!#$@!!
Someone set us up the bomb, so shine we are!
Stereoscopic vision is a very VERY useful thing for all things which percieve their environment through visual apparatus.
Huh? What do you mean? Well, close one eye (or put an eyepatch on) and look at your flat world. How far away is that streetlight? Hmm. How tall is that man? Hmm..
Of course, we as people are much MUCH MUCH better at percieving (interpreting) our visual environment than computers are. Humans generally have little trouble correctively percieving things such even through partial occlusions, changes in scale, orientation, distortion (glasses might make you able to see, but straight lines become anything but..) and changes in intensity and color.
Being able to get 3d information about objects aids greatly in interpreting what they are.
An image (2d) of a hand is (almost always) full of occlusions. These occlusions are diffucult to interpret in 2d because the edges in 2d have less differentiation than a depth-map would. (The distance metric is less ambigious!)
Wouldn't you like your computer to interact with you as if it was (a very obedient) human? This helps.
As the presenter put it "take a pair of sunglasses, smear them with light machine oil, put on the sunglasses, put on thick gloves, and then use a pair of chopsticks (alone) to try and pick something up". It was not clear at the time if they were using anything steroscopic or not. However it was shown that video inputs for the computers were from multiple angles.
One of the robots (an arm) that was under-development was made for use in a car factory to perform point spot welding. One of these things could easily punch a hole through the side of a car, so distance measurement was important at that time.
How much this has progressed since is unknown (to me at least). However the technology featured was specific and not necessarily within the price range that someone can just grab off the shelf.
Hopefully this imaging technology will give someone who wants to make their own robot a head start (somewhere). However I would rather stay away from it if it has any of the "capacity" of the robot in the Television article that I saw.
...New ways to bomb afghans!
(don't you worry, if you're worrying. it can also be used to bomb iraqis.)
I've been working with this library now for about 6 months. I'm not even sure why Intel chose to make a press release about it now - they didn't even upload any new code! (Maybe they will soon). Hmm. Maybe there is a computer vision conference coming up?
/.ers who read the report didn't go and look at the site or see what the library can do.
This is a super library for computer vision. The best I've seen. Highly optimized. Goes way beyond anything out there in the Open source domain.
Sadly, it seems to me that many of the
Good thing you asked me to read that again...
First time around, it looked like something a lady would be doing at 'this time of the month.' -_-;
Intel did this in response to my post here[1] (on April 26th of last year!), in which I outlined just this kind of program, but mentioned "All this is very processor-intensive, but so far it's very straight-forward." So, of course, Intel releases it openly. :)))
6 32 31
[1]
http://slashdot.org/comments.pl?sid=11896&cid=2
Already done.
So, basically !
Middle finger on it's own is Ctrl-Alt-Del
A fast pumping motion is WebPorn
Small Finger and Index are for launching windows
A close fist to Quake some arse...
A shooting motion to "Kill" a process...
Yeah, I can see how this will improve the pantonime in front of my computer... 8)
It takes 40+ muscles to frown, but only four to extend your arm and bitchslap the motherfucker
For instance, this could be used in a car and help the computer make the difference between the SUV in front of me that is breaking and the SUV on the AD in front of me...
...
Could also help the car find it's place on the road, caus 3D would allow positionning
Could have REAL Biometrics
Could have Real 3D Pron...
Uh, I think my brain just took the wrong turn here 8)
It takes 40+ muscles to frown, but only four to extend your arm and bitchslap the motherfucker
The other big headache is tracking a camera move. You basically feed the footage into a camera tracking program and define tracking points in the image; features in the frame which the computer follows as they move - the software uses a lot of maths to work out where in 3D space these 2D points are and creates a CG camera in your 3D app to match the move, you usually then have to build a rough proxy version of the set in 3D to go with it (unless the production has the bucks to spend and they get a LIDAR scan of the set). THEN you get to finally start putting in your 3D elements, that is if you haven't shat yourself and run out of the building screaming after a week or so of staring at the same bloody footage 10-12 hours a day...
Ahem... anyway - where this new system could come in useful is using depth perception to generate a z-buffer, which would allow the computer to isolate foreground and background objects - no need to blue screen you can just point and click an actor to get a matte. Tracking would be made easy(er, anyway) as you have an actual 3D plate to work with, feed it to one of those programs that can auto-model 3D geometry from photos and you get your proxy set for free too...
Big blue screen shoots are tough on actors, just ask anyone that worked on one of the new star wars movies. They have to spend hours waiting for the screens to be set-up and lit, and the choreography of shooting a scene with a digital character is painful to learn, not to mention the hassles and expense of shooting with a motion control camera. Not only would a system like this speed up production but presumably with a real-time z-buffer being generated the cast and crew could interact with lo-res versions of the CG characters in real-time on monitors to get a better feel of what they are doing.
In fact as a wider application, once we all have depth-percieving videophones you could matte in any image behind yourself you want - great for phoning in sick from the beach :)
The Intel computer vision library is not the only such resource available. The TINA machine vision system has been developed since 1986 and provides functionality for the machine vision researcher at both the infrastructure level (datastructures and functions for an enormous range of mathematical, statistical and image processing tasks) as well as state-of-the-art solutions to many machine vision problems. These include low-level feature extraction, robust primitive fitting, object tracking, 2D object recognition and 3D object location. Indeed the stereoscopic subsystems in TINA (PMF, Stretch Correlation) have been viewed for many years as the standard for edge based stereo. TINA is almost unique as a resource and living archive of over 70 man years of research and over 200 peer reviewed publications in machine vision and medical image analysis. Functionality in TINA has practical utility in several industrial contexts.
For the past 5 years TINA has been provided as open source under an LGPL license and development is now based at the University of Manchester, UK.
Whilst I am very pleased that Intel recognise the importance of machine vision research and can only commend them on their open source approach I have some reservations regarding the use of OpenCV by the research community at large. Certainly their motives are business orientated (and one cannot argue with this). Therefore, however, the contents of their library are ultimately dictated by what Intel want not necessarily what the research community might need or indeed what is even possible (such as dense estimates of stereo).
Open Source software is vital in research disciplines where there is a significant software component. What better way to disseminate your results than to encapsulate your entire experimental apparatus in a tar file! Why should others in the field waste time reimplementing your algorithms (probably incorrectly) in order to duplicate your results. A process which sits at the very heart of any scientific endeavour.
TINA has recently received direct funding from the European Union for developed as the open source environment for machine vision and medical image analysis research. For more details of TINA visit the website at http://www.tina-vision.net
Sorry to rant a bit but it is not often I read something on here that I know so much about!
-- "Can't sleep, clowns will eat me!"
I work in vision robotics as a daily task. Using cameras looking down to determine a x-y plane for positioning parts to a tolerance of 0.0005 inch. Then placing parts on that targeted subject. One of the only problems is being able to determine the height of said subject. Currently we use mechanical means of taking in variances of two thousands of a inch. This make me want to get my calcuator out and determine the angle between the cameras that would be required to make the z height determination that would enable me to know the height to 0.0005 inch. Yes, there are applications for this!
From the project coding standards page:
//Look at this
if( a > 5 )
{
int b = a*a;
c = c > b ? c : b + 1;
}
That's it Intel, I refuse to partake in your silly project.
Currere potes, sed oculare non potes.
hey, wait a second, that sounds familiar
I guess if we want a private conversation, we'll have to remember to rotate the pod around so Hal can't see in through the window...
pictures look wonderful!
But watch out for too much stereo separation. If you take the second shot too far from the first, you will lose the sense of "fusing" the two images for many people. The picture of your plum car is a prime example.
look! with a program like visual basic one could create a program to teach oneself lipreading, by concatenating multimedia clips of typical lip positions for component phonemes. (disregaring the common problem of aphoneticity in english and most other languages). this is a really interesting techenology and inspires one to ask why it hasn't been done yet, or why it causes revulsion to the powers that be. whereas speech recognition is mastered by most except the deaf, the opposite occurs in the case of lip reading. read my lips: you ain't seen nothing yet, and you are already asking the computer to do it for you!