Slashdot Mirror
Intel Releases Open-Source Stereoscopic Software
Eslyjah writes "Intel has released a software library that allows computers to "see" in 3D. The library is available for Windows and Linux, under a BSDish license. Possible early applications include lipreading input. Check out the CNN Story, Intel Press Release, and project home page."
Nobody needs it. Nobody. Those who do - will write their own stuff..
<^>_<(ô ô)>_<^>
Being able to see all those lusty ladies in 3d, it might use too much bandwidth downloading pr0n.
When it is missing the leading "<"! Try this
Try the CNN story here.
Possible early applications include lipreading input.
Didn't we learn anything from 2001? You would think that people wouldn't be so eager to teach computers to read lips.
Very useful; especially being somewhat open.
Sure, it might sound a bit far fetched.
But.. a computer being able to actually construct something in 3 dimensions instead of simply a colorfield has *huge* implications with regards to image recognition.. it really does.
Take a single image of, say, a user's hands, for gesture recognition. How do you recognize the hands from the background? Color.... heuristics.. and what not.
But now.. it's simple. It's the stuff that's close to you! It's a completely different way to look at things.
I can never see the pictures, except sometimes when there's glass in front.
-
Let's not stir that bag of worms...
This is not for rendering in 3d, but for allowing a machine to build a 3d model (internally) of the environment it uses. I assume it is based on the same sort of binocular mechanism as animal eyes, but the algorithms to build the internal structure are probably pretty advanced.
A cool application (I haven't seen if they've done this yet) is rendering in Open GL the internal view of what the robot eyes see. It would allow you to walk through a building, and then have a 3D model for various other uses. Reverse engineering blueprints.
THis would be great technology to have on any mars lander, or even just to analyze the data sent back.
Open Source Identity Management: FreeIPA.org
How about better control for computerized surgery and other medical procedures?
Mapping, and lots of different kinds of aerial/satellite photo analysis. You don't have to be looking directly at something to see a 3D view of it--you can look at stereo images--and so could a computer. Examples at Lunar and Planetary Institute: 3D Mars images.
Eventually, autopilots for cars.
...will be a terminator device. Hook this library up to stepper controlled aiming device and your burglar alarm can now be lethal. Imagine the possibilities -- your nosey relatives walk into your workshop and four .30 cal machine gun replicas suddenly swing into position, targeting said relative... As they walk, they're tracked. Perhaps mount a camera and superimpose terminator-like vision crosshairs on them...
What's the use? Well, besides the obvious uses in architecture, etc., how about being able to play (insert favorite 1st person shooter game here) in your house?
Geeze, now that I think about it, maybe this isn't such a good idea. What would JonKatz think?
If all this should have a reason, we would be the last to know.
Too bad /.'ers here (at least the first 18 posts) don't see the benefits of this.
How about a way to have a PC recognize the position of your fingers and hands. You could use this to manipulate shapes in 3D in a 3D rendering and animation program WITHOUT SPECIAL GLOVES. You'd simply gesture into something like 3DStudioMax, Lightwave, or Caligari TrueSpace and create shapes by molding them with your fingers.
Or wouldn't it be cool to develop a "hand gesture API" which you could use to say play a karate game??? Think about a 3D Bruce Lee in front of you kicking, and you moving your OWN hands in front of the monitor to block it (and if you wear those cool 3D shutter glasses now common on graphics cards you will essentially have a low-budget VR system).
Or how about a driving game where you use no driving wheel but rather simply move your hands IN THE AIR. The game could be smart enough to recognize when you shift your hands away from an invissible steering wheel to grab an invissible gear stick on your side.
Or how about a tool to allow people like Stephen Hawkins gesture expressions and small movements in the air and have the computer react to these actions (like moving a wheelchair around, turning lights on and off, calling on the phone, changing TV channels, etc).
Think about the possibilities!!!
"While AMD processors may be slightly cheaper and run legacy x86 programs more quickly, SSE-optimised code compiled with Intel's compiler completely creams even the new Athlon XP "1900". Intel doesn't need to make up marketing numbers in order to make their processors look faster than they actually are"
I have a few issues with what you said. 1)The fact that intel had to release a compiler specially designed to work with it's processor to get it to get all the perfomance out of it is goofy, I can probably bet that if AMD wanted to, they could release an AMD optimised compiler and do the same thing. Also, to get the performance out of that compiler you would have to recompile everything to use it(or wait for your closed-source software makers to provide you witha build) 2) The marketing numbers are made by benchmarking intel processors and AMD processors. The AMD marketing number is the approx. clock speed of a comprable intel processor. If anything, you should flame intel for making their instruction pipeline hideously long to get their processors to ramp up to high clock speeds(sacrificing performance at the same time).
"Our web server used to run on a 1.4GHz Thunderbird, which was cheap but notoriously unstable" The processor probably wasnt unstable, I have a similar system without problems, it's probably another part of your system. Processors either work or they dont you can't have a half-broken processor
"And when the fan blew out a month ago, the whole computer was taken with it!"
Why would you use a cheap fan in a webserver for a buisness? Why didnt you use monitoring software to automagically shut the system down when the fan went out?(and I know about the video...that's a different situation, in the Tom video, they took the heatsink off, with the heatsink on, there should be enough time to shut it down.
-Bucky
Sounds like a project being worked on by Carnegie Mellon University researchers. I know CMU has a close relationship with Intel. Anyone know any more about the connection to this research?
Ceci n'est pas un post
I honestly cannot think of any other logical uses for this library.
One word: mensuration.
Read that again. It's not a typo. Mensuration (which generally means the act or process of measuring) specifically means figuring out how tall buildings or features are from satellite photographs. Traditionally it involves calculating heights trigonometrically based on the time of day, latitude, and angle of lens inclination from which the photograph was taken, using shadows as points of reference.
This is not a research project. It's a very practical process, with tons of applications in the commercial world.
My point is that there are a lot of image analysis techniques that you've probably never heard of before. Don't mistake a lack of experience on your part for a lack of usefulness on theirs.
Why would we want or need computers to 'see' in 3d? Sure, if might be of an advantage to people who need their computer to read their lips. All in all, though, it is of very little use.
Oh, you small-minded moron. See my post elsewhere in the thread about mensuration. There is more image processing going on in the scientific, commercial, and government worlds than you realize.
This is pretty neat, but reminds me of something from a few years ago.
I interviewed with Intel and during the interview they said in no uncertain terms that they were actively trying to keep people upgrading their systems, and hence keep the dollars rolling in. At the time, the interviewer said that the technique was largely by helping Micro$oft to keep new OSs coming that required more and more horsepower to run properly.
This is very cool in its own right (or could be, I haven't looked at it completely), but strikes me as another way they can push that curve...
1. 2.
This is great!
Now the field of Stereoscopic software (I know, I never even thought it existed) could open up thanks to the pushing of a sorta-powerful company.
Hey, at least we'll see an open standard file format.
In my windowmanager I want to switch from click to focus to 'focus follows eyeballs'. I don't even think you need stereoscopic code to handle it. It seems possible, I just don't know how to do it.
I've heard this called Photogrammetry .
... from the Third Edition of the Manual of Photogrammetry, is somewhat simpler in statement and that definition of photogrammetry is, the science or art of obtaining reliable measurements by means of photographs.
t.
Voice recognition is the next major advancement in computer user interfaces. Lip reading will increase the accuracy of voice recognition software. It is exciting that Intel is furthering the field of cybernetics.
I look forward to the day when I can dictate to my PC by just mouthing the words. Voice recognition and touchscreens will save the office worker from Repetetive Stress Injury and Carpal Tunnel Syndrome. Lipreading will make voice recognition practical for large offices and many other areas.
-Mike_L
Given that Intel wants this to be availible at a consumer level in the near future, why couldn't people create robots with the ability to map terrain in front of them.. hmm...
You see, I'm a mumbler. That's right, I talk to my machines all day long...coaxing, cajoling, cursing, crowing...I do it all.
I'd hate for my corporate desktop to be recording everything I'd say for the posterity of HR - like my emails are today.
Even worse, imagine if my machine decided to take all of that personally...*shudder*
While i have to say that Intel's OpenCV library rocks (for a number of reasons), stereoscopic vision is nothing new. The cnn article is more or less crap ("Until today, computer vision applications has been restricted to two dimensions
;-) The inclusion of stereo vision will be cool for people trying to write vision applications, but it's not advancing the state of the art.
"? nice try...) It's mishmash of reporter hype and stock text which describes computer vision in general ("Over the next 5 to 10 years, Intel Corp. expects computer vision to play a significant role in simplifying the interaction between users and computers"). The Sussex Computer Vision Teach Files page has a reasonable description of stereoscopic vision from 1994. Lip reading is not really a 3D problem, so stereoscopic capabilites aren't going to help much. Many of the other uses- 3D environment modeling, object modeling and recognition, etc, are being worked on (again, the algorithms aren't new, this is just a new open source implentation) but they're not easy.
I don't mean to sound pessimistic, though. OpenCV is really cool, both as a corporate contribution to open source, and as a programming library even if you never look at the code. And the Matlab interface means fewer MSVC++ sessions which end with me feeling homicidal
Here's how you do make stereoscopic images with a digital camera:
Take a picture like you normally would, but be mindful of the position and angle of your camera.
2) Snap a picture.
3) If the subject you're photographing is close to you, take a small step to the right. If the subject is far away, take a large step to the right.
4) Aim your camera at the subject and photograph it again.
5) Pull up both images in the photo editor of your choice.
6) Arrange the photos side by side. The first image you took should be on the left, the second image you took should be on the right.
7) Sit directly infront of your monitor, and blur your eyes. If you cant blur them, try crossing them slightly. Try to focus on "the picture in the middle". If you still cant do it, hold up a pencil (eraser-side up) exactly halfway between your eyeball and the screen. Focus on the eraser. The image on the screen should pop out at you in stereoscopic 3D.
For some good examples of a stereoscopic images I took, go here. Try the picture of the steering wheel first...Its really easy. You'll also see a number of stereo photos of Tumacoccori, an 18th century Spanish mission that got the shit beat out of it by native americans. You'll also find another picture thats rather interesting---It's a downward view of a deactivated nuclear missile still in the silo at the Titan Missile Museum outside of Tucson. The view extends about 20 floors below ground. If I were to have taken this photo in 1981 versus 2001, I would have been shot on sight.
Cheers,
Bowie J. Poag
Point Grey likes to use three-camera systems, with the cameras arranged in a triangle. This eliminates most ambiguities found with two-camera systems.
Algorithms to do this have been around for years, but only in the last few years has it become possible to do it in real time on commodity processors. Hans Moravec was the first, almost 30 years ago, back when it took him 20 minutes of mainframe time to process a stereo image. Point Grey was selling a DSP-based solution a few years ago. Now you can do it on consumer hardware.
Mobile robots should be getting much better shortly. Systems based on Polaroid sonars have the resolution of probing the world with the big end of a broom. Laser rangefinders cost way too much and have moving parts. Millimeter wave radar is complicated to use as an imager (although it opens most supermarket doors in the developed world.) Affordable, fast vision is finally here.
Yeah, a couple of web cams for target acquisition and tracking, two torquey servoes to aim the Nerf blaster, one more to pull the trigger, this software to track the perps, and whammo! I could even calculate trajectories for the nerf darts (within their uncertanty, which is large), and give the target a few seconds to yell out the password in case they don't want to get shot. Very cool. Automatic cubicle defense systems - I could make a fortune here!
I'm being a pedantic bastard, but according to webster it doesn't specifically mean that. Where are you getting your definition from?
From some remote sensing guys that I work with. I don't pretend to have all the jargon right, but I'm pretty sure about this one.
Actually, IMHO, pure monocular vision is a more interesting (and challenging) problem-- it's pretty clear that human stereo vision is an exercise in redundancy, since we can do pretty well with one eye closed, not to mention the fact that we perceive all kinds of 3d structure in 2d contexts (like your favourite pr0n- umm, quake screenshot ;-). The fundamental question is how do we interpret 2d images into 3d models (or whatever representation we use in our heads)? This a distinctly different (and more difficult) problem from building a 3d model from a motion sequence or stereo pair.
So long, and thanks for all the Phish
Stereoscopic vision is a very VERY useful thing for all things which percieve their environment through visual apparatus.
Huh? What do you mean? Well, close one eye (or put an eyepatch on) and look at your flat world. How far away is that streetlight? Hmm. How tall is that man? Hmm..
Of course, we as people are much MUCH MUCH better at percieving (interpreting) our visual environment than computers are. Humans generally have little trouble correctively percieving things such even through partial occlusions, changes in scale, orientation, distortion (glasses might make you able to see, but straight lines become anything but..) and changes in intensity and color.
Being able to get 3d information about objects aids greatly in interpreting what they are.
An image (2d) of a hand is (almost always) full of occlusions. These occlusions are diffucult to interpret in 2d because the edges in 2d have less differentiation than a depth-map would. (The distance metric is less ambigious!)
Wouldn't you like your computer to interact with you as if it was (a very obedient) human? This helps.
Intel did this in response to my post here[1] (on April 26th of last year!), in which I outlined just this kind of program, but mentioned "All this is very processor-intensive, but so far it's very straight-forward." So, of course, Intel releases it openly. :)))
6 32 31
[1]
http://slashdot.org/comments.pl?sid=11896&cid=2
The other big headache is tracking a camera move. You basically feed the footage into a camera tracking program and define tracking points in the image; features in the frame which the computer follows as they move - the software uses a lot of maths to work out where in 3D space these 2D points are and creates a CG camera in your 3D app to match the move, you usually then have to build a rough proxy version of the set in 3D to go with it (unless the production has the bucks to spend and they get a LIDAR scan of the set). THEN you get to finally start putting in your 3D elements, that is if you haven't shat yourself and run out of the building screaming after a week or so of staring at the same bloody footage 10-12 hours a day...
Ahem... anyway - where this new system could come in useful is using depth perception to generate a z-buffer, which would allow the computer to isolate foreground and background objects - no need to blue screen you can just point and click an actor to get a matte. Tracking would be made easy(er, anyway) as you have an actual 3D plate to work with, feed it to one of those programs that can auto-model 3D geometry from photos and you get your proxy set for free too...
Big blue screen shoots are tough on actors, just ask anyone that worked on one of the new star wars movies. They have to spend hours waiting for the screens to be set-up and lit, and the choreography of shooting a scene with a digital character is painful to learn, not to mention the hassles and expense of shooting with a motion control camera. Not only would a system like this speed up production but presumably with a real-time z-buffer being generated the cast and crew could interact with lo-res versions of the CG characters in real-time on monitors to get a better feel of what they are doing.
In fact as a wider application, once we all have depth-percieving videophones you could matte in any image behind yourself you want - great for phoning in sick from the beach :)
The Intel computer vision library is not the only such resource available. The TINA machine vision system has been developed since 1986 and provides functionality for the machine vision researcher at both the infrastructure level (datastructures and functions for an enormous range of mathematical, statistical and image processing tasks) as well as state-of-the-art solutions to many machine vision problems. These include low-level feature extraction, robust primitive fitting, object tracking, 2D object recognition and 3D object location. Indeed the stereoscopic subsystems in TINA (PMF, Stretch Correlation) have been viewed for many years as the standard for edge based stereo. TINA is almost unique as a resource and living archive of over 70 man years of research and over 200 peer reviewed publications in machine vision and medical image analysis. Functionality in TINA has practical utility in several industrial contexts.
For the past 5 years TINA has been provided as open source under an LGPL license and development is now based at the University of Manchester, UK.
Whilst I am very pleased that Intel recognise the importance of machine vision research and can only commend them on their open source approach I have some reservations regarding the use of OpenCV by the research community at large. Certainly their motives are business orientated (and one cannot argue with this). Therefore, however, the contents of their library are ultimately dictated by what Intel want not necessarily what the research community might need or indeed what is even possible (such as dense estimates of stereo).
Open Source software is vital in research disciplines where there is a significant software component. What better way to disseminate your results than to encapsulate your entire experimental apparatus in a tar file! Why should others in the field waste time reimplementing your algorithms (probably incorrectly) in order to duplicate your results. A process which sits at the very heart of any scientific endeavour.
TINA has recently received direct funding from the European Union for developed as the open source environment for machine vision and medical image analysis research. For more details of TINA visit the website at http://www.tina-vision.net
Sorry to rant a bit but it is not often I read something on here that I know so much about!
-- "Can't sleep, clowns will eat me!"