Slashdot Mirror
Software Converts 2D Images To 3D
eldavojohn writes "Dr. David McKinnon from Queensland University of Technology, has recently launched a site that turns your sets of 2D images into 3D bump maps by way of 8 years of his research. The catch is that you need to have between five and fifteen photos of your object and they must overlap at least 80 to 90 percent. So with a video of an object, one might be able to extract every nth frame and use this site to generate a 3D model. Doctor McKinnon said, 'The full version of this software would be great for realistic learning simulators and training software, where you want everything to look like the real thing. This technology could also be great for museums wishing to turn their display objects into 3D images that can be viewed online. We are even looking into making 3D models of cows to save farmers spending thousands of dollars transporting their cattle vast distances to auction sites, allowing for an eBay style auction website for cattle. Films, animations and computer games could also benefit, since 3D film making is taking over from the traditional 2D method of filmmaking. Another application is allowing people to create 3D models of their own face to use on their avatar in computer games or 3D social networking sites such as Second Life or Sony's Home.' Physorg has more details."
old news.
between five and fifteen photos of your object and they must overlap at least 80 to 90 percent.
So the 3D object in question will only have a front side? That's nowhere near enough for all sides.
We are even looking into making 3D models of cows to save farmers spending thousands of dollars transporting their cattle vast distances to auction sites, allowing for an eBay style auction website for cattle.
-So... you spent the last 8 years of your life to develop a 3D generator so that one day you may help farmers model their cows instead of spending thousands(!!!) of dollars on transfering them for auction?
-Yes.
-OK, just checking.
I am the lawn!
Cows are spherical, as every mathematician knows.
Do ya think that will happen? Looking forward to my future 3D pr0n stash
Favorite quote from my Mechanical Systems professor. I was surprised how much ME's use that in the real world.
Bump maps are so 20th century.
Okay, so I'm not as dumb as this post will seem to make me by asking, but for the sake of the uninitiated...
What is a bump map? and how is it significant in relation to photos and 3D?
Article title is misleading. A bump-map is less exciting than converting 2D to 3D. It's not like it's going to build a perfect model of your head from 15 photos.
Photosynth is far more interesting if you're excited by this concept.
Porquoi?
. . . http://www.mndl.hu/works/fractalcow
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
I assume this is a more fancy version of something like Microsoft PhotoSynth?
Shame it doesn't involve lasers. :-P
But...
I've been doing some side research in computer vision for a month or two in order to solve a problem regarding constructing a fairly accurate 3D model of a cat walking in front of a webcam. I'm totally ignorant about the entire field, so I've been trading ideas with another friend of mine who actually brought up the idea in the first place. Some of the ideas went from some sort of "averaging" between rough 3D sketches of a cat between multiple frames (with some sort of checking to see if they are, indeed, "topologically equivalent" [within reason of course, we don't want cats to be equivalent to beach balls]) to simply checking for the cat by first getting the edges of all of the objects in the scene, and then checking out the shadows on the cat in order to check for features varying along the depth axis.
In any case, does anyone know of any good resources / articles that deal with this very problem?
"Hegelians, who love a synthesis, will probably conclude that he wears a wig." - Bertrand Russell
is uploading about 15000 pics of Halle Berry as we speak. Man I'm gonna have a blast tonight!
You take the cow to auction to sell it - to get it off your farm and on to someone else's. The point of the auction is to move the cow. It might be somewhat more efficient to move the cow directly from farmer to farmer, but this intermediate stop at an auction house can't be that big an inconvenience, can it?
It sounds like a solution in search of a problem.
Obviously since videos are just sequences of overlapping images, coupled with relatively rare scene changes, could this program be used to generate whole 3D models of videos?
A displacement map actually distorts the surface of your 3D object. A bump map simply creates the illusion of surface detail based on the angle of light relative to the geometry face normals. Anyone with a rudimentary knowledge of 3D modeling and rendering should know the difference between the two, so their use of this terminology is strange.
Oh, this is simple. If it's moving and ignores you - it's likely to be a cat.
Bonus points if you can project laser dot and move it around. If the object tracts that, it's almost assuredly a cat.
Faster! Faster! Faster would be better!
...except Crazybump (http://www.crazybump.com/) is faster, funnier, and has more features. Indispensable for 3D shader development.
since 3D film making is taking over from the traditional 2D method of filmmaking.
This is absolutely... positively... WRONG!
Quark tried to take 'images' of Major Kira so he could re-create her 3-D image in the holodeck for some pervy customer.
Dear Beardo:
The initial VERY VERY is mine.
BTW, here's what Google Search returns for
2D to 3D Conversion.
Please change my initial comment to VERY, VERY, VERY old news.
Yours In Stereo,
Kilgore Trout
Hahah, well, yeah. But what if there was a person who did the same exact thing? No, what if there was a person who did the same thing, AND he had a beach ball on top of his head with two ears pasted on top of it?
"Hegelians, who love a synthesis, will probably conclude that he wears a wig." - Bertrand Russell
What happens if we use this for videos (which are just sequences of generally overlapping images)?
If any progress could be made in this department, we could make video game maps by simply recording a factory with a video camera.
This would be very useful on dating websites where you need to know if the girl has a big ass. They often provide a very vague 2D image of their frontside.
This is Great! Now we can feed in old episodes of Gilligan's Island, extract 3D facial maps of each of the castaways, and paste them onto different actors. Finally we can produce new episodes! We can replace Fake Ginger with Real Ginger in the movie. Imagine the possibilities!
Now, if only voice reproduction and voice morphing technology was moving at the same pace as video.
When our name is on the back of your car, we're behind you all the way!
I've always thought it would be cool to have a tool that could take scenes from old movies where the camera was pointing out the window of a car and convert it into a perfect 3-d map.
You could even extract the people and build models from them including movement.
It's kind of the same as when they put all those dots/lines on a person's body to be able to model the exact movements of the body, just using smarter software instead of dots...
You could gather massive amounts of data from a single shot once a computer can fully comprehend what's going on in the shot.
They could combine this with: http://www.cyberware.com/products/software/fileFormats/iges124.html to create some more useful file outputs than .ply
Why is "porn" in the tags?
'Cause those cow pics aren't always too flattering. So someone needs to clean them up to put her from her best side.
So the boss will say "Those udders aren't big enough. I want big udders. Frickin' huge. And they should be bright pink. Get rid of all the veins and stuff". And I'll just give the udders a bit of a polish, make her a bit more toned, sort out any spots or skin problems. If the guy wants to meet her, it's not like it's not the same cow, maybe she just has a bad hair day.
You're right, we even read of that on... Slashdot!
Animoog.org
When your robot can navigate any foreign environment or your Natal 2 can work without a time-of-flight sensor remember its because of the work on "Structure and Motion" by guys like this (and me).
The Mars Exploration Rovers convert stereo-pair photographs into 3-D terrain models every day, and have been doing this for five years. It's not at all clear what this guy is doing that's new, although I expect if I had the time to drill down through the popularizations to the actual technology, it would be clear.
Now, having a robot that understands what it's seeing, without human input, is much harder.
http://www.geoffreylandis.com
This is how 3-d terrain mapping has been done for almost two decades.
Overlapping photographs (from a plane) are scanned and then processed by software that computes parallax for thousands of points across the overlapping images to compute depth changes.
what's new here ? Three dimensional reconstruction from images or video are not really new. The INRIA does it for a long time ( http://grimage.inrialpes.fr/index.php ) and even forked a startup that works with the cinema industry ( http://www.4dviews.com/ ). I even recall a demo which is two or three years old where a guy playd street fighter in front of several cameras. I was not able to RTFA (slashdotted), so can someone enlighten me ? Is the novelty the webservice ?
before someone comes up with stereoscopic porn?
...I now know that Chocolate milk is imported from Scotland. Thanks for the enlightenment. :D
If you are really going to try and take your technology mainstream, you may as well go and get a bunch of Shroud of Turin pictures, use your technology to reconstruct Jesus in 3d, and get yourself a guest TV spot on Fox. If your Jesus winds up looking like Peter O'Toole, so much the better!
This is my sig.
this is a one-week assignment in the computer vision course at Carnegie Mellon...
- Ryan
The FIRST WORD of the story is 'Dr', closely followed by his name. Did that not clue you into the fact that he already has a PhD?
There's already a biometrics security firm that puts your face on Second Life avatars using their facial recognition technology and database, for $10/face with bulk discounts, which can turn out very crappy or very real depending on the lighting used, the angle of the head, and the photo quality. I suppose that the main customers have probably been people from companies that want to maintain an air of professionalism as they appear in a virtual world, since several IT companies like Cisco use Second Life for meetings with the public. Hopefully what this would do is create competition and lower prices, though.
What if the cow was cardboard?
Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
Beee beee beee boo-boo-boo--chicka-chicka-chicka-chicka.
Troll it perhaps may be, but offtopic it certainly is not.
Why couldn't I just submit a copy of the same photo five to fifteen times? That would have at least 80-90% overlap.. In fact I'd go so far as to say it would have 100% overlap.
"The purpose of learning is growth, and our minds, unlike our bodies, can continue growing as long as we live." - M.J. A
First thing to test? Pretty obvious......porn!!!!!!! Son of a bitch.......it just struck me just now but why hasn't anyone made 3d porn yet? Or am I missing something and its already been done?
Already has a product, that builds 3d models based on pictures.
Autodesk Image modeler 2009
With a web-camera? My guess, next to impossible. An array? Maybe a chance. A cat is fairly soft and elastic, which makes model based approaches hard. The fur likely has to few identifiable features to provide enough depth information for a 3D-model.
Best chance Structured light. Preferably in the near-infrared spectrum, this can be captured by your web-cam, but doesn't scares the cat.
If I'm not mistaken, your run-of-the-mill projector does (also) emit near-infrared light. Band-pass filter for the camera, low-pass filter (or band-pass) for the projector, and of you go.
"Between strong and weak, between rich and poor [...], it is freedom which oppresses and the law which sets free"
You can create distance estimates using two cameras in stereo. I would recommend starting there. Check out a paper called "Remote Gaze Estimation with a Single Camera Based on Facial - Feature Tracking without Special Calibration Actions" by Hirotake Yamazoe, et al. You might be able to use some of the distance estimating formulas in there.
Indeed. There are also folks who use 2-d to 3-d technology to produce a 3-d head model of a subject based on 2-d images, rotate the head to the proper perspective, and then render the head at that perspective in order to compare it to a current 2-d image for face recognition.
The tricky part is picking points of correspondence in the images. The overlap requirement in TFA is probably to ease the difficulty of this. Usually, some kind of iterative error minimization is performed over the parameters of an affine transform. If two images are already pretty similar, the transform parameters will be more accurate and there will be more of the same "interesting" points in each image to check for correspondence. With face images, though, it's a bit easier, because you can also use contextual clues such as recognizing eyes/nose/mouth from the image first, and then using key points on those known features to get you started.
Still sounds like a cat to me.
This is my footer. There are many like it, but this one is mine.
I wrote this software and thought that I should reply to some of these threads to clear up any misconceptions about how it works.
It basically uses the parallax between the consecutive images in order to determine the depth from the camera of each point in the scene. You can create full polygon models as evidenced by the .ply downloads on the site. There is also a ply2dae convertor there if that suits you better and the in built view is displaying texture obj models (trickier to download because there are 3 files).
You can do full surrounding object models but in the interest of not overloading our server with 1000's of huge images we have limited the upload to small number as a bit of taster. I have run sub-millimeter accurate reconstructions with hundreds of images. The software is heavily accelerated by GPU processing and would be one of the quickest and most accurate incarnations of such technology available in the research literature.
The underlying tech is very similar to photosynth but we have gone a step further and reconstructed the 3D models of the scene as opposed to some sort of 3D mosaic. We are hoping to commercialise the tech so we have not published the core algorithms. Any one interested?
I'm posting this really late in the thread so maybe nobody will read it (or care) but...
If there is one place on earth that is crying out to see this technology used it is the KILOMETERS (really!) worth of intricate stone carvings at Angkor Wat (Cambodia). I've thought about borrowing (stealing?) a friend's $500,000 laser scanner to capture them but the 1) he (his institute really) probably wouldn't let me 2) the thugs who run Cambodia would probably not let me use it without me paying some extortionate amount. There really is no-where else on earth where you can see the results of thousands of man-years of skilled stone carvers. This priceless cultural heritage should be captured before pollutants like acid rain slowly erodes it or thieves literally dynamite it to pieces.
Now perhaps anyone with a good video camera, a steady hand, and a LOT of patience can get this done! Perhaps if this job is too large for any one individual to complete it could be done in sections and the individual video sequences shared over the internet. Anyway, I hope this software is modified to handle video (subject to certain restrictions such as shooting in progressive mode).
If I'm not completely mistaken, some video tools already available for quite some time would be able to do just that... e.g., Boujou is able to convert several tracking points in video footage to points in space, in order to later put other stuff in or alter the footage ...