Slashdot Mirror
Software Converts 2D Images To 3D
eldavojohn writes "Dr. David McKinnon from Queensland University of Technology, has recently launched a site that turns your sets of 2D images into 3D bump maps by way of 8 years of his research. The catch is that you need to have between five and fifteen photos of your object and they must overlap at least 80 to 90 percent. So with a video of an object, one might be able to extract every nth frame and use this site to generate a 3D model. Doctor McKinnon said, 'The full version of this software would be great for realistic learning simulators and training software, where you want everything to look like the real thing. This technology could also be great for museums wishing to turn their display objects into 3D images that can be viewed online. We are even looking into making 3D models of cows to save farmers spending thousands of dollars transporting their cattle vast distances to auction sites, allowing for an eBay style auction website for cattle. Films, animations and computer games could also benefit, since 3D film making is taking over from the traditional 2D method of filmmaking. Another application is allowing people to create 3D models of their own face to use on their avatar in computer games or 3D social networking sites such as Second Life or Sony's Home.' Physorg has more details."
between five and fifteen photos of your object and they must overlap at least 80 to 90 percent.
So the 3D object in question will only have a front side? That's nowhere near enough for all sides.
We are even looking into making 3D models of cows to save farmers spending thousands of dollars transporting their cattle vast distances to auction sites, allowing for an eBay style auction website for cattle.
-So... you spent the last 8 years of your life to develop a 3D generator so that one day you may help farmers model their cows instead of spending thousands(!!!) of dollars on transfering them for auction?
-Yes.
-OK, just checking.
I am the lawn!
Cows are spherical, as every mathematician knows.
Favorite quote from my Mechanical Systems professor. I was surprised how much ME's use that in the real world.
Bump maps are so 20th century.
You know what, AC, you're right.
This guy has wasted his life. EMC2, an Ubuntu-based CNC milling program, has a JPG to G-code generator. A company I've worked for did this with some commercial success turning baby handprints and footprints into custom milled embossed plaques.
You take your picture, finesse it, and turn it into a 3D-object. Several CAD programs, such as Rhino and Solidworks, have plugins that do the same thing. There are dozens of programs that do this for various prices.
Even better, they only require ONE image.
---
ECHELON is a government program to find words like bomb, jihad, plutonium, assassinate, and anarchy.
Okay, so I'm not as dumb as this post will seem to make me by asking, but for the sake of the uninitiated...
What is a bump map? and how is it significant in relation to photos and 3D?
Quite, He either wants geek cred points or is trying to get his PHD with this. If they know this is already out he could loose his grants
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
Article title is misleading. A bump-map is less exciting than converting 2D to 3D. It's not like it's going to build a perfect model of your head from 15 photos.
Photosynth is far more interesting if you're excited by this concept.
Porquoi?
. . . http://www.mndl.hu/works/fractalcow
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
I assume this is a more fancy version of something like Microsoft PhotoSynth?
Shame it doesn't involve lasers. :-P
isn't that rule 34?
But...
I've been doing some side research in computer vision for a month or two in order to solve a problem regarding constructing a fairly accurate 3D model of a cat walking in front of a webcam. I'm totally ignorant about the entire field, so I've been trading ideas with another friend of mine who actually brought up the idea in the first place. Some of the ideas went from some sort of "averaging" between rough 3D sketches of a cat between multiple frames (with some sort of checking to see if they are, indeed, "topologically equivalent" [within reason of course, we don't want cats to be equivalent to beach balls]) to simply checking for the cat by first getting the edges of all of the objects in the scene, and then checking out the shadows on the cat in order to check for features varying along the depth axis.
In any case, does anyone know of any good resources / articles that deal with this very problem?
"Hegelians, who love a synthesis, will probably conclude that he wears a wig." - Bertrand Russell
It is impossible to in general extract a true 3D image from a single 2D image. There are ambiguous cases. In fact, there are even ambiguous cases for two images. When one uses CAD and such to make 3D out of 2D images one is making implicit choices (generally using heuristics about how the objects are likely shaped). One can however use multiple images from slightly different angles to extract a close to unambiguous result. However, doing that is not easy and that seems to be what they are doing here.
is uploading about 15000 pics of Halle Berry as we speak. Man I'm gonna have a blast tonight!
You take the cow to auction to sell it - to get it off your farm and on to someone else's. The point of the auction is to move the cow. It might be somewhat more efficient to move the cow directly from farmer to farmer, but this intermediate stop at an auction house can't be that big an inconvenience, can it?
It sounds like a solution in search of a problem.
Even better, they only require ONE image.
In other words, it's not the same problem.
This guy has wasted his life.
Ouch. So if it's not a huge discovery in an entirely new research area, it's worthless? Would you be willing to apply this criterion to your own accomplishments?
This guy has wasted his life. ... There are dozens of programs that do this for various prices. Even better, they only require ONE image.
Okay, smart-ass, here's your ONE image.
Oh, this is simple. If it's moving and ignores you - it's likely to be a cat.
Bonus points if you can project laser dot and move it around. If the object tracts that, it's almost assuredly a cat.
Faster! Faster! Faster would be better!
...except Crazybump (http://www.crazybump.com/) is faster, funnier, and has more features. Indispensable for 3D shader development.
Quark tried to take 'images' of Major Kira so he could re-create her 3-D image in the holodeck for some pervy customer.
Hahah, well, yeah. But what if there was a person who did the same exact thing? No, what if there was a person who did the same thing, AND he had a beach ball on top of his head with two ears pasted on top of it?
"Hegelians, who love a synthesis, will probably conclude that he wears a wig." - Bertrand Russell
If it's supposed to extract 3D data from a single 2D image clearly there's a catch. Model tweaking required? Limited types of images that can be processed? Inaccurate? Suppose I took a photo of a photo, or there was a picture on a wall; how is it supposed to know what's 2D and what's 3D, or what depth things have? Shadow analysis can only go so far.
If this software works flawlessly but requires more pictures it's much more important than something theoretical/flawed. If you want a 3D model of something you have to expect to take more than one picture.
// MD_Update(&m,buf,j);
What happens if we use this for videos (which are just sequences of generally overlapping images)?
If any progress could be made in this department, we could make video game maps by simply recording a factory with a video camera.
This would be very useful on dating websites where you need to know if the girl has a big ass. They often provide a very vague 2D image of their frontside.
This is Great! Now we can feed in old episodes of Gilligan's Island, extract 3D facial maps of each of the castaways, and paste them onto different actors. Finally we can produce new episodes! We can replace Fake Ginger with Real Ginger in the movie. Imagine the possibilities!
Now, if only voice reproduction and voice morphing technology was moving at the same pace as video.
When our name is on the back of your car, we're behind you all the way!
I've always thought it would be cool to have a tool that could take scenes from old movies where the camera was pointing out the window of a car and convert it into a perfect 3-d map.
You could even extract the people and build models from them including movement.
It's kind of the same as when they put all those dots/lines on a person's body to be able to model the exact movements of the body, just using smarter software instead of dots...
You could gather massive amounts of data from a single shot once a computer can fully comprehend what's going on in the shot.
They could combine this with: http://www.cyberware.com/products/software/fileFormats/iges124.html to create some more useful file outputs than .ply
You're right, we even read of that on... Slashdot!
Animoog.org
When your robot can navigate any foreign environment or your Natal 2 can work without a time-of-flight sensor remember its because of the work on "Structure and Motion" by guys like this (and me).
The Mars Exploration Rovers convert stereo-pair photographs into 3-D terrain models every day, and have been doing this for five years. It's not at all clear what this guy is doing that's new, although I expect if I had the time to drill down through the popularizations to the actual technology, it would be clear.
Now, having a robot that understands what it's seeing, without human input, is much harder.
http://www.geoffreylandis.com
Case in point: Steinmetz Solids, or mouhefanggai, that appear to be spherical from some angles, and hexagonal from other angles. Or the solid letters on the cover of the book Godel, Escher, Bach, that only make sense if viewed isometrically: a picture along any axis gives a completely different set of information than the other axes.
Nostalgia's not what it used to be.
Yes. This is actually a fairly common graduate (Masters) level project in computer vision courses. My version was even made for the web...
what's new here ? Three dimensional reconstruction from images or video are not really new. The INRIA does it for a long time ( http://grimage.inrialpes.fr/index.php ) and even forked a startup that works with the cinema industry ( http://www.4dviews.com/ ). I even recall a demo which is two or three years old where a guy playd street fighter in front of several cameras. I was not able to RTFA (slashdotted), so can someone enlighten me ? Is the novelty the webservice ?
It's quite possible to build one, but it only works from one angle.
If you were in the tree (where the girl's left arm is pointing) you'd see something like:
/
\|
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
before someone comes up with stereoscopic porn?
Awesome image choice.
...I now know that Chocolate milk is imported from Scotland. Thanks for the enlightenment. :D
"This guy has wasted his life."
Saith the foolish geek posting on Slashdot.
Pot, meet kettle.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
If you are really going to try and take your technology mainstream, you may as well go and get a bunch of Shroud of Turin pictures, use your technology to reconstruct Jesus in 3d, and get yourself a guest TV spot on Fox. If your Jesus winds up looking like Peter O'Toole, so much the better!
This is my sig.
this is a one-week assignment in the computer vision course at Carnegie Mellon...
- Ryan
I have mod points but I just had to reply while the laugh is still fresh.
An ASCII picture of a Penrose triangle. Never thought I'd live to see the day. Well done Sir.
This post contains benzene, nitrosamines, formaldehyde and hydrogen cyanide.
The FIRST WORD of the story is 'Dr', closely followed by his name. Did that not clue you into the fact that he already has a PhD?
Here is another one in Australia from multiple angles. http://upload.wikimedia.org/wikipedia/commons/3/38/Perth_Impossible_Triangle.jpg
There's already a biometrics security firm that puts your face on Second Life avatars using their facial recognition technology and database, for $10/face with bulk discounts, which can turn out very crappy or very real depending on the lighting used, the angle of the head, and the photo quality. I suppose that the main customers have probably been people from companies that want to maintain an air of professionalism as they appear in a virtual world, since several IT companies like Cisco use Second Life for meetings with the public. Hopefully what this would do is create competition and lower prices, though.
What if the cow was cardboard?
Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
I am wondering if you had any links to either the thing you put up on the web or for any papers that you've published or whatever about this topic? I've posted on this news item already about it, but I figure I should give this a try as well. I've been interested in these sorts of problems for a few months now, but I haven't a clue where to start in terms of reading any sort of articles or whatever. I'm mostly outside the realm of computer science myself.
"Hegelians, who love a synthesis, will probably conclude that he wears a wig." - Bertrand Russell
Why couldn't I just submit a copy of the same photo five to fifteen times? That would have at least 80-90% overlap.. In fact I'd go so far as to say it would have 100% overlap.
"The purpose of learning is growth, and our minds, unlike our bodies, can continue growing as long as we live." - M.J. A
It reminds you of speaking Swazi?
"The purpose of learning is growth, and our minds, unlike our bodies, can continue growing as long as we live." - M.J. A
First thing to test? Pretty obvious......porn!!!!!!! Son of a bitch.......it just struck me just now but why hasn't anyone made 3d porn yet? Or am I missing something and its already been done?
With a web-camera? My guess, next to impossible. An array? Maybe a chance. A cat is fairly soft and elastic, which makes model based approaches hard. The fur likely has to few identifiable features to provide enough depth information for a 3D-model.
Best chance Structured light. Preferably in the near-infrared spectrum, this can be captured by your web-cam, but doesn't scares the cat.
If I'm not mistaken, your run-of-the-mill projector does (also) emit near-infrared light. Band-pass filter for the camera, low-pass filter (or band-pass) for the projector, and of you go.
"Between strong and weak, between rich and poor [...], it is freedom which oppresses and the law which sets free"
You can create distance estimates using two cameras in stereo. I would recommend starting there. Check out a paper called "Remote Gaze Estimation with a Single Camera Based on Facial - Feature Tracking without Special Calibration Actions" by Hirotake Yamazoe, et al. You might be able to use some of the distance estimating formulas in there.
Indeed. There are also folks who use 2-d to 3-d technology to produce a 3-d head model of a subject based on 2-d images, rotate the head to the proper perspective, and then render the head at that perspective in order to compare it to a current 2-d image for face recognition.
The tricky part is picking points of correspondence in the images. The overlap requirement in TFA is probably to ease the difficulty of this. Usually, some kind of iterative error minimization is performed over the parameters of an affine transform. If two images are already pretty similar, the transform parameters will be more accurate and there will be more of the same "interesting" points in each image to check for correspondence. With face images, though, it's a bit easier, because you can also use contextual clues such as recognizing eyes/nose/mouth from the image first, and then using key points on those known features to get you started.
I suppose a 3D image is only true if you can see it in 3D. Since your monitor is 2D, it is in fact a very impressive 3D simulation. If you visit the site, you will see that there are some problems but overall, looks very interesting. Maybe the future of pr0n.
Views expressed do not necessarily reflect those of the author.
It only requires one post to get karma, right? And once someone has posted and successfully acquired karma legitimately, there's nothing further to be done and all other posts are a waste. :)
Seriously, I agree with you. The AC who started this thread is an idiot.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
I wrote this software and thought that I should reply to some of these threads to clear up any misconceptions about how it works. It basically uses the parallax between the consecutive images in order to determine the depth from the camera of each point in the scene. You can create full polygon models as evidenced by the .ply downloads on the site. There is also a ply2dae convertor there if that suits you better and the in built view is displaying texture obj models (trickier to download because there are 3 files).
You can do full surrounding object models but in the interest of not overloading our server with 1000's of huge images we have limited the upload to small number as a bit of taster. I have run sub-millimeter accurate reconstructions with hundreds of images.
The software is heavily accelerated by GPU processing and would be one of the quickest and most accurate incarnations of such technology available in the research literature.
The underlying tech is very similar to photosynth but we have gone a step further and reconstructed the 3D models of the scene as opposed to some sort of 3D mosaic. We are hoping to commercialise the tech so we have not published the core algorithms. Any one interested?
You take your picture, finesse it, and turn it into a 3D-object.
That last bit is the doozy. If any of them reliably generate a 3D model from a single picture with no human interaction, then... well, that's different.
Rampant carbon sequestration destroyed the Dinosaurs' tropical paradise. I'm here to help repair the damage.
Still sounds like a cat to me.
This is my footer. There are many like it, but this one is mine.
I'm posting this really late in the thread so maybe nobody will read it (or care) but...
If there is one place on earth that is crying out to see this technology used it is the KILOMETERS (really!) worth of intricate stone carvings at Angkor Wat (Cambodia). I've thought about borrowing (stealing?) a friend's $500,000 laser scanner to capture them but the 1) he (his institute really) probably wouldn't let me 2) the thugs who run Cambodia would probably not let me use it without me paying some extortionate amount. There really is no-where else on earth where you can see the results of thousands of man-years of skilled stone carvers. This priceless cultural heritage should be captured before pollutants like acid rain slowly erodes it or thieves literally dynamite it to pieces.
Now perhaps anyone with a good video camera, a steady hand, and a LOT of patience can get this done! Perhaps if this job is too large for any one individual to complete it could be done in sections and the individual video sequences shared over the internet. Anyway, I hope this software is modified to handle video (subject to certain restrictions such as shooting in progressive mode).
Displacement maps and bump maps contain exactly the same information. The only difference is how they are used. I am a professional 3d artist, by the way.
Mod the parent/autor up !! And hopefully his post will help to clear all the misunderstandings.
If I'm not completely mistaken, some video tools already available for quite some time would be able to do just that... e.g., Boujou is able to convert several tracking points in video footage to points in space, in order to later put other stuff in or alter the footage ...