Slashdot Mirror
2D To 3D Object Manipulation Software Lends Depth to Photographs
Iddo Genuth (903542) writes "A group of students from Carnegie Mellon University and the University of California, Berkeley have developed free software which uses regular 2D images and combines them with free 3D models of objects to create unbelievable video results. The group of four students created the software (currently for Mac OS X only) that allows users to perform 3D manipulations, such as rotations, translations, scaling, deformation, and 3D copy-paste, to objects in photographs. However unlike many 3D object manipulation software, the team's approach seamlessly reveals hidden parts of objects in photographs, and produces plausible shadows and shading."
No longer is it Photoshopped, but instead we say it's been Carnegie Melloned.
I don't do anything like this for a living, but I must say I'm impressed. I'm fairly certain someone will say this was done back in 1997 though so it's nothing new.
Tic-Tac-Toe, Global Thermonuclear War, and relationships all have the same winning move.
How can images be admissible in court in our modern technological age of 3d manipulation of 2d images? Sure, they still have visual artifacts (like in the video presentation for this technology, when the airplanes are turned into 3d, their propellers are not changed, the same image of a propeller is kept for 3d model as was on the original 2d picture) but eventually all of these will go away, it may become impossible to detect that an image in front of you was manipulated at all.
Eventually this will also apply to video footage.
Add the digital augmentation of reality into the mix (Google Glass, etc.) and you can't rely even on the recorded information. We know that people are not good at remembering the details of what they saw, but if cannot be sure of images and video (and obviously audio) either, then this type of data becomes useless in courts. That's an interesting development in itself, never mind the fact that you can now turn a picture into a movie if you want.
You can't handle the truth.
Pictures and video are used in court but someone testifies that it hasn't been modified. If the defense argues that it has been modified then a jury weighs the merits of that claim.
While those results look impressive, in some of the demos where objects are seamlessly moved around, how are they filling in the original background (or what looks like it)? The video largely explains how the model is textured, lit, environment mapped, rendered with shadow projection with calculated perspective and depth of field, but I didn't hear much about re-filling the background. I assume they're cloning or intelligently filling texture ala photoshop, or perhaps in all cases where they showed something being animated it was a new clone of an existing object into a new area of the photo?
ACADEMIC OR NON-PROFIT ORGANIZATION NONCOMMERCIAL RESEARCH USE ONLY
I am not really here right now.
...informations to better builds the good!
Bad informations with for the good people so making of the understanding isn't!
Blogging because I can...
Comment removed based on user account deletion
The problem is that as technique improves, the theory that the photo/video was altered in a way that can't be detected becomes ever more plausible.
It was easy to take the witness's word for it when the alternative would involve millions in equipment and would likely be trivial to detect.
This looks pretty cool, but I have a lot of questions.
On it's surface, it looks like a lot of the results they're getting wouldn't currently be outside of the realm of student level work, such as the simple practice of projecting and baking textures into materials from photographs, the innovation seems to be that they're quickly automating a lot of that stuff into a UI with a fast lighting solution. One of the things I find most rewarding about 3d is that you sometimes get this huge burst of increased productivity, as long as you're not too bummed out about things you've spent time and energy learning how to do becoming obsolete. This isn't that different, fundamentally, than setting your viewport background in Maya, 3ds Max, etc. to be a photograph after properly matting your foreground objects and projecting textures with adjusted reflectivity, just without all of the manual tediousness. Also, there's also been other, similar work done on the subject, that I've heard of, but this still looks pretty neat if it's something you can use right now without a billion dollar computer.
One of the big things this tech might be doing is streamlining the process of match lighting. I personally can't wait till the major software packages have integrated solutions for easy lighting from photo sources. Currently the setup for photo matting is a pain, it requires stitching together panoramic photos of reflective chrome spheres - on location - or carefully using observation skills to recreate the lighting by hand (which can be very difficult for glossy surfaces). It would appear, however, that we're on the brink of not needing those things anymore. That being said, this software still has a bit to go, however.
For example, the lighting information baked into the diffuse textures of the objects, in these examples, does not appear to be dynamic - if you watch the taxi-spinning segment you'll notice that the specular highlights on the hood of the car do not properly update as the orientation of the model changes in relationship to the light sources, making the taxi appear to have white paint streaks once rotated out of alignment with the light source. The car falling off the cliff example is probably the most apparent in final results, as the strong baked lighting makes the coloring look off. The way we 3d artists get around this problem is to eliminate the lighting information in our diffuse textures as much as possible before reapplying them as flat color, and then let our lighting rigs take care of the reflections, shadows, and such. As they mention this software doesn't support transparency, and I would guess is rendering everything as matte objects, meaning the renderer probably isn't robust enough to handle anything coming close to complicated reflections/refractions and so on, making this software's usefulness very situational, currently. It would be a great way to quickly populate photos with hordes of smaller objects, for example. However, with a more powerful renderer, feature wise, this tech could be really useful for the Photoshop crowd. I wish Autodesk/Mental Ray would focus on stuff like this instead of the boring crap updates we usually get (Maya's new fluids are pretty cool though, tbh...).
Yes. I bought a bunch of hardwares the other day! Either that, or you are one idiot.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
a jury weighs the merits of that claim
Unfortunately, I wouldn't trust the average juror to weigh a head of lettuce.
-
- - You can't take something off the Internet! That's like trying to take pee out of a swimming pool.