PhotoSketch Image Manipulation Tool Taking the World by Storm

PhotoSketch, a new image manipulation program that combines stick-figure sketches, internet image search and pattern matching, seems to be spreading like wildfire. Created by five Chinese students at Tsinghua University and the National University of Singapore, the tool takes a basic sketch and simple labels and turns it into a polished image. "Although online image search generates many inappropriate results, our system is able to automatically select suitable photographs to generate a high quality composition, using a filtering scheme to exclude undesirable images," say the PhotoSketch team in an abstract outlining the tool. "We also provide a novel image blending algorithm to allow seamless image composition. Each blending result is given a numeric score, allowing us to find an optimal combination of discovered images. Experimental results show the method is very successful."

Comments

correct links

[sopssa]

Since the link to homepage in the article is some old-dated one, here's a correct one:

And the binaries (it's a few command line programs, so no fancy UI)

Re:correct links

[sopssa]

http://cg.cs.tsinghua.edu.cn/montage/main.htm

Damn slashcode!

Re:How does it mask?

[StreetStealth]

It appears from the video that it's running a fairly sophisticated series of algorithms to compare backgrounds and determine how difficult it would be to do a convincing mask-out of the foreground object, of which it appears to have a sort of heuristic expectation of shape from the user's sketch.

For instance, if your background is a grassy field, the user has requested a dog running, and you have a photo of a dog running over grass and a dog running over pavement, the grass one will allow a greater margin of error in the masking and thus it gets selected.

Overall, this looks like a fantastic step forward for computer vision, bringing the computer ever closer to the non-Cartesian way our brains see.