Image Processing By Example

Aaron Hertzmann writes: "My collaborators and I will present a paper called Image Analogies at SIGGRAPH 2001 this summer, where we describe a machine learning method for 'learning' image filters for example. For example, given a Van Gogh painting, the algorithm can process other images to look somewhat as if they were painted by Van Gogh."

"It can also 'texturize' images based on a sample textured image, e.g. to create landscape photos. It can do many other types of filters, as long as you give appropriate 'before' and 'after' examples to learn from." I especially like the idea of inferring a high-resolution image from a low-res one. The software is available for Windows and Unix, and "the source code is freely distributed for educational, research and non-profit purposes."

Comments

Wow

[AntiFreeze]

This is actually quite impressive software.

What I've started to wonder is where else it's underlying principles could be used, or where this sort of technology could lead in the future.

Could it be used to analyze text from certain authors (hey, text and art are no different to a computer - treat words as "pixels" and sentences and structures flow like colors) and mimic their style? Could this one day be used to turn my dull crud into something Fitzgerald or Hemingway or even Asimov or Heinlein might have written?

I also have the following few questions:

• What happens when one feeds a Van Gogh through the Van Gogh filter? Does the resultant image change much?
• Does the program apply the "filter" differently depending on what type of input it encounters, or is the same method applied to all input?
• Conversely, can the program be used to recognize when a work is of a certain artist?
• Or can it be used to see if an image has already been passed through a certain filter?
• Are there cases which cause the method to fail or create an undecipherable image? And if so, are these cases unique or do they conincide with a certain type of artistic style? [e.g. Monet -> Van Gogh just won't work right?]

I think that sums up my feelings. This stuff is really impressive guys, I hope the conference goes well.

---

Close, but no severed ear...

[garagekubrick]

This is interesting and all well and good, but ultimately where it fails is that the produced image is entirely dependent on the original photograph's perception of the world. A reproduction of an image through halide crystal activation, which is enough for human memory and recognizance, but it lacks the true meaning of the artist. Van Gogh never used contrast or flat lighting as exhibited in the source pictures, and he often burst highlights with striking colors that may not have been actually present to his eye. It's what seperates him from a Turner - not just his brush stroke or how thick he worked in paint but how he saw the world. It's pretty churlish to adopt the first real expressionist painter (who deliberately attempted to paint their perception of the world rather than reproduce it) as an example of this algorithim, as the resultant images show that without an interpretation or perception this is pretty useless stuff. All I see here is a souped up photoshop filter.

Van Gogh

[Greenrider]

Terrific...a computer that behaves like Van Gogh.

Next thing you know I'll come home from work to find that my PC has severed its own mouse cord in a fit of psychosis.