Slashdot Mirror

← Back to Stories (view on slashdot.org)

Algorithm Seamlessly Patches Holes In Images

Posted by kdawson on from the mind-the-gap dept.

Beetle B. writes in with research from Carnegie Mellon demonstrating a new way to replace arbitrarily shaped blank areas in an image with portions of images from a huge catalog in a totally seamless manner. From the abstract: "In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database that are not only seamless but also semantically valid. Our chief insight is that while the space of images is effectively infinite, the space of semantically differentiable scenes is actually not that large. For many image completion tasks we are able to find similar scenes which contain image fragments that will convincingly complete the image. Our algorithm is entirely data-driven, requiring no annotations or labelling by the user."

9 of 198 comments (clear)

  1. Re:Finally... by mwvdlee · · Score: 3, Insightful

    The algorithm requires images of similar content which can be used to fill the holes.
    Where are you going to find such images?

    --
    Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
  2. Re:Immediate Application by CaptainPatent · · Score: 4, Insightful

    Unfortunately, I think this particular algorithm would need a base set of data to begin working. While I'm sure portions of this algorithm could be implemented for such an application, it seems a base set is needed in a single image, therefore a full blank screen from a dropped frame or damaged images showing bad colors would not be successfully mended.
    If, on the other hand, you were a movie producer and needed to get rid of the frame change holes after loosing the master print of a film, you perhaps would be able to use such a program to mend those holes in the upper corner.

    --
    Well, back to rejecting software patent applications.
  3. Image compression? by grimJester · · Score: 4, Insightful

    If any hole in the image can be filled with a part of another pic, can't you compress an image by replacing one piece at a time with a reference to a patch? Also, how about replacing with patches of higher resolution than the original? I realize it would all be technically lossy as hell, but the compression artifacts should not be very noticable to the human eye, right? Additionally, how about using this for movie compression? Filling in based on info from previous and next frame.

    I may have to actually RTFA this time.

  4. doesn't generate new info by 192939495969798999 · · Score: 3, Insightful

    It takes an existing image and finds a very similar image in a huge catalog, then adds in a similarly-shaped piece to the existing image where applicable. So it's more like a puzzle solver than an image completion engine. If you don't have a huge, huge catalog of images, it won't really work for any given image as well as their samples.

    --
    stuff |
  5. Re:GREYCstoration by cluke · · Score: 2, Insightful

    Are you kidding me? Removing stuff like fences and overlaid captions from photographic stills and making a damn decent fist at filling in what would be underneath? You are a hard man to impress!

  6. Re:Finally... by kayditty · · Score: 1, Insightful

    Fake celeb slips will of course come first, but why stop there? That cute girl at the coffee shop? Snap her with the camera phone, erase all those pesky clothes, and let this algorithm do its thing. Of course, I could also see this used for more nefarious (even "sick") purposes... Ex-GF cheated and you don't have any nude pics to release to the web? You do now. And if you "repaired" a fully-clothed original of someone underage, would it still count as child porn?
    You could also just.. draw these images. It's almost exactly the same thing, except the computer-assisted method takes less effort (and talent).
  7. Re:Let me know when... by UbuntuDupe · · Score: 2, Insightful

    Very honoroble Locster:

    I agreeth with your merriment regaurding such tinkery. I myself saweth the play "InfoTrek" about the 21st century, in which a character, upon wishing to lern of a song he herd a few worrdes fromme, simple provided those words to his "computer" and shortly thereafter, generated an image sequence of a bande performing that song! I neerly interrupted the play with my laughter!

    --
    Apology to Ubuntu forum.
  8. Re:What does "semantically" mean? by Venik · · Score: 2, Insightful

    I think this type of classification would be pointless. Even if the computer could somehow differentiate between photos of boats and photos of men in suites floating face down in the bay, this would not help with seamlessly patching holes in images. But the truth is, while software can recognize particular elements of a photo, there is no understanding of of the subject or its context. Thus a missing mountain range in the background can be replaced by an ocean or a parking lot. Whatever categories this method uses, these are not categories humans use. It is based on mathematical analysis of the image data. "Semantic" is just a word developers like to use, just "AI", or "knowledge-based computing".

  9. Re:Finally... by sbate · · Score: 2, Insightful

    I got this in my head the other day and obsessed about it. I thought this is it. The end of posting any pictures of my kids on line. I thought soon I will want those button sized anti-camera diodes pinned to their shirts like little Orthodox crosses. Of course I thought it through and well what are you going to do? The best thing I can think of is to actually ask the cute girl if she has any pictures of herself naked she may just have some you never know till you ask....

    --
    Added Pressly: "Oh, and by the way, milk is nothing but liquid meat."