Slashdot Mirror
Content-Aware Image Resizing
An anonymous reader writes "At the SIGGRAPH 2007 conference in San Diego, two Israeli professors, Shai Avidan and Ariel Shamir, have demonstrated a new method to shrink images. The method is called 'Seam Carving for Content-Aware Image Resizing' (PDF paper here) and it figures out which parts of an image are less significant. This makes it possible to change the aspect ratio of an image without making the content look skewed or stretched out. There is a video demonstration up on YouTube."
I thought JPEG already did this.
The author's website was pegged serving that 20MB PDF before slashdot got ahold of it, I doubt it'll survive now. The paper is also hosted by the ACM, if you're a subscriber.
I Browse at +4 Flamebait
Open Source Sysadmin
I really like the way where the people are erased and you can hardly notice it. But the when it comes to the faces, the algorithm seems to need more work.
I wonder when we can get their software that does this...? The "erase" function almost seems like a better then image inpainting.
Watching the video, this seems pretty impressive. It's also neat to see them use the same technology to remove items (people in this case) from an image with only about 10 seconds of work.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
It seems like a little bit of work is left to make it as completely automated as you would need to have it just "always work" on any platform or device, but it seems like they're already working on that...
;)
:)
Other than that though, that's pretty awesome... I'm sure there's more instances where it doesn't look right than what they showed, but it's definitely cool how well it works as it stands!
I can imagine it would be extremely useful for ex-boyfriends or ex-girlfriends; just load up all their photos of them and their ex, wave the magic eraser, and *boom* you don't have to delete all your old vacation shots
I wonder how well it would work for the porn industry too; nice automatic resizing of breasts without ruining the picture! Fetishists will be SO happy!
ìì!
Finally, a way to reduce the space between surgically augmented breasts and lengthen wangs on Flickr!
Laughter is the Spackle of the Soul.
Okay, I get that they remove the pixels with least energy, so the unimportant information is lost when shrinking, it kinda works, looks a bit strange, but it's okay. however, when they make an image larger they also add the least information so you end up with a large image- but the useful information is the same size and the extra/useless low energy or background gets duplicated- to me, I think thats kinda pointless, I mean, you're adding stuff you've analysed and found NOT to be the focus of the picture. This may work for pictures with no obvious background, but lanscapes like one of the examples, have such an obvious background that only that gets enlarged and just gives you more background. You may aswell just add a nice blue frame round the edge of the picture to make it fit.
I'm really impressed. Again, maybe not too hard to implement at first, but probably damn hard to get working perfectly, and I might just be ignorant (and I'm entitled too, it's far from my field of work), but I've not seen anyone doing it before.
So does this mean you're taking some of those words away?
There are probably a few situations where the 'unimportant' bits of an image are still as relevant as the rest. Sports photos for instance - especially those played on grass - would not give you a true picture (literally) of what's going on in the scene.
This'd be good for reference photos - like the animals at the start of the YouTube video, but applications where precision and distance are required wouldn't benefit. Nice bit of work though and I reckon with some smart scaling embedded too (rather than its 'folding effect'), it'd cater for most image retargetting requirements.
This technology could render very visually-convincing (but not computer/analytically convincing) image censorship or alteration. I am strongly reminded of this example of photo-editing from the 1940s:
h es/vanishes.htm
http://www.newseum.org/berlinwall/commissar_vanis
I don't mean to instigate a knee-jerk, authoritarian censorship discussion. I think it's obvious that this technique is just plain cool and has great potential for beneficial use, even if it might be used for ill. That's just an intersting historical example that it might have made easier in execution (har-har; gallows humor and pun 2-for-1!).
This method is quiet interesting, though it falls over in situations where the detail level
or entropy of the background is as great as the foreground. Also the paper doesn't go into
too much details about the dynamic programming approach they used to find the path of least
energy, I guess that aspect of it is patentable. Another thing they could investigate is the
use of diagonal seams instead of just staggered vertical and horizontal seams.
All in all a very interesting read.
Arash Partow's Philosophy: Be a person who knows what they don't know, and not a person who doesn't know.
10 Seconds of work there, most probably a good deal longer finding a picture that is easy to do it to...
The technique was already invented by the Soviets in the '30s:
Before
After
Insignificant person removed.
Ths s rly gret !
\u262D = \u5350
"It's also neat to see them use the same technology to remove items (people in this case) from an image with only about 10 seconds of work."
Newspapers rejoice!
Verily, no good will come from this!
Begone, Ye image distorters from Hell!
Although they demonstrated on Windows, a friend of mine is one of their graduate students and was peripherally involved. He said it was originally developed as a GIMP plug in, but moved to a separate Windows app to show off the realtime resizing, etc. Hopefully they'll release the GIMP plugin? More likely Adobe will write them a check and license it to make sure that never happens.
Do you even lift?
These aren't the 'roids you're looking for.
Shrink the rest of your body, and increase you penis size by up to 20 pixels!
I find a small irony in the fact that the video is posted on youtube, a site which stretches and squeezes video to fit into a 4:3 aspect ratio
What is unmentioned in the PDF is that in order to test significance-measuring algorithms for parts of an image, they used goatse. After thousands of tests they were only satisfied once the method returned a blank image.
Those performing the evaluation of the test results before the final version were forced to be institutionalized.
I would like to take this opportunity to personally thank those brave soldiers for their unblinking fortitude in this ultimate self-sacrifice. It's times like this when I become truly aware of my own gaping inadequacies, and feel the deep, deep obligation to rectify my own short comings.
Now that I think about it, I'm pretty sure everything I just said is completely wrong.
Yea, even the Fat Unemployed Internet Artiste has taken exception to this devious work of The Prince of Darkness!
"More likely Adobe will write them a check and license it to make sure that never happens."
Is that check going to cover the removal of their paper from above and the ACM archives, let alone OUR archives?
I TOLD you it's 10 inches! See? SEE???
(Yes, I know, this thread is worthless without pictures)
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
The scheme relies heavily on edge detection. For the simple images shown in the demo (beach sccenes, open sky, plain color backgrounds etc) edge detection is easy. For more realistic photos with complex backgrounds, the approach breaks down.
This introduces very obvious artifacts, and looks worse than simply distorting the aspect ratio. At least in that case you can undo the distortion in your head. Here you are throwing away information at a much higher level than just the details of textures. I'm sure it would look even worse on images that weren't carefully chosen for the demo.
I thought it was pretty cool, so I made my own version after seeing the video. It obviously won't be as awesome as their one, but if you want to play around with it, you can get my C source and have a play around. It is GPL3.
Take a look to the end of the video, this one is an awesome method to remove objects from pics. Very interesting.
Copyright infringement is "piracy" in the same way DRM is "consumer rape"
In Soviet Russia,
eraser erases you.
What about artistic photographs? Most photos in that sense are planned to have a certain layout, composition, empty spaces, etc. Say I make a nice panorama shot with a 6:1 aspect ratio. Now my photo that took careful planning is reduced to a 4:3 with all the 'unimportant' spaces removed? Maybe it's just me, but there seem to be lots more instances where this would hurt than help. Journalistic images? Sports photos? Oh, the image can't fit, let's get rid of everything between the 50 and 20 yard lines. There aren't any players standing there. I really only see this being beneficial for web ads. Instead of creating square, vertical, and horizontal versions of the same ad, just make one and let the image be 'resized' accordingly.
Higher quality mov video here(direct link) if you're interested. (Or you know,if you just want to completely hose their server...)
---
Contronyms: for people who are chuffed by antonyms
---
That eraser... Start using "Lossy" tecnology for sum good...
because Youtube has outstanding video quality. I mean, Youtube does to video what the Playstation does to audio.
By far the most intersting part of the youtube video was the removal of the two out of five runners on the beach. I realize the removal would probably be a lot more noticeable if the image was higher detail, (what we get to look at on youtube is very low res compared to many practical applications) but it's still pretty stunning how easily and quickly the images could be altered. Very little post-removal editing would be required to selectively and convincingly remove content.
There's no reason why they could not use this same method to insert content. I'd like to see them add a few people on that beach for example. Simply stretching an area by cloning the region could do so much more if you could define what to put into the region besides similar content.
I work for the Department of Redundancy Department.
... not to be confused with Adi Shamir (the cryptographer).
http://outcampaign.org/
You can't take the sky from me...
Too much caffeine in the blog, couldn't sleep... I can't get my hand on the paper but the youtube presentation was extremely clear and I just wrote this C code based on libgd2. Basically it lowers the height of an image by 1 pixel, you can run it multiple time to remove more line.
http://rafb.net/p/jinioy45.html
(yeah my coding sucks but it produces awesome results and I reversed engineered the algorithm from youtube so please grovel...)
I'll improve it soon to remove an arbitrary number of line, horizontally or vertically
- no recalculation of gradient, only the gradient near the line needs to be recomputed
- precomputes a file that store the order of the pixel needing to be removed
I need help with something though, I understand how the algorithm can precompute a file which says in which order pixel should be removed, but I don't see how this can work in *both* direction. Suppose you want to reduce vertically and horizontally at the same time, the horizontal change should completely break the precomputed vertical changes. How would you handle that?
\u262D = \u5350
"Which reduces the total size by the same amount, but retains more information than treating every bit of information the same."
So what MPEG does for video, this does for still images?
I presume, as soon as a movie version is available. Not that they cannot live without the technology now, but they can rightfully say when distorting things that they leave things into perspective.
Bert
...as seen in the last part of the video: just separated ? Simply scan your favorite photographs and resize your ex away ! Great stuff, although - ofcourse - on Slashdot this would be a bit useless ;=)
Slashdot: stuff for news, nerds that matter, matter for news, stuff that nerd
...what this technology could do to Goatse... I leave the end-result as an exercise to the more mentally disturbed individual... =(O)=
so...fucking...cool
Clicky
Tm
Support TBI Research: http://www.raisinhope.org
Seems Orwellian to me. Think of how Oceania's Ministry of Truth could have used this in its rewriting of history.
It seems to be a way to make distortions of the original information less noticable and therefore more esthetic. Is that really what we want?
I don't like to derail the conversation. However, here is yet another brilliant Israeli innovation. One of thousands made in a country under constant threat, with an absolute commitment to research and intellectual advancement. However, like Israeli cancer treatments, high-tech and agricultural innovation, will you hear much about this in the media? Of course you won't.
Heaven forbid however, that Israel should shoot a few terrorists borne from a culture whose only significant export is Jihad. Boy will you read all about it then. Such a shame.
Your comment seems to be similar to the headline on tabloids.. Just because a technology could be used for negative purposes does not mean that it should not be developed.. If your reasoning was used, we should have all been living in caves by now..
By your reasoning
Cars can be used by criminals to travel faster.
A knife can be used to kill
Electricity can be used to kill
Computers can be used by the govt to collect more information abt us effectively
Is that really what we want?
see the flaw in the logic?
"Slashdot - News and Chat Sites Deviant". (Click "homepage" link above for details).
I see a lot of bitching and moaning about loss of information, about context, etc. People, if you trust something made with pixels you are already way behind. The wise person must be a skeptic, of all things. This technology, like every other, has its place and will be suited for certain things and abused to do others. So I guess what I am saying is it surpises me that a community like Slashdot is getting so many comments point out the obvious. I thought we were progressive enough to avoid that.. guess I should've been more skeptical.
I do not respond to cowards. Especially anonymous ones.
All but one person I know with an HDTV stretches their 4:3 content to fit the entire screen. It drives me nuts because nothing looks right. I wonder if this method would produce more natural looking images that fill the screen with 4:3 content. If so these guys could make a fortune licensing to the HDTV manufacturers!
The easiest way to insert additional content would seem to be to paste an image of the additional runner in with a similar background, then mark the seam as "unimportant". Similarly, to improve gluing together panoramas you could mark the seams between individual photos as unimportant.
Adobe put out a lot f advertisements targeting artists pushing PDF as a superior format to HTML because with PDF the reader can't mess with your layout.
When I looked at those ads my thought was "man, the PDF version is unreadable at this resolution, but the HTML might look ugly but I can read all of it".
As PDF came into use my fears were realized. PDF documents are all too often unreadable on anything but the largest screens, and I sometimes have to blow them up even on my 23" cinema display.
When it comes to preserving your artistic intent, the alternative to "losing layout" may well be losing all the content.
It's still impressive, but when you pause and examine the frames in the hi-res movie you see a lot of distortion. The people on the beach look like famine victims by the end of the sequence. Boosting the importance of the other figures would help.
This is a great paper even though the implementation is fairly simplistic. A good idea is one which makes you immediately think of a dozen ways to change it. This is a good idea.
Consider the simple example:
hello world
32332 32333 = 27
after three cuts we have:
hll wrld
333 3333 = 21
The next cut could be bad. But, with error diffusion
hll wrld
445 4433 = 27
That's better. Now, replace letters with pixels in an image. Then apply a better diffusion method (linear or exponential falloff).
There are lots of fun ways to modify this basic algorithm.
For me, it's a matter of how we broke up. Cheating hurts. Even now, I don't hate her. It's been almost a freaking year, she's taking my son away from me hour by hour (I'm down to less then ten a week), and even though she's settled on one guy to bang (as opposed to when we first split up when she flitted around like a fucking humming bird), it still hurts. I hate her because I hurt. I love her still today. I guess you could call it mixed feelings, eh? Wow. I'm a bitter fuck.
use this for torturing Palestinians?
"It's not compression as we know it, Jim"
No its not compression as such, but it looks in theory, possible to use this methodology, as another way of doing lossy image compression. It would be interesting to see what kind of results are possible by using this as a means of image compression?.
It looks very interesting research work. Perhaps it could also be combined with other forms of compression, to get even better image compression ratios?.
There are 10 kinds of people in the world... those who understand binary and those who don't.
While I disagree with your complaint, I have to say I'd like to try the famous "erase that darn Trotsky" trick.
To do list for Windows
If you want an app (for Windows and Linux) where you can try out this new resizing technique hands-on, take a look at http://www.thegedanken.com/retarget/ - very cool and comparably good performance as well.