Slashdot Mirror
Video Inpainting Software Deletes People From HD Video Footage
cylonlover writes "In a development sure to send conspiracy theorists into a tizzy, researchers at the Max Planck Institute for Informatics (MPII) have developed video inpainting software that can effectively delete people or objects from high-definition footage. The software analyzes each video frame and calculates what pixels should replace a moving area that has been marked for removal. In a world first, the software can compensate for multiple people overlapped by the unwanted element, even if they are walking towards (or away from) the camera."
Background has to be static for it to work.
Nevertheless, an interesting accomplishment.
Researchers have developed video inpainting to remove the character Jar Jar Binks from the Star Wars Prequals.
Don't the new Galaxy S4 have a similar feature, if I read correctly? Although only for photos.
/ The Arrow
"How lovely you are. So lovely in my straightjacket..." - Nny
I liked the fact that you could still see the pedestrians in the reflections of the display window in the video of the musicians, even though they had been erased from the front end. Like the vampire test, but the other way around.
So combine this tech with Google Glass and identify people you just don't want to see ever again, and you may end up walking right into them without even knowing.
You can't handle the truth.
Ten years ago, I predicted a "nudie button," which, instead of removing people from live video, would simply remove their clothing (through interpolation). I do not endorse the use of such a button on your TV's remote control, I merely predict its future existence.
can you guys remove the motorcycle from this picture?
Where the reflections masked? I would think you could remove them too, you just need to select them as well. I'm at work, so am unable to see the videos till I get home.
Memory is deceptive because it is colored by today's events. - Albert Einstein
You got that wrong. Now they will say see, we told you this existed. But if this is what they are willing to show us, think how more powerful the government version is that they won't show us? They can remove you from one video walking down the street, and put you in the same scene last week (ATM time stamp) robbing someone!
Memory is deceptive because it is colored by today's events. - Albert Einstein
If we extrapolate this, perhaps we won't be able to trust video as evidence any longer, so there's no reason to have all these surveillance cameras around.
This course on Coursera describes the basics of that technology. It's in its final week but I'm sure it will be reissued later on. They have other courses about computer vision announced for the next months.
The course was pretty interesting but you don't really have to do any programming to get a grade (programming assignments are optional). Lucky for me, because I have a job and no time to spend on lengthy programming assignments, but one can't become an expert of that subject just by listening at the lessons and doing the multiple choice quizzes.
Joseph Stalin would have loved this.
People have been doing this for far more than the last 5 years. It is a trivial application of so-called 'optical flow' where motion vectors are used to identify independently moving objects within a scene.
One interesting application (seen, for instance, in the Will Smith film "I am legend") takes video footage of a real environment, and converts the footage into a virtual static 'texture' for the background elements. Artists can then repaint over this 'texture' to add damage to buildings etc. The new texture can now be reapplied to the original footage, so the moving shot appears to show the artistic changes in visual context. Clearly this method will not stand up to the same scrutiny as remodelling buildings in CGI, and inserting them into a virtual set, but it works well for backgrounds.
Films today frequently use a so-called skybox- a 360 panorama stitched from multiple still photos shot on location. This skybox allows a virtual background to be 'projected' behind the actors (say when they are pretending to be on top of a tall building or mountain) that can track the rotational movement of the camera.
The idea of element extraction forms the basis of various camera enhanced video games found on the current consoles. Usually, the technique is the reverse of the example in the article, where it is the background that is removed so that the player may be isolated and inserted into a virtual scene.
Slashdot needs editors that know something about technology, but that isn't going to happen while the owners of Slashdot use the tech stories to draw readers to the constant anti-Iranian warmongering propaganda that appears here almost daily.
There is one if you really cared to look through the second link provided. There's a nice HD video demonstrating the technique.
or Star Wars without Jar Jar Binks
Software of this type has existed for a long time. It's commonly used for rig removal, but can be used to remove any object that is 'outlined' for removal. Next-and-last frame comparison is what 'batch clones-out' the outlined object. It's the same tech that Boujou used (vector analysis, per-pixel tracking via next-current-last frame comparison), but that app is/was used more for creating a virtual camera path for a 3D environment... Mokey was pretty good at this type of object removal, too (it's called Mocha, now - www.imagineersystems.com ). This type of software is pretty common and many companies make their own in-house if they have the need, I'd think. Remember how they removed Denzel Washington's character from the remake of The Manchurian Candidate? The real-time aspect is where we're going.. like The Running Man w Schwarzenegger. They used this type of tech in it, but in near real-time. That's scary.
No sig for you! Come back one year!
... I'd pretend I was one of those deaf-mutes.
This would be a cool usecase. I dont mind JarJar so much though, theres lots of stranger things in starwars, but i'd make a good technology demo.
Hivemind harvest in progress..
I have seen this done in After Effects 5 years ago. In fact I remember a script/template that was floating around that tried to automate it quite well.
Do not look at laser with remaining good eye.
You got that wrong. Now they will say see, we told you this existed. But if this is what they are willing to show us, think how more powerful the government version is that they won't show us? They can remove you from one video walking down the street, and put you in the same scene last week (ATM time stamp) robbing someone!
That isn't doing the conspiracy theorists justice!
These days there is no need to go through all the work of framing you for a crime and then inventing evidence of it. Oh no.
Instead, they will simply say you broke a secret law, but can't tell you what that law is (for national security after all), but they DO have evidence, but can't show what that is for more national security reasons.
A judge will agree that is acceptable, and without seeing the evidence will say it is enough to prove your guilt, and thus you are guilty!
That way they can more efficiently use the secret alien captured computers from the early 50s for reading your thoughts instead of wasting cycles processing video frames.
Keeps costs down. Do you have any idea how much CPU time is to rent on secret alien captured computers from the 50s?? Plenty mister, plenty!
Not sure if you're making fun of conspiracy theorists or making fun of people who call others conspiracy theorists?
What you describe was put into law by the 2012 NDAA. There isn't even a "judge" required.
Government says: "You're a terrorist!" The evidence is secret so you have no right to examine or challenge it. You have no right to legal counsel, no right to ever see a judge or jury, and it's off to prison you go for as long as the government says. Or maybe they just kill you, which they also claim the power to do.
Think outside the box - just delete episodes 1-3 inclusive.
Or nuke it from orbit, it's the only way to be sure...
the sponsor which pays for the broadcast paints over the stadium ads for the sponsor's competition. for example: in an auto race, the TV sponsor, Budweiser, paints the Miller car.
Anyone reminded of the Laughing Man from Ghost in the Shell: Stand Alone Complex?
A hacker who was able to hack the cybernetic vision of others in real-time to make himself invisible...
** We're sorry, the author of this thoughtcrime has been vaporized. **
...make sure you turn it off before going into a Walmart or KFC etc.
If that happens, I predict some nerds with Google Glasses gouging their eyes out...
They'd have to develop a similar algorithm for audio though.
I think I'd find a mute Jar Jar far easier to tolerate than an invisible one.
I believe this is just an extension of the research done by this guy.
Now I have installed an Active People Filter to remove people from all those Internet videos and movies and DVDs I watch. Along with speech and vocal removal filters for music, speech removal.
The filters have learned what people look like and do a fair job at stamping them out of still images too.
Now my life is like a series of paintings in still-life. Sitcoms are rooms of silent furniture and no stupid laughter, the Olympics and football a breathtaking vista of grand spaces and odd sporting equipment lying around.
I take long walks through the Internet, I am surrounded by the sounds of nature and gallivant in sun dappled glade. All is well.
Now I am alone in the whole world. And then the silent spell is broken shrilly. Microsoft wants to install updates on my computer.
My life is light, waiting for the death wind,
Like a feather on the back of my hand.
Dust in sunlight and memory in corners
Wait for the wind that chills towards the dead land.
~TS Eliot
<blink>down the rabbit hole</blink>
What does this have to do with the number of pixels of the video format? Why even mention the words "HD"? Is this software unable to operate at higher compression formats or smaller aspect ratios?
... and replace him with walkie talkies so as not to scare the children!
Sure enough, the cow costume was hanging up next to the superhero outfit and sailors uniform. (S,Spud)