Slashdot Mirror
Increasing Video Detail Using Super-Resolution?
Cecil Esquivel asks: "I'm looking for ways to increase the quality of video by using super-resolution algorithms which use the visual information across multiple frames of video to increase the resolution of individual frames. I have found very little on the web that can do this effectively for the entire length of video. There is commercial software, VideoFOCUS, which produces hi-res stills from video, but doesn't seem to have a product for producing hi-res video from video. There is a thesis from Duke U. which is 6 years old, monochrome only and is mostly proof of concept.) Anybody out there have more information or is willing to help me develop some software that can do this? Darwin/Mac OS X solution that can work with QuickTime DV, preferred." Typically, super-resolution uses image samples generated from low-resolution and high-resolution samples of the same source, which is then converted into source independent information that can be used to increase detail for other low resolution sources. Has anyone seen programs that use super-resolution techniques for increasing the resolution of your typical digital video clip?
Looking at the state of deinterlacing technology and some of the "detail enhancing" resizing filters would be a good area for study.
I'd *love* to see this used to help correct data errors in video streams as well. A DirecTV receiver with this built in would be cake++.
Just imagine, you too could have a Beowulf cluster of X-10 cameras...
This user account is inactive account replaced by the PDA
I imagine increasing the spatial resolution for a frame of footage decreases the temporal resolution of the set of frames. My gut feeling is that you would see more after/pre-image than in the source.
That sounds pretty interesting. Typically any software that deals with movies tend to reduce quality or at most keep it the same. Is this the first time software can actually be made to dramatically increase the quality of a movie, even if the source sucks?
There is commercial software, VideoFOCUS, which produces hi-res stills from video, but doesn't seem to have a product for producing hi-res video from video.
You can't figure out why that is?
http://www.salientstills.com/product/pr
So...just run it through video focus, make an avi with all the frames, add the original audio track, then compress them using some video compression...you have yourself a high res video!
I'm guessing something will look strange, like funny blurs, or background motion or something. Who knows. I don't think it would do good with fast moving objects in the video...probably blur it real bad.
It's worth a try though.
Okay. Let me debunk you. What you are describing is impossible. You can't get more "hidden" detail out of any image. What the poster of this article is describing, is using additional detail from nearby frames in a video sequence to gain additional pixels. In effect, a higher resolution image is created because the amount of data is there. It just needs to be juggled properly.
This principle applies to astronomy. If you want to find faint galaxies, you simply increase the exposure time for your video device (CCD or photo film) and make sure your tracking is right. Then you just let it go and in effect do the same thing as is done here digitally.
Ok, after googling for a little bit (I will never MSN anything!), I found a site who has done just what you were describing with MPEG video.
a /
http://www.ai.mit.edu/~brussell/research/sres/dat
Anyways, it seems that without proper filtering, the output looks REALLY weird. (look at they guy walking in a circle in front of the quilt) It seems that the motion vectors from the MPEG get taken in as part of the detected edges somehow! Thus, this would be most useful for uncompressed analog video as an input.
It's got pretty lame security. Any hacker that can access the Internet can connect to the NSA server and download top secret documents without even having to know the PASSWORD.
I realize the root was probably a troll (the AC, not you). Also, the submitter was technically asking about nearby frames. Realizing that, however, I'll respond anyway, because I feel the technique is at least worth a mention:
There is nevertheless a form of super-resolution which works on standalone single frames. It depends on what you mean by "additional detail". Normally when magnifying images you would use nearest-neighbor, or better yet, bilinear interpolation. But a magnification using these methods will still look blocky.
Now suppose you first detect the edges in your lo-res image. When you magnify, use that information to determine which pieces of the newly created pixels should be colored what. You can use a similar concept for textured areas without edges. The result? Through some simple assumptions, you get much better magnification than through other schemes.
If a star in your original image is a 5-pixel plus pattern, a super-resolution image might turn it into a circle. Though not strictly accurate, this is in fact not a bad result in many situations. However, you're right that there is really no more detail; if a star was not visible in the original image, it never will be.
Dude! Can you like get me some pictures of my neighbor's yard. SHe is like so hot and shit. And she sunbathes in the nude all the time. If you can email them to me that would be like awesome.
This seems to be implying that it can take a video, run it through the process, and spit out a higher quality video...
It looks like you're claiming that this isn't possible. Technically, you're right --- you can't necessarily get bits from the air. However, you're ignoring the fact that video contains a good deal of redundant information. Video codecs try to avoid including redundant information this, but only do so in the simplest ways.
Suppose I show three consecutive frames of an eggbeater. Now, if you knew nothing about this video, if the frames had no correlation between each other, and were just each random static, then I couldn't improve the quality at all.
When I'm showing an object that I know is the same object from frame to frame, however, I can use detail from each of the frames and add it to the other frames.
May we never see th
This might be a slightly related note -
I know many web sites have a video available in multiple formats; i.e. real, mp4, and sorenson, or whatever.
I'd like something that takes the different streams and assembles for me a high res final product, based on the idea that there are going to be different details available in each compression scheme... does this make sense? Does it sound like a reasonable idea?
Spoon not. Fork, or fork not. There is no spoon.
When I worked at HP (R&D) we had some pretty cool stuff for doing very similar things. Most of it came from Purdue or Palo Alto and then we made it actually work. All patented as fast as we got them working and sometimes sooner. Betcha you're going to run into that in real life. Even if you think of something 'novel' the patent's probably already filed or granted for what you're wanting to do.
Here. Granted this is about creating high resolution stills, but read the ideas in the /. comments.
I remember years back seeing a demo by Prof Barnsley where he demonstrated a low resolution image of a parrot that he had compressed using fractal compression. The resultant image compared with the original could be blown up in size to much higher resolution than you would expect and it seemed to add more detail than there was in the original. There was talk at the time of using this as a method for upconverting old video material to high resolution for projection screens. Not really a cure for noisy video of course.
Is this the first time software can actually be made to dramatically increase the quality of a movie, even if the source sucks?
Image processing systems have contained "noise reduction" processes for a long time. However, no software can save a bad script.
Will I retire or break 10K?
what's the answer??
I've been working on this for some time and have had a decent amount of success using multiple captures of digital broadcast (broadcasts are still generally encoded on the fly, so each stream will be different). This can also be applied (with great success) to analog captures of broadcast video. Just sum each capture with equal weighting and you'll be surprised how much detail you can gain. Of course, this won't work at all if you are trying to improve a DVD, since you have only the one stream.
If you want to combine frames temporally, a temporal filter will do this - that's what it's for. I tend to dislike temporal filters because they add all sorts of other artifacts I find exceedingly objectionable, but if you are working on relatively low motion video you can gain a bit of detail like this. I can't comment on mac software because I don't have one, but I've worked a good bit on and with AVISynth and I know a couple of people have been working on temporal filters that use motion vectors. I rolled my own, but it's a macro op specific to avisynth and I don't know if you could duplicate all the specific filter behaviors using mac software. Maybe if someone would port ASVISynth to mac or linux...
What you're looking for can be done by VISAR as displayed here
Although, as it's apparently only available though Intergraph I'm sure it's rather costly.