Slashdot Mirror
Colorizing Images and Video by Scribbling
Guspaz writes "Up until now, colorizing a video or image has been a painstaking and mostly manual task. However, researchers in Israel have come up with a new way of colorizing images just by making a few scribbles. The technique works on the premise that 'neighboring pixels in space-time that have similar intensities should have similar colors,' and also allows colorization of videos by 'marking' about one in ten frames."
I'm colorblind! That whole site is very confusing! :P
Back when voyager(s) were flying by planets I recall reading how the cameras worked. From what I remember, the cameras actually capture images in black and white. The cameras can detect much more "color" depth than color cameras could (or can?). The scientist would process the pictures to colorize them, you identify one area of color you know and the algorithm would process the rest of the 1 billion shades of gray into a color mapping for people to view. Now why cant identify this gray shade as the color red; anytime you see it then that is red. Go on for each color spectrum or have the algorithm adjust what a little red hue is for a given little hue of gray. It appears that is what the scribbles are doing which is quite clever and the algorithm doesn't have to work (guess) so much.
Personally I can't wait until there is a Photoshop filter for this. :D
Support Liberty, Support Ron Paul
from looking at the before and after images, this technique looks pretty cool and will probably have applications for recoloring an image that is already color. For instance, the image where he recolors the fabric on the chair.
Video Compression !
Only save the intensity channel and a few bits of markup and you compress the stream quite a bit.
The bikini - security through obscurity since 1943
Now it's even easier for corporations re-releasing films to completely destroy the original beauty of a film by adding unnatural and unnecessary color!
Coming soon, new dubbing techniques will allow easy substitution of the original actors' voices and dialogue with trite teen-angst to appeal to younger generations.
...and raise enought money to rebuild the smoking ruins of their server room.
I'm not wrong. You haven't thought about it hard enough.
The folks at slashdot can take down a webserver just by making a few scribbles on their website.
Isn't that just a fill tool? Paint does that.
No. Fill just goes until it meets a boundary. This colorization is a lot smarter than that. It appears to notice the boundarys by the sudden changes of the temperature in the color of pixels. That way it can then make an educated guess on how much to color and when to stop. You can then optimize this by putting in more than one input of the colors you want to change. This effect is really quite amazing. Scroll down and look at the gif video of the birthday party. JUST AMAZING.
I'm a virgo and on Slashdot. Coincidence? Yes.
I am so curious what this could do for so many old movies...
:) A lot of the appeal of older B&W movies is the fact that they aren't in color. You get a much broader range of contrast when it's filmed on B&W film than a color image which has been desaturated.
Ruin them?
If you meant older color movies which have degraded, then I agree. This seems like a very useful technique for restoring the original vibrancy of colors to films whose media hasn't stood the test of time.
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
This work is very similar to some work that was presented at last years siggraph using graph cut optimization titled Interactive Digital Photomontage by some researchers at the University of Washington. This stuff is really cool and has applications outside of just re-coloring black and white. For example, compositors in the film industry adjust the color composition of scenes that were filmed during the day to look like they were filmed at night. Sometimes they just need to tweak the color because the art director isnt happy with it. Other times it's because they introduced CG elements into live action scenes and they dont quite match. If they can tweak those colors interactively, without authoring masks, it is faster than re-rendering the scene and that saves money.
Very cool stuff.
Pete
What's a sig? Pete Brubaker
"What's wrong?"
"It's as if millions of lawyers stampeded the patent office and then suddenly... prior art."
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
-Don
Take a look and feel free: http://www.PieMenu.com
The site is slashdotted so I cannot read it, but i wonder if something akin to this could be used for compressing motion video. For example the intensity is encoded with currrent techniques, but instead of the color being encoded at a lower resolution, instead only a very small amount of colored points are encoded. Then during the decoding, the decoder uses an error function, intensity, and the time domain of previous and future frames to 'fill' the colors out.
-Don
Take a look and feel free: http://www.PieMenu.com
You b/w film purists. If all you can see is a threat to your bizarre, luddite idea of what film should be, you need to get your heads checked, or at least you need to listen to your inner geek. Stop using these folks' achievement as an opportunity for chest-thumping.
The idea that one could color correct video with a few strokes from mspaint is staggering. Imagine if one could do this to color video, in real time... you could color-highlight an object and the computer could follow it without sensors or other pre-implanted devices, and that's not even a particularly original idea. This is awesome technology with applications probably well beyond what we see here.
vk.
That feels better now.
Take a look and feel free: http://www.PieMenu.com
I tried out their matlab code and put a few example colourings on my web page, for the interested:
http://www.inference.phy.cam.ac.uk/cjb/
Intensity actually takes most of the bandwidth of an MPEG stream, because human eyesight tends to notice changes in luminance more than changes in chroma. The chroma channels are compressed *extremely* heavily compared to the luma channel, and are actually even at a lower resolution.
Make me a friend and I'll mod you up
I could see this working best as a "realtime" colour filter, especially if you're using a pen or something similar. Scratch near a border and view the result... if it goes a bit beyond where you want scratch on the other side of the border. If it's not quite enough lengthen your scratch.
I wonder how much CPU power is required, if you could do this realtime or close to it would be quite awesome, but having to make your scratches and click "apply filter" then wait for 30 seconds would not be nearly as useful/efficient.
What a brilliant plan! What if we were to take groupings of four pixels, store luminance for each and an average of their red and blue weights, netting a savings of:
... this sounds an aweful lot like the YUV encoding used in MPEG compression ... probably has something to do with it actually being the YUV encoding used in MPEG compression.
uncompressed: 24 bits per pixel X 4 pixels = 72 bits
compressed: 8 luminance bits X 4 pixels + 8 Red bits + 8 V bits = 48 bits
100 - (48 / 72 * 100) = 33.3%!
Wait
"Because Science" is one step from "Because old book". Try "Because of my experiment testing my falsifiable assertion".
Hey, that is actually true!
What's this? a slashdotter who knows what he's talking about?
He's a witch!
It's easier to fight for one's principles than to live up to them.
" The technique works on the premise that 'neighboring pixels in space-time that have similar intensities should have similar colors"
Interestingly, the retina exploits that same property of natural scenes to compress images. This correlation between luminance and color is an opportunity to throw out redundant information. The eye multiplexes color and luminance information over a single channel, transmitting luminance while discarding color at high spatial frequencies and transmitting color while discarding luminance at low spacial frequencies. First reported by C.R. Ingling, color/luminance multiplexing is an inherent property of the linear color-opponent center-surround receptive field. For a good explication of the subject, see:
Vision Res. 1985;25(1):33-8."The spatiotemporal properties of the r-g X-cell channel."
Ingling CR Jr, Martinez-Uriegas E.
Abstract: Analysis of the simple-opponent r-g receptive field of the X-channel shows that it is tuned to both high and low temporal frequencies, high and low spatial frequencies, and that its spectral sensitivity is both chromatic and achromatic.
Ceci n'est pas une signature.
Take a few deep breaths, you'll get over it. In the meantime, I can spread some disinformation if you want. Hmmm...
The MPEG standard, knowing that porn tends to drive the industry, actually contains several optimizations for drawing things such as skin tones and nipples. Really.
Make me a friend and I'll mod you up
24 bits per pixel X 4 pixels = 72 bits
In other news, mathematicians still agree that 24 times 4 is 96.
YUV 4:2:0 saves 50% bits over YUV 4:4:4, more info on wikipedia (per usual) http://en.wikipedia.org/wiki/Chroma_subsampling