Slashdot Mirror
Colorizing Images and Video by Scribbling
Guspaz writes "Up until now, colorizing a video or image has been a painstaking and mostly manual task. However, researchers in Israel have come up with a new way of colorizing images just by making a few scribbles. The technique works on the premise that 'neighboring pixels in space-time that have similar intensities should have similar colors,' and also allows colorization of videos by 'marking' about one in ten frames."
I'm colorblind! That whole site is very confusing! :P
Back when voyager(s) were flying by planets I recall reading how the cameras worked. From what I remember, the cameras actually capture images in black and white. The cameras can detect much more "color" depth than color cameras could (or can?). The scientist would process the pictures to colorize them, you identify one area of color you know and the algorithm would process the rest of the 1 billion shades of gray into a color mapping for people to view. Now why cant identify this gray shade as the color red; anytime you see it then that is red. Go on for each color spectrum or have the algorithm adjust what a little red hue is for a given little hue of gray. It appears that is what the scribbles are doing which is quite clever and the algorithm doesn't have to work (guess) so much.
Personally I can't wait until there is a Photoshop filter for this. :D
Support Liberty, Support Ron Paul
from looking at the before and after images, this technique looks pretty cool and will probably have applications for recoloring an image that is already color. For instance, the image where he recolors the fabric on the chair.
Video Compression !
Only save the intensity channel and a few bits of markup and you compress the stream quite a bit.
The bikini - security through obscurity since 1943
Now it's even easier for corporations re-releasing films to completely destroy the original beauty of a film by adding unnatural and unnecessary color!
Coming soon, new dubbing techniques will allow easy substitution of the original actors' voices and dialogue with trite teen-angst to appeal to younger generations.
Here is the coralized mirror.
Avantslash: low-bandwidth mobile slashdot.
...and raise enought money to rebuild the smoking ruins of their server room.
I'm not wrong. You haven't thought about it hard enough.
The folks at slashdot can take down a webserver just by making a few scribbles on their website.
Isn't that just a fill tool? Paint does that.
No. Fill just goes until it meets a boundary. This colorization is a lot smarter than that. It appears to notice the boundarys by the sudden changes of the temperature in the color of pixels. That way it can then make an educated guess on how much to color and when to stop. You can then optimize this by putting in more than one input of the colors you want to change. This effect is really quite amazing. Scroll down and look at the gif video of the birthday party. JUST AMAZING.
I'm a virgo and on Slashdot. Coincidence? Yes.
Already done. It's called NTSC!
Looks like a very cool idea. The videos are pretty amazing, however; the recolor job in the birthday party clip makes it look like a bad acid trip.
I am so curious what this could do for so many old movies...
:) A lot of the appeal of older B&W movies is the fact that they aren't in color. You get a much broader range of contrast when it's filmed on B&W film than a color image which has been desaturated.
Ruin them?
If you meant older color movies which have degraded, then I agree. This seems like a very useful technique for restoring the original vibrancy of colors to films whose media hasn't stood the test of time.
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
Read about it earlier this week, it looks cool, colorization looks nice, but with so few samples its hard to tell if it will work with a wide range of inputs, or only ones with contrast ratios like those shown.
Video Production Support
This work is very similar to some work that was presented at last years siggraph using graph cut optimization titled Interactive Digital Photomontage by some researchers at the University of Washington. This stuff is really cool and has applications outside of just re-coloring black and white. For example, compositors in the film industry adjust the color composition of scenes that were filmed during the day to look like they were filmed at night. Sometimes they just need to tweak the color because the art director isnt happy with it. Other times it's because they introduced CG elements into live action scenes and they dont quite match. If they can tweak those colors interactively, without authoring masks, it is faster than re-rendering the scene and that saves money.
Very cool stuff.
Pete
What's a sig? Pete Brubaker
"What's wrong?"
"It's as if millions of lawyers stampeded the patent office and then suddenly... prior art."
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
I'm not sure if this was intended to sound like some new ground breaking technique, but it really isn't. I am a masters Electrical Engineering student and am currently taking an Image Processing class. Using neighboring images to reconstruct an image is a VERY VERY common task - in fact, it's almost the only way to do it. How else are you supposed to guess the colors (or what pixel is *supposed* to be there) without knowing what's around it. It's obvious that the highest correlation will be between the nearest neighbors (except on some edges).
Maybe next time we can make a program that just guesses the colors and look at how interesting those come out!
I don't necessarily believe that old black and white movies are good BECAUSE they are black and white. Granted, a lot of "colorized" movies look like crap, and I'll also grant you that a lot of black and white movies are good. I think that the correlation between the two is probably imagined. Colorizing a good movie doesn't necessarily lessen the movie, and can add considerably to it I'd say. The act of adding color (if done well), by itself, is not going to ruin the movie, in my opinion. Adding color, and doing it poorly could, but that's neither here nor there.
-9mm-
-Don
Take a look and feel free: http://www.PieMenu.com
The site is slashdotted so I cannot read it, but i wonder if something akin to this could be used for compressing motion video. For example the intensity is encoded with currrent techniques, but instead of the color being encoded at a lower resolution, instead only a very small amount of colored points are encoded. Then during the decoding, the decoder uses an error function, intensity, and the time domain of previous and future frames to 'fill' the colors out.
-Don
Take a look and feel free: http://www.PieMenu.com
You b/w film purists. If all you can see is a threat to your bizarre, luddite idea of what film should be, you need to get your heads checked, or at least you need to listen to your inner geek. Stop using these folks' achievement as an opportunity for chest-thumping.
The idea that one could color correct video with a few strokes from mspaint is staggering. Imagine if one could do this to color video, in real time... you could color-highlight an object and the computer could follow it without sensors or other pre-implanted devices, and that's not even a particularly original idea. This is awesome technology with applications probably well beyond what we see here.
vk.
My Emphasis added:
Flood fill would be described as:
See the differences? They are important.
There are no tiger attacks in my area and it's all because this rock I'm holding keeps the tigers away.
That feels better now.
Take a look and feel free: http://www.PieMenu.com
The director didn't add colour because he probably couldn't! The question is would he have wanted to see the movie in colour if available?
I saw the Sign, and it opened up my eyes
I tried out their matlab code and put a few example colourings on my web page, for the interested:
http://www.inference.phy.cam.ac.uk/cjb/
Intensity actually takes most of the bandwidth of an MPEG stream, because human eyesight tends to notice changes in luminance more than changes in chroma. The chroma channels are compressed *extremely* heavily compared to the luma channel, and are actually even at a lower resolution.
Make me a friend and I'll mod you up
I could see this working best as a "realtime" colour filter, especially if you're using a pen or something similar. Scratch near a border and view the result... if it goes a bit beyond where you want scratch on the other side of the border. If it's not quite enough lengthen your scratch.
I wonder how much CPU power is required, if you could do this realtime or close to it would be quite awesome, but having to make your scratches and click "apply filter" then wait for 30 seconds would not be nearly as useful/efficient.
But I actually AM colorblind, you insensitive clod!
Stop filling with that color when you hit something that marks a definitive boundary.
That's the "and then a miracle occurs" step.
What a brilliant plan! What if we were to take groupings of four pixels, store luminance for each and an average of their red and blue weights, netting a savings of:
... this sounds an aweful lot like the YUV encoding used in MPEG compression ... probably has something to do with it actually being the YUV encoding used in MPEG compression.
uncompressed: 24 bits per pixel X 4 pixels = 72 bits
compressed: 8 luminance bits X 4 pixels + 8 Red bits + 8 V bits = 48 bits
100 - (48 / 72 * 100) = 33.3%!
Wait
"Because Science" is one step from "Because old book". Try "Because of my experiment testing my falsifiable assertion".
For example, on Farscape, given Virginia Hey's problems with makeup and contact lenses... heck, any of these humanoids-with-funny-skin-color shows would benefit from not having to put in the hours upon hours of makeup. Instead, we'd see hours upon hours of post-production...
-- Still waiting for the Nike endorsement
In the days when colorized videos of black-and-white films were common, I watched a few. The so-called "colorization" had some very serious problems, and I wonder whether this new method addresses them.
The problems tended to be in the background, and they probably thought people's attention would stay on the foreground, but I think like many things in film you notice them subconsciously. Either the background is out of focus, in which case there are no sharp edges for the colorization to work on, or it contains a basically infinite quantity of detail as the background gets farther and farther way. Either way, it was extremely common to see uncolored areas in the background.
It was fairly common to see black-and-white paintings hanging on walls, for example. The walls would be some fairly uniform wash of plausible wall color, but nobody was going to take the time to handcolor the paintings hanging on them.
A similar problem concerned scenes with machinery in them, or anything with lots of complex, detailed motion (so that successive frames weren't similar). Thus, you'd see black-and-white printing presses operating in a colorized newspaper building...
In addition, the fact that the colorized faces, for example, were a uniformly colored wash, rather than varying in color as well as brightness, created a subtle kind of phoniness. To me, the result was the conveyance of a sort of emotional coldness. The colorized movies looked colored, but they didn't feel colored.
The exact opposite of the kind of lift you couldn't help feeling in the fifties when you saw a Technicolor spectacular--in the days when "Technicolor" meant that by golly you were watching genuine dye-imbibation prints from real color separations. Sweet as candy, but irresistable. (The effect does come through in the best DVD restorations).
"How to Do Nothing," kids activities, back in print!
Hey, that is actually true!
What's this? a slashdotter who knows what he's talking about?
He's a witch!
It's easier to fight for one's principles than to live up to them.
" The technique works on the premise that 'neighboring pixels in space-time that have similar intensities should have similar colors"
Interestingly, the retina exploits that same property of natural scenes to compress images. This correlation between luminance and color is an opportunity to throw out redundant information. The eye multiplexes color and luminance information over a single channel, transmitting luminance while discarding color at high spatial frequencies and transmitting color while discarding luminance at low spacial frequencies. First reported by C.R. Ingling, color/luminance multiplexing is an inherent property of the linear color-opponent center-surround receptive field. For a good explication of the subject, see:
Vision Res. 1985;25(1):33-8."The spatiotemporal properties of the r-g X-cell channel."
Ingling CR Jr, Martinez-Uriegas E.
Abstract: Analysis of the simple-opponent r-g receptive field of the X-channel shows that it is tuned to both high and low temporal frequencies, high and low spatial frequencies, and that its spectral sensitivity is both chromatic and achromatic.
Ceci n'est pas une signature.
Take a few deep breaths, you'll get over it. In the meantime, I can spread some disinformation if you want. Hmmm...
The MPEG standard, knowing that porn tends to drive the industry, actually contains several optimizations for drawing things such as skin tones and nipples. Really.
Make me a friend and I'll mod you up
Except that groupings of 4 pixels is not necessarily a good approach. For one thing, it may be far more color information than is really necessary. Besides that, two adjacent pixels may have significantly different colors. Carefully designed "scribbles" of color could very well take up less space and give better quality.
I'm sure there are good reasons for the JPEG/MPEG method, and I'd be a bit surprised if the groups in question didn't think of this possibility, but I still think it should theoretically give better results (at the cost of higher complexity and computational requirements).
Some folks seem to be excited (or angry) with the possibility of coloring B&W movies with this technique. Forget realistic coloring, this looks amazing for artistic recoloring.
Go take a look at the "recoloring examples" in the coral cache. Also look at what a slashdotter did with the code. Photographers, designers and painters could do neat things with a filter like this in Gimp...
Prescriptive grammar:linguistics
Apparently, one of the critical differences between Octave and Matlab is that Octave apparently doesn't support multidimensional matrices/arrays with dimensions greater than 2.
:(
Which means that the current code is completely incompatible with Octave, as it depends on Matlab's implementation of imread() which returns image data as a three-dimensional matrix.
Going to see if I can get it to work easily, but there's a good chance I won't be able to.
retrorocket.o not found, launch anyway?
24 bits per pixel X 4 pixels = 72 bits
In other news, mathematicians still agree that 24 times 4 is 96.
YUV 4:2:0 saves 50% bits over YUV 4:4:4, more info on wikipedia (per usual) http://en.wikipedia.org/wiki/Chroma_subsampling
A colour image is desaturated compared to a B&W image? Hello McFly?
Please look up saturation in regard to colour.
Hint: Try raising the saturation of a colour image in photoshop or gimp (ctrl-u in photoshop). Observe. Then try using the "Desaturate" function and see what you get.