Slashdot Mirror
Reverse Engineering Doctor Who Into Color
Lanxon writes "In 1967, the BBC set about junking its Doctor Who archive: a moment sci-fi fans wish they could travel back in time to prevent. There are 108 vintage episodes missing, but since 1978 a number have been rediscovered as 16mm black-and-white films. The BBC shot many of these series in color, but made monochrome copies for countries such as Australia, where many TV companies were still broadcasting in greyscale. The reels had sat in archives since. Now, the Doctor Who Restoration Team, an independent group contracted by the BBC, is using a new technique to regenerate The Doctor in color."
Since this is the BBC, they shot *none* of them in color but many of them in *colour*....
So the article was devoid of anything of particular interest other than some jargon. The jargon, on the other hand, led to fascinating little technique about reconstructing the color of the grayscale image from "chroma dots". The actual method was discovered by a BBC engineer, and you can read more about it here: colour-recovery.wikispaces.com.
Can they reverse engineer the scripts instead? Color or black and white, those old episodes are damn unwatchable. We'd be better off giving Wikipedia descriptions of the episodes to the writing staff of Golden Girls. Those old droning 5-part episodes would be turned into 22.5 minutes of tightly scripted comedy starring Bea Arthur as the Doctor. And any of the other old hags as K-9.
Fascism trolls keeping me up every night. When I starts a preachin', he HITS ME WITH HIS REICH!
The good news is that they've figured out how to restore colour to the B&W negatives. The bad news is that it requires Kodachrome processing...
When you have nothing left to burn you must set yourself on fire
It almost is reverse engineering. The chroma subcarrier in a video signal has a center frequency picked to allow the sidebands to fall between those of the main lumanance (black and white) video. The spectrum of those extends out from the main visual carrier frequency (or up from D.C. for the baseband signal) at multiple of the horizontal scan rate. The goal was to add color broadcast information to an existing greyscale system while introducing a minimal amount of interference. Here people are figuring out what is going on from the visual interference.
The added signal amplitude represents the amount of color added/subtracted from the greyscale white, and the phase represents the hue. The phase of the signal is compared with a short burst (a minimum of eight cycles) sent just after the horizontal sync pulse prior to the start of video on each scan line. PAL, as used by the BBC, is very similar to NTSC, except the scan rates differ, the phase of the reference signal is inverted on every other line to help cancel out the effect of small phase errors on tint.
Basically, those trying to recover color from the back and white films of on-air video have to use a comb filter to pick off the frequency (precisely related to the inverse of the spacing) of the resulting dots that are there from the color signal. The position of the dots from left to right carries the phase information. Considering that the dot pattern is probably quite weak, the resulting color would be noisy. Depending on the filtering used, the bandwidth (detail) may also suffer. But it is still a good starting point to know what the colors were.
The dots aren't on/off like pixels. It's actually a sinusoidal intensity variation. I recall some older Zenith B&W sets had particularly good detail (and maybe some video peaking - enhancement) making it easy to see which programs were broadcast in color, and what parts of the picture were deeply saturated. In addition to a notch in the video response at 4.5 MHz to filter out patterns from the sound, some sets rolled-off or notched centered at 3.58 MHz (3.579545 actually) video response to reduce the interference. Better later sets (and color generally) used "comb" filters to separate the interleaved spectral components without those loss of detail seen with more primitive methods. Failure to filter color signals could cause wild colors/patterns on things like striped neck-ties when a shot zoomed in/out.
It's pleasing to see that there are still a few around that understand the old analog technology well enough to realize there were visual color cues remaining. Even those that understand the electronics well often don't associate a particular visual characteristic with the responsible signal attributes.
Although partial signal recovery is easy to envision with analog electronics, something along the lines of a GIMP/Photoshop plugin could work as well. Some might think of it as being similar to watermark detection.
Like everything from Colin Baker. Seriously, aside from Peri's chest, there was nothing of interest in those episodes.
Does this sig remind you of Agatha Christie?
I think the clue is in the name, you are speaking English not American.
...oh, and yo momma's so fat, her Schwarzchild radius is visible to the naked eye.
On the restoration processes used in the past can be found on the RT's website, if you dig around a bit: http://restoration-team.co.uk/
This post expresses my opinion, not that of my employer. And yes, IAAL.
The politics behind the Chroma Dot story is intriguing and in some places unpleasant. The instigator of the team was James Insell, and a method was created to perform the chroma dot extraction by a man named Richard Russell. Insell became a bit proprietorial over it all, and he and Russell parted ways, and now Russell it doing it alone. The original Colour extraction blog is here but they don't seem to have made any huge advances since Russell left. There is some more info, plus a link to Russell's own work (including software download) on my own Dr.Who webpage here
My web domain.
My mom told me once she was watching a black and white TV with her family, and someone walked on the screen with green hair. Everyone watching the TV instantly started laughing because the guy had green hair. I don't entirely understand your post, but it does verify that my mom was not crazy, and average people watching in those days could distinguish even if they didn't know what was going on.
"First they came for the slanderers and i said nothing."
If the colour subcarrier is there then why doesn't it just show in colour when displayed on a colour TV?
I think the point is that the color subcarrier isn't there; all you have are the errors from the color subcarrier bleeding into the luminance part and now those errors are being used to extrapolate and restore color to the film.
This along with the MGM fire that destroyed the original Tom and Jerry prints, is more proof of how piracy can help us. If this stuff had been pirated all over the net like it would be today, it wouldn't have been lost in the first place. Hopefully they would have used a loss less format though... :-)
Ah, memories of watching Tom Baker followed by an episode of 'The Tripods' then to be lulled to sleep by 'The Star Hustler'.... On Fridays and Saturdays I could stay up WAY past 9:00...
I seem to remember that episodes starring his immediate predecessor weren't bad either.
Before that it was black and white. I think I only ever saw Dr #1 discover the daleks, and Dr #2 probably wasn't bad, but he did look like one of the Three Stooges...
I also remember not liking to watch Doctor Who anymore starting with the Fifth Doctor there may have been more doctors? I stopped caring.
Then as an adult, starting with Christopher E. , I've been a fan again. I've really enjoyed every one of the new Doctors since then.
It's been long enough since watching the old Tom Baker episodes that I don't remember the plots anymore, but having gone back and watched a few on Netflix, I don't see myself watching any more of them.
The special effects are very dated of course. I mean, green slime covered green lightbulbs wrapped in bubble wrap skin are obviously just what I described. But that's not the real reason. I just don't think the old episodes have much to offer someone who can/has viewed the new ones. The best of the old is part of the new character and plotwise, with HOUR long episodes to flesh things out more deeply.
...
The colour distortions NTSC and PAL suffer from on high frequency luminance changes are interesting and a fascinating insight into the world before digital. Every geek should IMO understand this because it's interesting and cool and easy to understand. I'll try and simplify the explanation...
The artefact in question results from a colour (analogue) TV trying to display a picture where the luminance (brightness to you and I) changes rapidly, i.e. with a high frequency. Viewers of old will be used to people with stripy shirts suddenly and stragely developing psychedelic colour patterns. Why do you get them? That's the interesting part:
When NTSC was devised they had to design a system that was backwards compatible i.e. used the same bandwidth to transmit a colour signal. Worse than that they had to have the signal such that the colour signal would be interpreted by a B&W receiver as a B&W image correctly.
So what they did was look at the signal and realise that although the bandwidth allocated allowed for rapid luminance changes, no set at the time (in domestic usage anyway) was capable of displaying them. Therefore they could transmit a high frequency signal on top of the luminance signal that existing sets would filter out, they would act like a low pass filter, and average it out back to the original signal. They could therefore superimpose a high frequency signal on top of the old luminance only signal. Now, how to carry information with this signal?
Well you kept the frequency of this superimposition constant but you could vary both its amplitude and its phase* relative to a reference signal. This combined with some maths behind luminance and chrominance processing (wikipedia is your friend here) meant that you had good old luminance information for old sets, and two colour components that could be extracted by a compliant set, but would be ignored by older sets.
Now where the artefact in question comes from should now be obvious, a high frequency luminance change looks indistinguishable from no intensity change with a certain colour information. The system was designed to minimise these, but they had to happen at least slightly. As TV sets and studio cameras have improved over the years there have become more of these artefacts as we have got better at displaying fine detail, so what wasn't a problem now shows the limits of the system.
If you want information about how the colour reference signal was sent or to explain more please do ask, PAL and NTSC are one of my favourite geeky subjects. It's engineering from an almost forgotten era and a sign of just how clever some engineers are/were.
*The difference between phase modulation and frequency modulation are somewhat academic for the purposes of this discussion, let's just say that phase modulation makes more sense because we have a reference waveform to compare it to.
"The weirdest thing about a mind, is that every answer that you find, is the basis of a brand new cliche" -
Because the film is distorted, and includes both interlaced frames in each image. So you scan it at a high resolution first (2k lines, for example).
So they need to de-distort. And as you will remember from your old CRT monitors and TVs, the image distorts according to the image displayed, so each frame needs to be de-distorted individually.
Then de-interlace. Then extract the information required. Then re-construct.
It was supposed to be appalling.
Best Slashdot Co