Slashdot Mirror
600 PowerMacs Make One DVD
vaporland writes "NYTimes.com has this story about using a network of 600 PowerMac G5's to scan original movie negatives at 4000 lines per inch and create high-resolution digital recreations of classic movies."
← Back to Stories (view on slashdot.org)
It's not like these are crisp, sharp modern prints. Jesus, at 4000 dpi, the film grains will be dozens of pixels in diameter...
Honey, I shrunk the Cygwin
I guess having a crazy high-res version will help when they scale it down for DVD/VHS/Broadcast.
Vote for global prefs bug
Given that DVD's perform lossy compression, to fit an entire movie into one disc, is there going to be much noticable difference between using the original final cut and a 3rd/4th generation copy?
Macs are great for stuff like this, sometimes I wish they had had the marketing smarts to get the market share PCs have now. They have alot going for them...
Ah well, "Macs for productivity, Linux for stability, Windows for solitaire"
once digitised, could they not be processed to remove those? I don't know much about image processing but I'm sure someone would be able to come up with a filter that would pick up such spots and remove them (based on previous and next clean frames maybe?).
So now I'm going to have to go out and buy a whole new set of DVD's when they release the '4K Edition' of all my favourite films. And I thought I was safe until Blu-Ray came out...
click!
I've already pre-ordered mine. Hurry now, while supplies last!
What are the Macs being used for?
Yes, I RTFA, and they mention the Imagica 4000 lines/frame scanner and the 600 Macs, but not what the Macs are used for. Only that the frames are offloaded to a server with a large hard disk.
So WHAT part of the process are they being used for? Someone enlighten me please.
The Official Steve Ballmer Webpage
the poster got it right wrong. The film isn't scanned 4000 times per square inch, the entire film is scanned at 4000 LINES of resolution.
Current HDTV displays 1080 lines interlaced.
Film grain represents the physical resolution of the film, it's not dust or something which can be removed by duplicating adjacent pixels. Moreover film grain is aestethically much nicer than any rounding and blurring the kind of filter you are proposing would produce.
once digitized, they could be processed to replace the guns in the movie with walkie-talkies.
More music, fewer hits
a beowulf cluster of... oh wait.
> all of the secondary processes will catch defects such as film grain.
Saying that film grain is a defect is like saying pixels are a defect..
Great, so he's doing optical at 4000 lines per inch.
But what about the sound? Is he using non-compressed 24-bit samples at [at least] 96KSS [kilo samples per second]?
Your ear is a vastly more sophisticated sampling device than your eye; I don't know of a single sound compression technology on the market that can fool the human ear.
It would be a real tragedy to go to all that trouble to make good digital copies of the optical prints, only to try to cheat on storage space by downgrading the soundtracks to one of these abominable undersampled, compressed audio standards.
It won't be crazy high res. 35mm prints are one inch tall. that's 4000 vertical resolution, which in the scheme of things isn't much different to scanning an A4 document landscape at about 450dpi.
High res for detail, but not as crazy as dozens of pixel sized film grains
I would imagine that, as with anything else that has components that can be categorized as either "good" or "popular", sales of the "popular" stuff will subsidize the production of the "good" stuff.
Face it - they're going to sell more copies of "Dr. No" with Ursula Andress wearing the New & Improved High Resolution Digital Bikini than they are of Singin' in the Rain, starring Gene Kelly and the Incredibly Vivid High Resolution Raindrops.
This has been a test. If this had been an actual Sig, you would have been amused.
So, in 500 years, the copyrights will be expired, right?
I can only wish.
How you sample analog material plays a big part in the overal quality of the finished product. For music, you typically think of samples per second (CDs play at 44.1khz). But typically for the initial digitization of analog material, you oversample (perhaps sampling the analog music at 88.2khz, or even higher). This gives you something that's much closer to the original work than normal, and allows you to work with a higher quality, well, sample. Performing digital transformations, including cleaning up the video, removing scratches, etc. always works better if you have more samples to work from. So a higher resolution picture will make it easier to get rid of any scratches or imperfections in the original film.
Eventually, of course, you have to downsample to fit the format that you will be distributing. For CDs, you downsample to 44.1khz. For DVDs, you downsample (the resolution) to 720x480 NTSC or 720x576 PAL. Note that that's somewhere around 1/8th the resolution that they're scanning.
The idea is simple. With this one scan, they can be prepared for format changes. Once high definition DVDs come out, they can downsample to whatever that resolution will be. If they want to broadcast a movie on an HD television channel, they can downsample to 1080i or whatever HD format they wish.
This seems to be about making a high-resolution copy now for archival purposes, so that if the film itself degrades (as it is prone to do) there will still be something really close to the original to work from. Not a bad idea, I think.
"MGM has hired Lowry Digital to make 4K digital masters of nine James Bond films, including all of those starring Sean Connery."
Seems authorised to me. The other movies mentioned, are MGM productions as well.
Singing in the Rain
Casablanca (1942)
Once Upon a Time in the West (MGM/UA)
Nice artice, but where are the screenshots?
on the 2nd page... they talk about that:
##
Since then, he has bought hundreds of computers, hired a staff of 30 and worked on 80 DVD's -- including the long-awaited DVD of "Star Wars" -- erasing wear, tears, dirt, scratches and other ravages of age. (In the early days, he sometimes erased too much. By his own admission, his restoration of "Citizen Kane" is too clean; the natural grain of film is gone; it looks like a video. He later figured out how to fix flaws while preserving grain.)
##
I'm guessing lucas considers "greedo shooting first" wear, tear, and scractches!
e.
Build Your Own PVR/HTPC news, reviews, &
You digitise your originals, then "offline" edit with your scaled down versions on a PC/mac. Once you have everything editied to your liking, you get back on the big, expensive "online" system and it can build your film - even going to the point of writing out your 35mm print.
The news here I guess is that they are using this technology to archive old films. I still don't see where the 600 macs fit in however.
Actually the workstation that controls the scanner runs linux.
You can see an overview here of the machine.
If you look at the press releases they came out with an add-on that allows the machine to scan at 10k lines in 12 seconds.
As an aside, the smaller film scanners that capture 35mm slides have Digital Ice to remove surface blemishes. Part of it works by shining an infrared light through the film. The infrared light is unaffected by the different shades of color, but the dust "stops" it and therefore is detected. Quite ingenious.
I imagine as expensive as this machine is, it uses this and other techniques to remove surface and film imperfections. If you use an original to scan that has been well cared for, the results should be impressive.
I toyed around with the idea of homebrewing such a machine to convert some old family super8 movies.
The two problems that you are going to have is the film transport, and the amount of time it takes to scan the film. As it stands, it would be time intensive to build such a machine and technically challenging. That and not having a workspace, it will have to wait for another day.
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
Film studios have always been using higher quality masters and they have never leaked. This doesn't change anything.
Who's gonna bother to steal it (it being hundreds of gigabytes) and then downscale it to regular resoluiton for hours just to have something at the same quality that's available at blockbuster for $5?
Or are you implying that people would like to download the original and store it on a terabyte disk array?
How do they store these digitized movies? Even better, how do they transport them?
Some back-of-the-envelope calculations assuming a 4000x4000 image, 24 bit color (too low?), lossless (optimistic) 4:1 compression and 24fps show that a 2 hour movie takes up over 1.8TiB.
Is it just a box of 300GB tapes, or do they have something even cooler?
Can you imagine the restore times for a movie from tape...
- mib
Actually, it's about 2-3 film grains per pixel.
I used to make 35mm slides from computer files with my Agfa QCR-Z slide writer (and I still do from time to time for the few places that still use 35mms for projection).
It has the same resolution of 4k (4000 lpi) that these films are being scanned at. The pixels are significantly bigger than film grains, but are just about too small to bring into focus with a really good 35mm projector.
Later on, they made 8k and 16k resolution versions, which were mostly used for larger format than 35mm output because of the film grain issue (and the fact that the damn device used an RS-232 connection and therefore took 4-5 minutes to image a 4k line file)
A pizza of radius z and thickness a has a volume of pi z z a
That article is so full of incorrect statements, its sad it got published in such a reputable paper.
It confuses horzontal and vertical resolutions left and right, mixing the 4k horizontal resolution of a 4k scan with the 1080 vertical resolution of HDTV and extrapolating silly figures from the result, as one example.
4k scans of film aren't uncommon, although this might be the first time it was done for archival purposes.
No matter what the article author says, you'll see zero difference between a 4k, or 2k scan on a DVD transfer. A 2k scan is aproximately HD resolution, so there would be a benefit for HD formats to have a 4k scan, to eliminate noise, etc.
The article was also unclear why such horsepower is needed for such a mundane process as scanning and storing film. Thats a problem thats been solved for a decade or more by the film industry, where working with 4k frames is commonplace.
Another dimension I hope projects like this expand into is capturing a much higher dynamic range of the color information stored on the film. If you scan a negative (or positive) at 128-bit color depth instead of 32-bit color depth or the more standard 24-bit color depth, capture very subtle diferrences of light and shadow which are not visible to the naked eye, but with careful image processing you can enhance and amplify those subtle color shifts and nearly normalize an under/overexposed picture, pulling details from the light/shadow/color which no one has ever seen before. Some might argue that the director did not intend for the audience to see Brando's face in full light in "The Godfather" and that the heavy shadows were intentional, but in most cases any director would agree that some of the detail they wanted in some shots was obscured by poor lighting/exposure and they would like to tweak that.
On the consumer side, putting a wide screen high-res video track on a DVD is one thing, but making that video (plus audio and subtracks) fit within 4.7GB (if you want to keep it all one disc)*and* having it play back reasonably well on the average consumer-level DVD player (which can only handle around 7Mbs bitrate) means you have to compress the hell out of each track which means reducing the quality of the picture with compression artifacts. So it seems to fully appreciate a high-res film-to-DVD transfer you'll have to have a nearly uncompressed DVD transfer (very little MPEG2 compression applied, probably spanning 6 discs or more) and a high-bandwidth DVD player that can handle a very high bitrate.
...most definitely affects the final product. I am currently working on a digital film myself with some friends where the original images are being done at Hi-Def resolution (1080 lines) and then downsampled to 525 for output to DVD. In the event this does wind up going to celluloid (unlikely, but possible), we might need to ramp things back up to 2,000 lines. If we're stuck halfway through, rather than redraw a lot of the material, we might be able to use a product like PhotoZoom Pro to make up the difference (at a slight cost).
I suspected we would need to start making 4K digital safeties of film as a standard practice at some point. Hi-Def telecines are good as telecines, but not for archiving.
Honorary Member of Jackie Chan's Kung Fu Process Servers
Oh, I dunno, I was hoping they might be able to colorize Young Frankenstein. It's a shame Brooks could only afford B&W film for that movie...
Ooh, a sarcasm detector. Oh, that's a real useful invention.
Artists have known since at least the time of Rembrandt [i.e. almost 400 years] that the human eye can be fooled into seeing what it wants to see; in the case of Rembrandt and his pointilism, the eye [or the part of the brain responsible for processing data collected by the eye] merges small dots of color into a larger whole that it would prefer to see.
You've just described compression. A particularily artful, beautiful form of compression (especially Monet,) but it's compression nonetheless. You just proved point the previous poster made: nobody is going to be fooled into believeing that a pointilist painting is actually a scene taking place in front of them. You may admire it for its beauty, for the technical and artistic prowess required to render it to the canvas, for any number of reasons. But it's not a "perfect" rendition; if you 'believe' you're at the seashore any more or less than you would by staring at a photograph of the seashore it's an emotional decision, not a rational one. And you certainly wouldn't settle for seeing James Bond rendered in a pointillistic style for two hours, not when you know you can see it in all of its Technicolor glory in the next theatre over. It's different -- it's an art form.
Now, there's almost nothing artful about audio compression. (I say almost because there are artists applying all sorts of distortion to their sounds to create new ones, including overcompression.) For the most part, the distortion caused by compression is just a nasty side-effect. But the ear is indeed "fooled" by the compression. When you listen to a compressed audio stream, you hear music. It may be poorly reproduced, tinnily digitized, and companded down to the level of a phone line, but you still hear the music behind it. That's "fooling" the ear -- at least as much as pointilist art "fools" the eye (and without the art.)
Anyway, setting all "golden ear" arguments aside and getting back on topic, I very seriously doubt they'd use compression at all on the audio. The imaging they're doing on each frame is lossless (each frame is probably around 40MB RAW), and this guy didn't get funding for 800 Macs by being stupid and cheap.
John
From what I gathered, from this article and the profile of Lowry on Apple's website, the software doesn't just remove dust and scratches but also film grain, by comparing each frame in the context of the surrounding frames and then softening or even removing irregularities. Yes, the difference will be huge.
Even on DVD; a 4th generation copy is like a movie that has had compression added 4 times, and each copy is progressively worse. Ideally, you want the cleanest print possible before you add lossy compression.
We're talking Apple here. The guns will be replaced with stylish iPods.
This is similar to the techniques that Disney did in the restoration of Snow White for DVD.
n Ca mera/oct2002/snowwhite.shtml
Disney took the original camera negative, hand cleaned it frame by frame, and then scanned it one frame at a time using a specialized Kodak hi-res 6000 line scanner. If you have ever seen one of the pre digital restoration prints in the theatres and then see the DVD you will realize the miracle this restoration is.
http://www.kodak.com/US/en/motion/newsletters/i
It may also be possible to construct a virtual frame in memory at a much higher resolution, then use positional manipulation of the frame (I.E. move it) while imaging it. Just as the handheld "scanner" technology for cellphones etc will allow you to wave a camera over a printed page and build a high resolution scan based on multiple passes, correlation, and interpolation, so we could do with movies. The problem with digital scans is of course that your scan quality is limited by the CCD pixel element size, the film grain size, the difference in their sizes, and the correlation (or lack thereof) of their positions.
As for duplicating adjacent pixels, no one uses that for a scaling algorithm any more unless they are a complete nincompoop, since so many other algorithms are readily available, but you're correct (obviously) in that data is always lost when using digital enhancement, which makes it useful for things like trying to decipher what license plate is on the back of a car, but not so useful for improving the quality of digital media.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
This story was posted in more and less confusing detail on Apple's own pro-user webspace months ago. The article written by Joe Cellini is much better at explaining why the high resolution of scans, etc. The primary purpose of this studio is to remaster degraded and degrading films.
Here's the link:
http://www.apple.com/pro/film/lowry/
1 - scanning: the imagica XE can scan at a resolution of 4096x3112(1.31:1 aspect ratio), the just-announced
2 303.html
xe-advanced uses a 10k capture device that allows overscanning and subsequent downsampling from
8k(8192) to 4k. the 4096 pixels is the horizontal res. from perf-to-perf, and is nothing new(i've been
doing 4k since ~1995). the reason for 4k at the moment, is that 4096 pixels across is just _below_ the
grain of commonly used Oneg/intermediate stocks. using higher resolutions is a waste of processing
time and disk-space when your scanned resolution is higher than the source(this applies to t-grained
(tabular)films as well.)
anyway, you shouldn't see any pixels unless the color calibration is sub-optimal, you're looking at a digital
projection or there were hardware probs.
kodak(cinesite) has had "dust-busting" on their menu for quite a while now, although it was originally
done by hand, by artists using high-res paint programs(photoshop/matador, etc).
as correctly noted by another poster, the scanner is run by a linux based machine. the previous version
of their scanner used an SGI o2 running IRIX. see: www.imagica.com
Kodak used to make a commercial scanner(the cineon genesis scanner) that i believe is no longer avalable.
another scanner to look at is the Oxberry Cinescan.
this is the week to look for info as it's NAB time; new products and updates are typically announced there.
2 - color: the dynamic range of film is described in logarithmic terms(due to the sensitivity function of the
emulsion-processing chemistry) so it is appropriate to record/store using a log-based imaging format.
in this case, a 14bit DAC is used to generate 10bit log/pixel color data stored in the industry standard
Cineon format(created by Glenn Kennel @kodak and subsequently adopted industry-wide. see FIDO, Cineon)
10bits log is equivalent to 14 bits linear and covers approximately a 10-stop range or a density
range from zero(or film base) to somewhere around 2.048D to as much as 3.0D depending and the
scanner and recorder.
3 - lowry and warner: lowry and warner are both working on restoration systems. warner has a large library of
SE(sequential exposure) shows that will need duplicate archives and cleaning for DVD releases. SE is a method
for recording the RGB channels on individual-sequential frames. this process retains color integrity by
maintaining channel separation as long as possible avoiding channel bleed/crossover. lowry is using
the Macs to do the image processing; a feature-length film can be very, very large(90min x 24fps x @4k)
since each image can be ~50MB each - lots of disk space and processing time. as previously mentioned,
warner has a system which resizes/aligns each channel in a logical frame, resulting in a very clean image
with no(virtually no) fringing or edge artifacts due to sep misalignment. this is normally not an
issue with SE as each sep is on a single piece of film. for three-strip technicolor, the alignment is
more critical as there are three individual pieces of film that were run through a special camera(the
Technicolor camera) which i believe has a patent... for an interesting site with info on SE(w/pictures) goto:
thedigitalbits.com/articles/robertharris/harris07
4 - some resolutions:
HDTV - 1280x720 or 1920x1080
NTSC - 640x480(4:3)
PAL - 720x486
film - 2048x1536(1.33:1 AR)
4096x6144(vista-vision 8-perf)
i can expound more if additional details/info is needed.
"...that's as white as it gets; all the bits are on..."