Slashdot Mirror
Digital Video Capture and High Frame Rates?
Jeff asks: "So the folks at a place called Conniption Films (great name) developed a camera called the Millisecond Camera which can shoot 12,000 frames of film a second. I read the article and thought 'Hmm that's neat' but then realized they were still using an analog process for shooting this highspeed film. Being a geek, not necessarily into the film side of things but curious nonetheless, I wonder, shouldn't a computer be able to do a better job of such a thing? They say the film runs around a spindle going 500 mph (!). Wouldn't that be prone to failure and use alot of energy? Wouldn't it be more appropriate, easier, and overall cheaper to just hook up a high res CCD to a beowulf </duck> cluster of 2 ghz+ machines and capture high speed images that way? Why hasn't it been done yet? Or has it and I haven't seen it yet?" I did a double-take, when I first read this question, and then got curious and did a little digging. Turns out, high frame rates are not exclusive to the analog photography world, and to illustrate my point, I provide this link. It's woefully short on details, and the explanations as to why a camera that can record 1M frames per second is limited to a playback of only 103 frames, but the technology is out there. Has anyone seen any other digital cameras out there with high frame-rates? What visual mischief could you aspiring photographers get into with such a camera?
The explanation as to why it can only play back 103 frames is QUITE clear... the chip has 103 "on-chip" memory buffers per sensor, and they get cyclicly overwritten with the last 103 frames.
This overcomes the bottleneck of trying to transfer data off the CCD at such high frame rates in real time, but limits you to "downloading" the last 103 frames after-the fact from the chip.
MadCow.
I used to have a sig, but I set it free and it never came back.
A quick calculation on the bandwidth of capturing 12000 SVGA-resolution full color frames per second:
1024 (width) * 768 (height) * 4 (32-bit color) * 12000 (fps) = 377,487,360,00 bytes/second (35 Gbytes/s)
So no wonder they use film...
CCD simply needs a few milliseconds to regain their 0-volt signal level again before they can emit a new pulse. This recoverytime makes it unsuitable for high speed filming. Helas.
A. El Gammal, et al. published a 10,000fps imager with a 352x288 pixel resolution. This guy can maintain the full speed indefinately. Unfortunately is it not a commercial device, but something similar will probably be available within a few years.
Kleinfelder, S. SukHwan Lim Xinqiao Liu El Gamal, A. "A 10000 frames/s CMOS digital pixel sensor", Solid-State Circuits, IEEE Journal of. v38 n12, pp. 2049-2059. Feb. 2001.
The abstact is as follows:
A 352 x 288 pixel CMOS image sensor chip with per-pixel single-slope ADC and dynamic memory in a standard digital 0.18um CMOS process is described. The chip performs "snapshot" image acquisition, parallel 8-bit A/D conversion, and digital readout at continuous rate of 10000 frames/s or 1 Gpixels/s with power consumption of 50 mW. Each pixel consists of a photogate circuit, a three-stage comparator, and an 8-bit 3T dynamic memory comprising a total of 37 transistors in 9.4x9.4 um with a fill factor of 15%. The photogate quantum efficiency is 13.6%, and the sensor conversion gain is 13.1uV/e. At 1000 frames/s, measured integral nonlinearity is 0.22% over a 1-V range, rms temporal noise with digital CDS is 0.15%, and rms FPN with digital CDS is 0.027%. When operated at low frame rates, on-chip power management circuits permit complete powerdown between each frame conversion and readout. The digitized pixel data is read out over a 64-bit (8-pixel) wide bus operating at 167 MHz, i.e., over 1.33 GB/s. The chip is suitable for general high-speed imaging applications as well as for the implementation of several still and standard video rate applications that benefit from high-speed capture, such as dynamic range enhancement, motion estimation and compensation, and image stabilization.
"why the hell would anybody need 12000 frames per second. The human eye cant process all that plus nothing happens so fast you need 12000 shots of it in a second. This is just plain stupid and its a waste of film."
Because certian events, despite what you might think, *do* occur within 1000ths of a second. (The fireball from a nucelar blast for instance.)
Good cammera's shutter speeds tend to go up to 1/1000th of a second, and can go up to 1/8000th.
As far as the humman eye comment, well, just because you record at 12000 fps, doesn't mean you play it back at 12000 fps...
On the pure digital front, there are units that can record 1000 FPS continuous at 512 x 512 pixels. The system is data-rate limited. The imager can go much faster; if you cut the image size down to 32x128 pixels, you can get 32K frames/sec. At 128 x 128, you can get 11.2K frames/sec. The data goes into a buffer in the control unit (1 GB, typically), and is read out via FireWire. So this system can take a lot more frames than the device described in the article, which stores the images in memory within the imager and can only store 100 images or so.
You can get around any bandwidth issues with a sufficiently large amount of cabling. The whole idea of doing this in parellel implies that. Anyway, compare the bandwidth of digital photography with the physical bandwidth of looping film through an eyepiece at 12,000 frames per second and you come up with a very different problem -- you've got to use TINY film, with an effective resolution much lower than what some of you linux numbercrunchers are assuming. "SVGA resolutions?" Think more like 320x240 -- and don't expect more than a few seconds per cannister, high costs, etc.
.000083 s. With low light, you need extremely sensitive equipment to even detect it and even more sensitive equipment to detect the subtle variations in wavelength that make up colors. Today's CCD cameras are very slow to register intensity light -- much slower than film. The chemical reaction in film triggered by exposure can be controlled much better, simply by changing the tolerance of the film -- which is why your high end, high speed shutter digital cameras are so godawful expensive. The $2500 Canon I've been looking at has roughly the same shutter speed as an equivalent $300 film camera. The extra price is NOT a "coolness" tax...it's for the set of three extremely high res CCD sensors and the chips capable of processing their information at that speed. My film prof used to say "digital ain't digital"...there's a quality factor of all digital electronics that can be poiled down to the quality of interpolation, quality of the ADC and of transistors leading up to it.
No, the problem is light itself. You don't get much of it captured with a shutter speed of
CCD kind of sucks, man. For all its glorious promise, the best CCD chipsets aren't all that much better than the wonderful X-10 spycam.
Hey freaks: now you're ju
From an artistic point of view, the problem isn't which medium to develop...it's how to improve both technologies such that cost/energy/latency is not too different. I should have the freedom to choose the technology which best serves the intent of the piece free from those constraints. It could be film, it could be video. It really depends on how I want it to turn out in the end.
So more substance, less rant: here's how I think these technologies would be useful to end users, and thus what we should be thinking about here.
Video Tap: A major video breakthrough in the feature film making process was Jerry Lewis's video tap. This puts a prism or split field diopter in between the lens and the film plane, splitting it in two, one going to the film plane, the other going to a video camera. This is how a director is able to get immediate feedback on how the scene went (instead of waiting for the dailies the next night to see it). A high framerate video tap for high framerate film would be extremely handy. The quality wouldn't have to be great, it would just need fidelity to tell the director and cinematographer how well composed the take was, and making sure all the stuff thats supposed to be in the take are there...and nothing else (like a boom mic).
Internet/NLE: This also would help in modern, internetworked digital non-linear processes. This is where takes are digitized as they are shot (if not already doing initial capture in DV) and dropped into the timeline in a nonlinear edit suite (avid, cinerella, final cut pro) whos project files are shared in an internetworked data store (film crews on other ends of the world, and the CG shop instantly are able to see their shot in the context of the other units shots...in realtime) via a 3 point edit. Even with a film process, the tap could digitize the footage and insert it into the timeline...the print of the footage could be later scanned and conformed to the timeline. Very handy. So this ties into the throughput problem. You have to consider that the bottleneck isn't CCD voltage intervals, cache tomfoolery or writing to a non-volatile medium. It could be a crappy ADSL connection or satellite uplink set up by people who scarcely understand how that stuff works.
Noise and heat: One of the banes of film making and one of the big advantages of digital video is the noise that all those ratchet/crank/shutter type mechanisms in a camera create. A lot of the sound work in a film is dealing with the noise from the camera. Sometimes, the sound is recorded later after discarding the sound from the set wholesale. Now, in order for a cmos imager to be effective at these speeds, we'll need to keep it cool. Heat is more likely to degrade throughput than buffer speed or size. Hence, we're going to need to build hardware to cool the cmos. That hardware is likely going to be more exotic than the cmos, take more energy than the motor for a high speed film device and potentially create a lot of noise on it's own. So the advantages of the high speed DV cam over film are only possible if the apparatus that supports the camera don't reintroduce the same problems on an equal or greater scale than existed in film.
Personally, I feel that the single greatest and most useful application of this technology, from a creative standpoint is the high speed video tap. It would liberate crews from the burden of dailies and integrate high speed footage into modern production processes.
For non-creative uses (scientific/research), this technology can free users from the latent and toxic nature of film processing infrastructure.
"Let him go, Ralph. He knows what he's doing." --Otto Mann (simpsons)