Digital Video Capture and High Frame Rates?

Jeff asks: "So the folks at a place called Conniption Films (great name) developed a camera called the Millisecond Camera which can shoot 12,000 frames of film a second. I read the article and thought 'Hmm that's neat' but then realized they were still using an analog process for shooting this highspeed film. Being a geek, not necessarily into the film side of things but curious nonetheless, I wonder, shouldn't a computer be able to do a better job of such a thing? They say the film runs around a spindle going 500 mph (!). Wouldn't that be prone to failure and use alot of energy? Wouldn't it be more appropriate, easier, and overall cheaper to just hook up a high res CCD to a beowulf </duck> cluster of 2 ghz+ machines and capture high speed images that way? Why hasn't it been done yet? Or has it and I haven't seen it yet?" I did a double-take, when I first read this question, and then got curious and did a little digging. Turns out, high frame rates are not exclusive to the analog photography world, and to illustrate my point, I provide this link. It's woefully short on details, and the explanations as to why a camera that can record 1M frames per second is limited to a playback of only 103 frames, but the technology is out there. Has anyone seen any other digital cameras out there with high frame-rates? What visual mischief could you aspiring photographers get into with such a camera?

Comments

Cliff, did you READ it?

[MadCow42]

The explanation as to why it can only play back 103 frames is QUITE clear... the chip has 103 "on-chip" memory buffers per sensor, and they get cyclicly overwritten with the last 103 frames.

This overcomes the bottleneck of trying to transfer data off the CCD at such high frame rates in real time, but limits you to "downloading" the last 103 frames after-the fact from the chip.

MadCow.

CCD

[The Moving Shadow]

CCD simply needs a few milliseconds to regain their 0-volt signal level again before they can emit a new pulse. This recoverytime makes it unsuitable for high speed filming. Helas.

12,000 FPS isn't a breakthrough

[Animats]

You can rent an 11,000 FPS camera right now, for $200 per day. Photek makes a camera that can reach 40,000 FPS, although only with 8mm film frame size. Rotating-prism cameras like this have been around since at least the 1940s. The film advances continuously, and a rotating prism synchs the image to the moving film. Typically the synchronization has been mechanical, which means major problems at very high speed. An obvious upgrade to current technology is to feed the film with rollers or air jets rather than sprockets, detect the sprocket holes or some other form of clock track, and synchronize the rotating mirror prism electronically.

On the pure digital front, there are units that can record 1000 FPS continuous at 512 x 512 pixels. The system is data-rate limited. The imager can go much faster; if you cut the image size down to 32x128 pixels, you can get 32K frames/sec. At 128 x 128, you can get 11.2K frames/sec. The data goes into a buffer in the control unit (1 GB, typically), and is read out via FireWire. So this system can take a lot more frames than the device described in the article, which stores the images in memory within the imager and can only store 100 images or so.

Several points...

[My Third Account]

1024 (width) * 768 (height) * 4 (32-bit color) * 12000 (fps) = 377,487,360,00 bytes/second (35 Gbytes/s)

Well, For one thing nobody records at that resolution. As another reply stated, DV is 720x480.

Another problem with your simple calculation is that video is never stored as 32-bit color. That's totally unrealistic. The common way to store video is not RGB, but YUV. Because of the way the human visual system works, the color components (U,V) are typically stored at 1/4 the resolution of the luminance (Y), meaning that an 3*X pixel RGB image would be stored as a X+X/4+X/4=1.5X image in YUV, half the number of pixels.

More significant, though, is that fact that just about every digital image recording mechanism stores information compressed onto the storage media. This is true from consumer digital cameras to DV cameras to the Sony HDTV cameras Lucas used for Star Wars.

Consider what it means to take 12,000 frames per second. You're probably recording a single nearly-instantaneous event, or getting many images of a very fast event. In the former case, there will be a series of frames before the event in which nothing is going on, and the difference between the frames is close to zero, which compresses extremeley well with MPEG-style compression. Your data rate could be 1/100th of the uncompressed rate. When the event occurs, the instantaneous data rate goes up, but buffering can solve this, since it probably lasts a few frames.

In the latter case, recording a fast event at a fast framerate, is essentially the same as recording a normal-speed event at normal frame rates. In this domain as well MPEG-style compression is extremeley effective. At the maximum you would need 1/5th or 1/10th the uncompressed rate, but 1/100th is a pretty reasonable number given current technology.

The only challenge with realtime compression at this speed, of course, is sufficiently fast hardware. I think it could be done in parallel -- capture several GOPs worth of data (15-45 frames perhaps) and send it to a compressor, and then switch the buffer output to a new compressor, round-robin style.

In any case, video is usually stored at rates many factors smaller than the uncompressed rate. So if you change the variables of your equation to a more realistic resolution and color depth, then divide that number by 10 or 100, you'll have a more realistic data rate.

720(w)*480(h)*1.5(color)*12000(fps)= 6.2GB/s, divide by 100 for agressive compression but reasonable results = 600MB/s

Still too fast, but not completely unrealistic if you've got a healthy budget. ;-)