Slashdot Mirror
High-Speed Video Using a Dense Camera Array
karvind writes "Researchers at Stanford have demonstrated multi-thousand frame-per-second (fps) video using a dense array of cheap 30fps CMOS image sensors. A benefit of using a camera array to capture high speed video is that we can scale to higher speeds by simply adding more cameras. Even at extremely high frame rates, our array architecture supports continuous streaming to disk from all of the cameras. Now we know where to use 100TB tape drives and what to expect in the next sci-fi movie."
It's not that I don't think this is cool, because I can see all kinds of uses for this sort of thing.
But my question is this...
Are there any uses for high speed video capture that existing technologies weren't already well suited for, or is this just a cheaper and more readily available option?
"Everything you know is wrong. (And stupid.)"
Moderation Totals: Wrong=2, Stupid=3, Total=5.
Instead of having a hawkwardly swinging background, why wouldn't they use a set of rotating mirrors to sequentially distribute the light to the different sensors from a single entry point ?
If you were to capture at 1000fps, then each camera would ideally have a sample window within that 1/1000 sec.
Are the CMOS sensors designed for 30fps sensitive enough to capture a picture without having long exposure time? i.e. Can it handle the 1/1000 sec exposure time without a very dark image that is destroyed by the noise floor.
Would the sensor analog circuits have fast enough rise/fall time to have bandwidth(1) for that type of frequence response? The overall system bandwidth & sampling rate limits the time resolution and accuracy of such a system. (2)
The parent post is asking for the time uncertainty of when the accuracy sampling occurs. Nothing to do with time coding. Time coding is meaningless when you trigger is not accurate.
Note:
1. bandwidth = frequence bandwidth not data transfer rate in this context
2. This is reason why you have to use a scope (A/D) & probe (sensor) that are a few time faster than the rising/fall edge of the signal you are trying to measure. In a digital sampling scope, the A/D works very slowly by repeatly sampling on the repetitive waveform by introducing a time delay in each capture cycle to build up the information. The scope still need good frequence response to do so accurately.
Reading between the lines, they seem to have custom hardware and (maybe?) an MPEG encoder behind each camera, and a huge amount of software and general hassle to get an unwieldy and inflexible system to work at all. The upper limit on frame rate is about 5K/sec due to the integration time, but they would need about 160 cameras to achieve this continuously, and a hell of a lot of processing to produce sensible output. A lot of effort for something that isn't actually very useful.
For the same or less money/effort I have no doubt they could have either bought a purpose-made high-speed cam, or built one using something like This chip from Micron, which costs less than $2K and does 500 full-frame megapixel images per second, faster for partial frames. One neat feature is that it can effectively image individual lines at arbitary places in the frame at 500,000 per second - I'm sure these academic types could do some interesting interpolaty stuff with this to synthesise full-frame-like images at pretty high rates instead of messing with a system that doesn't have any realistic practical use.
First, this work is part of a larger research effort. In the past several years, cameras have become cheap, commodity devices, and you still get more processing power for the buck every year. I designed the Stanford Multiple Camera Array (http://graphics.stanford.edu/projects/array) not to be a high-speed camera, but to be a research tool for exploring the potential of large numbers of cheap image sensors and plentiful processing. High-speed video is one example of high-performance imaging using an array of cameras. We have also used our array for synthetic aperture photography, using many cameras to simulate a camera with a very large aperture. Such a camera has a very narrow depth of field, a property we exploit to look through partially occluding foreground objects like foliage. We are interested in view interpolation (Matrix-like effects, but with user control over the virtual camera viewpoint), too. If you want to learn more about the array and these applications, check out the links to our papers and my dissertation on the camera array website.
About the high-speed video work in particular, there are plenty of commercial high-speed cameras that run at higher frame rates than our camera array. If you want high-speed video camera, I recommend buying one of them. Using an array of cheap cameras has its disadvantages. You have to geometrically and radiometrically calibrate the data from all the different sensors, and in our case, we had to deal with the electronic rolling shutter. One benefit of this work for us was developing accurate and automatic (very important for 100 cameras) calibration methods for our array. An interesting property of the camera array approach is that parallel compression reduces the bandwidth so we can stream continuously. By contrast, as frame rate increase, most high-speed cameras are limited to recording durations that will fit in memory at the camera, usually well under one minute. That said, one could certainly design architectures to compress high-speed video in real-time.
What's most interesting to me about the high speed work is combining it with other multiple camera methods. One example is spatiotemporal view interpolation--capturing a bunch of images of a scene from different positions and times, then generating new views from positions and times not in the captured data. Think Matrix again, but with user control over the virtual camera view position and time. While the BulletTime setup from Manex captured one specific space-time camera trajectory, my goal is to capture images in a way that would let us create many different virtual camera paths later on. Traditional view interpolation methods use arrays of cameras synchronized to trigger simultaneously so they can reason about shape of the "frozen" scene, then infer how the scene is moving. In my thesis, I discuss how using the high-speed approach of staggered trigger times increases our temporal sampling resolution (effective frame rate) and can enable simpler interpolation methods. The interpolation algorithm I describe is also exactly the correction needed to eliminate the jitter due to parallax in the high-speed video sequences.
I've described just a few of the applications we've investigated using our camera array, but we hope this is just the tip of the iceberg. We're hard at work on new uses for the cameras, so stay tuned.