Slashdot Mirror
High-Speed Video Using a Dense Camera Array
karvind writes "Researchers at Stanford have demonstrated multi-thousand frame-per-second (fps) video using a dense array of cheap 30fps CMOS image sensors. A benefit of using a camera array to capture high speed video is that we can scale to higher speeds by simply adding more cameras. Even at extremely high frame rates, our array architecture supports continuous streaming to disk from all of the cameras. Now we know where to use 100TB tape drives and what to expect in the next sci-fi movie."
This is a very interesting development. If you watch the movies (especially the movie with balloon popping, I think its the third movie), you will see that this is an extremely accurate capture of the event. I would be interested to see how this could present itself in a regular consumer atmosphere...multiple cameras would not exactly make the cut. But yes, it does give a good idea on how to use the 100TB tape drives
Nothing to see here.
Quite convenient for a story about a slashdotted camera.
liqbase
How do they put all the footage together in the correct 'order', that is to say where each frame is in sequence.
How can they be sure that none of the cameras capture the same instant of the action?
To be honest, for the examples given, the image quality is somewhat lacking.
I can imagine this working REALLY well for crash simulation studies where the subject is a greater distance from the camera array.
The baloon popping movie needs quite severe modification to the captured images, and doesn't do much justice.
The effect has already been used in bullet time type fx effects anyway, they used an array of cameras firing in a sequence whilst laid around a subject.
The effect with bullet time is a lot cleaner because the captured images are not expected to be spatially aligned, and instead are made to give the effect of moving the camera around a subject.
liqbase
..ultra slow motion capture of a melting /.ed server. Hey I can see the individual /.ers GET / packets flowing through the fast ethernet port !
T think "cheap 30fps CMOS image sensors" simply refer to webcams. From the quality i've seen, they might be in the order of 20$ per unit, which makes to whole camera array about 1000$. Also those webcams do not produce thousand of megabytes, even at rates of 1MB/min you can get decent quality, which makes the video stream to be about 50MB/min.
I don't think so.
The more I look at this, the more I think they are making life difficult for themselves, and the resultant image quality shows.
Since making my first postings on this discussion, I decided to have a look around at how the professionals handle high speed photography and came up with some nice results.
Theres a company called Photron that have a range of single digital cameras capable of megapixel images at 2000fps.
In their gallery, they even have an example of a water filled baloon popping, and tbh it looks a lot better than this multi camera version.
Agreed, this is a way to do it on the cheap, but because of the spatial issues and timing complexities, it may be more trouble than its worth, and may well be wise to buy a camera from the professionals.
liqbase
Instead of having a hawkwardly swinging background, why wouldn't they use a set of rotating mirrors to sequentially distribute the light to the different sensors from a single entry point ?
I believe the point in producing this is to show that instead of purchasing expensive high-framerate cameras that has some sky-high pricetag, one can use an alternate and cheaper solution by using CMOS sensors. So let's try placing a couple of things into perspective.
Say you require a camera that can record say 90fps. To a manufacturer of electronic parts, this can be achieved with a little bit of engineering. Basically, take 3 of those 30fps CMOS sensors, pack them together, set a uniform color correction setting, and an interface to send the captured information to. All this would probably costs lots less than a specialized camera that can capture 100fps or so.
If you were to walk around in a computer shop or your local Best Buy, CompUSA, Microcenter, Circuit City, or Fry's, you might have noticed that there are quite a few webcams around, all of which uses the same 30fps CMOS described at the link. The CMOS themselves are likely cheap enough that, in theory, you can assemble a few together and have a 90fps camera. I would assume that the spatial distortion would be next to nil since 3 sensors are closely together.
Please correct me on any points. I believe the concept is interesting and would have some useful uses out there (ie. spectator sports, research, film).
~ Old Warriors Society
Reading between the lines, they seem to have custom hardware and (maybe?) an MPEG encoder behind each camera, and a huge amount of software and general hassle to get an unwieldy and inflexible system to work at all. The upper limit on frame rate is about 5K/sec due to the integration time, but they would need about 160 cameras to achieve this continuously, and a hell of a lot of processing to produce sensible output. A lot of effort for something that isn't actually very useful.
For the same or less money/effort I have no doubt they could have either bought a purpose-made high-speed cam, or built one using something like This chip from Micron, which costs less than $2K and does 500 full-frame megapixel images per second, faster for partial frames. One neat feature is that it can effectively image individual lines at arbitary places in the frame at 500,000 per second - I'm sure these academic types could do some interesting interpolaty stuff with this to synthesise full-frame-like images at pretty high rates instead of messing with a system that doesn't have any realistic practical use.
All of the CCDs run at 30 FPS. So, yes, they can scale this almost endlessly (until the parallax variation from non-colocated image captures becomes extreme, basically, or until the trigger timing required becomes too hard to achieve and jitter begins to be large enough to significantly alter your frame timing/sequencing).
The trick is that A runs on 0.000, 1.000, 2.000, etc. while B runs on 0.001, 1.001, 2.001, etc., C is on 0.002, 1.002, 2.002, etc. (units are frames relative to a starting time), and then the frames are sequenced appropriately (ABCABCABC etc.). This gives a very high frame rate while using relatively low-cost sensors - effectively, they're exploiting parallelism as a way to increase the array's effective sampling rate.
Basically, if you have N sensors capable of sampling X times per second each, and are capable of accurately triggering each sensor to a high degree of time accuracy, your effective sampling rate can be NX. Neat trick.
---
Mod me down, you fucking twits. Go ahead. I dare you.
(I read with sigs off.)
I downloaded their sample videos, but they keep playing really too slowly. I'm affraid their technology isn't quite ready yet ..
First, this work is part of a larger research effort. In the past several years, cameras have become cheap, commodity devices, and you still get more processing power for the buck every year. I designed the Stanford Multiple Camera Array (http://graphics.stanford.edu/projects/array) not to be a high-speed camera, but to be a research tool for exploring the potential of large numbers of cheap image sensors and plentiful processing. High-speed video is one example of high-performance imaging using an array of cameras. We have also used our array for synthetic aperture photography, using many cameras to simulate a camera with a very large aperture. Such a camera has a very narrow depth of field, a property we exploit to look through partially occluding foreground objects like foliage. We are interested in view interpolation (Matrix-like effects, but with user control over the virtual camera viewpoint), too. If you want to learn more about the array and these applications, check out the links to our papers and my dissertation on the camera array website.
About the high-speed video work in particular, there are plenty of commercial high-speed cameras that run at higher frame rates than our camera array. If you want high-speed video camera, I recommend buying one of them. Using an array of cheap cameras has its disadvantages. You have to geometrically and radiometrically calibrate the data from all the different sensors, and in our case, we had to deal with the electronic rolling shutter. One benefit of this work for us was developing accurate and automatic (very important for 100 cameras) calibration methods for our array. An interesting property of the camera array approach is that parallel compression reduces the bandwidth so we can stream continuously. By contrast, as frame rate increase, most high-speed cameras are limited to recording durations that will fit in memory at the camera, usually well under one minute. That said, one could certainly design architectures to compress high-speed video in real-time.
What's most interesting to me about the high speed work is combining it with other multiple camera methods. One example is spatiotemporal view interpolation--capturing a bunch of images of a scene from different positions and times, then generating new views from positions and times not in the captured data. Think Matrix again, but with user control over the virtual camera view position and time. While the BulletTime setup from Manex captured one specific space-time camera trajectory, my goal is to capture images in a way that would let us create many different virtual camera paths later on. Traditional view interpolation methods use arrays of cameras synchronized to trigger simultaneously so they can reason about shape of the "frozen" scene, then infer how the scene is moving. In my thesis, I discuss how using the high-speed approach of staggered trigger times increases our temporal sampling resolution (effective frame rate) and can enable simpler interpolation methods. The interpolation algorithm I describe is also exactly the correction needed to eliminate the jitter due to parallax in the high-speed video sequences.
I've described just a few of the applications we've investigated using our camera array, but we hope this is just the tip of the iceberg. We're hard at work on new uses for the cameras, so stay tuned.