Slashdot Mirror
High-Speed Video Using a Dense Camera Array
karvind writes "Researchers at Stanford have demonstrated multi-thousand frame-per-second (fps) video using a dense array of cheap 30fps CMOS image sensors. A benefit of using a camera array to capture high speed video is that we can scale to higher speeds by simply adding more cameras. Even at extremely high frame rates, our array architecture supports continuous streaming to disk from all of the cameras. Now we know where to use 100TB tape drives and what to expect in the next sci-fi movie."
This is a very interesting development. If you watch the movies (especially the movie with balloon popping, I think its the third movie), you will see that this is an extremely accurate capture of the event. I would be interested to see how this could present itself in a regular consumer atmosphere...multiple cameras would not exactly make the cut. But yes, it does give a good idea on how to use the 100TB tape drives
Nothing to see here.
Quite convenient for a story about a slashdotted camera.
liqbase
How do they put all the footage together in the correct 'order', that is to say where each frame is in sequence.
How can they be sure that none of the cameras capture the same instant of the action?
To be honest, for the examples given, the image quality is somewhat lacking.
I can imagine this working REALLY well for crash simulation studies where the subject is a greater distance from the camera array.
The baloon popping movie needs quite severe modification to the captured images, and doesn't do much justice.
The effect has already been used in bullet time type fx effects anyway, they used an array of cameras firing in a sequence whilst laid around a subject.
The effect with bullet time is a lot cleaner because the captured images are not expected to be spatially aligned, and instead are made to give the effect of moving the camera around a subject.
liqbase
It's not that I don't think this is cool, because I can see all kinds of uses for this sort of thing.
But my question is this...
Are there any uses for high speed video capture that existing technologies weren't already well suited for, or is this just a cheaper and more readily available option?
"Everything you know is wrong. (And stupid.)"
Moderation Totals: Wrong=2, Stupid=3, Total=5.
If you are hard up for disk space for this, may I suggest emailing frames to this free email account
I know it's a hack, but whatever gets the job done, right??
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
will be a 2 hour long film of a pin dropping.
..ultra slow motion capture of a melting /.ed server. Hey I can see the individual /.ers GET / packets flowing through the fast ethernet port !
T think "cheap 30fps CMOS image sensors" simply refer to webcams. From the quality i've seen, they might be in the order of 20$ per unit, which makes to whole camera array about 1000$. Also those webcams do not produce thousand of megabytes, even at rates of 1MB/min you can get decent quality, which makes the video stream to be about 50MB/min.
And now we know what the quality aware consumer as a minimum should expect from our beloved video producers:
No less than 1000 fps facials.
Instead of having a hawkwardly swinging background, why wouldn't they use a set of rotating mirrors to sequentially distribute the light to the different sensors from a single entry point ?
I believe the point in producing this is to show that instead of purchasing expensive high-framerate cameras that has some sky-high pricetag, one can use an alternate and cheaper solution by using CMOS sensors. So let's try placing a couple of things into perspective.
Say you require a camera that can record say 90fps. To a manufacturer of electronic parts, this can be achieved with a little bit of engineering. Basically, take 3 of those 30fps CMOS sensors, pack them together, set a uniform color correction setting, and an interface to send the captured information to. All this would probably costs lots less than a specialized camera that can capture 100fps or so.
If you were to walk around in a computer shop or your local Best Buy, CompUSA, Microcenter, Circuit City, or Fry's, you might have noticed that there are quite a few webcams around, all of which uses the same 30fps CMOS described at the link. The CMOS themselves are likely cheap enough that, in theory, you can assemble a few together and have a 90fps camera. I would assume that the spatial distortion would be next to nil since 3 sensors are closely together.
Please correct me on any points. I believe the concept is interesting and would have some useful uses out there (ie. spectator sports, research, film).
~ Old Warriors Society
Reading between the lines, they seem to have custom hardware and (maybe?) an MPEG encoder behind each camera, and a huge amount of software and general hassle to get an unwieldy and inflexible system to work at all. The upper limit on frame rate is about 5K/sec due to the integration time, but they would need about 160 cameras to achieve this continuously, and a hell of a lot of processing to produce sensible output. A lot of effort for something that isn't actually very useful.
For the same or less money/effort I have no doubt they could have either bought a purpose-made high-speed cam, or built one using something like This chip from Micron, which costs less than $2K and does 500 full-frame megapixel images per second, faster for partial frames. One neat feature is that it can effectively image individual lines at arbitary places in the frame at 500,000 per second - I'm sure these academic types could do some interesting interpolaty stuff with this to synthesise full-frame-like images at pretty high rates instead of messing with a system that doesn't have any realistic practical use.
All of the CCDs run at 30 FPS. So, yes, they can scale this almost endlessly (until the parallax variation from non-colocated image captures becomes extreme, basically, or until the trigger timing required becomes too hard to achieve and jitter begins to be large enough to significantly alter your frame timing/sequencing).
The trick is that A runs on 0.000, 1.000, 2.000, etc. while B runs on 0.001, 1.001, 2.001, etc., C is on 0.002, 1.002, 2.002, etc. (units are frames relative to a starting time), and then the frames are sequenced appropriately (ABCABCABC etc.). This gives a very high frame rate while using relatively low-cost sensors - effectively, they're exploiting parallelism as a way to increase the array's effective sampling rate.
Basically, if you have N sensors capable of sampling X times per second each, and are capable of accurately triggering each sensor to a high degree of time accuracy, your effective sampling rate can be NX. Neat trick.
---
Mod me down, you fucking twits. Go ahead. I dare you.
(I read with sigs off.)
I downloaded their sample videos, but they keep playing really too slowly. I'm affraid their technology isn't quite ready yet ..
California Governor Leland Stanford employed Eadweard Muybridge to settle a bet whether a horse gallopss with all four feet off the ground. Muybridge took the first motion picture by chaining 16 cameras together. The horse farm of this experiment is tucked away in a corner of the Stanford college campus which was founded ten years later.
Yeah, they mention Photron in their paper. As nice as that camera is, it can only store a few seconds at 800x600. The system you are looking at will run till you run out of space. The paper is a well written 320kB pdf and more worth your download time than the movies themselves.
Now, here are a few thoughts of my own. Some of the image quality problems you notice might be a side effect of reducing the movie to something that can be downloaded and played by the average web surfer. Higher quality image capture devices will become cheaper and this method will improve with that. More importantly, this system seems to not take any non free software to use. A wizzbang camera soon becomes a big pain in the ass if it's tied to special drivers that tie you to a specific operating system on a specific computer. it may be more trouble than its worth, and may well be wise to buy a camera from the professionals.
What you use is up to you and your needs, but these people are NOT making their life difficult in a pointless exercise. They have met their needs in a real way and could have kept it to themselves. I'm happy they decided to share and realize that much of the difficult work is now simply done. The authors, by the way, are members of the EE and CS departments of Stanford University. That makes them pros to me, and I'm about as well off reading their manual as I am reading one from a camera maker.
Friends don't help friends install M$ junk.
Sure, it's a lot more expensive, but there's dedicated camera systems that'll do a million frames per second - and more.
:
One of the bigger problems, especially with this 'array', though has been noted above : exposure time.
This might be correctible post-shooting, though. As each frame's exposure will overlap the next, whatever is similar in both could be presumed a no-motion area. Gets quite tricky, though.
And of course the array posted about has parallax issues, etc. etc.
Here's a fun high-end-ish camera
http://www.cordin.com/productsie.html
The 510 at 25,000,000 fps for example. Only captures 48 frames, but that should be enough for something fun...
Light travels at ~300,000,000m/s
In the delta between frames*, light should thus travel 12 meters.
Over 48 frames, it should travel 576 meters.
In other words... if you set this camera up, hooked the shutter to a flash so that the flash fires the exact moment the camera starts its run, then you should be able to see the light travel down, say, a hallway.
Better yet...if the flash is short enough, you should see a 'shelled sphere' sort of shape pass through the hallway, and bounced light bounce off the walls to other objects where the direct light from the flash wouldn't reach.
Can't say I've seen any real-life animations of this, though. There's a few temporal raytracers that can do this.
* again: exposure time means there's some blurring. You don't take a picture of a single moment in time. If you did, you would likely get no picture at all as no photon / electron / film-state change would occur to be recorded.
First, this work is part of a larger research effort. In the past several years, cameras have become cheap, commodity devices, and you still get more processing power for the buck every year. I designed the Stanford Multiple Camera Array (http://graphics.stanford.edu/projects/array) not to be a high-speed camera, but to be a research tool for exploring the potential of large numbers of cheap image sensors and plentiful processing. High-speed video is one example of high-performance imaging using an array of cameras. We have also used our array for synthetic aperture photography, using many cameras to simulate a camera with a very large aperture. Such a camera has a very narrow depth of field, a property we exploit to look through partially occluding foreground objects like foliage. We are interested in view interpolation (Matrix-like effects, but with user control over the virtual camera viewpoint), too. If you want to learn more about the array and these applications, check out the links to our papers and my dissertation on the camera array website.
About the high-speed video work in particular, there are plenty of commercial high-speed cameras that run at higher frame rates than our camera array. If you want high-speed video camera, I recommend buying one of them. Using an array of cheap cameras has its disadvantages. You have to geometrically and radiometrically calibrate the data from all the different sensors, and in our case, we had to deal with the electronic rolling shutter. One benefit of this work for us was developing accurate and automatic (very important for 100 cameras) calibration methods for our array. An interesting property of the camera array approach is that parallel compression reduces the bandwidth so we can stream continuously. By contrast, as frame rate increase, most high-speed cameras are limited to recording durations that will fit in memory at the camera, usually well under one minute. That said, one could certainly design architectures to compress high-speed video in real-time.
What's most interesting to me about the high speed work is combining it with other multiple camera methods. One example is spatiotemporal view interpolation--capturing a bunch of images of a scene from different positions and times, then generating new views from positions and times not in the captured data. Think Matrix again, but with user control over the virtual camera view position and time. While the BulletTime setup from Manex captured one specific space-time camera trajectory, my goal is to capture images in a way that would let us create many different virtual camera paths later on. Traditional view interpolation methods use arrays of cameras synchronized to trigger simultaneously so they can reason about shape of the "frozen" scene, then infer how the scene is moving. In my thesis, I discuss how using the high-speed approach of staggered trigger times increases our temporal sampling resolution (effective frame rate) and can enable simpler interpolation methods. The interpolation algorithm I describe is also exactly the correction needed to eliminate the jitter due to parallax in the high-speed video sequences.
I've described just a few of the applications we've investigated using our camera array, but we hope this is just the tip of the iceberg. We're hard at work on new uses for the cameras, so stay tuned.