Slashdot Mirror
High-Speed Video Using a Dense Camera Array
karvind writes "Researchers at Stanford have demonstrated multi-thousand frame-per-second (fps) video using a dense array of cheap 30fps CMOS image sensors. A benefit of using a camera array to capture high speed video is that we can scale to higher speeds by simply adding more cameras. Even at extremely high frame rates, our array architecture supports continuous streaming to disk from all of the cameras. Now we know where to use 100TB tape drives and what to expect in the next sci-fi movie."
This is a very interesting development. If you watch the movies (especially the movie with balloon popping, I think its the third movie), you will see that this is an extremely accurate capture of the event. I would be interested to see how this could present itself in a regular consumer atmosphere...multiple cameras would not exactly make the cut. But yes, it does give a good idea on how to use the 100TB tape drives
Nothing to see here.
Quite convenient for a story about a slashdotted camera.
liqbase
Great, a new SFX trick.
How do they put all the footage together in the correct 'order', that is to say where each frame is in sequence.
How can they be sure that none of the cameras capture the same instant of the action?
It's not that I don't think this is cool, because I can see all kinds of uses for this sort of thing.
But my question is this...
Are there any uses for high speed video capture that existing technologies weren't already well suited for, or is this just a cheaper and more readily available option?
"Everything you know is wrong. (And stupid.)"
Moderation Totals: Wrong=2, Stupid=3, Total=5.
If you are hard up for disk space for this, may I suggest emailing frames to this free email account
I know it's a hack, but whatever gets the job done, right??
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
http://www.ranaventures.com/mirror
will be a 2 hour long film of a pin dropping.
First of all, the idea is as old as moving pictures: using sequentially triggered multiple cameras was the first approach for capturing motion sequences ever used. This work, using digital cameras, doesn't actually seem to do much about the problems that arise from such an arrangement.
I can almost hear it hitting the floor in time to his brain counting all the money from reissuing all six movies in his new patented "MegaFrame 3000" format. You know he won't be able to help himself.
What use is a high-speed camera if it's shutter time is fixed to 1/30s (I guess that's what it is, since those are 30fps cameras, no further information on the homepage).
Yeah, you can get super-smooth animations of "slow" events but there's no way you could capture a bullet penetrating an apple, or any other high-speed events.
I've seen the bullet hitting the tomato in slomo before..
What makes this special?
..ultra slow motion capture of a melting /.ed server. Hey I can see the individual /.ers GET / packets flowing through the fast ethernet port !
T think "cheap 30fps CMOS image sensors" simply refer to webcams. From the quality i've seen, they might be in the order of 20$ per unit, which makes to whole camera array about 1000$. Also those webcams do not produce thousand of megabytes, even at rates of 1MB/min you can get decent quality, which makes the video stream to be about 50MB/min.
And now we know what the quality aware consumer as a minimum should expect from our beloved video producers:
No less than 1000 fps facials.
:P
-William Shatner can be neither created nor destroyed.
What about the shutter speed of the CCD? Obviously, you cannot scale this endlessly.
Instead of having a hawkwardly swinging background, why wouldn't they use a set of rotating mirrors to sequentially distribute the light to the different sensors from a single entry point ?
anyone have any idea if this technology will be applicable for robotics, and studying organic motion? i know high speed cameras have been used in the past to study insect motion and stuff, any idea if this will aid in that area of research, or are current cameras already fast enough?
Famous Last Words: "hmm...wikipedia says it's edible"
Reading between the lines, they seem to have custom hardware and (maybe?) an MPEG encoder behind each camera, and a huge amount of software and general hassle to get an unwieldy and inflexible system to work at all. The upper limit on frame rate is about 5K/sec due to the integration time, but they would need about 160 cameras to achieve this continuously, and a hell of a lot of processing to produce sensible output. A lot of effort for something that isn't actually very useful.
For the same or less money/effort I have no doubt they could have either bought a purpose-made high-speed cam, or built one using something like This chip from Micron, which costs less than $2K and does 500 full-frame megapixel images per second, faster for partial frames. One neat feature is that it can effectively image individual lines at arbitary places in the frame at 500,000 per second - I'm sure these academic types could do some interesting interpolaty stuff with this to synthesise full-frame-like images at pretty high rates instead of messing with a system that doesn't have any realistic practical use.
I downloaded their sample videos, but they keep playing really too slowly. I'm affraid their technology isn't quite ready yet ..
Now they are CMOS, instead of plate cameras...
"Flyin' in just a sweet place,
Never been known to fail..."
Sorry, the required cliche here involved beowulfs ;)
It would be pretty cool (and extremely expensive) if 50 of those 'Photron FASTCAM-X 1280s" were setup in a similar fashion (16,000 fps each). Though I cant really think of any uses for 800,000 fps cameras..
A more recent application is the "bullet time" developed for "The Matrix" movies.
This is a neat tool for the amature scientist who can't afford hundred-thousand-dollar high speed cameras for doing research. Unfortunately, it is just hack, and the constantly shifting/rolling perspective makes it impractical for research. The builders might consider stacking the camera units so that the lens apertures are closer to the centerline, since a cm or two in focal length won't distort the resultant video as much as a few degrees of divergence.
Another thought which would make this both competitive in image quality and economical is if the CMOS imagers - which are actually only a few millimeters in size - were etched onto a single piece of silicon in a phased array. Then, instead of being a few centimeters divergent, they'd be a millimeter. A few oildrop lenses, and the whole array might be the size of the palm of your hand, coming out to less than a degree at a meter or two.
even more ridiculously drawn out slowmotion scenes than in alexander?
world was created 5 seconds before this post as it is.
Judging from the images, they could have benefitted from use of evacuated glass bulbs containing resistance-heated tungsten filaments, arrayed in quantity such that the pictures aren't so *damned dark*. (They can afford 52 CMOS sensors, but where's the friggin lighting?)
I'd have a personalized plate on my car, but "toxic bachelor" won't fit into 7 letters.
I can see where this might be developed into some interesting tech but I think they need to come up with a way to overcome the slightly shifted perspective problem. The moving background is interesting but ultimately distracting.
If they were to channel the optics through a single lens somehow and then divided the light among the many cameras, they'd come up with something much more seamless. I think that would be really REALLY expensive and maybe even impossible. Another possibility would be to create a projected image of whatever is going through the main lens (thereby fixing the perspective) and then training the multiple cameras on the single projected image. It would fix the shifting perspective problem but the slightly varying angles of the projected image might be just as bad... I guess it depends on the amount of variance in the angles.
Anyway, That's my additional input on the idea... could make a really interesting project for sure.
It will be a galaxy-spanning space epic about the disaster that befalls the new purser on the Star Galleon "WangChung" and his deserate fight to defeat the evil thing that made the space-virus that turns his shipmates into zombies and save the planet Zorkon-9 from a Terrible Fate. The working title is "Faster Wolfenstein! Kill, Kill!", and early reports say that Dave Callahan has been attached as scriptwriter.
--
"Outlook not so good." That magic 8-ball knows everything! I'll ask about Exchange Server next.
This was captured with my digital 'sideline' camera as I captured the individual frames on film.2 0.jpg
_ RPD_PPost_lut.jpg
http://www.gotsheep.com/~hirsch/Photos/DCP_0492_3
I've found the digital file but not the film that I scanned it from- blowing up eggs is MUCH more fun.
http://www.gotsheep.com/~hirsch/Photos/EGG_3_crop
(Slashdot is doing wierd things to the links- so you'll have to remove the %20's it's sticking in in the spaces)
Has anyone thought about the fact that cheap CMOS cameras have a particularly long charge time, meaning that by the time you get enough energy onto the CCD to render an image, the object has undoubtedly moved enough to produce a blur (and more than the desired amount of time has probably elapsed before you go on to the next image)? Either an extremely bright light will be needed, or you will get significantly sub-par image quality.
Technology sufficiently advanced to capture the finer details of my cumshots.
The women should be so lucky.
Use ISO 8601 dates [YYYY-MM-DD]
One reason you haven't had camera arrays capture your body movements and translate them into 3d for cool fighting video games is that the frame rate on cameras was too low. You'd get blurs in frames using a 30fps camera. I wonder if you still get blurs, or if you get an exact picture of where someone is at. Street Fighter where you actually punch and dodge would be nice, or some midevil sword game.
God spoke to me.
California Governor Leland Stanford employed Eadweard Muybridge to settle a bet whether a horse gallopss with all four feet off the ground. Muybridge took the first motion picture by chaining 16 cameras together. The horse farm of this experiment is tucked away in a corner of the Stanford college campus which was founded ten years later.
Um, hello people! There's been prior art here...
FYI, I'm not affiliated to the manufacturer, but I do operate this kind of camera as a part time job. Hopefully it's OK for me to post to this thread, since I'm too far away (Finland) from most of you to sell my services.
Citius Imaging manufactures affordable digital high speed cameras. AFAIK, you can get one for under 15000 euros.
Some sample videos which I have shot can be found here.
It's not a *new* idea. . .
(think Muybridge)
These are my friends, See how they glisten. See this one shine, how he smiles in the light.
If you have a CRT monitor and a toy gyroscope, you can demonstrate this yourself. Give the gyro a good spin and hold it up in front of the CRT. Look at the CRT through the spinning spokes of the gyro. You will see a strobe effect, and on one side the gyro spokes will appear compressed, and elongated on the other side. If the gyro is spinning clockwise, it will be the left side which appears compressed, and vice versa.
This happens for the same reason: the screen is illuminated by a downward-scanning raster, and one side of the gyro is rotating with this motion, while the other side rotates against it.
Yeah, they mention Photron in their paper. As nice as that camera is, it can only store a few seconds at 800x600. The system you are looking at will run till you run out of space. The paper is a well written 320kB pdf and more worth your download time than the movies themselves.
Now, here are a few thoughts of my own. Some of the image quality problems you notice might be a side effect of reducing the movie to something that can be downloaded and played by the average web surfer. Higher quality image capture devices will become cheaper and this method will improve with that. More importantly, this system seems to not take any non free software to use. A wizzbang camera soon becomes a big pain in the ass if it's tied to special drivers that tie you to a specific operating system on a specific computer. it may be more trouble than its worth, and may well be wise to buy a camera from the professionals.
What you use is up to you and your needs, but these people are NOT making their life difficult in a pointless exercise. They have met their needs in a real way and could have kept it to themselves. I'm happy they decided to share and realize that much of the difficult work is now simply done. The authors, by the way, are members of the EE and CS departments of Stanford University. That makes them pros to me, and I'm about as well off reading their manual as I am reading one from a camera maker.
Friends don't help friends install M$ junk.
Sure, it's a lot more expensive, but there's dedicated camera systems that'll do a million frames per second - and more.
:
One of the bigger problems, especially with this 'array', though has been noted above : exposure time.
This might be correctible post-shooting, though. As each frame's exposure will overlap the next, whatever is similar in both could be presumed a no-motion area. Gets quite tricky, though.
And of course the array posted about has parallax issues, etc. etc.
Here's a fun high-end-ish camera
http://www.cordin.com/productsie.html
The 510 at 25,000,000 fps for example. Only captures 48 frames, but that should be enough for something fun...
Light travels at ~300,000,000m/s
In the delta between frames*, light should thus travel 12 meters.
Over 48 frames, it should travel 576 meters.
In other words... if you set this camera up, hooked the shutter to a flash so that the flash fires the exact moment the camera starts its run, then you should be able to see the light travel down, say, a hallway.
Better yet...if the flash is short enough, you should see a 'shelled sphere' sort of shape pass through the hallway, and bounced light bounce off the walls to other objects where the direct light from the flash wouldn't reach.
Can't say I've seen any real-life animations of this, though. There's a few temporal raytracers that can do this.
* again: exposure time means there's some blurring. You don't take a picture of a single moment in time. If you did, you would likely get no picture at all as no photon / electron / film-state change would occur to be recorded.
Does anyone know, of camera or camera units that can autoscan a given area, like a playground and generate images of various sections of the area ? I believe, one would have to auto mechanically move camera to aim at a section, click and move on to the next.
First, this work is part of a larger research effort. In the past several years, cameras have become cheap, commodity devices, and you still get more processing power for the buck every year. I designed the Stanford Multiple Camera Array (http://graphics.stanford.edu/projects/array) not to be a high-speed camera, but to be a research tool for exploring the potential of large numbers of cheap image sensors and plentiful processing. High-speed video is one example of high-performance imaging using an array of cameras. We have also used our array for synthetic aperture photography, using many cameras to simulate a camera with a very large aperture. Such a camera has a very narrow depth of field, a property we exploit to look through partially occluding foreground objects like foliage. We are interested in view interpolation (Matrix-like effects, but with user control over the virtual camera viewpoint), too. If you want to learn more about the array and these applications, check out the links to our papers and my dissertation on the camera array website.
About the high-speed video work in particular, there are plenty of commercial high-speed cameras that run at higher frame rates than our camera array. If you want high-speed video camera, I recommend buying one of them. Using an array of cheap cameras has its disadvantages. You have to geometrically and radiometrically calibrate the data from all the different sensors, and in our case, we had to deal with the electronic rolling shutter. One benefit of this work for us was developing accurate and automatic (very important for 100 cameras) calibration methods for our array. An interesting property of the camera array approach is that parallel compression reduces the bandwidth so we can stream continuously. By contrast, as frame rate increase, most high-speed cameras are limited to recording durations that will fit in memory at the camera, usually well under one minute. That said, one could certainly design architectures to compress high-speed video in real-time.
What's most interesting to me about the high speed work is combining it with other multiple camera methods. One example is spatiotemporal view interpolation--capturing a bunch of images of a scene from different positions and times, then generating new views from positions and times not in the captured data. Think Matrix again, but with user control over the virtual camera view position and time. While the BulletTime setup from Manex captured one specific space-time camera trajectory, my goal is to capture images in a way that would let us create many different virtual camera paths later on. Traditional view interpolation methods use arrays of cameras synchronized to trigger simultaneously so they can reason about shape of the "frozen" scene, then infer how the scene is moving. In my thesis, I discuss how using the high-speed approach of staggered trigger times increases our temporal sampling resolution (effective frame rate) and can enable simpler interpolation methods. The interpolation algorithm I describe is also exactly the correction needed to eliminate the jitter due to parallax in the high-speed video sequences.
I've described just a few of the applications we've investigated using our camera array, but we hope this is just the tip of the iceberg. We're hard at work on new uses for the cameras, so stay tuned.
looking for practicality? try watching boobies at 1560fps.
[ogle]
fascinating!
[/ogle]
I bet even with a camera that fast the wedding shots will still manage to get someone with their eyes shut.
A big part of the research of the group is to come up with the mathematics to create a "constant" image. Right now I think they're using simple perspective projections to make objects in the plane of the balloon appear in focus.
I think that it is possible to make objects in a particular depth plane appear non-shifty (even from the same set of sample data). Making the entire background non-shifty would be a matter of properly segmenting the video so that various regions can be mapped to the right depth planes.
It wouldn't work, because at any one instant in time, several of the cameras are taking a picture at once. This is why in the fan video, the blades are warped - they are moving as the image is being scanned from the sensor.
This means that to get an image from a single moment in time, you need to take strips from all the cameras that are taking a picture at one time and splice them together.
So the difficulty with a rotating mirror system would be splitting the light between several cameras at once. Also, the images would be darker, as you're effectively splitting one camera's light between several.
I couldn't help but notice that this looks like bullet time shot from a circular array, which gave me ideas about improved "3D" recording. Of course the left-right anthropic 3D model is still cool for human viewing (even that might benefit by recording more frames/sec and multiple angles for display). I suggest the MIT folks consider the possible applications of such an array (or perhaps a circular or spherical array) in robotic vision.
... but why? It's not like the terrorists need thosuands of frames per second. Although I'm sure a thousands-of-frames-per-second film clip as the sniper bullet hits would spread like a bad social disease across the Internet.
This sig has absolutely no significance and serves only to take up screen space and waste the time of the reader.