Slashdot Mirror
Digital Video Capture and High Frame Rates?
Jeff asks: "So the folks at a place called Conniption Films (great name) developed a camera called the Millisecond Camera which can shoot 12,000 frames of film a second. I read the article and thought 'Hmm that's neat' but then realized they were still using an analog process for shooting this highspeed film. Being a geek, not necessarily into the film side of things but curious nonetheless, I wonder, shouldn't a computer be able to do a better job of such a thing? They say the film runs around a spindle going 500 mph (!). Wouldn't that be prone to failure and use alot of energy? Wouldn't it be more appropriate, easier, and overall cheaper to just hook up a high res CCD to a beowulf </duck> cluster of 2 ghz+ machines and capture high speed images that way? Why hasn't it been done yet? Or has it and I haven't seen it yet?" I did a double-take, when I first read this question, and then got curious and did a little digging. Turns out, high frame rates are not exclusive to the analog photography world, and to illustrate my point, I provide this link. It's woefully short on details, and the explanations as to why a camera that can record 1M frames per second is limited to a playback of only 103 frames, but the technology is out there. Has anyone seen any other digital cameras out there with high frame-rates? What visual mischief could you aspiring photographers get into with such a camera?
are often a bit delayed. they're slow. that's it.
The problem is the bandwidth.
Small, 8bit color uncompressed movie at 300x300 pixels would require something like 8 billion bits per second. (300 * 300 * 12000 * 8)
Now we probably want more resolution & a higher bit depth, so multiply apporpriatly.
What are we going to use to transfer that much data around a cluster? Or even just from the camera to the cluster?
The explanation as to why it can only play back 103 frames is QUITE clear... the chip has 103 "on-chip" memory buffers per sensor, and they get cyclicly overwritten with the last 103 frames.
This overcomes the bottleneck of trying to transfer data off the CCD at such high frame rates in real time, but limits you to "downloading" the last 103 frames after-the fact from the chip.
MadCow.
I used to have a sig, but I set it free and it never came back.
A quick calculation on the bandwidth of capturing 12000 SVGA-resolution full color frames per second:
1024 (width) * 768 (height) * 4 (32-bit color) * 12000 (fps) = 377,487,360,00 bytes/second (35 Gbytes/s)
So no wonder they use film...
CCD simply needs a few milliseconds to regain their 0-volt signal level again before they can emit a new pulse. This recoverytime makes it unsuitable for high speed filming. Helas.
What visual mischief could you aspiring photographers get into with such a camera?
I have to say the obligatory ultra slo mo pron!
Actually fact is, the adult industry often drives the need for newer technologies I've read.
Is CCD the ONLY way to capture light digitally ???
Sig (appended to the end of comments you post, 120 chars)
also consider that most of the time, people that are interested in such frame rates, are also *very* interested in having detailed high-resolution frames of the event at 'interesting' times.
This probably means having to shoot images of around 4-6 megapixels, and I really don't see any way of doing that at the speed needed for this kind of application.
The only way might be exactly what the poster of the topic didn't grasp: have a camera that can take 100-1000 pictures at a 1Mpics/sec frame rate and store them in ultra-fast local memory, and transfer them out at leisure, with a good triggering setup, 100-1000 microseconds worth of data might just be enough for certain applications.
-- the cake is a lie
I have a Sony DSC-F707. It takes beautiful pictures but only has enough buffer memory for 3 burst pictures. With higher resolution images (akin to film level quality) you'd need way more memory and throughput than can be supported with traditional flash memory. A external drive mechanism won't work either (i.e. bluetooth) because of the throughput necessary to sustain something at the rate discussed. I mean camera's today can't even do mpeg compression decently.
At 640x240x24, you're talking 7372800 bits per frame. At 1000 frames/second, we'd need to be transferring 7Gbps. That would be a bit hard to handle. You could cut the rate by dropping colors. At B/W, it'd be pretty manageable, but that's probably not what you want. You probably also want higher resolution. No matter what, you wouldn't be able to swallow the stream for long.
Oh, and by the way. The confusion about a million frames/second versus 103 was just poor word choice in the article. What they mean is a 1 microsecond shutter speed - 1 microsecond frames with 9707 microsecond gaps. Great stop-action to cut blurring, but manageable transfer rates.
A. El Gammal, et al. published a 10,000fps imager with a 352x288 pixel resolution. This guy can maintain the full speed indefinately. Unfortunately is it not a commercial device, but something similar will probably be available within a few years.
Kleinfelder, S. SukHwan Lim Xinqiao Liu El Gamal, A. "A 10000 frames/s CMOS digital pixel sensor", Solid-State Circuits, IEEE Journal of. v38 n12, pp. 2049-2059. Feb. 2001.
The abstact is as follows:
A 352 x 288 pixel CMOS image sensor chip with per-pixel single-slope ADC and dynamic memory in a standard digital 0.18um CMOS process is described. The chip performs "snapshot" image acquisition, parallel 8-bit A/D conversion, and digital readout at continuous rate of 10000 frames/s or 1 Gpixels/s with power consumption of 50 mW. Each pixel consists of a photogate circuit, a three-stage comparator, and an 8-bit 3T dynamic memory comprising a total of 37 transistors in 9.4x9.4 um with a fill factor of 15%. The photogate quantum efficiency is 13.6%, and the sensor conversion gain is 13.1uV/e. At 1000 frames/s, measured integral nonlinearity is 0.22% over a 1-V range, rms temporal noise with digital CDS is 0.15%, and rms FPN with digital CDS is 0.027%. When operated at low frame rates, on-chip power management circuits permit complete powerdown between each frame conversion and readout. The digitized pixel data is read out over a 64-bit (8-pixel) wide bus operating at 167 MHz, i.e., over 1.33 GB/s. The chip is suitable for general high-speed imaging applications as well as for the implementation of several still and standard video rate applications that benefit from high-speed capture, such as dynamic range enhancement, motion estimation and compensation, and image stabilization.
You normaly use this kind of speed if you want to do motion analysis.
Say your company does crash tests or you want to find out what happens when something explodes.
if you frame size was 30cm squared (about a foot) then 12000 frames per second would allow you to capture 10 frames of somthing travelling at about the speed of sound(i think my math is correct!)
or a few frames of a bullet.
thank God the internet isn't a human right.
From reading the article, the bandwidth problem was solved by giving each pixel of the camera it's own memory. One problem that I can see is that: This is going to eat space on the chip that would normally be used for imaging. If you put too much memory around a pixel, you're going to start suffering in the quality of the image. (and they already had to increase the size of each pixel to be able to capture the light fast enough)
It would seem that they pegged the usable tradeoff at 103 samples per pixel, so that's how many images you can store.
Free Software: Like love, it grows best when given away.
Many people seem to think that digital image quality is superior to analog. This is untrue. So anything you want to analyze by looking at it, you want analog film. Digital advantages are: computer analysis, reproduceability.
Now to your question, the primary digital disadvantage is bandwidth.
Digital images have a very specific size: 1024x768x32 = 3 MegaBytes
Analog images have virtually unlimited sizes (infinite x infinite x infinite). Some people have tried to estimate the resolution of analog images, and they best they come up with is a vertical and horizontal resolution in the thousands, however this is unreasonable. Analog images are more detailed than that.
Now bandwidth calculation:
(size of single frame)(frame rate) = (bandwidth)
(3MB)(12000) = 36000 MBps
So, we are looking at processing and storing about 3.6 Gigabytes per second. I mean processing because we want to use lossless compression, and this would require some very specialized hardware to handle this framerate. This cannot be processed or stored in real time on any modern generalized computer. It should be possible to build a specialized machine to accomplish this.
I have discounted limitations in CCD speed, possibility of using mulitple cameras, high-end hardware I don't know about.
Conclusions: "Digital" is not the panacea. Visual image analysis should always be done with analog film. Digitization is good for reproducing images, and transporting them intact. A camera that does 12k fps is mostly for image analysis of high velocity and high acceleration objects, for analysis in a lab. There are applications of high speed digital imagery, but I don't know any offhand.
Finally, using a computer to process the resulting data takes a substantial amount processing time. So the answer to the question "why not use digital cameras" is "why would you need to?" If you can justify the need, do it. It will require, however, substantial resources which also need to be justified.
For amateur photography, don't worry about a 12k fps camera, stick with the 30fps DV handicams.
Torsten
There are a lot of limits when it comes to cameras connected to PC's. I've worked in a lab where we used cameras that generated 640x480x4 (32 bit color) frames at 60 Hz. Guess what. You can't even buy a HD that can sustain that kind of transfer rate for any period of time. Good thing those computers had about a gig of ram each ;-)
There is actually a few limitations. Bandwidth is the most important one. Here you're looking at the connection between camera and the computer. We used special frame grabber boards, fireware or USB - well... nothing that I know of can handle 12,000 Hz. Next, somewhat smaller limitation is the bandwidth to memory. When you're talking about 12,000 Hz - that will become a factor. And of course - unless you've got about 40 Gigs of Ram (at least) you would want to save the stream. There are Video Vaults which are basically raid arrays, but again - they can't handle this kind of data stream.
Technology is coming along though. The new CMOS based cameras can have fairly high frame rates. You can actually select between resolution and framerate. Last time I checked the fastest they could go was about 500fps (at low resolution), the limit being again the link between the camera and PC. I believe the theoretical limit of the CMOS type camera is at either 5000 or 8000 fps (I don't really remember which - sorry).
"why the hell would anybody need 12000 frames per second. The human eye cant process all that plus nothing happens so fast you need 12000 shots of it in a second. This is just plain stupid and its a waste of film."
Because certian events, despite what you might think, *do* occur within 1000ths of a second. (The fireball from a nucelar blast for instance.)
Good cammera's shutter speeds tend to go up to 1/1000th of a second, and can go up to 1/8000th.
As far as the humman eye comment, well, just because you record at 12000 fps, doesn't mean you play it back at 12000 fps...
As some have said already... the bandwidth is the problem, but even after that, you still have to write it pretty damn fast... so here is the obvious solution... a cluster... per pixel.
Rather than just have ram per pixel as the article says they did, for digital, have a single computer unit per pixel. So say you want a 4megapixel resolution full motion video. Then you get 4 million computers each processing a single pixel. That should be plenty fast enough to get some very high speeds (assuming the ccd can handle it).
Of course, the problem now is to tie all that data together into a single video... and even then to find a machine to play something like that, though i supposed you could take each 4 million machines and have them each play their data into a single pixel on an lcd.
But then, why not just use film?
There is another form of image sensor called a CMOS (complementary metal oxide semiconductor) sensor, and it is usually used in cheaper devices like web cams, or digital cameras under $100, since it's much cheaper to manufacture. As a result CMOS images generally worse.
To-do List: Receive telemarketing call during a tornado warning. Check.
On the pure digital front, there are units that can record 1000 FPS continuous at 512 x 512 pixels. The system is data-rate limited. The imager can go much faster; if you cut the image size down to 32x128 pixels, you can get 32K frames/sec. At 128 x 128, you can get 11.2K frames/sec. The data goes into a buffer in the control unit (1 GB, typically), and is read out via FireWire. So this system can take a lot more frames than the device described in the article, which stores the images in memory within the imager and can only store 100 images or so.
--Mike
"Not an actor, but he plays one on TV."
Most beowulfs use a gigE interconnect... perhaps have two or three NICs per node, one for the system interconnect, and the other two for connection to the CCD. The CCD module could easily be broken down into a grid of virtual segments, each with its own cache and a gigE interconnect.
Another option is to wait for 10gigE (along with the rest of the supercomputing world) or go with Myrinet, which has recently broken the 1 gigabit barrier.
As I pointed out in another post, a good option would be to use several gigE interconnects to connect the CCD module to the many nodes of a beowulf cluster. Besides, you're going to need a cluster to manage that much data anyway.
Picture an image sensor as a one-inch-square array of pixels. If the frame rate is 30 per second, then 1/30 (3.3%) of the light that falls on the array makes up each frame.
If the frame rate is 12k/second then only 1/12000 (0.0083%) of the light can be used to make each frame. That means that the CCD must be 4000 times more sensitive to light, or you must use a light source that is 4000 times brighter, to get the same results.
And that ignores the fact that solid state light-to-electricity convertors like CCDs have a certain "latency" or "stickiness". Like the effect that the eye sees after a watching a flashbulb, CCDs suffer from after-images, and the brighter the light the worse the problem. Film doesn't have that problem because each frame is exposed on a new "receptor", i.e. a new piece of film.
Shouldn't it be ?
You can get around any bandwidth issues with a sufficiently large amount of cabling. The whole idea of doing this in parellel implies that. Anyway, compare the bandwidth of digital photography with the physical bandwidth of looping film through an eyepiece at 12,000 frames per second and you come up with a very different problem -- you've got to use TINY film, with an effective resolution much lower than what some of you linux numbercrunchers are assuming. "SVGA resolutions?" Think more like 320x240 -- and don't expect more than a few seconds per cannister, high costs, etc.
.000083 s. With low light, you need extremely sensitive equipment to even detect it and even more sensitive equipment to detect the subtle variations in wavelength that make up colors. Today's CCD cameras are very slow to register intensity light -- much slower than film. The chemical reaction in film triggered by exposure can be controlled much better, simply by changing the tolerance of the film -- which is why your high end, high speed shutter digital cameras are so godawful expensive. The $2500 Canon I've been looking at has roughly the same shutter speed as an equivalent $300 film camera. The extra price is NOT a "coolness" tax...it's for the set of three extremely high res CCD sensors and the chips capable of processing their information at that speed. My film prof used to say "digital ain't digital"...there's a quality factor of all digital electronics that can be poiled down to the quality of interpolation, quality of the ADC and of transistors leading up to it.
No, the problem is light itself. You don't get much of it captured with a shutter speed of
CCD kind of sucks, man. For all its glorious promise, the best CCD chipsets aren't all that much better than the wonderful X-10 spycam.
Hey freaks: now you're ju
Because certian events, despite what you might
think, *do* occur within 1000ths of a second
Yes, like when I get fragged playing quake.
Michael
There is no cryptographic solution to the problem where the intended receiver and the attacker are the same entity.
There's also a different sort of CCD highspeed camera that's used in various types of racing.
That system uses a single row of pixels which can be scanned at extremely high rates - the picture is built from objects moving in front of the pickup row, rather than the camera actually taking a full-resolution image.
Sort of a high-tech slit-camera.
Perhaps not 100% on-topic, but still interesting.
The other factor when talking about extreme high-speed photography (when people are calculating bandwidth):
Most really high-speed cameras shoot in black and white afaik.
If you drop the calculations from 32bpp down to 8bpp for a nic greyscale image, you're starting to get to manageable numbers... Also, adding cheap hardware based compression (RLE or the like), would be able to reduce the data stream to even more manageable levels.
You're not going to be able to shoot 6 megapixel pictures that fast, but 320x240 or 640x480 images should be possible at high framerates. I doubt it would replace film, but it might be handy for quick playback without having to get negs developed.
If you watch the "Bad Boys" DVD (the Will/Martin ver of Bad Boys), they have some very cool high-speed photography of different guns being fired into different objects. They used some sort of kodak high speed imager afaik - around 2000fps.
"Nothing strengthens authority so much as silence." - Charles de Gaulle
There have been many good replies to this thread, though most are talking theory, some experimental at best. A camera module with the ability to capture 12,000 high resolution frames per second is bound to cost a fortune, and I really doubt there will be much competition for a long time. Perhaps a cheaper alternative would be to purchase several currently-available high speed CCD/CMOS camera modules and use a series of mirrors and lenses to allow the cameras to work together in a round-robin fashion to achieve the a much higher framerate. This would certainly keep the project from being locked into proprietary hardware -- be it a single interface type, manufacturer, or other monopolistic attribute.
The idea of "parallel" items is nothing new, we've already seen success with drives, clusters, and even an array of projectors to create a high resolution projected wall.
Just a thought...
As far as the humman eye comment, well, just because you record at 12000 fps, doesn't mean you play it back at 12000 fps...
:-)
Well, don't tell that to the hard core gamers - I'm sure some of them would pay alot of money to get a video card that does even 1000 fps.
Actually, motion looks fairly fluid above 25 frames per second (although your monitor will need to refresh at least 3 times that speed to avoid flicker).
There is no cryptographic solution to the problem where the intended receiver and the attacker are the same entity.
anyone else remember eg&g's high speed nuke cameras?
Rapatronic Camera Shots
Treatment, not tyranny. End the drug war and free our American POWs.
See my user info for links.
Let's do some arithmetic:
The wavelength of visible light (in a vacuum) is between 4x10^(-7) and 7x10^(-7) m.
The speed of light is 3x10^8 m/s (in a vacuum). Planck's constant is 6.6x10^(-34) J s.
Put these together, and a single photon of visible light has an energy of between 2.8x10^(-19) and 5x10^(-19) J.
Suppose you want to get 24-bit colour. As an absolute minimum, you'll want to be able to detect 4096 photons per colour per pixel per frame. CCDs are typically 50% efficient, which means you need 256*3*2 incoming photons per pixel. At, say, 1024x1024 pixels and a million frames per second, that means 3*4096*2*1024*1024*1000000 = 2.6x10^16 photons per second, at an average energy of 3.9x10^(-19) J each.
That's an absolute minimum of 1.0x10^(-2) W of incoming radiation.
How much light is available? Well, at "bright sunlight" is approximately 30 W/m^2 of visible light.
That means that you'd need an aperture roughly 28mm across... which isn't impossible, but is certainly not going to be desireable.
So how does ultra-fast photography work? They use really bright flashes of light... which is why you don't want to be filmed for more than a fraction of a second at once.
Tarsnap: Online backups for the truly paranoid
The problems abound and are covered quite clearly in other posts.
How I would tackle the problem is to setup a series of CCD (Foveon's X3 would be my choice), with each pixel element feed directly into a huge RAM cache where the data could be loaded off into yet slower storage.
Since we had to deal with charge time (hence the first C in CCD) of 1ms, we'd need 1000 CCDs each with their own data cache and so on.
Then comes the problem of making an image - since we'd be dealing with 1000 CCDs, we're going to have to figure out how to place each pixel so than when that pixel's series fires, we can capture an image which would look like any other series' image. So this is what you'd be dealing with:
DATA RATE = 3 * (W*H) * 1000 * Time Duration
For giggles:
3*(1024*768) * 1000 * 1s = 2,414,592,000,000 (2,358,000,000 Bytes per second).[1]
Thank you, I'd like a stiff drink now, and film looks mighty good.
[1] If I didn't fuck up my math...
III.IIVIVIXIIVIVIIIVVIIIIXVIIIXIIIIIIIIVIIIIVVIII
This ain't true anymore.
The flagship prosumer digital SLR from Canon, the D60, uses a Canon-made CMOS which renders incredible pictures.
Take a look at some sample image on www.dpreview.com and see for yourself! Things have changed since 2 years...
On another notem, CMOS sensors tends to be slower. Thats probably why their pro SLR like the 1D does use a CCD.
I'd rather be sailing...
Why? I remember when I first got a digital video camera, the first thing I went out and did was set it up on a tripod, pointing to a full pop bottle, and then shoot it. I ran back to my computer, put it into Premiere, slowed it down, and was vastly dissapointed that I couldn't see every instant of the process. Mostly the bottle's there, then in the next frame, the bottle has a hole in it, then it's up in the air, and it falls. How much fun would it be to see the bullet make its way towards the bottle, see every moment of the impact, and the exit of the bullet in a fine spray of mist? I'd love to shoot in 1000 fps instead of 29.whatever.
Mod my comments down. It'll be fun.
Ever seen the inside of a car impact test facility? You've seen this on some commercials, even if you didn't realize what you were looking at. There's a huge bank of lights on during a test. Literally hundreds of lights, each of them on par with the lights used to light stadiums.
They have to be turned on in sequence, because if you tried to turn them all on at once, the current draw would kill the power grid. This despite the extra-hyper-ultra-industrial strength wiring into the grid.
You also can't leave them on for more then a few seconds because the heat generated burns them out. (It is actually a challenge to balance these two conflicting priorities, to turn them on quickly, yet slowly.)
The power draw for a single test is enough that they actively try to minimize the amount of time these lights are on. This despite the fact that electicity is normally so cheap we really don't think about it much. (Think in business terms; this means it's worth someone who is being paid $50+/hour to actively spend time worrying about how to minimize the time these lights are drawing current.)
All of this for the ultra-high-speed photography that takes place. I don't recall exactly how bright they said it was in the facility I was in, but I think it blows sunlight away by several times.
I mention this as an example application where "bright flashes of light" (emphasis mine) aren't practical, so they have to go whole hog. Kinda cool.
This is the same problem that film would have, though. You can't get sub-photon resolution of light, because it doesn't exist.
For high speed photography, you need lots of light. This is just generically true. But the quantum efficiency of CCDs is virtually 100%, as opposed to film which is much lower. So, this is a strong point in favor of a CCD system - you'd need less light. There's a poster above talking about how in car crash tests they need massive lighting systems to be able to see things. This'd cut down on their power bill quite a lot.
Why the hell would anyone want more than 640K in their computers?
Capability drives application.
Hell, FILM technologies can't offer rates of 12,000 fps: once you expose a piece of film, it takes thousands upon thousands of years before it settles back so that you can take another frame!
The movie industry, of course, solves this by having tons of pieces of film, and rotating between them. This is, of course, directly applicable to CCDs/digital camera solutions: have a LOT of sensors, and a prism to shift between them all.
Take, for instance, two of those SI sensors: they'd then be able to do 1400 fps. Take 20 of them, and you've got 14K fps.
Well, For one thing nobody records at that resolution. As another reply stated, DV is 720x480.
Another problem with your simple calculation is that video is never stored as 32-bit color. That's totally unrealistic. The common way to store video is not RGB, but YUV. Because of the way the human visual system works, the color components (U,V) are typically stored at 1/4 the resolution of the luminance (Y), meaning that an 3*X pixel RGB image would be stored as a X+X/4+X/4=1.5X image in YUV, half the number of pixels.
More significant, though, is that fact that just about every digital image recording mechanism stores information compressed onto the storage media. This is true from consumer digital cameras to DV cameras to the Sony HDTV cameras Lucas used for Star Wars.
Consider what it means to take 12,000 frames per second. You're probably recording a single nearly-instantaneous event, or getting many images of a very fast event. In the former case, there will be a series of frames before the event in which nothing is going on, and the difference between the frames is close to zero, which compresses extremeley well with MPEG-style compression. Your data rate could be 1/100th of the uncompressed rate. When the event occurs, the instantaneous data rate goes up, but buffering can solve this, since it probably lasts a few frames.
In the latter case, recording a fast event at a fast framerate, is essentially the same as recording a normal-speed event at normal frame rates. In this domain as well MPEG-style compression is extremeley effective. At the maximum you would need 1/5th or 1/10th the uncompressed rate, but 1/100th is a pretty reasonable number given current technology.
The only challenge with realtime compression at this speed, of course, is sufficiently fast hardware. I think it could be done in parallel -- capture several GOPs worth of data (15-45 frames perhaps) and send it to a compressor, and then switch the buffer output to a new compressor, round-robin style.
In any case, video is usually stored at rates many factors smaller than the uncompressed rate. So if you change the variables of your equation to a more realistic resolution and color depth, then divide that number by 10 or 100, you'll have a more realistic data rate.
720(w)*480(h)*1.5(color)*12000(fps)= 6.2GB/s, divide by 100 for agressive compression but reasonable results = 600MB/s
Still too fast, but not completely unrealistic if you've got a healthy budget. ;-)
Ok ok ok ok I feel stupid now.
I divided 2^20 by 24 not by 3, heh.
make that 349525 frames with 1 megabyte of storage per pixel.
Quite frankly with that much storage space I would say store 64bits for the visible spectrum and use another 24 for the infrared spectrum, and if CCDs ever get advanced enough, another 16bits or so for the ultraviolet.
With the visible + infrared though, that is 11 bytes, or a mere 95325 frames per individual pixel, hehe.
Need help treating your acne? Come here!
Yep I built a electonic video camera that had megarhertz frame rates 8 years ago. I patented it too. Actually two different designs.
C.E.M. Strauss, "Synthetic Array Heterodyne Detection: A single Element Detector Acts As An Array", Optics Letters, Vol 19, No. 20, 1609(1994)
and
B.J. Cooke, A.E. Galbraith, B.E. Laubscher, C.E.M. Strauss, N.L. Olivias, Grubler, A.W. laser field Imaging through fourier Transform Hetrodyne, proc of SPIE, 3707, 390-408, (1999)
the problem with pixelated detectors is reading out the darn pixels fast enough. Normally this is done by some sort of bucket brigade across the ccd or some sort of serial memory access across a cmos array. very slow. And parallel access to an entire ray is absurdly complicated and expensive
In my approach I solved this problem by multiplexing all of the pixel signals onto the same single wire. Each pixel when activated creates an osciliory signal at a unique frequency. All of these are combined on a single wire out put (amplified by a single fast amplifier) and then the AC signal is digitized by a single fast digitizer and streamed to a hard disk. The frame rate is determined by the frequency separation between the pixels, so if the oscillation frequency is a megahertz then a frame can be resolved every microsecond. This process is continuos and can go on for as long as you have disk space.
the other cool feature is that the chip you do this on is a single pixel chip! not a pixelated array. the pixels come from painting the chip with a rainbow of light. for a 1-D example, imagine red light on the left edge and blue light on the right. when a reference signal comes in it beats with the light. the beat frequency that gets ouput is determined by where (left to right) the incoming beam hit.
of course the good news and the bad news is that this is intended for active remote sensing where one is illumunating a target with a single frequency laser. It does not work with ambient light (note the second articele referenced above will work with polychormatic light) . The good news is that the detection method is hetrodyne detection which has shot noise limited detection sensitivity even on a crappy photo detector. thus the system is capable of detecting a single photon of light.
Another cool feature is that one can do doppler detection with this too since any frequency shift in the target's reflected shifts the pixel frequency. This could be used for example the image bllod flowing in veins, find moving objects in noisy scenes (e.g. submarines, air planes) or any number of flow imaging concepts. The heterodyne detection means its sensitive enough to do at very long distances (say space), or to use it for imaging through very dense media (for example, imaging through the side of a vein or through breast or brain tissue.
A description of how it works in stilted patent language can be read on line here
From an artistic point of view, the problem isn't which medium to develop...it's how to improve both technologies such that cost/energy/latency is not too different. I should have the freedom to choose the technology which best serves the intent of the piece free from those constraints. It could be film, it could be video. It really depends on how I want it to turn out in the end.
So more substance, less rant: here's how I think these technologies would be useful to end users, and thus what we should be thinking about here.
Video Tap: A major video breakthrough in the feature film making process was Jerry Lewis's video tap. This puts a prism or split field diopter in between the lens and the film plane, splitting it in two, one going to the film plane, the other going to a video camera. This is how a director is able to get immediate feedback on how the scene went (instead of waiting for the dailies the next night to see it). A high framerate video tap for high framerate film would be extremely handy. The quality wouldn't have to be great, it would just need fidelity to tell the director and cinematographer how well composed the take was, and making sure all the stuff thats supposed to be in the take are there...and nothing else (like a boom mic).
Internet/NLE: This also would help in modern, internetworked digital non-linear processes. This is where takes are digitized as they are shot (if not already doing initial capture in DV) and dropped into the timeline in a nonlinear edit suite (avid, cinerella, final cut pro) whos project files are shared in an internetworked data store (film crews on other ends of the world, and the CG shop instantly are able to see their shot in the context of the other units shots...in realtime) via a 3 point edit. Even with a film process, the tap could digitize the footage and insert it into the timeline...the print of the footage could be later scanned and conformed to the timeline. Very handy. So this ties into the throughput problem. You have to consider that the bottleneck isn't CCD voltage intervals, cache tomfoolery or writing to a non-volatile medium. It could be a crappy ADSL connection or satellite uplink set up by people who scarcely understand how that stuff works.
Noise and heat: One of the banes of film making and one of the big advantages of digital video is the noise that all those ratchet/crank/shutter type mechanisms in a camera create. A lot of the sound work in a film is dealing with the noise from the camera. Sometimes, the sound is recorded later after discarding the sound from the set wholesale. Now, in order for a cmos imager to be effective at these speeds, we'll need to keep it cool. Heat is more likely to degrade throughput than buffer speed or size. Hence, we're going to need to build hardware to cool the cmos. That hardware is likely going to be more exotic than the cmos, take more energy than the motor for a high speed film device and potentially create a lot of noise on it's own. So the advantages of the high speed DV cam over film are only possible if the apparatus that supports the camera don't reintroduce the same problems on an equal or greater scale than existed in film.
Personally, I feel that the single greatest and most useful application of this technology, from a creative standpoint is the high speed video tap. It would liberate crews from the burden of dailies and integrate high speed footage into modern production processes.
For non-creative uses (scientific/research), this technology can free users from the latent and toxic nature of film processing infrastructure.
"Let him go, Ralph. He knows what he's doing." --Otto Mann (simpsons)
For 35 Gbytes/sec (280 Gbit), you'd need 280 gigE interconnects, and that's assuming you can perfectly divide up the data amongst them in realtime with no performance hit.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
Note that he said 35 Gbytes/s. THat's 280 Gbps. Even 10GigE would be woefully inadequate. Hell even 100GigE would just barely cut it. You'd need terabit ethernet to do it properly.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
All image capture is analog. It can be electronic (CCDs) or chemical (film), but there's always an element that "charges up" as it's hit by photons.
Compared to film, CCDs are extremely low-res (top quality 35 mm film has resolution equivalent to a 50 megapixel CCD) but, more importantly, they're slow. At very short exposure times, CCDs have so much noise that the final result is useless. The problem isn't the transfer rate, it's the time the CCDs take to "charge up" to meaningful values.
There is one alternative: use very large CCDs. The larger the CCD, the more light hits it, and the faster it can charge. But larger CCDs are more expensive and require special lenses.
Recording directly to digital does have one big advantage: you don't have to pay for the film. But the CCDs simply aren't up to film quality yet (and probably won't be for another 5 years or so). So the solution is simple: shoot on film, then digitise it.
RMN
~~~
There is an Astronomy project called Ultracam that uses special CCDs to capture astronomical events at highspeed.
Interestingly the computer interface that they use for the special CCDs uses Linux.
I am sure that you can get an idea of what is involved from Ultracam and use it in other real world applications (patents not withstanding).
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
I can think of one good reason why this isnt a good idea. As still digital cameras push the limits of how many megapixles they can fit into an image some profesionals are noticing an interesting problem. at a certain point, adding more resolution to a camera actualy decreases the image quality. THat's because when you decrease the size of the sensor which records the data for a pixel in a camera you decrease the amount of light that will hit that sensor.
This means that the signal to noise ratio for each sensor goes up. At 3 megapixels you wont see any degredation from this but as the resolution increases youre going to see more and more loss of acuracy and less acurate images. now this is for a camera which takes photo's at the equivalent of 50 ISO (1/50th of a second) if you want a camera that takes images at 12000 images per second that means each image has to be captured in about 1/36000th of a second, or 720 times faster, or 1/720 of the light will hit each pixel. inorder to maintain acuracy at that speed youre going to have to drop down the resolution to 4000 pixels per image (100x40?) inorder to maintain image quality.
--aiee
%s/Slashdot/google/g
Search for high framerate digital cameras. Let the poor slashdotters get back to writing code.
(+1 Funny) only if I laugh out loud.
I haven't been keeping up with Myrinet... until they license their design to other manufacturers and/or drop their prices significantly, I'm not interested. If I wanted to be locked into a product built by only one company, I would have bought a Cray or SGI in the first place.
You were probably correct a few years ago, but you need to get with the times.
There is no film in the world that can outshoot a high-sensitivity CCD nowadays. Cameras like the Kodak 760x can shoot at ISO 6400 with reasonable quality, which film is utterly incapable of matching with any sort of quality, and CCDs are only getting better.
Yes, crappy consumer digicams suck at anything over ISO 100. But a serious professional digicam beats the pants off of film at high ISOs. (In case you were wondering, my wife is a professional photographer who shoots with a Nikon D1X. I do know a bit about this.)
ZFS: because love is never having to say fsck
Mechanically inclined.
When was the last time you even heard that phrase? We live in a physical world. A mechanical device is a perfectly acceptable solution to a problem. Not everything needs to be done with software. Just look at the guy's level of disappointment. "But there has to be a way to do this with electronics! Electronics are always better than mechanics, aren't they? It's impossible for mechanics to do something electronics cannot, ins't it? Hello?"
And Cliff's additional writeup is no help either. The reason the video in the example he found can only played back at 103fps is fully explained in the link he provides (and apparently didn't bother to read). Also, the 12,000fps film camera that got everyone talking in the first place not the first of its kind. High-speed film cameras have been around for decades. The real kicker is Cliff's silly statement at the end, which makes it sound as though an electronic high-speed camera would be the first high-speed camera ever. He says, "What visual mischief could you aspiring photographers get into with such a camera?" Gee, I dunno Cliff, how about the exact same things people have been doing with high speed film cameras for the past 50 years, eh?
Sheesh. The world goes beyond the bits in a CPU. Turn off the computer and take a look around at the tangible, physical world.
Free Hans!
Over at Micron Technology in the imaging department we are working on the next generation of digital imaging sensors. All the processes are CMOS based instead of CCD.
Of particular interest to high frame rates might be the MI-MV13, Micron Imaging - Machine Vision 1.3Mega Pixel CMOS digital image sensor. This particular sensor can do 500fps at 1.3Mega pixel but can also be windowed to do for example 4000 fps at 1280 x 128.
Bandwidth.
Pixel Depth.
Image Dimensions.
Bitch, bitch, bitch, bitch, bitch. To quote someone who dearly needs to be heeded in this case (Dennis Leary), "Shut the fuck up, NEXT!" I've heard enough crap, why don't we just call up Nikon and ask them for one of their explosive imaging cameras? If I remember my Guinness Book of World Records, that unit is a digital camera performing in the MILLIONS of frames per second! 12,000? Feh!
Gee, how about a simple Google search, even? Let's try that, shall we (since the Guinness world record site SUCKS!):
- "Fastest Camera" search
- "Ultrahigh-speed Imaging" search
.PDF in that first hit -- it's from "The Industrial Physicist", and has some nice info on a "gated still-video camera." A quote:
Another point I quickly found is that high speed (million-plus FPS) imagery has been around since the late 80s. Most of it's digital. (Imagine that.) You can thank the US military for funding that.First 3 links are about the same camera! A half-million dollars, 200 million frames per second.
Grab the
- "Multisensor, ultrahigh-speed electronic
imaging systems (such as that shown in Figure
1) are capable of recording sequences of
discrete images at frame rates of up to 100
million pictures per second. They incorporate
compact, intensified charge coupled
device (CCD) modules that exhibit virtually
no geometric distortion or intensity variation
and provide the user with digital images that can be analyzed using a personal computer."
Oh, one other thing: The article is from December 1997 when Pentium IIs were hot stuff, and you counted yourself lucky to have 64 MB of RAM and a 9.1 GB F/W SCSI-2 hard drive!"...America's great minds of today, teaching America's great minds of tomorrow. Poor bastards." -- A Beautiful Min
The benefit of converting to this Y'CbCr colorspace is that you can get cheap, easy compression by simply subsampling the 2 color-difference channels, Cb and Cr, and humans will probably not notice.
Now lets assume that your "healthy budget" means "professional" and not "consumer"... That 1/4 chroma resolution implies a 4:1:1 or 4:2:0 sampling format which is fine for consumer level products like DV and DVDs, but any professional-level hardware will use at least 4:2:2, bringing you up to 16bpp.
Let's try that calculation again:
720(w)*480(h)*16(color)*12000(fps)= 66.4 gigabits per second.
Now for those of you who want to compress this monsterous stream with a beowulf cluster, I'd like you to show me one that can suck a 66.4 Gbps data stream out of a camera. :) right.
That leaves us with the only option of compressing in the camera hardware as the previous poster suggested with an array of encoders, each working on 1 GOP. Assuming we use realtime hardware encoders, we'll need about 401 of them: 12000(camera fps)/(30000/1001)(NTSC fps) = 400.4. I'd recommend setting the 401 encoders to produce 5Mbps MPEG-2 streams to achieve a decent quality. That gives us about 2 Gbps of MPEG-2 output.
BTW, to hold a 30-frame GOP in memory for each encoder while they encode, we'll need almost 8GB of RAM: 720(w)*480(h)*2(color)*8(GOP)*401(encoders) = 7.74GB (Let's try 8-frame GOPs for 2GB of RAM)
To store that 2Gbps video stream, we'll need a single SCSI Ultra320 hard drive. (not bad)
Now go build it! :)