Camera that Sees through Smoke and Fog Underway

tomschuring writes "The Age has a story about IATIA, who have been given $2.7 million by the Defence Department to fund development of a military spy camera capable of seeing through fog, smoke and dust storms. The technology uses a highly sophisticated camera that captures three images simultaneously through a single lens. Images thus resolved from between the particles making up fog, smoke, and dust storms are formed into a single picture of the hidden target."

Comments

Warning: Registraton Required

[RKBA]

BugMeNot username and password:
Username: registrationsucks1 Password: asdoestheage

Re:Warning: Registraton Required

[RLiegh]

Why waste all the extra effort when you can just use a login already made for you?
Mostly, I was thinking because inevitably whenever someone posts a username/password to a site like that on /., someone decides to be an asshole and change the password.

If that happens, they can fire up a quick throw away account using mailinator.

density

[Coneasfast]

how dense can the fog particles be? this camera would have to have an extremely large resolution to do this kind of thing. anyone have any specs on this?

the uses for this are endless, eg, if the technology becomes cheap enough, we can have this in cars to help driving during foggy weather.

Re:density

[ajna]

This system does not rely on resolution. You might be imagining it as taking two (or more) pictures shifted horizontally, perhaps, and somehow subtracting the intervening particle's optical effects, leaving only the subject matter. This is not how the system works, however: instead, as the summary briefly but correctly stated it relies on three images being taken, one focused in the plane being studied and the other two focused before and after that plane. Quantitative Phase Microscopy is the process of extracting additional data about the subject in the plane from the data in all three images. Why it doesn't rely on the resolution of the sensor is because the addition information is derived from the optical properties of the light passing through/reflected off the surface, not from sensor trickery.

I guess this could be used on cars given enough processor speed, but it's really not applicable in this case, as it yields additional information about something in a plane (parallel to the sensor of the imaging device -- imagine a brick wall ahead of you when driving). When driving, the plane, say, 50m ahead of the car is moving just as fast as you are, and seeing ultra-crisp images of that plane for the instant that it is 50m ahead would be of dubious utility imo.

Re:density

[dew-genen-ny]

Regarding cars -

surely we could just sweep a range of values....

or are you of the mind that a TV is impossible because we can only draw one line of dots?

just a case of enough processing power, surely?

Re:density

[ajna]

There is the possibility, yes. TV generally features items in a single plane of focus (enhanced by depth of field) and the raster sweep is in this plane. The problem in applying this technology to cars, for example, lies in that the "sweep" is of the plane itself. Of course it's possible in theory, but it's a different problem than what has already been worked out so throughly in television.

Other versions available...

[physicsphairy]

"that captures three images simultaneously through a single lens." There is also a Kodak version, where one set of pictures is lost, another is misdeveloped, and the third is inadvertently sent to your ex with the same middle initial.

Re:also

[brocheck]

I wonder if it would let you see through the particles that many dresses consist of.

I'd buy one.

Unused links on how it works - some detail

[tqft]

Some detailed links on how it works

http://www.iatia.com.au/technology/insideQpi.asp

http://www.iatia.com.au/technology/applicationNo te s.asp

he algorithm has a number of key advantages, including:

* Returns phase and intensity information independently
* Provides quantitative, absolute phase (with DC offset)
* Is a rapid, stable, non-iterative solution
* Works with non-uniform and partically coherent illumination
* Offers relaxed beam conditioning
* Solves the twin image problem of holography
* Has been experimentally applied to a number of radiations

You can find their list of patents on theire site. Digging into these should give you more detail.

I don't care I am going on holidays for 3 weeks in 3hours

Can it see through smoke and mirrors?

[telly333]

Just in time for politics.slashdot.org!


telly

Re:Can it see through smoke and mirrors?

[arose]

Yes, in fact it can. Here's the resulting image: politics without smoke and mirrors.

3D?

[Nathdot]

The technology uses a highly sophisticated camera that captures three images simultaneously through a single lens

Unfortunately the image cannot be viewed without Red+Blue 3D glasses.

Dense Camera Arrays for seeing through bushes

[vectra14]

These guys at stanford have done some really amazing stuff that's directly related. Except that they has literally dozens of cameras (as seen in their ppt), and their research seems to concentrate on multifocal image reconstruction (see ppt slides, presentation is quite good)

Link (has cool results links)

Re:Dense Camera Arrays for seeing through bushes

[ajna]

The Stanford work is actually entirely different. They utilize parallax -- in other words, their cameras are in physically distinct locations and see the scene with different perspectives. The IATIA work utilizes a single point of view, with images captured with the focal plane at the desired location and then slightly fore and aft. Read more here, at a Columbia site.

Quantitative phase microscopy is a relatively new technique that can generate phase images and phase-amplitude images. In practice, to obtain a quantitative phase image one collects an in-focus image and very slightly positively and negatively defocused images, and uses these data to estimate the differential with respect to the defocus of the image. These images (a through-focal series) can be easily obtained in our system with our z-motion nano-positioner. The resulting data can be solved to yield the phase distribution by Fourier-transform methods. Results are obtained by essentially solving an optical transport equation. Significantly, the phase that is obtained does not have to be unwrapped, as is required for interferometry.

I'd be lying if I told you I completely understand the quoted paragraph, specifically what "essentially solving an optical transport equation" refers to, but I'm sure some cursory googling will lead the curious to specifics, certainly more than googling on terms in the article summary would yield.

Re:Dense Camera Arrays for seeing through bushes

[TiggertheMad]

What they are saying is this: They take three pictures. On a camera there is a point somewhere in front of the lens that is the 'focus point'. The distance it is away from the camera will vary by the lenses and their distance apart, but it is basically a fixed distance for any given setting. The first picture that point is set too far behind the subject, the second right on the subject (In focus) and the third in front of the subject. Because you know how the lenses were made, you can do some math and figure out how far away each element in the picture is by how the focus changes between the shots, and get a (quasi) 3d model of everything in the picture. The concept is simple enough, although having a proc that can do that in real time could be a challenge.

The real challenge is this: You are building a 3d model by interpolating data from a scene, but you are only doing it in one dimension. I bet a 3d picture would look like a scene from Doom1. You can create flat sprites and position them, but you can't capture any depth information without paralax interpolation either via lateral movement and reshooting or additional cameras.

Re:Dense Camera Arrays for seeing through bushes

[ajna]

You've basically said what I wrote elsewhere in this article's comments. We both agree that it would have limited utility for real life applications (as in not in fields where confocal microscopy gets people hot and bothered) in as many words.

Incidentally the IATIA link itself held the answer to my above musings, about what the transport equation actually is. I still don't understand it, but it can be viewed by one and all at the bottom of this page: http://www.iatia.com.au/technology/insideQpi.asp

Keith Nugent

[metlin]

Hmm, Keith Nugent is fairly well known in some niche areas of optics. If I remember right, his initial work on the use of x-rays and the like to compensate for normal visible hindrances were met with some opposition, but he is quite famous otherwise.

That was because, ironically, this was developed as a method to visualize biological stuff, and some felt that his methods would not quite be suitable for such a task. His ideas were to use various parameters such as phase, intensity and angle of vision to extract information which could be correlated and converge to recreate images with minimal amount of information, which later gained acceptance.

I guess he developed on that technique, and later on evolved to have the military to take notice. Interesting neverthless.

Article is short on details

[shoolz]

Much is left to the imagination in the article

I am imagining that since it not possible to "see" "through" an object, that these three images must be of various wavelengths (visible light, ultraviolet light, and infrared) and then are run through an interpolation process to get a probable image of what is behind the obstacle.

Am I out to lunch? Can anybody shed more light on how this works?

Re:also

[Darthmalt]

IIRC Sony accidentaly did that. If you engaged the night vision you could see through clothing. However, I think they recalled all the cameras that were capable of this.

Okay, pervs, here's what you want

[Stormwatch]

Here's what you want, a camera that sees through clothes . Sheesh...

Blatantly stealing my parent post's material...

[Anonymous Coward]

...to make it clickable.

http://www.iatia.com.au/technology/insideQpi.asp


http://www.iatia.com.au/technology/applicationNote s.asp

Re:We don't need this

[Darthmalt]

Probably because it's easier for him to get moner for his research from the military. Many things we use as consumers everyday were started by the military.

GPS, Radar, heck even the microwave (though that was more the British military.

I fought the law...

[Rendition]

Hope they can't make this work for speed cameras...

Re:Hi-res TV stills

[swb]

Doesn't changing angles on the subject (a result of moving the camera), cause you to collect not more but different data on the subject, resulting in a higher resolution image that's higher in angular/dimensional data, but not in 2D data?

It seems like you'd end up with a David Hockney-like image, not a higher resolution image.

vaporware

[Doc Ruby]

Wake me when journalists have a camera that can see through the fog of war, where the first casualty is the truth.

Re:vaporware

[powerlinekid]

Oh jesus christ...

Re:interesting but

[6169]

Actually it doesn't seem like they're using parallax, though that's what I first thought as well. I think it actually has to do with the fact that all transparent or semi-transparent substances change the phase of light passing through them.

As far as I can tell, the three images are taken slightly out of focus from each other. One is in focus, and the other two are positively and negatively defocused.

You then use fourier analysis to take the difference in phase of the images viewed from the three lenses and produce a "cleaned up" image where as much of the stuff that is shifting the light frequency is removed.

Re:also

[suckmysav]

"IIRC Sony accidentaly did that. If you engaged the night vision you could see through clothing.

You also needed an IR pass filter to do that, but otherwise you are correct.

"However, I think they recalled all the cameras that were capable of this.

I don't think they recalled them, they just stopped making them like that.

Re:also

[AndroidCat]

If they make one that can see through fingers and lenscaps, I am so there!

Re:also

[Rufus211]

ABCNEWS.com : Cameras Let Voyeurs See Through Clothes

Fog of war...

[Anonymous Coward]

... now I can stop feeling guilty for turning off fog of war in games!

Re:We don't need this

[rebelcool]

Who are you to say "we dont need this"? You can forsee all applications of a technology before its made? And you automatically assume just because the money is initially military its going to be used to "kill people"? What nonsense.

This would useful for finding people in a burning building full of smoke. Or imagine putting it onto a car as a warning system in heavy fog that you're approaching an obstacle too fast. Same with planes. Surely more creative people than I can dream up a dozen applications for this.

Here's a tip about research: The military has a ton of money, and they spend it on all kinds of things that have nothing to do with "killing people". As pointed out already, the internet was a defense project. So was GPS. So was radar. So was a million other extremely useful things.

"We dont need this" - we don't need you and your cluelessness.

Re:O-kaaayy...

[FooAtWFU]

Around these parts, it works the other way around... things are legal until they're illegal. Who shouldn't be allowed to use it?

Already exists

[leabre]

I was in the US Navy from 1994-1996 and the damage control teams already have a special camera (forget what it is called) that can see through dense smoke (the type you would expect from a jet fuel fire or amunition fire on a ship) and help you to see clearly through the smoke.

Wonder what makes the camera in this article so different from the technology the Navy already uses... I'm sure the current navy breed is much more advanced than it was 10 years ago.

Thanks,
Leabre

Re:Already exists

[Anonymous Coward]

Walked through one of the carriers docked in Newport around that same time and they let us use a device (looked like old style night vision binocs) that sounds like what you are talking about. It worked off heat, and wasn't color though. It was cool to see the heat from a lighter swirling around for 2 min after the flame was extinguished.

Re:Already exists

[GooberToo]

I'm guessing that it's an IR type camera, which is more commonly being used by fire fighters these days. They use these to quickly search burning houses (etc). Without this gear, they must search the same way rescue workers have for the last several hundred years (blind belly crawl), if not longer. Even these cameras have their limitations, based on the ambient temp and the number/size of particulants in the air. These generally work well for rescue because the air on the floor is cool and a warm body still makes for good contrast.

I can't remember the brand of this system, but IIRC, they are super expensive. Expect to have to pay something like $40-60k per camera, per fire fighter. This is why communities often do fund raisers to get these for their local firehouses. Bluntly, most firehouses can't afford one, let alone the multiples, which are considered ideal.

At any rate, after all that, there is a huge difference between IR and what this article is talking about.

Re:Already exists

[Phoebus0]

Actually, they are called "Thermal Imaging Cameras", and they are cheaper than that. Multiple companies make them, ours are made by MSA. My department just purchased one for around $12k, so they are cheap enough to carry one per apparatus. You can get them even cheaper if you want fewer features. Almost every volunteer department I am aware of has at least one.

Generally they aren't restricted by particles in the air until the particles get large enough to block the infrared. In some situations the we can actually find the victim based upon their body temperature being cooler than ambient, rather than warmer than ambient. Also a good trick with the modern TICs is that you can put your hand on a window for a few seconds, pull it off, and you scan still see the shape of the hand in the warmth on the window.

Re:Already exists

[Stormshadow]

You're thinking about the Navy Firefighter's Thermal Imager AKA NFTI. It's an infrared camera, nothing really special. Just got my Surface pin today, so I could tell you all sorts of useless knowledge about it... runs off 12 AA batteries, can't see through glass, lasts about 90 minutes on a full charge.
-ET3(SW)

Parallax

[Anonymous Coward]

"The technology uses a highly sophisticated camera that captures three images simultaneously through a single lens. Images thus resolved from between the particles making up fog, smoke, and dust storms are formed into a single picture of the hidden target."

If it uses the concept of parallax, how can it possibly do this both using the same lense AND at the same time? Isn't parallax based on the concept of different images of overlapping fields of view? IR: two or more eyes/lenses or two or more images slightly timed apart if the object(s) in the foreground are moving?

If it's based on image analysis using different algorythms for three copies of the same original image, wouldn't it be liable to have errors? (Think of those optical illusions of inverted masks...) Or is the third one used to reduce/remove these errors?

Re:Parallax

[Anonymous Coward]

It doesn't use the concept of parallax, and it doesn't take multiple copies of the same original image. Rather, it takes three images focused at different planes, and uses the phase differences in light from traveling different distances to reconstruct the original.

my thoughts

[Large Bogon Collider]

I'm not a 100% sure, but the technique involves phase shift. As light of a single frequency passes though an medium, its phase is altered and light propagation is delayed. If you can computationally filter out all out of phase shift information caused by fog, for example, you can "see" what the hidden object looked like. This process is quite CPU intensive. It seemed that about a grayscale SVGA sized image (0.41 mp) took 1.5 secs on a PIV 2.4GHz to calculate. This should improve with algorith tweaking and using FPGAs.

This may also have medical applications in terms of optical imaging - see through the patient (arms and legs only, probably). Shine a bright light at the patient. Capture the ealiest photos that emerge (the ones that had a direct path to the camera). Ignore slow photons (ones that were absorbed and release or bounced around). Voila, instant imaging without x-rays. IIRC, this was in development years ago.

Better solution

[huge colin]

Just use an uncooled microbolometer-based infrared thermal imager. BAE Systems has been producing these for years. They're low-power, lightweight, and efficient.

When receiving this wavelength of IR, you can see through smoke, fog, some plastics (regardless of opacity to visible light), and independent of visible light levels. And seeing radiated heat is, of course, an obvious benefit. A fraction of a degree F is all that's needed to note a difference -- you can even see where things used to be because of the heat shadow they leave.

--Colin

Re:Better solution

[KD5YPT]

The problem is that those camera only works will in areas with low temperature (night sea, night sky, moderately cool weather). In desert where sand storm kicks up or in picking out idle targets (stationary artillery for example that was not moving for a long time), infrared sensors are pretty useless.

Re:We don't need this

[whorfin]

Yeah, nothing that was ever funded by military research has ever come to any good for society.

Well, except for computers and the internet. Everything else was crap. And I guess those satellites that let us talk all over the world and get sports and softcore porn beamed into our house are pretty neat too, except for the lite beer ads. And did I mention the GPS I've got on my cell phone?

Yeah, military research is a total dead end.

Re:I for one...

[hunterx11]

Tin foil, of course.

Re:Nope

[Anonymous Coward]

Except that has nothing to do with how the these cameras work, which is not by relying on individual particles moving out of the way to give a clear view of different parts of the image, but rather by reconstructing the image from phase contrast.

Re:also

[wwelch]

Hopefully fire departments will be able to afford this technology so that fire fighters will be able to see people through the smoke of the fire...

Re:Let's redesign the wheel!

[KD5YPT]

Actually is different, if you read the two article, you'll notice a major differences.

The one you stated is an infrared camera. Which means its only good at seeing objects that give off an appreciative amount of infrared radiation (in this case, runway lights, other planes, and etc).

The one stated in the main post is completely optical. It merely take three consecutive image and a computer compare the images and extract objects that are obscured by fog, dust, and such. Of course, this system would require that SOME visibility exist (in another word, no seeing through solid walls).

Alternate uses...

[d474]

Here are some good targets to test this camera out on, after all, these have some thick fog around them. Maybe we can get a clearer picture of:

1. CBS - the "true" source of the forged documents
2. 
   Dick Cheney's secret Energy group (who are the members)
   CIA - Tenet's "slam dunk" intelligence source on Iraq's WMD (who fabricated that intelligence - afterall, it wasn't real)
   White House - who outed the CIA agent
   FBI & John Ashcroft - why is Sibel Edmond's testimony being "re-classified" after 2 years of being in public domain
   Halliburton - wait, maybe we shouldn't. We don't want to break the camera...

Finally!

[lewp]

Every day for the last six months.

Where's my spy camera?
Where's my spy camera?
Where's my spy camera?

Here's your stupid spy camera!

Re:also

[Gordonjcp]

Actually, it was because it *didn't* have an IR filter - that was how the "NightShot" stuff worked. Images had an odd greyish tinge with weirdly glowing eyes. If you can stand it, look at the dark bits in the video for All Saints - Pure Shores for an example.

Nearly all CCD cameras are sensitive to infrared. You can test IR emitters by pointing a camcorder at them and watching for the flashes. I made a very effective IR surveillance camera by popping the front off the lens of a Philips Vesta Pro webcam (get the blade of a table knife into the little groove a couple of mm back from the front and twist) and removing the IR filter.

I don't know about this...

[NathanM412]

Sounds like vaporware to me!

OASys

[philipsblows]

In college my clinic team worked with Northrop Electronic Systems on their OASys project, or Obstacle Avoidance System. It was a laser + computer navigation system that would scan the horizon through smoke or other aerosols and generate a "safe passage" navigation image to the helicopter pilot using it. Supposedly it worked pretty well (they were still working on it after our 9 months on our piece of the project). It was basically a rotating laser optics assembly that would trace a cone in space, and the assembly would scan in the horizontal plane to yield the losenge shape (they used that term).

Here's a funny little twist. When we went to the site to visit the developers of the project at Northrop, we stopped off in a meeting room that had on one of the walls a poster for the OASys project, featuring a helicopter with a losenge-shaped window of visibility depicted against some trees with some smoke and other debris in the air.

Nearby on the same wall was another poster for a weapon system, the name of which escapes me. It was the same poster, but in the middle of the losenge-shaped window of visibility was a little gunsight, and I think the helecopter had some weapons slung.

We asked our liason person whether the two projects were related, and he assured us they were completely different as we were brought to another area.

Our professor on the project was a Yugoslavian National, and this was in 1992, so you can imagine how fun the rest of our visit was when they found that out....

Re:We don't need this

[d474]

You brought up some good points. I'm just playing the Devil's advocate here...just for sake of discussion:

This would useful for finding people in a burning building full of smoke... and once the targets have been acquired, neutralize them.

Or imagine putting it onto a car as a warning system in heavy fog that you're approaching an obstacle too fast... or taking advantage of a dust storm and locating the enemy before he can locate you.

Same with planes... same reason, faster visual target acquisition is an advantage.

the internet was a defense project... that could allow us to maintain communication after a nuclear strike which is necessary if orders for a counter-strike are no be disemminated

So was GPS... to guide precision munitions to targets to increase kill ratios

So was radar... to detect any and all potential aerial and sea going enemy targets

"We dont need this" - we don't need you and your cluelessness... nor your innocence.

Just wanted let you know that there is always a way technology can be used by the military that is related to killing people. Especially if the military is involved in it's development.

Two uses immediately come to mind:

[io333]

1. Car in fog. It would be nice to have a heads up display on my winshield, kind of like Cadillac did with night vision some years ago... Whatever happened to that anyway?

2. Airplanes! No more grounding because of fog.

Re:Two uses immediately come to mind:

[GooberToo]

Airplanes! No more grounding because of fog.

As long as there is ground visibility, taking off is the easy part. It's landing that'll kill ya if you're not careful. In other words, as long as you have some visibliity to taxi and roll down a runway, you can easily get into the air. The problem is, getting safely back on the ground. ;)

Re:also

[WhiteDeath]

I found this site about 6 years ago...

they sell the filters, and give a good run-down on the theory.

Re:also

[itwerx]

I wonder if it would let you see through the particles that many dresses consist of.

It would. The technology has actually been around for a long time in spy satellites.
It's devilishly simple. Take pictures/video along a number of wavelengths (e.g. IR through X-ray) along with the fact that they each reflect/refract at different angles of incidence and add some majorly intensive computation and you can "subtract" virtually any sort of dynamic occlusion, including the shifting fabric of a dress. If a woman were walking it would only take a few steps to get a remarkably clear image of what she looked like underneath.
Of course spy satellites (or, rather, some huge rendering farms down below) use it to remove distortion caused by clouds and shifting layers of air but it's all the same process, really.
People doubt that spy satellites can read the time off your watch but if you think about it there's not much you can't see if you've got good enough optics and distortions are no longer an issue.
Now getting all that computation into a camera would be very cool! Although, unless quantum computing makes a giant leap it'll be an analog computer rather than digital...

Pfff, had that already in...

[thrill12]

..[name your favourite war-game here] - it's called:

Turn fog of war off

Camera not required.

[raehl]

All you need to see through smoke or fog is a few extra OpenGL commands. It's not limited to just fog/dust either, you can even snipe through walls!