Sharpest Images With "Lucky" Telescope

igny writes "Astronomers from the University of Cambridge and Caltech have developed a new camera that gives much more detailed pictures of stars and nebulae than even the Hubble Space Telescope, and does it from the ground. A new technique called 'Lucky imaging' has been used to diminish atmospheric noise in the visible range, creating the most detailed pictures of the sky in history."

Comments

Yawn

[Anonymous Coward]

So what is so special? Many telescopes can resolve better than the Hubble in the visible range.

I'd only be impressed if they somehow made a ground telescope that could resolve in the infrared better than the Hubble.

Re:Yawn

[Anonymous Coward]

The article says the images formed by this process are not just better than Hubble in the visible range, it claims that they are the most detailed ever. Seems pretty impressive to me.

Re:Yawn

[QuickFox]

According to the second article on that page, it's the other way around:

Images from ground-based telescopes are invariably blurred out by the atmosphere. Astronomers have tried to develop techniques to correct the blurring called adaptive optics but so far they only work successfully in the infrared where the smearing is greatly reduced.

Re:Yawn

[hackus]

Not likely to happen anytime soon.

As everyone knows, and for those that do not, infra red wavelengths are absorbed by water vapor.

Keeps us toasty at night, but sadly blocks the infra red for observations at the same time.

-Hack

Re:Yawn

[Cecil]

Actually, near infrared is not blocked by water vapor, in fact water vapor is extremely transparent to near infrared light even moreso than visible light. That's why satellites can use infrared to see through clouds, and also why adaptive optics work so well in the near infrared range.

Far infrared is a different story, and you're absolutely correct there.

Re:Yawn

[0123456789]

Adaptive optics works so well in the IR due to the wavelength dependence of the Fried parameter, r0, and hence Kolmogorov turbulence. There's less turbulence in the IR, hence it's easier to correct it.

See here, for example, for more information.

There are wavelength ranges in the NIR where the atmosphere is indeed transparent (J,H and K bands, for example); but the atmosphere is opaque at most NIR wavelengths (and, even at those IR wavelengths where the atmosphere is transparent, the transmittance is lower than at visible or radio wavelengths). See here for more info.

Re:Yawn

[Shag]

I'd only be impressed if they somehow made a ground telescope that could resolve in the infrared better than the Hubble. Bah, too easy.

I'm holding out for besting Hubble in ultraviolet.

Pass the SPF-100, wouldja?

Re:Yawn

[Lawrence_Bird]

isn't it a bit off to be comparing a 200" to Hubble's 94" once you have removed atmospheric effects? I'd like to see the same shot off similar sized scopes to see the real improvement over (a) unaltered ground based observation and (b) space based.   This is not to say the technique is not useful, of course it is.  Just seems the article is doing apples to oranges comparisons.

Re:Yawn

[sedman]

Without this technique, the Hubble is better. With it, we can use the larger ground based telescopes to get better images without the expense of launching them into space. Sounds pretty neat.

Re:Yawn

[ceoyoyo]

I guess the better comparison would be by price. What does it cost to produce a ground based telescope that matches Hubble's resolution? With image techniques that compensate for the atmosphere, the answer is a LOT less. Or you can spend more and get BETTER resolution. The answer without atmospheric compensation techniques is that in many cases you cannot build a ground telescope that matches Hubble's resolution.

The article is comparing the images produced and using the price as a basis. Aperture isn't a particularly meaningful variable to keep constant when you're talking about space and ground based telescopes because it's immensely cheaper and easier on to get large aperture on the ground.

Lucky Imaging

[dlawson]

First post, huh.
This technique is often used by amateur astrophotographers using newer CCD cameras and even webcams. Astronomy Picture Of the Day http://antwrp.gsfc.nasa.gov/apod/astropix.html is a great site to see this stuff. I haven't checked Googles pictures, but I am sure that there would be a number of them there, too.
The quality of some of these photos is amazing.
davel

Re:Lucky Imaging

[ackthpt]

CCD cameras need not all cost £££ or $$$. I'm in the midst of converting a Philips SPC900NC to an astro imaging camera. Alas, I don't think I'll finish in time for a trip with the scope to high elevation next weekend.

Re:Lucky Imaging

[Anonymous Coward]

Apologies for not having an account - but I would really like to ask a question for someone who understands the process.

the wikipedia entry on this subject http://en.wikipedia.org/wiki/Lucky_imaging states that new procedures take, '... advantage of the fact that the atmosphere does not "blur" astronomical images, but generally produces multiple sharp copies of the image'.

Does the correction algorithm apply a single vector to each image (ie the entire frame is shifted in unity) to produce the composite, or is a vector field applied to every pixel point in the image to shift individually the pixels toward their correct centres? Also if it is pointwise what type of transform is being applied, affine , perspective etc.

Re:Lucky Imaging

[YttriumOxide]

Please mod parent up so that someone knowledgeable can answer the question - I don't know the answer myself, but would love to, and the Anonymous Coward score of zero means many people may miss this great question.

Re:Lucky Imaging

[theckhd]

From this paper, which is linked to in the Wikipedia article:

The frame selection algorithm, implemented (currently) as
a post-processing step, is summarised below:
1. A Point Spread Function (PSF) guide star is selected as a
reference to the turbulence induced blurring of each frame.
2. The guide star image in each frame is sinc-resampled by a
factor of 4 to give a sub-pixel estimate of the position of the
brightest speckle.
3. A quality factor (currently the fraction of light concentrated
in the brightest pixel of the PSF) is calculated for each
frame.
4. A fraction of the frames are then selected according to their
quality factors. The fraction is chosen to optimise the tradeoff
between the resolution and the target signal-to-noise ratio
required.
5. The selected frames are shifted-and-added to align their
brightest speckle positions.

(bolding mine)

So it looks like each frame is shifted as a whole rather than each individual pixel. Which makes sense from the description of the process, since the theory is that the images you're picking in the Lucky Imaging technique are high-quality images with a random offset due to the atmosphere.

Re:Lucky Imaging

[dapsychous]

You did independent research and asked an intriguing question. You're obviously new here. You didn't need to tell us. :)

Re:Lucky Imaging

[Anonymous Coward]

I have another question that someone who knows more about image processing and astronomy might be able to answer.

This approach seems to select "the best" images among a series and combine them somehow.

Wouldn't a blind source separation/unsupervised learning/latent variable approach applied to the images work just as well, or better? I mean, instead of selecting "the best" images, why not model them all simultaneously, to statistically infer what the "underlying" source is?

Re:Lucky Imaging

[Rei]

What are you using to do the conversion? I tried using my DSC-H2 for afocal astrophotography, but due to its fast lens, I had really severe vignetting (to the degree of the pictures being nearly useless). I really need eyepiece projection or prime focal, but for that, I need something that I can remove the lens on (i.e., a DSLR or CCD). I've also heard good things about using quality webcams for astrophotography (esp. for lunar & planetary).

Are you familiar with any shift-and-add or automated lucky imaging software for Linux? I've been having the darndest time finding any. Hugin does the "shift" part very well (it's a photo stitching tool, after all), but it can't do the add, and does poorly at keypoints on images like you get with astrophotography. While it's great with normal photos, it's really poorly suited for astrophotography processing, and I need to find something better.

Re:Lucky Imaging

[kindbud]

The amateur stacking program Registax seems to be more sophisticated than this. It allows multiple alignment stars or points and shifts the corresponding subregions of the image as needed. Otherwise, the method for selecting the images is very similar.

http://www.astronomie.be/registax/html/multi_opera tion_1_.html

Re:Lucky Imaging

[Anonymous Coward]

i can understand the application of a global vector shift since the problem and solution are dictated by the physics of the problem. here we have a very narrow appature looking up into the sky so that with a given state of turbulence in the atmosphere it could be expected that the induced refraction would be constant across the field of view for the instant of time that the exposure is made (or ccd sampled).

but i agree that the application of the image selection approach is a bit weird - surely by discarding images you are discarding useful information as well. this is almost a classical motion estimation problem except that the delta shift is being used to realign the images rather than as a product in its own right. I was thinking about using a function that captures the spacial difference between fields as an error (perhaps squared) that could then be differentiated and solved with corresponding statistical confidence coeffecients computed.

what do you mean by a latent variable approach ?. i understand that (un)supervised learning approaches are the stuff of neural networks. Very interested to know of other approaches to the problem .

Re:Lucky Imaging

[Rei]

The various "solutions" have been reinvented a thousand times over by amateurs. Part of the problem, however, lies with the scientific community for keeping their programs proprietary and closed source, leaving amateurs no choice but to reinvent the wheel. There are a number of "speckle" imaging techniques that are better than the conventional shift-and-add algorithms (if I recall correctly, there are several calibration steps, and then they basically decompose the images into first or second order FFTs, average those, and then reconstruct the image). They're only useful for medium to large telescopes; as an astronomer that I wrote to about the subject pointed out, for small aperture scopes, you get no significant difference between true speckle imaging (where each part of the image is dealt with independently) and shift-and-add (where the whole image is moved at once). Telescopes have a diffraction limit based on the aperture, and the sky has a seeing limit. Either one can limit your ability to see details smaller than them. Shift-and-add effectively reduces the seeing limit by a certain amount (I forget the specifics, and don't have the email onhand), putting it below the diffraction limit of small amateur scopes. In "normal" sites, seeing limits are usually 2-3 arcseconds, and on such sites, you need to have something like a >8" scope for true speckle to be noticably better than shift-and-add. In "good" sites (mountaintop, desert) with a seeing limit of 1 arcsecond, you need something like a >20" scope for it to matter. And of course you need some sort of sidereal tracking and quality optics (i.e., not a junk brand) for this to be relevant. Such an 8" with a motor-driven equatorial mount means an investment of 1-2k$, and that's only where true speckle imaging *starts* to make a difference, and only if you're in a normal observing site. A quality 20" scope may run you something like 50k$. As a consequence, doing better than shift-and-add usually isn't of great importance to most amateur stacking software.

Re:Lucky Imaging

[Anonymous Coward]

what do you mean by a latent variable approach ?. i understand that (un)supervised learning approaches are the stuff of neural networks. Very interested to know of other approaches to the problem .

Latent variable modeling is basically the same thing as blind source separation or unsupervised learning, but from the perspective of traditional statistics. The models are often the same, but estimated differently (although sometimes it's just a terminology thing). Latent variable modeling is fairly popular in epidemiology, population genetics, psychology, economics, and has been used in other fields, such as chemistry.

In its most simple form, you have

Y = x + E

You don't ever observe x (i.e., it's latent), but you do have some observations Y, which are assumed to be measures of x contaminated by random errors E. Often, the goal is to infer x based on the noisy observations Y. The errors E are often assumed to be normal/Gaussian. There are different estimation methods for the model, but the most common are maximum likelihood or least squares-based approaches.

Latent variable models get much more complicated than that, with nonnormal errors, complicated functional relationships between Y and x, relationships between x and E, etc.

A hot topic of research in epidemiology currently is spatial latent variable models, where the data are spatial, and the x variables represent locations or spatial variables. In epidemiology, the interest is in locating sources of disease, whether that be some pool of an infectious agent, or a chemical pollution source, or some area affected by infrastructural problem, or whatever. It has particular relevance, here, though, in that you're again dealing with spatial data, just of an entirely different nature.

Re:Lucky Imaging

[cbacba]

It's been called video astronomy for years. In more recent times, webcams have started to dominate due to the direct input into the computer. Programs such as Registax, a free program on the web, have been around for years and through many iterations. In essence, this is done using relatively small telescopes and relatively insensitive equipment, mostly limiting it to planets, sun and moon. It consists of alignment since both the mount and the sky tend to vary the position of the object somewhat. In fact, this random noise of position permits higher resolution imaging than the pixel resolution of the camera. (random noise gets reduced or eliminated by statistics)

The easiest way to think of it is that the atmosphere does all sorts of weird things over a short time frame. It can displace the object slightly. It can bring it into perfect focus briefly and then blur it out tremendously. It can do this even to a portion of the image. Programs such as registax have alignment sections and stacking or combining sections. The use of fourier analysis and point spread functions can be done to measure quality - as can manual efforts. There are usually quite a few options that can be specified as well as settings. On the combined image, registax uses wavelet processing to balance 'noise' with data in the final image and it is strictly an artistic and personal preference decision.

Typically, the planetary stuff is averaged which provides lower noise per number of raw images than summing, which is needed for dim objects. One finds a few hundred to a few thousand raw images combined into a final image of extraordinary quality.

This stuff was pioneered by the amateur community. However, a few years back it was used to get an image of mercury's surface using a 60" telescope in california and tens of thousands of images were analyzed and combined.

Due to the nature of low photon count imaging and high noise, video astronomy has been quite limited in achieving much in the way of extended exposures based on short time frames, such as 1/20 second. However, fairly dim scenes have been brought into the realm of being recognizable out of a series of video frames that appeared to be total noise. Image intensifiers have also been used to image dim deep sky objects with some bit of success.

Since typical good seeing tends to vary somewhat in the subsecond realm, even exposures of a few seconds tend to blur and keeping telescopes tracking to that level is virtually impossible. Fortunately, so many deep sky objects of interest to amateurs are quite large and require mostly time.

What actually sets this new effort apart is the low noise camera which actually permits short sub second exposures to be summed at the lowest of intensities without the injection of random noise - typically readout noise which is for the normal ccd somewhere in the 3-20 electron count average per pixel. Since the sensitivity of the typical astronomy camera ccd ranges from 20-70 or 80% quantum efficiency (100% is one electron per photon) at least at the peak sensitivity wavelength of the ccd, short images aren't that practical when short is under 1 second for a dim deep sky object. That's where this new camera is making waves. If versions can be had for under $10k US, there will even be a market in the amateur world. However, the thinned ccds I noticed described on their website can cost as much as a house which is outside the realm of the dedicated fanatic (and the small college imaging program).

The article is nice to see and it's nice to hear about the new camera. Seeing it named 'lucky' reminds me of a fortune cookie and it really isn't luck but rather persistance and the ability to isolate clear seeing instants from blurry averages. And besides, the technique already has a name - it's called video imaging techniques and they did not invent it. What's more, it's also known as the poor man's adapative optics.

Re:Lucky Imaging

[Anonymous Coward]

In fact you jogged my memory about latent variables from an economics class a few years ago - seems like there are lots of applications.

Thanks for your explanation.

Re:Lucky Imaging

[geowiz]

ironically one can still achieve many great things by hooking up and old videocon tube video camera to even a 200 mm photo lens and aiming it at the stars. analog videocon tube camera have unlimited resolution (the imaging coating has no defined pixels - only the readout scan lines are limited and can be varied.) ccds have some advantages over videocon tubes but the primary reason they went into consumer cameras was lower cost of production and adjustment rather than their advantages in operation. in fact the first ccd consumer cameras had LOWER resolution than old videocon tube consumer cameras at the time. I can still nurse a near HD resolution picture out of an old video con tube camera i have from the 80's. tubes do suffer from blooming and are naturally additive in their imaging so the coating sensitivity must be picked based on expected exposure duration of each frame but they have nearly unlimited imaging resolution in the coating and -driven by modern electronics- the scanline readout resolution can real be high.

Re:Lucky Imaging

[geowiz]

taking this tube idea further - one could paint an old huge tv display tube of many inches across with videocon type coating and read so many scan lines from that huge image that you would get a resolution that would cost millions if done with a one off ccd. one could even read different parts of such a huge videocon type system using different circuits get near parallel outputs of different parts of the image screen thus increasing possible resolution even further.

Re:Lucky Imaging

[Anonymous Coward]

Someone please mod this Informative so I can make comments instead of modding. Thanks.

If they can do this from earth...

[Ant P.]

...can the same be applied in space telescopes to get rid of the interference of the gas clouds they're looking at?

Re:If they can do this from earth...

[Anonymous Coward]

Well sure. All you have to do is bounce your laser off of those gas clouds to find out how to compensate for them. That should only take a couple hundred or a couple of thousand years with a laser that would consume more power than all of the Earth uses. Oh, and you better hope that that gas cloud doesn't change in the transit time.

Re:If they can do this from earth...

[Aesir1984]

The distortion they are trying to get rid of is caused by motion of the air in the atmosphere. It's similar to the waves and blurring you see looking across a parking lot on a hot day. They put space telescopes out of the atmosphere to get above these effects. The objects they're looking at don't have this problem because the thing being imaged is what is giving off the light, it's not something between the source and the viewer like the atmosphere is and so does not cause diffraction to the same extent. I would expect that this technique works rather well for bright objects, however I wouldn't expect it to work well for the very dim objects that the Hubble is normally tasked to look at. It order for them to use this technique they have to take many images per second. For very dim objects this would only mean a few photons per picture, not nearly enough to figure out if this image is any sharper than any other. So we won't be able to get rid of space telescopes or adaptive optics just yet.

Re:If they can do this from earth...

[vtcodger]

***Well sure. All you have to do is bounce your laser off of those gas clouds to find out how to compensate for them.***

That's adaptive optics. 'Lucky imaging' looks to be something different. Sounds like Lucky Imaging tries to catch and merge portions of the image that occasionally, by chance, make it through the ever changing atmosphere with minimal distortion.

But I think that the answer to the original question is probably still 'No" It doesn't sound like Lucky Imaging per se is an answer to the question "How can I see objects obscured by cosmic gas clouds?"

Re:If they can do this from earth...

[theckhd]

I think your suspicions are probably correct.

Lucky Imaging relies on the fact that every so often, a really high-quality image makes it through the atmosphere almost unperturbed (based on the Kolmogorov model of turbulence). While I don't know whether the same model can be applied to cosmic gas clouds, there may be another model that could accurately model the phase distortions those clouds impress upon a wavefront.

To achieve this one must take many very short-exposure (compared to the time-scale of atmospheric turbulence, or gas-cloud turbulence in the case we're considering) images of the source. However, distant (or dim) objects often require reasonably long exposure times in order to collect a large enough amount of light to be able to see the image. The problem with this technique may simply be that the exposure time necessary for the Lucky Image algorithm to work is too short to actually collect enough light to create a good image in the first place.

Re:If they can do this from earth...

[TMB]

Not exactly, but there are some techniques in the radio that are kind of similar. The problem is that the timescale for gas clouds between us and a given object to change configuration is usually longer than an observation. But in some cases you can look at how fast you see certain kinds of fluctuations that are due to intervening gas clouds and infer the size of the object, even when that object is too small to have been resolved.

[TMB]

Re:If they can do this from earth...

[sbohmann]

Not entirely true. Even a significantly underexposed picture can be adjusted accurately to the right position by statistical measures. Do this for 100,000 for them, sum them up, and you've got a result.

In the extreme case that atmospheric noise completely shadows an object, though, the Hubble telescope can show off the real advantage of looking directly through free space: no dispersion of light in vacuum.

Even in a new moon, perfectly clear winter night somewhere deep in Canada, the light from the brighter stars creates a certain level of glowing within the atmosphere itself that suffices to hide many astral objects' light completely, as far as statistics go.

But surely...

[click2005]

Amateur Lucky Imaging is popular because the technique is so cheap and effective. The low cost means that we could apply the process to telescopes all over the world."

can't they use the same techniques with the HST itself?

Re:But surely...

[ScrewMaster]

I'm just blowing smoke here, but it seems to me that a technique designed to compensate for atmospheric distortion might not be all that useful when there's no atmospheric.

Re:But surely...

[click2005]

The technique takes the clearer portions from many images and merges them. The article says that some portions are less smeared than others but doesn't say if the atmoisphere was also magnifying the target or not. I know astronomers have used gravity from intervening distant objects to magnify other distant objects so couldn't a similar technique be used there?

Re:But surely...

[drudd]

As the previous poster noted, there isn't any atmosphere and thus the technique isn't useful for HST.

Additionally, while they don't mention details in the article, I presume they have a specially designed camera. This is an old technique, but it's generally limited to very bright objects due to something called readout noise. Basically all CCD's produce an additional signal due to the process of reading out the data. This limits the effectiveness of repeated short observations to sources which are much brigher than this noise, since the noise also grows linearly with the number of images taken.

To image distant galaxies you typically have to take exposures of one to several hours, and thus this technique isn't useful.

Doug

Re:But surely...

[BattleApple]

Possibly, if the HST was as large and powerful as the land based telescopes

"The images space telescopes produce are of extremely high quality but they are limited to the size of the telescope," Dr Mackay added. "Our techniques can do very well when the telescope is bigger than Hubble and has intrinsically better resolution."

Re:But surely...

[Anonymous Coward]

Actually you are mistaken. If you research it a bit the reason the excitement is out there is that the CCD imager used is capable of working with low contrast images at high speed. It was only recently created. Pretty cool stuff.

Re:But surely...

[hazem]

Additionally, while they don't mention details in the article, I presume they have a specially designed camera.

They are using a new kind of CCD that somehow lowers the noise floor. Details are at:
http://www.ast.cam.ac.uk/~optics/Lucky_Web_Site/LI _Why%20Now.htm

In fact this site (same basic place) is much more informative than the press release and answers a lot of questions:
http://www.ast.cam.ac.uk/~optics/Lucky_Web_Site/in dex.htm

Re:But surely...

[Tablizer]

[use it on Hubble also] I'm just blowing smoke here, but it seems to me that a technique designed to compensate for atmospheric distortion might not be all that useful when there's no atmosphere.

Drats! There goes my thesis.

Re:But surely...

[CalSolt]

Well shit, I'm waiting for the array of 1,000 foot telescopes on the moon. THEN we'll be doing some serious astronomy.

Re:But surely...

[andersa]

To sum up, the problem is readout noise. The faster you read out the CCD, the more noise you get. When you image a faint object the readout noise exceeds the signal level. The reason amature astronomers can use this technique anyway is because they are imaging bright objects (like planets), so the signal is easily discernable from the readout noise.

Now there is a new type of CCD with a built in digital signal multiplier that precedes the readout step in each individual pixel. You can simply select an appropriate multiplier that gives pixel values that fall nicely in the middle of the register width and when you read out the value, any noise can simply be subtracted away because you know that it will be much less than the signal value you are looking at.

Re:But surely...

[SillySlashdotName]

Just wanted to add that amateur astronomers can image Deep Space Objects (DSOs) using modified (for long exposure) web cams as well as Lunar/Planatary Imaging (LPI) using unmodified web cams.

Either one is enhanced by 'stacking' images and processing a bunch of then. This is because the S/N ratio improves with additional images added to the stack.

The Signal increases by the square of the number of images, while the Noise increases by the sum of the number of images - so the Signal increases faster than the Noise, and with enough images to work with, the S/N ration can be manipulated to get some good images of both LPI and DSO.

I have an 8" reflector on an equatorial mount, and have a modified QuickCam 3000 Pro (google for Steve Chambers web site for modification instructions used) for DSO and an unmodified SPC900NC web cam I use for LPI. I use RegiStax and K3CCD software - K3CCD to control the modified QC3k. Total price of equipment was under $300US

I got some adequate shots of the lunar eclipse on the 28th of August, but even using a 60mm refractor, I was not able to get enough panoramic views to make it really interesting. I got some good shots of the lunar surface, but the equipment brought the subject too close for good ECLIPSE imaging. A tele-reducer would have been nice... maybe by February 21, 2008 when the next total lunar eclipse happens!

Re:But surely...

[Anonymous Coward]

So if the Lucky Imaging gets rid of the atmosphere effects, then how can these images be of *better* quality than what the Hubble telescope is capable of?

One would figure they could be *at most as good as* the Hubble images...

Re:But surely...

[geowiz]

however by taking a black image with the ccd covered first, you can get a direct readout of the noise for the ccd and then subtract it from each image thus canceling the noise becuase the noise doesn't change much between images since it is ccd specific and not image or target specific. so noise doesn't limit usefulness to a large degree.

Re:But surely...

[drudd]

Imaging with the shutter closed measures a different source of noise known as dark current (thermal noise of the ccd will cause a small number of free electrons rather than those freed by incoming photons). This noise is proportional to the length of the exposure, unlike readout noise which is a single constant factor.

As far as I know (I'm just a theorist), it is impossible to remove readout noise, since it is affected by the actual distribution of electrons on the ccd image. Someone who actually deals with astronomical imaging should feel free to correct me here.

Doug

awesome...

[Anonymous Coward]

gonna get me some new posters to fa..look at!

ummm...

[rstruzik]

using 'Blue Peter' technology

sounds painful

Re:ummm...

[German_Dupree]

That's what happens when you play with string...

Exposure Time?

[MonorailCat]

TFA states that the camera takes 20 frames per second. Aren't most exposures of deep space objects on the order of seconds or minutes (or longer). Seems like 1/20th of a sec wouldn't cut it for all but the brightest objects.

Re:Exposure Time?

[gardyloo]

Add up 1000 of those frames, and you have a 50 second exposure.

Re:Exposure Time?

[QuickFox]

Seems like 1/20th of a sec wouldn't cut it for all but the brightest objects. One of the short texts below the two initial articles says that it's a new camera capable of detecting individual photons:

This new camera chip is so sensitive that it can detect individual particles of light called photons even when running at high speed. It is this extraordinary sensitivity that makes these detectors so attractive for astronomers. Unfortunately it doesn't give any details on how much light is needed compared to other techniques.

Re:Exposure Time?

[ackthpt]

Use something like Registax.

Re:Exposure Time?

[PhunkySchtuff]

No, you don't. There is a certain threshold for signals to be recognised above noise in an image sensor. If the signals you're trying to detect are so weak, many rapid samples will only give you more noise

Re:Exposure Time?

[gardyloo]

Signal-to-noise should go down as roughly 1/N^(1/2), where N here is your number of exposures (as long as your signal isn't changing between frames). More exposures means a better signal-to-shit ratio.

    Now, if you're basing a real signal as being above some threshold, and noise as being below, then you need only one exposure, if the signal is present in that exposure. Otherwise, just keep snapping frames. No different than exposing film or a long-exposure sensor for a longer time.

Re:Exposure Time?

[JRIsidore]

Well, yes, but then why don't you expose 50 seconds in the first place? If you simply add up the images you'll suffer the same distortions that prevented good images in the first place. Unless you do some math tricks before summing them up you gain nothing at all. But to do that the single frames must have a clearly visible signal to work with, hence you either need a very sensitive camera or limit yourself to the brighter objects up there.

Re:Exposure Time?

[zippthorne]

Because you can throw away a few of 'em for various reasons, and you can register them electronically, which takes some burden off of the telescope mount.

Re:Exposure Time?

[SillySlashdotName]

If you simply add up the images you'll suffer the same distortions that prevented good images in the first place.

Expose for 60 min, giving (at 20 exposures per second) 72000 exposures.

Pick only the best 1000 exposures, keeping only the best, sharpest, clearest 1.4%, or getting rid of the worst 98.6%

Run them through your imaging algorythm.

You now have a 50 second exposure without all the blurring, distortion, and general cruft you threw away with the 98% (of the total) crap exposures you got rid of.

...the single frames must have a clearly visible signal to work with...

Nope. The stacking/summing algorhythm does not need a visible signal to work on, it used the electronic signals from the CCD at a level that would not (neccesssarily) be visible to the eye. It does need a discernable signal (not neccessaryily the object of the photograph, not neccessarily visible to the naked eye) to reference when stacking the exposures, but that is not often a problem.

...hence you ... need a very sensitive camera ...

That is the point of the article, the new, low noise CCD. The stacking, processing procedure is nothing new, the sharp images are nothing new - the hyperbole about 'Better than the Hubble!' not withstanding - only the new cammera is news in the Press Release. To bad they did not give more information about the camera.

Re:Exposure Time?

[JRIsidore]

The procedure you described only works under a certain premise: if you make thousands of images you can only take those that show the same type of blurring (or assure the distortions don't change during the time of exposure) so adding them will actually improve the image quality . When the signal is hidden in the noise and not discernible (and I don't mean necessarily by the naked eye) you have no way of telling which images are the best, and simple adding up won't help. A sensitive camera is therefore a "must", which is the point of the article, I agree.

Re:Exposure Time?

[geowiz]

wrong. you can subtract the noise by using a balck covered noise refence image fromt he ccd. then you can judge the distortion of each rapid image by measuring the contrast between pixels. un distorted images will have more contrast. add only the image with highest contrast and you sum the un aberrated images. you can even sum just the parts of each image that have the greatest contrast and tooss parts of each image that are more blurred. you can tell which images are blurred by analysis of tre frwequency spread on the pixels. you donlt even have to knwo target characteristics.

Re:Exposure Time?

[geowiz]

i agree that the only real news in the release here is the advance of the ccd senstitivity and that has nothing to do with the academic institution claiming credit in the press release. furthermore it is a natural advance of a technique invented rather than the discovery of a new technique. Academic institutions have great PR departments - better than us amateurs - however it is painful to see institutions get so much credit for things often discussed by amateurs openly on the net long before an institution publicizes their duplication of it. The breakthrough technique of summing very short ccd electronic exposures is the key innovation which I invented in 1995. To my knowledge no one publicly stated that idea before then but if I am wrong I would love to see it as I never mean to take credit for something that someone else did first.

Re:Exposure Time?

[JRIsidore]

You just repeated what I said... you need a discernible signal in order to do this. Maybe our understanding of "discernible" differs.

Compared to adaptive optics?

[kebes]

One of the main limitations to ground-based optical telescopes (and one of the reasons that Hubble gets such amazing images) is that the atmosphere generates considerable distortion. Random fluctuations in the atmosphere cause images to be blurry (and cause stars to twinkle, of course). The technique they present appears to be taking images at very high-speed. They developed an algorithm that looks through the images, and identifies the ones that happen to not-blurry (hence "lucky"). By combining all the least blurry images (taken when the atmosphere just happened to be not introducing distortion), they can obtain clear images using ground-based telescopes (which are bigger than Hubble, obviously). I imagine the algorithm they've implemented tries to use sub-sections of images that are clear, to get as much data as possible.

Overall, a fairly clever technique. I wonder how this compares to adaptive optics, which is another solution to this problem. In adaptive optics, a guide laser beam is used to illuminate the atmosphere above the telescope. The measured distortion of the laser beam is used to distort the imaging mirror in the telescope (usually the mirror is segmented into a bunch of small independent sub-mirrors). The end result is that adaptive optics can essentially counter-act the atmospheric distortion, delivering crisp images from ground telescopes.

I would guess that adaptive optics produces better images (partly because it "keeps" all incident light, by refocusing it properly, rather than letting a large percentage of image acquisitions be "blurry" and eventually thrown away), but adaptive optics are no doubt expensive. The technique presented in TFA seems simple enough that it would be added to just about any telescope, increasing image quality at a sacrifice in acquisition time.

Re:Compared to adaptive optics?

[Cadallin]

Both are employed pretty heavily by advanced "Amateur" astronomers. I put amateur in quotes because people at the high end of the hobby may have setups costing $50,000-$100,000+ dollars, going up to as much as people are willing to spend. There are several companies (http://www.sbig.com/ for example) that specialize in producing imaging equipment and software for these setups. It's pretty amazing what these people are able to do.

Re:Compared to adaptive optics?

[Phanatic1a]

ObRTFA: RTFA. It's not used *instead* of adaptive optics, it's used together with adaptive optics.

The camera works by recording the images produced by an adaptive optics front-end at high speed (20 frames per second or more). Software then checks each one to pick the sharpest ones. Many are still quite significantly smeared but a good percentage are unaffected. These are combined to produce the image that astronomers want. We call the technique "Lucky Imaging" because it depends on the chance fluctuations in the atmosphere sorting themselves out.

Re:Compared to adaptive optics?

[garvon]

http://www.ast.cam.ac.uk/~optics/Lucky_Web_Site/LI _vs_AO.htm
Is an article comparing Lucky to Adaptive. It looks like Lucky can work with dimmer objects then the Adaptive.

Re:Compared to adaptive optics?

[jstott]

Overall, a fairly clever technique. I wonder how this compares to adaptive optics, which is another solution to this problem.

The two techniques are unrelated; either one or both at the same time can be used to improve the images. Actually, the sample images from the original article were taken through a telescope (Palomar) using basic adaptive optics to improve the image before the "lucky" software even saw the data.

As you suggest, this also works with sub-sections of the image. I saw this same technique described at a conference about five years ago for imaging through terrestrial turbulence (I think it was funded by the US Army) and they were using image sub-sections at the time, having seen the videos they had then subsectioning is clearly both possible and effective.

-Jonathan

Re:Compared to adaptive optics?

[Tablizer]

Random fluctuations in the atmosphere cause images to be blurry ... The technique they present appears to be taking images at very high-speed. They developed an algorithm that looks through the images, and identifies the ones that happen to not-blurry (hence "lucky"). By combining all the least blurry images ... they can obtain clear images using ground-based telescopes

Actually, this is kind of what the human eye/brain does when observing thru telescopes. Human observers have generally been able to discern details better than (regular) astrophotography. This is because the brain remembers the details of the sharpest moments.

However, I see a potential downside of the Lucky technique. For dim objects you need a lot of light, or samples in this case. If you toss the majority of them because they are blurry, then you don't get a lot of samples. Hubble can keep just about every sample. Thus, such a technique may work better for obtaining clarity of brighter objects than for obtaining the details of faint objects.

Re:Compared to adaptive optics?

[TheMCP]

Adaptive optics does not require a guide laser. The system often works by identifying an object in the image which is essentially small: a far away star that will register as more or less a point source, for example. It then uses that as its guide, and distorts the mirror to minimize the image of that object: essentially, to focus it. If that object is "focused", then objects near it generally are too.

Using this methodology, a large ground based telescope can easily achieve better imaging than the Hubble, and this has been true for a decade or so. The Hubble is more famous, however, because NASA often uses it to make pretty pictures and puts them out as press releases, while most large ground based telescopes are used to do actual science and the results are a lot of un-photogenic numbers.

Re:Compared to adaptive optics?

[edunbar93]

ObRTFA: RTFA. It's not used *instead* of adaptive optics, it's used together with adaptive optics.

No, they propose that it be used together with adaptive optics. The research that was done to produce this press release was actually done at the Mount Palomar observatory, which was completed in 1947 and most certainly does not feature adaptive optics.

From the article:

The technique could now be used to improve much larger telescopes such as those at the European Southern Observatory in Chile, or the Keck telescopes in the top of Mauna Kea in Hawaii. This has the potential to produce even sharper images.

(Emphasis mine)

Re:Compared to adaptive optics?

[john83]

Both are employed pretty heavily by advanced "Amateur" astronomers. I put amateur in quotes because people at the high end of the hobby may have setups costing $50,000-$100,000+ dollars, going up to as much as people are willing to spend. There are several companies (http://www.sbig.com/ for example) that specialize in producing imaging equipment and software for these setups. It's pretty amazing what these people are able to do. I attended a lecture a year or two ago by a respected academic in adaptive optics (Chris Dainty, for the curious). He described efforts to put together an AO kit for amateur astronomers. I think he said that he wasn't able to get it under a few thousand Euro. It's not a cheap science, for sure.

You Too Can Get Lucky.

[Erris]

DIY.

Re:You Too Can Get Lucky.

[[rvr]]

This is indeed no news to amateur astronomers. This technique has been used extensively by planetary imagers in recent years to take amazing photos of Jupiter, Mars and Saturn. The basic tools are a good webcam to take AVI files and Registax to proccess the frames. Take a look to Damien Peach's best images.

As for pro, there is even an article in Wikipedia about it: Lucky imaging: "Lucky imaging was first used in the middle 20th century, and became popular for imaging planets in the 1950s and 1960s (using cine cameras or image intensifiers). The first numerical calculation of the probability of obtaining lucky exposures was an article by David L. Fried in 1978."

In order to throw away many frames and retain only those of high quality, better have a bright object or a big telescope. In this case, the astronomers had been able to image a faint nebula.

Re:You Too Can Get Lucky.

[Anonymous Coward]

What? A twitter post without "M$ Windoze" Inconceivable!

Re:You Too Can Get Lucky.

[Trogre]

Cool. Anyone know if there's a GIMP plugin for this sort of thing?

Performance vs. Adaptive Optics

[Anonymous Coward]

Can anyone comment as to whether this method would be superior to the adaptive optics currently planned for the Thirty Meter Telescope (of which Caltech is a participant)? Or would this potentially be used as a supplement to the adaptive optics rather than a low-cost substitute?

Re:Performance vs. Adaptive Optics

[gardyloo]

Can anyone comment as to whether this method would be superior [...] You must be new here.

Re:Performance vs. Adaptive Optics

[Aesir1984]

It's not really superior, it's just cheaper. If you're only looking at bright objects (planets, Messier objects, nearby galaxies, and some stellar remnants) this method would work very well. The problem comes when you try to take a picture of dim objects because you have to have enough light be caught by the optics in the exposure time (1/20th of a second in their example) to get an image good enough to tell if it's a "lucky" image or not. If you're looking at something really dim then those few photons you'll get in that amount of time just won't cut it. One black picture is just as sharp as another as far as the computer algorithm is concerned.

Re:Performance vs. Adaptive Optics

[Anonymous Coward]

In the article, they mention that this technology would best be suited to large ground telescopes. What I infer from this is that due to the lesser ability of smaller space telescopes to gather light, that this technology would be less suited, but a suitably large mirror/lens/whatever-is-used-in-large-telescopes could probably achieve significant enough light-gathering capabilities as to make for a good picture even with fast shutter speeds.

Re:Performance vs. Adaptive Optics

[Anonymous Coward]

You infer wrong. This technique has zero application in space telescopes because they SIMPLY DON'T NEED atmosphere compensation tech.

Spider-sense

[BitwizeGHC]

That is really quite amazing, and reminds me a bit of the jumping spiders whose retinas vibrate to increase their optic resolution.

Re:Spider-sense

[Anonymous Coward]

jumping spiders whose retinas vibrate to increase their optic resolution.

Well that's me tiptoeing nervously round my own home for the next week.

Re:Spider-sense

[QuickFox]

Don't worry, they'll only jump on you when you're asleep.

Errors?

[WPIDalamar]

Since it's running through a computer algorithm and piecing many together, and isn't just a single "lucky" picture, I wonder how much error is introduced by the algorithm. I mean sure, an algorithm like this might work well most of the time, but what happens when it produces an image that looks clear, but isn't accurate.

Re:Errors?

[Anonymous Coward]

"...what happens when it produces an image that looks clear, but isn't accurate. "

This is a real possibility. Although probability is against this happening very often, or seriously obscuring the reality of what we're imaging, it is quite possible that simple common patterns of image noise could show up very often on these lucky images because they pass as "sharp" segments. But the more complex the pattern, the greatly less probable it becomes. Reducing this effect would require a better algorithm that is able to correctly identify what is really there, and what is artifact.

Re:Errors?

[Score Whore]

Assuming the noise is random, and the object isn't, then these should be pretty close to what the same picture taken with the same optical equipment in a vacuum would produce with a slight bias towards the center of the noise. That is, if your noise is evenly distributed between 0.0 and 5.0, then averaging a few hundred slices through time would result in a fixed noise of 2.5 across the board. Which is the same thing as no noise.

There are things this wouldn't be useful for. Mainly anything that might be changing quickly. Like a light chart. Or some objects that emit in wavelengths that are absorbed by the atmosphere.

Re:Errors?

[Max Littlemore]

The principal is that by taking lots of pictures of the same thing, you can correct the error. The larger the sample you take, the closer you get to the true image. For error to be amplified you would almost need the same random dust particle arrangement from the telescope to the edge of the atmosphere in a significant sample of the images, which is very unlikely.

Of course you probably understand that.

what happens when it produces an image that looks clear, but isn't accurate.

In answer to your actual question, there are two possibilities. One is that life goes on as normal, the other is that it doesn't. Given the gravity of the consequences of even a small error in these images, we must be completely sure of their accuracy.

I therefore nominate you to travel to the cat's eye nebula to check. In return I promise to do everything in my power to ensure that the language is still alive on your return, so that someone can understand your report.

Dr. Mackay?

[comrade k]

Dr Craig Mackay is happy to be contacted directly for interviews

Man, the whole Stargate franchise has been really going down the drain since they cancelled SG-1.

Government gets lucky.

[Anonymous Coward]

"The researchers, from the University of Cambridge and the California Institute of Technology (Caltech), used a technique called "Lucky Imaging" to take the most detailed pictures of stars and nebulae ever produced - using a camera based on the ground. "

This also works when looking the other way.

Re:Government gets lucky.

[Artifakt]

This is probably an adaptation of something the U.S.'s Office of Strategic Reconnaissance has been using for the last 10 years or so to do just that. (But remember - If I actually knew that for sure, I couldn't say anything).

Re:Government gets lucky.

[Tablizer]

This is probably an adaptation of something the U.S.'s Office of Strategic Reconnaissance has been using for the last 10 years or so to do just that. (But remember - If I actually knew that for sure, I couldn't say anything).

You might be right. The "sky spies" may have had this technology for many decades, but couldn't publicize it so that the enemy wouldn't have it also. Its a shame that there may be many such military tech advances that we can't use for science. Somebody in the Pentagon probably said, "oh shit, the cat's out of the bag" when it was (re) discovered by civilians.

Re:Government gets lucky.

[patchvonbraun]

This happens a lot.

The NSA knew about Differential Cryptanalysis a long, long time before Shamir et al re-discovered and perfected the technique
    in the civilian world. Back when DES was being approved for use as a FIPS (1978 timeframe), the NSA suggested new S-boxes,
    but never gave clues about *why*.

When Shamir developed Differential Cryptanalysis (DC), it became clear that the S-boxes in DES as it appears in the FIPS were
    just-about perfectly robust against DC, but any other randomly-chosen S-boxes were definitely *not* robust against DC.
    The NSA eventually admitted that they knew about DC long before the civilian cryptographers did.

But then Linear Cryptanalysis was born in the civilian crypto community, and *that* completely blindsided NSA.

Now, on a more (ahem) focussed topic.

I'm an amateur *radio* astronomer from time to time. Averaging is the bread and butter of making radio astronomy work at all.
    With the exception of the Sun, all other celestial objects of interest are *below* the noise floor of my instrument. By using
    long-term averaging, the noise floor diminishes with the sqrt() of the averaging time. I sample at 8megasamples per second,
    and typically average between 15 seconds and 60 seconds of data. I can see through the atmosphere and clouds without any issue.
    But there are effects from variability of the interstellar mediaum that can be seen from time to time.

Many amateurs already do this

[szyzyg]

THere's several pieces of software which do som parts of this - Registax is what I use, but amateurs usually only have enough aperture to make this work for bright objects like planets. You can take a good quality webcam (the top of the line Phillips webcams are the best bang for yout buck), record some video of a planet through a telescope and then pick out the least distorted images before adding them together to create the final image. Now, the trick is getting the best measurement of which images are undistorted, and getting enough light in each frame while keeping the esposure time short enough to beat the atmosphere.

Look at the planetary images here for my attempts at this technique.

Re:Many amateurs already do this

[Tsiangkun]

nice work, some incredible shots in that collection.

Re:Many amateurs already do this

[edunbar93]

The difference is the resolution of the camera. The Phillips Toucam can produce movies at 640x480, whereas I would expect that the cameras they were using in this research produce research-grade resolutions in excess of 1280x1024, which is no small feat to get working at 30fps. Also, to make this work with a Toucam requires very bright objects and/or very large telescopes.

However, it's worth noting that amateurs' results today are typically much better than those of professional astronomers 30 or even 20 years ago, and are no doubt the inspiration for this new camera.

Re:Many amateurs already do this

[Anonymous Coward]

"resolutions in excess of 1280x1024, which is no small feat to get working at 30fps"

Presumably you've not heard of HDTV, then? Blu-Ray? HDV? HD-DVD? HD-CAM? DVC-Pro HD? HD-CAM SR? No?

Re:Many amateurs already do this

[TMB]

Aren't most amateur telescopes still below r_0? Depends what wavelength you're looking at, obviously, but I wouldn't have thought that you would get multiple speckles until you get up to diameters of 50cm or more (or maybe my sense of amateur telescopes is way out of date...).

[TMB]

Comparison to hubble...

[Anonymous Coward]

TFA mentions that they can achieve images better than Hubble. The sample image they show, of the Cat's Eye Nebula, isn't as sharp as the Hubble image of the same object.

Probably they can push their technique harder than this initial image suggests (it was mainly comparing the "lucky" image with a conventional, blurry, ground-based image)... But I just thought it would be good to show Hubble's pictures alongside.

Re:Comparison to hubble...

[Anonymous Coward]

I was going to mention the same thing. While the Hubble image is indeed many times sharper, perhaps by "can achieve" they were simply conjecturing that a ground-based telescope could be designed to fully exploit the benefits of this post processing technique, so that superior resolution to Hubble is possible (at reduced cost no less).

Re:Comparison to hubble...

[Bemopolis]

Yes, it would take a lot more explanation on their part to convince me that that image is sharper than the corresponding HST image. This could be the result of press office "manglage" (which, sadly, I have experienced firsthand). What I do know is that once they can remove all atmospheric effects from the 200-inch telescope, then the pictures would be twice as sharp as HST, since it is half its size (92 inches).

But my eyes tell me they haven't removed all atmospheric effects yet, and their words aren't convincing.

Re:Comparison to hubble...

[JavaManJim]

Thank you for the Hubble version of the Cat's Eye Nebula picture.

1. Slashdot introduction summary should say not as good as Hubble (HST). Instead mistakenly says better than Hubble.

2. The TFA linked site should (chuckle) show Hubble pictures along with the other ground based pictures.

God bless John Grunsfeld and the other NASA space walking astronauts who fix HST. Also the vast supporting cast for those missions.

Thanks for the update.
Jim

Re:Comparison to hubble...

[Dr. Zowie]

It appears that they simply picked a bad demo image. The Caltech site has a much more compelling sample at http://www.astro.caltech.edu/~nlaw/lamp_pics/.

Re:Comparison to hubble...

[kindbud]

I love LAMP.

Lucky Telescope

[s0m3body]

Just give me a dope and you won't believe what my resolution is !

Ugg, Background

[Nutty_Irishman]

You would think that someone that developed a start of the art method to remove noise and distortions from atmospheric images would think twice about using a salt and pepper bitmap background.

50,000 times cheaper, so what

[schwit1]

Technology improves over time and it gets cheaper. The HST is 20 years old, and the technology to design and build it are even older. New inventions will come along in the next decades to make Lucky seem overpriced. But that doesn't stop people from deploying it now.

Re:50,000 times cheaper, so what

[Dr. Zowie]

Unfortunately, space travel isn't subject to Moore's Law. Spaceborne stuff is going to remain much, much more expensive until space access is routine -- and even then it will remain very (rather than insanely) expensive so long as we are using chemical rockets and not reusable fusion drives or some other science fiction gizmo.

Re:50,000 times cheaper, so what

[ceoyoyo]

Space telescopes like Hubble have this unfortunate requirement that they be launched into space. An equivalent telescope made today, or twenty years from now (barring something revolutionary) will still cost a lot more than a ground telescope. Note that most of the telescopes they're talking about are at least as old as Hubble.

Not convinced by TFA

[Oligonicella]

Just went and looked up the Cat's Eye Nebula as taken by the Hubble. Lot more detail. What gives? Someone able to explain that, please?

Re:Not convinced by TFA

[darkmeridian]

This is all a guess.

The Hubble images probably resolve fainter objects but the Lucky images are sharper. Sharpness means resolution of distinct objects is better. The Hubble may see more while the Lucky sees them sharper but misses out on faint objects. The big question for me is how good the Earth-based telescope is at picking up faint images, which appears to Hubble's strength. The Hubble Telescope can peer at an object for hours at a time with an open aperture. A ground-based telescope cannot because the Earth turns.

I am also going to guess the Hubble image was processed more heavily after the fact.

Re:Not convinced by TFA

[Anonymous Coward]

Fixing your knowledge:
- The Earth rotates in 24h
- Hubble orbits in 90 min
so Hubble cannot peer "hours at a time", but ground telescopes can.

Fixing you lack of imagination:
- apertures can be closed and reopened.

Re:Not convinced by TFA

[Anonymous Coward]

Fixing your knowledge:
- The Earth rotates in 24h
- Hubble orbits in 90 min
so Hubble cannot peer "hours at a time", but ground telescopes can. Hubble can actually produce millon-second-long exposures. That's 400 orbits, but stacking 21 minute exposures.

Re:Not convinced by TFA

[Oligonicella]

Your 'knowledge' fix was explained away subsequently as baseless. Your aperture 'fix' is nonsense as both can do that. Good thing you posted AC.

Basically TFA is loaded. The photos they provide pale in comparison to Hubble shots. At the moment, the article is tripe. Let's see if things improve.

Re:Not convinced by TFA

[Anonymous Coward]

True, but the point was "at a time", which means "without break".

Let's see it beat Hubble at:

[tjstork]

I would think that before the scientists claim victory over Hubble, let's see their camera best some of Hubble's best work:

http://hubblesite.org/

There's a number of excellent Hubble images of just about everything in our solar system to the most distant galaxies.

I would put my money on Hubble, for two reasons.

First, the averaging algorithm is not without its flaws. They make the assumption that by averaging out a bunch of images, you eliminate distortion. For this to work, you have to assume that the probability of a particular pixel being in the right spot is higher as the distortion would essentially be random, and that could theoretically not be the case. If the distortion is completely random, then, averaging a set of images would essentially lose the pixel that is being pushed around its "real" spot by the atmosphere, and you can actually see that, as the corrected images still look muddy compared to their HST or even adaptive optic counterparts.

Secondly, the atmosphere doesn't just distort light, it also filters it. You can use averaging to remove distortion "noise", but, there's really no way to ascertain what information was removed by the atmosphere.

The bottom line is, yes, you can get some pretty good results with averaging software, but, if you have money to spend, the best images are going to be space based, and its still going to cost a billion dollars. Given the promise the heavens hold for the advance of human understanding, let alone essentially infinite resources, one only hopes that policy makers will not be mislead by the outrageous claim that one can get the best images from the ground. You can't. HST should not be thought of as an aberration made obsolete by adaptative optics or the low budget averaging. Low budget averaging and adaptive optics really need to be thought of as getting by until we can put larger, and better visible wavelength telescopes into space.

Imagine what a space based Mt. Palomar sized mirror could do, if in space!

Re:Let's see it beat Hubble at:

[SoupIsGood Food]

There's also the issue of deep sky surveys - these require looooooooooooong exposures. If you park your butt in front of a telescope in your backyard for a look through it, you will see more detail the longer you're parked. Your eye is able to pick out more information with longer exposure. So it is with imaging. Yeah, a really big mirror like on Palomar means you don't need to spend as much time imaging the same section of space as a smaller scope to capture equivalent detail, but here's the deal: the HST can look at the same thing for 20 minutes at a time, ekeing out every last little photon it can from the scene. Then it will do it all over again on the next orbit. And again, and again... each 20 minute pass adding to the cumulative light captured, until it's equivalent to an eleven day exposure. You're not going to get lucky enough to get eleven days worth of distortion-free viewing from a terrestrial scope without inviting in all sorts of software artifacts, sorry.

So, Lucky Imaging's a great step forward for some dirtside scope applications... less so for others, and not a replacement at all for a decent space-based instrument.

Re:Let's see it beat Hubble at:

[kindbud]

First, the averaging algorithm is not without its flaws. They make the assumption that by averaging out a bunch of images, you eliminate distortion.

No, they don't assume that. Their assumption is that an average of a bunch of images selected because they are probably sharper than average, will be sharper than an average of a totally random selection of images. And that is a sound assumption. The trick is in selecting images automatically that have a high probability of being sharper than average. A person can do this by sight but with hundreds of thousands of images, it takes a great deal of patience and time. Better to use a computer running a clever algorithm for selecting images.

Surprised this hasn't been asked yet here

[earthforce_1]

Is the algorithm used to pick the best image, or part of an image open source?

Re:Surprised this hasn't been asked yet here

[Tablizer]

Is the algorithm used to pick the best image, or part of an image open source?

Ah, the Google Web-Search Telescope.

link

[Anonymous Coward]

Hubble: Cat's Eye Nebula

Is the Crusade Over?

[Anonymous Coward]

I mean the crusade to save the Hubble.

Funny how something that had the scientific community up in arms and invectives flying at NASA and the Bush Administration (what's new eh?) is now a moot point. Twice as sharp, waaay cheaper. Time to put away the banners, boys!

Blue Peter for non-Brits

[ackthpt]

using 'Blue Peter' technology

Blue Peter is a BBC childrens show. Blue Peter Technology is effectively something so simple a child could do it.

Re:Blue Peter for non-Brits

[poena.dare]

Just wait... they'll be using "squint" and "move the menu back and forth at arms length" technology next.

Re:Blue Peter for non-Brits

[monk.e.boy]

Did anyone understand that comment?

Re:Blue Peter for non-Brits

[Cruise_WD]

Have you never watched a person with failing eyesight try and read the small-print on a menu (or anything else)?

Well, I thought it was a funny comment, anyway.

Re:Blue Peter for non-Brits

[33MHz]

"'Blue Peter' technology"? "20 frames per second"? "computer software to choose the best images"? Blimey, I must have missed that episode. Tracy Island pales into insignificance compared to this!

Re:Blue Peter for non-Brits

[isorox]

Blue Peter Technology is effectively something so simple a child could do it.

Given a quantity of washing-up bottles and plenty of sticky-back plastic

Re:Blue Peter for non-Brits

[Rei]

You laugh, but there's a legitimate method for seeing fainter objects with the naked eye through a telescope involving looking to the side of it, since your peripheral vision is more sensitive to faint objects. It can be tricky to pull off, though, because you instinctively want to look at what you're trying to perceive.

Not quite squinting, but still an eye trick ;)

Re:Blue Peter for non-Brits

[geowiz]

you can increase your eye light sensitivity breifly by eating buillbery jam too. it contains compounds that temporarilty increase the retina's light sensitivity - old british ww@ pilots did this before night flights. and of coarse limit your eyes exposure to non red light for a long period before viewign to avoid narrowing your irs.

Interesting but picture quality unjustified

[bit01]

The technique they're using, while interesting, needs more justification.

I'm wary when I see people doing any selection on random data because there's the problem of selection bias; throwing away the hundred results that don't match what they want and keeping the one that does. Just getting an image that seems plausible is not good enough.

Their quality measure isn't one I'd use. They should be comparing the technique-plus-low-resolution-optics against high-resolution-optics directly. That is, doing image differencing of images taken at the same time and seeing what differences there are. They may well have good reason for assuming it's all okay but until somebody does that test they cannot assume they've removed all the variability that the atmosphere provides; there could be all sorts of hidden biases due to various atmospheric, molecular and statistical effects.

---

"Intellectual Property" is unspeak. All inventions are the result of intellect. A better name is ECI - easy copy items.

Re:Interesting but picture quality unjustified

[thePig]

An interesting point -
Since we are doing science, is it a good idea to throw away nonconformist images away as improper?
Are we not bringing our own bias also to this? If we are only looking at what we expect to find and throw away the unexpected, wouldn't science take a hit?

Re:Interesting but picture quality unjustified

[edunbar93]

Well, I guess that all depends on whether or not you can *prove* that the that you throw away is worth throwing away. If you take a thousand images of exactly the same thing over the course of an hour, and keep only the best images, that's a little different than taking toxicology readings from a thousand different patients and keeping only the best results. The cat's eye nebula isn't going to change measurably from our perspective over the course of an hour. If you keep careful documentation on what you do to the images during processing, (after all, the results must be reproducible) that should keep problems to a minimum.

However, there's also a term called "overprocessing", by which some amateurs can create images with detail that doesn't actually exist in the original images. But if you keep careful documentation on what you do to the images during processing, (after all, the results must be reproducible) that should keep problems to a minimum.

Common use with amateurs, but has issues

[edremy]

As many have pointed out, there are a whole pile of applications that do the same thing for amateur telescopes. I've taken my Dad's 40-year-old 6" Dynascope, fixed up the motor drive, bought a $60 webcam (Philips SPC900), adapter and UV filter and gotten some quite nice photos of the Sun, the Moon, Jupiter and Saturn by capturing a few thousand frames and running them through Registax. (I'm working on Mars and Uranus- a whole lot harder with a small scope from a suburban backyard.)

I'm curious though about how they deal with some of the "features" you get to see with this technique. It's *very* easy to stack a few hundred images, run Registax's sharpening filter and get some interesting pictures of stuff that doesn't really exist. I'm not sure I really trust the fine detail in my photos- unless I see it in another taken a few hours later it may well not be real.

Would they be able to try this with hubble?

[Anonymous Coward]

Imagine, say, taking a movie using hubbles ccd camera, as hubble very slowly rotates across a field of view, and then using the slightly different positioning and color difference of the ccd pixels to extrapolate the color of the slice that a given ccd pixel would now cover.

don't worry

[Lehk228]

we're taking pretty pictures of the sky not doing brain surgery.

inverse process also useful

[Anonymous Coward]

I've found the inverse process to work well on pictures of my wife ;)

Google Earth & satellite pics

[Circlotron]

I hope the next time the pics used by Google Earth are updated they use something like this, assuming it is applicable.

They've got two things going on at once here

[Anonymous Coward]

You can't tell at all how much good the "lucky" camera is doing, although it is a tried and true
technique. Notice the before pictures in each case are wihtout adaptive optics and without the "lucky"
camera. The "after" images have both and EACH is likely to improve the image quality. I'd bet the adaptive
optics is doing most of it, but it's a pretty shoddy way to present the data.

Space-based telescopes aren't dead yet...

[XNormal]

Even if this technique can eventually produce better pictures at lower cost it is still limited to wavelengths that can penetrate the atmosphere. Some of the most exciting recent discoveries are in infrared (Spitzer) and X-ray (Chandra). The next big telescipe (James Webb Space Telescope) is also for infrared.

Does everything have to include jackassery?

[Quiet_Desperation]

and it's 50,000 times cheaper than Hubble

That's a bit of a cheap shot. Hubble has been in operation for 17 years and has been a vital research tool. The tech for this new technique is, well, NEW.

Re:Does everything have to include jackassery?

[Anonymous Coward]

Also Hubble was new technology at the time and much of the cost was the R&D. A new Hubble would cost a fraction of the original.

No, and this is why.

[edunbar93]

Interstellar gas clouds are pretty static. You would have to take one image every, say, year or maybe 100 years to really get any difference in the image quality. Whereas the earth's atmosphere produces an effect almost exactly the same as if you were to look at the bottom of a swimming pool, and in about the same timeframe.

No, the images we get right now from space telescopes are the best we can get at any given epoch, and that's just the way it is.

i invented the lucky telescope concept in 1995.

[geowiz]

I invented this process in 1995.
here is my original post on
the sci.image.processing newsgroup
my old email address is no longer active.
new one is geopiloot at mindspring.com 9 reduce the numbers of ooo's in pilot to one
it was ironic that many people jumped out to say it wouldn't work at the time.
it does work and it works well. In fact most of the additive image processing now done by amateur astronomers everywhere using pc's software is based on my invention which I did not patent.

George Watson

From: George Watson (71360.2455@CompuServe.com)
Subject: virtual variable geometry telescope
This is the only article in this thread
View: Original Format
Newsgroups: sci.image.processing
Date: 1995/12/11

Has anyone implemented a virtual variable geometry telescope using
only a CCD attached to a normal non variable telescope?

It would work like this:

Take extremely short duration images from the CCD at a frequency
faster than the frequency of atmospheric distortion (1/60 sec I have
read is the minimal needed timeslice for physically corecting
atmospheric distortion in real time so maybe an exposure of 1/120 sec
would be short enough).

Choose via computer a high contrast image as a reference image.

Continue to take rapid short duration images and keep only the high
contrast ones with that have minimal displacement/offset from the
reference image.

Sum each of those acceptable images to a storage that will become the
final image.

What you should end up with is a final image that has minimal
atmosperic based distortion because all the low contrast and non
matching images will have been discarded.

Obviously you build an image over a longer period of time than with
real time optical correction but at perhaps lower cost.

Anyone know whether this has been proposed/done or researched?

--
George Watson

The opinions expressed here are those of the fingers
of George Watson only; not those of George Watson himself.

Please reply via this newsgroup. No Email unless requested,
Thanks.

View this article only
Newsgroups: sci.space.policy
Date: 1995/12/30

Re:i invented the lucky telescope concept in 1995.

[140Mandak262Jamuna]

Looks like you thought about the idea way back in 1995. Would it qualify as invention, if you have not done the actual follow up work, crunch the numbers, did the heavy lifting to show it actually works?

Re:i invented the lucky telescope concept in 1995.

[geowiz]

It is not necessary to build something to invent it. Otherwise inventors would have to be craftsmen. they don't have to be. the patent office got rid of the requirement to build a working model decades ago. There were no heavy numbers to crunch. the only numbers needed were recognizing that if you took shots at a faster frequency than that of atmosphereic distortion and summed only the highest contrast pictures or parts then you would have a photo that had lower atmosperic distortion which i clearly discussed. That was novel at the time. then Astronomical exposures with film were always taken over long periods of time to increase light gathering. the introduction of ccd's meant someone did not have to physical change film between shots and thus the concept of taking short duration shots and summing them was a breakthrough. I knew it would work. It does work. Amateur software programs used since then do this. I can't explain it but like most inventors i have the abuility to visualize something in my head and have it demonstrat all physical processes in my head. i could "see" the light being gathered by the ccd and the high and low contrast images being formed and summed etc. most inventors can do this. ( one inventor test - count the number of corners of a cube. If you figure it out by vizualizing the cube in your head and then physically counting the cubes corners as you rotate it in your mind you probably are a good inventor). At the time the expense of patenting kept me from getting many patents. As a result I spent much time speaking out againt the high cost of patents saying the government should not tax the seeds (the patents) but instead the corn (the products fromt he patents) thus patents should be free and then the government should take its patent fees from licenses of the patents. This encourages the maximum recording and patenting of more ideas which are made available to benefit the world.

Re:i invented the lucky telescope concept in 1995.

[hidave]

The technique works only for a very small area of the sky at any one time since the jitter associated with the atmosphere varries depending on where one looks. Invented in 1995? The idea was around long before that and successfully demonstrated in the early 80's -- yes, in the eighties.

Re:i invented the lucky telescope concept in 1995.

[geowiz]

I looked for prior examples b ut could not find any. i foyu have them please post them. ccd's weren't around then. was it done photographically? thanks

Re:i invented the lucky telescope concept in 1995.

[hidave]

I'm not positive what detector material was used for the specific application, but CCDs have been around since the mid 1970s in military research. Bob Fugate is the real expert in this area. See http://findarticles.com/p/articles/mi_m0RBE/is_200 4_Annual/ai_n8589053

Re:i invented the lucky telescope concept in 1995.

[sploxx]

Hi, I know how it feels to be almost completely ignored (even here on /.) for a too long time when having a good idea. Kudos!

I still think too many people's ideas are lost because people too often want to stay on the main path.

For example, I myself thought (although surely not as the first person) about wireless self-organizing mesh networking (including car networks) a long time ago (must have been the modem days) - before it got popular/mainstream. People thought I was crazy.

i invented lucky imaging in 1995.

[geowiz]

I invented this process in 1995. here is my original post on the sci.image.processing newsgroup my old email address is no longer active. new one is geopiloot at mindspring.com 9 reduce the numbers of ooo's in pilot to one it was ironic that many people jumped out to say it wouldn't work at the time. it does work and it works well. In fact most of the additive image processing now done by amateur astronomers everywhere using pc's software is based on my invention which I did not patent. George Watson From: George Watson (71360.2455@CompuServe.com) Subject: virtual variable geometry telescope This is the only article in this thread View: Original Format Newsgroups: sci.image.processing Date: 1995/12/11 Has anyone implemented a virtual variable geometry telescope using only a CCD attached to a normal non variable telescope? It would work like this: Take extremely short duration images from the CCD at a frequency faster than the frequency of atmospheric distortion (1/60 sec I have read is the minimal needed timeslice for physically corecting atmospheric distortion in real time so maybe an exposure of 1/120 sec would be short enough). Choose via computer a high contrast image as a reference image. Continue to take rapid short duration images and keep only the high contrast ones with that have minimal displacement/offset from the reference image. Sum each of those acceptable images to a storage that will become the final image. What you should end up with is a final image that has minimal atmosperic based distortion because all the low contrast and non matching images will have been discarded. Obviously you build an image over a longer period of time than with real time optical correction but at perhaps lower cost. Anyone know whether this has been proposed/done or researched? -- George Watson The opinions expressed here are those of the fingers of George Watson only; not those of George Watson himself. Please reply via this newsgroup. No Email unless requested, Thanks. View this article only Newsgroups: sci.space.policy Date: 1995/12/30

coherent averaging

[Ixlr8]

The technique resembles coherent averaging: you know there is a still image which is degraded by atmospheric noise. By sampling over and over again and averaging, the noise (only gaussian!) is removed. It goes with the squareroot of the amount of samples used: i.e. for each quadrupling in samples, halve of the noise remains.

Of course there are other sources of error whose contribution also determine your detection limit.

Email from the principle investigator

[tfield98]

I emailed the principle researcher on this project, asking him what was novel about his approach, since amateurs have been "stacking" images for years. Below is his response: From: Craig Mackay [mailto:cdm@ast.cam.ac.uk] Sent: Tuesday, September 04, 2007 5:20 AM Subject: Re: What's new with Lucky? Dear Tom Thank you for your message. What is new about this (and gets rather lost with the media coverage) is being able to use lucky imaging on a much larger telescope. With a 2.5 meter telescope we are able to use typically 10% of the images. With a five meter telescope and four times the area we would be able to use only 0.01% of the images, a completely useless fraction! For the first time however we have managed to do it by using an adaptive optic system in front of our lucky imaging camera. That is what is new and that is what has made all the difference. The AO system gets rid of the larger scale low order turbulent distortions leaving lucky imaging to work on the higher frequency ones which it does rather well. Hence the new image quality which is twice as good in terms of resolution as Hubble, something that has never been achieved before either from space or from ground. If you look on the lucky website you will find a lot of information about amateur lucky imaging for which I have a very high regard. Best wishes Craig Mackay.

Re:Email from the principle investigator

[geowiz]

If that is their breakthrough their press release was poorly written. one would have to wonder also whether the hubble would be better than their ground based technique if a lucky imaging system without the unnneeded adaptive optics system was mounted on the hubble. since being above the atmosphere is better than any adaptive optics system one would think the same low order turbulence would be better eleminated by being on the hubble. from his response it seems they are not using partial image summing. atmosperic distortion happens in commonly occuring sizes since it occures from actual columns of air in motion that have limited sizes. i ownder what radius he is assuming for such column of distorting air if he is making a difference between 2.5 meter and 5 meter scopes.

Reinvention

[maroberts]

http://www.ast.cam.ac.uk/~optics/Lucky_Web_Site/in dex.htm/ refers to a 1978 reference (Freid). It seems that some ideas keep popping up, only the technology actually available to do it has progressed from imaginary to real.

Poor choice of demo image -- M13 much better

[Dr. Zowie]

It's not clear to me why they chose the image they did -- but the imager does much better (and appears to perform as well as the headline claims) in the M13 core -- check out the sample images at "http://www.astro.caltech.edu/~nlaw/lamp_pics/".

Common practice among amaterus for years

[kindbud]

Amateurs have been doing this for years with video cameras and then web cams. Registax is an one of the workhorse programs for automatically selecting frames from a digital video stream for stacking. There's a couple of others but that one's been the most popular for years.

You do it too

[Moraelin]

You do it too. See, Saccades.

Re:You do it too

[jafuser]

I wonder if this may be one of the contributing factors to the uncanny valley effect in 3D animation. One complaint is that the faces, and especially the eyes look "dead". Perhaps the 3D studios should hire some anime artists, who sometimes greatly exaggerate the saccade behavior.

Mod Parent UP

[screen404]

Mod Parent UP

8bit tech?

[sepelester]

http://www.ast.cam.ac.uk/~optics/Lucky_Web_Site/LI _Results.htm
Is that an 8 bit NES i see at the third picture from the top?

I wonder about the uncanny valley

[Moraelin]

You know, by now I wonder if the uncany valley effect actually exists at all. Remember, it's just a hypothesis.

The thing is, if you carefully cherry-pick your examples, and/or are allowed to hand-wave where any given example should fall, you can convincingly argue the uncanny valley effect. But the problem is when you anchor two examples which should, for example be in the valley, yet a third in the middle is not. Although by the shape of it, the third should be there too.

For example, the FF movies were supposed to be in the valley, and EQ2 is easy to argue as being in the valley too. A _lot_ of people's suspension of disbelief was tripped by EQ2. But take Oblivion: the graphics are half-way between EQ2 and the FF movies, yet they don't cause the same reaction. Hmm...

For example, if think it's as unidimensional as in the uncanny valley theory, then just degrading quality on something that's in the valley should eventually move it out on the left bank of it. I.e., it should become pretty. Right? Well, zombies are the classic uncanny valley example, right from the paper. So reducing texture quality and polygon counts on zombies should eventually make them pretty cute, right? I can't think of any game where that was true.

Could it be that some things are repulsive because of cultural associations, and _not_ because of an uncanny valley effect? For example, zombies are disturbing because they remind us of death and have millenia of being a genuine fear for the living, not because they're slightly imperfect humans. When people feared revenants in medieval europe, it wasn't because they looked imperfect, it was a genuine fear of the idea that the dead could come back and hunt the living. Could it be that emotional baggage that's the _real_ cause of our repulsion there? Just a possibility.

Now I don't claim I can disprove the uncanny valley. Just that it looks highly unbelievable to me by now. I actually started as a believer, and genuinely collected examples of why game designers and artists should avoid it. But then it just started to not add up. I ran into more examples as to why it doesn't work the way I think, than into examples supporting that hypothesis.

Mod parent up! VERY informative.

[Eivind Eklund]

See even subject: Email from the principle investigator

Eivind.