Slashdot Mirror
How Spirit Takes Pictures
Some Clown writes "MSNBC has a great article on the details of the camera system on the Mars Rover titled How Sprit makes great photos. Apparently the high resolution images are all done with a 1-megapixel camera. All the money is in the CCD and Lens. The hardcore digital photographers in the crowd will probably find the article to be only a teaser on the technical specs, but the rest of us in the unwashed masses should find it interesting."
Why not link directly to the original article?
The unofficial
It's important to note that in a color digicam each "pixel" only senses 1 color. The NASA cam is a black and white, and to make color they take 3 shots with different filters. This makes it equivalent to a consumer 3MP camera.
They also have a nice lens and a large sensor which helps as well.
Tim
Resolution in cameras (both digital and film) is really determined by optics. By taking pictures of a smaller area and stitching them together, they can probably get better pics than most pros get with their high end Digital SLRs, because they've put more money into the optics than the sensor. Also, the higher density CCDs and CMOS sensors going into digital cameras now tend to be more prone to noise than some of the very high quality, lower density models.
Also, remember that the cameras in the rover had to go through a lot more testing than a typical consumer camera, so it's probably using three, four, or even five year old components in the imaging systems.
It's good to use your head, but not as a battering ram.
Quality is expensive, the survival rate of craft going to Mars is less than 1/3. They tried to cut costs, but that leads to failure. Build them with enough attention that you don't throw $3-400MM away after years of effort...
Discussion of Spirit camera uses LOCO
"Sorry, your browser is not compatible with this feature!"
@%**! MSnbc
Click on the "Interactive feature" if you don't know what I mean,
then curse microsoft,
then go straight to http://marsrovers.jpl.nasa.gov to see the images without paying the microsoft tax. I vowed a long time ago to stop clicking on msnbc links.... sucker that I am to keep coming back for more...
Many believe buying a better camera with greater megapixels etc will make them a better photographer. Sadly mistaken are they.
A great photographer can take an old Brownie and develop some GREAT photos...
Anyone can point and shoot a digital camera...but it really takes someone with talent to get a GOOD image using one.
The greatness of a digital camera is you can snap those 500 shots to get the 3 good ones and not worry about film and developing costs...
Professional wedding photographers shoot 300+ pics per event and rarely get better than 25% that come out with any sort of quality, but people buy them just the same because of 'who' is in the picture.
Anyway just a rant on about people who think the latest and greatest will actually help their choice of shots, lighting, and perception become better...
"Just Smile and Nod." --Huck
Here's a story about some of the software involved.
Brady
I am not very surprised that no journalist understand that, I am more surprised that /. readers missed the point: it is simply nonsense to say that the camera is 1M pixels.
/. readers, please, if you are geeks, always read the small lines. Do not expect NASA or a journalist to do that for you. It is not the interest of the former, and the latter is just stupid.
Indeed, the CCD has 1 million pixels, but look at the published pictures: they are assembled from a great number of small 1 megapixels squares!! Simply have a look at the raw pictures on marsrovers.jpl.nasa.gov: some of them are not fully transmitted yet, consequently parts of the pictures are black.
To make a "normal" picture, like one you would naturally do with your 5M pixels camera, the pancam needs to take shots from, say, 20 different angles. And it is even worse than that: each pictures must be taken 4 times with each filters to get colors. Do not even dream of taking photographs of moving subjects!
There is another drawback: there is two cameras, for stereo. But if you look at the tech specifications on Cornell website, you'll see that each camera has filters that can cover only one half of the color spectrum. Hence, to get color pictures, you have to combine the photographs taken by both the left and right cameras. That's why there is some weird colored patterns on big objects: to put it simply, the left camera sees the red, the right one sees the blue! But both cameras do not see exactly the same thing!
And nonetheless, there was a hint: do you really expect 1M pixels raw pictures to weight 7MB? Huuh?
This is how virtually all consumer digital cameras work (more or less). They paint a pattern of color filters over the CCD. Then they use interpolation, based on the relative intensities, to figure out the most likely color of each pixel.
Different vendors use different masks, and there is a lot of debate about the best approach. See DP Review's Glossary section for more information.
Eloi, Eloi, lema sabachtani?
www.fogbound.net
Yes, the camera is 1 megapixel, but the published images are often made from multiple* shots, sometimes hundreds: for instance the panoramic images.
*No, I am not refering to 3 shots it takes to get red, green and blue data for each pixel.
Apparently, the skies are indeed blue, even though I used to think the Martian atmosphere was to tenuous to filter much light. But when you think about the conception of a muddy colored, reddish sky, that doesn't make much sense (unless a dust storm's happening): once again, the atmoshpere is so thin that it can only filter some colors leaving a bluish tinge for example, but it won't disperse much light (or block out all higher energy light in favor of red).
It's in the interpolation stage that most consumer cameras turn to junk. The fact that the mars rover takes a picture using an identical array (rather than a very-similar-array) with 3 different filters is what makes the image crisp. It's totally impractical in the consumer arena, however, because people would need to stand exactly still while their camera took 3 pictures.
Multi-layered sensors are in the works, however, one of which has been slashdotted. This would provide true image color with no interpolation, but failed to materialize in the year promised (last one).
If anyone has the slashdot link from a few years back, I'm sure it would be relevant to this discussion.
The ______ Agenda
the sensor advetized as a "2mp" sensor typically has a bayer pattern. since the sensor can only record B&W information, they put a patter of RBG filters in front of each sensor lement like so:
RGRGRGRG
GBGBGBGB
so your 2mp sensor is capturing 500k pixels of red tones, 500k of blue tones, and 1M of green tones.then software will interpolate this pattern into a 2mp image with all three colour elements.
I believe sex is highly over rated... unless it involves me
Hope that was useful.
For those not familiar with it, the multiple exposure they talk about in the article has been long used in the darkroom and can be done easily with modern scanners with good software. It brings out extreme details in parts of images that are normally burnt out.
Take a single slide that you scan. With a program like VueScan, you can set the exposure of the scanner, so you can do a dark scan (thus exposing properly the light part of the image), a normal scan and a light scan (exposing the dark part of the image).
Import all 3 into a graphic program, superimpose them and cancel the parts that you don't like (which is the creative part and not as easy as it seems).
Note that you can also do that taking 3 pictures with various exposure with the camera on a tripod, and it's the way the Mars rover does it.
Non-Linux Penguins ?
huh? To get stereo, you need separation. You could do this with a couple of prisms, or maybe the biggest fresnel lens you ever saw, but why not just move the camera sideways between shots?
If your camera can take several pix in a row, use that, and simply move the camera laterally during the shooting (assumes you have fast shutter time).
Lastly, no. As I understand it, a CMOS sensor cut into 1000x1000 pixels will give you "better" pixels than the same die at 2000x2000 pixels, and coupling the pixels 2x2. This has several causes:
1) you can only average your combined pixels after sampling: thus you get quantisation noise (and hypothetically phase interferernce, although I've never heard anyone comment on this)
2) if you couple 2x2 pixels, you will get 1xR + 2xG + 1B pixels. Most pixels will be predominantly one of these colors, removing the other 3(2 for G) from the picture. This also means that a blue photon heading towards the 2x2 metapixel must hit the 1/4 area which can see it, else it is lost.
3) (I don't quite get this one. As close as I understand it:) The size of the sensor feature size is coming close to the wavelength of light: Sony's new 8mp sensor is 0.008 m long, with 3000 pixels. That makes each pixel 2.6e-6 m. Compare with Red light, at a wavelength of 0.7e-7 m. Each sensor is three wavelengths wide(!). This apparently means that you can't usefully use an fstop higher than 11ish on the new sony f828. Search photo.net for a technical discussion.
4) I guess that we also get effects from the fact that each pixel sensor is basically in a well, and the smaller the pixel becomes, the harder it becomes for a photon to hit the sensor, rather than the well wall. I never hear this discussed either
I'm sure there is some reasoning behing NASA's decision but that article doesn't say what it is!
But the funny thing is that NASA don't even have decent software for tiling those images so they have seams everywhere (and I don't just mean from the color variance).
Doesn't it make you feel good to know that our freedoms are protected by politicans, lawyers and journalists.
That would be the correct usage of the mega prefix. If you check the definition of SI prefixes (which mega is one of) you will find they are all defined in terms of base 10, and no other base. Kilo is defined as 10^3, mega as 10^6 and so on. Thus a mega-anything is 1,000,000 of that thing by definition.
However, computer people hijacked the prefixes and started using them incorrectly. Since computers are base 2, base 10 numbers don't divide down nicely. 1,000,000 isn't remotely near a nice round number in base 2. So they took the SI prefixes and used them to indicate base 2 numbers. Kilo was used to mean 2^10, mega 2^20 and so on.
Well this is an incorrect usage, and one seen ONLY in computers. Everything else, it is base 10. If I say I have a kilogram of something I mean 1000 grams. Likewise with calories, metres, whatever.
Actually its significantly better than a 3MP consumer. A consumer sensor uses a bayer pattern which leads to a significant loss of sharpness in the final image, since the color sensors are in physically different locations (close but not close enough). The Foveon sensor addresses this problem directly. In consumer terms, its probably 5 MP equiv.
Using color filters for high quality digital color photography is an old technology (and even older for analog). Its obvious limitation is that the subject has to be still.
Bigger pixels at lower resolution are not necessarily a good tradeoff: you can do almost as well in terms of noise and sensitivity by using more smaller sensors and performing the averaging in software.
Compositing lots of low resolution images into a single high resolution image is also completely standard: you can get both free and commercial software to do it.
Altogether, I suspect that if you take something like the new Sony 8Mpixel camera and take raw pictures with it, and reduce it to 1024x768 using good software, you are going to be pretty close to the measured quality and sensitivity of Spirit's sensor (in practice, you'll see little or no difference under normal circumstances, however). Then, you can use compositing software to composit multiple images for panoramas.
The Spirit tradeoffs make sense for a Mars rover, also taking into account power and weight requirements, but they do not result in a level of picture quality that you couldn't achieve with the digital cameras you can buy at the local store.
It varies depending on whether it's transmitting to Earth or to Mars orbit and possibly varies by some other factors, but it's slower than a cheap DSL line and often slower than a 56k modem. I think the highest number I ever saw was around 152 kbits/second.
Tim
Omnia vestra castrorum habetur nobis.
Got this as an email the other day on my University of Minnesota account...it talks about the compression used. Haven't read up on it, though.
"The Mars Exploration Rover (MER) is currently landing a pair of rovers on Mars (one landed last week and the other will be landing soon).
Well over half of the bits transmitted from the rovers will consist of compressed image data gathered from the unprecedented nine cameras
on-board each rover. This compression is based on the ICER and the LOCO [1] image compression technologies. LOCO was developed by Dr. Marcelo Weinberger and Dr. Gadiel Seroussi from Hewlett-Packard Laboratories and Prof. Sapiro from the Electrical and Computer Engineering Department (while he was at the HP Labs before joining the University).
The JPL/NASA hardware implementation of LOCO on-board the rovers is used when maximum geometric and radiometric fidelity is required. The LOCO technology, patented by Sapiro, Seroussi, and Weinberger at Hewlett-Packard Laboratories, is also the core of the international standard JPEG-LS for
the lossless and near-lossless compression of still images."
[1] M. J. Weinberger, G, Seroussi, and G. Sapiro, ``The LOCO-I lossless image compression algorithm: Principles and standardization into JPEG-LS,'' IEEE Trans. Image Processing 9, pp. 1309-1324, 2000.)