The Forgotten Huygens Experiment

jdray writes "An experiment onboard the Huygens probe didn't run as planned because someone forgot to turn it on. The team lead for the experiment has put eighteen years of his life into the project, just to watch it not happen after a seven year ride to its destination on Titan."

Comments

Redundancy...

[Burb]

I understand that half the camera pictures were lost because they were transmitted on channel A. Interestingly enough, an article in New Scientist quoted one of the mission planners as being scathing about the scientists' choice to use the 2 channels for increased bandwidth...

This post is from memory. Please feel free to correct errors and ridicule me for factual inconsistencies.

Shit happens.

[KiloByte]

... especially in this field of work. If you have a project this big, the chance that nothing will go wrong are simply infinitessimal. Do you remember the last time when you wrote a program of 100 lines without doing a single error?

We should really praise the gods that the rest of Huygens mission was a grand success.

Re:Shit happens.

[grozzie2]

Do you remember the last time when you wrote a program of 100 lines without doing a single error?

I may not have got it all right on the first go around, but you can rest assured, i got it right after the testing and before it was deployed...

In my primary field of work, 'shit happens' is just not an acceptable excuse, I'm a pilot. We use checklists precisely for that reason, to make sure that shit doesn't happen. Every flight has a few phases where even one minor screw up can have serious consequences, so we have checks and balances built into the system to make sure that small screw up does NOT happen.

I know the software folks here on /. always want to make excuses about 'its hard' and 'its to complicated', but, it's actually not hard, and not to complicated. complex systems are designed and built every day in the aerospace field, systems that many lives depend on. We take it for granted that they are properly designed with failsafe modes, they can deal with problems on the fly, and they do not puke up and die when things become abnormal. Same goes for our crews, they train extensively to make sure they fully understand all operational modes, and they can deal with them. Once that's all done, we write books full of checklists, to make sure the details do not get missed at a critical time.

'I forgot' or 'shit happens' is just not an excuse. In reality, it's an admission of unprofessional conduct. Billions of euros spent, many many man years of effort, and you want to take 'forgot' or 'shit happens' as an acceptable excuse? there is no acceptable excuse, those are just admissions of shoddy management and operations. Those are terms that are not even in the vocabulary of true professionals.

Every time I read here on /. about how 'professional' programmers seem to think that it's to hard to actually take the time and effort to write failsafe code, and test it as such, I ask myself how many people would die if thier attitudes were used developing the flight management systems in our aircraft.

Thanks to government regulations, i can only fly 9 days a month, that leaves me with a lot of time to operate my other business. We do software development, embedded systems for mission critical applications. We do deploy equipment into life critical situations, so, for our work, 'shit happens' and 'i forgot' just dont exist in the vocabulary. We use checklists to ensure that all testing covers all forseeable abnormal conditions, up to and including partial failure of various hardware. for your typical 'desktop' developer, equivalent testing would be along the lines of making sure programs handle gracefully things like having the hard drive removed from it's computer while the program is still running. They may not function at full capacity anymore, but it's not reason enough to have the thing just puke up and crash, it needs to fall into a failsafe mode that's prepared to deal with the detail of 'no local storage available anymore'. the code to handle this scenario will likely not 'get it right' on the first try, but, it'll surely be right before the product goes into release.

Looking at the money spent, and the multitude of man years spent on developing the lander for this mission, to hear that a significant experiment was lost becase somebody forgot to turn it on, is just beyond comprehension. this goes way beyond unprofessional, and well past the line we would draw for 'incompetent'.

Re:Shit happens.

[JaredOfEuropa]

I know the software folks here on /. always want to make excuses about 'its hard' and 'its to complicated', but, it's actually not hard, and not to complicated.
[...]
We do deploy equipment into life critical situations, so, for our work, 'shit happens' and 'i forgot' just dont exist in the vocabulary. We use checklists to ensure that all testing covers all forseeable abnormal conditions, up to and including partial failure of various hardware.

You're right... up to a point. The amount of robust coding, testing, and many other things like security, are always subject to a balance of costs and benefits. Rigorous testing is expensive, and in many software applications it might be wise to, say, not do a complete regression test on a minor release since the cost of that test outweighs the risk of a bug slipping through.

In your field of business, I imagine you cannot easily deploy quick fixes (to embedded systems), and major bugs in life critical situations are obviously not acceptable. So you do rigorous tests and code reviews. In my line of business however, bugs are acceptable. Sometimes a bug makes it into production... users will moan, and we'll have to spend a bit extra on writing and deploying the fix, but the cost is lower than doing a full test on every release.

I agree with you that software developers should realise the importance of testing, and take a critical look at their own testing and coding procedures... often it isn't that hard or expensive to make real improvements.

Re:Shit happens.

[Oxygen99]

Well. Precisely. Coding is hard, but not any more so than designing a building, an aircraft or an automobile. However, neither is it any less hard, so why is software engineering not accorded anything like as much respect as other disciplines? Do you see Airbus outsourcing airtcraft designs to the far east to save a few Euro's? No. Yet for some reason management always believes software can be written cheaper and quicker.

Admittedly lives don't depend on 90% of the software any of us here writes, but that isn't to say it isn't complex or demanding and requires complex, demanding testing to ensure high standards of reliability.

If those resources aren't allocated, then I'm afraid 'Shit Happens' is very definitely an excuse.

Re:Shit happens.

[Doomdark]

It's never about costs. Mistakes ALWAYS cost more than thorough testing.

Well, that's kind of nitpicking. Although "time is money" is just a slogan, it does point to the fact that both timeline and money are constraints that affect test coverage that can be done. And cost/benefit analysis should be done for testing as well as for implementation: proper amount of testing to do is a compromise based on many things (type of system, expertise of implementers, aggressiveness of implementation/release schedule etc. etc.). So I would argue that it's ALWAYS about cost, in broad sense (delaying a release costs money -- that's the main reason to avoid delays).

And finally, there are cases where defects just are cheaper to have, than doing rigorours testing. Like everything in software engineering, impact of defects is relative; there are no absolute guidelines.

What about the grad students?

[Jonathan]

I assume (like practically all scientific projects) grad students were involved in the design. While the failure to turn on the experiment may be an embarrassment to the primary investigator, how does it affect the grad students? Do they just leave the "results" section of their dissertations blank? Do they need to restart their graduate research with another project?

Re:What about the grad students?

[imsabbel]

This probe lauchned years ago. Every grad student involved in building/designing would long now have his PhD...

Re:Shame they were only black and white.

[Speare]

Black and white sensors have higher resolution, just as black and white film has higher resolution. Resolution is more than the number of pixels, it's the valuable ability to resolve actual data with those photosensors.

Your little consumer digicam that did not cost a hundred thousand dollars is arranged with cheap little colored filters, cutting out over half of the photons that arrive in the camera, just so you can get the right shade of pink on your girlfriend's tummy. Scientists would rather collect all the photons they can, thanks.

Scientists do use filters now and then. Spirit and Opportunity use black and white cameras, but they can use something like NINE different filters to block out all frequencies except certain bands of interest. They don't just select Red, Green, Blue, but also various bands of near and far Infrared and Ultraviolet too. Those probes were designed later, and were going to be used on a longer mission, where power and available light energy would be greater. Huygens was built earlier, and going to a distant and dark moon where they'd be lucky if the probe lasted a couple of hours.

Is their logic still a mystery to you?