Slashdot Mirror
Why Hubble Broke and How It Was Fixed
angry tapir writes "I recently had the opportunity to sit down with Charles (Charlie) Pellerin, who was NASA's director of astrophysics when the Hubble Space Telescope launched with its seemingly fatally flawed optical system. Pellerin went on to head up the servicing mission that finally fixed the telescope and for that was awarded NASA's highest honor, a Distinguished Service Medal. Since Hubble he has done a lot of thinking about the problems that led up to the error and how organizations can best avoid making similar mistakes."
The article mentions that the contractor was afraid to bring up problems.
That, plus the mentality from management that people who bring up problems are "troublemakers," "negative," "not team players," etc. (because they've put too much of their ego or political capital into a project) has got to be responsible for more disasters, large and small, than any other deadly combination.
I worked for a large nonprofit that blew money on doomed projects as though money grew on trees. Each time, it started with somebody, usually a contractor or somebody else who stood to gain from it, flattering the leadership that this was huge and visionary and would make or save them millions. Then the organizational mind control started, where everybody was saying that it was the greatest thing ever. Then the flawed project management started. Then when the cracks were obvious, people who pointed them out were vilified as naysayers. It was only the lower-downs who said anything because to rise, one had to be a "team player," and the organization was hierarchical enough that lower-downs were ignored. Then denial that there were problems, together with tossing more money at it (including adding more people to a software project at the last minute because that always works). Then even when the leadership [sic] team [sic] all realized there were problems, they all waited until the person responsible for the project was willing to concede defeat. because in a political environment, nobody wants to confront somebody who might retaliate
Those elements are the inevitable recipe for disaster for any project, but it's fear that drives virtually all of them. Fear of not looking good (note that the Congresscritter didn't yell about wasting taxpayer money, she yelled about being made to look bad), loss aversion, fear of admitting a mistake, fear of speaking up.
Pellerin was brave enough to do something technically illegal and scrape up the funds for servicing it.
That is what a leader does.
It's not only the $50k for the test. Most likely there will millions in cuts and this test happened to be in the mix. It would be nice if you only had to pay for the tests that showed problems. It would make engineering much easier. Unfortunately you have to test for everything even the stuff that works fine.
I love Jesus, except for his foreign policy.
Your anecdotal story is intersting, but it fits right into what he was talking about with the Management failures at NASA. Clearly it wasn't the lack of that test that caused the problem. It was a management decision to not perform it. Probably under the exact same pressures. Even if it had been performed though, who's to say they wouldn't have rationalized away the results like they did all the other failed tests?
"We tested that mirror over and over and over with a different kind of device, the old style refractive null corrector," Pellerin says. The results? "Half wave of error, half wave of error, half wave of error." "So some people sat down and said, 'What's going on?" Pellerin recalls. "The mindset was that the mirror can not be other than perfect. So something else is happening. They concluded that the mirror was sagging under the force of gravity in the three point mount rather than being on the bed of nails by half a wave. "Well it turned out that was wrong. But they rationalised, rationalised, rationalised.
...
The project had suffered other challenges beyond fabricating and mounting the mirror; staff were being "hammered" all the time, Pellerin says. In addition there was constant angst about how far the project had gone over budget. "Hubble's initial budget was $434 million we closed it at $1.8 billion just for the flight segment; big overruns." "So the way it works is you tend to blame the people doing the work," Pellerin says. "So we're hammering on them, hammering on them so they had no free time or no inclination to track down anything that wasn't a critical problem because we have other critical problems. Difficult technical things that we couldn't solve yet." The review board also found that a hostile environment had been created for the contactor, which meant "they told us about any problem at their peril," Pellerin says.
You should read the article, because your comment is exactly the kind of thing he is talking about. Technical people who think they have found a technical problem, therefore the solution is to correct that problem. If the problem was really that measurement system the US uses is 'wrong', then how can the US have so many successful space missions? The problem is not that there are multiple measurement systems, or that one is somehow superior to the other. The problem is that the teams did not communicate successfully - not a technical problem at all. And don't say 'well, if the stupid US would use the same system as the rest of the world it wouldn't be a problem', because that just shows you completely missed the point. The point is that there was ineffective communication - a leadership problem - not simply a technical problem.