Slashdot Mirror
Why Hubble Broke and How It Was Fixed
angry tapir writes "I recently had the opportunity to sit down with Charles (Charlie) Pellerin, who was NASA's director of astrophysics when the Hubble Space Telescope launched with its seemingly fatally flawed optical system. Pellerin went on to head up the servicing mission that finally fixed the telescope and for that was awarded NASA's highest honor, a Distinguished Service Medal. Since Hubble he has done a lot of thinking about the problems that led up to the error and how organizations can best avoid making similar mistakes."
The real hero of that project was a man called Story Musgrave. http://en.wikipedia.org/wiki/Story_Musgrave There was a lot of planning put into fixing it, but without him actually up there in space improvising when stuff went south, the Hubble would be useless today.
I would also like to know the juicy details behind the Mars Climate Orbiter not orbiting, but amm.. slamming into Mars.
I worked at Ball Aerospace years ago and found out the real story. NASA cut the budget for Hubble so that a final optical train alignment task was never done. The engineers had designed a laser test to check the optical path but NASA wanted to save the $50000 the test would take. So until it was turned on, in space, they had no clue how bad it was. Working with NASA was tough mostly due to their arrogance.
The article mentions that the contractor was afraid to bring up problems.
That, plus the mentality from management that people who bring up problems are "troublemakers," "negative," "not team players," etc. (because they've put too much of their ego or political capital into a project) has got to be responsible for more disasters, large and small, than any other deadly combination.
I worked for a large nonprofit that blew money on doomed projects as though money grew on trees. Each time, it started with somebody, usually a contractor or somebody else who stood to gain from it, flattering the leadership that this was huge and visionary and would make or save them millions. Then the organizational mind control started, where everybody was saying that it was the greatest thing ever. Then the flawed project management started. Then when the cracks were obvious, people who pointed them out were vilified as naysayers. It was only the lower-downs who said anything because to rise, one had to be a "team player," and the organization was hierarchical enough that lower-downs were ignored. Then denial that there were problems, together with tossing more money at it (including adding more people to a software project at the last minute because that always works). Then even when the leadership [sic] team [sic] all realized there were problems, they all waited until the person responsible for the project was willing to concede defeat. because in a political environment, nobody wants to confront somebody who might retaliate
Those elements are the inevitable recipe for disaster for any project, but it's fear that drives virtually all of them. Fear of not looking good (note that the Congresscritter didn't yell about wasting taxpayer money, she yelled about being made to look bad), loss aversion, fear of admitting a mistake, fear of speaking up.
Pellerin was brave enough to do something technically illegal and scrape up the funds for servicing it.
That is what a leader does.
imperial vs metric!
Sent as ripples into the electromagnetic field. No single photon has been harmed in the process.
In order for the lens snafu to have been a fatal flaw, it would have had to have rendered the Hubble inoperable -- or at least incapable of doing science that couldn't be done with existing telescopes. By for the years that Hubble flew before the fix, it returned pictures that were hundreds of times better than anything previously seen. Within days of its activation, we were seeing so far, and so clearly, that our understanding of the nature of the universe was being rewritten. We were seeing things beyond what we could have imagined, and the days of modern space porn began. Astronomy was catapulted into the future, and astronomers lined up to permanently pack the schedule of the most capable telescope ever made.
Yeah, that sure sounds like a "fatal" flaw to me.
Comment removed based on user account deletion
I don't remember the last time I so thoroughly enjoyed and appreciated an article. To the original poster, and the /. editors that approved this - thanks. This one was a winner.
Great post. Another magnificent management failure that comes to mind is the launch of the 787 Dreamliner. I was close to that project in 2008 - subcontractors were getting drawings of structural parts 7 days before initial quantities were due to ship, and Boeing was still saying "right on time!" And that wasn't the half of it.
I enjoyed that thoroughly. Just like Mythical Man-Month is required reading in virtually all CS programs, something like this needs to be adapted and given as required reading, too - with more anectdotal details of the failure and solutions and costs, and with more pragmatic approaches to avoiding pitfalls, of course. Really good find for us. Thanks /. editors.
What's the name of the psychopathic congresswoman who sprayed Pellerin with spittle? She needs to have her name dragged through the mud at every opportunity.
Contribute to civilization: ari.aynrand.org/donate
Totally enjoyed the article. 1) I believe this type of mismanagement is a systemic flaw which occurs today despite all the Challenger / Hubble lessons learned. This seems to happen when any project is driven by unspeakable layers of management and the top layers have goals and metrics far different than the bottom layers 2) Was the Apollo project which put man on the moon any different? How did we get all these people to put a man on the moon while today we cannot even get a rocket off the ground?
This is a classic! It shows that when doing exacting science/engineering one should NEVER hurry the staff! Things will get done when they are done, otherwise someone will apply a field-expedient solution to a problem, with the usual "humorous" results...
I think the article was in some ways flawed. It gave a good description of how the error occurred. Then it moved on to a huge tirade against the focus on "individual abilities" which it blames for the whole error. Firstly, even taking the description of how the error occurred at face value, it is not at all clear that the error had anything to do with a focus on "individual abilities". On the contrary, it seems this was just an instance of really poor management that - due to cost overruns - pushed their employees to work harder, to the point that they lost their focus on quality and maybe even started cutting corners in the fabrication process. This has absolutely nothing to do with a focus on "individual abilities". However, let me address the "anti-individual abilities agenda" anyway.
The anti-individual abilities agenda is routinely promoted by managers, project managers and other people engaged in the management layers (management consultants, business schools etc.). The motive is pretty clear: Many bosses don't like admitting that the success of their project comes down to individual abilities of a few core members on the project. After all, what is the value of management then, they ask? It's like the tail wagging the dog.
However, this is just denying reality. I can firmly say that on any project of major size I worked on, the was a few 5-10% of people on the project running the show. This in itself is not very surprising, what is surprising is the fact that these 5-10% were not centered at the top of the pyramid. Rather, it was evenly spread out over all 'layers' from 'highest to lowest'. These people (by virtue of their skills and dedication to the project, something that is often lacking with the project management itself!) automatically assume a role of authorities whether management likes it or not. It's simply the only way to get things done. Let's face it, on any project there's going to be a lot of 9-5'ers that don't really care. They are never the ones driving the car, nor should they. It's the 5-10% who has both the ability to and the interest in getting the job done that counts. Those that dream about the project at night and who feel their personal honour is at stake in making it succeed. Also, as Fred Brooks noted in 'a mythical man month', some (sub)projects are like surgery. You need one highly skilled person to be in charge and carry out the job, and the rest of the team members are really just accessories of that person. Their contribution can be important of course, but at the end of the day, all choices, responsibility resides with the 'surgeon' etc.
I think the lesson to be learned from these observations is that management needs to accept that this is the structure that projects will generally fall into, no matter what they do. The job of management is to get the best result out of it. On projects with poor management that creates obstacles for progress and makes lots of bad choices (this often happens on politically infested projects as well as on projects where management doesn't have a clue about the technical aspects), often the project finds a way to completely bypass management. Decisions by management may be outright ignored, or important decisions are never brought up to this level but are just made behind the scenes. This is a very dangerous situation since important decisions may not be properly reviewed and may not even be known by all stake-holders. While most decisions taken may have been correct, it takes just one bad decision to jeopardize the project, and problems related to this kind of "skunkworks decisions" tend to surface very late where they may cause huge problems, sometimes disasters.
The job of management is to embrace the individual abilities, and to listen carefully (but of course not uncritically) to arguments brought forward, no matter if it is from a project manager or a "lowly" techie. They need to make a decent effort to try to understand what they are talking about, even if the explanations are not always clear and even if it can sometimes be highly technical.
This article reminds me higher ups talk about what really happened on major programs, as compared to our perception while it was active. i.e. Space Shuttle development as perceived by most (laymen and enthusiasts) during 1970s/80s/90s as compared to later mentions i.e. http://waynehale.wordpress.com/ and http://ocw.mit.edu/courses/aeronautics-and-astronautics/16-885j-aircraft-systems-engineering-fall-2005/
Another example is Sputnik, how did USSR succeed when USA failed? There are two answers from the experts. One is what they said in 1957, and the other is what they said in 2007. Both are very different.
So....... I cannot wait to hear what "they" really gonna say what happened and why regarding VSE/Constellation/SLS/. But damn have to wait 20 years to find out the real scoop.
mfwright@batnet.com
The REAL real hero would be the advisor against naming your child Story.
"Love heals scars love left." -- Henry Rollins