Elon Musk: Faulty Strut May Have Led To Falcon 9 Launch Failure

garyisabusyguy writes: This Forbes article provides the best analysis of the loss of the last Falcon 9 mission based on information released by Elon Musk to reporters. Highlights include:

• 1. Sound triangulation led them to identify a strut holding helium tank as root cause where the falling helium tank pinched a line causing overpressure in the LOX tank.
• 2. The failure occurred at 2,000 pounds of force, and the struts were rated at 10,000 pounds of force. They initially dismissed this as a cause until sounds triangulation pointed back to the strut
• 3. Further testing of struts in stock found one that failed at 2,000 pounds of force, with further analysis identifying poor grain structure in the metal, which caused weakness
• 4. It will be months before the next launch while SpaceX goes over procurement and QA processes all struts and bolts, and re-assesses any "near misses" with Air Force and NASA
• 5. Next launch will include failure mode software, which will allow recovery of the Dragon module during loss of the launch vehicle since they determined that it could have saved the Dragon module in this lost mission

.

Comments

Transparency

[The Raven]

I am amused by the fact that a private company does better than our government at disaster transparency. That said, it is pretty stupid that Space X has not been testing random parts to confirm they meet the requested specifications. Spec verification is a basic part of outsourcing. All outsourcing fails if you can't verify that you're getting what they promised you.

Re:Transparency

[bobbied]

You can test random struts, but you can't test ALL struts or you're left with no struts. Sounds like they didn't test the right ones is all.

Quite true, but there are OTHER ways of figuring out the strength of the part other than testing them to failure or causing damage to the parts. Some of these methods are quite expensive, but effective. How you get a one part that's 1/4th the strength the design requires through a manufacturing process, onto a flight ready hardware assembly and not know it says serious problems lurk within. Seems Space-X has some defects in it's quality assurance processes, and that should scare you more than this one launch failure. They are playing way too fast and loose with quality and I'd be very worried about their efforts towards getting "human" rated are not going well.

This was a structural failure. A failure that can likely be traced to a part that was too weak for the designed loads. This isn't a DESIGN failure, it's a QUALITY failure, and that puts the whole program into question. What has to happen now is that the whole QA process needs to be revisited and revamped to prevent structural components from sneaking though which are not strong enough to do what they are designed to. THEN you have to go though your whole stock of parts, sub assemblies, and flight ready assemblies and figure out what you can verify as trustworthy using your NEW QA process, throw out the rest and order, assemble and test replacements.

This isn't an easy or quick fix...

Then there is the whole, our supplier lied to us, approach, which will be quicker to deal with, but only because you just have to obtain a batch of replacement struts, fully tested and verified, replace all existing hardware that used the old ones by either reworking the assemblies or building new ones that have the new parts.....

I'm guessing Space-X will opt for the latter in public, but unless they sue the supplier for damages, the problem really is the former, which scares me..

Re:Transparency

[bobbied]

That said, it is pretty stupid that Space X has not been testing random parts

So uh... "Further testing of struts in stock found one that failed at 2,000 pounds of force"

Sounds to me like they likely were testing them at random and then decided to start testing a vastly significant number of them to troubleshoot.

Which tells me they may have a QA process issue that compounds a design issue. NEVER do you take a single failure risk in a human rated system's structural components unless you are SURE the components are sound. So if this strut was a critical structural component without any backup and you are not totally sure it can take the load before you fly the assembly, it's a process problem.

They do want to make this a human rated system right? That goal is in serious jeopardy now. You cannot afford to "learn on the fly" like this and arrive at a human rated design. You are going to build and launch a LOT of vehicles, fixing the problems as they crop up, before the statistics will be good enough. The cheaper way is to design sufficient safety margins into the system, do a few launches and deal with the issues as they arise, but unless you REALLY mess up, any system you hope to human rate is not going to have this many loss of vehicle failures... Structural failures should NEVER happen if you are on this track, because structure should be the easiest thing to get right because it's testable on the ground...

IMHO Space X is doing this wrong, and a structural failure is a serious problem that goes beyond this one failure....

Re:Transparency

[Kjella]

Statistics is your friend in this case. Random testing should show a large standard deviation (assuming they test to failure). You should then be able to calculate the probability of failure.

No, just no. Statistics helps if you have a process variation that approximately follows a normal distribution. It does nothing to protect you against freak failure, like say you're making creme brulee using a torch to caramelize the sugar on top but the spark to light it is only 99.9% reliable. That means 999 out of 1000 desserts will be fine and one will be a total failure, but you don't know it until it happens. Failing at 1/5th the design load is clearly outside any normal variation and tells you none of these struts can be relied on no matter what their average or standard deviation is.

Re:Holy Jebus

[Rei]

Elon is surely really fuming about this one, as I know from past interviews with him that he really doesn't like having to source hardware from outside suppliers. He has the old "robber baron" mindset of wanting to get the whole production chain start-to-finish in house, and it's one of the things that really frustrated him when he started Tesla: at the time of the last interview I read on the subject (something like 3 or 4 years ago), he had gotten SpaceX up to 80% in-house, but Tesla was only up to 20% in-house. Car manufacture has long been all about sourcing parts from a wide range of outside suppliers.

But even at 80% in-house at SpaceX, looks like that remaining 20% still bit them : Seriously, failing at 1/5th the rated failure value? The vendor might as well have given them a cardboard cutout with the word "strut" written on it in sharpie.

Re:$10,000 toilet seats

[Moof123]

Yep. Once you have thousands of parts all designed to trade off as much strength for weight reduction as possible it doesn't take much of a manufacturing hiccup to cause an expensive "excursion". Vendors end up having to rigorously test every widget, and custom design it just for you.

Before long $10k each for a batch of a half dozen toilets seat that are space rated to not outgas funny chemicals that foul optics, handles 10g's, has 6 sigma of de-rating for the bolt hole strength, weighs under 500g, and is non-flammable starts sounding like a deal.