Slashdot Mirror
Elon Musk: Faulty Strut May Have Led To Falcon 9 Launch Failure
garyisabusyguy writes: This Forbes article provides the best analysis of the loss of the last Falcon 9 mission based on information released by Elon Musk to reporters. Highlights include:
- 1. Sound triangulation led them to identify a strut holding helium tank as root cause where the falling helium tank pinched a line causing overpressure in the LOX tank.
- 2. The failure occurred at 2,000 pounds of force, and the struts were rated at 10,000 pounds of force. They initially dismissed this as a cause until sounds triangulation pointed back to the strut
- 3. Further testing of struts in stock found one that failed at 2,000 pounds of force, with further analysis identifying poor grain structure in the metal, which caused weakness
- 4. It will be months before the next launch while SpaceX goes over procurement and QA processes all struts and bolts, and re-assesses any "near misses" with Air Force and NASA
- 5. Next launch will include failure mode software, which will allow recovery of the Dragon module during loss of the launch vehicle since they determined that it could have saved the Dragon module in this lost mission
.
Now THAT is how you summarize.
See, you need unit tests AND integration tests AND end-to-end tests. Some testing at every level is better than exceptional much focus at just one level.
The supplier who gave them bad or subgrade steel should be, they're about to get reamed...
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
So after weeks of investigation it turns out it's a failure mode that even the most amateur of KSP players recognize.
They should have added a lot more of them, clearly. It's not like struts have any mass.
"They love to watch her strut!"
the heilum bottle would have shot to the top of the tank at high speed
That sounds a lot different than "a hose may have been pinched" Has anyone been able to find audio of the actual conversation?
Sell them faulty metal.
Or faulty parts made of metal.
It's just a thought, but would a competitor stoop to that? Even if not now, at some point in the future?
I'll see your senator, and I'll raise you two judges.
I am amused by the fact that a private company does better than our government at disaster transparency. That said, it is pretty stupid that Space X has not been testing random parts to confirm they meet the requested specifications. Spec verification is a basic part of outsourcing. All outsourcing fails if you can't verify that you're getting what they promised you.
"I will trust Google to 'do no evil' until the founders no longer run it." Hello Alphabet.
When your making something that has to withstand x units of load, it has to be certified of a certain grade of material, known to give the required strength + safety factor in design or you are guessing and trusting to luck.
SpaceX should be buying certified raw stock to process into parts, if someone sold them stock with forged certs, that someone is in bad doodoo, if they knowingly bought stock without checking the certifications themselves, they are as much to blame as the supplier.
Part of me doesn't want to think this is a result of building down to a lower price than traditional space exploration costs. There is a element besides pork making some things seem insanely expensive, and maintaining a genuine certified material stock process is one of those seemingly un-needed expenses...
Stop using monospace, you're not a special snowflake.
Everybody who sees posts in monospace just skips over them.
THIS is why aerospace and aeronatical parts cost so dang much.
By manually numbering each item of <ul> instead of simply using <ol>?
In Soviet Washington the swamp drains you.
Pilot wouldn't have needed to. Dragon 2 has automatic abort capabilities (even when unmanned). It would have separated from the second stage - probably firing its SuperDraco thrusters - and then automatically deployed parachutes once it was a safe distance away.
Dragon 1 doesn't have the SuperDracos (only the much smaller Draco attitude control thrusters) so it wouldn't have been able to put as much distance between itself and the booster, but from the video and the telemetry it looks like the capsule survived the (accidental) separation anyhow. It could have deployed its parachutes and probably survived the landing, but it wasn't programmed to do so. They have added it to the Dragon 1 programming now though.
Failures that occur high enough to land under parachutes, slow enough to get away from the inevitable explosion without heavy rockets, and early enough in flight that there's no time to manually enable the landing sequence are... really, really rare in rocketry. Usually you either fail at liftoff (see Orbital's last attempt to launch Antares), fail rapidly and catastrophically during liftoff (any number of examples), or fail once in orbit (often, though not always, at stage separation). In orbit you have time to make a decision and send orders. On the launchpad you can't land safely (without abort rockets). In midair you *usually* can't get away in time (without abort rockets). This was an exception to the "in midair" usual failure case; there were nine seconds from beginning of the failure to loss of vehicle, and in fact the capsule had already tumbled free (and probably *could* have used its ACS thrusters to put some extra distance between itself and the booster.
One thought, though: what about, in the case of a pre-separation second-stage failure, executing MECO 1 (Main Engine Cut Off, when the Falcon 9 first stage kills its rockets) early and doing an emergency stage separation? Normally there's no point - the first stage on most launch vehicles has no purpose if the launch fails and nowhere to go even if it separates safely - but the Falcon 9 first stage is designed for reusability. Emergency MECO, separate the stages, use the ACS and/or grid fins to steer clear of the second stage, and then fire up the main engines again and aim for the droneship or other landing pad. You'd need to be quick about it, and it might still not work, but if it does you've saved a booster worth $70,000,000 USD. Well, that and demonstrated the first successful first stage recovery ever, but assuming that becomes as routine as Musk wants it to be...
Actually, it would have been super cool if the first successful recovery of the first stage had been an emergency abort!
There's no place I could be, since I've found Serenity...
Shut the fuck up, nameless twit
The sound triangulated was in cryogenic liquid oxygen at 50 PSI. The speed of sound in that is approximately 1 kilometer per second.This paper is about calculating the exact speed. Elon talked in the conference about reading telemetry with millisecond accuracy. But this would yield only 1 meter resolution.
Bruce Perens.
Trolololol lol lol lol lollll
way to go.
Honesty may be the best policy, but by process of elimination, dishonesty is the second best policy.
Alternatively: Control your browser. You *do* have control over your browser, right?
"So long and thanks for all the fish."
How does that help the monospace-fetishist improve their readership? If someone points out to you that your hat is on fire, do you thank them for helping you or snipe back that they should stand further away from you if they don't want to get burned?
It helps the reader not have to worry about such rather than expecting others to conform to their behavior. Control your inputs and outputs. This is not a difficult concept. There will always be some moron who wants to post in monospace format. You can not change them. You can change you.
"So long and thanks for all the fish."
And as an engineer, I know that most engineers want to act in good faith. Some are inept or inexperienced but they still have good faith. The problem lies in management. Once you get the lawyers and bean counters involved is when asshole decisions like that get made.
I'm an engineer but I'm also an accountant (aka a bean counter). I also am management in my company and I'm in charge of the engineering, production and accounting among other things. You are absolutely right that most of the time the fault for most failures ultimately is due to management decisions. At the end of the day the buck stops with them and that is how it should be. HOWEVER, management ultimately relies on the expertise of engineers and the reasoned opinions of those who report to them. If management gets bad information (happens all the time) then management decisions are more likely to be bad ones. That's not to say that management can't introduce cock-ups all their own (we all know they routinely do) but bad management decisions normally don't happen in a vacuum. Most serious screw ups in a company happen because people at multiple levels in a company made a mistake. It's not just management or engineering or accounting alone. Rockets blow up when all these mistakes made by multiple people line up in just the right way. Business is a team sport and most failures in business involve more than one person.
Engineering is kind of like playing chess. You can see the entire board and you know (or should) what is possible at any given moment because you have close to perfect information. If you are good enough mistakes are largely predictable. Management is more like playing poker. You rarely have perfect information and you have to guess based on your assessment of the probabilities and sometimes you'll be wrong even if you play the hand perfectly. The mindsets needed for success in each for each are very different and can be hard to reconcile at times.
It has also been my experience that there are FAR more inept engineers that most people realize. Specifically too many engineers are inept at product and process documentation. It's boring and despite its importance it tends to get overlooked and engineers often spend as little time with it as they can get away with. I run a contract manufacturing company that makes wire harnesses. I can count on my fingers the number of product drawings that I've received in the last 5 years from all our customers combined that I could manufacture without having to get substantial clarification from the engineering staff at our customer. I routinely see ambiguous dimensions, incompatible parts, improper or incomplete specifications, missing part numbers, internal part numbers (useless to anyone else), incoherent diagrams, unnecessary lookup tables, obsolete or hard to get parts, and more. Most engineers I've run into are really quite terrible at documentation. I see drawings daily that were clearly not written with the expectation that anyone else might ever read them despite the fact that someone else reading it is the entire point of the document.
I never read monospaced posts. Can someone summarize what the parent said?
Most commercial and large-scale shipping endeavors (hell, even UPS) require the sender to insure their package if they want coverage in case of loss. This is the same with commercial container ships, and I suspect the same with airline freight.
Hardly. This points more to poor vendor selection and a crap procurement process. In many cases the costs of faults should be contractually passed back to the vendor. The end result is a more costly product but with higher quality as a result.
Even good vendors sometimes make mistakes. Even good engineers sometimes overlook important details. I can assure from personal experience you that even draconian claw-backs (if you can get them) will not result in a good product and frankly are unnecessary in most cases. It is FAR more complicated than that. You are right that the problem is with flaws in procurement but the answer isn't just claw-backs. Good vendor relationships require quite a lot of oversight, interaction, structure and cooperation. New vendors typically require a lot of oversight until the interaction, structure and cooperation can develop.
The problem is that salesmen are lying bastards.
That should not be relevant once the engineering staff gets their hands on the product specifications which would be necessary prior to quotation for anything built for a company like SpaceX. To build any custom product the engineers will have to evaluate and sign off on the production process. Purchasing would have to sign off on the procurement. Any company that simply lets their sales people throw out randomly generated numbers for engineering intensive products isn't going to be around for long. I know because I run a manufacturing company that makes such products. The dumbest thing we could possibly do would be to give our sales representatives carte-blanche. We'd be out of business within a year if we did that.
When you get the technical people on the phone (if you can get them) the complete and typically trustworthy story comes out.
I am an engineer (among other things) and I work with engineers all the time. I can assure you that getting a complete answer out of engineers at other companies can be exceptionally challenging at times. It's not that they lie but rather that they are busy, hard to track down (esp in big companies), you are a distraction, sometimes they are lazy and more than you would think are not especially competent. I run into a LOT of engineers that are really quite bad at writing engineering documentation. I run into quite a few others who have an exceptionally poor concept of design for manufacturing. Working with engineers can be great but I can assure you from first hand experience that it can be quite a challenge at times too.
Apollo 13 was a near miss also likely due to quality.
No, Apollo 13 happened because of TESTING. Had they not tested the oxygen tank, it would have been fine.
Oh, and had the oxygen quantity gauge not failed, they would have died. The failed gauge led to them stirring the tank more often, so it blew up before the reached the Moon. In the original mission plan, that oxygen tank stir would have happened while the LEM was on the lunar surface.
So, if anything, Apollo 13 is an argument for less testing, not more.
I thought things like x-ray examination and grain analysis are mandatory for every semi-critical product in the steel industry.
And they putting some untested struts in the rocket that costs many millions? How can it happen?
I hope the Ministry of Silly Walks is conducting a full investigation.
The book Normal Accidents takes a look at several famous failures (Kansas City Walkway, Space Shuttle, etc) and a common phenomenon emerges: a last minute test or modification, often to increase safety, plays a significant part in the system failure.
If anything, that Google's monopoly blocks it is a reason for "everybody sane" to give it a benefit of the doubt.
But, whatever — as long as it is used by actual developers — such as the guy behind UDT — and offers its vast network of mirrors for downloading open source code, I'll keep using them, thank you very much.
Whatever the site does in its attempts to monetize its numerous Winblows users is of no concern to me.
In Soviet Washington the swamp drains you.
If they did not then there would be just as many people complaining that they could not. I over-ride fonts with my browser's settings. It works for me. There is always going to be a dumbass in every crowd and there is usually more than one. I can not control them but I can control me.
"So long and thanks for all the fish."
I'm wondering why they didn't turn on Dragon's emergency abort system. Space X missed a golden opportunity to test it. Recovered the payload in-tacked after a launch failure would have been more impressive than a successful launch.