Slashdot Mirror


Could a Dirty Rag Take Out a $2 Billion Satellite?

An anonymous reader writes "The alleged rescue of a U.S. military communications satellite underscores some of the weaknesses in U.S. space efforts. Quoting: 'The seven-ton “AEHF-1,” part of a planned six-satellite constellation meant to support radio communication between far-flung U.S. military units, had been in orbit just one day when the problems began. The satellite started out in a highly-elliptical, temporary orbit. The plan was to use the spacecraft’s on-board engine to boost it to a permanent, geo-stationary orbit. But when the Air Force space operators at Los Angeles Air Force Base activated the engine, nothing happened. The Government Accountability Office would later blame the failure on a rag left inside a fuel line by a Lockheed worker.'"

45 of 297 comments (clear)

  1. Lockheed gonna get sued? by Anonymous Coward · · Score: 5, Insightful

    Seems like the gov't should sue lockheed for failing to deliver the working satellite as contracted.

    Hopefully that'll happen (which will probably leave that worker jobless) and we'll get some of our tax dollars back.

    Shhh... I can dream!

    1. Re:Lockheed gonna get sued? by JonahsDad · · Score: 5, Informative

      TFA states that they are seeking compensation from Lockheed. Hopefully, that'll happen without an actual suit.

    2. Re:Lockheed gonna get sued? by davester666 · · Score: 5, Funny

      Yes, the gov't will have to pay for that space-rag now. Lockheed forgot to bill them for it.

      --
      Sleep your way to a whiter smile...date a dentist!
    3. Re:Lockheed gonna get sued? by Anonymous Coward · · Score: 4, Insightful

      Seems like the gov't should sue lockheed for failing to deliver the working satellite as contracted.

      Hopefully that'll happen (which will probably leave that worker jobless) and we'll get some of our tax dollars back.

      Shhh... I can dream!

      Lockheed wouldn't piss off their biggest spender. They'll pay back in the form of a "credit" for some kind of services that have the highest margin for Lockheed. The guy who screwed up and his boss will get fired for sure, and then they will have some business analyst examine their QA process and add a little redundancy in the inspection policies. Nothing to see here folks.

    4. Re:Lockheed gonna get sued? by Ouchie · · Score: 5, Funny

      Yes, the gov't will have to pay for that space-rag now. Lockheed forgot to bill them for it.

      The bill also included the fines levied by the TSA for failing to file an export declariation on the space rag.

      --
      "Of all the things I've lost, I miss my mind the most." ~Ozzy Osborne
    5. Re:Lockheed gonna get sued? by Anonymous Coward · · Score: 5, Insightful

      Seems like the gov't should sue lockheed for failing to deliver the working satellite as contracted.

      Hopefully that'll happen (which will probably leave that worker jobless) and we'll get some of our tax dollars back.

      Shhh... I can dream!

      Lockheed wouldn't piss off their biggest spender. They'll pay back in the form of a "credit" for some kind of services that have the highest margin for Lockheed. The guy who screwed up and his boss will get fired for sure, and then they will have some business analyst examine their QA process and add a little redundancy in the inspection policies. Nothing to see here folks.

      Isn't that what should happen? I mean, when did the world suddenly decide that anytime anyone makes an honest mistake they should be crucified for it forever? If there is restitution for lost funds as well as improvements to try to prevent a repetition of the same problem, shouldn't everyone involved be satisfied? I'm fairly certain that the OP's hope that we all get some kind of tax refund is probably not going to happen, and even if it did, you'd be talking about a few dollars per person at most.

    6. Re:Lockheed gonna get sued? by z00_miak · · Score: 5, Funny

      TFA states that they are seeking compensation from Lockheed. Hopefully, that'll happen without an actual suit.

      That would be quite the space suit.

    7. Re:Lockheed gonna get sued? by pixelpusher220 · · Score: 3, Insightful

      49% don't actually pay income tax

      Because they didn't make any money above federal poverty rates....but hey, don't let details get in the way of a perfectly overused irrelevant statistic.

      the top 1% paid 40%

      When they account for 50%+ of the income, the should be paying *more* taxes, not less.

      --
      People in cars cause accidents....accidents in cars cause people :-D
    8. Re:Lockheed gonna get sued? by Anonymous Coward · · Score: 3, Informative

      The to 5% make 38%. But this is Slashdot, it's not as though you guys are going to let something like MATH get in the way of Socialism.

    9. Re:Lockheed gonna get sued? by DJ+Jones · · Score: 5, Insightful

      Actually that whole 49% figure is misleading. The element of "time" is conveniently left out. 49% of Americans at any given "time" do not pay income tax, not because they are lazy, poor, freeloading citizens but because they are either too young to earn income or retired. In actuality, over 90% of Americans pay income tax at some point in their lifetimes.

    10. Re:Lockheed gonna get sued? by Hatta · · Score: 5, Insightful

      I mean, when did the world suddenly decide that anytime anyone makes an honest mistake they should be crucified for it forever?

      Forgetting a rag is an honest mistake. Failing to plan for honest mistakes by implementing the appropriate checks into your process is negligence.

      --
      Give me Classic Slashdot or give me death!
    11. Re:Lockheed gonna get sued? by syousef · · Score: 3, Insightful

      I mean, when did the world suddenly decide that anytime anyone makes an honest mistake they should be crucified for it forever?

      Forgetting a rag is an honest mistake. Failing to plan for honest mistakes by implementing the appropriate checks into your process is negligence.

      The engineer following the process is not necessarily the person that created the procedure. Also even if a procedure is in place double failures do occur - they are just less likely.

      I love the way so many people are willing to judge that a man should or should not be fired based on 3 minutes of reading a slashdot story. Really enhances my faith in human nature. Hope none of you ever sit on a jury. What disciplinary action if any should be faced by various staff involved is something that would require at least weeks of investigation, IF you want to go in that direction and waste the time on a witch hunt instead of just fixing the issue.

      --
      These posts express my own personal views, not those of my employer
  2. Hmm...scale does not compute. by fahrbot-bot · · Score: 5, Interesting

    blame the failure on a rag left inside a fuel line

    Must be a really small rag or really big fuel line. Seriously, how would this happen? It's a freaking satellite engine, not the shuttle main.

    --
    It must have been something you assimilated. . . .
    1. Re:Hmm...scale does not compute. by localman57 · · Score: 5, Funny

      Yeah. Damn it people! This is just rocket science, not brain surgery!

    2. Re:Hmm...scale does not compute. by Richard_at_work · · Score: 5, Informative

      You can use tiny squares of cloth, impregnated with cleaning solution, to clean the inside of valves and metal lines - gets rid of metal filings which are left over from the boring process.

      Quite easy to leave one behind. Which is why there are processes in place designed to prevent such issues.

    3. Re:Hmm...scale does not compute. by 0123456 · · Score: 3, Informative

      How do they know a rag was left in the fuel line? Do they have a sensor in the fuel line that checks for the presence of rags?

      I don't know about this case, but AFAIR NASA required forms signed in triplicate saying that any tool taken into the shuttle was later removed from it. Perhaps there's similar tracking in this case and a check showed up a rag that wasn't signed out for being removed.

      It seems to be a common problem, I'm sure I remember a couple of rocket launches which were blamed on rags in the fuel lines.

    4. Re:Hmm...scale does not compute. by ClickOnThis · · Score: 4, Insightful

      Yeah. Damn it people! This is just rocket science, not brain surgery!

      Actually, compared to rocket science, brain surgery is a walk in the park.

      --
      If it weren't for deadlines, nothing would be late.
    5. Re:Hmm...scale does not compute. by Maximum+Prophet · · Score: 4, Interesting

      You can use tiny squares of cloth, impregnated with cleaning solution, to clean the inside of valves and metal lines - gets rid of metal filings which are left over from the boring process.

      Quite easy to leave one behind. Which is why there are processes in place designed to prevent such issues.

      So, they built a tool to make sure the rag was removed. Then they built another tool to check that the first tool was removed...

      More seriously, why wouldn't groundside testing notice that there was a rag in the line?

      --
      All ideas^H^H^H^H^Hprocesses in this post are Patent Pending. (as well as the process of patenting all postings)
    6. Re:Hmm...scale does not compute. by Richard_at_work · · Score: 4, Informative

      More seriously, why wouldn't groundside testing notice that there was a rag in the line?

      Yup, why wouldn't it?

      Obviously it didn't. Multiple times. In multiple different situations - this isn't the first space mission to be ruined because of something left where it shouldn't have been.

      The obvious answer to your question might be because it didn't block anything during testing, so there was the appearance of nothing wrong. Turn on the fuel flow, after the experience of the launch, and it might have been jostled free from wherever it had chosen to hide - from there it might be a short ride to a bottle necking point such as a crimp in the line, a sharp bend, or a valve, and thus begins the blockage.

    7. Re:Hmm...scale does not compute. by treeves · · Score: 3, Insightful

      No, not paper. These things are assembled in cleanrooms, in which ordinary paper is not allowed, due to the particles/fibers it sheds. Cleanroom cloths are usually lint-free polyester cloth squares about 8 inches on a side, IME.

      --
      ...the future crusty old bastards are already drinking the Kool-Aid.
    8. Re:Hmm...scale does not compute. by SwedishChef · · Score: 5, Insightful

      More seriously, why wouldn't groundside testing notice that there was a rag in the line?

      Some of these positioning rockets are single-use. If you test one you have to build another to replace it. And then test it. And then.....

      --
      No one ever had to evacuate a city because the solar panels broke!
    9. Re:Hmm...scale does not compute. by fuzzyfuzzyfungus · · Score: 3, Interesting

      I'm a bit surprised that it wasn't caught purely as a side effect of other procedures... If I were planning on running fuel through something important that might theoretically be contaminated with lubricants/condensation/whatever, I'd strongly consider blowing $5 worth of compressed nitrogen through it until the outflow is clean...

      Catching every little thing that might gum up the fuel lines during assembly, testing, and cleaning seems like it could be a genuinely hard problem. Doing a combination pressure test/gas flush seems like it would be a cheap, simple, brute-force solution to that entire class of potential problems...

  3. The answer appears to be a yes. by jandrese · · Score: 5, Interesting

    Isn't this sort of like asking if a $5 wrench could wreck a car engine if it were left inside of a cylinder? Is anybody going to say "no"?

    And yes, I went with the car analogy right from the start. Deal with it.

    --

    I read the internet for the articles.
    1. Re:The answer appears to be a yes. by Suki+I · · Score: 3, Interesting

      Exactly! Any old blockage could prevent fuel from getting through the fuel line. Same with the oxidizer. Even smashing a bug under an electrical component could cause a failure.

    2. Re:The answer appears to be a yes. by xrayspx · · Score: 4, Funny

      Luckily, in the aerospace industry, there's no such thing as a "$5 wrench". Hell that was probably a $700 dirty rag.

    3. Re:The answer appears to be a yes. by roc97007 · · Score: 3, Informative

      I have assembled zero satellites. But back in my military contracting days, I did the electronics for several military applications and was present when satellites were built. The boosters on those things are fairly small, and the fuel lines tend not to be big enough to stuff into what we think of as a rag. Maybe a cleaning tool or some other implement. I think whomever wrote that was either lazy or didn't fully understand what they were writing about.

      > Why has Slashdot suddenly fallen into the trap of "I've never seen one so it can't possibly exist"?

      Have we so soon forgotten that us slashdotters come from a variety of backgrounds? For instance, legal articles are often responded to by actual lawyers in this group. There are actual astronomers, actual physicists, actual biologists, and I'm certain, actual rocket scientists, who read and participate in Slashdot. We're not all gamers living in our parent's basement. Although there are some.

      --
      Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
    4. Re:The answer appears to be a yes. by RingDev · · Score: 3, Informative

      Most starters aren't strong enough to bust up a wrench or socket. Take out a plug maybe, possibly bend a valve, but in all likelihood, the motor would turn the engine till contact and stop.

      That is assuming you are hitting the engin with the starter before hooking up the fuel and plugs. Which is usually a good idea to get the oil pump primed and heads lubricated firing it up.

      That said, I have a number of wrenches that could easily fit in a cylinder with the piston at BDC. A GM 350 for instance, has a 4" bore and 3.48" stroke. On the diagonal that gives you over 5 1/4" clearance at BDC, not including the combustion chamber in the head.

      9-11mm wrenches and 1/4" wrenches are common tools under the hood. Wiring brackets, trim plates, grounding lines, battery terminals, oil pan bolts, valve cover bolts, etc... They all fall into that size range.

      -Rick

      --
      "Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
  4. anything can take out a satellite by apcullen · · Score: 5, Funny

    It's hard. Any little thing that goes wrong will likely cause the whole thing not to work.
    That's why it's rocket science.

  5. Test Sequence? by Anonymous Coward · · Score: 4, Interesting

    Who puts an engine together without a test fire? Seems to me that some simple checks would have prevented a very big waste of funds and effort. I guess it won't be a total waste if they can learn from it.

    1. Re:Test Sequence? by codegen · · Score: 3, Informative

      Actually, the Hubble mirror isn't supposed to be flat, its shape is a particular function. It was actually manufactured exactly to spec, but the spec was wrong.

      Actually,the hubble was spec'd to be a conic constant of p=-.0023, but was polished only to p=-.0139 (i.e. over hyperbolic). The error was due to a problem with the tester. The null reference element was out of position by just over a millimeter. The interesting thing is two other testers reported that the mirror was wrong, but they were ignored because they were not the 'primary' testing instrument. You are correct that it wasn't supposed to be flat, but it definitely wan't built to spec.

      --
      Atlas stands on the earth and carries the celestial sphere on his shoulders.
  6. Lots of failures there. by Lumpy · · Score: 5, Insightful

    Assembly failure - leave a rag.
    Inspection failure - did not check for rag.
    Pre-flight final inspection - still did not find the rag.

    Wow, complete failure all the way down the line from assembly to mating with the launch vehicle.

    --
    Do not look at laser with remaining good eye.
    1. Re:Lots of failures there. by Massacrifice · · Score: 5, Funny

      XXI century new space programs motto : It's failures all the way down, man!

      --
      -- Home is where you eat your heart out.
    2. Re:Lots of failures there. by geek · · Score: 4, Interesting

      It reminds me of those surgery horror stories where the surgeon or staff leaves behind clamps and sponges inside the persons body.

      Shit happens. All we can really do is our very best to try and prevent it, but ultimately, we're human and prone to mistakes.

    3. Re:Lots of failures there. by Overzeetop · · Score: 4, Interesting

      Actually pre-flight final won't catch that kind of thing; it's already buried in the system (and you don't fire thrusters on a flight unit prior to launch). This is likely one of those cases where a scrap of cleaning"rag" was torn off within the path in an area not visible at either end and went unnoticed. To save money, a visual of the system prior to final assembly was determined to be sufficient and the endoscope procedure was eliminated, saving several thousand dollars (combined on all the lines). Sure, in hindsight a compressed air test would have been sufficient, but it's a little late to play what-if now.

      --
      Is it just my observation, or are there way too many stupid people in the world?
    4. Re:Lots of failures there. by LWATCDR · · Score: 5, Informative

      You left out Slashdot summary failure.

      FTFA
      "On Oct. 24, AEHF-1 reached its originally planned orbit. Testing began soon afterward. The Air Force expects to bring the satellite into service in March. Meanwhile, two more AEHFs are slated to launch in 2012."

      They got it into the correct orbit over two months ago using the small thrusters.
      In other words...
      More sensationalistic headlines to get clicks and comments from the new Slashdot.
      Really? Oh and the answer is "no a dirty rag did not take out a 2 billion dollar commsat."
       

      --
      See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
    5. Re:Lots of failures there. by roc97007 · · Score: 4, Interesting

      > It reminds me of those surgery horror stories where the surgeon or staff leaves behind clamps and sponges inside the persons body.

      Funny you should mention that. I had emergency surgery last year for severe traumatic internal bleeding (won't bore you with the details -- or maybe I already have) and things happened so quickly that they did not have enough time for an instrument inventory. (Apparently it's someone's job to keep track of how many tools get used and then count them before final suture.) So after they got me stable they ran me back through x-ray to look for stuff. Didn't find anything, fortunately.

      But really -- it's not that much of a horror story, they just have to open you back up at some point to retrieve the objects. It's not something you want to have happen, but it's a fairly well known procedure. Horror stories to me are things like taking off the wrong limb [1] or prescribing catastrophically wrong medication.

      [1] Before I went in for knee surgery, the doctor gave me a sharpie and had me mark the correct knee. Just in case.

      --
      Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
  7. Foreign object debris seems to be common... by Richard_at_work · · Score: 4, Interesting

    At least one of the recent Soyuz failures was put down to a similar issue - debris left in a fuel line by a worker.

  8. Send up some Midol? by rts008 · · Score: 5, Funny

    So, the problem is the satellite is 'on the rag'?

    --
    Down With Slashdot BETA!!! I've been around the corner and seen the oliphant; you can only abuse me from your perspecti
  9. Speculation, not fact. by pla · · Score: 4, Insightful

    FTA: "They didn’t know it at the time, but a fuel line had become clogged. The blockage “was most likely caused by a small piece of cloth inadvertently left in the line during the manufacturing process,” according to the Government Accountability Office." (bolding mine).

    So no, we don't know that a dirty rag caused a two billion dollar satellite to fail. We think a fuel line became clogged, and some government bean-counter pulled the dirty-rag hypothesis straight out of their derriere so they could sign off on this one and go home.

  10. Heading hyperbole by biometrizilla · · Score: 5, Informative

    Already been established that they were able to overcome the rag and get the satellite into a functional orbit where it can fulfill its mission objective. http://www.spaceflightnow.com/atlas/av019/120103rescue.html

  11. Glad they didn't go to a backup! by wisebabo · · Score: 4, Insightful

    "Finally, it speaks to the size and age of the U.S. space arsenal that the Air Force felt it had no choice but to rescue AEHF-1 instead of replace it with a back-up spacecraft. 'The asset inventory is getting so tight that they spent months limping the heap to its proper orbit,' the insider lamented."

    Look guys, before you throw away (replace with a backup) a $2 Billion satellite, I damn well hope you try some pretty heroic measures. Those are my tax dollars in (the wrong) orbit! So I'm very glad you didn't have (to use) a backup satellite.

    Anyway, does anyone know if the low power thrusters which were eventually used to put this satellite into the correct orbit used the same fuel tank as the clogged thruster? Otherwise 1) I'm very surprised they had enough fuel to get there and 2) they would probably have very little left to last the lifetime of the mission. So let's hope that all the thrusters used a central (hydrazine?) fuel tank and there's plenty left.

    Space is hard and while the U.S. program has certainly had its ups and downs at least it hasn't seen the near total collapse as what happened to the Ruskies. They had quite a bad year last year and that blogger walking around their factory just exposed their problems more. If Mars is going to be a "Red" planet it will because of China not Russia.

  12. Check the logs? by Maximum+Prophet · · Score: 4, Funny

    So why do they not check the forms before launching the satellite into orbit?

    --
    All ideas^H^H^H^H^Hprocesses in this post are Patent Pending. (as well as the process of patenting all postings)
    1. Re:Check the logs? by hedwards · · Score: 4, Insightful

      Probably the same reason why things end up being left inside of patients. Accidents happen, even if it's something that should never happen because it was on the checklist.

  13. Re:Blowing out lines with air... by netwarerip · · Score: 3, Funny

    They had one, but none of the engineers had 2 quarters to put in the machine.

  14. CHECKLIST by bussdriver · · Score: 3, Informative

    The airlines did it and improved by amazing amounts (nobody remembers how bad it was) and the things were much less complex to fly back in those days; the pilots were insulted by it as well. CHECKLISTS WORK.

    Something that important should involve multiple checklists; to error is human no matter how good and smart you are. Doctors are the most arrogant pricks I've ever met so they'll put up a huge fight and have a hard time admitting it when the error rate goes down by half. It likely would go down by half; that is how badly it is needed.

    Nurses too... a friend of mine fought off his nurse violently (as much as he had strength post op) she had to call people in to hold him down and sedate him and luckily somebody heard his screams and READ the chart and realized she had the wrong person! he would have died and without a proper autopsy the cause wouldn't have been known. Mistakes killed my father too. Checklists must be mandatory by law like the pilots who have no issue with them today.