Slashdot Mirror


Japanese Probe Finds Miswiring of Boeing 787 Battery

NeverVotedBush writes in with the latest installment of the Dreamliner: Boeing 787 saga. "A probe into the overheating of a lithium ion battery in an All Nippon Airways Boeing 787 that made an emergency landing found it was improperly wired, Japan's Transport Ministry said Wednesday. The Transport Safety Board said in a report that the battery for the aircraft's auxiliary power unit was incorrectly connected to the main battery that overheated, although a protective valve would have prevented power from the auxiliary unit from causing damage. Flickering of the plane's tail and wing lights after it landed and the fact the main battery was switched off led the investigators to conclude there was an abnormal current traveling from the auxiliary power unit due to miswiring."

52 of 201 comments (clear)

  1. Yay, time for finger pointing by DavidRawling · · Score: 2

    Who will it be? Maintenance? Boeing?

    1. Re:Yay, time for finger pointing by the+eric+conspiracy · · Score: 4, Insightful

      Outsourcing contractor.

    2. Re:Yay, time for finger pointing by Anonymous Coward · · Score: 4, Insightful

      Really? I am actually quite impressed. The degree of investigation over lighting failures and back up safety systems and all that is pretty awesome. Putting aside my condemnation of corporations like Boeing, this mess isn't damning, but rather assuring. Any finger pointing should be met with a reminder that the plane landed just fine. Granted, I'd be annoyed if my flight was grounded for this nonsense but degree of blame should reflect the problem caused.

    3. Re:Yay, time for finger pointing by Anonymous Coward · · Score: 4, Insightful
    4. Re:Yay, time for finger pointing by bobbied · · Score: 3, Interesting

      Who will it be? Maintenance? Boeing?

      All of the above!

      I'm skeptical of this story. They are basically saying that somehow the wiring got messed up in such a way that everything still worked, but the battery was improperly charged/discharged by the APU. The evidence they have is some lights that flickered. This seems fishy to me.

      If something is miswired, then it's going to be possible to PROVE that as fact. Even if the unit was cut from the aircraft, it would be possible to physically inspect and verify what wire went where. Flickering lights are NOT PROOF of anything being incorrectly wired.

      If the drawings don't match the design, you can PROVE that by inspecting the drawings. If the aircraft doesn't match the drawings you can PROVE that by inspecting the aircraft. We have NO proof here.

      I'm guessing that somebody in Japan wants to get these aircraft back into the air, bad enough to come up with some story with flimsy evidence and managed to get Japan's version of the NTSB to agree.

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
    5. Re:Yay, time for finger pointing by bobbied · · Score: 4, Insightful

      I was just thinking that..the media will now have their blame game but at the end of the day it was a plane mishap that didn't include charred bodies strewn on the countryside. It was a glitch, that was easily fixed. I could have been much worse.

      This whole idea of a wiring error sounds fishy and it seems to be based on flimsy evidence. These kind of things are proven by hard inspection of the aircraft, drawings, and designs not by observing flickering lights. Somebody in Japan wants these aircraft in the air really bad, and I'm betting they managed to talk Japan's version of the NTSB into this idea.

      I'm waiting for the final report on this... Before I decide to get on one of these.. Because if this flimsy sounding reason is what I think it is, another plane is going to have a battery fire pretty soon and this time we might not be so lucky.

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
    6. Re:Yay, time for finger pointing by marcosdumay · · Score: 2

      Add to that that there were other, less severe but similar problems with the battery on other planes.

      Also, I'd say (but nobody listens to me anyway) that if the battery can be misswired like that, it's a design flaw and Boeing should issue a correction. Of course, there is a lot of needed research before stablishing that the battery in fact has this problem, but that'd be the proper action.

    7. Re:Yay, time for finger pointing by ColdWetDog · · Score: 5, Informative

      You do realize that the flickering lights pointed investigators in a particular direction. THEN, after more analysis, they discerned the problem lay in miswiring. The flickering lights are not prima facie evidence of a wiring fault.

      A bit more detail would be welcome. As it is, one cannot tell what happened or how many aircraft are affected.

      --
      Faster! Faster! Faster would be better!
    8. Re:Yay, time for finger pointing by PolygamousRanchKid+ · · Score: 2

      Boeing should have called the Call Center Help Line. The first question they always ask, is, "Is the device plugged in correctly?"

      I find it mildly amusing that the Airbus A-800 also had problems with the wiring. They blamed that on a mismatch in CATIA system between French and German engineers.

      It's amazing, all those high-tech doo-hickies, whatchits and gadgets in the plane. . . and in the end a wiring problem causes the system to fail. Maybe in the future, they can just all use one bus, and get rid of the wiring.

      --
      Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
    9. Re:Yay, time for finger pointing by anubi · · Score: 5, Insightful

      When I read of it, I felt more vindicated than surprised.

      During my tenure in aerospace, I had witnessed more and more of a disregard for detail work. What used to be a good thing called "attention to detail" started being regarded negatively as "being a perfectionist".

      The devil is in the details. Thousands of things work perfectly. One does not. This is the inevitable result of overlooking just one detail.

      --
      "Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]

    10. Re:Yay, time for finger pointing by Solandri · · Score: 3, Insightful

      The 747 has something like 150 miles of wiring. The 787, which was specifically designed to reduce the amount of wiring, still has some 60 miles of wires. There's a lot of opportunity for miswiring something.

    11. Re:Yay, time for finger pointing by sjames · · Score: 2

      That is important to keep in mind. Because of the things they did right, the thing they did wrong hasn't killed anyone.

    12. Re:Yay, time for finger pointing by Anonymous Coward · · Score: 2, Interesting

      You do realize that the flickering lights pointed investigators in a particular direction. THEN, after more analysis, they discerned the problem lay in miswiring. The flickering lights are not prima facie evidence of a wiring fault.

      A bit more detail would be welcome. As it is, one cannot tell what happened or how many aircraft are affected.

      the Japanese government is not big on providing details. the culture is one where you trust your elders, and the government is the ultimate parent. personally, I resent that.
      *bows head to dodge trolls*

    13. Re:Yay, time for finger pointing by buybuydandavis · · Score: 4, Insightful

      The length of the wires isn't a useful metric - it's the complexity of the wiring that causes miswiring.

    14. Re:Yay, time for finger pointing by Runaway1956 · · Score: 4, Funny

      It wasn't me! I swear it wasn't me! I've never worked on an aircraft in my life!

      Sux2bthatguy!!

      (Note that Runaway is color vision impaired, and has in fact wired things wrong from time to time.)

      --
      "Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
    15. Re:Yay, time for finger pointing by Runaway1956 · · Score: 3, Insightful

      At a guess, I'd say the total length of wiring might be indicative of complexity. The machines that I have worked on that have only a few hundred feet of wiring are generally less complex than machines with thousands of feet of wiring in them.

      For comparison, find an old Farmall or John Deere tractor, and compare the wiring to your modern automobile. An elementary school child can figure out the wiring on an 50 to 80 year old tractor. Good luck with your car - experience mechanics have problems chasing down problems, especially intermittent shorts.

      --
      "Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
    16. Re:Yay, time for finger pointing by aaarrrgggh · · Score: 3, Insightful

      Only if you assume the topology is the same. The 747 is likely to be much more of a "star" topology with traditional circuit breakers. The 787 is more of a "bus" topology with solid-state relays.

    17. Re:Yay, time for finger pointing by viperidaenz · · Score: 3, Insightful

      Then compare a "modern" car with a very modern car. The huge mess of wires is being replaced by CAN and LIN buses.

    18. Re:Yay, time for finger pointing by athmanb · · Score: 4, Funny

      They originally planned to use 60 miles of wiring but then they only ordered 60 kilometers of wires so two thirds of the devices are not connected. It's not that big of a problem though since most things are covered by redundancy.

    19. Re:Yay, time for finger pointing by router · · Score: 4, Insightful

      I agree.

      It shouldn't have been possible to "miswire" an aerospace battery, the connectors should have been coded, the wires, and the inspectors should have seen and tested this. Battery failure is still a process failure. Unfortunately, process failures are the most systemic failures possible. Lets hope I'm wrong....

      andy

    20. Re:Yay, time for finger pointing by ShanghaiBill · · Score: 4, Insightful

      A big chunk of the blame should go to whoever designed the connectors. For safety critical systems, it should be physically impossible to connect them in an unsafe configuration.

    21. Re:Yay, time for finger pointing by plover · · Score: 2

      They learned at least two things from this incident, not one. The first lesson is that it was "miswired" (agreed, a fishy statement), but it means they can test some wiring or insulation in existing and future planes to make people think they're doing enough to get the planes back in the air. Second, and more importantly, they learned that the batteries can burn as a group, and that they need to minimize the damage a battery fire can cause by better restricting the ability of the fire to spread. So the next time this happens, the plane won't be at as much risk.

      --
      John
    22. Re:Yay, time for finger pointing by fluffy99 · · Score: 2, Informative

      A valve is also another term for a diode, which only allows current to flow in one direction.

    23. Re:Yay, time for finger pointing by semi-extrinsic · · Score: 5, Insightful

      Yes, and it's extremely annoying if you want to do anything to your own car. It's bad on the same level as proprietary connectors for phones and all that, but unfortunately the amount of people improving their own cars is too low to cause any consumer feedback to manufacturers.

      And I don't mean adding stupid spoilers and boost chips and sillyness, I mean stuff like adding an extra pair of high beams that can be operated with the same button as the regular high beams. That will take some serious hacking on a modern car. If car manufacturers were good at making things, this wouldn't be a huge problem, but modern cars do so many things wrong that it's infuriating. Like putting lambertian leds in places where they should have put batwing ones, forcing me to put a diffuser in front of it so that my daughter is able to sleep in her car seat. Or making it a fifteen-minute job to remove the battery for charging it during the winter, when it should take two minutes. Or putting the light that activates when you open the trunk in the far left corner of the trunk, so that it doesn't light up anything if you actually have something in the trunk. I could go on about this for a while...

      --
      for i in `facebook friends "=bday" 2>/dev/null | cut -d " " -f 3-`; do facebook wallpost $i "Happy birthday!"; done
    24. Re:Yay, time for finger pointing by dwywit · · Score: 2

      Yes, and how much are you willing to pay for your ticket/s?

      --
      They sentenced me to twenty years of boredom
    25. Re:Yay, time for finger pointing by aethelrick · · Score: 5, Insightful

      a wiring problem could be as simple as using an incorrect thermistor on a Li-ion pack or not wiring a thermistor in at all. These are often used to alter charge/discharge rates in response to the battery pack temperature. A battery will still work in every other respect, except it won't respond accordingly in response to overheating. This is a fairly simple example of what could go wrong to cause a fire that would not stop the battery from working (until it failed by going on fire). The trouble with Li-ion packs is that if this happens (and it does) then the fire can very easily spread to the surrounding cells. I can see how this could cause short voltage spikes that would overcome resistance in a line to "flicker" a light.

      I'd just like to add, I may be totally wrong, but I thought I'd weigh in for the fair minded rather than the conspiracy theorists on this one. Also, before anyone assumes I'm a Boeing employee, I'm not. I'm just a bloke who works with Li-ion batteries and who has seen faults similar to this in the past.

    26. Re:Yay, time for finger pointing by Luckyo · · Score: 2

      Those thinking this is "nonsense" should stop for a moment and recall what happened when pilots didn't take smoke warning seriously:

      https://en.wikipedia.org/wiki/Swissair_Flight_111

    27. Re:Yay, time for finger pointing by andydread · · Score: 2

      a Valve lets the Steam out.

    28. Re:Yay, time for finger pointing by 140Mandak262Jamuna · · Score: 2

      You got it backwards buddy. Not connecting most things would mean there can not be a fire. What happened was they ordered 60 miles of wiring when they needed only 60 km. The extra wire was looped round and round, held together with plastic ties, just in case we need it later. That is what causing the fires.

      --
      sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
    29. Re:Yay, time for finger pointing by nomorecwrd · · Score: 4, Insightful

      Did I ever told you about this guy, I met some years ago, that used an abrasive file over the edges of the HardDisk power connectors, because they "didn't fit" the way he wanted to connect them?

      Q: What is worse than a dumb guy?
      A: A dumb guy with initiative.

    30. Re:Yay, time for finger pointing by dywolf · · Score: 2

      ever actually have to trace wires on an aircraft?
      i've worked on Huey and Cobras...small helos. And they can be a PITA to track things down.
      And yes, i've found things that were caused by miswired connections, usually in the solder terminals of a switch.
      so yes, i can easily see something simple like putting the wrong wire into the wrong terminal of a terminal lug/connector as all it was.
      wouldnt be the first time, just like it wouldnt be the first time a tech manual drawing was unclear or even incorrect.

      --
      The guy who said the election was rigged won the presidency with the second-most votes.
    31. Re:Yay, time for finger pointing by bobbied · · Score: 2

      I agree.

      It shouldn't have been possible to "miswire" an aerospace battery, the connectors should have been coded, the wires, and the inspectors should have seen and tested this. Battery failure is still a process failure. Unfortunately, process failures are the most systemic failures possible. Lets hope I'm wrong....

      andy

      One must think about how aircraft are actually assembled. In most cases, wire bundles are installed without the terminating connectors installed on at least one end. This is because the connectors are too bulky to easily pull bundles though the small spaces required and it is difficult to know the exact length necessary to provide the proper clamping and clearances. It is simpler and cheaper to just install the wires and then cut them to length and install the connectors.

      Manufacturing processes for aircraft usually include a comprehensive double check of wiring harness installation. This includes manual and automated testing using machines the connect to the huge number of connectors in a wired aircraft, followed by extensive functional testing of just about everything. Errors are not uncommon, but they are generally caught and corrected long before the aircraft gets signed off as airworthy.

      Usually, manufacturing designs for aircraft include specific keying for connectors which might be miswired. This means that it would be impossible for an avionics mechanic replacing a battery to connect up something incorrectly, unless they altered the keying on the connectors or did something really stupid like re-routing an existing wiring harness to "make it reach". Both activities would be physically obvious.

      Beyond routine maintenance, you have ongoing modification processes, where wiring can get changed due to design changes. This process is very strictly controlled and involves a verification process that should have independent review of all the work done. If someone miswired the aircraft, and the review missed it, then ALL three entities are going to be at fault. The guy who re-wired it, the guy who signed it off as complete and the manufacturer who designed the verification test. Assuming everybody was following procedure and nobody is guilty of not doing their job.

      Again, this whole "it was wired incorrectly" idea sounds fishy to me given the "blinking lights" as evidence. Either the plane was wired wrong or it wasn't, and you can grab the design drawings and verify the wiring pretty quick. Now if they are saying there is a *design* issue, that too should be something that can be clearly explained by looking at the drawings. Reports of blinking lights might be an indicator of where to look, but it is not proof we found the problem.

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
  2. User error by girlintraining · · Score: 5, Interesting

    So basically, the user reached back behind the power supply while fiddling and bumped the 110/220V switch, and it caught fire. Naturally, they didn't say anything to the tech after setting the switch back besides, "It just caught fire! All by itself!"

    The user in this case is a giant airline company, and tech support would be Boeing. The FAA, of course, is the QA manager, who reviewed the call, and after reading the ticket closure notes, facepalmed, leaned back into his chair, and took a deep draft of coffee.

    --
    #fuckbeta #iamslashdot #dicemustdie
    1. Re:User error by Cassini2 · · Score: 2

      Many power supplies are designed to autoswitch between 110V and 220V for just this reason. Cheap power supplies aren't.

      I knew one customer that said: "We didn't know that it was a 220V machine when we connected it to 600V!" That bang was audible.

    2. Re:User error by girlintraining · · Score: 4, Interesting

      No no, I know. I was just reframing the "black and nebulous art" of airplane maintenance into something easier to digest for slashdotters. It was either that, or a car analogy, and turning a plane into a car just felt wrong. :) The truth is a bit more complicated; But it still boils down to operator error and not a design flaw. Of course, a design that allows someone to plug in one component backwards and have the entire device go up in flames is not a good one, but it's not flawed in the strict sense of the word. It's disappointing that my $500 laptop has a feature that prevents the battery from being plugged in backwards, but a multi-million dollar state of the art aircraft does not.

      --
      #fuckbeta #iamslashdot #dicemustdie
    3. Re:User error by number11 · · Score: 3, Informative

      Actually from my admittedly limited experience, FAA and airplane mfgrs are downright obsessive about making connections idiot proof and failsafe. It's pretty difficult to find places in an airplane where it's possible to plug the wrong things together or backwards. FAA has been dealing with Murphy for a very long time. In this case, if that's what happened, then it's one that slipped through the design and development process. FAA will mark this as a design failure and require Boeing to make it impossible to connect wrongly.

      Looking at that Japanese powerpoint, it looks like that may be exactly what happened. The battery cells are rectangular with a stud on each side of the top. Not even any prominent markings to indicate polarity, though the two studs seem to be mounted with different colored rivets. You'd think they'd at least have different diameter studs for the positive and negative, and jumpers with holes to match.

  3. Re:What? by girlintraining · · Score: 4, Funny

    Can't they make an idiot proof power plug?

    Because idiots are much more resourceful than ordinary people.

    --
    #fuckbeta #iamslashdot #dicemustdie
  4. Re:What? by Areyoukiddingme · · Score: 2, Insightful

    Yes, but commercial airliners aren't built with plugs and sockets. For weight savings, everything is directly hardwired. At least, in pretty much every airliner prior to the 787, and I can't imagine Boeing changing that. Military aircraft are built with plug and socket connectors, but both sides of the connection are big bulky heavy metal components. When your plane is a cockpit and wings strapped onto a giant oversized turbine, you basically don't care about weight, but commercial airliners are the exact opposite. They're obsessed with weight savings, so the miswiring happened during initial assembly and their quality control procedures were too poor to catch it. Boeing has fallen a loong long way.

    Someday, people are going to look back on the outsourcing mania of core competencies by MBAs over the past generation as sheerest idiocy.

  5. Re:What? by Sponge+Bath · · Score: 3, Interesting

    Because idiots are much more resourceful than ordinary people.

    Behold!

  6. Happened before by Grayhand · · Score: 4, Funny

    It's the damned metric +/- that causes all the confusion.

  7. A protective valve? by dgharmon · · Score: 2, Informative

    "the battery for the aircraft's auxiliary power unit was incorrectly connected to the main battery that overheated, although a protective valve would have prevented power from the auxiliary unit from causing damage"

    What is a power diode

    --
    AccountKiller
    1. Re:A protective valve? by the+eric+conspiracy · · Score: 4, Informative

      "Valve" is a generic term, slightly archaic for an electronic switch. Some vacuum tubes are called valves.

      http://en.wikipedia.org/wiki/Vacuum_tube

      Since a transistor is simply a crystal triode, the terminology is reasonable.

      http://www.beatriceco.com/bti/porticus/bell/belllabs_transistor.html

    2. Re:A protective valve? by PPH · · Score: 2

      That might be a translation of the Japanese term. By a non-technical translator.

      --
      Have gnu, will travel.
  8. Re:What? by hawguy · · Score: 4, Insightful

    Yes, but commercial airliners aren't built with plugs and sockets. For weight savings, everything is directly hardwired. At least, in pretty much every airliner prior to the 787, and I can't imagine Boeing changing that. Military aircraft are built with plug and socket connectors, but both sides of the connection are big bulky heavy metal components

    Do you have a reference for that? It doesn't make sense that field replaceable parts are hardwired in - you'd have to clip the wires to take it out, and every time you clip the wire it gets shorter, so eventually you'd have to run a new wire back to the source.

    Even for parts that aren't replaced often, it seems that hardwiring would just increase the chance of error - if everytime they replace an engine someone has to sit down and manually splice 200 separate wires, that seems a lot more trouble prone than plugging in a dozen connectors that were wired in at the factory and tested on the factory test harness to be sure every wire was connected to where it should be.

  9. Japanese Probe? by Charliemopps · · Score: 4, Funny

    When you say "Japanese Probe" I had an entirely different idea in my head regarding what this story was about.

    1. Re:Japanese Probe? by MyHair · · Score: 2

      When you say "Japanese Probe" I had an entirely different idea in my head regarding what this story was about.

      Weren't you surprised nothing was pixelated?

  10. Re:What? by Anonymous Coward · · Score: 2, Informative

    Slightly different, but my friend works for a company that makes in-flight video systems for planes including Lufthansa. While not mission critical, they still have to follow FAA and other regulations... one of which is some of the plugs they use plug in and then are secured in place with 12 to 16 screws even though the signals being passed are just network/video/audio.

    I don't see why they couldn't use plugs of the same fashion instead of hard wiring everything

  11. Re:What? by garyebickford · · Score: 3

    AFAIK you are completely incorrect. Just using the 787 for example, the entire thing is built in modules including wiring and all, that are built and tested by subcontractors and then plugged and bolted together at the Boeing assembly plant. All commercial aircraft that I am aware of at least since the 1940s has had connectors. The biggest problem with connectors is not the weight but the unreliability. Each connector is a potential point of failure, so aircraft electrical connectors are actually heavier - they have positive scraping between the two parts of each connection, often have moisture resisting / sealing. and have a threaded ring that holds them together. Then (IIRC) there is an additional set of tabs through which a wire is threaded and itself positively bound - used to be twisted, now I think they use a mechanical crimp. The wire assures that the threaded ring can not unscrew itself due to vibrations.

    The most expensive cost for a commercial aircraft after fuel is the cost of downtime - time spent fixing things costs thousands of dollars per hour. Therefore everything on an aircraft is designed to be removed and disconnected quickly, efficiently and safely - including things like wings, tail fins, etc. FAA is not going to allow the mechanics to cut wires and fasten them back together, so again the connectors are designed so that each one can only go one way.

    --
    It's easier to be a result of the past, but more fun to be a cause of the future! http://www.spacefinancegroup.com/
  12. Re:What? by confused+one · · Score: 2

    I'm sorry; but, you're wrong. Work for a sensor manufacturer that sells to the aerospace industry and I can tell you, commercial aircraft cabling is full of connectors. Same kind of locking connectors found on military aircraft.

  13. Re:What? by c0lo · · Score: 2
    --
    Questions raise, answers kill. Raise questions to stay alive.
  14. Re:What? by Copperhamster · · Score: 2

    Never underestimated the ingenuity of dumb. I gave a guy a hard drive to replace the one in his computer. He 'knew all about it' and I knew he had the knowledge to format and reinstall his OS no problem.
    He brought the drive back to me, claiming it didn't work, in fact the power cables would not fit till he took his dremel to them, so it must not have been the right drive for his machine.

    He had shaved off the corners of a standard hard drive power connector so he could fit it upside down in a used 512 MB IDE hard drive. Which of course killed the drive, but I also determined it killed that connector on the motherboard's IDE bus.
    The reason for this mistake?
    His computer had been built with some oddball brand of hard drive (I can't remember... Paladin or Palladium? I think it started with a P) which put the controller board on TOP of the drive instead of on the bottom.

  15. Lost in translation by KH · · Score: 2

    I see that the discussion here is based on a sketchy summary from the originally Japanese press conference. More coherent information is available if you could read Japanese but I know it's too much to ask for...

    Here is the latest update of the on-going investigation from the JTSB issued 20 Feb, 2013; this mentions the mis-wiring:

    http://www.mlit.go.jp/jtsb/flash/JA804A_130116-130220.pdf

    More in-depth information is given at

    http://www.aviationwire.jp/archives/16032

    According to this article, the mis-wiring was in the original specs/design, and the design had been corrected. The aircraft in question was manufactured in accordance to the earlier specs but no modification was made to comply with the new ones. One can infer that the bug was considered insignificant to compromise the safety of the aircraft. The JTSB currently does not think this mis-wiring was the cause of the battery incident although they will keep looking into it as a potential cause of anomalous voltage readings.