Slashdot Mirror


Excessive Modularity Hindered Development of the 787

TAGmclaren writes "The Harvard Business Review is running a fascinating article exploring the issues facing Boeing's Dreamliner. Rather than simply blaming outsourcing, as much of the commentary has been focused on, the article delves into the benefits of integration and how being integrated when developing a new product gives engineers more degrees of freedom. From the article: 'Historically, Boeing understood that, and had worked with its subcontractors on that basis. If it was going to rely on them, it would provide them with detailed blueprints of the parts that were required — after Boeing had already created them. That, in turn, meant that Boeing had to design all the relevant pieces of the puzzle itself, first. But with the 787, it appears that Boeing tried a very different approach: rather than having the puzzle solved and asking the suppliers to provide a defined puzzle piece, they asked suppliers to create their own blueprints for parts. The puzzle hadn't been properly solved when Boeing asked suppliers for the pieces. It should come as little surprise then, that as the components came back from far-flung suppliers, for the first plane ever made of composite materials... those parts didn't all fit together. Time and cost blew out accordingly. It's easy to blame the outsourcing. But, in this instance, it wasn't so much the outsourcing, as it was the decision to modularize a complicated problem too soon.'"

13 of 200 comments (clear)

  1. No specs? by BVis · · Score: 4, Insightful

    So Boeing told the contractors what they needed to build, but didn't give them hard specifications? What the hell? Two things:

    Boeing needs to have their collective asses kicked for doing it this way, and:
    The subcontractors should never have agreed to the work without specs first.

    The first one is probably the result of Boeing not wanting to spend the engineering dollars to develop the blueprints, and the second is due to the enormous amounts of money involved in making the parts.

    Now that I know this, you'll never catch me on one of those abominations. What the hell was Boeing thinking?

    --
    Never underestimate the power of stupid people in large groups.
    1. Re:No specs? by bunratty · · Score: 4, Insightful

      No, the problem wasn't no specs. The problem was that the system was designed on paper first, without actually building it. Then the specs for the individual pieces were created, and those individual modules were built from the specs. The idea was that then the parts were completed, they would be integrated and work perfectly together. Of course, that never happens because when the pieces come together for the first time, unanticipated problems occur. This is why early integration is a good idea and is part of the philosophy of release early, release often.

      --
      What a fool believes, he sees, no wise man has the power to reason away.
    2. Re:No specs? by idontgno · · Score: 5, Informative

      This.

      I'm a systems engineer, which means that integration is pretty much the only reason my job exists... for projects (hardware/software/everything) which are too big to continuously integrate. Projects that are modularized by design, and very often subcontracted as well.

      If the first time you're integrating your product is the first production run, you're too late. You should have had a prototype. You should treat the first production samples AS prototypes. (The wisdom of the "never buy the zero revision of anything" is in this.)

      But, yeah, that's expensive. It's cheaper to assume that every subassembly will be perfectly built to perfect specifications, and that interfaces just magically happen, and that integration is just sticking the pieces together and turning a few screws.

      --
      Welcome to the Panopticon. Used to be a prison, now it's your home.
    3. Re:No specs? by vlm · · Score: 4, Insightful

      Having lived for over 5 years in Japan, I doubt the Japanese subcontractors would build anything without clear specifications.

      The problem is the toilet seat bracket had to be made 1/10 mm thicker for supersized passengers, and that was properly annotated by the seat mfgr on blueprint revision #24352. Unfortunately the news never reached the design engineers for the landing gear who need to adjust blueprint revision #7652 foward by 2 mm

      My extensive experience with electronic design is if the Chinese say they'll give you a container full of old fashioned thru hole 1K resistors at a tenth of a penny each or whatever they will in fact do so. Maybe they painted the resistor color code with lead paint and the assembly line workers are political prisoners, but the resistance and power dissipation specs will be more or less as per the data sheet. And you can talk the Indonesians into providing a container full of microwave medium power bipolar transistors with a Pd of one watt and a Ft of 25 GHz for two bucks each and they will in fact do it. But god help you if you tell both of them, "I'd like a class A biased driver amp assembly so you two kids cooperate mkay?" Now multiply that by one zillion subcontractors all operating more or less without adult oversight by design to save money as a new project management technique, and you've got a recipe for disaster.

      "I've got an idea, lets improve the obvious metrics, then you little guys can work together to design and build it which will make me a bunch of money, mkay?" That stuff doesn't fly.

      --
      "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
    4. Re:No specs? by OzPeter · · Score: 3, Interesting

      Isn't that how all modern airliners are created? I'm trying to remember a documentary about the development of the A380, IIRC the entire thing was designed in CAD, the factories were then tooled and the first plane was created from parts from the same production line as the production run would come from.

      The Airbus also suffered from manufacturing problems as the German and Spanish facilities were using a different version of the CATIA CAD tool than the English and French facilities. This resulted in hilarity when modules from different locations did not mate as intended.

      --
      I am Slashdot. Are you Slashdot as well?
    5. Re:No specs? by DerekLyons · · Score: 5, Insightful

      Any manufacturer worth their salt (and Boeing is one) should be able to fully prototype their product prior to outsourcing bits of its production (except for things like batteries). But any parts that needs to fit together very precisely should be prototyped first.

      Welcome to the 21st century, I hope your trip from 1950 was a comfortable one! Here in the 21st century computers have advanced to the point where we don't need to physically prototype things anymore - it can, and has been, done digitally for well over a decade now. In fact, one of the most complex things that man is currently building (a nuclear submarine, something else new to you but take my word on it) are now routinely and successfully designed and built without any physical prototype.
       
      Seriously - you and a bunch of other commenters are utterly clueless as to the state-of-the-art of over a decade ago. Boeing has built (IIRC) three new aircraft now (plus major upgrades like the new 747) using completely digital design, visualization, and validation tools. While it's not entirely a mature technology, it's not new and very complex vehicles are and have been in service for years that were designed and built using it.
       
      Prototyping persists with smaller items because the requisite systems and software are so expensive, and is enabled by the fact that the teams involved are relatively small and simple, physically located in one place, and the prototypes are relatively cheap and new ones can be turned around (at worst) in a few weeks. On the other hand, a mockup/prototype of something like a nuclear submarine or a major aircraft can cost tens of millions of dollars (or more) and take a year or more to assemble. To assume that the latter must prototype because the former do is... ludicrous at best.
       
      The problem here isn't lack of prototyping, it appears that they tried to extend the process too far and the management systems weren't weren't set up properly to handle the new process.

  2. Interesting problem by astralagos · · Score: 3, Insightful

    Systems design in engineering basically involves drawing a box around a bunch of parts and saying "this is a system". The interfaces after that are hopefulyl clean -- good systems design does that, but implicit in the choice of a system breakdown is efficiency loss. I might not, for example, think about the fact that the giant engine at the heart of my car could also run heating. There's this long term conflict in engineering between the need to abstract, which enables all forms of delegation, including outsourcing, subcontracting and even building teams, and the loss of efficiency. Good engineers learn things at an almost inexpressible level,developing jargons for the systems under their purview -- in the case of Boeing, there was literally one guy who was their expert on cabling. If you wanted to submit a drawing change, he could envision the change in the cabling of the plane and whether the change was physically possible. That's always been the bane of system abstraction - you find these things that have to cross systems and, if you don't recognize them early enough, they come back to bite you in all sorts of creative ways. Kelly Johnson was a big believer in this. His rules for skunkworks explicitly required that engineers had to be within a specific number of feet of the shop floor -- that way they weren't too divorced from the reality of the products they were making. You see this in the design of a lot of the early computer systems as well, parts bolted together in weird ways before we started developing this high-level view of what systems actually made up a computer.

  3. C'mon, losers, we solved this in the 70's! by fuzzyfuzzyfungus · · Score: 4, Funny

    Obviously, Boeing should simply have specified that all the contractors deliver components that accept and output plaintext, and then used pipes and awk to cobble the pieces together into a working system! What could possibly go wrong?

  4. Engineering by the+eric+conspiracy · · Score: 3, Insightful

    Boeing didn't want to hire all the engineers needed to design the 787. So when they outsourced these subsystems they also counted on their suppliers to do the engineering of these subsystems.

    The problem is that engineers are not fungible. Boeing didn't appreciate this, any more than the software industry did when it started outsourcing.

    An aerospace structural frame engineer is not the same thing as a marine structure engineer. There are huge differences in the body of experience despite the fact that they both use the same tools.

    This was the primary cause of the delays Boeing had. It will continue to be a problem for anyone who tries this sort of outsourcing.

  5. Re:Lithium batteries considered dangerous by SternisheFan · · Score: 3, Interesting
    Elon Musk of Tesla Motors agrees with the un-safeness of these batteries.

    ..." Musk, who has run Tesla Motors [NSDQ:TSLA] for several years, laid out his thoughts on battery design in a detailed e-mail to the website Flightglobal.

    In it, he termed the architecture of the GS Yuasa battery packs supplied to Boeing "inherent unsafe," and predicted more fires from the same causes due to its design.

    Specifically, Musk criticized the use of large-format lithium-ion cells "without enough space between them to isolate against the cell-to-cell thermal domino effect."

    He also noted that when thermal runaway occurs in the larger cells, more energy is released by the single cell than comes from a small-format "commodity" cell, of the type used by the thousands in Tesla battery packs.

    And he went on to highlight what he viewed as the dangers of batteries using those large-format cells, saying they have a "fundamental safety issue" because it's harder to keep the internal temperature of a large-format cell consistent from the center to the edges.

    Not surprisingly, Mike Sinnett--Boeing's chief engineer for the 787 project--counters that the company designed the pack to cope with not only a single cell failure but to contain runaway thermal events as well."

    http://www.greencarreports.com/news/1082007_tesla-ceo-musk-boeing-787-batteries-inherently-unsafe

  6. Boeing by Anonymous Coward · · Score: 5, Interesting

    There was a short article on the Dreamliner in the latest New Yorker magazine REQUIEM FOR A DREAMLINER? . Quote Surowiecki :The Dreamliner was supposed to become famous for its revolutionary design. Instead, it’s become an object lesson in how not to build an airplane.
    To understand why, you need to go back to 1997, when Boeing merged with McDonnell Douglas. Technically, Boeing bought McDonnell Douglas. But, as Richard Aboulafia, a noted industry analyst with the Teal Group, told me, “McDonnell Douglas in effect acquired Boeing with Boeing’s money.” McDonnell Douglas executives became key players in the new company, and the McDonnell Douglas culture, averse to risk and obsessed with cost-cutting, weakened Boeing’s historical commitment to making big investments in new products. Aboulafia says, “After the merger, there was a real battle over the future of the company, between the engineers and the finance and sales guys.” The nerds may have been running the show in Silicon Valley, but at Boeing they were increasingly marginalized by the bean counters.

    Read more: http://www.newyorker.com/talk/financial/2013/02/04/130204ta_talk_surowiecki#ixzz2JTGx7SPc

  7. Bad drawings are the rule not the exception by sjbe · · Score: 5, Informative

    When it comes to mechanical parts, geometric dimensioning and tolerancing is a solved problem. When it comes to electrical interoperability, one'd think that's a solved problem as well.

    "Solved problem"? HAAHAHAHHHAAHHAAAAA....

    You don't manufacture things for a living do you? I run a company that makes wire harnesses. We're a contract manufacturer - we don't design things, we just take prints and build what is on the prints. I can count on my fingers on one hand the number of prints we have gotten from customers which were correct and sufficiently detailed such that the product could be built without asking any questions. There pretty much always are critical details left out of the prints. About 2/3 of the prints we see have incompatible parts specified. About half are missing at least one important dimension such as length. About 10% have missing parts and about 25% have incompatible parts. About 20% specify needlessly expensive parts like gold plated terminals that cost more but provide no actual performance benefit. Most of them leave off at least one critical tolerance. I've even seen drawings with dimension in inches and tolerances in metric.

    Why does this happen? For the most part because an alarming number of engineers doing the drawings aren't actually very good at their job. Some of them are just plain lazy. The electrical engineers usually can specify a wire schematic but often have no idea whether something can actually be built or know much about industry standards. The more mechanical engineers (yes mechanical engineers can and do design circuits) tend to create bad designs and specify the wrong parts because they don't know any better. Sometimes they are trying to do a good job but they don't bother to consult manufacturing during the design process and they come up with a stupid design or something that is impossible to build.

    I have run into some good engineers but they are the exception.

  8. Re:This is what happens... by Wilf_Brim · · Score: 3, Interesting

    This. If you get the the bottom of TFA you see what really was driving the decisions about how to design and produce the 787. At the time of the critical decisions for the 787 the head honchos at Boeing were not really Boeing people (a corporation where the key competency for the last 60 years has been the production of profitable commercial airliners.) They came from McDonnell-Douglas, whose key competency was more in the production of military aircraft. The development process of current military hardware is intolerably broken. The old method of subcontracting the design of subsystems and then trying to get them to work together, then just getting more money from Uncle Sam when the result didn't work now results in the aforementioned F22 and F35 (the latter of which may never enter volume production, or at least some variants may not) because complexities have expanded, and costs have likewise increased exponentially. As it turns out, you can't do that with civilian airliners. There aren't friendly Senators and Representatives (whom you have paid off with campaign contributions and subcontractors in their district) to give you more money. And friendly Generals and Admirals (whom are expecting 6 and 7 figure jobs when they retire) who will accept your explanations why things aren't working correctly, and why it's going to be another 3 years to get their gizmo, which doesn't work quite as anticipated. You have shareholders who expect profit, airlines who expect a product in line with what they ordered and expect to pay, and regulators who do not take kindly to aircraft whose electronics bays burst into flames at odd times.