Slashdot Mirror

← Back to Stories (view on slashdot.org)

Boeing To Make Key Change in 737 MAX Cockpit Software (wsj.com)

Posted by msmash on from the up-next dept.

Boeing is making an extensive change to the flight-control system in the 737 MAX aircraft implicated in October's Lion Air crash in Indonesia, going beyond what many industry officials familiar with the discussions had anticipated. From a report: The change was in the works before a second plane of the same make crashed in Africa last weekend -- and comes as world-wide unease about the 737 MAX's safety grows. The change would mark a major shift from how Boeing originally designed a stall-prevention feature in the aircraft, which were first delivered to airlines in 2017. U.S. aviation regulators are expected to mandate the change by the end of April.

Boeing publicly released details about the planned 737 MAX software update late Monday [Editor's note: the link may be paywalled; alternative source]. A company spokesman confirmed the update would use multiple sensors, or data feeds, in MAX's stall-prevention system -- instead of the current reliance on a single sensor. The change was prompted by preliminary results from the Indonesian crash investigation indicating that erroneous data from a single sensor, which measures the angle of the plane's nose, caused the stall-prevention system to misfire. Then, a series of events put the aircraft into a dangerous dive.

11 of 211 comments (clear)

  1. I guess the incredibly obvious question is... by ZorinLynx · · Score: 5, Interesting

    Why the hell wasn't this the case before?

    Aren't flight control systems supposed to be triple-redundant anyway? Everything I've read about them says they are; three systems and if there is incorrect data it uses the two that agree.

    1. Re:I guess the incredibly obvious question is... by lgw · · Score: 4, Interesting

      I assume they're talking about the sensor behind the pitot hole here. Making that the only sensor, and non-redundant, is particularly questionable. It's well known that pitot holes are very easily thrown off: an insect building a nest inside it (or ice forming, or etc) will throw off the sensor enough to crash a plane, if it's all you rely on.

      --
      Socialism: a lie told by totalitarians and believed by fools.
    2. Re:I guess the incredibly obvious question is... by bobbied · · Score: 5, Interesting

      Why the hell wasn't this the case before?

      Aren't flight control systems supposed to be triple-redundant anyway? Everything I've read about them says they are; three systems and if there is incorrect data it uses the two that agree.

      Well.. I believe the way the system works allows the control inputs of the pilots are able to overcome anything the system does. It's basically like an autopilot, where the pilot can override the system by applying pressure to the controls. This system is designed to apply backpressure as the aircraft approaches a stall, making it harder for the pilot to continue to increase the angle of attack and hopefully avoiding the stall. So you can still stall the aircraft, just pull harder and keep increasing the AOA...

      The problem though, is that pilots are conditioned to change the trim to deal with unusual pressures for the desired pitch angle. So if the system believes the sensor and it's saying "STALL" but you are actually not, the system applies pressure to lower the nose, which the pilots will be conditioned to trim out. IF the stall doesn't go away, the system keeps the pressure there and unless the pilots realize what's going on they will keep trimming nose up. Eventually, the process ends up with an aircraft that's severely out of pitch trim which will be very confusing to the pilots, with really high control pressures required to do anything to the pitch. Thus "control problems" seems to describe exactly what I imagine was going on. It was a vicious cycle that makes the aircraft really hard to control.

      So, I understand the engineering and using one AOA sensor. Kind of makes sense... Hey, the pilots can just override this anyway, we are stopping them from actually stalling the aircraft, just making it harder to do. We've don't this before in fighter aircraft and other fly by wire systems w/o any problems. But I think there wasn't enough thought given to what happens when that sensor fails and if they can implement some cross checks between airspeed, rate of climb, rate of turn, they might be able to more gracefully fail the system and disable it, or at least not get into the vicious cycle that leads to a pitch trim issue.

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
    3. Re:I guess the incredibly obvious question is... by Solandri · · Score: 4, Interesting

      Usually there are 3+ pitot tubes. Looks like the 737 has 5, with 3 of them dedicated to measuring airspeed. It's incredibly rare that a single fault causes a crash. Reporters just like to write up their stories that way to give their stories more impact, even if it twists the truth.

      This isn't the first time faulty airspeed readings led to a flight computer has led to a crash. It isn't even the second time. In all previous cases, the plane was flyable. It was the confusion as the pilots tried to diagnose the problem based on the bizarre behavior of the plane and the flight control software and alarms which doomed the flights. It requires a deep and thorough understanding of when different flight protection modes in the software are triggered and kick in, to work backwards from the behavior you're seeing, to what problem(s) could be triggering those modes. If you've debugged software, you've encountered this. Unlike natural laws like physics, software can be designed arbitrarily. So your intuitive feel for how things should work becomes useless for tracking down the problem. You're totally dependent on how thoroughly you understand the software's arbitrary design.

      Bear in mind that the stall warning is pretty much a "you're gonna die if you ignore me" warning. So it takes quite a bit of convincing before pilots will decide it's the warning that's faulty, not something else that they're doing wrong. That may be the cause of the reluctance of pilots to simply shut it off and fly the plane "by the seat of their pants" based on how the throttle settings, altitude, and attitude. So while theoretically the stall warning triggering incorrectly is a recoverable problem, it may take pilots a long time to diagnose and clear up the problem. Long enough for the plane to crash.

    4. Re:I guess the incredibly obvious question is... by Anonymous Coward · · Score: 2, Interesting

      This is what happens when you stop expecting your companies to compete on the free market and instead protect them with a combination of defacto and real terms state aid, such as trying to destroy competition such as Bombardier with illegal trade acts.

      As soon as you let your companies stop competing and instead give them a position of immunity, determine them too big to fail, and no longer deemed in need of competing on the free market, then they'll get lazy, they'll get incompetent, and shit like this will happen.

      Boeing desperately needs to face real competition, and stop being protected by an protection racket artificially created by US government protectionism, and that extends to military contracts too; it can't keep just getting given them on a plate even when it's the worst option.

      It's sad that people have to lose their lives for the growing complacency of companies like Boeing to be unveiled and tackled. Even now it's refusing to admit any real fault, claiming the aircraft is still safe when it's very clearly not.

    5. Re:I guess the incredibly obvious question is... by uncqual · · Score: 5, Interesting

      My lay person's understanding...

      In order to increase fuel efficiency on the 737 MAX, the engine fan diameter was increased. These "underwing" engines would have been too close to the ground if mounted as on other 737 models. Thus, the engineers moved the engines forward and upward to achieve necessary ground clearance. This, along with some other changes, moved the force of thrust forward which made the plane more prone to lift its nose too high and stall. To guard against this, Boeing introduced the Maneuvering Characteristics Augmentation System (MCAS) which activates automatically when the autopilot is off in some conditions which include when the angle of attack (AOA) is too high. The MCAS system, when needed, attempts to prevent a stall by adjusting the horizontal stabilizer trim upward and will do this over, I believe, about 10 seconds or until the pilot overrides it or the angle of attack is within limits. If the pilot activates the trim control switch on the yoke, MCAS will be disabled -- but, five seconds after the switch is released, MCAS will reengage if the conditions call for it (esp. AOA). When MCAS is altering the trim, the manual trim controls on each side of the center "console" will be spinning away and, if a pilot looks down, they will see that motion as there is a white stripe extending outward from the center in order to make the movement obvious.

      The best speculation I've heard about the Lion Air crash was that there was a problem with one of the AOA sensors. There are two such sensors - one on both side of the 737 Max.

      As in most crashes, due to the redundancy of systems and procedures, it's rarely one thing that causes a crash but rather a cascade of events.

      There had been problems with at least one of the AOAs on previous flights but maintenance attempts appear not to have solved the problem. So, first there was a failure of maintenance, but of course AOA sensors will fail from time to time, so one can't blame the crash on that failure.

      I've not heard how MCAS handled conflicting AOA sensor readings but I suspect this is one of the big areas of change that they will push in the April "patch". But, it's likely that the failing AOA caused the MCAS to activate when it shouldn't have and push the nose down by adjusting the trim - but this actually pushed the plane's nose down too far. When the pilots tried to correct, they ended up disabling MCAS (although perhaps not explicitly aware that they were doing so) only to have it start undoing what they had accomplished five seconds after they released the trim control on the yolk - and this was a vicious loop.

      Had the pilot recognized what was happening, they simply would have ran the "runaway trim" procedure (which would have disabled MCAS and some other automatic trim controls completely via a switch on the center "console") and flown the plane manually with no problems. Unfortunately, the pilots likely didn't figure out what was causing the problem and failed to execute the necessary procedure. So, that was a pilot error (and, that's probably what will be determined to be the main problem here, with contributing factors).

      There is much debate on why the Lion Air pilots may have failed to recognize what was going on. Many pilots and their union claim that they were not told about the existence of MCAS. Boeing hasn't been talking a lot, but they seem to assert that there was no need to train the pilots on MCAS beyond what the manuals/training did as it was a classic "runaway trim" scenario and the training was sufficient to cause the pilots to detect that case and initiate the proper procedure. Boeing did, however, issue documentation updates to operators worldwide soon after the Lion Air crash.

      After Boeing issued the documentation updates, every 737 MAX pilot should have been fully aware of MCAS and what to do if was doing the wrong thing. This, coupled with the witness reports that the Ethiopian Airline 737 MAX that crashed was spewing smoke and fire from the back of the plane a

      --
      Why is there an "insightful" mod and why isn't it "-1"? If I wanted insight, I wouldn't be reading /.
    6. Re:I guess the incredibly obvious question is... by caseih · · Score: 3, Interesting

      The MCAS spins the same trim knobs that the pilot spins. So the pilot can trim the nose back and after MCAS spins it down. They might fight each other, but ultimately they are both adjusting (and potentially undoing) the same thing. I'm sure it's initially confusing to pilots for sure, especially because older planes would cancel the automatic trims when the stick was pulled on, but apparently this is not the case with MCAS. If it turns out the MCAS contributed to the Indonesian crash, then it was a matter of training. But Boeing screwed up the design.

  2. We joke, but I never thought it would come to this by Anonymous Coward · · Score: 0, Interesting

    The software industry has become a hot mess of unleashing garbage beta code onto the world, and figuring that they can finish it later and fix everything with patches. Being fast to market trumps all other considerations. It seems like this mindset has made it into the world of aviation finally, which you would expect might be the last bastion of common sense and doing things carefully and correctly. And people are paying for it with their lives by the hundreds. If there is not a billion dollar lawsuit against Boeing over this, something is very, very wrong.

  3. Next one will crash due to stall by jfdavis668 · · Score: 3, Interesting

    That would have been prevented by the current system.

  4. Re:The Tesla People by PvtVoid · · Score: 4, Interesting

    It's funny how they point to aviation as nearly infallible when they talk about self-driving cars.

    Self-driving cars don't have to be infallible. They just have to be safer than the average person, which is a really low bar.

  5. Re:seems like the logic here is flawed. by BostonPilot · · Score: 5, Interesting

    No, you're trying to grossly oversimplify the problem, and it's causing you to say things that are silly.

    Having worked as a vendor to the avionics group at Boeing, and having had a student who wrote test code for the 777, I can tell you that the testing / verification process for their software is mind boggling. They've had decades to fine tune their processes for creating reliable computer software. Believe me, you sound idiotic second guessing them, and it doesn't sound like you're a pilot either...

    The one thing I will agree with you about is that the system should trust the crew. However, I must say that some of my airline captain buddies would strongly disagree with that. Just look at Air France Flight 447 as a perfect example of why trusting the crew can go wrong. However, I still lean towards this... if you don't trust the crew then it's like the old joke about the perfect crew:

    The ideal flight crew is a pilot and a dog.

    The pilot is there to feed the dog, and the dog is there to bite the pilot if he touches anything.

    Seriously, if the automation is so complicated and opaque that the crew can't tell what it's doing and why... that's a problem. The move towards more automation seems to be to make up for an inexperienced crew... I think more training / sim time is the right solution, not more automation. Still, both Airbus and Boeing seem to think more automation is the right way to go.

    I'll be interested to hear what they learn from the FDR...