Slashdot Mirror

← Back to Stories (view on slashdot.org)

Crashing iPad App Grounds Dozens of American Airline Flights

Posted by Soulskill on from the have-you-tried-pushing-the-button dept.

infolation writes: American Airlines was forced to delay multiple flights on Tuesday night after the iPad app used by pilots crashed. Introduced in 2013, the cockpit iPads are used as an "electronic flight bag," replacing 16kg (35lb) of paper manuals which pilots are typically required to carry on flights. In some cases, the flights had to return to the gate to access Wi-Fi to fix the issue.

31 of 263 comments (clear)

  1. NB4 by Anonymous Coward · · Score: 5, Funny

    holding the plane wrong

  2. Shoulda run Linux by Anonymous Coward · · Score: 4, Funny

    or at least the Android variant thereof. Fools.

    1. Re:Shoulda run Linux by jellomizer · · Score: 3, Funny

      Let the Zealots have their fun.
      Lets really ignore the fundamental architecture between Android (Linux) and iOS (BSD) Are actually very close in design.

      --
      If something is so important that you feel the need to post it on the internet... It probably isn't that important.
  3. Wow ... by gstoddart · · Score: 5, Interesting

    Now there's a technology fail for you.

    Reminds me of a US naval ship being towed to shore because Windows NT crashed.

    I guess this is a problem when you have consumer technology being used in mission critical environments.

    --
    Lost at C:>. Found at C.
    1. Re:Wow ... by bondsbw · · Score: 3, Insightful

      What part of gstoddart's post didn't you understand? I don't see any mention of iOS or the app store in that post.

      A bit overly defensive, eh?

      --
      All my liberal friends think I'm a conservative, all my conservative friends think I'm a liberal.
    2. Re:Wow ... by gstoddart · · Score: 4, Insightful

      What part of "no fucking kidding" don't you understand?

      I didn't say it was the exact same thing, I said it reminded me of a time when another epic technology fail caused a similarly huge cluster fuck.

      I don't give a crap what the crash was ... I care that a piece of technology barfed all over the place and left people sitting around going "what the hell do we do now?"

      When an airline has to halt operations because of something like their iPad crashing, that's a sure sign that someone hasn't really been doing a sufficient job of testing.

      I used to work on a project which dealt with people who do aircraft maintenance .. this is not an industry who collectively takes risks. But apparently their software vendor doesn't see it that way.

      --
      Lost at C:>. Found at C.
    3. Re:Wow ... by NatasRevol · · Score: 4, Interesting

      eFlight books were switched to to save millions of dollars in fuel costs every year. They're that heavy.

      --
      There are two types of people in the world: Those who crave closure
    4. Re:Wow ... by Richard_at_work · · Score: 5, Informative

      The US naval ship Windows NT crash meme is somewhat of a myth - there was a testbed ship (USS Yorktown CG-48) running an experimental ship management and integration system. The crash did indeed occur, but it had nothing to do with Windows NT and everything to do with invalid data being entered into the apps management system causing all linked systems to stop working. While everyone jumps on the "Windows NT" aspect of this, it would have happened under Unix as well.

    5. Re:Wow ... by Mr+D+from+63 · · Score: 3, Insightful

      Redundancy doesn't need to be hardcopies. Either bring backup I-pads, or better maybe a backup windows tab so you have both diversity and redundancy.

      Also, since this happened to many at about the same time, I assume an update or change was to blame. Don't update these unless there is a reason. And test if you do update, or keep a non-updated backup on hand until the update is proven reliable.

    6. Re:Wow ... by gstoddart · · Score: 3, Informative

      Because they were expecting to save millions in fuel from not schlepping them around.

      So bringing the physical copy would have been almost 40 pounds of crap, which would defeat the purpose of having the iPad.

      Not saying I agree with not having a backup. But I can see why airlines wanted to get rid of it.

      A little known fact about aircraft manuals ... pretty much no two are identical since the production of planes changes over the years, and they all have slightly different pieces and parts. So this 737 is unlikely to be identical to that 737.

      You cant' have one manual, you need one for each damned aircraft. Which is part of the appeal for having it in electronic form.

      --
      Lost at C:>. Found at C.
    7. Re:Wow ... by bobbied · · Score: 4, Insightful

      They DO have a backup...

      Both pilots carry IDENTICAL I-Pads.... What amazes me is that nobody thought of the single point of failure, the application the I-Pads run.. OOPS..

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
    8. Re:Wow ... by jellomizer · · Score: 4, Insightful

      At the time Windows NT wasn't a consumer technology, Windows NT was a serious contender in the server space, for mission critical systems. At that time Linux was still considered a Hobby OS. Other alternatives were Unix variant, but during those stages they weren't really that much better. It was just when we heard that Windows NT crashed, we all laughed at it, because of allegiance towards Linux.

      However today... Consumer technology today Windows, Android, iOS. Are really based on Professional Server Grade OS Kernels. They are just running on cheaper hardware.

      The issue for this isn't blaming the iPad or iOS but the maker of that App for those documents. They screwed up, This would have happened if they had a Million Dollar professional system in their hands too.

      --
      If something is so important that you feel the need to post it on the internet... It probably isn't that important.
    9. Re:Wow ... by gstoddart · · Score: 3, Interesting

      Yeah, how they failed to have two distinct sets which are never updated at the same time eludes me.

      That just pretty much guaranteed it would eventually go wrong on them.

      --
      Lost at C:>. Found at C.
    10. Re:Wow ... by Dutch+Gun · · Score: 3, Insightful

      Those flight bags, at 35 lbs, were also very uncomfortable for the pilots to lug around. I remember how heavy a backpack full of textbooks was as a student, and wouldn't wish to repeat that experience at my age, which is still younger than many pilots. I wouldn't be surprised if the pilots were pushing for this as well.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    11. Re:Wow ... by Megane · · Score: 3, Interesting

      I think that's a likely cause. I doubt they're updating the app (executable) on a regular basis and pushing the update, when it's only the data that changes regularly. All it takes is one glitch in a weekly data update, and one bad switch statement to cause a program to crash.

      Proper error handling is one of the most important things in keeping things running (especially in unattended systems), but one of the harder things to get right, because it's hard to test (as in QA) for every possible unexpected input. You have to get a bit paranoid with your coding, because garbage input really is out to get you.

      --
      #naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
    12. Re:Wow ... by Megane · · Score: 4, Insightful

      And then someone has to print some of those books on a regular basis. And then someone has to dispose of them when they expire. And the pilots probably carry regularly updated information for every airport in the countries they might fly in, whether they go to that airport or not.

      --
      #naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
    13. Re:Wow ... by tlhIngan · · Score: 4, Informative

      I'm guessing that their solution will be to put Pilots and Copilots on different update schedules and also allow for the immediate roll back of any software updates by the user. Where I don't think having one application on one OS is necessarily all that risky, what cost them in this case was the inability for the pilots to roll back to the last version that worked right after an upgrade or grab a 'backup device" from the pilot's lounge if theirs is somehow messed up. Given that the issue is not safety but more about keeping the schedule here, I imagine that the logistical costs of their solution will be a primary consideration.

      No can do.

      The problem isn't the iPad. Or the application. It's that one particular updated doc caused a problem.

      And by flight regulations, EVERYONE has to carry the latest revision of the document. And every document is on a different update schedule.

      Some documents are changed only when there are updates. Other documents have fixed expiry dates and must be updated to the latest version before that.

      And at all times you must have the latest available updates - sure there's maybe a week of grace when the new edition comes out before the old edition expires, but that's about it.

      In the paper world, people were actually employed to go through all 35lbs of documents ensuring the latest versions of every page were present (pages are usually supplied as differences in binders, so you remove the old page and stick in the new page. Pages were versioned (typically by date) and there's often a cover sheet saying what's the latest version of each page (updated every time there's an update).

      Of course, if you have hundreds of pilots each having to do this, eventually the human version of patch(1) will screw up, so you need to double check for this.

      It's why EFBs have been so widely embraced - not having to have someone check 35lbs of documents practically daily, not having to have a whole infrastructure set up to distribute updates, not having to spend time updating documents, etc, it's a terrible chore.

      In fact, given the number of updates and how long it's been going on, it's surprising it's only happened once that an update screws up - I'm sure in the past with paper it happened dozens or hundreds of times a day because updates happen that often, usually to different subsets of the pilots.

    14. Re:Wow ... by jittles · · Score: 3, Insightful

      The update was likely to some item of data the application uses, not the application itself.

      I don't see how you have any information to support this assumption. I'd guess its more likely an IOS update and some resulting incompatibility. Updating the data itself is probably the least likely change to cause error. Also, that data probably doesn't change very often, so it would have been pretty obvious if that were a root cause.

      The evidence to suggest that it was a data change and not an application update was that their entire fleet of 737's was down. There was no report of another airframe being affected. Also, the data does change on a regular basis. From the FAA:

      if your chart is more than 8 weeks old it probably isn't current. Aeronautical information changes frequently; more frequently than every six months. That is why we publish the Aeronautical Chart Bulletin in the back of each Airport/Facility Directory (AFD) every 56 days and why it is important that you consult the Notices to Airmen (NOTAMs) prior to each flight.

    15. Re:Wow ... by fustakrakich · · Score: 3, Insightful

      I'm boggled by the fact that you flyboys brought only ONE type of tech onboard for this map stuff.

      Maybe your argument is with Jeppsen. They've had a pretty big monopoly for a very long time now.

      --
      “He’s not deformed, he’s just drunk!”
    16. Re:Wow ... by bobbied · · Score: 3, Informative

      The emergency handbook for the aircraft isn't the issue here, it's the maps and approach plates which are constantly changing and must be kept current. The maps are legally required to fly IFR so it's part of the checklist before you kick the tires and light the fires you make sure you have the necessary maps and approach plates for your destination and alternates.

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
  4. Before we start blaming or laughing at Apple... by JaredOfEuropa · · Score: 5, Interesting

    Let's see these AA iPads and the software for what they really are: pieces of business-critical software / hardware. Which means that they have to treat it like any other combination of business critical software and hardware. The entire configuration is frozen, software, OS, patches and all, and any change is thoroughly tested before it is pushed to the production devices.

    So what happened? One news item hints at a recent update causing the issue. Where did the update come from? Was iOS updated, or the app? Was this update tested before being rolled out?

    --
    If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
  5. cost recoup by ftobin · · Score: 3, Insightful

    I wonder how long it takes to recoup the cost of this disruption by continuing to carry lighter manuals.

  6. Re: Why do they not have the paper as backup? by davidwr · · Score: 4, Insightful

    Weight.

    --
    Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
  7. as an executive maybe i can clarify. by nimbius · · Score: 5, Funny

    Here at american, we know you've come to expect the broken traytables, rotted seatback pockets, and permanently reclined seating prominently featured on our aging reagan-era Boeing fleet. We know none of you understand what the hell a gold line american star alliance partner is, but are well aware it means you're about to board a 42 seat brazillian rust-bucket with misaligned landing wheels and a weird styrofoam smell. Each year we add more rare earth metals and precious gems to our flight upgrade programs in an in incorrigible effort to confuse and infuriate weary passengers. What is Americium? Shouldnt platinum be more worthy than sapphire? who knows, who cares. We recognize your supreme discomfort at 4 AM as our cancelled connector to newark hobbles mercifully into the hanger for 20 years of well-earned repair to be condensed into 9 minutes of speed tape and air fresheners. We know you choose American because our 35 year old concourse seating has gone from suede to patent leather from use, and its foam long since evaporated to a fine haze of formaldehyde. And we, American, appreciate your undying commitment to sit in an airplane that smells canned soup and farts while futile attempts to adjust your weight merely prolong your encounter with the threadbare frame of a seat no more comfortable than a bus stop bench. But we cannot sacrifice our commitment to swiping, clicking, and tapping on a device that makes our second hand aeroflot cockpits look like modern museums to supercomputing and hence have cancelled numerous flights.

    --
    Good people go to bed earlier.
  8. Why such crap? by aaaaaaargh! · · Score: 3, Insightful

    What do they put on it? Checklists? Airport charts? Or even approach/departure charts? What if it crashes during taxiing on a busy airport? What if it crashes in the middle of a complicated approach procedure? What if it crashes during checklist and the pilots forget to check a point?

    In other words: Why would anyone use cheap crap such as an iPad in a professional passenger airplane? How stupid is that?

    1. Re:Why such crap? by FooAtWFU · · Score: 4, Insightful

      Do you have a better and more reliable tablet system in mind, or are you suggesting that they should have stuck with the 35-pound suitcases full of printed material?

      --
      The World Wide Web is dying. Soon, we shall have only the Internet.
    2. Re:Why such crap? by gstoddart · · Score: 5, Insightful

      I could have designed and built a system in probaby 1/10 the time it took for them to PAY OFF APPLE and buy those shiny shitslates.

      they used consumer grade 'auto updatable' fashon accessories for mission critical things

      Horseshit. You are completely talking out of your ass.

      Because they sure as shit didn't do this without approval from the FAA:

      The iPad has been used in General Aviation in conjunction with its paper backup counterpart, which is mandated by the Federal Aviation Administration (FAA). There are many applications available which include everything that would be on the paper charts plus aviation tools including navigation charts, taxi procedures, weather maps, GPS, Minimum Equipment List, Company Policy Manual, Federal Aviation Regulations and flight controls. Although these tools have been used in the private sector, the use of the iPad in commercial aviation is just taking flight.

      The Federal Aviation Administration finished a three-month testing project which included putting the device thru adverse conditions such as rapid decompression testing and tests to make sure the tablet did not interfere with the avionic equipment. Early in 2011 the FAA authorized charter company Executive Jet Management to use iPad records without the backup paper charts.[1] This helps make way for the iPad to become an aviation instrument for the rest of the industry. Alaskan Airlines,[2] Delta Air Lines,[2] and American Airlines[3] planned test programs.

      Why must everybody on Slashdot keep acting like they could whip up a half-assed solution in a week, or that regulated industries just make shit up as they go?

      The reality is, this has not a fucking thing to do with paying off Apple or a hastily thrown together solution.

      This sounds entirely like an update from the vendor was poorly tested. In which case, they have some lessons to learn about working in that industry -- which is about as risk averse as you can get. Precisely because the FAA holds them to a very high standard.

      But, hey, don't let reality get in the way of your claims you could do a better job in your pajamas.

      --
      Lost at C:>. Found at C.
    3. Re:Why such crap? by ColdWetDog · · Score: 4, Funny

      I blame this mind set on the Avenger movies. If Scarlet Johansson can save the world in while running around in spandex underwear, your average Slashdot coder should be able to outperform a couple dozen programmers, managers and QA staff with just a six pack of Mountain Dew and a jumbo bag of Doritos.

      --
      Faster! Faster! Faster would be better!
  9. It is a Jeppesen software failure by Anonymous Coward · · Score: 5, Informative

    Electronic Flight Bag (EFB) software is an essential tools for aviation. One iPad can handle multiple charts, maps, and devices which would can weight of more than 20 lbs. Jeppesen software is the American Airlines is the corporate EFB software. A recent update crashed. The Jeppesen tool is a well known company and has Aerospace level of testing. It still failed. There are other EFB tools out there. This has nothing to do with WiFi and everything to do with software development.

  10. USS Yorktown crash nothing to do with Windows NT? by DougPaulson · · Score: 5, Informative

    Software glitches leave Navy Smart Ship dead in the water

    a database overfow error (resulting from a divide by zero operation) caused the ship's propulsion system to fail.

    The Yorktown Affair

  11. Re:Android, iOS, and Windows Atom by ShanghaiBill · · Score: 5, Informative

    It seems reasonable to have three tablets on the flight deck, running iOS, Android, and Windows 8 for Atom.

    The app crashed, not the OS. So having multiple OSes may help in some situations, but not in this one. Some mission critical applications are implemented by two teams working independently. Since this app is basically just a PDF reader with a customized menu, that should not be difficult.