'IT Issue' Grounded All United Airlines Flights In The US (nbcnews.com)

← Back to Stories (view on slashdot.org)

'IT Issue' Grounded All United Airlines Flights In The US (nbcnews.com)

Posted by EditorDavid on Sunday January 22, 2017 @02:14PM from the no-fly-zone dept.

For two and a half hours -- no take-offs. An anonymous reader quotes NBC News: All of United Airlines' domestic flights were grounded Sunday night because of a computer outage, the Federal Aviation Administration said as scores of angry travelers sounded off on social media... U.S. officials told NBC News that the Aircraft Communications Addressing and Reporting System, or ACARS, had issues with low bandwidth. No further explanation was immediately available for what United described only as "an IT issue."
An hour ago United tweeted that they'd finally lifted the stop and were "working to get flights on their way." 66 flights were cancelled just at Chicago's O'Hare Airport, the Chicago Department of Aviation told the Associated Press, and though the article doesn't identify the total number of flights affected, "Chicago-based United Airlines and United Express operate more than 4,500 flights a day to 339 airports across five continents."

58 of 117 comments (clear)

Min score:

Reason:

Sort:

No Take offs? by rossdee · 2017-01-22 15:13 · Score: 3, Funny

Well at least thats better than no landings
1. Re:No Take offs? by Scoldog · 2017-01-23 15:01 · Score: 1
  
  All planes will land eventually land, it'll just depend if the landing was survivable.
  
  --
  This space for rent
Windows 10 Auto Reboot by zenlessyank · 2017-01-22 15:24 · Score: 5, Insightful

Gotta love rebooting when MS wants you to reboot.
1. Re:Windows 10 Auto Reboot by ole_timer · 2017-01-23 03:25 · Score: 1
  
  ACARS was used to track MH370's flight path curve and was used in the investigation Malaysia Air was too cheap to buy the service so more details were not available such as GPS location.
  
  --
  nothing to see here - move along
2. Re:Windows 10 Auto Reboot by ole_timer · 2017-01-24 01:58 · Score: 1
  
  yep.
  
  --
  nothing to see here - move along
One more reason... by jcr · 2017-01-22 15:26 · Score: 1

Why UAL is my last resort airline. I will only use them when there is no alternative.
-jcr

--
The only title of honor that a tyrant can grant is "Enemy of the State."
1. Re:One more reason... by hambone142 · 2017-01-22 15:38 · Score: 1
  
  I hear ya. Charging to use the overhead bin really took the cake. Add to that, no reserved seats for cheapo fares until the last minute (potentially separating you and your partner), rude folks behind the counter and rude old bags for flight attendants.
  They are my last resort also. The merging of two bags of shit (American Airlines/UA) in to one large bag of shit.
2. Re:One more reason... by Anonymous Coward · 2017-01-22 15:58 · Score: 1
  
  .... You do know they merged with Continental not American right? American is still its own entity and the two airline's inner workings could not be more different.
3. Re:One more reason... by dunkelfalke · 2017-01-22 21:53 · Score: 3, Funny
  
  Dave Carroll, is that you?
  
  --
  "It's such a fine line between stupid and clever" -- David St. Hubbins, Spinal Tap
4. Re: One more reason... by ColdWetDog · 2017-01-23 03:41 · Score: 2
  
  If you need 5 cubic feet of batteries and drugs on your flight ... well, I sure hope to hell you are having a good time.
  
  --
  Faster! Faster! Faster would be better!
What is up with airlines IT structure by SuperKendall · 2017-01-22 15:29 · Score: 4, Interesting

This is not the first airline this has happened to, I think now something like three airlines in about a year? How on earth do all of these separate companies have the same problems where ANY breakage of the system mean planes with schedules pre-determined ages ago cannot fly? Is there some kind of Intuit Turbo Airline Manager software they all run??
WTF!
This is probably the strongest demonstration yet that we are all living in a computer simulation.

--
"There is more worth loving than we have strength to love." - Brian Jay Stanley
1. Re:What is up with airlines IT structure by l0n3s0m3phr34k · 2017-01-22 15:35 · Score: 5, Informative
  
  Well, probably because this back-end system, is managed by what used to be Hewlett Packard Enterprise. They've had so many layoffs, and shuffled from company to company so many times, that the people who actually know what they are doing are all long gone.
2. Re:What is up with airlines IT structure by Anonymous Coward · 2017-01-22 16:08 · Score: 3, Informative
  
  Actually capitalism does work. Airline accidents are at an all time low even with much higher passenger miles, costs for a ticket are accessible to most people, delays and lost luggage aren't nearly as common, and there are plenty of choices in scheduling a flight.
  But you meant something else, and I proved you wrong. How does it feel to be a socialist and wrong?
3. Re:What is up with airlines IT structure by dbIII · 2017-01-22 16:13 · Score: 1
  
  One amusing thing that happened near me was an airport IT failure not an airline one. Several airports under the same owner in different states ended up with all baggage handling operations run from one site. So when the obvious happened and two backhoe incidents took out both connections one airport had baggage conveyors that could not be operated either locally or manually.
  If someone put all these recent airline/airport failures in a book and sent it back to 1970 they would think it was a satire.
4. Re:What is up with airlines IT structure by Anonymous Coward · 2017-01-22 16:16 · Score: 1
  
  Airline accidents are at an all time low
  The FAA and NTSB say ... you're welcome.
5. Re:What is up with airlines IT structure by dbIII · 2017-01-22 18:02 · Score: 4, Funny
  
  What you attach to a backhoe is your own business.
6. Re:What is up with airlines IT structure by dunkelfalke · 2017-01-22 20:50 · Score: 4, Insightful
  
  How exactly airplane accidents being at an all time low is a result of capitalism? Are you one of these crazy people who worship capitalism as a deity?
  Airplane accidents don't happen that often anymore because of strict regulations and aircraft being generally more intelligent. Capitalism has directly caused a lot of accidents, like Alaska Airlines Flight 261 (airline was too cheap for proper maintenance), Turkish Airlines Flight 981 (manufacturer was too cheap to fix a known design error), American Airlines Flight 191 (again, airline too cheap to do proper maintenance), JAL Flight 123 (yep, again maintenance) and so on. Yay capitalism. Same goes for delays and lost luggage, by the way. Strict regulations making it difficult for the airlines to weasel themselves out have helped, not capitalism.
  
  --
  "It's such a fine line between stupid and clever" -- David St. Hubbins, Spinal Tap
7. Re:What is up with airlines IT structure by im_thatoneguy · 2017-01-22 21:16 · Score: 3
  
  How on earth do all of these separate companies have the same problems where ANY breakage of the system mean planes with schedules pre-determined ages ago cannot fly?
  
  Because all of them depend on software to handle their flight scheduling. Imagine this scenario: a plane lands 30 minutes late into Denver with 200 passengers. 50 of them will now miss their flight. Those 50 passengers are going to set off a cascade of modifications automatically to hundreds of dependencies. No human being can keep thousands of flights and millions of passengers in their head at once. There used to be a lot more slop in the system and margin for error. Now a plane delayed landing is almost certainly a plane delayed departing. The entire system has to minimize the damage by deciding whether it makes sense to delay a handful of flights to ensure they make it or attempt to accommodate them on later flights. And if they delay those later flights how will that impact all of the passengers on those flights, etc etc etc. What about that storm in Chicago which will undoubtedly delay any aircraft who don't leave *now*? There are millions of variables and millions of dependencies and if the satellite tracking system goes down suddenly the system won't know if a plane is going to be on-time or if it'll be an hour late. The only safe conclusion is to just stop all traffic until everything is sorted.
  When you stop and think about it an Airline's IT system *IS* the company's day-to-day/minute-to-minute management. So if part of the system goes down in one region that will affect the whole system since a passenger currently in Tokyo very well may be flying to Singapore on a plane arriving from Denver. It's a global network of millions of interdependent pieces.
  Many airlines are taking measures to minimize these impacts. It used to be that a plane would leave New York, arrive in Chicago, leave Chicago, arrive in Phoenix, leave Phoenix arrive in Seattle, stay for the night and work its way backwards. If the plane was delayed along any stop everything would get delayed. Airlines are trying to reroute their networks so that a plane just circulates back and forth between 1-2 places in a day.
8. Re:What is up with airlines IT structure by tomhath · 2017-01-23 01:14 · Score: 1
  
  There are many different forms of capitalism. The most common is what's known as a mixed economy. It works quite well here in the US.
9. Re: What is up with airlines IT structure by 140Mandak262Jamuna · 2017-01-23 02:21 · Score: 1
  
  Capitalism is a vengeful God. It will smite any corporation that kills people in pursuit of profits, .... eventually, ... once it becomes readily visible to all consumers that "this company will kill to make a few cents", and in the meantime its competitors who tried to be safe would have been undercut and driven to bankruptcy... The CXOs would have cashed in their stock options and unloaded the stock on to a bunch of chumps holding worthless stock....
  Yay! For capitalism.
  But in reality, these morons would think, "yeah, so I'm going to bail out before the company crashes and burns"
  
  --
  sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
10. Re:What is up with airlines IT structure by Tisha_AH · 2017-01-23 03:42 · Score: 1
  
  ACARS has several component systems; some are common to all airlines and others are unique to each particular carrier. At its simplest form it provides very basic information regarding the status of a flight (takeoff, landing, gate departures and arrivals) and in-flight weather information (making each plane in to a temporary weather station).
  Some air carriers use ACARS for much more comprehensive functions like monitoring the status of equipment on aircraft in flight (maintenance scheduling), passenger information or cargo requirements. Those carriers are going to be more susceptible to periods of high demand across bandwidth limited VHF frequencies or availability to a satellite uplink.
  ACARS has become so useful that it is now hitting some limits because of the high demand for the functionality.
  
  --
  Tisha Hayes
11. Re:What is up with airlines IT structure by atheos · 2017-01-23 04:23 · Score: 1
  
  forget playing Chess, Poker and Go - This is the problem that AI needs to be solving.
12. Re:What is up with airlines IT structure by dunkelfalke · 2017-01-23 05:00 · Score: 1
  
  It was more or less comparable if we consider aircraft from the same era - it would be dishonest comparing today's accident rates with the rates of the 1970ies, no matter which aircraft. The reasons, however, were quite different - pilot errors and general technical backwardness were the most prominent reasons - soviet passenger aircraft was technically about a decade behind. Soviet aircraft designed shortly before the breakup (Il-96, Tu-204) caught up and are generally about as safe as western aircraft and have all the modern (as in late 1980ies) airframe features, but the engines aren't as economical, hence very few of them were ever built.
  
  --
  "It's such a fine line between stupid and clever" -- David St. Hubbins, Spinal Tap
13. Re:What is up with airlines IT structure by dunkelfalke · 2017-01-23 05:03 · Score: 1
  
  Dude, you are preaching to the choir here - that is exactly what I have written.
  
  --
  "It's such a fine line between stupid and clever" -- David St. Hubbins, Spinal Tap
Re:Replaying the Oracle commit log by Anonymous Coward · 2017-01-22 15:59 · Score: 1

Turn off your phone and go for a curry.
from TFA by nimbius · 2017-01-22 16:05 · Score: 4, Interesting

U.S. officials told NBC News that the Aircraft Communications Addressing and Reporting System, or ACARS, had issues with low bandwidth.

so you're complaining that a 60wpm VHF-based system from the late seventies that was never designed for high bandwidth communication beyond 300 baud, has problems delivering bandwidth intensive data? the average ACARS datagram is only 8 tuples. it sounds like one United's H1B candidates didnt take the time to RTFM before rolling out their code and immediately clobbered the system with garbage XML or metadata some middle manager wanted to include to improve productivity.

--
Good people go to bed earlier.
1. Re:from TFA by Anonymous Coward · 2017-01-22 17:27 · Score: 1
  
  Yes, this is true. ACARS is about as low bandwidth as it gets. Pretty robust in terms of demod too. It uses AFSK, so simple correlater is used to detect the ones and zeros for the bit slicer. Can't imagine how the problem could be anything to do with bandwidth... 8-bit microcontroller could easily decode it.
2. Re:from TFA by AmiMoJo · 2017-01-23 00:49 · Score: 2
  
  Actually this is a genuine issue with the shear number of aircraft in the sky. Radio systems have issues with hidden transmitters, that is if you have two aircraft who both want to send a message at the same time but can't hear each other they both start transmitting, and a receiver in the middle hears both simultaneously and neither is intelligible. It's particularly bad with wifi, and now air traffic is getting the same problem.
  As things get busy the problem is amplified by re-transmissions, additional messages due to equipment failures and warning conditions like low fuel, more and more aircraft joining parking areas waiting to land and so forth.
  The system could do with an overhaul, but it's well established, works over long ranges and is relatively cheap.
  
  --
  const int one = 65536; (Silvermoon, Texture.cs)
  SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Re:Must be Russia by Anonymous Coward · 2017-01-22 16:08 · Score: 1

Only pro TRUMP posts have the high quality we need to Make Slashdot Great Again.
Heil Trumpler!!!
Re:Must be Russia by Anonymous Coward · 2017-01-22 16:10 · Score: 1

Now that Trump is President, the entire US Government will be flying on his new "discount" airline, Trumpoflot, and all of the profits will be donated back to the US government. Mysteriously, the airline never seems to make a profit. But at least the prices are low, the lowest ever, and anybody presenting "alternative facts" is a clearly a Nazi.
Clearly a Nazi.
Who IS John Galt? by Ungrounded+Lightning · 2017-01-22 16:15 · Score: 1

And is that question scribbled on airport walls country wide?

--
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
Re:Must be Russia by lucm · 2017-01-22 16:24 · Score: 3, Funny

Good thing Obama is no longer president, or we would be nuking Russia for their "hacking" of a US airline...
It would never happen. They would just call the Scorpion guy and get him to rollback a buggy Windows Update like he did at LAX in 2014.

--
lucm, indeed.
Children of the Magenta by Anonymous Coward · 2017-01-22 16:25 · Score: 1

The children of the magenta have taken over the airlines, and now the op specs won't let us hand jam the flight data in. This is, of course, fucking retarded, because all airline pilots are completely capable, competent, and routinely train to flying these things by hand, using reversionary navigation. I can, indeed, fly the 'Bus across the ocean without ACARS, without GPS, without the computer figuring out takeoff and landing data, and without datalink weather. The airplane is actually well designed, and the FMS navigation is excellent on DME/DME and good enough on raw INS to at least find Ireland, at which point I can get VOR navigation back and take an ILS into any commercial airport in the Europe, the Americas or the Pacific (there are sketchy places in Asia and Africa.)
Nervous? by Vinegar+Joe · 2017-01-22 16:41 · Score: 3, Funny

"There's no reason to become alarmed, and we hope you'll enjoy the rest of your flight. By the way, is there anyone on board who knows how to fly a plane?" - Elaine Dickinson

--
"The average reporter we talk to is 27 years old......They literally know nothing." - Ben Rhodes
and once again, cutting corners in IT backfires by Indy1 · 2017-01-22 16:48 · Score: 5, Insightful

I love reading when a company that is critical dependent on their IT infrastructure to function, cuts as many corners (and jobs) as possible in IT to save a buck, then has it all blow up in their face.
Target, Home Depot, United, Yahoo, etc.....they'll save millions, until they end up losing billions.

--
Lawyers, MBA's, RIAA? A jedi fears not these things!
Apollo System? by AlanObject · 2017-01-22 16:52 · Score: 1

Is United still using the Apollo system on IBM 370 mainframes? Last time I paid any attention to it that's what they were doing.
obligatory... by mbaGeek · 2017-01-22 17:43 · Score: 1

did they try turning it off and then back on?

--
It ain't what they call you. It's what you answer to. http://mylyceum.us/
In other words ... by CaptainDork · 2017-01-22 17:47 · Score: 1

... had issues with low bandwidth ...

DDoS.

--
It little behooves the best of us to comment on the rest of us.
Re:It's not Russia, it's the squirrels by BlueStrat · 2017-01-22 18:05 · Score: 3, Funny

Squirrels cause more damage to infrastructure than humans and natural disasters. No joke. NSA even acknowledged it. http://cybersquirrel1.com/

Nyet!
Moose and Squirrel sold out and are now double-agents working for Dear Leader!
Strat

--
Progressivism (aka US 'Liberalism'): Ideas so good they need a police/surveillance-state to enforce.
Somewhere by quantaman · 2017-01-22 18:15 · Score: 2

A software developer reaaally doesn't want to come in to work on Monday.

--
I stole this Sig
1. Re:Somewhere by archilochus · 2017-01-22 22:28 · Score: 3, Funny
  
  Anyone else remember this (yes, I'm that old): http://dilbert.com/strip/1996-01-31
Many Guitars were saved... by REden · 2017-01-22 19:07 · Score: 2

Many Guitars were saved.. because we all know United Breaks Guitars.

--
--- If it's worth doing, it's worth doing in Perl!
ACARS security by aberglas · 2017-01-22 19:07 · Score: 1

So, what type of encryption did they use in the 1970s? My guess is none at all.
ACARS must have become a critical component. Otherwise they would have just taken off without it. Used the voice radio instead. If it is critical, is it secured? Nope.
(Voice communication is also unsecured, and idiots with VHS radios have occasionally caused nuisance. But voice is between human beings who can generally figure out what is going on.)
1. Re:ACARS security by Zak3056 · 2017-01-23 05:57 · Score: 1
  
  idiots with VHS radios have occasionally caused nuisance.
  To be fair, the replacement radios are in Beta.
  
  --
  What part of "shall not be infringed" is so hard to understand?
Low bandwidth by edtice1559 · 2017-01-22 20:24 · Score: 2

I hope that somebody who has worked on these systems will comment here. ACARS is a satellite-based system and I don't think that each airline has their own satellite. So how is it that this only affected United and not everybody? Something doesn't add up here. There are a lot of snarky comments here but on-board aircraft software doesn't get updated en mass. The only thing I can really think of is that the ground-based IT systems were unable to process ACARS data. But that wouldn't be for bandwidth-related issues!
1. Re:Low bandwidth by Anonymous Coward · 2017-01-23 01:28 · Score: 1
  
  ACARS is not satellite. The transmitting system is on each aircraft and sends out short telemetry messages using very narrow, low bandwidth, AFSK modulation in the clear. Not to be confused with the other telemetry on all of these planes called ADS-B. ADS-B sends out GPS location, velocity, ID.
2. Re:Low bandwidth by edtice1559 · 2017-01-23 02:46 · Score: 1
  
  This is a great post. Wish I could mode it up. Can you confirm or refudiate this part of Wikipedia >Because the ACARS network is modeled after the point-to-point telex network, all messages come to a central processing location to be routed Even though it's not satellite, it still seems to be shared infrastructure that should affect everybody equally unless it is the final ground link.
Re:Must be Russia by Anonymous Coward · 2017-01-22 23:30 · Score: 3, Insightful

Non-sequitor. We haven't nuked anything, and not for events much more egregious than preventing takeoffs of a single major airline which experienced a rather trivial IT outage - the type of thing that has happened all-too-often at the airlines without any outside interference.
You missed the sarcasm.
Under 8 years of Obama, the US completely ignored hacking by China and Russia - China stole the entire OPM database of cleared US government workers, and Obama did nothing. No response at all.
Hell, Obama knew about Russian hacking of the DNC last summer - and did nothing. Why the hell do you think the Russians thought they could get away with hacking the DNC? Because Obama had let it happen for years.. Obama didn't even bother with some bogus "red line" threat of retaliation.
"OMG RUSSIAN HACKING!!!" didn't become a big deal until Hillary needed an excuse.
The Repsonse by Murdoch5 · 2017-01-23 01:35 · Score: 1

Take all of those whinny little travelers and reply back: "If you think it's easy keeping computer systems up, running and online for US wide air traffic control, please submit your resume or shutup!"
ACARS and NextGen ATC by PilotKnob · 2017-01-23 02:30 · Score: 1

From what I've gleaned from the measly information dripping out our way, the basis of the NextGen ATC system is going to be based on ACARS. This scares me greatly, as ACARS in my experience in using it over the past decade is that it's utterly unreliable. "ACARS NO COMM" is seen in the MCDU scratchpad far more than any other message. They better have a super-duper improved communication network ready for this, or I'm calling it right now - it's a dead-in-the-water system if it attempts to use today's utterly fragile ACARS network.
Got another accident for your list by Zontar_Thing_From_Ve · 2017-01-23 02:48 · Score: 1

You can possibly add Air France 447 to that list. Air France chose not to replace the air speed indicators until the plane returned to France on what would be its last flight. The passengers and crew lost that gamble. I have no inside knowledge as to whether that was for cost reasons, lack of training in Rio where they simply didn't know how to do the job there, or control freak reasons (I worked for a French company and until you actually work for them you really have no idea how much of a control freak the French are about everything) but failure to replace the air speed indicators prior to that flight started the chain of events that led to the crash.
1. Re:Got another accident for your list by WhiplashII · 2017-01-23 03:21 · Score: 1
  
  However the true cause of the crash was actually nepotism. The copilot that caused the crash (by continually stalling the plane until it hit the water) was not the best for the job, he just had the best connections. He did not know that continuously pulling back on the stick would stall/crash the plane.
  Of course, the real root cause of the crash was that there was no obvious feedback that he was pulling back on the stick. The PIC did not know he was doing that until he mentioned it right before impact, by which time it was too late to recover. (from the transcript: copilot: I'm pulling back on the stick, why doesn't the nose go up. command pilot: NON!)
  
  --
  while (sig==sig) sig=!sig;
2. Re:Got another accident for your list by tlhIngan · 2017-01-23 06:21 · Score: 1
  
  However the true cause of the crash was actually nepotism. The copilot that caused the crash (by continually stalling the plane until it hit the water) was not the best for the job, he just had the best connections. He did not know that continuously pulling back on the stick would stall/crash the plane.
  Of course, the real root cause of the crash was that there was no obvious feedback that he was pulling back on the stick. The PIC did not know he was doing that until he mentioned it right before impact, by which time it was too late to recover. (from the transcript: copilot: I'm pulling back on the stick, why doesn't the nose go up. command pilot: NON!)
  And the real problem is that Airbus planes traditionally do not stall. They are fly by wire, but the pilot controls go through a flight envelope management computer that ensures the plane stays within the flight envelope.
  The problem is, that piece of avionics requires a lot of sensors ot be operational, including airspeed. One failure and it goes offline, and the Airbus becomes like any other plane - susceptible to stalling.
  The stall horns went off. THe stick shaker and pusher went off. The pilot did not respond appropriately to the stall.
  This would be important in the Miracle on the Hudson, where Sully turned on the APU immediately. That had the effect of keeping the flight envelope system running, which when you're low and slow, is very easy to stall the plane. That one step likely turned a terrible situation (ditching) into one that was more manageable. And in an emergency, you use all the resources and aid you got.
Re:Must be Russia by ole_timer · 2017-01-23 03:23 · Score: 1

Obama said the OPM hacking of the SF86's was "allowable espionage" - he reacted, but in the way we wanted... He was way too late on Russia and thus ineffective...

--
nothing to see here - move along
Re:Must be Russia by AvitarX · 2017-01-23 03:26 · Score: 2

I'd think getting Saudi Arabia to pump more oil was all that needed to be done.
You don't think Putin is aware that his behavior and that are related? You don't think that's why he wanted to make sure there wasn't a democrat in office?

--
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
Problems delivering bandwidth intensive data? by khz6955 · 2017-01-23 04:41 · Score: 1

@nimbius: "so you're complaining that a 60wpm VHF-based system from the late seventies that was never designed for high bandwidth communication beyond 300 baud, has problems delivering bandwidth intensive data?

Why would local congestion take out all of United Airlines' domestic flights? I suspect the problem occured at the centeralized message routing system. Which apparently doesn't come with any build-in failure modes.
IT issue? by mu51c10rd · 2017-01-23 06:18 · Score: 1

Did they turn it off and on again?
Re:They are still using private infrastructure by Eristone · 2017-01-23 08:15 · Score: 1

Bwahahahahhahaha....
Wait. You were serious. Let me laugh louder...
BWAHAHAHAHHAH.... HAHHAHA....HAH....
Anonymous Coward who started computing sometime after 2008 and has no memory of the history of why things are done the way they are done. Either that or someone working on their MBA and ..well, there goes that history statement again.