Slashdot Mirror
More Airline Outages Seen As Carriers Grapple With Aging Technology (reuters.com)
An anonymous reader writes: Airlines will likely suffer more disruptions like the one that grounded about 2,000 Delta flights this week because major carriers have not invested enough to overhaul reservations systems based on technology dating to the 1960s, airline industry and technology experts told Reuters. Airlines have spent heavily to introduce new features such as automated check-in kiosks, real-time luggage tracking and slick mobile apps. But they have avoided the steep cost of rebuilding their reservations systems from the ground up, former airline executives said. Scott Nason, former chief information officer at American Airlines Group Inc, said long-term investments in computer technology were a tough sell when he worked there. "Most airlines were on the verge of going out of business for many years, so investment of any kind had to have short pay-back periods," said Nason, who left American in 2009 and is now an independent consultant. The reservations systems of the biggest carriers mostly run on a specialized IBM operating system known as Transaction Processing Facility, or TPF. It was designed in the 1960s to process large numbers of transactions quickly and is still updated by IBM, which did a major rewrite of the operating system about a decade ago.
Well, mainframe computers have such excellent uptimes (you almost never reboot one) because everything is hot-swappable. CPU failure? Remove the CPU module, insert new one, and continue - all while powered up. The OS takes care of suspending the failed one and scheduling around it. Ditto all other components. Effectively, you should never reboot them.
Of course, the thing is, when you eventually do reboot them, they take hours to boot all the way up as they perform comprehensive integrity checks (who knows why it was rebooted?).
This is, IMHO, the far bigger issue. Airlines are scheduled tight - if the plane's not flying, it's costing money. Ultra-low cost carriers have very right schedules to ensure the planes are always in the air.
Getting the crews and equipment all prepositioned in the right place and ready to fly is a delicate balance at the best of times and a complete nightmare when you have to start from scratch.
I worked on the first big web-enablement for AA's Sabre system, back in 97-99. Saabre was the key to inter-airline reservation scheduling. Travel agents used this as their main system, and some other folks around here may remember the gateway with CompuServ. eAAsySaber. LOL.
It was unlike any dotcom experience I had around that time. Super legacy. Impossible to change anything - and grave uncertainty that changes were even possible!
The Sabre core compute and data storage stack was built on a series of different mainframe and mid-range systems, back when instead of writing new business functions, you instead attached new business systems.
The glut of stuff crossed vendors occasionally. Mostly IBM. Parts dated to the 70's and through the 90's. I never met anybody who had a "mastermind" view of how it all worked. Instead, lots of analysts with diagrams - mostly from vendors and "big five" firms. Any proposed change had to be run through an exercise that called on the various experts in different parts of the system. Most were not so much expert, as "acquiring some expertise". ;-)
Our work became the basis for travel services like Expedia, and customer offerings by American Express Travel, etc.
I'm sure that this may have changed only somewhat. Saber was sold off, and became a core to Travelocity - who in turn were finally bought by Expedia, who consumed Saber information. Behind it all, there are a 360 and some front-end processors ported to AS400 systems, I'm nearly certain.
"Flyin' in just a sweet place,
Never been known to fail..."
Reservation system could be implemented in chapter 10 of your first programming book. It seems trivial thing?
It's actually really really complex.
It's not just a "reservation system" where you lock out a ticketed space for X seconds until someone completes a transaction. You really have to view it as "The Company" when you're talking about airlines. Let's say a pilot has been in the air too long due to a delayed departure in New York. He hits his max flight time for the next 24 hours but he was scheduled to fly from his destination to another leg. So now you need to replace the pilot. Which pilot? Well there is a plane coming into the airport around the same time as the NY flight. But is that pilot rated to fly that same aircraft? Ok he is, great. But because 30 of the 300 passengers are going to miss their connections now because of the delayed arrival they need to be moved to different flights. But those flights are maxed out. So you have to bump some passengers on a scheduled flight and move them to a later flight as well. Because the plane is getting in late it's also going to depart late. So you also need to either arrange all of the passengers at the next destination to be on different flights and set of a chain reaction or you need to pull in a different plane at the 2nd destination to short circuit the chain reaction. But where can you get a plane from for the cheapest? And how much will it cost to put people up in a hotel vs flying an extra crew in on overtime?
This is all simple enough to calculate with like 1-2 planes. But when you have 1,000 aircraft and all of the seat assignments effectively being interdependent along with business interests (profit/loss of changes), customer service interests such as ticket class... and you have to stay up to date instantaneously with dozens of terminals all trying to do the same thing manually in addition to the automatic callbacks for unexpected events... it's big engineering effort to not create some sort of automatic-trading style feedback loop that accidentally sets off a chain reaction that cancels every flight in the country.
Every change has a cost. No human can orchestrate thousands of interdependent variables with millions of passengers manually. You have to have a central director system which instantaneously handles all of the callbacks and dependencies for a change throughout the entire graph.
It's actually very cool when you stop and think about how well it does at keeping everything relatively straight.
Do reporters even read these stories as they are writing them?!? "Airlines will likely suffer more disruptions like the one that grounded about 2,000 Delta flights this week because major carriers have not invested enough to overhaul reservations systems based on technology dating to the 1960s... [TPF] is still updated by IBM, which did a major rewrite of the operating system about a decade ago."
Big, complicated system, written by a big, experienced company, still maintained... Do they think we'd be better off if it were rewritten from the ground up as a Ruby on Rails app or something?
Psst, I don't want to cause a panic, but I heard that large, important chunks of the Internet run UNIX, which also dates back to the '60s.
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.