Delta Air Lines Grounded Around the World After Computer Outage (cnn.com)

Delta Air Lines says it has suffered a computer outage throughout its system, and is warning of "large-scale" cancellations after passengers were unable to check in and departures were grounded globally. The No. 2 U.S. carrier said in a statement Monday that it had "experienced a computer outage that has impacted flights scheduled for this morning. Flights awaiting departure are currently delayed. Flights en route are operating normally." A power outage in Atlanta at about 2.30 a.m. local time is said to be the cause of computer outage. CNN reports: "Large-scale cancellations are expected today," Delta said. While flights already in the air were operating normally, just about all flights yet to take off were grounded. The number of flights and passengers affected by the problem was not immediately available. But Delta, on average, operates about 15,000 daily flights, carrying an average of 550,000 daily passengers during the summer. Getting information on the status of flights was particularly frustrating for passengers. "We are aware that flight status systems, including airport screens, are incorrectly showing flights on time," said the airline. "We apologize to customers who are affected by this issue, and our teams are working to resolve the problem as quickly as possible."

Comments

Incompetent IT

[sjbe]

A power outage in Atlanta at about 2.30 a.m. local time is said to be the cause of computer outage.

Kind of amazing they haven't figured out how to make their system redundant, distributed, and/or robust. It makes zero sense that a power outage in Atlanta should have any effect on a flight going from Salt Lake City to Seattle. If this was the first time something like this had ever happened I could see them being caught off guard but stuff like this is nothing new and multiple airlines have been affected. You would imagine that having a robust network would be job number 1 for their IT people since one failure like this can easily cost tens of millions of dollars.

Re:Incompetent IT

[Anonymous Coward]

AFAIK pretty much all airlines run scheduling software from a single company (I remember reading an article about how Southwest moved from an in-house system to the same as everyone else due to complexity issues), so it's not so much the airlines but this 3rd party that seems to have somewhat fragile software.

Still though, this begs to be something hosted in a datacenter/cloud with an online shadow in the background of another location replicating everything and ready to take over at a moment's notice, or something similar. Pretty standard these days, but airlines are so tight for money that they end up sometimes shooting their own feet...

That's about $100 Million per day in lost revenue

[billrp]

You would think they would have a backup for the backup power. But like someone earlier said, this outage sounds suspicious.

For those claiming bad managers and saving money:

[Anonymous Coward]

Most of y'all probably don't know what you're talking about. Here's what's going to happen:

1) Delta will file a loss-of-business / data system failure claim after things are stable again
2) They'll haggle with their insurer long after this little story is forgotten (and yeah, lots o' heartache today, but it's still probably going to be little.)
3) Delta will get a settlement of some dollar amount
4) Some bean counter will eventually tally the cost of that policy versus the payout versus how much all those redundant backups would have cost. The accountant will most likely conclude that it was a smart idea to have bought that insurance policy and NOT paid out the multimillions of dollars IT was asking for in redundant systems.
5) The insurance company will note the payout as a blip on its financials (probably already expected by the actuaries.) Insurance company will keep making profit.

The little air traveller is screwed and blued, but Delta and its insurer will keep flying. Doing business today without a data loss rider on your business insurance would be the really stupid idea, much more so than wasting money on redundant systems that are more expensive than said rider.

Paperless Tickets

[Vlad_the_Inhaler]

This story brought to you courtesy of paperless tickets. Yes they are cheaper, yes it is simpler if people can print their own tickets, but the IT has to be up and running.
I remember an airline IT outage back in September 2004, there was a bug in the OS's error-handling routine for a particular class of error. This had all been tested with this particular OS level and had worked, but they had been forced to change the OS configuration to accomodate some new software and the bug was in place. Moving to new discs required a reboot, an additional configuration error caused problems. If it had been fixed within (I think) 90 minutes all would have been fine. The outage was 8 hours.
Passengers turned up at the airports with their paper tickets and were allowed to board. Any pre-allocated seating was ignored. People were laughing about flying the way things used to be, a good time was had by most.

Then came paperless tickets. The next outage had effects more like those we see in this case.