Slashdot Mirror
British Airways CEO Won't Resign, Says Outsourcing Not To Blame For IT Failure (bbc.com)
British Airways CEO Alex Cruz insisted he would not resign on Monday as he sought to draw a line under three days of chaos at the UK flag carrier after IT problems left tens of thousands of passenger stranded. In an interview -- the first since a global computer outage all but shut the airline down -- Cruz said he doesn't think "it would make much of use for me to resign." Separately, he also denied an outsourcing deal was to blame for the IT problems that hit on Saturday, causing the airline to cancel almost all its services over the weekend. From a report: A leaked staff email revealed Mr Cruz had told staff not to comment on the system failure. When asked about the email he told the BBC the tone was clear: "Stop moaning and come and help us." The airline is now close to full operational capacity after the problems resulted in mass flight cancellations at Heathrow and Gatwick over the bank holiday weekend. Questions remain about how a power problem could have had such impact, said the BBC's technology correspondent Rory Cellan-Jones. One theory was that returning systems were unusable as the data had become unsynchronised. [...] Cruz told the BBC a power surge, had "only lasted a few minutes," but the back-up system had not worked properly. He said the IT failure was not due to technical staff being outsourced from the UK to India.
They pissed me off more than a decade ago, and I swore never to use their services again. Since then I flew all across the world, for scientific conferences, cooperation, or just fun. This includes even many flights to the US.
I'm not surprised BA sucks this bad, with a CEO like Alex Cruz.
"The agriculture ministry is not in charge of Gundam" - Japanese ministry official.
What I can't explain is why he is STILL the CEO.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
I think generally backups are badly managed. I don't think most management sees them as all that critical. I suspect admins likely get tired of trying and just go with the flow after awhile.
In all my years professionally I've only ever really needed enterprise backup (i.e. not my desktop etc...) twice (Oracle DB). Both times it was useless. The first was a scheduling issue where the last backup that was done was 9 months old which is really unacceptable. In that case we had to use some complicated data harvesting from log tables (which fortunately we had in this instance), though some data was lots due to format differences. The second time apparently the backup process was broken, and it was under maintenance to fix it, for a month, but no one decided that it might be a good idea to tell anyone, so when we deployed a new version of an application into production which caused a number of data issues, the last good backup was 3 weeks old, meaning we had to get creative with the existing data and live with the rest putting it on users to manual confirm a couple weeks worth of possible changes.
Anyway from my own experience whenever it's been needed, it's not there. Personally I think I am way more fastidious about my backups, but there seems to be a thing about corporate culture, and perhaps the idea of risk management and passing blame and responsibility off on somebody else..
Maybe it wasn't a single supply, but rather a series of failures over the last 6 months which never got fixed.... and then the last one failed as well.
Or some forgot to plug in the monitoring cables for the redundant power supplies? I worked at a company where a hallway suddenly smelled like an open sewer for several weeks. What made it mysterious was that no sewer line went through that part of the building, leaving the building owner and plumber puzzled about the source. The smell came from leaking batteries inside a redundant UPS in the network closet on the other side of the wall. Since the monitoring cable wasn't plugged in, the one-man IT department didn't know that the UPS stopped working a long time ago.
I often recall this piece of sagely advice from guitarist Robert Fripp, talking to author Tony Bacon:
TB: What advice would you give a young musician?
RF: Never fly Air Iberia.
TB: No, seriously.
RF: Seriously. Never ever fly Air Iberia.
IT is hard and how it works is invisible to those who don't understand it. BA might be screwed. Not only have they outsourced IT but it looks like they don't have the expertise anymore to even evaluate the quality of their IT or even prioritize and fund what their IT should be doing. So now not only is BA not good at IT they are doubly handicapped in that at least from their CEOs statements they can't even evaluate IT.
Um, no. This is an interesting fallacy that I see all of the time - 'go back to paper'. Let's face it - the ONLY way you can run a modern airline, hospital, utility or whatnot is with a computerized system. When it goes tits up, you go tits up until you can get backups on line. Just finding the requisite paper products (and manual credit card imprinters - I'm going to bet that half the BA employees have never even seen one) could take days.
Can you imagine trying to hire and train 5000 temps to fill out complicated forms while the rest of the staff has complete meltdowns?
Fat chance.
Now, BA should have been able to handle anything short of force majour with backups and redundant systems. The power supply theory is laughable. But paper isn't going to solve the problems on any sort of reasonable time scale.
Faster! Faster! Faster would be better!
This is an interesting fallacy that I see all of the time - 'go back to paper'. Let's face it - the ONLY way you can run a modern airline, hospital, utility or whatnot is with a computerized system.
There is a modern airline - Air France I think, that does just this. Their systems fail often, but they have a robust paper system to keep people from being stranded.
Thing is, you don't to allow people to buy new tickets in order to function. Lots of what a modern airline does can just be ignored. You need to verify tickets and boarding passes - which can be done by straining the phone network back to a central office with lots of temp workers, and you need to keep aircraft inspection/maintenance logs current, but that's still mostly paper anyhow.
You can make very complex systems work without computers if you care enough to do so. You can also make disaster recovery systems that actually work when you need them - though you do need to follow the expensive advice of professionals, so maybe some corporations are culturally incapable.
Socialism: a lie told by totalitarians and believed by fools.
Businesses can and do have contingency plans to work without computers. It's absolutely possible with a little foresight and planning.
Foresight, planning *and* training. I work for a Retailer with about 20 branches. What we do, we disconnect every branch from central IT once a year for a day (granted, on one of the slower days), so that they know how to handle the backup procedures.
Oh, there's incompetence here, but it's not the India that's the problem.
In my experience India has an incredible number of talented, capable people, but like talented capable people everywhere they cost more than ignoramuses. But even a country of a billion people has a finite pool of top-notch talent. On the other hand India does have an almost limitless supply of subpar talent, and Indian businessmen are enterprising to a fault. If a Western CEO jis willing to shell out good money for sub-par people, there's a killing to be made.
So who, exactly, is the fool in this scenario?
The British Airways debacle was an instance of a catastrophic failure being brought on by an unusually but statistically predictable event. Therefore, the new vendor the CEO brought in wasn't up to the job he hired them for. That's the CEO's fault, end of story.
The real problem is that people who are good at IT operations make their job look too easy. A fool looking at the lack of drama in a well-run data center is apt to mistake that for the job being easy.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.