Slashdot Mirror
Software Upgrade Crashes UK Air Traffic Control System
pitpe writes "Earlier today the computer system controlling most of the UK's airspace failed, after tests in preparation for an upgrade failed. The original failure occurred at the West Drayton centre, which is an old (70's) system, as opposed to the new system at Swanage, which has had its own problems. A system wide reboot to fix the system resulted in the entire system being taken down temporarily."
A dutch friend of mine once remarked that she didn't understand the mentality of the British. "You" she said, "have an amazing tendency to run things into the ground and then get around to fixing them rather than spending money on continually maintaining them so they never fall apart."
It's a very good point.
Avantslash - View Slashdot cleanly on your mobile phone.
Yes, I think that the software structure of a critical realtime system like ATC is much more important than which OS or language it's written in. It should be built like a strange composite stranded cable, with different strands of simple structure that can survive sporadic (even systemic) failure of its parts. In such a system, there should be no such thing as a system-wide reboot, since the only thing that is truly system-wide is the data.
Without this structure, Linux would probably fail at an unacceptable rate too.
[You have a stable society when some nut guns down a schoolyard and the law doesn't change.]
To quote from the NATS (National Air Traffic Services) press release:
"The FDP was being tested overnight for a future upgrade. The system was successfully returned to service but at 06.03 errors were detected in the distribution of flight data between Centres. As a precaution, we decided to restart the FDP (known as a cold restart) causing an interruption to full service. The data processing system was restored at 06.42 and declared fully operational at 07.03. Flight capacity restrictions were lifted at 08.05. The system is now fully operational and we are confident that it is stable.
Through the response team at West Drayton, we have been working with airports and airlines to clear the delayed departures, and expect the backlog to be cleared quickly.
Our investigation into the cause of the problem is continuing."
Let me get this straight: they ran a test on the FDP. The FDP glitched. They rebooted the FDP. They are still investigating the problem.
Now, unless I am mistaken, I can only infer from their statement above that they are now running the FDP which is still susceptible to the problems highlighted by the test.
To be fair (not that I hold any affection for Mrs T. in my heart) the rot stretches back a lot further than the 70s.
I'd say the UK has been letting the infrastructure maintenance slide since at least WW2, maybe earlier. We inherited a fantastic installed base from the Victorians - the fact that it took 50 years of neglect to rot away is a tribute to how well they built - but the sad fact is this stuff was put together by a world-spanning Empire at the top of its game. What with paying for a couple of world wars and then trying to keep up Great Power appearances in the postwar world, we didn't have enough cash to keep this installed base up to scratch or replaced in anything like a timely fashion.
Unfortunately what has taken 50 years to fall to pieces is likely to take about as long to put back together again and (I have it on very good authority) *that* is the real reason why Blair and Brown are so keen on PFIs, despite them being such a poor deal for UK plc in the long term. Its not because they are a cunning dodge to keep spending off the treasury books and plump up the bottom line numbers for the current electoral cycle (although that's a handy side-effect); its because they know that they or their like-minded successors cannot stay in charge for the decades that a full infrastructure overhaul is going to take and they want to make damned sure that nobody raids the infrastructure warchest after their watch.
What one government gives another can take away after all, so from their perspective its no good kicking off a massive overhaul project now if a Conservative government is able to come along in a few years time and gut it for tax handouts just when its about to pay off. What handing out those juicy multi-decade PFI contracts does is lock in a powerful City-based constituency who will scream bloody murder if a future Chancellor tries to raid those revenue streams for a quick handout.
It doesn't make much fiscal sense, but politically its quite astute.
Regards
Luke
#include witty_one_liner.h