Slashdot Mirror


Software Upgrade Crashes UK Air Traffic Control System

pitpe writes "Earlier today the computer system controlling most of the UK's airspace failed, after tests in preparation for an upgrade failed. The original failure occurred at the West Drayton centre, which is an old (70's) system, as opposed to the new system at Swanage, which has had its own problems. A system wide reboot to fix the system resulted in the entire system being taken down temporarily."

20 of 233 comments (clear)

  1. Software doesn't rust... by Anonymous Coward · · Score: 5, Insightful

    "which is an old (70's) system". As long as it's not 30-year-old hardware then the software should still be fine. Why does everyone think that simply because software was written in the past it is bad?

    1. Re:Software doesn't rust... by GoofyBoy · · Score: 5, Insightful

      >Why does everyone think that simply because software was written in the past it is bad?

      What is implied is that its being pushed to its limits. e.g. it was designed for 100 flights a day, when today there are 1200 flights a day.

      Those small things which you could get away with before start to become factors in usability and stability.

      --
      The surprise isn't how often we make bad choices; the surprise is how seldom they defeat us.
    2. Re:Software doesn't rust... by Prendeghast · · Score: 4, Informative

      The 'fridge size boxes are 70's vintage (I suspect bits have been replaced over the years). The CPUs are only about five years old. The system consists of two identical computers for hot failover and they they had to get two custom CPUs made by the original manufacturer (IBM, I think) to deal with Y2K.

      As for the software? Written in some weird language called Jovial, and continually repatched - never rewritten.

      BTW, where the heck is Swanage? The new NATS center is in Swanwick!

  2. Lucky in the US... by Kredal · · Score: 4, Informative

    Considering that up until about 2000, all of the major Air Traffic Control centers in the US were running on vacuum tubes, we were lucky nothing like this ever happened here. Sure, there were glitches at regional centers, that required controllers to do everything by hand, but nothing that required a full reboot of the entire country's ATC system.

    Hopefully the UK will get the new system tested and online before it causes more problems!

    --
    Whoever stated that signature sizes should be limited to one hundred and twenty characters can just go ahead and kiss my
    1. Re:Lucky in the US... by aldoman · · Score: 5, Informative

      There is 2 ATC centers in the UK - West Drayton which is for the 4 major London airports only (Heathrow, Standstead, Gatwick and London City). This is a 70s system and is due to be replaced by 2006. This is the one that crashed, but because a large percentage of UK air traffic is destined for London, then it caused the other one to go to a standstill.

      The other one at Swanage handles the ATC for everywhere else. This was replaced with a new system in 2002.

      But, by 2006 hopefully all ATC in the UK will be running on new systems.

  3. More problems... by Mz6 · · Score: 4, Informative
    I found a similar article on MSNBC.

    It seems they have been having problems with their computer systems since 2001 when it was "privatized".

    "The air traffic service has been beset by problems since it was partially privatized in 2001. A $484 million center at Swanwick in southern England opened five years late in 2002.

    The opening was delayed by problems with computer software, and the glitches continued for months afterward, as controllers misread aircraft altitudes and destinations because of hard-to-decipher computer screens. In at least one case, controllers mistook the Scottish city of Glasgow for Cardiff in Wales.

    Now.. that seems like a pretty big mistake for me.. especially for an air traffic controller to do. However, the article later states that:

    "Transport Secretary Alistair Darling said Thursday's problem did not lie at Swanwick but at the older West Drayton center, which is due to be closed by 2007."

    Thank goodness that old one is closing, however it doesn't sound like its replacement is doing any better!

    "If you want to know what is wrong with transport in this country it is that over decades successive governments did not spend enough on the infrastructure and air traffic control is no different," Darling told BBC radio."

    Excellent quote! While terrorism is on everyone's mind, we sometimes forget that safety of transportation should also be just as high. I couldn't imagine pilots relying on themselves to fly airplanes amid the thousands of others without the aid of traffic controllers and their computers.

    --
    Hmmm.
    1. Re:More problems... by Mr_Silver · · Score: 4, Interesting
      "If you want to know what is wrong with transport in this country it is that over decades successive governments did not spend enough on the infrastructure and air traffic control is no different," Darling told BBC radio."

      A dutch friend of mine once remarked that she didn't understand the mentality of the British. "You" she said, "have an amazing tendency to run things into the ground and then get around to fixing them rather than spending money on continually maintaining them so they never fall apart."

      It's a very good point.

      --
      Avantslash - View Slashdot cleanly on your mobile phone.
    2. Re:More problems... by Anonymous Coward · · Score: 5, Insightful
      A dutch friend of mine once remarked that she didn't understand the mentality of the British. "You" she said, "have an amazing tendency to run things into the ground and then get around to fixing them rather than spending money on continually maintaining them so they never fall apart."
      That's a pretty fair point. The trouble is, since 1977, politics has been obsessed with who can provide the biggest tax cuts, which has meant all the state funded / supported industries have gone to hell in a handcart.
      <rant>
      I blame Margaret fucking Thatcher, who let the hospitals fall apart and flogged off the viable bits of the infrastructure to her friends (at well below market value). [We're still feeling the effects of this on the railways, which the private sector has run into the ground] Corrupt old bitch.
  4. What WAS the System that crashed? by Spencerian · · Score: 4, Insightful

    Perhaps a person experienced in ATC software or hardware could enlighten us on the specific system in use, its OS and other trivial bits.

    It would help to reduce the coming surge of Microsoft jokes, which is very likely not relevant here.

    --
    Vos teneo officium eram periculosus ut vos recipero is.
    1. Re:What WAS the System that crashed? by orbitalia · · Score: 5, Informative
      Hi, I worked on exactly this system for 4 years.

      The hardware is an IBM 9020 family mainframe, the application is written in Jovial (one of , if not THE first algebraic language), and BAL assembler (for the monitor mostly). The monitor is the operating system so it effectively is a custom written operating system for this application.

      Although MVS is also used for testing. The I/O capabilities of the mainframe are superb which means it can handle 2000+ flights with only 14 Megs of RAM (if I remember rightly).

      I believe the NAS application came as a freebee from IBM when the UK purchased the hardware and was the same NAS (national airspace system) application used all over the US. It has been continously developed since then (no mean feat when you consider that all variables are global in Jovial, It uses holleriths instead of ascii, and you are limited to 5 or 6 characters per variable name). The hardware has also been upgraded several times over its lifetime.

      It doesn't often go down, last time was 2002 sometime, and you can tell how important it is because everyone screams when it does go down. The people I worked with are extremely dedicated to their job, but one cannot test a system like this for absolutely every eventuality. No doubt some patch was applied and some special case came up that caused a FLOP (functional loss of operation). It happens, Radar is usually unaffected, so the safety implications are not large, but flow is affected.

      The UK approach to handling NAS is much different to the US, the US tends to not touch the NAS software and develop external systems that enhance the usage of airspace, where as the UK tends to delve into NAS and improve things directly in NAS. Jovial is a very interesting language it has been used heavily by the US military and exists in such applications as Cruise missiles and many other aircraft and missile systems. Read about Jovial here if you are interested.

      I can't say too much about it for various NDA reasons (OSA) I think most of the above is in the public domain.

      HTH.

  5. Re:Three fingers by Fishstick · · Score: 4, Funny

    Hey, it worked for Samuel L Jackson...

    this is a UNIX system, I know this!!!

    --

    There is much cruelty in the universe, John.
    Yeah, we seem to have the tour map.

  6. Links for reference by matthew.thompson · · Score: 4, Informative

    National Air Traffic Services http://www.nats.co.uk/services/index.html are the outfit responsible for this.

    They have a press release http://www.nats.co.uk/news/news_stories/2004_06_03 .html which explains quite nicely what they did and why.

    --
    Matt Thompson - Actuality - Insert product here.
  7. So what? by stratjakt · · Score: 5, Insightful

    There are redundant systems in place. Analog radar, humans with brains.

    At least there should be. Computers crash, break, have bugs, etc. They're a tool - a more efficient and convenient tool to be sure.

    But when they break, there are contingencies so that planes can still take off and land, and wont just fall out of the sky.

    This is also why Y2K was such a bunch of stupidity. We really aren't as reliant on computers as people think. We know they crash and are prepared to handle it when they do.

    --
    I don't need no instructions to know how to rock!!!!
  8. Same in Ireland! by pixelbeat · · Score: 5, Informative

    Much the same thing happened last
    week in Dublin

  9. And the Wizard said: by the_twisted_pair · · Score: 5, Funny

    It appears you are trying to land a plane. Would you like to:

    [x] Allow Windows to detect new hardware ?
    [ ] Allow planes to circle in uncertainty ?

    [x] Show this window at all airports

  10. Swanwick not Swanage! by perly-king-69 · · Score: 4, Informative

    The new centre is at Swanwick in Hampshire, not Swanage in Dorset!!

    --

    --
    This sig is inoffensive.

  11. Swanwick, not Swanage! by Xilman · · Score: 4, Informative
    The new system is at Swanick near Southampton, not Swanage as posted here.

    Swanage is a pleasant little seaside resort. I know it well and stayed there a few nights when on my honeymoon.

    Finding Swanwick and Swanage on a map of southern England is left as a exercise. Hint: Mapquest may be a good place to start.

    Paul

    --
    Lasciate ogne speranza, voi ch'intrate
  12. Re:Damn... by InShadows · · Score: 4, Funny

    Microsoft has been testing around with flight software since XP called MSFlight.. here's the article to prove it..

  13. Golden rules.. by rf0 · · Score: 5, Insightful

    Never just test software upgrades on Live systems

    Rus

  14. ATC software is scary (aka, Know Your Userbase) by Knowbuddy · · Score: 4, Insightful

    My dad is helping the FAA and the US military design and roll out the next gen ATC software here in the US. He comes home and tells stories that make my skin crawl.

    The first version of the software was built using standard current interface guidelines and widgets and the testing group that had no experience with older ATC systems were wowed at how simple and yet powerful it was. Pretty much any random person off the street could look at the screen and easily figure out what was going on and how to do various basic tasks. When that version was demoed to the ATC union the union freaked out at how different it was and thus began a cycle of making it more and more backwards.

    So, nowadays the next gen ATC software almost exactly replicates the UI of the old non-computerized and semi-computerized systems. On-screen toggle switches and dials, that sort of thing. The FAA and the ATC union have decided that retraining all of their ATCs to use modern computer interfaces would be a Bad Thing. When the computer screen doesn't exactly replicate the interface of the 50+-year-old systems, they freak out and scream bloody murder. On the flip side, kids coming into the field today that have been using computers most of their lives are finding the interface to be counterintuitive to the point of being almost unusable. Middle-aged workers who are both highly proficient ATCs and home computer users report that switching between the two types of interfaces each night when they go home requires conscious effort on their part, since they are so orthogonal.

    So who wins? Historical inertia, of course. Why fix the problem today when you can wait for your successors to fix it in 25 years?