The London Stock Exchange Goes Down For Whole Day

Colin Smith writes "TradElect, the Microsoft .Net based trading platform for the London Stock Exchange, was offline for about seven hours, meaning that their 5-nines SLAs are shot for approximately the next 100 years. The TradElect system was launched back in June of 2007 and was designed for increased speed and system capacity."

Comments

Re:That's some strange math...

[pyite]

Since when is 7 hours even close to "a whole day"? Maybe you meant "almost a whole business day"?

It's a whole trading day--and that's all that really matters when it comes to a major market.

Re:Good lord, they're running on Windows? Why?

[Hyppy]

Perhaps the bit you're missing is that windows isn't quite as bad as the /. crowd likes to say it is. Especially if its an older (translation: fixed & stable) variety like win2k or even nt4.

I'm not sure if you're serious or not, but surely you aren't trying to compare NT4 uptime with the 5 9s of a solid System z platform?

Re:Good lord, they're running on Windows? Why?

[erroneus]

Oh please. Persuasive marketers can get Windows installed just about anywhere including US war ships.

While it is commonly accepted by many techies (and strongly denied by others) that Microsoft Windows is not a suitable platform for that level of computing, sales people often bypass the techies who know better and sell to managers and executives who still believe "you can't get fired for using Microsoft."

With all this said, it will be quite some time (and possibly never) that we will ever know for certain what is at the root cause of the failure. You can be sure that Microsoft is all over this problem both technically and P.R.-wise. They won't let the facts get out if they are damaging. Recall the major power outage that many still believe was caused by a worm attacking Microsoft servers? As far as I can see, the true cause of that failure has yet to be revealed.

But if this was a planned event, or an unplanned disaster resulting from a planned event gone bad (updates, upgrade, other maintenance), you would think they would have provided for mishaps in some way or another.

But as this news story is all I have to go on, there is no indication of cause and so I will not presume this is a Microsoft problem. But it says a lot that NYSE runs on Linux and not Microsoft. It seems SOMEONE did listen to the techies.

Re:Misleading summary

[tgatliff]

Why the heck they were using MS Windows for this type of environment is stunning... Transactional processing which is the bulk of this type of setup is where Solaris and Linux excel. Any company that builds a system like that on .Net should be thown out on the street.

In short.. Not to rock on Windows, but different platforms always offer different strengths..

Re:Still don't know why...

[morgan_greywolf]

Wait! Are you suggesting that downtime can be caused by application problems, network problems, hardware problems, dumbass systems administrators and a whole slew of other things completed unrelated to the platform on which it is running?

I am *shocked*! *Shocked* I tell you!

Re:Oh, my.

[Coraon]

Followed by the youngest member of the team becoming the scape goat and being fired.

Re:Misleading summary

[japhering]

As is normally the case M$ threw lots of money at the exchange to get it to switch unix/linux base to windows net so that M$ can tout that a major exchange is running windows.

Full page ads touting the switch and the reasons they cited were better through put and better up time.

They even had ads touting it here on /.

Re:How many failures before..

[stinerman]

When I said there were companies who could provide excellent Linux support, he said his ass was on the line if something broke so he wanted to be able to justify his software choice to the the C-level guys. And those guys knew the name Microsoft. So he didn't see anything else as an option.

In other words, he used the "no one ever got fired for buying IBM" defense.

Tee Hee

[mengel]

Oh, ye of lesser cynicism. I also, long ago, used to believe that language features could improve software reliability. Nowadays the idea just makes me cackle -- in actuality the universe just invents better idiots.

Re:Tee Hee

[Simon Brooke]

I also, long ago, used to believe that language features could improve software reliability. Nowadays the idea just makes me cackle

Why? Certain languages have features that eliminate large classes of errors. Whilst its possible that programmers will find other ways to screw up, I'd have though that reducing the set of errors that are actually possible would go some way to improving reliability.

Out of curiousity, what languages are you familiar with? Have you worked much in languages with very tough compile-time checks, like Haskell?

Y'know, I agree with the grandparent. On my first coding job there was a guy (Chris Burton) who'd worked on the Manchester Mark One. He was retirement age when I met him. We had a new model of inkjet printer, which had a new processor none of us had ever seen before. It printed characters, we needed it to print bitmaps.

Chris took the datasheet for the printer and the datasheet for the processor home on the train with him, and came back next morning with new code for the printer PROM written out - in opcodes, not assembler mnemonics - in longhand on a pad of paper. That code was blown into the PROM and worked first time, and continued to work without any errors reported for the three years I was on that project.

Programmers like that just don't seem to exist any more. Automatic memory allocation, bounds checking, type checking, etc. are great technology, and I wouldn't choose to live without them. But they mean we are all sloppy and careless, because we can get away with it, and when humans can, they do.

Re:Oh, my.

[mrjb]

These "better languages" are easier to use which allows for less experienced coders to perform the tasks.

I couldn't disagree more. Although automatic garbage collection is nice, this doesn't mean that you'll get "five nines uptime" systems by working with "less experienced" coders.

If you're building a system that must guarantee 999.99% uptime, you wait until your best professionals become available, because it doesn't only involve code. You DON'T give the job to the less experienced ones, no matter how great the programming language. Five nines uptime requires a very robust design and very solid code quality running on a very solid platform which is running on a very solid OS on a very solid infrastructure. You'll want everything to be tested by unit tests, integration tests, regression tests, and whatnot. That involves a whole lot more than 'just' coders, but whoever works on it, they better be good at it.

Re:Bad upgrade

[Locutus]

this same kind of thing( replace *nix with Windows ) is what took out the LAX comm system a few years ago and left dozens and dozens of airplanes in the air and on the ground at/over LAX without communications.

What blows me away is that for years, UNIX systems were one of the defacto standards for mission critical OSs. Along comes a marketing company, Microsoft, and people are saying it is capable of mission critical use even when there are constant disruptions from virus attacks, Ctl-Alt-Del and BSoD are a well known features, and any of a hundred other reasons it is NOT ready for mission critical systems.

What kinds of morons are running the show anyways? And it is about time people start getting fired for this junk. From my experience on operating systems, UNIX was the one OS where when you wrote code, you dealt with the business logic/code and not OS issues. Only once in a blue moon did an OS patch or structure tweak get in the way of coding the application(s). OS/2 was pretty good but not as good as UNIX and Windows was the worst. Gawd, I still hear people complaining about that little Windows Mobile OS crashing. They can't even get a small chunk of code working properly let alone the behemoth that is the Windows desktop and server OS.

LoB

Re:Using Microsoft for a 5-nines SLA? Is that a jo

[Capt James McCarthy]

I have a feeling that the 'normal' IT situation was to blame for this.

Preamble: Technical Expertise provided a wonderful architecture that was HA and robust, fast, and scalable.

Bean Counters looked at the cost and said "You Tech guys spend too much money."

IT architects: "How much is your data worth?"

Bean Counters: "Not this much. Look we don't really need all of these systems. My home system has been working for 4 years with no problems. And I've talked with Microsoft Execs and they will cut us a deal for their platform. Now go away, I've just decided how the architecture will be done. Why did we hire you anyways?"