Slashdot Mirror
NZX Moves To Oracle On Linux
sn00ker writes "In this story in The New Zealand Herald, we learn that the NZX stock exchange has moved their database systems to Oracle running on RedHat Linux, running on commodity Intel-based hardware. What's really impressive are the performance numbers they're claiming. Quoth the article, "One key query - searching the data on historical trades to identify maximum trade values - has been cut from 36 seconds to 0.03 seconds." An improvement of over 1000 times is spectacular in anybody's books, and is one hell of a boost for the proponents of Linux at the back-end of the financial world."
Oh come on! They consolidated 21 databases and moved to Oracle. That's why it is 1000 times faster. The move to Linux is a footnote as far as the performance issue is concerned -- as stated in the article, the move to Linux was for cost. I'm sure Solaris or god help me, Windows Server 2003 would have given similar performance results. Now if they had moved to MySQL...
Dr. Rick
- "It's such a fine line between clever and stupid" (Nigel Tufnel)
- Zort! (Pinky)
Obviously a much-needed index was added during the migration...
I'm inclined to think that having a request suddenly run 1000 times faster might be due to something a DBA has done, rather than a change of OS.
Of course, if you want to yell from the treetops "Linux runs 1000 times faster..." I'm sure people will back you up.
A 1000 fold improvement in performance, just by moving to linux. Incredible. Unbelievable even.
Comon guys. What kind of idiots do you take us for?
I can't believe you'd get a three order of magnitude improvement in a single function simply because of a change in operating system. I mean, unless they had been using SCO or something.
Sure, a more efficient process scheduler, a more efficient IO scheduler, but really. It would make a lot more sense for the difference to be in the DBM, or even more likely, in the design of the database itself.
Just because someone works for a big company doesn't mean they know what their doing. The most likely reason for the speedup would have been an optimization in their own software, or their database schema. Followed by an improvement in the RDBM, and finally the OS.
A thousand fold increase in speed simply from changing the OS is just impossible to believe -- unless there was something very wrong to begin with.
autopr0n is like, down and stuff.
I'm no windows sympathizer, but in the world of enterprise software, only optimizations at the database layer (or reworking badly written networking layer) can yield those kind of results.
Sounds like they data warehoused and redesigned the schema/indexes to better match usage.
"We went for Linux, not just because we hated Microsoft, but because the cost was compelling," Phillips said.
(Insert funny remark here because I'm unfunny)
The article states a really big improvment, but is seems hard to swallow. The article fails the mention what system they were running before, aside from mentioning "propietary Unix". I don't know, maybe they had some 10-year old system running the database before and with that I could buy the big improvement but with crucial information omitted in the article, feels kind of like puffed up hype.
Apart from being able to consolidate 21 databases into one, the new NZX system runs faster, more reliably and at less cost, says the company's tech team.
One key query - searching the data on historical trades to identify maximum trade values - has been cut from 36 seconds to 0.03 seconds.
Well yeah. They consolidated 21 databases. It sounds like they had an 'overgrown' design, with lots of hacks. That's why it was slow, the consolidated the whole thing into one. Probably with help from Oracle themselves on optimization. Anyone would get a huge speedup out of that.
autopr0n is like, down and stuff.
Financial organizations are very conservative but even Deutsche Bank are migrating to Linux some of their less important processes.
In all the cases the future of the financial industry is in cheap linux clusters.
According to the artcle, they built a cluster using Oracle Real Application Cluster, (I guess Beowulf is just for toy apps :P) which allowed them to spread the core DB over multiple machines (!).
autopr0n is like, down and stuff.
The improvement is impressive - but I would credit the overall architecture, rather than some single specific factors - like Oracle10g+Redhat or DBA or systems consolidation.
I mean, every part of the architecture has its role.
Some other contributing factors not mentioned, I suspect, would includes - focused performance requirements, specific purpose optimised query framework.
Can someone point to some public material on the architecture? It would be a interesting read.
Sunset over the lake, cool mist over the bridge; A leave upon the ripples, the snow reflects its glow.
I doubt they're your garden variety "OMG BillG iz teh debil" Loonix fanbois, friend.
They are a serious enterprise, and there must be a reason something as provocative as " not just because we hated Microsoft" would come out in an interview.
IOW - It's likley that Microsoft's products and/or policies have left a very, very bad impression with these people, and they're glad that they have a compeditor with which to smack Microsoft in the head with.
Soko
"Depression is merely anger without enthusiasm." - Anonymous
As slashdot only accepts conspiracies for posting this has never really be disseminated:
The NZX (ex NZSE) runs the Computershare ASTS trading system for their equities and bond trading. They have done so for 4 years.
This system runs under Linux (Redhat) on Compaq machines.
That they aggregated some of their databases and achieved better performance is non news but the increase in performance stated is worth a conspiracy post!
C'mon, it ain't nice to call NZ that!
Unless specifics about the query and the physical database model are comparable in both systems this isn't really impressive.
Comparable - not equal - since each database engines optimizer has it's individual quirks and strength.
Assuming that you have large joins on huge tables a couple of good indexes, which make the optimizer happy can reduce execution time from hours to seconds.
Table scans are expensive in database speak.
ich bin der musikant
mit taschenrechner in der hand
kraftwerk
...for using the term "unices" to refer to different flavours of Unix (I like Chocolate Chip Meself).
On a more serious note, the statement made above just about applies to any operating system i.e.
If the world goes*insert OS name here*, *insert company name here* won't need to spend so much making sure its technology can run securely and reliably on every weird combination of hardware and operating system its customers adopt.
My Favourite Meme
I cannot believe the spin on this post. Even for slashdot this is way below the bar. Anybody who knows the slightest thing about databases knows that a performance improvment like this is not attributed to which operating system or database you use. They would have had to be running Access on Windows98 on a 386 ACER laptop to see a performance to increase like this. Obviously there's something else going on. For example, a simple change in how tablespaces are organized could be responsible in which case it would be possible do precisely the same thing with just about any reasonably DB/OS.
I have achieved increases of 10^4 and 10^6 in production systems by recoding a small critical part of an application (usually less than a page of code).
Most of the time the problem is stupid code or operational ignorance. Rarely is hardware, O/S or data base software changes the sole or main solution in performance problems. Hardware is only a factor when the system is underspecified to save money.
Given that they consolidated 21 databases into a single database the problem could simply have been network latency between separate physical servers.
The simplest way to get performance problems is to test on developers personal machines with tiny test databases and implement without full scale testing.
For those of you who wish to ensure that Microsoft SQL server is slow, invoke a user defined function as part of the where clause that the optimizer cannot recognize as a determinate function when joining two tables. This will ensure a nested loop join that will take an eternity.
When I was young, I had to rub sticks together to compute.
What the Oracle guy said was key, that if a software company can target a restricted range of kit, rather than every possible third party gizmo and buggy driver that can be installed under Windows, they've got a vastly easier job.
For some software applications it makes sense to refuse to ship the software on its own and insist on giving away free hardware with the deal, with the operating system of your choice (it isn't really going to matter which operating system) fully configured and installed. That way you know what the client is running your software on, you've tested it, and you've got an identical setup back in the lab to research problems on, and you know it isn't going to crash because the client's box is running some crap driver you've never heard of.
Let's see how many hours NZX is down during next five years due hw/sw malfunction. That's meaninful.
Dyslexics have more fnu.
"CREATE INDEX"
It's amazing what you can do to optimise a query or two...
Advice to the sarcasm-impared: do not take this posting literally.
Have To Smile :)
Everyone seems to be thinking that the story is all about a thousand times performance increase because they switched to Linux.
I don't see the article make that claim... they just said that they changed a bunch of stuff, and they now have a different system in which one sample query is 1000x faster.
This could be (and probably is) due to a number of reasons:
-- consolidated many separate databases into 1
-- probable new data model
-- probable new application design
-- upgraded system resources (more RAM, better CPUs, faster SAN, etc.)
-- different OS
-- Oracle tuning / kernel tweaking
It doesn't make sense that they'd just re-implement the exact same system and application design... they probably spent a lot of time redoing the apps to make them smarter and faster.
To assume that Linux is singly responsible for the performance increase is kind of silly.
$0.02 (CDN)
I've discovered that Oracle is pretty much OS agnostic because it pretty much takes over the system it is installed on. That aside, when a server is pure anything, the OS really isn't relivant. When all it does is run one app, the performance is pretty much tied to that app. All modren OSes provide good disk, memory, network, etc services. Now you can argue specifics till you are blue in the face, but when running one app, it doesn't much matter.
Where an OS can shine is if you are running lots of stuff (eg webserver, scripts, database server, media server all on one box) and espically when you are screwing around and hence likely to cause problems. However when you do a DB install and run nothing but that, the OS is just a helper. It talks to the hardware and provides some simple APIs. Which OS it is isn't of much consequence to performance.
The cost thing makes me curious too. We tried Solaris on Linux. The DBA couldn't get it to work, and neither could I. Then I looked at the requirements. We are trying SUSE, since that was listed... Well, sorta. It didn't run on normal SUSE, just SUSE Enterprise Server. Likewise not RedHat, but RHEL, and also UnitedLinux. In otherwords, high dollar server Linuxes. Oracle tech support wouldn't even talk to us unless we used a supported OS. We ended up option for Windows XP Pro, since it was supported. As I said, OS didn't much matter, just that it ran Oracle.
Now while I'm sure (or at least pretty sure) Oracle could be made to run on a non-enterprise Linux, what would be the point? They wouldn't support you and support is one of the big reasons to buy Oracle (not cheap in case you were wondering).
I really question the cost thing. I am guessing someone pushed Linux, and is justifying it as a cost based decision.
Now, I'm sure people are about to jump on me, given that Linux is free... But WAIT! We are talking Linux for running Oracle here. Well, if one checks Oracle requirements you find that in additon to Windows, HP-UX, xOS and such, it does run on Linux, but it's pickey. They require and only support enterprise Linuxes such as RHEL and SUSE Enterprise.
Ok, fair enough, but these AREN'T free. RHEL is to the effect of $800. Hmmmmm... Given that ORacle will also run on XP Pro, doesn't seem like such a deal any more.
We've dealt with Oracle in this regard and found out that:
1) It won't work on stock SUSE or RedHat systems. Dunno why, but there must be something different in the enterprise versions because it won't install properly on the normal ones.
2) More importantly Oracle REFUSES to support you if you aren't on a supported OS. They just say "run a supported OS" and that's it.
Well, given that, for the kind of apps one would want an Oracle database, support is important,I'm not seeing them running on a normal Linux distro hacked to make Oracle happy. So given that they are probably on an enterprise Linux, I'm not seeing the cost savings.
The whole thing sets off my zealotry bells. It sounds like that had a horrible hacked-ass, old database system. They needed to modernize it. So they elected to use Oracle, Makes sense, when it comes to unlimited scalibility and rock solid reliability, Oracle just has it. However then someone sold them on doing it on Linux. No problem, except it sounds like cost was the selling point, which isn't really valid for Oracle.
So now we have the justification scramble. Make sure everyone, espically the bosses, buy the cost argument. Pointing out the speed increase is also a good idea, never mind what caused it, obviously it was your brilliant decisions.
I've seen this happen plenty, and it's not limited to people advocating Linux, any platform that they like will work. You get a zealot for platform X, that uses BS arguments to sell it. They then produce lots of hype, to make sure people think it was the right choice.
A significant section of the IT world think of linux as nothing more than a hobby Linux - that you can't rely on it for anything mission critical.
What this is saying is that Redhat Linux CAN foot it with the big (commerical) boys like Sun and Microsoft.
No, I did not read the f***ing article!
It does say this was just one operation. I'm betting the first time they ran
DELETE FROM clients
It took 36 seconds to return. The second time it pretty much came straight back.
We tried Solaris on Linux,,,,,
Man, let me stand up and take my hat in genuine appreciation.
This ladies and gents, is a real hero.
IANAL but write like a drunk one.
Just because someone works for a big company doesn't mean they know what their doing.
I'd say successfullly pulling off a massive consolidation project on a financially critical system probably puts an upper limit on their incompetence.
Just because what they say doesn't really make all that much technical sense doesn't mean they are incapable of making technical sense when it serves their purpose. Getting things done in any organization involves using your successes to push you agenda. It also means sensitivity to the other messages your audience may be receiving and what message you need to counteract it.
For example, the PHBs may have been hearing that Linux was an unsophisticated system cobbled together by a bunch of amateurs from 1980s technology. If you don't think that message is out there, or that it can't possibly be effective , you are extremely naive. If you think you can counter this argument with technical arguments about file systems, virtual memory schemes and schedulers you are even more naive. So, here's a countermessage: "Look, this Linux based system works great. It's a thousand times faster in some important tasks than the systems we spent millions on before. How 'unsophisticated' can that be?"
You might not think this mode of reasoning is entirely valid, and you'd be right. But it's not without its virtues. Successful decision makers put a higher premium on things being demonstrably "good enough" than on their being "best". And this argument meets the admittedly relaxed corporate standards of truth: it is not literally false and its advanced with the best interest of the company in mind.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
But those linux costs are nothing - compared to the oracle licensing costs:
- $40,000 / CPU for base product
- $10,000 / CPU for partitioning
- $10,000 / CPU for RAC
So, even a trivial Oracle cluster is just not going to come in under a quarter million dollars. Saving a few thousand dollars by going from windows to linux isn't going to make any difference at all.
Unless you have a large unix support staff you can leverage, want to diminish security-related patching & vulnerabilities, etc, etc. But those numbers are a little tougher to quantify.
I was working on a large project where we tested platforms for Oracle. We ran two servers with similar hardware. One on Solaris+Oracle and one on NT+Oracle.
After serveral months, it became obvious there wes no comparison in performance. The Solaris-based server out-performed the NT-based box easily by a factor of 4-to-1.
More importantly, the NT system has to be routinely rebooted in order to remain stable. I actually had to schedule reboots just to keep the system from running out of resources!
That was more than four years ago. Since then, the NT server was repurposed into a workstation; the Solaris Oracle server is still running with an uptime of more than two years.