Slashdot Mirror
HP Publishs First Linux TPC-C Benchmarks
The first ever official TPC-C benchmark on a Linux system has been published.
This was run on a cluster of 32 HP servers with Intel Xeon CPUs, running Redhat Linux and Oracle RDBMS. The system had over 18 terabytes of storage, and cost over 2 million US dollars. Performance was higher than a similar system running on MS Windows.
Some more TPC-C benchmarks of big iron kit... found here
Linux cluster:
Windows cluster:
Note that the number of clients in the windows tests is higher 24 instead of 16), with smaller CPU's. Also, the server's aren't identical.
Besides from the small differences in setup, it's plain that hardware-costs greatly outnumber software costs. Yeah, linux has a small bit more performance (less than 1%) for a bit lower price (6%) but these aren't real shocking numbers. Of course, I'll get flamed for not bashing microsoft, but the difference really isn't that big.
--
If code was hard to write, it should be hard to read
I noticed a few things when looking at the break-down of costs:
... and a whopping 15" CRT. The clients only make up 5% of the overall cost though.
For the clients, a 1G 133Mhz SDRAM DIMM is $880! ouch. The cost per client is $7172, for which you get a dual 1.4Ghz Pentium III with 4Gb of RAM
The Oracle software is $35000/CPU, for 3 years. What happens after 3 years? Does Oracle lease the software or something? Oracle support is shown as 2,000 * 24. Where does the 24 come from?
It does scale! The cluster TPC-C Benchmark is not very conclusive IMHO, it has very few intra-process communication, so you can basically scale it almost indefinitely.
AL
I have done an extensive study on the effect of compiler flags on performance and I can tell you that to begin with some of them have near zero effect, some others interact badly (ie they speed things up when used separately but slow them down when used together), that some of the flags you give are already enabled when you use -O2 (BTW the -Ox flags are a just convenient way to enable a miriad of "minor falgs"). That -O3 is a double edged sword since in gcc 2.x it is equivalent: to "-O2 -finline-functions" thus making the code faster on small programs (sinced there is no the overhead of function call) but bigger (and thus it could slow large programs if it makes you run of TLB entries, cache or RAM), and it also slows things down when the program (eg the Linux kernel) has already had its significant functions inlined by the programmer (if you use -O3 on it the compiler will inline functions who are small but that the programmer didn't inline because they are rarely called, so you get no benefit but the enlarged size could make that two functions who were in the same page are no longer and thus you get TLB faults). Finally due to bugs on all versions of gcc's optimizer there are significant cases where -march=i686 will be significantly slower on the 686 family (PPro, PII, PIII, Celeron) than using plain -march=i386. Ah, it is not a foregeone conclusion that the Athlon behaves better with a program compiled for the 686 than for the 386: due to differences in architecture and timetable all the 686 family runs more than 10% faster on programs compiled for the 386 than on programs compiled for the Pentium.
Now that we are at it: RedHat 7.x are compiled with -mcpu=i686 (that means: optimize for the 686 family but using only 386 instruction so it runs everywhere). Using -march=i686 like you do only gives a 2% performance increase. For programs who have parts written in assembler (kernel, glibc, ssl) RedHat provides versions for the 686 and the Athlon and these are compiled with -march=i686 and -march=k7 respectively (ie full optimization).
Nearly forgot: the effect of the minor flags not included in -02 (except for -finline-functions who is part of -O3) is quite minimal when comparing the best combo of them to plain -O2 or -O3. From a memory it was couple percent at most. defintely not worth the risk since exotic combos are not so well tested and you have a chance of gcc generating wrong code.
The Windows TPC benchmark was on last years hardware (IBM xSeries 370, released last year).
The Oracle TPC benchmark is on next years hardware (HP ProLiant DL580R - not available till May 2003).
Err... are you talking about the TPC benchmarks refered to in the links? Both were run on 8 ProLiant DL580's with a total of 32 Xeon 900s. The Linux setup is marked in the full disclosure as being available March 5 2003, Hardware available now.
The big problem in getting the cost lowered by replacing Windows w/ Linux, Oracle w/ PostgreSQL, etc. is that most of these benchmark test systems cost huge bucks -- bucks contributed by the hardware & software manufacturers. If you were HP, would you rather foot the entire cost of putting one of these systems together with open source, or split the cost with Oracle and/or MS? Your hardware lands on the list either way.
The real challenge here is finding the money to build competing open source - based machines. MS has lots of monopoly money to dedicate to getting their products on the lists. Linux has no similar war chest.
Ok.. I may get myself in some trouble here... Please be kind.. Red Hat Advanced Server is the first version of Linux to meet all the TPC requirements and provide the performance level of this benchmark. Numerous open source performance technologies were incorporated, without which, the performance would be significantly below 100,000 tpmC. The 7x24 - 4-hour response, 3 year SW and HW maintenance is also required. The TPC audit (done by a certified TPC auditor on-site) requires the demonstration of a full system crash, as well as a single node, without losing a single committed transaction. The audit for TPC benchmarks take many days... I hope the Linux community doesn't miss the point that the TPC benchmarks are the toughest benchmarks to run and pass the audit. 138K tpmC will run 95% of the OLTP business environments today. This publication provides input to the many large corporations looking more closely at Linux in the "bet your business" application areas (databases). Although I agree that the absolute numbers of performance and price/performance may not be applicable directly to many real customer applications, it is a very fair environment (level playing field) to compare HW and SW performance and price/performance. This, IMHO, is a significant step forward for Linux in the enterprise.
Is there some reason that the article mentions that the Linux solution "beats" the Windows solutions, but everywhere on the site the Windows servers (whether it be IBM's DB2 or MS-SQL) are the leaders of the pack many times over?
That article link is like something a politician would say. Do you realize that Oracle is optimized for UNIX? It never should have been ported to Windows. Such a comparision is akin to saying "Apache 1.x runs better on Linux than on Win32!" No shit, Sherlock. Anyone who looks at the top ten results will clearly see that the point this article is trying to make means jack as far as the real numbers go.
By the way, I hear the latest Linux kernel is better at running Linux programs than CygWin!