Slashdot Mirror
HP Publishs First Linux TPC-C Benchmarks
The first ever official TPC-C benchmark on a Linux system has been published.
This was run on a cluster of 32 HP servers with Intel Xeon CPUs, running Redhat Linux and Oracle RDBMS. The system had over 18 terabytes of storage, and cost over 2 million US dollars. Performance was higher than a similar system running on MS Windows.
One wonders where the slowdown lies. It would be nice to find out if it lies withing the Win32 version of Oracle or within Windows. Seeing as Windows does have the top score in the Clustered TPC-C results, I'm leaning more towards Oracle being the slow one. (everyone else here would disagree no doubt)
If you look at the top results in a price/performance comparison, Windows 2000/SQL server are still winning.e sults.asp?resulttype=cluster&version=5
http://www.tpc.org/tpcc/results/tpcc_price_perf_r
It's interesting, though, that Red Hat had the cheapest Oracle implementation.
This just in: ditors at Slashdot ar no longr prmittd to us th lttr '' in articl titls!
On the other hand, the difference in performance is 17.21 vs. 18.46, i.e. approximately 6%. I think that the result is inconclusive (except that Linux can complete in this area at all).
Linux cluster:
Windows cluster:
Note that the number of clients in the windows tests is higher 24 instead of 16), with smaller CPU's. Also, the server's aren't identical.
Besides from the small differences in setup, it's plain that hardware-costs greatly outnumber software costs. Yeah, linux has a small bit more performance (less than 1%) for a bit lower price (6%) but these aren't real shocking numbers. Of course, I'll get flamed for not bashing microsoft, but the difference really isn't that big.
--
If code was hard to write, it should be hard to read
As many other people have pointed out, the system configuration was vastly different despite being 'similar' - yes, I want to see Linux out perform Windows but I would rather see it run on identical systems! When benchmarks are weighted like this, it just makes Linux look like the inferior product which is not the kind of image it should be getting for itself!
"Hey! Unless this is a nude love-in, get the hell off my property!!"
Of course, I'll get flamed for not bashing microsoft
:)
Don't be silly.
From this FAQ you'll see:
In general, TPC benchmarks are system-wide benchmarks, encompassing almost all cost dimensions of an entire system environment the user might purchase, including terminals, communications equipment, software (transaction monitors and database software), computer system or host, backup storage, and three years maintenance cost. Therefore, if the total system cost is $859,100 and the throughput is 1562 tpmC, the price/performance is derived by taking the price of the entire system ($859,100) divided by the performance (1562 tpmC), which equals $550 per tpmC.
Most people would focus on the hardware cost, but in reality the highlighted maintenance cost took the precedence.
Most midrange UNIX server has outragous maintenance cost. The maintenance cost of a UNIX server in the third year could be exceeding the cost of the hardware itself. It's due to the fact that older parts are difficult to find, thus make maintaining older servers more difficult. Besides, they really want to cut older production lines in favor of newer servers production.
x86 platform is known to have flat and lower maintenance cost, due to the low cost hardware and high compability with older hardware, i.e. older parts can be found easily. That's why Microsoft could easily beat the TPC pissing races.
Doomsday finally comes to Microsoft when Linux is entering the database market. Although at this moment big corps are still offering Linux maintenance with cost comparable to UNIX package, that's not surprising when Linux engineers are not as abandon as MCSE. However, it'll not be the case in the future. I think Microsoft would eventaully lose this pissing race in the long run.
There was a followup message on the lkml from hp discussing how these numbers can improved even further, you can see it at http://marc.theaimsgroup.com/?l=linux-kernel&m=103 237327405644&w=2
The Windows TPC benchmark was on last years hardware (IBM xSeries 370, released last year).
The Oracle TPC benchmark is on next years hardware (HP ProLiant DL580R - not available till May 2003).
Crowing about how performance is higher under Linux is FUD. It's not a fair comparison. Or didn't the story submitter understand that hardware always affects performance?
The reason to use Linux is all aspects of its openness and compatibility with other systems. With Linux, you aren't locked into a single vendor. You use tools and APIs that have been around for nearly two decades and are available, in multiple implementations, from dozens of vendors. And you control how you upgrade, when you upgrade, and what path you follow with the software. And if you don't like Linux anymore, you can switch to any of a dozen other, compatible platforms.
With Windows, you are locked into a single, proprietary implementation and Microsoft has you by the proverbial precious body parts; there is no other vendor you can get a compatible implementation of Windows or all the Windows libraries from. Every couple of years, Microsoft completely changes their computing paradigms to ape what they perceive is a threat from some other company, and when the threat is gone, they just drop the initiative and move on to the next thing.
You can get stability buy paying a premium to a company like IBM, which is committed to providing it, or through open systems available from multiple vendors or open source, where you control your future. But building a large, long-lived infrastructure on Microsoft platforms is a costly folly--the company has proven that they will change approach every couple of years and that they will force their customers to move along.
As far as price/performance it was in the middle of the top ten. I think if they had used PostgreSQL (or maybe SapDB, dunno haven't used it) instead of Oracle it could have been higher in the results.
> Yeah, linux has a small bit more performance (less than 1%) for a bit lower price (6%) but these aren't real shocking numbers.
Huh? The cost of the Windows software adds $150K more. That is a lot of money. Think of it this way: if I have two laptops studded with diamonds that cost 2.3 mil each (before the OS is installed), and the OS for one is free while the OS for the other is 150K - the latter seems very expensive when you look at the software budget
I think that you should break out the costs of these systems and look at the hardware and software seperately. "The systems cost the same in hardware, but there is a $150k difference in software." is a much cleaner analysis.
The Windows benchmark is running on a DL580 as well. That being said, your statement is valid. The systems are NOT identical. Click on the executive summary links at the bottom of each and you will see the detailed hardware specs for each.
Also, the clients are not only different in number, but the Linux clients are running a faster CPU as well (1400 Mhz vs. 1000 Mhz).
The only conclusion I think we can draw is that Linux plays "in the same ballpark" as Windows, performance-wise.
I'm not bashing you, I just don't see your point.
If you get a car for 20000$ at dealer A and the same car for 19500$ at dealer B, will you say "Hey, it's just a small difference" and buy from dealer B?
Let's not forget, this is about database-servers here and both run the very same database (Oracle). The underlying OS is irrelevant, you don't have to run MS Office or "the Sims" on this thing.
So please tell us stupid Microsoft-bashers what is your point.
Ok.. I may get myself in some trouble here... Please be kind.. Red Hat Advanced Server is the first version of Linux to meet all the TPC requirements and provide the performance level of this benchmark. Numerous open source performance technologies were incorporated, without which, the performance would be significantly below 100,000 tpmC. The 7x24 - 4-hour response, 3 year SW and HW maintenance is also required. The TPC audit (done by a certified TPC auditor on-site) requires the demonstration of a full system crash, as well as a single node, without losing a single committed transaction. The audit for TPC benchmarks take many days... I hope the Linux community doesn't miss the point that the TPC benchmarks are the toughest benchmarks to run and pass the audit. 138K tpmC will run 95% of the OLTP business environments today. This publication provides input to the many large corporations looking more closely at Linux in the "bet your business" application areas (databases). Although I agree that the absolute numbers of performance and price/performance may not be applicable directly to many real customer applications, it is a very fair environment (level playing field) to compare HW and SW performance and price/performance. This, IMHO, is a significant step forward for Linux in the enterprise.
Is there some reason that the article mentions that the Linux solution "beats" the Windows solutions, but everywhere on the site the Windows servers (whether it be IBM's DB2 or MS-SQL) are the leaders of the pack many times over?
That article link is like something a politician would say. Do you realize that Oracle is optimized for UNIX? It never should have been ported to Windows. Such a comparision is akin to saying "Apache 1.x runs better on Linux than on Win32!" No shit, Sherlock. Anyone who looks at the top ten results will clearly see that the point this article is trying to make means jack as far as the real numbers go.
By the way, I hear the latest Linux kernel is better at running Linux programs than CygWin!