Slashdot Mirror

Firstly, niether Oracle or MS post meaningful stats. Both of them require you to make a non-disclosure agreement with them - you are not allowed to reveal benchmarks.

Secondly, there are the TPC benchmarks, but take them with a huge pinch of salt. They are not about what the average IT shop can produce - they are about what hordes of MS/Oracle/IBM/Sun/Compaq engineers can produce with enormous budgets. They are also about finding the latest loophole in the TPC rules.

Thirdly, Oracle, at least allow you to download their RDBMS to try it for yourself - and that is exactly what you should do.

I wouldn't worry too much about limits on the number of rows in tables etc. unless you are storing absolutely obscene amounts of data - more than supermarkets in the UK who keep records of every purchase made with loyalty cards etc.

I'm sorry, but you just don't have a clue.
But if it's so vague, why would anybody pay thousands of dollars for it?
Quick question for you Jon,
If you were an advertiser, what information would you find more valuable:
a) Suzy Radcliffe age 9 likes to read Kuro5hin and keep abreast of the latest benchmarks on tpc.org.
b) Children who use altavista rather than yahoo also prefer pepsi to coke.
Well?
I think it's obvious that general information is more valuable than specific information.
--Shoeboy

I do believe that C# will adapt, but how can it compete with a 'relatively' open and cross-platform language

C# and the CLI have been submitted to the ECMA for standardization. It will be more open than Java.
The .NET framework is also being ported to other platforms.

Java is really one of the few great enterprise server solutions (for the internet) that is both robust and scalable. (I don't consider a 4-cpu Dell running windows 2000 scalable).

No, this is scalable.

That 400 megahertz processor operates on about 4 times more CPU instructions per clock cycle than your X86 chip. You're comparing apples and oranges. And I have bad software support problems on my IBM Aptiva running Windows that crashes every 5-7 days. What problems do SPARC chips have that x86 chips don't?

You are wrong. An Intel P3 or an AMD Athlon absolutely smokes the USII in terms of instructions per clock cycle. Fortunately most people don't use these machines for their raw CPU power, they use 'em for the I/O throughput. Current Sun architecture is quite a bit faster than current Intel architecture in addition to the fact that Sun uses huge amounts of L2 cache. The USIII and the Intel P4 will be on par with each other in terms of I/O throughput (they'll both be at 3.2GBps).

What exactly is standard about needing a massive image editing package with your server? Dumb statement.

How will you ever get a job in the real world when you equate Microsoft Paint with Oracle in the same sentence. I'm a sys admin and haven't touched a graphics program for work in over 5 years.

You seem to imply here that graphics programs are not valid applications for a server. A lot of very strenuous supercomputing that is done is directly related to graphics. I know quite a few people that are using SGIs on the same scale as the E10k to do rendering. Incidentally, SGIs work quite well for this as they have slightly better fp performance as well as a more scalable MP architecture requiring simpler programming (multi-threaded vs. clustered). The fact remains though that neither MIPS nor SPARC have the FP performance of an Alpha, Intel, or AMD which is what some people use to build render clusters (think Titanic).

Calling the original poster dumb was an argument ad hominem. It also makes you an asshole (another argument ad hominem).

Most Sun hardware is pretty reliable though it is overpriced in comparison to Intel hardware of equivalent quality. I can buy a dual-processor E220 for ~$20,000. I could buy an equivalent Compaq, HP, or Dell rack-mounted server for half that. It would have slightly better better CPU performance and slightly less I/O throughput.

Sun's specifically aren't good at hosting dynamic web pages because they can require quite a bit of CPU and relatively small amounts of data. Sun machines do better with huge amounts of data and relatively smaller CPU loads. The make great Oracle servers. Which is what I gather you are using them for.

Additionally, MS SQL Server 2000 on Win2k on a quad-processor Compaq (Xeon 700Mhz) can be faster than a quad-processor E450.

Yes, you are a Sun Bigot. You are also an asshole (once again, there goes that ad hominem thing again). You're almost as bad as a mainframe guy. People that think that they have a concept of system architecture but get their judgement clouded by their own zealous behaviour annoy the shit out of me.

Additionally, I'll provide something that you didn't provide while debunking the earlier poster's comments.

Check out specint and specfp marks on Intel P3s versus ultrasparc IIs. go to http://www.spec.org for the info. For database results that prove my point check out tpc-c benchmarks at http://www.tpc.org. Granted, they don't have results for Oracle 8i on a quad-processor E450, but they do have results from other rdbms vendors. The E450 scores about a third slower than the quad-processor compaq.

Have a nice day.

The www.top500.org only lists 1 NT system and 499 UNIX systems. And to keep things fair of the 5 "self-made" models there is 1 nt system and 4 UNIX systems. So, I guess that I'm not the only one who cannot tune NT properly.

How depressing. I wonder if Red Hat Enterprise Edition for Oracle would be applicable? I think that given the context, you may have little choice.
But why was the hardware platform selected before the OS? I'd say the mainstream recommendation is Solaris/SPARC. Windows, like Linux, is a red-headed stepchild to Oracle, though they won't say that.
I'm afraid Linux is totally indefensible as an enterprise DB platform for now. It doesn't even show up at tpc.
I'm sorry to say I predict a lot of pain with your Win2K servers. Maybe the key to keeping them stable is not running any unnecessary apps or scripts on them - use an external Linux box for all cleanup and monitoring tasks. If you experience the growth you're expecting, you'll wish you'd thrown these machines in the dumpster and bought Suns.

Quite. Let's see how well these "open source" databases do in TPC-C and -H.

Quite. Let's see how well these "open source" databases do in TPC-C and -H.

Of course Microsoft can sue. SQL Server is just plain better than Oracle. Has anyone actually tried to use Oracle? It's a nightmare compared to SQL Server.

At any rate, thank goodness for competition. Have you seen those numbers? Without compteition, we'd probably still be in the dark ages of 100,000 tpmC. Hah.

Of course Microsoft can sue. SQL Server is just plain better than Oracle. Has anyone actually tried to use Oracle? It's a nightmare compared to SQL Server.

At any rate, thank goodness for competition. Have you seen those numbers? Without compteition, we'd probably still be in the dark ages of 100,000 tpmC. Hah.

Of course Microsoft can sue. SQL Server is just plain better than Oracle. Has anyone actually tried to use Oracle? It's a nightmare compared to SQL Server.

At any rate, thank goodness for competition. Have you seen those numbers? Without compteition, we'd probably still be in the dark ages of 100,000 tpmC. Hah.

Of course Microsoft can sue. SQL Server is just plain better than Oracle. Has anyone actually tried to use Oracle? It's a nightmare compared to SQL Server.

At any rate, thank goodness for competition. Have you seen those numbers? Without compteition, we'd probably still be in the dark ages of 100,000 tpmC. Hah.

The conflict between Oracle and Microsoft on the topic of database performance has been extremely nasty of late. Essentially, Oracle was taken entirely by surprise when some respected tests revealed that SQL2000 on Win2k beats the pants of Sun/Oracle peformance.

You can see the benchmarks here: TPC tests

The margin by which Oracle is beaten must be pretty humiliating when they are competing for the same account. Oracle and Sun have attempted all sorts of dirty tricks to disqualify the results, and TPC even removed them at one point, but MS has always fixed the technicality and pumped out even higher results at even lower prices.

Hence Oracle's recent market promise that if they can't triple your speed they will give you a million dollars. Some of us inside have joked about setting up a really badass SQL service and getting that million as a stunt :-)

At any rate, I am not surprised that the two companies eventually are coming to slights and legal maneuvering. Oracle knows quite well that they are one of the companies Microsoft has marked as "Make irrelevant in five years" and they really don't want to go the way of Lotus, Corel, etc.

-konstant
Yes! We are all individuals! I'm not!

Look, it's larry that's being a jerk. He can't take it that Microsoft owns all the top benchmarks set up by and independent (truly) testing organization that's been around a while. And if you look at the bang for buck figures, it's not even close. all TEN of the top ten are SQL server (some version) see the results yourself As for an app demonstration, I can write an app that will run faster with (piece of crap) microsoft Access than Microsoft SQL server.
---

SPECWeb99 is a terrible benchmark for making a decision of which web server you want to run for a real business. Almost all of the pages in the SPEC benchmark are statically generated. A more useful benchmark is TPC-W, which essentially is Amazon.com (see the TPC website for more info). TPC-C is the standard benchmark for measuring databases, and TPC-W is likely to become the standard for web servers.

The software is targeted at businesses and vs. Oracle 8i and IBM's DB2, Microsoft SQL Server not only is much cheaper but according to the Transaction Processing Performance Council tests performs comparably.

This software is targeted at businesses that routinely pay tens to hundreds of thousands of dollars for software. Not college kids who want a DB to host their guestbook application.

What was the purpose of your post besides displaying your ignorance?


(-1 Troll)

If you visit TPC.org , you will find that they don't have one benchmark, but rather about four, with substantially different purposes:

• TPC-C is intended to determine throughput of a transaction processing system in creating transactions;
• TPC-H measures performance on what is intended to be an "ad-hoc DSS environment."
• TPC-R measures performance on "business reporting," intended to be more like "typical DSS reports."
• TPC-W measures performance on a "web transaction" workload.

The notion that there can be a comparable benchmark between the databases is something of which people should disabuse themselves.

If you need to have high performance transactional behaviour, I would point out that ODBC is NOT the issue; regardless of whether the SQL-CLI drivers suck, the important point is that neither database fully supports the industry standard SQL/XA or X/Open DTP and XA standards.

Serious transaction systems commonly use transaction monitors like BEA Tuxedo or Encina, interfacing via XA to a relational database (like Oracle, Sybase, DB/2, Sleepycat DB, TimesTen, ...). From that perspective, MySQL and PostgreSQL are both still "toys," although SDTP - A Multilevel-Secure Distributed Transaction Processing System outlines how an XA interface to PostgreSQL was constructed in Common Lisp for use in a set of applications running on FreeBSD.

If you build a benchmark based on an application exercising the strengths of MySQL, it will probably perform badly when used with PostgreSQL, and vice-versa.

Take these systems seriously when they start supporting things like XA, and when BEA makes Tuxedo available for use with them.

From the article:(I use Oracle here as an example, as they press release implicate that they are using Oracle for testing; If not, they are not testing against the proprietary database leaders).

Did anyone read my comments here? If not, check out the tpc.org site. Look at the top ten results by price/performance and the top ten results by database vendor. Gosh -- MS SQL hands down. So why is it taken as an article of faith by many on sladhot that Oracle is the best? (what happened to "the best tool for the job?") Oracle is a fine database, but I can't in good faith reccommend it to a client when MS SQL is SO MUCH cheaper and faster.
---

From the article:(I use Oracle here as an example, as they press release implicate that they are using Oracle for testing; If not, they are not testing against the proprietary database leaders).

Did anyone read my comments here? If not, check out the tpc.org site. Look at the top ten results by price/performance and the top ten results by database vendor. Gosh -- MS SQL hands down. So why is it taken as an article of faith by many on sladhot that Oracle is the best? (what happened to "the best tool for the job?") Oracle is a fine database, but I can't in good faith reccommend it to a client when MS SQL is SO MUCH cheaper and faster.
---

From the article:(I use Oracle here as an example, as they press release implicate that they are using Oracle for testing; If not, they are not testing against the proprietary database leaders).

Did anyone read my comments here? If not, check out the tpc.org site. Look at the top ten results by price/performance and the top ten results by database vendor. Gosh -- MS SQL hands down. So why is it taken as an article of faith by many on sladhot that Oracle is the best? (what happened to "the best tool for the job?") Oracle is a fine database, but I can't in good faith reccommend it to a client when MS SQL is SO MUCH cheaper and faster.
---

I posted this yesterday but probably no one read it due to the fact that it was one of the latter posts (past 400). Anyway here goes...

According to the original article the names of the proprietary databases benchmarked were not given because it violated licensing agreements. Flawed benchmarks like this are the reason why.

As someone who has downloaded, installed and used Oracle 8i, IBM's DB2 and Borland's Interbase I can testify that configuring any of these DB's properly is a non-trivial task that can easily be messed up by someone who has no idea what he/she is doing. Using non-native drivers, not indexing tables properly, improper tablespace sizing, choosing an improper number of data files, improperly managing data blocks, etc can all lead to creation of a suboptimal database application performance.

Most of the major DB companies provide DB's for independent benchmarking from organisations like the Transaction Processing Performance Council. As can be seen from this story these tests involve several thousand transactions per second and not several hundred as reached by this Great Bridge sponsored benchmark. I suggest that someone with a deep pockets or a vested interest in seeing Open Source DBs succeed should enter PostgreSQl or MySQL in these TPC-C tests.
The Queue Principle

Was it not www.tpc.org that ran these tests? In that case, I don't put much stock in them. Take a look at the top ten results by price/performance or even the complete results by database vendor shows no mention of Postgres. I'll believe it when I see these results ratified by the TPC.
---

Was it not www.tpc.org that ran these tests? In that case, I don't put much stock in them. Take a look at the top ten results by price/performance or even the complete results by database vendor shows no mention of Postgres. I'll believe it when I see these results ratified by the TPC.
---

Was it not www.tpc.org that ran these tests? In that case, I don't put much stock in them. Take a look at the top ten results by price/performance or even the complete results by database vendor shows no mention of Postgres. I'll believe it when I see these results ratified by the TPC.
---

Quad Xeon machines are doing around 25,000 transactions per minute on the real tpc tests (here) so for a 2-cpu machine to do 300 per minute is not terribly impressive. I think it's probably the trivial hardware that was holding those test back, though, rather than postgres per se.

With only two disks the tests were almost certainly disk-bound, which would explain the striking similarity in the TPC-C results for all three vendors. I doubt any of the database systems really got a chance to hit their stride.

So the bottom line is that we still don't know what postgres can do given reasonable HW, by which I mean at least 4 CPU's, 2GB memory, and 16 disks.

Hello, Great Bridge?

...as someone who has downloaded, installed and used Oracle 8i, IBM's DB2 and Borland's Interbase I can testify that configuring any of these DB's properly is a non-trivial task that can easily be messed up by someone who has no idea what he/she is doing.

Most of the major DB companies provide DB's for independent benchmarking from organisations like the Transaction Processing Performance Council. As can be seen from this story these tests involve several thousand transactions per second and not several hundred as reached by this Great Bridge sponsored benchmark.
The Queue Principle

What more can be said? I think you've done the Open Source community the biggest favour anyone could: you've proven our tools ARE INDEED enterprise-ready.
Betcha Ellison and Gates (oh, come on, couldn't you have figured that one out yet?) are gonna sweat over this one for a long time... Well, bwaaahaaaaaa loosers!

I'm not very clear on the numbers, though. They say PostgreSQL achieved 1127.8 transactions per second, or 6766.8 transactions per minute. Recent TPC-C benchmark shows IBM with DB/2 an absolute winner with 440879 transactions per minute, Compaq came next with MS SQL 2000 with "measly" 262243 transactions per minute.

I'm quite sure I'm getting something wrong here, just don't know what exactly. Any knowledgeable DBAs here to enlighten us, common mortals?
-----

This 'article' is nothing more than a press release from Great Bridge.

There may be some additional information learned by reading the results of the benchmark from
http://www.tpc.org/New_Result/TPCC_ Results.html

Although I am having a hard time finding any reference to Postgres on that page. Can anyone find any better references?

-k

You may not like Microsoft, but Windows 2000 is a very capable operating system. The fastest databases in the world run on w2k. Sure, FreeBSD might be even faster at certain things, and being free is a big win, but this doesn't mean that w2k sucks. Heck, even Lycos is switching to it.

Slashdot has become the Weekly World News of tech sites. What a pot of crap.

My recommendations if you are on a budget: Stick with Linux and Sybase and get some vendor support. Definitely stick with x86 hardware since you are on a budget. The size of the database is less important than the actual design. How much data is going to be used at any one given time? Figuring that out will tell you if you need to add another gig or three of RAM. A good dual-processor machine should be sufficient, perhaps a quad if there are lots of simultaneous users. Bottlenecks in a database are rarely at the CPU.

IMHO, you should concentrate on your RAID setup. Get ~20 4GB disks and set them up with RAID 10(full mirroring+striping). That alone is going to give you much, much better performance than a solution with say 4 20GB disks. At $200 per disk this will run you about $4,000. Paying careful attention to this will get you your best database performance while still spending a hell of a lot less than you would with an RS/6000.

You need the performance, but obviously you can't fit the whole database in RAM. So get a good RAID controller and buy as many small disks for it as you can.

Try the TPC-C reports here. These are not specifically comparisons between vendors, but they are all reports of systems running a common benchmark. The reported benchmarks all went through approved TPC audits before being released. (On an aside, Microsoft just withdrew their results on SQL Server 2000 because they failed the audit, it's on ZDNet somewhere, I don't have the link).

• He gave a very vague description of the layout of his database and the nature of the queries. Without that, it is impossible to reproduce his results.
• He did not describe his client/driver configuration, for all we know, they could have been programs running on the same machine as the DB server
• For database benchmarks such as TPC-C (you can see the stringent reporting specifications at www.tpc.org) it is common to allow the driver program to warm up the database for a long amount of time (45 - 60 minutes seems to be common now, depending on the size of the database) then allow the database to reach steady state after at least 20 minutes. He does not mention whether he gave time for the database to stablize. Transient numbers are not a good indication of actual system performance.
• His reported statistics are lacking. Saying that PostgreSQL is 2 - 3 times faster than MySQL leaves a 50% uncertainty in his measurements. At times, he reports 2 - 3, and others he reports simply 3. In fact, I don't see any solid measured numbers for queries/second, transactions a second or some other metric.

What does the server DO?

How much I/O is done per transaction?

How much CPU-bound processing is done per transaction?

Does the server use threads? Is there a pool of threads (how many) or is a thread created for each transaction?

What OS calls does the Linux port use? Which socket calls? Is a new connection created for every transaction? select() or poll()? blocking or nonblocking sockets? etc, etc...

Is the data file opened with O_SYNC to emulate transactional properties?

How were the binaries compiled? Which compiler?

Given that this site seems to be very windows centric, how do I know that the Linux port is done properly by someone who know what she is doing? Can I see the source please?

It would also be nice to see some profiling data from the benchmark. Where is the bottleneck?

CPU utilization

Kernel/user mode utilization per processor

network packets per second

disk traffic per second

Also note that their term "transaction" is probably not a database transaction with ACID properties (as the industry standard TPC benchmark mandates). An ACID transaction needs to store updated data permanently on disk (not file buffers) before the transaction commits. I don't think the test does this.

I am implementing a database manager on Linux and there are some areas where I think NT has better capabilities than Linux (disk I/O for ACID transactions), but that is another story, since this benchmark probably doesn't do ACID transactions.

per@nospam.mimer.se
www.mimer.com

It's also worth noting that until June 30th, Microsoft Windows 2000 running SQL Server 2000 on a Compaq machine had taken the price/performance lead AND the sheer performance lead. Alas, I haven't read SPEC's rules yet, but their disqualified numbers are here.

Yes, I know... I found this out on the Register.

But it looks like IBM's toasted those disqualified numbers anyhow... cool!

In that case, check out the last set of Win2K results which Compaq/MS have withdrawn, presumably because either a) they've been overshadowed or b) they want to put something even faster up. Effectively, at this moment in time, Win2K has the top three performance slots and the top ten price/performance slots.
--
Cheers

I do not understand what the benchmark is trying to measure.

They claim to be measuring transactions per second. The industry standard for measuring transactional processing power is TPM (per minute), and the T applies to ACID transactions, which have absolutely nothing to do with application serving. See the Transaction Processing Council for more info.

Mixing apples and oranges in an attempt to confuse a few hapless MIS manager types.

I for one never disagreed that Windows is a fine application server. If nothing else, there's lots to serve. But as far as transactional processing, I would not bet my MIS career on M$

This just went up on the TPC website Monday, there is a monster leader in transaction processing price/performance and that is:

IBM Netfinity with Intel Xeon processors
IBM DB2
and Windows 2000.

You will not believe this unless you see it!


Read'em and weep

Now this was the benchmark that Ellison was saying "Beat us and we'll give you cash" on, yet when MS did it, he didn't pay *grin*

For TPC-H see http://www.tpc.org/new_result/h-ttperf.i dc (although perhaps Orcale hasn't submitted any).

For an explanation of the TPC-C benchmark see http://www.tpc.org/faq_TPCC.html.

Satisfied?

Now this was the benchmark that Ellison was saying "Beat us and we'll give you cash" on, yet when MS did it, he didn't pay *grin*

For TPC-H see http://www.tpc.org/new_result/h-ttperf.i dc (although perhaps Orcale hasn't submitted any).

For an explanation of the TPC-C benchmark see http://www.tpc.org/faq_TPCC.html.

Satisfied?

Now this was the benchmark that Ellison was saying "Beat us and we'll give you cash" on, yet when MS did it, he didn't pay *grin*

For TPC-H see http://www.tpc.org/new_result/h-ttperf.i dc (although perhaps Orcale hasn't submitted any).

For an explanation of the TPC-C benchmark see http://www.tpc.org/faq_TPCC.html.

Satisfied?

Now this was the benchmark that Ellison was saying "Beat us and we'll give you cash" on, yet when MS did it, he didn't pay *grin*

For TPC-H see http://www.tpc.org/new_result/h-ttperf.i dc (although perhaps Orcale hasn't submitted any).

For an explanation of the TPC-C benchmark see http://www.tpc.org/faq_TPCC.html.

Satisfied?

Ah Well. So much for getting all sides of a story on this forum.

This guy is right: check out tpc.org and you will see an industry standard ranking of database servers. Windows 2000/SQL Server does indeed blow away all other contenders, running the heaviest IBM and Sun iron and Oracle.

However, I don't think Sun is really in trouble. Rather, Microsoft is. The are still the belligerent company they always have been, and I for one have developed a severe distaste of having to deal with incompatible software and devious methods to hook people into staying their customers. So what if it performs better, I need software that works well with everything else I use.

Take a lesson from IBM, and quit trying to dominate the world. Treat people as humans rather than competitors to be crushed, and perhaps the anti-Microsoft sentiment will fade. Until then, expect people to go with other solutions.

Take a look at the benchmark results on http://www.tpc.org/ and you'll see some Windows doing some *real* work faster and cheaper than many UNIX based solutions.
</flamebait>

The overall performance will depend on the type of applications you are running. To that end I also wondering if are you planning on running any standard benchmarks and making the results public? I would be particularly interested in seeing the results from the TPC-C benchmark (http://www.tpc.org). I'm not sure if it will be even possible to run this benchmark on your system since I don't know how it is configured but it would be nice to see how your system compares in terms of enterprise computing solutions.

I checked the numbers, did you? Clock for clock, the PA-Risc outdoes the alpha at integer ops. For database systems, this is generally more relevant than floating point speed.

Yeah, and I checked the other numbers too. All the evidence I see suggests that the top reported Alpha numbers are higher than the top reported HP numbers. In fact, Alpha seems to be ahead of HP in spite of the fact that the most recent offical TPC numbers for Alpha are almost two years old. (I'm really looking forward to seeing transaction numbers for the GS series.)

I fail to see how "work per clock cycle" is a more relevant measure of performance that "work per unit wall clock time" (aka "throughput").

That's better

Well, SQL Server is easily the fastest RDBMS out there (ye s, much faster than Oracle 8i and MySQL</A>), as well as being far cheaper than Oracle.

Why do people use Oracle+NT ?

How exactly is Microsoft SQL Server not a quality product? Be specific, I'm not trying to flame you, but rather I'm looking for good information.

Might surprise some of you Oracle gurus to know that your vaunted database platform is no longer the fastest DBMS. Windows 2000/SQL server trounced Oracle running on the heaviest Sun and IBM iron out there. Check out www.tpc.orgfor the results. This was the very test that Oracle used for years to claim it was superior to other database platforms.

Windows 98 being low quality, yes. I would never program on a system that crashed every time I made a programming error. But SQL Server? I can't get the thing to crash, and I've tried.

Ron Soukup, a lead developer of SQL Server and the author of Inside SQL Server, has repeatedly stated that SQL Server has never used unpublished APIs to acheive its performance. This is illuminating, since SQL Server is perhaps Microsoft's most complex application program. I personally believe Ron's statement, and it would surprise me if you contested it.

If SQL Server can acheive its world dominating performance without using unpublished APIs, I really think that makes the whole "MS uses unpublished APIS to its advantage" argument moot, don't you think?

That said, there is a great site that compares the servers and databases you mention, and will likely give you the stats you are looking for. Its www.tpc.org.