Slashdot Mirror
Postgres Beats MySql, Interbase, And Proprietary DBs
Mike writes: "An independent organization tested Postgres 7 vs. MySql, Interbase, and two leading commercial databases using the ANSI SQL Standard Scalable And Portable benchmark and found that Postgres was the clear winner. In fact, Postgres was the only open source database to offer similar performance to the two commercial applications. The results are detailed here."
You should try backing up your claim. Obviously, MySQL is a toy database. It is used for fast tabular data access, not true database work. What makes PostgreSQL a toy? It supports things such as table inheritance, very nice transaction support, and other things.
I'm curious, what makes open source automatically have lesser products? I'm not saying that currently PostgreSQL has everything you may need, but does that have to do with open source or just the specific project? Also, I'm not impressed with your amount of data. If you are using a pre-written application, most of them hold and keep _way_ more data than they ever need to. Oracle Applications, for instance, for our 300-person company, took up 1 Gig just for the table definitions! So your 4Tb system could be more from mismanagement than having that much useful data.
Actually, for database applications, the critical component is the hardware (assuming you have a _real_ RDBMS - not MySQL). Sun hardware beats just about anything. If you want to see PostgreSQL being used in the real world, take a look at http://www.pgsql.com/
Have you taken a look at the features in PostgreSQL? They are very, very nice.
Engineering and the Ultimate
Even if there have been "a pretty huge amount of changes" as you state, it's still marked as a beta product. As for myself, I wouldn't use a beta product in a production environment until it's been marked as stable by the developers, no matter how stable other people might say it is. That's why I use MySQL v3.22 right now. If something is marked beta, there is more than likely a reason for it. It would be irresponsible of me to risk using it and risk losing customer's data. I'm sure other businesses and individuals can't afford to take those risks with their data either.
Prevent email address forgery. Publish SPF records for y
9 out of 10 people say Statistics can be made to mean anything you want.
2 out of 13 people don't understand statistics.
45 out of 51 statistics are made up on the spot.
3 out of 4 people don't know how to spell Statistics.
A Dick and a Bush .. You know somebody's gonna get screwed.
War is necrophilia.
>They don't whine about their competition not
>being "SQL compatible" (a debatable term in any
>event). They don't lie about how the competition
>"isn't relational." They don't sneer about how
>the competition is "just a front end to the file
>system."
Have you read the MySQL docs?
Their section on why not to use foreign keys at best makes me laugh, at worst makes me cringe. This sentiment is even shared by most of the MySQL fans I know.
They gloss over their lack of sub-selects with an example that doesn't require the feature.
They gloss over the lack of transactions and commit/rollback syntax by suggesting table locking, which simply is not practical in high volume environments. Moreover, their examples neatly omit the idea that you may need to lock a *LOT* of tables, stalling updates to most of the database in order to circumvent the lack of MVCC.
Whether you want to admit it or not, the drawbacks in MySQL are *VERY* real, and the MySQL documentation trys to play them off as minimal annoyances.
I find their lack of responsibility in this more offensive than a few causally posted insults.
>Anyway, if there was a standard benchmark to run
>against both DB's from a vanilla install (and by
>that I mean running './configure' with no flags)
>then I think that would go a long way towards
>giving a more balanced view.
The tests they used ARE standard database benchmarking tests - AS3AP and TPC-C.
As for configuration, I can't rightly say what they did with it, as I wasn't there.
However, running them without any options does not necessarily put them on equal footing. For example, Postgres by default does an fsync after every single write, whereas MySQL does not. This SERIOUSLY impacts performance.
Not to mention the fact that in the real world, it's not really a viable option to recompile programs/kernels every other week or even shut the server down more than once a month.
- My password is slashdot
A year ago - that's about the timeframe in which some VERY serious memory leaks were uncovered and fixed, at the prodding of some folks using it in a web environment.
When you've got memory leaks, your server will go down, it's just a matter of time. Once the leaks were fixed, the folks I know using AOLserver+Postgresql who were forced to reboot periodically found their problems disappeared.
We at OpenACS (http://www.openacs.org) have been running our Postgres installations for months at a time with no need to reboot.
It truly has improved dramatically in the last 15 months or so.
Am I a biased Postgres fanatic? Not exactly - when I evaluated 6.4 in January, 1999 I decided it was useless for web work, far too slow and crash-prone. When 6.5 was released it was so dramatically improved that I changed my mind.
And, the OpenACS web toolkit project intends to support both Postgres and Interbase (now that the latter's Open Source), so we're not Postgres-only bigots.
...as someone who has downloaded, installed and used Oracle 8i, IBM's DB2 and Borland's Interbase I can testify that configuring any of these DB's properly is a non-trivial task that can easily be messed up by someone who has no idea what he/she is doing.
Most of the major DB companies provide DB's for independent benchmarking from organisations like the Transaction Processing Performance Council. As can be seen from this story these tests involve several thousand transactions per second and not several hundred as reached by this Great Bridge sponsored benchmark.
The Queue Principle
The review says "proprietary" because they're not allowed to release the names. It's true, I've read the contracts (I use Oracle as well as Postgres, depending on whether or not I'm picking up the tab). It's not their fault.
Clearly, if they were open source, their names could be revealed. They have to use SOME word to describe them and owe their users an explanation as to why they don't name names, don't they?
These contract terms really suck, and that's where you should be venting your anger, rather than accusing the benchmark authors of "FUD" just because they adhere to contract terms.
They explained this right in the article, too, maybe you should've read it instead of falling asleep.
Were you using PG 7.0, or PG 6.5? The optimizer's been improved in PG 7.0, and in my case one particularly annoying query that had a very poor plan chosen by PG 6.5 had a very good plan chosen for PG 7.0.
You can turn off various join strategies in PG 7.0 via "set" commands, not as flexible as Oracle's hints notation to the optimizer but it can help in situations like you describe. At least you get SOME control over the optimizer.
frankly... i somehow doubt they will be able to fix everyon's concerns.
as many have said before you cannot add transactions to a non-transaction database. transactions are complex entities which require pervasive changes to the code base, and require some _real_ thought how to deal with the issues they bring up. MVCC for example...
anyways, we have postgreSQL, its advanced, its cool, it has a better interface than mysql! why not improve postgreSQL rather than hacking up mysql to make it do something it fundementally wasnt designed to do?
version 7.0.2 is the current reccomended production stable release.
"It looks like they got those DB/2 results on a system with 128 Xeon CPUs and a system price tag of just $14,232,696! I wonder how well DB/2 would do on the same hardware as the Postgres tests."
Conversly I'd like to see how well Postgres would do on that hardware.
as another poster said, maybe encapsulating your queries in a transaction would help?
maybe you should try version 7.0.2? Its much better, I have been using it, and there are no problems.
As for the 8k issue, if you need to stick blobs in, thats what large objects are for, but if you need just a bit more, you can increase max tuple size via a compile time option.
....but what about real world databases?
I routinely work with large (4Tb) Oracle and Sybase databases and, to be perfectly honest, these figures don't mean a whole lot to us people living in "The Real World" that use proper commercially supported databases rather than toy MySQL and PostgreSQL databases.
This isn't meant to be flame-bait, but it's about time that you lot in the geek open-source world realised that open source isn't always the be-all-and-end-all!
Open source may well be the buzz-word of right now, but I fear for any company that relies on either MySQL or PostgreSQL to look after their data.....
Oh, and I don't mean to whore karma, but if this gets moderated down it'll show once and for all that the regulars that visit this site are well blinkered towards Linux/Open source and really can't see the real world.
Foxpro is quite ridiculously fast.
It beats SQL Server for many basic tasks and can cope with tables up to 2GB in size quite happily.
It can also use ODBC to connect to the back end of your choice, allowing scaling to SQL Server or Oracle or whatever you like.
If you'd like to select some data, I'll happily run off some benchmarks for you.
My Journal
Mama says that you've been had if you can't talk about it. People like that just are not your friends.
Was approval denied?
Friends don't help friends install M$ junk.
Pfhreakaz0id,
..." tells me you didn't read the actual article itself. The original article answers your question: "The tests were conducted by Xperts Inc. of Richmond, Virginia, an independent technology solutions company, using Quest Software's Benchmark Factory application..."
Go back and reread the original article. I don't mean to flame, but I hate it when people say things ignorantly. So here's where I rant.
Your first question of "Was it not www.tpc.org
Secondly, I looked at both those links... They both have prices attached to the products. The first "top ten results by price/performance" stuck strictly with MS SQL Server, which we all know doesn't run on Linux. Besides, how do you rate a product in price when it has no price? Postgres can be downloaded and used freely. And I would like to point out that both lists that you link to are commercial databases. If you reread the original article posting, Postgres was rated as a non-commercial database, as an alternative to commercial databases.
Rant over. Just make sure you read things in the future.
Seth Anderson BTW, I'm not 23 anymore -- I am TexasCowboy26 now. =)
I read the article. I was asking rhetorically. My point is, they were running the TCP benchmark without the benefit of TPC's expertise. I'll wait until the TPC itself runs it. Also, the price/performance thing is hardware/software. Theoretically, that should give free(beer) databases an edge, right? hmmmmm, wonder why the top ten is MS SQL? could it be it's the best product for the money? Nah.....
---
DO NOT DISTURB THE SE
It is indeed benchmarketing.
The names of the two 'industry leaders' couldn't be released due to restrictive licenses, which to me means one of them is Oracle. (Not that that would be tough to figure, even without prior experience with them).
Icebox
Interbase was one of the databases they tested:
open source competitors MySQL 3.22 and Interbase 6.0, demonstrated some speed with a low number of users but a distinct lack of scalability...Neither Interbase nor MySQL could be tested for TPC-C benchmarks...Interbase 6.0, recently released as open source, does not have a stable ODBC driver yet; while Xperts was able to adapt the version 5 ODBC driver for the AS3AP tests, the TPC-C test would not run. 'With MySQL it's an inherent design issue. Interbase 6 should run the TPC-C test, and perhaps would with tweaking of the test's code.
rage, rage against the dying of the light
It was added a while back. And still, mysql (when using MyISAM) is a lot faster than competing databases. Sorry I've done my independant benchmarks in the "real-world".
Don't believe me, just test them out for yourself. MySQL opens a can of whoopass, but people just don't realize that I guess...
I really strongly encourage everyone to benchmark their database servers independantly, instead of trusting these "independant" companies like.... well we all remember MindCraft, can we trust these organizations?
Ever need an online dictionary?
A Dick and a Bush .. You know somebody's gonna get screwed.
War is necrophilia.
hello, what about Progress?
you can love your data + the transactions + not have to shell out so much green.
The tests were performed using ODBC drivers. ODBC drivers vary widely in quality but are always inferior to the native API of any given database. For Interbase they used the ODBC driver from Interbase 5. This is a notoriously poor performing driver. I've found Interbase to be as fast as SQL Server 7 and nearly as fast as Oracle 8i when accessed using native methods.
I am very dissapointed that slashdot posted this article. It is a marketing ploy buy a new company trying to make a buck off Posgres.
There is little or no real comparison data in this report. There are paragraphs about "Postgres performs 4-5 times as fast as Mysql and others" What are we comparing here. Air? They don't even tell you what type of machines were used or OS for that matter. We are simply left to believe what we read?
If I was looking for a benchmark from Oracle or any other database (or CPU for that matter) I would expect Graphs and numbers representing the facts. Not paragraph after paragraph of non-technical data that investors and CEO's like to loot at. Come on guys. this is not data that is relevant to any developer.
Anyone who knows how to create and use a true enterprise database and has toyed with postgresql has known these results. Mysql, interbase, are truly just sql frontends to the file system. Both are screamingly fast for basic tables with no real design. Once you begin to join tables, perform concurrent queries, etc. they come apart like a firestone tire.
Although they are only the beta version. ftp://ftp2.interbase.c om/pub/products/beta6.0/ib_b60_doc.zip
About 10 meg, PDF format.
Mirrored at Interbase2000.org
>I had heard that Postgres was slow as hell and a
>serious resource hog, but I'll have to do some
>testing of my own. Is there a Postgres admin here
>who'd like to tell us what kind of resources
>Postgres demands?
Best tests are always your own. But the memory footprint is actually quite small.
SIZE RSS SHARE
2196 2132 1672 postmaster
444 300 252 postmaster
156 4 4 pg_ctl
The first process is an active connection to the datbase. Only uses about 524k of non-shared memory.
The second is the process that listens for incoming connections, and the third is the control program. You'll only have one of each of these.
E.g, when serving 100 simultaneous connections, the postmaster processes should use around 52MB, including the shared libraries.
Of course, you'd probably also want enough memory to keep your indexes cached. That'll depend on the size of your tables.
As for CPU utilization, I haven't really hit any walls there, since I only support a handful of clients, but I rarely see utilization by one postmaster process even break 2%, and thats on a complex five table join doing regular expression matching on two fields.
3 out of 4 Teletubies walk the straight and narrow.
--
Sheesh, evil *and* a jerk. -- Jade
I'm no lawyer, but I would have to wonder if there isn't a constitutionally protected right to free speech that prevents this sort of "license" from having any force.
I do not have a signature
*IF* you are in a production environment, you need a *real* RDMS, NOT MySQL (a file system with a SQL interface).
PostgreSQL sounds cool, will look deeper for my next DB server.
Visit DC2600
Eve Fairbanks says I drive a hybrid!LOL
Check that. If so, try benchmarking on an Alpha system rather than an Intel box.
--
Ben Kosse
--
Ben Kosse
Remember Ed Curry!
That is because it is not a REAL RDBMS.
Visit DC2600
Eve Fairbanks says I drive a hybrid!LOL
I think that this is a load of crap. From the TPC home page The TPC offers a full membership for a yearly fee of $9500 and an associate membership for a yearly fee of $1500.
$1500 is a lot of money for a free sofware project to shell out every year.
Maybe GreatBridge could join TPC on behalf of the PostgreSQL community(Yuck. I hate that word).
Its just not a really fair representaiton since they used OLD MySQL, and they used lame drivers for interbase giving the only competition to be the two closed proprietary databses which we can assume are... Oracle and SQL ? Or is that what they want us to think, I dont know, im gonna compare those numbers to some tests we run at my work.. I amnot sure if postgres is all that just yet, but ive always had a suspicion it can really perform. *is lame and replies to peoples sig's* ;)
:)
I just wanted to say Dylan Thomas was cool and that is pretty much my favorite poem
If you think education is expensive, try ignornace
Gee, that is useful... any database ought to give awesome performance under those kind of conditions. In fact, all DB optimizations boil down to 2 things: 1) minimize number of block reads necessary for a given query (indexing) and 2) Reuse data from a single read many times (caching). If an application requires performance in excess of what a database system can handle while entirely in memory, it is time to think about using specialized data structures.
In fact, I got the O'Reilly book on MySQL(don't get this one, get the New Riders book if you need to have a book.) and I found myself referring to the MySQL help file for most things. I certainly trust it more than the book for correct information.
No, Thursday's out. How about never - is never good for you?
My experience with Access (97) was that it is not to bad until the database size exceeds what can be kept in RAM.
I find this hard to believe. I've been an Interbase fan for a long time, and it is FAST. PostgreSQL has always seemed huge, slow, and klunky to me. Again, I'm flabbergasted.
F I R S T P O S T ?
This is pretty surprising to me. The reason I originally chose MySQL was because of Linux Journal's reporting of how it stacked up with Oracle and the other commercial DB's. Hmmmm...
--
Quantum Linux Laboratories - Accelerating Business with Linux
* Education
* Integration
* Support
*Condense fact from the vapor of nuance*
Actually, the AS3AP test that was run was the IR (information retrieval) test, just quick reads. See http://www.benchmarkresources.c om/handbook/5-3.html.
One would think that would be an environment in which MySQL would shine. But the results are actually not out of line with an earlier test run by PC Week.
Regards,
Ned Lilly
VP Hacker Relations
Great Bridge
...Gilbert of Great Bridge!!!
Cool name.
-Derek
They used ODBC drivers for testing. IMHO I don't think anybody uses ODBC drivers except on MS platforms.
"using each product's own ODBC ensured the most valid 'apples to apples' comparison."
I've used the ODBC drivers for MySQL and they're were a joke. This may, ofcourse, have improved since I last used them about 2 years ago, but using the ODBC drivers for comparing these databases is the worse possible way to compare them.
I personally would prefer a second test from a different outfit before accepting the numbers. I understand your misgivings; I also have them. These "independent" studies are at least more independent than the original poster thought. He was implying that Great Bridge had done the testing.
Moderators, start stocking up on your points, because it'll get ugly.
--
Ben Kosse
--
Ben Kosse
Remember Ed Curry!
<bart
--
--
Somehow I'm not surprised. Of course they didn't mention the "proprietary datarbase leaders" names (MS-SQL and Informix anyone?).
My rules for picking databases for the last several years has been:
If you really care about your data and have $: Oracle
If you really hate your data and don't need transactions: MySql
If you like your data but don't have the green for Oracle: Postgres
OBSlashdot- (Useful first post)
People. I hate to break this to you...
.75, not bad for a P300 with 128M.
But perl's DBD::CSV (comma seperated values) is just about as powerful as MySQL.
Take a look at the tables - they are 2 files, they aren't even tabulated relationally in a database (the 'R' in RDBMS).
For instance - take the table files from one database, slap them in another database directory. Blammo! You can use that table there!
(NO - THIS SHOULD NOT BE A FEATURE)
Where are the stored procedures? the subselects? the objects?
WAIT! They aren't there! Why you ask? BECAUSE MySQL IS NOT RELATIONAL.
MySQL IS NOT A RELATIONAL DATABASE. It's a daemon used to interpret network-based sql queries and insert/retrieve plaintext data in files. It then formats this data and presents it to you, the user. This is what makes it so fast. It's also what makes it useless for any real world application.
Fast, whatever, but it's not a *real* database in the terms of these benchmarks. If you want a real database, you use something that's not going to crumble while doing simple operations such as locking tables or handling 100+ users in a join of 2-3 20+ column tables.
Our oracle box does this fine at work, in fact, constantly, and never exceeds a load of
My postgres tests are similar if not better personally, with large and awkward select statements, and even without indexing it performs better.
MySQL is great for your web BBS or whatever. But anyone who uses it to store sensitive client information is simply a fool. (although I'll still let them pay me exorberantly inflated prices to let me code for them)
And if I haven't pissed the horribly uninformed mysql crowd off enough, ACCESS statements are more feature rich than MySQL - and it's only really a client.
Simply put, a lot of us here at work are getting a laugh out of this. MySQL is a toy, simply put.
-Erik-
(by all means, moderate me down, i'm still right)
- But mysql is faster! - who cares, mysql doesn't do transactions
- who cares, most web apps don't need transactions
- well i hope you don't expect to do anything important without transactions!
etc etc etc etc
At least MySql is GPL now so noone can go off on THAT. But seriously, I find this story interesting, but geezus, let's get another section for these types of story. like a Database section, or maybe Flamebait section.
sig:
sig:
See the "..for smart people" banners Wired runs here? Look elsewhere guys.
If you want details of high performance testing, then you need to visit www.tpc.org. Until I see Postgres up there then this testing is useless to the business world.
Why do you suppose the "2 leading commercial databases" were never named? It would be interesting to know what they were
--
-- I invented COBOL! What have you done lately?
They make the point that AS3AP test is read-only and should have been a piece-of-cake for MySQL, if you believed MySQL's slashdot-enhanced reputation as the ultimate in web automation.
Instead, it's looking as though MySQL is only fast when it doesn't matter: when there aren't very many people using it.
The Postgresql people insist that the results of these benchmarks were completely unexpected... they knew that they'd improved things a lot, but they'd never made an attempt to measure it before. It's understandable to suspect that the "independant agency" was watching which side it's bread was buttered on, but it's also obvious that Great Bridge really needed to know this information, *someone* had to do these tests, and who else was going to pay for them?
It's all very well and good to take benchmarks with the proverbial grain of salt, but you can't just throw out data because you don't like the result.
thank you for posting a clear, technical, and non-gee-golly response to a solid technical question. . it's really good to see developers saying, "well, this can deifnitely use some work" when that is in fact the case.
Only on a Negotiated CONTRACT - and even THEN - they can NOT prevent free speech. You can NOT agree to give up your right to free speech, any more than you can agree to be sold into slavery.
Duh!!
http://www.mrbrklyn.com/amsterdam.html http://www.brooklyn-living.com
This is really great news. I've used postgresql own and of for the past few years. I've found it really intuitive to use, and configure. It's also got it's own array of assorted public domain utilities, so that you can graphically manage your databases, or whatever.
I'm really glad to hear that it has performed so well in terms of performance, and the handling of large amounts of data. As that would have been my only concern about recommending it for a large project.
As somebody who has got to work with large databases on a daily basis, I look forward to implement the next one's using postgress.
It's not even just the fact that it's free, or the fact that it is faster. But just think of all the head aches I'll save. Configuration of Oracle is a big pain. Even doing a simple backup is a complex procedure. To maintain an Oracle DBMS you need proactically need a degree just on that.
Opposed to that Postgresql's setup is simple and straight forward. As well as backups which are easily done in a text format, instead of Oracle's binary.
With this announcement I feel confident that Oracle and it's sleazy Larry Ellison have now reached their peak.
And we'll all be able to look forward to a new golden age of humanity where databases are fast, efficient, and not needlessly complex to use.
BTW
If you even THINK that UCITA will prevent publications from printing Benchmarks and software reviews, you really need to get your head on straight.
http://www.mrbrklyn.com/amsterdam.html http://www.brooklyn-living.com
This 'article' is nothing more than a press release from Great Bridge.
There may be some additional information learned by reading the results of the benchmark from
http://www.tpc.org/New_Result/TPCC_ Results.html
Although I am having a hard time finding any reference to Postgres on that page. Can anyone find any better references?
-k
Whoa! Lookit all the pro-MySQL comments!
Just because the results were executed in a rather clandestine fashion, it doesnt mean its wrong. Sure, I'd love to see some truely impartial tests done over as many DBMSs as possible, each test done both through the ODBC drivers and native drivers, so we can all see where the weaknesses and strengths lie. However...
The PostgreSQL folk have done a lot of work for version 7, both in terms of speed improvements and a really kick-ass transaction system (yes, you should be using transactions). Perhaps some of you folk should consider giving it another look, rather than saying MySQL is better just because it has been in the past.
That is very true (but I picked BSDi 0.3.3 beta over Linux 0.9 around the same time and stuck to it, so at least there's consistency in my position :-)
I'm sorry if you got the impression I though PostgreSql still sucked as badly as when I stopped looking at it. I know a few folks that run PostgreSql and are very satisfied with it.
To answer the question, what keeps me from looking at PostgreSql are two things:
I fully realize that every persons situation is different (yes, I still use Berkeley db 1.86 if I feel it's the right screwdriver to hammer a particular nail).
I'm curious to hear if PostgreSql has the ease of rebuild these days that I so admire in MySQL. Frankly, the speed advantage MySQL used to have never was critical in my apps (at least not unless compared with Oracle).
Bert Driehuis -- All I asked was a friggin' rotatin' chair. Throw me a bone here, people.
There is a lot to think off for any company when looking at a database, and one of the first words you will hear is support!! Well following the great OSS model like many other companies in many other areas of technology we have a company that can support PostgreSQL!!Landmark has now publicly announced the formation of their company, called Great Bridge, to provide commercial support for PostgreSQL. -- PostgreSQL webpage
The test shows that postgresql is good, and having used it for some coursework some time ago I can say that I am kinda suprized. MySql was a hell of a lot faster at the time and was so cause it provided next to nothing from the SQL-92 standard (BUT then again its not trying to say it does... its freaking awesome for webstuff... the winner handsdown).
Have always seem postgresql as the DB you use when you can't afford a copy of the two unamed db's!! ORACLE and INFORMIX....unless they were talk about FoxPro and DB2 ? I guess it is the choice when you want to learn about databases and dont want to be limited to what MySql can offer. I used it and it was just what i needed. Since then they have gone ahead and added better mechanisms for foriegn keys, better support for triggers and procedures and a slew of other stuff that Oracle and Informix have had for years.
The GUI interfaces for it have also become much nicer to use and all in all its pretty good for most tasks..
BUT, i am not sure how this article can claim it beat the TWO BIG DB's cause they still have a hell of a lot more functionality and that will of course cause a little bit of a speed hit! I mean I am all for this software, i used it as an alternative to MicroSoft Access (hehe could this be one of the two big ones ? or SQLServer ? ?) and it rocked! SO its good to see it was a clear winner... in speed of transactions processed! The PostgreSQL team is doing some good work and thats always good to see, but maybe we should take some of these benchmarks with a pinch of salt!! Just a warning, having used it, Informix and Oracle, i would be careful to just say speed is all you want. I mean there are a lot of things to consider, like how locking works with each, how rollbacks are handled and what kind of isolation levels are available ? This is something you dont have to worry about till you are doing a full product! well just my views.... a warning i guess!
Non-Deterministic Finite Automata
The first "top ten results by price/performance" stuck strictly with MS SQL Server, which we all know doesn't run on Linux. Besides, how do you rate a product in price when it has no price? Postgres can be downloaded and used freely.
um... Not unless your hardware and personnel are free. From http://www.tpc.org/faq_TPCC.html
Q: What do the TPC's price/performance numbers mean?
A: TPC's price/performance numbers (e.g. $550 per tpmC) may not be what you think they are. When first analyzing the TPC price/performance numbers, most people mistakenly believe they are looking at the cost of the computer or host machine. That is just one component, and not always the major component of the TPC's pricing methodology. In general, TPC benchmarks are system-wide benchmarks, encompassing almost all cost dimensions of an entire system environment the user might purchase, including terminals, communications equipment, software (transaction monitors and database software), computer system or host, backup storage, and three years maintenance cost. Therefore, if the total system cost is $859,100 and the throughput is 1562 tpmC, the price/performance is derived by taking the price of the entire system ($859,100) divided by the performance (1562 tpmC), which equals $550 per tpmC.
What more can be said? I think you've done the Open Source community the biggest favour anyone could: you've proven our tools ARE INDEED enterprise-ready.
Betcha Ellison and Gates (oh, come on, couldn't you have figured that one out yet?) are gonna sweat over this one for a long time... Well, bwaaahaaaaaa loosers!
I'm not very clear on the numbers, though. They say PostgreSQL achieved 1127.8 transactions per second, or 6766.8 transactions per minute. Recent TPC-C benchmark shows IBM with DB/2 an absolute winner with 440879 transactions per minute, Compaq came next with MS SQL 2000 with "measly" 262243 transactions per minute.
I'm quite sure I'm getting something wrong here, just don't know what exactly. Any knowledgeable DBAs here to enlighten us, common mortals?
-----
I took a look at http://www.tpc.org/new_result/c-db-results.idc They don't even test MySQL, PostgreSQL, etc... not exactly a very conclusive study.
Of the little I've used MySQL it seems to be fine as long as things stay pretty flat. I wonder why it was included in a benckmark like this though (other than to make PostgreSQL look good). Much like Access or FoxPro, it isn't optimized to do what they asked it to do, shouldn't be a big deal when it doesn't keep up.
Icebox
Someone who was very familiar with Postgres and not very familiar with Oracle, Informix or whatever else might easily obtain that sort of result. A misconfigured database can creep along at a snail's pace.
--
As a matter of fact, I am a lawyer. But I play an actor on TV.
What could happen if one did publish benchmarking results -- they yank the license? I guess that maybe a big deal since Oracle licenses are so expensive but I'm surprised noone has tried publishing benchmarks.
Ha! I kill me!
Huh? We're not affiliated in any way with Mindcraft.
Regards,
Ned Lilly
VP, Hacker Relations
Great Bridge
OTOH, Oracle can be slow if you don't configure it right. Which is one good reason to prohibit benchmarks :-)
Another (bad) aspect to Access is it's 1GB filesize limit. Thats not very useful for big projects. I'm quite sure PostgreSQL can grow to be much much larger than that.
Sig (appended to the end of comments you post, 120 chars)
This is very very true.
The native interface to Oracle is *much* quicker. Having used Native drivers to Oracle on ColdFusion machines I can definately say it works quicker than ODBC.
At my work we write scalable applications and in the near future I would like to sit down with a copy of Oracle, tune the hell out of it and spend a week optimizing the atabases, PostgreSQL and Oracle, and MsSQL 7.0 because Benchmarking just one system like a Database is not really telling, Stick it on one of my Cluster of ColdFusion machines and lets see if the DB can keep up with the rest of my Application.. THEN I will be impressed not until then.
I read the TPC Benchmarks I even know how they work, but they still seem useless to me because We cant afford some 70K / Year Oracle DB Admin to come and constnatly Tune our Databases for us, if I cant learn enough to get it resonably well configured in a certain time frame I will go with something that *CAN* handle the load, and if only Oracle can ( which I dont believe ) Then we would be forced to find a solution that performs, But I have a copy of DB2 as well and you guys would be very suprised at the little places support for Databases come from.
I have read and installed ColdFusion on Linux and in the documentation and release notes Allaire has instructions on how to set up Both MySQL *AND* PostgreSQL via ODBC, so its not like a native driver, but it is support and I find it rather impressive that a big company that is more or less leading the way for commercial web application development platforms reaches out to support both MySQL and PostgreSQL.
My opinon is if you cant do much with the database what good is it?
Jeremy
If you think education is expensive, try ignornace
We also ran the same tests on Interbase 5.6 for Linux. The 6.0 release showed some very modest improvement over the 5.6 results (even with the ODBC hack), so that's what we reported in the press release. When and if there is an open source ODBC driver for Interbase 6 (or even a finished closed one), we'll re-run the tests.
Regards,
Ned Lilly
VP Hacker Relations
Great Bridge
Has anyone noticed that some open-source marketeers are beginning to really resemble their closed-source peers? This press release just reeks of FUD; how many times is the word "proprietary" snidely used? If it were "innovat*" this could have been written by a Micros~1 lackey - there aren't a lot of other differences.
I'm as big an open-source and free software proponent as anyone I know, just as opposed to proprietary code, and I've even specifically used and recommended Postgres in the past. And still I think that adopting these methods is not the way, because they represent a lot of what was wrong with the proprietary model. The MBAs who put this little exercise together have no idea what they're selling or why - they just want to sell it now, through any means and at any cost to the truth. The real virtues of Postgres, open-source, or anything else get lost in the hype.
I personally fell asleep halfway through reading that mess. I've only just now woken up, and dammit, I'm cranky.
-- Life is short. Forgive quickly. Kiss slowly. ~ Robert Doisneau
We paid for the test. Xperts worked as a contractor to Great Bridge. Look, this isn't vendor FUD. This is an open source company sharing its research with the open source community. Feel free to ignore the results if you find them somehow tainted. Our hope is that by releasing our findings, others will try the same tests. Regards, Ned Lilly VP Hacker Relations Great Bridge
knacker (v) (1) to bollox something up; (2) to tea leaf something; (3) (n) an Irish tinker
Which version were you using?
Which probably explains why one of the biggest goals in the slashcode is database independence.
With 200,000+ users, slashdot is not exactly a small application anymore.
No, Thursday's out. How about never - is never good for you?
This made me laugh so hard.
:M
The fact that MySQL was involved implies one thing
1) There were no stored procedures or triggers used in the tests [1]
which leads to:
2) This is not a real-world test because of 1). The "Real World" uses stored procedures and transactions. The fact that this test didn't use either puts it in the same league as filesystem performance tests.....
[1] If the test was fair this must be a fact, 'cos MySQL doesn't support either.
The government isn't taking away any of your rights. They can't. People and companies can, though. That's the difference. The us gov't is not saying "your'e not allowed to publish benchmark data". Individual (companies) are. That's the difference.
And to be honest, it's fair. Those things are SO hard to properly set up and optimize, if anyone who bought the software could publish benchmarks, there'd be a wave of confusion as people with the exact same systems came up with different scores. Not just that, but due to the complication factor, it'd also be incredibly easy to "accidentally" misconfigure a given system in order to hobble it and call it's competitor the victor in a given benchmark.
Hence the reason they don't want results published. When they are being done to be published, generally, they'll ship a system and/or a few techs to configure the benchmarkers' systems in order to make sure that they get shown in the best possible light.
I drifted a little.. all i meant to say was that the government is the only person according to the constitution that can't take away your free speech. And they're not supressing speech in this case. Stop pointing fingers at them...
Performance/Price
When Price is 0 then the ratio is always infinitely better than MS SQL Server??
Nuh? Big boy like you can do math?
http://www.mrbrklyn.com/amsterdam.html http://www.brooklyn-living.com
That's incorrect. From the PostgreSQL-hackers mailing list (msg):
Nope. If it were due to the ODBC driver, then MySQL and PostgreSQL would not have had comparable performance in the 1-2 user case. The ODBC driver is a per-client interface so would have no role as the number of users goes up.
I couldn't find the information either, however I suspect that at least one of them was MS-SQL Server.
Karma: Non-existant. Due mostly to the fact that you smell funny and nobody likes you.
Yes, thats the thing about open source, you can make it do what the hell you want it to do.
looks like another Micro$lave.
What does that have to do anything?
Do you mean to imply that /. is running solaris? That's untrue. See this.
Ummm, maybe you could read my post above that the price includes hardware, thus free(beer) databases should have an edge?
---
DO NOT DISTURB THE SE
Now is one of those times when I wish we could all moderate the original story down. Then we could maybe make it disappear quickly and save us from the inevitable troll-baiting that's bound to happen in the Linux newsgroups - i.e. "[] and you linonuts pissed and moaned about Mindcraft and now you pull the same crap." Oh well....
Yeah, of course, the DBMS stayed up. Great, just nobody could use it.
f.
The two industry leaders cannot be mentioned by name because their restrictive licensing agreements prohibit anyone who buys their closed source products from publishing their company names in benchmark testing results without the companies' prior approval.
Apparently, if you succumb to the MS, or Oracle virus (or whomever it was they tested), you're not allowed to talk about your experiences comparing them to other products. I wonder exactly how legal that clause is in the license....
D
Now that's humour.
wrighty.
(As this is already offtopic I'd like to add that the articles today have been particularly fine. Good work fellas.)
Most likely MySQL was included because - as anyone who looks at the comments on any thread about any DB on /. - it has hordes of ravening fan, many of whome have swallowed a lot of nonsense about databases in general and MySQL in particular. Or, to put it more politely, MySQL has a lot of claims made about it, and it's interesting to see how it stacks up in testing, as well as more thorough comparisons.
--
My name is Sue,
How do you do?
Now you gonna die!
There is no spoon.
Moof!
An overall benchmark for DBMS (and especially the ANSI-SQL one) tells you just the information you don't need: performance on a random task. Normally you have to find a DBMS for your specific application, so this information is completely useless to you! After you have specified the requirements for your database, you can single out the one that fits best. And believe me: there are applications where you don't choose Postgres ...
Xperts is not a "benchmarketing" company. There would have been absolutely no reason for them to favor one dbase over another. They use many different databases/systems/languages for their applications(both open and closed source), and really can be considered a neutral party in these matters. A single post with no qualifications may not be very convincing, but try to trust me on this one.
... can be found here: http://www.greatbridge.com/news/p_081420001.html
--
--
If code was hard to write, it should be hard to read
I don't know how it compares with the rest of the other databases on Linux though. --e!- ---
-------------------------------------------
-----------------------------------------------
Unix _is_ user friendly, it's just particular about who its friends
MySQL does have a completely different way to make releases than PostgreSQL, so it no completely fair to just compare a PostgreSQL 7.0 with MySQL 3.22, even if MySQL 3.23 is marked as beta. The reason is that if PostgreSQL would follow the same release mechanism as MySQL, where there is one month without nothing but bug fixes between -alpha, -beta and -gamma releases, PostgreSQL 7.0 would still be marked -alpha.
Agree. And it was even faster before Microsoft bought it. They've done two things with it since:
The original FoxPro team worked wonders with that code. The reason FP can't do the SQL benchmarks is that the SQL interpreter in it has basically been static since they were engulfed by The Borg.
Got time? Spend some of it coding or testing
Surprise, fear, and ruthless perseverance... Our three main weapons are fear, surprise and ruthless efficiency .... and an almost fanatical devotion to the Pope. Our four...
Sorry 'bout that. No one expects the Spanish inquisition.
Bert Driehuis -- All I asked was a friggin' rotatin' chair. Throw me a bone here, people.
We like Postgres. But it couldn't get us home when we needed it to.
bart
I'm really sick and tired of the flaming directed at MySQL for no other reason than that it is successful.
The proponents of Postgresql have completely lost any credibility with me. Their unending whining about MySQL's limitations are topped only by how they duck the touchy issue of Postgres performance.
Today's article is supposed to address that. So Great Bridge plants a deliberately biased "benchmark" (oops, mortal sin, I used the B word) which, unsurprisingly, puts the competitors at a disadvantage by artificially forcing down their performance via ODBC drivers so they will match Postgres' slug-slow performance.
And what we have is a PRESS RELEASE about a BENCHMARK. Can it get any lower than that? Shame on Postgres and shame on Great Bridge. When MySQL does comparisons, they publish entire tables showing the features for each database in the test, they show the run results, and they make the source code available.
They don't whine about their competition not being "SQL compatible" (a debatable term in any event). They don't lie about how the competition "isn't relational." They don't sneer about how the competition is "just a front end to the file system." By the way -- many are now arguing that it is better to run Oracle, DB2 and other systems that have a native disk mode through the file system anyway -- see the very informative Slashdot discussion on this a few weeks ago.
No, Monty and the MySQL folks just keep doing their thing, improving an already good product and sticking to a clearly charted development course. They have responded to the market and GPLed their code and are adding other features. When they get to subselects I'll probably be using it.
That's right, I'm not even a MySQL user these days. I still use R:Base, everyone's favorite database to piss on in the early 1990s, just like you losers are pissing on MySQL now. Only problem with R:Base (aside from it not being open source, I'm still trying to get their attention on that) is that it has virtually no marketing. But it is rock-solid, as ANSI92 SQL compatible as anyone, supports all the features of Postgres and then some, and runs like a bat out of hell.
If the Postgres crowd will stop flaming, whining and lying, I will start taking Postgres seriously again.
-------
Bill Gates Is My Evil Twin.
Um. Sorry.
WWJD? JWRTFM!!!
they didn't even do one of the tests with mysql ... benchmarks always seem to be biased in favour of one competitor.
And in fact I am currently migrating one of my web sites (in PHP3) from PostgreSQL to MySQL because MySQL is significantly faster and stable compared to PostgreSQL.
I used to be a fan of PostgreSQL, and always thought it was a more superior product than MySQL because of more completed SQL support. The web site I have been building has been running on PostgreSQL for more than a year now (from 6.4 to now at 6.5.3). As the amount of data grows larger, there are constantly problems with the backend DBM. The postmaster sigfaults every now and then, and for a couple of times I have to restore to backup because it refuses to function anymore. Most annoyingly, PostgreSQL still has the row size limit of 8k, which is just ridiculous for developing any application.
I have been running TWIG on the same host, and initially I used PostgreSQL as its backend. That was a big mistake because it takes around 5 seconds just to perform all the queries in a page! I then tried MySQL, and that makes a huge difference.
I do understand that MySQL does not support many nice features that I like from PostgreSQL, especially the subqueries and foreign keys. However, it is still a better DB for web based application.
If you just want to use it (and not admin it), O'Reilly's Programming the Perl DBI has some info on accessing a PostgreSQL DB (hint: it's not that different from any other DB when seen through DBI). Oh yeah, MySQL & mSQL, also from O'Reilly has a little bit about it (but not very much at all). I guess readmes, man pages and HOW-TOs are your friends for the next couple months.
If you're really curious, throw it on a test machine and (if possible) "port" some apps to use Postgres instead of MySQL or whatever. You probably won't reach any real conclusion (or do nearly enough work to justify moving to another DB for a production environment), but the effort will very likely get you very familiar with how it works, how to set it up, how to admin it, its performance, its quirks, etc. That's both a good and a bad thing, BTW... :-)
-B
Ash and Hickory, straight-grained and true, make excellent bludgeons, dandy for the cudgeling of vegetarians.
They ran the tests through ODBC, meaning that the performance of the driver (or lack thereof) becomes a huge bottleneck. All this benchmark tells us is that Postgres has a well-optimized ODBC driver. It says very little about the underlying performance of the RDBMSs in question.
Well, considering that they couldn't mention Oracle by name even if it was involved in performance testing (because of its draconian no-reviews license agreement) I don't see how you can rule it out.
Do any other RDBMS' license agreements have such clauses?
Comparing a bleeding edge product (postgres 7.0) vs. an older version of mysql (3.22) and saying it doesn't support transactions? Experimental Transaction support has been added see here for details. Notice how they didn't interview anyone besides the guy from Great Bridge. Why can't we all just get along?
I did my own benchmarking a while back using PostgreSQL 6.5 and Access 2000. The table structures and data were identical, each table had identical indexes on it and everything.
... its got lots of features and is truely Open Source. But you shouldn't be using it if speed is your utmost concern.
PostgreSQL 6.5 was about 10-15% slower than Access 2000. Not to mention it was even on a faster box with a lot more memory (K6-266 w/256MB Linux vs. P233 w/64MB Windows 2000).
Now, I love PostgreSQL
Of course, maybe version 7.0 _is_ that much better...who knows.
Sig (appended to the end of comments you post, 120 chars)
Yeah, it has the 8k tuple limit and you can recompile to have it use 16k or 32k. There are other ways to work around the tuple limit, like breaking anything more than 8k into chunks.
Despite the 8k limit, I managed to make a message board with postgres and php that has no message size limit. The message board of mine could really use some people to hammer on it! Its new so I hope to find bugs and fix them.
-- CompTechNews Message Board: http://comptechnews.com/ --
The article/press release/marketing FUD does not lend itself well to peer review, which is as important to technical journalism as it is in the scientific circles.
Here are the questions that came to my mind:
Of course, they may not want to reveal these informations for fear of peer review. *sigh*.
--
Postgres has a serious limitation that is not being taken into account.
The limitation is that you can only store a maximum of 8KB in a row! This limitation is deeply rooted into the postgresql source tree.
Version 7.1 promises to fix this, however, this version is long overdue! I've head that they are re-implementing the way data types are stored completely! I hope the new implementation is not slower.. Also, and if they are changing the way data is stored and retrieved, then it would make benchmarks like the ones outlined in the story posted useless!
Well, English is not my native language, especially not Southern American English, so if I misread anything in the message I replied to, please enlighten me. There was some language in that message that was, errr, hard to parse, so I cannot exclude the possibility of a hint of misunderstanding a priori.
Bert Driehuis -- All I asked was a friggin' rotatin' chair. Throw me a bone here, people.
What can I say - I've used MySQL for so many things in the past, and only once had a problem (a 3.21 release) with stability.
One server running 3.23.5 has now been up for 59 days, and whilst it has only handled a rather meagre 470,000 queries in that time, I have still yet to have any issues with it, from data contingency to stability.
I'm not in a position to comment about Postgres, having not really looked at in it in detail - but from where I stand MySQL has always behaved perfectly. I don't know where this benchmark got the info with regards to slowing queries down with a large number of users - a web-based chat system I wrote a while back for a childrens TV company quite happily handled 130+ concurrent connections to the database with about 160 page requests per second.
For what is essentially free software, that's easily good enough for me.
Ok, I know very little about the competing databases. I'm just starting to use MySQL. When I was deciding which db I wanted to spend my time learning I went and asked an experienced friend what they use, and the slashdot FAQ said... go MySQL! So I would like to hear what CmdrTaco and the his motley crew have to say about MySQL's performance under pressure. I mean Damn! they get slashdott'd all the time :^)
Please be patient, I'm a work in progress! --Alan Jackson
Quad Xeon machines are doing around 25,000 transactions per minute on the real tpc tests (here) so for a 2-cpu machine to do 300 per minute is not terribly impressive. I think it's probably the trivial hardware that was holding those test back, though, rather than postgres per se.
With only two disks the tests were almost certainly disk-bound, which would explain the striking similarity in the TPC-C results for all three vendors. I doubt any of the database systems really got a chance to hit their stride.
So the bottom line is that we still don't know what postgres can do given reasonable HW, by which I mean at least 4 CPU's, 2GB memory, and 16 disks.
Hello, Great Bridge?
Hmm... I may have to take another look at Postgres. I've been using MySql in the name of speed and because, for what I'm doing these days, I don't really need Postgres' more advanced features. I had heard that Postgres was slow as hell and a serious resource hog, but I'll have to do some testing of my own. Is there a Postgres admin here who'd like to tell us what kind of resources Postgres demands? After all, the horror stories I've heard might have been coming from people trying to run it on a 386 with 6 megs of RAM.
--
Proud member of the Weirdo-American community.
It's one thing to claim to be enterprise ready as a databse product. It is quite another to be one.
Before I get started, I should hasten to mention that I work on developing DB2 UDB and therefore anything I write is biased and should be viewed as such :-)
Enterprise-ready is one of those phrases which gets bounced around a lot. But what does it mean for relational databases? In my opinion, it at least includes the following:
At this point, I don't know what the score is for PostgreSQL on the above. Any expert care to comment?
Cheers,
Toby Haynes
Anything I post is strictly my own thoughts and doesn't necessarily have anything to do with the opinions of IBM.
Sadly, the docs are not free. Now, I'm not really whining about shelling out the $40 or whatever for the 'interbase handbook' or whatnot; my complaints are these:
Timeliness. Programs CHANGE! Open projects change QUICKLY! You must have open docs, or at least docs not committed to paper, to revise them.
Sheer quantity of docs! Ever used Oracle? Oracle8: The Complete Handbook is not all you need! You inevitably end up buying the whole goddamn bookshelf! RBDMS's are complex beasts! With that in mind, closed docs means that such a project can't be called 'free' after all.
I guess it's 'Raymond Open' not 'Stallman Free'.
The second one, of course. The court and the U.S. Marshalls = gov't power. Thus: Gov't coercion is the problem. No gov't coercion, no problem.
If I tell someone to rob you, both me and the person doing the robbing are guilty of a crime (legally and morally). If a corporation gets a court to issue a judgement against you which is unfair, both the company and the gov't are morally guilty of theft.
You may be able to escape liability on this though: Contract clauses which are "contrary to public policy" can be found invalid. Preventing the exercise of free speech could be considered "contrary to public policy" enough to prevent a court from levying damages against (i.e. stealing from) you. Any lawyers (of which I am not) care to confirm or deny that?
Just because it CAN be done, doesn't mean it should!
I don't want to start a big argument about this, because it's already off-topic enough, but...how does a company take away your rights? Through a legally binding (i.e. enforced by laws and the government) contract. Sure, you can agree to the contract and thus give up your rights. But is it legal to create a contract, enforcable by government power, that requires you to give up your Constitutionally guaranteed rights? Furthermore, for a company to suppress your speech, they have to have the backing of the government in most cases. Witness libel, slander, etc.
Besides, it's better to point too many fingers at the government than not enough IMO.
--jbBullshit! If that were "fair" then Consumer Reports would have nothing to do. Buying competing products, and pitting them against each other under identical circumstances is a time-honored tradition. You're even allowed to make TV commercials about it (to wit, the "Pepsi Challenge" ads are nice examples, as are any which have statements like "FooWidget outperforms BarWidget 2-to-1 on {whatever}", which happen all the time. It'd be VERY hard to enforce that clause of the license, methinks. D
I have been weighing the two DBMS's for a few weeks for a project I am developing. I have learned that Postgres has some other features that might make you want to consider it. One very nice one is the ability to embed a Perl interpreter in the application so that you can write functions in Perl rather than Postgres's own procedural language. (There has been some talk of doing the same with MySQL and PHP/Zend, but AFAIK nothing so far has come of it). The query language that Postgres supports is, as is well known, much richer than that of MySQL, allowing for views and subselects; two very addictive features.
Postgres, however has an 8k (32k if you recompile) column data limit, meaning that storing large amounts of data in a row can be difficult. Although for most people this should not matter, it's a make or break feature for my work and so I had to go with MySQL. This will supposedly be corrected by version 7.02 of Postgres.
Both are great projects and now that MySQL is under the GPL maybe some cross-pollination will occur and enrich each one.
My site runs slashcode, and yes, mysql does tend to get somewhat slow. It also leaves about 1000 extra threads for some reason (rh6.2 box, p3-500). Since I run slash though, im stuck.
or am i?
--
It's a pitty that users new to database concepts and database-backed websites are misled to believe that MySQL is a robust database product. It's NOT ! It wasn't designed nor written to be.
Without
- Transactions (recently partially added in MySQL)
- Procedural languages (PG supports at least 3: PL/PGSQL, PL/Tcl, PL/Perl)
- Subqueries (e.g: SELECT foo FROM bar WHERE bar IN (SELECT bar FROM foo2) )
- Multi-Variant Concurrency Control (absolutely essential for websites - Readers don't wait for writers and writers don't wait for readers)
- Referential Integrity
- Much of the SQL92 standard
MySQL should only be used in applications that did not require any security or where the integrity of the data is not important (why would you want a database of inconsistent data?)The features that I mentioned above are crucial to any database product. Without them you will be spending much more time and effort trying to implement in the application level, things that should be handled by the backend.
Look at the Sourceforge bug reports page and you'll notice that most of the problems they face are due to their poor choice: MySQL. I am not saying that MySQL does nat have its space... it does, but it's not in the enterprise and it's not in important data.
Quit whining about the ODBC deal. Stick to the facts: without these features MySQL is very limited.
People who scream "FUD" without reading the article really annoy me. Mind the allcaps, please; But I feel it is necessary, so that this actually gets through some of the skulls out there.
"THE TWO INDUSTRY LEADERS CANNOT BE MENTIONED BY NAME BECAUSE THEIR RESTRICTIVE LICENSING AGREEMENTS PROHIBIT ANYONE WHO BUYS THEIR CLOSED SOURCE PRODUCTS FROM PUBLISHING THEIR COMPANY NAMES IN BENCHMARK TESTING RESULTS WITHOUT THE COMPANIES' PRIOR APPROVAL."
I benchmarked Oracle and Microsoft-SQL against one another for box weight (that is, how heavy the software, packaging, and associated manuals are) and found that the differences are scale-dependent. Overall I found that Oracle was heavier.
Go ahead, guys, sue me. Good luck; you'll need it.
"'It's just astounding, and unexpected,' said Xperts' Brosnahan of Postgres' performance.
'I ran the test twice to make sure it was running right. Postgres is just a really powerful database.'"
Uhm, shouldn't it have been run more than twice? I mean I thought part of benchmarking was to run a single test a LOT around 10,000 trials or so, with each competitor.
--Dave
There's no such thing as free speech. Haven't you noticed that the First Amendment to the Constitution reads:
Libel, slander, copyright trademark or patent violations, licensing agreements, saying posting or printing anything that someone with more money and more lawyers than you doesn't like...Free speech is a dream. --jbEveryone seems to be missing the fact that it is a press release by a company providing support/etc for Postgres. Now, I don't know about you, but this raises the "benchmarketing" alarms for me.
Not to say that this isn't true. However, as I browsed the release, I noticed things like "1-100 client connections", which tell me that there is a lot of maneuvering room to pick the best values.
Jason PollockGreat Bridge did not do the testing, therefore, they can be considered independent.
According to those benchmarks, postgres is very impressive. I'd like to learn more about it.
I liked the New Riders book on MySQL. Is there a similar book from any publisher for postgres?
Or maybe some good web sites?
Torrey Hoffman (Azog)
Torrey Hoffman (Azog)
"HTML needs a rant tag" - Alan Cox
Bartwol had it right -- it was a combination of memory leaks and having to take the db offline to Vacuum it.
I have yet to see a database that doesn't require a large amount of "rigamarole": some of it nightly, some of it weekly, and maybe even some of it annually. When you're dealing with something as complex as a modern RDBMS, maintenance is a given. A good DBA will automate most of it, with notification of exceptions, of course
Well, if you were running Oracle, once the sucker is running it never comes down -- not for backup, schema changes, moving tables to different disks -- nothing. It takes some expertise, to be sure, but it all makes sense.
You're right that there's a lot of work in a production database -- capacity planning, performance analysis and tuning. A lot of it's kind of fun. On the other hand, things that I consider rigamarole are when the admin has to do stuff that should be the system's jobs: freeing memory pointers and reusing disk space.
It sounds like the memory leaks have been fixed, and what remains is the vacuum process. For applications that don't need to run 7x24, this is not so bad.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
also, i'm all for postgres, but doesn't it seem funny that their business is based on postgres solutions, and now they come out with this "independant benchmark" claiming that postgres is the best?
This could be another attempt at "benchmarketing" ;)
-Doviende
"The value of a man resides in what he gives,
and not in what he is capable of receiving."
"The value of a man resides in what he gives,
and not in what he is capable of receiving."
--Albert Einstein
This reads more like an ad than a benchmark. Then again, it wouldn't be tough to beat MySQL. Maybe the two 'leading closed source' databases were MS Access and FoxPro.....
Icebox
Or... you could use Postgres.
Right?
Upon seeing the box was too small, Schrodinger's Elephant breathed a sigh of relief.
I'm not sure if Microsoft bans such reporting. However, that wouldn't nescesarily stop a US based magazine from reporting it. After all, that has got to be covered by the constitution. I realize that the constitution rarely applies to business affairs, but I have a hard time believing that a court wouldn't see it as covered.
-- Superlame http://catpro.dragonfire.net/joshua/
Just goes to show you that both sides spew the same amount of FUD.
Only the State obtains its revenue by coercion. - Murray Rothbard
Agree.
"I Want to Believe (TM)", but does the name
"Mindcraft" mean anything to you?
I don't know about the others but InterBase is definitely ready for production use! I've been using IB for the last five years in a car rental company in UK. We have 120 rental locations with a SCO server at each running InterBase. Each site has between 2 and 30 users. We couldn't run our business without InterBase! Reasons we chose it: 1. ANSI 92 Compliant and supports transactions and triggers (and event alerters which are good) 2. Native drivers and ODBC 3. JDBC, direct API and E/SQL (precomilers supplied) 4. It's pretty fast (if used properly) 5. Optimistic locking/versioning means its very forgiving for developers and also good for performance 6. Virtually ZERO maintenance - just do a backup and restore once in a while - our DBAs don't know it's there (good job as they spend most time firefighting/nursing the central Sybase and Oracle!) 7. It's multi platform - we use it on SCO, Solaris (sparc) and Linux (dev only right now) 8. It scales well - we also have a >20GB data mart in Interbase 9. It's small (freakin small). We install it onto our SCO servers off TWO FLOPPYS! 10. arent nine reasons enough? Basically, it's rock solid, very flexible and hassle-free. Like I said, our business couldn't do without it (and we're the no1 car rental company in UK). Check it out at interbase2000.org.
G4 Hackintosh
This is worse than the mindcraft stuff. This is essentially a press release. they refer some databases as just "a leading propeitary database". What are they afriad to say Oracle or something? Lets see actual results, this press release doesn't move me at all.
--
Be insightful. If you can't be insightful, be informative.
If you can't be informative, use my name
Be insightful. If you can't be insightful, be informative.
If you can't be informative, use my name
To me there seems to be a perennial problem with mysql vs pgsql. If Great Bridge skewed the benchmarks in favour of pgsql, then fine, but a criticism of the mysql benchmarks is that they are skewed towards mysql. Funny that.
Is there, or how you do you go about writing, an *independent* set of benchmarks.
The pgsql, mysql ppl sure do come out of the woodwork when something like this happens, but I've yet to see a truly *independent* benchmark happen. Speaking of which - is there such a thing as an independent benchmark for anything? hmm Bogomips maybe?
Anyway, if there was a standard benchmark to run against both DB's from a vanilla install (and by that I mean running './configure' with no flags) then I think that would go a long way towards giving a more balanced view.
G
This article suprised me as well. I really thought MySQL stacked up better. At least it seems to be getting new features rapidly (such as replication). Missing subselects have annoyed me at times for sure! I for one though, am VERY excited about Interbase coming open source. Go Borland!
It's like buying a Ford, measuring its 0-60 times, and publishing them in a magazine. Hmm, I wonder if anybody does that.
Uwe Wolfgang Radu
Borland has just recently released its source code and so what we have now is an open-source, royalty-free, Borland-quality database to use and abuse.
Download links are:
Client and server source code
Server Linux binaries
From personal experience, Interbase is perfect for a tight budget situation where you need to server a medium-size userbase.
--
Kiro
Er... I really can't comment, other than to say that it's not IBM, Informix, or Sybase ...
Why can't you comment? Is it because of restrictive EULAs that won't let you disclose the results of benchmarks?
I have discovered a truly marvelous sig, unfortunately the sig limit is too small to contain i
...against DB2, Oracle, or SQL Server.
Probably the 2 "commercial databases" were 8 year old copies of Informix and Paradox or something similar...
DrLunch.com The site that tells you what's for lunch!
This isn't surprising - everything I have ever read/seen/experienced about MySQL says that it's a pretty fast and simple database for smaller applications, but isn't very scalable (both in terms of features and performance).
I am quite surprised that the Postgres results are almost 3 times that of proprietary databases. That seems kind of fishy - surely the big proprietary databases aren't THAT slow.
But either way, it confirms what lots of people already knew: MySQL is a nice, simple fast database for smaller applications. But for larger stuff, there are better RDBMS' out there.
>Issue two. They compared the bleeding edge
.17 or so) I had it crash once, lock up once, and had the client segfault on me twice.
>postgres (7.0) with the old-as-heck mysql (3.22)
Again, they benched the current STABLE releases of the product in question.
Postgresql 7.0 has been released for months now, and was in beta for months previous to that.
The latest development branch of MySQL, on the other hand, is still undergoing some heavy changes, such as support for additional back-ends for storage (sleepycat, myisam), replication, etc.
This shows in the stability of the product. Last time I tried to run it (around
The occasional (!) crash might be fine and dandy for a home datbase, or static data for a non-crucial website. But a company that has to depend on the database being available 365/24/7 simply CANNOT tolerate this. Especially considering the speed of isam checking extremely large tables.
A clock stopped -- not the mantel's Geneva's farthest skill Can't put the puppet bowing That just now dangled still. An awe came on the trinket! The figures hunched with pain, Then quivered out of decimals Into degreeless noon. It will not stir for doctors, This pendulum of snow; The shopman importunes it, While cool, concernless No Nods from the gilded pointers, Nods from seconds slim, Decades of arrogance between The dial life and him.
----------------------------
----------------------------
Anonymous Emily Dickinson LIVES!
Sigh. The label says "new and improved". Does that mean I've been eating "old and inferior" for all these years?
I used Postgres (back before the SQL was bolted on to it), and it worked but sorta sucked. Having a working database was all I asked for so I was happy.
I then tried one of the earlier releases of PostgreSql, and gave up on it. Then came this dark horse, MySQL. It worked well enough, was fast, and survived the "pull the power plug" test that Postgres failed. Back then, MySQL and PostgreSql both sucked from an SQL compliance viewpoint.
The reason (and I mean the single reason) I'm still running MySQL is its speed and operational ease. It never crashes. If someone messes something up (using explicit commands or code to do so :-), I rebuild the database in record time from the last valid dump, with the changes rolled forward from the change log (a manual process, but it beats *racles automatic roll forward hands down).
Bottom line, use the tool that's best for you. PostgreSql is far more correct than MySQL, and according to this independent press release by the company that intends to make money off it, supported by independent research paid for by same, it is even faster. More power to you, but unless PostgreSql gets the upper hand in the operational arena, you'll have to pry MySQL from my cold dead fingers.
Bert Driehuis -- All I asked was a friggin' rotatin' chair. Throw me a bone here, people.
Independent, or not, these `studies' have
nothing about the real life. How often do you
really _port_ your DB? In most cases you design
it from scratch, taking decisions best to your
knowledge. What would be done with a trigger in
PostgreSQL, might be better as a UDF in InterBase (a bad example).
Porting a DB 1:1 is impractical, each system has
its merits and caveats. But when you have ported
your DB considering all the tips and tricks, you
get something totally different. Still apples to
oranges.
There are some `objective' parameters, but they
don't require benchmarking. You can find about
page size from the docs. Anything that is not in
the code, is `subjective' and the field for
speculations is vast.
I like PostgreSQL, but this press release makes me
upset. Really good product talks for itself, it
needs no hype. If something like this happens, a
trouble is most likely to follow. Just wondering
when they will realize that keeping marketoids
away is a Good Idea? Quousque tandem?
KuroiNeko
They didn't do the test because MySQL doesn't support the SQL standard. Is there something wrong with a benchmark biased against non-conformant programs?
Most people who know much about MySQL know it doesn't have transactions and is best used as a read mostly write some database. It's fine for that. It is not fine for big systems.
--
Infuriate left and right
This article is all snake oil. I'm disappointed that I read the whole thing and didn't learn anything about comparative DBMS architecture and implementations. The simply *is* no *how or why* to the results. What a useless piece of sales drivel! I feel no better equipped to decide on a favorite database backend.
--- Nothing clever here: move along now...
Obviously you didn't read that paragraph and the next correctly.
What does MySQL's lack of minimal SQL tranasction got to do with bias?
Ahh, it hasn't taken long for the MySQL wwenies to crawl out of the woodwork.>[?
PostgreSQL 7.0 is the current production release of PostgreSQL. Try getting your facts right.
--
My name is Sue,
How do you do?
Now you gonna die!
Microsoft has commissioned many "independent" studies in which NT wins handily, and people very rightly don't consider those to be truly impartial. Why the difference here?
Yes, but since one of those "features" omitted by MySQL is a sensical locking mechanism, MySQL (which uses table-level locking, a la the flat file dbs of 1960s) tends to grind to a halt when you have multiple readers and writers.
The reason MySQL was slower was because they used the ODBC drivers. The MySQL ODBC drivers are known to be significately slower then the native drivers. Although I do agree it is probably the best way to test a large number of databases using ODBC drivers, I would like to see the results of all the databases tested using native drivers.
-Tripp
Was it not www.tpc.org that ran these tests? In that case, I don't put much stock in them. Take a look at the top ten results by price/performance or even the complete results by database vendor shows no mention of Postgres. I'll believe it when I see these results ratified by the TPC.
---
DO NOT DISTURB THE SE
Something I think many people, benchmarkers included, forget is that the drivers are important too. What I mean is, if I am going to use a piece of hardware/RDBMS/etc... then what drivers I can use with it are going to be an integral part of my overall experience with the product.
If one DBMS/3D Card has better drivers, even though it is "slower in theory", then that means that my overall experience will be a better one than the "theoretically faster, but with crappier drivers" product.
What does this mean? Trying to equalize products on drivers is often an exercise in finding which product has the most tuned driver. IN the case of Postgres, it appears their ODBC driver is tuned much better than the others. However, very few people I know use ODBC drivers for MySQL, and not many use them for Oracle either. They all use the native drivers. Thus, this benchmark doesn't mean anything to them, because a non-real-world situation was benchmarked.
I wish people would perform real-world benchmarks: i.e., run what people would actually run. That's one thing that I really like about the gamer-oriented hardware review sites. They post a bunch of meaningless BusinessMark2000 and AppMark2k scores, but they also go in and show you how fast the actual games will play on the hardware. That is CRUCIAL to my purchasing decisions. RDBMS vendors should benchmark one database with its best-performing driver vs. another database with its bets performing driver. Then we could really get an inkling of an idea as to how the thing will really perform in the field.
Testing with the same drivers only looks fair; in reality, as in this Postgres benchmark, it was likely the deciding factor to making Postgres "trounce" the competition.
A few comments... The most noticible and glaring issue is the "independant study" was (surprise, surprise) commissioned by Great Bridge, and great bridge's reason for existance is to sell support and services for postgres. Not the strongest indicator of impartiality. The entire press release is designed to sell postgres, not to provide a fair comparison.
Issue two. They compared the bleeding edge postgres (7.0) with the old-as-heck mysql (3.22) - they're now up to revision *22* of the development series for mysql - that's a pretty huge amount of changes. I would have been much more impressed with this if they had ran the comparison between 3.23.22 and 7.0. As with everything, folks, don't believe benchmarks, especially ones in press releases. Believe real-world tests. I've used both, and I'm using MySQL 3.23.22 for my site.
Where postgres excelled was handling >4k simultaneous hits. Where it failed was in that it corrupted data. mySQL never corrupted data, was only a tad slower (but could improve with better written ODBC or JDBC drivers), and benefitted from more RAM. To say postgres "won" is more than disingenuous. Isn't data integrity THE most important criterion?
The benchmarks are irrelevant unless conducted by the TPC. Last time I checked, MS held the top 45 spots in TPC-C price/performance and 4 of the top 5 spots in TPC-C performance.
"Saddam Hussein cavorts with terrorists."
Hey-
This article at apachetoday is just a copy of the press release from GreatBridge themselves (who are now working on postgres to expand and support). I looked around Xperts, who conducted the test, but couldn't find any information about the test, how they were conducted, etc. So, I wonder how fair things were. And who paid for the test.
Jason
is that PostGres has a kickass ODBC driver. I'd like to know how things would've worked out with the PHP, Perl and C interfaces.
And why didn't they just come out and name MSSQL and Oracle?
D
The first, last, and only tech news site on the net
http://www.tpc.org/new_result/ttperf.idc
http://www.postgre sql.org/mhonarc/pgsql-general/2000-06/msg00390.htm l
The "rigamarole" you speak of is the need to run the "VACUUM" command regularly. This command essentially does garbage collection, cleaning up space wasted by previous database updates. As I understand it, Postgres (as part of its transaction processing scheme) creates a new copy of a row every time that row gets updated. VACUUM reclaims the space occupied by old rows and updates indexes accordingly.
A major problem with VACUUM is that it requires exclusive access to the database and can take quite some time to execute. Effectively, a production database must be taken out of service for this routine maintenance, and if you don't execute VACUUM periodically, the remnants of old rows in an active database can eat up all your available disk space.
<bart
I have yet to see a database that doesn't require a large amount of "rigamarole": some of it nightly, some of it weekly, and maybe even some of it annually. When you're dealing with something as complex as a modern RDBMS, maintenance is a given. A good DBA will automate most of it, with notification of exceptions, of course.
I'd be surprised if your installation (you don't say what you chose) doesn't have a lot of maintenance associated with it, even if cron's doing all of it for you.
--
--
E_NOSIG
OK, this is great news, but performance isn't everything. I didn't go with Postgres for one of my projects last year for a couple of reasons.
The first and biggest reason is that some people who ran it said was that it wasn't as stable as the commercial databases. There was some rigamarole you had to do every week or so like rebooting the daemon or running some utility or another to keep the system from losing its mind. Sorry I can't recall better, but this was a year or so ago. In any case once I set a database server up and running I don't want to have to do anything with it unless something changes in its environment. Does anybody remember this?
Also, this is really more of a nit, but Postgres also lacked some important SQL constructs like outer joins and foriegn keys. OK, you can get around this, but the solutions are ugly and this also makes porting stuff a pain. Last time I checked, Foreign keys had been done, but while outer joins were on the list of features to add no work had started.
It would be nice if it supported all ANSI SQL intermediate language constructs. This would greatly facilitate porting to Postgres.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
A clock stopped -- not the mantel's
Geneva's farthest skill
Can't put the puppet bowing
That just now dangled still.
An awe came on the trinket!
The figures hunched with pain,
Then quivered out of decimals
Into degreeless noon.
It will not stir for doctors,
This pendulum of snow;
The shopman importunes it,
While cool, concernless No
Nods from the gilded pointers,
Nods from seconds slim,
Decades of arrogance between
The dial life and him.
----------------------------
----------------------------
Anonymous Emily Dickinson LIVES!
Hmmm. If you're so sure of your facts, then don't friggin' post as Anonymous Coward.
I'm an acknowledged BSD bigot. But that said, even though I dislike the GPL for the same reason I dislike forced outings of gays, I have yet to see anything come out of it that stifles innovation. There is a tremendous amount of positive innovation that came from GPL based projects. Why would one want to deny such an obvious fact?
List the contributions to this world made by project Bob. List same by the BSD users. List same by Linux. Rank them. Any questions?
Bert Driehuis -- All I asked was a friggin' rotatin' chair. Throw me a bone here, people.
where are the details? the two links in the post point to the same press release. can anyone point me to the actual detail of these benchmarks?
It wouldn't take much to get around this "thou shalt not publish benchmarks" restriction.
:).
Let's just agree right now that the next time we do a benchmark, we'll refer to Oracle as "commercial database 1", Informix as "Commercial database 2", MS-SQL as "Commercial database 3", or something like that.
It would be a little awkward to defend your choice of numbers if you only did the bm for 2 and 3, though
You could have something more subtle, though. Pick any databases you want as "commercial 1" and "commercial 2", and then state somewhere in your writeup that "people are very interested in knowing how PosgreSQL stands up agains the established giants like Oracle, Informix, MS-SQL, and DB2". Then we just know to read into that that commercial 1 was Oracle, and commercial 2 was Informix. Any list will do--just casually mention at the end of the writeup that
"the following databases have EULA's that prohibit the publishing of benchmarks:
Oracle
Informix
DB2
MS-SQL
"
There's no way that the companies could prove that you were doing that, and you would just never confirm or deny that there was any correlation.
Liberty uber alles.
I think it would be a great step forward in OSs to be able to take an RDBMS for granted.
- Tom
- Tom
"O, to grace how great a debtor daily I'm constrained to be."
Actually, it depends. If you're talking about raw speed then it's often hard to beat MySQL since it lacks most of the features that slow down databases. MySQL has a reputation for being fast, but it doesn't have a reputation for being a replacement for a true DBMS.
If it is available for instant access, please help me find what I'm doing wrong. I should be able to just start it up and set up a table. (I'm not root, BTW.) If the DBs aren't set up where any user can just go in, then this should be made easier.
- Tom
- Tom
"O, to grace how great a debtor daily I'm constrained to be."
Mysql are fixing everyone's concerns about transaction and becoming ansi complaint. If those problem are a concern for you then you can fix them yourself.
MySQL didn't take part in the TPC-C part of the test, because MySQL can't handle TPC-C. It doesn't implement enough of SQL2 for that. :)
In my professional opionion, MySQL needs to drop the My part and adhere better to the SQL part. At least that would make my life easier.
There was a comment on the article at the bottom that gives a link to the original story, which has some pictures, you can find it Here
"The first time I got drunk, I got married. The second time I bought a chimpanzee, after that I stayed sober" Arian Seid
There is way too little detail to make any sort of judgement based on these figures. I would need to see a lot more of the configuration information before I would even dream of considering these results. For example there is no kernel configuration options or NT settings. How much swap space was uaes what sort of filesystem was used, how were the partitions laid out, what were the configuration options used for the databases. There is only a couple of sample results, but the words out-performed is all over the place. If we are so ready to slam Microsoft results we should hold these to the same standard.
I am not surprised the commercial vendors would not allow their names to be mentioned (were they even asked, wasn't clear from report). Remember with any of these benchmark reports cavet emptor.