Slashdot Mirror
MySQL Developer Contests PostgreSQL Benchmarks
Lucas Marshall writes: "Developer Shed has an article written by the main MySQL developer Michael "Monty" Widenius in response to Great Bridge's press release reported here on Slashdot on Monday. He makes some pretty good points about fair benchmarking and software competition."
ACID stands for Atomicity, Consistancy, Isolation and Duribility. Atomicity means, basically, a transaction. A series of commands are issued, then either commited or rolled back. All happens, or none happens. The classic example is doing an update on Joes_Account_Balance to subtract 100 dollars, then failing to do the update on Suzys_Account_Balance to add 100 dollars. The transaction failed, so it should be rolled back so that the 100 doesn't get removed from Joe. Consistency means that the database must always be in a valid state, and that usually means rules are enforced. If there's a database rule that states that any subtraction of 100 dollars from an account needs to add a record to the audit table, so you subtract 100 dollars from Joes_Account, but the audit table cannot be updated due to disk full/network break/act of Gods/whatever, then the subtract needs to be rolled back, or not happen in the first place. Isolation means that users don't see other users. If two people are both doing multi-stage updates to Joes_Account, user one doesn't see the fact that user two just did an insert, and an update. Durability means that if bad things happen, the database survives. Logs, backups, that sort of thing.
Vintage computer games and RPG books available. Email me if you're interested.
Take a close look at what you are writing people. Especially those who have said "MySQL is useless in the real world." I can't imagine any sane definition of "real world" that doesn't include Slashdot.org, countless e-commerce sites and other non-Web applications.
Of course MySQL has limitations! Of course it does not meet the definition of an RDBMS! It meets the needs of a significant set of *real world* problems with great efficiency.
PostgreSQL meets a different, but overlapping, set of problems with ever increasing speed and efficiency.
They are both open source.
They are both improving at open source speed.
They both rock.
>As they haven't released any information about :(
>the test or even the results one can just assume
>that they have done a test that doesn't have
>anything to do with the real world
"In the AS3AP tests... Postgres peaked at 1127.8 transactions per second with five users, and still processed at a steady rate of 1070.4 with 100 users... MySQL reached a peak of 803.48 with two users, but its performance fell precipitously under the stress of additional users to a rate of 117.87 transactions per second with 100 users."
The test is standard, so the definition of what it entails is easy to come by and they provided what the results were. So your above complaint really has no substance to it.
However, I do agree there should have been more information - namely the software configuration.
>PostgreSQL has of course also a lot of weak
>areas, like slow connection, slow insert, slow
>create/drop tables, running long multiple
>transactions where you get a conflict
>at the end (in this page/row locking is better)
Yes, Postgres is slower to connect, though not grotesqely so.
Yes, Postgres is slower to do inserts, but updates and inserts don't block. So though MySQL may be impressive if you bench insert speed without considering its impact on overall performance, in real world applications you'll see very different behaviour.
I'm not sure exactly what you're refering to when you mention "conflicts" at the end of transaction blocks. This is one area where I find MySQL falls short, since a) it has no transactions and b) it doesn't have the framework for transactions (e.g. multi-version concurency control, etc).
>We here at MySQL has always tried to design very
>fair test that no one can misinterpret or lie
>about.
Unfortunately the benchmarks MySQL uses are all single client load, which is completely meaningless in a production environment and covers up the weakest parts of MySQL (e.g. all selects stalling while an insert or update is happening).
>It's a shame that Great Bridge funds a test that
>is solely done to confuse users instead of
>telling the truth; PostgreSQL is good in some
>areas, and bad in others, just like all other
>databases.
I fail to see where they state what you are claiming they did. They merely point that a) on the AS3AP test, using the current production release of MySQL with its ODBC implementation, the performance was lackluster, and that MySQL currently lacks the feature set to run TPC-C.
>The article doesn't mention anything about this >or even with which ODBC driver they used the
>different databases.
As opposed to the benchmark on the MySQL website, where they make no mention of how the databases were compiled, what limits were set, what version of Perl was used, what version of DBI was used, what version of the DBD drivers were used, what hardware platform it was run on, what operating system it was run on, etc, etc.
And futhermore they are benchtesting an alpha product against a faily old stable release (6.5 being current over a year before these benchmarks were posted, with a half dozen released in the meantime).
>I also don't agree with the argument that they
>are not testing MySQL 3.23 as this is still
>marked 'beta'. We have here at MySQL a completely
>different release schedule on our versions...
>Compared to our release schedule, PostgreSQL 7.0 >would be called alpha.
Nice way to imply that the Postgres team doesn't properly beta test. Fact of the matter is that the 7.0 release spend many many months in its alpha/beta/gamma cycle.
There were a couple bugs that slipped through the cracks, but the subsequent releases resolved all known issues, and came very promptly (15 days between 7.0 and 7.0.1, and then 3 days between 7.0.1 and 7.0.2).
As for the development cycles being different, I agree. Though I would go the other direction. The goals of the 7.0 development series were much less ambitious than those of the 3.23 series of MySQL has; e.g. replication, two new backends, transaction support, a real locking mechanism, etc. Not to mention that they stopped adding new features to 7.0 about 6 months ago, whereas MySQL is continuing to add features (and change existing features) of 3.23.
>The net result is that the posted test is about
>as wrong as you can do a test, the important
>thing is just to get the people that reads that
>page to understand that.
Funny, I would say the epitomy of both misleading and incorrect testing would be the sql-bench suite run by MySQL.