Slashdot Mirror
MySQL Beats Commercial Databases in Labs Test
An anonymous reader writes "Many of the big players now offer free or 'light' versions of their databases, some would call them crippleware. Builder AU compared databases from Oracle, IBM, Microsoft and MySQL, and the open source offering came out on top."
I thought the reason MySQL don't post their own comparisons is because the EULAs of Oracle etc. specifically forbid public posting of benchmark results?
And what about PostgreSQL? It should fare very well.
Sig is on vacation
I personally believe that sort of condition in a EULA is unenforcable (even assuming the EULA proper is enforcable - which I don't believe either), as it is anticompetitive. Either way, the test was done by an Australian company, and that could lend a legal hand by setting up international roadblocks to EULA enforcement.
So, look at the pages dedicated to MS SQL Express and IBM DB2. DB2 costs thousands of dollars, MS SQL Express is free. DB2 has a slightly superior feature set and additionally runs on Linux... and they rate it drastically higher, even though it's ridiculously expensive in comparison. Don't even get me started on the fact that the MS SQL version they tested was a beta (almost every Beta MS releases is far slower than the release versions, and contains tons of additional debugging code - VC# Express Betas were drastically slower than the release version of VC# Express.) Of course, none of this is really a suprise, since the 'labs test' is pretty obviously nothing of the sort.
And of course, absolutely no mention of stability, reliability, bugs, robustness, etc... what a suprise, considering that both MSSQL and MySQL are arguably far behind in those areas.
Where are the test cases? Where is the testing methodology? How about some explanation of particular cases where one solution didn't compare with the others, or where one solution excelled? This 'labs test' reads more like a sales pitch than anything resembling an actual test.
using namespace slashdot;
troll::post();
SQL Server Express is not really meant for any serious production environments. I suppose one could use it for a personal web site here and there, but it is a tool primarily meant for developers.
Instead of having to have access to a full fledged SQL Server, you use SQL Server Express to develop your application and then deploy it to a full SQL Server when that server becomes available.
Since SQL Server Express supports the vast majority of the features that a developer might need, it is very useful during the initial development of an application.
In my experience, SQL Server Express is great for basic projects (like a personal web site or blog) and for the initial phases of development of a "real" project. Once you start getting into the realm of serious applications, where one might need finer grained control of isolation and locking, or when you are at the point where you need to do performance testing of your application, you really do need to move up to the full SQL Server box.
At any rate, I'm not really sure this comparison is all that fair. MySQL makes an attempt to be a database server for "real" applications, where as SQL Server Express is more of a development tool / MS Access replacement that is targeted at personal projects.
Actually, it's quite true, and if you look at what's actually going on behind the scenes inside the database, it makes sense.
:)
If there's nothing going on in the database, then the 'summary' value that MySQL keeps is probably spot on accurate. But, if there's lots of simultaneous inserts and deletes, then it's really going to be very approximate. Until things are all flushed, the summary may include all the inserts and none of the deletes, or vice versa. If you wanted to make the summary information accurate, then you'd have to establish locks and the like around that summary value, and THAT will slow the database down. As it stands, inserts and deletes can be executed with ZERO regard to each other.
Postgresql has a similar problem, except instead of offering a summary value and informed that it's an estimate, whenever you do a count(*) it actually scans the entire table file looking for 'valid' rows. Ie, count(*) is not instantaneous. I think they were going to address this issue in a later release (or perhaps it's in 8.1 already), but it's NOT a simple thing. However, if you wanted instant answers in Postgresql NOW, you can do it by setting up a trigger on insert and delete, and maintaining your own summaries. This is a performance hit, of course... but you'd get the same, or a similar hit, if the database was maintaining for you.
What the 'big guys' do, I don't know. But... don't knock MySQL for doing something weird
Postgresql, for comparison, will give you an 'accurate' value, but it actually has to create rows: it can't rely on summary information.
Both you and the parent aren't reading what they wrote. SELECT COUNT (*) is accurate, it was SHOW TABLE STATUS that gives the estimate (as it should, IMO).
I just finished reading it, and the didn't run any performance tests (which I would have liked to have seen). They were comparing options, features, prices, etc. The only mention of performance was things like "here are some neat tunable options that affect performance." and "but performance won't scale that well because you are limited to one CPU" (not direct quotes). This article is an overview, not a "lab test". I'm not sure where that came from. The original article says they were "tested" also. They installed all the products and poked around, but no performance numbers were presented at all.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
This is THE most retarded review of modern database systems that I've ever read. From the moment I read the overview of MSSQL Express, I knew what the writer's opinion was going to be, and that was completely tilted in MySQL's favor. The basic descriptions of product feature were in most cases wrong. One would get the impression from this article that a major RDBMS would always allow dirty reads. And while it's true that you CAN do that, it is not the default behavior for any of them. It has to be explicity done and you have to go out of your way in your SQL code to make that happen.
It's things like that where you just ultimately conclude that the writer(s) of this article just does not know what the hell he's talking about and doesn't have a basic understanding of the concepts or products under review. It's just more OSS nonsensical propaganda in my opinion. And don't fool yourselves into thinking that an article like this is going to change any IT manager's mind about what DBMS he's going to deploy in his enterprise.
The reviewers know databases about as well as my grandma knows sports cars. They seem to mean well, and admit that this comparison was complex and hard. In the end they were unfortunately over their head.
PRODUCT SELECTION
1. where's postgresql? This is the product that the commercial vendors need to be the most nervous about. Sure, they're loosing more low-end revenue to mysql right now, but postgresql is getting picked up by some big players. It is far more mature than MySQL, doesn't have the quality issues, isn't partially owned by Oracle, etc.
2. where's at least a mention of all the various other solutions - from Firebird to Derby (Cloudscape)
FUTURE PROOFING
1. They mistakenly say that mysql doesn't require scaling up to enterprise versions like db2/oracle do. This is incorrect because mysql lags behind oracle & db2 for performance in many situations:
- since it doesn't support query parallelism (which provides near linear performance improves to db2/oracle)
- since it doesn't support partitioning (which can provide 10x performance improvements to db2/oracle)
- since it doesn't have a mature optimizers (which means that queries with 5 table joins can tank)
- since it lacks memory tuning flexibility
Together this means that as your data increases you have to continue moving a mysql database to larger & larger hardware.
In other words, if you need to scan a table with 10 million rows in it, then join that data against 6 other tables - db2/oracle can:
- leverage partitioning so only scan 1mil rows or so instead of 10mil
- split the scan across four cpus
- leverage more efficiently tuned memory (ensuring little tables & indexes stay in memory)
- use the best possible join
and probably complete the query in 1/60th the time that mysql would take. And that means that you could get better performance from db2/oracle on a $25,000 four-way smp than from mysql on a $2,000,000 32-way.
2. They fail to mention that Oracle now owns the most valuable parts of the MySQL solution (Innodb). Oracle has obviously purchased this component (which is how mysql supports transactions, pk/fk constraints, etc) in order to harm MySQL. Since there is no other viable replacement for Innodb the MySQL future is in serious doubt.
3. They probably weren't aware that MySQL is the least ANSI-SQL compliant database in the market. This is means that porting mysql code to another database is a royal pain in the butt compared to code supporting postgresql, db2, etc. Though, to be fair, it is getting much better.
LICENSING COSTS:
1. mysql isn't necessarily free, and can cost more than the commercial alternatives for small distributed commercial apps
2. db2 licensing only provided for DB2 Express- which is the low-cost 2-cpu model. That's often ok, hardly compares to Oracle standard edition also included. Also, I think they may have gotten their db2 costs mixed up between express & workgroup editions.
CONCLUSIONS & MISC
They mentioned some of the great mysql features like clustering and fault tolerance. Sorry, but mysql cluster solution is a separate telecom product that they purchased, that stores your data in memory - limiting your database size to however much memory you can afford. Not a practical solution for very many.
The mysql fault tolerance is really just replication. That's sad.
They mention one strength of mysql is their maximum database size of 64TB - which is nonsense, just because its internal registers and pointers can handle a theoretical maximum of 64TB doesn't mean that it would ever make sense to put more than 20 GB on it. DB2 & Oracle can go to 64TB, but today almost nobody is going beyond 10 TB just due to backup performance, cp
Actually, even the crippled versions of DB2, Oracle, and MSSQL still have the underpinings for advanced features that MySQL doesn't support. From real replication to actual performance monitoring (all three of the big guys provide detailed hooks into the guts of the DB) to support for multiple filegroups and indexes and databases spread across filegroups, the big DB's have features that are important but impact performance.
SHOOT!! you want to see MySQL get its bum kicked on performance? Run a test on a filesystem against MySQL.
Comparing performance among databases is only meaningful if all of the candidates have the features of which you need. MySQL has come a long way, and I use it in production every day, but this is kind of a silly comparison. The free versions of the big DB's are meant to provide an easy migration path to more feature-complete versions; if you use Sequel Server Express and want to upgrade to something that that supports clustering and log shipping, you may your money and get your features. With MySQL, if you outgrow it, you either need to start writing code, migrating to something else, or sitting on your hands waiting for it to get there.
Recap, for those who won't RTFC and want to slag me: I like MySQL. I use it for mission critical purposes in production environments. However, comparing a simpler product's performance to (crippled versions of) more robust products is silly.
Cheers
-AC
This is utterly false.
MSDE is based off SQL Server 2000, which itself a revision to SQL Server 7. MS Access has NOTHING to do with SQL Server (excpet proving nice single DB front ends via ADPs). When your dishonest (or just stupid) so early in a article, you loose your reader.
Lab test? What test? This was a list of features from the product documentation. What a disingenuous title.
Slashdot - where whining about luck is the new way to make the world you want.
Give me a break. This guy has reviewed databases on the basis of features, with, as far as I can tell, not a single real performance evaluation in different kinds of applications (OLTP, DSS, data warehouse), data volumes, or query complexity.
It gets better. In discussing Oracle, he explains: That is not to say the other databases serve up incorrect data but with some database engines when the workload is high, uncommitted data can be flushed from buffers to disk potentially creating a dirty read. MVRC also ensures that readers do not block writers and visa versa. HUH? I can't speak for EVERY database out there, but for most of them, you'd have to specifically set a "read uncommitted" isolation level to actually read dirty data. The majority of the databases would simply give a lock-and-block situation while the second reader waits for the writer to complete. Oracle's MVRC (and PostgreSQL's scheme) both prevent this lock-block situation. But, really, to say that this would potentially create a dirty read situation is just silly.
He also didn't speak of Oracle's new Express Edition. Yeah, it's limited to 1 CPU and has a cap on its data volume, but you get all of Oracle's core features (including PL/SQL) for FREE.
Nothing to see here, folks. Just move along.
If you want to distribute MySQL with your application to a customer, you have to pay a license fee. That means that for many people, MS SQL Express may be better.
If I wanted to do some complex database logic, I'd probably consider MS SQL Express, as stored procedures on MySQL haven't been out there for long.
If you are building a database to go on low-cost LAMP hosting, MySQL does the job well.
For a piece of shareware requiring a small database, something like SQLLite is probably better than these options.
And that is fixed year ago...