Comparing MySQL Performance

An anonymous reader writes "With the introduction of the 2.6 Linux kernel, FreeBSD 5-STABLE, Solaris 10, and now NetBSD 2.0, you might be wondering which of them offers superior database performance. These two articles will show you how to benchmark operating system performance using MySQL on these operating systems so you can find out for yourself if you're missing out. While this may not necessarily be indicative of overall system performance or overall database application performance, it will tell you specifically how well MySQL performs on your platform."

Comments

What no Windows benchmarks?

[sjrstory]

I thought Windows XP was supposed to be the Fastest and the Most Reliable OS in the World

...never believe everything you read on the Intarweb. ;)

That's all well and good, but...

[Sheetrock]

MySQL is a different animal from PostgreSQL, which is itself a horse of a different color than DBI. To truly profile these operating systems you must take into account the differences in:

• Tuple calculus
• Transaction journaling
• Operator space/system call overhead
• Disk cache timings

And much more... in essence, you can't be certain these benchmarks hold true for the performance of all databases and it may even be a mute argument -- the same operating system may be tweaked differently if you're fileserving or mailserving or networkserving or if you're only dataserving. A useful tool, but one that must be run on each server.

Re:That's all well and good, but...

[dubl-u]

and it may even be a mute argument ...

or even more likely, a moot argument

So you think. It's all the rage now to have technial meetings where nobody speaks and all arguments are made through pantomime. I'm sure that's what he's talking about.

My favorite thing to explain that way is an elevator algorithm, although stochastic fair queueing is a close second.

Re:postgres

[TheWingThing]

If you R the FA, which not many do these days, he says he didnt have the time though he planned to do both, so he did MySQL. This is mentioned in the first link, which was /.ed last week. The first link was Part 1 which explains the setup and procedure. Part 2 (second link) explains the results.

Tuning on FreeBSD

[zulux]

On a PostgreSQL install, I almost quadrulpeled performance on FreeBSD 4.10 by bumping up the SHMMAX in FreeBSD, then tweaking PostgreSQL to use it for queries and indexes.

Make sure FreeBSD has DMA turned on as well, and make CFLAGS somthing other than a 486.

All of the *BSD are *VERY VERY* conservative and will do a lot better when properly configured.

Re:Tuning on FreeBSD

[a11]

Valid point. RedHat Linux is tuned for Oracle - including a larger shared memory region. A more valid comparison would be if each OS was first tuned for the database.
Something like shared memory, which is used for sorting, caching, and hashes, would slow a database down quite a bit if there was not enough.
I work with multi-terabyte databases daily, and by my observation, the flavor of UNIX is irrelevant if the IPC resources are adequate. When you're scanning a gig from disk, all DMA, an extra second in the kernel doesn't count in the O().

If TFA gets slashdotted, these are his conclusions

[TheWingThing]

Conclusion/final thoughts

Both Linux 2.4 and 2.6 had the strongest showing overall for these tests, dominating just about every benchmark no matter the workload. Scalability for both kernels was also excellent with addition of an extra processor. In fact, I was surprised how well 2.4 had done, as I had somewhat expected 2.6 to show at least a noticeable, if slight, increase over 2.4. Instead, they took turns besting each other from test to test -- and in scalability -- for a fairly even overall showing.

Solaris 10 had a very strong showing as well, having great speed as well as great scalability. I think the results show that Solaris 10 is a great platform for MySQL. Of course, I didn't have Super Smack results as I couldn't get Super Smack to port to Solaris (as detailed in the previous article), so bear that in mind.

NetBSD 2.0 also had a very strong showing, although it was tarnished by two issues. One, MySQL on NetBSD 2.0 doesn't scale with the addition of CPUs. The results would seem to indicate that it might be wise to run a uniprocessor kernel even if two processors are available. The other issue was the poor I/O performance for the 10M row SysBench test. The SMP scalability issue is easy to understand since, to be fair, this is the first NetBSD release to support multiple processors. The I/O issue is more of a mystery, however.

FreeBSD 5.3 did relatively well in both KSE and linuxthreads mode, although with all the work that's been done in the SMP and threading realms, I was a little disappointed with the results. Still, it seems that the native threading model for the production release of FreeBSD-5 is ready for prime time, and can replace the long-standing FreeBSD convention of using linuxthreads with MySQL.

For FreeBSD 4.11, however, linuxthreads definitely helped with performance (and in many cases outperformed FreeBSD 5.3). With libc_r, performance lagged far behind linuxthreads for many tests, and there was little scalability. I would say it's highly advisable to build your FreeBSD 4.11 MySQL binary with linuxthreads.

For all the time it took, I think the tests were worth it. I learned quite a bit about MySQL performance in general, and I'd like to again thank Peter Zaitsev for his methodology recommendations and input, as well as Jenny Chen from Sun for her input.

use windows!

[mboverload]

I use Windows Server 2003 for all my SQL needs. It is 20% faster than an equivilent Linux machine!

Well, at least thats what Microsoft told me...

How relevant are those benchmarks

[Donny Smith]

Does anyone go: "OK, I need the OS for my mySQL project. I'll benchmark BSD, Linux, Windows, and choose the fastest OS."

Difference among OS should be pretty much unimportant unless one's an ISP or big enterprise. I would choose the OS based on completely different criteria:
1) Existing skillset (advantage to existing skills)
2) Existing deployed OS (advantage to OS already deployed)
3) My company's OS strategy (advantage to the OS and the CPU platform we chose to standardize on)
4) Existing software (if I already have X vendor's backup agent for mySQL on Linux or database tuning tools, I wouldn't use BSD just to (potentially) gain an extra 5% in some ludicrous benchmark result).

Today's hardware (and operating systems) are so cheap that it's almost irrelevant what OS and hardware goes into many a project.
Look at the new HP's 25p and 35p blades (Opteron-based) - a 2 processor 1GB RAM version is just some $1,700 more expensive than a 1 processor 512MB RAM version.
It's easy to lose that $1,700 in downtime, spend it on a Windows engineer's new RHCE or such...

Re:Other Benchmarking

[E_elven]

Does anyone know of a nice computational/science benchmark that runs on Linux/Windows/BSD and is free?

Try:

int main(int argc, char** argv)
{
for (long i = 0; true; ++i)
{
char* x = new char[i*i];
std::cout << "I love Stevie Hawking!" << std::endl;
}
}

You'll need a stopwatch.

Re:Why do people use MySQL over Postgres?

[SerialHistorian]

I've written a bunch of enterprise-class stuff on MySQL.

The first and second answers are inertia. All of my tools work with MySQL and I'd have to spend a week or two re-writing them for PostgresSQL, and I can't shake loose that kind of time right now.

Also, I have a set of redundant, mirrored MySQL servers in my colo box that run all of the websites I've built, and I'd have to get more rack space or convert everything over to Postgres at the same time. Neither of which are cost effective when what I have ... works.

The third answer is that MySQL is blazingly fast at doing simple things. Where Oracle (The other RDBMS that I'm familiar with) can return simple select queries or complex insert or joined select queries in .5 to 1.0 seconds each, MySQL can return simple queries in .01 seconds and stupidly complex queries in 5-10 seconds. Since 100% of what I'm doing is simple selects or can be hacked very quickly to seem like simple selects, there's no reason to use anything more powerful for what I'm doing.

I don't need to have "good habits" ... I don't need to have nth degree optimized queries. I don't use 99.99% of the features that MySQL has, not to mention all the features that Postgres has that I wouldn't use. (And don't get me started on Oracle.) It's also faster for me, in both database query return and programmer time, to execute 5 simple, general, fast queries that are part of a code library (and when the database structure changes, edit that one code library) than it is for me to write one really complex query for each code module (and have to edit every module when the database structure changes).

What it comes down to is that it works well as a lightweight database for websites and web apps, and there's a ton of community support and literature. It's not Oracle. It never will be. It's not useful for everything. But when you need a lightweight database to handle a ton of simple select queries without melting down, .... MySQL fits the bill. Why swat a fly with a sledgehammer?

Procedural problem with NetBSD multiprocessor

[Bushcat]

It seems like the performance of NetBSD will be re-evaluated, so expect the results to be recast in the next few days.

See the message thread titled "NetBSD performance" at http://software.newsforge.com/article.pl?sid=04/12 /27/1243207: an anonymous reader asks "Did you enable PTHREAD_CONCURRENCY? You have to set that variable to the number of CPUs in your system, else you won't be able to run more than one thread at a time, even you have more than one...". He replies "Sunofa. The $PTHREAD_CONCURRENCY environment variable wasn't set, as I had no idea it was an option. ... It could very well be the issue. In the next few days I'll re-run the NetBSD tests with that set."

Amazing

[cranos]

Somebody posts a comparison of an application running on different OS's as a system benchmark, and what do people do? Attack MySQL.

God guys get over it, MySQL is here and it has actually proven itself to be usefull. Yes its missing features and has issues, but it fills the niche it is aimed at.

Re:MySQL vs PostgreSQL

[agilen]

He was modded offtopic, because this article was about OS benchmarks, not database benchmarks. The title of TFA is "Using MySQL to benchmark OS performance"...so its seeing which OS mySQL runs best on, not which database is best.