Slashdot Mirror
MySQL Falcon Storage Engine Open Sourced
An anonymous reader writes "The code for the Falcon Storage Engine for MySQL has been released as open source. Jim Starkey, known as the father of Interbase, is behind its creation; previously he was involved with the Firebird SQL database project. Falcon looks to be the long-awaited open source storage engine that may become the primary choice for MySQL, and along the way offer some innovation and performance improvements over current alternatives." This is an alpha release for Windows (32-bit) and Linux (32- and 64-bit) only, and is available only in a specially forked release of MySQL 5.1.
I'm a developer, but I've never used MySQL.
Isn't MySQL already open source? If so, how does the Falcon storage engine differ from the "regular" storage engine that comes with MySQL?
The most rabid believers in American Exceptionalism are the exact same people whose policies are destroying it.
I've been very excited since I first heard about this new storage engine adapted from Netfrastructure. Not only does it give MySQL a transactional storage engine that is not controlled by a hostile company, but the engine appears to be designed from the bottom up to support web traffic. Jim gave a great talk at the Boston MySQL meetup that you can watch here http://video.google.com/videoplay?docid=1929002440 950908895
Get developed by a company that doesn't hate MySQL, for starters.
Dewey, what part of this looks like authorities should be involved?
Ugh, no you haven't. The storage engine is how MySQL stores the actual data on disk. There are different formats, each have their own benefits, issues and work arounds. Think of MySQL as a SQL interpreter talking to different storage engines on the back. The default is MyISAM, but InnoDB and ISAM are others.
OK, although I don't agree with it, I know that a lot of people use MySQL because it's the most common database supported by web hosts. Isn't it almost as likely for a hosting company to have PostgreSQL, though, as to upgrade to a bleeding-edge version of MySQL when this is finally readied for public consumption? Will this new backend give any extra functionality that PostgreSQL doesn't offer?
Dewey, what part of this looks like authorities should be involved?
Actually, a good question would be : What is the difference between Mysql and Mysql falcon?
Mysql maximizes rich channels and empowers cross-platform convergence letting you drive mission-critical niches whereas Falcon utilizes scalable initiatives by scaling end-to-end networks for reintermediate granular platforms (win32 and linux 32/64).
Like others said already .. First of all, Falcon allows MySQL to not be as dependent on InnoDB as it is right now. When Oracle got hold of InnoDB, people had serious doubts about MySQL's future. For a solution like MySQL, it's always good to have competing (Even if similar) engines for it's own survival / growth.
So if Falcon uses MVCC, does it require something like PostgreSQL's VACUUM? Or does it have some other way to detect and remove dead tuples?
Also, has anyone looked at making PostgreSQL a storage plugin for MySQL? :-)
I was forced to learn Postgres because I wanted to do a SELECT * FROM foo WHERE bar NOT IN ( SELECT x FROM y WHERE z > 4 ) type query - basically a negative subselect. I couldn't do (or couldn't work out how to) use table JOINs to perform the same thing, and at the time, MySQL didn't do it.
I'm so thankful. Postgres is so much better. And the table JOIns I did have ran so much faster in PG. s/mysql_/pg_/g *.php
Get your own free personal location tracker
But in something as mission-critical as a database, of all things, reliability trumps everything. I don't think they could have developed PostgreSQL any other way and still supported its primary goal of safety.
What gave you the (wrong) impression that PostgreSQL folks have been sitting around twiddling their thumbs? Version 8.2 just came out within the month and includes several performance boosts that make it fly on our production systems.
Dewey, what part of this looks like authorities should be involved?
Falcon looks to be the long-awaited open source storage engine that may become the primary choice for MySQL, and along the way offer some innovation and performance improvements over current alternatives.
Come on. Give me PostgreSQL any day. After fiddling with MySQL at work for a few custom developments (both versions 4 and 5.0.2) i'm ready for anything else. Gave PSGSQL a shot at home and it runs very nice, with lots of advanced features. Anyone with real-world deploying feedback to share?
for smaller databases, but limiting the tablespace to a single file per database/schema doesn't sound very flexible, and won't allow DBAs to maximize their disk throughput.
That isn't necessarily the case. Recently it has become popular to aggregate spindles into single stripe/mirror volumes with large stripe widths. This spreads I/O operations uniformly across disks. All disks contribute their IOPS capacity to all operations. Large stripe widths attempt to leverage high sequential IO bandwidth.
Oracle calls this policy SAME; Stripe and Mirror Everything. Their Automatic Storage Management (ASM) is an implementation of SAME, and it is recommended for use with both OLTP and OLAP applications.
If I had to make a MySQL Falcon instance fast I would build a large stripe/mirror volume for the (single) tablespace file. The point is that isolating high load tables on separate spindles isn't necessarily the best policy, according to current thinking. Of course there are other reasons for isolating parts of a database, but you only mentioned throughput.
Lurking at the bottom of the gravity well, getting old
Nah, I'm just giving you a hard time. I've worked with a couple of people who have only ever worked on MySQL, and they tend to not know some pretty essential things for a DBA like ACID compliance and such.
It's just a function of how easy MySQL is to set up. It's trivial to set up, but a lot of the default decisions are generally bad for an SQL database, and the documentation -- while good -- never encourages you to go beyond the defaults.
It's like hearing someone say they can design websites, and then finding out they mean with FrontPage.
Drupal is one example of something that works great with MySQL. I can argue about MySQL's faults over and over, but at the end of the day it's easy to use and it's good enough for most people. CMS systems and forums are where MySQL really shines.
The road to tyranny has always been paved with claims of necessity.
That depends on the database.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
The big thing Falcon brings is MVCC which allows safe simultaneous reading and and writing without locks.
Here is a good explanation of PostgreSQL's MVCC.
We don't see the world as it is, we see it as we are.
-- Anais Nin
Microsoft SQL Server has had almost all of these features since its first release in the early 90s. MVCC was just introduced in Microsoft SQL Server 2005. There is no row-level compression in SQL Server (or Oracle, or DB2, or PostGreSQL... which is probably a *good thing* from a performance perspective).
This is a nice step forward for MySQL, but for the most part it is just a means for catching up to the other commercial DBs and PostgreSQL. ACID compliance, granular locking, MVCC, and multithreading are *not* differentiating features in the database world.
Umm, Postgres has had automatic row-level compression (automatic compression of TOASTed) values for some time now.
Actually, compression can improve performance. Often disk I/O can slow you down more than the extra CPU work to compress/decompress. I have written some flat-file processing code at my work and it runs much faster reading from and writing to zipped text files instead of uncompressed text files. The space savings are just a bonus.
I've ended up having to support and/or migrate several Access databases designed by end users ...
... ;)
These guys who've set up MySQL are gurus by all comparisons
"The large print giveth, and the small print taketh away" -- "Step Right Up", Tom Waits
MySQL *have* done so with Falcon. MySQL *do* own Falcon.
[Jim has worked on other RDBMS in the past] != [MySQL do not own the one he's currently working on]
Il n'y a pas de Planet B.
Meh, I liked it better when they called it Sybase.
"I use a Mac because I'm just better than you are."
Why would you be concerned about the license for a database? The licence is only important if you are modifying the code AND distributing the modified code. Most database users I know just install the binary and use that in which case you don't care if it's BSD or GPL licenced.
evil is as evil does
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
I've read through all comments with 2 or more in rating, and it seems that people really underestimate what Jim is doing here.
We're talking in-memory MVCC here. This means you can add 1000 records, do a rollback, and the harddisk hasn't been accessed. Even if you commit, performance will eventually be magnificent compared with on-disk MVCC systems. You can run larger systems on one server with this, than you would be able to run on a cluster with other database systems.
This system has been designed to provide very good performance improvements for those who do know how to create SQL statements, but probably even better performance improvements for those who don't. And we don't have a tradeoff between performance and transactions any more - transactions and better performance are both included.
Also, please note that this technology will make MySQL a trustworthy data storage for many commercial applications out there, giving added value to their apps and their businesses. It will also enable small but very skilled development teams able to use MySQL as a trustworthy database for specialized applications - previously only Firebird and Postgresql were able to provide this for free, and even though Firebird has a very high deployment in USA's top 500 companies, postgresql seems to be very much *nix only in deployment statistics.
I have been programming database applications for more than 20 years, and have been programming Oracle, MSSQL, MySQL, postgresql, Firebird, dBase, Paradox, Access and other databases. I see Jim's contributions to MySQL as extremely important for the database market. Instead of having "just" a transaction layer on top of a storage layer, MySQL now provides mechanisms that give this design an advantage over those database systems where the transactions are stored on disk (like Firebird, Postgresql).
And - by the way - this has NOTHING to do with "optimizing for web applications". Web applications are just as diverse as GUI applications and other systems, and GUI applications will benefit from this as much as web applications.
InnoDB is licensed under the GPL. InnoDB is no more dependent on Oracle then Linux is on Red Hat. MySQL could if need be develop InnoDB themselves.
Except that a large chunk of MySQL AB's revenue would disappear as they'd no longer be able to sell a commercial version of their database with InnoDB support. And believe me, few people are going to buy the commercial version if the preferred storage engine is MyISAM. MySQL pay InnoSoft for the right to distribute the InnoDB engine as part of their commercial version - whether Oracle allows them to renew this agreement when it expires is somewhat doubtful.