I would be glad to see your work. If you don't get a speedy reply, don't be suprised - I am flying from Germany to San Francisco shortly and may not have stable network access for about the next 10 days.
As your humble MySQL community advocate, I exist to serve.;/
To address your concerns with our understanding of the GNU GPL, we are happy to try and improve the quality and clarity of our work.
Feel free to review the following minor modification of the licensing summary? Does it help clarify the issue for you?
--
Our software is 100% GPL licensed, and if your use of it is 100% GPL (or OSI) license compliant, then you may use it without a fee. In all other instances, you are better served by our commercial licence.
--
As for what restrictions were historically imposed on the ISP, I have not run across any old documents to support this (but frankly I have not looked too hard nor do I care too much. I will ask David Axmark about it next time I am in touch with him.)
Where does it contradict the GPL? We explictly state in the first sentence that GPL compliant software can use MySQL for no cost.
As for the exact details of the licence, oh great, now I have to go dig out an old version of MySQL. Dum, deee, dum, dum... here we are:
The MySQL Free Public License grants the right to modify the program, create works based on the program and distribute copies of such throughout the world.
blah blah blah... some paranoid psuedo-legal verbiage...
To roughly sum up, then the license states that you can't resell the program.
So yes, we had a free (as in free speech, though and will) license that also ensured that you could only share your changes to it if you kept them free as in free beer, donuts and love.
However, the license did not force ISPs to purchase license - at least nothing in my reading of it could be interpreted this way.
(Please note that this misperception was a glitch in my personal circuitry, rather than some ominous evil corporate misdirection.:)
Yes, the Windows licensing was different - thought that we had been over that already.
We have changed the license - read the license file in the source.
As for whether this change is for the better or the worse, who knows - we will find out in a few years.:) Ethically we are comfortable with the choice and it makes it easier to explain the licensing to people which is always an added bonus.;)
Licensing is a tricky issue and I know few people who would claim to be able to see the end effect of the licensing choices that we make today.
The holder of the copyright has a choice to allow exceptions and extensions. For example, the Free Software Foundation decided to allow Bison to be used to develop non-free programs, even though the programs it helps to generate contain GPL'd code.
See the GPL FAQ for details: http://www.gnu.org/licenses/gpl-faq.html #CanIUseGP LToolsForNF
Mindshare is a very important reason why we promote the software - we want people using, testing and developing with it.:)
We label the software as alpha, beta, etc. not just to err on the side of caution, but to present developers with an accurate view of the software.
With the 4.0 beta code, we don't want people to blindly put the code into production without finding out why it is labeled as something other than stable.
In many cases, I feel that alpha MySQL code is better than a lot of proprietary stable code. It gets used in heavy production scenarios very successfully. This does not mean that it is mostly bug-free yet. When we close all the open bugs, then we label a branch stable.
Hrm. Nothing like responding to dead thread to drag out the hours of the night, but here goes...:)
The MySQL 4 client library is GPL not LGPL.
Regarding the licensing explanations, what is unclear about:
"This is our licensing policy in brief: Our software is 100% GPL, and if yours too is 100% GPL (or OSI compliant), then you never have to pay us for the licences. In all other instances, you are better served by our commercial licence."
-- http://mysql.com/support/arrangements.html
As for never wanting to be Open Source, we have had an open license for non-Windows versions of the software from day one. For a period of time we did sell the Windows version of MySQL as shareware, but this opened up after we gained a bit more confidence and stability as a company.
Of course, we want people to buy licenses, but we don't have to do a bait and switch to get them to do it.:)
Your friendly neighborhood mysql evangelist... --zak
I am glad that you are trying to help! The post seemed inflammatory due to certain phrases like, 'Why MySQL is Not Suitable for Enterprise or High-Volume Use', 'MySQL.com misleads you about it's capabilities' and 'PISS POOR'
I think that people misunderstood some of what you said because you painted broad generalizations over specific issues. Stating 'Inability to use more than one index on a table in a query' is very different than how you later phrased your concern.
I would certainly like to see some tests on the optimizer - I have been having some discussions regarding benchmarks with the PostgreSQL and Firebird developers and I am sure that we would all find the benchmarks interesting.
As for browsing the optimizer code, you are brave individual.:)
Part of our performance gains did come from the query cache. Windows does a poor job of caching disk I/O and the query cache helps make up for this. I believe that Oracle (and others) use a similar strategy as one of their optimizations. It is interesting with the benchmark to note how similar the performance curves were until each database hit some sort of internal limit. I expect that if Oracle and MySQL had been allowed to continue beyond the limits shown then we would have seen a similar type of drop-off for both at some later point.
Regarding high-use: We have users who are really putting MySQL through its paces under high load - it works quite well and has a low TCO. Low enough to displace Oracle, SQL Server and other databases from existing setups.
Currently, you can ask MySQL to become ANSI compatible by starting the server with the --ansi flag. When this is enabled, MySQL uses double quotes as the identifier delimiter.
The drawback with this is that it still has to be enabled on a per-server basis and many people do not run with it enabled (nor do they want to have it enabled).
This does make life difficult for tool developers and porters.
I will raise the issue with our developers and will reply to you as soon as I hear something.
Thanks for the critical feedback - however, you have some information that is not accurate.
On the positive side, your criticism has provided some interesting ideas!:)
REPLICATION You are correct that the binary log stores the SQL data modification statements that were applied to the master, rather than the actual changes that were made to it. You are also correct that if something goes wrong, the DBA is the one who gets to fix it.
It does not assume that the master and slave are equal - it does assume that the slave contains an linear subset of the data on the master.
We have users working with replication in high-demand situations and it is performing well. Could we improve it? Of course - I was just browsing through the slides from the PostgreSQL Replication talk at OSCON and it looks like some very cool things are going on - we should watch and learn.:)
TABLE LOCK LATENCY It is true that table-level locking used by MyISAM performs poorly under heavy concurrent read/write.
If this is an issue switch to the InnoDB storage engine (which uses low-cost, non-escalating row level locking) or the Berkley DB storage engine (which uses page-level locking).
InnoDB uses a single bit to indicate if a particular page stores rows that are locked. If any rows are locked in the page, then a few more bits are needed for the page to indicate what particular rows are locked.
FILE SYSTEM BUFFERING File system buffering can be a thorny issue. The InnoDB storage engine forces a flush to disk upon the commit of every transaction and then writes a checkpoint so that it knows when the last flush was made.
INDEXING This is just plain wrong. Of course MySQL can use more than one index in a query!
CLUSTERED INDEXES MyISAM tables can be optimized so that the order of the rows in the table matches the order of the indexes on the table.
InnoDB tables already use clustered primary key indexes. Secondary indexes refer to the primary key values.
ON-LINE BACKUPS Replicate the database out to another server (even one on the same machine), then stop the slave to take a backup. This has the added benefit of being able to ask the slave to take over from the master when you need to maintain the master.
BACKUP FORMAT Use the binary log instead of the mysqldump tool - it uses a compressed binary format that is much more compact.
Also, if you have to recreate a table from a mysqldump, then disable indexes until you have recreated all the rows - will save a good deal of time on bigger tables.
Dumping the indexes is probably not a bad idea though.:)
DUMPING TO MULTIPLE FILES An option to allow dumping to multiple files would be convenient.:)
In the past, I just asked mysqldump to dump to stdout and have a perl script handle the segmenting.
The binary log does get segmented into multiple files automatically. I will ask the developers if it would make sense to make a new dump tool that works with the binary log format so as to get the benefits of that format.
BETTER TOOLS... Duly noted - the output from EXPLAIN is cryptic.
DELVING DEEPLY... Absolutely - we could use more tools to give detailed performance information. The current tools that report the number of active threads, the number of questions run, the memory currently used, the max. memory consumed, etc. are not enough. We have some graphical tools like MySQL-Graph (A GPL'd app) to make review of the data easier.
Also, slow queries is more than a counter. The slow query log stores details on every query that ran over the value of the long_query_time setting. Use the mysqldumpslow tool to give a summary of data in the log file. The log can also record queries that did not use an index.
QUERY OPTIMIZATION On what knowledge do you base your assertion that the query optimizer is 'PISS POOR'? Do you understand the code behind it? Have you run benchmarks to compare its performance to another optimizer?
Of course we keep optimizing for specific cases. We want to continue to improve performance whereever possible. We have spent a good deal of effort doing broad optimizations - the optimizations that have the greatest benefit for the most queries. Now we are work more on tweaking specific cases.
MISSING FEATURES The Berkley DB storage engine uses page level locking.
The InnoDB storage engine uses row level locking (without resorting to lock escalation:)
We have excellent transaction support - likely the best of any available database today. InnoDB supports the repeatable read transaction isolation level. However, due to how we implemented our multi-versioning support, we don't get phantom reads. This is a higher level of transactional isolation than MS SQL, Sybase, PostgreSQL, Interbase, Ingres, etc. IIRC, only FireBird and Oracle may be the same.
Sub-selects should be out very soon. We are still working on stored procedures, views, triggers and full support for referential integrity. We know that these features are important. However, we are working on doing truly robust implementations - rushing them out will not help anyone.
END NOTES Why does no one mention the stuff that MySQL is good at?:)
We are fast - we have third party confirmation that we are faster than DB2, MS SQL and Sybase. The test even confirmed that we perform about as well as Oracle (a bit slower:) when running under Windows using a JDBC driver.
We don't need to stop the database to vacuum or do many maintainance tasks.
We know that we can run in critical environments because we have users like Yahoo! Finance *and* Slashdot:) using us for critical, high-load applications.
We are a fully-threaded app and can take full advantage of SMP machines.
We can run natively under a bunch of OSs - including Windows.
Our ability to use different storage engines gives users great choice in how to manage their data. If someone needs a lightweight format in a non-transactional environment, use MyISAM - it has very little storage overhead and is speedy in situtations where you do not have many concurrent reads/writes.
If you need really robust transaction support that ensures the integrity of your data, use the InnoDB storage engine. Storage overhead is more than with MyISAM tables, but that is not generally an issue for enterprise level users.
I can hear people grumbling - well, the transaction support isn't integrated, so it isn't valid... That is complete junk - since when is choice a bug? That is like saying that the Linux is not a modern operating system because its default filesystem does not use journaling. Usually, is it only the proprietary and/or less advanced operating systems do not give you a choice of file system.
Also, we can easily add in new storage to support specific needs. Look at how quickly InnoDB was integrated - it suddenly took us from having no transaction support to having great transactional support. Without the storage engine concept, we would have had to do a lot more work to get it integrated.
Slashdot Mirror
We have supported ANSI SQL 92 conformant transaction for about 22 months now.
Granted, we did use BEGIN as the keyword to indicate the start of a transaction. This syntax is not ANSI SQL 92 conformant.
However, in version 4.0.10 we introduced the comformant START TRANSACTION keyword as a synonym for BEGIN.
I would be glad to see your work. If you don't get a speedy reply, don't be suprised - I am flying from Germany to San Francisco shortly and may not have stable network access for about the next 10 days.
Weeeelll, it goes something like this:
... two thousand and foooooooo......
What year will the 4.0 series of MySQL become stable?
In the year of our lord, two thousand and three, no wait
... and I was asking for your input on the changes that I will suggest to the rest of the MySQL team.
Do these changes clarify the issue?
As your humble MySQL community advocate, I exist to serve. ;/
To address your concerns with our understanding of the GNU GPL, we are happy to try and improve the quality and clarity of our work.
Feel free to review the following minor modification of the licensing summary? Does it help clarify the issue for you?
--
Our software is 100% GPL licensed, and if your use of it is 100% GPL (or OSI) license compliant, then you may use it without a fee. In all other instances, you are better served by our commercial licence.
--
As for what restrictions were historically imposed on the ISP, I have not run across any old documents to support this (but frankly I have not looked too hard nor do I care too much. I will ask David Axmark about it next time I am in touch with him.)
Where does it contradict the GPL? We explictly state in the first sentence that GPL compliant software can use MySQL for no cost.
... some paranoid psuedo-legal verbiage ...
:)
:) Ethically we are comfortable with the choice and it makes it easier to explain the licensing to people which is always an added bonus. ;)
As for the exact details of the licence, oh great, now I have to go dig out an old version of MySQL. Dum, deee, dum, dum... here we are:
The MySQL Free Public License grants the right to modify the program, create works based on the program and distribute copies of such throughout the world.
blah blah blah
To roughly sum up, then the license states that you can't resell the program.
So yes, we had a free (as in free speech, though and will) license that also ensured that you could only share your changes to it if you kept them free as in free beer, donuts and love.
However, the license did not force ISPs to purchase license - at least nothing in my reading of it could be interpreted this way.
(Please note that this misperception was a glitch in my personal circuitry, rather than some ominous evil corporate misdirection.
Yes, the Windows licensing was different - thought that we had been over that already.
We have changed the license - read the license file in the source.
As for whether this change is for the better or the worse, who knows - we will find out in a few years.
Licensing is a tricky issue and I know few people who would claim to be able to see the end effect of the licensing choices that we make today.
The holder of the copyright has a choice to allow exceptions and extensions. For example, the Free Software Foundation decided to allow Bison to be used to develop non-free programs, even though the programs it helps to generate contain GPL'd code.
l #CanIUseGP LToolsForNF
See the GPL FAQ for details:
http://www.gnu.org/licenses/gpl-faq.htm
I am always stunned by the number of hardcore netware geeks that I talk to at conferences. Novell has some pretty freaky staying power.
Mindshare is a very important reason why we promote the software - we want people using, testing and developing with it. :)
We label the software as alpha, beta, etc. not just to err on the side of caution, but to present developers with an accurate view of the software.
With the 4.0 beta code, we don't want people to blindly put the code into production without finding out why it is labeled as something other than stable.
In many cases, I feel that alpha MySQL code is better than a lot of proprietary stable code. It gets used in heavy production scenarios very successfully. This does not mean that it is mostly bug-free yet. When we close all the open bugs, then we label a branch stable.
Hrm. Nothing like responding to dead thread to drag out the hours of the night, but here goes... :)
:)
The MySQL 4 client library is GPL not LGPL.
Regarding the licensing explanations, what is unclear about:
"This is our licensing policy in brief: Our software is 100% GPL, and if yours too is 100% GPL (or OSI compliant), then you never have to pay us for the licences. In all other instances, you are better served by our commercial licence."
-- http://mysql.com/support/arrangements.html
As for never wanting to be Open Source, we have had an open license for non-Windows versions of the software from day one. For a period of time we did sell the Windows version of MySQL as shareware, but this opened up after we gained a bit more confidence and stability as a company.
Of course, we want people to buy licenses, but we don't have to do a bait and switch to get them to do it.
Your friendly neighborhood mysql evangelist...
--zak
Matt,
:)
I am glad that you are trying to help! The post seemed inflammatory due to certain phrases like, 'Why MySQL is Not Suitable for Enterprise or High-Volume Use', 'MySQL.com misleads you about it's capabilities' and 'PISS POOR'
I think that people misunderstood some of what you said because you painted broad generalizations over specific issues. Stating 'Inability to use more than one index on a table in a query' is very different than how you later phrased your concern.
I would certainly like to see some tests on the optimizer - I have been having some discussions regarding benchmarks with the PostgreSQL and Firebird developers and I am sure that we would all find the benchmarks interesting.
As for browsing the optimizer code, you are brave individual.
Part of our performance gains did come from the query cache. Windows does a poor job of caching disk I/O and the query cache helps make up for this. I believe that Oracle (and others) use a similar strategy as one of their optimizations. It is interesting with the benchmark to note how similar the performance curves were until each database hit some sort of internal limit. I expect that if Oracle and MySQL had been allowed to continue beyond the limits shown then we would have seen a similar type of drop-off for both at some later point.
Regarding high-use: We have users who are really putting MySQL through its paces under high load - it works quite well and has a low TCO. Low enough to displace Oracle, SQL Server and other databases from existing setups.
Currently, you can ask MySQL to become ANSI compatible by starting the server with the --ansi flag. When this is enabled, MySQL uses double quotes as the identifier delimiter.
The drawback with this is that it still has to be enabled on a per-server basis and many people do not run with it enabled (nor do they want to have it enabled).
This does make life difficult for tool developers and porters.
I will raise the issue with our developers and will reply to you as soon as I hear something.
Hey,
:)
:)
:)
:)
:)
:)
:) when running under Windows using a JDBC driver.
:) using us for critical, high-load applications.
:)
Thanks for the critical feedback - however, you have some information that is not accurate.
On the positive side, your criticism has provided some interesting ideas!
REPLICATION
You are correct that the binary log stores the SQL data modification statements that were applied to the master, rather than the actual
changes that were made to it. You are also correct that if something goes wrong, the DBA is the one who gets to fix it.
It does not assume that the master and slave are equal - it does assume that the slave contains an linear subset of the data on the
master.
We have users working with replication in high-demand situations and it is performing well. Could we improve it? Of course - I was just
browsing through the slides from the PostgreSQL Replication talk at OSCON and it looks like some very cool things are going on - we should
watch and learn.
TABLE LOCK LATENCY
It is true that table-level locking used by MyISAM performs poorly under heavy concurrent read/write.
If this is an issue switch to the InnoDB storage engine (which uses low-cost, non-escalating row level locking) or the Berkley DB storage
engine (which uses page-level locking).
InnoDB uses a single bit to indicate if a particular page stores rows that are locked. If any rows are locked in the page, then a few more bits are needed for the page to indicate what particular rows are locked.
FILE SYSTEM BUFFERING
File system buffering can be a thorny issue. The InnoDB storage engine forces a flush to disk upon the commit of every transaction and
then writes a checkpoint so that it knows when the last flush was made.
INDEXING
This is just plain wrong. Of course MySQL can use more than one index in a query!
CLUSTERED INDEXES
MyISAM tables can be optimized so that the order of the rows in the table matches the order of the indexes on the table.
InnoDB tables already use clustered primary key indexes. Secondary indexes refer to the primary key values.
ON-LINE BACKUPS
Replicate the database out to another server (even one on the same machine), then stop the slave to take a backup.
This has the added benefit of being able to ask the slave to take over from the master when you need to maintain the master.
BACKUP FORMAT
Use the binary log instead of the mysqldump tool - it uses a compressed binary format that is much more compact.
Also, if you have to recreate a table from a mysqldump, then disable indexes until you have recreated all the rows - will save a good deal
of time on bigger tables.
Dumping the indexes is probably not a bad idea though.
DUMPING TO MULTIPLE FILES
An option to allow dumping to multiple files would be convenient.
In the past, I just asked mysqldump to dump to stdout and have a perl script handle the segmenting.
The binary log does get segmented into multiple files automatically. I will ask the developers if it would make sense to make a new dump tool that works with the binary log format so as to get the benefits of that format.
BETTER TOOLS...
Duly noted - the output from EXPLAIN is cryptic.
DELVING DEEPLY...
Absolutely - we could use more tools to give detailed performance information. The current tools that report the number of active threads,
the number of questions run, the memory currently used, the max. memory consumed, etc. are not enough. We have some graphical tools like MySQL-Graph (A GPL'd app) to make review of the data easier.
Also, slow queries is more than a counter. The slow query log stores details on every query that ran over the value of the long_query_time
setting. Use the mysqldumpslow tool to give a summary of data in the log file. The log can also record queries that did not use an index.
QUERY OPTIMIZATION
On what knowledge do you base your assertion that the query optimizer is 'PISS POOR'? Do you understand the code behind it? Have you run
benchmarks to compare its performance to another optimizer?
Of course we keep optimizing for specific cases. We want to continue to improve performance whereever possible. We have spent a good deal
of effort doing broad optimizations - the optimizations that have the greatest benefit for the most queries. Now we are work more on
tweaking specific cases.
MISSING FEATURES
The Berkley DB storage engine uses page level locking.
The InnoDB storage engine uses row level locking (without resorting to lock escalation
We have excellent transaction support - likely the best of any available database today. InnoDB supports the repeatable read transaction
isolation level. However, due to how we implemented our multi-versioning support, we don't get phantom reads. This is a higher level of transactional isolation than MS SQL, Sybase, PostgreSQL, Interbase, Ingres, etc. IIRC, only FireBird and Oracle may be the same.
Sub-selects should be out very soon. We are still working on stored procedures, views, triggers and full support for referential
integrity. We know that these features are important. However, we are working on doing truly robust implementations - rushing them out
will not help anyone.
END NOTES
Why does no one mention the stuff that MySQL is good at?
We are fast - we have third party confirmation that we are faster than DB2, MS SQL and Sybase. The test even confirmed that we perform about as well as Oracle (a bit slower
We don't need to stop the database to vacuum or do many maintainance tasks.
We know that we can run in critical environments because we have users like Yahoo! Finance *and* Slashdot
We are a fully-threaded app and can take full advantage of SMP machines.
We can run natively under a bunch of OSs - including Windows.
Our ability to use different storage engines gives users great choice in how to manage their data. If someone needs a lightweight format in a non-transactional environment, use MyISAM - it has very little storage overhead and is speedy in situtations where you do not have many concurrent reads/writes.
If you need really robust transaction support that ensures the integrity of your data, use the InnoDB storage engine. Storage overhead is more than with MyISAM tables, but that is not generally an issue for enterprise level users.
I can hear people grumbling - well, the transaction support isn't integrated, so it isn't valid... That is complete junk - since when is choice a bug? That is like saying that the Linux is not a modern operating system because its default filesystem does not use journaling. Usually, is it only the proprietary and/or less advanced operating systems do not give you a choice of file system.
Also, we can easily add in new storage to support specific needs. Look at how quickly InnoDB was integrated - it suddenly took us from having no transaction support to having great transactional support. Without the storage engine concept, we would have had to do a lot more work to get it integrated.
blah blah blah...