Slashdot Mirror
MySQL to Adopt Solid Storage Engine
hmart writes "As seen on ZDNet's Open Source Blog MySQL is taking another step to defend from Oracle's recent acquisitions of InnoBase and Sleepycat. From the article: 'MySQL responds by getting Solid Information Technology, a proprietary database vendor, to take its solidDB Storage Engine for MySQL open source, under the GPL, starting in June.'"
It seems that Oracle is not taking the threat from MySQL lightly. The purchases they have made have been quite public, but they also have been quietly hiring developers from many different open source projects. It is scary how many of them have dropped off the radar. On a side note: I was always under the impression that MySQL was a 'amateur' database. I was very surprised when I was forced to switch from Microsoft SQL 2000 - MySQL had a much smaller memory footprint, and the performance just blew me away. To this day I still cannot believe the performance difference. It really speaks to the power of Open Source.
buy out Solid Information Technology.
-Larry Ellison
So mysql will now have solid storage? Finally! Oh wait, it's " Solid Information Technology". Never mind...
___
If you think big enough, you'll never have to do it.
Does anyone know where I can find more info on the sorts of options/features this storage enginge supports?
For that matter can anyone point me somewhere which describes the deliniation of responsibilities between generic MySQL code and the storage engines? In particular is MySQL just an SQL query parser/optimizer stuck on top of storage engines?
If you liked this thought maybe you would find my blog nice too:
No matter which storage engine you use in MySQL, the SQL syntax is still the same. All you have to change is the ENGINE= line in your schema definitions.
We used to use a Solid SQL database a few years back, before I really knew anything about databases or SQL. It was extremely reliable, fast enough for everything we did, and very easy to administer (even for people who didn't know anything about databases or SQL). I haven't used it recently, but if their technology is still top-notch, this will be very good news.
My Web Page
You might adopt different strategies in code to deal with storage layers that supported different levels of transactionality or locking atomicity.
As Mysql continues to catch come towards the Mid-tier in the market, Postgresql is charging at Oracle.
The entire Database market is heading towards OSS.
And lets all be glad that MSSQL is secretly Sybase under the covers. Because of this we have the beautiful FreeTDS packages for Perl that happily talk to MS SQL server. Those of us stranded in MS shops can actually work this way, imagine the bloodshed if MSSQL was completely proprietary!
I want to delete my account but Slashdot doesn't allow it.
Within two clicks of the Slashdot story page, I found THIS.
If you want news from today, you have to come back tomorrow.
I guess somewhere down the line MySQL users will be pressured to move off of innodb onto this Solid DB. (At which point, we might see some community backlash and a big controversial fork where the community makes a GPL-only version of MySQL to maintain InnoDB support -- which would please Oracle greatly.)
I hope the MySQL has gotten something in paper that will make sure that this is the last time that users have to migrate. Sure, they can claim that a migration will be seamless, but who's going to be foolhardy enough to risk their data on a transition like this without significant testing?
Actually, now that I think about it, I don't know if it is reasonable to expect users to switch "just" so that MySQL will be able to sell commercial licenses to their drivers and what not. The switch to InnoDB took years (and it's probably still ongoing for many users), but in return, those who switched got a lot of new features that made it worthwhile (probably). This time around, there is not much of an incentive for end users -- assuming that the feature sets will be roughly comparable between InnoDB and Solid.
MySQL is about choice... You can choose to use it or you can choose to use something else. You can choose to store your data in InnoDB, BerkeleyDB, Cluster, MyISAM, Memory, CSV
"Choices" is positive spin. Some might say that MySQL is all about sacrifices. Here are some examples:
(1) "MySQL has many applications written for it"
* Not if you enable strict mode, or if you use storage engines that don't support the features you need.
(2) "MySQL is optionally SQL compliant with strict mode"
* Only if you want to forego 99% of the existing MySQL applications, and start fresh.
(3) "MySQL has transactions"
* Not in MyISAM, which means no full text indexing.
(4) "MySQL is free"
* Only if your application which links against the client library is also GPL.
(5) "MySQL is fast"
* Only if you use MyISAM, which means no transactions or many other features that aren't available in MyISAM.
I could go on. Anyone can talk about how MySQL has a feature, but you have to make sacrifices for those features. And I think many of those are bad, unnecessary sacrifices. MySQL implements features not to give their users choice, but to give MySQL AB a marketing advantage. Their advocates and salespeople will always say "yes" to all of those features above, but it's not until later that the customer realizes that they can't use the features together.
Social scientists are inspired by theories; scientists are humbled by facts.
I'd suggest checking out some of the fulltext search libraries if you want flexible, fast fulltext search. It may not be as easy as using the database's built-in support, but the extra effort may pay off in performance and extensibility. For starters, check out open source Apache Lucene, which has ports to several languages, including Python, C, and .NET (C# I believe), and the original in Java. It's easy to get up and running quickly, and designed well enough to be customizable later.
If your application is designed for MySQL specificially without strict mode, then strict mode is not for you. It is more useful to people porting their applications from some other product to MySQL than to the existing MySQL user base.
Ditto above. Perhaps the existing MySQL user wants to go to strict mode in order to support databases *other* than MySQL. Have you concidered that?
MyISAM was never designed with transactions in mind. It performs its intended function excellently. Not all data is useful to keep in some kind of transaction context - take for example a table mapping UPC codes to product names and descriptions: it will never need to change in a transaction so having transactional overhead would be wasteful.
Myth used to scare people away from opensource GPL code.
Some people would disagree with that statement. Quite a lot of people have found that for their needs, MySQL is lighter and faster (or at least, as fast as the big name products) even when never using MyISAM tables.
There will be people whose needs are best served with products from Oracle, IBM, Microsoft, Sybase... I am sure that no one in MySQL claims that it is the only solution to all problems: MySQL is a young database product which is actively being developed so it will improve.
No sig. Move along - nothing to see here.
I'm just worried about this quote from Solid's V.P. of Marketing: It sounds like MySQL will use only a subset of the Solid product. So how good this news is will depend on how complete a subset of its product Solid opens.
If you took a look at the BDB source code, you're understand immediately why forking this is not an option unless you happen to have hired the developers.
My blog
I have been using Perl for 5 years. After 1 month with Ruby, I'm not going back.
That's like saying "i have been eating dogshit for 5 years. After 1 month with steak & eggs, i'm not going back.". Hardly an impressive statement at all.
There will be no need to pressure existing users off from InnoDB .... InnoDB is GPL code and as such will always be available: This is the strength of open source software, it is very hard for players to pick up their toys and leave the sandbox. If they leave the sandbox, they have to leave their toys there. Besides, Oracle has stated that they do not intend to change Innobase's business. Lets hope that they keep that promise.
Of course, MySQL can encourage people to use a particular product choice based upon future roadmaps but I don't imagine that InnoDB support will ever be axed.
(opinions are my own, not of my employer nor anyone else)
No sig. Move along - nothing to see here.
Perhaps if your SQL is trivial and/or you're writing with portability in mind then abstraction is a good idea. If you're doing anything with transactions, locking, indexing, anything which requires the use of SQL which isn't available or identical on all databases then you can't (fully) abstract. If you're not writing with portability in mind abstraction is a waste of cycles and, more importantly, development time.
// MD_Update(&m,buf,j);
Insiders claim that the liquid storage plans went down the drain when an executive bumped into the file server and spilled hundreds of critical files.
This is typical LAMP programmer thinking. What do I need transactions and data integrity in a database for? I'll just code the checks into my application. I prefer instead to put checks both in the database and in the application.
This is no myth. MySQL's client libraries are definitely GPL. If you link to them you must abide by the terms of the GPL. Alternatively, you can purchase a license from MySQL AB. MySQL AB spins this up so much saying that they are open source but what they really mean is that they are open so long as you are open.
I'll stick with PostgreSQL. Unlike MySQL, PostgreSQL is a serious alternative to Oracle or MS SQL. It is also BSD licensed and thus there are zero restrictions on its usage within a commercial product and there is no need to purchase a commercial license.
MySQL is okay for the LAMP mentality but when you start getting in to ORM/ERM (Object/Entity relational mapping) with packages like Rails's Active Record or WebObjects's Enterprise Objects you need a more serious database. As another poster has pointed out in this thread I give it only a few more years before mainstream open source web development moves into the ORM camp.
Hmmm, any real reason why they don't just take postgresql and use that as their backend?
Mmmm'kay... so if there is a performance bottleneak in accessing a table which is essentially constants, the developer decides to simply hard code all that rather than keeping it all in the database. Great. Seen it done many times.
I personally know of many companies which make use of MySQL and have never publically released any code and are still abiding by the GPL. MySQL obviously encourages support contracts for companies using MySQL internally but does not force people to license MySQL in the non-GPL form if their use of MySQL is strictly internal. ie: no binary distribution etc.
I personally use BSD-style licenses for many of my personal project code (see for example the PC-Net simulation code in Xen) but I will equally defend the GPL against FUD attacks. There is nothing stopping companies from using GPL code within their organisation.
(opinions are my own and not of my employer)
No sig. Move along - nothing to see here.
Postgres used to be superior to MySQL in many ways two versions ago. It was much stronger and faster and it used to allow you some actions that were imossible with MySQL like DELETE in multiple base at once. But Postgres also had some problems like the impossibility to alter a DB once it had been created, forcing you to re-create a DB if a minor modification was needed. Now MySQL has evolved a lot and is basicaly at the same level but indeed with a lot more support.
There is a world of difference between "MySQL to Adopt Solid Storage Engine" and getting "Solid Support", as in, from TFA: "...getting Solid Information Technology, a proprietary database vendor, to take its solidDB Storage Engine for MySQL open source, under the GPL, starting in June".
Solid is just offering YA storage engine that works with MySQL, and they happen to be GPLing it. This does -not- mean MySQL is adopting anything from Solid.
Slashdot editors: please read TFA once in a while and please stop writing misleading headlines.
Says here that the marketing VP quoted in TFA, Paola Lubet, used to work for Oracle.
Now I wouldn't want to read anything into that, I'm guessing there's a lot of rotation between database companies, and it's a private company held by a bevvy of investors, so Larry Ellision presumably won't be able to drop it in his shopping basket when he pops out for some milk. On the other hand the investors sound like venture capitalists (I may be wrong, corporate finance is not my field) who might be interested in any short-term returns on their investment that a large, cash-rich company could offer.
I found this page which states that DBD::mysql uses the MySQL client libraries (It says this for Net::MySQL).
.. Any takers?
If you perldoc DBD::mysql, it says it's licensed under the same license as perl (the artistic license).
Unless DBD::mysql is saying that their code is licensed under the artistic license, but as soon as it's built against the MySQL client library it becomes GPL (and just leaving out the last part), I don't see how the MySQL client libraries could possibly be GPL only.
So MySQL is keeping up to Oracle's increasing power - that's good! What about PostgreSQL though? What are they doing to keep up? Anyone know?
... while others (PostgreSQL) have all one needs. I guess that's an unavoidable dynamic of free software (and a good one as well -- just like in other competing open source projects). So probably one day MySQL will have everything that PostgreSQL has today, but why wait ? It's mostly transparent to move from one to the other, and there are tons of guides and scripts to convert from MySQL to PgSQL. Even PhpMyAdmin has its Pg pendent (PhpPgAdmin).
And it's painful to go back when you've learned to do proper logic separation using stored procedures, triggers, views and transactions, things that way too many projects duplicate in the PHP or Ruby front-end, relegating the DB to the "simple" role of being a storage backend (even Ruby on Rails forces you to think that way via the MVC architecture).
For that purpose, use SQLite -- it does what you need, is very high performance (on-disk, non-concurrent access -- can't get any faster than that). For real solid DB work, with garanteed integrity, just the way RDBMS were meant to be, go for PgSQL.
we're using HP data protector backup/restore software, which uses solid as its database for storing all data information.
HPDP itself is as stable as an italian taxi driver stuck behind two priests in a skoda, but thank GOD, the database is rock solid and doesn't corrupt when HPDP goes belly up again.
so all in all, a good pick of mysql!
On a long enough timeline, the survival rate for everyone drops to zero.
Well, according to Google:
"Eating Dogshit": 1,220
"Eating Steak & Eggs": 43
I'd say more people are eating dogshit than eating steak & eggs. Godbless the interweb!
I'm going to go create my own technology news site, with blackjack and hookers. You know what? Forget the news site.
> Isn't it better to have a choice of technologies?
uh, better than what? If the answer is stability, then no.
So, MySQL has the following storage plug-ins: MyISAM, Innodb, Sleepycat, etc, etc, etc. as an enterprise data architect (ok, sounds pompous, how about someone that implements a *lot* of database solutions) I really don't care about a wide variety of storage engines that mostly don't meet my needs. Of all of the above, only Innodb is a good general-purpose solution. The others are useful only in specialized circumstances or are obsolete.
The idea that multiple storage engines is a great idea is just market spin.
Oracle, DB2, Informix, Sybase, SQL Server, Teradata, Postgresql, Firebird, etc - all have their own. There is no risk to the user that another company is going to buy out their storage engine, force the database vendor to perform an expensive switch and possibly impact functionality or licensing for my applications.
If MySQL wants to support multiple storage engines that's fine. Not terribly attractive to me, but fine. But first - it needs to own its own. And its own needs to support ACID as well as non-logged activity. This shouldn't be news to anyone: this is what every single commercial database provides and has been the way to do it for 20+ years. It's still the way to do it.
> Freedom of choice is always a good thing.
> MySQL is not about having cheap database... MySQL is about Freedom of choice.
How about cheap, full-featured, standards compliant & stable RDBMS? Why isn't this a possibility? This is what I want, not freedom to pick from a pack of obsolete, special-purpose or dead-end storage options.
> What would be the list of priorities for you for features/changes to be made to MySQL in order for
> you to seriously concider it. Bear in mind that MySQL does not have an infinite amount of developers
> so delivering on all of them overnight would not happen.
Now that MySQL has implemented most of the standard database features, the next step is to harden them and the rest of the codebase. At this point I'm not interested so much in features as I am high-quality & robust implementations.
- this means that if I add a column to a table I won't have to spend hours copying the data back & forth under the covers
- it means no more gotchas - with broken date handling, exception handling, type conversions, value truncations, etc.
- it means that individuals clients shouldn't be able to turn off strict mode.
- it means that non-strict mode should be deprecated - and gone in a few years
- it means that the optimizer should be able to handle very complex joins and almost always come back with the very best possible query
Now, *after* the above have been addressed there are some additional features to add. These are mostly oriented towards reporting functionality, where I think mysql is pretty weak right now. My list would include:
- oracle-style range partitioning: this means that you can using mysql for large reporting & warehousing applications. We're not talking union-all-views here, but the ability to attach & detach partitions, and disregard partitions of data from queries. I often see 10x performance gains from partitioning over indexing in this kind of app.
- query parallelism: need to be able to split reporting query work across CPUs. These large queries, often scanning a 100,000+ rows can typically get linear performance improvements in my experience from query parallelism.
- automatic query rewrite: need to be able to automatically rewrite queries in order to hit summary tables if they exist. This is critical for some types of powerful reporting tools, or reporting performance boosts for commercial apps.
- materialized views: simple method to manage summary tables. Not critical since this can be developed in other ways, but this has become the preferred way to manage summary tables.
- federation: the ability to redirect a query against a table to another database to be resolved. This allows you to define reference tables on one database, then share them across many.
- query failover: the ability for the mysql client to automatically reroute a query to a failover server. This allows you to provide failover without any code changes at the application layer.
that's all i've got time to write, but should cover most of what I'd look for first.
Support for this is being implemented but it does rely on the storage engine to know what to do. Otherwise, the fallback is to do it the old slow way.
Good suggestions. I'll try to make sure that they're not forgotten.
This would mean breaking backward compatibility - something that I think will be undesirable. There are some embedded applications which do not benefit from strict mode for example.
This is the "holy grail" of database systems... What may seem trivial to us to decide what query plan is best may be non-trivial to calculate. There may be a large number of optimal paths but the optimum one may be very hard to deduce due to local minimas. Many databases have such large and complex query optimizers that the query plans have to be precalculated and cached. MySQL has a skilled query optimizer team and I am sure they will do great stuff.
Already actively in development and may be previewed in the 5.1 tree.
Only available in MySQL Cluster to a limited degree. Tricky to make a general solution but it is something we are aware of.
Already available and in 5.0 tree, although improvements are possible.
AFAIK, JDBC connector already provides this and I think the
No sig. Move along - nothing to see here.