Slashdot Mirror


Google Releases MySQL Enhancements

An anonymous reader noted that "Google has released its internally developed enhancements to MySQL to the open source community this week. Changes include improvements in replication, high availability configuration, and performance." It'll be interesting to see if the changes they made are of interest to other places using MySQL.

51 of 208 comments (clear)

  1. Great! by glwtta · · Score: 5, Insightful

    Did they fix that thing where it always sacrifices data integrity for speed?

    (I'm not even trolling, I do want to know if they fixed that)

    --
    sic transit gloria mundi
    1. Re:Great! by Bigby · · Score: 4, Insightful

      You can go with a slower speed and higher data integrity by switching from MyISAM to InnoDB tables. The choice is there, so I would use the term "always sacrifice".

    2. Re:Great! by xelah · · Score: 4, Insightful

      He doesn't mean 'foreign keys actually work', or 'inserting nonsense dates gives an error' or anything of that kind. He means things like:
          - The database doesn't corrupt tables. Ever.
          - If the power fails or the kernel goes away at an arbitrary instant, then when the database starts up again all of the data will be there, with committed transactions entirely present and uncommitted ones entirely gone.

      Secondly, it's not justified to just assume that MySQL will be faster even with it's limits on data integrity. It depends on your workload. Consider differences in locking strategy and query plans, for example. There's a benchmark showing scaling behaviour in one particular set of circumstances here: http://tweakers.net/reviews/674/6 ; this shows a fairly striking difference in scaling with load on a specific machine.

    3. Re:Great! by Matt+Perry · · Score: 2, Insightful

      You can go with a slower speed and higher data integrity by switching from MyISAM to InnoDB tables. The choice is there, so I would use the term "always sacrifice".
      According to the docs MyISAM is the default table format. So how about "sacrifice by default." Sure, you can change the default to InnoDB but why isn't it set to InnoDB already? I find it strange that you have to enable data integrity features in a RDBMS.
      --
      Slashdot: Failed Car Analogies. Amateur Lawyering. Anecdote Battles.
    4. Re:Great! by rnicey · · Score: 2, Funny

      Because then they'd have to admit that they're a fancy SQL parser, and most of the real meat is contributed by a company called Oracle (ahem Innobase). That said, InnoDB is a real nice data engine.

    5. Re:Great! by jbellis · · Score: 2, Interesting

      "MyISAM is faster than InnoDB" is certainly the conventional wisdom. It's also wrong.

  2. Fit for duty? by Anonymous Coward · · Score: 2, Funny

    Those Googlians are stupid, don't they read slashdot? If they did they'd know MySQL isn't fit for ANYTHING! Sheesh.

    1. Re:Fit for duty? by alienmole · · Score: 3, Insightful

      Yes, but imagine the world's biggest Beowulf cluster of MySQL servers.

      Now imagine them in Google's data centers.

      Which, in fact, is where they are. Now do you see?

  3. Re:so... by bigman2003 · · Score: 4, Interesting

    No, all hail MySQL.

    I'm a recent convert from SQL Server, and I love MySQL now.

    The fact that I can download a copy and get it running in about 20 minutes is the best part. The fact that the admin is pretty damn good, and easy, is the part that keeps me using it.

    Maybe there IS something to this whole open-source thing.

    --
    No reason to lie.
  4. Google-y goodness by loafing_oaf · · Score: 5, Insightful

    Ah, now this is how it's supposed to work. No bull like, "We're releasing improvements as MSN-SQL," or any other nonsense. Yay Google.

    --
    Always someone has power over you. The thing to consider is this: Is the power good, or bad?
  5. Wont be included in MYSQL... by emj · · Score: 4, Insightful

    MySQL is dual licensed so if they add this code they can't sell their product under another license..

    1. Re:Wont be included in MYSQL... by Albanach · · Score: 5, Informative

      MySQL could license the code for distribution. Equally Google could submit the code to MySQL using their existing mecanism for code contributions and transfer copyright to MySQL.

      To suggest that the code can't be included because someone else owns copyright is just wrong - MySQL didn't develop the Innodb stuff, they license it. They could do the same here.

    2. Re:Wont be included in MYSQL... by BinaryPower · · Score: 3, Insightful

      Google doesn't sell a lot of software products. Some, but not a lot. It's intended purpose was to speed up their own internal system, and so they decided to share their work because it probably works well.

      --
      Patience is a virtue. Acquire it as fast as you can.
    3. Re:Wont be included in MYSQL... by MightyYar · · Score: 2, Informative
      That's true, but only part of this release is a patch - the other two items are tools:

      • mypgrep.py - a tool, similar to pgrep, for managing mysql connections
      • compact_innodb.py - compacts innodb datafiles by dumping and reloading all tables
      • patches - patches to add features to MySQL 4.0.26

      It will be interesting to see what happens with the patches.
      --
      W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
    4. Re:Wont be included in MYSQL... by vivaoporto · · Score: 2, Insightful

      Yes it does. And it was GPL, as it's said in TFA. So, no dual license if it is included.

    5. Re:Wont be included in MYSQL... by diegocgteleline.es · · Score: 2, Insightful

      Also, because it's just cheaper for them if they get it merged in the mainstream mysql tree. Basically, it's stupid not to release it.

    6. Re:Wont be included in MYSQL... by Bacon+Bits · · Score: 3, Informative

      No, these won't be included because the patches are for MySQL 4.0. To give you an idea, MySQL 4.0 is what shipped with Debian 3.1 (Sarge). It's that old. Mysql.com doesn't even host binaries for MySQL 4.0 anymore!

      4.0 is not actively developed at all, and lacks basic requirements such as subqueries, accurate math for DECIMAL or NUMERIC types (they're identical to FLOAT in 4.x so decimal errors for aggregate functions are common), and VIEWs. That's in addition to the normal flaws for data integrity in MySQL (MyISAM database engine, fsync disabled by default) that make it one of the least ACID compliant databases around that can still understand SQL.

      This is like Google releasing patches for Apache 1.2.

      --
      The road to tyranny has always been paved with claims of necessity.
    7. Re:Wont be included in MYSQL... by MightyYar · · Score: 4, Informative
      If you look at the subversion repository, they also have patches for MySQL 5. It seems as though they are not going to "release" those patches until they are feature-equivalent to the MySQL 4 patches:

      These will have the same functionality as the MySQL 4 patches. Unfortunately, only some of the features have been ported. This will be updated when the patches have equivalent functionality.
      --
      W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
  6. The patch... by dfdashh · · Score: 5, Informative
    --
    df -h /my/head
  7. This illustrates a problem with commercial OSS by MikeRT · · Score: 4, Interesting

    At least with the GPL, anyway. You have a company like MySQL which actually makes a product, rather than bundling and providing services as the majority of the business. However because they can't exclusively control the app, they can't make a large amount of money except on other services and such. Then a company like Google comes in, with tons of resources, and makes great modifications to your code. The only problem is that you can't merge them into the commercial distribution without their permission. What's ironic about this is that being dual-licensed, the code would remain free for everyone to use under the GPL.

    MySQL chose this approach, but I could see how this might make others less inclined to dual-license under the GPL.

    1. Re:This illustrates a problem with commercial OSS by 99BottlesOfBeerInMyF · · Score: 4, Insightful

      This illustrates a problem with commercial OSS At least with the GPL, anyway.

      I think this illustrates a problem with trying to sell OSS as if it were closed source software, instead of relying upon contract work for improvements, customizations, services, and other closed source add ons or using that OSS as a tool yourself for some other market.

  8. Silly Question by bill_mcgonigle · · Score: 2, Informative

    Did they fix that thing where it always sacrifices data integrity for speed?

    It's a silly question, really. MySQL is about performance. If you can trade off integrity for speed, and you need speed, use MySQL with myASM tables. If you need that integrity and would benefit from MVCC use PostgreSQL. If you need MySQL for some reason (web hoster, etc) and want the integrity, use InnoDB tables (for version 5 - something else for version 6), though there's a performance hit and you still don't get MVCC.

    --
    My God, it's Full of Source!
    OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
  9. WTF? by mekane8 · · Score: 3, Interesting

    MySQL is the second most popular database behind Microsoft SQL Server, according to a winter 2006 report from Evans Data. Oracle comes in second if you combine its database offerings. The situation is similar among wireless developers, almost 30% of who use Microsoft SQL Server compared to the 20% who use MySQL.
    What exactly does Oracle offer besides database offerings? How do you not include its "database offerings" when you're ranking the most popular databases? Is information week retarded? The vague reference to "wireless developers" is also weird. Wouldn't the Google page be more worth reading and more coherent?
    1. Re:WTF? by 0racle · · Score: 2, Informative

      What exactly does Oracle offer besides database offerings?
      Application Server, ERP, some other stuff.

      What they are talking about is counting Oracle Standard, Oracle Enterprise and other Oracle database editions as separate offerings to make MySQL look more important.
      --
      "I use a Mac because I'm just better than you are."
    2. Re:WTF? by Smidge204 · · Score: 3, Informative

      The word is "combine" not "include" - Oracle offers a variety of Database "flavors" (editions) as well as services and utility suites. If you "combine" these offerings, Oracle comes in second.

      =Smidge=

    3. Re:WTF? by Em+Ellel · · Score: 2, Insightful

      What exactly does Oracle offer besides database offerings? Erm, check out their web site - look at the other two columns of products besides database - they offer quite a few products including little things like Oracle ERP, Peoplesoft and Siebel.

      -Em

      --
      RelevantElephants: A Somatic WebComic...
  10. Re:so... by Matje · · Score: 5, Informative

    I wouldn't be so sure: the mysql administrator is not always able to restore a backup correctly. you are well advised to restore backups through the command line. it is also impossible (at least last time I checked) to automatically include all databases in a scheduled backup. You can only select the databases that are present at the time you schedule the backup.

    furthermore, the mysql query browser screws up when you paste a query containing tabs. this particular bug was reported to mysql over a year ago. it was somewhat fixed, but not really.

    I really like mysql but claiming that the admin is better than sql server is well, pushing it.

  11. InnoDB...? by suv4x4 · · Score: 2, Interesting

    Apparently something has changed in Google's usage of MySQL. They have this to say for their patch:

    "In a perfect world, each feature would be provided as a separate patch and all code would be as portable as MySQL. We are not there yet. These have been implemented and deployed on Linux. Also, some of these features only work with InnoDB, because we use InnoDB."

    InnoDB is the slower, but safer/transaction-able way to use MySQL. In an earlier blog entry "let's get a real database", Google revealed they run AdWords/AdSense on MySQL, and they rolled up their own transactions as they went for speed with MySQL. Now we see they changed their mind.

    So I suppose that goes to show, never mind how limited your application, you better stay away from MyISAM: it'll bite you sooner or later.

    1. Re:InnoDB...? by fruey · · Score: 2, Interesting

      I've seen a lot of people saying InnoDB is slower, but last time I converted a forum from MyISAM to InnoDB it was *faster* on reads, it was the writes that were slower...

      And, of course, it didn't crash all the time.

      --
      Conversion Rate Optimisation French / English consultant
    2. Re:InnoDB...? by suv4x4 · · Score: 4, Informative

      Umm, InnoDB is an engine for MySQL, they are still using MySQL. They just said they use the InnoDB engine exclusively rather than the other engines such as MyISAM. They probably never used MyISAM since it doesn't have foreign keys or transactions. No respected DBA would ever use MyISAM.

      Where did I say InnoDB wasn't MySQL ..? And no, they used MyISAM initially, you can check the comments after their original post.

      In fact the irony in your post is big, as the original post addresses just people like you who are very quick to decide what a "respected DBA" would do, or not (versus reality).

      I'll quote one of the Google devs answering questions about it:

      Q: holy crap, you ran adwords on a transactionless database?

      A: Yep.

      Q: you have balls. and you're lucky as hell nothing (else) went wrong.

      A: Nope. Luck had nothing to do with it. If you don't have transactions you just roll your own. It's actually not hard at all.


      Bottom line is, you use transactions as it easier to retain data integrity and makes for simpler code. You don't use transactions to be "respected DBA". Whatever reasons they had to use MyISAM was probably sound, but after certain level of sophistication, they switched to InnoDB to make things easier to manage.

  12. Nevermind - found the Wiki with more info... by xxxJonBoyxxx · · Score: 2, Informative

    Nevermind - found the Wiki with more info...
    http://code.google.com/p/google-mysql-tools/w/list

  13. Re:Wireless Developers? by fm6 · · Score: 5, Funny

    (Snicker.) In the building where I work, there are special supply cabinets for nomadic employees who use the flex offices. These have signs on them saying "These supplies are for flexible employees only!" Every time I see one, I want to add an addendum: "Inflexible employees fuck off!"

  14. Re:so... by bigman2003 · · Score: 2, Interesting

    I don't want to get all niggly...

    But I didn't claim that MySQL admin was better than SQL Server admin- I just said it was damn good.

    I have had much better success with backup and restore on MySQL than I have had on SQL Server. I find the interface and functionality much easier to understand and much more obvious. The damn DTS in SQL Server is a black hole to me...admittedly I had a database replicate in the wrong direction about 4 years ago and I am still gun-shy when using it. Yes, it was my fault, but MySQL lays things out a little more clearly.

    --
    No reason to lie.
  15. Re:so... by cdwiegand · · Score: 2, Informative

    I call shill. I used Sql Server 2K and 2K5 at my last job, and I would LOVE to have SQL Server Studio for MySql - the admin tools SUCK. The ones that use the native dll crash, and the ones that use .Net bindinds (i.e. Toad from Quest) have bugs of their own, and crash every once in awhile too. Also, none of them hold a candle to the usability of SQL Server Studio - executing part of a script, and wanting to cut and paste the result to show what I got (standard practice here) is difficult in MySql's tools (even third party ones), but SQL Server makes it easy.

    --
    . Define sqrt(x) as something really evil like (x / rand()), and bury it deep. Watch your coworkers go nuts.
  16. Re:Hep Me Understand... by bill_mcgonigle · · Score: 4, Insightful

    What good is speed without data integrity? If I can't trust my data, It doesn't matter how fast I can retrieve it.

    If you have a read-only situation there's no need for full ACID compliance. I've seen some contrivances where MySQL reads happen from myASM databases, and the writes go into an InnoDB database, and something on the backend happens to replicates the changes into the 'read-only' databases reliably. I've just never had, myself, an application so speed critical that it was worth doing that instead of doing PostgreSQL for everything. But my use cases aren't everybody's use cases.

    --
    My God, it's Full of Source!
    OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
  17. Re:so... by Architect_sasyr · · Score: 4, Interesting

    Whilst the link escapes me there is a set of admin tools for MySQL that are almost identical to the 2k5 ones for MSSQL. Truth be told it doesn't look as flash, but they are still really nice to use. The one installed on my laptop (hence not being able to remember the name) allows for highlight execution of part of script a-la the SQL Studio and cut and pastes fine, I'm sure there is more than one tool out there that can do this!

    I use both MS SQL and MySQL in corporate environments for various purposes and have to say that I like both for different reasons. MySQL is "damned good" in its own right, but there are some ANSI features available in MSSQL I really have trouble living without (for example: DELETE FROM tblData WHERE fldValue = '3' OUTPUT INTO tblBackup). My thought would be to see what Google have to offer, if it helps me, then great, if it doesn't, it's great for the community, and in a few years time when MySQL has all the features I require I can finally throw out the MS SQL servers.

    My $0.02 AU

    --
    Me failed English...
    FreeBSD over Linux. If my comments seem odd, this may explain...
  18. Better replication is a start by thsths · · Score: 5, Interesting

    Because the replication support in MySQL is very basic. A single master for the whole database, isn't that a bit limited? I had to run two database processes just because I had two data sources to replicate from.

    Unfortunately, the patches from Google don't really fix the basic problem. I think MySQL needs a complete redesign of the replication function. Oracle seems to handle this much better: the replication is controlled on the master, not the client, and it works in just about any combination you can imagine. That is not to say that the MySQL replication is not useful in certain situations (it fans better), but it is very much limited to one kind of application.

    Way to go.

  19. Re:Hep Me Understand... by _|()|\| · · Score: 2, Informative

    What good is speed without data integrity?

    In this context, data integrity refers to mechanisms that prevent you from doing something bad. If you never do anything bad, these mechanisms aren't strictly necessary. It's analogous to the difference between dynamically and statically typed program languages.

    What do I mean by "something bad"? Without referential integrity, you could have the database equivalent of dangling pointers. Without concurrency controls (either through locking or the previously mentioned MVCC), one user could accidentally blow away another's changes. (Look up isolation levels for some specific examples.) Without transactions, a failed update could leave the database in an inconsistent state (e.g., the money left your savings account, but never made it to your checking account).

  20. Re:so... by gfxguy · · Score: 4, Insightful

    Gee, thanks. What about us jack-of-all-trade schmoes who only use the basics and aren't running company wide databases? The amount of data I use the database for is actually quite small, relatively speaking. I run (among all my other jobs) our internal webserver, which means I need to write all the code, including the presentation part, and administer the database.

    Sorry, we can't all be gurus, but I still need to back up my tables every so often.

    --
    Stupid sexy Flanders.
  21. They differentiate between 9i and 10g by brokeninside · · Score: 3, Interesting
    The numbers from the Evans Data report:
    1. MS SQL - 61%
    2. mSQL - 40%
    3. MS Access - 38%
    4. Oracle 10g - 22%
    5. Oracle 9i - 20%

    The survey ``asked developers at 517 companies in its 2006 winter survey what database they developed with'' with developers allowed to give multiple answers. It isn't clear to me why Oracle was split between two versions while none of the other databases were.

  22. Re:so... by Matje · · Score: 2, Informative

    [blatant selfpromotion] if you're looking for something similar (but with much less functionality) written as a dos batch script, try this http://www.jijenik.com/projects/mysqlbackup/. It will even email you an compressed and encrypted copy of the backup file. [/blatant selfpromotion]

  23. Re:so... by chrispycreeme · · Score: 5, Funny

    Because it's a hell of a lot more fun than using a mouse trap and the mouse is just as dead.

  24. Re:Hep Me Understand... by Safety+Cap · · Score: 2, Informative

    If you never do anything bad, these mechanisms aren't strictly necessary.

    Show me one programmer who hasn't done something 'bad' (i.e., made a mistake) and I'll say, "Just let him write his second line of code," just after I pry the delete, backspace and ctrl keys off his keyboard.

    --
    Yeah, right.
  25. Re:so... by tempestdata · · Score: 5, Insightful

    Well said, not everyone can be the guru of everything. This macho geek attitude of 'If you cant do it in this super efficient, optimal way, you have no business doing it' is very detrimental to the OSS community. I am a software engineer, but I do basic admin stuff too, there are admins in my company who are obviously better than me.. but that doesn't mean that I cant take care of some of the duties too. I dont have to be supremely competent.. just competent enough.

    --
    - Tempestdata
  26. Finding what's "under the covers" by VampireByte · · Score: 2, Informative
    Also, way too much of what go's on in Enterprise Manager is undocumented. Oh, the way to do it in the GUI is documented, but under the covers it is impledmented with undocumented Transact SQL.


    You can use the SQL Server Profiler tool to capture the SQL that's run when you do something in Enterprise Manager. I've done this when I wanted to do something in my own admin tools that I know can be done in EM.

    --

    Run and catch, run and catch, the lamb is caught in the blackberry patch.

  27. Re:so... by Krazy+Nemesis · · Score: 3, Insightful

    Yeah, but the setup time and learning curve is atrocious, and you run the risk of shooting your own foot off.

  28. Re:so... by jazzkat · · Score: 5, Insightful

    BigMan, If you're coming from SQL Server, you'd be better off downloading and using Postgres 8.2.4 for windows, from here. PG is BSD licensed, which means you can bundle it with your commercial .NET-based apps for free.

    The management interface for PG is on-par with SQL Server Studio; I use both on a daily basis. It's also "20 minutes to set up and start populating data". As an added plus, Postgres has all of the "standard" syntax and referential data integrity turned on out of the box.

    You use MySQL if: a) you're developing a LAMP app for an inexpensive webhost that only allows MySQL databases, or b) all of your developers cut their teeth on MySQL and therefore productivity will drop if you ask them to use standard compliant syntax, or c) You're using an app (like SugarCRM or WordPress), the developers of which insisted on using funky MySQL-only features (instead of standard portable syntax) and therefore it's too much work to port to a standard syntax.

    In all other cases, you use Postgres or some other commercial database. Postgres scales much better than InnoDB on any combination of a) larger numbers of read-write transactions, b) larger numbers of connections, c) more processors, d) larger datasets (including and beyond 400-500GB).

    Cheers, -J

  29. Re:license incompatibility? by linhux · · Score: 2, Informative
    Actually, the announcement blog post says:

    [...] thus we have released the changes with a GPL license for the MySQL community to use and review.
    It seems to me that while their Python tools are Apache-licensed, the actual MySQL patches are GPL.
  30. Re:so... by xappax · · Score: 2, Insightful

    I dont have to be supremely competent.. just competent enough.

    I agree in general, because I'm in the same situation, being "Administrator of Computer Stuff". But come on, folks. Being an expert in mysql is one thing, but understanding how to write a shell script or use a simple command-line utility like mysqldump is pretty basic stuff that even "master of none" types like me are comfortable with.

    There is a lot of reflexive elitism and egotism among techies, but if someone doesn't have a foundation of basic "linux literacy", telling them not to meddle with important systems is often warranted. I think people who consider themselves computer-savvy can sometimes get defensive when their knowledge about a particular topic is shown to be lacking because they feel it's a reflection on their overall "geek cred". So they point out all the other technologies they're competent in as though that somehow mitigates their lack of knowledge in this one. This, too, is a kind of "macho geek attitude", in that people will refuse to admit that they're n00bs and need to spend a lot of time learning before they can use a new technology.

    When I come up against something I don't know how to do well, I spend my energy trying to learn how to do it better, not vehemently asserting why I don't need to.

  31. Re:license incompatibility? by Pollardito · · Score: 2, Informative
    that's really odd, i had remembered the article mentioning that they were released under the GPL and sure enough the first paragraph is:

    Google on Tuesday released new programming code under the GPL license to enhance MySQL databases.
    but then the actual tools code page lists :

    License: Apache License 2.0
    as for the original question, the Apache FAQ for their license says they think that they're compatible but FSF does not:

    Is the Apache license compatible with the GPL (GNU Public License)?

    It is the unofficial position of The Apache Software Foundation that the Apache license is compatible with the GPL. However, the Free Software Foundation holds a different position, although we have not been able to get them to give us categorical answers to our queries asking for details on just what aspects they consider incompatible.

    Whether to mix software covered under these two different licenses must be a determination made by those attempting such a synthesis.
  32. Re:so... by Erik+Noren · · Score: 2, Funny

    What part of fun don't you understand?