Slashdot Mirror

← Back to Stories (view on slashdot.org)

The Biggest MySQL Cluster, Ever?

Posted by Cliff on from the an-RDBMS-that-needs-a-diet dept.

ExcerLee asks: "Our team is going to build a MySQL (load-balancing and fail-over) cluster for the Cluster World Expo in the end of June. This cluster is going to have at least 10 nodes, and will use dual-opteron systems from Polywell Computers and SuSE Enterprise Linux for AMD64 from SuSE. While we are working on this hefty cluster, we wonder if this is the biggest MySQL cluster has ever been built. I googled with terms like 'biggest mysql cluster', but didn't find much." If you've run a MySQL cluster before, how large was it and how well did it perform? Krow: I have been told of much larger clusters then this; Slashdot DBs total 6 machines with 18 processors, and LiveJournal has one of the most complicated clusters I have ever seen set up.

19 of 55 comments (clear)

  1. Why would they think that? by Hanashi · · Score: 4, Insightful

    IT environments are so complex today, why would anyone think only 10 of anything is "the biggest"?

    --
    Check out my eclectic infosec blog at InfoSecPotpou
    1. Re:Why would they think that? by moosesocks · · Score: 2, Informative

      10 of these ought to be more powerful than anything else that's been built. 1060 processors, 5760gb of RAM.

      It would definitely be the world's largest MySQL cluster. Of course, I dobut that anybody would even WANT a cluster that big. Once you get a database that size, you would probably want to custom-tailor your own solution, or not use a traditional database at all. The only databases big enough to warrant this type of power are probably those of the government (ie. the IRS); that being said, the government is probably using an ancient antiquated system which works just fine for their purpose.

      --
      -- If you try to fail and succeed, which have you done? - Uli's moose
  2. Wow! Correct use of the Cliff Comma Subject! by JUSTONEMORELATTE · · Score: 4, Funny

    Mark this day! Cliff's trademark comma actually landed in an appropriate place!
    Assuming, of course, that he was trying to sound like Fat Comic Book Guy from The Simpsons.

    --

  3. One Question.... by clonebarkins · · Score: 3, Funny

    ...what are you going to put in the db?

    --

    "The evil of the world is made possible by nothing but the sanction you give it." -- Ayn Rand

    1. Re:One Question.... by PerlGuru · · Score: 3, Funny

      *everything*

    2. Re:One Question.... by clonebarkins · · Score: 3, Funny
      *everything*

      You forgot the evil "Mwahahahahahaha" after it!

      --

      "The evil of the world is made possible by nothing but the sanction you give it." -- Ayn Rand

  4. BEOWULF by Anonymous Coward · · Score: 2, Funny

    Imagine a BEOWULF cluster of these CLUSTERS.
    Someone had to say that stupid shit. :-(

    1. Re:BEOWULF by ciaran_o_riordan · · Score: 5, Interesting

      I know you were only joking but I thought I'd mention that this is impossible.

      BEOWULF support has to be compiled into an application for it to be used. MySQL doesn't have BEOWULF support, and I don't see a patch for it on the web.

      OpenMosix performs kernel level clustering. You apply the OpenMosix patch to Linux, compile, reboot, and Linux will now migrate processes automatically to other OpenMosix enabled computers on your network.

      Beowulf is more efficient, but it would probably take a month or so to program MySQL to use it. OpenMosix takes about 30 minutes to set up and it then works with every application you run.

      I thought this was interesting.

      Ciaran O'Riordan

      --
      Expert in software patents or patent law? Contribute to the ESP wiki!
    2. Re:BEOWULF by The+Bungi · · Score: 2, Funny
      You mean something called 'Beowulf' that has something to do with clusters actually exists??

      All these years...

    3. Re:BEOWULF by h3 · · Score: 3, Interesting

      OpenMosix takes about 30 minutes to set up and it then works with every application you run.

      Well, not *every* application. Some don't migrate, depending on I/O, threading, etc. But a lot do, and it really is easy to setup.

      We use it at work on a bioinformatics cluster and the best part is that it'll transparently balance user jobs without them having to do or learn anything new.

      Ob topic: we happen to run MySQL on one of our cluster nodes, but it doesn't migrate ;).

      -h3

  5. Documentation by wixu · · Score: 4, Insightful

    Are you planing to write some documentation of this work and showing it to internet?

  6. Why powerful servers? by samjam · · Score: 2, Interesting

    It would say more if you used less powerful machines and still did well.

    If you show how well it can run on stock hardware (as opposed to buying more) it may impress more people.

    That means 800MHz to 1GHz maybe?

    Just a thought.

    Sam

    --
    blog.sam.liddicott.com
  7. Re:livejournal? by aeinome · · Score: 5, Informative

    Here's an XML file detailing all of the clusters. It's a little hard to read, but it works.

    --
    When you don't have a leg to stand on, don't even get up.
  8. Wow! Amazing! by FroMan · · Score: 4, Funny

    That is truly amazing. Oh, not the cluster, that doesn't sound so impressive I guess. I really don't know, I don't work with clustered databases.

    The truly amazing part is that the postgresql folks have not arrived yet.

    --
    Norris/Palin 2012
    Fact: We deserve leaders who can kick your ass and field dress your carcass.
    1. Re:Wow! Amazing! by cperciva · · Score: 4, Funny

      The truly amazing part is that the postgresql folks have not arrived yet.

      Not at all. Postgresql is a database. This story has absolutely no connection to databases.

      --
      Tarsnap: Online backups for the truly paranoid
  9. Re:They didn't [Was: Re:Why would they think that? by Hanashi · · Score: 2, Informative
    I'm sorry, but I don't think you are correct. See the following excerpt:

    While we are working on this hefty cluster, we wonder if this is the biggest MySQL cluster has ever been built.

    They're clearly asking if theirs is the biggest ever built.

    --
    Check out my eclectic infosec blog at InfoSecPotpou
  10. mobile.de by willamowius · · Score: 2, Interesting

    mobile.de (German car marketplace) has over 100 MySQL database servers in their cluster and is performing very well.

  11. I'll bet the biggest is Google... by ubiquitin · · Score: 2, Informative

    Google has clusters of 100+ CPU's, and they run MySQL. I haven't seen any public confirmation that they are clustering MySQL, but it would make a lot of sense for them to do so. They would be smart to use a relatively flat db to store all incoming search requests for analysis.

    --
    http://tinyurl.com/4ny52
    1. Re:I'll bet the biggest is Google... by philj · · Score: 4, Informative

      How google works. Enjoy.