Slashdot Mirror

← Back to Stories (view on slashdot.org)

How Twitter Is Moving To the Cassandra Database

Posted by kdawson on from the big-table-doesn't-capture-the-half-of-it dept.

MyNoSQL has up an interview with Ryan King on how Twitter is transitioning to the Cassandra database. Here's some detailed background on Cassandra, which aims to "bring together Dynamo's fully distributed design and Bigtable's ColumnFamily-based data model." Before settling on Cassandra, the Twitter team looked into: "...HBase, Voldemort, MongoDB, MemcacheDB, Redis, Cassandra, HyperTable, and probably some others I'm forgetting. ... We're currently moving our largest (and most painful to maintain) table — the statuses table, which contains all tweets and retweets. ... Some side notes here about importing. We were originally trying to use the BinaryMemtable interface, but we actually found it to be too fast — it would saturate the backplane of our network. We've switched back to using the Thrift interface for bulk loading (and we still have to throttle it). The whole process takes about a week now. With infinite network bandwidth we could do it in about 7 hours on our current cluster." Relatedly, an anonymous reader notes that the upcoming NoSQL Live conference, which will take place in Boston March 11th, has announced their lineup of speakers and panelists including Ryan King and folks from LinkedIn, StumbleUpon, and Rackspace.

7 of 157 comments (clear)

  1. Don't believe them! by smellsofbikes · · Score: 4, Funny
    They keep saying that the Cassandra database is better, but somehow I don't believe them. I can't imagine they know what they're talking about. Maybe in the long-term they'll be proven right but I really don't think they are. I don't know why, though...

    heh heh heh.

    --
    Nostalgia's not what it used to be.
  2. Cassandra, eh? by maugle · · Score: 4, Funny

    I hear Cassandra can even predict when disastrous system failures are going to occur! Unfortunately, for some reason nobody ever believes the warnings.

    1. Re:Cassandra, eh? by einhverfr · · Score: 2, Funny

      Especially when trojan horses are the cause of such a disaster....

      --

      LedgerSMB: Open source Accounting/ERP
  3. Re:And this is front page news, why? by BarryJacobsen · · Score: 4, Funny

    Why is it that whenever twitter makes any random change to some part of its infrastructure that we need a front page story about it?

    Because the change prevented them from posting it to twitter.

    --
    Track your TV Shows with your iPhone - FREE
  4. Don't want to install Cassandra by einhverfr · · Score: 2, Funny

    I hear Cassandra is really a trojan. Can anyone verify? I don't want a trojan on my computer.....

    --

    LedgerSMB: Open source Accounting/ERP
  5. Re:network issues? by Anonymous Coward · · Score: 1, Funny

    you just need to get bigger, faster pipes

    That's what she said!

  6. Re:Java / JVM Wins Again ... by codepunk · · Score: 3, Funny

    Until recently I thought the same way, I would never endorse a solution that involves java. However
    a recently came to the same realization that sun did when they created it. Java is a fantastic
    way to over sell gobs of expensive hardware. I am a system administrator so the more hardware it takes to
    run a solution the better off I am, more machines, more money and better job security. So I have now
    fully jumped on the java bandwagon, java makes me smile.

    --


    Got Code?