Slashdot Mirror

← Back to Stories (view on slashdot.org)

Open Source Solution Breaks World Sorting Records

Posted by Soulskill on from the out-of-sorts dept.

allenw writes "In a recent blog post, Yahoo's grid computing team announced that Apache Hadoop was used to break the current world sorting records in the annual GraySort contest. It topped the 'Gray' and 'Minute' sorts in the general purpose (Daytona) category. They sorted 1TB in 62 seconds, and 1PB in 16.25 hours. Apache Hadoop is the only open source software to ever win the competition. It also won the Terasort competition last year."

15 of 139 comments (clear)

  1. Overlords by Narpak · · Score: 3, Funny

    I for one welcome our new datasorting overlords!

    --
    The Long Now Foundation
    1. Re:Overlords by Jurily · · Score: 4, Funny

      I for one welcome our new datasorting overlords!

      With a name like Apache Hadoop, I wouldn't be surprised if they came from Star Wars.

    2. Re:Overlords by davester666 · · Score: 3, Funny

      Fastest implementation of BubbleSort EVER!

      --
      Sleep your way to a whiter smile...date a dentist!
    3. Re:Overlords by ModMeFlamebait · · Score: 5, Funny

      datasorting for I new one our overlords! welcome

      --
      Pavlov. Does this name ring a bell?
    4. Re:Overlords by Anonymous Coward · · Score: 1, Funny

      Actually, it came from Google. Sorta.

      i actually like the name 'Google Sorta' better than Apache Hadoop

  2. I'm sure that I can rock their scores by Anonymous Coward · · Score: 1, Funny

    Just give me a few minutes to patch together a bubblesort from my highschool Pascal class. I'll show them record speed!

    1. Re:I'm sure that I can rock their scores by Thinboy00 · · Score: 5, Funny

      My sort will totally beat yours!

      --
      $ make available
    2. Re:I'm sure that I can rock their scores by Midnight+Thunder · · Score: 2, Funny

      Bogosort: for when you have you are paid by the hour, but aren't penalised for being late.

      --
      Jumpstart the tartan drive.
    3. Re:I'm sure that I can rock their scores by bmajik · · Score: 3, Funny

      I've asked lots of interview candidates to implement randomSort. They've never heard of it, so then I describe the algorithm.

      Watching their eyes go wide is the highlight of the interview, typically.

      Occasionally some person who has overcome their interview nervousness will, with eager honesty, try to implore to me that this is not a very good sort algorithm, and that much better ones are taught in universities these days.

      Good Times.

      --
      My opinions are my own, and do not necessarily represent those of my employer.
    4. Re:I'm sure that I can rock their scores by Anonymous Coward · · Score: 3, Funny

      Bogosort: for when you have you are paid by the hour, but aren't penalised for being late.

      with my luck, bogosort would get it right the first time.

    5. Re:I'm sure that I can rock their scores by Anpheus · · Score: 3, Funny

      No, he clearly changed roles from developer to Evil HR. He's probably directly subservient to Catbert.

  3. Is it settled? by Jah-Wren+Ryel · · Score: 4, Funny

    So, it appears they have finally sorted out whether open source beats proprietary.

    --
    When information is power, privacy is freedom.
  4. Re:What data? by rackserverdeals · · Score: 4, Funny

    They sorted 1TB in 62 seconds, and 1PB in 16.25 hours.

    This doesn't say anything if we don't know what kind of records were supposed to be sorted.

    It's amazing what you can learn if you actually RTFA.

    All of the sort benchmarks measure the time to sort different numbers of 100 byte records.

    If that's not good enough for you, post your email address and maybe someone will be kind enough to send you the 100TB and 1PB data files they used.

    --
    Dual Opteron < $600
  5. Re:Use C++ and save 10x the hardware by Anonymous Coward · · Score: 1, Funny

    Use C++ and save 10x the hardware

    You tell em brutha! I'm so tired of carrying 10 cell phones to play java games.

  6. Re:Great! It's open source! by turbidostato · · Score: 3, Funny

    "Why isn't this illegal"

    Because they made it legal by passing it on a Totally Unrelated Bill.