Slashdot Mirror

← Back to Stories (view on slashdot.org)

North America's Fastest Linux Cluster Constructed

Posted by CowboyNeal on from the where-was-the-lightning dept.

SeanAhern writes "LinuxWorld reports that 'A Linux cluster deployed at Lawrence Livermore National Laboratory and codenamed 'Thunder' yesterday delivered 19.94 teraflops of sustained performance, making it the most powerful computer in North America - and the second fastest on Earth.'" Thunder sports 4,096 Itanium 2 processors in 1,024 nodes, some big iron by any standard.

21 of 325 comments (clear)

  1. Very great and all... by irokitt · · Score: 5, Interesting

    But why did they use itanium processors? Were they acquiring parts before Opterons were availabel? Did they have a problem with Xeon processors? Or did they have too much cash lying around?

    --
    If my answers frighten you, stop asking scary questions.
    1. Re:Very great and all... by Too+Much+Noise · · Score: 2, Interesting

      Check SpecFP benchmarks - Itanium2 smokes pretty much everything else. Reason? it was meant to be a fp monster from the beginning. Integer math is weak (Opterons kick Itanium on that pretty hard), but FP math, especially vector FP math is Itanium's selling point. Why do you think the vast majority of I2 sales were to scientific research groups? (check the target profile for SGI's I2 clusters - research and defense)

    2. Re:Very great and all... by tap · · Score: 2, Interesting

      I wish those graphs were easier to read! From the looks of it, at ~32 processors the 1.5 GHz It2 has a 12 second run time and the 1.8 GHz Opteron is 24 seconds. The It2 cluster has Quadrics high end Elan4 interconnect, while the interconnect of the Opterion cluster isn't listed. It might have GigE for all we know.

      The It2 probably cost around 5 times as much as the opterons, so a real comparison would be 32 It2 processors vs 160 Opterons. With the scaling shown for that model, the Opterons of equilivent cost would be 2-3 times faster than the Itaniums.

  2. "Most" powerful by Alomex · · Score: 4, Interesting

    Look, any way you cut it the 100K computers Google is reputed to have is the most powerful Linux cluster anywhere in the world.

    1. Re:"Most" powerful by irokitt · · Score: 4, Interesting

      4,096 iTanium processors versus ~8,000 boxes sporting Pentium II, III, and 4 processors. But remember that the interest Google has is in disk access and redundacy, not complex mathematical computation. So it isn't configured as a 'supercomputer' per se.

      --
      If my answers frighten you, stop asking scary questions.
    2. Re:"Most" powerful by Boone^ · · Score: 2, Interesting
      You're right, but this still only uses an off-the-shelf interconnect from Quadrics. Quadrics bills themselves as the "price/performance leader", not the performance leader.

      There are many purpose-built supercomputers coming up (like Sandia's Red Storm) that use custom yet pricy interconnects that end up smoking anything Quadrics can put together. Anytime your interconnect relies on a PCI-type bus, you take a latency penalty on each end. Real supercomputers access memory on other nodes directly, not through a generic shared bus to a fancy network card.

      Read some of this if you're bored, it goes through Sandia's entire thought process. http://www.lanl.gov/orgs/ccn/salishan2003/pdf/camp .pdf

  3. vs google by docl · · Score: 2, Interesting

    This is probably a stupid question, but would anyone care to explain how this is different than a really large cluster. For example, if people estimate google to approach 100K nodes, how does this compare?

  4. Another Article by Flashbck · · Score: 4, Interesting

    And only 55 people were needed to build it!

  5. Re:I'd hate to be the guy... by MrRuslan · · Score: 2, Interesting

    You know what would be cool...if it would be posible to somehow recycle part of the heat from this clust into energy.perhaps tunneling all of it into one source and it boil water into steam to make energy...kidding about that last part ;)

  6. Second fastest on earth? by Stevyn · · Score: 2, Interesting

    yeah, that we know about. I remember the article on google a few weeks ago that made everyone think just what they hell they're running over there. I wouldn't be surprised if governments kept other supercomputing clusters secret. I don't mean anything tin-foil-hatish here, I'm just thinking that some governments have test facilities that they don't let the public know about.

  7. Re:apple's response will be interesting by Jeff+DeMaagd · · Score: 2, Interesting

    I love G5s, but Virginia Tech's cluster IMO can't say much until they get the G5 Xserves, because the PowerMac G5s don't have ECC memory. ECC is very important for such a large scale project that runs simulations where data is stored in RAM for any meaningful duration.

  8. Re:apple's response will be interesting by gerardrj · · Score: 2, Interesting

    There's also the difference in the interconnects, that has a lot to do with the efficiency of the system as a whole.
    Lets see what the VTech system does with ECC RAM installed when some node's aren't double-checking other node's results.

    --
    Article X: The powers not delegated... by the Constitution...are reserved...to the people
  9. Re:apple's response will be interesting by System.out.println() · · Score: 2, Interesting

    And don't forget that the current round of G5's are currently almost a year old... and long due for an upgrade. I hope some other instituion builds a 1,500 G5 2.6 GHz cluster :) (Or something to that effect.)

    --
    I've got more mod points and GMail invi
  10. Re:apple's response will be interesting by fmorgan · · Score: 2, Interesting

    I heard a presentation from VTech on why they selected the G5 over the Itanium (for scientific calculations, with lot of floating poing operations, both are faster than AMD chips; not a big problem for AMD, of course; how many of us need to simulate nuclear explosions in our desktops? well, at least until the next generation of strategy games, of course).

    At the time - this was a study done in July/Aug 2003, remember - the speed of the G5 and the Itanium2 were similar for the same clock speed (for scientific calculations; before someone flames me with something off topic, remember that this is a very particular kind of application); then what happened was that Intel was simply "out-clocked"! Kind of funny when Intel was the big champion of "clock-speed" over AMD, Motorala and IBM.

    This was in a presentation by VTech at an O'Reilly conference; coverage for this with several articles (including /. discussions) can be found here:
    http://www.macdevcenter.com/mac/osx2003/

  11. Re:Did hell freeze over? by Ralph+Wiggam · · Score: 2, Interesting

    Nuclear weapons are dinosaurs. They did their job from 1945 to 1991. Who are we going to nuke now? The North Koreans, who are proposing a peace treaty? Canada? Nukes are weapons of deterence. Osama isn't sitting in his cave thinking, "We shouldn't mess with the US, they might nuke us."

    -B

  12. Probably OT, but... by Trogre · · Score: 3, Interesting

    ... if you want a practically guided tour of LLNL, watch TRON sometime. They filmed it there (the science-lab live action stuff anyway).

    --
    "Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
  13. Of course... by MaestroSartori · · Score: 2, Interesting

    ...you realise that it isn't a linear scale. Trying to make a G5 cluster which achieved 4.8 gigaflops per processor would take more than the 4400 processors, and thus would easily take more than 300 more processors than are used for the Itanium cluster.

    300 processors. Thats 150 dual-processor boxes. I can't be bothered working it out now, but how far that goes to eliminating the power & heat advantage the G5 has would be interesting to find out...

    --
    Game dev and music blog
  14. Re:Did hell freeze over? by NEOtaku17 · · Score: 2, Interesting

    I don't know if people realize this but one of the major reasons computers exist as they do today is because of military research. Yeah building nukes was one of the driving forces behind more complex and powerful computers. Both nuclear power and computers grew up as a result of WW2. Now stop complaining about military research because you wouldn't even be typing messages on Slashdot if it weren't for the need of more advanced military technology pushing science and application of science further ahead.

    --
    Creative Demolition
  15. Re:Nah, but who knows what the NSA has cooking. by DAldredge · · Score: 2, Interesting

    The NSA has their own small scale fab to make their own custom chips. Hell, they do not every try to keep it secret and have let more than one file crew film part of it.

    They can design all the custom chips they want with their rather large budget.

  16. Your forgetting by IAmAMacOSXAddict · · Score: 2, Interesting

    It also has twice the processors, to generate the X2 times speed that they claim. Something tells me, now that VA is recieving the XServer G5 cluster nodes, that they may want to add some more units. they can put 48 units in each rack now, rather than 12 of the full size G5 Desktop form factor. According top my primitive calculations that would allow them to run 4 times as many machines in the same space (would be over 8000 CPUs. I figure that will likely kick the crap out of this new linux cluster...

    --
    MacOSX, because making *NIX better is a lot better than waiting for Micro$loth to fix Windows
  17. Re:before everyone starts shouting at once... by ajp · · Score: 2, Interesting

    Speculation: very cool. Lots of registers, yes, but the Register Stack Engine makes it feel like you never run out: there are automagically more. EPIC: bundles are actually between 3 and 5 instructions. Rotating registers: most important for passing parameters. Predicated instructions? 128 of them.

    However your post, and my post, are wasted on /. These people will always hate Itanium because it's not kewl. But they're not the ones who will be buying the processors.