Slashdot Mirror


North America's Fastest Linux Cluster Constructed

SeanAhern writes "LinuxWorld reports that 'A Linux cluster deployed at Lawrence Livermore National Laboratory and codenamed 'Thunder' yesterday delivered 19.94 teraflops of sustained performance, making it the most powerful computer in North America - and the second fastest on Earth.'" Thunder sports 4,096 Itanium 2 processors in 1,024 nodes, some big iron by any standard.

325 comments

  1. Imagine a ... by Anonymous Coward · · Score: 5, Funny
    pineapple on a monkey.

    And you thought I was going to say something else...

    1. Re:Imagine a ... by Nasarius · · Score: 1

      I dunno, where's the pineapple?

      --
      LOAD "SIG",8,1
    2. Re:Imagine a ... by Anonymous Coward · · Score: 0

      It's your imagination, so it's up to you.

    3. Re:Imagine a ... by Zordak · · Score: 1

      That is the only good "Beowulf cluster" joke I am likely to read in this thread (that includes the "We're going to see so many Beowulf cluster jokes" posts). You actually made me laugh out loud.

      --

      Today's Sesame Street was brought to you by the number e.
    4. Re:Imagine a ... by Anonymous Coward · · Score: 0

      Pleased to be of service ;-) Hope the coffee didn't spray over the keyboard!

    5. Re:Imagine a ... by Anonymous Coward · · Score: 0
      It's not a "Beowulf cluster" joke. It's the old "Pineapple on a monkey" joke. You must be a slashdot newbie...

      ;-)

    6. Re:Imagine a ... by JPriest · · Score: 5, Funny

      I see your pineapple monkey and raise you a rabbit with a pancake on its head. (cache)

      --
      Saying Java is nice because it works on all OS's is like saying that anal sex is nice because it works on all genders.
    7. Re:Imagine a ... by Anonymous Coward · · Score: 0

      You do realize that freecache ignores files less than 5 MB, right? Happy slashdotting.

    8. Re:Imagine a ... by Anonymous Coward · · Score: 0
      I'm impressed! Not many people could tell you where to find a photo of a rabbit, with a pancake on it's head, on the Internet.

      Nice to see the Internet I know and love is still out there surviving.

    9. Re:Imagine a ... by nocomment · · Score: 3, Funny

      Imagine SETI running on that!!!

      --
      /* oops I accidentally made a comment, sorry */
      /* http://allyourbasearebelongto.us */
    10. Re:Imagine a ... by Anonymous Coward · · Score: 0

      No, No, this is the pineapple on a monkey joke. You're getting it confused with the the typewriter and monkey joke.

    11. Re:Imagine a ... by Anonymous Coward · · Score: 0

      It's sort of like a million monkeys on typewriters, only they produce conga music instead of Shakespear.

    12. Re:Imagine a ... by ndogg · · Score: 1

      Oh yeah, well I raise you a monkey on a chocobo (mirror)

      --
      // file: mice.h
      #include "frickin_lasers.h"
    13. Re:Imagine a ... by ndogg · · Score: 1
      --
      // file: mice.h
      #include "frickin_lasers.h"
    14. Re:Imagine a ... by Anonymous Coward · · Score: 0

      Nah, the rabbit's still winning.

    15. Re:Imagine a ... by Anonymous Coward · · Score: 0
    16. Re:Imagine a ... by SEWilco · · Score: 1
      I'm impressed! Not many people could tell you where to find a photo of a rabbit, with a pancake on it's head, on the Internet.

      Does it count if he stumbled across the photo a year ago and bookmarked it, because he uses it every two weeks? Or maybe he took the picture a while ago and looks for suitable places for it.

      Or maybe you just don't recognize the traditional Wales agricultural ownership brand, the Welsh rare-bit.

  2. Whoa. by Anonymous Coward · · Score: 0, Offtopic

    That is indeed one badass piece of silicon.

    Now, this may sound stupid, but, who has the fastest computer in the World?

    1. Re:Whoa. by TravisWatkins · · Score: 4, Informative

      That would be the Earth Simulator in Japan.

      --

      "But I'm still right here, giving blood and keeping faith. And I'm still right here."
    2. Re:Whoa. by BiggerIsBetter · · Score: 1

      Or to put that another way...

      Japan: PWN3D!
      USA: Doh!

      --
      Forget thrust, drag, lift and weight. Airplanes fly because of money.
    3. Re:Whoa. by Trejkaz · · Score: 1

      Holy shit man, imagine a Beowulf cluster of those.

      --
      Karma: It's all a bunch of tree-huggin' hippy crap!
    4. Re:Whoa. by Anonymous Coward · · Score: 1, Insightful

      Yes, and even before the Earth Simulator was built the US was falling behind on environment and climate research, to be behind even some smaller European nations.

      Even so, the US preaches to the rest of the world how they should do and think. I think you have reached the state of "being so dumb you don't even KNOW that you are dumb".

    5. Re:Whoa. by m.h.2 · · Score: 1

      Huh?! How did this get modded up? Yeah, OK. You're smarter than us, Anonymous Eunich, er, Coward.

      If you're going to assert the superiority of your intellect, at least have the gonads to not do it anonymously.

  3. Very great and all... by irokitt · · Score: 5, Interesting

    But why did they use itanium processors? Were they acquiring parts before Opterons were availabel? Did they have a problem with Xeon processors? Or did they have too much cash lying around?

    --
    If my answers frighten you, stop asking scary questions.
    1. Re:Very great and all... by Anonymous Coward · · Score: 0

      About the only think the Itanium II is any good at is FP. Well, when you use a 6MB L2 cache variant ...

    2. Re:Very great and all... by MBCook · · Score: 5, Insightful
      I like the Opteron as much as the next guy and I'm no fan of the Itanic. But the fact is that for some types of calculations the Itanium can smoke Opterons. If you want the fastest, in many cases you want the Itanium. If you want the best value (which still performs quite close to the fastest), you want an Opteron. I don't remember which operations are better on which, so you'll have to look that up (or someone will reply with the answer).

      Depending on budget, price (I wouldn't be suprised if Intel cut them a sweet deal to get this cluster publicized to help our their product's sales), and other factors, the Itanium could have been a good choice.

      Especially if they were using software that had been designed for the Itanium (like they were replacing an older cluster) then they wouldn't have to port the software which would have saved real money.

      I'm not a fan of Intel lately, but the Itanium isn't overpriced garbage no matter what. That smacks of fanboyism. Interesting you didn't add G5s to your list, BTW.

      ALSO: Don't forget that the Itanium 2 was DESIGNED FOR big iron, while the Opteron was designed for servers and small iron. They can be used in other ways (you could run a web site off an Itanium 2), but the Itanium was designed for these kind of applications.

      --
      Comment forecast: Bits of genius surrounded by a sea of mediocrity.
    3. Re:Very great and all... by DAldredge · · Score: 1

      The Itanium 1/2 has much better compilers for science related workloads.

    4. Re:Very great and all... by Anonymous Coward · · Score: 0

      I don't know about you but the last I looked at the intel compilers they sucked major ass! I believe they probably used gcc compilers which have been optimized quite well.

    5. Re:Very great and all... by Wesley+Felter · · Score: 1

      Since when is a 4-way system "big iron"?

    6. Re:Very great and all... by tap · · Score: 5, Informative

      Do you have any kind of benchmark where the Itanium smokes the Opteron? The Itanium does have a greater memory bandwidth, but not by a lot. If you look at the spec benchmarks, it can be faster on some of them, but not by a lot. However, the Itamium is a lot more expensive!

      Compared to a Xeon or AthlonMP cluster, the Itanium faired poorly in price/performance. The only reason to use Itaniums was if you needed 64 bits for more than 4GB of memory, or needed high single CPU performance for a pooly parallized application. (Of course if your application parallizes poorly, a cluster is probably a bad choice to begin with). Then Opterion came out and changed all that. It's 64 bits, it's fast, and it's a fraction of the price of the Itanium2.

      I just purchased a new Beowulf cluster. The decision was between Xeons vs Opterons. The Opterons had better price/performance, but the Xeons would fit in better with our existing Pentium3 Beowulf, other ia32 servers, and existing software. In the end, we went with Opterons. Itanium2 was never even in contention. Just one look at the price and performce of a Itanium2 system was all it took to cross it of the list.

    7. Re:Very great and all... by evilviper · · Score: 2, Funny
      I don't remember which operations are better on which, so you'll have to look that up (or someone will reply with the answer).

      Wow! What a great argument strategy! Let me try...

      I like slashdot as much as the next guy. But the fact is that CmdrTaco is an evil blood-sucking cyborg who kills a puppy for each and every slashdot subscriber. I don't remember where I found this irrefutable proof, so you'll have to look it up yourself (or someone will reply with it).
      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
    8. Re:Very great and all... by Roydd+McWilson · · Score: 2, Informative

      GCC? On Itanium? Optimized quite well? Whatever. Check out Trimaran for the HP/Illinois/NYU compilers which basically inspired Itanium.

      --
      THE NERD IS THE COMPUTER.
    9. Re:Very great and all... by Anonymous Coward · · Score: 0

      You just missed the point. As soon as you said "price/performance", you clearly didn't get it. The grandparent made it really clear that he was talking about the *absolute* fastest.

    10. Re:Very great and all... by Yenya · · Score: 2, Informative
      The problems of Opteron against Itanium2 are:
      • You cannot order the bigger L2 cache (Itanium2 can have 6MB).
      • For "randomly branched" code you need as short pipeline as possible. This is the reason Athlon outperformed PentiumIV at the same clock speed. Now Itanium2 has 6-stage pipeline, while Opteron has 20-stage, IIRC.
      OTOH, for full performance you need _much_ finely-tuned compiler for VLIW CPUs such as Itanium2 than for a generic CISC or RISC CPU.
      --
      -Yenya
      --
      While Linux is larger than Emacs, at least Linux has the excuse that it has to be. --Linus
    11. Re:Very great and all... by medelliadegray · · Score: 0, Redundant

      "If you want the fastest, in many cases you want the Itanium. If you want the best value (which still performs quite close to the fastest), you want an Opteron."

      You make good points throughout your reply, but if you're clustering--the idea of buying the fastest available just doesnt make sense, unless underlying it really is that much faster in even a cluser environment?

      I'm guessing, as you also mentioned, that intel probably cut them a sweet deal if they used intel's flagship.

      only other option would be they thought intel would hold up better/be more stable. /shrug

      --
      Troll, Troll, go away and flame again some other day
    12. Re:Very great and all... by identity0 · · Score: 2, Informative

      I am not an expert, but in general, Opteron seems to be targeted more for the workstation/server market than the supercomputer market. It's not like they really need x86 backwards-compatibility in the supercomputer field, so Opteron doesn't seem to be optimized for that market. I think Intel may have made IA-64 with supercomputers in mind than AMD did with x86-64.

      Some reps from SGI came to my LUG the other day, and talked about their clusters and supercomputers. The guy doing the Q&A said that he personally liked the Opterons and x86-64, and that the Opterons were fast, but for what SGI does they preferred Itanium. The Opterons have their memory controller embedded in the chip itself, which is great for 1 or 2 or even 8 processors. However when you go up to a 512 processor single-system image supercomputer like SGI's Altix, a lot of the memory controller stuff is done in the switches or otherwise off-chip. Itanium allowed for more flexibility in how they did memory controllers, because they don't have an on-chip one.

      There were some other reasons too, like having more registers, etc. that made SGI choose Itanium over Opteron. I don't know how applicable they are to this situation, as this doesn't seem to be a SSI supercomputer.

    13. Re:Very great and all... by Anonymous Coward · · Score: 0

      Yes, here is just one (of many)

      NWChem Results

      Itanium does very VERY well on computational chemistry codes, to mention one area.. (OK, the one I work in ;)

    14. Re:Very great and all... by fupeg · · Score: 3, Informative

      Try any from SPEC, for example. Maybe you're thinking about x86 because otherwise, the Itanium2 is way out of the Opteron's league (as well as price range, but that is besides the point.)

    15. Re:Very great and all... by Too+Much+Noise · · Score: 2, Interesting

      Check SpecFP benchmarks - Itanium2 smokes pretty much everything else. Reason? it was meant to be a fp monster from the beginning. Integer math is weak (Opterons kick Itanium on that pretty hard), but FP math, especially vector FP math is Itanium's selling point. Why do you think the vast majority of I2 sales were to scientific research groups? (check the target profile for SGI's I2 clusters - research and defense)

    16. Re:Very great and all... by tap · · Score: 4, Informative

      Ok, checked them again. The best 1.5 GHz Itanium2 SPECfp2000 score is 2148 while the opteron 248 is 1691. That's 27% faster. I'd hardly call that smoked.

      The Opteron 248 is $670 on pricewatch, while the 1.5 GHz It2 is $5200! The motherboards are like $1400 vs $400.

      You have to keep in mind that this isn't a single machine, it's a cluster. You could take the money spent on an Itanium2 cluster, and buy an opteron cluster with five times as many processors. I am well aware that one does not get perfect scaling. But if you are running something on a cluster in the first place, I have a hard time imagining something that is faster with one fifth as many 27% faster processors. Yes, there are codes that would be faster on 1000 Itanium2 vs 5000 Opterons, but you would never runs these on cluster, because they would be faster still shared memory system.

    17. Re:Very great and all... by Xrikcus · · Score: 1

      And of course the other problem is that randomly branched code often has short code segments between branches and less compile time instruction scheduling, so less use of the 6 ins/cycle static pipeline in the i2. Whether that balances out against the long opteron pipeline and all the pipeline flushes is a different question though.

      The i2 also detects branch prediction misses fairly efficiently, so that helps.

    18. Re:Very great and all... by Anonymous Coward · · Score: 0

      Yes, there are codes that would be faster on 1000 Itanium2 vs 5000 Opterons, but you would never runs these on cluster, because they would be faster still shared memory system.

      This machine basically is a shared memory system: OK, it isn't, but the performance is the same. Remember your Athlon or Pentium 3? It probably had a memory bandwidth of around 1GB/sec.

      Guess what the bandwidth between any two nodes of Thunder is? You guessed it, slightly under 1GB/sec. QsNet II is not _quite_ as fast as SGI's NUMALink 4 (which they use to make their shared memory Altix systems but it's in the same ballpark.

    19. Re:Very great and all... by tap · · Score: 2, Interesting

      I wish those graphs were easier to read! From the looks of it, at ~32 processors the 1.5 GHz It2 has a 12 second run time and the 1.8 GHz Opteron is 24 seconds. The It2 cluster has Quadrics high end Elan4 interconnect, while the interconnect of the Opterion cluster isn't listed. It might have GigE for all we know.

      The It2 probably cost around 5 times as much as the opterons, so a real comparison would be 32 It2 processors vs 160 Opterons. With the scaling shown for that model, the Opterons of equilivent cost would be 2-3 times faster than the Itaniums.

    20. Re:Very great and all... by boots@work · · Score: 3, Insightful

      You make good points throughout your reply, but if you're clustering--the idea of buying the fastest available just doesnt make sense, unless underlying it really is that much faster in even a cluser envrironment?

      That sentence doesn't even parse, but anyhow: single-thread performance still matters to clusters. There is a limit to how much you can effectively parallelize many problems. If that limit is 1, then you need a Cray or something. If the limit is extremely high, you can use distributed.net, or a cluster of recycled C64s.

      In the middle, you might be able to parallelize the task to a limited extent. If you can only split your work into 500 parallel tasks, then you want 500 of the fastest processors you can get. For many applications, that means 500 Itaniums. Even if you could buy 800 Opterons for the money, they might not be as fast.

      only other option would be they thought intel would hold up better/be more stable. /shrug

      Itanium has slightly better manageability; you can find out when a memory module or CPU is likely to fail for example. There is a heap of error detection/correction in the CPU, far beyond Xeon or Opteron afaik. If you have hundreds of machines being able to easily detect failures is worth something.

      (Or you can just take the google route and let it fail and replace the whole box. But that really requires your whole application to be written to accomodate it.)

    21. Re:Very great and all... by Anonymous Coward · · Score: 0

      That's 27% faster. I'd hardly call that smoked.

      What DO you call 'smoked'? You don't see huge differences between modern processors.

      The Opteron 248 is $670 on pricewatch, while the 1.5 GHz It2 is $5200!

      Apples and Oranges. The 248 is a workstation and 1U server chip. Check the Opeteron 8xx -- it's the Big Iron chip. As an AMD Fanboy, you should know that.

    22. Re:Very great and all... by tap · · Score: 2, Insightful

      I don't know if I would call NUMALink a true shared memory system. It is NUMA after all! I was thinking of 32-64 way machines with a true shared memory system, or large vector machines based on SX-6 processors for example.

      But look at NUMALink4, its got 6.4 GB/sec per link bandwith and 240ns latency.

      QsNetII is just under 1 GB/sec bandwidth, the limit of PCI-X, with a latency of 3us.

      So, NUMALink4 has 6.4 times the badwidth and 12.5 times less latency than QsNetII. That a much larger performance difference than Opteron vs Itanium!

    23. Re:Very great and all... by Toraz+Chryx · · Score: 1

      Okay then, an Opteron 848 is $1299 (according to pricewatch anyway)

      can't find a quad Opteron mainboard price though, the Tyan K8QS looks like it's the business, but I can't find anywhere in my limited searching that's actually selling them.

    24. Re:Very great and all... by tap · · Score: 1

      How is it apples and oranges? Why would I build a cluster with four way opterons when I could get a faster cluster using two way? If there was a $670 Itanium2 processor for clusers, I'd use that. But there isn't, and you have to look at what exists. The fact is, Itanium just isn't very impressive for a cluster. If you think Itanium is so great, show me some numbers. Calling people names just shows you don't have any facts to back up your argument.

      Are you saying that clusters built with Opteron 248 chips don't work, because they're not "Big Iron" chips? That's absurd.

      All this business about what is "big iron" and what is workstation, what is designed for floating point performance, what has an optimized instruction set, and so on is irrelevant. It all comes down to GFLOPS, dollars, watts, and floor space. All the rest is just meaningless marketing buzzwords of interest to no one but fan boys with an agenda that isn't supported by real numbers.

    25. Re:Very great and all... by Anonymous Coward · · Score: 0

      You're so smart. You should get a job at Lawrence Livermore and tell them how to really build clusters. But instead, you flaily impotently at ACs.

    26. Re:Very great and all... by joib · · Score: 3, Informative


      There is a limit to how much you can effectively parallelize many problems. If that limit is 1, then you need a Cray or something.


      Well, Crays are also parallel computers, so they won't help you much in this situation. Some Crays do have vector processors, but that is also a sort of parallelism. It's just that you use that parallelism through tuned BLAS libraries or with a vectorizing compiler (e.g. Fortran 95, HPF and such things), instead of doing it manually with MPI or threads or something like that. So if you're problem is totally serial, a vector processor won't help you either.


      (Or you can just take the google route and let it fail and replace the whole box. But that really requires your whole application to be written to accomodate it.)


      Not necessarily. Most supercomputers are not used to run a single job taking months, but rather they run lots of smaller and shorter jobs. On the p690 cluster where I do my stuff, I (and apparently most users) mostly run jobs using about 8-16 cpu:s , with a runtime of a few hours to a day. If one node would fail, the jobs that are executing on that node would also fail. It's no big deal, just resubmit the job to the queue when you get around to it.

      Of course, if you're programming one of the very few and far between applications that has a runtime of months, you certainly want to save intermediate results once in a while. Not only to guard against hardware failure, but also so that the user can check the intermediate result and see if the app is still on the right track. It would be quite a bummer to use months of cpu time only to realize the entire thing is wasted because you specified the initial values wrong.. :-)

    27. Re:Very great and all... by boots@work · · Score: 1

      Well, Crays are also parallel computers

      I think in their heyday they were the fastest straight-line processors. I don't know what holds that record now. It may well be I2, for some tasks.

      The other stuff you described, including checkpointing, falls under my category of writing your app to expect failures.

    28. Re:Very great and all... by Anonymous Coward · · Score: 0

      You forgot to factor in the cost of electricity, airconditioning, space and maintanence for the extra 4000 boxes!

    29. Re:Very great and all... by cyb97 · · Score: 1

      OP didn't mention gcc, and I'm pretty sure (s)he thought about Intels own compilers. They usually smoke GCC when it comes to optimizations... not surprisingly

    30. Re:Very great and all... by mczak · · Score: 1
      How is it apples and oranges? Why would I build a cluster with four way opterons when I could get a faster cluster using two way? If there was a $670 Itanium2 processor for clusers, I'd use that. But there isn't, and you have to look at what exists.
      Actually, there is an Itanium2 "for clusters" (ok, intel says it's for workstations). Itanium2 DP is in fact much cheaper - the only difference to the Itanium2 MP is it has less L3 cache (only 1.5MB vs. the 1.4Ghz Itanium2 MP's 4MB), so its SpecFP score will no longer look that impressive ;-)). And it can only be used in dual systems (not sure if this is really true, but it's advertized at least that way).
      Are you saying that clusters built with Opteron 248 chips don't work, because they're not "Big Iron" chips? That's absurd.
      They used quad-cpu nodes for their clusters for some reason probably (only half the nodes, more easily manageable, less space...). So it's probably fair to assume that if they would have used Opterons, they would have wanted quad-cpu nodes too.
      Of course, Opteron 848 vs. Itanium2 MP is still much cheaper (as is Opteron 248 vs Itanium2 DP).
    31. Re:Very great and all... by SuperQ · · Score: 4, Insightful

      the problem is not that you couldn't get the processors, the problem is scale.

      A system like this will use a high-speed interconnect, not gige. The popular choice right now is infiniband, and that stuff isn't cheap, and also has limits to the number of ports per IB switch. The system at LLNL has 4 procs per node, which reduces the number of IB switches involved. 5000 dual proc (you suggest 248 proc) machines would require 2500 IB ports, instead of 1024.

      now if you considered the opteron 848 ($1300), in 8proc nodes, that would be something to think about, reduce the number of IB ports in half, and be able to double the processors.

      the other consideration is also processor scale. the 27% per CPU is signifigant, because even with dual proc SMP, you loose some % of the CPU time. There was a posting on an article about how processors scale this way. I forget how the principle works.

    32. Re:Very great and all... by Anonymous Coward · · Score: 0
      Ok, checked them again. The best 1.5 GHz Itanium2 SPECfp2000 score is 2148 while the opteron 248 is 1691. That's 27% faster. I'd hardly call that smoked

      The Opteron 248 has a clock speed of 2.2 GHz compared to the Itanium2 at 1.5 GHz. That is almost a 50% clock speed advantage yet only delivers 73% of the performance. I definitely call that getting smoked.

    33. Re:Very great and all... by Anonymous Coward · · Score: 0

      The best 1.5 GHz Itanium2 SPECfp2000 score is 2148 while the opteron 248 is 1691. That's 27% faster. I'd hardly call that smoked.

      SPECfp is general compiled code. Both the Top500 benchmark and a lot of scientific applications rely solely on matrix multiplications. The Itanium2 has two floating-point units, where each unit can complete a Fused-multiply add (FMA) instruction per cycle. For matrix code you get incredibly close to the theoretical peak, meaning a single ia64 chip performs at 6Gflops.

      The Opteron and Xeon don't have FMA instructions, which means they achieve much lower performance on this particular application (peaks out around 3.5Gflops). As a sidenote, this is alos the reason the G5 cluster at Virginia tech scored quite well - for general code the difference is much smaller.

    34. Re:Very great and all... by megarich · · Score: 1

      You're right about that but unfortunately itanium1 is. My company was too cheap to buy me a new machine so they gave me this itanium1. While it is fast, I can't upgrade past redhat 7.2 which flakes out on me alot. Pretty much I'm stuck in itanic hell.

    35. Re:Very great and all... by tap · · Score: 1

      I wouldn't say Infiniband is the popular choice right now. Other than VT's G5 cluster, there really isn't anyone else using it. For Beowulf interconnects, the most popular seems to be Myrinet for smaller systems and Quadrics for the largest systems. LLNL used Quadrics' QsNetII for their cluster, and you can get the price list for it. A 1024 node interconnect was just over $4 million, about $4k per node.

      The thing is, if you look at the price difference of an Opteron system vs Itanium2 system, and even when you factor in a pricey interconnect like QsNetII in addition to the smaller per-node costs like cases, power supplies, etc., the Opteron still comes out ahead by a large margin.

    36. Re:Very great and all... by tap · · Score: 1
      SPECfp is general compiled code. Both the Top500 benchmark and a lot of scientific applications rely solely on matrix multiplications.

      Ok, let's look at top500 and see what kind of LINPACK performance you get.

      LANL has a 2816 processor Opteron 2GHz cluster, with Myrinet, with a Rmax of 8051.

      PNNL has a 1936 processors Itanium 2 1.5Ghz cluster, with Quadrics, with an Rmax of 8633.

      Per processor, that's 2.85 GFLOPS for the opteron vs 4.45 for the Itanium2. Scale the Opteron 246 to a 248 and you should get 3.135 GFLOPs. That puts the Itanium2 at 42% faster, per CPU.

      Now look at the cost. Those Opteron nodes cost about 1/4 as much as the Itanium2 nodes. For half the money, you could buy an Opteron cluster twice the size, and get more overall performance. That's what matters in the end, what total performance will you get for your budget.

    37. Re:Very great and all... by Performer+Guy · · Score: 1

      NUMA can be true shared. Especially ccNUMA. You can read from any memory and write to any memory, it can all be addressable from any node. Routing is done transparently through the infrastructure and can be kept low latency. It is true shared memory but I suppose you may be familiar with other implementations that aren't.

    38. Re:Very great and all... by Noren · · Score: 1
      Not necessarily. Most supercomputers are not used to run a single job taking months, but rather they run lots of smaller and shorter jobs. On the p690 cluster where I do my stuff, I (and apparently most users) mostly run jobs using about 8-16 cpu:s , with a runtime of a few hours to a day. If one node would fail, the jobs that are executing on that node would also fail. It's no big deal, just resubmit the job to the queue when you get around to it.

      I expect it'll fall somewhere between the two extremes. On the linux cluster of Itanium2 processors as similar as it gets to this new one (mpp2), there are 1794 processors currently running jobs-

      one 600 processor job
      three 256 processor jobs
      one 120 processor job
      ... and 16 more jobs ranging from 2 to 40 processors totaling the remaining 306 processors.

      The largest job currently in the queue requests 900 processors for 24 hours. (it's rare for people to request more than 72 hours for a single job)

      So, it's true that the majority of the jobs running are small (40 processors or less)... but three quarters of the processors are doing jobs of at least 256 processors. Most of the clock cycles do go to larger jobs, but it's far from a single job using the whole machine for months limit. Some of the few-processor jobs are run on mpp2 because of unusually large memory/disk requirements which make them impossible or inefficient to run on smaller, cheaper clusters; many also are test runs for larger jobs. (Most people running on mpp2 also have access to more typical smaller clusters and use those for easier calculations.)

    39. Re:Very great and all... by Noren · · Score: 1
      Well, looking at the top500 list-

      #5 is a 1936 x 1.5 GHz Itanium2 processors linux cluster with an Rmax of 8633
      #6 is a 2816 x 2.0 GHz Opteron processors linux cluster with an Rmax of 8051

      It's just one benchmark which depends in part on other aspects of the systems, but it seems to me to be as good a one as any. At least in this case the numbers are published and are comparable.

      Like a lot of other decisions, it really all comes down to what you want to use it for. For some jobs Itanium2 is enough better than the competition that it is cost-efficient, for others the choice should be Opteron (or Cray, or Xeon, or G5, or a truly custom system...) The top 6 supercomputers on the top500 list each use different processors at the moment!

    40. Re:Very great and all... by gbulmash · · Score: 1
      First, let's assume that all overhead being equal as a percentage of performance, that the base equipment costs (processors, MoBos, controllers, interconnects, RAM) would come out to a 4:1 price advantage for Opteron.

      Assuming the 16,384 Opteron processors have a speed of "1", and the 4096 Itaniums are 27% faster or "1.27", the Opteron system would have a speed of 16384 vs. a speed of 5202 for the Itanium system.

      Seems like apples to apples... but 4x as many processors, motherboards, RAM, power supplies, etc. are going to generate a LOT more heat, requiring more cooling equipment and more electricity to power it. Add to that the added space requirements. Add to that all the additional electricity the greater number of systems will use. Last, recoding from IA64 to X86-64 is another cost.

      It's never apples to apples. I'd love to champion AMD against Intel, and I have an AMD FX-51, but I can't make it fit any purpose. Itanium is good for big iron and will own that for a while yet... IMO.

    41. Re:Very great and all... by Anonymous Coward · · Score: 0

      while the 1.5 GHz It2 is $5200! The motherboards are like $1400 vs $400.

      I'm sure when they purchased 4096 processors, they got a bulk discount.

    42. Re:Very great and all... by Anonymous Coward · · Score: 0

      Then you're a fool who believes the megahertz myth. Look, it doesn't matter HOW you get the speed, it just matters that you have it.

      Now look at the price. Hmm, you have to spend more than 5x the price to get that Itanium2 chip than you do to get the opteron chip, and you only get 27% more power out of it. I'd say the Opteron smokes it for price/performance.

    43. Re:Very great and all... by joib · · Score: 1

      Hmm, that looks about how I would assume a supercomputer would be used.

      If only...

      \begin{rant_mode}
      I think the reason we have a very different usage pattern over here is that the cluster is configured so that us normal users are not allowed to submit jobs using more than one node. Thus we are limited to a maximum of 32 cpu:s (1 node). On top of that, a backfill scheduler is used, meaning that smaller and shorter jobs can jump the queue. In practice this means that a 8-16 cpu job can usually start right away during non-office hours or with only a few hours waiting time. By comparison, when I once submitted a 32 cpu job, I got tired of waiting after it had been in the queue for a week..

      Now, it makes me wonder why the supercomputer center blew shitloads of tax-payer money on that fancy federation/colony or whatever its called switch, when noone is allowed to use it.. :(
      \end{rant_mode}

      I'm not saying the p690 is a bad computer, in fact it's about 4 times faster than the Cray T3E it replaced. I just think that for the same money they could have gotten a much faster computer. Like an Itanium2/Quadrics cluster, for instance (they already have an Origin for shared memory jobs, so IMHO MPI performance should have been their top priority).

    44. Re:Very great and all... by Anonymous Coward · · Score: 0

      And I would say that you're a fucking idiot that either doesn't understand the megahertz myth or my post. The megahertz myth argues that clock speed is not the best determinant of real CPU speed and has been used against Intel for having pushed highly clocked chips that can't beat a much lower clocked AMD part, regardless of price.

      Now that an argument has been made that Intel's 64bit chip smokes AMD's 64bit chip on a clock-for-clock basis, you suddenly want to get pissy and change the argument to price/performance.

      Why don't you try figuring out what the argument is before you post on it. Better yet, just suck it AMD fanboi.

    45. Re:Very great and all... by Roydd+McWilson · · Score: 1
      OP didn't mention gcc

      Uh, yes they did.

      --
      THE NERD IS THE COMPUTER.
    46. Re:Very great and all... by 4of12 · · Score: 1

      If you want the fastest, in many cases you want the Itanium.

      I would have wanted whatever would have been the current chip in the Alpha line if it had received even half the development money that the Itanic did.

      --
      "Provided by the management for your protection."
  4. I'd hate to be the guy... by krammit · · Score: 3, Funny

    ...who gets the electric bill.

    I cringe when I leave the A/C on for too long..

    --
    "Watch your cornhole, bud."
    1. Re:I'd hate to be the guy... by Anonymous Coward · · Score: 2, Insightful

      Pay taxes? Then you are the guy who gets the bill.

      That said, I think our national labs are pretty great when they aren't designing nukes.

    2. Re:I'd hate to be the guy... by MrRuslan · · Score: 2, Interesting

      You know what would be cool...if it would be posible to somehow recycle part of the heat from this clust into energy.perhaps tunneling all of it into one source and it boil water into steam to make energy...kidding about that last part ;)

    3. Re:I'd hate to be the guy... by Forge · · Score: 1

      Actualy that's per CPU
      $2,863,104 total.

      --
      --= Isn't it surprising how badly I spell ?
    4. Re:I'd hate to be the guy... by Anonymous Coward · · Score: 0

      Yeah.. presumably this machine probably uses something like a megawatt of electricity.

      But there are much bigger users out there. Fermilab takes around 40-50 MW. Some industrial sites use considerably more than that, even. Some aluminum processing facilities require hundreds of megawatts of power.

    5. Re:I'd hate to be the guy... by Anonymous Coward · · Score: 0

      Actually co-generation plants are a great way to use waste heat.

      The problem is that they usually need to be designed as one-offs for a particular building and heat load, so the extra upfront expense is rarely taken to reduce operating costs.

    6. Re:I'd hate to be the guy... by ozzmosis · · Score: 1

      if you live in the USA you and I pay the electric bill =x

    7. Re:I'd hate to be the guy... by Anonymous Coward · · Score: 0

      You've never left me on in your life....

  5. "Most" powerful by Alomex · · Score: 4, Interesting

    Look, any way you cut it the 100K computers Google is reputed to have is the most powerful Linux cluster anywhere in the world.

    1. Re:"Most" powerful by 0xC0FFEE · · Score: 5, Insightful

      If google's cluster is interconnected via ethernet, there is a whole range of computational problems it can't tackle. If you want to simulate a spatial phenomenon with lot of things going back and forth in a volume, you're bound to have a _lot_ of communications. The cost of the interconnect system in those simulation systems is often a substantial proportion of the total cost of the installation.

    2. Re:"Most" powerful by irokitt · · Score: 4, Interesting

      4,096 iTanium processors versus ~8,000 boxes sporting Pentium II, III, and 4 processors. But remember that the interest Google has is in disk access and redundacy, not complex mathematical computation. So it isn't configured as a 'supercomputer' per se.

      --
      If my answers frighten you, stop asking scary questions.
    3. Re:"Most" powerful by smitty45 · · Score: 4, Insightful

      Powerful = fastest computation, not biggest. A roomfull of Chevettes do not make a Corvette.

    4. Re:"Most" powerful by Anonymous Coward · · Score: 1, Insightful
      Bad example. Considering they are for transportation I am sure the roomfull of Chevettes would actually get more done in less time.

      If I was going to move my stuff to the other side of the country, how many trips would that take me in a Corvette?

    5. Re:"Most" powerful by Boone^ · · Score: 2, Interesting
      You're right, but this still only uses an off-the-shelf interconnect from Quadrics. Quadrics bills themselves as the "price/performance leader", not the performance leader.

      There are many purpose-built supercomputers coming up (like Sandia's Red Storm) that use custom yet pricy interconnects that end up smoking anything Quadrics can put together. Anytime your interconnect relies on a PCI-type bus, you take a latency penalty on each end. Real supercomputers access memory on other nodes directly, not through a generic shared bus to a fancy network card.

      Read some of this if you're bored, it goes through Sandia's entire thought process. http://www.lanl.gov/orgs/ccn/salishan2003/pdf/camp .pdf

    6. Re:"Most" powerful by Rosyna · · Score: 1

      I want to know which test reported 19.94. Certainly doesn't seem to mentioned anywhere.

    7. Re:"Most" powerful by Glonoinha · · Score: 1

      I think the guesstimate for Google was more like 80,000 boxes sporting P2/P3/P4 processors. That's an order of magnitude difference. The Ethernet backplane may cause some latency issues for some types of problems but remember what it currently processes every minute of every day right now.

      --
      Glonoinha the MebiByte Slayer
    8. Re:"Most" powerful by Anonymous Coward · · Score: 0
    9. Re:"Most" powerful by tap · · Score: 5, Informative

      I think you've got that backwards, Quadrics is the performance leading, not the price/performance leader. Myrinet, SCI, and Infiniband all beat it in price/performance. Quadrics is faster, and scales to more nodes than the others.

      According to Quadrics latest price list, the cards are $1200 each, $913 per port for a 64 node switch, and $185-$265 for a cable. That's $2300/node.

      Myrinet cards are $595, the switch is $400 per port for 64 nodes, and the cables are ~$50. That's $1050/node.

      Quadric's price for a 1024 node interconnect is $4,176,094. That's hardly chump change. The bandwith is about 10x higher than gigabit ethernet, and the latency about 100x lower.

    10. Re:"Most" powerful by lightray · · Score: 1

      Powerful = fastest computation

      Mod parent up! Now let's get out the amp meters.. let's compare these computers kilowatt to kilowatt.

      Then we can be quite literal about which computer has "more horsepower," as one kilowatt is about 1.34 horsepower. (-:

    11. Re:"Most" powerful by irokitt · · Score: 1

      Missed that when I previewed, I didn't mean to drop the zero...

      --
      If my answers frighten you, stop asking scary questions.
    12. Re:"Most" powerful by Wolfier · · Score: 1

      > A roomfull of Chevettes do not make a Corvette.

      How many Chevettes can you fit in an everyday room? bedroom/washroom/living room?

      Probably 2 or less and most likely 0.

      A better analogy would be "A football field of Chevettes...". ;)

    13. Re:"Most" powerful by spectre_240sx · · Score: 1

      Off topic I know, and I apologize... but this post just got me wondering how often one of those 80k boxes goes down. I wonder what maintainance is like over in googles datacenter.

    14. Re:"Most" powerful by Nikker · · Score: 1

      How do you rate computers? by how much they can pull?

      All joking aside these computers have been pulled together in such a tight system that they can make almost 90% effiency. All that it takes once they get to 100% is money to throw at it.

      Oh ya and if a room full of chevettes were cabled up and were set to tow your corvette it would get draged along way before they felt the drag ;)

      --
      A loop, by its nature, continues. If that didn't make sense, start reading this sentence again.
    15. Re:"Most" powerful by smitty45 · · Score: 1

      the point of the analogy is that google has more machines that suck individually much worse than each node in the LLNL cluster. google is hardly powerful when compared to computation. it *is* powerful when you're talking about redundancy, because even tho they have thousands of shit boxes, if they lose some (and they do, many per day) the site still can be covered.

    16. Re:"Most" powerful by tesmako · · Score: 1

      Actually not true, Google solves the probably largest matrix problem in the world once a month. To compute page-rank they solve a 4,285,199,774 x 4,285,199,774 connectivity matrix through power-iteration. As it happens it is a very well-known and easy to distribute problem, but it is a huge math-problem none the less.

    17. Re:"Most" powerful by Anonymous Coward · · Score: 0

      I read somewhere that if one of the computers dies, they don't do anything, just leave it sit on the racks, since it happens often enough that employing enough people to find each individual box in the warehouses and figure out what hardware needs replacing and all is more expensive than just adding a few newer computers.

    18. Re:"Most" powerful by boots@work · · Score: 1

      Dozens or more go down per day, according to Google sysadmins. But they have many spares already powered up and ready to take over the work. The applications are written to spread around pretty transparently.

      At one point, I hear, they didn't even bother pulling out dead machines. They just left them in the rack until the whole rack was replaced. That seems... creepy, or something. I'd hate to have dead coworkers just left in their cubes...

      As another poster wrote, this is only suitable for certain applications, and Google have to write their apps to fit on it.

    19. Re:"Most" powerful by Anonymous Coward · · Score: 0

      Yup. And as far as we're talking specialized astrophysics or particle physics hardware, let's also not forget the 64-Teraflop (!) GRAPE-6 supercomputer, which might also be considered to be one of the fastest machines in the world.

      http://grape.astron.s.u-tokyo.ac.jp/~makino/grap e6 .html

    20. Re:"Most" powerful by Anonymous Coward · · Score: 0

      "If google's cluster is interconnected via ethernet, there is a whole range of computational problems it can't tackle. If you want to simulate a spatial phenomenon with lot of things going back and forth in a volume, you're bound to have a _lot_ of communications."

      Luckily, that's not going to be a problem that Google have to solve just yet.

      And if google did want to do scientific processing, wouldn't they be more likely to write a program and ask people to run it at home? Google screensaver and all that?

    21. Re:"Most" powerful by Anonymous Coward · · Score: 0

      But a huge pile of sand grains near water do make a beach.

  6. how fast is it? by chickenrob · · Score: 4, Funny

    Is it fast enough to run all the latest spyware, adware, and viruses and not slow down your solitaire game?

    --
    People say my sig is the best thing about me.
    1. Re:how fast is it? by Man+in+Spandex · · Score: 1, Funny

      its linux only you insensitive clod!

    2. Re:how fast is it? by Anonymous Coward · · Score: 1, Funny

      i hear it will be able to run longhorn decently

    3. Re:how fast is it? by chickenrob · · Score: 1

      useing vmware naturally....

      --
      People say my sig is the best thing about me.
    4. Re:how fast is it? by Anonymous Coward · · Score: 0

      Is it fast enough to run all the latest spyware, adware, and viruses and not slow down your solitaire game?

      I know this is totally offtopic, but ... the funny thing about your statement is that my grandpa (81 years old) uses his computer for exactly three things: email, web browsing, and Solitaire. Darn thing keeps getting hit by spyware and viruses, and how does he notice? Solitaire doesn't run well, and he gets popup ads over it.

      So I can totally relate. :-)

      (Incidentally, I can't just "switch him to Firefox and Thunderbird" like I'd like to; I tried and he got all confused because the icons are different, even with the most IE-like/OE-like themes I could find. So the software can't be changed. He even gets confused by antivirus software popup messages, thinking THEY are viruses....)

    5. Re:how fast is it? by MrRuslan · · Score: 1

      maybe in wine because windows can't take that much abuse no matter what you run it on I can bet anything on that.

    6. Re:how fast is it? by chickenrob · · Score: 1

      >He even gets confused by antivirus software popup messages, thinking THEY are viruses.... Who can tell anymore.. "warning your computer is not protected" -r

      --
      People say my sig is the best thing about me.
  7. Awesome! by haxeh · · Score: 2, Funny

    That's amazing!

    Now we can... uhh... what are we supposed to do with that much power again?

    1. Re:Awesome! by MrRuslan · · Score: 5, Funny

      It's all for reserved for Doom III on longhorn.

    2. Re:Awesome! by shaitand · · Score: 1

      Obviously we must use it for something constructive, like calculating the next few hundred billion digits of pie or processing random white noise in space on the extremely unlikely chance we'll be listening to the exact right piece of the sky at the exact right moment and recognize the completely alien transmission as being something other than noise.

      Nah, seriously though, we'll use it to support the largest growing industry. That industry which powers and drives the human imagination and recognizes our greatest achievements. Obviously this is going to be used as a streaming pr0n server.

    3. Re:Awesome! by GNUALMAFUERTE · · Score: 1, Funny

      There are only 3 things you can make with such a cluster:

      a) Play Duke Nukem Forever

      b) Install LongHorn

      c) Benchmark it and talk about the incredible performance you get using the aformentioned games :)

      --
      WTF am I doing replying to an AC at 5 A.M on a Friday night?
    4. Re:Awesome! by haxeh · · Score: 1

      mmm, pie...

      seriously though, it should be able to generate its own porn, complete with a good plot, with that much power

    5. Re:Awesome! by System.out.println() · · Score: 1

      You finally have the ability to really imagine a Beowulf cluster of those things. Rather than telling all the Slashdotters to imagine it for you!

    6. Re:Awesome! by spectre_240sx · · Score: 1

      Porn with a good plot?! Jeesus, you're asking for another ice age!

    7. Re:Awesome! by physick · · Score: 1

      You could simulate the membrane of one human cell, 20 microns in diameter, plus the enclosed water for 2 nanoseconds. Oh, and it would take 7 years.

      Just the membrane, not the DNA, not the proteins, just the lipids and water. It takes an Intel Xeon 40 cpu-days to simulate 2 microseconds of a patch of membrane 50 nm wide. Scale this up to 20 microns and it would take 1000 nodes almost 7 cpu-YEARS to simulate the cell membrane for 2 nanoseconds; and in that time each molecules would only have moved a fraction of its own length.

      That's a very, very expensive simulation of mostly water.

    8. Re:Awesome! by Anonymous Coward · · Score: 0

      Of course it is..

      Doom is the only reason computers are worth bothering with these days.

    9. Re:Awesome! by Anonymous Coward · · Score: 0

      Do what we do every day Pinky... Try to take over the world.

  8. but but but by Anonymous Coward · · Score: 4, Funny

    Can it run Windows?

    1. Re:but but but by phisheadrew · · Score: 1

      I hear it just barely missed the recommended Longhorn Specs. But once you turn of some of the extra graphical crap it should run alright.

  9. Re:Just imagine by Anonymous Coward · · Score: 0

    You missed "All your base..."

  10. Re:Just imagine by kgayer · · Score: 1

    3)????? Not everyting my friend..

    --
    2 + 2 = 5. Big Brother's watching you. bonglord.com
  11. Did hell freeze over? by SuperBanana · · Score: 4, Funny

    LLNL built a supercomputer, and it's going to do things besides simulate nuclear weapons?

    Quick, someone ring Satan and ask how the sno-cones are.

    1. Re:Did hell freeze over? by cyfer2000 · · Score: 1

      maybe super powerful laser weapon, consume more power than the computer.

      --
      There is a spark in every single flame bait point.
    2. Re:Did hell freeze over? by geek · · Score: 4, Insightful

      I grew up in Livermore, the lab was some 500 yards from my bedroom window. They work on a lot more than nuke simulations, including alternate fuels (my brother in law was driving a hydrogen fuel car from the lab 10 years ago as a test), laser technology and about a million other things. Why is it people like you who hear "Nuke" rant on and on like biased little children and post inflamatory things like this?

      The lab is a GOOD thing damnit. Do you even know what nukes are? What nuclear research has done for us? Grow up man.

    3. Re:Did hell freeze over? by nate+nice · · Score: 1

      That had to be one of the passionate posts I've ever seen. Cheers, man.

      --
      "If you are a dreamer, a wisher, a liar, A hope-er, a pray-er, a magic bean buyer ..."
    4. Re:Did hell freeze over? by Ralph+Wiggam · · Score: 2, Interesting

      Nuclear weapons are dinosaurs. They did their job from 1945 to 1991. Who are we going to nuke now? The North Koreans, who are proposing a peace treaty? Canada? Nukes are weapons of deterence. Osama isn't sitting in his cave thinking, "We shouldn't mess with the US, they might nuke us."

      -B

    5. Re:Did hell freeze over? by Anonymous Coward · · Score: 0

      I grew up in Livermore, the lab was some 500 yards from my bedroom window.

      Sure, but how far away was it from the kitchen window?

    6. Re:Did hell freeze over? by geek · · Score: 1

      Which is why we haven't made nukes in 10 years and in fact have been REDUCING THE FUCKING ARESNAL IN A TREATY WITH RUSSIA. God you anti-nuke retards are so behind the fucking times. Get over it already.

    7. Re:Did hell freeze over? by psychokid · · Score: 1

      LLNL and LBL are part of UC. They do alot of bio research there. If I'm correct, the majority isn't even related to nuclear research at both labs.

    8. Re:Did hell freeze over? by NEOtaku17 · · Score: 2, Interesting

      I don't know if people realize this but one of the major reasons computers exist as they do today is because of military research. Yeah building nukes was one of the driving forces behind more complex and powerful computers. Both nuclear power and computers grew up as a result of WW2. Now stop complaining about military research because you wouldn't even be typing messages on Slashdot if it weren't for the need of more advanced military technology pushing science and application of science further ahead.

    9. Re:Did hell freeze over? by Anonymous Coward · · Score: 0

      He says Tux the Hell-penguin is having a great time.

    10. Re:Did hell freeze over? by Anonymous Coward · · Score: 0

      Livermore also used to be a camp that was condemed to make the lab. It was a very neo-military place designed to craft young boys into hard men. In an interesting note it was the first place that William S. Burroughs was buggered.

    11. Re:Did hell freeze over? by F34nor · · Score: 1

      They guy's name is Ralph Wiggam for Christ sakes, what do expect from him?

    12. Re:Did hell freeze over? by Anonymous Coward · · Score: 0

      Um, geek(5680), why should we believe you? What makes you think you have any shred of credibility anyway?

      "The new treaty does not significantly alter the number of existing nuclear delivery systems and therefore only marginally affects the residual nuclear potential of the United States and Russia,"observed Jack Mendelsohn, a former member of the U.S. Strategic Arms Limitations Talks and START negotiating teams.

      http://www.armscontrol.org/aca/sortmay02.asp

    13. Re:Did hell freeze over? by Saeed+al-Sahaf · · Score: 1
      the lab was some 500 yards from my bedroom window

      No, the fence was 500 yards from your window. On the other hand, had you been associated with China Lake, you would have lived smack dab in the God-damn middle of it.

      --
      "Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
    14. Re:Did hell freeze over? by geek · · Score: 1

      No, the LAB was 500 yards from my window. My house was on Arlene way, my backyard WAS the fence. If you don't know the geography then don't comment on this. I lived there 20 years.

  12. Google Cache by nadolph · · Score: 2, Informative

    http://www.google.ca/search?sourceid=navclient&ie= UTF-8&oe=UTF-8&q=cache:http%3A%2F%2Fwww%2Ellnl%2Eg ov%2Flinux%2Fthunder%2F

    --
    With the moo and the cow and the fish. Minesweeper Record: 7 sec
  13. I don't care what anyone says by MrRuslan · · Score: 5, Funny

    this thing should do doom 3 with a software renderer at a very playable 47 FPS...

  14. vs google by docl · · Score: 2, Interesting

    This is probably a stupid question, but would anyone care to explain how this is different than a really large cluster. For example, if people estimate google to approach 100K nodes, how does this compare?

    1. Re:vs google by complete+loony · · Score: 4, Informative
      Google have lots of little (in comparison only) jobs that have to process heaps of data, googles cluster(s) wouldn't perform well in the top 500 list since they don't concentrate on link speed, which is the main factor in performace for supercomputers, but on raw data processing power.

      The GFS article that appeared a while back said they used standard 100MBit ethernet, this is not going to get you a good score in any supercomputer benchmark.

      --
      09F91102 no, 455FE104 nope, F190A1E8 uh-uh, 7A5F8A09 that's not it, C87294CE no. Ah! 452F6E403CDF10714E41DFAA257D313F.
    2. Re:vs google by Anonymous Coward · · Score: 0

      The difference is that Google has 20,000? machines that are handling maybe a million different requests per second. These computers are all working on different parts of the same (usually one) program. Google returns results in a fraction of a second. The problems they feed these computers are usually intentionally limited to run for a maximum of two weeks. It's not that they stop the program early, they wait for the results, but usually initially limit the problem to run for an estimated two weeks (or less). It's not that the machine is bad/broken/slow, it's just that the problem is very big and very hard.

    3. Re:vs google by unixbugs · · Score: 1

      100k nodes doesnt mean anything if half of them were part of the original DARPA project.

      --
      You are about to give someone a piece of your mind, something which you can ill afford...
  15. Finally... by Fry-kun · · Score: 2, Funny

    ...I can back up my brain

    --
    Did you know that "FTW" ("for the win") is a direct translation of "Sieg Heil"?
  16. Re:Imagine a Beowulf cluster of..... by Anonymous Coward · · Score: 0

    What's with the weather? It's been so DRY lately.

  17. The way I see it... by blackula · · Score: 2, Redundant

    ...there are basically three type of clusters: 1) shared nothing: in this, each computer is only connected to each other via simple IP network. no disks are shared. each machine serves part of data. these cluster doesn't work reliably when you have to aggregations. e.g. if one of the machine fails and you try to to "avg()" and if the data is spread across machines, the query would fail, since one of the machine is not available. most enterprise apps cannot work in this config without degradation. e.g. IBM study showed that 2 node cluster is slower and less reliable than 1 node system when running SAP IBM on windows and unix and MS uses this type of clustering (also called federated database approach or shared nothing approach). 2) shared disk between two computers: in this case, there are multiple machines and multiple disks. each disk is atleast connected to two computers. if one of the computer fails, other takes over. no mainstream database uses this mode, but it is used by hp-nonstop. still, each machine serves up part of the data and hence standard enterprise apps like SAP etc cannot take clustering advantage without lot of modification. 3) shared everything: in this, each disk is connected to all the machines in the cluster. any number of machines can fail and yet the system would keep running as long as atleast one machine is up. this is used by Oracle. all the machine sees all the data. standard apps like SAP etc can be run in this kind of configs with minor modification or no modification at all. this method is also used by IBM in their mainframe database (which outsells their windows and unix database by huge margine). most enterprise apps are deployed in this type of cluster configuration. the approach one is simpler from hardware point of view. also, for database kernel writers, this is the easiest to implement. however, the user would need to break up data judiciously and spread acros s machines. also adding a node and removing a node will require re-partitioning of data. mostly only custom apps which are fully aware of your partitioning etc will be able to take advantage. it is also easy to make it scale for simple custom app and so most of TPC-C benchmarks are published in this configuration. approach 3 requires special shared disk system. the database implementation is very complex. the kernel writers have to worry about two computers simultaneously accessing disks or overwriting each others data etc. this is the thing that Oracle is pushing across all platforms and IBM is pushing for its mainframes. approach 2 is similar to approach 1 except that it adds redundancy and hence is more reliable. so what type are we talking about here?

    1. Re:The way I see it... by twoslice · · Score: 3, Funny
      ever

      hear

      of

      Paragraph

      tags?

      --

      From excellent karma to terible karma with a single +5 funny post...
    2. Re:The way I see it... by Anonymous Coward · · Score: 1, Informative

      Next time you cut and paste from someone else's blog, perhaps take the time to restore the paragraph tags? Smells like plagurism to me...

    3. Re:The way I see it... by sapbasisnerd · · Score: 2, Informative
      Had to decide to reply to this or mod it down, decided to reply.

      That's a wildly inaccurate summary of the landscape of RDBMS clustering technology.

      Problem is, that's not what we are talking about here.

      So the answer to your question at this end is almost certainly "none of the above" or probably more correctly "some bits of all of the above". Functionally most of the kind of stuff you do here doesn't need shared concurrent access to the same data files however for simplicity of implementation they probably nevertheless run GPFS so that all nodes can see all files.

    4. Re:The way I see it... by skdffff · · Score: 2, Informative

      There are basically two types of clusters - HA (High Availability) and HPC (High Performance Computing). They both called "clusters" (what confuses some people) but designed for completely different purposes. You're talking about variations of first type while cluster in the article is HPC cluster.

    5. Re:The way I see it... by Anonymous Coward · · Score: 0
      Congratulations, douche bag. You've single-handedly convinced me to return to meta-moderating.

      What I don't get is why you'd plaigarise this to begin with? To get karma? Are you one of those trolls who tries to get karma so he can then burn it with stupid comments? Or, perhaps even stupider, are you one of those idiots who thinks karma is a measure of his personal self-worth, and feels better about himself when he gets high moderations for poster utter crap?

      Seriously, man. Seek help.

    6. Re:The way I see it... by I_am_the_man · · Score: 1

      What is even more sad is that I could sware the the original is more or less taken from some keynote speeches Larry Ellison did when Oracle RAC first came out. The part that does not seem like it was taken from Ellison was the mention that everybody uses #3 for TPC-C benchmarking. This is not the case. They actually either use Shared Nothing (which will give you big time clustered results) or they go unclustered (which is the benchmark that is closest to a realworld setup and is what people actually consider these days; albeit it is not exactly real world either). Other than that, every major point was given by Ellison to the best of my memory.

    7. Re:The way I see it... by Sykil · · Score: 1

      Well, if you want to get all semantics about it, it's and ordered list, rather than a paragraph. ;)

  18. Another Article by Flashbck · · Score: 4, Interesting

    And only 55 people were needed to build it!

    1. Re:Another Article by Anonymous Coward · · Score: 0
      And only 55 people were needed to build it!
      Why's that? 1 person to turn on the switch, and the other 54 people to plug in the power cord?
  19. Re:longhorn by MrRuslan · · Score: 1

    Betya this thing could run longhorn in bochs faster than you will on real hardware :P

  20. Clarification by Anonymous Coward · · Score: 2, Funny

    19.94 teraflops??

    Gimmy something I can grasp; what's this in BogoMips?

    1. Re:Clarification by Anonymous Coward · · Score: 0

      Roughly 8,584,617 (only counting the compute nodes)

      (no, that figure isn't made up)

    2. Re:Clarification by MrIrwin · · Score: 1

      Overflow

      --

      And if you thought that was boring you obviously havn't read my Journal ;-)

  21. 2nd fastest supercomputer by m1kesm1th · · Score: 5, Funny

    Also in completely unrelated news, Bill Gates announced the first fully installed test of Longhorn happened today.

  22. OK, here goes... by rco3 · · Score: 3, Funny

    Hey, with a Beowulf cluster of these, I can run Longhorn!

    OK, I'm done. Sorry. Mod away!

    --

    Ce n'est pas un vrai mouvement de robot!
  23. apple's response will be interesting by Twid · · Score: 4, Insightful

    If I calculate right, they are claiming an Rmax of 19.94 teraflops with 4096 processors.

    The Virginia Tech cluster for Apple had an Rmax of 10.28 teraflops with 2200 processors.

    So, the Itaninum 2 delivered 4.8 gigaflops per processor, the G5 delivered 4.6 gigaflops per processor.

    This seems like a pretty poor showing for Itanium 2, overall. It's a much hotter chip than the Opteron or the G5, so cooling and power costs are likely much higher than a comparable apple cluster. The Xserve G5 is also likely cheaper than a similarly equipped Itanium 2 server, given that the Itanium 2 is $1398 per chip on Pricewatch, and a dual processor Xserve G5 cluster node is $2,999 list. Even with 4 cpus in a single box, I think the Itanium 2 server would easily top $6,000.

    But anyway, good game to Lawrence Livermore. I'll be curious to see if Apple has another volley to fire before the top500 list closes for this round.

    --
    - "When you want something with all your heart, the entire universe conspires to give it to you" -Paulo Coelho
    1. Re:apple's response will be interesting by DAldredge · · Score: 1

      One doesn't buy from vendors listed on pricewatch for systems like this. I am sure that they got a large volume/make intel look good discount.

    2. Re:apple's response will be interesting by Jeff+DeMaagd · · Score: 2, Interesting

      I love G5s, but Virginia Tech's cluster IMO can't say much until they get the G5 Xserves, because the PowerMac G5s don't have ECC memory. ECC is very important for such a large scale project that runs simulations where data is stored in RAM for any meaningful duration.

    3. Re:apple's response will be interesting by gerardrj · · Score: 2, Interesting

      There's also the difference in the interconnects, that has a lot to do with the efficiency of the system as a whole.
      Lets see what the VTech system does with ECC RAM installed when some node's aren't double-checking other node's results.

      --
      Article X: The powers not delegated... by the Constitution...are reserved...to the people
    4. Re:apple's response will be interesting by System.out.println() · · Score: 2, Interesting

      And don't forget that the current round of G5's are currently almost a year old... and long due for an upgrade. I hope some other instituion builds a 1,500 G5 2.6 GHz cluster :) (Or something to that effect.)

    5. Re:apple's response will be interesting by Divine_Entity · · Score: 0

      Well, Apple doesn't make processors Intel does, Apple just puts them in a pretty package and gives you software to go with it. So what I beleieve you mean is won't " IBM's response be interesting " since they make the G5 and sell it to apple

    6. Re:apple's response will be interesting by System.out.println() · · Score: 1

      They haven't got the XServes installed yet? I know they were selling their old G5 PM's as a special refurb sort of deal.

      As for "can't say much"... I think that having the world's third fastest supercomputer for a fraction of the cost of its peers says plenty.

    7. Re:apple's response will be interesting by Twid · · Score: 1

      I checked California Digital's site. Those servers are 4U behemoths. The Xserves are 1U. So the Xserves actually take up half the space (2U for 4 processors versus 4U for 4 processors) for roughly similar performance.

      Like I said, I'm surprised the Itanium 2's performance was so low, given that it's a newer architecture than the PowerPC 970.

      --
      - "When you want something with all your heart, the entire universe conspires to give it to you" -Paulo Coelho
    8. Re:apple's response will be interesting by fmorgan · · Score: 2, Interesting

      I heard a presentation from VTech on why they selected the G5 over the Itanium (for scientific calculations, with lot of floating poing operations, both are faster than AMD chips; not a big problem for AMD, of course; how many of us need to simulate nuclear explosions in our desktops? well, at least until the next generation of strategy games, of course).

      At the time - this was a study done in July/Aug 2003, remember - the speed of the G5 and the Itanium2 were similar for the same clock speed (for scientific calculations; before someone flames me with something off topic, remember that this is a very particular kind of application); then what happened was that Intel was simply "out-clocked"! Kind of funny when Intel was the big champion of "clock-speed" over AMD, Motorala and IBM.

      This was in a presentation by VTech at an O'Reilly conference; coverage for this with several articles (including /. discussions) can be found here:
      http://www.macdevcenter.com/mac/osx2003/

    9. Re:apple's response will be interesting by prockcore · · Score: 4, Insightful

      This seems like a pretty poor showing for Itanium 2, overall.

      It does? You know that clustered computing doesn't scale linearly. If virginia tech were to double the amount of processors used, they wouldn't double their performance.

    10. Re:apple's response will be interesting by fmorgan · · Score: 1

      That's true. But there is also a matter of $$/performance. And in here the G5s are the cheapest.
      Check the prices for a Dell with 2 Itanium2. Not exactly "Wal-Mart PC" prices.

    11. Re:apple's response will be interesting by sapbasisnerd · · Score: 1

      And if density really matters and you're running Linux then you go with IBM JS20 blades. If I'm reading the specs right you can get 14 2Way 1.6Ghz 970 blades in 7U (twice the density of Xserve, similar list price for the blades themselves but one must still buy the Bladecenter chassis so the IBM solution would be a little more expensive).

    12. Re:apple's response will be interesting by adpowers · · Score: 1

      Yeah. Also remember that Steve-o said we would have 3 GHz G5s by this time next year. That was said at WWDC and the next one is coming up in a month and a half (end of June). The second revision should also fix any design problems that have been discovered. With the G5 at 3 GHz and Intel dropping the P4 for the Pentium M, Apple will soon have some of the fastest clocked chips around.

    13. Re:apple's response will be interesting by Anonymous Coward · · Score: 4, Insightful

      Actually, there's more to it than that. Virginia Tech's machine only gets ~55% of its peak performance, whereas Thunder gets 87%. Given that Thunder has twice as many processors, that's an EXCELLENT showing. Remember, the actual work that's going to run on Thunder won't scale anywhere near as well as the easily scaled LINPACK benchmark, so the performance gap between "benchmark" and "real world" will only get wider in practice.

      Thunder is an absolutely remarkable machine.

    14. Re:apple's response will be interesting by Anonymous Coward · · Score: 0

      Did someone say something about 1U servers?

      Here you go!

    15. Re:apple's response will be interesting by the+quick+brown+fox · · Score: 1
      Mod parent up.

      87% does seem surprisingly high. For some reason I thought only vector machines like Earth Simulator and Cray could get that close to theoretical peak performance, while clustered microprocessors languished at 75% and below.

      But now that I think about it, wouldn't the ratio be driven primarily by link bandwidth, not processor architecture?

    16. Re:apple's response will be interesting by Too+Much+Noise · · Score: 1

      Actually, he shas a point. The G5 setup was only useful for short jobs due to the non-ECC memory - any longer ones would have had to be rerun to check the results for consistency. There goes your computing power when you have to do the same thing at least twice to make sure it's ok. So it's 3rd fastest for peak power, but sustained power including the redundancy was quite lower.

      If you don't believe it, calculate the failure rate for the installed RAM they had - or look it up in one of their interviews, the guys actually understood it pretty well. Why do you think they switched to XServe? just to spend more money?

    17. Re:apple's response will be interesting by bhima · · Score: 1
      Actually this *is* the most interesting comparison!

      . What is the cost, volume, and energy dissipation of an equivalent (in performance) IBM JS20 blade setup

      --
      Nothing in the world is more dangerous than sincere ignorance and conscientious stupidity.
    18. Re:apple's response will be interesting by Anonymous Coward · · Score: 0

      Thanks for the mod request. Yes, 87% is very high.

      Well, processor architecture does come into it in a number of ways. First of all, LINPACK is all about matrix multiplication, and the most efficient way to do that on a cache-based architecture is to tile it up into little chunks. Itanium offers four advantages here:

      - big cache (in this case, 8 times larger, per CPU, than the G5/PPC970)
      - fast cache (an L2 hit on Itanium is only 5 cycles: this is close to the performance of the *L1* cache on Opteron and PowerPC chips, moreover, an L1 hit on Itanium is "perfect" - just a single cycle.)
      - architectural support for counted loops and software pipelining (by this I mean ia64 instructions like 'br.cloop') - what this means is that you can perform loops of any size with basically *zero* overhead, which is quite amazing when you first see it, but basically it's realising that the IA64 is such a wide, inherently parallel architecture (that's the EP in EPIC: explicitly parallel) that it has the brainpower to manage loop control at the very same time it is doing something else (in LINPACK's case, lots and lots of multiplication and addition)
      - IA64 offers a fused multiply+add instruction, which is nice not only because it makes the instructions smaller (saves I-cache etc), and not only because they run at the same time, but also because it increases accuracy. An ia64 'fma' instruction is equivalent to performing both the multiply and addition to _infinite_ precision, then applying a _single_ floating point rounding error. The math behind proving this is quite cool, and not that trivial.

      If you want to learn more, you should check out the book "Scientific Computing on Itanium-based Systems" by Cornea, Harrison and Tang - I'm sure you can find it on amazon easily enough.

    19. Re:apple's response will be interesting by Anonymous Coward · · Score: 1, Insightful

      I love how you missed the contradiction in your own (short) post!

      In the blue corner...

      I know they were selling their old G5 PM's as a special refurb sort of deal.

      and in the red corner..

      I think that having the world's third fastest supercomputer

      *dingdingding!*

      They don't have anything right now - you are quite right, they are selling their old G5s, but the new machine is nowhere to be seen. Does this sound like value for money to you?

      The sooner you realise that VT have been basically used by Apple as an advertising exercise, the better.

    20. Re:apple's response will be interesting by boots@work · · Score: 1

      mod parent up!

      I just got one of the rx1600 machines, and they are seriously cute.

      Itanium2 processors with no CPU fans. After seeing the Intel Lion box with about 20 fans I would never have believed it.

    21. Re:apple's response will be interesting by System.out.println() · · Score: 1

      As much as I want it to happen, I doubt we're going to see 3GHz G5's at WWDC. Faster than the current ones, certainly, but I think 3 is too much, too suddenly.

      IBM has had manufacturing problems aplenty and I certainly am not trying to call our buddy Steve a liar, but he made promises that IBM turned out to be unable to keep.... probably.

      Personally I expect 3GHz around Septemberish.

  24. Rejoicing at Intel by Animats · · Score: 4, Funny

    "We sold the Inaniums! We sold the Inaniums!"

  25. Re:Imagine a beowulf cluster of beowulf jokes by wjwlsn · · Score: 0

    Okay... imagine a Beowulf cluster of these! Actually, here it is.

    --
    Getting tired of Slashdot... moving to Usenet comp.misc for a while.
  26. Second fastest on earth? by Stevyn · · Score: 2, Interesting

    yeah, that we know about. I remember the article on google a few weeks ago that made everyone think just what they hell they're running over there. I wouldn't be surprised if governments kept other supercomputing clusters secret. I don't mean anything tin-foil-hatish here, I'm just thinking that some governments have test facilities that they don't let the public know about.

    1. Re:Second fastest on earth? by BCW2 · · Score: 1

      Who but the Japanese or the British have the technology? All US national labs have them, + CIA, NSA, DOD, and several other Govt agencies. My guess is the US Govt has at least 35 supercomputers spread out all over the place. Thats a bunch of power. Most is used in research and weapons design. NSA for code making and breaking. They probably have several more on order.

      Treasury could use one to figure out the tax code, nothing else has worked.

      --
      Professional Politicians are not the solution, they ARE the problem.
  27. Re:It's all about sticking it to the mac. by smitty45 · · Score: 1

    x86_64 will be the arch of choice, and this proves it.

  28. Heat by Rick+Zeman · · Score: 3, Funny

    4,096 Itanium 2 processors in 1,024 nodes

    So THAT'S what's causing our heat wave!

  29. Wow by 0xC0FFEE · · Score: 2, Informative

    Here's a picture: http://doc.quadrics.com/quadrics/QuadricsHome.nsf/ DisplayPages/3A912204F260613680256DD9005122C7

    1. Re:Wow by Anonymous Coward · · Score: 1, Funny

      These are not the nodes we are looking for...

    2. Re:Wow by Anonymous Coward · · Score: 0

      Umm...yeah - that would be Los Alamos National Laboratory in New Mexico. We want Lawrence Livermore National Laboratory. Anyway, the LLNL site has pictures of their cluster that work fine. No need to go to the networking company's site for pictures, man.

  30. Yes, but does it run Linux? by Anonymous Coward · · Score: 0

    Oh... right.

  31. Take that Apple! by callipygian-showsyst · · Score: 2, Funny

    Now you can't say you have the fastest "Thupercomputer" any more! You've been beat by Intel and Linux!

    1. Re:Take that Apple! by stang7423 · · Score: 1

      Are you looking at my headgear....

    2. Re:Take that Apple! by Anonymous Coward · · Score: 0

      It used to be impossible to make fun of Apple users here without getting modded to -100. I'm glad things are improving!

    3. Re:Take that Apple! by Anonymous Coward · · Score: 0

      Don't worry. The evening shift of mods usually come in and mod down Apple comments. Happened to me just a few weeks ago. I think those worthless editors are doing it, personally.

    4. Re:Take that Apple! by Anonymous Coward · · Score: 0

      Yeah I know it's a troll, but what the hell.

      Apple never had the fastest and never claimed to. They claimed to have the best price/performance ratio of the power players, and they still do. The only thing that's changed is the third-place position, which presumably (since this one got second) is going to drop a place.

    5. Re:Take that Apple! by Anonymous Coward · · Score: 0

      And with 1000 more years of basement hackers that Linux supercomputer might even be able to run a desktop similar to OS X!

      OH THE GLORY!

    6. Re:Take that Apple! by Anonymous Coward · · Score: 0
  32. Re:It's all about sticking it to the mac. by MJOverkill · · Score: 1

    Itanium's don't run X86_64, they run IA_64.

  33. Re:It's all about sticking it to the mac. by Anonymous Coward · · Score: 0

    God damn you are stupid. That is all.

  34. Fanboy showing ignorance by Anonymous Coward · · Score: 0

    "How about a real os,"
    Hmmm.. FreeBSD 4.4 base with the best gui isn't real?

    "I can't get any nic laying in a box to work on a mac"
    This sounds like a user error to me. Almost every piece of hardware I plugin is automatically picked up.

    you keep referring to x86 when this article is not based on IA32, but Itanium IA64.

    "But chances are I can buy *cheap* video editing hardware and *cheap* good soundcards that work sweet with these things."
    The sound cards in Macs are already top notch as is core audio - just look in any sound studio worth their salt.

    I like linux as much as the next guy, but don't shout off about things you aren't intelligent about.

  35. I really hope by Lord+Kano · · Score: 3, Funny

    that they didn't build this just to win 2 grand from distributed.net.

    LK

    --
    "Hi. This is my friend, Jack Shit, and you don't know him." - Lord Kano
  36. Let me guess.... by Hamster+Lover · · Score: 0, Offtopic

    Microsoft counters that their new Windows Server farm is 25% faster and has 50% less TCO.

  37. And whats so wrong with AMD? by Imidazole · · Score: 0

    And whats so wrong with AMD?

  38. before everyone starts shouting at once... by painehope · · Score: 4, Insightful

    yes, they're hot as hell and eat power the way oprah eats twinkies, and yes Intel has made a poor handling of the Itanium line, but the Itanium architecture is very interesting, and is actually very appropriate for a HPC environment. Not the part of the HPC market that clusters dominate, but the segment that Cray, SGI, HP Alphaservers, etc. have traditionally dominated. The segment that doesn't give a shit about cooling, power consumption, or price-performance, but who just need to get the job done as quickly as possible.

    Some of the coolest features of the Itanium are also some of the reasons why a lot of people don't want to use it. The EPIC ISA, for example. It was designed ( along w/ the physical hardware ) to expose a lot of the internal workings of the processor to the user. But rather than recompile and re-optimize their code, people would rather bitch about migration. That's fine for workstations and servers, but in an HPC environment, you want the nifty features, you want to occasionally hand-tune code segments in assembler, etc.

    Anyways, I'm not a fanboy ( well, maybe an AMD and MIPS fanboy ), just wanted to get in a few honest points before everyone started shooting holes in the Itanic.

    --
    PC moderators can suck my White pierced, tattooed dick. If you think pride == hate, s/dick/Aryan meat mallet/g.
    1. Re:before everyone starts shouting at once... by slamb · · Score: 5, Informative
      Some of the coolest features of the Itanium are also some of the reasons why a lot of people don't want to use it. The EPIC ISA, for example. It was designed ( along w/ the physical hardware ) to expose a lot of the internal workings of the processor to the user. But rather than recompile and re-optimize their code, people would rather bitch about migration. That's fine for workstations and servers, but in an HPC environment, you want the nifty features, you want to occasionally hand-tune code segments in assembler, etc.

      I just coded some IA-64 assembly and from what I've seen, this comment is dead-on. They've got a lot of interesting features:

      • Speculation. The idea is to do memory fetches far in advantage to avoid waiting for the (much slower) memory system. You can do a LD.S operation that tells the machine something like "I might want the value from this memory address in a few instructions." It fetches it from memory, if it's in a good mood. If the address is paged out, it doesn't get it. (Instead, it sets a NaT (not a thing) bit to tell you nothing useful is there.) Later, you do a CHK.S. If it turns out that the speculative load fails, it jumps to some "recovery" code which gets it for real.
      • Lots of registers. 128 general-purpose 64-bit registers. Floating point registers. Some specialized ones, I think.
      • EPIC. (Explicitly Parallel Instruction Computing.) It has different types of instructions, aimed at different execution units. In the current incarnation, there are two sets of these in each processor. You give it bundles of three instructions, more broadly divided into groups. Instructions in a group don't depend on any earlier results calculated by the group, so they can be executed in parallel.
      • Rotating registers. This lets you make different iterations of the same loop work with different registers, to take advantage of EPIC more fully.
      • Predicated instructions. There are a bunch (16? 64? don't remember) of predicate bits, set by the CMP instruction and the like. Every instruction has an associated predicate. (p0 is hardcoded to true, so you normally don't notice.) So you can do conditional execution without jumping. More efficient, especially if it's just a few instructions that differ.

      If you just have a simple sequence of operations, each dependant on the one before, you can't really take advantage of these capabilities. (My code was like this. Even though performance wasn't my reason for writing assembly, it was a little disappointing that I couldn't play with the new toys.) If you're expecting these features to make Word start faster, you'll probably be disappointed.

      But if you're doing intensive computations in a tight loop, you can do amazing things. If you can get all the execution units working simultaneously, it will fly. And the features like rotating registers are designed to make that possible. You need a very good compiler or a very smart person to hand-tune it. You may need to recompile to tune if your memory latency changes (affecting how many iterations to run at once) or they come out with a new chip with more sets of execution units. But in a situation like this, none of that is a problem. They'll have applications designed to run as fast as possible on this machine. They may never be run anywhere else.

    2. Re:before everyone starts shouting at once... by Anonymous Coward · · Score: 0

      Hmm, I thought most of those companies gave quite a few shits about cooling, power consumption, and price-performance.

      It's just that you almost need your own power plant to drive it no matter how efficient the design for that size of machine and the cooling system has to be pretty hardcore. As for price/performance, Linpack isn't too helpful. People who merely read the top 500 get a pretty skewed perception of the relative merits of an architecture.

    3. Re:before everyone starts shouting at once... by customjake · · Score: 1

      I can't think of anyone, or any organisation that doesn't care about price-performance. I positive that if a solution that offered the same performance came around at 1/3 the cost, you'd be willing to take the cheaper of the two. And if you're serious about getting it done now, just buy 3 times the stuff.

      I think the Itanium line is interesting, and provides decent performance, but the cost, heat and energy consumptions make it a bear to work with.

      i'd be interested in seeing how much they paid for each one of those servers and compare it to Big Mac. You might very well be able to buy 2 big mac's for the price their monster. Which for those ppl counting, would be twice the performance.

    4. Re:before everyone starts shouting at once... by joib · · Score: 1

      I'm somewhat familiar with supercomputing, and my impression is that the only place where you'll find assembler is in the (usually vendor-supplied) BLAS libraries (and perhaps in some of the tuned user-space MPI libraries too). The applications are all in Fortran (and C in some cases).

      Given that supercomputer architectures change every few years, nobody has the time to rewrite their apps all the time.

    5. Re:before everyone starts shouting at once... by painehope · · Score: 1

      Of course people care about price-performance. But there are a good deal of problems that cheaper architectures can't solve. Opteron has changed that a good deal, but nonetheless, until Cray comes out w/ Red Storm or whatever they are calling it, there's noone selling > 8-way Opteron machines, and IIRC none of those have > 16 GB of main memory.

      Bear in mind that some problems scale well on a cluster. Some sorts of signal processing ( such as the kind done in seismic processing ) are embarassingly parallel, as the saying goes. Other things, like crash simulations, whether simulations, etc., require either a very expensive interconnect ( what, you think Quadrics, Infiniband, Myrinet, SCI, all of those are cheap? ) w/ shared memory, or a large SMP machine. Those are the architectures that Itanium is aimed at. They weren't aimed at competing w/ Xeon, Athlon, Opteron clusters. They were aimed at competing w/ the POWER architecture, SGI, PA-RISC, Alphaservers, large Sparc machines, etc. Monsters that have 64-512 GB of main memory, 64 CPUs, etc.

      So, if given the choice between a 5 million USD machine that can't solve the problem as quickly as it needs to be done, and a 15 million USD machine that can, which do you think a lot of people will pick? Not that this deployment is one of those situations ( it's a 4-way cluster environment ), but you can also bet your ass that Intel gave them one sweet deal.

      And for the person that commented about not hand-tuning applications, bite me. Find out where your app spends most of it's time, which will generally be a loop somewhere, compile down to assembler, look at the code, and figure out how to optimize it. More likely than not, you can optimize it w/ 50 lines of assembler, if even that much.

      --
      PC moderators can suck my White pierced, tattooed dick. If you think pride == hate, s/dick/Aryan meat mallet/g.
    6. Re:before everyone starts shouting at once... by ajp · · Score: 2, Interesting

      Speculation: very cool. Lots of registers, yes, but the Register Stack Engine makes it feel like you never run out: there are automagically more. EPIC: bundles are actually between 3 and 5 instructions. Rotating registers: most important for passing parameters. Predicated instructions? 128 of them.

      However your post, and my post, are wasted on /. These people will always hate Itanium because it's not kewl. But they're not the ones who will be buying the processors.

    7. Re:before everyone starts shouting at once... by Junta · · Score: 1

      With respect to the memory, at least this box has 32 GB of memory and 4-way Opteron. I know, still no where near the big iron, but very respectable at the price points OEMs are selling them for. Put Myrinet or Infiniband in these suckers for the money you save by not going Itanium, and you have a cluster configuration that scales really well.

      Itanium is a fascinating architecture, but Intel *has* to do something about power consumption, cooling, and just plain cost before it can be a good competitor to other systems. The price difference between cooling, power, and the initial price and the competitors means competing configurations equally priced either have many many more nodes or a much better interconnect between nodes.

      --
      XML is like violence. If it doesn't solve the problem, use more.
  39. Re:It's all about sticking it to the mac. by maeltor · · Score: 1
    Your post doesn't have ANYTHING to do with the topic.

    You honestly think they built this machine to "stick it to the mac"

    Wow, talk about zealot distorted reality. I'm amazed by your total lack of intelligent thought.

    "Some might argue that powerpc and sparc64 is more scalable than x86. But ask yourself, is it really?"

    Last I checked, this ITANIUM2 machine isn't about proving scalabity on x86 arch machines (btw ITANIUM isn't your typical home consumer x86 boxen). In fact I think they were going for specific types of computation when this was developed, but you know...I may wrong. :rolleyes:

    Its all about choosing the right hardware for the job.

    Your last statement is bullshit too. x86 is the most common arch because of original ease of use and relatively cheap hardware. Just because you can get drivers for a lot of "NIC's" on Linux boxes doesn't exactly prove that x86 is more scalable.

    FLAMEBAIT
  40. Re:It's all about sticking it to the mac. by Anonymous Coward · · Score: 0

    I'm going to guess and call troll. I almost bit on it but then I restrained my self when I realized that the poster didn't really say anything intelligent. For a consumer there is really no need to upgrade a computer other than memory. When is the last time your mom went out and bought a new motherboard for her PC? Thats what I thought, it probably never happened.

    I'm still waiting to see how much this cluster costs. I'm going to guess it was a pretty penny with quad Itanium processors.

  41. What about SCO? by watsondk · · Score: 4, Funny


    do they have the nerve to go after this cluster?

    afterall they are trying extortion by lawyer against other large Linux users

    1. Re:What about SCO? by Triumph+The+Insult+C · · Score: 1

      Darl: LLNL, you owe us, $500k for Linux licenses. We're cutting you a deal.

      LLNL: Sure thing Darl. We'll send it right over. Nevermind the appearance of the delivery truck, we want to make sure your 'payment' gets their safely. It will be in a small suitcase and don't worry about the ticking noise.

      Darl (to Chris): Heh. Dumbasses. We should hit up the gov't more often.

      --
      vodka, straight up, thank you!
  42. Re:Just imagine by Anonymous Coward · · Score: 0

    No...... 1) Imagine a beowulf cluster of insensitive soviet russian clods belonging all your base that run linux 2) Don't forget to pay your $699 licensing fee you cock-smoking teabaggers 3) ?????? 4) profit

  43. Re:Just a little ahead of Apple when you look at.. by Flashbck · · Score: 1

    did you copy this post or did you both just copy from some other forum?

  44. Sadly... by System.out.println() · · Score: 5, Funny

    "We sold the Inaniums! We sold the Inaniums!"

    "The Itaniums, however, remain unsold."

    *hopes that was not an actual mistake but rather a poorly conceived pun on "inane"...*

  45. Re:It's all about sticking it to the mac. by damiam · · Score: 2, Informative

    Sorry to burst your bubble, but Itanium isn't x86.

    --
    It's hard to be religious when certain people are never incinerated by bolts of lightning.
  46. Re:Just a little ahead of Apple when you look at.. by Anonymous Coward · · Score: 0
    From the date stamp, this one (above, by callipygian-showsyst) is the original post. The other one is the dupe! He's probably using some auto-post Karma script that copies high score posts and reposts as your own.

    Back in the days of Karma points, there was a perl script called "Karma50.pl" that did that trick...

  47. Yes but... by goombah99 · · Score: 2, Funny

    Can it run WINE?

    --
    Some drink at the fountain of knowledge. Others just gargle.
    1. Re:Yes but... by Anonymous Coward · · Score: 0
      Can it run WINE?
      If you poured some WINE on it I can guarantee it wouldn't be running for much longer...
  48. Comment removed by account_deleted · · Score: 1, Redundant

    Comment removed based on user account deletion

  49. Big Iron? by nacturation · · Score: 4, Funny

    Thunder sports 4,096 Itanium 2 processors in 1,024 nodes, some big iron by any standard.

    If the government gets a hold of that, we're going to need some big tinfoil...

    --
    Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
  50. Nah, but who knows what the NSA has cooking. by tukkayoot · · Score: 2, Insightful
    I thought of Google too, but yeah since their network isn't built for sheer computational speed, I doubt it's anywhere near the fastest/most powerful system for many processing tasks.

    The NSA, on the other hand... I would guess that they have the most powerful cluster of machines in the world for breaking encryption. Though perhaps not as powerful as the article's supercomputer for other tasks.

    Plus there are undoubtedly several other highly classified supercomputers designed to chew on other problems.

    So it would seem that you'd have to caveat any claim of regarding the "fastest computer" by saying it's the fastest known, non-secret computer. But then the headline loses some of its appeal.

    1. Re:Nah, but who knows what the NSA has cooking. by DAldredge · · Score: 2, Interesting

      The NSA has their own small scale fab to make their own custom chips. Hell, they do not every try to keep it secret and have let more than one file crew film part of it.

      They can design all the custom chips they want with their rather large budget.

  51. LLNL's usefulness by SuperBanana · · Score: 1, Troll
    my brother in law was driving a hydrogen fuel car from the lab 10 years ago as a test)

    I think that speaks volumes as to the usefulness of LLNL's research. After all, it's been 10 years, and there are still no hydrogen-powered cars available for purchase by consumers. Furthermore, there is extremely little research needed in the area; hydrogen conversion kits were developed by numerous companies and individuals decades ago.

    Why no hydrogen cars? Well, it could have something to do with hydrogen being a net-loss fuel; it takes more energy to make than it provides. But hey, Dubya loves it. Why? Well, you can extract hydrogen from natural gas(which pollutes just as much as burning natural gas, but it moves the emissions out of sight of the consumer, yay!)

    Why is it people like you who hear "Nuke" rant on and on like biased little children and post inflamatory things like this?

    I'll stop ranting when my nation's defense budget isn't the largest in the world several times over both in total and per capita- we spend OVER 50% of our budget on "defense" and our budget is larger than our top 3 potential enemies -combined-. Those are 2001 stats, mind you- and the defense budget has only gone up.

    1. Re:LLNL's usefulness by geek · · Score: 0, Flamebait

      You can get hydrogen from water, sorry you missed that one in 1st grade science. As to your cry baby routine over their usefulness I'll just overlook it as utter immaturity and ignorance. LLNL isn't an automaker, they simply did research which is what good labs do.

      I find it laughable how little losers like you turn every single thread into a Bush bashing fest. Grow up and get a life. I hate the man too but seething rabid ignorant posts on a message board don't change a god damn thing. Get over yourself.

    2. Re:LLNL's usefulness by Anonymous Coward · · Score: 0

      After all, it's been 10 years, and there are still no hydrogen-powered cars available for purchase by consumers

      Not true -- fleets can purchase hydrogen cars, and I suppose you could too if you had a means of fueling it up.

    3. Re:LLNL's usefulness by tap · · Score: 1

      You can get hydrogen from water, sorry you missed that one in 1st grade science.

      Water just doesn't break apart into oxygen and hydrogen on it's own, it takes energy! More energy that you get back when you burn the hydrogen by recombining it with oxygen.

    4. Re:LLNL's usefulness by slamb · · Score: 2, Informative
      Why no hydrogen cars? Well, it could have something to do with hydrogen being a net-loss fuel; it takes more energy to make than it provides.

      That's thermodynamics. It's true for any fuel. It's even true for oil and nuclear energy - the difference being only that the energy wasn't put in during our lifetime. (And in the case of nuclear, that the pre-existing energy is all but inexhaustible.)

    5. Re:LLNL's usefulness by amorsen · · Score: 1

      Contrary to popular belief, the accesible uranium deposits are very exhaustible. If the world switched to nuclear for all its energy needs, the uranium price would go way up. Here are estimates of uranium reserves.

      --
      Finally! A year of moderation! Ready for 2019?
    6. Re:LLNL's usefulness by slamb · · Score: 1
      Contrary to popular belief, the accesible uranium deposits are very exhaustible. If the world switched to nuclear for all its energy needs, the uranium price would go way up. Here are estimates of uranium reserves.

      I did not know that, but I'm not too surprised. (How efficient are our reactors, by the way? I.e., how much energy capacity do those uranium reserves represent?)

      I was actually thinking of fusion with that statement. A man's gotta dream. :) We will not run out of deuterium any time soon. And though we've had little success with fusion before, I think it's only a matter of time.

    7. Re:LLNL's usefulness by Anonymous Coward · · Score: 0

      How about this... Instead of trying to use hydrogen as a fuel for a car, by either combustion or fuel cell, use it for backup power for a semi-off-grid home. Say your house (in Arizona...) has a photovoltaic array on the roof. During the day you gather energy from that(hopefully while you are at work using someone else's power), and during this use that power to electrolyze water and store the hydrogen. Use a fuel cell at night to provide power and voila! you have a killer, safer, and much more regnerative battery.

      And it is true that fuel cell cars just push the hard part of energy transformation away from the consumer, usually larger power plants are more efficent per kW/hr in terms of pollution and money than, say, your average gasoline burning minivan...

    8. Re:LLNL's usefulness by IceAgeComing · · Score: 1

      But hey, Dubya loves it. Why? Well, you can extract hydrogen from natural gas

      It's also one of the few alternative fuel models that involves a centralized power generation plant. So if it succeeds, big oil companies will maintain control over power generation.

      Solar, wind, water, geothermal, etc. can be done on a smaller scale by smaller companies or individuals. Big oil companies will lose their strangehold on energy generation if these succeed. Hence they have no interest in helping these technologies advance.

  52. Re:Just a little ahead of Apple when you look at.. by Anonymous Coward · · Score: 0

    Twid (67847)'s post is dated Thursday May 13, @11:10PM

    callipygian-showsyst (631222)'s post is dated Thursday May 13, @11:27PM

    Now, which one was posted first again?

  53. Probably OT, but... by Trogre · · Score: 3, Interesting

    ... if you want a practically guided tour of LLNL, watch TRON sometime. They filmed it there (the science-lab live action stuff anyway).

    --
    "Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
  54. but.... by charstar · · Score: 0, Redundant

    ...can i play warcraft3 on it?

  55. But, it DOESN'T run Linux! by Anonymous Coward · · Score: 0

    Gotcha. Ha, ha!

  56. A lot of modification by callipygian-showsyst · · Score: 0, Redundant
    Pineapple on a lot of modification! Shared everything: in this, each disk is connected to all the data. standard apps like SAP etc can be used in other ways (you could run a web site off an Itanium 2), but the segment that Cray, SGI, HP Alphaservers, etc. have traditionally dominated.

    The segment that doesn't give a shit about cooling, power consumption, or price-performance, but who just need to break up data judiciously and spread acros s machines. Also adding a node will require re-partitioning of data. Mostly only custom apps which are fully aware of your partitioning etc will be able to take advantage.

    It is also likely cheaper than a similarly equipped Itanium 2 server would easily top $6,000. But anyway, good game to Lawrence Livermore. I'll be curious to see if Apple has another volley to fire before the top500 list closes for this round.

    - "When you want to occasionally hand-tune code segments in assembler, etc."

    Anyways, I'm not a fan of Intel lately, but the Itanium could have been a good choice. ALSO: Don't forget that the Itanium 2 is $1398 per chip on Pricewatch, and a dual processor Xserve.

  57. Nope. But they can't do what google does either. by Anonymous Coward · · Score: 3, Informative

    Google's cluster isn't a computational cluster.

    You have several types of clusters, each are designed to do a specific task, although you can easily mix-n-match for different purposes.

    1. Server clusters. Bunches of machines running together, providing services that compliment each other.

    For example you have a file server that is mirrored to another that is hooked up to a different part of a Lan/Wan backbone in order to improve service. Lot's of databases are clusters like this.

    2. High avaiblity clusters.

    You have a machines that are backups of other machines. If one machine fails a backup is activated instantly and replaces the failed machine without ANY loss in services.

    Sort of like a RAID harddrive setup. Hotswappable computers, that sort of thing.

    Google is the first 2 types. It has several clusters with nodes. Each node is made up of a few computers, if a node fails then another backup can back it up instantly, giving the techs time to correctly fix the issue. The computers each take some of the burden, too, so that it seems that they would have to be running mega-machines to provide the performance when in reality they just run a bunch of PC-style computers.

    3. Computational clusters. Clusters that are designed to pool their resources to create a single big computer that is used to proccess large amounts of data and intense mathmatical functions.

    2 types of these are Beowolf clusters and OpenMosix clusters.

    OpenMosix cluster is easy to setup if your a little bit familar with linux and even have knoppix cluster cdroms you can build ones quickly and easily.

    Beowolf is used for big number crunching and programs that use it are generally written to run a specific cluster, although libraries and tools are portable.

    Used lots in astromony for example. 10-12 PCs in a college lab can make a nice number crunching machine.

    There are some clusters that do all 3, lots can do only 1 or 2 of the types easily. Different types can compliment each other.

  58. Re:It's all about sticking it to the mac. by Roydd+McWilson · · Score: 0, Troll

    Too bad this article is about an Itanium cluster, not x86.

    --
    THE NERD IS THE COMPUTER.
  59. North America's? by hdd · · Score: 1

    I thought North America is suppose to be the leader of technology. I guess i was wrong...

    --
    This Sig is removed due to factual inaccuracy
    1. Re:North America's? by Anonymous Coward · · Score: 0

      You are wrong. During the last year the US was passed by Europe and Asia when it came to number of scientific papers published. Also with the current US govs decrease in science funding, there have been some indications that the flow of scientists are reversing, going away from the US to other countries.

  60. When we all know... by Chris+Brewer · · Score: 1

    That a roomfull of Chevettes means that someone has an unhealthy obsession with Vauxhall's...

    Nothing is as whiney as a Vauxhall Chevette doing 125km/h down the motorway, knowing that it ain't gonna get any faster, except maybe on a slope.

    --
    Consultancy: If you're not part of the solution, there's money to be made in prolonging the problem
    1. Re:When we all know... by sharkey · · Score: 1
      Nothing is as whiney as a Vauxhall Chevette doing 125km/h down the motorway,

      What about your wife sitting in the passenger seat of the brand new Corvette you just bought, instead of paying to go on vacation with her mother?

      --

      --
      "Outlook not so good." That magic 8-ball knows everything! I'll ask about Exchange Server next.
  61. Itanium vs Opteron by vlad_petric · · Score: 4, Insightful
    Itanium's instruction set is actually a lot more geared towards scientific computing than server benchmarks. Scientific stuff usually is made of very regular code, that is quite easily schedullable by the compiler. Server stuff is generally memory-bound and very irregular, so the processor usually gets less than one instruction executed per cycle - bundling instructions (static schedulling by the compiler) is completely pointless.

    "Big Iron" is a very vague term - server benchmarks behave very differently than scientific computation as far as performance is concerned; if you don't believe me I can easily point you to a couple of research papers analyzing them.

    The humongous on-die caches makes the Itanium perform well on servers, and definitely not the instruction-set architecture. So "WAS DESIGNED FOR" is only 50% true.

    --

    The Raven

  62. But... by Canberra+Bob · · Score: 0, Flamebait

    At least we have girlfriends and dont live in our parents basement.

  63. Re:But... by System.out.println() · · Score: 1

    Seriously. The basement's too cold.

  64. Linux support by linuxguy · · Score: 2, Insightful

    Intel provides excellent Linux support for Itanium. Also if you use the Intel compiler, which Lawrence Livermore does, you get considerable speed boost on Intel CPUs.

    See: http://www.llnl.gov/linux/linux_basics.html#compil ers

    Intel can afford to provide little niceties like this. Can AMD? I doubt it.

  65. hydrogen has negative net energy by SuperBanana · · Score: 1
    You can get hydrogen from water, sorry you missed that one in 1st grade science.

    Actually, I majored in physics.

    Re-read my post. Hydrogen, like gasohol, is a negative net fuel. Which means it takes more energy to make the fuel, than you get out of burning it. Same as Gasohol, which is produced from corn.

    Hydrogen is just the New Gasohol, politically. Gasohol is a huge piece of pork barrel spending that mostly lines the pockets of Archer Daniels Midland(ADM) and to a much lesser extent corn growers...to the tune of $1.4B+/yr since the early 70's.

  66. Big Mac Caveats by Anonymous Coward · · Score: 0

    19.94 is the first result we see for this new cluster, and likely the only. The first result for the Big Mac cluster was closer to 7 than 11, but the cluster was tuned and linpack rerun and resubmitted several times over several months. Although there is nothing wrong with that, it is quite unusual for the top500 linpack benchmark. This is a significant edge for the Big Mac. We can't say how much since other clusters usually don't get this special treatment.

    This reason is likely because without ecc memory there wasn't much point in running real workloads on the Big Mac. That has been corrected now, with the XServe G5 systems.

  67. Re: A Better Way To See It by value_added · · Score: 2, Insightful

    Ed Note: Unless the author wishes to narrow his/her audience to a small subset of Slashdot users, standard formatting and non-cutesy sentence case is always appropriate.

    There are basically three type of clusters:

    1. Shared Nothing: In this, each computer is only connected to each other via simple IP network: no disks are shared. and each machine serves part of data. These cluster doesn't work reliably when you have to aggregations. For example, if one of the machine fails and you try to to "avg()" and if the data is spread across machines, the query would fail, since one of the machine is not available. Most enterprise apps cannot work in this config without degradation. For example, IBM study showed that 2 node cluster is slower and less reliable than 1 node system when running SAP IBM on windows and unix and MS uses this type of clustering (also called federated database approach or shared nothing approach).

    2. Shared Disk Between Two Computers: In this case, there are multiple machines and multiple disks. Each disk is at least connected to two computers. If one of the computer fails, other takes over. no mainstream database uses this mode, but it is used by hp-nonstop. Still, each machine serves up part of the data and hence standard enterprise apps like SAP etc cannot take clustering advantage without lot of modification.

    3. Shared Everything: In this, each disk is connected to all the machines in the cluster. Any number of machines can fail and yet the system would keep running as long as at least one machine is up. This is used by Oracle. All the machine sees all the data. Standard apps like SAP etc can be run in this kind of configs with minor modification or no modification at all. This method is also used by IBM in their mainframe database (which outsells their Windows and Unix database by huge margin).

    Most enterprise apps are deployed in this type of cluster configuration. The approach one is simpler from hardware point of view. Also, for database kernel writers, this is the easiest to implement. However, the user would need to break up data judiciously and spread across machines. Also adding a node and removing a node will require re-partitioning of data. Mostly only custom apps which are fully aware of your partitioning etc will be able to take advantage.

    It is also easy to make it scale for simple custom app and so most of TPC-C benchmarks are published in this configuration. Approach 3 requires special shared disk system. The database implementation is very complex. The kernel writers have to worry about two computers simultaneously accessing disks or overwriting each others data etc. This is the thing that Oracle is pushing across all platforms and IBM is pushing for its mainframes. Approach 2 is similar to approach 1 except that it adds redundancy and hence is more reliable.

    So what type are we talking about here?

  68. Mod parent down by BashDot · · Score: 1

    As redundant. As someone else has pointed out, they just copied and pasted someone else's post. Honestly, finding an origional opinion is getting harder and harder these days...

    1. Re:Mod parent down by callipygian-showsyst · · Score: 1
      As someone else has pointed out, they just copied and pasted someone else's post

      Wait a minute! What type of perverted logic is this. If SOMEONE ELSE just pointed this out, then YOUR post is JUST as redundant.

      I'm going to ask /. for a REFUND on my subsciption. I didn't pay good $$$ to be surrounded by idoiots and jerkoffs!

    2. Re:Mod parent down by Anonymous Coward · · Score: 0

      I'm going to ask /. for a REFUND on my subsciption. I didn't pay good $$$ to be surrounded by idoiots and jerkoffs!

      Maybe you should kill yourself then, otherwise you'll see and idiot and jerkoff ever time you look in the mirror or open your mouth.

  69. Virginia paid retail by Anonymous Coward · · Score: 0

    Even better. The virigina dudes paid retail for the stock macs. If you can get adeail on itaniums you can get a deal on G5s.

    1. Re:Virginia paid retail by Anonymous Coward · · Score: 0

      They paid retail on the macs, but not the interconnect. Can you give me a quote on 1000 Infiniband host adapter PCI cards, please?

    2. Re:Virginia paid retail by Anonymous Coward · · Score: 0

      Wrong. They didn't pay "retail", but they did pay the same price any other huge academic order would have (much less than retail).

  70. Wait a minute... by Golgafrinchan · · Score: 1, Funny

    I thought Deep Thought was the 2nd greatest computer in the Universe of Time and Space?

    --
    My userid is prime!
    1. Re:Wait a minute... by MrIrwin · · Score: 1

      In fact if anybody had actually thought to ask Marvin he would have told them.

      --

      And if you thought that was boring you obviously havn't read my Journal ;-)

  71. Re:Just imagine by Trejkaz · · Score: 0

    LOL! And just when I'm out of mod points.

    --
    Karma: It's all a bunch of tree-huggin' hippy crap!
  72. But... by Trejkaz · · Score: 2, Funny

    But Windows only has 50% less TCO if your time is worthless.

    --
    Karma: It's all a bunch of tree-huggin' hippy crap!
  73. Grr!!! by Trejkaz · · Score: 0, Offtopic

    You forgot Natalie Portman, naked and petrified with hot grits down her pants, you insensitive clod!

    --
    Karma: It's all a bunch of tree-huggin' hippy crap!
  74. Of course... by MaestroSartori · · Score: 2, Interesting

    ...you realise that it isn't a linear scale. Trying to make a G5 cluster which achieved 4.8 gigaflops per processor would take more than the 4400 processors, and thus would easily take more than 300 more processors than are used for the Itanium cluster.

    300 processors. Thats 150 dual-processor boxes. I can't be bothered working it out now, but how far that goes to eliminating the power & heat advantage the G5 has would be interesting to find out...

    1. Re:Of course... by MaestroSartori · · Score: 1

      Errata: it would take 4335 G5s to get a 19.94 teraflops cluster assuming linear scaling, not the 4400 I stated. Whoops :)

  75. Re:It's all about sticking it to the mac. by customjake · · Score: 0, Offtopic

    HAHA, my mom just bought a new Asus board and a 2500+ Barton and some ram from crucial.com to replace her AMD K6-3 400. Oh the irony

  76. Re:It's all about sticking it to the mac. by Anonymous Coward · · Score: 0

    How about a real os, with SCALABLE architecture.

    But this cluster runs Linux, so much for THAT idea.

    Linux is the new OS of choice.

    You keep telling yourself that.

    x86 has ALWAYS been the arch of choice

    Too bad this cluster is IA-64 then. I guess all the embedded processors don't count either?

    If you disagree, why do I have every driver for almost every NIC on linux for x86, but I can't get any nic laying in a box to work on a mac?

    Because you are a moron and are trying to use an OS designed primarily for x86 on a non-x86 architecture. In other news, square pegs don't fit in circular holes. Man you are a genious.

    MACS ARE DIEING!

    And Linux is still born and hoping to one day be alive.

  77. No worries about that... by NevDull · · Score: 1

    According to the distributed.net speeds page, the rc5-72 rate of an I2 1.4 is about 1Gkey/sec


    Itanium 2 speeds

    Since a P4M running at the same clock does 3 times as much, it wouldn't be so efficient... though we are talking about the US Gov't.

    -Nev

  78. Re:It's all about sticking it to the mac. by smitty45 · · Score: 1

    Indeed, that is true. I stand corrected, but also like and have seen x86_64 run insanely fast. :)

  79. Not any more! by brodin · · Score: 2

    >It's all for reserved for Doom III on longhorn.
    Sorry, I'd played Doom III yesterday at E3. That's joke is (in your best Iron Chef voice) o-vah!

  80. can't believe nobody mentioned favourite vendor! by nighty5 · · Score: 2, Funny

    $2,863,104 in license fees going SCO's way!

    I can see the investors now rubbing their 2 cents together....

  81. The fastest? by NerveGas · · Score: 1


    4096 Itanium2's, pumping out 20 teraflops, and it's supposed to be the fastest? I thought that Red Storm (10,000+ Opterons) was pumping out around 40 teraflops/second.

    Did I miss something?

    steve

    --
    Oh, you're not stuck, you're just unable to let go of the onion rings.
    1. Re:The fastest? by convolvatron · · Score: 1

      red storm wont be installed at sandia until later this year. it will be > 40 Tflops peak..but neither of them will be close to peak in actual application code. they should remain roughly within a factor of two. i have a strong hunch that red storms network will be considerably faster, but i have my doubts that that will translate into a substantial performance benefit on mpi code

  82. Sounds nice, but... by Anonymous Coward · · Score: 0

    Everyone knows Magus Casper, Magus Balthazar, and Magus Melchior are far superior. Fear the Magi system. (I almost pulled root on Casper too, damnit.)

  83. The mice can't be happy... by mr_z_beeblebrox · · Score: 1

    after not more than 30 seconds the thing clicked and whirred and spat out a little piece of paper that said "2*3*7"

  84. Top500 list not updated yet by mrjb · · Score: 2, Insightful

    This is the official top 500 list of supercomputers (not updated yet although thunder is mentioned as '*possibly* the second-most powerful computing machine on the planet'). Linux moving up to second place (from fifth a bit ago, iirc), woohoo! Only one left to beat!

    --
    Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
  85. Support Costs: Wow! by Wingsy · · Score: 1

    Quote from: http://www.cbronline.com/print_friendly/fcd941d5f0 31c96e80256e130060a00a "The service provider's six-year, $328m deal to provide a range of enterprise-wide IT services to the US Agency for International Development (USAID) was...." Yikes! That's over 50 mil per year, for IT costs???? And this company only has 52 employees? This has to be wrong. Even Windows IT support isn't that high. :)

    --
    If I didn't have absolutely NOTHING to do, I wouldn't be here.
  86. why you need a fast processor in your cluster by ianturton · · Score: 1
    or needed high single CPU performance for a pooly parallized application. (Of course if your application parallizes poorly, a cluster is probably a bad choice to begin with).

    Your application may parallelise at a high level but still need a good FP CPU to do the parallel bits. Thats why Cray's X1 has big vector processors linked together in a cluster. If you need to know more buy my book

    Ian

  87. But not as fast as google :-) by Anonymous Coward · · Score: 0

    Didn't the Stashdot article called How Many Google Machines, Really? claim that google consisted of 79,000 computers. That makes google the fastest computer in the world :-)

  88. Something else more cost-effective? by Anonymous Coward · · Score: 0

    Thunder: 2x the performance of the VATech cluster, 4x the cost. mmmmmm...... (OK, 3.8 times the cost if you paid the students who assembled it)

  89. using waste heat. by ianturton · · Score: 1

    When Edinburgh University bought a Cray T3D they did use the waste heat to heat the building. Cray T3Ds used a liquid freon cooling system to shift the heat from the 256/512 alphas this went through a huge heat exchanger with the output of this feed to the heating/hot water system. This went some way to offset the costs of the electricity. The power requirement was such that a new electrical sub station was required.

  90. Re: A Better Way To See It by sapbasisnerd · · Score: 1

    Better formatting doesn't solve the fundemental problem here which is that the content is wildly inaccurate.

  91. Your forgetting by IAmAMacOSXAddict · · Score: 2, Interesting

    It also has twice the processors, to generate the X2 times speed that they claim. Something tells me, now that VA is recieving the XServer G5 cluster nodes, that they may want to add some more units. they can put 48 units in each rack now, rather than 12 of the full size G5 Desktop form factor. According top my primitive calculations that would allow them to run 4 times as many machines in the same space (would be over 8000 CPUs. I figure that will likely kick the crap out of this new linux cluster...

    --
    MacOSX, because making *NIX better is a lot better than waiting for Micro$loth to fix Windows
  92. Too bad.. by SQLz · · Score: 1

    To bad its Intel. They would have had more power and paid 1/2 as much for opterons.

  93. Fast?! It's not that fast .... by jonykaos · · Score: 1

    whatever this (Thunder) cluster can do in terms of performance will be peanuts compared to IBM's BlueGene/L, also at LLNL: 360 TeraFlops ~130K processors http://www.llnl.gov/icc/lc/OCF_resources.html

  94. Re:can't believe nobody mentioned favourite vendor by IAmAMacOSXAddict · · Score: 1

    They can't, they no longer have the two cents to rub together, now that SCO is falling apart...

    --
    MacOSX, because making *NIX better is a lot better than waiting for Micro$loth to fix Windows
  95. ...When there's 1024 of them solving one problem by Glasswire · · Score: 1

    (end of msg)

  96. or make me King... by bblackfrog · · Score: 1

    click here for my proposed solution to the energy crisis.

    Unfortunately, could only be enacted by a dictator.

  97. Cray's Red Storm by Anonymous Coward · · Score: 0

    60% more performance, but 150% more processors and 500% more cost. Opteron isn't really a great value proposition in this space, although it can perform very well (supposing Red Storm is completed and performs as expected).

    Scaling is a hard problem on these clusters, so it is quite possible that Opteron could put on a competitive showing at the same performance level as Thunder, even though Red Storm has a much worse price/performance ratio.

  98. Finally... by sparcnut · · Score: 1

    A computer that can run Longhorn.

    --
    perl -e 'print $i=pack(c5, (41*2), sqrt(7056), (unpack(c,H)-2), oct(115), 10);'
  99. Anybody... by Anonymous Coward · · Score: 0

    ...know what framerate they get with this thing?

  100. I'm pretty lost on this processor business by Knights+who+say+'INT · · Score: 1

    and I'm about to gey myself a new PC for running Linux after having this one for over five years. What is a good high-end-consumer solution that doesn't heat too much? This is Rio de Janeiro, and we ain't got no air conditioning either.

  101. Bull fucking shit by daveschroeder · · Score: 1

    we spend OVER 50% of our budget on "defense"

    Bullshit

    http://das.doit.wisc.edu/misc/outlays.jpg

    Defense spending, INCLUDING veterans affairs and foreign affairs, is 20%. Nice try.

    We spend more on "social programs" alone, and twice as much on social security and medicare.

    Not saying defense spending still isn't a lot, but it's no where near "over 50%".

  102. It is not big iron at all. by Anonymous Coward · · Score: 0

    No, its not big iron by any definition, its a cluster. Big iron are big powerful machines, not lots of ordinary little machines attached together with a fast interconnect.

  103. But what distro? by Zemplar · · Score: 0

    "I want to try Gentoo"

    "NO. We're using Debian!"

    "Bob's already started loading Slackware!"

    -- Seriously --
    Anyone know specifically what OS they will be using?

  104. Re:PEOPLESPRIMARY.COM MUST DIE!!!! by Spudnewt · · Score: 0

    peoplesprimary only got you fired because you were surfing slashdot while at work. At any rate, you would have been fired anyways for wasting your company's time. You are exactly what is wrong with this world. You are employed to do work, not waste time browsing slashdot.

  105. Okay, let's take the current #5 then! by DeeKay · · Score: 1

    Pacific Northwest National Laboratory
    1936 Itanic2s 1.5Ghz, 8633 Rmax -> 4.45 GFLOPS per CPU

    Less than the G5. Where's the big Floating point monster Itanic now i wonder?

    G5 and Itanic can both do 4 DP-FPops per Cycle. However, the G5 has 25% more clock, so bad luck for Intel!

    Sorry, but the SPEC-Scores (Itanic2 SpecFP = 2x G5-SpecFP) simply fail to show... SPEC has become as much of a fake as almost any Benchmark!

  106. Re:Buy My Book by Ed+Almos · · Score: 1

    I might consider buying your book if you didn't want a hundred and twenty books per copy !! Unless each page is printed on sheets of platinum I don't see how you can justify such a price.

    Ed Almos
    Budapest, Hungary

    --
    The more corrupt the state, the more numerous the laws. - Tacitus, 56-120 A.D.
  107. Re:It's all about sticking it to the mac. by Anonymous Coward · · Score: 0

    How is this a troll?