Slashdot Mirror

← Back to Stories (view on slashdot.org)

Virginia Tech to Build Top 5 Supercomputer?

Posted by michael on from the computer-hogs dept.

hype7 writes "ThinkSecret is running a story which might explain exactly why the Dual 2GHz G5 machines have been delayed to the customers that ordered them minutes after the keynote was delivered. Apparently, Virginia Tech has plans to build a G5 cluster of 1100 units. If it manages to complete the cluster before the cut-off date, it will score a Top 5 rank in the Linpack Top 500 Supercomputer List. Both Apple and the University are playing mum on the issue, but there's talk of it all over the campus."

104 of 460 comments (clear)

  1. Just imagine,... by Anonymous Coward · · Score: 3, Funny

    ...a Beowulf cluster of....

    Oh wait, it is a cluster. DAMN!!!!!

  2. is that so? by carpe_noctem · · Score: 2, Funny

    there's talk of it all over the campus ....and we all know how reliable campus rumors are! C'mon guys, don't forget to say hi to the Olson twins when you see them on campus next year!

    --
    "Quoting famous computer scientists out of context is the root of all evil (or at least most of it) in programming." - K
    1. Re:is that so? by daeley · · Score: 4, Funny

      C'mon guys, don't forget to say hi to the Olson twins when you see them on campus next year!

      Wow! Imagine a Beowulf cluster of...

      Er, ah, forget I said anything.

      --
      I watched C-beams glitter in the dark near the Tannhauser gate.
    2. Re:is that so? by carpe_noctem · · Score: 2, Insightful

      Wait a sec...you mean the thinksecret story, or the macrumors one? I think "real" is a bit of an overstatement here...

      --
      "Quoting famous computer scientists out of context is the root of all evil (or at least most of it) in programming." - K
    3. Re:is that so? by NASAKnight · · Score: 3, Interesting

      It's not just a campus rumor. I attended an informational meeting about it today (which was actually postponed when I got there, so there will be another one soon). Basically, they are recruiting a few computer geeks here at Tech to help set it up and all. Should be fun :P

      Stephen

      --
      Fault loves the past, worry loves the future, but content enjoys the present.
    4. Re:is that so? by ReallyQuietGuy · · Score: 2, Funny

      Er, ah, forget I said anything. it's too late!!!! a beowulf cluster of Olsen Twins!?!?!? OH MY GOD!!!!!

  3. What about latency? by Thinkit3 · · Score: 2, Interesting

    That's the one thing that favors huge amounts of processors in the same box. All this "the internet is one giant distributed computer" doesn't acknowledge this. A box designed to be separate just will not have the latency advantage of a supercomputer designed from the ground up.

    --
    -Libertarian secular transhumanist
    1. Re:What about latency? by Anonymous Coward · · Score: 3, Interesting

      Who builds 2-8K processors in a box? Have you thought for a second what it might take to power or cool this? Or if it could be wired? The actual engineering of actually doing this are much more challenging than talking crap on Slashdot.

      You win the moron of the article award. Congrats.

    2. Re:What about latency? by andrewl6097 · · Score: 3, Interesting

      Actually, the amount of cache coherency checking needed in a large shared memory machine drives the latency right up. Besides, myrinet is ~15 usec latency. That's pretty goot for most things.

    3. Re:What about latency? by Glonoinha · · Score: 3, Insightful

      -You win the moron of the article award. Congrats.

      Now you are one optimistic AC. The day is still young. I am giving 30:1 odds that there are going to be way better morons than Thinkit3 before this thread is archived.

      --
      Glonoinha the MebiByte Slayer
    4. Re:What about latency? by purdue_thor · · Score: 5, Informative

      >> A box designed to be separate just will not have the latency advantage of a supercomputer designed from the ground up.
      I suggest you look at the list of the top supercomputers in the world. Most are clusters, ie. separate, distinct machines (just a quick glance shows the top 25 all are). It's just too darn hard to make a shared memory computer with 1000's of processors. So the common architecture is to make a cluster of smaller shared memory machines.

      Besides, most clusters built utilize special interconnects like Myrinet that offer low latency connections. They're more expensive than ethernet, but it's a supercomputer so you spend it.

      >> All this "the internet is one giant distributed computer" doesn't acknowledge this.
      On the contrary... people know this very well. That's why we see rendering and SETI processing as distributed. They don't really need to communicate with others often.

    5. Re:What about latency? by Junta · · Score: 5, Informative

      There are tradeoffs actually. This isn't like distributed.net or seti@home, this is a controlled network. They have complete control over the network switches, technology, and topology used and can design the network to accomodate tho problems the cluster will be designed to solve.
      For example, you could use Myrinet to get 2 Gigabit, super low latency connectivity, or Quadrix, or Infiniband, or just a well laid out Gigabit Ethernet with high end switches.

      In multiple processors in a box, the processors have to fight for the resources that box has to offer. NUMA alleviates demand on the memory, but IO operations (when writing to disk or to network) in a multiprocessor box block a good deal as the processor count in a node rises.

      The idea with clusters is that inter-node communication in most cases can be kept low. Each system can work on a HUGE chunk of a problem on its own, with its own dedicated hard drive, memory subsystem, and without having too much competition for the network card. A lot of problems are really hard to solve computation wise, but are *very* well suited to distributed computing. A prime example of this is rendering 3D movies. Perhaps oversimplifying things, but for the most part, a central node divides up discrete parts (a segment of video), and each node works without talking to others until done, so the negative impact is minimal. Certain problems (i.e. nuclear explosion simulations where time and spacial chunks interact more with one another) are much more sensitive to latency/throughut. Seti@Home and distributed.net are *extremely* apathetic to throughput/latency issues (not much traffic and very infrequent communication).

      --
      XML is like violence. If it doesn't solve the problem, use more.
    6. Re:What about latency? by Aapje · · Score: 4, Insightful

      It all depends on what you need the cluster for. Some computations need constant communication, others can go on for hours, days or even weeks without feedback. If you're smart, you use supercomputers for the first kind of tasks and clusters for the second kind.

      Universities (and big business) often work together and exchange resources. Virginia Tech gets a large amount of bargaining power by having control over a large amount of processing power. They can easily trade CPU time on their cluster for CPU time on a low-latency supercomputer.

      --

      The Drowned and the Saved - Primo Levi
    7. Re:What about latency? by Kibo · · Score: 5, Interesting

      I wonder if any universities have tried to write a distributed computing app along the lines of seti. Require it to connect to the university network, it grabs itself maybe 50 megs of hd space, and a fraction of all the new computers people bring to campus, in addition to all the computer lab gear belong to their massive number crunching problems. Make another version available to alumni, or even institutions as some form of corporate sponsorship.

      Then if it got popular, and they were really clever, they could sell off a part of that computational power they amassed to solve other peoples problems providing for funding for new versions and new supercomputing clusters.

      --
      --Jimmy has fancy plans; and pants to match.
    8. Re:What about latency? by Anonymous Coward · · Score: 4, Interesting

      Latency is paramount for some tasks, less important for those that *can* make a good distributed project over the Internet of today.

      Now, since today's supercomputers are *all* massively parallel constructions, the difference between a commercial design and an off-the-shelf cluster is in the quality and speed of the interconnects. NEC's Earth Simulator, the prime example of 'custom' supercomputer architecture, puts many processor units on *ridiculously* fast 'local' buses, and its racks are all interconnected with still_pretty_insanely_fast (and rather expensive) custom links.

      Meanwhile, more 'commercial' designs use various interconnects. IIRC, NEC's 'regular' supercomputers, which formed the design basis for the Earth Simulator architecture, use Fibre Channel 'mesh' networks between racks. The Opteron - sure to be an up-and-coming player in this market - offers HyperTransport, which it looks like Cray will be stretching to its limits on Red Storm; I'm not sure *how* long an HT bus can be, but one gets the impression they'll be stretching it as far as possible, and it's certainly high throughput/low-latency versus the technologies you'd usually find in use for 'networking.'

      Anyhow, point is, those designs pack a lot of CPUs together with *very* fast interconnects (equivalent to 16, 32, 64+-way SMP), and have lots and lots of racks of those. (The Opteron/Red Storm approach sounds sexy to me, because I think Hypertransport should let them pack 'lots and lots' of CPUs together versus existing designs. I've yet to read anything about what they're actually doing with it, though.)

      Now.. In contrast, an 'off the shelf' cluster is usually going to stick with Ethernet, and will only have 1 to perhaps 4 processors per [node-unit-where-the-CPUs-are-connected-on-a-fast- local-bus], depending how affordable 'cheap' multiprocessor systems are at the time. But *everyone* building supercomputers bumps up against the latency/routing problem; it's just a question of whether it's a problem for, say, 50 Earth Simulator racks (aren't there quite a few more?) vs. 1100 PowerMacs. Experimenting with 'lots of little nodes' has led us to better understand the problem, and learn how to produce tuned topologies that can compete favorably with 'purpose-built' hardware. See: http://aggregate.org/KASY0/

      Now, the question *is* one of cost-benefit. Large supercomputers tend to be built with maintenance features and power efficiency in mind. In turn, a totally 'off the shelf' cluster like KASY0 has some advantages because each machine is a cheap, practically disposable 'module' unto itself, and can doubtless be downed off the cluster, pulled out and replaced with another while being easily bench-repaired (since, after all, it's a self-contained PC, rather than a CPU blade or some other random card that would require an expensive test rack to troubleshoot). Meanwhile, if you absolutely demand low-latency, you want one sort of design (Red Storm seems to be acheiving it 'on the cheap,' by combining off-the-shelf - and thus cheap - chips and buses with smart 'custom-design' engineering) while if you can sacrifice some for throughput (jobs with few conditionals), you want another... (like 1100 G5 Macs on a shelf, wired with 'boring' gigabit ethernet, especially if Apple is giving you a bulk discount on the hardware).

      So what I'm trying to say is... this is a *combination* of PR stunt and intelligent planning, and there's certainly a lot of 'good science' they could do with the beast - both in number-crunching and 'computer science' a-la cluster topologies. Whether they'll actually *use* it for such, or if it'll be solely a topology toy is anyone's guess.

      I think there's some hope that it'll be the "Real Thing," though, since this would explain some of the weird rumors about FC-on-the-mainboard Macs. So they get a Real Monster, made of what will be revealed as "the new G5 Xserves" at the unveiling. The best of COTS *and* fresh d

    9. Re:What about latency? by BobGibson · · Score: 3, Interesting

      Or how about this: your bandwidth is dependent upon the amount you contribute to the distributed processing.


      Hopefully there would be some sort of minimum service level, maybe 64kbps; presumably people dropping tens of thousands expect at least a modicum of return on their investment. People who didn't want to install the client could trudge along at those speeds.


      Eventually there would be a market system, whereby people would trade their completed blocks for other commodities, like food vouchers, prints, copies, cash, and sexual favours.


      Good luck and godspeed,


      Branch

    10. Re:What about latency? by Anonymous Coward · · Score: 2, Informative

      In multiple processors in a box, the processors have to fight for the resources that box has to offer. NUMA alleviates demand on the memory, but IO operations (when writing to disk or to network) in a multiprocessor box block a good deal as the processor count in a node rises.

      Seymour Cray once said that a supercomputer is a device for turning compute-bound problems into I/O-bound problems. In other words, if your job is I/O bound, great! That means you're making the best possible use of your compute resources.

      In the real world, most jobs require far more compute resources than they do I/O resources. So scaling to a thousand processors or more makes sense, because we can already scale I/O up to gigabytes per second, either disk or network, very easily.

      The idea with clusters is that inter-node communication in most cases can be kept low.

      Alas, the idea with supercomputers is that inter-node communication cannot be kept low. Consider terabyte-scale data set visualization, for example. There's simply no way to do that job without distributing a copy of the entire data set to every node. That makes is a really bad job for a cluster, but a perfect job for a supercomputer.

      Do not be fooled into thinking that clusters are superior to supercomputers, or vice versa. There are tasks that clusters can do cost effectively that supercomputers cannot do as cost effectively. However, there are tasks that supercomputers can do that clusters simply cannot do, no matter the cost. So a one-to-one comparison between the two is inevitably going to be incomplete and misleading.

    11. Re:What about latency? by stingerman101 · · Score: 2, Insightful

      Doesn't it also depend on the interconnect technology they decide on? Since the G5 architecture uses hypertransport for the PCI-X connection and switches the communications to the processor, we are not talking about the same level of latency that older generation PC's had, are we? Isn't it premature to draw conclusions until we better understand how Apple and Virginia Tech plan to architect this new super type computer cluster?

  4. Hmmm... by Undaar · · Score: 4, Funny

    "Both Apple and the University are playing mum on the issue, but there's talk of it all over the campus."

    Must be a pretty boring campus...

    --
    ~ "When I'm of that age I'm just going to live up a tree."
    1. Re:Hmmm... by danielobvt · · Score: 2, Funny

      Must be a pretty boring campus...
      Says the person posting a comment on slashdot on the weekend.....
      Was that tinkling sound the noise of your glass house falling down? ;)

  5. Run Photoshop ! by orthopodreloaded · · Score: 4, Funny

    Are they gonna run Photoshop on that supercomputer ?

    1. Re:Run Photoshop ! by TheRaven64 · · Score: 2

      What about X-Code? It supports distributing a compile around a network. I imagine that you could get some pretty sweet compile times with a cluster of that size...

      --
      I am TheRaven on Soylent News
  6. 1100 G5s still can't... by Anonymous Coward · · Score: 3, Funny

    play Doom III.

  7. As a VT student... by Julius+X · · Score: 4, Informative

    but there's talk of it all over campus.

    Funny, I haven't heard anything about it prior to today. Guess I'm just out of the loop then...

    --

    -Julius X
    remove "-whatkindofspamdoyoutakemefor-" from email to send
    1. Re:As a VT student... by Kirby-meister · · Score: 4, Informative

      As a CS student at VT, I received word of it days ago - Hello all, This email is to serve as invitation and notice of impending Terascale Facility assembly assistance. For those receiving this info for the first time know that Virginia Tech is building a top 10 supercomputer from scratch and we need your assistance. We do have one stipulation to volunteerism and that is you must not be a wage employee of the university. Grad students on GTA/GRA are fine as well as others outside the university that may wish to volunteer. We are expecting to receive machines next week!!! Yikes! In preparation for the assembly process, we need to get volunteers together at the AISB (Andrews Information Systems Building), 1700 Pratt Dr., this weekend. We are planning to have a process orientation session start at 10:00 AM on Saturday, August 30, and last no longer than an hour. We can give you a few more details about the project if you show up and have not been before. :-)

    2. Re:As a VT student... by Creosote · · Score: 5, Interesting
      Funny, I haven't heard anything about it prior to today. Guess I'm just out of the loop then...
      As a University of Virginia staff person, I can tell you that VT's impending purchase of 1100 G5's was announced on our Mac user's group email list back on 28 July. By Apple's regional Higher Education user's group rep, who kiddingly asked when they could expect UVa's purchase order for 1200...
  8. That's just Hokie by leviramsey · · Score: 4, Funny

    And it'll get skunked by 40 teraflops by Duke's supercomputer every year!

    1. Re:That's just Hokie by TiMac · · Score: 2, Informative
      And which supercomputer might that be?

      I'm at Duke...let's just say I'm "in" on a lot of computing stuff...and I don't know of any supercomputer on campus of any significant magnitude. There's a couple clusters....

      Maybe you were just making a joke....I had no idea. :)

      --

    2. Re:That's just Hokie by bencvt · · Score: 2, Funny

      Yeah, in basketball at least... but the football score difference will be about the same. :-)

  9. Problems with my supercomputer. by Anonymous Coward · · Score: 5, Funny

    I don't want to start a holy war here, but what is it with you with you G5 zealots? Ive been sitting at my 1100 CPU G5 supercomputer for 20 minutes as it computers a fast fourier transform of an 8Ghz guassian. 20 minutes! At home, on my 60 cpu linux beowulf cluster, the same operation would take 2 minutes if that. Also, while this operation is takiing place, Doom III won't start, and everything else grinds to a halt, even my multithreaded emacs is struggling to keep up as i type this.

    My Sun Enterprise 5000 is faster than this machine at times. Super computer addicts, flame me if you want, but I'd rahter hear some inteligent reasons why I should use the G5 supercomputer over cheaper, faster clusters.

    1. Re:Problems with my supercomputer. by Anonymous Coward · · Score: 2, Funny

      Doom III won't start

      Doom III isn't out yet.

      flame me if you want

      'kay. Wanker.

      but I'd rahter hear some inteligent reasons why I should use the G5 supercomputer over cheaper, faster clusters.

      Well, you see, 5 is a bigger number than 4, and intel is stuck at 4, so ours is better. And AMD doesn't even *do* numbers!

    2. Re:Problems with my supercomputer. by carpe_noctem · · Score: 2, Funny

      Are you buy any chance the guy emailing out these spams?

      --
      "Quoting famous computer scientists out of context is the root of all evil (or at least most of it) in programming." - K
    3. Re:Problems with my supercomputer. by Rosco+P.+Coltrane · · Score: 3, Funny

      even my multithreaded emacs is struggling to keep up as i type this.

      May I suggest you stop emacs to free resources for your FFT?

      --
      "A door is what a dog is perpetually on the wrong side of" - Ogden Nash
    4. Re:Problems with my supercomputer. by Viadd · · Score: 5, Funny
      I don't want to start a holy war here, but what is it with you with you G5 zealots? Ive been sitting at my 1100 CPU G5 supercomputer for 20 minutes as it computers a fast fourier transform of an 8Ghz guassian.

      And what's with having only 1100 mouse buttons? At the price they're charging, why didn't Apple provide 2200 mouse buttons and 1100 scroll wheels. And why did they use a dead operating system like BSD anyway?
    5. Re:Problems with my supercomputer. by darc · · Score: 2, Funny

      You idiot! A google search would have solved all of your questions and comments. You should have visited the google usenet groups. alt.supercomputers.help.1100CPUG5, like, duh.

      All these newfangled newbies posting dumb questions to ask slashdot. Sheesh. Next thing you know, they'll be asking legal advice.

      --
      Tired of legitimate data sources? Try UNCYCLOPEDIA
    6. Re:Problems with my supercomputer. by Lars+T. · · Score: 2, Funny

      Sure AMD does numbers, they even add a +.

      --

      Lars T.

      To the guy who modded me down from perfect to terrible Karma - Apple haters still suck

  10. Re:What? by inkswamp · · Score: 4, Informative
    Slashdot posting vaporware news from an unreliable (Thinksecret et al) source?

    I take it you don't look at Think Secret on a regular basis. It is, easily, the most accurate Mac rumors site out there. In fact, they have posted info on numerous occasions that has caught the attention of Apple's lawyers, and have been forced to pull down and issue their standard disclaimer. Say what you will about other rumors sites (most of them simply feed off each other) but there are some startlingly reliable sources informing Think Secret. Frankly, I don't recall the last time they were wrong about anything they've posted.

    --
    --Rick "If it isn't broken, take it apart and find out why."
  11. Macs ? by Rosco+P.+Coltrane · · Score: 4, Interesting

    Virginia Tech placed the dual-2GHz G5 order shortly after the G5 was announced. Multiple sources said Virginia Tech has ordered 1100 units

    Wow, that'll make Apple's quarter for sure :-)

    Seriously though, why PowerMacs ? I've always been under the impression that intelloid machines are the cheapest commodity hardware around for equivalent processing power, if not the most exciting. Would anybody know why Powermac G5s are a better choice here?

    (Note to computer zealots: it's not a flamebait, it's a genuine question, from someone who is rigorously ignorant of the Mac world. And just in case, the first sentence is a joke, too ...)

    --
    "A door is what a dog is perpetually on the wrong side of" - Ogden Nash
    1. Re:Macs ? by jbolden · · Score: 5, Informative

      Altivec. Certain types of vector code when compiled to only run on a G4 outperform a pentium even at 3+x the ghz range (i.e. a 800 mhz G4 beating a 3ghz PIV). Assuming similar numbers for the G5 and the increase across the board on all the non vector operations + the fact that the 970 work together so much better....

      I can see it making a lot of sense. NASA and lots of bio companies use the G4s this way.

    2. Re:Macs ? by WasterDave · · Score: 5, Interesting

      why PowerMacs?

      A couple of things make them suitable for clustering:
      * There's heaps of processor-processor bandwidth and memory bandwidth.
      * On board gigabit ethernet.
      * Monster fast execution of properly written vector code.
      * Well designed cooling.

      Of course, the bang/buck ratio could be an issue for some debate but there's little doubt that in comparison to other commercial unices it's an absolute bargain.

      Dave

      --
      I write a blog now, you should be afraid.
    3. Re:Macs ? by damiam · · Score: 5, Insightful

      For certain types of processing (rc5 cracking is one example), Macs completely smoke PCs. For example, distributed.net stats show that a 667Mhz G4 can process more keys/second than a 2.8Ghz P4. Considering how much faster a 2Ghz G5 would be, a 1100-node cluster would be damn powerful if you were doing work that mapped well onto Altivec.

      --
      It's hard to be religious when certain people are never incinerated by bolts of lightning.
    4. Re:Macs ? by Llywelyn · · Score: 4, Interesting

      1) As others have mentioned, AltiVec and the dual-FPU on the G5 probably were significant factors in this decision. The Earth Simulator is comprised of processors that are very slow at most tasks, but are designed to scream at vector-optimized code and, honestly, AltiVec makes SSE2 and 3Dnow! look like toys by comparison.

      2) You would be hard pressed to configure a dual-opteron or dual-Xeon which could trounce the G5 in terms of speed and cost significantly less. MacOS X server also costs less than any version of windows (pure capital cost here for an 1100 seat license), which may also have factored in.

      3) My guess is that they have struck a fairly significant deal with Apple (even so low as Apple provides them at cost, though I doubt its quite that low) in exchange for some degree of publicity when this thing is built.

      --
      Integrate Keynote and LaTeX
    5. Re:Macs ? by Llywelyn · · Score: 2, Insightful

      True, but it is entirely possible that they are building this for more than just to say "we have a computer on the top list and you all sux0r5". They are depending on the dual-floating point units on the G5 for LINPACK and AltiVec for whatever else they put it to.

      Without more details its hard to tell

      --
      Integrate Keynote and LaTeX
    6. Re:Macs ? by 11223 · · Score: 4, Informative

      Oh, and I suppose you were asleep during the discussion of the G5's dual, independent double precision floating-point units? And the out-of-order-execution engine that makes them usable? And the memory architecture that will make it scream on large data sets?

      The G5's floating point hardware is the most advanced to be found right now, either in standard double-precision or vector double precision.

      (FYI: yes, this cluster exists, or will exist. Unfortunately I believe they will be using MPICH which might put a dent into their numbers.)

    7. Re:Macs ? by Erich · · Score: 3, Informative
      Altivec
      Doubtful. Altivec can only do single-precision floating point. It's pretty good at it, it can do operations 4 wide, but only single precision. Linpack needs double precision (at least, for the benchmark).

      The dual floating point units on the G5 will help, but it's nothing extraordinary. P4s and Athlons both have multiple floating point units. P4's are relatively orthogonal, Athlons less so. However, SSE2 allows for vectorized double precision operations. It is likely that for the linpack benchmark, best-in-class P4 or Athlon architecture-based machines would outperform best-in-class G5 machines.

      Altivec is extremely powerful. However it is only useful for applications that don't require their floating point to be double precision. SSE2 is less powerful, but allows for double precision SIMD processing.

      --

      -- Erich

      Slashdot reader since 1997

    8. Re:Macs ? by Lars+T. · · Score: 2, Informative

      Apple's list of (third party) clustering tools.

      --

      Lars T.

      To the guy who modded me down from perfect to terrible Karma - Apple haters still suck

    9. Re:Macs ? by Lars+T. · · Score: 2, Interesting

      Heat. CPUs create quite a lot, some more some less (like the G5). A few years ago a German university (Chemnitz IIRC) build a large Linux cluster. For cost and speed reasons they wanted to go with Athlons, but decided to use P3s instead, mostly because they created less heat. It is one thing to pack a couple of hundred computers into a building, it's an other to also buy a new air-conditioning for that building.

      --

      Lars T.

      To the guy who modded me down from perfect to terrible Karma - Apple haters still suck

    10. Re:Macs ? by tmattox · · Score: 2, Interesting

      I am one of the designers of KLAT2 and KASY0, and the guy who ran the Linpack benchmarks on both. Over 3 years ago when we submitted our results for KLAT2 to the top500 list, there was no public indication that 64-bit floating point was required. It took them awhile, but the top500 website now has a FAQ that indicates "full precision" is required, and they interpret that as 64-bit for most machines. FYI, 32-bit FLOPs are useful in many situations, and machines had been on the top500 list that had used 32-bit FLOPs. You might take a look at our KASY0 FAQ on GFLOPS. As a means to rank the top500, I think it is quite legitimate to require 64-bit FLOPS, but that doesn't make it "illegal" to use 32-bit Linpack FLOPS for other comparisons.

      As for the G5, it won't need AltiVec to get good Linpack numbers due to its fused multiply-add capability in its dual floating point pipes. That's 4 FLOPs per clock peak! I hope VT was able to get Apple to leave out, and not charge for, the components not needed in a cluster node. The PCI-X slots in the G5 should allow VT to better use a high-speed cluster network technology. Commodity x86 boxes tend to only have 32-bit 33MHz PCI, limiting the usable link bandwidth between nodes to under a gigabit per second. For 64-bit Linpack GFLOPS per dollar, a cluster of G5's could be competative. I look forward to seeing their results, and any similar work using the upcoming Athlon 64.

      --
      Tim Mattox
    11. Re:Macs ? by Scudsucker · · Score: 2, Interesting

      They use HyperTransport which was invented by AMD and used in Opertons.

      So what? Did he say it was exclusive to PowerMacs? And Apple is a member of the HyperTransport consortium.

      * On board gigabit ethernet.
      Huh? Such things exist in the x86 world as well.

      But they usually aren't standard on the motherboard.

      AltiVec cannot be used; it can only perform 32 bit floating point calculations which is not legal for the Linpack benchmark used at the top500 site.

      Wonderful. Once you've gotten your "top 500" rating you can turn it back on and start spanking other clusters. Next?

      The Apple G5 has far more fans than your typical x86 box

      Which was done to reduce the amount of noise, not because the 970 puts out enormous amounts of heat.

      Oh yeah -- they can't strike out the cost of buying a copy of MacOS for each machine, can they?

      Hmm, lets think about this for a second. They're buying over a thousand brand new machines and giving Apple some great PR. Do you really think that Apple wouldn't come down on the price of the included operating system?

      Any more jerky questions?

    12. Re:Macs ? by jcr · · Score: 3, Informative

      Altivec can only do single-precision floating point.

      Not quite correct. Apple has extended-precision libraries available for Altivec.

      -jcr

      --
      The only title of honor that a tyrant can grant is "Enemy of the State."
    13. Re:Macs ? by jcr · · Score: 2, Informative

      Go look up the altivec instruction set and tell me which instructions work on packed double-precision values.

      As I said, Apple's published extended-precision libraries that use Altivec. You can indeed use altivec for double precision operations, you just have to use multiple passes.

      You can do double precision on PPC, but you don't get anything from the vector unit. Only the FPUs.

      You are mistaken.

      -jcr

      --
      The only title of honor that a tyrant can grant is "Enemy of the State."
  12. Actually they're not that mum by Nefrayu · · Score: 5, Informative

    I got the following email the other day:
    Virginia Tech is in the process of building a Terascale Computing Cluster which will be housed in the Andrews Information Systems Building (AISB). For those who are interested in learning more about this project, we will host an information session on Thursday, September 4th from 11 a.m. to noon in the Donaldson Brown Hotel and Conference Center auditorium.
    We look forward to seeing you there
    Terry Herdman Director of Research Computing.

    I'll try to remember to take notes on this and let you all know if there's anything interesting...

    --
    Friends help you move. Real friends help you move bodies.
  13. Not fast enough by cperciva · · Score: 2, Informative

    By my count, they'll have an R_peak of 8800 GFLOPS; unless they've got more efficient linpack code than anyone else, that will put them around 7th or 8th place.

    --
    Tarsnap: Online backups for the truly paranoid
    1. Re:Not fast enough by bspath1 · · Score: 5, Informative
      Reposting my AC post:
      You left out a factor of 2. 2 processors * 1100 * 4 FLOP * 2 GHz = 17600 GFLOPS. Remember, the 970 has 2 FPUs each capable of 2 FLOPs/ cycle.
      Leaving out a factor of 2 in this case significantly alters the R_peak value. -Bruce.
  14. Not! by Bones3D_mac · · Score: 2, Interesting

    Of all the Macintosh rumor websites, Think Secret is, by far, one of the most reliable sites I've seen. If it wasn't, I wouldn't be investing $1,200 a year in them for their message board. Of course, if you think you can do better... ;-)

    --


    8==8 Bones 8==8
  15. Talk of it all over campus? by Control-Z · · Score: 5, Funny
    there's talk of it all over the campus

    Yeah, chicks dig massive...computers.

    No wait, no they don't!

  16. Memory by rf0 · · Score: 2, Informative

    One thing against clusters rather than machines designed for the ground up is memory access. If on a n Single System Image (SSI) system is that any node can access memory of another over fast internconnects. With a cluster the memory has to be transfered over ethernet which even if using 10GB Ethernet is still a number of magnitudes lower than memory

    Rus

    --
    Cheap UK and US VPS
  17. Sounds familiar... by jettoblack · · Score: 2, Interesting

    Right after the Sony Playstation 2 launch, there was a big shortage. Several media stories blamed it on some "unnamed" Middle East country buying them all up to power their missles and supercomputers (because, the rumor claimed, the PS2 was just so powerful).

    Wonder if Apple is trying to "pull a Sony" here...

  18. What operating system will they be using? by Sonicboom · · Score: 3, Interesting


    The article makes no mention of the operating system that will be running on this supercomputer. I for one would like to see them get this done w/ OS X rather than use GNU/Linux.

    --
    [Connection closed by foreign host]
  19. I get to help build it =D by Anonymous Coward · · Score: 2, Informative
    In my email the other day, I received this letter:
    Hello all,

    This email is to serve as invitation and notice of impending Terascale Facility assembly assistance. For those receiving this info for the first time know that Virginia Tech is building a top 10 supercomputer from scratch and we need your assistance. We do have one stipulation to volunteerism and that is you must not be a wage employee of the university. Grad students on GTA/GRA are fine as well as others outside the university that may wish to volunteer.

    We are expecting to receive machines next week!!! Yikes! In preparation for the assembly process, we need to get volunteers together at the AISB (Andrews Information Systems Building), 1700 Pratt Dr., this weekend. We are planning to have a process orientation session start at 10:00 AM on Saturday, August 30, and last no longer than an hour. We can give you a few more details about the project if you show up and have not been before.

    There are many things that need to be covered and many new volunteers needed. We have posted an electronic sign-up sheet for proposed shifts at (link deleted) We will need folks to sign-up as either a primary volunteer or on-call/backup person that we can call and bring in if we are short people. We know this is a very busy time for everyone and we want to get this done and over with quickly so it will not affect other work that needs to be done across campus. Once we have a definite date for the deliveries we will send out notification to those folks that said they were available on that day. We will have 48 hours notice of shipment, 72 hours notice of delivery. The machines will arrive on a staggered, every other day, schedule. Three shipments are expected for the total number of machines.

    Orientation today was postponed, however, so I won't have more details until Wednesday =/ I'm looking forward to helping out, though.
  20. Getting a bit ahead of ourselves by Richard+Mills · · Score: 2, Insightful

    "If it manages to complete the cluster before the cut-off date, it will score a Top 5 rank in the Linpack Top 500 Supercomputer List."

    Err... I think somebody's getting a bit ahead of themselves here. =) Building parallel computing systems is complicated, and it may end up being quite a bit harder to realize the predicted performance than thought (not an uncommon occurrence). I'll believe it when they have the actual Linpack numbers.

  21. Top 5? I dont think so by rawgod0122 · · Score: 3, Interesting

    Currently the top 5 consist of 4 machines that have a Therotical maximum speed (Rmax) the are larger then the 10TFLOPS this machine will have. Then you have to translate that into peak speed which is what matter and what this list uses to rank the machines. Peak will be a good deal less, but this mostly has to do with the way the systems are interconnected and not the machines themselves. Say what you may about the G5 but the interconnect is more important.

    There is only one machine in the top 5 that this cluster could beat. The rest of the world has had 6 months to build machines too.

    This should be a top 10 machine for sure. Good to see more fast machines being built every day.

  22. Re:Do they have a need for it? by harks · · Score: 2, Informative

    Schools have many different accounts set up to fund many different things. This is due to how donors donate money and specify that they want it to go toward a certian project or department. One department, say the CS department might have recieved donations from CS alumni. Also, having large projects like this can generate lots of revenue through grants.

  23. The Altivec stuff is the key, I'll bet. by Above · · Score: 4, Interesting

    Several years ago I did some work on some Virginia Tech "supercomputers" (actually, baby versions of ones on campus that were the same as huge ones they leased time on elsewhere), and I think the people talking about Altivec are on track. I never knew exactly what they did, but at that time the Math, CS, and Engineering groups were working together to simulate wing designs for the YF-22 jet figher prototype. Since I was more of a "sysadmin" (althoug h with a math and CS background) I ignored most of what was going on, but one thing I can tell you was vectors, vectors, and more vectors. The vector is king. It's an assumption, but I'll bet they are still working on similar type studies, and if built, this will be just the beast for it.

  24. Re:Do they have a need for it? by absoluthokie · · Score: 4, Informative

    Exactly, Virginia tech has a goal to become a top 30 research university. Having known about the plan for some time, this makes perfect sense. The departments who are building the cluster, have gotten very large grants and donations from our great alumni to build this, and become a better university for it. This construction can be compared to the stadium expansions. The stadium expansion is paid out of a different set of funds, as is research. Academic fund is hurting because alumni rarely give money for academic reasons, but more for football or research.

  25. Fucking A by waldoj · · Score: 3, Funny

    Damn, I've been at this school for a week and I haven't found a single redeeming value. Finally, a cause to hang in there for the next couple of years.

    -Waldo Jaquith

  26. AltiVec by charnov · · Score: 3, Interesting

    While the AltiVec unit is very impressive, The SSE2 unit on the P4 or the Opteron would have nearly the same performance and cost a whole heck of a lot less (I am betting if this rumor is true at all, then Apple has given the units to the school).

    What I am wondering is, what OS is this cluster going to run? I mean, have the BSD folks figured out how to scale? No chance it will be OS X...maybe AIX?

    --
    [RIAA] says its concern is artists. That's true, in just the sense that a cattle rancher is concerned about its cattle.
    1. Re:AltiVec by Space+Coyote · · Score: 4, Informative

      While the AltiVec unit is very impressive, The SSE2 unit on the P4 or the Opteron would have nearly the same performance and cost a whole heck of a lot less (I am betting if this rumor is true at all, then Apple has given the units to the school).

      Real world numbers don't bear this out. Check out the Photoshop and other application performance numbers for this. The gcc version used by the SPEC benchmarks used by Apple didn't even take advantage of AltiVec. When accounted for, and any institution making such a purchase would definitely have considered this, the AltiVec-enabled PowerPC chips totally spank x86 and others in number crunching tasks.

      What I am wondering is, what OS is this cluster going to run? I mean, have the BSD folks figured out how to scale? No chance it will be OS X...maybe AIX?

      An OS doesn't need to 'scale' to be a member of a cluster. It just needs to run the code locally and send the result back to the cluster master node.

      --
      ___
      Cogito cogito, ergo cogito sum.
    2. Re:AltiVec by discstickers · · Score: 5, Insightful

      The SSE2 unit on the P4 or the Opteron would have nearly the same performance and cost a whole heck of a lot less.

      Uh, no. 2 years ago, my roommate and I were both running the distributed.net client. I have a 500 Mhz Powerbook G4 (100Mhz bus). He had a 1.4GHz P4 with rambus RAM. I got 4Million keys/sec. He got 2MKeys/sec.

      So clock for clock, my machine was nearly 4 times faster.

      --
      I have a shitty sig!
    3. Re:AltiVec by JDWTopGuy · · Score: 4, Informative

      No kidding. My 667Mhz Powerbook G4 gets 6.8 MKps, my friend tried it on his 2.53Ghz P4 and got 3.4 MKps, and my 1.4Ghz Athlon XP 1600+ gets aroung 4.5 MKps. I calculate a Dual 2Ghz G5 around 40 MKps, probably more. Imagine them running distributed.net on this bad*** cluster!!!! YOW baby!

      --
      Ron Paul 2012
    4. Re:AltiVec by sevenofnine · · Score: 2, Insightful

      Yeah right, so what your saying is that they should build their own cluster by putting together 1100 units of grey boxes?
      In your "calculations" i think you forgot to add the added manhours for doing this...
      and thats not even thinking about the order / failure rate...
      buy mem for 1100 machines is bound to have 1, 2 or more (properbly the ladder) mem blocks not working...
      And im sure there is alot more reasons to this that you ( and i) missed...

  27. Hoax by Anonymous Coward · · Score: 2, Insightful

    The cluster has nothing to do with Apple. They were contemplating using PPC970 CPUs (basically becasue they're cheaper than the Power4's), but AFAIK there were never plans to use complete G5 computers, or anything else made by Apple. And I doubt IBM would take CPUs from Apple to give them to V.Tech. The delay is mot likely due to performance issues; they're probably fine-tuning the OS so it's competitive with x86 / Windows systems.

  28. Re:Do they have a need for it? by Kibo · · Score: 5, Interesting

    The grant money that flows into a public research and occasionally teaching institution can be stagering, and absolutely dwarf the money students pay in tuition (sometimes by a factor of 10!). A better question might be, why don't the gradstudents donating their labor, possibly to patents that will be controlled by the university, recieve more consideration, and fair labor law protections.

    But I would bet this will be not too dissimilar in use from the HP Itanium2 referenced earlier on slashdot. I would bet one of the paramount concerns this cluster would look at is the effect of farm runoff, and probably climatology too among other things.

    --
    --Jimmy has fancy plans; and pants to match.
  29. XServe? by diesel66 · · Score: 2, Funny

    This won't help sell their clustering hardware:

    http://www.apple.com/server/clustering.html

    --



    eleven plus two / twelve plus one
  30. Another brilliant idea, inspired by Apple by Anonymous Coward · · Score: 5, Interesting

    With 1100 machines in the cluster, there must be _at least_ 2200 DIMMs. Since these must be 400MHz (PC3200) DDR, they can't be on a large 0.15 micron DRAM process, but most likely between 0.11 and 0.13u.

    Who cares?

    APPLE G5'S DO NOT SUPPORT ECC.

    The random bit error rate for 2200 DIMMs with 0.13u cells is roughly one '1' bit dropped to '0' every 9 hours. In other words: good luck getting any reliable, large-scale computation done with this cluster. (And I do mean "good luck" - they might get a run of two or three days without any problems once in a while.)

    Now if only Apple would support PC3200 ECC DIMMS, which certainly do exist:

    http://www.intel.com/technology/memory/ddr/valid /d imm_results.htm

    this cluster might be a bit more useful for real work.

    1. Re:Another brilliant idea, inspired by Apple by dbirchall · · Score: 4, Insightful
      With 1,100 machines in the cluster, you'll probably be running into something's mean time between failures pretty darn often, whether it's memory getting a bit wrong, or one of the 9,900(!) fans needing replacing. :)

      But... a cluster should be redundant enough to withstand that sort of minor inconvenience and go on functioning without the errant node while it gets fixed, reboots or whatever.

      I'll admit that building something smart enough to say "Node 206, you have a memory error. Bad G5, no donut!" is beyond the scope of my understanding.

  31. Re:Poor choice on Apple's part by Glonoinha · · Score: 4, Funny

    -I'm about to buy another in a day or two...

    You mean you are about to order another in a day or two ... there is a pretty big gap between asking for one and actually getting one. Tell you what, let me finish this gig I got happening in Virginia and then we can start dealing with the customers that want to order G5 machines in the onsies - twosies quantities.

    Sincerely,
    Steve.
    sjobs@apple.com

    PS - Seven Mac's in ten years isn't hardcore. 1100 units ordered on one purchase order before they even ship (August, 2003) ... now THAT is hardcore.

    --
    Glonoinha the MebiByte Slayer
  32. Re:MOD PARENT DOWN by class_A · · Score: 3, Insightful

    ooops, yuo = own3d!!!!
    http://n0cgi.distributed.net/speed/query.php?cputy pe=99&cpumhz=1000&recordid=1&contest=rc572&multi=0
    Power PC 7450/7455 G4 1000 MacOS X 10.2 2.9005 RC5-72 10,594,666.00

  33. Won't someone please think of the social sciences? by Decaffeinated+Jedi · · Score: 2, Funny
    Ah... now all the budget cuts to Virginia Tech's Department of Political Science make sense. ;)

    DecafJedi

    --
    DecafJedi
    my weblog: apropos of something
  34. Re:MOD PARENT DOWN by ZackSchil · · Score: 2, Funny

    That first column says "Report as Error" It's a link to report the number as an error if it is unusually high compared to its neighbors, not to state that the actual figure is an error. Click the link, it will explain the system to you. I must be off because being a homosexual, a Macintosh user, and a priest takes a lot of my time! At least the slashdot trolls seem to think so! Toodles!!

  35. so no G5 Xserves soon? by johnpaul191 · · Score: 5, Interesting

    i would take this story to imply that a G5 powered Xserve is not going to be shipping anytime soon..... the Xserve is made to cluster and run in situations like this. i guess the rumor sites can speculate if it's G5 parts available or some other holdup on a G5 Xserve.
    unless there is some reason the desktops are better for this project that i did not pick up on?

    as for the above question about Macs.... depending on what they want to really do with this, Altivec is really efficient for some computations. all flame wars aside there have always been people clustering Macs for certain uses. i do not know how much of it was user preference or the software they wanted to run or the simplicity of getting the cluster running.
    it is supposedly VERY simple to cluster Macs. there was a story on /. a year or so ago about a group that went from building a rack and unboxing their G4s to a running cluster in part of a day. i really don't remember the specifics but i think it was something like 30 G4s? i would guess the G5 is not that much harder... and they seem to have Apple helping. maybe they hooked up the optical cards from the Xserve...... we'll see i guess.

    1. Re:so no G5 Xserves soon? by jo_ham · · Score: 3, Interesting

      Have you seen the size of the heat sinks on the G5? I saw one in an Apple store today and was very impressed with the engineering of the whole machine.

      The heatsink is a large oblong about 5"x4"x6" with a thin grille like construction. It's just too big to go in the 1U Xserve. Give them some time to work on designing it to fit though. The G5 is an ideal CPU for the Xserve as you say.

  36. Bragging Rights by Michael_Burton · · Score: 4, Funny

    I've been tempted to order a dual G5. I've resisted the temptation by realizing that my only real reason for wanting it would be to awe friends and co-workers. Pretty shallow. I was ashamed.

    What a surprise to find that the folks who buy multi-million dollar supercomputers seek some of the same shallow satisfaction that moves me--bragging rights.

    Still, if a single order for 1100 units causes significant delays filling orders for other customers, Apple must not have been expecting to sell many of these things. Maybe I should place an order just to help out.

    --
    When all you have is an axe, everything looks like a grindstone.
  37. It's true. by Mekabyte · · Score: 2, Funny

    For those unbelievers, here's a little proof and maybe a bit more

    Here's an official word (search for Teraflop).

    Also, here's the original e-mail that went out (a month ago) They never mentioned Apple though:

    > Date: Mon, 28 Jul 2003 17:36:46 -0400
    > From: Jason Lockhart <multimedia@vt.edu>
    > Subject: Terascale Assembly Assistance
    >
    > Hello all,
    >
    > As you may know the College of Engineering in conjunction with the
    > university Information Systems and Computing organization are
    > building
    > a 10 TeraOp, 1100 node supercomputing cluster. We are in need of
    > volunteers to assist in three areas of assembly; cabling, RAM and PCI
    > card install, and machine racking. We would like to get as many
    > volunteers as possible.
    >
    > Some logistical things, there is not adequate parking for every
    > volunteer to drive and park at the Computing Center. We
    > would ask that
    > volunteers either carpool (4 or more to a vehicle) or take the BT.
    > Initial cabling will be done with the power cables beginning this
    > coming Friday, August 1st. We have yet to set a start time,
    > but I want
    > to get an idea of who is available to assist on that day as well as
    > availability for the weekend and early next week. I will forward the
    > time to arrive as soon as I nail it down...should be tomorrow. The
    > power cabling represents one third of the overall cabling that will
    > need to be done.
    >
    > I need to get a list of those who will be assisting to the facilities
    > people at the ISB as soon as possible. This list will be
    > used to allow
    > volunteers access to the building and the machine room. If
    > you're not
    > on the list you will not be allowed in! We want to have at
    > least 30 to
    > 40 people working at any one time on the project. If you know others
    > that will be interested in assisting, please have them email me so I
    > can get them on the roster.
    >
    > Thank you for your willingness to participate, and please respond and
    > have others respond with "Terascale" in the subject so I can
    > filter the
    > volunteers properly.
    >
    > Thanks.
    (E-mail signature removed)

  38. Think Secret's Record by Nick+dePlume · · Score: 5, Informative

    As Zack pointed out, iWalk was not a Think Secret report; in fact, we debunked it. For WWDC, we reported that Apple would announce 64-bit Power Macs as well as a videoconferencing camera that we said would be called "iSight," -- I think we're in the clear there. iWorks? I maintain that it is still a future Apple release. As for 12-inch and 17-inch PowerBooks, while we raised the possibility of a release that week, we specifically said we couldn't confirm the delivery date: "It's unclear when Apple plans to announce the upgrades..."

    Bottom line? Like any other news organization, Think Secret has occasional misses. But those misses don't appear to include any of the items mentioned here. I think our record speaks for itself.

    Nick dePlume
    Publisher and Editor in Chief
    Think Secret

    1. Re:Think Secret's Record by Nick+dePlume · · Score: 2, Insightful

      I'd argue that it's solid in that regard as well.

      I believe we're honest and up-front about the reliability of what we report. If something has not been confirmed by multiple reliable sources, we say so. If something is mere rumor, we also say so, but most of the time, we would rather not report it if we can't confirm it.

      As for mistakes, after Macworld Expo, for example, we typically take a look at the announcements and compare them to our pre-expo reporting to see what, if anything, we got wrong. We don't hide anything.

      Nick dePlume
      Publisher and Editor in Chief
      Think Secret

  39. Talk about a ton of desktops in a server room by Kalak · · Score: 4, Informative

    For the ones who are questioning this existence, the order is shipping, the racks (a ton of them) are there in the main Computing Center server room. First they required all servers to be moved innto racks. Then they started moving servers around, including removing the Petaplex. The power has been upgraded in the server room (the UPS backup generator actually). This caused a morning of basically all the important servers on campus having to go down for one day in the summer - I hated waking up to go switch off machines for that one. The AC has been upgraded to accomidate the huge amount of heat to be put out. It was't until I heard about the cluster that all the chages in the Machine Room made sense. Now they're recruiting help to do the grunt work of putting all the machines in the racks.

    The stated objective was to be on the next 500 list. Dell and HP were considered, but they couldn't fill the order in time (possibly as they have made announcements of other large clusters recently) and Apple promised delivery after someone leaked the story of the cluster meetign with Dell and HP to Apple and Apple jumped at the chance.

    Basically, the story is not a rumor from the point of view of the geeks on campus who have been effected by the preperations. I'll probably post the /. link to the campus geek list (If someone hasn't beaten me to it).

    I'm disapointed about this being only on the Apple section of /. since a cluster this size is noteworthy of the frontpage. (Rumor - and this is rumor sice I haven't goe to direct sources on this - is that it will not be running OS X, and probably BlackLab or YellowDog or SuSE.)

    --
    I am, and always will be, an idiot. Karma: Coma (mostly effected by .hack)
    1. Re:Talk about a ton of desktops in a server room by valdis · · Score: 3, Informative

      Actually, this summer's outage for the new diesel backup generator was something else entirely - that was merely replacing two older natural-gas fired generators that were no longer sufficient to fully back up all the existing hardware. That install has been in the planning stages for a long time, and was needed for current operations.

      Do the math - the new generator is rated at 600kva and is already carrying several hundred machines (including a very power-hungry Sun E10K and a number of E6K-class machines). There's not enough capacity on that generator for 1,100 more systems.

      (And I just wanted to say "This is All Kevin's Fault" - except for a few unrelated parts we blame on Randy ;)

    2. Re:Talk about a ton of desktops in a server room by valdis · · Score: 2, Insightful

      Exposure to the facilities? All I'll say there is that I know of at least 4 people who have posted to this story that could throw a paper airplane from their cubicle and have a reasonable chance of landing it on the blueprints. ;)

      I said there wasn't capacity on that generator. I didn't say anything about the existence of other generators - and there's a distinction between pulling copper to get power grid capacity to the cluster and having emergency power backup for same. That diesel is for emergency backup, not local power generation - we've got a nice coal-fired beast on campus for that.

      In addition, you have to remember that the cluster most likely has different backup power requirements than our production systems. Our production systems (the central mail server, the large database machines, etc) really need to stay up - we're talking about things that if the power substation near the airport goes out, the whole campus is screwed till it comes back online, if we don't have backup power.

      Look at how many services were out when we did the cut-over - everything was affected, from E-mail to the library catalog. That's what emergency power is for - so you don't have an event like that without the multi-week notice that the install had.

      On the other hand, does a compute cluster really need that level of power backup, or can we significantly trim the budget by merely having a really good power conditioner to make sure there's nice clean non-ripply power, and enough battery backup to allow the cluster to do a suspend-to-disk and poweroff cleanly? Yes, if we take a power hit in that scenario, some researchers have to wait for their results - but the business of the university as a whole isn't disrupted.

      Or possibly the plan is to have the conditioner and batteries for now, and have someplace to cable in a backup diesel generator in the future - remember that 1,100 G5s, a bunch of Infiniband switches are a chunk of change - and then you get the expense of the power/cooling work. Suddenly you start thinking about how to put the non-essential costs into the next fiscal year. ;)

      (The above is not intended as an actual statement of the actual design or plans, just an illustrative discussion of the fact that there are a lot of tradeoffs that need to be made when planning and designing large-scale installations of any sort).

  40. Re:Poor choice on Apple's part by runenfool · · Score: 4, Insightful

    You got a link for dual Opterons for 1500 bucks with all the goodies in the G5? If yes, Id love to see it.

    (seriously - I can take out a loan :) )

  41. Re:Do they have a need for it? by KiahZero · · Score: 4, Informative

    Obviously you haven't looked at VT recently. Tuition and fees is only $7,500, out of state. I can only wish that my tuition were that low. Hell, for in-state students, the room and board is the same price as tuition (around $2,000). But of course, you're modded up insightful, because you pulled a random idea out of your ass and presented it as fact.

    --
    I'm a lawyer, but not yours. I wouldn't represent someone who thinks taking legal advice from Slashdot is a good idea.
  42. 1% of G5 orders by Viadd · · Score: 4, Insightful
    Just for perspective, there have been over 100,000 G5s ordered, so this cluster is about one percent of the backlog. In other words, assuming that Apple ships all pending orders in about a month, the G5 I ordered will be delayed by about 8 hours.

    And 8 hours@12.4GFlops...damn you Virginia Tech, you owe me a third of a quadrilion floating point multiplies!

  43. 1100 machines is ~1.1% of the placed orders by vnv · · Score: 2, Interesting

    According to Apple, there were "over 100,000" pre-orders for the G5. Now this includes single processor models, but the university's alleged order of 1100 machines is not going to make a big impact on everyone else.

    Besides, the real reason that Apple's machines are late is case defects and AGP problems, amongst other issues that Apple has not been forthright about. At the keynote an honest Apple employee told me the machines wouldn't ship until October as there were many little problems and I should wait for the January refresh so I don't get a flaky machine.

    And one has to wonder why anyone building a cluster would build it using desktop machines and not use the forthcoming G5 rackmount machines from Apple and IBM... which is supposed to include a quad-processor from IBM.

    1. Re:1100 machines is ~1.1% of the placed orders by vnv · · Score: 2, Interesting
      That still doesn't change the fact that Apple apparently delayed tens of thousands of G5 shipments for a month, making other companies and individuals all wait just to please this one university. If I had lost my place in the queue, I wouldn't be happy having to wait longer for a machine I likely ordered three months ago.

      Now what maybe true is that because these machines are going into a cluster, they don't care about cosmetic problems with a patched case door and they don't care if AGP 8X doesn't work right for high-performance 3D cards. So Apple could be dumping 1100 "good for clusters" machines on the university while waiting for an inline revision of the first batch of G5's. That makes more sense.

      And you know... I'm quite glad I took an honest Apple employee's advice and don't have to worry about this stuff. Come January (Feburary is what I put on the calendar), I'll have a G5 with some of the bugs worked out and for less money.

  44. G5 Vs. Itanium2 and Opteron: Some perspectives by Anonymous Coward · · Score: 5, Informative

    1. The PPC970 draws from the Power4 lineage, which I have used for a long time. The PPC970 has 2 double precision FPUs, each capable of fused multiply add instructions leading to 4 flops/cycle/processor (2 units*2flops/cycle). This is identical to the Itanium2 FPU microarchitecture. The Opteron on the other hand can only do 2 double precision flops/cycle, which makes it only half as powerful on matrix heavy scientific computations, when compared to the PPC970 or the Itanium 2. The PPC970 should really be compared in FP terms to the Itanium2 at 1/10th of the cost, and at 2GHz it is clocked higher than the top-end 1.5GHz Itanium2 Madison. Moral of the story, read thy arstechnica. 2. The standard benchmarking process (LINPACK) only uses double precision FP. If this rumor is true, then this machine is capable of an Rpeak (LINPACK) of 17.6 Teraflop, which those of you who follow top500 will realize is quite substantial. 3. If they are really using Infiniband, this should be a nice machine. Infiniband provides 10 Gbps (20 Gbps full duplex) of bandwidth, which is much faster than either Myrinet or Quadrics. Also Infiniband latency is 10us and the benchmarking process is bandwidth not latency sensitive. On the other, this stuff is really expensive. If all of this is true, this would be a major engineering endeavor. Also, it is probably cheap. However, all in all, this could well just be a rumor (come on it is thinksecret - remember iWorks). If not, this should be a fairly substantial machine.

  45. SGI Origin 3000, 1024 processors... by green+pizza · · Score: 4, Interesting

    It's just too darn hard to make a shared memory computer with 1000's of processors. So the common architecture is to make a cluster of smaller shared memory machines.

    It's hard, but not too hard or impossible. The Silicon Graphics Origin 3000 supports 512 processors in a single image system with the stock IRIX kernel and 1024 processors with the "XXL" kernel.

    Rumor has it Origin 4000 will support 2048 processors, as will Altix once SGI has done some major work with their kernel patches. (Altix is currently limited to 64 processors per system image).

  46. Why G5's? I helped set up the racks by Anonymous Coward · · Score: 3, Informative

    They went with the G5's because they were the cheapest 64 bit solution and because they would use less power and generate less heat than alternative systems. That is the whole of it.

    They are having someone write infinaban drivers for OS X just for this cluster.

    I look forward to helping install 4GB of ram + the infinaban cards in each of these bad boys.

    It is great having connections!

  47. Re:Poor choice on Apple's part by benh57 · · Score: 2, Insightful

    And that opteron system will use 3x the power and heat. Universities don't spend millions on a 'non efficient choice'. Smart people did the math, and they had facts - which you don't.

  48. Re:I, for one... by scrod · · Score: 4, Funny
    ... welcome our... oh wait... I hate MACs...

    Damn, ethernet controllers must really piss you off then, huh?
  49. Bragging rights... by ca1v1n · · Score: 2, Funny

    As long as they put it on Legion, U.Va. will get to maintain state bragging rights. I don't keep up with the football rivalry, but this is much cooler anyway.

    --
    WARNING: there is a trojan on your
  50. Re:a Hokie alumn speaks by stingerman101 · · Score: 2, Informative

    If the money granted was earmarked for a specific project their is nothing the university can do to transfer it to another need. It is either they use it or lose it. Fortunately these grants usually include monies for maintenance of the systems and the staff required, which means that personnel can be put under this new grant area and alleviate costs from another area. So in the end, grants like this help other areas of the school and will attract and create other sources of revenue.

  51. Re:Do they have a need for it? by steve_bryan · · Score: 2, Interesting

    So how does this anti-intellectual tripe qualify as insightful? Any yahoo can point at and complain about just about any non-trivial project at a research university whether it is public or private. If they were building it just to attain a certain ranking without any research proposals or plans it wouldn't be hard to find fault. Does anyone that could possibly be the case here? I think this sort of empty headed bushwhacking is a cheap shot and contemptible.

    Is there something particularly about building any clusters today that is ill advised? Anything specifically about a cluster built with these parts? Why do any science that involves a large expense when the money could be applied to "lowering tuition"? Maybe because an important part of the mission of some universities is to advance the state of knowledge by performing research that would not be done by other segments of society.

  52. But, what will they *do* with it? by chiph · · Score: 2, Insightful

    Sure, having a top-5 supercomputer is cool, and the bragging rights will garner the school some additional funding and scholarships, but at the end of the day after the benchmarks are run and the empty Jolt(tm) cans are recycled, what will they *do* with it?

    Chip H.

  53. Re:AMD x86/64 by stingerman101 · · Score: 2, Informative

    Well the only AMD alternative is the 2GHz Opteron whose processor alone costs about 900/processor. You won't be able to get a dual 2GHz Opteron for less than Apple's retail dual 2GHz G5. Too, you have to make sure PCI-X slots are there for the interconnect cards they are using. But as was stated elsewhere, the G5's can process twice the scalar FP instructions of the Opteron or the Itanium per cycle. The University as also relies on vector math and the G5's have a significantly better SIMD unit.