Slashdot Mirror

← Back to Stories (view on slashdot.org)

A Look Inside Virginia Tech's New Super Computer

Posted by Cliff on from the taking-a-peek dept.

Mr Bob "The original" bougert brings us "...a video of the Virginia Tech super computer centre. How many people think that super computer centres like this, with their reasonably cheap cost should be created in more places? This video of the infamous super computer should be interesting to some and pretty to look for others." It views like an ad for Apple, but Virginia Tech has scored quite an achievement with this milestone, and this should serve as a decent introduction for those unfamiliar with the project.

22 of 420 comments (clear)

  1. Imagine... by Zelet · · Score: 5, Interesting

    ...what they could do now for the same cost using the new Xserve dual 2 Ghz G5.

    Dual 2GHz PowerPC G5
    512K L2 cache/processor
    1GHz system bus/processor
    512B DDR400 ECC SDRAM
    80GB Serial ATA drive
    Dual Gigabit Ethernet

    All for only $3000. They could really built a small, inexpensive cluster with a couple thousand of those.

    --
    ...And when they came for me, there was no one left to speak out for me." - Martin Niemoeller (1892-1984)
    1. Re:Imagine... by bedmison · · Score: 5, Interesting
      The rumor here on campus is that Apple is going to let them trade the G5 towers for G5 XServes, 1:1. So if you are wondering where the first 1100 XServes are going, look no further than Blacksburg.

      There is some question as to whether they are going install more nodes than the 1100 they have. Given that there are 96 52-U racks currently housing the 1100 towers, they will have plenty of space for more XServes. We did get an email saying the info systems building will be without power over the coming weekend as new power lines are installed...Coincidence? I think not.

    2. Re:Imagine... by be-fan · · Score: 2, Interesting

      Actually, they couldn't do as much. They'd have to throw another 4GB of RAM in there, as well as buy the expensive Mellanox Infiniband interconnect. To get the same total price as the VaTech cluster, they'd have to get each machine at around $2500.

      --
      A deep unwavering belief is a sure sign you're missing something...
    3. Re:Imagine... by stang7423 · · Score: 2, Interesting
      I am wondering just what the real market is for the XServe.
      That market is me. We just got one of the older xserves where I work. It is a college newspaper that employs a couple hundred students. OS X Server and our new Xserve allows us to lock down our machines to keep the students from doing all the destructive things that they tend to do.

      I just spent 1(one) day setting up our xserve and getting open directory setup. Now we have network logins and have locked down all our workstations so our reporters can't break our machines by installing AOL, MSN or any other unauthorized app, or leech our bandwidth by installing limewire, etc... Sure I could have done all of this on a Linux/BSD box as a server but i don't have that kind of time. The research and installation alone would have taken up all of my christmas break (I'm a student too). With the Xserve I was able to get a bundled product that worked pretty much out of the box. For that i'm willing to pay the premium. Plus it looks so much nice then any of the other white box servers we have in the building :-)

      Now don't get me wrong I like linux just as much as the next slashdot reader (we have 4 linux servers in the building), but I have never setup a linux box in the short period of time I was able to set up the xserve.
    4. Re:Imagine... by Doc+Tagle · · Score: 2, Interesting

      Va Tech invested a tremendous amount in developing the air conditioning system for 1100 G5's, not to mention the racks and Inifiniband architecture.

      I find it had to believe that they would tear it down after only six months of use.

      Of course if they took out all of the the G5's They would be left with 92 racks (1100 G5's @ 12/rack) They could fill those racke with ~3850 G5 XServes. At 7700 processors they might hit the 30+ Teraflops to knock the earth simulator out of #1 :)

  2. The most telling statistic for me by dgrgich · · Score: 5, Interesting

    I was amazed at the cost/performance ratio that they were able to achieve with Big Mac. Over at Barefeats.com, they point out that a Dual 2ghz G5 is roughly 17% faster AND more expensive than a Dual 1.8 G5 - keeping the cost/performance ratio fairly equal. Taking this out to supercomputer levels, the #1 supercomputer is three and a half times faster than Big Mac but cost 60x as much money!!! Amazing.

  3. Pricing by TechnoWeeniePas · · Score: 5, Interesting

    The video states that the top two cost in the hundreds of millions to build...but never says how long ago. Dont get me wrong the Mac cluster is quite impressive and inexpensive but the price to power ratio has been changing quite rapidly just in the last few months! So if you rebuilt the top two today how would they rank pricewise?

    1. Re:Pricing by lquam · · Score: 5, Interesting

      The Earth Simulator (#1) and Asci Q (#2) were both completed in 2002, although I know planning on the Earth Simulator goes back to the mid-90s. No idea on when Asci Q was planned, but it's 8192 1.25GHz Alphas (SC45 servers) which is current technology for the Alpha line. But with TES you're talking about something that's nearly two orders of magnitude more expensive than VT's X. If you could build it today for the same price or perhaps 75-80% and get another 10-20% performance out of it it would still be way more expensive on a $ per GFlop basis.

      More interestingly, #4 on the list in the NCSA's Tungsten with 2500 3Ghz P4s. It's about 15% slower with 300 more desktop procs than X and was also made operational in '03. I suppose if they were to run around plugging 3.2 GHz processors into their 1250 Dell boxes one could perhaps sneek up on X, but you'd likely have to wait for the 4 GHz P4e to actually steam past it.

      Basically, the supercomputers which were completed most recently ARE the ones at the top of the list. X just happens to be insanely cheap compared to the ones above it.

      Len Quam

  4. PCs to Crays by fdicostanzo · · Score: 3, Interesting

    Does anyone know the relative speeds of todays PCs vs. an old super computer from the 80s?

    UMass had one of those Connection Machines with the 65k processes and the blinking lights sitting unused in the basement for awhile and I was always curious to know whether it was any faster than what could be done serially with a 3GHz PC.

    --
    Synergies are basically awesome, and they're even better when you leverage them. -PA
    1. Re:PCs to Crays by flaming-opus · · Score: 3, Interesting

      The connection machines never were real speed deamons in their day. They were built to be used for AI codes (lots of one-bit integer ALU's), but AI groups don't have any money so they re-tooled it to do floating-point math, but they really only sold them with the help of DARPA subsidies.

      http://www.cray.com/company/history.html

      As for the old crays, you probably don't want any of those from the 80's. Even the New cray X1 processor's have a theoretical peak of 12.8 Ghz, a little less than twice the G5. But it's important to remember that this doesn't tell the whole story. The X1 has 34 GBps/CPU memory bandwidth and 77 GBps/CPU to cache. It also has a 400GBps of interconnect bandwidth for a single cabinet. It's always been cheaper per flop to buy small computers and gang them together. It's not ALWAYS the best solution.

      What the big mac performs really well on are hugely parallel computations with few dependancies between each piece of the computation. (like Linpak for example.) When there is a lot of dependancies between peices of the computation, large shared-memory machines work much more efficiently. Thus a bunch of DOD and DOE labs (plus meteorological sites and boeing) are still interested in paying the premium for custom vector supercomputers.

  5. Re:Apple Hardware? by Aqua+OS+X · · Score: 2, Interesting

    Dell doesn't really develop a lot of software. Apple does.

    And, yes, Apple builds it's machines outside of the US as well.

    --
    "Things are more moderner than before- bigger, and yet smaller- it's computers-- San Dimas High School football RULES!"
  6. Re:Virginia Tech purchased those Macs at full pric by junkgoof · · Score: 1, Interesting

    Most hardware vendors (even big ones like Dell) are lucky to pull off a 10% markup. Apple used to get 40%, and I think they are still up there. It's depressingly obvious now, too, because they use the same nvidia/ATI cards, the IDE drives, same everything except for motherboard, chip, case, and firewire. At least when they still used SCSI and custom video cards you could sort of avoid seeing it.

    Apple made the decision long ago, and they chose the Sun model, not the M$ model. They could have licensed their OS and slugged it out for market share, but they chose to shoot for high margins and low penetration. It works, they make money. As long as they keep making cool stuff some people will spend the premium.

    --
    You got me into this! You were the ideologue! I'm only a poor assassin! - Twenty evocations, Bruce Sterling
  7. Cluster Software by thomas536 · · Score: 1, Interesting

    I have looked and looked, but can't seem to find any information about the software that they have developed to make these G5's into a cluster. I'm curious to know what software was developed, and who developed it, i.e. Apple or the internal VA Tech team. I assume that MPI has been around for FreeBSD/OSX for a while. There must be some type of process migration, scheduling, and load balancing. If anybody has any links to any white papers or otherwise anything similiar, I'd appreciate to see them.

    Thomas

  8. Re:Virginia Tech purchased those Macs at full pric by Wesley+Felter · · Score: 4, Interesting

    OS X server comes with an unlimited client license and you can put it on as many computers as you please.

    Nope. Unlimited clients mean that unlimited clients can connect to the server. You still have to have separate OS licenses for your client machines. But this has nothing to do with clusters that run regular OS X.

  9. Re:Virginia Tech purchased those Macs at full pric by djupedal · · Score: 2, Interesting

    The story goes that the person doing the project contacted Apple and they blew him off at first. He eventually purchased from the online Apple Store.

    I'm sure there was room for greasing each other's palms, but I think the price was so low to begin with that there was little point it argueing over a few hundred thousand, what with the deadline they were facing.

  10. Why trade 'em back to Apple by amichalo · · Score: 3, Interesting

    My first thought when I heard about the 1100 VT deal was that when g5 xServes are available, VT will replace the PowerMac cluster and distribute the 1100 over the campus.

    Any rumor of that?

    --
    I only came here to do two things; kick some ass, and drink some beer...looks like we're almost out of beer.
  11. Re:Apple Hardware? by JeffTL · · Score: 3, Interesting

    Apple builds outside, but they seem to do most of their R&D in America. Dell, in my experience, does just the opposite excepting their apparent comparatively low amount of R&D: they do their construction in Round Rock, but more or less stick together the innovations of others. The last thing Dell actually invented that I can recall is their business model. And new ways (e.g. flaps) for a computer to collect dust bunnies unbeknownst to the owner.

  12. Re:Virginia Tech purchased those Macs at full pric by Vaystrem · · Score: 5, Interesting

    "Apple doesn't place a giant markup on its products. They put a lot of money into product and industrial design. Therefore, Apple computer's cost more."
    Wrong. Anyone who has sold computers or has industry experience will probably let you in on a little secret: there is a tiny markup in computers. Period. Money is made in accessories and service plans.

    And for (hopefully the last time)Macintoshes are not more expensive! This point has been made many many times on Slashdot. But to make it one more time (IANAMU [I am not a Mac User]):

    $6,174.00
    Dual 2GHz PowerPC G5
    4GB DDR400 SDRAM (PC3200) - 4x1GB
    2x250GB Serial ATA - 7200rpm
    ATI Radeon 9800 Pro
    56k V.92 internal modem
    SuperDrive (DVD-R/CD-RW)
    Apple Keyboard & Apple Mouse - U.S. English
    Mac OS X - U.S. English

    $6,634
    Dell Precision Workstation 650
    2 xIntel(R) Xeon(TM) Processor,3.06GHz,512K Cache
    Microsoft(R) Windows(R) XP Professional
    Hyper-Threading feature preset to "ON."
    Memory: 4GB,DDR266 SDRAM Memory,NECC
    Keyboard: Entry Level, PS/2, No Hot Keys
    Mouse: PS/2,Dell, 2 button w/no scroll
    Monitor: No Monitor Option
    Graphics Cards: nVidia, Quadro NVS 280, 64MB, dual monitor VGA capable
    Speakers: Internal Chassis Speaker
    Productivity Software: Dell Precision Workstation
    4X DVD+RW/+R AND 16XDVD-ROM,DVD Decode/Sonic SE(for Professional Authoring) DRWDV4X
    2x250GB SATA, 7200 RPM Hard Drive with DataBurst Cache(TM) SARC RAID
    Floppy Drive: 3.5 inch 1.44MB Floppy Drive

    So what was that about macintoshes being more expensive?

    (Note I had to reformat the Dell Quote so that it would look ok)

  13. Re:Virginia Tech purchased those Macs at full pric by PopCulture · · Score: 2, Interesting

    DELL
    Intel(R) Celeron(R) Processor at 2.4GHz
    Microsoft(R) Windows(R) XP Home Edition
    256MB Shared DDR SDRAM at 333MHz
    40GB ATA/100 Value Hard Drive
    48x CD-ROM Drive
    DVD+RW Drive
    17" monitor $399 after $100 rebate

    Apple
    256MB SDRAM - 1 DIMM
    40GB Ultra ATA drive
    Keyboard/Mac OS - U.S. English
    Combo drive (DVD-ROM/CD-RW)
    1GHz PowerPC G4
    $850 after rebate (DELL x 2)

    welcome back my reality- where dropping 6 grand for a computer is simply not an option!

    --

    Here's to finally giving Bush his exit strategy in November
  14. The real star here is.....Infiniband!! by kap1 · · Score: 2, Interesting

    PC's, Apples, who cares. What made all this stuff possible, both technically and economically, is Infiniband. What is Infiniband? Well, think of all those special purpose, packet switched data buses and high speed networks of the past (mucho $$) and make them an industry standard (less $$) and you've got infiniband. Blows 10GB ethernet away.

    That's what all the HPC (high performance computing) guys have been using to glue their clusters together. See www.infinibandta.org for more info.

    Will Infiniband make it to a server near you? Only time and the economy will tell.

  15. Re:Virginia Tech purchased those Macs at full pric by PopCulture · · Score: 2, Interesting

    thats exactly what I was geting at. People (mac-heads) gripe at those who complain that Apples are more expensive. Well, for the average consumer (web surfing, e-mail, word processing) they are!!!

    you show me any consumer who needs Dual 2GHz PowerPC G5 with 4 GB of ram, I'll show you 100,000,000 people who don't.

    seriously, I've got an 800 MHz pentium III with 512MB RAM and I can watch a dvd, burn a cd, listen to mp3's, have MS Visual Studio.NET and NetBeans open working on 2 different projects, host Oracle, SQLServer, and mySQL and the postgreSQL client, and be browsing on several web browsers for where the hell I went wrong with my code... AT THE SAME TIME.

    its great bragging rights to have a dual 2 GHz Mac with 4 GHz RAM... but that is a fscking unrealistic comparison. Really now.

    --

    Here's to finally giving Bush his exit strategy in November
  16. Re:My Beef With Apple by Anonymous Coward · · Score: 1, Interesting

    Seriously, if you have *that* much difficulty getting a windows PC to work correctly, you shouldn't be building them for people :-P

    You don't have to do jack in Windows to get it to work well. Just get WHQL-certified drivers and you're set. As for chipset-optimized? What on earth do you do with your windows computers? :-P Everything I've ever seen which has chipset optimizations give you a choice of AMD/P3/P4/Xeon/PPro... hardly confusing (and not something that changes on your computer every day).

    I've got nothing against macs (in fact, I set up a powerbook to be our PS2->network wireless bridge last night), but to say that windows machines are tougher to get working than macs is sheer sensationalism. There is absolutely nothing you can't do with a PC. The best games, the best software, the most support, the best choice of _working_ hardware, everything.

    Oh, and as for the games remark, don't make me mention Medal of Honor on the mac. Please. That didn't work out too well.