Virginia Tech Upgrade: PowerMac G5 to Xserve G5

An anonymous reader writes "Virginia Tech officially announced that they will be migrating their G5 Supercomputer from PowerMac G5s to Xserves. According to the article, the Xserve G5s will reduce power consumption, heat production and decrease the system size by a factor of three. The pricing of the upgrade is still being determined, and according to Srinidhi Varadarajan, they are working on getting "very good homes" for the PowerMac G5s which will be replaced."

Comments

Speed Improvments

[beelsebob]

While providing no real speed improvement this should actually speed up the cluster by a factor two - The XServe G5s have error correcting RAM in which should stop them having to run jobs twice just to be sure of getting the right result. They may even get a slight speed boost from having a 1.1Ghz bus rather than a 1Ghz one.

Bob

Re:Speed Improvments

[linwoes]

I thought I might point out that ECC is actually slower per access that non-ECC RAM. There are a few clocks per access to compute and check the syndrome.
Another thing that struck me is why we assume that VT would buy current generation XServes. I would imagine that they are going to be purchasing the next version of the XServe with a faster clock speed. I imagine that size matters but they'll get a speed boost also.

Re:Speed Improvments

[Anonymous Coward]

Yes, VT will be purchasing the second revision of the XServe G5, which will use the PowerPC 970FX. With clock speeds of 2.2, 2.4, and 2.6 GHz and bus speeds of 1.1, 1.2, and 1.3 GHz there will be a marked speed improvement as well as reduced heat and power dissipation.

Re:Speed Improvments

[Anonymous Coward]

ECC checking only adds a single clock cycle on reads and imposes no additional delays on writes since the ECC generation executes in parallel with other tasks. This delay only matters for the first cycle of a transaction; adding to the overall first access latency. The rest of the data transfers are pipelined so no additional delays are incurred. For well designed memory controllers(SDRAM,DDR) FAL is ~13+ base bus clocks for poorly designed ones ~17+ base bus clocks. 4-8% percent degredation in overall system performance is not likely(perhaps a single test can show this).

Re:Well duh?

[beelsebob]

I'm sure VirginiaTech realised that there would be 1U boxen along soon... What they also realised was that they had a tight deadline to get their computer tested for inclusion in the Top500 list - without the P3 rating they would have lost a LOT of jobs.

Bob

Re:The Cost?

[jocknerd]

They've already covered their costs in publicity and research. Getting on the Top500.org and being ranked #3 is huge. Well worth the $5 million they spent.

I know where they'll end up.

[blackchiney]

Trust me, the university is not letting anything out of their hands that can't be obsoleted first. It's a state school so they have a pecking order. My first bet is a large majority ends up at the Empo' followed by professors (who are also looking to build a smaller farm), faculty, staff, other state schools, and if we are so fortunate (and this is really a long shot) you can scoop one auctioned[PURCH].

WTF???

[kinnell]

+4 Interesting?!? Did any of you moderators actually read more than 2 lines. +4 Funny maybe...

Misprint on Reuters. They've already found homes!!

[dylan_baxter]

It looks as if our hopes are dashed. Hi, That statement on Reuters should have read "found new homes" not "finding" Regards, Srinidhi Dr. Srinidhi Varadarajan Director, Terascale Computing Facility Virginia Tech -----Original Message----- To: Srinidhi Varadarajan Subject: Request for information regarding liquidation of the G5's Greetings Mr. Varadarajan, I am sure my email is only one of a slew of letters to grace your inbox following the press release (http://biz.yahoo.com/rc/040126/tech_virginiatech_ apple_1.html). I sincerely hope this letter is no intrusion. I am writing in hopes of obtaining enough information to keep up to speed with VT's liquidation of System X. Hopefully, there will be an opportunity for myself and others like me to finally purchase a G5 without breaking the bank! Any leads you could give me would be greatly appreciated. Thank you for your time, Dylan Baxter Computers are like air conditioners - they stop working properly if you open Windows.

Re:PowerPC 970FX

[harakh]

The Register article about PowerPC 970FX. Seems plausible to me atleast.

Re:too soon to initial install

[J. Charles Holt]

"...is apple giving them another deep discount..." Apple didn't give them a deep discount the first time. Jobs says that the only special treatment that they got was that their orders for G5s were filled before the general public's. But they paid full price for 'em.

Re:Upgrade cost

[Alan Partridge]

skeptical

cynical

Have you considered an English tutor?

Re:Upgrade cost

[daviddennis]

Normally, you would be right.

But with the G5s being in very short supply, I don't think it happened this time. Remember, at that moment everyone and his brother (including me!) was looking for one.

Virginia Tech had a huge time constraint, and Apple had a reason to delay delivery as much as possible. They compromised at an "educational list" deal, which is also not unheard of.

That being said, I'm sure they DID get a generous deal on the Xserves, since time pressure was not involved, and there was no pressing need to upgrade immediately.

D

Re:VATech's rise to prominance

[ginbot462]

You obviously went to a school that stressed doing research instead of speaking out of your ass. Just cause you hadn't heard of them, doesn't mean other people and employers did not.

from http://www-2.cs.cmu.edu/~landay/school-rankings94. txt
For 1994 (see #26):

GRADUATE SCHOOLS OF
ENGINEERING WITH THE HIGHEST SCORES IN THE U.S. NEWS SURVEY

1. MASSACHUSETTS INSTITUTE OF TECHNOLOGY
2. STANFORD UNIVERSITY (Calif.)
3. PURDUE UNIVERSITY (Ind.)
4. UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
5. UNIVERSITY OF CALIFORNIA AT BERKELEY
6. CALIFORNIA INSTITUTE OF TECHNOLOGY
7. UNIVERSITY OF MICHIGAN AT ANN ARBOR
8. CORNELL UNIVERSITY (N.Y.)
9. UNIVERSITY OF TEXAS AT AUSTIN
10. CARNEGIE MELLON UNIVERSITY (Pa.)
10. GEORGIA INSTITUTE OF TECHNOLOGY
12. NORTHWESTERN UNIVERSITY (Ill.)
13. UNIVERSITY OF WISCONSIN AT MADISON
14. PENN STATE UNIV. AT UNIVERSITY PARK
15. UNIVERSITY OF MINNESOTA AT TWIN CITIES
16. PRINCETON UNIVERSITY (N.J.)
17. UNIVERSITY OF CALIFORNIA AT LOS ANGELES
18. RENSSELAER POLYTECHNIC INSTITUTE (N.Y.)
19. OHIO STATE UNIVERSITY
20. TEXAS A&M UNIVERSITY AT COLLEGE STATION
21. UNIVERSITY OF SOUTHERN CALIFORNIA
22. UNIVERSITY OF COLORADO AT BOULDER
23. UNIVERSITY OF FLORIDA
24. RICE UNIVERSITY (Texas)
25. JOHNS HOPKINS UNIVERSITY (Md.)
26. HARVARD UNIVERSITY (Mass.)
26. VIRGINIA TECH
28. NORTH CAROLINA STATE UNIVERSITY
29. UNIVERSITY OF WASHINGTON
30. CASE WESTERN RESERVE UNIVERSITY (Ohio)
31. MICHIGAN STATE UNIVERSITY
31. UNIV. OF CALIF. AT SANTA BARBARA
33. UNIVERSITY OF PENNSYLVANIA
33. DUKE UNIVERSITY (N.C.)
35. LEHIGH UNIVERSITY (Pa.)
36. UNIVERSITY OF VIRGINIA
37. UNIVERSITY OF MARYLAND AT COLLEGE PARK
38. UNIVERSITY OF CALIFORNIA AT DAVIS
39. UNIVERSITY OF CALIFORNIA AT SAN DIEGO
40. UNIVERSITY OF ROCHESTER (N.Y.)
41. COLUMBIA UNIVERSITY (N.Y.)
42. VANDERBILT UNIVERSITY (Tenn.)
43. YALE UNIVERSITY (Conn.)
43. DARTMOUTH COLLEGE (N.H.)
45. UNIVERSITY OF IOWA
46. BROWN UNIVERSITY (R.I.)
47. UNIVERSITY OF DELAWARE
47. COLORADO STATE UNIVERSITY
49. UNIVERSITY OF CINCINNATI
50. IOWA STATE UNIVERSITY

Re:Upgrade cost

[godzilla808]

I remember one of the project managers saying that the thought was to eventually cycle the G5 towers into labs on campus anyway. So they'll have a supercomputer and kick-ass lab machines. :)

Re:Instead of going 3x smaller

[aaarrrgggh]

Actually, the heat problem is a lot harder to deal with as you triple the density. They are looking at close to 12kW/rack, which pushes the envelope on what you can do with air.

For every five racks you need one computer A/C unit, without any redundancy. Anywhere you have a cable dam or piping, your ability to cool quickly goes to hell, even with a 24" raised floor.

I predict lots of problems with this upgrade... based on the marketing video they did with Apple. Just not set up to cool that kind of density.

Simple:

[daveschroeder]

Because the next list in June 2004, and the one after it, and the one after that, will include many new very, very expensive clusters (some $100M+) with performance far, far beyond 10Tflops.

So, yes, someone can build a 1101-G5 cluster right now, and be faster than VT's cluster. But they won't be on any list, and they definitely won't be anywhere near #3 on the next Top 500 list. And neither will VT.

That's why the whole VT #3 thing is the coup that it is: the timing was *perfect* for them to take the #3 spot for a mere $5.2M. The PR and grants they'll get *because of* that are more than worth it. That will never happen again for anywhere near that small an amount of money anytime soon.

See some of the new clusters that will be in the Top 10 on the next list:

http://www.bayarea.net/~kins/AboutMe/GIFs/TOP500_l ist_for_CPU.gif

Nope

[daveschroeder]

As has been pointed out countless times again, the cost was NOT a "mere" $5M. Their total hardware cost was ~$7M ($5.3M for the computers and memory, another $1.7M for the infiniband hardware), there was another $1M to upgrade an existing building.

"The total cost of the asset, including systems, memory, storage, primary and secondary communications fabrics, and cables is $5.2M." (Source: http://don.cc.vt.edu/tcfslides.pdf)

That $5.2M INCLUDED the Infiniband cards, switches, and cabling.

"The total cost of the asset, including systems, memory, storage, primary and secondary communications fabrics and cables is $5.2mil. Facilities upgrade was $2mil. 1mil for the upgrades, 1mil for the UPS and generators." (Source: Interview with Dr. Varadarajan)

There was then an additional $1M for "facilities upgrades", and $1M for power infrastructure.

They also had the benefit of free labour (millions of Mac zealots)

Huh? Millions of "zealots", eh?

Even if we GROSSLY overestimate labor, let's say a MILLION DOLLARS, the total cost is still $8M. So screw the free labor argument: even if they paid a MILLION DOLLARS to put it together (which is a huge, gross exaggeration), they're still much, much cheaper than anything close. Also, ANY academic institution has this same benefit.

and have not factored in the cost of power and cooling (at 2MW total power and cooling, this is a pretty significant expense, about $5,000 a day) or the support costs.

Sorry. Other clusters don't include power in their capital costs. And cooling *is* equipment included in the VT cluster. Ongoing support costs are NOT included in the costs of any of the other Top 500 clusters. The only thing different about the VT cluster was that the $5.2M figure didn't include some of the infrastructure costs other clusters have. But even the ASCI clusters are asset + infrastructure only, and do NOT include buildings, energy, or support costs. So, sorry again. And even at $7M + our imaginary labor, it's still ridiculously cheaper.

Even NCSA's new Tungsten cluster is $12M for the ASSET ALONE. That does not include building, support, infrastructure upgrades, or anything. Just the computer. And Dell installed it for free. So are they "PC zealots", since it was free labor?

Re:Why upgrade to dual 2 Ghz?? They won't be 2 ghz

[shawnce]

I think you may have your facts wrong...

Apple announced the G5 base Xserve systems only a couple of weeks ago and they top out at 2GHz currently using a 90nm version of the PPC 970. They have not announced any systems based on G5s with higher clock speeds.

Can you point to documentation about systems using 2.3+ GHz G5s (PPC970/fx etc.) and when those will be available and when VTech / public will be getting them?

The main reason, so far stated, that they are swapping the systems is for space saving and power savings (electrical and cooling) thanks to the 90nm G5s being used.

Re:Has anyone else heard.....

[cbustapeck]

Last I checked, users don't spend too much time in server rooms, so this shouldn't be much of an issue.