Earth Simulator, G5 Cluster Drop In 'Top 500' List
daveschroeder writes "The November Top 500 supercomputer list has been published at SC2004. Topping the charts is IBM and the US Department of Energy's 'BlueGene/L DD2' beta system, at 70.72 TFlops, followed by NASA's 'Columbia' at 51.87.TFlops. For the first time in several publications of this list, Japan's Earth Simulator is no longer in the number one slot, falling to third. Virginia Tech's 'System X' Xserve G5 cluster, while 20% faster than the original cluster that debuted at number 3 last November, has fallen to number 7 due to the new entries, but remains the fastest supercomputer at an academic institution. Here's an excellent cost comparison (Google cache) of the top machines ('System X' is significantly cheaper than anything else in the top 20, not to mention cheaper than many things far below it in performance)."
Err, I'm not sure if the costs can be accurately compared in this way. One needs to remember that a cluster of separate computers acting as a supercomputer compared to a custom designed hardwired system isn't exactly the same thing! Otherwise you can start comparing stuff like SETI which I'm sure is the world's cheapest supercomputer because it technically didn't cost anything to SETI themselves.
I hear they're using it to convert heat into electricity for the rest of the government. Hence their name.
No sig for you!!
A few more thoughts...
Before anyone says "Of course System X is cheaper! Virginia Tech had free student labor to put it together! They paid them in pizza!"
The only thing anywhere close to System X is NCSA's Tungsten, a 2500 processor Pentium IV Xeon Dell Linux cluster. It cost $12 million, just for the asset (comparable to System X's $5.8 million overall price, including the upgrade to Xserve G5s). That's twice the cost, and over 2Tflops less performance. 2Tflops is a top 100 supercomputer...so it's a whole top 100 supercomputer poorer in performance, for an extra $6.2 million.
Another example is PNNL's 1936 processor Itanium2 cluster: 3.5Tflops less performance than System X, for $25 million.
Any way you slice it - no pun intended - System X is still a LOT cheaper, even if you allot, say $2M for professional installation and systems integration - an EXTREMELY liberal estimate, probably by an order of magnitude.
System X also has the highest Rmax per CPU of any system on the list, except for specialty non-commodity systems like Earth Simulator.
And on top of it all, last November, they hit #3 in the world, #2 in the US, and #1 academic, as well as the first academic site to ever exceed 10Tflops, all for less than $7 million in total - including all improvements to buildings, physical plant, and other infrastructure.
That first system might not have had ECC, but what it did do is break into the top 5, following all the rules of the Top 500 organization, for relative pocket change - for a price that was absolutely unheard of, sharing the spotlight with systems that cost $100 million or more - and also catapulted Virginia Tech to a supercomputing center of national prominence overnight, able to attract additional attention, funding, grants, and publicity. Not to mention testing and proving the suitability of a completely new OS, platform, processor, and interconnect for high-performance computing, increasing choice for all (and resulting in new clusters based on the same technology, such as the US Army/COLSA cluster). And even as new systems enter the top ten in the tens and hundreds of millions of dollars, System X retains the title of #1 at any academic institution, and shares the top 10 with the best of the best.
Seems to me that Virginia Tech pulled a real coup here, and a full year later, is still considerably cheaper that anything else. And now, it's being used for real scientific work. To bring a whole new platform onto the scene in essentially under a year and break into the ranks of the supercomputing elite virtually overnight, and to do it significantly, and sometimes ridiculously, cheaper than everyone else, is a feat that can't be ignored.
5 of the 10 top machines use the Power archtecture, either the Power4 or PPC family.
It's nice that Off The Shelf boxes like Apple and Intel can make a super computer cluster. When do the stories stop? We know that if you put enough PCs together, you get a very powerful machine. What we should be looking at is cutting edge technology in specialized CPUs. Give me 10,000 vanilla boxes and some good custom software, but give me a cutting edge CPU designed for super computing, that's science. We already know that it is possible to fill a fucking building with Pentiums, or better 68000s.
"Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
Earth Simulator spent a while at the top of the list - that's a lot of TFLOPs under the curve - a lot of seconds. What did it accomplish while it was king of the hill? How much Earth was simulated?
--
make install -not war
I was down at Virginia Tech last year when I was looking at colleges. They would not let anyone near that computer. Even the guy who was giving the tour was complaining about the limited amount access Tech students were given. The main reason he cited was that the companies developing the supercomputer had technology that they didn't want people who had not signed NDA's to see. Anyway, the point was that while the computer may be owned by the university, students aren't even allowed to see what $5 million of their tuition bought.
That was a close one... I was starting to worry that Apple might be dying again.
IIRC then IBM just came out with their entry very recently. Perhaps they know how to play supercomputer sniping. It's easy to learn on ebay.
A Multiplayer Strategy Game for Mac OS X, Windows, and Linux
These numbers seem surreal, like thinking about Monopoly money. I'm sitting here at my old PII-300, analyzing the cost/power ratio of machines costing a mere $6M, or as much as $350M. This one cost, uh, nothing.
On any one of those systems, you could emulate a Beowulf cluster of this one, and still have time to play Thermonuclear War.
sigs, as if you care.
Don't discount infiniband... it has one of the lowest internode latencies available (quadrics has a lower latency, but lower max bandwidth). So if an OS that supported multi-machine spanning was used in a senario like VA tech, you'd be dead wrong. The hardware is there, just not the software, just not yet.
The Earth Simulator computed that global warming will cause major climate change in the next 50 years.
Clearly it suffers from liberal bias.
"I'm not sure if the costs can be accurately compared in this way. [
Actually that sounds like a perfectly valid comparison, SETI included. In bang for the buck SETI deserves to win hands-down in that scenario, and fairly. System X deserves its place as well.
You can compare CPU benchmarks here.
AMD is beating the crap out of Intel.
The Internet is full. Go Away!!!
Is there some fantasy supercomping league I don't know about?
The evaluation of an action as 'practical' . . . depends on what it is that one wishes to practice.
adminstration and maintenance similar perhaps... but what about power?a few watts per core adds to a lot more heat PLUS the cost of cooling. i think it would be interesting if they printed a FLOP/$ per annum for each of the top 500. the cost of acquisition being spread evenly over the lifetime of the cluster.
Your CPU is not doing anything else, at least do something.
You raise good points, and the team at Virginia Tech did do something remarkable. That said, cost per flop of the LINPACK benchmark is interesting but not particularly meaningful. For instance:
"Another example is PNNL's 1936 processor Itanium2 cluster: 3.5Tflops less performance than System X, for $25 million"
What is not captured by the LINPACK scores is that PNNL's machine will absolutely spank the BigMac cluster at what the PNNL machine is intended for: running computational chemistry codes such as NWChem. A lot of the cash for the PNNL machine went into large memories and fast I/O that simply does not show up in the LINPACK benchmark. Furthermore, there are a lot of very high-profile scientific publications that have come out of the computational chemistry abilities of the PNNL machine. That's something else extremely important that doesn't show up in the rankings.
There are a lot of similar examples, but the PNNL one is one that I know something about, so I chose it. Basically, I'm saying to not read too much into those cost comparisons. It really is comparing Apples to oranges... er, HPs in this case. =)
Now Apple markets good computers. Tend to be on the expensive side, but they are usually high quality.
The Power970 is decent enough in itself. The opteron is more powerfull, but is also much more energy hungry. The Intel Itanium is nice but it's very expensive. etc etc
But what is this worship of Apple? It makes no sense.
Story 1: Earth simulator.. blah blah blah., but Mac cluster!
Story 2: SGI supercluster.... blah blahblah, But Mac cluster!
Story 3: Blue Gene cluster, 65000+ cpus... blah blah blah, but Mac cluster!
Realy? Who gives a fvck about the 7th place computer, and who gives a damn about cost analysis at this point? What about the Top5?
Did you know that Blue Gene is PowerPC?
Did you know that Linux now runs the majority of top super computers...
Did you know that Blue Gene proccessors only run a 700mhz??!!!
Did you know that #4 is 3564 Power970's running at 2.2 ghz? And that beats out 4000+ Intanium2's running at 1.7ghz?
This is a Geek site.. what about OSes?
By ranking:
1. Linux, 2. Linux, 3. Unix, 4. Linux, 5. Linux, 6. Unix, 7. OS X, 8. Linux, 9. Unix, 10. Linux (most powerfull x86 btw), 11. Unix, 12. Unix, 13. Linux, 14. ?, 15. Linux, 16. Linux, 17. Linux, 18. Linux, 19. Linux, 20. Unix.
Were is the most powerfull Windows computer? Well there is one cluster that is probably still on the top500. I dare you to find it, though. It's probably around #200 or #300, which is stil freaking fast.
Ok, So the big Mac is still #7. That's great, but there are 6 wonderfull computers that have all sorts of great technology that your completely ignoring because Apple wets your pants.
Did you know that Blue Gene will eventually have over 65,000 proccessors??
While you are correct that clusters are not the ultimate solution for high performance computing, single-image computers are not a great solution either. They require specific optimizations to be done for the particular system and do not lend for easy system upgrades.
kc8apf
> How much Earth was simulated?
Well, I've noticed a vew glitches (disappearing keys, poor AI in girlfriends, crazy presidents in some countries, etc.), but I'd say most of the Earth has been running reasonably well.
DB Error: connect failed
Apparently, the top 500 list is not actually hosted on one of the top 500 machines.
200 pizzas a week. ;-)
Bagdes? We don't need no steenking bagdes!
My computer is number 44,286,551 and I'm gunning hard for position 44,286,550.
You only have to buy 1100 of them to get a discount.
- It's not the Macs I hate. It's Digg users. -
Just a minor comment...
;-)
I work at UPC and there has been a lot of hype here for machine #4, which is (or is going to be) a >4500 PPC970s machine running linux (nice work, ibm). I disagree with the claim that the Virgina Tech cluster is the first academic supercomputer. As far as I'm concerned the Technical University of Catalonia (UPC) is also an academic institution.
Anyway. we now got europe's fastest supercomputer. That's what matters. ha!
Ok, this is just one of those things that bothers my proofreading brain. What exactly is "two times less" supposed to mean? Literally, it means that they should be shoving as much power back into the grid as these comparable systems. 'Cause "one time less" would be zero.
Try "half as much." Damn, I hate grammar nazis like me.
The cost he quotes for the Blue Gene ($200 million) was the cost of some government contract that included BG/L, ASCI Purple (a huge cluster of POWER5 servers) and some R&D as well.
Recently IBM announced their commercial prices for BG machines (see e.g. theregister.co.uk or news.com.com). Prices start at $1.5 million (1 fully equipped cabinet). Using this price and published linpack figures one arrives at about 2.9 Mflop/s/$, compared to the maximum value of 2.2 Mflop/s/$ he quotes for the best apple system.
Add in the fact that the BG uses much less space and power than a comparable xserve cluster, that it has a faster and lower latency network, and we have a winner.
uh-huh. If G5 runs so cool, then surely they could have kept the original cooling-system for the 2.5GHz model, instead of going for an complicated liquid-cooling system? Really, why did they move from heatsink/fan to liquid-cooling? AFAIK the original G5's were already quiet. And looking at reviews such as this seems to suggest that the G5 does indeed run very hot.
And looking here and here I can see this:
2.5GHz G5: 75-85C during load
2.2GHz Opteron: 48C during load
G5 runs cooler? Hardly.
Lesbian Nazi Hookers Abducted by UFOs and Forced Into Weight Loss Programs - -all next week on Town Talk.
Does anyone know what this guy is talking about? Who is the wise prince? What is happening?
bash-3.00$ uname -a
SunOS panda 5.10 Generic sun4u sparc SUNW,Ultra-2
Virginia Tech's 'System X' Xserve G5 cluster [...] remains the fastest supercomputer at an academic institution.
Beg to differ: #4 is about 5 mins from home (by bus), in the northern campus of universitat politècnica de catalunya. And, yes, part of the institution, not some loaned space or something. Mind you, one wishes Spanish Universities involved their students a tenth as much. S-2.
"Be a little more logical, open minded, less fanatic people."
So we should say "It's really no achievement to have a supercomputer in a pricerange available to institutions other than military. The fact that they use Apple's G5 and OS X, an almost out of the box solution is totally irrelevant. If you like you can build your own courtesy of Virginia University, but who would want a supercomputer that's cheaper than the other twenty first contenders in the list of supercomputers. Remember, they're Apple, so they're crap. And expensive, whatever the calculations say. They must be. They're Apple. I repeat, they're Apple. Crap. Be realistic, don't be a fan-boy."
Fan-boy indeed.
I think, therefore I am...I think.
For not being sufficiently deferential to Apple.
It's because the die size is smaller, the G5 has less surface to dissipate heat, that's the reason for the high eficiency liquid cooling.
Mexico: 100% conservative's America now!
It doesn't make much sense to compare operating temperature differences between machines with different cooling systems. There's a much easier way to figure out how much heat a processor generates: just look at how much power it consumes. An Opteron at 2.2 GHz sucks 89W. A PPC 970fx at 2.5 GHz uses around 50W.
This space unintentionally left unblank.