Earth Simulator, G5 Cluster Drop In 'Top 500' List

daveschroeder writes "The November Top 500 supercomputer list has been published at SC2004. Topping the charts is IBM and the US Department of Energy's 'BlueGene/L DD2' beta system, at 70.72 TFlops, followed by NASA's 'Columbia' at 51.87.TFlops. For the first time in several publications of this list, Japan's Earth Simulator is no longer in the number one slot, falling to third. Virginia Tech's 'System X' Xserve G5 cluster, while 20% faster than the original cluster that debuted at number 3 last November, has fallen to number 7 due to the new entries, but remains the fastest supercomputer at an academic institution. Here's an excellent cost comparison (Google cache) of the top machines ('System X' is significantly cheaper than anything else in the top 20, not to mention cheaper than many things far below it in performance)."

Comments

Hmm

[foxalopex]

Err, I'm not sure if the costs can be accurately compared in this way. One needs to remember that a cluster of separate computers acting as a supercomputer compared to a custom designed hardwired system isn't exactly the same thing! Otherwise you can start comparing stuff like SETI which I'm sure is the world's cheapest supercomputer because it technically didn't cost anything to SETI themselves.

Re:Hmm

[Anonymous Coward]

Viruses are the new supercomputers.

Re:Hmm

[Wesley Felter]

All the systems on the Top 500 list are benchmarked running Linpack. If you can run Linpack on SETI@home, you're welcome to count it.

Re:Hmm

[Ibanez]

Well then, why not compare the earth? What other supercomputer has gotten closer to "42" than it?

Re:Hmm

[dustpuppy_de]

Deep Thought, actually. This supercomputer calculated that very number, as the answor to the Ultimate Question of Life, the Universe and Everything. Earth, Deep Thought's successor, was pretty close to calculating the question, without which the answer obviously doesn't make very much sense - but as we all know, it was destroyed by the Vogons to make way for a new hyperspace bypass.

More Information here.

Re:Hmm

[flonker]

Norton Antivirus: Fighting Back Against Skynet!

Re:Hmm

[mjeppsen]

Wow. Parent is modded "Insightful"? Incredible. I guess "+5 Funny" was disabled...

The Dept. of Energy

[aztektum]

I hear they're using it to convert heat into electricity for the rest of the government. Hence their name.

Pizza arguments

[daveschroeder]

A few more thoughts...

Before anyone says "Of course System X is cheaper! Virginia Tech had free student labor to put it together! They paid them in pizza!"

The only thing anywhere close to System X is NCSA's Tungsten, a 2500 processor Pentium IV Xeon Dell Linux cluster. It cost $12 million, just for the asset (comparable to System X's $5.8 million overall price, including the upgrade to Xserve G5s). That's twice the cost, and over 2Tflops less performance. 2Tflops is a top 100 supercomputer...so it's a whole top 100 supercomputer poorer in performance, for an extra $6.2 million.

Another example is PNNL's 1936 processor Itanium2 cluster: 3.5Tflops less performance than System X, for $25 million.

Any way you slice it - no pun intended - System X is still a LOT cheaper, even if you allot, say $2M for professional installation and systems integration - an EXTREMELY liberal estimate, probably by an order of magnitude.

System X also has the highest Rmax per CPU of any system on the list, except for specialty non-commodity systems like Earth Simulator.

And on top of it all, last November, they hit #3 in the world, #2 in the US, and #1 academic, as well as the first academic site to ever exceed 10Tflops, all for less than $7 million in total - including all improvements to buildings, physical plant, and other infrastructure.

That first system might not have had ECC, but what it did do is break into the top 5, following all the rules of the Top 500 organization, for relative pocket change - for a price that was absolutely unheard of, sharing the spotlight with systems that cost $100 million or more - and also catapulted Virginia Tech to a supercomputing center of national prominence overnight, able to attract additional attention, funding, grants, and publicity. Not to mention testing and proving the suitability of a completely new OS, platform, processor, and interconnect for high-performance computing, increasing choice for all (and resulting in new clusters based on the same technology, such as the US Army/COLSA cluster). And even as new systems enter the top ten in the tens and hundreds of millions of dollars, System X retains the title of #1 at any academic institution, and shares the top 10 with the best of the best.

Seems to me that Virginia Tech pulled a real coup here, and a full year later, is still considerably cheaper that anything else. And now, it's being used for real scientific work. To bring a whole new platform onto the scene in essentially under a year and break into the ranks of the supercomputing elite virtually overnight, and to do it significantly, and sometimes ridiculously, cheaper than everyone else, is a feat that can't be ignored.

Re:Pizza arguments

[thpr]

Before anyone says "Of course System X is cheaper! Virginia Tech had free student labor to put it together! They paid them in pizza!"

No, my real question would be: What is the ongoing operating expenses of System X? After all, I'm interested in total cost of ownership, not in acquisition cost.

Re:Pizza arguments

[daveschroeder]

I'm sure it would be similar to any Linux cluster in the top 10; there's no reason to believe it should be any different or require significantly different levels of system administration and maintenance.

Re:Pizza arguments

[CODiNE]

I remember the power usage and heat generation was lower than the competition... they were able to save a lot of money on cooling systems by going with the G5's. That has to be saving them a lot of money day to day. I don't have the wattage numbers on hand right now, but I do remember G5's beating P4's, Opterons and especially Xeons, so there's no way Virgina Tech is paying what those x86 top ten people are in energy bills. The XServe's keep track of their fans and internal temperature, automatically letting the admin know when a machine is GOING TO FAIL and preemptively swap out faulty components before they cause serious damage to the systems. I don't know what sort of Apple Care that kind of system comes with, but I'm sure it's competitive with their consumer systems, I'd imagine any hardware failures are completely covered absolutely free for at least the first year, probably three or more if they pay Apple for any support.

-Don.

Re:Pizza arguments

[hernick]

A major cost is power and cooling requirements. According to Apple, a single drive, dual processor 2.0GHz XServe will use about 250W peak. Virginia Tech has 2.3GHz machines and Infiniband PCI-X cards. There's also networking gear and other support equipment to consider. So, we'll use a high figure of 350W per node, giving us a 40% overhead. As for heat production, same overhead, 1200BTU/h per node.

We're going to consider the worst-case scenario, under which we have a 100% load, year round, on all 1100 nodes. That gives us a power consumption of 385kW and 1320kBTU/h of heat generation.

Now, we need to get rid of that heat, and that's going to require a lot of power. My research indicates up to 300kW may be required, but that's a high number and actual requirements may be lower.

So, here we are, with 685kW required for power and cooling. That means a 6000MW/h a year.

Now, the cost of power is high, since you need to amortize and maintain the UPS equipment and the generators. We'll use a figure of 0.15$/kW/h, or 150$/MW/h. Very generous.

So here we are. The absolute worst case for power and cooling. Full load, year round, expensive cooling, overpriced power and amortized UPS and generators.

900 000$/yr. Below a million. It's not that bad, is it ? The real cost is likely below a half-million.

As for the rest, well, how much pizza is really required to entice graduate students and professors to work on that machine ?

Re:Pizza arguments

[adiposity]

Well, I'm not going to disagree with anything you've posted here, but are we sure Apple charged these people anywhere near full price for the hardware? Just a question, I don't actually know the answer.

-Dan

Re:Pizza arguments

[Anonymous Coward]

The XServe's keep track of their fans and internal temperature, automatically letting the admin know when a machine is GOING TO FAIL,

Huh? Very few of all the possible faults in a computer have any kind of temperature change associated with them. When a memory stick, a HD, a CPU, a system chip, a connector fails, you don't get any overheating. (You may get a slight temp decrease after the component has failed and stopped consuming power.)

Your scenario only applies to failing fans and heatsinks prying themselves loose... not very common occurrences.

Sorry, but you have bought into some not very tech-savvy information somewhere.

BTW, it has been establisehd in every discussion here and in specialised hardwarwe sites that the power draw of a 2 GHz G5 is on par with a 2 GHz K8. This comparison is made harder by IBM not readily giving out the max power but a "typical" power, but valid estimations have been achieved nevertheless.

There is a reason why the 2.5 GHz G5 couldn't be air-cooled quietly. (It could have been air-cooled, but not quietly. I have a 2.4 GHz Athlon 64 system that is whisper quiet with a Zalman CNPS7000-AlCU at 1300 rpm and two 120mm case fans at 1600 rpm. Go figure.)

IBM is a fabulous foundry, but so has AMD been recently (avoiding most of Intel's 90 nm problems, for instance). The 970FX is just a really good processor based on a damn good architecture, not an engineering miracle.

Re:Pizza arguments

[frankie]

Virginia Tech paid Apple's standard educational price, which is about a 10% discount on retail (and tax exempt). The only special break from Apple was that VTech got the first 1000 G5 towers off the assembly line, before other customers and stores.

Re:Pizza arguments

[LinuxHam]

The XServe's keep track of their fans and internal temperature, automatically letting the admin know when a machine is GOING TO FAIL and preemptively swap out faulty components before they cause serious damage to the systems

Just like the IBM xSeries servers I'm deploying right now. Our servers automatically order replacement components if/when components fail. This includes CPUs, memory, fans, hard drives.. just about anything. IBM big boxes have been doing this for decades, and distributed systems for a number of years now. Don't get too excited.

Power architecture does well

[blamanj]

5 of the 10 top machines use the Power archtecture, either the Power4 or PPC family.

Re:Power architecture does well

[Brett Johnson]

It is actually 6 of the top 10, and 13 of the top 25.

Re:Power architecture does well

[Brett Johnson]

I think it is interesting that 11% of the top 500 are Power architecture, and 64% of the top 500 are intel based systems. Yet 50% of the top 10 are Power architecture and only 20% of the top 10 are intel architecture. Also interesting is that the Power based systems seem to have twice the Mflop/dollar ratio over the intel systems.

Re:Power architecture does well

[Anonymous Coward]

I think it is interesting that 11% of the top 500 are Power architecture, and 64% of the top 500 are intel based systems. Yet 50% of the top 10 are Power architecture and only 20% of the top 10 are intel architecture. Also interesting is that the Power based systems seem to have twice the Mflop/dollar ratio over the intel systems.

That's because x86 is a horrible architecture. On top of that, x86 instructions are translated into microcode before they're executed, so you end up with an unknown (maybe you could ask the folks at Intergraph about it) architecture emulating a crappy architecture in hardware. Better architectures exist (ARM, MIPS, POWER, 68000, PA-RISC, toy architectures used in introductory computer architecture classes, everything else), but Intel won out in the marketplace. You can still get better chips, but you pay more and have less support.

That's why you'd be better off investing in AMD over Intel. AMD hit upon what Intel should've done years ago. The x86-64, for those who don't know, supports x86 binaries as well as its own new architecture. Think of it like an x86 chip with the underlying hardware exposed. If Intel had exposed the hardware that x86 instructions get translated to, they'd have had a clear upgrade path instead of having to dork off x86 out of the blue. AMD embraced and extended x86, and marginalized its future without doing any actual damage to it or x86 users. It's flat out genius.

In the meantime, almost anything performs better than x86, and with less power consumption. It makes those mini-ITX boards look like jokes, because instead of engineering a low powered MIPS board/processor, the VIA folks did another x86. It may have been good from a business point, but it's horrible from an engineering standpoint, and that sums up Intel and x86 fairly nicely.

Re:Power architecture does well

[kai.chan]

5 of the 10 top machines use the Power archtecture, either the Power4 or PPC family.

Coincidentally, my top machine use Microsoft Windows and it has a record of AlwaysFlops.

Who has coffee?

[Saeed al-Sahaf]

It's nice that Off The Shelf boxes like Apple and Intel can make a super computer cluster. When do the stories stop? We know that if you put enough PCs together, you get a very powerful machine. What we should be looking at is cutting edge technology in specialized CPUs. Give me 10,000 vanilla boxes and some good custom software, but give me a cutting edge CPU designed for super computing, that's science. We already know that it is possible to fill a fucking building with Pentiums, or better 68000s.

super computations?

[Doc Ruby]

Earth Simulator spent a while at the top of the list - that's a lot of TFLOPs under the curve - a lot of seconds. What did it accomplish while it was king of the hill? How much Earth was simulated?

Re:super computations?

[Zork the Almighty]

They simulated the entire earth up until the creation of slashdot. Then everything suddenly ceased.

Re:super computations?

[roine]

How much Earth was simulated?

The closest they got was 41.9998645234, which is pretty good in my book.

VA Tech Supercomputer

[dretay]

I was down at Virginia Tech last year when I was looking at colleges. They would not let anyone near that computer. Even the guy who was giving the tour was complaining about the limited amount access Tech students were given. The main reason he cited was that the companies developing the supercomputer had technology that they didn't want people who had not signed NDA's to see. Anyway, the point was that while the computer may be owned by the university, students aren't even allowed to see what $5 million of their tuition bought.

Re:VA Tech Supercomputer

[jdog1016]

It's not even technically on campus. Its at the nearby Virginia Tech Corporate Research Facility. And in any case, you can arrange a tour if you want.

Re:VA Tech Supercomputer

[monkeyboy87]

Tech has a long tradition of this. We weren't allowed to come near the IBM 3090 (a near super computer in the late '80's) when I was an undergrad there either.

it was housed in the CRC about 1 mile off campus in those days. Probably freed up the room for the cluster when they decomissioned the old 3090 behemoth.

Re:VA Tech Supercomputer

[Chiron Taltos]

$5 million of their tuition probably landed several hundred million dollars worth of research grants. Paying tuition does not entitle you to everything a university does with the money. And that's even if the assumption is correct that it was tuition money in the first place.

Can anyone shed light on where the $5 million actually came from?

Thanks for that post

[commodoresloat]

That was a close one... I was starting to worry that Apple might be dying again.

Erm ...

[phoxix]

What exactly did the Big Mac do anyways ?

I can assume it was put to some sort of use. But I honestly get the feeling it was more to have fun, and look cool (which means more bling bling from sponsors, alumni, etc)

Sunny Dubey

Re:Erm ...

[BJH]

Bagdes? We don't need no steenking bagdes!

hehehe, IBM knows how to play

[BortQ]

IIRC then IBM just came out with their entry very recently. Perhaps they know how to play supercomputer sniping. It's easy to learn on ebay.

Funny MIPS

[RealProgrammer]

These numbers seem surreal, like thinking about Monopoly money. I'm sitting here at my old PII-300, analyzing the cost/power ratio of machines costing a mere $6M, or as much as $350M. This one cost, uh, nothing.

On any one of those systems, you could emulate a Beowulf cluster of this one, and still have time to play Thermonuclear War.

cluster operating system

[MrMartini]

I get a kick out of the fact that System X runs Mac OS X.

Only with Mac OS X can you get the combination of commercial software (such as Microsoft Office and Adobe Photoshop), user friendliness, no known viruses, best available security, and stability/scalability suitable for world-class superclusters.

Re:EARTH TO MAC ZEALOTS:

[Anonymous Coward]

Don't discount infiniband... it has one of the lowest internode latencies available (quadrics has a lower latency, but lower max bandwidth). So if an OS that supported multi-machine spanning was used in a senario like VA tech, you'd be dead wrong. The hardware is there, just not the software, just not yet.

Earth Simulator has liberal bias

[Anonymous Coward]

The Earth Simulator computed that global warming will cause major climate change in the next 50 years.

Clearly it suffers from liberal bias.

Sounds right, actually

[MooseByte]

"I'm not sure if the costs can be accurately compared in this way. [ ... ] Otherwise you can start comparing stuff like SETI which I'm sure is the world's cheapest supercomputer"

Actually that sounds like a perfectly valid comparison, SETI included. In bang for the buck SETI deserves to win hands-down in that scenario, and fairly. System X deserves its place as well.

Re:Sounds right, actually

[Anonymous Coward]

Depending on application, System X's 'bang for buck' could actually be an unmitigated loss compared to IBM's offering. Any problem that isn't embarassingly parallel, for instance, would destroy System X in a value comparison with even crazy-expensive Cray machines. On such a problem even SETI, were it forced to utilize all nodes, would be orders of magnitude slower than a dual opteron workstation (if not, it might be faster if any single node (or tight cluster of nodes) is faster than the aforementioned workstation and all work is delegated to it, idling the rest).

System X only has respectable bang for buck if you limit the application domain to problems System X was specifically designed to solve effectively, which, incidentally, aren't the same set of problems that BlueGene/L can solve effectively. So no, it is not a valid comparison. It's like saying that the space shuttle transport is less cost-effective than a fleet of pickup trucks with an equivalent combined cargo weight capacity.

CPU benchmarks

[3770]

You can compare CPU benchmarks here.

AMD is beating the crap out of Intel.

Re:CPU benchmarks

[Anonymous Coward]

This is the best specint AMD has to offer (yes, the fastest operon is slower):
AMD Athlon (TM) 64 FX-55 1 core, 1 chip, 1 core/chip 1750 1854

Here is the best intel:
3.4 GHz, Pentium 4 Proce 1 core, 1 chip, 1 core/chip with HT Technology enabled 1667 1705

Here are the best specfp numbers for AMD:
AMD Athlon (TM) 64 FX-55 1 core, 1 chip, 1 core/chip 1741 1782

Which just edges out the best Pentium
3.6GHz Xeon) 1 core, 1 chip, 1 core/chip 1700 1721

But gets beaten by a low end I2
1300 MHz, Itanium 2 1 1808 1808

Also by a Fujitsu SPARC
PRIMEPOWER900 (1890MHz) 1 core, 1 chip, 1 core/chip 1510 1803

Gets trounced by the second-best I2 available
1500 MHz, Itanium 2 1 2161 2161

And gets demolished by the top end POWER5
1900 MHz, 1 CPU 1 core, 1 chip, 2 cores/chip (SMT off) 2576 2702

Re:CPU benchmarks

[Rufus211]

Usefull site for getting usable spec numbers is ace's SPECmine. Try SPECint2000 or SPECfp2000 and check "CPU MHz". The opteron's fare quite nicely.

Why does slashdot keep posting these stories?

[The Pim]

Is there some fantasy supercomping league I don't know about?

power costs?

[CaptainPinko]

adminstration and maintenance similar perhaps... but what about power?a few watts per core adds to a lot more heat PLUS the cost of cooling. i think it would be interesting if they printed a FLOP/$ per annum for each of the top 500. the cost of acquisition being spread evenly over the lifetime of the cluster.

Re:power costs?

[daveschroeder]

That's easy...the Xserve G5s consume a LOT less power (and therefore generate less heat, resulting in lower cooling costs) than any competitive products (Xeon, Itanium2, Opteron)...and this was true even when they were using the 970 (as opposed to the 970fx they are using now).

Several of the researchers at Virginia Tech have referred to this in various news stories numerous times - one estimate was over two times less power than comparable systems.

Don't get carried away with LINPACK flops/cost!

[Richard Mills]

You raise good points, and the team at Virginia Tech did do something remarkable. That said, cost per flop of the LINPACK benchmark is interesting but not particularly meaningful. For instance:

"Another example is PNNL's 1936 processor Itanium2 cluster: 3.5Tflops less performance than System X, for $25 million"

What is not captured by the LINPACK scores is that PNNL's machine will absolutely spank the BigMac cluster at what the PNNL machine is intended for: running computational chemistry codes such as NWChem. A lot of the cash for the PNNL machine went into large memories and fast I/O that simply does not show up in the LINPACK benchmark. Furthermore, there are a lot of very high-profile scientific publications that have come out of the computational chemistry abilities of the PNNL machine. That's something else extremely important that doesn't show up in the rankings.

There are a lot of similar examples, but the PNNL one is one that I know something about, so I chose it. Basically, I'm saying to not read too much into those cost comparisons. It really is comparing Apples to oranges... er, HPs in this case. =)

What is with the Apple fan-boyism?

[Anonymous Coward]

Now Apple markets good computers. Tend to be on the expensive side, but they are usually high quality.

The Power970 is decent enough in itself. The opteron is more powerfull, but is also much more energy hungry. The Intel Itanium is nice but it's very expensive. etc etc

But what is this worship of Apple? It makes no sense.

Story 1: Earth simulator.. blah blah blah., but Mac cluster!

Story 2: SGI supercluster.... blah blahblah, But Mac cluster!

Story 3: Blue Gene cluster, 65000+ cpus... blah blah blah, but Mac cluster!

Realy? Who gives a fvck about the 7th place computer, and who gives a damn about cost analysis at this point? What about the Top5?

Did you know that Blue Gene is PowerPC?
Did you know that Linux now runs the majority of top super computers...

Did you know that Blue Gene proccessors only run a 700mhz??!!!

Did you know that #4 is 3564 Power970's running at 2.2 ghz? And that beats out 4000+ Intanium2's running at 1.7ghz?

This is a Geek site.. what about OSes?
By ranking:
1. Linux, 2. Linux, 3. Unix, 4. Linux, 5. Linux, 6. Unix, 7. OS X, 8. Linux, 9. Unix, 10. Linux (most powerfull x86 btw), 11. Unix, 12. Unix, 13. Linux, 14. ?, 15. Linux, 16. Linux, 17. Linux, 18. Linux, 19. Linux, 20. Unix.

Were is the most powerfull Windows computer? Well there is one cluster that is probably still on the top500. I dare you to find it, though. It's probably around #200 or #300, which is stil freaking fast.

Ok, So the big Mac is still #7. That's great, but there are 6 wonderfull computers that have all sorts of great technology that your completely ignoring because Apple wets your pants.

Did you know that Blue Gene will eventually have over 65,000 proccessors??

Re:What is with the Apple fan-boyism?

[MacDork]

Ok, So the big Mac is still #7. That's great, but there are 6 wonderfull computers that have all sorts of great technology that your completely ignoring because Apple wets your pants.

Fine, I'm willing to talk about number 6. ;-)

Re:What is with the Apple fan-boyism?

[xenoandroid]

Apple worship because we all knew they were going to die and are overpriced and whatnot, then we see their products being used to make one of the most powerful clusters in the world for very cheep.

Apple worship because it's a smack in the face to those who still continue to bash Apple for reasons that no longer exist.

"OS 9 sucks!"
"We're on OS X now, and it's unix-like"
"oh...um...Well one button~"
"And yet I'm still more productive on it than my Windows box"
"Well um, I want linu-"
"You can install that too."
"Well, they're slow and-"
"I suggest you actually try using one before saying that."
"They're overpric-"
"Really? I didn't think $900 was that expensive for a mid-range machine."

We do the Apple worship thing just to fustrate the anti-mac crowd even more.

Re:EARTH TO MAC ZEALOTS:

[kc8apf]

While you are correct that clusters are not the ultimate solution for high performance computing, single-image computers are not a great solution either. They require specific optimizations to be done for the particular system and do not lend for easy system upgrades.

It's still running

[Rui del-Negro]

> How much Earth was simulated?

Well, I've noticed a vew glitches (disappearing keys, poor AI in girlfriends, crazy presidents in some countries, etc.), but I'd say most of the Earth has been running reasonably well.

Re:Just imagine

[mlk]

Using such a beast for a simple FPS would be such a waste.
So true, we should use a complex, awe-inspiring game, which will push the limits of any machine. I suggest Nethack.

Not on the list

[leapis]

DB Error: connect failed

Apparently, the top 500 list is not actually hosted on one of the top 500 machines.

VT ongoing cost of ownership evaluation...

[MacDork]

200 pizzas a week. ;-)

Re:VT ongoing cost of ownership evaluation...

[LittleBigLui]

You USians and your funny units... how much is that in Libraries-of-Congress?

I'm still hanging in

[serutan]

My computer is number 44,286,551 and I'm gunning hard for position 44,286,550.

Re:And we thought Macs were expensive

[cbreaker]

You only have to buy 1100 of them to get a discount.

Software

[Mardak]

It's not just about how fast the systems perform in linpack. The machines should be calculating something useful, and if you feed it inefficient code, it'll be going nowhere fast.

Apple has created software development packages specifically designed for their G5's with optimized code for the 64bit architecture such as complex math functions.

So not only is Apple providing a cheaper and power efficient system for academic institutes, they make it easier for professors and assistants to create the software to run on those clusters.

#4 is also academic

[az4 h0th]

Just a minor comment...

I work at UPC and there has been a lot of hype here for machine #4, which is (or is going to be) a >4500 PPC970s machine running linux (nice work, ibm). I disagree with the claim that the Virgina Tech cluster is the first academic supercomputer. As far as I'm concerned the Technical University of Catalonia (UPC) is also an academic institution.

Anyway. we now got europe's fastest supercomputer. That's what matters. ha! ;-)

Re:EARTH TO MAC ZEALOTS:

[kevingolding2001]

Yeah, but imagine a beowulf clus... Ummmm.. sorry, what was the question again?

Re:two times less

[jbarlow]

Ok, this is just one of those things that bothers my proofreading brain. What exactly is "two times less" supposed to mean? Literally, it means that they should be shoving as much power back into the grid as these comparable systems. 'Cause "one time less" would be zero.

Try "half as much." Damn, I hate grammar nazis like me.

Not to mention that BG beats BigMac in flops/$

[joib]

The cost he quotes for the Blue Gene ($200 million) was the cost of some government contract that included BG/L, ASCI Purple (a huge cluster of POWER5 servers) and some R&D as well.

Recently IBM announced their commercial prices for BG machines (see e.g. theregister.co.uk or news.com.com). Prices start at $1.5 million (1 fully equipped cabinet). Using this price and published linpack figures one arrives at about 2.9 Mflop/s/$, compared to the maximum value of 2.2 Mflop/s/$ he quotes for the best apple system.

Add in the fact that the BG uses much less space and power than a comparable xserve cluster, that it has a faster and lower latency network, and we have a winner.

Re:Not to mention that BG beats BigMac in flops/$

[Anonymous Coward]

Recently IBM announced their commercial prices for BG machines (see e.g. theregister.co.uk or news.com.com). Prices start at $1.5 million (1 fully equipped cabinet). Using this price and published linpack figures one arrives at about 2.9 Mflop/s/$, compared to the maximum value of 2.2 Mflop/s/$ he quotes for the best apple system.

Actually, that 2.2MFLOP/s/$ figure for VT's cluster isn't the best for an apple XServe based system. The University of Maine built a 512 CPU cluster of XServes for $680K, or 3.012MFLOP/s/$, according to the price comparison's chart, edging out a BlueGene cabinet.

Of course, now I get to be flamed by a bunch of mac haters who think pointing out a factual error in your statement means I don't know anything else about clustering and will blindly chose a mac above all others.... *sigh*

Re:Not to mention that BG beats BigMac in flops/$

[Lars T.]

Since his point was PowerPC beats the crap out of x86 and Itanium, yeah, I'm convinced he will be disappointed it takes a PPC system to beat a PPC cluster.

Re:Not to mention that BG beats BigMac in flops/$

[iammaxus]

This is not a flame:

That UoM 512 CPU one is significantly smaller than the top 5 super computers. You should probably only be comparing in class. Costs do not increase proportionally with size of the computer.

Re:need?

[mosb1000]

Your key mistake here is your use of the word "needs". The data I've seen indicates that the G5 draws an equivalent amount of power as comparable Intel and AMD systems. Also, the G5's in the x-servs are air cooled. I think they mostly liquid cool the dual 2.5 Ghz G5 just to keep the noise down.

Re:need?

[10Ghz]

Your key mistake here is your use of the word "needs". The data I've seen indicates that the G5 draws an equivalent amount of power as comparable Intel and AMD systems. Also, the G5's in the x-servs are air cooled. I think they mostly liquid cool the dual 2.5 Ghz G5 just to keep the noise down.

uh-huh. If G5 runs so cool, then surely they could have kept the original cooling-system for the 2.5GHz model, instead of going for an complicated liquid-cooling system? Really, why did they move from heatsink/fan to liquid-cooling? AFAIK the original G5's were already quiet. And looking at reviews such as this seems to suggest that the G5 does indeed run very hot.

And looking here and here I can see this:

2.5GHz G5: 75-85C during load
2.2GHz Opteron: 48C during load

G5 runs cooler? Hardly.

Fan boys read only what they want to see!

[mdemirha]

Well, I really am getting sick of this Apple fan-boyism. What is up with that? People here scratch their heads all day long and try to find some calculations that may show that Apple higher than the others even though the comparison is a just an apples to oranges comparison. One writes that Flops per $ for apple's are better? How the hell can they make a real comparison? Why dont they look at the Flops per CPU? What are the other hardware in those systems? Why dont they compare all the machines? Do they have the same HDD/RAM/..etc other parts. If not (and it is not) this is a totally useless comparison.

Now another fan boy writes that 3600 2.0 GHz G5's beat 4000 1.7 GHz Itanium 2s. What a comparison! My god!

One other fan boy writes that AMD Opterons beats the crap out of Intel systems. Looking at the link he provided, it is pretty clear that Intel Itaniums beat the crap out AMD systems and then the fan boy defends himself by saying that AMDs are cheaper!

Oh come on now people, be a little more objective! The article says the following: A total of 320 systems are now using Intel processors. Six months ago there were 287 Intel-based systems on the list and one year ago only 189. # The second most common processor family is the IBM Power processor (54 systems), ahead of PA Risc processors (48) and AMD processors (31). # At present, IBM and Hewlett-Packard sell the bulk of systems at all performance levels of the TOP500.

These are much more important numbers than some uber-geek-fan boys calculations. It is apparent that Intel has increased its percentae *A LOT*. AMD also started to putting many systems into top 500. In my opinion, Apple's success in this list is MUCH MUCH LESS than Intel's or AMD's or IBM's or HP's successes.

Be a little more logical, open minded, less fanatic people. Apple is just a freaking computer like any other computers around. It is not some sort of a super/splendid/magnificent/God-Like computer.

Re:Fan boys read only what they want to see!

[zpok]

"Be a little more logical, open minded, less fanatic people."

So we should say "It's really no achievement to have a supercomputer in a pricerange available to institutions other than military. The fact that they use Apple's G5 and OS X, an almost out of the box solution is totally irrelevant. If you like you can build your own courtesy of Virginia University, but who would want a supercomputer that's cheaper than the other twenty first contenders in the list of supercomputers. Remember, they're Apple, so they're crap. And expensive, whatever the calculations say. They must be. They're Apple. I repeat, they're Apple. Crap. Be realistic, don't be a fan-boy."

Fan-boy indeed.

A supercomputer application

[tezza]

Keeping track of the very high frequency of postings of these supercomputer rankings on Slashdot.

Can I vote for a supercomputer thread so that I can elect to have it not displayed in my preference? I wouldn't want to miss out all the other tasty hardware goodness. I don't mind news about new Supercomputer technology, but whoever holds the most teraflops at a certain point in time is not of interest.

http://apple.slashdot.org/article.pl?sid=04/11/09/ 0126220&tid=137&tid=126&tid=181&tid=1
November 9th, 2004

http://science.slashdot.org/article.pl?sid=04/11/0 6/2239245&tid=136&tid=137&tid=14
November 7th, 2004

http://slashdot.org/article.pl?sid=04/11/03/161424 5&tid=137&tid=139&tid=1
November 3rd, 2004

http://science.slashdot.org/article.pl?sid=04/10/2 7/0147206&tid=137&tid=139&tid=14&tid=106
October 26th, 2004

http://apple.slashdot.org/article.pl?sid=04/10/26/ 0636230&tid=137&tid=3
October 26th, 2004

http://science.slashdot.org/article.pl?sid=04/10/2 0/1727255&tid=137&tid=136&tid=14
October 20th, 2004

Re:What i really like is

[AusG4]

Does anyone know what this guy is talking about? Who is the wise prince? What is happening?

Virginia fastest academic?

[shiruf]

Virginia Tech's 'System X' Xserve G5 cluster [...] remains the fastest supercomputer at an academic institution.
Beg to differ: #4 is about 5 mins from home (by bus), in the northern campus of universitat politècnica de catalunya. And, yes, part of the institution, not some loaned space or something. Mind you, one wishes Spanish Universities involved their students a tenth as much. S-2.

Coup? Cuckoo!

[PingPongBoy]

5.7 TFlops in less than a square metre coming to a store near you, an IBM Blue Gene. If these start appearing at Wal Mart, it's time to redefine supercomputer.

Re:funny or funny?

[Zork the Almighty]

I think cheap clusters will be old news in a year or two. IBM's Blue Gene designs are so space efficient, I think they will shift the whole market in that direction.

Re:EARTH TO MAC ZEALOTS:

[UnknowingFool]

System X is not a single system image machine. It will not work well for problems requiring low internode latency and high internode bandwidth in comparison to a real supercomputer.

Christ, clusters are not the end all and be all of high performance computing systems.

I guess I don't get the arugment because many of the other entries are clusters and not not single image either. Of course, given OS and architecture differences, all the supercomputers may perform differently in real world applications than the benchmark tests. The point is that System X was built using off-the-shelf components at a fraction of the cost of comparable systems. The entry right above System X is the US Army cluster of Apples (Mach 5). Obviously Apple is starting to be considered for high end computing.

In your face, Amiga! In your face! :)

MOD PARENT DOWN

[Anonymous Coward]

For not being sufficiently deferential to Apple.

We need a Steve Jobs Benchmark!

[Anonymous Coward]

How fast does it do a 500 pixel Gaussian Blur on a 300MB Photoshop file, convert it to CMYK and rotate it 22 degrees?

(betcha it comes out #1!)

Re:need?

[Kyusaku Natsume]

It's because the die size is smaller, the G5 has less surface to dissipate heat, that's the reason for the high eficiency liquid cooling.

Re:need?

[znu]

It doesn't make much sense to compare operating temperature differences between machines with different cooling systems. There's a much easier way to figure out how much heat a processor generates: just look at how much power it consumes. An Opteron at 2.2 GHz sucks 89W. A PPC 970fx at 2.5 GHz uses around 50W.

Supercomputer Window-shopping

[speedbump]

The top 500 list is interesting, and fun to read, but LinPack and other benchmarks only give us a rough feel for how well these monsters perform.

We're not comparing apples to oranges, more like ORCHARDS of apples to oranges...

Re:Are prices on that comparison adjusted for date

[99BottlesOfBeerInMyF]

Of course, if you want the numbers to be even more meaningful you should reference the cost of the systems, not today, but the length of time it takes to build them before today. Right? Of course there are advantages to quick deployment that in many cases are just as important as cost, so perhaps you should also add to the cost, the price of hiring out comparable computing resources for the time it takes to build them. Of course all of this is moot since the Big Mac cluster was so much cheaper AND faster to build than anything else in the top 10 it wins any non-biased comparison hands down. (Given that your goal is to cheaply and quickly develop a cluster for a purpose for which the LINPACK test is a good benchmark.)

the floating point operations thing (Altivec)

[kardar]

I think if I understand this properly, it's the G5 processor (and the previous ones too) that have the Altivec engine, which provides a level of "vector" capabilities that other CPU's simply don't have.

While there may be a bias (personally I am sick of the way EVERY FreeBSD, OpenBSD, and NetBSD post is being mishandled - it's unprofessional and destructive, and irresponsible given /.'s place in the IT community :), but I think it's worth mentioning that (correct me if I am wrong) - that the G5 (and G4 before it) have floating point, or "vector" capabilities, which really set them apart from the other PC processors such as the AMD and Intel. And probably better vector capabilities than a lot of RISC CPUs as well.

The G5 is an elegant, very cool processor, it's unfortunate that it's so locked into Apple's proprietary format - it would be so cool to build your own with parts, and have a choice of parts.

It's really an awesome processor. As to whether or not there's a bias, perhaps there is... but I think that when it comes to supercomputing clusters, the G5 has some unique capabilities that set it apart from other processors (unless, of course, you want to buy Crays and things like that). It would be interesting to see if there were some way (and this is way in the future, but just in terms of where things are headed) - it would be interesting to see DragonFlyBSD paired with a G5, or maybe at that point in the future it will be a G6 (who knows?). DragonFlyBSD is working on a way to (I can't find the info right now) improve clustering performance - if that comes to pass, and their PowerPC port comes to pass, even with purchasing a machine from Apple directly, you might have some very cool things going on.

Re:These are the ones you're allowed to know about

[Oddly_Drac]

"Well the "Beast" in Brussels UN is probably up there also"

Only if we're counting fictional computers thought up by conspiracy theorists.

So, no.

"Big Brother uses in the very near future if not already."

You mean filming some no-marks around the clock in the name of entertainment? Or the fairly silly idea that Europe is spearheading an effort to slap everyone into a database. Have you ever seen the EU decide anything? Do you know that the EC meets in Brussels, not the UN?