Slashdot Mirror
Earth Simulator, G5 Cluster Drop In 'Top 500' List
daveschroeder writes "The November Top 500 supercomputer list has been published at SC2004. Topping the charts is IBM and the US Department of Energy's 'BlueGene/L DD2' beta system, at 70.72 TFlops, followed by NASA's 'Columbia' at 51.87.TFlops. For the first time in several publications of this list, Japan's Earth Simulator is no longer in the number one slot, falling to third. Virginia Tech's 'System X' Xserve G5 cluster, while 20% faster than the original cluster that debuted at number 3 last November, has fallen to number 7 due to the new entries, but remains the fastest supercomputer at an academic institution. Here's an excellent cost comparison (Google cache) of the top machines ('System X' is significantly cheaper than anything else in the top 20, not to mention cheaper than many things far below it in performance)."
I was down at Virginia Tech last year when I was looking at colleges. They would not let anyone near that computer. Even the guy who was giving the tour was complaining about the limited amount access Tech students were given. The main reason he cited was that the companies developing the supercomputer had technology that they didn't want people who had not signed NDA's to see. Anyway, the point was that while the computer may be owned by the university, students aren't even allowed to see what $5 million of their tuition bought.
It is actually 6 of the top 10, and 13 of the top 25.
You raise good points, and the team at Virginia Tech did do something remarkable. That said, cost per flop of the LINPACK benchmark is interesting but not particularly meaningful. For instance:
"Another example is PNNL's 1936 processor Itanium2 cluster: 3.5Tflops less performance than System X, for $25 million"
What is not captured by the LINPACK scores is that PNNL's machine will absolutely spank the BigMac cluster at what the PNNL machine is intended for: running computational chemistry codes such as NWChem. A lot of the cash for the PNNL machine went into large memories and fast I/O that simply does not show up in the LINPACK benchmark. Furthermore, there are a lot of very high-profile scientific publications that have come out of the computational chemistry abilities of the PNNL machine. That's something else extremely important that doesn't show up in the rankings.
There are a lot of similar examples, but the PNNL one is one that I know something about, so I chose it. Basically, I'm saying to not read too much into those cost comparisons. It really is comparing Apples to oranges... er, HPs in this case. =)
I think it is interesting that 11% of the top 500 are Power architecture, and 64% of the top 500 are intel based systems. Yet 50% of the top 10 are Power architecture and only 20% of the top 10 are intel architecture. Also interesting is that the Power based systems seem to have twice the Mflop/dollar ratio over the intel systems.
That's easy...the Xserve G5s consume a LOT less power (and therefore generate less heat, resulting in lower cooling costs) than any competitive products (Xeon, Itanium2, Opteron)...and this was true even when they were using the 970 (as opposed to the 970fx they are using now).
Several of the researchers at Virginia Tech have referred to this in various news stories numerous times - one estimate was over two times less power than comparable systems.
A major cost is power and cooling requirements. According to Apple, a single drive, dual processor 2.0GHz XServe will use about 250W peak. Virginia Tech has 2.3GHz machines and Infiniband PCI-X cards. There's also networking gear and other support equipment to consider. So, we'll use a high figure of 350W per node, giving us a 40% overhead. As for heat production, same overhead, 1200BTU/h per node.
We're going to consider the worst-case scenario, under which we have a 100% load, year round, on all 1100 nodes. That gives us a power consumption of 385kW and 1320kBTU/h of heat generation.
Now, we need to get rid of that heat, and that's going to require a lot of power. My research indicates up to 300kW may be required, but that's a high number and actual requirements may be lower.
So, here we are, with 685kW required for power and cooling. That means a 6000MW/h a year.
Now, the cost of power is high, since you need to amortize and maintain the UPS equipment and the generators. We'll use a figure of 0.15$/kW/h, or 150$/MW/h. Very generous.
So here we are. The absolute worst case for power and cooling. Full load, year round, expensive cooling, overpriced power and amortized UPS and generators.
900 000$/yr. Below a million. It's not that bad, is it ? The real cost is likely below a half-million.
As for the rest, well, how much pizza is really required to entice graduate students and professors to work on that machine ?
Deep Thought, actually. This supercomputer calculated that very number, as the answor to the Ultimate Question of Life, the Universe and Everything. Earth, Deep Thought's successor, was pretty close to calculating the question, without which the answer obviously doesn't make very much sense - but as we all know, it was destroyed by the Vogons to make way for a new hyperspace bypass.
More Information here.
Virginia Tech paid Apple's standard educational price, which is about a 10% discount on retail (and tax exempt). The only special break from Apple was that VTech got the first 1000 G5 towers off the assembly line, before other customers and stores.
It's because the die size is smaller, the G5 has less surface to dissipate heat, that's the reason for the high eficiency liquid cooling.
Mexico: 100% conservative's America now!