New Top500 List Released at Supercomputing '06

Guybrush_T writes "Today the 27th Edition of the Top 500 List of World's Fastest Supercomputers was released at ISC 2006. IBM BlueGene/L remains the world fastest computer with 280.6 TFlop/s. No new US system in the top10 this year, since they all come from Europe and Japan. The French Cluster at CEA (French NNSA equivalent) is number 5 with 42.9 TFlop/s. The Earth simulator (no 10) is no longer the largest system in Japan since the GSIC Center built a 38.2 TFlop/s Cluster, reaching the 7th place. The German cluster at Juelich is number 8 with 37.3 TFlop/s. The full list, and the previous 26 lists, are available on the Top500.org site."

Comments

What, no microsoft?

[harris s newman]

Its supprising that no microsoft systems are listed....

Re:What, no microsoft?

[Kadin2048]

Its supprising that no microsoft systems are listed....

Well, they only published the requirements for Vista a few weeks ago; I'm sure they'll do better next year.

Re:What, no microsoft?

[Anonymous Coward]

I would find that suprising too, except that there are 3 Microsoft systems on the list (5 OSX systems too). I actually read the page though.

Re:What, no microsoft?

[Kadin2048]

Just to follow up, you can get OS information here: http://www.top500.org/stats/27/osfam/ (by family)

OS (# systems) (Percent)
Linux 367 73.40%
Windows 2 0.40%
Unix 98 19.60%
BSD 4 0.80%
Mixed 24 4.80%
Mac OS 5 1.00%
Totals 500 100%

Alternately there's a more refined breakdown listing them by Operating System type and version. Oddly, "Linux" is listed both as an operating system family and as a distinct flavor/distro ... I can only assume that the systems using "Linux" as the particular operating system are using a custom-made distro, instead of one of the commercial ones (which are listed separately on the detailed chart). Unless they just failed to report one in particular.

As for the Windows-based systems, there were one each for Windows 2003 Server and Windows Compute Cluster Server 2003.

Re:What, no microsoft?

[MrFlibbs]

Also, what about massively distributed XP "supercomputers" like SETI@home? We just had a slashdot article discussing how the SETI version dominates the other spare-cycle-using background programs. If you summed up all the cycles in use at a given point in time, how would this stack up to the supercomputers on the list?

I realize internet-linked PCs are a different beast, but given the wide range of architectures on the top 500 supercomputer list, is it such a stretch to consider this a "supercomputer"? Anyone know how the SETI@home project would place?

Re:What, no microsoft?

[HermanAB]

Well, now that you mention it, *nobody* beats MS in distributed botnets...

Re:What, no microsoft?

[awing0]

With over 900,000 computers in the system, SETI@home has the ability to compute over 250 TFLOPS (as of April 17, 2006).

http://en.wikipedia.org/wiki/Seti@home

IBM's Blue Gene is faster (and more flexible). Big networks of computers like SETI's are good at crunching static radio telescope data or brute force RC5 cracking. When it comes to most real world problems, the nodes must communicate and share data, which over the internet makes it far too slow. Real supercomputers do not use any type of networking between nodes, they have a shared memory bus.

Re:What, no microsoft?

[flaming-opus]

shared memory bus is a pretty subjective measure. Sine the mid-90's no supercomputers have used true flat-memory shared buses. Instead they are connected by some sort of switched point-to-point network. In many of the mpp machines, these networks are built into the memory controllers on each node. (blue gene, crays, earth simulator, etc) The bandwidth and latency of these networks is orders of magnitude better than gigE, but it's still a network of sorts.

Re:What, no microsoft?

[after fallout]

Real supercomputers do not use any type of networking between nodes, they have a shared memory bus.
Don't say that, all it does is show how little you know.

There are 2 common types of interfaces between nodes on a supercomputer: shared memory and message passing.

Shared memory is where all the nodes can access memory over some sort of network. In order for communication to happen all 2 nodes need to do is read and write to the same location in memory. There is little talk about the network protocol used at this level because for the most part it is an emulation of layer 2 of the OSI (as if all you are doing is ordering the hardware around).

Message passing would best be described in terms of layer 7. Communication occurs between 2 nodes via messages that are sent back and forth (hence the name). The most common message passing scheme is MPI. In MPI, there is a concept of a sender and a reciever. The reciever calls MPI_Recv and the sender calls MPI_Send and a message is sent from send to recv. You could almost think of this as an HTTP communication; the server is listening, the client sends information, the server sends back, except in MPI the reciever must be calling MPI_Recv and waiting for a send from a specific sender and the sender must call MPI_Send to send the information to the reciever (there really [well, sorta] isn't a concept of a timeout). In my experience, this makes MPI (I use MPICH2) difficult to debug, if A calls send to B and B calls send to A at the same time, your program blows up (often with very little useful information).

On the cluster I do my work on, the implementation of MPI sends TCP/IP packets over ethernet (much like 256 of this top500 list). The libraries could be written to do the work over Myrinet or any other network.

For future reference please learn some factual information before you go spouting bull. If you follow this link, and choose interconnect family, you would find that most of the supercomputers in the top500 list are using some standard network interconnect.

Re:What, no microsoft?

[Miles Von Blitzkrieg]

I believe the numbers next to the OS names were referring to the number of supercomputers using that OS. Easy mistake.

280 flops not fast enough

[Anonymous Coward]

To keep up with the rate that humans make mistakes.

Damn...

[ReidMaynard]

Spanky's Cluster'O'Porn just missed the top 500 :-(

From 11 to 451...

[engagebot]

Dang, when our SuperMike was built (Lousiana State University), we were 11th on the list. A quick look now and we're at 451.

I feel old... ;0)

Re:From 11 to 451...

[sconeu]

Dude, your cluster is on fire!!!!

how many aren't listed?

[rritterson]

How well does this represent the real top 500?

If you look at the list, several of the computers/clusters are known simply as "Classified". It makes me wonder if those at the top really represent the top 10 most powerful supercomputers out there. I'm willing to be the US government, for one, has a couple of military use supercomputers up there that they aren't even willing to acknowledge the existance of.

At the other end of the spectrum, how many smaller clusters aren't on the list simply because the administrator doesn't have time to shut the entire thing down to run a LINPACK benchmark? The cluster I/we use would easily make it into the top 450, and maybe higher, but our research is deemed more important than the glory that comes with being on the list.

Re:how many aren't listed?

[Entropy]

US government, for one, has a couple of military use supercomputers up there that they aren't even willing to acknowledge the existance of. Just so long as one of them isn't named SKYNET, I'm content ..

Re:how many aren't listed?

[flaming-opus]

I would say it's unlikely that the classified computers are in the top 10, and here's why: the top500 list is constructed using linpack to measure floating point performance of highly parallel computation. Much of the work done by intelligence agencies is data-mining. It's integer tasks that are probably I/O bound rather than cpu-bound. If there were a top500 list of high performance storage systems, I bet the classified systems would own the top of the list, just not for raw fp-compute power.

Take, as an example, the Cray MTA. It's a product that's not even mentioned in their products page on their website. Yet, if you surf the net very carefully, you'll find out they're building the next version for their single customer of the product-line: the NSA. Even at the maximum configuration, the machine wouldn't make the top500 list, but it has features that make it uniquely suited to a few very peculiar application kernels. (single-virtual-cycle access to any memory within the distributed system)

Sure the department of defence uses supercomputers to predict the weather, improve weapons systems and simulate, but these are probably not done on systems we don't know exist. That sort of stuff is done at AHPCRC or ERDC, or at Beoing/lochead-martin/Ratheon/etc. All of these sites have huge HPC resources, just not the hugest of the huge.

Re:how many aren't listed?

[phasm42]

United Airline's intranet is called SkyNet.

Re:how many aren't listed?

[BeBoxer]

Yeah, but is the NSA sitting on a supercomputer as a "last resort" for encryption problems?

Not supercomputers in the sense they are being discussed here. The top500 list is computers that excel at floating-point operations. I have never seen an encryption method which uses floating point at all. They all use integer operations. DES, RSA, AES, MD5, SHA-1, etc. All 100% integer. In most cases cracking encryption algorithms really boils down to some sort of a search algorithm, so it wouldn't suprise me if there is overlap between systems that excel at data-mining and ones that excel at cracking encryption.

Google

[celardore]

There doesn't seem to be any mention of the GoogleNet. While it may not be used for figuring out sums and what-not, it does have an estimated 126 terraflops of computing power. I'd say that's notable. I bet at least half those terraflops are devoted to advertising aswell.

Re:Google

[zlogic]

That's not a supercomputer, that's a really large computing grid. Supercomputers are needed for tasks that require insane amounts of RAM and a really fast system bus. When you have many computers, the data transfer speed between computers is limited by the network bandwidth, and any network is slow compared to your CPU-memory bus. That may be OK for tasks like searching but not acceptable for physics simulation/capture. Sensors connected to a modern collider generate tens of gigabytes of data per second. So to simulate it you'll need to transfer all that data through your network, which will require hours or even days to simulate just one second.

Re:Google

[Rhys]

Google's full computing power is no more a supercomputer than distributed.net or seti@home is. Yes, both supercomputers and seti@home are parallel computation, but they are very different beasts.

Does google have a supercomputer? Maybe. I'm not actually clear what use a "traditional" supercomputer would be to them. For one thing, "disk" IO is genrally not the forte of a supercomputer, at least compared to processing power.

Re:"Flop/s"???

[VoxCombo]

I believe it's:
FLoating point OPerations / second, with the / representing "per"

131072 Processors!

[ackthpt]

Shit! I can remember when processors had that many transistors!

hello, olde programmers home, i'm enquiring for a vacancy...

Re:131072 Processors!

[Dadoo]

I can remember when processors had that many transistors!

You know, I reall like that metric, especially when you consider each of those processors probably has somewhere in the neighborhood of 100 million transistors.

Don't feel old, though. I cut my programming teeth on a processor with only 3500 transistors (6502). The transistors were probably so large, you didn't even need a clean room to manufacture it. :-)

Missing Computer

[Anonymous Coward]

What about the computer that processes Bill Gates' IRS filing?

Rmax vs Nmax

[G3ckoG33k]

Comparing the Rmax and Nmax values it seems that the list would look quite different if sorted on Nmax instead of Rmax. Can someone explain in plain English the difference, as I didn't understand their explanation. Thanks! :)

Re:Rmax vs Nmax

[SSCGWLB]

Rmax and Nmax are two measures of the capability of a supercomputer.

The first, Rmax, is the LINPACK benchmark. The LINPACK benchmark is a measure of floating point operations per second for the cluster. They usually include theoritical ((Max FLOPs for one CPU) * (number of CPUs) == Rpeak) along with the actual. Obviously, theoretical values will always be larger then actual due to wasted CPU cycles.

Nmax is the size of the problem (i.e. the dimension of the solved linear equation)

So, Rmax is the maximum performance obtained by the LINPACK benchmark during the test, Nmax is the size of the problem the best performance was achieved at.

~nate

Re:Rmax vs Nmax

[Junta]

Rmax represents the maximum acheived measured FLOPs as a result of an xhpl run.

Nmax represents the problem size. Nmax generally is aimed to be a problem that consumes as much memory as possible without swapping.

Rpeak is the theoretical max FLOPs possible according to the processor used. For example, a PPC chip is theoretically capable of 4 Flops per clock, so multiply the clock by the number of cores in the cluster. x86_64 is theoretically capable of 2 flops per clock, so multiply cores by two. Note that AMD clock for clock doesn't do any better than intel in *this* particular benchmark, so Intel clusters inherently can climb this list better, despite poor memory performance and other factors that make them less useful in a general supercomputing sense. Itanium can acheive better floating point (I believe 8 flops per clock).

And for anyone seeking to compare Rpeak/Rmax numbers with published Cell figures, keep in mind that game consoles (and by extension cell) brag about their single precision (32-bit) floating point performance, whereas this list only deals with double precision numbers (64 bit). Cell actually is nothing special at get top500 relevant benchmark results.

Many people feel this very specific benchmark is a poor indicator of the overall effectiveness of a cluster, and consider hpcc (which includes hpl as a subset) to be a better holistic method to evaluate the value of a cluster.

US is doing badly

[twfry]

The editors comment that there are no new 10 top US based computers is an odd comment. The US has 6 out of the top 10. Thats hardly doing poorly.

Re:US is doing badly

[stupidfoo]

Actually, as I posted above, there is a new system in the US. It's currently the #3 system.
http://www.top500.org/system/8128

Can't they use water cooling

[spicydragonz]

I see water cooling rigs all time take 2ghz CPUs to +4ghz. Why not use this for these machines? Perhaps a motherboard that could be bathed in cooling fluid...

Re:Can't they use water cooling

[DAldredge]

Because, maybe just maybe, they want accurate results?

Re:Can't they use water cooling

[scoobrs]

Because quite a few of them do use liquid cooling, but they don't order hobbyist solutions. Many Crays still use 3M Fluorinert, which doesn't damage sensitive electronics when you spray it right on a processor.

Re:Can't they use water cooling

[Rhys]

Given that clusters these days are made from commodity components (Xserve G5s, for instance) and how large clusters are these days, you end up with a pretty astounding failure rate. We lose roughly two peices of hardware (in order of most to least common: memory, cpu, motherboard, disk, power supply) a week, and our cluster (Turing, 640 nodes) is fairly small. We aren't even into the late-in-life crazy-disk-failure mode that most machines get at 3-5 years old. Think about the logistical nightmare if we had to try to "drain" a system of coolant before pulling it out to service it.

Plus then you'd have to have all that (very custom) cooling equipment, pumps, etc. You'd have to watch for leaks closely, which is also a problem with air cooling and the refrigerant lines, but those have a lot less surface area of pipe/connectors to go wrong: a loop per rack for rack-mounted cooling, not a loop per machine.

Plus, as other posters have said: we'd like accurate numbers.

Googleplex?

[neonprimetime]

How does Googleplex compare with the #'s in this top 500 list? (# Processors, max, peak, etc.)

Re:I for one....

[ackthpt]

welcome our new European and Asian supercomputer overlords.

With the Chinese quest to show they're as good as anybody, if not better, they'll probably be claiming something which dwarfs the USA DOE's Blue Gene chart-topper in a few years. To be powered and cooled at the site of the Three Gorges Dam.

probably use it to run world economic models and such to plan for when the USA defaults on hits 9+ trillion $ debt.

Re:Cue the Vista / Linux / Beowulf cluster jokes

[bcat24]

I for one welcome our new Beowulf cluster of Linux-running, not quite Vista-capable supercomputer overlords.

In Soviet Russia, a Beowulf cluster of Linux-running, not quite Vista-capable supercomputers imagine YOU!
On the other hand, in Korea, only old people imagine a Beowulf cluster of Linux-running, not quite Vista-capable supercomputers.

Oh! Won't somebody please think of the Beowulf clusters of Linux-running, not quite Vista-capable supercomputers?

Did I forget anything?

Blue Gene?

[theheff]

You're telling me that the fastest computer in the world is a pair of pants??

Re:Blue Gene?

[blibbler]

Well I know all my thinking takes place in my pants.

Super computers on our desks?

[davonshire]

Just a passing thought looking at this list when you peek at the bottom of the list.
you see a 2.8Ghz system with 1024 processors or some such.

Sorry I remember working on repairing a Univac computer when I was in the Navy and how amazing it sounded that Cray had produced this a super computer that could do 800
million operations a second.

(Circa 1980 or so)

You could have one of these computers for I think it was 13 Million dollars.
And how fabulous that the power supply was actually under the circular bench
so you could sit on your investment.

Consider the processing power we have now a days on our desks. A lowly
3 Ghz P4 Laptop with 2 GB of dynamic ram and 60 GB of Hard drive storage.

I've yet to see a pair up with our single or dual desktop computers today
and where they sit back in the super computer days of old. If anyone
has a link or info I'd love to hear about it.

Thanks,
  Nestalgia is the romance of historic madness.

Re:Super computers on our desks?

[stratjakt]

Correction, MMX added SIMD-type functionality to the original pentium line, but MMX only worked on integers, and reused floating point registers, making the proc incapable of doing FP math and SIMD at the same time..

SSE was when intel "got it right".

And it's still not that commonly used a feature.

But does it run...

[saleenS281]

But does it run Windows Cluster Edition? (Bet you didn't see THAT one coming)

Yeah...

[tpjunkie]

Bite my shiny Beowulf clustered, Linux-running, not quite Vista-capable supercomputing ass!

I wonder

[Itninja]

I wonder where my old Packard Bell 486/sx 33 would fall in this list. Which makes me wonder if there's a 'bottom 500' list somewhere. I would love to see a list of the slowest computer still in use.

What? No botnets listed?

[HermanAB]

A typical MS Windows botnet will outperform any of these machines on the SOPS (SPAM operations per second) benchmark...

Re:Cue the Vista / Linux / Beowulf cluster jokes

[Rob T Firefly]

That's no Beowulf cluster of Linux-running, not quite Vista-capable supercomputers.. it's a space station!

Re:Cue the Vista / Linux / Beowulf cluster jokes

[drawfour]

Did I forget anything?

Yes. You forgot to be funny.

Non-American nationalism

[amightywind]

A common story occurance on slashdot for the submitter to highlight some deep deficiency in American business, technology, or way of life, which is then inevitably followed by non-American dick-swinging nationalism of why someone else does it better. This is a perfect example.

The Network is the Computer

[Doc Ruby]

Isn't the Slashdot DDoS network the most powerful "computer" in the world?

What about...

[Cr0t]

Hmmm... where is the NSA's Super Computer?

Apple is still holding up well

[Darth Cider]

Notice that #21 and #28 use Apple XServes still running with G5 dual processors. The Virginia Tech system, #28, has fallen only 8 places, from #20 last year.

It's too bad this list doesn't mention cost. When Virginia Tech built its first cluster, the big news was how absurdly inexpensive it was in relation to other systems. It would be interesting to learn if that still holds true.

petaflop in 2008-2009?

[peter303]

The speed doubling time is still about 18 months (== 10x in five years). Two more doublings from the 2005 or 2006 280 TFlops is around 2008-2009. Its a version of Moore's law for supercomputing. Though processor speed hasnt been gaining as fast in recent years, improve clustering technology and software seems to be compensating.

"Exaflops in 2020!"

450K servers + 90 petabytes?

[peter303]

A recent Wired story about their twentieth-something server farm in Oregon (near cheap electricity) has them at about 450K blades. Assuming a mix of old and new commodity disks averaging 200GB per blade, gives close to a 100 petabytes. Plus MicroSoft was blathering about 800K server farms recently which hints at its estimate of a "beat-google" number might be.

Distributed Computing Wins Again!

[BrianWCarver]

Individuals contributing their spare processor cycles via BOINC are currently producing over 380 TeraFLOPS putting them clearly in first place (if such distributed systems were counted).

SETI@Home is now operated exclusively through BOINC and it alone is doing over 167 TeraFLOPS right now, putting the SETI@Home network in second place, only behind BlueGene/L (if such distributed systems were counted).

You can contribute your spare processor cycles too by downloading the BOINC client and attaching to a cool project such as Rosetta@Home which folds proteins as part of an effort to cure human diseases. Join the biggest "supercomputer" today!

Supercomputing '06

[breckinshire]

Supercomputing '06 was awesome. They threw this foam party... and well, 12 people died of electrocution, but it was still a great time. At one point, IBM's Deep Blue took off its chassis... you should've been there.