Slashdot Mirror

As another reader pointed out there is a serious drool factor in a dual core AMD Opteron. Other than the gamers and overclockers one does not need dual cores or multi-GHz clock speeds for most applications. My desktop system is a dual processor 200MHz Pentium Pro system (circa '97) and my web server which was /.'ed in August is a dual processor 100 MHz Pentium (circa '95). "Dual" processors is *not* new. Both systems work fine for the jobs they need to perform most of the time. The only times they don't is when clueless programmers overengineer their web sites. I guess it takes a lot more than dual cores or GHz clock speeds to make me drool. :-;

But getting back to the topic at hand. Most of my dual processor systems *rarely* use both processors. I think I've only noticed one program (a commercial OCR program) that seems to be programmed to take advantage of dual processors). I think the older versions of Netscape may have an explicit problem with dual processors if Javascript is enabled. [I suspect this is because Javascript may try to run as a separate thread and both Javascript and the HTML image display code use the memory heap at the same time without single threading the code and end up corrupting the heap. But this is just a guess on my part.]

The real reason for going to multiple cores is (a) the drool factor; and (b) competitive edge [for example Sun is pushing on 4 & 8 cores to distinguish themselves from the commodity processors]; and (c) true supercomputer applications. With respect to (c) a lot of people in government and research were pretty upset with the fact that the U.S. didn't have the #1 spot in the Top500 list for the better part of 2 years. While having multiple cores helps put you back at the top of the list -- supercomputer architectures are complex. You reduce the processor delays between the processors on the same chip but for problems which require the CPU and memory of thousands of processors (protein folding for example) you still have "speed-of-light" message passing delays between the multi-core processors. That requires a very sophisticated message passing network. [See theoretical discussion here.] You can reduce these delays by packing the processors closer together but then you have heat and reliability problems.

These problems aren't quite as significant with server farms because the data is coming from and going out into the world and interprocessor delays are not as important as they are in supercomputers and large database applications that have to be concerned with concurrency issues.

It is interesting to note that IBM is running the Blue Gene clock speeds at rates significantly below what AMD and Intel chips are at and even IBM's PPC chips are capable of. And this is from the company that used to build ECL based mainframes that had to be water cooled (so they know the technology). I presume this is because the want to keep the heat production down, reliability up and perhaps to minimize the excess space (and therefore interprocessor delays) that water cooling might require.

Look here.

The speed you quoted is the theoretical peak, not the actual maximum achieved in a real world calculation (like the Top 500 organization's use of Linpack).

System X's equivalent theoretical peak is 20.24 TFlops.

I'm also not indicting Linux clusters in the least; they've clearly shown they can outperform traditionally architected and constructed supercomputers for many tasks, with the benefit of using commodity parts - at commodity pricing. All I'm saying is that there's a new player here, and it's a real contender, and has done a lot for very little money...which was the whole goal of Linux clusters in this realm in the first place.

(Also, as I said, the volunteer labor model is irrelevant - let's just pretend it was professionally installed for an additional $1M, or even $2M if that would satisfy you. It's still several million dollars cheaper, and 3Tflops greater performance. These are BOTH rackmount clusters with similar amounts of nodes and processors, running a commodity OS with fast interconnects. There are differences, yes, and perhaps even differences in goals. But looking past that, price/performance for something like this is still an important metric.)

Perhaps take some work in the Scientific or Defense area's so you can see SGI is a long time from going away. I am not saying they never will, but right now they have a lot of stuff in those areas'. We are getting ready to make another very large investment in SGI. Take a look at the latest NASA supercomputer that is number 2 in the TOP500, or IBM's BlueGene that is number 1 (www.top500.org/lists/2004/11/).

Being that I work on Irix, Solaris, Linux, HP-UX, and AIX on a daily basis I can say that there are things you get with Irix that no other platforms have available to them (yet). Before I came here I never worked on Irix and I didn't like them. But, now when it comes to Supercomputing environments I would have to lean toward Irix because they have created their OS specifically for that purpose. Where all the rest have created a more general purpose OS. I will say this though; out of all the above platforms Linux tends to be our least reliable platform. Linux is great, but it has some maturing to do.

Linux though is FAR more reliable than anything Micro$oft puts out.. Shall we say probably 1000 times more reliable? Anyone?

Granted it does not perform well comared to AMD.

Trust me. IA64 is not going to die any faster than *BSD or Apple. As soon as Intel lowers the price and the low voltage higher clock rate chips come out. IA64 is good. Trust us.

Granted it does not perform well comared to AMD.

Trust me. IA64 is not going to die any faster than *BSD or Apple. As soon as Intel lowers the price and the low voltage higher clock rate chips come out. IA64 is good. Trust us.

Granted it does not perform well comared to AMD.

Trust me. IA64 is not going to die any faster than *BSD or Apple. As soon as Intel lowers the price and the low voltage higher clock rate chips come out. IA64 is good. Trust us.

Granted it does not perform well comared to AMD.

Trust me. IA64 is not going to die any faster than *BSD or Apple. As soon as Intel lowers the price and the low voltage higher clock rate chips come out. IA64 is good. Trust us.

Unfortunately, the vast majority of oil co's are Microsoft shops.

Not when it comes to data processing. Plenty are using Linux clusters for heavy duty data crunching.
Most of the data processing houses also use Linux clusters.

Look at http://www.top500.org/ and see how many are oil and gas/seismic data processing related.

Just my $0.02.

itanium has not delivered on a single design goal since its inception. intel went full steam ahead on itanium, placing bets on a number of key technologies to pan out in order to sustain itanium development -- all of which never happened.

And which of the world's leading microprocessor companies do you run or even work for?

How do you explain the Itanium failing so badly in its design goals that it is #1 in memory bandwidth? How do you explain their failure when creating 2 of the top 5 computers in the world?

right now amd is eating them for lunch with amd64

Actually, its the Opteron that is competing with the Itanium processor, but you get extra /. bonus points for mentioning that AMD is better than Intel.

Windows Server 2003 Compute Cluster Edition is for up to 128 processors, however they boast it might do 256 processors.

That might not make in in the top 500 of super computers:

last 500 super computers

Most computer in the bottom of the top 500 heave at least 256 processors.

MS themself claim that every factor 2 gives it's own set of problems.

However selling 128 licenses of windows 2003 server is interesting.....

But this article isn't a surprise. ia64 is just presently a pretty crappy CPU for clustered computing because it's very hot, sucks a lot of power and very expensive.

FWIW, a Xeon uses slightly more power than an Itanium chip, and yes the Itaniums are very expensive. However, I believe that both of these are going to change. The Itanium already has a low power model at 1GHz, and Intel is looking at upping the speed of these low power offerings. And they better start reducing their prices.

And being that the current 2nd and 5th fastest clustered computers are based on the Itanium chip, ovbiously someone with more decision making power than you believes that these processors are OK for clustering. The first AMD offering is at #17.

NASA intends to use the neural map of his brain to build the next supercomputer in order to regain the top spot and predict the outcome of all Jeopardy games until 3025.

Fortunately, they anticipated the slashdotting, and provided their own mirror. It even has an apposite fqdn.

The best thing that AMD can have happen for them on the corporate front would be to get major vendors like Dell, HP, and IBM to offer their chips in their products.

IBM and Sun are already offering AMD based workstations, in addition to HP blades and supercomputers. At least at the workstation and server level, it seems as if the major vendors are already offering them.

Actually... The Top500 rankings system does establish some limits to prevent vendors from building, benchmarking, and publishing results on systems which are not available to customers. The restriction is that only 5% of a single vendor's total listed processing power can be at vendor owned facilities. See the Call for Participation.

We're not sure if BlueGene will be included on the list to be published this week, as it's an engineering system on IBM's manufacturing floor, and does not (yet) belong in a comparison of production systems. IBM needs 1,330 Terraflops of non-internal systems on the list to qualify BlueGene. The current list has ~400 TF worth of IBM systems, so BlueGene is unlikely to be on Tuesday's list unless the list maintainer decides to include it anyway.

What's certain is the the Earth Simulator will no longer be #1: NASA's SGI cluster was installed on site and in production even before being benchmarked, and the announced results of 42TF will push it down a notch just as easily as the "quietly submitted" results of 51.

IBM's system will certainly be high on the list when it enters, but subject to the list maintainer's discretion, they may have to wait until actually shipping it before they can have it included on the list. If they can manufacture, ship, and benchmark a 70TF system in the next 6 months, they likely will be #1 when the next list is published. Unless of course somebody buys an even larger Altix cluster or another vendor releases something competitive.

The restriction on submissions was designed for exactly this type of situation, and there's two parts to it:
What can be built in a lab is a function of money, and the larger vendors with other revenue streams can afford to build large systems for no purpose other than to post a benchmark - while smaller vendors can only afford to build what they can sell.
Posting more flops than anyone else is good work, but it really doesn't mean much to the HPC industry or user community until people can buy one, have it installed, and run their code on it.

The test is called linpack.

http://www.top500.org/lists/linpack.php

The vast majority of clusters are for simulating very complex systems that require lots and lots of calculations.

You can get a few hints by looking just at their names.

The number one "Earth Simulator Centre" is fairly self-explanatory, going to their website show they create a variety of models for things such as weather, tectonic plate movement, etc.

The number 3 LANL supercomputer "is a key part of DOE's plan to simulate nuclear weapons tests in the absence of actual explosions. The more powerful computers are designed to model explosions in three dimensions, a far more complex task than the two-dimensional models used in weapons design years ago." I imagine that most US government simulations would be doing something simmilar.

IBM's BlueGene is the king right now(well for the time being), but I don't see Big Mac(either version) beating the earth sim. Still, 2 out of the top 4 isn't bad.

If we're assuming the Army's COLSA MACH5 will meet the October deadline, then that would make it 2 out of the top 6 machines being G5 based. But if we're counting installed but not-yet-operational supercomputers, then we should probably include Red Storm (41.5 Rpeak). Does anyone know if MACH5 and Red Storm will make the deadline?

IBM's BlueGene is the king right now(well for the time being), but I don't see Big Mac(either version) beating the earth sim. Still, 2 out of the top 4 isn't bad.

If we're assuming the Army's COLSA MACH5 will meet the October deadline, then that would make it 2 out of the top 6 machines being G5 based. But if we're counting installed but not-yet-operational supercomputers, then we should probably include Red Storm (41.5 Rpeak). Does anyone know if MACH5 and Red Storm will make the deadline?

IBM's BlueGene is the king right now(well for the time being), but I don't see Big Mac(either version) beating the earth sim. Still, 2 out of the top 4 isn't bad.

If we're assuming the Army's COLSA MACH5 will meet the October deadline, then that would make it 2 out of the top 6 machines being G5 based. But if we're counting installed but not-yet-operational supercomputers, then we should probably include Red Storm (41.5 Rpeak). Does anyone know if MACH5 and Red Storm will make the deadline?

IBM's BlueGene is the king right now(well for the time being), but I don't see Big Mac(either version) beating the earth sim. Still, 2 out of the top 4 isn't bad.

If we're assuming the Army's COLSA MACH5 will meet the October deadline, then that would make it 2 out of the top 6 machines being G5 based. But if we're counting installed but not-yet-operational supercomputers, then we should probably include Red Storm (41.5 Rpeak). Does anyone know if MACH5 and Red Storm will make the deadline?

You needn't be so snippy about it. If you had done any research at all into it you'd know that it was, indeed, in the #3 position but wasn't ranked at all last time around because it was down for an upgrade. They're moving from dual processor G5 desktop machines in the cluster to all G5 Xserves and since all the nodes weren't up during the official ranking period it doesn't appear on the list. Look for it to make a strong appearance again in the near future.

You seem like the type that needs proof, so here's the previous list.

I sure dont see it given that the 'official' word back in 2003 is that it was 3rd fastest. The Top 500 list (June of 2004) I can't even find it on that page. And last, if it did reach the 10.6TFlops it'd be #5 after the 11.6TFlop BlueGene/I

Four teraflops? I thought the Earth Simulator was capable of 35-45 Gigaflops. http://www.top500.org/list/2004/06/

The peak of VTs System X cluster was about 17 Tflops, and the sustained rate was just over 10 (which rendered it the third place on the Top500 list).

Except that it's not on the most recent Top 500 list anywhere.

Remember how Va. Tech replaced all 1100 G5 nodes with G5 XServes a few months ago? Well, when you do something like that, you have to rerun and resubmit the benchmark. Va. Tech were not able to get the machine back together soon enough to rerun the benchmark in time to make the last list; there's even a big caveat about it on the Top 500 home page.

(It's also not clear that the original version of the Va. Tech machine ever did anything other than run that benchmark, but that's another matter.)

--Troy

The peak of VTs System X cluster was about 17 Tflops, and the sustained rate was just over 10 (which rendered it the third place on the Top500 list).

Except that it's not on the most recent Top 500 list anywhere.

Remember how Va. Tech replaced all 1100 G5 nodes with G5 XServes a few months ago? Well, when you do something like that, you have to rerun and resubmit the benchmark. Va. Tech were not able to get the machine back together soon enough to rerun the benchmark in time to make the last list; there's even a big caveat about it on the Top 500 home page.

(It's also not clear that the original version of the Va. Tech machine ever did anything other than run that benchmark, but that's another matter.)

--Troy

I know that when the Mac G5 Cluster was developed they claimed tremendous speed, but when the sustained rate was calculated, it turned out to be much lower ...

From what I know when Virginia Tech's G5 cluster's results were submitted they looked OK. You can see the results here. The measured result was about 58% of the theoretical peak which is on par with other similarly configured systems. Now, why Tech spent $5 mil and rushed to get this system put together for the November 2003 top500 list, and then dismantle the machine is another story. Maybe it was just an Apple advertisement like many /. posts. Dunno.

I know that when the Mac G5 Cluster was developed they claimed tremendous speed, but when the sustained rate was calculated, it turned out to be much lower ...

Speaking of the Big Mac (lame name), where is it now? I don't see it in the list and the the news page on their site doesn't list it as coming back on the 24th edition in November.

While we make every attempt to verify the results obtained from users and vendors, errors are bound to exist and should be brought to our attention.

While we make every attempt to verify the results obtained from users and vendors, errors are bound to exist and should be brought to our attention.

1) When the IA-64 design first became public, it was clear that they'd made some incredibly poor decisions. For example, the architectural design was based on the assumption that the chip would not do out-of-order execution in hardware. Such deficiences were to be remedied by a god-like compiler that would emerge at some later date. Unsurprisingly, it never has.

The Intel fortran, C, and C++ compilers for the Itanium for Windows and Linux are pretty godlike in my experience. Look at AMD benchmarks and usually they are done with the Intel compiler. Intel has also licensed the compiler technology to at least Microsoft for VC++.

2) These predictions were borne out by the fact that Itanium performance has always sucked, especially considering the enormous die size, cost and heat dissipation.

Your definition of sucked differs from mine. Yeah, the Itanium1s did suck. Yeah, the Itaniums put out tons of heat, but much of the current work at Intel is not with uping the MHz game, but in lowering power consumption and heat dissipation. Currently available examples are the PentiumM and the low voltage Itanium2 processors which are only available at 1GHz now, but the whole line is supposed to be available in 2005.

3) It looked like Itanium might win in the market despite its technical limitations, just because of Intel's vast marketing budget, its momentum, and its monopoly leverage forcing OEMs to stay away from technically superior alternatives like AMD64.

Coke does the same thing over RC cola. Windows does this over OS X. It just comes from being the market leader. People buy what they are familiar and comfortable with and can afford.

4) Thankfully this hasn't happened. The technically superior, open solution is winning. Thanks AMD.

No comment.

I dont think they have to much to worry about in that department this beast has way more than 32 processors.

Huh? Contiguous in what sense? Attached to the same motherboard? In one DIMM? Addressable in one chunk by the OS?

I've only been to one supercomputing conference, but when I was there most all of the people there ran clusters and the top500 site (although this list is produced by the same supercomputing conference people) lists many clusters there.

In other words, where does this contiguous memory requirement come from?

Theres no reference to WETA DIGITAL's second cluster, #77 on the list. It contains 588 computers as opposed to 504.
top 500 page for the cluser here
Why don't they just combine the two. That would surely grant them a top 10 spot...

It has 1080 processors (540 blade servers) and is at rank 80.

Weta's primary cluster (not for rent!) has 1176 processors and is at rank 77.

It has 1080 processors (540 blade servers) and is at rank 80.

Weta's primary cluster (not for rent!) has 1176 processors and is at rank 77.

What about all the other supercomputers in the world? Why are you so concerned about this particular site?

Hmm... if you look at the top 500 it appears that 80th was their previous place and that they have since upgraded their cluster to become 77th.

This is the top 500

The Read more here link leads to a few pity sentences framing lengthy excerpts from the IEEE article.

BTW, the machine in question, the Max-Planck-Gesellschaft MPI/IPP, is currently ranked 66th. It looks to be a fairly ordinary cluster with none of the exoticism that Cray says we so desperately need

What was big mac pushing, around 3-5 Tflop at a price tag of $6mil?

IBM scams the top500 with benchmarks replicated from one never-shipped-to-customers benchmarking-only cluster. Look at the numbers on the list... Notice a lot of them the same? Gee, I wonder how many customers actually ran (or could possibly get) those numbers. Yeah, you guessed it... NONE

The IBM cluster we have at our university is badly designed, maintained, and expensive. A strictly by the numbers underperforming heap of crap.
The IBM engineers actually de-tuned Linux with their alledged enhancements. Pretty funny.

IBM - no creativity. Very sad.

Wouldn't you think the government has had some non-public serious hardware dedicated to this for the past decade? If these are the Top 500 "known" supercomputers (http://www.top500.org/list/2004/06/), where are all the Defense Department listings?

The NSA has "worked" closely with vendors supplying encryption equipment since the 1990s (http://www.cnn.com/TECH/computing/9807/27/securit y.idg/

I would believe the government's NSA hardware is probably around 6-10 years ahead of what is commercially available or even known. http://www.hpcc.gov/pubs/blue94/section.4.6.html

Just curious. My guess is that Intel keeps pumping money into SGI to get Altix systems out and those who have them (LLNL and ...?) got them at practically no charge to run Linpack and look good on the Top500 list.

Hmm, maybe you should revisit linux cluster then - it all depends on what components you use to build your cluster. There are several mechanisms available today that will allow you to deal with the situations you describe - on the systems level, there is STONITH - if you are not that a machine is part of the cluster, kill it. Then on the application level there are a variety of solutions that can be utilised.

Do you honestly believe that the likes of (insert name of very big pharma, rhymes with "ein") or some of the examples in the top 500 will spend millions of dollars on DNS or Apache read only systems Supercomputers?

Scott McNeally must be jumping for joy everytime he sees FUD like yours printed - he sees his marketing dollars paid off handsomely. Linux clustering is a mature, solid and reliable solution to a large array of computational problems. There are problems that cannot be solved effeciently with a massively parrallel setup, and for those, you would look to the more expensive, but in it's own catagory, very competitive, NUMA system like the one designed by SGI. And it still runs Linux.

Does this mean that the applications running on the "old" clusters, presumably using some flavor of MPI to communicate between nodes, will have to be ported somehow to become multithreaded applications ?

NCSA still has plenty of "old" style clusters around. Two of the more aging clusters, Platinum and Titan are being retired, to make room for newer systems like Cobalt. Indeed, the official notice was made just recently--they're going down tommorrow. However, as the retirement notice points out, we still have Tungsten, Copper, and Mercury (Terragrid). Indeed, Tungsten is number 5 on the Top 500, so it should provide more than enough cycles for any message-passing jobs people require.

So, anyone has any insights as to why/how this matters for the programmers ?

What it means is that programming big jobs is easier. You no longer need to learn MPI, or figure out how to structure your job so that individual nodes are relatively loosely-coupled. Also, jobs that have more tightly-coupled parallelism are now possible. The older clusters used high-speed interconnects like Myrinet or Infiniband (NCSA doesn't own any Infiniband AFAIK, but we're looking at it for the next cluster supercomputer). Although they provided really good latency and bandwidth, they aren't as high-performing as shared memory. Also, Myrinet's ability to scale to huge numbers of nodes isn't all that great--Tugsten may have 1280 compute nodes, but a job that uses all 1280 nodes isn't practical. Indeed, untill recently the Myrinet didn't work at all, even after partitioning the cluster into smaller subclusters.

This new shared-memory machine will be more powerful, more convienient, and easier to maintain than the cluster-style supercomputers. Hopefully it will allow better scheduling algorithms than on the clusters too--an appaling number of cycles get thrown away because cluster scheduling is non-preemptive.

I'd also like to point out some errors in the Computerworld article. NCSA is *currently* storing 940 TB in near-line storage (Legato DiskXtender running on an obscenely big tape library), and growing at 2TB a week. The DiskXtender is licenced for up to 2 petabytes--we're coming close to half of that now. The article therefore vastly understates our storage capacity. On the other hand, I'd like to know where we're hiding all those teraflops of compute--35 TFLOPS after getting 6 TFLOPS from Cobalt sounds more than just a little high. That number smells of the most optimistic peak performance values of all currently connected compute nodes. I.e. - how many single-precision operations could the nodes do if they didn't have to communicate, everything was in L1 cache, we managed to schedule something on all of them, and they were all actually functioning. Realistically, I'd guess that we can clear maybe a quarter of that figure, given machines being down, jobs being non-ideal, etc. etc. etc.

As a disclaimer, I do work at NCSA, but in Security Research, not High-Performance Computing.

According to top 500 supercomputers 35 TFLOPS would put it ino second place after Earth simulation center - 35-40 TFLOPS

The odd thing is that in the AMD press release they note 30 AMD chips, but the top 500 site itself says 34 AMD processors. I wonder what the story is on the other 4.

Paul Sundling

Check out the June 1994 list. Ten years ago, supercomputers at about the 100th place on the list had gigaflop performance of today's desktops. Flashmob1, the University of San Francisco event in April that assembled a 180 gigaflop cluster in a single day, would have been at the number 1 spot. It's just cool to imagine the trend continuing, and it could, especially with wifi or wimax collective computing.

Use the Mirror