Slashdot Mirror

considerably more than doubling the performance of the current number 3 1152-node dual 2.4 GHz Xeon MCR Linux cluster.

Seti@Home already blows away all other supercomputers on the planet

Where did you get this idea? I'd honestly like to know. It's very misinformed.

In their defence, they did in fact have a decent sized list of cutomers, just no list of decent customers (where bleeding edge IT is concerned). I went into this with little SCO background, and was expecting something ... bigger. Their size really hit home at this thing. How they can call an OS that's run on some of the world's fastest boxes immature is now reely reely reely beyond me (where it was before only reely beyond me)...

I had no idea how much more powerful NEC's EarthSim was than the the "next best thing" was, as far as supercomputers go, but check Top500.org's current list (to be updated in November) out: NEC ES runs, at max, almost three times (!) the G-flops as the next runner-up. I always figured the supercomputer races would be like Cedar Pointe and their roller coasters... you know, somebody builds a bigger or faster one, so you build another that edges them out by just enough to reclaim the title. I had no idea NEC decided to take the "largest computational genitals, period" crown with such authority.

The TeraGrid is the NSF flagship for grid computing - be it good or bad.

The Grid.org people are some of the former SETI@home people gone more general purpose.

And of course, there is The Global Grid Forum which is meeting in Chicago in a week or so. GGF is the standards behind the Globus enabled grid.

We could ask why CERN/etal couldn't have come up with a slightly more imaginary name?

We can also ask why NSF are such suckers for the last 20 years of hype from the people who have run the national supercomputer centers in the USA? Ditto congress. But that is a (sad) story for a different day.

And finally we can ask what Top500.org is going to do when people begin reporting HPL benchmarks using these things? That HPL became the standard that people are designing supercomputers around argues just how totally screwed up high performance computing really is at the moment.

-- Multics

HP is very linux friendly. Look at their linux page. They support other distributions besides RedHat even like Debian and Suse and possibly others. They have linux drivers for winmodems on some of their PCs. On their Itaniums, 3 OSes are supported HPUX, Linux, and Windows.

Plus don't forget that HP Offers Linux Purchasers Indemnification for its customers using Linux on HP hardware.

They may be a mostly M$ shop, but they do have interests in other OSes. In fact, HP is the vendor with the most systems on the top500 list, and not one of those machines runs windows...

-
All you have to do if you want to show the scientific community's interest in Linux, is to show the scientific community's use of Linux: The 500 most powerful computer installations in the world... many of which run Linux, including the the 2nd fastest system in the world (and all of which are used by members of the scientific community).

You could also use these simple searches on slashdot (here is another) and google to collect some very interesting data.

For example, here's a nice tidbit that may be the exact community you're looking to impress.

Granted, these do not give you a timeline, but it should be enough data for you to be able to ask them "Why are we not using Linux?".

I believe they were in a rush to beat the cutoff date for the Linpack Top 500 Supercomputer List

Apple gave them the performance and, I'm sure, the sweet deal they needed to make it happen.

ROFL! Hurd. Hah. You made my day!

I stumbled on this a while back -- it's Linus' Usenet announcement of Linux back in 1991. But the part that's priceless is:

I can (well, almost) hear you asking yourselves "why?". Hurd will be out in a year (or two, or next month, who knows)

That was 1991! Nowadays Linux is running on everything from the biggest computer systems in the world (3rd fastest supercomputer (unclassified) in the world is running Linux -- here) to people's Tivos and PS2s.

I think the SCO Information Minister should try holding his breath until Lawrence Livermore National Laboratory buys a mountain of licences for the 2,304 processors in the MCR Cluster...

No one has told me why the timeline is so short for getting this thing up and running...

I hear they are rushing to try to get included in this year's Top 500 Supercomputers list.

As for the setup of the machines - even going through the Apple Store you can change the DVD-R for a Combo. I guess with a 1100 machine order, Apple lets you choose more (or rather less) than just that. Which brings us to the desktop boxes - many of the PC clusters in the Top 500 Supercomputer Sites list are build with plain-jane desktop PCs, why can't one build one with "gold-plated, fashionable Apple desktop boxes"? Because you say so?

If you look at the top500 list, you see Lawrence Livermore's Linux cluster is at 3rd place. It's an 1100 node cluster of 2.4 GHZ P4's. Looks like Apple is a bit late to the party if they're only shooting for #5.

"By the mid-nineties the new breed of computers made from off-the-shelf commodity chips arrived on the market. Those funded from the US ASCI programme and consisting of several thousand CPUs grabbed the headlines, but because of communication and memory bandwidth limitations, they often deliver very little of their potential peak performance to the user."

For truly high speed clusters, the per port cost of the interconnect is a big piece of the puzzle. Thus you still want the maximum per processor performance. If you look at the Top 500 list you will see a number of highly ranked Intel machines. This is a very big recent development but they are all Xeon and Itanium systems, not budget Pentium 4s. So the G5 is price competitive, especially for floating point apps where PowerPC has always shined.

It would be a big coup if Apple can pull off even a top 10 finish, and it's not out of the question. If you'll look at the second decade of the Top 500 list you'll see several ~1000 processor pSeries 690 Turbo clusters. These are all 1.3GHz Power4 machines, the sister chip of the PPC970. Double the number of processors, each with roughly 1.5x clockspeed, and you're in the ballpark of a top 5 finish. The recently announced IBM FORTRAN compiler will help a lot with this. The open question will be whether the interconnect, which is rumored to be Infiniband, can keep up. If it works, this will sure sell a lot of G5 Xserves!

For truly high speed clusters, the per port cost of the interconnect is a big piece of the puzzle. Thus you still want the maximum per processor performance. If you look at the Top 500 list you will see a number of highly ranked Intel machines. This is a very big recent development but they are all Xeon and Itanium systems, not budget Pentium 4s. So the G5 is price competitive, especially for floating point apps where PowerPC has always shined.

It would be a big coup if Apple can pull off even a top 10 finish, and it's not out of the question. If you'll look at the second decade of the Top 500 list you'll see several ~1000 processor pSeries 690 Turbo clusters. These are all 1.3GHz Power4 machines, the sister chip of the PPC970. Double the number of processors, each with roughly 1.5x clockspeed, and you're in the ballpark of a top 5 finish. The recently announced IBM FORTRAN compiler will help a lot with this. The open question will be whether the interconnect, which is rumored to be Infiniband, can keep up. If it works, this will sure sell a lot of G5 Xserves!

>> A box designed to be separate just will not have the latency advantage of a supercomputer designed from the ground up.
I suggest you look at the list of the top supercomputers in the world. Most are clusters, ie. separate, distinct machines (just a quick glance shows the top 25 all are). It's just too darn hard to make a shared memory computer with 1000's of processors. So the common architecture is to make a cluster of smaller shared memory machines.

Besides, most clusters built utilize special interconnects like Myrinet that offer low latency connections. They're more expensive than ethernet, but it's a supercomputer so you spend it.

>> All this "the internet is one giant distributed computer" doesn't acknowledge this.
On the contrary... people know this very well. That's why we see rendering and SETI processing as distributed. They don't really need to communicate with others often.

This history predates the rise of the beowulf cluster. Nowdays most of the new supercomputers are merely big stacks of rackmounted PCs with fast interconnects running Linux. If you look at The Top500 trends you will see that the number of intel supercomputers in the top 500 is up to 119 from 56 six months ago, and that the number of clusters has gone from 93 to 149 over the same period.

Well, I work for one of the companies that sells "classified" computers to the government. Typically, the sale itself isn't classified (especially since Cray is a public company) and sometimes, there's even a press release (that's one of ours from this spring). What "classified" usually means is that access to the system and the data on the system is classified. I don't have a security clearance, hence I can't look at, say, a crash dump from one of those sites. So, for your questions:

1) The NSA, Army, various other US and foreign government agencies.

2) Cray, SGI, IBM, HP (look at the Top 500 list for a good reference) and others. The Top 500 even lists a number of systems as "classified".

3) Uh, well, people *do* know about them.

Not really. The top500 list lists LANL's ASCI Q at 20.48 Peak TFLOPS and LLNL's ASCI White at 12.29 Peak TFLOPS.

You like my watch? It's #135 on the Top 500 Supercomputers chart.

I checked this list and could find nothing older than seven years, and those machines were in the 300's. The oldest in the top 100 was six years old and it was 81st, and a Cray, BTW.

Usage..... Count Share Rmax Rpeak Procs
Industry... 202 40.4 % 82398 182964 62869
Research... 131 26.2 % 187689 278030 120046
Academic... 115 23 % 77143 133564 45216
Classified.. 27 5.4 % 14167 20691 12892
Vendor...... 22 4.4 % 11033 15545 5230
Government... 3 0.6 % 1317 2256 528
Total...... 500 100 % 373749 633052 246781

Usage..... Count Share Rmax Rpeak Procs
Industry... 202 40.4 % 82398 182964 62869
Research... 131 26.2 % 187689 278030 120046
Academic... 115 23 % 77143 133564 45216
Classified.. 27 5.4 % 14167 20691 12892
Vendor...... 22 4.4 % 11033 15545 5230
Government... 3 0.6 % 1317 2256 528
Total...... 500 100 % 373749 633052 246781

Oh yeah, real concerned. The top US supercomputer can only do 20 TFLOPS or so. That will never do.

Imagine a beo...

This should do a lot for AMD's credibility as a server processor manufacturer. According to the current top500 list, you have to go to number 84 to find an AMD based supercomputer. If these articles are correct, you'll soon have 2 in the top 5. That's quite a change of events.

Interesting to note is that #3, #6, and #8 are all linux clusters. All three of which are at Livermore.

Cray's X1 also debuted, but it was much lower @112. However, it ought to be noted, that the examples out so far are only 60 processors at tops. As soon as the money gets ponied up, prolly at ORNL, they'll be waaaay up towards the top. My guess is, if all goes as planned, they'll be at #15 by year's end.

What I find exciting these days is actually the High Productivity Computing Systems Effort, the Blue Planet or Blue Gene. These are a little ways off from being on the Top500 list yet though. :D

I do wish there were more SC companies doing hardware development in the US. I love Cray, but a single vendor smacks of eggs in one backet syndrome...So, geeks, if ya wanna start a startup with a design, go for it...Betcha the NSA (aka Cthuhlu of HPC) would be happy to sponsor ya...;)

Yeah, not to mention the other 130-odd AIX clusters in the top 500 supercomputer systems in the world., some of which perform classified nuclear-explosion simulations and other fun things.

Not to mention also that JP Morgan Chase, Citigroup, Discover Card, and (IIRC) Bank of America all use AIX extensively for all sorts of financial applications.

Not to mention also that Nortel Networks uses Motorola FX fault-tolerant systems running AIX (licensed in turn from IBM) for their adjunct processors that collect information about phone calls...so a good portion of those telcos and large companies/ISPs who have a Nortel switch probably have at least one critical AIX system connected to it...

Talk about collapsing economy, try collapsing national infrastructure. Welcome to the slippery slope of professional risk management.

In terms of number of processors, ASCI Red at Sandia has had > 9000 Intel pentium pro (and them pentium II Xeon) procesors since the late 1990s.

It's still # 15 on the top 500 list

It's very difficult to make money on software in the Beowulf arena because, duh, it's FREE! You have to make your money on hardware and integration of the hardware and software. Seems that there'll be lots of overhead with all of the developers now on hand at Penguin. Maybe this is why the CEO of Penguin, Marty Sayer, left 2 years ago and is now a VP at AMD.

In addition, for the most part Clustermatic does the same thing and is setup the exact same as a Scyld distribution, granted Scyld does add some neat things of their own. Although Scyld actually has turned a profit here of late, don't get me wrong, I like Becker but I really don't see this one working out in the long run.

The National Supercomputer Centre in Sweden use multiply Linux beowulf clusters as well as supercomputers like the SGI Origin 3800. The fastest Linux cluster places 51 among the 500 fastest computers in the world.Monolith consists of over 200 PC computers. Each computer has two Intel Xeon processors at 2.2 GHz, 2 GBytes primary memory (ECC DDR). They're all used for research and development.

The advanages of clusters is allot of power at low cost. How well they perform compared to ordinary supercomputers depends on the task.

• IBM eServer pSeries 690 Turbo: 1462 MFlops/sec.
• Intel Pentium 4, 3.06GHz: 1414 MFlops/sec.
• Cray T94: 1129 MFlops/sec.
• Cray Y-MP EL: 41 MFlops/sec.
• Pentium Pro 200MHz: 38 MFlops/sec.
• Apple Macintosh: 0.0038 MFlops/sec.
• Palm Pilot III: 0.00081 MFlops/sec.

That's how much floating point work the CPU can do per unit time. That's the no-hype benchmark for CPUs.

Its really nothing huge, but you also need to consider the cost involved. AU$1.7 million is about US$1.1 million. So for about a million US$1 you could get in the top 100 supercomputers in the world.

Looking at the latest top 500 list this would put it as the third most powerful 'self-made' system in the world. For that reason I think it deserves at least a mention and add the relatively low cost and you've got a /. story.

Nahhh, that's not a computer. Now this, this is a computer.

Mike

And it's not as simple as hardware/software prices/support. There's some critical stuff that cannot and does not show up in the specs, and it's not cheap.
I agree. Perhaps, after all Linux can not seem to run very top-end systems. Worse yet, you will not find it in the enterprise systems As to the Desktop, Well skip that as well

You can solve yesterday's problems on tomorrow's computers quite cheaply.
Same thing here as well (with out the sarcasm). Tomorrow's problems are being solved on todays computers due to their low cost. Otherwise, we would be waiting till the costs of the computers were less than the costs of the problem.

If you look at the 1999 chart carefully, you'll notice that all the SGI machines in the top 150 are Cray vector computers. SGI got out of the vector supercomputer business a few months after this chart was compiled, selling all its Cray IP to Tera Computer. So even if this line of computers had managed to maintain its dominance of the Supercomputer market, it wouldn't matter to SGI.

So much for your big conspiracy theory.

If you look at the 1999 chart carefully, you'll notice that all the SGI machines in the top 150 are Cray vector computers. SGI got out of the vector supercomputer business a few months after this chart was compiled, selling all its Cray IP to Tera Computer. So even if this line of computers had managed to maintain its dominance of the Supercomputer market, it wouldn't matter to SGI.

So much for your big conspiracy theory.

You're talking about CrayLink, which is the memory sharing technology in SGI MIPS supers. Supposedly SGI bought Cray just to get the patent--a purchase that cost them a lot of money, very little of which they got back when they sold Cray off again. But IBM and Sun have similar technologies. Whether they're as good as CrayLink, I couldn't say. Still, SGI Origin series no longer dominates the supercomputer market.

Any one notice the picture of the Earth Simulator at the top of the article.

While it's not nearly the whole story, an interesting benchmark is the top 500 supercomputers list. Sun and linux are both on the list. Sun seems to be the main supplier of supercomputers these days. But they first appear on the list as number 156. There are only two linux systems, but they are numbers 5 and 45.

I guess that doesn't tell you all that much about the top end, other than that linux and Sun are both quite capable of supporting raw, top-end gigaflops power. The actual computing is done by processes, of course, so this benchmark really just tells you that neither OS has any showstoppers. Neither interferes with the ability of a process to crunch bits and bytes.

Now on to the other benchmarks ...

IBM already sells a more powerful server than anything Sun sell, and it runs linux as well as AIX, the p690 - check out the world top 500 computers to see the list of p690 clusters at http://www.top500.org/list/2002/11/

I guess 95% of these will be running purely AIX, but im sure quite a few will have linux installed alongside in a partition.

Ewan

All two of them? How many G3 / G4 can you find on this list...? Yes, didn't think so.

It's just a name for a really powerful computer to be used by scientists to run models of whatever it is they're studying. These things don't get covered in mainstream news. If you want more information on supercomputers, go to http://www.top500.org/ (No, your G4 didn't make it).
But as for the name "Earth Simulator", that's exactly what it is, a name. Who knows, in Japanese its name could be The Matrix.

Looks like it's been #1 since at least June of last year.

http://www.top500.org/list/2002/06/

Well that would, of course, depend on how much bandwith is running to said server now wouldn't it? =)

Found some nice info (good old google) on said Supercomputer though since the sites linked article didn't have much.

A Time Article on The Earth Simulator

Top 500 page on Earth Simulator

NEC page on the Earth Simulator

Google Translated Powerpoint presentation on the Earth Simulator

A snippet(s) of info:
"Based on the NEC SX architecture, 640 nodes, each node with 8 vector processors (8 Gflop/s peak per processor), 2 ns cycle time, 16GB shared memory. Total of 5120 total processors, 40 TFlop/s peak, and 10 TB memory. "

"Earth Simulator's processors are one-chip LSIs fabricated with 0.15 micron CMOS process and copper wiring. Highly optimized software and high-speed networks that pump massive amounts of data through 7.8TB/s bandwidth connecting the 640 processing nodes are key to the amazing efficiency of Earth Simulator."

_ALL_ Sun servers are very stable, but slow. SPARC speed is poor, take a look at SPEC CPU2000 Results. The memory bandwidthis _very_ low. In Linpack-top500 you won't see SUN in the 100 first places.

The Fujitsu SPARC64 V is better chip and 100% compatible with SUN solaris/SPARC. And better servers with 128 CPUs !!!!

LiNUX is a better alternative below 8 CPUs: Migrating Oracle9i - Based Sun Servers to Dell Servers Running Linux and Migrating Oracle9i - Based Sun Servers to Dell Servers Running Linux, Part 2. LiNUX+x86/ia64 , and soon AMD x86-64, is cheaper and faster than Solaris/SPARC

DEC/Compaq/HP have the best chip (Alpha EV7) and the best UNIX servers (ES47,ES80,GS1280) in RISC arch. It's a pity that Alpha is going to die to put intel ia64 instead.

And if you need NUMA machine, SGI Altix is for you.

Why do you need to buy a SUN server?

Lawrence Livermore's new Linux-based cluter is currently the fifth most powerful supercomputer in the world. I can't speak to your experiences, but 11.2 Teraflops, 4.6 TB of memory and 138.2 TB of storage sounds pretty "super" to me. There are many more lesser Linux-based high-end computers as well. Google should give you a nice list.

Trouble is, you haven't seen/worked/heard about everything, so blanket statements like yours are typically what turns out to be blatantly false. Get some facts, and try to stay away from sweeping judgements.

-B

Clustering has definitely won out in the United States mostly due to the appeal of cheap processing power, but that doesn't mean that clustering is always best. Like another poster mentioned, it depends on what you're doing. For Google, clustering is probably a good solution, but for high end supercomputing, it doesn't always work.

Check out who's on top of the TOP 500 supercomputers. US? Nope. Cluster? Nope. The top computer in the world is the Earth Simulator in Japan. It's not a cluster of lower end processors. It was built from the ground up with one idea -- speed. Unsurprisingly it uses traditional vector processing techniques developed by Cray to achieve this power. And how does it compare with the next in line? It blows them away. Absolutely blows them away.

I recently read a very interesting article about this (I can't remember where - I tried googling) which basically stated that the US has lost it's edge in supercomputing. The reason was two fold: (1) less government and private funding for supercomputing projects and (2) a reliance on clustering. There is communication overhead in clustering that dwarfs similar problems in traditional supercomputers. Clusters can scale, but the max speed is limited.

Before you start thinking that it doesn't matter and that the beowulf in your bedroom can compare to any Cray, recognize that there are still problems within science that would take ages to complete. These are very different problems from those facing Google, but they are nonetheless real and important.

_ALL_ Sun servers are very stable, but slow. SPARC speed is poor, take a look at SPEC CPU2000 Results. The memory bandwidth is _very_ low. In Linpack-top500 you won't see SUN in the 100 first places.

The Fujitsu SPARC64 V is better chip and 100% compatible with SUN solaris/SPARC. And better servers with 128 CPUs !!!!

LiNUX is a better alternative below 8 CPUs: Migrating Oracle9i - Based Sun Servers to Dell Servers Running Linux and Migrating Oracle9i - Based Sun Servers to Dell Servers Running Linux, Part 2. LiNUX+x86/ia64 , and soon AMD x86-64, is cheaper and faster than Solaris/SPARC

DEC/Compaq/HP have the best chip(Alpha EV7) and the best UNIX servers (ES47,ES80,GS1280) in RISC arch. It's a pity that Alpha is going to die to put intel ia64 instead.

And if you need NUMA machine, SGI Altix is for you.

Why do you need to buy a SUN server?
- because my programs _only_ run with solaris/sparc

News.com is reporting that IBM has won a $290 million contract with the federal government to build what are expected to be the world's two fastest supercomputers at Lawrence Livermore National Laboratory. The other machine, the Linux powered Blue Gene/L will be 10 times faster than the current #1 , NEC's Earth Simulator with a speed of 360 teraflops, according to IBM.