Slashdot Mirror

The article got it wrong. The best they could do a year ago was actually 43rd on the cluster here at PKI (I'm a student). top500.org lists it as running Linux but actually runs both RHEL and HPC Server concurrently.

The FLOPS are measured by a benchmark program, The Linpack Benchmark that runs under OS overhead, so it would differ with different OSes and probably different configuration parameters in the same OS. I wouldn't be surprized if MS hadn't supplied significant engineering support to get the system tweeked to the T to nail down good numbers on the benchmark suite.

What's missing in the article is that there are only a few windows-based systems in the top500 and there numbers have been declining over the years.

Actually, Microsofts share has increased, they went from nothing to 5 installs in a few years.
http://www.top500.org/stats/list/32/osfam

"OS Family" "Count" "Share %"
Linux 439 87.80 %
Windows 5 1.00 %
Unix 23 4.60 %
BSD Based 1 0.20 %
Mixed 31 6.20 %
Mac OS 1 0.20 %

I congratulate Microsoft on making the top ten. I'm not sure if the 5 HPC Windows installations do anything useful other than provide PR for Microsoft Marketing (TM). This is from a company that charges a CAL to print to a server.

That being said, I'm pretty sure the Microsoft solution won't allow you to mix and match different computers (and OS to a certain extent) like you can do with Linux HPC. Knowing Microsoft, you can't reuse your valid NT/2000/2003/2008 server licences within the cluster. Past history has shown any update from Microsoft will take down the whole cluster instead of a single node (London Stock Exchange). Microsoft probably will provide better cluster management software making it a better choice for customers requiring HPC without having in-house HPC knowledge.

Linux Magazine has some good articles on HPC for linux http://www.linux-mag.com/solutions/hpc

Enjoy,

Problem is they've missed the boat. Linux already has compilers for multiple CPUs

Look at this chart..

http://www.top500.org/stats/list/32/os

Windows HPC 2008 is on 4 machines out of 500. (+1 is windows 2003 if you want to count that)
Linux is on 454 out of 500 super computers

Which Operating System do you think is going to have better tools to support Super Computing?

Also I am hoping you mentioned Direct3D as to get a point across and you're not suggesting that Direct3D be used on these machines?

Researching the Shanghai Supercomputer Center, they were #10 in 2004 but have had same system for about 3 years now. Looks like they finally got a new system and were able to get #10.

In very low end(cell phones, PDAs, etc.) this doesn't hold and in some high performance or high reliability scenarios(mainframes, exotic supercomputer architectures) it is also not the case

x86 accounts for 429 of the fastest 500 computers in the world. I think it's pretty well suited to high-end computing.

According to this it's on 439 of them, 87.80 %. But that seems a bit general, maybe it's counting Solaris as Linux? Or would that be in Unix?

More info here: http://www.hpcwire.com/topic/processors/Tokyo_Tech_Boosts_TSUBAME_Super_with_GPUs.html

From the list http://www.top500.org/stats/list/32/osfam
  the Linux share is 87%, Windows is 1% (Less than the Desktop share of Ubuntu).

CNL (Computing Node Linux), SLES (SUSE Linux ES) are Linux as well. The only ones that aren't Linux are AIX, Windows (duh!) and Others, which add up to 11.2% in 2008 from the charts, otherwise confirmed from here

Here's the droids you've been looking for:
http://www.top500.org/charts/list/32/os

Linux has dropped to only 77.8%, although specific other linux versions make up another 5% (approx)

And windows has had a huge increase to .2%

Good going! :)

The only benchmark that matters when it comes to speed and optimisation is the top 500..

Linux - 378 - 75.60 %
MacOS X - 2 - 0.40 %

HPC is also not trending toward Windows even though Steve Ballmer thought it would.

We can speculate about why, but I do so like that first graph link. I wonder if desktops are going to swing like that, or if laptops are beginning to take that curve.

Our IT deparment must be nuts for running Windows Compute Cluster on their system then. Running on 4604 cores no less. It's also the largest Windows cluster (2:50 mark) in the world (and before anyone's head explodes, yes it does run Linux).

How much the cluster is actually used on the Windows side I don't know, but it is there and available for use.

You obviously never used any of the big DoD supercomputers. Have a look at the (old, by now) ASCI Red configuration here. It has two interfaces, one for classified jobs and one for non-classified jobs.

This was the same state Unix was in around the early 1990s. We're not dead yet! In fact, we've taken over the large computer market since then.

Ahem. Linux Is Not UniX. Linux owns the big iron these days, holding over 85% of the Top500. It's pretty dominant on the small end too, with home routers and file servers being the extreme of that bracket. The middle is getting squeezed out as thin-is-in netbooks and nettops push into the mainstream.

ISO has lost its street cred so expect an Open Source replacement. Open Standards benefit everyone, so I expect someone to fill in the gap.

Unix was never open source until Open Solaris (the provenance of which is still subject to vigorous debate).

But of course you knew that. I was a Unix admin in 1984. At the time it was the stuff. Unfortunately because it was born before the age of software as property it wasn't designed to be protected from the greatest threat progress has ever faced: intellectual property lawyers. Linux was.

This was the same state Unix was in around the early 1990s. We're not dead yet! In fact, we've taken over the large computer market since then.

Ahem. Linux Is Not UniX. Linux owns the big iron these days, holding over 85% of the Top500. It's pretty dominant on the small end too, with home routers and file servers being the extreme of that bracket. The middle is getting squeezed out as thin-is-in netbooks and nettops push into the mainstream.

ISO has lost its street cred so expect an Open Source replacement. Open Standards benefit everyone, so I expect someone to fill in the gap.

Unix was never open source until Open Solaris (the provenance of which is still subject to vigorous debate).

But of course you knew that. I was a Unix admin in 1984. At the time it was the stuff. Unfortunately because it was born before the age of software as property it wasn't designed to be protected from the greatest threat progress has ever faced: intellectual property lawyers. Linux was.

You can go to the store and buy an off-the-shelf PC today which will do 2 teraflops.

...no. even the best cell processors are pushing 50 gigaflops, if I recall. typical home CPUs are in the 5-10 gigaflops range.

the number 500 supercomputer has an Rmax of 9 teraflops; it wasn't long ago (~4 years) when half a teraflop would get you on this list.

#s 1,2, and 3 as of June of this year according to http://www.top500.org/list/2008/06/100 [top500.org]

From the same source I can pull stupid stats out of my ass and make some useless remarks too : PowerPC more popular than AMD CPUs - What you should buy for your next home computer! or maybe "0% of the top 500 computers use Nvidia 9800 graphics cards - gaming is a dying on the PC"

Linux runs 85% of the top 500 computers in the world troll. Maybe it's your skills that are lacking.

A hundred years? BlueGene/L has 213k cores - a mere factor of three increase shouldn't take long.

http://www.top500.org/list/2008/06/100

HP has had for more than a decade their own Linux distribution called "Insight Diagnostics". You can download it from HP.com and use it to do diagnostic tests on almost all the hardware that HP computers come with. It's special purpose and has no install to HD option. This disk includes at the least drivers for almost all HP provided hardware, even if it's third party. Most come with a physical disk, but you can always get the ISO from HP.com no matter what other OS the PC was shipped with.

I really don't understand why HP Linux wasn't mentioned in TFS. You can get all HP servers with Linux, and most desktops as well. HP has thousands of FOSS projects they fully fund. They partner with standards organizations at the highest levels. You can get many different distros from them including Debian, Mandrake, RedHat, RedFlag and SuSE. Almost all of the 36% of the top500 supercomputers that use HP nodes are Linux. They don't make a big deal of it, but the option is there. Is this a matter of complaining about the people who are on the fence so we can get them to change?

/Disclosure: I don't work for HP, but I fix most of 'em - laptops, desktops, workstations servers and blades (but not superdome or nonstop). My opinions are most definitely not my employer's.

The kind of work that you're talking about would more likely be done on Road Runner at LANL, instead:
http://www.top500.org/system/9485
Both are petascale systems, but they are different tools for different jobs.

I wonder how much power this sucker is gonna drain from the local power grid

Judging by the current machines, the most power-efficient one using some 19.1 Watts per core, and assuming further advances in efficiency (say down to 10 W per core by 2011), about 2 GigaWatts.

Yeah, that was my thought. Roadrunner at Los Alamos sits at the top of the 500 list with Rmax 1,026,000. I don't know enough about benchmarks to distinguish between "Rmax" and "sustained petascale," but it is achieving over a petaflop. Maybe someone here can tell us more about linpack vs. whatever they're using for this new one. I notice the article linked in the story mentions Roadrunner at the end, but without saying how it compares in speed. It doesn't seem to say by what specific measure this new computer's speed surpasses a petaflop.

Yeah, that was my thought. Roadrunner at Los Alamos sits at the top of the 500 list with Rmax 1,026,000. I don't know enough about benchmarks to distinguish between "Rmax" and "sustained petascale," but it is achieving over a petaflop. Maybe someone here can tell us more about linpack vs. whatever they're using for this new one. I notice the article linked in the story mentions Roadrunner at the end, but without saying how it compares in speed. It doesn't seem to say by what specific measure this new computer's speed surpasses a petaflop.

What if you need real uptime with a load of 80 on a 32 cpu system? Can Linux handle the load and have years of uptime?

Over 85% of the top 500 super computers in the world run Linux. http://www.top500.org/ as best I can tell almost none run Solaris as most of the Unix is AIX. So all you "Linux's uptime, stability and processing power sucks compared to Unix" old ass fanboys go back to your clubhouse and cry.

Ask any *real* Unix admin who uses both and more than likely they will say Linux is great for small jobs but Solaris is king for anything else.

Wow...if you are a *real* Unix admin it is no wonder Linux came along and is so successful.

In Sweden we already have this extension. At least
the law passed quite recently that gives the
Swedish National Defence Radio Establishment right
to eavesdrop on all Internet traffic crossing
Swedish borders.

BTW, the NDRE bought one of the worlds most
powerful super computers, it is now in 11th
place, before the law was passed. How could
they be so sure that they would be able to put it
to Good(TM) use?

I think this part of the computing timeline is going to be
one that is well remembered. I know I find it fascinating.

This is a classic moment when tech takes the branch that
was unexpected. GPGPU computing will soon
reach ubiquity but for right now it's the fledgling that is being
grown in the wild.

Of course I'm not earmarking this one particular project
as the start point but this year has gotten 'GPU this' and
'GPGPU that' start up events all over it. Some even said
in 2007, that it would be a buzzword in 08.

And of course there's nothing like new tech to bring out
a naysayer.

Folding@home released their second generation
GPU client in April 08. While retiring the GPU1 core in
June of this year.

I know I enjoy throwing spare GPU cycles to a distributed
cause and whenever I catch sight of the icon for the GPU
client it brings the back the nostalgia of distributed clients
of the past. [Near the bottom].

I think I was with United Devices the longest.
And the Grid.

Now we are getting a chance to see GPU supercomputing
installations from IBM and this one from MIT.
Soon those will be littering the Top 500 list.

I also look forward most to the peaceful endeavors the new
processing power will be used for... weather analysis,
drug creation, and disease studies.

Oh yes, I realize places like the infamous Sandia will be using
the GPU to rev up atom splitting. But maybe if they keep their
bombs IN the GPU it'll lessen the chances of seeing rampant
proliferation again.

Ok, well enough of my musings over a GPU.

-AI

Since you are citing anecdotal evidence, so shall I. I am not a zealot. While I use and prefer a Macintosh, I have made use of windows machines quite often, even in the 'creative' fields. I set up a few Gigastudio computers in mac-only sound studios.

Let me see...while graphics are common, there are some other uses:

Gov't Agencies:

The Military Sea Lift Command used/uses Macs for inventory management. I bought some surplus G4s and a 9500 series server off them. I don't know their current system, but they tend to use some custom inventory code, so they still might be on macs.

Let's not forget the US Army contractor COLSA, which bought a 1,566 Xserve cluster to run aerodynamic simulations.

Small businesses:

I do hardware maintenance for some sound studios, all Mac based. This is expected. The contractor who did the build-out for one of the studios uses a Mac laptop. The company that handled the specialty double-pane window frames for the control room was Mac based. The Piano tuner used a windows computer though.

The video rental store near me uses Macs as their checkout platform.

Schools:

Go to any university where Macs and PCs share a lab. The macs almost always fill up faster than the PCs. I know that American University uses a couple XServes as part of their IT needs.

And you say Apple doesn't have a place in web servers, datacenters or programming? Let's go over to Virginia Tech, why don't we. Take a look at : http://www.top500.org/site/systems/2024. How does 1,100 Dual 2.3 GHz XServes on an Infiniband network sound? That sounds like the 280th fastest supercomputer in the world to me. When it was introduced in 2003, it was number 3. That, to me, looks like a pretty strong showing of the datacenter capability of the Mac.

-----
Macs are not a total solution, but they work in FAR more areas than you give credit for. It is not based on zealotry either. If Linux or Windows could give me the same TCO, and the same ease of maintenance, I would switch in a second.

The myth that 'serious applications' only run on PCs is FAR from accurate. For financials, there is MYOB and Quicken. For graphics there is Vectorworks and BRL-CAD. The list goes on. When you start to talk about "serious functionality" and programs that require proficiency beyond the OS, you need to realize that the platform should aid in that functionality of that software, not hinder it. In my experience, and in that of many power users, Windows hinders, OS X helps, and Linux can go either way.

Ok, you've got $50B in the bank and you want to build the ideal desktop, server, supercomputer and portable device OS for all people. In addition to the money you have accumulated the world's largest collection of professional programmers, systems engineers and managers from around the globe. You've got a first class distribution network, peered server farms and media pressing manufacturers the world over. You've partnered with every major OEM since the beginning of time, so you have full specifications for all the hardware there is. Let's have some criteria...

1. Take out the trash. Interpret that as you will.
2. Now is a good time to consider security. Now keep it in mind throughout the rest of this post. It's hard, I know. Try. Try really hard. If you make a good start the NSA might help you. If anybody knows about IT security, they do.
3. Employ some black hats to keep you honest on security. Give them fiat to break your stuff. Pay them well to keep your secrets, then try to compromise them with strippers. If it doesn't work they'll still respect you and you'll have improved the strippers' economy.
4. Architect your solution using proven practices -- separate functions by critical elements and include only the necessary in the core system. Use peer review. You can afford a subscription to Communications of the ACM. When your system architects have mastered the patent expired (pre-1990) material, they might be ready to lead a team of programmers.
5. Choose a good set of toolchains that include every programming language since CP/M was kicking your butt. Because developers don't like to be told which toolchain to use.
6. Cross platform is not "runs in the last two versions of Windows". Make sure the thing can be ported to every hardware architecture there is -- including systems designed to prevent just that like the XBOX. Include alien processors and systems like Sparc, Power and Cell. Don't forget to include obscure crap like SNOBOL and APL - the few freaks who use that stuff really love it. Remember that every build must run in the user's choice of VM environments. When you let the users do what they will, they do the most amazing stuff.
7. Allow for multiple user interfaces based on user choice -- web-based, terminal based, GUI are only major categories of options, not individual choices. Some people like thin clients, so make sure they're supported fully.
8. Choices are not hierarchical. The subchoices of a major choice often overlap in interesting or useful ways.
9. Build a separate version for every conceivable field of endeavor. Archaeology? That's going to need GIS software, modelling software, a good browser and office package and a hundred other things. Make that many separate versions. Even specialists like choice. Architects? That's another suite. Don't be stingy. Every desktop needs an office suite or three, a CAD program, several browsers for the users to choose from and many other things. That way end users (or network admins) can choose. Try to get each one to install from a CD if you can, or a DVD at worst. Larger volume distribution media should be reserved for distributions that also include considerable multimedia content.
10. Make sure the thing scales from the feeble 386 processor available in some old embedded devices to the largest supercomputer currently in use, with additional consideration for how extreme the next 20 years might expand that horizon. Absurd Limit Theory is your friend.
11. Let go of stupid licensing. Your product's licensing cannot be so obscure that it takes three months with legal six months into a project to discover that licensing is not available for this use. It's also not acceptable

http://www.top500.org/system/8757

Look at the description. Does it run RH? If it exports a Lustre filesystem, I think Lustre only runs on *nix.

Does anyone know the real implementation details behind this system? Is it part Linux, part Windows? Was it linux and now Windows? Did they port Lustre to Windows?

Tha majority are around the 1-4k mark, but in the top 25 the range is from 6720 cores to 212992 cores. Only 2 entries in the top 25 have fewer cores than Microsoft

http://www.top500.org/list/2008/06/100

Basically, it's all brute force if you want to get into the top 25.

First, the Top500 list has plenty of value. What most people do not realize (or should realize) is it is one data point on the HPC spectrum. If your HPC program does not perform the same or similar matrix operations as HPL then the ranking is meaningless to you. To some the list has become a public relations contest.

Second, performance is virtually independent of the OS (unless you are using TCP). Most big clusters use InfiniBand and run applications in "user space" by-passing the kernel. The rest of the code is crunching numbers.

Third, for the right cost, anyone can get a system on the Top500 list. It is a rather simple price/performance calculation, by the way. Breaking into the top 10 might be a little more difficult.

Finally, HPC and Linux are synergistic. Take a look at Why Linux on Clusters? to get the full story. The Windows model does not work very well in this space.

While Linux is dominant, other systems do make it into the list. . After Linux, comes mixed, then Unix. The #4 cluster is a Sun cluster created for The University of Texas at Austin.

But does it run linux?

But does it run linux?

But some encryption can be broken.

Especially if you have the the eleventh fastest computer in the world (and yes, I've heard that just that computer was bought for reasons like this new law)

Do you think this linux box was on the bargain rack at wal-mart?

beowulf cluster of hot grits, etc.

WRONG. At it's peak it hit #3 on the top500.org chart and at the time (Nov 03) was the cheapest of the top 10 by several orders of magnitude:
http://www.top500.org/site/history/2024

http://www.top500.org/node/13224

This was the cluster. At its peak it was #14 on the top 500 list.

There's no reason to believe that Apple systems (XServes, etc) couldn't be used for a supercomputer cluster, but since they now use the same Xeon processors as everyone else, there's no compelling reason to choose them over another vendor of similar hardware.

Extrapolating from the performance development chart which shows a 10 fold increase about every 4 years (desktop computers should be pretty similar), and assuming top desktop computers today hit around 100 gigaflops, then you can expect we'll hit that sometime around 2024.

I work with Argonne and am involved with the HPC world. Sadly this article doesn't include a link to the actual top500 that would clear this mess up.
http://www.top500.org/lists/2008/06

The #1 computer, the one over a petaflop, is RoadRunner at Los Alamos.

#2 is a bluegene machine from the DOE

#3 is Intrepid at argonne.

It's not clear to me how they could be so wrong in the article.

The top500 list clearly show that roadrunner is #1. What's this one then?

I was thinking the Intrepid was the "Fastest in the World", but actually it's the fastest for open science. The DoE owns the top three on the list. Why do they need so many? If you're protecting the nation's energy, why not set and example and use less of it?

Top500 has the actual list. Would have been nice to have this in TFA or TFS.

Well, the top 500 website even shows Roadrunner as the fastest here. Blue Gene/P looks to be #2.

If anyone wondered what FRA will be using its fairly new 13728-core, 102 Tflop/s (Rmax) Xeon cluster for, I guess this is it. When it was new on the previous list (November 2007), it held the fifth place. Here is an article about it in Computer Sweden (in Swedish). Maybe now is a good time to upgrade to 2048-bit keys...