Slashdot Mirror
Fastest-Ever Windows HPC Cluster
An anonymous reader links to an eWeek story which says that Microsoft's "fastest-yet homegrown supercomputer, running the U.S. company's new Windows HPC Server 2008, debuted in the top 25 of the world's top 500 fastest supercomputers, as tested and operated by the National Center for Supercomputing Applications. ... Most of the cores were made up of Intel Xeon quad-core chips. Storage for the system was about 6 terabytes," and asks "I wonder how the uptime compares? When machines scale to this size, they tend to quirk out in weird ways."
Enough power to run vista.
But does it run linux?
"Your cluster has just finished downloading an update, would you like to reboot now?"
Just disrupt the deflector shield with a tachyon burst.
does it blend?
/. comments out of the way)
(figured we needed to get all the regular
The Windows Server 2K8 code base must be better than previous versions of Windows. From what I understood, Windows didn't scale for clustering due to problems with file locking (IIRC, the overhead for tracking locks grew quickly enough that the performance was marginalized past about 4 nodes). Unless they're using an iSCSI SNS server that handles the locks over a clustered file system. Still, this is leaps and bounds beyond previous versions of Windows WRT clustering!
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
Bah! Beat me to it ;)
Shameless plug alert: Game server control panel
So does everyone else.
You are using English. Please learn the difference between loose and lose; they're, there, and their; your and you're.
And with the easily affordable CALs, up to 11 users will be able to use it at the same time! (well 8, 2 CALs will prolly be used by junior admins, and one for "test")
If you don't know what AltaVista is (was), get off my lawn.
Anyone know the fastest Windows HPC cluster *not* built by Microsoft purely as a marketing exercise to say 'look we can do HPC?'. And that actually gets used.
How does a Windows HPC cluster present itself? Do you submit batch jobs from a GUI?
Clustering in the sense I think you are discussing is the HA-clustering stuff. HPC clustering is a tad different.
XML is like violence. If it doesn't solve the problem, use more.
In other news, IBM debuts world's fastest punch card reader...
"Not an actor, but he plays one on TV."
How does a Windows HPC cluster present itself? Do you submit batch jobs from a GUI?
. . . maybe with a tossed chair . . . ?
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
Such a powerful cluster should get from power-up to BSOD instantly!
McCain/Palin '08. Now THAT's hope and change!
Actually... When America tested it's first fighter jet it had a dummy propeller.
Sure, just pop in a Rocks cd and hit the power switch. That will format the harddrives for you as well.
my favorite pet opensource project, and while I'm at it, "Imagine a Beowulf cluster of those ..."
So.... six terabytes... isn't that horribly small by today's standards ? I mean, our small backup server here is 2 teras, it's just a cheap PC with a bunch of SATA drives in it.
Does that mean my gaming rig and media server, when combined, constitute an "HPC Cluster" worthy of the top 100 ?
Ghey.
-Billco, Fnarg.com
I don't, but there's a lot of information at the home page. Including links to case studies for NASCAR, Daresbury, etc., etc.
Including FAQs. And, finally, the answer to the burning question: will it run Linux?
Supercomputing is the one area where Linux is the dominant operating system. Period. AIX still plays, but that's about it. Just check out top500.org if you don't believe me, the June list just came out. Though the IBM Blue Gene machines do run a proprietary microkernel on the individual computational nodes (the user-accessible nodes are Linux).
PC moderators can suck my White pierced, tattooed dick. If you think pride == hate, s/dick/Aryan meat mallet/g.
What about taking a super shit? God I gotta dump out.... I got my eye on YOU!
"It looks like you're breaking into the top 25 fastest supercomputers. Would you like me to fix that?"
stuff |
and I have a very hard time believing most of the claims of fact in this story.
"When we deployed Windows on our cluster, which has more than 1,000 nodes, we went from bare metal to running the Linpack benchmark programs in just four hours,"
Hmmm. And what installer was this? Is it available commercially? How much is the license for the version with this mythical four-hour installer?
"The performance of Windows HPC Server 2008 has yielded efficiencies that are among the highest we've seen for this class of machine," Pennington said.
What "class" would that be? I imagine it would explicitly exclude Free clusters.
One should question whether the efficacy of any institution/research project using their grant money wisely given the amount of money required to fulfill Microsoft's licensing requirements.
Furthermore, If research projects are actually considering wasting their grant dollars on Microsoft licenses, then the outlook for American R&D is grim.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
...that the only thing which counts is the 'Total Cost of Ownership'? Do I have to pay every installed node running Windows or every CPU? And how much do i have to pay for every registered copy of Windows and it's support service?
I mean, it can't accelerate with more than 9.82m/s, and the article doesn't say a word about the terminal velocity.
War is one of the most horrible things a human can be exposed to. And one of the worlds largest industries.
But the statistics for the top500.org show that over 9000 processors is way above normal for a supercomputer cluster up there. In fact less than 5% of machines in the entire 500 have more than 8000 processors, with the majority around the 1-4k mark. Oh, and 85% run Linux-only with an amazing 5 (not percent, actual projects) running Microsoft-only. So it looks like MS did this through hardware brute-force, not some amazing feat of programming. But then, that's true of them all. Although being in the top500 list is "good PR", it doesn't mean that much.
I wonder what the licensing is like for a 9000-processor Windows Server, though?
Should be enough for everyone.
Is this euphemism for "botnet"?
I did not know Microsoft was advertising on slashdot these days...
Can someone explain why anyone could possibly want Windows on a scientific computing cluster? What does Windows offer that Linux doesn't?
Much of my work involves running molecular dynamics simulations. By HPC standards these are tiny calculations (in my case, usually 32 CPUs at a time). All science HPC software I'm aware of is Unix-oriented, and everything runs on Linux. At my institution we have an OS X cluster and we are in the process of purchasing a Linux cluster. We didn't even consider Windows - given the difficulties we've experienced administering Windows on the desktop, a Windows cluster just seems like an expensive exercise in frustration.
Spammers must be just drueling for this to go into production environments. This improved speed will be great for viruses as well. MS is really starting to seperate themselves from the rest of the server quality OS's. Qudo's!!!
So what are the costs on this sytem? My guess is that it is double what a similar speed Linux system is. And yeah, it would be interesting to see what uptimes/cpu is.
A Windows MSCS cluster is essentially for fail-over/HA purposes. This is for high-performance purposes, and explictly excludes use as an application or database server. From the FAQs (although this is for 2003):
but does it run Vista? :P
I bought an external WD hard drive for $200 that was 1 TB. Yay, it's fast, but it isn't going to be doing much with so little storage.
Support a great indie game: http://www.abaddon360.com
but also dreaming that having to tinker with underlying code base will in the future not be needed anymore [1] and hence increase their share in the HPC market. Even if true - why pay loads of money if you can use Linux for free? [1] http://port25.technet.com/archive/2008/06/18/is-high-performance-computing-naturally-open-source-ie-for-tinkerers.aspx
First, the Top500 list has plenty of value. What most people do not realize (or should realize) is it is one data point on the HPC spectrum. If your HPC program does not perform the same or similar matrix operations as HPL then the ranking is meaningless to you. To some the list has become a public relations contest.
Second, performance is virtually independent of the OS (unless you are using TCP). Most big clusters use InfiniBand and run applications in "user space" by-passing the kernel. The rest of the code is crunching numbers.
Third, for the right cost, anyone can get a system on the Top500 list. It is a rather simple price/performance calculation, by the way. Breaking into the top 10 might be a little more difficult.
Finally, HPC and Linux are synergistic. Take a look at Why Linux on Clusters? to get the full story. The Windows model does not work very well in this space.
HPC for Primates. Read Cluster Monkey
Have more than 1 BSOD at one time.
... but does it freeze while formatting a floppy?
I saw an "article" about Windows-powered (dual-boot!) "supercomputer" being installed in Umeå, Sweden, just yesterday or so. In a traditional Microsoftese way, the article claimed the system to be fastest supercomputer in Europe - except in one place, where it was correctly referred as fastest Windows-based supercomputer in Europe. The linpack numbers were certainly a fraction of fastest European system on the top500 list.
I conclude that Microsoft has launched a fact-reshaping marketing campaign for their seriously underdog HPC platform, and their obvious primary target is not to get into high performance computing field, but give big bosses false ideas of platform scalability to make them to commit into big "future" Microsoft-only environments, primarily 2008 Server.
I never fail to wonder why people calling themselves journalists pass Microsoft marketing releases almost unchewed to their publications and lose all credibility in more critical circles (like serious journalists).
For a lot of the fairly typical stuff, I actually am prepared to admit the base OS overhead may not be that different. A lot of HPC clusters are not set up particularly fundamentally different from a typical linux server randomly set up. This is mainly because it's just easier to understand and set up this way.
However, the ones that do implement something highly efficient or sophisticated at the OS level would have a very very hard time achieving analogous results. The petaflop system, for example a) uses cell processors and b) is mostly ram-resident in terms of OS. Neither one of those is Windows friendly (Windows PE can be run from ramroot, but the base platform is about two orders of magnitude larger than what I would call a 'base' linux platform). And officially, Windows PE is specifically designed to frustrate as a platform (timebombed reboots), because MS fears it being too widely used and cutting into their more traditionally licensed/extorted base.
On top of that, the general business scenario is mind-numbing. HPC is a market dominated by linux, for which there is a nicely competitive market of vendors. Red Hat, Novell, Canonical, and others all can offer a platform supporting linux apps and all have power to help. A cluster going for a monopolistic company providing a platform that isn't open to replacing the vendor, and a platform not particularly interesting in and of itself from a technical standpoint... Well, I just don't get it.
XML is like violence. If it doesn't solve the problem, use more.
If linux was faster on this cluster they would be listing it on the top 500 with linux not windows HPC. Also one of the most important things to look at is how efficient the cluster is, this one had 77.7% application efficiency on 9,472 cores which is very impressive. Windows HPC deployed and was testing linpack on over 1000 machines in less then four hours, I would like to see rocks do that.
While I don't agree that Microsoft Windows HPC Server is the best software to manage a supercomputer, the linux diehards out there should pay attention to a problem that Microsoft is trying to tackle: accessible supercomputing. See one of their case studies as an example.
The bottom line is, these days pretty much anyone has access to a few TFlops of compute power, but the learning curve for getting something running on these machines is pretty intimidating, especially for non-CS based disciplines. I've had to take a 1-2 day class, plus futz around with the clunky command-line tools for a few days or so, on every supercomputer I've used, just to get simple jobs running. In my experience, people learn to game the various batching and queuing systems such that their jobs run faster than everyone else's, further shutting out the newcomers.
HPC vendors would be wise to focus more attention on the tools and interfaces so that Joe-researcher can set the number of nodes and go, rather than having to manually edit loadleveler text files, sending them to the queue, and then coming back next day to find the job failed due to a typo in the startup script.
On multi-TFLOP systems, not everyone needs 99.5% efficiency with all the implementation details that requires. These days, many people just want their job to run reasonably quickly, with no fuss.
The same thing happened several years ago with the move to high level languages like Python and Ruby. Sure, they're slower than C++ and FORTRAN. But for the vast majority of applications, you wouldn't know the difference on modern processors. And the turn around time and user-friendliness on these languages is so much better, using them is a no-brainer.
Hopefully Microsoft can spur the industry in this direction.
Ignore any merits and start the random ad hominems...
I'd like to be enlightened as to the benefits of running a Microsoft cluster in a scientific environment. Really. You've given me nothing new to compare my experiences running Microsoft-based clusters versus my experience running Linux clusters.
I suppose if you REALLY ran windows clusters for a living you'd know which tools. /etc. But it's a whole lot less effort than getting an MCSE.
That would be Cluster Administrator. Which is higher-level than the "mind-boggling" array of files in
Please, provide some facts. Otherwise your response is, well, zealous and overwrought.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
now see how fast the identical hardware runs with Linux on it... bet it goes way faster...
Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
There's an obvious application to run on a Windows cluster.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
From your case study:
"""
In addition, it is investigating ways to allow users to connect remotely to the cluster. It expects to complete the project and move the cluster into production by March 2009.
"""
By time the cluster in the case study allows users to remotely log in, the hardware will have lost at least 1/2 of its value.
While more work is needed to make things user friendly, you have to remember that the funding is there for CPUs; not many folks are forward looking enough to realize researchers really need funding into making stuff easier.
Whereas Server 2008 and Vista share a tad more of their code base.
and *that* is relevant.
And could be humorously be alluded to because of the mis-detection of some software.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
"Microsoft is trying to tackle: accessible supercomputing"
..
.. are accessible to people interested in pursuing science, simulation or modeling"
.. :)
Assuming MS was responding to this imagioned problem
"The contest showed that supercomputers
"but the learning curve for getting something running on these machines is pretty intimidating, especially for non-CS based disciplines. I've had to take a 1-2 day class, plus futz around"
You actually programed a supercomouter - cool. What type and where exactly? How does HPC Server differ in respect to other solutions?
"the Blue Gene family of supercomputers has been designed to deliver ultrascale performance within a standard programming environment"
"Hopefully Microsoft can spur the industry in this direction"
You mean like continually inventing Apple, badly
davecb5620@gmail.com
Thanks,
Now I don't have to post it
8-)
How did Novell and IBM manage it on Blue Gene ..
"Linux has dominated the marketplace for high-performance computing,"
Mark Seager, Lawrence Livermore National Laboratory, Calif
davecb5620@gmail.com
http://www.top500.org/system/8757
Look at the description. Does it run RH? If it exports a Lustre filesystem, I think Lustre only runs on *nix.
Does anyone know the real implementation details behind this system? Is it part Linux, part Windows? Was it linux and now Windows? Did they port Lustre to Windows?
To tie this with another post. Somebody said that the OS is bypassed by application anyway, so what role does Linux or any other OS have ona a cluster like this?
If the only reason to have an OS is to load and run an application, and to do some other unavoidable file maintenance. I think there could be competition from Windows.
What would the Microsoft per-core license fees run for this 9,472 core server? What would they run for the same system running Linux? Methinks Microsoft cut them a break on license fees for the Beta software...
that this computer doesn't *come* in the top 25.
So you can't really use them as your "benchmark".
Are you the type of person that puts a research cluster on the public internet? If so please stay clear of my machines...
this is a step backwards in terms of the development of safe and robust super computing. I'm curious, dose the admin have to have a license for every machine in the cluster?
However, MPI itself has serious issues. The master copy of the program starts/stops slave programs vis SSH, although some implementations also support inetd-style starts. Messages sent to multiple machines are sent sequentially, rather than via a reliable multicast, wasting bandwidth and wasting CPU cycles.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
The top500 score is a very particular benchmark. Whether it is the best measure of things is a matter of debate, but comparing numbers isn't a good idea. Essentially, if it can't/doesn't submit a top500, it isn't directly comparable to any of those scores in a meaningful fashion.
For example, with the PS3 clients figuring so prominently, my suspicion is that these are 32-bit floating point operations, and top500 only counts 64-bit floating point operations. I couldn't clarify it readily, so I might be wrong, but it seems that way at first glance.
XML is like violence. If it doesn't solve the problem, use more.
I read the link, here's a particular quote that caught my eye:
"The consortium is committed to making supercomputing resources more widely available. However, its high-performance computing (HPC) systems were Linux-based, and CINECA realized that an increasing number of researchers, especially those in private industry sectors, were unfamiliar with Linux-based tools and interfaces. Acquiring the necessary expertise to use the consortium's resources was too time-consuming and costly for many enterprises."
Smells like fud to me. WTF?! *If* you're going to write a program that needs an HPC system to run *then* you're going to have to learn how to program for an HPC? Hello?! It's call mpi, upc, shmem, hell, even pvm. And they've been around for what, 19 years? What's it got to do with linux? Nothing, *except* that Linux has *excellent* support for that clustering software.
WTF do you need windows for? A cluster version of notepad? Duke Nukem Forever? And lastly, what's so *time consuming* and *costly* about doing the following:
1) Open a browser. any damn browser.
2) Type in the url: www.google.com
3) Type in the search box:
mpi upc shmem pvm site:wikipedia.org
4) Follow the freakin' links and *READ* the info.
Cost: $0
Getting a freakin' clue about the current standard of cluster programming: priceless.
What of kind of researchers from private industry do they have that they don't know how to use google or wikipedia? Gartner experts? Microsoft employees?
is this some new category of competition, like "fastest-ever cluster with blue cabling"?
the software on a cluster should, ideally, get out of the way of the jobs as completely and quickly as possible. it's hard to imagine windows doing that, let alone offering some advantage over the incumbent OS (Linux). in spite of smelling a bit like jato-equipped pigs, "because we can" doesn't really seem like a good rationale here.
...Beowulf cluster of these?
They had an Rmax of 68.48 out of an Rpeak of 89.59. That is a Linpack efficiency of 76.4%. With an inifiniband network, that's hardly interesting. It's hard to find a good comparison nearby (must be IB and Intel Core2). The nearest ones I see are #31 (81.5%), #11 (69.9%), and #10 (87%). 76% is about right smack in the midst of them. If comparing it to ethernet based installs, then yes, it is better (ethernet will easily get you down to the 50s), but that isn't particularly a constructive comparison. I guess he did say 'among' the best, meaning it doesn't suck as much as he would have thought, but for a more expensive product, it doesn't get you anywhere.
If speaking about utilization (odd choice of words if so), he isn't comparing linux to windows, but whatever job scheduler (maui, moab, pbs, etc) to MS solution. If that is the case and he is being genuine, I suspect he picked a bad scheduler for that.
XML is like violence. If it doesn't solve the problem, use more.
The top500 has some visibility, Linux dominates it, no Windows *at all*, they probably spent some of their own money to get at least one spot.
Which Wiki article? The problem is *now* there are two varaints of 3.2 ghz cell processors. One used in the Petaflop system, and one in the PS3. The one in the petaflop run I heard *explicitly* sought to make the double-precision number repsectable.
And just because both folding and linpack are "scientific" computing, doesn't mean they have the same computational demands in terms of precision *nor* floating point efficiency. Even if they are the same, which I can't determine still, there can be a wide variation in the processor instruction stream and how a processor copes. It's because of this very complexity of HPC computing that a lot of people justifiably call the top500 benchmark a tad artificial. hpcc tries to cover a greater range, but doesn't have the same marketing clout that xhpl does.
Folding@home is a great endeavor, but it isn't so easy or simple to guess how it stacks up against this list.
XML is like violence. If it doesn't solve the problem, use more.
It looks as if this cluster was first mentioned in the Top500 in 06/2007. At the time it was ranked #8. Since then it's slid down to #14, and now it's down to #23.
http://www.top500.org/system/ranking/8757
When new, it was running RHEL and the Lustre file system.
http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/Intel64Cluster/TechSummary/
Now in its Windows guise it's unclear what type of shared storage / cluster file system that it's using. Maybe this? :)
"Using CIFS as a Global File System for HPC Clusters"
http://download.microsoft.com/download/a/9/f/a9fd6c73-3745-494c-be4e-651505fe608a/CIFS%20HPC%20Storage.doc
Actually, yes... It looks like they've run RHeL 4 on it.
http://www.top500.org/system/8757
Clippy :
"Portable Batch System - can I help you ?"
User :
"Maybe".
Clippy :
"Enter Windows Genuine Advantage process and enter the License number. Its below your laptop."
User :
"Wot laptop ?"
Clippy :
"I mean your Portable Batch System."
User :
"You do not understand ...".
Clippy :
"Please call the manufacturer of the laptop for further help."
Did this answer your question ?
1. Yes.
2. Hell, yes.
Some kids in Estonia and Nigeria just got very very happy.
How fast do you want your Nigerian oil money today ?
This is just the hardware normally known as abe.ncsa.uiuc.edu. They tried out Windows on it to get the benchmark, and whatever other experiments desired by whoever commissioned the study, then booted it back to linux so it could actually be useful.
...I'd love to have a beowulf cluster of those.
You are irrelevant, and you have failed. Pack it up and go back to hunting possums in the bayou.
What I'm interested in, is seeing how this compares to a Linux based cluster using the same hardware...
Last time i saw a windows cluster in the top500 list, it was 150 places behind a redhat cluster using the same hardware but with less nodes (around 5% less if i remember).
http://spamdecoy.net - free throwaway anonymous email - avoid spam!