Slashdot Mirror
Windows Compute Cluster Server 2003 Released
grammar fascist writes "According to an Information Week article, on Friday Microsoft released Windows Compute Cluster Server 2003." From the article: "The software is Microsoft's first to run parallel HPC applications aimed at users working on complex computations... 'High-performance computing technology holds great potential for expanding opportunities... but until now it has been too expensive and too difficult for many people to use effectively,' said Bob Muglia, senior vice president of [Microsoft's] Server and Tools Business unit, in a statement."
"but until now it has been too expensive and too difficult for many people to use effectively,' "
So the MS solution is cheaper then linux and easier to use the Mac clusters? I don't think so.
Ms is the "me too!" guy from the usenet. Everytime anybody else comes up with something Ms comes in afterwards and says "Me too!".
evil is as evil does
It takes some serious marketing balls (and/or or a lack of marketing brains) to release a product branded "2003" when we're already halfway through 2006.
:)
I actually have to applaud the naming move; it accurately lets everybody know that this product is based on Windows Server 2003. It would have been quite misleading if they'd passed it off as " Windows Compute Cluster Server 2006".
Wonder what the meetings between the marketing team and the engineering team were like for this one.
OtakuBooty.com: Smart, funny, sexy nerds.
"...but until now it has been too expensive and too difficult for many people to use effectively."
I know many people take exception to that remark. But not everyone knows how to build Beowulf clusters.Some of us thought it was insane when, in the 90s, Microsoft said they were going to enter the server market. Yet here they are. And who in their right minds would run their web services out of IIS? (Then again, Apache now runs on Windows.)
The point is, just because the idea is absurd doesn't mean it won't happen. If corporate consolidations put support for technical computing under the IT department, and support for Linux is considered toodifficult for the IT folks, it's only a matter of time before the decree to port technical computing applications to Windows.
The fact is, M$ has access to software vendors, hardware vendors, and large customers in ways that Linux companies do not. They can create markets where they shouldn't be justified (unless you think all operating systems really require anti-virus software).
I'd love to be wrong about this. But I've finally come to the conclusion that sound technical judgement does not stop absurdity from happening.
Another, perhaps even more popular Linux cluster distro is Rocks Clusters.
While I don't have personal experience with OSCAR, Rocks is really good. These days, doing a cluster with a "normal" distro is insane. I think MS will have to think long and hard before they come up with something equally easy to install and manage as Rocks.
That being said, I think MS is not targeting Win CCS at academic supercomputing, which has a long history of using Unix/Linux, but rather they want to expand HPC to business customers who otherwise have a 100 % MS environment.
The idea behind Windows clusters being 'cheaper' has nothing to do with the individual price of the OS (versus, for instance, free Linux); the named price is low, not free, but that is not the point of your savings with a Windows HPC cluster. The point is that most programmers work on a Windows platform and have experience with it. And if you program with/for Windows and, for instance VS 2005, MS counts on the effort of building programs that run on HPC to be considerably less effort than it is on a Linux (or Xgrid) cluster. Making existing Windows 'hits' clusterable (i heard mention somewhere of image, movie and 3d processing software) is easier because of this too; making it work on other clusters is a pain because there you would have to work in an environment you are not used too. Like all things with MS; they count on the familiarity and ease of use to make this all run. That is what makes it cheaper; you cannot get a Linux HPC programmer and if you find him/her he will be godawful expensive; for WinHPC it will just be 'another VS programmer' of which there are a lot. Look for MS to add testing, debugging and development aids for HPC in the upcoming versions of VS.
As I understand it this sort of thing can be done on just about any kind of computer. And at every university I've ever been to there's usually stacks of old pcs laying around.
As opposed to running email and word, HPC is one of these things where CPU power actually matters. Those 500 MHz PC:s aren't worth the hassle to set up and maintain. Not to mention that heterogeneous hardware (which a random bunch of discarded PC:s probably is) is a nightmare to maintain and program efficiently in parallel.
Most clusters consist of quality rack servers from a reputable vendor. TCO matters, not the cost of the hw alone.
"But I've finally come to the conclusion that sound technical judgement does not stop absurdity from happening"
Something that the majority of Slashdot readers seem not to understand (and with justification) is that purchasing decisions are not rational.
A basic training course on sales techniques will, unless it's totally bogus, emphasise the fact that purchasing is based on emotion, not rationality. Some 80-90% of all sales are emotion-driven and then sometimes post-facto justified by selectively picking facts.
As the world becomes a more complex place and huge amounts of information become available to prospective purchasers there's a kind of paradox emerging that will horrify economists who cling to the theory that perfect markets are based on rational purchasers with perfect information, because the reverse is happening.
Most purchasers are not analytic personalities. People who hang around Slashdot underestimate how much they have (in general) honed their own analytic skills with years of practice while most middle-tier managers in corporates never did. For those non-analytic people, being asked to rationally evaluate a mass of facts and statistics is a SCARY proposition. That's not how they got their job, they did that by looking good in a suit and licking backsides more or less assiduously whilst being ok at judging how the politics are shaping up. Their skillset is way different from yours and they react differently.
The more information you make available to those people, the less they are likely to use and the more they will look around for 'safe' decisions. This will be especially true if their promotion prospects may depend on the outcome. THEY ARE NOT SPENDING THEIR OWN MONEY, it's the company's. Their decision will be based on the likelihood of retaining their job or getting promoted before their mistakes are discovered.
So, figure for yourself. On the one hand some technical guy they distrust because he's smart can 'download an ISO from the interweb and build a cluster myself' or 'buy from Microsoft'.
The first bit of irrational figuring will be 'the Microsoft stuff costs tens of thousands but the geek says it's free - that does not compute, he must be wrong'. The second will be 'if it goes wrong who will get the blame'. Guess the outcome of that one for yourself.
The result is fairly predictable IF you understand the parameters. Microsoft's marketing does understand where it's operating and will be well aware that its customer base is heavily loaded with irrational people. Most likely they are hearing squeals from that customer base asking where Microsoft's compute cluster solution is because 'we want to buy one'. It would be foolish not to give them one surely?
Sure, it can be.
In addition to the issues with interconnects, raw performance of individual nodes, and heterogeneous clusters...
Reliability becomes a big deal with such old computers. Sure, a well designed cluster will be able to route around a significant number of failed nodes, but computing efficiency will plummet and won't be terribly predictable(often, predictability becomes more important than raw burst performance). You might have 20 nodes working today, 12 tommorow, back up to 20 for a few weeks, then lose 6, 2 of which are completely dead leaving you with 18 after the repair... You see the problem here?
That said, such things can make for interesting projects, and might make decent production systems in some contexts. They are not, however, a panacea to a universities HPC needs.
Speaking of learnign experiment HPC systems, does anyone know if any of the virtual machine solutions available for PCs can be used to create a software simulation of an HPC cluster? I realize such a thing would be near useless for real work, but might make for an interesting learning exercise and useful testbed before you deploy to the live cluster.
Thing is; 'most' programmers wouldn't know what HPC, let online how to program for it. MS wants companies to believe that this kind of programmer (or admin) can set-up, program against and run HPC clusters with little training other than Win2003 & VS.NET 2003/2005. If this is true or not is irrelevant; as long as company CTO's/CIO's/CEO's believe it, no one cares about the real technical merit of the statement. ... Linux is 'better' than Windows in most cases, but still those markets grow for MS...).
And they do, because they are usually managers who use Word and PPT sometimes and play golf with one of Ballmers' boys; dunno if this is true worldwide, but it is here in the Netherlands. And yes I know this from experience, not from reading it in a blog.
The MS marketing story to managers that are not indept technical is a very strong one; choose Windows, you'll pay, for instance 20 times $500 and use your current Win2003 admins for installing/running and your current devvers or *any* Indian $3/hour company to write your software, versus; pay nothing for the licenses, hire new, hard to get Linux admins for $3000/month, hire even more hard to get HPC / C programmer (also hard to come by when outsourcing...) for the Linux variety. Bottom line; pay, as company, Windows; 10k + 20k computers one time extra for your environment, Linux: pay 20k computers + 40k/year + x * 100k for development.
True or not, this is a strong salespitch which does work on the golfcourse and a lot of companies needing this will go for it, as they have done for other MS technologies which make no sense (embedded? webservers? datacenters? databases? storage software?
University researchers may have limited funding but a lot of researchers at large corps (oil/med/etc) don't have much trouble getting funds for their research, bear in mind also that not everything is research, for instance engineers may simply want to run some large numerical models etc. I have personally seen parallel processing on windows clusters implemented at a large corp, plenty of funding there.
Microsoft haven't even noticed clusters until now, so it will be a few years before anything of note is written to run on clusters on that platform - plus the whole poorly documented moving target operating systems which makes it pointless to port some software that has developed over decades and is run on clusters today.
Your comment should be modded funny, not insightful.
.net are *not* fast enough. I've tried).
The only reason you need a HPC cluster is -- indeed -- High Performance Computing. That means you're going to use as many cycles (or messages passed) as you can get from that $50K+ cluster you've just bought. This easily precludes anything but a fast compiled language like Fortran or C (and no, Java with JIT or
The comment about domain specific languages in your blog for HPC purposes is true: Fortran is exactly that.
I made this comment on Ars Technica, and I'll just repeat it here as well.
I just can't see any future in this. There are few things working against MS here:
a) Price. In the large-scale, the price they are asking would mean lesser nodes. Instead of paying for Windows, the customers could just use Linux and add extra nodes to the cluster.
b) source. Yes it does matter. In markets like this, the people running the cluster do fiddle with things in order to make it go faster. They can't do that with Windows.
c) Ease of use. Well, the people who make clusters are usually not morons, so I don't really see any real need for "point 'n click" GUI for creating clusters. And maybe that GUI could impose a bit more overhead to the system? And creating Linux-clusters is relatively easy.
d) Momentum. Linux has companies like SGI, Cray, IBM and others using and improving it. And there are universities involved as well. Those companies really know Linux and they REALLY know HPC. Microsoft has no real know-how regarding HPC.
e) Familiarity. This time, people know Linux. MS is trying to beat an entrenched competitor. MS has succeeded in doing this before, but they did it by undercutting the competition. This time they are competing against something that is free. And their competitor has the advantages mentioned in A, B, C and D, all of which matter to the target-audience.
Lesbian Nazi Hookers Abducted by UFOs and Forced Into Weight Loss Programs - -all next week on Town Talk.
...emphasise the fact that purchasing is based on emotion, not rationality. Some 80-90% of all sales are emotion-driven and then sometimes post-facto justified by selectively picking facts.
Maybe with things like cars and clothes, but clusters are merely machines to crunch numbers. Kinda like a big calculator, and little emotion goes into designing and using them. Its bang/buck. Thats it.
Microsoft _may_ be able to sell this HPC edition to some PHB out of emotion who is completely clueless and has clueless admins as well, but an OS has little to do with an HPC system. In fact, the less of the OS the better. Most of the time, HPC apps are in user land. The OS does basic memory management and I/O, but that is it.
Most all clusters are Linux. Why? Its good and cheap. You don't need the scalability and robustness of say Solaris, because you (typically, almost 100% of the time) only have one thread per processor. Yes, I know with large SMP machines, the OS can and does matter, but those rarely have the bang/buck ratio of clusters. The two big guys that have done this over the years (large SMP/NUMA/NUMAcc, etc) are SGI and Cray. And both of those companies are hard for cash right now. IBM probably does not make money, or much money off of their large number crunching systems, but they are probably viewed as RND, not a "for profit" good or service (I could be entirely wrong here regarding IBM, but thats my hunch).
I don't know what Microsoft is doing with this product. Like someone else said, its probably just a "me too!" thing. In looking at their "details", they do not mention using desktop machines at night. The is a BIG miss by them, because that would be one of the only things that could even make this a marketable item for an already primarily MS outfit.
The more I think about this, the more silly this sounds. Yeah, I'm an anti-MS guy, but I try to give them the benefit of the doubt, but this product seems completely worthless. Actually, now that I learned that this is an only 64bit offering, I believe this is a way for MS to sell a product for beta/stress testing of their 64bit server offerings.
To close this post, from the FAQ:
Q. How does a Windows-based compute cluster compare with a cluster running UNIX or Linux?
A. There is little substantive difference, but UNIX-based solutions should be fully ported to Windows to realize the full benefits of the Windows operating system. There are several differences between UNIX-based operating systems and Windows. For example, I/O operations and threading are different on UNIX-based systems than they are on Windows. I/O intensive applications will benefit from using Windows native I/O APIs rather than UNIX style I/O APIs.
Emphasis mine. The second bolded part is important. That porting is expensive and time consuming. Especially when its common for codes to be 30+ years old and designed for UNIX systems. Sounds like vendor lockin to me. Wow, typical Microsoft.
There are exceptions: some folks do wind up digging up racks of old servers, at rock bottom prices or even for free, as their data centers or deployed installations decommission them. You can inherit quite a lot of slightly outdated hardware this way: if you can justify the electrical expense of running them, they're quite convenient for massive, lengthy computing jobs.
A lot of cluster managers also mistake "really expensive, physically robust servers for "will always be working". The complexities of such setups and the general frequency of failure of "high availability" software itself means that the much vaunted 99.99% uptime of such systems is usually based on serious cooking of the numbers, not any metric actually used in the field. After the crops of failures of things like the old IBM deskstar drives, the run of bad tantalum capacitors in Dell motherboards, and other failures that strike entire classes of brand new hardware, it's often better to use older, cheaper, burned in hardware that's had the BIOS updates and the kinks worked out, and save the extra money for the next round of upgrades in six months or a year.
now that super computing has been turned into clustering and there are lots of people doing it (like it hit >$x billion,) it has apppeared on microsoft's radar.
:-)
Unfortunately for Microsoft, the terrain's already covered by Linux and those systems are a moving target with cost-benefit lines that Microsoft CAN'T possibly over take. (The software is $-free and open source and the users WANT collaboration.)
Its a technological death trap for Microsoft. (I can just hear the SNAP.
MSBPodcast.com The opinions expressed here are my own. If you don't like 'em... Think up your own stuff.
should be fully ported to Windows to realize the full benefits of the Windows operating system
I got the impression from TFA that the major feature HPC Edition had to offer was its handy and clean interface with Visual Studio, allowing all the CLI masses to write software for a compute cluster all their own (without having to deal with learning a different platform).
Would this have any effect on the HPC market? I can't see people with existing installations+software biting, but I can see it tempting to businesses building new installations and wanting to minimize training costs, etc, even if what they're buying won't really give you a full "H" in HPC. If a business IT department has passed the dictum: "This is a Windows only environment," and a manager in that business needs a compute cluster, he can either pick a fight with his IT department or bend over and take the Windows HPC software (which MS will give them for peanuts this first time around).
Unrelated, Near future prediction:
Don't blame me, I voted for Baltar.
There were papers on what the product was. I got a little USB memory key, it was only like 32 megs so not super useful, but free, that had some documents on it. I'll assume that they were smart enough to print those documents out to hand out after the USB keys ran out.
Right next to the booth with the 'cookie' were stations with somewhere between 5 and 8 different companies showing off their software sitting on top of the CCS. There were fluid dynamics, car crash simulation, and a couple others. Some of the demos were kinda lame so I didn't pay much attention to them, and a couple of the monitors were showing the same software doing different things, so I'm not sure the exact count.
There were developers and feature PMs (the guys who write the specs that the devs implement) standing around. I bet that they had a passing knowledge of how their system worked.
I didn't stick around to watch the MS presentation so I can't comment on that. You are right that there were a bunch of other booths that were absolute disasters though.