Slashdot Mirror
How Well Does Windows Cluster?
cascadefx asks: "I work for a mid-sized mid-western university. One of our departments has started up a small Beowulf cluster research project that he hopes to grow over time. At the moment, the thing is incredibly weak... but it is running on old hardware and is basically used for dog and pony shows to get more funding and hopefully donations of higher-end systems. It runs Linux and works, it is just not anything to write home about. Here's the problem: my understanding is that an MS rep asked what it would take to get them to switch to a Microsoft cluster. Is this possible? Are there MS clusters that do what Beowulf clusters are capable of? I thought MS clusters were for load balancing, not computation... which is the hoped-for goal of this project. Can the Slashdot crowd offer some advice? If there are MS clusters, comparisons of the capabilities would be welcome." One has to only go as far as Microsoft's site to see its current attempt at clustering, but what is the real story. Have any of you had a chance to pit a Linux Beowulf cluster against one from Microsoft? How did they compare?
well, it cluster fucks pretty well...
har har
FP - woohoo!! BTW - I have played with clustering on windows and linux. The windows was purely load balancing. Beowulf for computation of linguistics was FAR better.
What do you call a cluster of Windows machines
when they Blue Screen?????
A Cluster Bomb!!!!
Service guarantees Citizenship! Questions Guarantee GITMO.... Amerika Uber Alles!
a blue screen domino effect... starts with one, and next thing you know, its a sea of blue screens. The problem with MS clusters is that stability really becomes a question... a crash spreads like an infection...
Fire in the hands of the village idiot is no tool, but a weapon of mass destruction
Licensing would seem to be the first thing that comes to mind.
Software costs for 100 linux machines are close to nill.
Software costs for 100 Windows machines probably won't be.
Granted I have read the licensing on the MS Clustering link but if it like anything else you'll need either a license of some kind on every machine.
--"Karma is justice without the satisfaction"
Anyhow, imagine how much you're paying in software licensing for a large cluster? For a univeristy project, this just doesn't seem to make sense.
One has to only go as far as Microsoft's site to see
Ah, so this is a typical Ask Slashdot then?
--saint
To start, it would take a few hundred dollars per box to put W2000 on it, since they presumably don't want you to just copy their evaluation version.
So unless they're willing to give you their OS for free, why would you even consider it? Suddenly your supercomputer cluster would cost like a real supercomputer... then you could have just bought a real supercomputer!
From what I understand from reading Win 2k Advanced Server's help section on Windows clustering, it is mostly for stability. Kind of like a massive mirror raid system. I really don't see any performance advantage if you're looking for supercomputer speeds, unless your measure performance by uptime. As a side note, what were you using for clustering? I'm currently doing a cluster using mosix for my school and it seems to be going nice. I'm just curious as to what gives the best speed performance on the linux end.
can't sleep slashdot will eat me
Why pay money when you can set up a cluster for the cost of the hardware? I assume as you increase the size of a Windows based cluster you'd get more power(!) but also more costs.
Would'nt the license fees become prohibitive?
Well, my company clusters exchange on Win2K Advanced Server.
We run in what's called an active/active cluster. But for the most part, the machines are just sharing responsibility. In most windows clusters the other server is just sitting there waiting for the first to fail. They share two drives (on seperate hardware either fibre channel or SCSI attached) and when it fails, the other server picks up those drives. Windows writes data to those drives so when the other server picks it up it can import that data and pick up where the other server left off.
Clustering is PERFECT for fault tolerance. It is useless for most intents and purposes for load balancing. If you want load balancing you can use NLBS, but it just plain sucks, and never works right.
Mike @ The Geek Pub. Let's Make Stuff!
With their vaunted stability, and marketing
savvy, their new Cluster Product will be called:
The Cluster Bomb!
Service guarantees Citizenship! Questions Guarantee GITMO.... Amerika Uber Alles!
to get them to switch to a Microsoft cluster?" How about not shafting everybody who's ever used your OS? ;)
"from the please...no-more-beowulf-jokes dept."
too late
Sent from your iPad.
If you're running on old hardware, M$ isn't the way to go.
;)
I still have a useful BSD and a Linux server running on a 486. It doesn't do much, but it works for my purposes. I can't imagine the pain that I would experience trying to run something like XP on it.
Considering how much you can tweak on an open-source OS, I wouldn't begin to consider MS. I mean, Office XP is slow, but how much more do you need to get out of it?
// Agent Green (Ian / IU7 / KB1JQO)
// IEEE 802.3: All 10base Are Belong To Us
MCS (Microsoft Cluster Services) are designed for load balancing and fault tolerance, as where Beowulf Clusters (AFAIK) are more for distrubuted processing load for performance increases (massive threading). MCS works quite well, especially well on Fibre Channel and Brand Name Hardware such as Dells and Compaqs.
Simply put, it works well (but the cost is often an issue due to the cost of hardware in an enterprise) but it is not the same clustering you see with the Unices. E-mail me at my account if you have more specific questions.
My intent is not to start or participate in a flame war, but the term clustering simply implies different things on different OS'.
Easy solution, run wine to please the MS observers.
The date on the M$ "CCTP" website is over a year old... but this is the first I've heard of it. Why hasn't M$ been marketing this? I mean, the fact that it doesn't even work (if that is the case) has never stopped them in the past, eh? ;-)
Has anyone tried this thing?
--jrd
XML is like violence. If it doesn't solve your problem, you're not using enough of it. --AC
It would take a lot of money for licensing and for RAM.
And the cluster would perform like SHIT.
I'd tell any MS rep to fuck off if he came to me with such a god damn stupid idea.
A beowulf cluster is not limited to Linux, it could run on top of any OS. I believe NASA did the original design work to be OS agnostic.
http://www.windowsclusters.org/projects.htm gives a list of current Windows clusters.
Finally, are you out of your tiny little mind? I wonder why M$ is so keen to help. There is no such thing as a free lunch, espically from M$.
Q: Can Microsoft Windows do *X*?
A: Who gives a shit?
How about free licensing in perpetuity, ability to run on older hardware (which further reduces cost) and access to the codebase so that changes can be made (if necessary or desired) to better support your clustering needs.
If that Microsoft Rep can even touch those, then tell him you are open to the idea of beginning negotiations.
C0deM0nkey
Ok, you have a solution in place. It works. Some sales guy wants you to change your solution that works.
Make him convince you that the time and cost of the switch is going to gain you something.
Does your current setup not do what you need it to do?
...he talks of being in a educational institution and you guys think of licensing costs? most have campus licenses (for a low fee) allowing them to use as much windows as they want without paying an additional fee. Seems you trolls dont have even a clue why you dont beat the hell out of Microsoft. Simple: They are not that stupid. Just because you morons have to pay the full price does not mean that anyone who is worth something for MS has to pay - I get my MS Softwarte for free (for a long time by now), and heh as with a VERY high propability access to a similar deal to. My advice: GET IT GOING ON MS. Why? Obviously MS wants you to get it going, so ask your rep for all support you can get. That will be more and better (a.k.a. professional) than anything you can get for LinSux.
That would be....what, a Grendel cluster?
Hello,
We run a MS cluster here. VERY big app... so big, I am loathe to name figures, because that would identify to MS just who is talking here...
But, we use MS clustering for our web app. Our setup is that we have a database server with 4 procs, and a growing array of web servers with 1 proc each, all of which use disk space on a SAN. W2K clustering manages the load balancing as well as allocating disk space out of the SAN to virtual partitions as needed. The original poster is correct; MS clustering is for load balancing, not computation. I have seen many times Microsoft sales reps don't have a clue of what they're trying to sell; they're just told from on high to replace Linux with Microsoft wherever they can. I think this is clearly a case of that.
My advice? Ask the sales rep to demonstrate how MS clustering will solve a common comp-sci problem with more MIPS than each box alone has. Point out that you're not running a web server or any such service on these boxes, but that they're for raw computation. Even better, see if he'll let you talk to a technician on how W2K clustering can meet your 'unique' (at least to MS) needs.
Now, for everyone else... Don't get me wrong. W2K clustering is a great technology for building highly performant, highly reliable, highly scalable applications quickly and easily. But it scales in the direction of millions of users, not millions of computations.
I am disrespectful to dirt! Can you see that I am serious?!
You should check out the Cornell Theory Center for info on clustering with MS 2000. They sold there souls to the devil for free machines...signing a contract forbidding them to use UNIX type OSs. I have installed and fiddled with a cluster of MS 2000 machines, and it is a complete pain. We ended up dual-booting with Redhat 7.1 and it hasn't been in windows for 8 months now!
Clustering for windows requires Windows 2000 Advanced Server, and a great deal of patching. And with old hardware you are out of luck trying to run Windows 2000 Advanced Server.
Distributed computing for Windows has been around for a while though, Seti@home has been doing it for years.
Looks like Microsoft is busy being slashdotted.
Chances are the MS rep didn't understand MS clustering. He just knew that you had a Beowulf cluster and he wanted to sell you MS software so he figured he'd sell you a MS cluster, regardless of whether or not it would do what a Beowulf cluster could do.
However there is a server solution I saw demoed at a MS DPS I attended called Application Center. It allows you to manage your cluster and distributes workloads throughout the cluster.
Now, I'm not sure if you NEED this to take advantage of Windows 2000 clustering. The last time I worked with a MS cluster was under NT 4 and it was failover only. The load balancing was "faked" by a router that would just alternate which server the request was sent to.
(insert "yeah but MS is evil" comment here)
(insert "yeah but Linux Beowulf clusters cost less" comment here)
(insert "yeah but who wants to have to reboot your cluster all the time" comment here)
(insert "I wish the sigs were longer because that's a really good quote by Richard Feynman" comment here)
"For a successful technology, honesty must take precedence over public relations for nature cannot be fooled." -Feynman
My managers will only buy windows products as they have a site liscense with MS. They are looking into Linux a little bit because of the Terminal server w/ load balancing does not load balance and the clusterd computers do not talk to each other. The profiles on the 3 clusterd servers do not update each other at all. This was much better than the last attempt my boss did using an IBM pre configured configured box the whole cluster got a BSOD and corrupted a drive losing data for 3 days. People were not happy.
I can only hope MS's poor performance will make them switch.
Vote early. Vote often. Vote CowboyNeal.
an MS rep asked what it would take to get them to switch to a Microsoft cluster.
first the rep needs to prove that $199.00 per node for software fees has to provide major benifits over the Linux cluster. How many windows clusters can he list for you to call and ask about it? refrences, ones you can call and talk to the guys running/maintaining it. Show where microsoft provided increased profits or savings over an open alternative.
If they cant give you a dollar amount that shows increased profits or major savings then be sure to tell the rep that he shouldn't let the door hit him in the ass on the way out. It isnt MS versus Open anymore in today's economy.. it's what can get it done and save me money or can give me more profits... and this is what makes Open solutions win... microsoft can't give savings and the performance difference isnt enough to give profits that will more than overcome the added expense of Microsoft.
Get real numbers, talk to real people running real clusters on all platforms. if you have real numbers then you can make solid decisions.
Do not look at laser with remaining good eye.
Windows clustering works as advertised for the most part, but is expensive. Some exceptions include heavily loaded machine pulling from fiber channel arrays and NAS. Both of the network attached devices seem to have some problems. Driver issues? Don't know.
Haven't seen the reported "bsod round table" where one machine crashes, shortly followed by another and another. The problems we have seen is a single machine bsods, and the other machines in the cluster don't realize it's down.
If your already in the MS camp, it will work, it look at other solutions. I think they will be more cost effective.
"Science is about ego as much as it is about discovery and truth " - I said it, so sue me.
...with M$'s "Computational Clustering Technical Preview":
* PLAPACK package (open source software)
heh.
-JT
MPI Pro 1.6 from MPI Software Technology, Inc.
Cluster CoNTroller 1.0.1 from MPI Software Technology, Inc
These found on the MS website. It sounds like it's more than load balancing if you ask me... (mpi pro, that would be a commercial version of MPI? I've never used that particular implementation before, only mpich)
Cluster lizards
You can do a windows cluster thing, but it's still not as good even as Condor for Unix. All in all, I'd say to tell them to go screw themselves unless they want to give you money for a LOT more hardware as well as software, to make up for the fact that you're not going to be able to do as much with it. If MS wants to be taken seriously as a hardcore number-crunching OS, the bastards can EARN it instead of trying to bribe academics.
I've been looking at this a lot myself now, as I'm also building a cluster for use in a computational bio lab at Florida State. It certainly seems that Linux is the only way to go right now. In case anyone cares, my cluster right now is 16 nodes of:
Tyan S2460 with 2 Athlon MP1800+ processors per node
1 gig PC2100 RAM per node
20 gig 7200 RPM Maxtor HD
3Com Gigabit over copper Ethernet
low-end cheapass video and floppy, etc.
All in these really nice rack cases, with a big black 2001 monolith-esque rolling rack to shove it all around in. It cost just about $26,000 to build so far, but the plans are to expand it to as many as 512 nodes within the next year or so. Whee!
How good is any MS product in its v1.0 release?
Seems to me that historically, MS rushes a v1.0 product out to stem the tide of a competing product and then spend the next couple releases getting a "real" product out the door.
I have zero experience with unix clustering but would be suspicious of the MS offering until it has a chance to mature.
That said, with the three computers I have at my place (a p3 desktop, a celeron I use as a low grade server, and my p3 notebook) I'd love to be able to set up a cluster for encoding. Such operations will be the killer app for clustered systems IMHO.
www.lonseidman.com
There's your opportunity!
N quad xenons. All required licencesx2. Gig-e switches. Hardware hardware hardware. no use in lowballing here.
Plus cash. A couple of hundred K$. A connection to internet2.
Airborne Pigs might be useful too.
I believe Indiana University has two hardware-similar clusters, one running Unix and one running some flavor of DOS. I don't have to URL but it shouldn't be hard to find.
At my last job, we had a COW (Cluster of Workstations) running all sorts of operating systems. Except Windows. Why? Because they won't run in a production environment for more than a few days without freezing or crashing, and the system administrators refused to babysit them. With Windows 2000, I've had my home machine run for upwards of 28 days without a reboot, but only if all the video drivers are stable and the machine is not doing too much at any given point (say, burning cds while watching movies and keeping my net connection above 200k/s). But so help you if a driver freezes. There's no way to reset them. Your hardware will play into your decision as much as the operating system, I believe, due to stable driver support.
In terms of performance, Windows kernels have pretty good latency compared to 2.2.x linux kernels, so running a full screen dos app might give very good performance, but there's a lot of overhead munching into your RAM, which is likely to be an expensive premium on older hardware.
Lastly, with Windows, I've never heard of doing channel bonding for ethernet (3 100TX cards ~= 1 gigabit), nor diskless booting that I know of. These can be really necessary for large clusters to keep maintenance down and performance up without buying higher end equipment.
Any connection between your reality and mine is purely coincidental.
but based on personal experience, Windows ME is pretty much a cluster.
'Same speed C but faster'
gee, after a fudge up like that, I would think that I would dump Windows rather than take them back after they promiss it will be better.....
Ahhh well, PHBs are like abused wifes I guess.
I am the Alpha and the Omega-3
While I haven't been near a Microsoft Cluster in a while, I do remember a couple of things that really stand out about them:
The number of systems able to be part of the cluster is severely limited. At the time, it was limited to 2, but I'm pretty sure that has increased to a somewhat larger single digit number.
The number of applications available to run on the cluster is just as severely limited. Again at the time, there were exactly zero applications, but I know that there is at least one (Exchange) now.
Given the limitations of what uses you can put an MS cluster to, I wouldn't bother with it in the first place.
"Suppose you were an idiot..... And suppose you were a member of Congress... But I repeate myself."
Could you imagine a Beowulf cluster of MS Windows clusters?!
pr0n - keeping monitor glass spotless since 1981.
Imagine a Beowulf cluster of these!
That MIGHT make sense if he didn't need hardware too. Since THAT'S the case, he should head out to IBM, get a decent midrange, and cluster linux on virtual machines.
No worrying about Software licenses, AND "Professional" support from IBM for both hardware and software.
"I can't give you a brain, so I'll give you a diploma" - The Great Oz (blatently stolen sig)
The only way an NT cluster would make any sense is if you already have a license that has a bunch of unused seats on it. If you don't have to pay anything extra, it might be worth the trouble to use NT for the convenience.
On the other hand, this is one area where MS sorely lags behind in software. For a research cluster, you'll have very little in the way of applications and be very, very alone when something goes wrong.
Also, the whole ethos of computational clusters is for performance. You won't be able to compile the operating system with just what you need, tuned to your hardware and processors. You'll also blow memory with a desktop you don't need. This just isn't a good idea for any serious cluster.
there are some (as far as i understand) very good macintosh clusters that are very easy to use and very fast. especially if nothing (significant) has been done yet, a macintosh cluster computing G4-optimized code would blow away anything else in its price range. I can't say I have ever used one of these, or any other cluster for that matter, but the genuine power and versatility of the mac tells me its gotta be good.
We researched MS Clustering very extensively. We're already an MS shop and even still it was cost prohibitive.
Notes from experience:
1) Clustering with Windows requires one of the following OS setups: Win2K Server WITH MS Application Center, OR Win2k Advanced Server. (Similarly with the XP platform)
2) OS Licenses therefor will run between $1000-2000 _per-machine_!
3) If you need Application center, which you likely will, you're talking (If I remember correctly) about another $1g per.
4) Of course MS is just getting into this so don't expect it to be easy, well documented or stable.
Finishing Notes:
Obviously, Linux would be mucho cheaper
Easiest, and still cheaper than MS would be the Plug-n-Play Mac solution!
No Comment.
Persuade one of your mates to sell them a site license for Linux. If that doesn't work, find some pro-Linux company and offer them some easy cash.
if microsoft could do a computational cluster, wouldn't they have a 5000 box cluster in the super computer list just like the linux clusters.
I run a large B2C e-commerce site with Win2k server. We are running a "cluster environment" with a Cisco Content Swtich providing the load balancing functions, MS SiteServer providing file replication, and SQL server doing DB replication (where the content switch manages failover).
We have tried using Windows 2000 native clustering (file/database). We have found that with limited resources and budget this is difficult to do. You generally have to be running active directory to have the cluster work effectively in terms or load balancing and cluster management. This is a total pain in the ass to manage especially if you are doing a one-off active directory setup.
I have been told MS application center manages this correctly and allows for true software level load balancing. But, I only know of a few large installations using this. Generally, I have found a Ciscoed solution to be the most prevalent.
Bottom Line:Microsoft makes clustering and load balancing to work because you can't rely on one solo server (not matter how big the server is).
Since the middleware runs on WinNT and friends, so will your applications (when properly ported ofcourse).
For example PVM, there are MPI implementations for WinNT too, I just don't remember any links of the top of my head.
A: Give away your OS and clustering software, complete with source code.
Anyone here run version 1.0 of the Linux Kernel? How well did it run.
Face it, most software sucks in it's first version. It's a sad fact.
"For a successful technology, honesty must take precedence over public relations for nature cannot be fooled." -Feynman
Check out ACME at Perdue University. It was setup by a couple grad students on the cheap and really is a model of inexpensive high-performance computing. I think they only spent a coupe grand on the whole thing with help from the school scrap yard. Some good lessons in there. Oh, and they run FreeBSD which, as it's name suggestes is FREE!!
Have a Happy.
The main point of clustering is to take a bunch of "cheap" computers and make them work like a super computer.
If you really want a performance comparison look at the minimum sys req for the clusters:
Linux i386+ with 16mb ram and 200mb hd space.
(unlimited nodes/users)
Windows Win2k/XP server p2/300 64mb ram 800mb hd space.
(+$200 per node/+$500 per every 5 users)
with all of the overhead for windows gui and other apps (whihc are mostly usless in a cluster) you would need three times the system for a windows box to get the same perfrom as a *nix/bsd box.
Unless of course you can cluster DOS then whoa buddy watch out!
I sig therefore I am...
This troll has a point, most Universities have a licensing deal with Microsoft that makes licenses much, much cheaper. So the OS cost should be inconsequential, what matter more is how it performs doing the tasks you cluster computers for and how hard it is to write code for that platform vs. Beowulf. I would think the CPU overhead of Windows would make it not fare well when compared with Beowulf Linux. I realize Windows can arguably outperform Linux on web serving but that's much less an issue of maximizing CPU use.
The only reason I can think of to go with the Microsoft clustering is if its the only way you can get some decent hardware. If you have 20 PIIs now, I would think 20 P4s would trump them even running Windows.
really? anyone wanting to cluster all those licence fees has got to be out of their minds.
Can you imagine a beowolf cluster of these?
Can you imagine a beowolf cluster of these?
Will can you?
Can you imagine a beowolf cluster of these?
I've been lucky enough to have access to a Beowulf in my current job, and the way we use it, Windows just wouldn't fly, because there isn't a powerful command line.
We're primarily using the Beowulf for computations which are "embarassingly parallel" - in other words, tasks for which it is trivial to partition the input into 16 equal-sized pieces, give one to each node, and then collect the results and paste them together. For example, multiplying incredibly huge matrices and brute-force keyspace searching are embarassingly parallel.
For us, the primary advantage of running Linux on the Beowulf is that most of the time we don't need to write custom software to speed up a calculation. We just write a shell script that rsh's into each box and runs a program with slightly different command-line parameters on each one.
Obviously for some computational problems it's worth using MPI to have the processes communicate with each other, or load-balancing software so that we can run lots of smaller, but different-sized jobs, and these techniques would probably work equally well whether you're running Windows or Linux.
But for experimentation and prototyping, and quickly distributing easy problems, I think there's an incredible advantage to having a command line. (Of course you could install Cygwin on all of the Windows boxen...but why?)
There are plenty of resources on the net that provide specific details about building clusters and how to optimize the performance. don't forget applications need to be re-written to make them friendly to distribute/parallel processing.
If there's one place where Linux excells and Microsoft needs to be kept out of with armed guards, constentino wire, and rabbid dogs it's the computing research centers in higher education. Scraping by to live and make post graduate tuition can suck, but having to fight for grant money that only lines the pockets of the richest man on the planet just so you can do your thesis is adding way too much insult to injury. For the sake of future scholars, show this salesweasle the door with the help of your foot.
*** Sigs are a stupid waste of bandwidth.
If cost is an issue (since you are running on old hardware) then why would you switch to Microsoft? Not to mention the fact that Microsoft Cluster isn't something to write home about and it most certainly won't run on old hardware. Can you give more specifics on the hardware you have and what you want to do?
Everyone talks about setting up beowulf clusters. It's pretty easy to set them up, just make sure there is a lot of usable bandwidth between the systems.
The question here that isn't being asked is about the application. Sure, you have a cluster. But just what is it doing? What numbers are you crunching with that many gigaflops? To take the beowulf idea out of the realm of geek bragging rights into actual useful production takes an application, and you can bet that most are customer designed in house.
Very little of the OS itself is involved in the real applications that make beowulfs useful and money-making. Take a look at your intended application, and see what its requirements are. If you are writing it in house, tell the MS rep to take a leap, since you wont have to worry about 100+ MS licenses, Visual Studio licenses, or whatever else. If your intended application requires MS OS underneath, hold out on the rep until he agrees to a dramatically reduced price on the software. But worrying about the OS in a cluster before looking at the application is counter productive.
Toodles D. Clown
HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA HA
Can you imagina a beowuld cluster of these? I know i can or cant!
boom.
where are my glasses?
Can you imagin a a beowuld cluster of these?
Oh there they are. up your ass. i wonder what they are doing there.
Can you imagina a beowuld cluster of these?
Slashdot zealot are almost anti-ms users. I think they will reply Ms sucks and linux rocks even if they actually don't know what a cluster is.
For a computational cluster, the OS itself shouldn't really matter. What matters is, do you have the tools you need, and does the environment allow you to work with the cluster in a flexible way.
For a typical compuatational cluster, what determines the performance will be the quality of your application. Only if you pick an OS with some extremely poor basic functionality (like, horribly slow networking), will the OS have an impact on performance.
People optimize how their application is parallelized (eg. how well it scales to more nodes). The OS doesn't matter in this regard. They optimize how well the simple computational routines perform (like, optimizing an equation solver for the current CPU architecture) - again, the OS doesn't matter.
So, in this light, you might as well run your cluster on Windows instead of Linux, or MacOS, or even DOS with a TCP/IP stack (if you don't need more thatn 640K
However, there's a lot more to cluster computing than just pressing "start". You need to look at how your software performs. You need to debug software on multiple nodes concurrently. You need to do all kinds of things that requires, that your environment and your tools will allow you to work on any node of the cluster, flexibly, as if that node was the box under your desk.
And this is why people don't run MS clusters. Windows does not have proper tools for software development (*real* software development, like Fortran and C - VBScript hasn't really made it's way into anything resembling high performance (and god forbid it never will)).
Furthermore, you cannot work with 10 windows boxes concurrently, like they were all sitting under your desk. Yes, I know terminal services exist, and they're nice if you're a system administrator, but they are *far* from being usable to run debuggers and tracing tools on a larger number of nodes, interactively and concurrently.
Last but not least, there are no proper debugging and tracing tools for windows. Yes, they have a debugger, and third party vendors have debuggers too. But anyone who's been thru the drill on Linux (using strace, wc -l
So sure - for a dog&pony show, windows will perform similar to any other networked OS with regards to computational clusters. But for real-world use ? No, you need tools to work.
cunt ass fuck fecal slam drip shit fucker cum ballz dick tits nigger cock suck
The ability to run on really old/unused hardware
One free license per machine
Simple... ask for :
- Modifiable source code... essential for University level research.
- Blanket permission to publish research methods and results, including code.
- No NDAs that could limit a student's job oportunities. (i.e. "No Compete" clauses etc.)
- Free or low cost would be nice
:o)
Jonathan Weesnermy right is indifferent.
If you want to do some kind of renderfarming or number crunching across a network, why would you need *MANY* copies of win2k Server? I might have missed a point, but win2k datacenter is about load balancinglike bandwidth managing, IO requests, and uptime if one of the machines fails, etc...
:)) , but in your case I'd say keep with what you've got unless you get a buttload of funding and a very good reason to move to win2k (which I don't really see), because a datacenter plus admin will cost you in the 6 digits to maintain and license.
If the server holds the data and you have a potential of a lot of clients doing requests (thus I/O, Bandwidth, like a P2P crunching system to name a popular example) In that example, I don't see why you'd want to switch to microsoft if you got it to work on linux, you'll need to have a very good knowledge (or hire someone with) of Microsoft Server products if you want to move to anything more than a standalone server. Also last time I checked with M$ for that solution because I wanted a safer domain and maximum uptime, everything was doubled for 2 machines, I thought it would be a bit cheaper than that but heck, for the price of the Advanced server VS the standalone, with 25 users, you can get an extra tape drive and cheap RAID1 to mirror your critical drives (on a small buisness server)
So if you mention that you WISH you'll get donations, and you want raw computing power, instead of buying MS licenses, concentrate on the goal you try to acheive: distributed crunching power with scalable servers, so basically you'll need HARDWARE to crunch. (I still don't get why you'd NEED server to run number crunching, workstations can do the same and transmit to a server, like I was stating before). Check what you have, check what you need, design around that, do a cost analysis since it seems to be very critical in your case.
There are some cases where you'll want MS servers, here at work I've setted up a MS server to have less configuration and troubleshooting issues with my win2k Pro machines (at least I know when something screws up it's MS related for sure
--- Metamoderating abusive downgraders since my 300th post.
i had built a dual-boot NT / linux cluster at my previous job. there was an mpi forNT at that time and i used a samba server to unify user homes across all the machines and both OS'es. we had a similar experience with microsoft, but they were trying to throw hardware at us to keep the cluster "NT only". Opensource tools like samba , VNC, and MPI, were instrumenta in the functioning of the NT cluster. On of the biggest annoyances occured when a job would crash and the mpi would call win32 to create a window with an error message. The shameful aspect was that the processes on that machine could not be killed remotely until someone went and clicked off those annoying error windows! The kill command in the reskit failed to stop those processes since they were being blocked in the win32 part of the kernel! (at least i think so).
You won't find a single Cluster running Window$ in the TOP500 http://www.top500.org
Try submitting this question to beowulf@beowulf.org or to the LinuxBIOS group linuxbios@lanl.gov
You'll get some pretty thorough responses from actual cluster builders and users as to why nobody bothers with Window$ in clustering for performance computations.
.
Quidquid latine dictum sit altum viditur
Try running W2K clustering on any but a P3 - a fairly fast one with a heap of memory (no pun intended.)
(Speaking of memory -- who in their right mind puts a GUI-based OS on a headless machine???)
For shoestring operations, Beowulf is recycled computing.
more like Cluster Fuck
Something to remember with Windows is all the licensing hassles. MS2k Server can requirement multiple licenses depending on what features you use. I spend 5% of my time dealing with this crap. My advise: RUN AS FAST AS YOU CAN AWAY FROM WINDOWS AND ANY OTHER MS PRODUCTS! RUN, I SAY, RUN! (sorry about the caps but it needed to be said).
I'm waiting for them to sell the "Microsoft Windows 2000 Server Keyboard License" for console access (The Control, Alt and Delete keys will be covered by separate license).
I run [the IT stuff for] three offices in two countries and if I could convince the bosses to switch to something else I'd do it in a "New York minute". (Man, I hate Windows).
AC
Much of the comment pool seen thus far is about the MSFT view of clustering, which is entirely possible what the MS rep was thinking. Computational science simply is not on MSFT's radar as far as I can tell. Thank your favorite deity for that.
Here are your issues. Most computational science codes are written for unix based systems. Cluster codes are either latency sensitive (so they use Myrinet/other) with protocol stack bypasses, bandwidth sensitive (need hundreds of MB/s to do their work, or dataflow sensitive (need packets arriving in the right order in a just in time mode). Most computing centers have hundreds or thousands of users, who need simultaneous access to the resources.
The pragmatic view is that your cluster needs to support many users simultaneously (windows cannot), manage large data flows correctly over high bandwidth low latency pipes (windows cannot), and do so in a manner whereby your costs (porting code, end user costs) do not rise as the number of users increase.
In short, unless MSFT donates lots and lots of new hardware (256 nodes or more of late model AMD/Pentium gear, with 2 GB ram, and 50->100 GB disk per unit, gigE adaptors, etc), you very likely could not effectively run windows in the first place, and your costs to run would skyrocket without some serious software license donation by MSFT. Not to mention the cost of the programmers you would need to port the Unix codes to the windows compilers. Not to mention the additional support headcount you would need to maintain the beast.
In short, MSFT based computing clusters are simply not viable. This is from a cost wise basis, a time wise basis, a headcount basis, and so forth. If MSFT is willing to help you offset all the costs you would incur, great, go for it. Otherwise, have a good long look at some of the cluster linux distros, and stay far away from MSFT products.
Disclosure: I work for a cluster vendor. We will sell what the customers want. Customers do not want MSFT clusters. If they did, our business would be brisk. Never had one inquiry.
pretty much all of the Intel server cards as well as several of the desktop cards support channel bonding. all compaq server NICs support this as well, and it works great.
however, I would take issue with your assertion that 3 100mbit cards are roughly equal to a gigabit card. while it's true that something like 4 100mbit cards will give you close to the real performance of a gigabit card when used on a low end PC, there is much to be gained by using actual gigabit (use of giant frames, better latencies, etc.)
if you're going to build a cluster, and you actually have a budget, you're going to buy decent yet cheap server boxes. these will most likely include 64bit PCI slots, and there lies your motivation for gigabit. the performance there is unparalleled when using a real wirespeed switch, without using faster technologies of a proprietary nature.
my 2 cents.
EOM
I have worked with Microsoft Clustering before and as one poster put it earlier, MS Clustering is mainly for reliability. If you use Advanced Server, which costs a couple grand PER install, then you can cluster TWO systems together. If you need more than two systems, you have to go with Enterprise Server which costs quite a bit more per seat. It handles fail-over fairly well, but for distributed computing, the software has to support it.
While I personally have not worked with Linux Clustering, my group is working on bringing a couple of 20 system Linux clusters online. For distributed computational projects, that is the way to go.
resistance is futile...beowulf jokes are unavoidable.
on a more serious note, are any of the distrubuted computing projects (SETI@home, AIDS@, Cancer@, the gene folding project, etc,) using windows?
no? maybe there's a reason.
FreeBSD for the impatient.
blah blah blah blah Linux cluster blah blah blah.
...Execute search MS - terms: cluster
Results: Microsoft Clustering, formally known as wolfpack.
...Execute talk: Yes...MS does clustering, what would it take to convince you to use ours.
I think if I was in the customer's position, I'd agree to it just to shove it back in their face when I ask how it distributes the computing load etc...of course that would be
blah blah blah computing load blah blah
...Execute search MS - terms: computing load
Fuzzy Logic Results: Microsoft Clustering, formally known as wolfpack. Use for load balancing.
Software Included in the Toolkit
Microsoft Windows 2000 Professional evaluation version
Microsoft Windows 2000 Server evaluation version
Microsoft Visual C++® 6.0 Standard Edition
MPI Pro 1.6 from MPI Software Technology, Inc.
Cluster CoNTroller 1.0.1 from MPI Software Technology, Inc.
Visual Fortran 6.5 Standard (Trial Version) from Compaq
Math Kernel Libraries 5.0 from Intel
Computational Cluster Monitor from Cornell Theory Center
PLAPACK package (open source software)
Does this mean that you can get all this stuff (including Visual C++ 6.0) for just $7.95 ???
But your only real choices for what you want are:s eed/appleseed .html
Linux cluster (Beowolf)
http://www.beowulf.org/
or
Mac cluster (MacOS or LinuxOS, AppleSeed):
http://exodus.physics.ucla.edu/apple
There is no real, available, and viable WindowsOS solution. You would be in beta-land, at best.
Consider it if MS will pony up big time (all the HW, SW, and maybe cash for overhead, salaries, gifts to the campus library). Consider it a research project (proof of concept).
You won't be getting your "real" work done with this, though. Use a proven solution if that's your goal.
that he says he "works for a mid-sized mid-western university" when his handle has a link to a Ball State University email address.
Just come out and say it.
"A plan fiendishly clever in its intricacies"- Homer Simpson
I guess the topicstater had his answer; Beowulf is for fast computing, Windows clusters are for load balancing.
What I would find more interesting, how is the stability managed?
High end clustering on Intel is still delivered by Calderas UnixWare (Openunix now). A cluster consists of 2 rootnodes which manage the cluster. If one rootnode goes down, another clustermachine becomes rootnode.
The rest of the system is also designed that there is the least possibility of a single point of failure.
I do not know if Linux and Windows can compare to UnixWare clustering.
On Linux it will not be Beowulf then, Beowulf is for speed, not stability or reliability.
On Windows I am not sure, but I do remember Bill Gates at some ComDex event standing by a cluster of 7 Windows 2000 machines, running Fords website. He turned off 1 machine the hard way, and the website kept on running. I do expect him to exactly know which machine to power off, and be sure not to hit a rootnode.
Well, don't worry about that. We can get you back before you leave. (Dr. Who)
:)
hawk
The company I used to work for was doing quite well in terms of IT, then we were bought out, and the new guys basically bent over and took it up the tailpipe from MS, doing everything MS asked, never questioning anything, and the networks went to hell.
Quasi-related. Lets say I have a bunch of machines sitting around. Is there anything the end user would use Beowulf for?
I don't mean - would it make vi run faster. I understand the very basics at least. But aside from having to write my own code to take advantage of the cluster, are there apps already beowulfized?
I could see where POVRay would be a good app to do this to. Have the controlling machine ship off each line of the rendering to a machines CPU and come back with the result a lot faster.
With enough machines that were fast enough, you could almost provide a real time rendered view of a work in progress.
IAABA. (I am a beowulf admin).
Beowulf clusters get built to support your application, not the other way around. Your choice of hardware and OS will depend on the parallel nature of your code. Do you need myrinet, or can you get away with fast ethernet? Will your code even compile under win32? Do the supporting libraries (PVM/MPI/BLAS whatever) run under win32? What about the queuing system?
How are you going to manage the cluster? You need automation, even for small clusters. How easy is it to add a new user, apply a patch or change a bios setting on your cluster without having to plug a keyboard and monitor into each node? What about central logging? How about automated OS installs when you add another 100 nodes when you get your funding?
Oh. Benchmark, benchmark, benchmark. That means your code, running your datasets, on your hardware and OS. Not vendor supplied numbers. If you have a serious hardware vendor, you should be able to wrangle demo mechines off them. Try before you buy.
The Cornell Theory Center, a supercomputer center with at least one machine on the Top500 list uses Windows clusters exclusively.
http://www.tc.cornell.edu/
This isn't Wolf Pack. These are Beowulf-style clusters using Windows. MPI, F90, LAPACK, the whole works.
I worked with several research organizations that did some serious number crunching and none even considered windows products for clustering for a few reasons.
One is the fear is that if you submit the job that may take a few days to complete you'll come back to a blue screen.
Also, most organization like that work on tight budgets from grants and stuff and need to get the most bang for the buck (non-profit).
If you look at any organizations that do serious computing (number crunching not web serving) don't use Windows, other than perhaps desktops.
All site licensees should remember who it is they are dealing with.
In the not too distant past:
At least one cancer research center had its budget seriously dinged when MS educational licenses ceased to apply.
Large corporations everywhere were pressured into Enterprise Agreements requiring them to keep current levels of Microsoft software.
And so on.
The Enterprise Agreements are especially interesting if you remember the original plans for Windows ME. Microsoft originally planned to strip out all network support in an effort to force businesses to upgrade to W2K.
They ended up backing down that time.
Now, however, imagine a plan that requires clients to upgrade their software but doesn't require Microsoft to guarantee that the upgrade will contain the same functionality.
Oh well. None of my business. People dumb enough to get into these situations deserve what they get.
That's one beowulf cluster that I DON'T want to imagine.
So, he should buy a single computer, emulate a bunch of computers on it, and then run software that to a certain extent allows all the little virtual computers to act as one?
I've got an idea. Maybe he should emulate 100 apple2's on each virtual linux, and then hand code 6502 asm, allowing each virtual a2 to do beowulf style clustering on each virtual linux box, with mosix clustering between each apple2 virtual beowulf on each virtual linux box, all running on a nice IBM mainframe.
I mean, if you're gonna add stupid overhead for overhead's sake, why not go for the gold?
Been there. Done that.
MS clustering is for load balancing and stability. As such, it does a reasonable job. Beowulf is for high-end scientific computing and does a resonable job.
To really do clustering well, you don't want either. You want AIX or Solaris, but you probably can't afford them. But, Linux clustering in load balancing style is developing quickly with IBM, TurboLinux, VMWare and Intel doing many interesting things. Beowulf is still cheaper.
In any case, though, for your situation, there's only one solution and that's Beowulf. Besides afford MS' licensing fees, you mentioned that you're running on older equipment. I sincerely doubt that those servers could run W2K Server in its standalone mode, much less in clustering.
Steven
Years ago, I worked at an ISP that ran partly on Solaris, mostly on Linux. A few MS reps came in to try and get us to switch to NT. We let them go through their routine, then walked them around the operations room, telling them the capabilities of what we had, and asking if NT would match them. The response was repetetively "no". When we pressed them on a few issues, they gave in rather easily. When we asked them why you couldn't bind another IP to an ethernet card under NT without a reboot, they admitted "lazy programming."
So, take the MS reps through the operation, tell them the capabilities. Ask them if they can meet or exceed them. If they say "Yes", you're either not using the real capabilities of your Linux machines, or they're lying.
steve
Oh, you're not stuck, you're just unable to let go of the onion rings.
Why would microsoft get a license fee per box ? Their goal is to rule out linux, so they ll probably do price-dumping, create some 'academic' style of contract with non-disclosure agreement and garanty of exclusivity.
I think that rather than post advertisment, slashdotters should face the danger, and adress it. (i.e. code, sue, boycott)
It's a .gov so they have to use some contract we already have with IBM hardware and the MS site liscense. The director of my department is a big MS fan (even after he upgraded to XP on his laptop and corrupted the drive). I hope to be moving to a different department tho where I can possibly run linux on my desktop pc.
Vote early. Vote often. Vote CowboyNeal.
Philosophy major, right?
It sounds like a good deal at the outset, and, yes you're right, the licensing issues, the cost for hardware, even the access to source code are all arguments that can be debunked.
The problem is that Microsoft forces you to give up your ability to choose other options. Microsoft can offer a lot, but in the end, educational institutions need to keep their options open.
It simply isn't wise for an educational institution to undercut the opportunities of their students by signing the Microsoft agreements that won't allow any competing software to be used.
That's the real issue.
And it's not the same as what you can do with Linux clustering. Here's how it works. Bear in mind this is with released software and hardware, not the vaporous new clustering that you can only see at dog-and-pony shows (ie Windows 2000 Datacenter).
.Net Advanced Server will be pricey. How pricey? Thousands of dollars, I'd imagine, per node. Evidence is becoming available that MS's pricing goal for servers has become to increase the hardware/software spending ratio to 50/50, so you'll end up paying so much to license the software that you could have had twice as much hardware. This isn't that cynical, actually, especially if you look at the new SQL Server 2000 licensing options for Enterprise Edition.
We've got a two-node cluster. Basically, it's two dual-CPU boxes, with a shared fibre channel array. The boxes are connected to the public network, they also are connected via a crossover cable to each other (you could use a switch or a hub if you have more than 2 boxes, we just used a crossover because it's more reliable since we only have two boxes).
The boxes are set up as an active-active cluster. That means there are two virtual machines set up on the cluster, so we can actually use both boxes.
The whole "Clustering" part is in failover, not in load distribution. Bascially, if one hardware node on the cluster goes down, the other node picks up the virtual machine that was hosted on the node. We've got to be careful not to load any virtual machine on the cluster to more than 50% of capacity or if a failover event occurs one hardware node won't be able to handle the stress.
There is no shared load balancing in a Windows 2000 Advanced Server cluster. The gentleman talking about the web server clustering is talking about a different solution, the Microsoft Application Server, which is Microsoft's software load balancer/web deployment system that makes a gob of web servers look like a cluster, as if you combined a good content management package with an Alteon LoadDirector or a Cisco content switch (whatever they're calling the Arrowpoint stuff these days). It's different software for a different purpose. The clustering stuff is for failover only.
If you're doing computationally intensive "stuff", and you want to load balance, Microsoft does not have a solution for that. The rep probably just knew you were doing something with "clustering" and that Microsoft sells "clustering", with no real knowledge of the fact that the clustering that Microsoft has is designed for something different than the clustering you need.
As far as "what would it take to get you to switch", I'd suggest that your answer should not overlook a statement like "Free licenses in perpetuity". Microsoft would probably be more than willing to hand you an eval, or even a few licenses for the current version, but the upgrade to
That's another point, if you want to use clustering, you have to buy "Enterprise Edition" products, like SQL Server EE, which costs 5 to 10 times the amount of SQL Server standard edition.
good luck,
rocketscientist.
It all depends on what application it is you want to dedicate the cluster to, and whether tools are already available.
If you wanted a cluster to do animation rendering and were based on 3D Studio, then you'd probably want your farm to run Windows. Web work could go either way, with Linux being a less expensive solution. A custom coded app would probably be best implemented under *ix, though I could see situations where there might be some resources more readily available under Windows.
They replaced an old IBM SP2 with an Intel cluster running W2K. It doesnt show up very high on the netlib list but I would certainly call them and maybe pay them a visit.
is that you'll have all your bugs in one place!
You should check out the AC3 project at Cornell University's Theory Center, which is "home to the largest Windows-based high-performance cluster complex in the world".
There are numerous machines, such as the 256 CPU Veclocity 1, that run MPI-Pro over MyraNet(?), that was one of the 500 fastest computers.
Windows is a very viable and high performance solution for running scientific parallel application, and you should order the $8.00 evaluation kit from MS and check it out for yourself.
I've developed for some of these systems, and have been very impressed. I've worked with Linux clusters too, but only on older, weaker machines, so it would not be fair to compare the two.
(Btw. all opinions here are my own, and in no way should be construed as those of Cornell or the TC).
One thing you might want to consider is administration time, scientists, who are already annoyed that programming destracts them from their real work, might not want to devote the time and effort to learn to and administrate all those linux boxes.
Anyway, if the MS rep is very eager, he might offer you some great deals. MS is very eager to be taken seriously as an HPC option.
Yeah I was helping a grad student the other day setting up a Windows cluster.. its not fun.. not terribly hard, but some of the requirments, and that stupid ass HCL that they have make things a bitch. I would think it would be more likely used for load balancing than anything else, sure you could use it for computational things, but the overhead that windows needs for the OS means that you would get less CPU power than if the system was a Beowulf one
"After all, we're all alike."
:)
hawk
-
Failover clustering. This is an OS service that servers like SQL Server and Exchange plug into that allows Active/Passive or Active/Active clustering over a shared SCSI/Fibre bus. In theory you could write your app to use this service but I think it would be overkill.
-
Network Load Balancing. This is just a software version of the standard kinds of NLB found in cisco boxes.
-
Component Load Balancing. This is the most suitable. It's provided by Application Center and it allows you to deploy COM+ objects on a cluster of machines and have the calls distributed according to the load on those machines. You can control the threading and lifetime of the objects and view the status of the machines pretty easily using the Application Center MMC plugin (or SNMP, I believe). You'd have to wrap the computational part of your application into one or more COM objects. Once you've done that then you can create and call those objects in the cluster as if it were one machine - the clustering is transparent to the client application. I played around with AC a bit when it was in beta for a project that I was working on. We didn't go with it in the end because the design of our application ended up not requiring it (we just went with hardware load balancing), but it seemed like pretty cool technology - if you're into the whole COM thing. It has a really cool rolling deployment feature where you can redeploy your components (and/or IIS application if you have one) to your cluster incrementally while it's still running.
Here's some links to docs on MS's site:Introducing Windows 2000 Clustering Technologies
Application Center home page
Component Load Balancing
Back when Beowulf started there were a few universities that had dual boot systems. I think they ran Linux half day, Windows the other. Windows NT and above have had implementations of MPI and PVM that have worked quite well for a while. MPICH and PVM are both freely available, the problem is that these systems do not integrate well with Visual Studio.You end up needing cygwin/gcc, which is basically Unix on Windows
Most clustering software today is either open source or based on open source and made for the Unix environment. So besides the obvious license and stability questions others have brought up Windows has limited tools and libraries.
ALSO:
There are problems with remote administration, being tied to graphical interfaces for such simple nodes. Wasted efficiency by running the GUI.
Finally Microsoft has a VERY limited knowledge base for this application of Windows nor do many HPC people know anything about Windows.
As you can see from the above, Linux on HPC is basically able to take those same horrible excuses for running Microsoft on the desktop and shove it down their throats.
Yes, Windows can do HPC, but why would you want to?
A few points:
You can set up any application or service to cluster & fail over if required, as long as:
Active/Active mode is more complicated, meaning instances of an application running on different nodes, all accessing the same data on disk. Only certain applications can do this successfully, e.g. Oracle, which does so by using a custom file system and effectively bypassing the Windows Cluster Service. Windows & most apps will normally throw a fit if there are clashing file requests from multiple nodes, since Windows caches file tables in memory and can thus lose track of the real situation on disk (bad news). I've seen it BSOD in such cases.
(this is not a
So the most important determinant of performance is the compiler and whatever libraries you need (such as an MPI library). It's hard to get identical versions of these under both systems, so a true OS-to-OS comparison is hard.
The OS can play a performance role in things like the TCP/IP connection. I recently reviewed a paper which included some comparisons of TCP/IP latencies, and Linux was about half the latency of NT, so communication-intensive apps that use standard sockets would run faster under Linux.
A few prominent institutions have become shills for MS clusters for Beowulf-style computation, but it turns out that MS funded their clusters. If your university has a good name than tell your MS rep you'll switch if MS buys you a big cluster of 2GHz processors and pays your sysadmins' salaries. (They make such offers to big-name universities.) And if you see any papers promoting NT Beowulfs, check carefully to see who funds them.
At my place we used to have some WLBS (Windows Load Balancing Services) systems.
I maintain a ms cluster which is a high availablity cluster. It runs exchange and works well. In looking for computational clusters, I found Top 500 cluster list. Most of the machinces are none intel based and can't run a ms cluster.
Alliance Supercluster Ranked as the World's Most Powerful Intel-based NT Cluster is a press release showing that a NT based cluster broke into the top 500 list in 207th place.
The microsoft site lists the cluster toolkit as a tech preview. It look to me that MS have found a new source of revenue and is now trying to get people to use it.
Microsoft brought us Windows XP. I bought a Mac.
Currently at home I'm running a cluster of 2 Windows XP .NET Enterprise beta 3 Servers. The cluster will not speed up the processing of a program. It will only provide more stability if the computer should fail.
The XP cluster provides load balancing for an application by making virtual clusters, which have resources like an IP address or computer name. These resources can then be moved between computers at will.
If a basic application (one that was not made for clustering) is assigned to a virtual cluster it will automatically start on the computer that is the main host for that application when the host is running. If the main host fails the program will be started on another system that the virtual cluster is assigned to.
If anyone wants me to, I'll post excerpts from the help file.
I just can't imagine a beowulf cluster of those...
If your admins are running Exchange2000 on adv.server using an active/active configuration
fire them.
Exchange 2000 has SERIOUS issues with active/active configs that result in loss of data among others. Tell your admins to take a look at the MS white papers sometime.
It cluster-fscks nicely thank-you-very-much.
this article without the obligatory Beowulf comments?
Escher was the first MC and Giger invented the HR department.
This wouldn't have happened if they had thier servers clustered to handle the load-balancing issues.
Can you say Charlie Foxtrot
Tell the Microsoft sales rep that you are using Linux because that's where many of the advances in clustering technology are being developed. In fact, they recently switched from using Windows as the basis of their development to using Linux, and one of their primary sponsors is Microsoft. Since Linux is clearly Microsoft's first choice for a clustering platform, yours should be too. After all, noone ever got fired for doing what Microsoft told them to!
about that mac solution....
/ hp c.shtml
Yellow dog linux sells a cute little piece of hardware designed for clustering around PPC. very cute...maybe the best balance of cost effective and easy in terms of clustering that ive seen.
http://www.terrasoftsolutions.com/products/briQ
-jef
my understanding is that an MS rep asked what it would take to get them to switch to a Microsoft cluster.
You've got a golden opportunity here! Microsoft does it your way or they don't get the sale.
Let them know the nature of a cluster in a research project. Nodes will be swapped in and out. New ones will be added. Different OSs will be used. So tell them you want a copy of Windows for each potential node, licensed to the University and not to any individual node. Tell them you need full rights to install, reinstall, and uninstall any particular copy on any particular node. Tell them you will not accept any terms restricting the cluster to Windows only.
If you really want to play hardball, tell them you don't even want licenses, but bonafide user-owned copies of Windows subject only to the provisions of copyright. In other words, you don't want to be subject to any EULA. Then you'll discover how much Microsoft wants your cluster to be a Windows cluster.
A Government Is a Body of People, Usually Notably Ungoverned
At some universities, they hold a universal license to certain products. Purdue University, for example, holds a universal license to Oracle and certain M$ products. This means that as long as the software is being used for universtiy purposes, it can be installed anywhere. (At least, this is my interpretation) If this is the case, then the licensing issue becomes moot, and the university could install it damn near anywhere they pleased...
"Goodness, how did you people live long enough to invent tools?" -Hobbes (the tiger, not the philosopher)
does somebody heard about alinka cluster ??? they are doing an excellent job, and it's a linux base !!! www.alinka.com
Try looking at Pooch from Dean Dauger. http://www.daugerresearch.com/pooch/whatis.html
This would allow you to use the Macs (OSX UNIXY goodness too) individually as personal workstations (for writing, graphics, computation, surfing the web) while at the same time using them in clusters for compute intensive work. This makes for a doubly productive machine and one that is much cheaper as more work can be accomplished with it than simply using it as a dedicated node.
Mac clusters are easy peasy to set up (even junior high students are doing it) as the one page instructions should indicate and Applescript'ability. Also pretty damn fast given the built in Gigabit of G4's and the Altivec (if taken advantage of like in Apple's version of BLAST).
Finally, the other item of interest. You can use any Mac you have. G3's, G4's of any model and speed as one does not have to balance everything like on typical clusters where all of your hardware has to be exactly alike. The Macs in your cluster can even have iMacs on the secretaries desk involved!
Visit Jonesblog and say hello.
Then, point out the scads of Beowulf clusters and Linux/Unix based systems.
Finally, inform the rep and your management that you've chosen to use the more cost effective, higher performance and standardized choice...Unix.
If management resists further, do a cost analysis. That'll convince them.
299,792,458 m/s...not just a good idea, its the law!
Galileo: "The Earth revolves around the Sun!"
Score: -1 100% Flamebait
...to ask a question that I wanted to ask as well. Granted, this topic seems a little strange, considering the Linux cluster is in place, and it seems like the kind of question which encourages a Microsoft vs. Linux world domination showdown for grandmaster of the universe. It also shows a limited business sense on the part of the poster (why change something that works well when you can't afford a replacement?).
Right now a coworker and I are looking at pricing and configuring a fault-tolerant cluster for a client who runs Windows 2000 and Exchange 2000. They're a bit paranoid, so they've decided they want a cluster. We've tried to educate them on exactly what a Microsoft cluster can and can't do, so it's difficult to understand exactly what they want (basically an entire network exactly like Microsoft's own, but for $1000).
Pricing on a two system cluster is around $50,000. Buying two copies of Exchange and Windows Advanced Server will total $20,000. Then there's the hardware costs. For our client, they've specifically requested this, so they're ready to pay.
My question to Whamo is are they really taking the Microsoft rep seriously? If they have to pay software costs for their new cluster that's going to mean two things: either buying less CPUs to add to the cluster, or not doing the project at all, because just the software will put them over budget. With Advanced Server running somewhere around $4000 that's a lot per machine when Linux costs at most $5 to burn a CD after downloading it via the university's T1/T3/etc. Whamo says "it is running on old hardware and is basically used for dog and pony shows to get more funding and hopefully donations of higher-end systems" and to me that is your answer. If you can't afford the hardware you can't afford to buy Microsoft's software...
Also, there's MOSIX as well, but I don't have much experience with MOSIX and thus cannot comment on it.
Comment removed based on user account deletion
a free license and source code, if their product can perform as well on the same shitty hardware.
GUFFAW!
(You may as well request that he throw in a blowjob in the main quad at noon.)
That's easy. "Can you beat Linux on price? No? Then why are you wasting our time?"
Electricity. No matter what clustering SW or OS you have, you still have to pay the power bill. Chances are the electricity costs for those ancient boxes are higher than the cost for faster, newer, lower-power boxes...
From the link to MS, it appears that although the versions of Win2k are evaluation versions, the version of Visual C++ isn't (I don't give a stuff about the rest).
If you can cope without the optimising compiler of the professional edition (which appears to make little difference which optimisation method you choose), $7.95 (+$1.95 to get it to the UK) for Visual C++ seems like a bargain.
Steve.
Now all we need is a beow....wait, we already have one. Darn!
> the software that you run has to to either be
:)
> custom or custom modified in order to take
> advantage of the cluster.
Yes...and seeing as the user of the cluster is usually either the developer or manager of development of the custom software, Linux vs. Windows doesn't really apply...other than the standard "I'd like to see VB run on a Beowulf cluster" rant - now you'll get your chance!
I've clustered Windows boxes plenty of times. I leave them clustered in a corner of a dark closet. Unplugged.
A cluster of Windows would be a Glass House, and you know what they say about those.
Does it just blow out the circuit breakers?
first off there are several MPI implementations that run under windows. I know PVM has a windows port - as I work with the group that writes it :)(http://www.csm.ornl.gov/torc). you need an rsh client and server for both I think (you do for PVM) I don't know of any free one that works well at the moment, we use ataman rsh services. The other way to go that I have seen is to use the cygwin tool. Another national lab runs a modest size cluster (~512 node) windows cluster but the use pvm and mpi through a cygwin window. As for stability -well they are about the same. On our 64 node cluster (processor craps out, os craps out, whatever). The fellows on the big IBM SP2 get about the same, and the windowz guys get about the same. When you are running the HPC clustering services you are not running much more than a kernel on each machine so no IE explorer and other BS to crash the system. Remeber when people are running real codes (such as weather modeling, nuclear modelling, and genome searching - what they run on our cluster) it would be the equivalent of you running a process that was computing on an aproximatly 6 gig file, using the processors nearly fully, puching the gigether card to it's limit, and using all available ram (512 per node in our case) for several months - the only way a box gets hit that hard is being used a computation node. So yes, it is entierly possible to build even large HPC clusters from windows, though unless microsoft is going to fund you (as they do the other national lab) I would not recoomend it.
------- Sorry about the spelling, I suffer from two problems. Dyslexia makes it difficult to spell well, lazy makes it
Well Get IBM to hook you up with Linux, they dont have all those Linux commercials for nothing.
please gimme half of whatever it is you are smoking.
"I am a warrior, and information is my weapon..."
It seems like the MS rep might actually be willing to "get" the stuff together to make the switch happen. Cost, at least initially, might not be an issue.
My concern is about future costs and scalability. Right now they can use whatever is given them. Some version of Linux that supports Beowulf clustering can be made to work on virtually any equipment they are given (you'd be surprised at the old stuff that has been cobbled together as it is). To me, the growth path is more evolutionary this way. They can ask civic groups, local government and small businesses for their cast-offs. However, the MS solution seems to make them have to switch to straight corporate giving, which can be a hard sell for a school our size.
Thanks for the input so far.
http://www.tc.cornell.edu/news/releases/2001/top50 0.asp
As of November 21st, the Cornell Theory Center's MS windows 256 node cluster was #320 in the top 500 fastest computers in the world. My roommate used to write device drivers for this cluster at CTC, apparently it's quite impressive.
When you instantiate a COM component, you can specify the server to run it on. You can also specify this in the registry.
Isn't this most of what you need to do Beowulf-style clustering?
A friend of mine who was involved with the Rocky Horror Picture show casts (in a coupl eof casts for several years) has said he is yet to meet anyone in a Rocky cast that hadn't been previously molested as a child or raped.
:)
<raises hand>
I don't know if you're trolling or what, but I've been in Rocky casts for years, and I wasn't molested or raped. Nor were several of my closest friends who are also in Rocky casts. Nor has anyone else in the casts I've been in ever even mentioned such a thing. Hardly conclusive evidence, but based on the sample I've taken, I think it's safe to say your friend is either crazy or lying.
- fader
I don't know if their cluster is the server serving the link above, but whatever it is it can't a little /.
For raw MPP numeric processing, W2k is too dam slow. You can boot Linux in 4MB of RAM and less than 64MB of disk, then, just load the libraries you need and nothing else, and you will have a preety decent system. Try thining W2K down and you will have a huge problem there. You can use Sun's GridEngine for Linux (http://www.sun.com/software/gridware/gridengine_p roject.html) and best of all, it's open source!
At the end, it all comes to your soft, if you develop a highly scalable, almost share nothing algorithm, Linux Clustering is the way to go. For fail-over Linux you have tha HA Linux project, once more, Open Source!
Regards!
It fits.
Good judgement comes from experience, and experience comes from bad judgement.
- W. Wriston, former Citibank CEO
Quad Pentium 500 MHz Pentium Xeon's
1 GB memory
6 disk array, three logical mirrored drives.
MS Windows NT Server 4.0 Enterprise Edition
MS SQL Server 7.0 Enterprise Edition.
It should be noted that you must have two NT Server licenses and two SQL Server licenses. If you want to do an active-active cluster it requires four licenses. The Enterprise Editions of these software packages was much more expensive than their standard counterparts. You can not use standard editions for clustering.
Installing cluster services was very easy. The Cluster Manager app was OK outside of the occasional hangs. (Although the manager app hung, the operations were completed, such as failover, failback.)
In order to do active-active clustering you must have two shared storage devices; the active node will only be able to access the shared storage it "owns".
SQL Server installed all right if you followed the MS White Paper exactly. I don't know why, but installation order was important; if you didn't follow it it didn't work.
Applying service packs was extremely painful. The instructions were straightforward but did not work. MS provided us with a program that backed out the SP snafu, which worked somewhat. If it weren't for google we'd have been dead.
MS support is useless IMO. No contracts just pay-by-incident. Have a credit card handy before you do any upgrades of any kind. You will most likely need it.
As long as the cluster was just doing SQL server, it worked great. Failover was seamless. Given the proper hardware, Windows behaved well. Make sure that you only attempt this with certified hardware. Very important.
Once we started adding third party reporting software things started to go bad. Adding it to the cluster services was remarkably easy. However, even though the servers had quad procs and a good amount of memory, simultaneous report requests ground the system to a halt. SQL Server behaved well, around 25% of CPU at most even in heavy load. The reports (JRE) would take up over 50% of the CPU in light load. Very bogus IMO.
A lot of third party apps do not support MS Clustering. Lot's of tweaking to get them to work.
If I were to do it again, I think I would not have used MSCS, but instead have two distinct systems that had some kind of data replication software.
This configuration is also limited to a two node cluster. Although you can run an active-active cluster the instances of SQL Server would be seperate. The data storage areas cannot be shared between the two nodes.
Although I prefer UNIX I try not to be an MS bigot. It does certain things well. I hope that clustering has improved with w2k.
Wake up and smoke some DMT. Clustering on
Windoze machines is about 5 years old.
Terminal server w/ load balancing does not load balance and the clusterd computers do not talk to each other. The profiles on the 3 clusterd servers do not update each other at all.
Maybe you should try setting things up correctly, then.
TS will not even install on a server cluster, but you seem to be referring to what's called a TS farm in the MS world.
I don't see why the TSs would need to 'talk' to each other if you set up connection affinity correctly (if someone disconnects they get the same server upon reconnect). Also, you should store data on any of the 'farmed' servers. Try using roaming profiles or mandatory profiles.
I'm sure the MS guy knows these guys are on a tight budget. It may be that he's asking in the hopes that MS will be able to provide the necessary software (and possibly hardware) gratis...in exchange, of course, for the bragging rights, and possibly use of anything neat they discover or invent.
It may be less of a sales job than an investment in publicity and tax deductions.
gm
Ad luna, Alicia! Ad luna!
The thing to remember is that MS Clustering, which I do use, is aimed at providing Applications. It is not designed to provide Hardware resources. MS Clustering provides either maximum avialiablity (fail-over) for an application or maximum resource distribution (load balancing). What you will not get is a mini-Cray in terms of processing power, memory useage, disk, etc. It was build for business user needs not for pushing hardware to the max.
The interesting thing about MPI (which, by the way, is the clustering software that's part of Microsoft's clusting kit) is that it's also available on Linux, Solaris, and other OS's if I recall correctly.
We used it in college, and we clustered between an array of Solaris SPARC machines and an array of PCs running Win2k. If you wrote your software correctly, it didn't matter what OS was on the thing as long as you had a version compiled for that environment.
So what I want to know is why on earth anyone would want to build a Windows cluster when they could avoid the bloat and use the exact same software in a more robust and secure environment of Linux.
In the Windows environment our accounts had to be granted administrative access in order for us to run our projects, we'd request the access and it'd be granted for the duration that we were in the lab, then revoked when we left, even if we were just "going to the bathroom quick."
Slay a dragon... over lunch!
you should store data
s/should/shouldn't/
Then run the new distributed computing, called The Distributed Folding Project (http://www.distributedfolding.org). It just started last month and there are clients for Linux, Mac, *nix and Windows (in other words: most platforms in use today). I believe this is the first distributed computing project that has a Beowulf cluster install guide :)
If you have questions, visit the forum here: http://groups.yahoo.com/group/distributedfolding/
Microsoft DOES have innovation, quality, and bug-free code, just not TRADITIONAL innovation, quality, and bug-free code.
What was its name now again.. Hmm.
Oh yeah, now i remember its loadlin.exe.
You need to find a large problem that you are trying to solve, or at least research. Find a problem, setup your models, and start crunching.
Then you go to the vendors/donors and say "we need to make this run fast". This is something they can understand.
Until you have a specified your problem, all your doing is saying "buy me a bigger computer to play with".
You really need to ask yourself the question of "what's the goal?". Is it high availability, or
high computation rate?
Here where I work, we have a 132 node (128, plus I specced four hot spares)x1GHz/node Athlon Beowulf, running Scyld. It's actually less reliable than a single processor (why? Same reason that dual-engine planes declare engine-out failures twice as often as single-engine planes... hardware sometimes really does fail, and when you have
nodes stacked like coal in a steelmill, the law of averages _will_ catch up with you).
The machines all boot via NFS, but have a local
hard drive just for distributed data storage (NOT for the OS; having a single bootable image really
helps system maintainability. )
I can't even consider what hellish world we'd be in if we tried to run Windows instead of Linux on
the machine. Imagine trying to install a service
pack on 132 nodes?
So, are you trying to scale to many users, or trying to scale to many computations?
I still have an old 100MB IDE drive that houses a 20MB swap partition and an 80MB bootable ext2. It was originally built to be just an X server AKA graphics terminal so that I could still surf/work while my kid was hogging the better box for her own entertainment. The base distro was Slackware. There are no compilers, no source, no dev-libs, no mozilla, no jvm/rte, no emacs, etc. Worked like a charm.
;-) Hmm, perhaps I should just publish the disk image?
It ran on a 386sx40 with 16MB of RAM(4 30 pin SIMMs remember those?). I still have all of the parts except for the case and power supply. Perhaps I'll put it back together, load a web server and/or telnet along with a published user/password and put it out there for all to see.
Good judgement comes from experience, and experience comes from bad judgement.
- W. Wriston, former Citibank CEO
Perhaps those managers who are also ACoA are driven by a deep, almost irrational, need to help others. It comes from a lifetime of helping their parents cope with what they ought to have been able to handle on their own. I don't know if the "telltale control issues" was the panelist's viewpoint or your editorializing but it doesn't reflect any substantiative understanding of the dynamics of alcoholism.
"the best balance of cost effective and easy in terms of clustering"
e ed .html
...is almost certainly AppleSeed.
http://exodus.physics.ucla.edu/appleseed/apples
Check out the MIT Press book
.
Beowulf Cluster Computing with Windows
Click here
Stupid sig of the week: Perl Hackers DIIMTOW
Check out Tom Sterling's latest book "Beowulf Clustering with Windows" available at Amazon. Tom is the Beowulf co-creator and has put out two excellent Beowulf books recently. The other is "Beowulf Clustering with Linux".
Also, the Cornell Theory Center has been very successful with Windows computational clusters. They have plenty of info online.
The combination of "Microsoft" and "clustering" is where we get the word "clusterfuck"....
Max
My god carries a hammer. Your god died nailed to a tree. Any questions?
A company out there provides commercial MPI libraries for a variety of operating systems, Windows included
http://www.mpi-softtech.com/
A couple years back, I was one of the MPI writers/maintainers for a number of platforms (worked with MPICH). As far as performance went with real applications (as well as synthetic benchmarks), Windows and Linux on the same hardware were pretty much the same. Typically computational problems were faster on Windows though because of the better compiler support at the time. Communications performance benchmarks were interesting.
Also, the at the URL above you can find cluster management software (batch scheduling and stuff).
Obviously, then, it is a repressed memory!! Now stop arguing! The troll is always right!
i hate pansy republicans
a Beowulf Cluster of these???
"The best laid plans of mice and men gang oft agley..." - ROBERT BURNS
Ive seen a few small NT clusters at SupterComputing 99. so yes its possible. but im guessing its gonna take moremachine to get the same results as a linux cluster of the same kind due to all the over head in NT (gui and whatnot).
I imagine 2k could be setup to be similar but why would you want to? again you have much more over head that an base install of linux which can be custom done to allow you put only what you want and need on saving space and resources.
Linux is free do you really think MS is really just gonna give you licences for a cluster? what if it grows more money will have to be forked out for a licence for that mahcine.
linux: cleaner, stabler, lighter. which probaly all makes the cluster faster. not to metnion cheaper
M$ : bigger, costmore, requires better hardware in many cases.
and dont fix it if it aint broken.
Sounds like good detail from experience...
Acquiescence leads to obliteration
According to this article, Macs are simple to use as clusters:
t ml
http://www.wired.com/news/mac/0,2125,50078,00.h
Whamo submitted the story ABOVE this one, not this one...
"The best laid plans of mice and men gang oft agley..." - ROBERT BURNS
So, why in heaven's name would anyone want to run Windows for clustering? Oh, I can think of one reason: Microsoft may give you a big bundle of money, like it did to these people, but other than "buddy, the first one's on me", they just don't have a point, or a product. The only area where Microsoft is somewhat credible is on the desktop; their product isn't great, but it's usable. For server applications, they are simply not competitive.
Seems that everyone is ignoring the fact that Linux or Windows is irrelevant if the hardware isn't that fast. The poster said that the system works, but isn't great because the hardware isn't great. Once you get new, better hardware, THAT will be the time to evaluate windows vs. linux. However, I suspect that the linux solution will still be the winner. Until then, the important thing is to get better hardware.
We use NT 4 EE (Enterprise Edition) clusters, running Microsoft Cluster Services (MSCS) extensively, on primarily IBM x86 hardware. Clusters run Microsoft SQL Server 7 Enterprise Edition, which is "Microsoft Cluster Aware" as well as applications which are NOT cluster aware.
One co-worker has coined the clustering as a "glorified re-boot" because if Node A crashes, the SQL Server there really becomes totally unavailable for a short period of time before it starts up on Node B (which includes moving the IP Address[es], disk-drives, NetBIOS Name, SQL Server, etc. over to the other node). Maybe Win2k is better about having a "hot standby" or seamless failover.
One huge drawback, at least with NT4 EE and SQL Server 7, is updating (service-packing) SQL -- had to totally UNCLUSTER the whole cluster [a timely endeavor] just to get SQL 7 updated to Service-pack 3.
Another drawback is having applications which are "not cluster aware" and sort of "forcing" them into the MSCS cluster environment -- they don't always behave properly in a failover situation, especially if you have applications which normally require lots of user-interaction to startup and shutdown properly. However, this problem with cluster "unaware" applications is probably true with any cluster.
As others have posted, the key is defining what you want from your cluster and seeing how that matches to the definition of "cluster" put forth by the various vendors you're looking at using.
I mean, it's not really an advantage to pay for licenses, throw away a lot of work you've done on the cluster you have, train for a new environment, etc. Other questions are: which applications are supposed to run on those servers, who is supposed to work with them in the production phase, will you develop for the thing and with which tools?
For the MS-person it'd be great if you switch, so he can tick off another customer converted to Windows, but what do you get out of the deal?
--
"By the way if anyone here is in advertising or marketing... kill yourself." -- Bill Hicks
VMS clustering *IS* the best implementation. But Windows clustering is nothing like the VMS version. Microsoft got Dave Cutler to reimplement the core VMS internals, but they failed to hire the cluster and file system people from DEC.
My experience is with Windows NT 4 Server Enterprise Edition. MS chose to use a "shared nothing" implementation - which, IMHO, means they don't do clustering. There is no cluster-wide locking, software runs on one node at a time, there was a limit of two nodes, and it required a shared disk.
Pfffff! I just need a good blowjob.
I worked on a Wolfpack (early version of Microsoft clustering) cluster a couple of years ago. Wolfpack clusters were limited to 2 nodes at the time, working as peers. Last time I looked, MS was currently working with 4 nodes, and hoping to reach 16 eventually. They were architected to handle failover and redundancy issues, but not compute intensive problems. (And early versions were not very robust, but I hear that has gotten better.)
Beowulf clusters run with up to thousands of nodes; for compute intensive problems, that would certainly be the choice I'd make. The cost savings re: licenses and reuse of older hardware are bonus items.
A signature always reveals a man's character - and sometimes even his name. -- Evan Esar
Microsoft Cluster services is designed for one thing: High Availability (little or no down time / load balancing). Beowulf clustering is designed for one thing: Parallel Processing (data analysis / number crunching). They are two different types of clustering. The debate on cost is a waste of time. While Linux is as capable of high availability clustering as Microsoft is, it has little cost. With Microsoft you have to buy a license of Advanced server for each cluster node and then have licenses for each application as well. For cluster aware Microsoft apps that means Enterprise editions. Advanced Server costs in the $4000 range. SQL 2000 Enterprise Edition cost in the range of $11,000 per node. If you are backending a website with a SQL cluster, just for SQL you are looking at around $20,000 per processor . If you are looking for a cluster to be online 24x7 then you go with Microsoft (and pay the additional money for support). If you are looking to predict weather patterns, analyse ocean currents, or predict the lottery, use Red Hat and Beowulf (and pay the additional money for support).
I forgot to add that Microsoft on supports 2 node clustering with Advanced Server and 4 node clustering with Datacenter.
... a "pile"? Oh wait, even single processor Windows machines are often called piles...
to err is human, to forgive is divine, to forget is... umm...
Switching to Microsoft is an example of herd mentality. You can't do that and still have a Beowulf cluster. You'd get a Beosheep cluster.
Give a man a fish and he eats for one day. Teach him how to fish, and though he'll eat for a lifetime, he'll call you a miser for not giving him your fish.
Terrorists can't threaten a country's freedom and democracy. Only lawmakers and voters can do that.
BSODs are an issue for Windows and not Unix / Linux primarily because in Unix video / device drivers don't run in the inner ring of the kernel, and can't bring the whole box to it's knees because of a minor bug in a driver or a hardware failure in (what should be) a seconday I/O device.
Windows NT 3.51 had the video drivers (and most other drivers as well) in the outer ring of the kernel where they couldn't down the whole machine, just certain services. I've seen Win3.51 boxes with horribly buggy video drivers just keep right on running when the video would lock up. NT 4.0 and above aren't the same.
The decision to move the drivers into the inner ring of the kernel is why BSODs are a Windows issue. Blaming the user for not setting up his box just right doesn't solve the real issue, poor OS design.
A real OS (unix/linux/OS390/VMS/even NT3.51) doesn't have these problems.
--- It is not the things we do which we regret the most, but the things which we don't do.
How did they compare?
Ask Mindcraft.
our beowulf team (http://home.cwru.edu/beowulf) used pvm for linux, but pvm runs on everything including the kitchen sink. there are new builds of the win9x/nt versions of pvm out on http://www.csm.ornl.gov/pvm/ . it is a good system, i suggest putting all the doc files in one huge binder. not that bad to work with though...
Imagine a Beowolf Cluster of THESE!!!
The beowulf-specific libraries may be OS-agnostic, but that doesn't mean that all OSes are equal.
How's the networking support? What about system services in general? How much overhead from the OS and mandatoy applications and services? What about libraries for the specific task at hand?
Linux/Unix can be stripped down to almost nothing. But can you kill the GUI of a clustered Windows system?
This isn't a trivial issue - I remember reports of some NT servers that were running near 100% capacity with minimal loads. The problem was traced to a screen saver! Once you know this you can disable screen savers, but what about other unneeded services?
For every complex problem there is an answer that is clear, simple, and wrong. -- H L Mencken
Not trolling... thats a good data point...
as I said, this was my friends experience. Maybe its just the people that he tends to end up getting close to and talking about stuff with (its not the sort of thing a person goes around telling everyone)
Again, right back to personality things... its been my experience that the people a person tends to attract tend to have certain similarities of character and experience.
However, just having known people of variou sbackgrounds, I wouldn't at all be surprized to see a disproportionate number of people with those experience in Rocky Casts (due to its subject matter)
My point was just that people do give off non-verbal clues about their past experiences often, and often people with similar major experiences seem to make similar life
decisions.
So no...not trolling...just relating my experience to the joke someone made about PHBs and battered wives. As one o fthe amotivational posters says "Dysfunction: The common link in all your dysfunctional relationships is you". There is often more truth there than one wants to admit.
-Steve
"I opened my eyes, and everything went dark again"
What? Where? How? I thought that only universities could get "site licenses" (no marginal cost per installation); all commercial organisations have to license each MS product separately (even using Open License or Select License, there's still a per-box cost).
It was almost a direct quote. His statement was that the adult children of alcoholics tend to have certain control issues.
He even related a story where his boss's son came in to work as an intern and one day he finnally got curious enough to ask and went to the son and said "Your grandfather wasn't by any chance an alcoholic", to which the kid answered yes.
Well the deep need to help others would be a reason to go into psychiatric work. Its many years of school and training.
He didn't go much into what these issues are, and I don't have much (knowing) experience with the adult children of alcoholics. However, I do know several people with other traumatic childhood backgrounds and they do, as a group, seem to have certain peculiarities that are hard to really pinpoint and list, but are definitly there well enough that I have gotten that intuitive twinge about a person weeks before they actually told me about their past.
So, what he meant by "control issues" im not sure... but management is an interesting style of job with very different stresses and requirements than that of the people being managed, so I would not doubt that there is a high correlation (if not a 1:1) with certain types of experiences and going into management.
(id like to see it correlated with management styles too)
-Steve
"I opened my eyes, and everything went dark again"
Question:
What would it take for you to switch to Microsoft clusters?
Thats easy
Answer:
Free liscensing for all the machines
Salesmen are too much.
/dTd
It sounds like you are talking about PVM (parallel virtual machine) which beowulf does and Windows DOES NOT out of the box. Now clustering used in it's correct term usually refers to load balancing applications which does not do PVM. I know there is a project out there that will allow you to do PVM tasks on a set of Windows servers but I would stay away from it. The MS rep is extremely uneducated (which most MS people are) about the uses that you are getting out of the Beowulf "cluster". There is no question here which to choose since between MS and Beowulf only one actually does what you want it to do.
We have two vendor imposed windows 2000 advanced, professional, super-duper, mumbo-jumbo systems, each of which was installed by a dell / microsoft engineering team. The rest of our env is Linux / Netware / Solaris / AiX.
Only a few weeks later, the clustering software was so badly broken and tangled up, that we had to separate the nodes, and remove the clustering software and hardware. This occurred for both of them. Nobody could fix it.
The good news, is that I got two new Linux / Oracle database servers out of it.
Thanks Microsoft !!!
- Fargo
I don't know or care how capable windows is at computational clustering. What I do care about is that windows is designed for several things that you just don't need in a clustering environment. It's designed to be compatible with DOS and older versions of windows - well, you don't need that for custom computational work. It's designed to be user-friendly - but in this sort of project, everyone is going to be technically inclined anyway, so a pretty GUI isn't that important.
Those are the main selling points of windows. But they are *selling* points - you pay for the privelege of using these features. I'm not an anti-microsoft fanatic, but why pay for windows here when Linux or several other free OSes would probably do just as well, with less overhead and cost?
I'm the stranger...posting to
M$ clusters? Haha! You gotta be kidding man!
Microsoft can FUD anything into reality ... but ask them to give you the names of three other contacts at universities that have successfully implemented a Microsoft clustering solution who are willing to talk to you about this.
... hopefully you can get more than one person at a given university and cross-check their stories.
If someone else has 'successfully' implemented one, you can find out for how large a cluster, what classes of problems it's solving, and how long it stays up (oh, and how many times they've had someone break in). Presumably administrators of an existing installation at a university won't egregiously lie to you
Then find three success stories at universities of Beowulf clusters and similar information for them. Side-by-siding these for your management should make the point verifiably clear. Ask the Microsoft sales rep for these contacts (don't settle for grossly postprocessed 'success stories') and maybe he'll disappear.
I think the point must be made to management that there is no better proof of concept than a working implementation that matches or comes close to your needs.
Design and Analysis of NT and Linux Superclusters for Computational Grids http://archive.ncsa.uiuc.edu/General/CC/ntcluster/ sc99/tutorial/index.htm
Tell your Microsoft rep, "Oh, I dunno.. Microsoft Linux 2002 has a nice ring to it."
AFAIK, Win32 has a two node limit, but i've heard rumors of 4 and even 8 node support floating around in the form of CA releases. It wouldnt be too wild an idea to see Win32 support 4 nodes in the not too distant future.
Windows is the runt of the clustering litter. Even Novell wipes the floor with Win32 in that arena. The hands down winner is still Beowulf.. you cant beat the flexibility, low-level control, support, documentation, and bang-per-buck ratio that PVM offers.
Cheers,
Bowie J. Poag
Having seen first hand how poorly the following setup ran, I'd say steer clear of Microsoft until they admit that reboots are not normal:
2 x HP Netservers, both dual p2 Xeon, 1gb ram, and a small raid shelf with 8x 9gb disks. Both NT4 installs with the correct patchlevels.
One machine ran oracle, the other IIS, these were clustered so that one would take over the task of the other, should there be a problem.
Problems:
1) Crashing (daily at least)
2) Slow (astonishingly poor, disk defrags once a week helped this)
3) Sometimes one host would freeze, and the other wouldn't actually notice
4) Often a shutdown of one node would move the services across, but upon rejoining the cluster - the node with both services would refuse to give one back.
5) Often, IIS would stop talking, and neither node would actually realise.
The attempted solutions:
1) Replaced CPUs, memory, disks, eventually nodes
2) Reinstalled clustering software, eventually total clean installs of operating system and applications
3) Support from Microsoft, and Oracle, and HP who made the (certified) kit. Oracle+HP both pointed the finger at the OS, Microsoft simply failed to help, when we got any response from them at all.
4) (this helped) I used one of the spare HP9000 servers to monitor them remotely by trying test transactions - it alerted people when they fucked up.
I think the above says it all really. Standard software on correct hardware - it just didn't work properly. Microsoft can stick their clustering "technologies" where the sun don't shine.
--
ALL YOUR BASE ARE BELONG TO US!
I can say that, I went to SIU
My beliefs do not require that you agree with them.
We ran four clusters: two file server and two database. The fileservers were using Microsoft Clusters and the database clusters were using Digital Clusters all of it running on Compaq hardware.
The Digital Clusters were stable enough, but they continually dropped the ball on automatic failover, load balanacing was all done manually. The fileservers were supposed to auto load balance and failover, and did neither. We continually had to reapply permissions and recreate shares on the file servers whenever we would manually failover drives, and BSOD's were an almost daily fact of life.
I would hope that things would be better under Win2k, but the technology under NT was just not there at all.
Wu-Tang Name: Half-Cut Skeleton Get your own Wu-Na
Yes you can build a Windows Beowulf cluster. They work very well. Thomas Sterling just came out with a book describing how: "Beowulf Cluster Computing with Windows" Its a Scientific and Engineering Computing Series book from MIT Press.
In my experience, the 'natural' way for such conversations to open is cryptic comments that only make sense to someone who has suffered a similar abuse. If there's a positive response the conversation can quickly open up.
My guess is that once your friend started the conversation it made it a lot easier for more and more people on the cast to share (or for that matter -- even remember!) their own abuse experiences.
Given that the person with the negative response hadn't had such an experience, it's not surprising that other cast members haven't spuriously mentioned it to him. -- and unless someone mentions something like that to you, it's unlikely that you're going to figure it out.
One of my sisters supressed memory of abuse that she had experienced as a child until she was almost 30, and I'd probably never have known that a girlfriend of mine and her best friend had been abused if they hadn't started talking about fear of spiders which led to them discovering each other's abuse (and they'd been friends for years).
Like said -- It's not the kind of thing that comes into the conversation space without a good trigger, so lack of any comment is quite a bit different than an explicitly negative responce.
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
I'd like to see you install Windows 2000 on a bunch of old machines... especially Win2k Server. I tried to install it on a pentium 133 and had nothing, but mess.
it's a sig, wtf?
In addition, each server in a Microsoft cluster can be active or passive. Active means it shares in the load (depending on how you configure the distribution of tasks), while passive mode means the server sits there waiting for the active node to fail.
So a M$ cluster is not meant for massive parallel processing operations.
As for Beowolf clusters, I was always under the impression that these were specifically designed for highly scaleable parallel processing (although I've never actually seen or played with a Beowolf cluster).
I can no longer read Dilbert. It's too depressing, because it is too real. -- Hyperhaplo
7-8 months ago I purchased two boxes with Win2K advanced super ultra server + MS Unix Services (basically I got the OS for free) and just out of curiosity wanted to see how would it work with the other 50+ Linux servers I have. I wanted to know if it could handle serving NFS to Linux clients etc. I swear it blue-screened on me after 5 minutes during setting up network interfaces. Win2K lived another 5 minutes on those machines: that's how long it took me to find RedHat CDs.
I don't know how would Windows cluster do compared to Linux cluster as far as the performance goes, but I can guarantee you that it will never be as reliable. Another things is (more of a question actually) is can you do some neat stuff like channel bonding etc. on Windows clusters. If you guys as CS students want to learn something about how modern OSs and clusters work, I don't think MS-way is a good idea.
Enough material about MS clusters to brag about.
Never underestimate the relief of true separation of Religion and State.
Set up 2 parallel clusters. Use the donated crap... er... donated systems and see how well Win2K runs a cluster on Pentium 200's with 64Megs of RAM. Submit the results to the MSRep, and ask him to explain. Make him squirm.....
Feed the need: Digitaladdiction.net
Let me guess. They both paid a lot of money to a "therapist" who makes a pretty penny keeping them as perpetual "victims" who need "help". The scientific evidence for the accurate recovery of so-called "repressed" memories is nonexistent.
Free, forever. Plus support and upgrades. Then I'll switch. I've used this aproach to chase off many a M$ sales droid while working for poor, underfunded public edumakashun institutes.
--
You sure got a purty mouth...
Yes, I dunno about universities, but MS doesn't sell "site licenses;" who ever told you that either a) doesn't know what he's talking about or b) blindly parroting the line of someone else who desn't know what he's talking about.
The MS rep should have gotten laughed out of the room. "At what cost? At what min. HW requirement? To what extent can we modify the source?" How would have left about 1 cm tall.
At the bottom of a deep pit. :-)
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
You are looking in the wrong place.
Win2K cluster is for fail-over.
WLBS (network load balancing) is for performance.
I know that on other OSes those two features are in a clustering product, but MS didn't have time to merge these products before Win2K shipped. Perhaps in the next rev. In the mean time, use WLBS (ships with Win2K Advanced Server).
Look for the load balancing support under networking/properties+Local Area Connection/Properties. Look for Network Load Balancing.
Man, we were doing so well, an article on /. about Beowulf clusters and no
beowulf cluster jokes. Then you came along!
"Windows Cluster"...seems like there's a word missing...
Ahh.. in a perfect world...
Dear Mr. Microsoft Guy:
I will gladly replace all Linux cluster servers with Windows under the following conditions:
1. Release the source code for the Windows operating system to me and all of my colleagues so that we may have a guarantee that bugs or shortcomings can be fixed and verified by us if we encounter them.
2. Allow me to redistribute any and all changes that I make to the source code of Windows to all of my colleagues so that they will not have to duplicate my efforts. Likewise, allow them to give their patches or revisions to me.
3. Allow me to install the operating system on any number of servers in the cluster, with no incremental cost, so that I may scale my solution up or down based on my needs without worrying about the cost of the software.
Of course it's possible. It's not *nice*, but it works.
... ;) It's not fast, but since all it does is pass packets through (cable modem) and block incoming packets it doesn't need to be.
For the record, I maintain a headless NT 4.0 web/database server at work for one of my projects (requires disabling the mouse driver to avoid error messages at startup) controlled via PC Anywhere and a headless Win98SE machine at home as my internet gateway (running SyGate NAT 3.0 and SyGate Personal Firewall) controlled via VNC.
Why NT 4.0? Mandated at the time (the main servers are in the US maintained by an external group - we're in Australia with the admin server for the same system).
Why Win98SE? I tried various linux and bsd distributions on the machine, and couldn't get any to work - Pentium 60MHz, SCSI, plain IDE (not ATAPI) so had to install an I/O card to get a CD-ROM to work, old intel ethernet cards, etc. I've configured it to reboot every night so I don't have stability problems
I was actually involved in presales on some hardware for some clustering and which eventually ended up running on UNIX but Microsoft was involved in the early meetings. They sent along the usual contingent of sell-em-anything sales droids but there was an honest engineer with the group who said:
"It's not really accurate to refer to this arrangement of servers as a wolf "pack". A pack is an organized group with a leader working towards a defined goal using a plan with a visible, known structure. These servers just sort of hang out together. It would be more accurate to call them "wolf buddies"."
Microsoft didn't get to put their software on the solution but I did tend to put more credence in what that particular engineer would tell me about the capablities of Microsoft products.
I can't wait till they make the switch to McDNS so the bloody site will spare me the hassle of resolving at all.
From the coments I've read, most of you seem to think that you will be competing against Beowulf with Win2K. This is not that case - you need to be running DataCenter in order to have more than 2 hosts in the cluster. OK, say you want only 2 hosts in the cluster... you still need to have Win2K Advanced Server (cha-ching!)... and THEN, not all apps will run active-active, some are still bound to active-passive rules. Add to that, your CALs and whatever commercial software you run on it... They don't miss a beat when it comes to licensing. Also, Satan will be stocking up on thermals when DataCenter or Advanced Server is capable of running on lower-end hardware that Linux is capable of. Bruce
That link you gave does talk about IBM making it easier to build Linux clusters. Unfortunately for your argument, it's making them out of many Intel-based boxes, not one of their bigger machines with virtual Linux servers.
Looks like it probably is a stupid idea to build a Beowulf cluster out of lots of virtual Linux servers.
The problem as I see it is that is a cluster you want to squeeze every cycle out of the chips for processing a large problem. You can create a 'cluster' of workstations to process a common problem by having each work on a sepaerate piece of the dataset (Seti at home, renderfarms etc) but that's not we're really about. The aim of beowulf and similar projects is to combine a number of distinct units in such a way so that they attack a problem as a single unit.
Yes you can do it on MS. No you would want to. Why?
1. You can't shave away the parts of the OS that are stealling cycles. Who really cares if the thing can allow you to click on icons or check email while it's processing.
2. You cant rebuild the kernel to include or exclude features. A number of Beowulf type sites rebuild the kernel to include interprocess communication channels to syn the nodes.
3. You're running the system as a single unit through a single user controller, so why the hell would you want to license each node....
4. Doing it through Linux would piss off MS
Personally, reason 4 is the main seller for me:>
There are more important things than Windows vs. Linux.
/.) a couple of weeks ago reporting that on a single machine, numerically intense code ran about 10% faster on Windows XP than on Linux (compiled with the Intel compiler in both cases).
It's not relevant to clustering, do but recall the story in Open Magazine (also discused on
This speedup is probably irrelevant next to questions of efficient distribution of processing power, but is worth knowing about.
More significant is the finding in the same story that numerically intense code compiled with Intel's compiler ran 47% faster than when compiled with gcc, so you should be looking at getting your hands on some funding to buy the Intel compiler rather than worrying about MS and Windows.
Better compiler optimizations could be worth the price of many nodes in your cluster!
Can Microsoft break the $1,000 per GFLOP barrier? Linux can, and has. See this link: http://aggregate.org/KLAT2/press.html
Looks like the Redmond engineers could really learn something from those folks in Kentucky.
MSCS allows for application failover. MS bought WLBS/NLBS and released it (formerly was called convoy cluster server) to allow for LOAD BALANCED TCP/IP APPLICATIONS (web, proxy, DNS, etc). Beoulf as we know, is usually for massive computation.
"You never truly understand a thing until you can explain it to your grandmother" -Albert Einstein
We have a windows 2k advanced server cluster here at the office. Since I installed it I feel safe to say that it doesn't do anything other than load balancing, and high availablity on the machines.
As it stands your current setup appears to be the best setup.
Gee -Hmmm...
;-)
I know - here, there you now have a site license to run Linux on any machine you want.
That is a pretty lame reason to only install windows, especially when it simply ISN'T the right tool for the job!
Have you compiled your kernel today??
No really apples. Use some Macs.
http://www.dell.com/us/en/biz/topics/power_ps3q00- beowulf.htm
0 1s e-cluster.htm
p 50 0.asp
a re /dellcs.htm
http://www.dell.com/us/en/biz/topics/power_ps4q
http://www.tc.cornell.edu/news/releases/2001/to
http://www.intel.com/eBusiness/affiliates/hardw
-cupid
For a related articles on academia and Windows Clustering go to ftp://catalogtool:catalog@ftp.us.dell.com/app/5q01 -hpc.pdf
or go to
http://www.tc.cornell.edu/ac3/tech/CTChardw.pdf
I think people should mod this reply up.
Quite clearly with so many 1 G cpus it is a rather diapppointing performer. Concidering IBM built two other clusters using 256 1 G cpus and it's about a 66 places higher putting out 66 GFlops more.
That's doesn't make the windows cluster look too hot. Maybe it's cause dell's not used to doing this kind of work. Or maybe the GUI does end up sucking quite a bit of resource.
Really, clustering means so many different things that a question like this just can't be answered.
Even companies like Sun that talk about their 'Clustering technoloy'... are really talking about application specific solutions.
Beowulf? What, you mean using PVM? Windows could do as well as linux, why on earth couldn't it?
Mosix type clusters?
Web clusters that are more of a farm?
It's a silly question.
What's an M$ site licence worth for that site? Ten grand? A hundred grand? That'd almost make it worthwhile suffering through ISO-9002 certification and all of the other bullshit necessary before selling to gummint... we'd sell a Linux site licence for half price and throw in site licences for OpenOffice, AbiWord, SIAG, Apache, Zope, PostgreSQL, MySQL, GIMP, PERL and Python site licences too. (-:
Got time? Spend some of it coding or testing
Getting past what are the wrong tools first: Beowulf is an architecture to do massively parallel computation, so we can eliminate two of the best known HA tools. Microsoft Cluster Service is two or four node high availability, similar to HA Linux's efforts. NLBS is a software form of a hardware load balancer, similar to Cisco Local Directors and only really good for web farms. So what does MS provide to do similar stuff as Beowulf?
COM+ and Queueing Components. AppCenter.
The way it works is this. You write a COM+ component that is transactionally queuing aware. Each component takes a work unit in, processes it, and then sends the result of the transaction to the queueing components for reassembly or re-issue (if a node fails to submit a result, for example, good for checkpointing).
You can use normal Windows 2000 Professional boxes for the worker bees, and use a few Windows 2000 Server boxes to co-ordinate the issuing of jobs and control, and munging the result sets coming back in.
If you need to submit a wide variety of jobs, obviously the COM+ components will be changing regularly, it'd be a good idea to go to AppCenter so that you can treat a bunch of machines as single whole. This allows you to upgrade or deploy an app in a few mouse clicks to literally thousands of machines in a few seconds. AppCenter also has pretty good resource management, something that might be necessary if multiple jobs are running at the same time.
The cool thing is the development environment is really friendly and you can make COM+ components pretty easily and test them locally (for the n=1 case) before deploying to the farm.
There are also specialist MP libraries for the Win32 platform, such as PVM or MPI (WMPI). These have the benefits of re-using the knowledge and API's that users might already be familiar with - one of the biggest thing when a place converts from one supercomputer to another is rejigging and reoptimizing the code for the new architecture.
Andrew van der Stock
Can you imagine a Beowolf cluster of these Beowolf clus...
Oh, never mind.
D'ya think this might be a response to Samba's CAL server ability?
BTW, don't knock savings on CALs. Get 1000 students onto Exchange and you're staring down the throat of at least $USD30,000 in CALs alone. Get 1000 students into PostFix and you're staring down the throat of, well, no licencing costs at all. And you have capacity on the same hardware to serve roughly four times as many mailboxes...
Got time? Spend some of it coding or testing
Actually, it does loosen the screw. The rellies on the farm use a hammer quite effectively as a screwdriver (both ways) and spanner where appropriate. It just doesn't do the screw in question much good...
I've used a hammer myself to gut a dead hard drive for the magnets, when I didn't have a small enough star driver. I just flattened the top of the bolt out to tinfoil thickness and pulled it straight through the metal cover. The technique with screws is different, some light taps can loosen them in their substrate (typically wood or sheetmetal) enough to winkle out by hand. Using somer CRC/WD40 often helps as well.
Got time? Spend some of it coding or testing
One could say that of my sister's case, but with my girlfriend and her best friend, it was a private conversation on the beach between the two of them. I just happened to be around when the conversation took place.
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
You're talking apples to giraffe comparisons here.
Beowulf clusters are primarily distributed computing. Microsoft clusters are for fail-over/high availability. closer analogy would be Unix HA clusters (which actually have the same friggin' limitations as the MS implementation).
synchronised BSOD! I Like It.
Windows 2000 clusters have some significant basic problems, totally discarding the fact that 2K clusters are more for fault tolerance than high computation:
1) RAM. Windows, to start up, will eat 64 megs of the RAM that you want to use for your process. Without a head and no X, you could get Linux down to 4.
2) Disk. Windows boot from a remote FS? Unlikely. To do so you'd need to get network cards that support booting from a network. Its also possible to have one app drive and alot of little drives... but to get Win2K on those, you're going to have to get them about better than about 300m in size. On Linux, you could install the basic stuff on 40 megs. Remember all it has to do is get instructions from your head computer. Or another option: Linux on a floppy, booting / from NFS. No HD needed. Just RAM, CPU, floppy drive, power, and something to put it in. That makes the reboot slower but so what? Whens the node going to go down? Which brings us to...
3) Stability. Lets assume for a moment all these computers arent the same kind and dont have heads. How long, even, till you even know a head has crashed? Linux offers stability that my servers never crash, their uptime is only limited by my want to upgrade them.
4) Price. Even if you are a school, if you get it for free, whats a salesman doing there anyway? I am sure that SOMEONE there is paying for those MS products. Linux? $0. Simple math:
Nodes - Linux cost - Windows cost
1 - $0 - $150
10 - $0 - $1500
100 - $0 - $15000
Overall, I would have to say, the whole concept of a Windows Cluster is just Not There.
I thought the most interesting quote on the Microsoft HPC cluster pages was that they bundle an Open Source application with Windows 2000 Server in the preview kit. Not sure if it is GNU but sounds like it is good enough for MS to use it Cheers VikingBrad
to put you into this shiny new Yugo this afternoon?
Microsoft is only interested in making money.
Opensource projects are only interested is in getting the job done.
Which do you want to go with?
Event Management Solutions : http://www.stonekeep.com/
It's up to the vendor to prove that it works and that it will be good for you. If they can't show you anything better than what you already have then ignore them, even if they are Microsoft. If they can show you something better, then look for holes and unanticipated costs. Also look at the companies with experience at that end of the market, and not just a big PC company starting to ease into the top end (which is really what they are - hugely successful, but a PC company). It may be a good solution, just get them to prove it.
The first thing that sprang to mind when I read this was, "Microsoft cannot compete, since Windows depends on the GUI."
Remember that one of Microsofts contentions in the anti-trust trial is that they cannot unbundle Internet Explorer from Windows, that the system is so interdependent that no elements can be left out and still function.
So they cannot compete on price, since all other things being equal a Windows machine must have a video graphics card.
They cannot compete on performance, since all other things benig equal a Windows machine must spend resources on storing and running the GUI.
Yesterday, I was showing a very happy WindowsXP owner (who also happens to be a somewhat savvy computer consultant with Unix and Linux experience) the beauty of Debian's apt and dselect packages. He was so happy with the granularity of not installing anything that he doesn't want, that I gave him my Debian 2.2r4 CD. (I'm running Woody anyway)
Bob-
The Ludwig von Mises Institute. The reasoning individuals economics
Microsoft changed the meaning of the word clustering. Much cheaper to do this than to actually offer a competitive product. Gates he speak with forked tongue.
An answer already lies on TPC, their top three performing machines are Windows clusters, because Windows 2000 and SQL Server 2000 cluster very well together.
Top Ten TPC-C by Performance
..after the posting about the MOSIX fork last week, I got off my ass and set up a small cluster at home. I took my old dual PII400 and a IBM Thinkpad (PIII, 750Mhz). I actually put another thinkpad in the cluster as well, but since it was running on a wireless link (Aironet) it was more than useless :-)
:-)
Setting up mosix required little more than a kernel patch (a good time to upgrade my boxes to 2.4.17 anyway), apt-get install mosix mosixview, and a couple of small tweaks. I've got Mosixview running over SSH, and with no tweaking at all, I was able to shave a minute off a 2.4.17 kernel build, launched from the 2XPII400.
I was just trying to pound the boxes, so I did builds with make -j 16 or so. I think I need to tweak it somewhat (I was messing with the box speeds throughout the session), as I expected more than just 1 minute off a 6 minute compile, and I could tell I wasn't using the laptop fully as it's CPU wasn't 100% all the time. I suspect this type of compile isn't the ideal test anyway, as it's alot of small files that are done quickly-- the time it takes to TX them over the ethernet would (IMHO) sometimes not be "worth" it given the size of the file. Since the ethernet wasn't maxed all the time either, I think I still have a way to go. I want my boxes to SWEAT when I tell them too!!
I'm the kind of guy who hates his CPU's to ever be less than 90% pegged each
-- I speak only for myself.
I can't speak to the quality of a Linux Beowulf cluster. I did, however, recently setup and install an MS Win2K two server cluster for a large manufacturer.
The hardware:The Software:
The purpose for using a clustered setup was for failover assurance. The active cluster controlled the manufacturing process, and if the cluster went down, the whole plant went down. We settled on a two node cluster. We later discovered that two nodes is the maximum that microsoft "recommends" (read supports). The cluster was to act as the gateway between the plant IT network, and the device network of PLCs for the machinery. The cluster would serve progress reports to the IT network, while controlling and collecting data from the device network. In the case of a failure on one node, the second node was to pick up the load.
After our initial research and development on the project, we became seriously disenchanted with MS clustering. No synchronization, no active replication, no load balancing.... about all it could do is fail gracefully from one node to the other. The three NICs on each node were a real nightmare. First NIC was a heartbeat. Second NIC to the plant IT network. Third NIC to the plant device network. We routinely encountered routing table loops through the heartbeat NIC, causing VERY long delays in connections.
The RAID acted as a quorum drive between the two nodes. The active node had control of the quorum, and the MSDE database was installed there. When a failover occurred, the last transactions committed were in the database. Unfortunately, any data from a transaction in progress was completely lost. The nodes often got confused over who owned the quorum, sometimes both fighting for it, other times both dropping it.
Fortunately, this system is only a prototype, and never controlled anything more important than a machine simulator and an isolated test network. I would highly discourage anyone considering using MS clustering. If you are hosting a web site, you may get some use from it, but if you are looking to perform distributed transaction processing, constant uptime, and at the same time provide a robust system, avoid MS clustering.
As others have pointed out it is *possible* to run a Windows based cluster. It isn't *practical* unless you are willing to put a lot of mony into very high end hardware and support.
The basic problem is system reliability. With commodity hardware, my experience is that a windows boxes have an uptime of less than 20 days. Thats translates in to a single machine crashing with probabilty 1/20 in any day (probably too low). I've successfully run linux clusters where the uptime was only limited by my propensity for upgrading kernels, etc. Given the probability of crashing on a given day to be, say, 1/200 (probably too high.)
Now, remember that most parallel programs will die if any of the nodes crash. Then do the computation:
P(No Nodes Crash during 1 day)= 1 - P(Any Node Crashes) = 1 - [1 - P(one node crash)] ^(# Nodes)
Play with the numbers and you'll see why no-one can afford to run really big Windows clustesr. For even a 20-node cluster there is 64% probability of a crashes every _DAY_. For a 20 node Linux cluster, that number is only 10%. It gets worse the more nodes you have.
I can't seem to get past the idea that a research project using Microsoft software would only be a marketing tool for Microsoft. I'm no programmer, nor am I a CS student, so I don't know... But I would expect that the interesting questions would require digging a little deeper than you can get with binary only code. And of course, if you get access to the source, do you get to distribute your networking optimizations or whatever?
Use free software so you can at least own your research.
MS Cluster - ACK!!! Your network person is going to wack you upside the head when they learn what you have done to them!
MS cluster has the feel of a VAX cluster. Multiple systems handling client requests load balancing between them selves.
The way a MS cluster alternates client request among themselves is to share the same MAC address among the units in the cluster. If you were on a shared media type of Ethernet, the incoming packet will have the MAC address of the receiving server. All servers can see the packet and one of them will process it. Well it seems to work or so it seems
Now for the "bad" part of it. Some one up there did not really understand networking or decide to make his own rules and ignore common sense. Put a network switch into the mess. A switch will learn which port a system is one by listening to the sending station MAC address. If multiple systems are using the same MAC address, which port will a new client request go out on? The last server that sent out data.
Himm.. Not working well here. Lets call MS and find out what we can do to make this work. There answer? Turn off learning for the switch.
Well, to quote my clustering prof:
Clustering apps and toolkits for all brands of *nix are multifluous and are in good use today.
Clustering projects for windows have fail... er, declared success and moved on.
Enough said?
I have used and supported WLBS/NLB on Windows NT 4.0/2000 for the last 3 years in Fortune 250 and 500 companies. I have used it on switches and hubs with many machines and a few.
WLBS works very well if configured properly. I will agree that some of the docs are confusing until you work with the technology for a while, but I can say the same for many man docs as well. WLBS doesn't cause reboots, typically the reboots stem from instabilities in IIS 4.0 (many of which are corrected in IIS 5.0), but many required reboots are caused by shoddy in-house developed code.
WLBS does do an interesting Layer 2 slight-of-hand, but this novel approach makes it very efficient from a server perspective, and on a properly configured switch, the network cost is minimal. WLBS allows scaling out with insignificant diminishing returns, which makes it very cost effective, and no additional hardware is required, especially hardware that would need to be made redundant itself.
As an additional item to many others who have posted here, you cannot flatly say that a Linux solution is cheaper that Microsoft. This is not just a rehash of the TCO arguments, but if your company has a large developer base that scripts for Active Server Pages, or a great application that is making your company serious cash, but uses COM+, Linux is not appropriate. There is no definitive right or wrong here. The point of commerce it to make money, and if you do that with Microsoft's technologies, then the business needs drive that decision.
We have 2 MS clusters for redudancy and we can't manage to keep them running, they are a turn key Dell solution. We two weeks ago Dell put a guy on a plane from Austin to Indianapolis with 2 hours notice to try to get this thing running. Thank god I am a network guy because our server guys have been spending several DAYS at work with VERY short sleep breaks. Our experience has been if you have something important don't use MS software.
The last time I looked at the Top 500 Supercomputers of the world list, MS was nowhere to be found. In the top 30 there were at least 2 Linux clusters.
We had a MS sales rep in our office and I asked him about this, his reply was that Windows does not scale to those levels.
a beowulf cluster has three main components, not including the application running on it.
1. A big pile of commodity computing hardware and a fast network interconnecting it.
2. An implementation of distributed computing API that handles things like message passing between nodes.
3. Utilities to manage the cluster as if it were a single machine (like starting a copy of the application on each node, etc)
Looking over this list, windows can run on the same hardware as linux.
There are windows implementations of the Parallel Processing APIs typically used by applications on linux beowulf clusters. Are they as efficient? I don't know. Maybe someone else does.
That leaves 3. The tools for managing the clustered application as if it were running on a single machine. I don't know if these exist for windows now, but I see know reason such tools couldn't be created with relative ease. Windows already has extensive infrastructure for remote monitoring and management.
I wonder what Google's TOC is. I realize that it is not an HPC cluster in the strictest of sense, but with the sheer number of systems they have (>8000 last I heard...) the TOC must be orders of magnitude smaller than if they did the same thing on Windows.
:-)
And when it comes to performance, in an HPC application I am reminded of what my professor used to say about MACs. "They would be really great computers if they didn't spend all their CPU time drawing trash cans." Why run a graphical OS for a pure HPC application? Seems you would have to have more hardware (nodes etc) to get to the same level of processing power since the OS has to spend some of its time keeping the Graphics bits going, not to mention the 3D screen savers - oh yeah and PC Anywhere for node management
I've worked for california based start up, and we've always been using MS servers. At the time when I was involved with this, we were clustering Windows 2000 adv. server betas, and from what I recall it was a pain in the ass until final came out.. From then on we haven't had any problems. It isn't that complicated.
A beowulf cluster of blue screens?
J
--- Worst tagline ever.
Asking slashdot about Windows Clustering is like asking the Pope about the benefits of Satanic worshipping...
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Try to open a UDP port on a Windows 2000 server, keep it open for 30 days, and see what happens.
Lone solution; reboot
see: http://shop.barnesandnoble.com/booksearch/isbnInqu iry.asp?isbn=0262692759
:-)
Interesting. I thought Beowulf meant Linux. How is it that this stuff has been going on for years and no one noticed? No-marketing, the latest in anti-competitive techniques. Are the end-users and researchers bypassing the datacenters and approaching MS directly (again)? We can't let that happen. Don't our users know they can't live without us, the Unix priesthood? We can't tolerate empowered users, especially if they decide to spend their grant money directly and not rent cycles from the computing center and not let us decide what is best for them (while taxing them severely for the privilege).
This stuff can't be made as simple to use as a desktop with commercial support and licensing fees less than the cost of an admin or else we'll be out of a job and have to do something more important to our school or business than hack on the plumbing (while looking like priests at the altar). Time to join the AFL-CIO! I feel a job action coming on.. a sit-down strike in the dean's office
/Ari
a whole cluster of Windoze boxen is analogous to a hitch for beetles ... sure it multiplies the power of the insect, but does nothing worthwhile. Try it.
Now that Carmen Electra is whoring out to anybody there's a new clause in my contract...
;O)
OOHH BABY! Oh yeah, plus the source code, yadda yadda yadda...
Whoa!! Could you imagine a Beowulf cluster of Beowulf cluster jokes?!! Man, that would RULE!
pr0n - keeping monitor glass spotless since 1981.
1. "Windows clustering" AFAIK, only includes load-balancing and some transactions processing, designed for web servers primarily. No one uses it though because application-specific designs are usually better. Cluster management also exists, but I don't think, it's helpful in general.
2. There probably won't be much trouble to put MPI or PVM on a bunch of Windows boxes, and run something clustered. It's however unlikely that this thing will be easy to manage, and certainly the performance is likely to suck because Windows networking was "optimized" in various ways that make sense in a, say, web or mail server, but offer no help otherwise.
3. Windows has no equivalent of MOSIX or other advanced clustering architectures, and messiness of its design doesn't give much hope for it.
Contrary to the popular belief, there indeed is no God.
The guy should just ask the MS rep to show a working
Clustering setup on similar machines, and tell him
if you are satisfied with the setup and all
features that you want are provided and they are
willing to match the cost then you will replace.
They must also match the serviceability. If the
source is not available then they have to come
and correct the problem, as long as the system is
in use.
The rep may not be able to show a working setup,
since these machines are outdated. Win2K may not
even run on these machines. Improving anything due
to clustering may be a very tall order. Then they
have to match the features, that would also be
difficult. Ultimately matching the cost would be
the most difficult job. And giving guarantees is
in the realm of impossible.
What the Windows machines drop on the floor is nothing compared to what they fling at the fan.
Leave the software mafia out of the argument. It is easier for an auditor to audit a machine with recognizable software than for an auditor to prove that there is no such software in a machine.
Boot up a machine, see a known software package is there, get its license info and confirm it. Easy.
Boot up a machine and not see any software on "The List". Have to search a lot to find something that is not there. You can't prove a negative.
Allow me to one-up myself, then:
ALL YOUR BEOWULF ARE BELONG TO CLUSTER!
pr0n - keeping monitor glass spotless since 1981.
Maybe finger was the trial version of Exchange? That would explain why the code name for the full version got censored...
Got time? Spend some of it coding or testing
1) Windows clustering is working well (and NO Beowulf code is stolen!!!)
2) Microsoft is a fantastic company that a) makes life easier wherever you look, b) makes you more effective and c) generally has a high coolness factor
3)Linux is a patchy soup, while Windows is serviced and packed
I wish you all could understand this.
Anyway, from now on it's "dotNET or haven't got it YET?", and all you linuxationists and wanna-rule-the-worlds will feel like being on the wrong planet.
As you are, lightyears away!
Fuck off with your shitty moderation, bitch. You moderators are just plain fuckwits. Stupid. Stupid. Stupid.
i hate pansy republicans
So far, No one seems to mention one of the more important aspects of the argument. Which operating system is more prone to a virus? And if one is more prone to a virus, wouldn't that require a reinstall for almost every node of the cluster?