Slashdot Mirror

• http://www.opensymphony.com/quartz/
• http://www.clusterresources.com/products/torque/
• http://www.OpenPBS.org/
• http://saturn.sourceforge.net/

There are many other open source cluster/queuing systems available.

The one I prefer is OpenPBS. It works very well for engineering compute clusters, and there are many different resource schedulers available which use the PBS job and node management system.

This is not a supercomputer. It is just a collection of individual laboratories and universities running plain old batch systems on Beowulf (or beowulk-like) clusters. To transfer jobs and data between sites they're using Globus. It's really not interesting stuff, certainly not newsworthy. This Grid is held together with duct tape and spit. The Globus software is woefully immature (Expect complete rewrites years. Expect refusal to fix crippling bugs because the next complete rewrite due out next year might fix it.) The Globus software is only a partial solution, it really only cares about moving data and jobs between sites. Anything more complex like automatically assigning jobs to the least busy site, distributed management of user accounts, or even ensuring that jobs eventually finish requires piling more software on to. This additional software is also constantly changing. The result is a system that is at best frustrating to install and configure, assuming someone else has assembled a known working package. If you're doing it by hand, expect days of effort. When things go wrong the system is a nightmare to debug as the many, many layers of software don't propogate errors (that is, when the error is logged at all, sometimes they'll just silently fail).

Since the article doesn't really have to do with grid computing. Here are some real Grid Computing links.

Globus Toolkit
LSF
openPBS
gridengine
OSCAR
ROCK MPP
maui

and last but not least: beowulf cluster
---

Pat Moran isn't an intern. I know, because I used to work at that very NASA site. It's been a while, but I think he works at the NAS division of NASA. (Here's a profile of the division.) I probably bumped into him at a few meetings, since the the NAS building was across the parking lot from my building.

Anyway, not only does NASA's Ames Research Center use a bunch of open-source software, but they've been using it for a long time. When I got there in 1994, everything in my building that wasn't a Mac was all Unix stuff, with gcc, emacs, X11, etc., etc. (Well, except for a few old Symbolics LISP machines, and a MicroVAX that one guy couldn't bear to get rid of.) Mostly SGI and Sun equipment.

In fact, we had exactly ZERO Windows machines in a building of more than 100 people. That changed when someone came up from NASA JPL and brought a Windows machine with him. He kept wanting to connect to our printers, etc., and we couldn't help him unless he could speak lpd or Appletalk. (I was a systems admin.) Supposedly NASA JPL is (or was then) the oddball NASA center because it didn't use lots of open-source software. But then, it's also supposed to be basically 100% contractors instead of NASA employees.

Of course, that could all have changed, since I left there in 1997, but they have had a pretty good record of using open-source software.

Not only that, but they've been a good Internet citizen as well. For a while, one of the root DNS servers was at NASA Ames Research Center, and I think it may have been in the NAS building. (In fact, I think NASA may still have one of the root nameservers. I remember talk of them giving it up, but I don't know if that every happened.)

By the way, the paper was about releasing (some) NASA-developed software as open source. And that's not a new idea. For example, the batch-queueing system PBS was developed for the very same division of NASA (NAS) and released as open-source. And (before that?) was another batch-queueing system developed by NASA and released to the public, called NQS.

I've got 62 Alpha machines, 2 Intel 32bit, and 3 Itaniums on the way plus my laptop and this is only a 3 day a week job.

You want me to compile what?

I have a hacked version of PBS, a stock version of Maui, and a number of scientific libraries/applications that are compiled from source. I think thats enough :)

Hmm, I thought about moderating the parent up, but surely the original poster will read _all_ of the answers :-)

Anyway, I wanted to give a vote for OpenPBS. It works pretty well, and the code is moderately ok (i.e., I could sit down and add some new features).

It is true that the license is not Open Source (whomever) compliant, it only restricts your rights to redistribute commercially. For many people this is not an onerous restriction. Sun probably makes you register as well; they seem to like registration forms :->

PBS can use the MAUI scheduler as well. One thing that PBS does, that condor does not, is support parallel jobs.

Anyway, I don't hate it, which is more than I can say for a lot of software.

• Condor - seems to be the best free as in beer scheduler, but it's not free as in speech.
  
• OpenPBS - This one is sort of Free, but it is being developed by a company that doesn't seem so sure it likes it that way. The code goes BSD after a couple of years, and they've been doing that for several years, yet they don't make the old (now BSD) versions available, and they make you register just to download.
• Sun GridEngine - Free, and it looks pretty sweet. I couldn't get it to work on Debian, but people on the mailing list said they were using it with Debian.
• Globus Toolkit - Not so sure about this one.
• Maui - Scheduler system for supercomputers
• OSCAR - Sweet project from IBM to put together all the best Free tools for clustering! They are using the Maui scheduler in their system.

What I would really like to see is a HOWTO that gives a good overview of scheduling and clustering. Everything I have found so far is not so good.