Slashdot Mirror

← Back to Stories (view on slashdot.org)

Advanced Job Scheduling?

Posted by Cliff on from the beyond-at-and-cron dept.

Kagato asks: "I'm trying to make my company's Unix boxes more mission critical in the area of job scheduling. Scheduling jobs in Unix has been around since the dawn of time. On most systems you have 'cron' and 'at' to provide most of your scheduling needs. But outside the basic world of 'do this at such time' there are a slew of commercial products that handle dependencies, failure routes, monitoring, dependent notification, etc. Commercial products of this type have been around for years. Is there anything like this available in the GNU and Open Source worlds? I've been looking at Freshmeat, SourceForge and Google. I've found the pickings for advanced scheduling are pretty slim."

6 of 24 comments (clear)

  1. Scripting by Anonymous Coward · · Score: 1, Informative

    Some of the stuff you talked about wouldn't be difficult to script at all. Once you knew what services you wanted to run and when, you could just script them to run and add all the error catching stuff in yourself. Each program has a startup script that fires it up, watches it for errors, etc, and reports back to the Master Poobah script. If something really goes wrong, the master script can page you at 2am. Too bad you wouldn't get paid for that kind off effort, other than your paycheck.

  2. Re:What commercial products? by pozitron969 · · Score: 3, Informative

    We are currently finalizing a scheduling tool purchase for the company I work for. We have taken a look at the commercial job schedulers available and we are down to two that best fit our needs. #1 -Tidal Enterprise Scheduler and #2 - Job Scheduler. We are choosing them for a Windows 2000 platform but they all have Unix agents and other platforms available as well. Here the others we looked at:

    ActiveBatch32
    UC4
    Unicenter Autosys Job Management
    Control-M
    I wish this was a post back in August.

    Good Luck!

  3. A few options by muleboy · · Score: 5, Informative
    I have been looking into this lately, and here are the options I have found:

    • Condor - seems to be the best free as in beer scheduler, but it's not free as in speech.
    • OpenPBS - This one is sort of Free, but it is being developed by a company that doesn't seem so sure it likes it that way. The code goes BSD after a couple of years, and they've been doing that for several years, yet they don't make the old (now BSD) versions available, and they make you register just to download.
    • Sun GridEngine - Free, and it looks pretty sweet. I couldn't get it to work on Debian, but people on the mailing list said they were using it with Debian.
    • Globus Toolkit - Not so sure about this one.
    • Maui - Scheduler system for supercomputers
    • OSCAR - Sweet project from IBM to put together all the best Free tools for clustering! They are using the Maui scheduler in their system.

    What I would really like to see is a HOWTO that gives a good overview of scheduling and clustering. Everything I have found so far is not so good.

    1. Re:A few options by d^2b · · Score: 2, Informative

      Hmm, I thought about moderating the parent up, but surely the original poster will read _all_ of the answers :-)

      Anyway, I wanted to give a vote for OpenPBS. It works pretty well, and the code is moderately ok (i.e., I could sit down and add some new features).

      It is true that the license is not Open Source (whomever) compliant, it only restricts your rights to redistribute commercially. For many people this is not an onerous restriction. Sun probably makes you register as well; they seem to like registration forms :->

      PBS can use the MAUI scheduler as well. One thing that PBS does, that condor does not, is support parallel jobs.

      Anyway, I don't hate it, which is more than I can say for a lot of software.

  4. You missed one: GNU Queue by crath · · Score: 3, Informative

    GNU Queue offers batch scheduling for clusters of computers; however, a cluster only needs to contain a single computer.

    One additional commercial tool we use where I work is Platform Computing's Load Sharing Facility. It works well, but it's expensive (read "over priced") and I suggest you try something else first.

  5. Re:What commercial products? by AtariDatacenter · · Score: 3, Informative

    FWIW, I'm in an environment that uses Autosys for intelligent scheduling. Seems to work pretty well. I really like the dependancies and all that you can set it. Of course, the only thing similar I used was cron, and this is light years better than cron when it comes to all the factors that were described in the article.