Open Source Batch Management?

Asgard asks: "My employer is currently running a commercial batch management platform. Unfortunately the licensing model makes it unfeasible to run it in the development / testing environments, leading to poor usage of the tool and unexpected failures in production. I'm looking for an equivalent Open Source tool and am wondering how others have approached the problem. Does Slashdot have any suggestions?" Imagine a system like cron, but with job dependencies. Are there any batch systems out there like this? "The tools I've found through web searches mostly treat 'batch management' from the cluster perspective -- a user submits an ad-hoc job and the tool figures out where and when to run it based on load and architecture requirements. Instead I am looking for something that manages daily schedules of jobs based on their dependencies with other jobs and external events, such as files arriving or time.

An example might be that every day jobs a, b, c, and d must run. Job a must not run before 9pm and requires file X to be present. Jobs b and c depend on a completing successfully. Job d must run after 2am and after b and c have completed successfully. If job c fails then an operator must fix the issue and rerun it, after which the tool will move on to job d. "

Comments

TORQUE Resource Manager

[Bryan_Casto]

I think TORQUE Resource Manager will do what you're looking for. From their page:

TORQUE (Tera-scale Open-source Resource and QUEue manager) is a resource manager providing control over batch jobs and distributed compute nodes. It is a community effort based on the original *PBS project and has incorporated significant advances in the areas of scalability, fault tolerance, and feature extensions contributed by NCSA, OSC, USC, the U.S. Dept of Energy, Sandia, PNNL, U of Buffalo, TeraGrid, and many other leading edge HPC organizations.

For example:

[Ayanami Rei]

in cron.daily...
make -j $NCPUs -C /working/dir /working/dir/Makefile -

all: tasks/1 tasks/2 tasks/3

tasks/1:
foo bar baz
frob fritz
touch tasks/1

tasks/2: tasks/2.1 tasks/2.2 some_make_test(tasks/2.3)
bar baz qyzzy and touch tasks/2

etc. etc. etc.