Slashdot Mirror

← Back to Stories (view on slashdot.org)

Open Source Experiment Management Software?

Posted by Cliff on from the open-source-for-science dept.

Alea asks: "I do a lot of empirical computer science, running new algorithms on hundreds of datasets, trying many combinations of parameters, and with several versions of many pieces of software. Keeping track of these experiments is turning into a nightmare and I spend an unreasonable amount of time writing code to smooth the way. Rather than investing this effort over and over again, I have been toying with writing a framework to manage everything, but don't want to reinvent the wheel. I can find commercial solutions (often specific to a particular domain) but does anyone know of an open source effort? Failing that, does anyone have any thoughts on such a beast?"

"The features I would want would be:

  • management of all details of an experiment, including parameter sets, datasets, and the resulting data
  • ability to "execute" experiments and report their status
  • an API for obtaining parameter values and writing out results (available to multiple languages)
  • additionally (alternately?) a standard format for transferring data (XDF might be good)
  • ability to extract selected results from experimental data
  • ability to add notes
  • ability to differentiate versions of software
In my dreamworld, it would also (via plugin architecture?) provide these:
  • automatically run experiments over several parameters values
  • distribute jobs and data over a cluster
  • output to various formats (spreadsheets, Matlab, LaTeX tables, etc.)
Things I don't think it needs to do:
  • provide a fancy front-end (that can be done separately - I'm thinking mainly in terms of libraries)
  • visualize data
  • statistical analysis (although some basic stats would be handy)
The amount of output data I'm dealing with doesn't necessitate database software (some sort of structured markup is ok for me), but some people would probably like more powerful storage backends. I can see it as experiment management 'middleware'. There's no reason such software should be limited to computer science (nothing I'm contemplating is very domain specific). I can imagine many disciplines that would benefit."

4 of 122 comments (clear)

  1. fp by Anonymous Coward · · Score: -1, Offtopic

    ask em about my First Post nigga!

  2. OT: moderation probs by Anonymous Coward · · Score: -1, Offtopic

    no ones seems to be getting mod what happned?

  3. jesus by Anonymous Coward · · Score: -1, Offtopic

    today is the conmemoration of the death of jesus in the cross, manly due the zionist influence. it's a sad day. i dont see hope in this world now.

  4. Sad news, Earl King dead at 69 by Anonymous Coward · · Score: -1, Offtopic

    I just heard some sad news on talk radio - New Orleans Blues/R&B musician Earl King was found dead in his home this morning. There aren't any more details yet. I'm sure we'll all miss him. Even if you weren't a fan of his work, there's no denying his contributions to popular culture. Truly an American icon.