Slicer Grid Computing
Items discussed
- Stephen Aylward (KitWare) presented BatchMake interface to Condor
- Neil Jones (BIRN) presented GridWizard
- Group discussed different data storage technologies: MIDAS, BIRN Federated Data/XNAT/HID, etc.
Where we are
Both BatchMake and GridWizard have (partially working) demos of launching multiple processes on remote machines. However, there's still a set of questions as to how these systems will interface with the Slicer environment to meet community needs. To that end, it's worth figuring out the answers to a few questions:
- What is the main community we should target? Will there be different modes of interaction for algorithm developers and domain scientists?
- What are two specific use cases of a Slicer user interacting with batch processing facilities through Slicer? Where does data come from, where does it need to end up? What data needs to get passed back to Slicer itself?
- What are two specific use cases of a NAMIC user interacting with batch processing facilities, perhaps outside of Slicer?
- Stephen (Aylward) presented a soup-to-nuts backend for managing "experiments", data, and displaying metrics computed from images in graphical form. Neil (Jones) discussed the infrastructure to manage the running of large number of jobs and the portal dashboard used to track the jobs.
- What facilities within Slicer or within other environements (e.g. portal or stand-alone applications) are necessary to track and manage these jobs.
- What look-and-feel of modules in Slicer is required? Do we need special "grid"-type parameters in the execution model, or should we just adhere to "min parameter", "max parameter", "step" notion?
- Can we handle configuration of either tool out of band from Slicer?
- Can we handle monitoring out of band from Slicer?
Until now, we have been focused on the infrastructure development: developing tools to perform generic batch processing. The tools are at a stage where progress (aside from just improving the tools but not deploying them) can be made in applying these to specific research needs.
To do
(Still pending: collected use cases on Wednesday; need to write up a summary and project plan with both immediate and long-term goals. In this case, "long-term" means "October".)
