2008 Summer Project Week:Batch Processing

Return to Project Week Main Page

Key Investigators

• Julien Jomier
• Steve Pieper
• Marco Ruiz
• Jim Miller
• Chris Churas
• Hans Johnson
• Killian Pohl
• Jeff Grethe
• Mark Anderson

Objective

Implement tests of batch processing techniques using slicer3 command line modules (EMSegmenter, Registration).

Approach, Plan

BatchMake and GWE are both ready to apply to sample datasets using distributed grid/cluster computing. This week we will flesh out the use cases and run actual jobs.

Also, review the concepts and issues documented on the Large Scale Experiment Brainstorming page.

Progress

• Identified separation between GWE and BatchMake
  • GWE: Running Slicer modules on parameter space from within slicer, console utility or scripts.
  • BatchMake: Scripting using Slicer3 modules (outside of Slicer3 GUI) and reporting

• Cases
  • 175 cases processed with BSplineDeformableRegistration on the SPL cluster using GWE.
  • 2275 cases processed with BSplineDeformableRegistration on the BIRN cluster using GWE.
  • 16 cases processed with FreeSurfer on the BIRN cluster using GWE.
  • Working out some Condor issues on the SPL cluster.
  • Prepared commands to process 50 cases with Killian's pipeline on the SPL cluster using GWE.
  • Reviewed Sylvain's cases requirements to verify they could be processed in a cluster using GWE.

• Future Work:
  • BatchMake
    • Implement support to run through GWE
    • Provide reporting tool as slicer execution modules
  • GWE
    • Detailed on the "future work" section of Project_Week_2008_Special_topic_breakout:_GWE
  • GSlicer3
    • Auto-deploy Slicer3 in clusters where module invocations will be parallelize.
    • Real time monitor interface to track status of requests submitted to clusters.
    • Integrate GWE with the "Slicer3 Chain Module Engine": Slicer3:JavaBasedChainCLMsEngine

References

• http://www.batchmake.org
• http://www.gridwizardenterprise.org