Difference between revisions of "Summer2009:Extension of the Command Line XML Syntax/Interface"

From NAMIC Wiki
Jump to: navigation, search
(Created page with '__NOTOC__ <gallery> Image:PW2009-v3.png|Project Week Main Page </gallery> ==Key Investigators== * Johns Hopkins/Vanderbilt: Bennett Landman * John...')
 
m (Text replacement - "http://www.slicer.org/slicerWiki/index.php/" to "https://www.slicer.org/wiki/")
 
(14 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
__NOTOC__
 
__NOTOC__
 
<gallery>
 
<gallery>
 +
 +
Image:JSIT-fig2.png|JIST Pipeline
 +
Image:JIST-fig3.png|Multi-Core and Grid Processing
 +
Image:JIST-fig1.png|MIPAV Plugins
 +
Image:JIST-3D.png|Example: Fully automatic surface, DTI, and sub-cortical analysis
 
Image:PW2009-v3.png|[[2009_Summer_Project_Week|Project Week Main Page]]
 
Image:PW2009-v3.png|[[2009_Summer_Project_Week|Project Week Main Page]]
 
</gallery>
 
</gallery>
−
 
−
 
  
 
==Key Investigators==
 
==Key Investigators==
 
* Johns Hopkins/Vanderbilt: Bennett Landman
 
* Johns Hopkins/Vanderbilt: Bennett Landman
 +
* GE Research: Jim Miller
 
* Johns Hopkins: Heba Mustufa (not attending in person)
 
* Johns Hopkins: Heba Mustufa (not attending in person)
  
Line 21: Line 25:
  
 
<h3>Approach, Plan</h3>
 
<h3>Approach, Plan</h3>
−
The XML syntax and parser implementation for the SLICER CLI interface will be studied. Recommendations will be made for extended (backward compatible) syntax. A prototype parser for the new schema will be implemented within SLICER.  
+
The XML syntax and parser implementation for the SLICER command line interface (CLI) will be studied. Recommendations will be made for extended (backward compatible) syntax. A prototype parser for the new schema will be implemented within SLICER.  
 
</div>
 
</div>
  
Line 27: Line 31:
  
 
<h3>Progress</h3>
 
<h3>Progress</h3>
−
To be filled in later.
+
<ul>
−
 
+
<li>Surveyed existing CLI interface and identified areas for extension
 +
<li>Explored integration of external modules into the s3ext extension framework
 +
<li>Implemented prototype JIST/MIPAV->CLI bridge and discovery procedure.  
 +
<li>Circulated drafts of revised "meta-protocol" layer for adaptation between existing CLI
 +
<li>Communicated with XIP team about generating a common/compatible interface layer
 +
</ul>
 +
''See detailed results below.''
  
 
</div>
 
</div>
Line 35: Line 45:
 
<div style="width: 970%; float: left;">
 
<div style="width: 970%; float: left;">
  
 +
==Results==
 +
'''Areas for Investigation with CLI and Existing Limitations'''
 +
<ul>
 +
<li>Multiple “programs” per executable.
 +
<li>De-couple dependence of executable query (program which is called with “—xml") from the executable which implements desired functionality.
 +
<li>Provide a mechanism to report non-resource (file-base) results, such as integer, vectors, etc.
 +
<li>Report to both stdout (via xml markup) [optional]
 +
<li>Generalize the calling syntax --- especially the ability to control argument structure.
 +
<li>Allow authors to provide more meta-information about the algorithm (which may be ignored).
 +
<li>Enable the concept of an “ignorable” parameter which need not be understood to use the program.
 +
<li>Increase structure for information that authors can provide about their algorithms and moved this to a separate category.
 +
<li>Increase documentation for “type-of” arguments. Attempted to provide more clarity on a “data-centric” view of the data rather than how these would be represented in ITK/VTK.
 +
<li>Provide preferred and supported format syntax for resource (file) based communication.
 +
<li>Ability to model dependencies between flags
 +
<li>Workflow level perspective – “sample a bunch of points” = harness level things
 +
<li>Ability to have variable number of output arguments and determine the number / names at RUNTIME
 +
</ul>
 +
 +
'''Slicer 3 Extension System'''
 +
<ul>
 +
<li>Recognized the need for a interface specific compatability checks (rather than build level checks) because external modules might not be built on the build farm.
 +
<li>Emphasized that module availability might vary by platform. Some modules could be win-only/redhat only/ubuntu only/linux kernel X.Y only/etc.
 +
<li>Suggested the use of a Ubuntu "multi-verse" system for accessing/discovering external modules
 +
<li>Recognized that external modules may be very large and require size checks/download monitor/cancel/resume/etc.
 +
<li>Suggested that EULA's would be required for modules. Users would need to agree to these licenses prior to download (e.g., difference between external multi-verse and Slicer-licensed modules on the build farm).
 +
</ul>
 +
 +
'''JIST/MIPAV->CLI Bridge'''
 +
<ul>
 +
<li> Automated discovery of JIST/MIPAV algorithms from the command line: edu.jhu.ece.iacl.jist.cli.discover
 +
<li>
 +
 +
<div style="width:400px; margin:0 auto;><pre>
 +
Adding Noise                            edu.jhu.bme.smile.demo.AddingNoise
 +
Chebyshev Fitting                      edu.jhu.bme.smile.demo.ChebyShevFitting
 +
Demo3: Image Algebra                    edu.jhu.bme.smile.demo.ImageAlgebra
 +
Image Arithmetic                        edu.jhu.bme.smile.demo.ImageArithmetic
 +
Demo2: Generate Random Volumes          edu.jhu.bme.smile.demo.RandomVol
 +
Demo4: Scale input image                edu.jhu.bme.smile.demo.ScaleImageDemo
 +
staple                                  edu.jhu.bme.smile.demo.SmileAlgorithmDemo
 +
...
 +
</pre></div>
 +
 +
<li> Automated self-documenting executables
 +
<li>
 +
<div style="width:99.5%; margin:0 auto;>
 +
<pre>usage: edu.jhu.ece.iacl.plugins.registration.MedicAlgorithmFLIRTCollection
 +
      [-h] [-inApply <arg>] [-inCoarse <arg>] [-inCost <arg>] [-inDegrees
 +
      <arg>] [-inFine <arg>] [-inInput <arg>] [-inMaximum <arg>]
 +
      [-inMinimum <arg>] [-inMultiple <arg>] [-inNumber <arg>]
 +
      [-inNumber2 <arg>] [-inOutput <arg>] [-inReference <arg>]
 +
      [-inRegistration <arg>] [-inSkip <arg>] [-inSource <arg>]
 +
      [-inSubsample <arg>] [-inTarget <arg>] [-inUse <arg>]
 +
      [-outExecution] [-outRegistered] [-outTransformation]
 +
      [-outTransformation2] [-outTransformations] [-xDir <arg>] [-xFile
 +
      <arg>]
 +
 +
Optimized Automatic Linear Registration 1.1.1.1 1.1.1.1 unk
 +
Linear Registration algorithm based on FLIRT.
 +
-h,--help              Print this message.
 +
-inApply <arg>          Apply rotation [option:All|X|Y|Z] (required,
 +
                        default=All)
 +
-inCoarse <arg>        Coarse angle increment [double] (required,
 +
                        default=15.0)
 +
-inCost <arg>          Cost function [option:Correlation ratio|Least
 +
                        squares|Normalized cross correlation|Normalized
 +
                        mutual information] (required,
 +
                        default=Correlation ratio)
 +
-inDegrees <arg>        Degrees of freedom [option:Rigid - 6|Global
 +
                        rescale - 7|Specific rescale - 9|Affine - 12]
 +
                        (required, default=Affine - 12)
 +
-inFine <arg>          Fine angle increment [double] (required,
 +
                        default=6.0)
 +
-inInput <arg>          Input Weighted volume [file] (optional)
 +
-inMaximum <arg>        Maximum angle [double] (required, default=30.0)
 +
-inMinimum <arg>        Minimum angle [double] (required, default=-30.0)
 +
-inMultiple <arg>      Multiple of tolerance to bracket the minimum
 +
                        [integer] (required, default=10)
 +
-inNumber <arg>        Number of iterations [integer] (required,
 +
                        default=2)
 +
-inNumber2 <arg>        Number of minima from Level 8 to test at Level 4
 +
                        [integer] (required, default=3)
 +
-inOutput <arg>        Output interpolation [option:Trilinear|Bspline
 +
                        3rd order|Bspline 4th order|Cubic
 +
                        Lagrangian|Quintic Lagrangian|Heptic
 +
                        Lagrangian|Windowed sinc|Nearest Neighbor]
 +
                        (required, default=Trilinear)
 +
-inReference <arg>      Reference Weighted volume [file] (optional)
 +
-inRegistration <arg>  Registration interpolation
 +
                        [option:Trilinear|Bspline 3rd order|Bspline 4th
 +
                        order|Cubic Lagrangian|Quintic Lagrangian|Heptic
 +
                        Lagrangian|Windowed sinc] (required,
 +
                        default=Trilinear)
 +
-inSkip <arg>          Skip multilevel search (Assume images are close
 +
                        to alignment) [boolean] (required, default=false)
 +
-inSource <arg>        Source Volumes [file collection: semi-colon
 +
                        delimited list] (required, default=)
 +
-inSubsample <arg>      Subsample image for speed [boolean] (required,
 +
                        default=true)
 +
-inTarget <arg>        Target volume [file] (required)
 +
-inUse <arg>            Use the max of the min resolutions of the two
 +
                        datasets when resampling [boolean] (required,
 +
                        default=true)
 +
-outExecution          Execution Time [string] (required)
 +
-outRegistered          Registered Volumes [file collection: semi-colon
 +
                        delimited list] (required, default=)
 +
-outTransformation      Transformation Matrix [matrix semi-colon
 +
                        delimited rows in text] (optional)
 +
-outTransformation2    Transformation Matrices [file] (optional)
 +
-outTransformations    Transformations [file collection: semi-colon
 +
                        delimited list] (required, default=)
 +
-xDir <arg>            Request Output: Processing Directory (default
 +
                        current) [directory] (optional)
 +
-xFile <arg>            Request Output: Results File (default output.txt)
 +
                        [file] (optional)
 +
 +
Provided by: JIST (Java Image Science Toolkit) Command Line Interface v0.1
 +
http://www.nitrc.org/projects/jist/
 +
</pre></div>
 +
<li> These features will be released in the next version of JIST.
 +
</ul>
 +
 +
'''Framework for Interoperability of Neuroimaging Software'''
 +
<ul>
 +
<li>Existing Slicer CLI can be bridge to ''almost'' any command line system through use of shell scripts.
 +
<li>These scripts are platform specific and are complex to extend for larger projects --- do we write script factories? Factories for platform specific factories?
 +
<li>If we are going to address the problem of inter-communication, perhaps it would be best to develop a SIMPLE architecture agnostic XML syntax for functionality query, programmatic direction, and result response.
 +
<li>Started a NITRC project (FOPA - Framework for Open Programmatic Access) to develop multi-platform reference implementations. http://www.nitrc.org/projects/fopa/
 +
</ul>
 
==References==
 
==References==
 
<ul>
 
<ul>
 
<li>MIPAV: http://mipav.cit.nih.gov/
 
<li>MIPAV: http://mipav.cit.nih.gov/
 
<li>JIST: http://www.nitrc.org/projects/jist/
 
<li>JIST: http://www.nitrc.org/projects/jist/
−
<li>SLICER CLI: http://www.slicer.org/slicerWiki/index.php/Slicer3:Execution_Model_Documentation
+
<li>SLICER CLI: https://www.slicer.org/wiki/Slicer3:Execution_Model_Documentation
 
</ul>
 
</ul>
  
  
 
</div>
 
</div>

Latest revision as of 18:07, 10 July 2017

Home < Summer2009:Extension of the Command Line XML Syntax < Interface

  • JIST Pipeline

  • Multi-Core and Grid Processing

  • MIPAV Plugins

  • Example: Fully automatic surface, DTI, and sub-cortical analysis

Key Investigators

  • Johns Hopkins/Vanderbilt: Bennett Landman
  • GE Research: Jim Miller
  • Johns Hopkins: Heba Mustufa (not attending in person)

Objective

We will extend the command line interface (CLI) to support more robust/generic communication between SLICER and third party modules. The primary object is to enable cross-compatibility between the MIPAV/JIST module framework and the SLICER CLI framework. Successful completion of this task would enable the Johns Hopkins CRUISE (cortical surface) and CATNAP (diffusion tensor imaging) modules to be run within SLICER. The secondary (and long term) objective is to establish a flexible and usable syntax for module/plugin communication with graphical front-ends which could be reused in other medical image analysis settings.

Approach, Plan

The XML syntax and parser implementation for the SLICER command line interface (CLI) will be studied. Recommendations will be made for extended (backward compatible) syntax. A prototype parser for the new schema will be implemented within SLICER.

Progress

  • Surveyed existing CLI interface and identified areas for extension
  • Explored integration of external modules into the s3ext extension framework
  • Implemented prototype JIST/MIPAV->CLI bridge and discovery procedure.
  • Circulated drafts of revised "meta-protocol" layer for adaptation between existing CLI
  • Communicated with XIP team about generating a common/compatible interface layer

See detailed results below.

Results

Areas for Investigation with CLI and Existing Limitations

  • Multiple “programs” per executable.
  • De-couple dependence of executable query (program which is called with “—xml") from the executable which implements desired functionality.
  • Provide a mechanism to report non-resource (file-base) results, such as integer, vectors, etc.
  • Report to both stdout (via xml markup) [optional]
  • Generalize the calling syntax --- especially the ability to control argument structure.
  • Allow authors to provide more meta-information about the algorithm (which may be ignored).
  • Enable the concept of an “ignorable” parameter which need not be understood to use the program.
  • Increase structure for information that authors can provide about their algorithms and moved this to a separate category.
  • Increase documentation for “type-of” arguments. Attempted to provide more clarity on a “data-centric” view of the data rather than how these would be represented in ITK/VTK.
  • Provide preferred and supported format syntax for resource (file) based communication.
  • Ability to model dependencies between flags
  • Workflow level perspective – “sample a bunch of points” = harness level things
  • Ability to have variable number of output arguments and determine the number / names at RUNTIME

Slicer 3 Extension System

  • Recognized the need for a interface specific compatability checks (rather than build level checks) because external modules might not be built on the build farm.
  • Emphasized that module availability might vary by platform. Some modules could be win-only/redhat only/ubuntu only/linux kernel X.Y only/etc.
  • Suggested the use of a Ubuntu "multi-verse" system for accessing/discovering external modules
  • Recognized that external modules may be very large and require size checks/download monitor/cancel/resume/etc.
  • Suggested that EULA's would be required for modules. Users would need to agree to these licenses prior to download (e.g., difference between external multi-verse and Slicer-licensed modules on the build farm).

JIST/MIPAV->CLI Bridge

  • Automated discovery of JIST/MIPAV algorithms from the command line: edu.jhu.ece.iacl.jist.cli.discover 
  • Adding Noise                            	edu.jhu.bme.smile.demo.AddingNoise
Chebyshev Fitting                       	edu.jhu.bme.smile.demo.ChebyShevFitting
Demo3: Image Algebra                    	edu.jhu.bme.smile.demo.ImageAlgebra
Image Arithmetic                        	edu.jhu.bme.smile.demo.ImageArithmetic
Demo2: Generate Random Volumes          	edu.jhu.bme.smile.demo.RandomVol
Demo4: Scale input image                	edu.jhu.bme.smile.demo.ScaleImageDemo
staple                                  	edu.jhu.bme.smile.demo.SmileAlgorithmDemo
...
  • Automated self-documenting executables 
  • usage: edu.jhu.ece.iacl.plugins.registration.MedicAlgorithmFLIRTCollection
       [-h] [-inApply <arg>] [-inCoarse <arg>] [-inCost <arg>] [-inDegrees
       <arg>] [-inFine <arg>] [-inInput <arg>] [-inMaximum <arg>]
       [-inMinimum <arg>] [-inMultiple <arg>] [-inNumber <arg>]
       [-inNumber2 <arg>] [-inOutput <arg>] [-inReference <arg>]
       [-inRegistration <arg>] [-inSkip <arg>] [-inSource <arg>]
       [-inSubsample <arg>] [-inTarget <arg>] [-inUse <arg>]
       [-outExecution] [-outRegistered] [-outTransformation]
       [-outTransformation2] [-outTransformations] [-xDir <arg>] [-xFile
       <arg>]

Optimized Automatic Linear Registration 1.1.1.1 1.1.1.1 unk
Linear Registration algorithm based on FLIRT.
 -h,--help               Print this message.
 -inApply <arg>          Apply rotation [option:All|X|Y|Z] (required,
                         default=All)
 -inCoarse <arg>         Coarse angle increment [double] (required,
                         default=15.0)
 -inCost <arg>           Cost function [option:Correlation ratio|Least
                         squares|Normalized cross correlation|Normalized
                         mutual information] (required,
                         default=Correlation ratio)
 -inDegrees <arg>        Degrees of freedom [option:Rigid - 6|Global
                         rescale - 7|Specific rescale - 9|Affine - 12]
                         (required, default=Affine - 12)
 -inFine <arg>           Fine angle increment [double] (required,
                         default=6.0)
 -inInput <arg>          Input Weighted volume [file] (optional)
 -inMaximum <arg>        Maximum angle [double] (required, default=30.0)
 -inMinimum <arg>        Minimum angle [double] (required, default=-30.0)
 -inMultiple <arg>       Multiple of tolerance to bracket the minimum
                         [integer] (required, default=10)
 -inNumber <arg>         Number of iterations [integer] (required,
                         default=2)
 -inNumber2 <arg>        Number of minima from Level 8 to test at Level 4
                         [integer] (required, default=3)
 -inOutput <arg>         Output interpolation [option:Trilinear|Bspline
                         3rd order|Bspline 4th order|Cubic
                         Lagrangian|Quintic Lagrangian|Heptic
                         Lagrangian|Windowed sinc|Nearest Neighbor]
                         (required, default=Trilinear)
 -inReference <arg>      Reference Weighted volume [file] (optional)
 -inRegistration <arg>   Registration interpolation
                         [option:Trilinear|Bspline 3rd order|Bspline 4th
                         order|Cubic Lagrangian|Quintic Lagrangian|Heptic
                         Lagrangian|Windowed sinc] (required,
                         default=Trilinear)
 -inSkip <arg>           Skip multilevel search (Assume images are close
                         to alignment) [boolean] (required, default=false)
 -inSource <arg>         Source Volumes [file collection: semi-colon
                         delimited list] (required, default=)
 -inSubsample <arg>      Subsample image for speed [boolean] (required,
                         default=true)
 -inTarget <arg>         Target volume [file] (required)
 -inUse <arg>            Use the max of the min resolutions of the two
                         datasets when resampling [boolean] (required,
                         default=true)
 -outExecution           Execution Time [string] (required)
 -outRegistered          Registered Volumes [file collection: semi-colon
                         delimited list] (required, default=)
 -outTransformation      Transformation Matrix [matrix semi-colon
                         delimited rows in text] (optional)
 -outTransformation2     Transformation Matrices [file] (optional)
 -outTransformations     Transformations [file collection: semi-colon
                         delimited list] (required, default=)
 -xDir <arg>             Request Output: Processing Directory (default
                         current) [directory] (optional)
 -xFile <arg>            Request Output: Results File (default output.txt)
                         [file] (optional)

Provided by: JIST (Java Image Science Toolkit) Command Line Interface v0.1
http://www.nitrc.org/projects/jist/
  • These features will be released in the next version of JIST.

Framework for Interoperability of Neuroimaging Software

  • Existing Slicer CLI can be bridge to almost any command line system through use of shell scripts.
  • These scripts are platform specific and are complex to extend for larger projects --- do we write script factories? Factories for platform specific factories?
  • If we are going to address the problem of inter-communication, perhaps it would be best to develop a SIMPLE architecture agnostic XML syntax for functionality query, programmatic direction, and result response.
  • Started a NITRC project (FOPA - Framework for Open Programmatic Access) to develop multi-platform reference implementations. http://www.nitrc.org/projects/fopa/

References


Retrieved from "https://www.na-mic.org/w/index.php?title=Summer2009:Extension_of_the_Command_Line_XML_Syntax/Interface&oldid=97735"