Difference between revisions of "Summer2009:Extension of the Command Line XML Syntax/Interface"
m (Text replacement - "http://www.slicer.org/slicerWiki/index.php/" to "https://www.slicer.org/wiki/") |
|||
(11 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
__NOTOC__ | __NOTOC__ | ||
<gallery> | <gallery> | ||
− | + | ||
Image:JSIT-fig2.png|JIST Pipeline | Image:JSIT-fig2.png|JIST Pipeline | ||
Image:JIST-fig3.png|Multi-Core and Grid Processing | Image:JIST-fig3.png|Multi-Core and Grid Processing | ||
Image:JIST-fig1.png|MIPAV Plugins | Image:JIST-fig1.png|MIPAV Plugins | ||
+ | Image:JIST-3D.png|Example: Fully automatic surface, DTI, and sub-cortical analysis | ||
+ | Image:PW2009-v3.png|[[2009_Summer_Project_Week|Project Week Main Page]] | ||
</gallery> | </gallery> | ||
− | |||
− | |||
==Key Investigators== | ==Key Investigators== | ||
Line 25: | Line 25: | ||
<h3>Approach, Plan</h3> | <h3>Approach, Plan</h3> | ||
− | The XML syntax and parser implementation for the SLICER CLI | + | The XML syntax and parser implementation for the SLICER command line interface (CLI) will be studied. Recommendations will be made for extended (backward compatible) syntax. A prototype parser for the new schema will be implemented within SLICER. |
</div> | </div> | ||
Line 31: | Line 31: | ||
<h3>Progress</h3> | <h3>Progress</h3> | ||
− | + | <ul> | |
− | + | <li>Surveyed existing CLI interface and identified areas for extension | |
+ | <li>Explored integration of external modules into the s3ext extension framework | ||
+ | <li>Implemented prototype JIST/MIPAV->CLI bridge and discovery procedure. | ||
+ | <li>Circulated drafts of revised "meta-protocol" layer for adaptation between existing CLI | ||
+ | <li>Communicated with XIP team about generating a common/compatible interface layer | ||
+ | </ul> | ||
+ | ''See detailed results below.'' | ||
</div> | </div> | ||
Line 39: | Line 45: | ||
<div style="width: 970%; float: left;"> | <div style="width: 970%; float: left;"> | ||
+ | ==Results== | ||
+ | '''Areas for Investigation with CLI and Existing Limitations''' | ||
+ | <ul> | ||
+ | <li>Multiple “programs” per executable. | ||
+ | <li>De-couple dependence of executable query (program which is called with “—xml") from the executable which implements desired functionality. | ||
+ | <li>Provide a mechanism to report non-resource (file-base) results, such as integer, vectors, etc. | ||
+ | <li>Report to both stdout (via xml markup) [optional] | ||
+ | <li>Generalize the calling syntax --- especially the ability to control argument structure. | ||
+ | <li>Allow authors to provide more meta-information about the algorithm (which may be ignored). | ||
+ | <li>Enable the concept of an “ignorable” parameter which need not be understood to use the program. | ||
+ | <li>Increase structure for information that authors can provide about their algorithms and moved this to a separate category. | ||
+ | <li>Increase documentation for “type-of” arguments. Attempted to provide more clarity on a “data-centric” view of the data rather than how these would be represented in ITK/VTK. | ||
+ | <li>Provide preferred and supported format syntax for resource (file) based communication. | ||
+ | <li>Ability to model dependencies between flags | ||
+ | <li>Workflow level perspective – “sample a bunch of points” = harness level things | ||
+ | <li>Ability to have variable number of output arguments and determine the number / names at RUNTIME | ||
+ | </ul> | ||
+ | |||
+ | '''Slicer 3 Extension System''' | ||
+ | <ul> | ||
+ | <li>Recognized the need for a interface specific compatability checks (rather than build level checks) because external modules might not be built on the build farm. | ||
+ | <li>Emphasized that module availability might vary by platform. Some modules could be win-only/redhat only/ubuntu only/linux kernel X.Y only/etc. | ||
+ | <li>Suggested the use of a Ubuntu "multi-verse" system for accessing/discovering external modules | ||
+ | <li>Recognized that external modules may be very large and require size checks/download monitor/cancel/resume/etc. | ||
+ | <li>Suggested that EULA's would be required for modules. Users would need to agree to these licenses prior to download (e.g., difference between external multi-verse and Slicer-licensed modules on the build farm). | ||
+ | </ul> | ||
+ | |||
+ | '''JIST/MIPAV->CLI Bridge''' | ||
+ | <ul> | ||
+ | <li> Automated discovery of JIST/MIPAV algorithms from the command line: edu.jhu.ece.iacl.jist.cli.discover | ||
+ | <li> | ||
+ | |||
+ | <div style="width:400px; margin:0 auto;><pre> | ||
+ | Adding Noise edu.jhu.bme.smile.demo.AddingNoise | ||
+ | Chebyshev Fitting edu.jhu.bme.smile.demo.ChebyShevFitting | ||
+ | Demo3: Image Algebra edu.jhu.bme.smile.demo.ImageAlgebra | ||
+ | Image Arithmetic edu.jhu.bme.smile.demo.ImageArithmetic | ||
+ | Demo2: Generate Random Volumes edu.jhu.bme.smile.demo.RandomVol | ||
+ | Demo4: Scale input image edu.jhu.bme.smile.demo.ScaleImageDemo | ||
+ | staple edu.jhu.bme.smile.demo.SmileAlgorithmDemo | ||
+ | ... | ||
+ | </pre></div> | ||
+ | |||
+ | <li> Automated self-documenting executables | ||
+ | <li> | ||
+ | <div style="width:99.5%; margin:0 auto;> | ||
+ | <pre>usage: edu.jhu.ece.iacl.plugins.registration.MedicAlgorithmFLIRTCollection | ||
+ | [-h] [-inApply <arg>] [-inCoarse <arg>] [-inCost <arg>] [-inDegrees | ||
+ | <arg>] [-inFine <arg>] [-inInput <arg>] [-inMaximum <arg>] | ||
+ | [-inMinimum <arg>] [-inMultiple <arg>] [-inNumber <arg>] | ||
+ | [-inNumber2 <arg>] [-inOutput <arg>] [-inReference <arg>] | ||
+ | [-inRegistration <arg>] [-inSkip <arg>] [-inSource <arg>] | ||
+ | [-inSubsample <arg>] [-inTarget <arg>] [-inUse <arg>] | ||
+ | [-outExecution] [-outRegistered] [-outTransformation] | ||
+ | [-outTransformation2] [-outTransformations] [-xDir <arg>] [-xFile | ||
+ | <arg>] | ||
+ | |||
+ | Optimized Automatic Linear Registration 1.1.1.1 1.1.1.1 unk | ||
+ | Linear Registration algorithm based on FLIRT. | ||
+ | -h,--help Print this message. | ||
+ | -inApply <arg> Apply rotation [option:All|X|Y|Z] (required, | ||
+ | default=All) | ||
+ | -inCoarse <arg> Coarse angle increment [double] (required, | ||
+ | default=15.0) | ||
+ | -inCost <arg> Cost function [option:Correlation ratio|Least | ||
+ | squares|Normalized cross correlation|Normalized | ||
+ | mutual information] (required, | ||
+ | default=Correlation ratio) | ||
+ | -inDegrees <arg> Degrees of freedom [option:Rigid - 6|Global | ||
+ | rescale - 7|Specific rescale - 9|Affine - 12] | ||
+ | (required, default=Affine - 12) | ||
+ | -inFine <arg> Fine angle increment [double] (required, | ||
+ | default=6.0) | ||
+ | -inInput <arg> Input Weighted volume [file] (optional) | ||
+ | -inMaximum <arg> Maximum angle [double] (required, default=30.0) | ||
+ | -inMinimum <arg> Minimum angle [double] (required, default=-30.0) | ||
+ | -inMultiple <arg> Multiple of tolerance to bracket the minimum | ||
+ | [integer] (required, default=10) | ||
+ | -inNumber <arg> Number of iterations [integer] (required, | ||
+ | default=2) | ||
+ | -inNumber2 <arg> Number of minima from Level 8 to test at Level 4 | ||
+ | [integer] (required, default=3) | ||
+ | -inOutput <arg> Output interpolation [option:Trilinear|Bspline | ||
+ | 3rd order|Bspline 4th order|Cubic | ||
+ | Lagrangian|Quintic Lagrangian|Heptic | ||
+ | Lagrangian|Windowed sinc|Nearest Neighbor] | ||
+ | (required, default=Trilinear) | ||
+ | -inReference <arg> Reference Weighted volume [file] (optional) | ||
+ | -inRegistration <arg> Registration interpolation | ||
+ | [option:Trilinear|Bspline 3rd order|Bspline 4th | ||
+ | order|Cubic Lagrangian|Quintic Lagrangian|Heptic | ||
+ | Lagrangian|Windowed sinc] (required, | ||
+ | default=Trilinear) | ||
+ | -inSkip <arg> Skip multilevel search (Assume images are close | ||
+ | to alignment) [boolean] (required, default=false) | ||
+ | -inSource <arg> Source Volumes [file collection: semi-colon | ||
+ | delimited list] (required, default=) | ||
+ | -inSubsample <arg> Subsample image for speed [boolean] (required, | ||
+ | default=true) | ||
+ | -inTarget <arg> Target volume [file] (required) | ||
+ | -inUse <arg> Use the max of the min resolutions of the two | ||
+ | datasets when resampling [boolean] (required, | ||
+ | default=true) | ||
+ | -outExecution Execution Time [string] (required) | ||
+ | -outRegistered Registered Volumes [file collection: semi-colon | ||
+ | delimited list] (required, default=) | ||
+ | -outTransformation Transformation Matrix [matrix semi-colon | ||
+ | delimited rows in text] (optional) | ||
+ | -outTransformation2 Transformation Matrices [file] (optional) | ||
+ | -outTransformations Transformations [file collection: semi-colon | ||
+ | delimited list] (required, default=) | ||
+ | -xDir <arg> Request Output: Processing Directory (default | ||
+ | current) [directory] (optional) | ||
+ | -xFile <arg> Request Output: Results File (default output.txt) | ||
+ | [file] (optional) | ||
+ | |||
+ | Provided by: JIST (Java Image Science Toolkit) Command Line Interface v0.1 | ||
+ | http://www.nitrc.org/projects/jist/ | ||
+ | </pre></div> | ||
+ | <li> These features will be released in the next version of JIST. | ||
+ | </ul> | ||
+ | |||
+ | '''Framework for Interoperability of Neuroimaging Software''' | ||
+ | <ul> | ||
+ | <li>Existing Slicer CLI can be bridge to ''almost'' any command line system through use of shell scripts. | ||
+ | <li>These scripts are platform specific and are complex to extend for larger projects --- do we write script factories? Factories for platform specific factories? | ||
+ | <li>If we are going to address the problem of inter-communication, perhaps it would be best to develop a SIMPLE architecture agnostic XML syntax for functionality query, programmatic direction, and result response. | ||
+ | <li>Started a NITRC project (FOPA - Framework for Open Programmatic Access) to develop multi-platform reference implementations. http://www.nitrc.org/projects/fopa/ | ||
+ | </ul> | ||
==References== | ==References== | ||
<ul> | <ul> | ||
<li>MIPAV: http://mipav.cit.nih.gov/ | <li>MIPAV: http://mipav.cit.nih.gov/ | ||
<li>JIST: http://www.nitrc.org/projects/jist/ | <li>JIST: http://www.nitrc.org/projects/jist/ | ||
− | <li>SLICER CLI: | + | <li>SLICER CLI: https://www.slicer.org/wiki/Slicer3:Execution_Model_Documentation |
</ul> | </ul> | ||
</div> | </div> |
Latest revision as of 18:07, 10 July 2017
Home < Summer2009:Extension of the Command Line XML Syntax < InterfaceKey Investigators
- Johns Hopkins/Vanderbilt: Bennett Landman
- GE Research: Jim Miller
- Johns Hopkins: Heba Mustufa (not attending in person)
Objective
We will extend the command line interface (CLI) to support more robust/generic communication between SLICER and third party modules. The primary object is to enable cross-compatibility between the MIPAV/JIST module framework and the SLICER CLI framework. Successful completion of this task would enable the Johns Hopkins CRUISE (cortical surface) and CATNAP (diffusion tensor imaging) modules to be run within SLICER. The secondary (and long term) objective is to establish a flexible and usable syntax for module/plugin communication with graphical front-ends which could be reused in other medical image analysis settings.
Approach, Plan
The XML syntax and parser implementation for the SLICER command line interface (CLI) will be studied. Recommendations will be made for extended (backward compatible) syntax. A prototype parser for the new schema will be implemented within SLICER.
Progress
- Surveyed existing CLI interface and identified areas for extension
- Explored integration of external modules into the s3ext extension framework
- Implemented prototype JIST/MIPAV->CLI bridge and discovery procedure.
- Circulated drafts of revised "meta-protocol" layer for adaptation between existing CLI
- Communicated with XIP team about generating a common/compatible interface layer
See detailed results below.
Results
Areas for Investigation with CLI and Existing Limitations
- Multiple “programs” per executable.
- De-couple dependence of executable query (program which is called with “—xml") from the executable which implements desired functionality.
- Provide a mechanism to report non-resource (file-base) results, such as integer, vectors, etc.
- Report to both stdout (via xml markup) [optional]
- Generalize the calling syntax --- especially the ability to control argument structure.
- Allow authors to provide more meta-information about the algorithm (which may be ignored).
- Enable the concept of an “ignorable” parameter which need not be understood to use the program.
- Increase structure for information that authors can provide about their algorithms and moved this to a separate category.
- Increase documentation for “type-of” arguments. Attempted to provide more clarity on a “data-centric” view of the data rather than how these would be represented in ITK/VTK.
- Provide preferred and supported format syntax for resource (file) based communication.
- Ability to model dependencies between flags
- Workflow level perspective – “sample a bunch of points” = harness level things
- Ability to have variable number of output arguments and determine the number / names at RUNTIME
Slicer 3 Extension System
- Recognized the need for a interface specific compatability checks (rather than build level checks) because external modules might not be built on the build farm.
- Emphasized that module availability might vary by platform. Some modules could be win-only/redhat only/ubuntu only/linux kernel X.Y only/etc.
- Suggested the use of a Ubuntu "multi-verse" system for accessing/discovering external modules
- Recognized that external modules may be very large and require size checks/download monitor/cancel/resume/etc.
- Suggested that EULA's would be required for modules. Users would need to agree to these licenses prior to download (e.g., difference between external multi-verse and Slicer-licensed modules on the build farm).
JIST/MIPAV->CLI Bridge
- Automated discovery of JIST/MIPAV algorithms from the command line: edu.jhu.ece.iacl.jist.cli.discover
-
Adding Noise edu.jhu.bme.smile.demo.AddingNoise Chebyshev Fitting edu.jhu.bme.smile.demo.ChebyShevFitting Demo3: Image Algebra edu.jhu.bme.smile.demo.ImageAlgebra Image Arithmetic edu.jhu.bme.smile.demo.ImageArithmetic Demo2: Generate Random Volumes edu.jhu.bme.smile.demo.RandomVol Demo4: Scale input image edu.jhu.bme.smile.demo.ScaleImageDemo staple edu.jhu.bme.smile.demo.SmileAlgorithmDemo ...
- Automated self-documenting executables
-
usage: edu.jhu.ece.iacl.plugins.registration.MedicAlgorithmFLIRTCollection [-h] [-inApply <arg>] [-inCoarse <arg>] [-inCost <arg>] [-inDegrees <arg>] [-inFine <arg>] [-inInput <arg>] [-inMaximum <arg>] [-inMinimum <arg>] [-inMultiple <arg>] [-inNumber <arg>] [-inNumber2 <arg>] [-inOutput <arg>] [-inReference <arg>] [-inRegistration <arg>] [-inSkip <arg>] [-inSource <arg>] [-inSubsample <arg>] [-inTarget <arg>] [-inUse <arg>] [-outExecution] [-outRegistered] [-outTransformation] [-outTransformation2] [-outTransformations] [-xDir <arg>] [-xFile <arg>] Optimized Automatic Linear Registration 1.1.1.1 1.1.1.1 unk Linear Registration algorithm based on FLIRT. -h,--help Print this message. -inApply <arg> Apply rotation [option:All|X|Y|Z] (required, default=All) -inCoarse <arg> Coarse angle increment [double] (required, default=15.0) -inCost <arg> Cost function [option:Correlation ratio|Least squares|Normalized cross correlation|Normalized mutual information] (required, default=Correlation ratio) -inDegrees <arg> Degrees of freedom [option:Rigid - 6|Global rescale - 7|Specific rescale - 9|Affine - 12] (required, default=Affine - 12) -inFine <arg> Fine angle increment [double] (required, default=6.0) -inInput <arg> Input Weighted volume [file] (optional) -inMaximum <arg> Maximum angle [double] (required, default=30.0) -inMinimum <arg> Minimum angle [double] (required, default=-30.0) -inMultiple <arg> Multiple of tolerance to bracket the minimum [integer] (required, default=10) -inNumber <arg> Number of iterations [integer] (required, default=2) -inNumber2 <arg> Number of minima from Level 8 to test at Level 4 [integer] (required, default=3) -inOutput <arg> Output interpolation [option:Trilinear|Bspline 3rd order|Bspline 4th order|Cubic Lagrangian|Quintic Lagrangian|Heptic Lagrangian|Windowed sinc|Nearest Neighbor] (required, default=Trilinear) -inReference <arg> Reference Weighted volume [file] (optional) -inRegistration <arg> Registration interpolation [option:Trilinear|Bspline 3rd order|Bspline 4th order|Cubic Lagrangian|Quintic Lagrangian|Heptic Lagrangian|Windowed sinc] (required, default=Trilinear) -inSkip <arg> Skip multilevel search (Assume images are close to alignment) [boolean] (required, default=false) -inSource <arg> Source Volumes [file collection: semi-colon delimited list] (required, default=) -inSubsample <arg> Subsample image for speed [boolean] (required, default=true) -inTarget <arg> Target volume [file] (required) -inUse <arg> Use the max of the min resolutions of the two datasets when resampling [boolean] (required, default=true) -outExecution Execution Time [string] (required) -outRegistered Registered Volumes [file collection: semi-colon delimited list] (required, default=) -outTransformation Transformation Matrix [matrix semi-colon delimited rows in text] (optional) -outTransformation2 Transformation Matrices [file] (optional) -outTransformations Transformations [file collection: semi-colon delimited list] (required, default=) -xDir <arg> Request Output: Processing Directory (default current) [directory] (optional) -xFile <arg> Request Output: Results File (default output.txt) [file] (optional) Provided by: JIST (Java Image Science Toolkit) Command Line Interface v0.1 http://www.nitrc.org/projects/jist/
- These features will be released in the next version of JIST.
Framework for Interoperability of Neuroimaging Software
- Existing Slicer CLI can be bridge to almost any command line system through use of shell scripts.
- These scripts are platform specific and are complex to extend for larger projects --- do we write script factories? Factories for platform specific factories?
- If we are going to address the problem of inter-communication, perhaps it would be best to develop a SIMPLE architecture agnostic XML syntax for functionality query, programmatic direction, and result response.
- Started a NITRC project (FOPA - Framework for Open Programmatic Access) to develop multi-platform reference implementations. http://www.nitrc.org/projects/fopa/
References
- MIPAV: http://mipav.cit.nih.gov/
- JIST: http://www.nitrc.org/projects/jist/
- SLICER CLI: https://www.slicer.org/wiki/Slicer3:Execution_Model_Documentation