2008 Summer Project Week:Nonlinear transforms

Return to Project Week Main Page

MRML scene graph.

Transform node in a MRML scene.

Key Investigators

• GE Research: Jim Miller, Bess Lee
• Isomics: Steve Pieper, Alex Y.

Objective

Extend Slicer's suite of transformations to include nonlinear transformations (B-spline, deformation grids) in the MRML scene, visualization pipeline, and command line modules.

Approach, Plan

• Develop new MRML nodes and MRML storage nodes for B-splines and deformation grids.
  • vtkITK wrapper around ITK classes
• Add attribute to transform nodes to indicate whether they represent a "Transform to World/Parent" or "Transform from World/Parent"
• Extend visualization system for "Transform to World/Parent" or "Transform from World/Parent"
• Extend command line modules to annotate a transformation as a "Transform to World/Parent" or "Transform from World/Parent"
• Identify places in slicer3 code where linear transforms are assumed - these either need generalization or informative warning/error messages

Progress

Concrete

• MRML nodes and storage nodes for B-spline and deformation grids "done".
• "Copied" the "Linear registration" module to "Rigid registration"
• "Affine registration" now a shared object library (much faster)

Discussions

• Eulerian/Langrangian viewpoint discussions (Gary, Steve)
• Transform file format "should" indicate the coordinate frames and direction a transform maps* "Add Transform..." menu added
• Coordinate frame manager to map between any two data sets verses mapping to parent/world. (Mike)
  • Named coordinate frames, coordinate frame for new data is labeled as the name of the dataset
  • If transformation between coordinate frames is "unknown", Slicer will initially assume they are the same coordinate frame (identity)

Todo

• itkDisplacementFieldTransform
• Transform attribute of coordinate frame
• Coordinate frame manager from Mike Halle
  • Maps between any two datasets not necessarily to world, have multiple transformation "estimates" for a single coordinate frame mapping.
• Visualization support?
  • TransformDisplayNode?
• Need to patch itkTransformFactory (trunk and release) to register BSplines for 2D and 3D for floats and doubles (Luis)

Design options

1. (Short term) Assume all transforms in ITK transform files are resampling transforms and hence need to be inverted on read/write from Slicer.
   • Nothing changes at the CLI interface layer
   • Nothing changes within the CLI module
   • Modify vtkMRMLTransformStorageNode to add an inversion to the sequence of converting Slicer <-> ITK (already doing an RAS to LPS conversion).
   • Affects all transforms loaded/saved to file (such as through the menu option to "Add Transform...")
   • Ugly for BSpline and GridTransforms (speed and accuracy concerns).
   • Prototype implemented.
2. (Short term) Assume all transforms to/from CLI need "to be" or "are" resampling transforms.
   • All changes are at the CLI interface layer
   • Nothing changes within the CLI module
   • Modify the vtkCommandLineModuleLogic (writing of transforms) and vtkSlicerApplicationLogic (reading of transforms).
   • Only affects transforms sent to/from CLI. Doesn't affect transforms loaded with "Add Transform...".
     • So transform files generated via batch running of CLI will have to be manually inverted within Slicer (or an invert option will need to be added to the "Add Transform..." dialog).
   • Ugly for BSpline and GridTransforms (speed and accuracy concerns).
3. (Long term) Add attribute to transforms to indicate transform sense (ToParent verses FromParent)
   • Modification needed at the CLI interface layer to manage what sense CLI are expecting and producing
   • Modify vtkMRMLTransformStorageNode to properly manage sense, provide the right sense to a CLI, etc.
   • Nothing changes within the CLI module
   • Modify all uses of transforms to take advantage of the sense of transform (skip unnecessary transform inversions)
   • BSplines and GridTransforms may be handled cleanly

• Should we add a mechanism in the CLI to indicate how a transform should be added to the scene?

    <image>
      <name>MovingImageFileName</name>
      <label>Moving Image</label>
      <channel>input</channel>
      <index>1</index>
      <description>Moving image</description>
    </image>
    <transform fileExtensions=".txt" reference="movingImageFileName">
      <name>OutputTransform</name>
      <longflag>outputtransform</longflag>
      <description>Transform calculated that aligns the fixed and moving image. Maps positions in the fixed coordinate frame to the moving coordinate frame. Optional (specify an output transform or an output volume or both).</description>
      <label>Output transform</label>
      <channel>output</channel>
    </transform>

Coordinate Frames

Mike Halle has a prototype coordinate frame manager written in python. This class allows one to name coordinate frames and then register transformations that map from one coordinate frame to another coordinate frame. The coordinate frame manager uses a graph search technique to construct transformations to map between any two named coordinate frames.

We could implement this manager in Slicer as is or distribute the information across several Slicer objects. For instance,

1. New MRML node type vtkMRMLCoordinateFrameNode would have an ivar to name the coordinate frame.
2. Add an ivar to vtkMRMLTransformableNode to keep the ID of the data's coordinate frame
3. Add two ivars to vtkMRMLTransformNode to keep track of the IDs of the "from" coordinate frame and the "to" coordinate frame

Each MRMLTransformableNode would initially define a unique coordinate frame. Slicer could treat coordinate frames with no mappings as being trivially aligned with the world coordinate frame. Or Slicer could force the user to define coordinate frame mappings between two data sets before those data sets are allowed to be shown in the same visualization or processed by certain algorithms.