Projects:RegistrationDocumentation:Developer

Back to ARRA main page
Back to Registration main page
Registration Use-case Inventory

Discussion Hotlist

• BSpline Resampling: Francois implemented a checkbox to choose the direction of interpretation for multiple transforms. I think this resolves the current conflict of definitions nicely without extensive recursion. What remains is a small residual difference between the volume generated by the BSpline module directly and the Resampling:

The residual is on the order of 1 voxel roundoff and hence would fit Andriy's suggestion:

 When you resample, you lose accuracy due to the unnecessary intermediate conversion of 
 point to voxel after the first affine transform T_1. When you apply bspline transform T_2 
 after this, instead of using T_1(input point) as the input to T_2(), you use 
 VoxelToPoint(PointToVoxel(T_1(input point))), which picks the center of the voxel.

Implementation Hotlist

1. GUI control parameters for multiresolution registration module
2. ROI box masking: enable use of ROI box as mask for fixed image
3. Enable Parameter Presets via Load/Save/Add to Menu
4. GUI cleanup for register images: remove Bspline and fiducial, consolidate all advance tabs, consolidate affine & pipeline affine etc., "save Xform" menu move to I/O
5. documentation (reference manual) for register images, multires and fiducial reg.
6. Module for Xform Concatenation
7. Add Link to Transforms module to Registration module menu (as: "manual interactive")

Registration Modules

Separate those functions that use different algorithms. Main distinctions are affine vs. b-spline' and intensity vs. fiducials. Combinations are done by the user as workflows/pipelines. This supports a Slicer staple of clean modular design where changes in one do not easily spill over to the other, less black-box behavior for the user and hopefully better portability of the code.

1. Intensity Affine (exists, under construction)

creates Affine Xform, Input: image pair.

Parameters: DOF (3,6,7,9,12), multi-res level, sampling %, cost function, stop criterion fail (iteration), mask image or ROI

2. Intensity BSpline (exists, scheduled to become a separate registration submodule)

creates BSpline Xform, Input: Affine aligned image pair.

Parameters: grid size/spacing, multi-res level, sampling %, cost function, stop criterion fail (iteration), mask image or ROI

3. Manual Affine (exists, but not as registration module, needs scale and shear added)

interactive visual alignment with 3-12 DOF using Transforms module. This is a good tool for cases in need of an initialization.

the earlier slicer2 ability to use mouse motion directly would be well received. Technical limitations noted, on wishlist.

4. Fiducials Affine (exists, needs to become a separate registration submodule)

solves Procrustes registration problem with 6-12 DOF using ICP or similar Input: fiducial pairs.

Parameters: none

5. Surface Based Affine (ICP) (does not exist as module, but ICP is avail)

seeks registration via ICP algorithm of two surfaces. Input: 2 segmentation models/surfaces.

Parameters:

6. Fiducials BSpline (does not exist)

creates an interpolation grid from fiducials (Krigging or similar): Input: fiducial pairs

Parameters: grid size/spacing, continuity constraints

this module could serve well for registration repair, i.e. aligning local regions after automated non-rigid or affine. Simpler version interactive by dragging single grid nodes.

Registration - Related Modules

6. Xform Concatenation

combines multiple Xforms

Status: basic "harden transform" exists, but needs to be expanded and obtain more prominent position as separate Registration Submodule, or as an extension of features for the data module for interactions with the MRML tree. Manipulation of Xforms likely will happen there. A graphical representation of the concept of nested Xforms would be useful, also explain terms of 'hardening Xform' etc. Possibility to copy/duplicate a Xform needed for effiient use. A strategic decision (from an object-oriented programming viewpoint, is where to anticipate the different formats: 1) transforms have the methods to apply themselves to different data (anticipate the different data types), 2) data types have the method to apply a Xform (anticipate the different Xform. It seems the current version 1) is the more feasible one.

7. Interpolator

applies Affine and/or BSpline Xform and resamples image data

Status: exists, integrated into indiv. registration methods, isolated on ITK level, needs more uniform representation in GUI

8. Evaluation

registration quality assessment.

visualization as a registration submodule

Parameter GUI Register Images

We want the parameters sufficient to give control over all relevant aspects of the algorithm. We consider highly the clinician user who is willing to convey/deal aspects of image content rather than abstract mathematical concepts, i.e. where possible we seek to translate the mathematical concept/parameter into a interaction with the image content or something that is read from the image. On the other hand we consider the flexibility of the registration tool as vital to the image analysis researcher, i.e. we want to avoid black-box behavior.

Parameter Register Images MultiRes

• DOF: radio buttons (3,6,7,9,12)
• Multi-resolution: check boxes (2x, 4x, 8x, 16x)
• Combinations of the two above as a single list of checkboxes for increasing DOF. Media:DOFHierarchy_GUI.jpg‎
• Cost function: radio buttons (MI, NC, MSq, IR)
• Mask Image/ROI: pulldown menu for MRML nodes (labelmap or ROI from ROI module)
• Histogram Mask: restrict sampling to voxels in a predefined intensity range. Often cases have distracting content at a particular intensity range that could improve registration if masked. This can currently be done by thresholding and using the labelmap as mask, but it might be worthwhile as a direct control?
• more candidates from the current reference manual

Masking / ROI

• Enable using a box ROI via the ROI Module. This will enable the user to quickly focus on the area of interest and restrict active registration to this region. The registration module will accept a MRML ROI node as defined with the ROI module.
• reduced overlap with smaller ROI can become an issue. Need documentation/user guidance that with smaller ROI one need better initial alignment.
• the box ROI could be implemented by first creating a subvolume and sending that to the original algorithm (as if it were a full image, or by building a labelmap first. The subvolume version has interesting possibilities for speed.
• This could be extended to 2 ROI boxes, that form a combined fiducial-intensity approach: the box locations themselves provide initialization as well as masking, and the refined alignment is guided by the image content/intensity within. Issues arise re. non-rigid DOF and box sizes, i.e. unless constrained by same box size, inadverted box size differences could lead to unwanted scaling.

Fiducials

• directly implement elements/submodules from the Fiducials module into the Fiducial Affine Registration module.
• consider use of "Dual Camera View to see both corresponding structures simultaneously.
• The Fiducials module allows selecting points off model surfaces. We consider this still a likely scenario, despite the alternative ICP for automated Surface-Based Registration
• Procedure:

1. obtain model of the 2 corresponding structures
2. create new fiducial list fixed, choose color.
3. set 3Dviewer mouse mode to pick
4. select points on fixed (non-moving) reference structure (at least 3, based on DOF)
5. create new fiducial list moving; choose alternate color.
6. select corresponding points (in same order as for the list above) from moving structure
7. feed both lists as input to Fiducial Affine registration

Presets

• Parameter Presets: To shield the non-expert user from parameter settings and avoid complex GUI design, all parameters are listed and controllable via a preset dropdown menu (see image). The menu reflects a subset of the full hierarchy accessible via a Wizard/Organizer. Users can save and add their own presets.
• Adding a shared or downloaded preset is done via "Import Scene". Adding presets to the permanent/deafault list is done by dropping the preset MRML file into a dedicated preset/settings folder and restarting slicer.
• Each preset is associated with a MRML file like this one. . Adding all possible combinations would make the menu too long. Consider hierarchical menu or an organizer to add/remove individual items.
• Parameter Settings Wizard: the wizard is basically the hierarchical form of the Parameter Presets. As the presets become comprehensive, the list evolves into a tree (which corresponds to our Use Case Hierarchy Tree). Traversing this tree can be done with a Wizard that asks questions to branch. At the end is a set of presets. So there's 3 ways to pick the parameters: 1) directly from the dropdown Custom Presets menu, 2) via the Wizard, 3) by setting each manually

Algorithm/Strategic

• Transform Concatenation: (partially available: harden transforms) Multiple spatial transforms applied to the same volume appear as nested MRML entries, which can be collapsed. Important to apply before a volume resampling is executed to avoid accumulation of interpolation effects. Affine is simple matrix multiplication, Bspline would have to send the gridpoints through all Xforms and reparameterize. A complete system of arranging multiple registrations also needs an easy copy function for all MRML entry data (images and Xforms). More Details.
• Recursive DOF hierarchy: Both multi-resolution and the sequence of calls with increasing DOF could be implemented as a single recursive scheme. The caller provides an array of DOF and a matching array of resolution levels, which is processed by recursive calls with the first/last element taken from the list. A simulation Matlab program and example is here: Projects:RegistrationImprovement:RecursiveScheme

• Offline WIKI: can the documentation WIKI be downloaded and used offline, e.g. for laptop users?
• Affine SVD: (in development) decomposition of affine into the 4 3DOF components: translation, rotation, scaling, shearing. Can an SVD do this for 12DOF vs. 3 DOF? Having values that relate directly to geometry can help greatly in evaluating and adjusting results. E.g. user can tell visually which of the 3 rotations is most out of alignment. Checking the result rotation angles already provides a quick check and valuable info about where things went awry if solution is suboptimal. Also enables to decouple individual elements, i.e. extract the shear component etc. The current "Transforms" module already does this for translation & rotation, but the scale and shear portions are missing.
• Distortion Correction: e.g. spherical harmonics Gradient Nonlinearity correction (BIRN) is a form of spatial Xform to be housed in the Registration or Transformations module.
• Volumes/Display' would it be possible to have a switch in the selected volume here automatically trigger a switch in the FG/BG toggle if all FG/BG views have uniform images assigned (i.e. all FG panels show same image)

Result Evaluation Tools

see metrics on the Registration Visualization and Evaluation page

Processing & Bug Notes

• to run slicer3.5 on fat node: on X11: ssh -Y george; cd /workspace/meier
• NRRD (.nhdr) files seem to load only when raw file is compressed (raw.gz). Should be fixed (i.e. specific error message suggesting zipping) or documented
• data loading method needs to cleanup: it leaves "X11" active objects behind: progress bars. Not a refresh issue. Objects close when X11 closes.
• MRML tree applies to all module and is important interactive element: have always visible and provide functionality (e.g. save) via context/right click menu
• save dialog not intuitive: allow to choose/change filename. The combination of auto-selecting what to save and the same filename can easily cause unwanted data deletion. Waiting on QT?