Projects:NonRigidRegistration

Requirements

Ron Kikinis we are in clear need of a non-rigid registration for grey scale images. The atlas based segementation, the fMRI effort, and the DTI efforts are all in need of an algorithm for non-rigid registration.

I would like to initiate a discussion in NAC and NA-MIC about the best available algorithms among the ones that have been published.

Some of the additional requirements are

• relatively robust, with few parameters to tweak
• runs on grey scale images
• has already been published
• relatively fast (ideally speaking a few minutes for volume to volume).
• can be implemented in ITK and parallelized.

I would like you to identify existing algorithms that comes as close as possible to this set of requirements.

Specification List

I have tried to organize the comments with respect to certain categories. It is my hope that we will be able to generate a specification list from these categeories. Please change/move /add any comment as well as category. I have moved the "Ongoing Discussion" List to here .
Thanks Kilian

Use Cases

1. An investigator has already completed analysis of an fMRI study on a cohort of subjects for each of which she has a good quality anatomical MRI scan. The investigator would like to make figures for their manuscripts (even cover art for their best work) that allows them to display the group average statistical maps showing the results overlaid on an exemplar subjects (or a average of the subjects if able to generate or import) anatomical scan. An additional feature that would be valuable would include the ability to visualize an anatomically defined ROI such as a Freesurfer segmentation or parcellation unit as is currently possible with the vtkFreesurferReader module. The strength of this would lie in the ability to support doing this with the statistical outputs of SPM, FSL, and AFNI toolkits most commonly in use. (Randy Gollub)

What registration problem do we want to address

• Alignment of Anatomical brain MRI
  • (Kilian Pohl) When aligning two anatomical MRI volumes with each other the results generated by BSpline or Demons generally look very nice visually. However, voxels within voxels within an area of similar intensity are randomly mapped from source to target. This is a huge problem for segmentation algorithms that rely on aligned prior information to identify weakly visible boundaries. For example, most boundaries between neighboring cortical structures are not clearly defined on anatomical MR images, such as between middle and superior temporal gyrus. Thus, segmentation algorithms use aligned prior information to distinguish these structures. However, the algorithm does not gain any information from the aligned prior information in that area as the deformation field maps voxels to random locations.
• DTI
• Non-Rigid registration of anatomical MRIs specifically for the cortical region
• EPI/MRI
  • (Sandy Wells) I have some concerns about the application of general-purpose non-rigid registration approaches to the EPI/MRI registration problem. While such approaches may produce pairs of EPI/MRI that "look better", I would be cautious about expecting that approach to be accurate. My feeling is that robust solutions to this problem will require some of the physics of the problem to be built into the solution, either by way of field maps, or physics simulations. I feel that this applies even more to Echo-Planar DTI data. (Randy Gollub) I agree.
• (Stephen Aylward) Should it handle or be a part of registration process for patient-atlas registration in the presence of large tumors or resections?
• Atlas - Image Registration
  • (Bruce Fischl) we have one that is part of our segmentation procedure that meets all these needs except the "relatively fast" one :) . It's quite robust, we've run it on hundreds of AD, schizophrenia, etc..., but it is also quite slow (15 hours or so). Gheorghe's NA-MIC project is also on non-rigid registration, although it's not published we're hoping to write it up soon.
    • (Kilian Pohl) Just to make sure that we are all on the same boat, I have a couple of questions:
      What type of registration do you use as part of your segmentation ?
      Is there a paper that describes the registration in detail ?
      Does the registration rely on tissue classification ?
      If so, can you register brains with pathologies, such as MS lesions or meningiomas ?
      • (Bruce Fischl) yes, the linear part was described in our Neuron 2002 paper (page 9 - 10), and the nonlinear extension in the IPAM thing that was published in NeuroImage NeuroImage (pages 5-7). It works fine with MS and white matter damage, not sure about tumors and such as we haven't really tried it. It is designed to be part of our segmentation (the MRF stuff), and I doubt it's optimal for functional alignment, but it works quite well for classification. It doesn't require classification - the classification requires it.
    • (Ron Kikinis) Excellent. Does it have the potential to be parallelized?
      • (Bruce Fischl) As far as parallelization, I don't see any reason why not. I think Anders may have been messing around with parallelizing it, I'm not sure.

How fast should it be

• Seconds per volume?
• Minutes per volume?
• Depends on the application: Fast is good, robust is better. If it takes more than an hour per case, its a practical problem. Ron

What type of tool

• just non-rigid
• combination of affine and non-rigid

What Metric should be used

• MI
  • (Sandy Wells) While the MI objective function can be nice to use, because it does not require domain knowledge, sometimes additional robustness can be gained by using objective functions that do, such as recent contributions by Chung and Zollei.
• KLD
  • (Sandy Wells) Albert Chung's KLD approach has been shown experimentally to be substantially more robust than MI on MRI/CT registration on the Vanderbilt data set, though somewhat less accurate, i.e., there is a "bias vs capture" tradeoff.
• "Dirichlet" approach
  • (Sandy Wells) Lilla Zollei's "Dirichlet" approach provides a natural generalizatin of the entropy approach that incorporates prior knowledge in a more controlled way, and it is better for EPI / MRI than MI is. It appeared in her thesis, and in a recent WBIR paper (see her web page for those... just google "Lilla Zollei").
• optimal transport
  • (Allan Tannenbaum) We could try the optimal transport one
• "Don't Care" feature"
  • (Stephen Aylward) Should it support the use of "don't care" regions across which the deformation is smoothly interpolated
• feature-image registration metrics
  • (Stephen Aylward) The metric used in HAMMER registration is one example from this class; however I am not promoting that particular metric - it is simply the most well known method from that class.
    • Features used in the metric could be tuned for the modalities/scales involved.
    • Using sparse, pre-selected features makes the metric very fast.
    • Parameters/features are limited by domain knowledge (e.g., presets for T1/fMRI registration).
    • The ITK image-image and feature-image registration frameworks support such metrics, in general - improvements are clearly possible/needed.

How Many Degrees of freedom

• affine
• Low order
  • (Jim Miller) I am currently fond of the idea of using a network low order transformations to model a complicated deformations.
• polynomial based
  • B-Spline
    • (Sandy Wells) One general purpose method that is used pretty widely in neuroimaging is Daniel Reuckert's combination of the MI objective function with a B spline mesh. My impression is that both of those things are in ItK already... do they play well?
    • (Guido Gerig) At UNC, we have excellent experience since over 7 years with Daniel Rueckert's Rview/cisgvtk tool (freely available), which combines linear and nonlinear registration using a polynomial approach with choice of grid spacing, choice of 6 different image match metrics like MI, NMI, cross-correlation etc., excellent esign of GUI/visualization/ROI selection/parameter settings and command-line execution as batch jobs, very nice design of cascading deformation fields, running the tool for calculation of deformation and separately for applying the deformation field etc. This tool is embeded into all our brain processing pipelines and we have experience on thousands of intra- inter-patient and inter-modality registrations and atlas building. This method/tool seems reimplemented in ITK but my students could never really made it work as robust as the original Rview version, and it is not clear to me if the ITK version is a re-implementation or complete redisign independent of the original developers. For our autimatic EMS brain segmentation, our PhD students even went that far just to reimplement MI linear/nonlinear registration a la Rueckert in ITK (Prastawa et al.). The Rueckert tool is not invertable and does mathematically not guarantee that there isn't overfolding of space.
• high-dimensional

Type of transformation

• cascading several transformations
• directed non-invertible
• diffeomorphic
  • (Guido Gerig) For diffeomorphic high-dimensional transformations, we use high-dimensional fluid deformation as developed by Miller/Christensen/Joshi, which is also a central part on the population-based unbiased atlas-building developed by Joshi et al. Since this method is diffeomorphic, and was extended to provide a symmetric transformation between pairs and populations of datasets, it can be used for transforming to an average/template but also to go backwards by mapping atlas segmentations back to the individual cases for automatic segmentation and statistical analysis. The fluid transformation is not part of ITK and there are speed issues with the Fourier transforms/backtransforms if integrated in ITK, but my programmer colleagues might know better if there was is an ITK-based fluid deformation available. Speed issues were a concern a few years ago, but current versions take 30 to 60' on standard cheap PCs, which is good enough for automatic batch processing.
• fully invertible transformations
• Multi-scale
  • (Stephen Aylward) insensitive to moving/fixed images being at different resolutions e.g., DTI/fMRI vs 3D T1

Type of Optimizer

• stochastic gradient descent
  • (Sandy Wells) I have found it, empirically, to be an effective (i.e., fast) choice. There was an interesting paper at the WBIR conference by one of Josien Pluim's students that evaluated a collection of optimizers on the Reuckert-style MI + B spline approach. This paper showed winning performance by SGC.

Example Datasets

• Dartmouth test data and MNI template: Dartmouth structural dataset acquired on a Philips Achieva 3T scanner; normalized dataset generated with SPM5, courtesy of the Brain Imaging Laboratory, Dartmouth Medical School. Download includes target dataset colin27 MNI template downloaded from http://imaging.mrc-cbu.cam.ac.uk/downloads/Colin/

• here.