2008 Progress Report NIH QnA

Urgent Items

We received this letter from our NIH program officers. Below are excerpted specific questions from this letter, with names of NA-MIC team members responsible for the answers.

MGH

• What progress has been made with the MGH subcontract in the past year?
  • There's no progress report summary following the Statement of Intent for MGH (page 42). We recognize that MGH's budget was significantly reduced this past year, as a good portion was re-budgeted to Washington University beginning in December 2007. Nevertheless, we expect some progress to have been made.
  • The scientific progress report lists MGH as participating in some efforts (Shape Based Segmentation and Registration; Spherical Wavelets; and Shape Analysis with Overcomplete Wavelets), but there are no MGH personnel listed as key investigators for that section.
• With regard to future work (Dr. Fischl's consultation), the statement of work refers to "integration of FreeSurfer with ITK and 3D Slicer." What does that mean? Wasn't that effort abandoned.
• The progress report is somewhat confusing as to whether FreeSurfer is being used to study cortical correspondence. On page 250 (Section 3.2) a collaboration between MGH and MIT is mentioned with respect to cortical correspondence, while the UNC progress report states that MOL is being used to explore cortical correspondence. Please clarify.

MGH Response (Bruce Fischl)

The following MGH Progress Report Summary was omitted inadvertently from the the final submission, as was the name of Bruce Fischl (MGH) from the mentioned list of projects:

• In the past year we have completed the development of bio-orthogonal spherical wavelets and made progress on the extension to overcomplete wavelets. The overcomplete wavelets provide invariance to rotations of the underlying coordinate system and hence should provide a more stable analysis platform for space-frequency decompositions of cortical folding patterns. We are currently investigating the expansion of different geometric invariants such as mean curvature with using the overcomplete representation.
• In addition we have given training sesssions to groups of slicer users in the generation and use of the cortical parcellations in FreeSurfer, and how they can be imported into Slicer for further analysis. These have focused on DLPFC, but can be extended to other cortical regions.

Going forward we intend to improve the FreeSurfer/Slicer/ITK interoperability. Towards this end we have worked with Core 2 to incorporate support for our internal formats in Slicer, and intend to include support for ITK formats such as NRRD directly into FreeSurfer. This will significantly increase the ease of development of algorithms that take advantage of the two platforms.

We also have an ongoing collaboration with Drs. Polina Golland, Mert Sabuncu and Thomas Yeo at MIT regarding improving cortical correspondence. Specifically, the existing FreeSurfer algorithms are being used to generate a baseline state-of-the-art accuracy measure that will be used to assess the performance of novel registration algorithms. Here we are working on both fast diffeormophic registration, and have a recent conference paper describing the "spherical demons" algorithm that improves execution time significantly without sacrificing performance. In addition, as part of Mr. Yeo's Ph.D. research we are developing techniques for the optimal alignment of either archectonically or functionally defined cortical regions, defined using whole-brain histology and functional MRI respectively. The existing FreeSurfer framework is also being used by UNC to comparatively evaluate the UNC cortical thickness framework, which employs group-wise cortical correspondence method within its Slicer based cortical thickness analysis framework.

UCLA

• What progress has been made with the UCLA subcontract in the past year?
  • There are no publications listed in progress report. A search of NA-MIC publications database shows only one paper with Art Toga or Nathan Hageman's name on it, and that's the iTools paper (from the Software and Data Integration Working Group).
  • Timeline information is out of date. It has not been updated to reflect the changes in the statement of work that was agreed upon in August 2007.

UCLA Response (Arthur Toga)

In the past year, we have developed a novel method for diffusion tensor imaging (DTI) tractography modeled on the dynamics of a viscous fluid described by the second order non-linear Navier-Stokes equations. We have also made progress towards making this method fully compatible with the NA-MIC ITK-based software infrastructure (i.e. Slicer) to promote its dissemination to the scientific community.

In addition, we have created a software tool to generate a phantom DWI dataset based on a helical shape. Its purpose is to provide a standard dataset with a known underlying ground truth for validation of DTI analysis methods. The user is able to specify the dimensions of the underlying shape of the helix and the desired DWI parameters to simulate the acquisition parameters in their own experiments. We have made the helical phantom generator available to the NAMIC group through the LONI website (http://www.loni.ucla.edu/Research/Phantom/index.html) so that they might use it to validate their own DTI/DSI methods.

• Q1: A paper citing the NAMIC grant that describes our fluid mechanic DTI tractography method has been accepted to IEEE TMI. It will become available in the NAMIC publications database shortly.
• Q2: We have updated the timeline based on the most recent NIH statement of work.

Group	Aim	Milestone	Proposed time of completion	Status
UCLA	1	Debabeler functionality	Year 1	Continued Progress
UCLA	2	SLIPIE Interpretation (Layer 1)	Year 1--Year2	In Progress
UCLA	3	SLIPIE Interpretation (Layer 2)	Year 1--Year2	On Schedule
UCLA	3	Developing ITK Modules	Year2	In Progress
UCLA	4	Integrating SRB (GSI-enabled)	Year2	Completed
UCLA	5	Integrating IDA	Year2	Completed
UCLA	5	Integrating External Visualization Applications	Year2	Completed
UCLA	6	Development of DTI Tractography Methods	Year3-4	Completed and Ongoing
UCLA	7	Integration of DTI tracotography methods into Slicer as ITK compatible modules	Year4-5	In Progress
UCLA	8	Development of DTI Registration Methods and integration into Slicer	Year4-5	Initiated
UCLA	9	Development of DTI white matter atlasing and tractography tools	Year5	Initiated

Queens

• Regarding the Brachytherapy Needle Positioning Robot Integration DBP:
  • Which grant is funding the patient data collection? If it is supported by NA-MIC, please let me know because NIBIB will have to approve your Data Safety Monitoring Plan.
  • There seems to be some scientific overlap between this project and other NIH-funded grants, R01 EB002963 (PI: Whitcomb [previously Fichtinger]) and R01 CA111288 (PI:Tempany). The statement of work for the NA-MIC subcontract states, "The deliverables of the contract is professional-grade clinical software engineering of the above modules based on the NA-MIC toolkit (to the extent reasonable and possible) and to develop end applications based on Slicer, for clinical trials in image-gu ided prostate biopsy." However, this goal also falls within the System Integration aim (Aim 3) of the NCI grant and the System Integration aim (Aim 3) of the NIBIB grant. Please provide us with additional clarification to distinguish these projects in terms of their aims.
  • Also, please confirm that Dr. Gobbi and Mr. Vikal are being supported at no more than 3 and 6 months, respectively by other grants (as they are listed for 9 and 6 months on the NAMIC subcontract).

Queens Response (Gabor Fichtinger)

Q1: Which grant is funding the patient data collection? -- The Queen's NAMIC team does not conduct clinical trials. For development and testing purposes, we use previously acquired anonymous image data provided by our clinical collaborators. The data we use does not contain patient identification information. Under the NAMIC grant, we shall not perform clinical trials. We will hand over the developed system to clinical collaborators who are supported by NAMIC.

Q2: There seems to be some scientific overlap -- There are indeed close synergies but no overlap. First: R01 EB002963 (PI: Whitcomb) concentrated on developing a transrectal prostate biopsy robot and its rapid clinical testing. For this purpose, a straightforward and minimalistic system was built to operate the robot. This system is not based on Slicer. The objective of our RoadMap project to empower this robot with superior capaboilities of Slicer and the NAMIC toolkit. The grants therefore are ideally synergistic, without any overlap. Also importantly, R01 EB002963 (PI: Whitcomb) will end on July 31, 2008, before the anniversary date of the NAMIC grant. Second, R01 CA111288 (PI:Tempany) is a Biomedical Research Partnership (BRP) that includes Acoustic Medsystems, Inc., a company whose responsibility is system's integration within the company's quality control environment. Dr. Fichtinger's NAMIC team at Queen's is not involved in system development for R01 CA111288 (PI:Tempany).

Q3: We confirm that Dr. Gobbi and Mr. Vikal are being supported at no more than 3 and 6 months, respectively by other grants (as they are listed for 9 and 6 months on the NAMIC subcontract).

Kitware

• Kitware is working on a text, "Practical Software Process", to document the NA-MIC software process. How will that be distributed? Will NA-MIC funds cover all the costs so that the text can be distributed free of charge?

Kitware Response (Will Schroeder)

Whenever possible, all works created under NA-MIC funding are distributed under an appropriate open source license (of course some publications and data have special requirements due to long-standing journal policy and privacy issues). Thus in the case of the text referred to here, "Practical Software Process", the material will indeed be available in an open source form. The idea is to capture the software process in a single document that will help other researchers and developers replicate the software processes that we use at NA-MIC.

Note that this does not preclude Kitware (and other organizations for that matter) from publishing and selling the book. This is fair game under open source licensing. We have precedent for such an approach, for example the ITK user guide http://www.itk.org/ItkSoftwareGuide.pdf is available on-line, yet the Company still sells the book.

Dissemination Core

• What does it mean when you say that you NA-MIC hosted the Workshop on Open Source and Open Data at MICCAI 2007? Did NA-MIC providing financial support? Set the agenda? Invite participants? Please clarify.

Dissemination Core Response (Tina Kapur)

NA-MIC personnel contributed their time to conduct typical tasks involved in chairing a workshop such as soliciting manuscripts and open reviews, setting the agenda, and inviting speakers for keynote presentations. No financial support was needed or provided by NA-MIC other than travel of NA-MIC presenters to the conference.

Questions for non-urgent consideration:

MIND

• Systemic Lupus Erythematous project
  • Has this project driven any new algorithm development?
  • Have the new tools been tested on other lesions? What other sorts of lesions are likely to benefit from them?
  • Has manual segmentation (which is serving as the gold standard for this project) been shown to have low inter-observer variability?

Mind Response (Jeremy Bockholt)

Has this project driven any new algorithm development?

Mark Scully has enhanced an algorithm developed by Vincent Magnotta to use a joint histogram method of feature extraction, instead of thresholding, in order to incorporate dependencies between sequences. All segmentation methods have been shown to be sensitive to intensity standardization so work has begun with Hans Johnson to build a Slicer tool implementing the methods presented by Florian Jager in "A new Method for MRI Intensity Standardization with Application to Lesion Detection in the Brain". Use of support vector machines for lesion identification and the minimum set distance for false positive identification, consistent with the methods presented in "Automated segmentation of white matter lesions in 3d brain mr images, using multivariate pattern classification" by Zhiqiang Lao, et al. In order to incorporate spatial information in the classification a Markov Random Field is being examined as a final step, consistent with the Jager paper mentioned and "Fully automatic segmentation of multiple sclerosis lesions in brain mr flair images using adaptive mixtures method and markov random field model" by Rasoul Khayati, et al. Combining all lesion segmentation methods in the NAMIC kit via boosting is also being examined. A method for detecting flow field interruption due to lesions in Diffusion Weighted Imaging is currently under development.

Have the new tools been tested on other lesions? What other sorts of lesions are likely to benefit from them?

We have used the new tools developed for lupus to compete in the 3D Segmentation in the Clinic Workshop at MICCAI 2008. We have in mind to develop the new tools generically, such that, they may be used to quantify lesions in lupus and MS. In future work, we will also test the tools on lesions found in vascular dementia, mild cognitive impairment, and Alzheimers populations.

Has manual segmentation (which is serving as the gold standard for this project) been shown to have low inter-observer variability?

We have looked at an initial sample of 5 comparing manual segmentation of white matter lesions in lupus between two human raters. The relative overlap between raters was calculated to be 0.78. This manual segmentation issue is the topic of our accepted poster to be presented at Society for Neuroscience 2008, we will present our entire sample of 15 lupus subjects with white matter lesions manually traced by two experienced human raters, as well, as results of each of the automated lesion methods.

Structural Image Analysis

• Is NA-MIC supporting the UNC-led (Martin Styner) 3D Segmentation in the Clinic Workshop at MICCAI 2008? If so, in what ways?
• Does NA-MIC have any process planned for eliminating an algorithm from its toolkit if a competing algorithm outperforms it?
• Are there any plans to integrate results from the segmentation workshop into documentation for Slicer in such a way that they are readily accessible to users choosing between Slicer modules?

Structural Image Analysis Response (Martin Styner with input from other Core 1 and 2 people)

Response to #1: NA-MIC is and has been supporting the MICCAI workshops on the comparative evaluation of image analysis tasks with real, clinical data ("3D Segmentation in the Clinic" workshops at MICCAI 2007 and MICCAI 2008). Due to NA-MIC's involvement, the two workshops thus far highlighted evaluations focusing on image analysis tasks of core importance to the NA-MIC DBPs (caudate segmentation in 2007 and lesion segmentation in 2008). NA-MIC's support is provided via time contribution of one of the organizers, Martin Styner, as well as by providing training and testing datasets. More than half of the training and testing datasets in the 2007 caudate segmentation evaluation were provided through NA-MIC by the PNL DBP I site.

Response to #2: NA-MIC is the provider of a research software platform. We do not believe that eliminating algorithms from the NA-MIC kit is in the best interest of the research community because even if a particular algorithm is not actively used as part of an end-to-end clinical solution today, or does not perform as well as another one in the context of a particular task, we want to remain open to the possibility that it could be a key enabler of solutions in the future. However, for algorithms that are part of end-to-end solutions to problems that we are actively working on, we are making a concerted effort to provide publicly accessible tutorial materials to explain and encourage their adoption.

Response to #3: While some participants especially of this year's workshop will make their work available as Slicer modules, this is no requirement on participation. We certainly encourage open source submissions, as well as availability of Slicer modules. The linkage of the online available workshop's results to the documentation of those submissions that are available as Slicer modules will provide information about the module's comparative performance. This information is readily available via the workshop websites that host the continued comparative evaluations (e.g. cause07.org for the caudate segmentation evaluation). Thus there exists an efficient way for the user to select the best performing Slicer module for the task at hand. As the evaluation is continued after the workshop, newly developed Slicer modules can be added at any time to the comparative evaluation websites.

Miscellaneous

Kitware (Will Schroeder)

Some concerns were raised regarding the blurring of the distinction between NA-MIC, ITK and VTK. We recognize the contributions that NA-MIC-funded programmers have made to both ITK and VTK and we recognize that the relationship between NA-MIC and the other toolkits is beneficial to a broad development and user community. However, in some cases there seems to be insufficient acknowledgement in the report of the tools that predated NAMIC and helped lay the groundwork for it; for example CMake and DART.

Response: Yes, in extended, long-lived open source communities it can be difficult to keep track of the original contributors and funding agencies. We believe that this is important, since the contributing parties are often asked to justify their existence; plus in the interest of fairness, not to mention the need to track the source of contributions when problems occur or changes are contemplated, a trail of ownership is desirable. We generally ask all developers to include a "Thanks:" field (or equivalent) in the file documentation to note contributions due to projects. Also, svn and cvs repositories contain extensive historical information about the developer(s) who add, modify and delete material to a project.

Another way of recognizing the contributions of funding agencies and organizations is to keep the historical narrative of a project up to date. For example, the "About" page in the ITK project http://www.itk.org/HTML/About.htm refers to the many projects that it spawned including CMake and DART. However, upon reading this document as it currently exists, it is clear that further historical information would help clarify contributions. Also, the "Sponsors" section (see http://www.itk.org/HTML/Sponsors.htm for an example) also can be helpful in ferreting out contributions.

To confuse matters more, there are many organization beyond NIH and NLM have contributed to these systems. In particular VTK has received large contributions from the user community, as sell as the US National Labs, NSF, Air Force Research Labs, Army Research Labs, DOE, and dozens of commercial entities who do not want their names publicized. For example, in the last two years Sandia National Labs have worked side-by-side with the VTK community to implement an extensive scalable information visualization system, which I predict will eventually by used in biomedical computing applications.

Having said this, we are taking this comment to heart and will revisit the historical descriptions of these projects to provide clarity on the source of contributions. We will also remind our developers that assigning appropriate credit is important and will update the documents as necessary to reflect this.

St. Louis (Dan Marcus)

XNAT is open-source using the XNAT License . What does that mean? Is it different than the NA-MIC license?

The XNAT license is quite similar to the NA-MIC license in that it gives users freedom to modify code without sharing it and to use it for commercial purposes. Its use predates the existence of NA-MIC, so we plan to stick with it. -Dan Marcus 7/28/08

Isomics (Steve Pieper)

Isomics' statement of work states: Significant effort will be devoted to re-architecting core components of 3D Slicer to make them better interoperate with other NA-MIC tools. Wasn't Slicer developed in parallel with NA-MIC tools? How have incompatibilities arisen?

Response: Version 2.x of 3D Slicer existed for a number of years prior to NA-MIC and is still used by NA-MIC participants for specific tasks. Development of version 3.x is now about 2 years old and has been developed with NA-MIC tools as the foundation. A number of functional blocks continue to be ported and re-architected to leverage the new environment. Examples include the interactive diffusion imaging modules, the Label Map Editor, and the Image Guided Therapy interfaces. In addition, as new functionality such as XNAT and Grid Wizard are added to the NA-MIC Kit, new interfaces are required.

UCSD (Jeff Grethe)

The report includes no specific information on progress at UCSD in developing and supporting grid computing for NA-MIC.

Response: The submitted progress report contains a detailed description of specific information on progress at UCSD in developing and supporting grid computing for NA-MIC, beginning on page 155. The report detailed the progress in the development of a package (GWE) that bridges the gap between biomedical researchers and tools such as Slicer and the Grid (i.e. grid middleware such as Condor, PBS, Torque used to schedule applications on a variety of individual cluster resources). Nearly 3 pages are spent on the architecture that was developed and an additional 3 pages were devoted to describing how GWE enables access to distributed computational resources through Slicer. In addition, work continues with further integration of new features in GWE and its integration with Slicer, such as the Chain Module Engine (work that was described as current work in the report) that was a project during the NA-MIC summer project week (http://wiki.na-mic.org/Wiki/index.php/Slicer3:JavaBasedChainCLMsEngine). In addition, the summer project week also had a special GWE session (http://www.na-mic.org/Wiki/index.php/Project_Week_2008_Special_topic_breakout:_GWE) and project that featured GWE and details progress on running Slicer3 modules on the SPL cluster in addition to the UCSD cluster (http://www.na-mic.org/Wiki/index.php/2008_Summer_Project_Week:Batch_Processing). The results of this project detail specific NA-MIC use cases that were enabled by GWE including: 175 cases processed with BSplineDeformableRegistration on the SPL cluster using GWE; 2275 cases processed with BSplineDeformableRegistration on the BIRN cluster using GWE; 16 cases processed with FreeSurfer on the BIRN cluster using GWE. The project also specified current and future work that continues to enable more processing on distributed resources via GWE and the integration of GWE and BatchMake.

Training Core (Randy Gollub)

Please tell us a bit more about the training core's program to provide one-on-one mentoring (it's mentioned in the timeline , but there's no information on the Wiki). How is it structured? Who can be mentored? How does one arrange for mentorship?

Response: The training core provides one-on-one mentoring within the NAMIC community to facilitate the on-going research and development projects. All members of the NAMIC senior leadership are charged with the responsibility of identifying individuals on their teams who require additional training to complete their work on any NAMIC sponsored project. The leadership use all meetings, T-cons and email to network among ourselves matching identified training needs with appropriate personnel. This mentorship is an integral part of all project team building that is steadily on-going. The NAMIC PI, Dr. Kikinis and Training Core Director, Dr. Gollub oversee the entire process. The key metrics of our success can be best appreciated by the proficiency of all our junior faculty, trainees and students in working in this multi-disciplinary environment and in their ability to advance their careers in the direction they choose.

Some updates to the timeline are needed:

Bruce Fischl

The timeline for MGH indicates that many of the tasks have been modified, but these modifications aren't listed in the table of timeline modifications.

Dan Marcus

Why not list Wash U in the timeline for the appropriate tasks?

These were inadvertently omitted from the progress report, and are listed below:

1) XNAT Desktop
1.1) establish requirements for desktop version of XNAT (complete)
1.2) develop implementation plan for prototype (complete)
1.3) implement prototype version (in progress, to be completed 3rd 
quarter, 2008)
1.4) implement alpha version(to begin 4th quarter, 2008)
2) XNAT Central
2.1) deploy XNAT Central, a public access XNAT host (complete)
2.2) coordinate with NAMIC sites to upload project data (ongoing)
2.3) continue developing XNAT Central based on feedback from NAMIC 
sites (ongoing)
3) NAMIC Kit integration
3.1) implement web services to exchange data with Slicer, Batchmake, 
and other client applications (ongoing)
3.2) add XNAT Desktop to standard NAMIC kit distribution (to be 
determined, likely 2nd quarter, 2009) Let me know if you need anything 
else or if these seem incomplete.

Ross Whitaker

• Certain tasks have been removed from the Utah aims because they have been "subsumed by Core 1-2 partners", presumably MGH. Now that the plans have changed so that MGH no longer plans to do this work, this needs to be updated.

Reponse: The work for head tissue classification was based on the paper "MRI Tissue Classification with Neighborhood Statistics: A Nonparametric, Entropy-Minimizing Approach" by Tasdizen et al. in MICCAI 2005. This was implemented in ITK and was the first tissue classification algorithm in the ITK toolkit (as far as we know). The EM-segmenter work, which came out of MIT and Harvard, did not use learned neighborhood statistics, but used probabilistic atlases more effectively. This is now in ITK, and is undergoing continuous improvements and updates. It works sufficiently well that further engineering of the "Neighborhood Statistics" approach, from Utah, is probably not going to have an impact on the DBPs and is not an effective use of NAMIC resources.

Will Schroeder

• Has there been progress in the migration from LONI to batchmake? It is not yet listed as complete in the Isomics timeline.
• Based on the timeline, Kitware has completed its tasks . Some new tasks are listed in the statement of work and should be entered into the timeline.

Response: Yes, there is a progress. We are building a batch processing framework using BatchMake and grid computing tools. Solutions have been demonstrated; however significant work remains to integrate the Slicer modules into the framework in a user-friendly manner.

As far as the timeline: Kitware still has obligations in year 5 including support and software releases. Based on our current contributions--CMake, CDash, CTest, CPack, KWWidgets, ITK, VTK, BatchMake, Slicer (support for GUI, registration, Slicer modules)--there is plenty to do. For us, improving the way these tools work together (e.g., see paragraph above), and producing solid software releases, is a significant work load, especially given reduced funding levels.

Tina Kapur

For the future (assuming and hoping there is a future!) , it would be nice to have publications listed at the end of each relevant section (in addition to the Additional Information links currently provided). Several members of the Center team commented on the lack of details in the progress report and wanted to follow up by reading the relevant publications.

Response: Thanks for the suggestion. We will follow it in the future.