2006 APR NIH Questions and Answers

Diffusion image analysis section

1a. How is the development of the software being coordinated?

Answer: Prototype software development is done at each Core 1 site in a manner that is most efficient for the individual investigator. Once the prototype is tested by one or more DBP investigators on their clinical data, and both Core 1 and DBP investigators believe that it would be beneficial to have a more persistent version of the algorithm implementation available, they invite Core 2 to give their input on the migration of the prototype to the NA-MIC kit. This migration step is typically initiated during the Programming/Project Week events that are held twice a year, in which investigators from all cores are present and weigh in on which software is ready to be migrated to the NA-MIC kit and how the software development should be coordinated. Specifically, for each project, one or more Core 2 investigators join the DBP and Core 1 team and help identify components that already exist in the NA-MIC kit and how those can be reused to efficiently migrate the latest algorithms into the kit.

1b. Why choose one method over another?

Answer: Diffusion Tensor Imaging and its application to understanding medical disorders is a relatively new area of scientific inquiry. Being new, there are no standardized methods or gold standards at this time to guide us in choosing one method over another. In fact, in some instances there is no method available to address a specific biological hypothesis, and therefore software needs to be developed for this purpose. And, even when DTI methods are available, it may not always be clear whether one method should be used or another. For example, an investigator may be interested in characterizing white matter between two gray matter regions. One approach to this problem might be to use tractography methods. The question then is which tractography technique to use, e.g., one based on clustering or stochastic tracking or curve evolution. This can only be answered by comparing the results of two or more methods purported to measure the same thing, using measures of reliability, reproducibility, and accuracy, i.e., which method best characterizes the biology of the specific hypothesis. Also, it may not be the case that “one size fits all,” and thus one method may be better for larger regions of interest while another method might be better for smaller regions of interest, etc. In work we are conducting in this U54, computer scientists from Utah, UNC, GATech, MGH/BWH, and MIT are working on different algorithms to address similar problems defined by the DBPs (e.g., clustering, stochastic tracking, curve evolution to investigate fiber bundles), as well as working on different algorithms to address very different problems (e.g., clustering algorithms to investigate meaningful fiber bundles in schizophrenics and controls versus the development of mode measures to better characterize anisotropic diffusion).

2. Give the biological significance for using a particular software algorithm.

Answer: In this U54, the goal is to have clinical investigators work with computer scientists and engineers to drive the development and implementation of algorithms that will enable clinical investigators to investigate biologically significant questions. Thus the biological significance for any particular algorithm is based on how well it addresses a biological question poised by one of the Core 3 investigators. There is no clear overlap among the software algorithms that necessitates head to head testing for any of the biological questions being addressed by the Core 3 investigators. Rather, the biological signficance of a particular software algorithm will be determined in time by the impact the final published results have on the research community and by the number of new investigators that utilize that algorithm to analyze and publish significant findings in additional data sets.

3. Will Fiber tracker and fiber viewer be integrated with the Slicer DTI module?

Answer: A proof of concept integration between FiberTracking and Slicer 3 (see here for details) was developed during the Summer 2006 Project Week that allows image data to be shared between Slicer 3 and FiberTracking. With the new Slicer 3 architecture, we can distribute FiberTracking/FiberViewer as plugin modules which will closely integrate with the Slicer user environment. Further development will allow us to share more complex image types, such as tensor and vector images, and fiber tract geometry information.

Structural analysis section

1. Why are there five segmentation development methods? Are they all solving the same problems? Do you plan to objectively benchmark their relative performance and later if appropriate correlate their results with clinical outcomes?

Answer: There is not single universal segmentation method that works for all imaging data. In some cases it is possible to use intensity information (e.g., for separating white matter from gray matter), in some cases one can try to use shape and/or anatomical atlas information (e.g., caudate nucleus), and in some other cases heuristics may be employed (e.g., such as those given to us by our Core 3 partners). For oriented domains or on tensor fields such as one finds in DTI, one may need other directional segmentation methods. That being said, it is important to emphasize that these methods may be combined: shape and directional information may be used in the level set framework, statistics may be combined with partial differential equations, and heuristics may be employed to drive Bayesian techniques. We benchmark our methods on clinical data, and some of the research we do concerns the formulation of new validation methods (e.g., using statistics or Laplacian based techniques). Any segmentation methods designed for similar tasks are compared regarding their relative performance on this clinical data. In our pipeline, Core 1 algorithms are given to Core 2 for the development of software, which in turn gives the corresponding application programs to Core 3. Core 3 then provides feedback on the methods to Core 1 in order to refine further the algorithms.

2. Are certain methods better suited to particular DBP problems? If so, how does one choose which method to use for which problem?

Answer: It is quite likely that some methods are better suited to particular DBP problems. For example, if one wants to know about changes anywhere in the brain in one subject population versus another, say between Alzheimer’s patients and normal aging subjects, and the question is a broad one, then correspondingly broad measures such as tissue segmentations, coarse structural segmentations or cortical parcellations are appropriate to address possible differences in the brain between these two populations. On the other hand, if the question is whether certain parts of the striatum are abnormal in patients with schizophrenia versus normal controls, then targeted and precise measurement are needed and thus detailed structural volume or shape measures are better suited. Thus the link between methods and problems needs to be carefully considered based on the kind of question being raised and the type of tool that will best address the specific question. This is why it is important that neuroscientists and computer scientists understand each others research areas sufficiently well to be able to work together in coming up with the best marriage between method and biological problem/question.

3. Related to the previous question, how are the methods being coordinated? Will grid computing be implemented to try out multiple methods on multiple biological problems and how would you form objective conclusions from such measurements?

Answer:

Grid computing is most applicable to fully automatic segmentation methods. Even for algorithms which are able to run unsupervised on populations of subject data, interactive tuning by an expert is typically required beforehand in order to establish a proper protocol. For other algorithms, a basic level of user interaction is necessary for each individual subject. Given this range of requirements, NA-MIC is providing a platform that can accomodate different styles of interactions. Specifically, new algorithms implemented in the NA-MIC Kit are able to execute either as part of the interactive Slicer3 environment or as command line executables. Importantly, it is possible for a user to develop a set of parameters using the interactive system and then apply those to larger populations in an automated fashion; as the populations become increasing large the importance of grid computing correspondingly increases. Another area which we have not yet explored is the applicability of grid computing to the problem of parameter space exploration. In this scenario the task of identifying the appropriate algorithm parameters can be simplified by launching large numbers of grid computing jobs with varying initial condidtions in order to rapidly iterate through the options.

The ability to pose end-user analysis tasks a automated batch programs simplifies the process of comparing various algorithms to find the best solution for a given data set. However it does not address the problem when skilled users are required for interactive parameter tuning. Because expert judgement is often required for to correctly interpret analysis results, our goal is not to provide a kind of 'high score table' to pick the best overall algorithm, but rather to provide a uniform experience for users who wish to try a variety of algorithms and parameter options for a give set of data. Then, when the user has determined the algorithmic approach, to provide the tools that will automate their analysis to the extent possible given the nature of the algorithm.

4. Are the methods driving the biology or is the biology driving the methods, or are some methods being developed independent of the problem?

Answer: In early neuroimaging studies of brain disorders, the method (e.g., CT, MRI analysis techniques) tended to drive the biology because without the method, there was no ability to view or measure the brain. Now, with more tools available for exploring the neurobiology of brain disorders, we are able to use the biology to drive the methods in terms of developing specific methods that will address questions about, for example, the neurobiology of schizophrenia. This is the model we have been aiming at within NAMIC. There is, however, still room for basic method development, not fully bound to a specific biological problem or question. The latter is important because scientific paradigm shifts tend to occur with new developments that don’t initially seem to have a particular application to a disease. So the answer truly is, and should be, that the methods sometimes drive the biology, that the biology sometimes drives the methods, and that method development can be independent of the problem, though the latter is not the focus of this particular U54.

5. What circumstances must be in place before a user moves from manual segmentation to automated segmentation?

Answer: If the given automatic segmentation method gives a very similar result to that of the manual segmentation, but allows the user to perform the segmentation much efficiently in a transparent robust manner, then this would certainly strongly motivate the user to switch from manual segmentation to automatic segmentation. Furthermore, the reliability of automated segmentation methods is higher compared to manual segmentations. Both efficiency and reliability are especially of importance in larger clinical studies, where inter-rater variability and the time and financial needs of manual segmentations are limiting factors. In order to improve reliability and efficiency, there is also a rather large middle ground, namely enhancing efficiency and reliability while necessitating still some user interaction. In this case, the automatic method gives reasonable approximations, which can then be improved upon by the clinical user. If such semi-automatic methods enhance reliabiliy and efficiency then this is a strong-selling point. For example, combining Jim Fallon's neuroanatomical rules with a Bayesian framework has reduced segmentation time by a factor 4 for a key cortical area (area 46) with reliable results that are within the intra-rater variability of the pure manual segmentation method.

6. The registration tools developed by Isomics are not described.

Answer: Slicer Registration Framework (Registration Tools): The registration framework is designed to support translation, rigid, affine, and deformable forms of registration between volumetric image data in Slicer. These tools are an important of the NA-MIC Kit, since linear and nonlinear image registration is a key component for segmentation, morphometric analysis, and data fusion.

Images that are registered by Slicer can have different orientation, for example axial and coronal. This was not available in ITK prior to the NA-MIC development. Users can supply initial transformations, also Slicer provides an interactive steering mechanism that allows user to translate and rotate mages while the registration algorithm is running. Slicer registration module provides different settings for a quick interactive registration as well as for a high-quality but slower registration. Development and refinement of these techniques was performed in cooperation with Core 3 users.

Registration framework is implemented in three levels of Slicer architecture: ITK registration classes, VTK registration wrappers, and Slicer registration module.

ITK registration classes contain ITK registration pipelines. The classes are templated by image type, transformation type, optimizer type, and optimization metric type, which make them easy to extend to a new combination of the types.

VTK wrappers provide the pipeline connections between the VTK input/output images used in Slicer and the ITK registration.

Slicer registration module provides User Interface widgets for settings of optimization/registration parameters for multiple levels of resolution and multiple levels of quality.

The linear and deformable transformations resulting from the Slicer's registration module can be applied to multiple other images by the way of Slicer TransformVolume module. This module is very flexible to support resampling of images at arbitrary resolutions and orientations. Together with the NA-MIC added Volume Export tool, this allows users to resample data through linear and non-linear transforms to make several data sets match at the pixel level. This allows subsequent processing such as segmentation to be performed on data that was originally acquired at differing orientations or resolutions.

Medical researchers from Martha Shenton's PNL Core 3 group at BWH use Slicer registration module on a daily basis in support of their segmentation and DTI research.

7. The report references industry-standard programming style and design. Please elaborate and comment on CMake and Doxygen in this context.

Answer: Doxygen is a documentation generation system that is widely used in the open source community. VTK has been using doxygen for the last ten years. Doxygen was used as the documentation generator for ITK since its beginnings in 2001. Doxygen is especially useful for object-oriented toolkits. CMake, was originally developed by Kitware under the ITK contract with the National Library of Medicine. These tools coninue to play a major role within NA-MIC, supporting the "extreme programming" development process advocated by Core 2.

Functional MRI analysis section

1. What software modules are being used for fMRI? Which of these is being developed?

Answer: fMRI analysis is a relatively new research direction identified by NAMIC last year to be of increasing interest to Core 3 investigators and to Core 1 activities. We are taking a mixed approach of using existing software packages to perform standard analysis of the fMRI data sets, extending capabilities of NAMIC kit to supporting such analysis, and developing novel methods for fMRI analysis and integrating them into NAMIC kit.

Slicer's fMRIEngine and Interval Browser modules are used for individual fMRI studies by Core 3 researchers who are providing user feedback for continuing software development of these two prototype modules. These tools represent important infrastructure for the NAMIC kit as their architecture is designed to incorporate new activation detection and inference algorithms developed by Core 1, in addition to standard GLM-based activation detection. External industry-standard software packages SPM and FSL are used by Core 3 for group analysis and other functionality that the Slicer prototypes do not yet accommodate. Incorporating fMRI analysis into Slicer offers seamless integration of functional, anatomical and DTI data, analysis and visualization.

As part of Slicer, both the fMRIEngine and Interval Browser modules are built on the NAMIC kit foundation; both are distributed in the Slicer2.6 release. New fMRIEngine tutorial, developed in collaboration with Andrew Saykin's BIL Core 3 Group at Dartmouth, is currently being refined for posting on the NAMIC wiki's Slicer Training page (http://wiki.na-mic.org/Wiki/index.php/Slicer:Workshops:User_Training_101). In Slicer's current release, the fMRIEngine can perform first level fMRI analysis using GLM-based activation detection. It currently supports visualization of the resulting parametric maps in combination with morphological and DTI data in a subject's native coordinates, analysis of regions-of-interest and interactive probing for voxel time course and region statistics information.

2. How is external software incorporated into Slicer?

Answer: We have identified two approaches to integrating Slicer with fMRI analysis software. We create compatibility with well established software packages by supporting the respective file formats. Algorithms developed by Core 1 groups are directly incorporated as Slicer modules or ITK classes.

For example, the algorithm for MRF-based regularization of fMRI data as reported in "Wanmei Ou, Polina Golland, From Spatial Regularization to Anatomical Priors in fMRI Analysis. In Proc. IPMI 2005, LNCS 3565:88-100, 2005", was recently incorporated into Slicer fMRIEngine. In this case, Core 1 researchers worked with Core 2 to transfer an understanding of the early implementation, testing and validation of the algorithm. Core 2 researchers then re-implemented the algorithm within Slicer's software paradigm, extending the fMRIEngine processing and visualization pipelines and graphical user interface to incorporate the new functionality. Core 1 and 2 researchers then collaborated to validate the Slicer implementation using the original datasets used to validate the first prototype. In continuing work, the fMRIEngine's help text functionality will also be extended to provide interface-level guidance for users of these tools, and the tutorial will be extended to cover their use as well. These tools will be delivered to the broader community in Slicer's next release.

3. Which software was developed by NAMIC and which was developed elsewhere? Where was it developed?

Answer: The algorithms developed at the Core 1 sites and their Core 2 software implementation are supported by NAMIC.

The fMRIEngine and Interval Browser have also been supported by the Neuroimage Analysis Center at BWH (NIH P41RR013218), the FIRST Biomedical Informatics Research Network (NIH NCRR 5 MOI RR 000827), and the Harvard Center for Neurodegeneration and Repair. All software development for these prototype Slicer modules takes place at BWH.

Core 3 also utilizes external software packages for performing standard fMRI analysis. Of those packages, SPM is developed by the Wellcome Department of Imaging Neuroscience at University College London, and FSL is developed by members of the FMRIB Analysis Group at Oxford University.

4. Describe the type of data for which a given piece of software is being used. Is it providing accurate results?

Answer: Most fMRI data analysis packages provide tools for statistical analysis of time series data from each spatial location (voxel) within the brain for the purpose of detecting activation patterns associated with cognitive or other behavioral stimulation events. Most fMRI software employ a general linear model approach relating the signal time series from each voxel to a vector coding for the expected result or model. Examples of widely used fMRI analysis tools include SPM, FSL, AFNI and Brain Voyager. Previously, Slicer had been primarily oriented toward morphometric analyses and DTI tractography. However, an fMRI Module has been added to facilitate analysis and visualization of structural, functional and diffusion data all within the same toolkit. The specific type of fMRI that can be analyzed with the Slicer fMRI Module includes a time series of volume scans with T2* signal weighting that is sensitive to susceptibility effects, i.e., the so-called blood oxygen level dependent (BOLD) contrast. At present, Slicer can analyze blocked or event related BOLD fMRI time series data on a single case by case basis. Through interoperability with SPM and other packages via standard data formats group statistical results can be visualized using Slicer and additional post-processing can be performed in conjunction with structural and DTI data. The issue of accuracy of fMRI software is an important and interesting problem. In most respects there is not a generally agreed upon “gold standard”. As a preliminary step, we have begun to compare results obtained using Slicer to those obtained using established fMRI packages such as SPM (latest version, SPM5). Initial results with standard fMRI motor tests with clearly established anatomical localization have been encouraging. Larger fMRI data sets and simulations can be used to quantitiate accuracy relative to other methods. In other contexts that are not part of NAMIC at this time, clinical outcomes of presurgical fMRI mapping and results using animal models of focal brain injury can serve as validity tests for the proposed fMRI algorithms and software implementation.

5. The preliminary results of the datasets are not clear, what are the aims?

Answer: The specific aims and hypotheses of the Driving Biological Problems are unchanged from the NAMIC application. The Core 3 Aims were synopsized in the fMRI component of the progress report in a section entitled: “Applications to Functional Brain Activation in Schizophrenia and Related Conditions”. As discussed above, the data sets collected during years one and two are now being prepared for analysis which will continue during year three.

6. Why the focus on dopamine?

Answer: Dopamine plays important roles in the functioning of pre-frontal cortex, and has long been associated with research in schizophrenic dysfunction and treatment effects. (A quick pass through PubMed looking for "dopamine" and "schizophrenia" reveals 765 articles, going back to dopamine antagonism factors in 1975, and currently considering dopamine receptor signalling in schizophrenic patients, dopamine receptor binding as a function of age in schizophrenia, the role of dopamine in learning in schizophrenia, etc.) NAMIC Core 3 researchers have shown that dopamine D1 receptor gene alleles at the Ddel SNP in the first intron are associated with clinical response as well as brain metabolic changes during clozapine treatment (Potkin et al. 2003). D3 receptor alleles (gly9gly) have been shown by our group and others to be associated with risk of tardive dyskinesia (Basile et al. 1999; Lerer et al. 2002). The catabolic enzyme for dopamine, catecholamine-O-methyl-transferase (COMT) has been linked to schizophrenia previously (Weinberger et al. 2001; Egan et al. 2001) and is differentially expressed in the prefrontal cortex over other areas. Given the imaging data also indicates the prefrontal cortex as an area of schizophrenic cognitive dysfunction, the dopaminergic system is considered a key candidate mechanism that cannot be ignored in schizophrenia research.

References: Basile V. S., Masellis, M., Badri, F., et al. (1999). Association of the MscI polymorphism of the dopamine D3 receptor gene with tardive dyskinesia in schizophrenia. Neuropsychopharmacology. 21(1), 17-27.

Egan M. F., Goldberg, T. E., Kolachana, B. S., et al. (2001). Effect of COMT Val108/158 Met genotype on frontal lobe function and risk for schizophrenia. Proc Natl Acad Sci U S A. 98(12), 6917-22.

Lerer B., Segman, R. H., Fangerau, H., et al. (2002). Pharmacogenetics of tardive dyskinesia: combined analysis of 780 patients supports association with dopamine D3 receptor gene Ser9Gly polymorphism. Neuropsychopharmacology. 27(1), 105-19.

Potkin S. G., Basile, V. S., Jin, Y., et al. (2003). D1 receptor alleles predict PET metabolic correlates of clinical response to clozapine. Mol Psychiatry. 8(1), 109-13.

Weinberger D. R., Egan, M. F., Bertolino, A., et al. (2001). Prefrontal neurons and the genetics of schizophrenia. Biol Psychiatry. 50(11), 825-44.

7. Needs citations.

Answer:

During year three, all of the fMRI-related NAMIC components will be analyzing data collected during years one and two and preparing results for dissemination including publications, conferences and training sessions. Citations to all new fMRI publications will be provided in the next progress report. Initial fMRI-related publications are listed below:

Onitsuka, T., Niznikiewicz, M. A., Spencer, K. M., Lucia, L. C., Kikinis, R., Jolesz, F. A., Shenton, M. E., & McCarley, R. W. (2006). Functional and structural deficits in brain regions subserving face perception in schizophrenia. American Journal of Psychiatry, 163, 455-462.

Roth RM, Koven, NS, Randolph JJ, Flashman LA, Pixley HS, Ricketts SM, Wishart HA, Saykin AJ. Functional magnetic resonance imaging of executive control in bipolar disorder. Neuroreport. 17(11):1085-1089, July 31, 2006.

Turner JA, Smyth P, Macciardi F, Fallon JH, Kennedy JL, Potkin SG. Imaging phenotypes and genotypes in schizophrenia. Neuroinformatics. 2006;4(1):21-49.

Wanmei Ou, Polina Golland. From Spatial Regularization to Anatomical Priors in fMRI Analysis. In Proc. IPMI 2005: Information Processing in Medical Imaging, LNCS 3565:88-100, 2005.

Wanmei Ou. fMRI Detection with Spatial Regularization. MIT Master Thesis, May 2005.

NA-MIC Kit section

1. Which parts were specifically developed by NAMIC?

Answer: There were several software tools developed specifically by NAMIC. These include Slicer3, CPack, DART2, KWStyle and adoption of the Wiki as a communication vehicle. While in some cases these products were inspired by previous activities, these were designed and created from the ground up to reflect changes in technology and user requirements.

Slicer3 is the next generation Slicer product. It is the principle delivery platform for NAMIC-developed technologies. Slicer3 has been thoroughly rearchitected to accommodate new requirements from the user community, to address NCBC open source licensing requirements, and to reflect recent changes in computing technologies. Its execution model enables biomedical scientists to readily add new plug-in modules into the Slicer3 framework. Further, these modules can be implemented in a variety of programming languages including different GUIs, communicating over the framework through Slicer3's common data model.

CPack is a novel effort without precedent. The idea behind this project is to create a cross-platform packaging tool for software, ultimately to aid in the dissemination process. In the past, software distributions have been manually constructured for each platform on which they were to run (platform meaning combination of hardware, operating system and compiler). CPack leverages CMake in order to automatically create packages for all platforms. This greatly simplifies and accelerates the distribution of software packages, to the point that it is technically feasible to create a complete distribution on all platforms on a daily basis.

DART2 is the next generation client-side testing server. Its purpose is to aggregate testing data across many distributed testing clients, summarizing the software quality aspects of the project in a concise and informative fashion. While DART2 builds on the experience of the previous DART Classic, it was necessary to redesign and reimplement the testing server because 1) DART Classic was a cobbled collection of code that was hard to maintain, and 2) DART Classic did not support dynamic queries, or temporal/longitudinal analysis of test results. Besides addressing these deficiencies, DART2 adds many other features such as a built-in database server, compression of results, and configurable dashboard display.

KWStyle is a small utility application that enforces programming style on a body of code. It is used as part of the software quality process.

The NAMIC-Wiki is a new vehicle for collaboration, communication and dissemination, having been adopted by the NAMIC community with the inception of the NCBC. While the Wiki technology pre-dates NAMIC, we are making significant changes to the technology to support rapid wiki-to-web publishing and publication database management.

2. Which parts are built upon previous foundations?

Answer: To answer this question, it is necessary to distinguish between open-source tools that NAMIC is using versus those that it is developing. The tools that are used in the NAMIC community generally are open source products developed by groups outside of NAMIC. This includes CVS, Subversion, Doxygen, phpBugTracker, MailMan, and several programming languages such as C++, Tcl, and Python. In general, no NAMIC effort goes into these tools with the exception of filing a rare bug report or providing feedback to the appropriate open source community.

The tools that NAMIC personnel are developing (besides the tools listed in the answer to the NAMIC-Kit Question #1 previously) include VTK, ITK, CMake, CTest, Loni, KWWidgets, and CableSWIG. In some cases significant NAMIC effort has gone into extending these systems. Here are some examples of extensions:

• In ITK, NAMIC has added support for DTI data structures and DTI-related algorithms.
• Several VTK 3D widgets have been created or extended to support NAMIC requirements.
• CMake and CTest have added capabilities due to the complexities of the Slicer software build process, and requirements on cross-platform execution and distribution.
• Several complex, 2D GUI widgets have been created and enhanced for incorporation into the Slicer3 application (i.e., modifications to KWWidgets toolkit). Related training material has also been created to disseminate to the user community.

Despite the fact that these additions were built on an existing foundation, these efforts are often significant in scope and are a reflection of on-going research efforts. For example, while Core 1 pushes DTI-related algorithms forward, changes in ITK must be added to support the implementation and evaluation of these technologies. Similarly, 3D human-computer interaction as represented by VTK's 3D widgets is an ongoing research topic in the visualization community, and augments many medical imaging algorithms (e.g., placing seedpoints for segmentation, or marking fudicials for segmentation).

General Issues

1. Please comment on the status of the human studies. When will analysis of prospective data be conducted?

Answer: At BWH, the IRB is in place at both BWH and the VA to have access to prospective data that is part of other ongoing studies. Accordingly, in addition to the retrospective data that has been uploaded to the wiki site, there is also new data being acquired on both structural and DTI that will be de-identified and made available to U54 investigators. In addition, as the fMRI system has been problematic at BWH, we have focused our efforts on developing some new DTI sequences that will provide U54 investigators with 3T DTI data that is of higher resolution than what they now have access to from the BWH/Harvard site. At Dartmouth, the IRB approval is in place to collect structural, fMRI and DTI data on the same participants at 1.5T and/or 3T and to share these de-identified data with other NAMIC sites and investigators. Progress has been excellent with collection of prospective data on patients with schizophrenia and controls on our 1.5T magnet. During year 3, Dartmouth will move nearly all NAMIC data acquisition to our new NCRR-funded 3T scanner because of its numerous technical advantages and greater accessibility. Dartmouth will also focus on accelerated data analysis during the third year. At UC-Irvine, IRB approvals are in place to collect DTI, fMRI, and genetic data on the same subjects. Licensing issues with MGH are being worked out to allow MGH's DTI sequences to be installed on the UCI scanner so that comparable data can be collected. Analyses have been progressing on legacy datasets as they are available.

Overall, we are beginning to see prospective data acquisitions from the DBP's appear during the last year of their participation in the NCBC effort. They will have to leave the NCBC before there will be time to analyze the data properly in a collaborative fashion, involving all three cores. There is no dedicated path to extend the interaction with the NCBC's. Furthermore, Cores 1 and 2 will have to refocus on the new DBP's, which will have a new set of issues, different from the first generation DBP's. There is a real concern that tremendous opportunities will be missed because of the sequencing of these efforts (see below for more on this topic).

2. Please provide your response to the NAMIC advisory board recommendations

Answer: The NAMIC Advisory Board made three recommendations: (1) the Center should actively pursue methods to distribute their software and other resources to the broader community; (2) the Center should consider additional techniques for quantifying algorithms; and (3) the Center should generate a set of success criteria.

With respect to the first recommendation, the advisory board also noted that while we are doing a good job at distributing our software toolkit, we are also creating a number of additional techniques and tools that would be useful to biomedical researchers and we should consider therefore getting the word out about these other resources. We agree with this observation and we think the development of new image processing algorithms is a particularly strong aspect of the project, and the dissemination of these new algorithms, in the form of open source software, is fundamental to the mission of the Center. Furthermore, the Core 1 investigators are all propopents of this dissemination strategy and are all actively engaged in the process of packaging sofrware and algorithms for consumption by the field. We have specific examples of new algorithms which are available to the community through NAMIC. However, there are practical and managerial considerations. We need to take some care in the release of new software and techniques to the larger research community, because mistakes can undermine the mission of the Center and the field as a whole. The pressure on Core 1 and Core 3 researchers to publish (by their own institutions and NIH) means that while algorithms and scientific results must be demonstrated as valid by the standards of those venues, software used for such publications need not be demonstrated as reliable. Furthermore, software hardening (making reliable) has associated time frames which do not easily fit in to the cycles of generating new, competitive scientific and alogorithmic results. It is a problem of which we are keenly aware. We have begun to address this problem in our Core 2 strategies, which compress the development and testing cycles of new software through the use of new tools for software management and new strategies for software development. In this sense we believe we are at the leading edge in open software development and we believe that our results to date compare favorably with other centers.

Regarding the second recommendation, the advisory board noted that we should consider additional techniques for quantifying algorithms. Here, the advisory board recommended that the Center work on techniques that verify that our algorithms (and software implementations) work on different data sets. We agree with the Board that this is an important topic, and we believe that the dissemination of new validation techniques and data sets is a potentially important contribution of the Center. However, we should not underestimate the diffuculty of this problem. The field of image processing is still not converged on the correct approach for every situation (e.g. real versus phantoms, specific versus general, evaluation criteria), and attempts to systemitize validation data and techniques within the ITK project generated a surprising degree of disagreement among researchers. Despite these hurdles, we are making progress on this issue within NAMIC. Already with some of our segmentation algorithms for cortical and deep brain structures employ data sets from James Fallon and Martha Shenton. Also DTI tractography is being evaluated with data sets from the Shenton and Saykin groups as well as data provided to us by colleagues at BWH (e.g., C-F Westin). Finally we have worked extensively on validation techniques to quantify the accuracy and efficiency of our algorithms. For example, we have proposed and tested a new algorithm for validating segmentation using the Laplace equation. Furthermore, all shape analysis algorithms at the different Core 1 sites have been extensively tested on the same clinical data and compared against each other both qualitatively as well as quantitatively. For validation purposes, additional synthetic test studies with known ground truth are also in development.

Regarding the third recommendation, the advisory board noted that we should generate a set of success criteria for evaluating our progress/success. This is an excellent suggestion and we plan to incorporate it into our Center. One metric is obviously the number of publications, but we are also trying to develop a set of success criteria for much of the work that goes unseen such as the engineering work which is critically important for software to be used by a large number of investigators but which is harder to evaluate in terms of success as much work is not easily translated to paper publications. The problem of evaluating Core 1 success was an agenda item at our spring Core 1 PI meeting at UNC, and that discussion resulted in several very interesting ideas. We are putting together a work group to develop a set of criteria for evaluating our success that can be used both to evaluate the current interactions with the DBPs as well as the next cycle of work to be conducted with new DBPs.

Finally, there was a discussion by the advisory board concerning the tenure of the DBPs, which is 3 years, and the difficulty this poses for DBP researchers who would essentially have to submit proposals within a year or two into the process in order to secure independent funding at the end of the DBP cycle. The advisory board believes that the current mechanism of limiting the DBP time to 3 years is not conducive to successful collaboration, and they recommended that NIH consider improved mechanisms for interaction between the Center and the DBPs. We agree wholeheartedly. Moreover, we think that the work being done now between the computer scientists and the DBPs needs to continue to flourish independently of the Center and this would be facilitated by the DBPs being able to complete the whole 5 years with the other Cores, which would put them in a much better position for independent funding as they would have completed work with the other Cores and be more effective in working with computer scientists and engineers. More thought about how to support continued collaborative work between the initial Core 3 researchers and the U54 team would be helpful, so that Core 3 researchers would be able to continue the kind of work begun in the Center with independent funding beyond their term in the project.

Recommendations for Next Year Report

1. Comment on obstacles and challenges encountered during the year.
2. Comment on outcomes of DBPs.
3. Create a context for the report in the Introduction.
4. Comment on how all the themes as a whole are moving forward.
5. Full citations should be given within the report as footnotes or in a bibliography, not just as hyperlinks.
6. Explain time-lines with text including reasons for delays. Link timelines to papers and citations.
7. Define terms used in the timeline to describe the amount of progress being made, especially with respect to software development.
8. Put priority weights on completed tasks within timeline. Note if a task is a stepping stone or a final goal.
9. Make sure the projects presented in the annual report are also listed in the project pages of the NAMIC Wiki and vice versa.
10. In the budget pages, please elaborate on the number and locations for travel.