2012 Progress Report Science Wiki Version Engineering

5.2. Engineering

Key Investigators

• Will Schroeder, Kitware
• Stephen R. Aylward, Kitware
• Steve Pieper, Isomics
• Jim Miller, GE Research

The Engineering component of the Computer Science Core (Core 1b) has been focusing on the infrastructure needed for the Algorithms component to implement their methods, and has been working closely with them so that the functionality we provide can serve to inspire new methods as well. Herein we provide more details regarding our accomplishments in those directions

5.2.1. End-user Platform: 3D Slicer

Steve Pieper

Release of 3D Slicer version 4.0 and 4.1

As shown in Figure 1, 3D Slicer version 4.0 (commonly called Slicer4) is having an immediate world-wide impact. Version 4.0, released at the Radiology Society of North America (RSNA) meeting in late November 2011, is the result of a major effort by the NA-MIC engineering cores, in collaboration with the wider Slicer community, to re-implement and streamline the software in response to feedback from NA-MIC DBPs and algorithm developers. A new 3D Slicer version 4.1 release, being finalized at the time of this writing, adds back many of the advanced features of Slicer3, notably the Extension system by which new functionality can be downloaded and installed independent of the main executable. As described more fully below, these sweeping changes required touching not only most of the code in Slicer, but also feeding important feature and bug fix changes back into the rest of the NA-MIC Kit and upstream libraries. As a result of these efforts, 3D Slicer in now an improved reference implementation of a modern medical image computing package and a strong foundation for research.

Figure 1: 3D Slicer version 4.0 end-user downloads during the first 3 months of 2012, a rate of 45,360 per year (over 100 downloads per day). Interactive geolocated download statistics are available at http://download.slicer.org/stats.

Major Developments and New Functionality in Slicer4

• Modern Cross-Platform Design Patterns: As a by-product of the port of the user interface from KWWidgets to Qt, a comprehensive suite of non-GUI OS abstractions and utility functions from Qt became available to support core application functions such as preference settings, multi-processing, application resource data. This was a direct benefit from the GUI port supported by an ARRA supplement to the Neuroimage Analysis Center, a NA-MIC collaborating P41 grant. Slicer4 supports native windows 64 bit environments and can be bundled into a standard Mac OS X application bundle.
• Efficiency and Robustness: Careful review of core data management and processing pipeline steps allowed us to remove much redundant processing. The result is that Slicer is easier for developers to understand and debug, and end users experience faster startup and more responsive behavior.
• DICOM Networking: Slicer4 includes a DICOM listener and DICOM Query/Retrieve capabilities for integration with standard clinical image management environments and workflows. For example, intraprocedural imaging obtained during image guided procedures can now be auto-routed to Slicer for analysis and navigation.
• Additional Features: Slicer4 includes: an improved flexible view layout system; a revised implementation of the Expectation Maximization (EM) Segmenter; faster hardware accelerated volume rendering; improved markups and annotations; improved atlas and model hierarchy support; a streamlined and revised diffusion MRI implementation.

Plans

With the introduction of the Slicer4 Extension system we plan to stabilize the core slicer distribution and move to less frequent releases with more of the algorithm innovation becoming available through as Extensions. This will allow further stabilization and streamlining of the core while speeding up the delivery of new technologies to end users for testing.

5.2.2. Computational Platform

Jim Miller

Efforts in the Computational Platform have focused on developing a general and flexible computing architecture and analysis platform to meet the needs of NA-MIC scientists and engineers as well as the larger medical imaging community.

Major Developments

• Interactive methods: The Editor module has supported interactive segmentation techniques that were deeply integrated with the Editor codebase. We have broadened the Slicer Extension mechanisms to support Editor Extensions, allowing interactive segmentation techniques to be developed separately from the Slice codebase. We have also refined the interaction patterns for interactive segmentation within the Editor and are starting to develop interactive registration techniques.
• Multivolume analysis: The infrastructure for Diffusion Weighted MRI (DWI) IO and visualization has been generalized to be used for other time varying acquisitions like Dynamic Contrast Enhanced MRI (DCE) and Gated Cardiac CT. Massively univariate processing of DCE to produce parametric maps is being developed as a Slicer Extension.
• Distributed computing: Slicer Execution Model modules (also known as Command Line Modules) are now available as Nipype tools, enabling local and distributed scripted execution of processing pipelines.
• Exploratory image analysis: Infrastructure for the interactive exploration of images and the relationships of features calculated over regions of images was developed, including: feature libraries for Gabor, Haralick, entropy, polynomial, and histograms; and charting capabilities to display line, bar, and scatter plots within Slicer.
• Compatibility with ITK version 4: Compatibility with ITK version 4 was developed and continuously maintained over the past year as ITKv4 matured. Slicer will officially switch to ITKv4 in the coming months.

Plans

At the time of this writing, 3D Slicer 4.1 is being finalized. After the release of Slicer 4.1, the Slicer development codebase will migrate to using ITK version 4. This migration will enable new image registration methods, introduce SimpleITK APIs, and introduce GPU support for image analysis algorithms. Other plans for the Computational Platform include further developments for interactive analysis methods, infrastructure for private cloud computing, and interfaces for statistical analysis.

5.2.3. Data Management Platform

Dan Marcus

Efforts in the Data Management Platform have focused on (1) developing an ergonomic user interface and internal networking logical to efficiently exchange data between Slicer and XNAT and (2) expanding Slicer's support for importing from and exporting to local DICOM Objects and networked DICOM PACS.

Major Developments

• User Interface: The XNAT development team is working with a web usability firm (Integrity St. Louis) to design and implement a web interface that can be deployed in Slicer 4's Qt framework for exploring data hosted in a remote XNAT data repository. Prototypes have been implemented and are currently being reviewed by stakeholders with the goal of a functioning interface by June, 2012.
• Networking logic: An alpha implementation of the infrastructure for actually exchanging data between XNAT and Slicer has been developed and revealed that significant additional work is required to handle parsing of MRML files within Slicer in the context of remotely hosted data. This work is now underway and will require several months to complete.
• Use cases: Several use cases have been identified, including projects within the Quantitative Imaging Network (QIN), to drive the development of the Slicer/XNAT integration.
• DICOM-RT and QIN support: Via DCMTK and custom classes, Slicer is being extended to support RT Plans, Images, Annotations, etc. Much of the support is being driving by the head-and-neck cancer core. Additional developments are being driven by an ongoing effort to integrate Slicer as an annotation module in the Quantitative Imaging Network (QIN).
• DICOM database and networking: Slicer DICOM support is approaching clinical quality in terms of speed of searching and IO by maintaining a database that indexes previously loaded DICOM objects. Via this database, it is no longer necessary to parse each object in order to search and/or load an entire series, study, or patient into Slicer.

Plans

The bulk of the effort in the coming year will continue to focus on (1) the user interface and networking logic for remote data management in XNAT and (2) import and export of Slicer results (i.e., entire MRML scenes) as DICOM objects. Regarding XNAT, as the initial development completes, we will turn to developing more efficient mechanisms for caching data locally and synchronizing local caches with remote repositories. Regarding importing and exporting Slicer data as DICOM objects, we are pursuing the concept of a DICOM Lollipop. In a DICOM Lollipop, a complete Slicer (MRML) Scene, with annotations, segmentations, viewing conditions, etc, can be saved as a binary payload in a standard DICOM object. In this manner, Slicer's data can be pushed/pulled from PACS for integration with and sharing across hospital workflows.

5.2.4. Community Software Process

Stephen R. Aylward

The goal of the Community Software Process effort is to provide tools and processes that make it easy for algorithms developers to contribute methods to the NA-MIC Kit, while maintaining the NA-MIC Kit's high-quality software standards.

Major Developments

This year, we have seen massive expansion and significant stabilization of the NA-MIC Kit. While these two accomplishments may seem at odds, they actually represent the planned progression of our community software processes. The specific aims and approaches we pursued to achieve these accomplishments are as follows:

• Provide a modern, stable platform: Slicer 4.0 has been released. This is a major milestone in the NA-MIC community. Slicer 4.0 represents a re-write of the majority of the slicer core to achieve stability, remove redundancy, refactor inefficiencies, and provide new pathways for growth. The most noticeable change is the conversion of the user interface to Qt - which provided speed, stability, and support from the well established Qt community.

• Provide a simple interface for algorithm developers to extend Slicer:
  • Python has been adopted as the preferred scripting language, and it has been tightly integrated into Slicer 4.0. Python is a powerful scripting language with strong scientific computing support via add-on libraries such as SciPy, NumPy, and NiPy. We have tightly integrated it with Slicer so that python scripts can modify and extend the Slicer GUI, manipulate Slicer's data representations (i.e., the MRML Scene), and call other extensions in Slicer to specify novel workflows.
  • Slicer Extension Manager is now the "Slicer Catalog." It is a new web-based system that builds upon the "App Store" concept that is familiar to Android and Apple users. Users can easily install, uninstall, rate, and comment on extensions. Developers can easily add new extensions, upload revisions, add screenshots, and respond to feedback from users.

• Provide access to the best algorithms: ITKv4 has been released by the Insight Software Consortium, and we have begun integrating this new version of ITK and its associated wrapping for Python (i.e., SimpleITK) into Slicer 4.0.

• Provide easy access to clinical data: Upgraded the version of the DICOM library (DCMTK) used by Slicer and provided improved DICOM RT support. We also have begun supporting Ultrasound (e.g., video) and 4D (e.g., gated CT) data in Slicer.

• Provide easy-to-create and easy-to-access documentation: We have integrated the extension writing and the documentation generation processes. The documentation created when an extension is written is now automatically ported to a web host for easier access from within and from outside of Slicer.

Plans

The planned efforts of the Community Software Processes are continuations of ongoing work:

• Provide a modern, stable platform: Slicer quality will continue to be improved via refactoring and via expanded emphasis on testing. Refactoring of the core is nearly complete and new efforts will be directed by the Algorithms team - to support their evolving needs and to inspire future research directions. Testing will evolve from code unit testing to GUI testing. Kitware's ParaView team has developed a semi-automated GUI testing system. It can record and then play-back user interactions. It can also verify that the user interface and Slicer output matches a pre-defined state. In this way, algorithm developers can more easily implement regression tests to ensure the continued operation of their modules.

• Provide a simple interface for algorithm developers to extend Slicer:
  • Python: We will continue to promote Python as the preferred language for scripting in Slicer. It will be used for algorithm prototyping, parameter exploration, and workflow development and delivery. In particular, we expect future development to facilitate scripts that feature interactive algorithms running within Slicers 2D and 3D visualizations.
  • Slicer Catalog: Future work will focus on extending the foundation introduced in Slicer 4.1. New developments will address hosting "extension packages" (e.g., the microscopy package or the DTI package of extensions) as well as hosting data, tutorials, videos, and other adjunct material.

• Provide access to the best algorithms: We will complete integration of ITKv4 and its associated SimpleITK (for python) into Slicer 4. Special attention will be given to ensuring that the upgrade will not disrupt the operation of existing modules.

• Provide easy access to clinical data: The OFFIS and CTK developers anticipate further updated to DCMTK and other clinical data systems used by Slicer. We will continue, in particular, to broaden and stabilize Slicer's support of DICOM, DICOM RT, Ultrasound, video, and 4D data.

5.3. NA-MIC Kit

The NA-MIC Kit is designed to accelerate the pace of research and facilitate clinical evaluation. It provides (a) a flexible yet stable execution and visualization engine with strong support for clinical data (Slicer), (b) methods for extending that platform and sharing those extensions with others, and (c) tools for community software development. The major components of the NA-MIC Kit are illustrated in Fig. 2.

Figure 2. The NA-MIC Kit is a collection of applications, libraries, and processes for bridging algorithm developers, engineers, and clinical researchers. Slicer is the primary vehicle for deploying algorithms. The other components of the NA-MIC Kit addresses the critical yet often overlooked elements of data sharing, integration into clinical workflows, code quality assurance, cross-platform development, and much more.

5.3.1. Maturation

Stephen R. Aylward

As mentioned in Section 5.2., Slicer has undergone massive expansion while also achieving improved stability. The same is true for the NA-MIC Kit as a whole. This evolution represents a maturation of our tools as well as the community. Open-source software, good software practices, and our tools are becoming the leading standards in our fields. Large communities of users and developers are forming around many of the components that we have chosen to use and that we have created for the NA-MIC Kit. The many improvements and new features of Slicer are discussed in Section 5.2. Below we highlights the evolution of the other tools in the NA-MIC:

• CMake 2.8.7 was released with support from NA-MIC. CMake is the foundation of the Slicer build process. It greatly facilitates building Slicer on multiple platforms, using a wide range of support libraries within Slicer, and packaging Slicer for redistribution. CMake is downloaded over 2,000 times per day. It has become the industry standard for cross-platform development, and NA-MIC has played a key role in driving and funding its development.

• CDash 2.0.2 (with CTest) was released with support from NA-MIC. CDash and CTest are responsible for the nightly regression testing of the core code and extensions of Slicer. NA-MIC has driven more of the evolution of this project than nearly any other end-user application. In the CDash releases during this past project cycle, one of the most significant contributions to CDash from NA-MIC was the package upload process. This process allows the many machines that are used to test Slicer every night to upload the executables and packages they create during testing to the main CDash server. This, in turn, allows users to download those testing packages and run additional tests or use them in their research. This complete automation of the test-release cycle is a massive time-saver for the Service core and has greatly reduced the time to discover and resolve bugs and to improve the stability of Slicer. More details on this process is available in a blog at: http://www.kitware.com/blog/home/post/249

• DCMTK 3.6 was released with support from NA-MIC. DCMTK is the DICOM toolkit used in Slicer for local object IO and for networking Slicer with DICOM PACS. This release offers improved support for jpeg compressed DICOM images, for structured reports, for large file support, and for RT objects. Further details are available at http://www.kitware.com/blog/home/post/88

• XNAT 1.5.4 was release with support from NA-MIC. XNAT is an open source imaging informatics platform, developed by the Neuroinformatics Research Group at Washington University. It facilitates common management, productivity, and quality assurance tasks for imaging and associated data. Thanks to its extensibility, XNAT can be used to support a wide range of imaging-based projects. The 1.5.4 release addressed security and DICOM handling as well as improved the overall stability of the system.

• BRAINSFit updated with NA-MIC support. BRAINSFIT is a collection of programs for registering images with with mutual information based metric. Several registration options are given for 3, 6, 9, 12, 16 parameter (i.e. translate, rigid, scale, scale/skew, full affine) based constraints for the registration. The program uses the Slicer execution model framework to define the command line arguments and can be fully integrated with Slicer using the module discovery capabilities of Slicer.

5.3.2. Expansion

The maturation of the foundation of the NA-MIC Kit (discussed in Section 5.3.1) has allowed NA-MIC to pursue new opportunities with less effort and greater confidence. Highlights regarding the expansion of the NA-MIC Kit into new areas include:

• Slicer Catalog: The NA-MIC community was introduced to the Slicer Catalog in Slicer 4.1. This system allows users to install, unstall, search, browse, and rank Slicer extensions. This user experience is available from within Slicer and over the web - much like the Android and Apple App Stores. Developers can contribute, update, document, and post screenshots on their modules and receive community feedback. We see this work as a launchpad for new levels and avenues for community involvement in Slicer.

• CDash Package Manager: We have automated the nightly release of pre-compiled packages for Slicer on multiple platforms. This new process is built on CDash and allows executables and packages created during nightly regression testing to be submitted to a Midas system for download: http://slicer.kitware.com

• CTK is the toolkit NA-MIC created in collaboration with other open-source toolkits (e.g., MITK from the German Cancer Research Center in Heidelberg, XIP from Siemens, GIMIAS from UPF in Spain, and OpenMAF from U of Bologna) to host custom Qt and DCMTK modules for crafting medical applications. CTK now provides several innovative GUI and DICOM elements that specifically save GUI space, user-time, and developer effort in medical applications. Examples of the widgets provided by CTK are discussed in the blog: http://www.kitware.com/blog/home/post/169

• GUI Testing is being offered in a maintenance release to follow Slicer 4.1 in May/June 2012. This work will allow user interactions with Slicer to be recorded on one machine and played back on another - and the results of those interactions can be compared. This GUI testing will be integrated into Slicer's nightly regression testing process. We propose to base the tests on the features demonstrated in the Slicer tutorials.

• DICOM Lollipops are a novel method for embedding entire Slicer (MRML) scenes into a DICOM object. Via this embedding, Slicer data can be read to/from PACS - enabling better integration of Slicer with clinical workflows.

5.3.3. Roadmap

We are proposing to offer quarterly releases of the NA-MIC Kit and Slicer. Highlights from the recent releases and plans for future releases are given next.

• Slicer 4.0: Major Changes (November, 2011)
  • Slicer 4.0 includes a major overhaul of the user interface, improved and simplified workflows for major tasks, simplified procedures for developers, and improved Python support.
  • https://www.slicer.org/wiki/Documentation/4.0/Announcements

• Slicer 4.0.1: Major Changes (January, 2011)
  • Notable changes in Slicer 4.0.1 include new support for Ubuntu 11.04 and Fedora FC 13, and for DWI tractrography modules on Mac OS X. VTK GPU raycast method support of ATI GPU cards for Mac OS X is also included in this release, as well as major improvements to restoring scenes, which now provides significantly faster speeds. Additionally, in Slicer 4.0.1, users can now drag-and-drop files, including volumes, meshes, annotations, etc., into Slicer from Window Explorer, Mac Finder, and Linux Nautilus.
  • https://www.slicer.org/wiki/Slicer4:QtPort/Releases#Slicer_4.0.1

• Slicer 4.1: Major Changes (April, 2012)
  • Charting support has been added with a new chart view in the 4.1 release, which is used by the MultiVolumeExplorer module. This new module introduces multi-volume (e.g. time series) support in Slicer. The Cache Settings panel has been ported from Slicer 3, to provide users with controls to display or clear the available cache space used when downloading sample data, and to store temporary filter outputs. Further support for importing VTK unstructured grids is also a new addition.
  • Several other modules have been updated or added, including the OpenIGTLinks, Welcome, and DICOM modules. The CompareViews and View Controller GUI have been revised and improved. The Modules settings panel has also been enhanced, enabling users to set Prefer Executable CLI loading option to decrease memory consumption by modules and select which module(s) to skip at startup, as well as to customize their Favorite modules toolbar. Many of the icons for the Core Modules have also been updated.
  • http://www.kitware.com/news/home/browse/401

Figure 3. Slicer 4.1 introduced support for 4D images and charting.

• Slicer 4.2: Plans (August, 2012)
  • The next major release of Slicer 4.2 is still in the planning phase, but certain features are likely to be present. In particular, we anticipate the following will be released in Beta/1.0 format in Slicer 4.2:
    • QtTesting
    • DICOM Lollipops
    • ITKv4
    • SimpleITK
  • Additionally, Slicer 4.2 will feature the maturation of two leading technologies:
    • Improved and stabilized Multi-Volume support, e.g., dynamic objects (4D surfaces/meshes)
    • Improved and stabilized Slicer Catalog integration
  • As with prior releases, community involvement is key in determining and providing the final set of features to be included.