Slicer3:EM

Project Summary

The goal of this project is the creation of interfaces in Slicer 3 that integrate the EMSegment algorithm (Pohl et al.), an automatic segmentation algorithm for medical images that previously existed in Slicer 2. As in Slicer 2, the user is able to adjust the algorithm to a variety of imaging protocols as well as anatomical structures and run the segmenter on large data sets. Thus far, the primary focus has been on developing a wizard-based module that simplifies the configuration of the algorithm by guiding the user through a work flow. Additional interfaces to the algorithm, such as a command line executable and simplified "one-click" Slicer3 module, are planned.

As of May, 2007, the core functionality of the wizard-based EMSegment module is complete and has been checked into the Slicer3 SVN repository. The module was previewed at the December 2006 NAMIC meeting in Clifton Park and demonstrated at the NAMIC All-Hands meeting in Salt Lake City. A tutorial has been developed and is available from this page. Currently, regression tests are being developed, a command line executable version of the module is under development, intensity normalization is being added, and the segmentation results will be validated during the summer 2007.

Contacts

• MIT/BWH: Kilian Pohl (pohl@csail.mit.edu)
• Kitware: Brad Davis (brad.davis@kitware.com)

EMSegment Workflow Module

The EMSegment worfklow module is the primary Slicer3 interface to the EMSegment algorithm. The target audience for this module is someone familiar with brain atlases and tissue labels, not a computer scientist. It allows the user to configure the algorithm---step-by-step---to a variety of imaging protocols and anatomical structures, and then apply the algorithm to segment data. Configuration settings are stored in an EMSegment parameters node in the Slicer3 MRML tree. These settings can be saved and later applied to new data via any of the EMSegment interfaces within Slicer3 or the command-line EMSegment executable.

A similar EMSegment module is available in Slicer2.6. The intent of this project is to implement a Slicer3 module with similar functionality while also improving the graphical user interface (GUI). While the GUI and data structure (MRML) code was completely rewritten for the Slicer3 module, the algorithm code was not modified.

High-level Module Description

The purpose of the module is to configure the algorithm to automatically segment anatomical structures in medical images. First the user has to specify parameters defining the image protocol and the anatomical structures of interests. This process results in a template that the module uses to automatically segment large data sets. The template is composed of atlas data and a non-trivial collection of parameters for the EMSegment algorithm.

Once the parameters are specified, the target images are segmented using the EM Segmentation algorithm (Pohl et al.). If the results are satisfactory, the template is saved and can be used later to segment new images (via the GUI or batch processing). If the results are unsatisfactory, the parameters can be modified and the segmentation re-run.

One important aspect of the project is the workflow wizard. This wizard simplifies the module by dividing the complicated template specification task into a number of smaller, intuitive steps.

Steps in EMSegment Workflow

• 1/8 Define Parameters Set: Select parameter set or create new parameters
• 2/8 Define Hierarchy: Define a hierarchy of anatomical structures
• 3/8 Assign Atlas: Assign atlases for anatomical structures
• 4/8 Select Target Images: Choose the set of images that will be segmented
• 5/8 Specify Intensity Distributions: Define intensity distribution for each anatomical structure
• 6/8 Edit Node-based Parameters: Specify node-based segmentation parameters
• 7/8 Edit Registration Parameters: Specify atlas-to-target registration parameters
• 8/8 Run Segmentation: Save work and apply EM Algorithm to segment target images

Status

A version of the Slicer3 EMSegment workflow module has been completed and checked into the Slicer3 SVN repository. Example data have been packaged with a tutorial and are available from this wiki page. A tutorial session was presented at the January 2007 NAMIC All-Hands meeting by K. Pohl and B. Davis.

While there is a completed working version of the module it will likely be under development for some time. The primary future development efforts will be for (1) bug fixing, (2) adding new functionality, (3) modifying the underlying code to make better use of the evolving Slicer3 base functionality.

Intensity normalization and image registration steps are under construction.

Completed

• finalize workflow description (Kilian, Wendy, Brad)
• define new MRML node structure (Kilian, Brad)
• implement workflow wizard (Sebastien, Luis)
• document workflow wizard and write tutorial (Sebastien)
• implement core MRML classes/attributes (Brad)
• implement logic class that manages MRML nodes and provides API to GUI (Brad)
• implement user interface for each wizard step (Yumin, Sebastien)
• create example data and parameter set (Kilian, Brad)
• port algorithm code from Slicer2 (Brad)
• create tutorial (Kilian, Brad)

In Progress

• bug fixes
• intensity distribution widget
• incorporate feedback from NAMIC community into priorities for future work
• intensity normalization (Kilian, Brad)
• atlas-to-target registration (Brad)
• target-to-target registration (Brad)
• command-line interface (Brad)
• regression testing (Brad)
• segmentation results validation (Sylvain, Brad)
• help text (Kilian, Brad)

Future Work

Prioritized future work for EMSegment Module:

Documentation & Outreach:

• (+1) template library---parameters sets that others can use (Kilian)
  • distribution of information/data/templates
  • link to wiki/web page for data/templates
• ( 0) expand developer documentation (some work critical)
• (-1) expand user tutorial (later)

New Features:

• (+1) one-click segmentation of new images (Brad)
• (+1) class overview widget (Brad)
• (+1) intensity distribution widget (Brad)
• (+1) atlas-to-image registration (Slicer community)
  • coordinate with other Slicer3 developers; Ron, Will, Bill
• (+1) change log-normal to normal (Slicer community)
• (+1) generate surface models (Slicer community)
  • coordinate with other Slicer3 developers; Ron, Will, Steve
• ( 0) controlled vocabulary
  • color---label from controlled set---need specification & use cases
• ( 0) rules for wizard transitions
• ( 0) volume computation
  • coordinate with other Slicer3 developers; Ron, Will, Steve
• (-1) tissue labels carry through to other Slicer3 modules
• (-1) PCA interface
• (-1) CIM interface

Updates to current code:

Usability:

• (+1) rename buttons with more intuitive names: (Kilian & Brad)
  • manual, manually segment, etc.: Edit, Sample Points, Automatic
  • when modifying tissue class tree: add "sub-class" instead of "child node"
• (+1) wizard buttons not scrolled out of view: maybe move to top (Brad)
• (+1) undo/redo (Brad)
• (+1) progress bar (threading?) (Brad)
• (+1) small gui tweaks (Kilian & Brad)

Engineering:

• ( 0) expand updating based on MRML events

Implementation Details

The module is implemented as a programmatic Slicer3 module because it requires a large degree of interaction with the user, the data stored in the MRML tree, and the Slicer3 GUI itself. Because the MRML node structure is rather complicated (for example the anatomical tissue hierarchy and a large number of interdependent nodes) the Logic class is solely responsible for maintaining and accessing these nodes. The Logic class provides an API that the GUI code uses to access and modify data. The Logic class also wraps the algorithm code itself.

EMSegment Command-line Executable

The EMSegment command-line executable is used to simplify the processing of large collections of images. The primary function of the work flow module is to step the user through the process of calibrating, via algorithm parameters, the segmentation algorithm to a particular set of input data. However, once a successful collection of parameters is established, the user will commonly want to bypass this detailed calibration process when segmenting new images. The command-line executable provides this batch processing capability.

The executable is included in the Slicer3 distribution.

Status

A first version of the EMSegmentCommandLine program is complete. We are currently testing the program with users and may modify the interface.

Interface

The interface to the executable is both simple and flexible. It is simple because the number of command-line parameters is minimized---only a MRML scene (containing algorithm parameters), a target image (or multiple target images, e.g., T1 and T2), and an output labelmap image need to be specified. It is flexible because any EM algorithm parameter can be modified, within the MRML scene, via the EMSegment GUI interface.

The interface is under construction and subject to change; here is the current version:

USAGE: /usr/local/dev/build//Slicer3-Shared-Release/bin/EMSegmentCommandLine

                                       [--processinformationaddress
                                       <std::string>] [--xml] [--echo]
                                       [--generateEmptyMRMLSceneAndQuit
                                       <std::string>] [--dontWriteResults]
                                       [--resultStandardVolumeFileName
                                       <std::string>] [--verbose]
                                       [--disableMultithreading]
                                       [--parametersMRMLNodeName
                                       <std::string>]
                                       [--targetVolumeFileNames
                                       <std::vector<std::string>>] ...
                                       [--resultVolumeFileName
                                       <std::string>] [--mrmlSceneFileName
                                       <std::string>] [--] [--version]
                                       [-h]

Where:

  --processinformationaddress <std::string>
    Address of a structure to store process information (progress, abort,
    etc.). (default: 0)

  --xml
    Produce xml description of command line arguments (default: 0)

  --echo
    Echo the command line arguments (default: 0)

  --generateEmptyMRMLSceneAndQuit <std::string>
    Used for testing.  Only write a scene with default mrml parameters.

  --dontWriteResults
    Used for testing.  Don't actually write the resulting labelmap to
    disk. (default: 0)

  --resultStandardVolumeFileName <std::string>
    Used for testing.  Compare segmentation results to this image and
    return EXIT_FAILURE if they do not match.

  --verbose
    Enable verbose output. (default: 0)

  --disableMultithreading
    Disable multithreading. (default: 0)

  --parametersMRMLNodeName <std::string>
    The name of the EMSegment parameters node within the active MRML
    scene.  Leave blank for default.

  --targetVolumeFileNames <std::vector<std::string>>  (accepted multiple
     times)
    File names of target volumes (to be segmented).  The number of target
    images must be equal to the number of target images specified in the
    parameter set, and these images must be spatially aligned.

  --resultVolumeFileName <std::string>
    The file name that the segmentation result volume will be written to.

  --mrmlSceneFileName <std::string>
    Active MRML scene that contains EMSegment algorithm parameters.

  --,  --ignore_rest
    Ignores the rest of the labeled arguments following this flag.

  --version
    Displays version information and exits.

  -h,  --help
    Displays usage information and exits.

  Description: EMSegment command-line module description and
  help.

  Author(s): Brad Davis

  Acknowledgements: Many people and organizations have contributed to the
  funding, design, and development of the EMSegment algorithm and its
  various implementations.

Example

In order to demonstrate the EMSgementCommandLine program, we show how to apply it to the EMSegment tutorial data.

1. Download and build Slicer3 (see the Slicer3 homepage).
2. The EMSegment command-line executable is built in the Slicer3 build directory
   1. On Windows, you will find the executable in (depending on the build type: Debug, Release, RelWithDebInfo) $Slicer3-Build/bin/Debug/EMSegmentCommandLine.exe
   2. On Windows, you will find the executable in $Slicer3-Build/bin/EMSegmentCommandLine
3. Download the EMSegment tutorial data (see below). Suppose that the tutorial data is stored in the directory $EMSegmentTutorial.
4. Run the executable to see the documentation
   1. $Slicer3-Build/bin/EMSegmentCommandLine --help
5. Run the segmentation algorithm on the tutorial data using the tutorial parameters
   1. $Slicer3-Build/bin/EMSegmentCommandLine --mrmlSceneFileName $EMSegmentTutorial/
   2. $Slicer3-Build/bin/EMSegmentCommandLine --mrmlSceneFileName $EMSegmentTutorial/Data/EMSegmentTutorialTemplate.mrml --resultVolumeFileName /tmp/segmentation.mhd --targetVolumeFileNames $EMSegmentTutorial/Data/VolumeData/target_t1_normed.mhd,$EMSegmentTutorial/Data/VolumeData/target_t2_normed.mhd --verbose
6. Run the segmentation algorithm on your data (for now, make sure that your target images are aligned with the atlas data)
   1. $Slicer3-Build/bin/EMSegmentCommandLine --mrmlSceneFileName $EMSegmentTutorial/Data/EMSegmentTutorialTemplate.mrml --resultVolumeFileName /tmp/segmentation.mhd --targetVolumeFileNames /MyImageDirectory/MyImageT1.mhd,/MyImageDirectory/MyImageT2.mhd --verbose
7. Run the segmentation algorithm using an atlas that is aligned to your image data
   1. $Slicer3-Build/bin/EMSegmentCommandLine --mrmlSceneFileName $EMSegmentTutorial/Data/EMSegmentTutorialTemplate.mrml --resultVolumeFileName /tmp/segmentation.mhd --targetVolumeFileNames /MyImageDirectory/MyImageT1.mhd,/MyImageDirectory/MyImageT2.mhd --verbose --atlasVolumeFileNames atlasBackgroundReg_small.mhd,atlasBackgroundReg_small.mhd,atlasCSFReg_small.mhd,atlasGreymatterReg_small.mhd,atlasWhitematterReg_small.mhd
   2. Note that the order of the atlas volumes is important---use the order above for the tutorial dataset
      1. use an order that matches the mrml parameters file if you design a different parameter set

What Can Go Wrong?

• Shared libraries were not found
  • If your run the executable and shared libraries are not found, then you need to add VTK, ITK, KWWidgets, and Slicer3 build library directories to your path. You can also run the executable using Slicer3 as a wrapper. This method will set up the paths correctly; run it like this:

  $Slicer3-Build/bin/Slicer3 --launch $Slicer3-Build/bin/EMSegmentCommandLine --help

• Are the images aligned?
  • The target images must be aligned with the atlas. You should visually check that your target image is aligned with $EMSegmentTutorial/Data/VolumeData/atlas_t1.mhd.

EMSegment Tutorials

2008 Tutorial

These slides and data describe the Slicer3 implementation of the EMSegmenter and demonstrate its use. A recent (7Jan08) version of Slicer3 is required to run the test data.

Slides - Data

Old (2007) Tutorial

NOTE: This version of the tutorial is not compatible with the latest Slicer SVN version. It is compatible with Slicer versions that were released around the Summer Project week 2007. An updated tutorial will be released soon (around the time of the January 2008 All-Hands meeting).

To try out the module with the tutorial data:

1. build the latest version of Slicer3
2. download and untar (or unzip) the data file below (it is approximately 43 megabytes zipped and 329 megabytes unzipped)
3. start Slicer3
4. choose File->Import Scene... and select the file EMSegmentTutorial/Data/EMSegmentTutorialTemplate.mrml from the location where you untared the data
   1. a number of images and MRML nodes will be loaded, this may take some time depending on the speed of you computer and file access
5. in Slicer3 change to the EMSegment module
6. in step 1 of the module make sure that the parameter node "TutorialTemplate" is selected (that is the one that was loaded in the previous step)
7. use the "Next" and "Back" buttons to navigate through the module, viewing and updating parameters as you go
   1. Note: no parameters need to be modified to run the segmentation using the default tutorial settings
8. on step 8/8, select "Run" and the segmentation will start
   1. the segmentation takes approximately 2.5 minutes on a dual core Intel processor with 2 gigabytes of memory
   2. by default the resulting segmentation is saved in the segmentation_result.mhd image

Tutorial: Slides - Data (tgz) - Data (zip)

Screen Shot

EMSegmenter Mini-Tutorials

EMSegmenter Frequently Asked Questions