AHM 2006:ProjectsSlicerIGSTK

Project

Integration of Slicer 3 and IGSTK

• http://www.slicer.org
• http://www.igstk.org
• http://public.kitware.com/IGSTKWIKI/index.php/Main_Page

Introduction

We will develop the software, Slicer-RFA, to navigate ultrasound-guided RFA therapy with pre-procedural CT images. The software will load the pre-procedural CT images for planning of needle insertion, perform patient-to-image registration using fiducial markers attached to the patients abdomen, and guide ultrasound images and RFA needles using electro-magnetic tracking sensor. The Slicer-RFA will have other support functions specific to RFA ablation: calibration of ultrasound probe and image space to the tracking device, automatic recognition of the fiducial markers in the CT images. The software is unification and extension of Image Guided Surgery Tool Kit (IGSTK) developed by the principal investigator of the proposal and 3D Slicer, prototype of which developed by the co-investigator of the project (Hata), and core software in NA-MIC. We will carefully design the software architecture so that IGSTK and 3D Slicer, are optimally unified. IGSTK will support functionalities specific for image-guided therapy, while Slicer 3.0 from NA-MIC will provide framework for image input/output, image processing, and user interface. We will make the most of novel features of new Slicer 3.0, but also modify those advanced to features to fit in the unique software requirements in RFA.

Goals

Develop an open source software architecture for a liver RFA planning/treatment workstation (Slicer-RFA) that builds on existing open source projects at the two collaborating institutions

Specifically, during this AHM 2006, we will

• Analyze the technical issues involved in integrating Slicer 3 and IGSTK

More specifically,

Before the trip, we will

1. Simply try the feasibility of linking IGSTK to Slicer 2.6
2. Modify MRML to incorporate IGT specific instances (possibly to Data model proposed in Slicer 3.0)
3. Create logger class in Slicer, from itkLogger, and log items in 2). (<- I like this part most).

During the meeting, we will

1. Design the integration of State machine in Slicer (internally and in Data model XML) and make proof-of-concept program
2. Design (centralized?) error and event handling throughout slicer, IGSTK, ITK, VTK
3. Incorporate state machine in Slicer UI (through 3.0's XML-based UI description)

After the meeting, we will

1. Feedback our outcome into the future grant
2. Submission to the Insight Journal
3. Extended work possibly submitted to MICCAI 2006

Architecture

We will develop new software, Slicer-RFA, by extending the Slicer 3.0 and incorporating RFA specific functions in IGSTK. A unique graphical user interface for RFA will be developed in Slicer following its coding and design guideline. Functions unique to Image-guided Therapy will be developed in IGSTK library. The new additions of features in IGSTK are, patient-to-image registration using fiducial markers, support tools for tracking device, calibration of US tracking probe (and images) to the tracker.

We will add IGSTK in list of libraries to be linked to 3D Slicer in its compilation. The IGSTK also shares cross-use the functions offered in the libraries linkied to the 3D Slicer. The IGSTK library will support various OS platform as does Slicer 3.0 and cross-platform compilation tool will be offered to support ease of porting. This approach helps us to develop a 'thin' application layer that is independent of the data and execution implementation to ensure future expandability to other ablation and image-guided therapy.

Identified Issues

Data model

Modification to new Data model for RFA

Data Model is proposed newly in Slicer 3.0 and is a data-centric approach to describe the scene graphics in XML format. The Data Model is implemented independent of the visualization and algorithmic components of the system. We will inheret the Data Model in Slicer-RFA to record the case specific library of image data and registration result. The basic Data Model in Slicer supports instances as,

• Volumes
• Scalar Types
• Label Maps (segmentation result)
• Reference to Lookup Table
• Models
• Named Field Data (scalars, vectors, labels) at points and cells (FreeSurferReaders)
• Color, Clipping State, Visibility, Scalar Visibility, LookupTable
• Transforms*
• Fiducials, Fiducial Lists
• Name, Label#, Diffuse/Ambient/Specular.

The Data Model API in Slicer allows adding, deleting, reading, and modifying medical image data types (Volumes, Models, Transforms, Fiducials, etc).

In addition to the Data Model provided by Slicer, we will develop additional instances required uniquely for RFA.

• State information
• Transoformation matrix for CT-to-patinet registration in the tracker’s coordinate system
• Predicted error from the CT-to-patient registration
• Locations of tracker attached to the RFA applicator and US transducer
• Transformation matrix from calibration of tracker to the US image coordinate system.
• Magnitude and gain of the US imager in the last state of the imaging.
• Location of fiducial markers

Time-coding

We will develop time-coded Data Model secondary to keep the log of the procedures, but primary ensure to recover the safe system recovery in case of system crash. The time-coded logging of the Data model can bring the system back to arbitrary time point. Safe and stable recording of the registration matrices is especially important, since repeating registration procedures is often time-consuming and impractical. In RFA, there are three registration matrices to be created from registration and calibration procedures. They are (1)registration matrix to correlate CT image’s coordinate system to patient’s coordinate system using the tracking devbice; (2) matrix representing the location and orientation of tracking device attached to RFA applicator and US probe; (3) and calibration result to convert image space of ultrasound to tracker. The matrix in (1) is measured measuring the fiducial markers attached to the patient abdomen, and this measurement procedure takes place after the CT imaging and before RFA applicator placement. The matrix (2) is continuously fed into the software through IGSTK’s tarcking data reader. The matrix (3) is measured one before the procedure by imaging specially prepared calibration phantom. To ensure secure recovery from unexpected shutdown or freeze of the system, one may need to keep log of all these registration matrix with time-coding. Such logged matrix can be used to review the procedure for reporting purpose. We will, therefore, extend the Data Model in Slicer 3.0 and develop a time-coded data model for internal recoding and status logging. All the relevant information, including fiducial location, tracking data, registration matrix, and calibration results from the RFA cases are stored in this extended Data Model.

Luis Ibanez's Note This functionality could be added by creating a Logger class in Slicer. This logger will listen to the reports sent by State Machines in IGSTK classes. The logger class derives from the ITK Logger http://www.itk.org/Insight/Doxygen/html/d7/dc0/classitk_1_1Logger.html

Coordinate system manager

We will implement the registration matrices for

• CT-Patient registration
• Tracker's frame
• US-to-tracker calibration

in Slicer Coordinate System Managerframe work.

Slicer Coordinate System Manager (SCSM) is a new feature proposed in Slicer 3.0. In RFA of liver, we will use the managet to provide patient-ct-us-tracking registration.

SCSM is an extension of scene description file formats such as VRML and Slicer's MRML, which offer a framework for describing objects and the coordinate systems they live in. The hierarchy-based transform model of VRML comes from the CAD world, where nested coordinate systems have proven effective for describing mechanical assemblies.

Extension to IGSTK

1. registration algorithms (ICP, point, accuracy assessment)
2. EM tracking module
3. calibration
4. Events and state machine concept in XML

'

Luis Ibanezs note''''' Current implementation of IGSTK doesn't use XML, there are some concerns about safety of XML as a mechanism for describing the State Machines. The testing requirements seem to indicate that a compilation time definition of the State Machines is safer.

The description of the State Machine could easily be exported in XML. It is already exported in "dot" graphviz format and in LTSA format.

[Patrick Cheng's note]
This might be a useful reference for designing our XML scheme for state machine. State Chart XML
Here is another interesting project Finite State Machine Editor written in C++ and Qt. fsme.sourceforge.net
It has three parts:

1. FSME — a Finite State Machine Editor. It is used to design Finite State Machine. FSM description is stored in XML file, which is language and platform independent.
2. FSMC — a Finite State Machine Compiler. It is used to make source code from XML description. By now, generators to C++ and Python exist.
3. FSMD — a Finite State Machine Debugger/Tracer. It is used to monitor Finite State Machines in realtime. Just pipe program to fsmd and inpect everything in graphic.

FSME User Interface

Slicer-RFA

1. (exists in slicer 2.0, but not sure in 3.0) Locator (EM tracking) driven image manipulation
2. (new development) dice similarity measure to assess overlap between two volume in slicer
3. (new development) fiducial marker detection and selection
4. (new development) path planning

Final proposal to Slicer3.0 development team

To be filled after the meeting.

Team Members

• Nobuhiko Hata - BWH
• Steve Pieper - Isomics
• Patrick Cheng - ISIS Center - Georgetown University
• Luis Ibanez - Kitware

Summary Presentation - One Slide

The one-slide summary of this project is available below:

File:2006 AHM Programming Half Week Slicer IGSTK Integration.ppt