2013 Project Week Breakout Session:Slicer4Python

Back to Summer project week Agenda

Goals

The material here provides a guided walk through of the resources available for python scripting in Slicer 4. It is based on what is available in the nightly builds as of project week (June 17, 2013, as of Slicer revision 22099).

One goal is to demonstrate the development and testing of a python scripted Slicer module.

Another goal is to provide A Guide to Python in Slicer for the Casual Power User which means that this walkthrough of the features of Slicer should give you an idea how to make a custom module that helps organize and automate your processing. This can be useful for your own research or you can create helper modules that simplify the work for users who are exploring an algorithm or working with a large set of studies.

In this tutorial, we'll make a module that allows you to quickly sequence through a set of volumes. We'll see how to make a custom GUI with a slider, how to implement the volume switching functionality, and how to write a self-test for the module.

Then we'll step back and look at how you could build on what you have learned in order to implement custom solutions to your research problems using Slicer's python interface.

Step-by-Step Scripted Module

Creating a scripted module from templates with ModuleWizard

This topic is covered in the ModuleWizard documentation. It allows you to create a skeleton extension with any combination of modules you want.

Prerequisites

We'll go through the steps on a Mac, but the steps are essentially the same on all platforms. We'll be using a locally built version of Slicer created using these instructions, but the steps can also be performed using a binary download as long as you have a checkout of the Slicer source code available. (From the source code you would need the ModuleWizard script and the extension templates).

Note that this tutorial is a companion to the Hello Python Programming tutorial and the Python Scripting documentation.

Make an Extension

For this example we'll make an extension with a single scripted module as a demonstration. We run the following command from the Slicer source directory.

./Utilities/Scripts/ModuleWizard.py \
          --template ./Extensions/Testing/ScriptedLoadableExtensionTemplate \
          --target ../VolumeTools VolumeTools

We call the extension "VolumeTools" because we will use it to host a demonstration scripted module that scrolls through the volumes available in the current scene.

Make a Scripted Module

Now we want to put a scripted module inside the extension. We can do this with this command (again from the Slicer source directory)

./Utilities/Scripts/ModuleWizard.py \
          --template ./Extensions/Testing/ScriptedLoadableExtensionTemplate/ScriptedLoadableModuleTemplate \
          --target ../VolumeTools/VolumeScroller VolumeScroller

Note that we've used the Module template, which is inside the Extension template and directed the result to go inside of our new extension.

Set up the CMakeLists.txt file

Since the template included a stand-in scripted module, we want to delete it and tell CMake to use our newly created VolumeScroller module instead. We also want to get rid of the dummy module. These commands on unix can do this:

rm -rf ../VolumeTools/ScriptedLoadableModuleTemplate
perl -pi -e 's/ScriptedLoadableModuleTemplate/VolumeScroller/g' ../VolumeTools/CMakeLists.txt

Note that you'll actually want to edit the CMakeLists.txt file by hand, since it contains the metadata about your extension, like the author, category, documentation URL, etc.

Configuring Slicer to use the module

Initial module as created from the template

You are now ready to test your module. The easiest way to do this is by specifying the path on the command line, like this:

./Slicer-build/Slicer --additional-module-paths ../VolumeTools/VolumeScroller

The path above assumes you are in the Slicer-superbuild directory of a local build directory, but you can substitute the appropriate path to start Slicer.

From there, you should find the VolumeScroller module in the Examples category of the module menu. Congratulations! You are now ready to start programming your scripted module.

Notes for "Real World" usage

For the purposes of this tutorial we don't worry about some things. But if you are planning to complete the process of making this into an extension you should refer to the information about how to build the module with CMake, how to bundle the module, and other extension-related topics on the "How To" column on the right of the Slicer developer documentation page. Even though we are making a scripted extension, as of now it is still necessary to use CMake and have a local build tree for at least your initial submission of an extension.

Also we would suggest you start using git for your extension directory from the beginning so that you have a complete log of your development. See github for a free repository hosting site and some links to git resources.

Understanding the Structure of the Scripted Module

Before we start coding, let's take a moment to look through the template. There are just over 300 lines, and much of it is boilerplate that you need only need to make small changes to. Other parts will require complete changes as you go from template to actual functioning module.

The classes defined in the template are as follows. Note that after using the ModuleWizard, the string ScriptedLoadableModuleTemplate" is replaced with "VolumeScroller"

• The ScriptedLoadableModuleTemplate is the hook by which the Slicer module factory recognizes the module and populates the menu and other internal structures. This is where you customize the metadata of your module. Note that the self-test code is also initialized here. The constructor is passed a parent, which is an instance of a qSlicerScriptedLoadableModule. This is not a parent in the QObject sense of the term, but it is the hook that allows you to bind your python code to the corresponding C++ calls.
• The ScriptedLoadableModuleTemplateWidget defines the GUI of the module. The constructor is passed an instance of a qSlicerScriptedLoadableModuleWidget which is a hook to C++.
• ScriptedLoadableModuleTemplateLogic is really just a helper class for you to define with the implementation of your algorithms and any helper functions you need. By convention this class should not reference any GUI elements (no QWidgets defined here).
• ScriptedLoadableModuleTemplateTest is a subclass of a standard python unittest TestCase. Note that this class responds specially to methods whose names start with the string "test", so follow the pattern of the template when adding test functionality.

Basic Development Cycle

Accessing your module via the python console

As a learning experiment, let's try manipulating our widget within the runtime environment. This is a very powerful feature of the python scripted modules in Slicer, and as you develop you'll find yourself using this a lot for debugging and for exploring new features.

The first thing to do is to bring up the python console with the View->Python Interactor menu (or the hotkey Control-3/Command-3).

In the console, you can access the following object:

slicer.modules.VolumeScrollerWidget

which is the instance of our scripted modules widget. Note that this console has tab completion and other nice features. Using this we can access any of the fields of the interface, and even manipulate them. Try some of the following while watching the interface:

b = slicer.modules.VolumeScrollerWidget.applyButton
b.enabled = True
b.down = True
b.down = False
b.clicked()

Note how invoking the 'clicked()' method caused the 'Run the algorithm' message? This is because the clicked method is a signal for the button and it is connected to a python callable that is part of our scripted module.

You'll find that on line 132 of our VolumeScroller.py:

   self.applyButton.connect('clicked(bool)', self.onApplyButton)

which makes the connection to the onApplyButton method on line 142.

Edit / Reload the code

The Reload & Test Collapsible Box

Notice the two big buttons "Reload" and "Reload and Test" buttons. These are useful during your development. Here's what they do:

• Reload:
  • removes the current instance of the modules GUI
  • reloads the python source code
  • re-creates the GUI in place of the old one
  • updates slicer.modules.<moduleName>Widget to point to the new instance
• Reload and Test:
  • performs the Reload
  • invokes whatever test code you have defined

By default, the test will download a sample dataset and confirm that it was loaded. We'll talk more about the contents of the test below.

Reloading

If you click the Reload button nothing much will seem to have happened, just a little flash of the GUI. But this is a very powerful button. If you edit the file VolumeScroller.py you can change the code an reload it. Of course you'd want to use your favorite text editor for this, but as an example, you can run this command from your Slicer directory:

perl -pi -e 's/Parameters/ParametersForMyKillerAlgorithm/g' ../VolumeTools/VolumeScroller/VolumeScroller.py

and when you click the Reload button: Voila! Isn't that just awesome?!? This is just the tip of the iceberg really, since you can completely redefine the GUI incrementally, by adding new widgets, changing the callbacks, updating the layout. And all of it without even needing to exit Slicer.

Advanced note: here we pointed Slicer to the source directory for the scripted module, so that is the code that is reloaded. If you compiled the extension and pointed Slicer to the build tree, then you'd need to edit the .py file from the build tree in order to use the Reload button. You can do this if it's convenient, but don't forget to copy the files back to your source tree or they might be lost.

Refine functionality

At this point it is a "simple matter" of writing the module code to implement your desired functionality.

We won't go through each line in detail, but we'll replace block-by-block to test the functionality at each step. If you get lost while following the editing instructions below, you can find the final version of the script on github.

Let's start by replacing the existing GUI with something that implements the VolumeScroller interface.

Refining the GUI

Let's start by replacing lines 81-148 of the template with the following code and then the interface. Use your favorite editor to open VolumeTools/VolumeScroller/VolumeScroller.py.

    #
    # Volume Scrolling Area
    #
    scrollingCollapsibleButton = ctk.ctkCollapsibleButton()
    scrollingCollapsibleButton.text = "Volume Scrolling"
    self.layout.addWidget(scrollingCollapsibleButton)
    # Layout within the scrolling collapsible button
    scrollingFormLayout = qt.QFormLayout(scrollingCollapsibleButton)

    # volume selection scroller
    self.slider = ctk.ctkSliderWidget()
    self.slider.decimals = 0
    self.slider.enabled = False
    scrollingFormLayout.addRow("Volume", self.slider)

    # refresh button
    self.refreshButton = qt.QPushButton("Refresh")
    scrollingFormLayout.addRow(self.refreshButton)

    # make connections
    self.slider.connect('valueChanged(double)', self.onSliderValueChanged)
    self.refreshButton.connect('clicked()', self.onRefresh)

    # make an instance of the logic for use by the slots
    self.logic = VolumeScrollerLogic()
    # call refresh the slider to set it's initial state
    self.onRefresh()

    # Add vertical spacer
    self.layout.addStretch(1)

  def onSliderValueChanged(self,value):
    self.logic.selectVolume(int(value))

  def onRefresh(self):
    volumeCount = self.logic.volumeCount()
    self.slider.enabled = volumeCount > 0
    self.slider.maximum = volumeCount-1

  def cleanup(self):
    pass

You'll see the new interface. You can try to click on the Refresh button, but you'll see an AttributeError in the python console. This is because we have not yet implemented the logic methods. Let's do that now.

Refining the Logic

For the logic, replace lines 222-239 with the following text:

  def volumeCount(self):
    return len(slicer.util.getNodes('vtkMRML*VolumeNode*'))

  def selectVolume(self,index):
    nodes = slicer.util.getNodes('vtkMRML*VolumeNode*')
    names = nodes.keys()
    names.sort()
    selectionNode = slicer.app.applicationLogic().GetSelectionNode()
    selectionNode.SetReferenceActiveVolumeID( nodes[names[index]].GetID() )
    slicer.app.applicationLogic().PropagateVolumeSelection(0)

Now we can test this manually by adding a bunch of volumes to the scene. Then click the Refresh button and drag the slider.

Advanced notes:

• Instead of using the Refresh button, we could attach the onRefresh callable to update automatically when the MRML scene invokes a vtkMRMLScene::EndBatchProcessEvent, vtkMRMLScene::NodeAddedEvent or vtkMRMLScene::NodeRemovedEvent event.
• The slicer.util.getNode(s) methods rely on the MRML scene allocating node IDs with the class name as the prefix. This is established behavior that you can rely on so that pattern matching constructs can be used as in the example to select nodes of various types.

What did we just do?

Develop self-tests

Scripted module self-tests are very powerful in a number of contexts:

• During development you can use them to automatically load data and invoke your code. This helps you debug and incrementally develop your code in terms of a working test case.
• The CMakeLists.txt of the scripted module template includes the needed directive so that your self test will be part of the ctest testing of your code. This means that when the code is submitted as a Slicer extension, these tests will be run automatically and will be reported on the dashboard. This means that you will be able to confirm correct behavior of your module on different platforms, and your tests will confirm your module is still working as the Slicer core is updated.
• The test is registered with the Testing->Self Tests module so that it can be invoked by users. This means that if you get an error report from a user, you can direct them to run the self test to confirm that the basic operations are working and can narrow down to issues related to their data. Or if the self-test fails, then you can investigate why their OS, graphics hardware, or other specific issue is causing the test to fail for them but pass for you.

To implement a self test for this, let's replace lines 306-309 at the bottom of the file with:

    volumeNode = slicer.util.getNode(pattern="FA")
    logic = VolumeScrollerLogic()
    volumesLogic = slicer.modules.volumes.logic()

    blurLevelCount = 10
    for sigma in range(blurLevelCount):
      self.delayDisplay('Making blurred volume with sigma of %d\n' % sigma)
      outputVolume = volumesLogic.CloneVolume(slicer.mrmlScene, volumeNode, 'blur-%d' % sigma)
      parameters = {
          "inputVolume": slicer.util.getNode('FA'),
          "outputVolume": outputVolume,
          "sigma": sigma,
          }

      blur = slicer.modules.gaussianblurimagefilter
      slicer.cli.run(blur, None, parameters, wait_for_completion=True)

    slicer.modules.VolumeScrollerWidget.onRefresh()
    self.delayDisplay('Selecting original volume')
    slicer.modules.VolumeScrollerWidget.slider.value = 0
    self.delayDisplay('Selecting final volume')
    slicer.modules.VolumeScrollerWidget.slider.value = blurLevelCount

    selectionNode = slicer.app.applicationLogic().GetSelectionNode()
    selectedID = selectionNode.GetActiveVolumeID()
    lastVolumeID = outputVolume.GetID()
    if selectedID != lastVolumeID:
      raise Exception("Volume ID was not selected!\nExpected %s but got %s" % (lastVolumeID, selectedID))

    self.delayDisplay('Test passed!')

This test does a lot of in a small amount of code:

• we run an CLI module over a range of parameter values to populate the scene with volumes
• we test that the GUI is responding to update requests
• we confirm that after running the test the correct volume has been selected

But the thing with testing is that there are an effectively infinite number of possible things to test. So your testing code should be driven by practical considerations:

• What high-level tests can I run that will be most likely to expose faults in the program?
• Can I confirm that I get a correct and consistent result even as I improve the logic or performance of the program?
• Can I detect when changes to other libraries have broken my code?
• Are there external unknowns that I can check for? (like, will my test detect if the user's machine has limited memory or failed network connection?)
• Have users reported odd behavior? If so you should try to capture that use case in a test to narrow down why it happens for them.

As you have learned in this tutorial, the GUI, Logic, and Test classes are all fully programmable and dynamic, so you can easily add additional test buttons with different functionalities to support your interactive code development. You may not want to expose all of these to the end users of your module, but it's good to keep them in the code (and in the CMakeLists.txt) so that you will know if anything breaks later.

Notice that after the test is run the data is still in the mrml scene, so we can use the slider to scroll through the volumes.

Extra considerations

It's worth noting that this entire tutorial was developed from scratch without exiting Slicer. This is one of the huge productivity gains of working with scripted modules and self-tests.

Although this module was written as a demonstration, you might think about how you could build on this for your own work:

• if you have a sequence of scans that you want to move through quickly, you could use the module as-is.
• if you want to explore the parameters of an algorithm that runs slowly, you could modify the self-test to run it and quickly review the results.
• If you have more than one parameter of interest, you could modify the GUI to create sliders to quickly run through pre-computed results at different setting combinations.

The final module GUI and example data after running the self-test and volume rendering one of the blurred images.

Document

If you are developing the Extension with a plan to distribute it via the Extension Manager, then you should carefully follow the Extension Documentation Tutorial.

The Harder Parts

If you've gotten this far you have seen that you can do a lot with this interface. You've seen that the interface can be controlled, and data can be manipulated, but you may also be wondering how you can figure out what is possible and what is not possible.

Automating Slicer

For controlling Slicer, the short answer is that anything you can do with the GUI can be scripted. The best resource for this is to review the tests in the Slicer/Applications/SlicerApp/Testing/Python directory. Each of these is a python scripted module with an elaborate self-test that exercises a large part of the slicer functionality. Many of these tests were created by translating the Slicer end user tutorials into the corresponding python code. These tests are run every night on multiple platforms and the results appear on the Slicer dashboard so that developers will know when something is not working correctly.

Extending Slicer

Sometimes it's not enough to simply automate existing Slicer functionality and you need to create something new. In order to do this, you need to understand something about how the current Slicer code operates and what other tools are available in various toolkits.

For this, there are a number of resources at your disposal. In this section we review the most important python modules available in Slicer and how to get more information about them.

Python

Most of the Python Standard Library is bundled with Slicer. Lots of good stuff there, so try to use it instead of recoding from scratch.

NumPy

NumPy is a very powerful tool for numerics in python, supporting N-dimensional arrays, linear algebra, and related functions. As of this writing, Slicer uses version 1.4.1. Note that it's very easy to get many Slicer data structures as NumPy arrays for manipulation.

fa = array('FA')
b9 = array('blur-9')
(fa-b9).mean()

VTK

Slicer uses a VTK version that is almost exactly 5.10.1 (some fixes are not yet in a VTK release).

VTK is well documented, and this page on the VTK wiki provides a lot of resources including many in Python.

Since much of Slicer's core functionality is written as C++ subclasses of VTK classes, VTK's wrapping is also applied to the Slicer code, making almost everything available in a scripted module.

Qt

Slicer uses PythonQt from MeVis Fraunhofer to expose Qt in Python. Note that this is not PySide or PyQt but for our purposes provides very similar functionality.

Slicer binaries are built with Qt 4.7.4 which is a very capable GUI toolkit, but also has very nicely coded OS abstractions for a wide variety of other features, such as networking and process management. It even contains a full implementation of WebKit. The PythonQt layer makes this available seamlessly in Slicer. Very rarely you may find an aspect of Qt that is only usable from C++ (namely if the feature can only be implemented by subclassing and implementing a virtual method). As we have seen in the tutorial above, the signals, slots, and properties of QObjects are available.

CTK

While Qt is a powerful general purpose tool, CommonTK is a community that extends Qt with features that help us implement medical imaging systems. Slicer developers have been founding members of the CTK effort so you will find that a lot of Slicer code makes use of CTK.

SimpleITK

SimpleITK has been available as an optional build to Slicer for about a year, and as of June 2013 is has been enabled in nightly builds of Slicer. SimpleITK provides direct access to a wide variety of ITK functionality, such as filtering and level set segmentation. Note that registration, meshes, and some other aspects of ITK are not available through SimpleITK.

Learn more about SimpleITK at the SimpleITK in Slicer Breakout Session.

Slicer Modules and CLIs

And of course, when designing new Slicer modules you can re-use much of the functionality directly through Python. For example, we used slicer.modules.volumes.logic to access helper functionality exposed by the Volumes module. And we used slicer.cli.run to request execution of a Slicer command line module.

As you implement extensions and modules, keep in mind that you can be exposing a reusable API that other modules can make use of. Typically this will be in the form of logic methods or self-contained collections of GUI code.

Debugging

The suggested methods to debug your Slicer scripted module is to use the Reload and Test option extensively. Since the interface is under your control, you can add print statements, dialog boxes, or other feedback to help you track what is going on.

By capturing the debugging scenario in a self-test, you can easily recreate the exact steps needed to recreate and investigate the issue.

With the slicer.modules.<ModuleName>Widget attribute, you can access the run-time state of your module in the Python console. From there you can look at the state, invoke methods, and experiment with parameter options.

If you are getting crashes that you cannot debug in Python, you can use standard C++ debugging techniques to generate stack traces and related information.

Optimizing Performance

In general Python code will be efficient as long as you avoid doing low-level operations on large amounts of data. For example, if you are doing pixel manipulations on a volume, then a set of nested Python for loops is likely to be slow. Instead, you should try to map your problem into either a NumPy array operation, or use one or more of the existing VTK or SimpleITK filters to implement the 'heavy lifting'.

If none of these are suitable, you can also provide custom C++ implementations as logic classes or CLI modules and bundle them with your extension. This approach allows you to optimize the core algorithm, while using Python for the interface, testing, and debugging capabilities.

Before jumping into possibly premature optimization, you should investigate what is really taking time in your application. Tools such as instruments for the mac have proven very useful for pinpointing where the bottlenecks really are.

Special Topics

The Slicer Architecture

Much of Slicer coding is essentially the same whether you are using C++ or Python, so you can refer to the Slicer doxygen and other topics on the Slicer developer page for reference.

A few python-specific items are worth noting:

• slicer.mrmlScene gives access to the main vtkMRMLScene instance and can access/manipulate all data
• slicer.app accesses the qSlicerApplication
• slicer.app.layoutManager() returns the qSlicerLayoutManager through which you can access the threeD and slice views.
• slicer.dicomDatabase is the ctkDICOMDatabase used by the DICOM Module
• slicer.util has a set of helper methods

Working with python modules that are not bundled with slicer's binary distribution

If you want to make use of python packages that are not bundled with the distribution, you can often easily install them into the Slicer embedded Python simply by using the Slicer application as if it were a standard Python executable.

For example, the following install pydicom into a binary distribution of Slicer on Mac.

hg clone https://code.google.com/p/pydicom/
cd pydicom/source
~/Desktop/Slicer.app/Contents/MacOS/Slicer setup.py install
open ~/Desktop/Slicer.app/Contents/MacOS/Slicer

Depending on their installation configuration, other packages may not be as easy to install, or they may require that compilers or other dependencies be available on the system.

Additional Resources

Nigel Goh of the University of Western Australia's Step by Step Guides (docx) provide detailed instructions. These are still in development, with the goal of migrating the content to the Slicer wiki.

To learn Python development, look at the Python home page or for interactive on-line learning we suggest Code Academy.