Revision as of 18:39, 20 October 2010

Home < Projects:ARRA:SlicerQA

Motivation

A core design feature of the Slicer 3 medical image data analysis and visualization platform, cross platform compatibility, enables the widest possible user community to benefit from this valuable research tool. To support this key functionality, all software code in the core and modules (built-in, loadable, scripted, and command line) must be fully tested on each of the supported platforms (Windows, Solaris, Linux 64 and 32-bit, Mac OSX Intel and PowerPC). All documentation and training materials (tutorials, demonstrations, practice data sets. etc) must also be developed and tested on each supported platform.

With the recent release of Slicer 3 (Stable Release version 3.4) the number of available modules that need testing and training materials has substantially increased. This is compounded by the fact that an increasing number of users are contributing new modules to the consortium. Unlike NA-MIC engineers, most members of the user community develop their algorithms and/or workflows on a single software platform, thus never encounter any bugs or other cross platform incompatibilities that must be resolved.

Funding of this supplement will enable the NA-MIC project team to more effectively and rapidly disseminate Slicer modules and capabilities to the user community and to minimize sources of errors in the use of the software.

Research Plan

The new Q/A engineer will work with the team of Core 2 Engineers overseeing Slicer 3 development to devise an efficient protocol for testing each type of module (built-in, loadable, scripted, and command line) on each supported platform. This testing forms a baseline so we can confirm that new versions of the code can be used to accomplish the same tasks. This workflow will be implemented for all existing Slicer 3.4 modules and applied to new modules as they become part of the standard distribution. The testing protocol will leverage the engineering methodologies adopted by the NA-MIC effort, which emphasize automated testing to the extent technically possible, augmented by human spot checking of functionality that cannot be efficiently automated. Specifically, the Slicer 3 effort relies heavily on core NA-MIC Kit components such as VTK and ITK that include extensive automated testing frameworks to ensure that fundamental operations are successfully completed on all supported platforms. These automatically testable operations include: compiling the code, reading and writing files, performing a defined set of image processing operations, etc. For each of these operations the testing system can confirm that a correct result is obtained. For some aspects of the software, no fully automated testing solutions exist; for example, the addition of a new button on a dialog box may make one or more other buttons impossible to access. For this type of functionality, we must rely on human spot checking of core functions as the code is developed. Our approach in these situations is to capture the essential functionality through the tutorials that describe the module. By assuring that the tutorial can be accomplished successfully on all platforms we develop confidence in proper functionality.

The Q/A engineer will then work with the Training Core team (R. Gollub and S. Pujol) to develop new tutorials or extend existing ones to all supported software platforms for each module as soon as it has reached a level of stability and maturity such that the interface and input/output values are not being rapidly revised.

Key Personnel

Stuart Wallace

Progress

Week ending 10/17/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Updating of Journal of Slicer Tests
Updating of Journal of Slicer Workshop Tests
Uploading of most recent Slicer3.6 slide versions for consistency and improved display, and updated logos across wiki pages
- update logos
- improve display
- specific version Slicer3.6 used ( Slicer.3.6.1-2010-08-23 (Windows & Linux64) and Slicer.3.6.1-2010-08-20 (Mac and Linux32))
Continued work on modules from tutorial submissions for Summer 2010 Tutorial Contest
- Fiducials
- RSS
- Longitudinal Lesion Comparison
- Prostate Biopsy
- Stochastic Tractography
- Perk Station
Continued slide revisions to reflect 3.6.1 UI
Preparation for UCSF Training Workshop
- Data Loading, Interactive Editor, and Manual Registration tutorials
Updating of Slicer 3.6 Slicer Test Table
HST DTI course preparation
- Revisions of MIT slides for DTI for Slicer3.6.1
Update of Dicom to Nrrd tutorial for Slicer3.6 with DicomToNrrd module using Dartmouth dataset

Week ending 10/10/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Updating of Journal of Slicer Tests
Updating of Journal of Slicer Workshop Tests
Uploading of most recent slide versions for consistency across wiki pages
Continued work on modules from tutorial submissions for Summer 2010 Tutorial Contest
- Fiducials
Troubleshooting for Robust Statistics Segmentation to 3.6.1 stable (Windows memory error)
Continued slide revisions to reflect 3.6.1 UI
Updating of Slicer build instructions wiki to reflect linux related build error
- svn version related error revolving around python/numpy building
Preparation for UCLA Training Workshop
- Data Loading, Interactive Editor, and Manual Registration tutorials
Porting of DICOM to NRRD module to 3.6.1
- Revision of slides for 3.6.1 UI
Creation and population of Slicer 3.6 Slicer Test Table

Week ending 10/3/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Updating of Journal of Slicer Tests
- Uploading of most recent slide versions for consistency across wiki pages
Continued work on modules from tutorial submissions for Summer 2010 Tutorial Contest
Porting of Atlas Label Fusion Tutorial to 3.6.1 stable
- Mac
- Linux64
- Linux32
- Windows
- Revision of slides to reflect 3.6.1 UI
Porting of Robust Statistics Segmentation to 3.6.1 stable
- Mac
- Linux64
- Linux32
- Windows
- Revision of slides to reflect 3.6.1 UI

Week ending 9/26/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Updating of Journal of Slicer Tests
Continued work on modules from tutorial submissions for Summer 2010 Tutorial Contest
Porting of Atlas Label Fusion Tutorial to 3.6.1 stable
- Mac
- Linux64
- Revision of slides to reflect 3.6.1 UI

Week ending 9/19/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Updating of Journal of Slicer Tests
Finalization of tutorials and slides for Chinese PLA General Hospital, Beijing
- Hello Python
Finalization of tutorials and slides for Beijing 2010 NA-MIC Workshop
- Neurosurgery Data
- Updating of Journal of Slicer Workshop tests
Assistance in preparation of Slicer tutorial for HST 583
Continued work on modules from tutorial submissions for Summer 2010 Tutorial Contest

Week ending 9/12/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Updating of Journal of Slicer Tests
Preparation of tutorials for Chinese PLA General Hospital, Beijing
- Hello Python
Preparation of tutorials for Beijing 2010 NA-MIC Workshop
- Neurosurgery Data
- Updating of Journal of Slicer Workshop tests
Assistance in preparation of Slicer tutorial for HST 583
Continued work on modules from tutorial submissions for Summer 2010 Tutorial Contest

Week ending 9/5/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Further revisions of Freesurfer tutorial slides
Revisions of Neurosurgical planning tutorial slides for 3.6.1 stable release
QA/porting of tutorials into 3.6.1 (stable versions, linux 32 bit)
- Neurosurgical planning
- Freesurfer
- Programming in Slicer (python)
- Diffusion
- Manual Registration
- 3D Visualization
- Interactive Editor
- Change Tracker
Updating of Journal of Slicer Tests
Preparation of tutorials for Chinese PLA General Hospital, Beijing
Preparation of tutorials for Beijing 2010 NA-MIC Workshop
- 3D Visualization Data
- Diffusion Data
- Neurosurgery Data
- Updating of Journal of Slicer Workshop tests
- Assistance in debugging of fiducial seeding module

Week ending 8/29/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Further revisions of Freesurfer tutorial slides
QA/porting of tutorials into 3.6.1 (nightly versions, all OS)
QA/porting of tutorials into 3.6.1 (stable versions, all OS)
- 3Minute
- Freesurfer
- Programming in Slicer (python)
- Programming in Slicer (cxx)
- Diffusion
- Manual Registration
- 3D Visualization
- Interactive Editor
- Change Tracker
Updating of Journal of Slicer Tests
Preparation of tutorials for Beijing 2010 NA-MIC Workshop
- 3D Visualization Data
- Diffusion Data
- Neurosurgery Data
- Updating of Journal of Slicer Workshop tests

Week ending 8/22/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Revision of tutorial slides
- - Freesurfer tutorial
QA/porting of tutorials into 3.6.1 (8-16-2010 version)
- 3Minute
- Freesurfer
- Programming in Slicer (python)
- Programming in Slicer (cxx)
- Diffusion
- Manual Registration
- 3D Visualization
- Interactive Editor
Updating of Journal of Slicer Tests

Week ending 8/15/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Final QA of tutorials for UNC/Duke tutorial event in 3.6.1. release
- Revision of tutorial slides
  - 3D Visualization
  - Interactive Editor
  - Diffusion
  - Manual Registration
QA/porting of other tutorials into 3.6.1
- 3Minute
- Freesurfer
- Programming in Slicer (python)
- Programming in Slicer (cxx)
Updating of Journal of Slicer Tests

Week ending 8/8/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
QA of tutorials for UNC/Duke tutorial event in 3.6.1. release
- Revision of tutorial slides
  - 3D Visualization
  - Interactive Editor
  - Diffusion
  - Manual Registration
QA/porting of other tutorials into 3.6.1
- 3Minute
- Freesurfer
- Programming in Slicer
Updating of Journal of Slicer Tests

Week ending 8/1/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Continued QA of tutorial slides and modules for tutorial submissions for Summer 2010 Tutorial Contest
- Additional testing of modules in OS's not specified in tutorial slides
  - GAMBIT
  - White Matter Lesion Segmentation
  - Automatic SPHARM

Week ending 7/25/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Continued QA of tutorial slides and modules for tutorial submissions for Summer 2010 Tutorial Contest
- Additional testing of modules in OS's not specified in tutorial slides
  - GAMBIT
  - White Matter Lesion Segmentation
  - Automatic SPHARM
Fixed 3.4 tutorials with bad links
Support for competing renewal (tutorial counts, etc.)
Begun porting of other 3.4 tutorials into 3.6
- Freesurfer Tutorial

Week ending 7/18/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Continued QA of tutorial slides and modules for tutorial submissions for Summer 2010 Tutorial Contest
Updating and curation of Slicer Tutorial Journal
Updating and curation of Slicer 3.6 Training wiki
Further testing of Summer 2010 Tutorials
- Update of tutorial completion dates of previously QA'ed tutorials for 3.6
- Additional testing of modules in OS's not specified in tutorial slides
  - Stochastic Tractography
  - Prostate Biopsy
  - Perk Station
  - Labelmap Fusion

Week ending 7/11/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Continued QA of tutorial slides and modules for tutorial submissions for Summer 2010 Tutorial Contest
Updating and curation of Slicer Tutorial Journal
Further testing of Summer 2010 Tutorials
- Update of tutorial completion dates of previously QA'ed tutorials for 3.6
- Additional testing of modules in OS's not specified in tutorial slides

Week ending 7/4/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Further edits of Slicer 3.6 tutorials
- Neurosurgical Planning
Finalized QA of tutorials for St. Louis Workshop
- Neurosurgical Planning
Continued QA of tutorial slides and modules for tutorial submissions for Summer 2010 Tutorial Contest
Updating and curation of Slicer Tutorial Journal
- Addition of Neurosurgical Tutorial QA
- Addition of Summer 2010 Tutorials QA
- Update of tutorial completion dates of previously QA'ed tutorials for 3.6

Week ending 6/27/10

Monitoring of slicer-users and -dev lists
Monitoring of nightly builds
Further edits of Slicer 3.6 tutorials
- Interactive Editor
- Diffusion
Finalized QA of tutorials for St. Louis Workshop
Continued QA of tutorial slides and modules for tutorial submissions for Summer 2010 Tutorial Contest
- Tutorial Contest judging support
- Discussion with tutorial authors for further edits in preparation for submission to 3.6 tutorial wiki page

Week ending 6/20/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
Refinements of QA for Slicer 3.6
Updates of Slicer Tutorial Journal
QA in progress for St. Louis Workshop
QA in progress for tutorial submissions for Summer 2010 Tutorial Contest
Met with Randy and Sonia to finalize preparations for Summer Project Week and tutorial contest.

Week ending 6/13/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
QA for Slicer 3.6 completed on
- Programming in Slicer
- Manual Registration
- Change Tracker
- Interactive Editor
Tutorial Slide Revisions for 3.6 for:
- Diffusion Tutorial
- Programming in Slicer (C++ and Python)
- Manual Registration
- Change Tracker
- Interactive Editor
Update of Slicer Tutorial Journal

Week ending 6/6/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
Revision of Slicer 3.6 QA
QA for Slicer 3.6 completed on Windows 64bit
- Slicer 3 minute
- Slicer Visualization/Data Loading
- Diffusion Tutorial
QA for Slicer 3.6 still in progress for:
- Programming in Slicer
- Manual Registration
- Change Tracker
Tutorial Slide Revisions for 3.6 for:
- Slicer 3 Minute
- Slicer Visualization/Data Loading
- Diffusion Tutorial (in progress)
- Programming in Slicer (in progress)
- Automatic Segmentation (in progress)
- Manual Registration
- Change Tracker
- Editing of Training Portfolio for Summer 2010 Project Week.
More work on tutorial revisions/QA Slicer Tutorial Journal

Week ending 5/30/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
Revision of Slicer 3.6 QA
More work on tutorial revisions/QA Slicer Tutorial Journal
- Work on valgrind scripts with Luis I. for linux and mac installs.
- Editing of Training Portfolio for Summer 2010 Project Week.

Week ending 5/23/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
Revision of Slicer 3.6 QA
More work on tutorial revisions/QA Slicer Tutorial Journal
- Work on valgrind scripts with Luis I. for linux and mac installs.

Week ending 5/16/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
Revision of Slicer 3.6 QA
More work on tutorial revisions/QA Slicer Tutorial Journal
Start of Journal of Slicer Workshop Tests in preparation for Summer 2010 Project Week.

Week ending 5/9/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
Revision of Slicer 3.6 QA
More work on tutorial revisions/QA Slicer Tutorial Journal
Preparation for 2010 Project Week

Week ending 5/2/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
Revision and completion of Slicer 3.6 QA
- T-Con with Luis, Andriy, and Steve to discuss, improve and automate QA procedures for all modules

Week ending 4/25/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
Revision and completion of Slicer 3.6 QA

Week ending 4/18/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
Initiation of Slicer 3.6 QA

Week ending 4/11/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds
Final testing of HelloNewWorld tutorial for Iowa training.

Week ending 4/4/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds

Week ending 3/26/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds

Week ending 3/19/10

Continued monitoring of slicer-users list
Continued monitoring of nightly builds

Week ending 3/12/10

Continued monitoring of slicer-users list
Continued work on AHM extension tutorials
- VMTK Centrelines module on brain CT data
- VMTK Centrelines module on liver CT data
- Robust Statistics Segmentation on liver CT data
Continued adding new tutorials to Slicer 101 page

Week ending 3/5/10

Continued monitoring of slicer-users list
Continued work on AHM extension tutorials
- VMTK Centrelines module on brain CT data
- VMTK Centrelines module on liver CT data
- Robust Statistics Segmentation on liver CT data
Continued adding new tutorials to Slicer 101 page
Continued monitoring of nightly builds
Preparations for Slicer3.6 release relating to extensions/plugins.

Week ending 2/26/10

Continued monitoring of slicer-users list
Continued work on revised 'Hello World' tutorial for python
Continued work on AHM extension tutorials
- VMTK Centrelines module on brain CT data
Started task of adding new tutorials to Slicer 101 page
Continued monitoring of nightly builds
Preparations for Slicer3.6 release relating to extensions/plugins.

Week ending 2/19/10

Continued monitoring of slicer-users list
Continued work on revised 'Hello World' tutorial for python
Continued work on AHM extension tutorials
- Robust Statistics Segmentation module
- VMTK Centrelines module on brain CT data
Continued work on wiki page for Slicer Training Data
Continued monitoring of nightly builds

Week ending 2/12/10

Continued monitoring of slicer-users list
Continued work on revised 'Hello World' tutorial for python
Continued work on AHM extension tutorials
- Robust Statistics Segmentation module
Continued work on wiki page for Slicer Training Data
Continued monitoring of nightly builds

Week ending 2/5/10

Continued monitoring of slicer-users list
Continued work on revised 'Hello World' tutorial for python
Continued work on AHM extension tutorials
- Robust Statistics Segmentation module
- VMTK Centerlines used with liver CT data
Continued work on wiki page for Slicer Training Data
Continued monitoring of nightly builds

Week ending 1/29/10

Continued monitoring of slicer-users list
Continued work on revised 'Hello World' tutorial for python
Continued work on AHM extension tutorials
- Robust Statistics Segmentation module
- VMTK Centerlines used with liver CT data
Created revised wiki page for Slicer Training Data
Continued monitoring of nightly builds

Week ending 1/22/10

More work on tutorial revisions/QA Slicer Tutorial Journal
- Worked on error messages found with DTI Neurosurgery dataset (thank you Alex Y)
Continued monitoring of slicer-users list
Met with Randy and Sonia to plan upcoming Slicer training workshops at HMS and UIowa
Initiated work on revised 'Hello World' tutorial for python
Continued work on AHM extension tutorials
Continued monitoring of nightly builds

Week ending 1/15/10

More work on tutorial revisions/QA Slicer Tutorial Journal
Followup for tasks given at 2010 AHM
Continued monitoring of slicer-users list

Week ending 1/8/10

Creation and initial population of Slicer Tutorial Journal

2010 AHM
- Work on new and revised tutorials for extensions in development
- Worked with Yi Gao to debug Robust Statistics Segmentation algorithm
- Tutorial contest polishing
- QA of DTI Neurosurgery dataset

Continued monitoring of slicer-users list

Continued QA of Slicer 101 tutorials

Week ending 1/1/10

Continued monitoring of slicer-users list.

Continued monitoring of nightly builds and extensions.

Preparations for NAMIC AHM (Robust Statistics Segmentation and Tutorial Polishing).

Week ending 12/25/09

Continued monitoring of slicer-users list.

Continued monitoring of nightly builds and extensions.

Week ending 12/18/09

Continued monitoring of slicer-users list.

Initial work begun for Robust Statistics Segmentation module work/discussion for AHM.

Met with Ron to refine QA procedures.

Continued tutorial testing on nightly builds.

Week ending 12/11/09

Continued monitoring of slicer-users list.

Post RSNA meeting with Randy and Sonia to discuss points of interest for moving forward with training/tutorial materials and AHM work.
- Set up meeting with Yi Gao for AHM to further work with and refine the Robust Statistics Segmentation module. Initiated some work on the modeule that could be done pre meeting.

Week ending 12/04/09

Work with EM Segmenter familiarization

Drafted email contact list for Slicer extensions for wiki

Continued familiarization with newer modules, especially those only available in 3.5 currently.

Continued monitoring of slicer-users email list.

Week ending 11/27/09

Last revision of liver segmentation tutorial for RSNA/CTSA. Used Python Surface Toolbox for improved smoothing results. Also tried out Robust Statistics Segmentation algorithm, but kept Python Sirface Toolbox results for tutorial.

Revised liver segmentation tutorial slides to reflect use of updated Slicer version (3.5) and segmentation techniques.

Finalized handouts for RSNA/CTSA.

Continued monitoring of slicer-users email list.

Week ending 11/20/09

Revision of liver segmentation tutorial for RSNA/CTSA. Used Python Surface Toolbox for improved smoothing results.

Finalized edits to RSNA/CTSA tutorial powerpoints and datasets (RECIST, PET/CT, etc).

Created and revised handouts for RSNA/CTSA.

Continued familiarization with newer modules, especially those only available in 3.5 currently.

Continued monitoring of slicer-users email list.

Helped Steve debug Partners PC specific bug; could not recreated problem on Partners PC with Slicer 3.4 on Win 32.

Week ending 11/13/09

Completed most recent round of smoothing for liver segmentation tutorial for RSNA. Worked with Steve and Ron on this; have both short term and long term solutions for the improvement of the smoothing methods.

Initiated a protocol for QA of nightlies and plug ins with input from Katie. Will be creating new wiki for this task.

Had first EM Segmenter T-Con with Andriy, Kilian, and Sylvain. Ironed out priorities and responsibilities or each of the group members, and will build on this work.

Continued to monitor the Silcer Users mailing list.

Continued familiarization with newer modules, especially those only available in 3.5 currently.

Week ending 11/06/09

Completed most recent round of 2009 RSNA CTSA tutorial and data QA. This includes updated liver segmentation, PET/CT Fusion, RECIST, and Change Tracker datasets. All tested in Slicer 3.5, Windows 32 bit build.
- Some of these also tested in Slicer 3.5 for Linux, also with no problems.

Met with Steve to go over protocol for improved segmentation protocol.

Met with Ron and Katie to discuss QA protocol for nightlies and plug-ins.

Week ending 10/30/09

Continued EM Segementer testing; also set up a monthly t-con with Andriy, Killian P., and Sylvain J. to discuss results of module testing.

Continued 2009 RSNA Conference preparation; Catalyst workshop tutorial dataset and powerpoints tested on Slicer 3.4. Will continue work on segmentation smoothing improvements and their implementation in both 3.4 and 3.5.

Meeting with Ron and Katie regarding QA and upkeep of extensions.

Week ending 10/23/09

EM Segmentation testing, including review and revision of tutorials, bug tracking, and introduction to vervet data. Done with Andriy F.

Followup from SfN 2009 DTI Workshop.

2009 RSNA Conference preparation.

Week ending 10/16/09

Continued work on SFN 2009 DTI Workshop tutorials. Confirmed that the DTI and 3D Visualization datasets worked in Slicer 3.4 stable version across all platforms. Final revisions to tutorial slides completed.

Continued work on Slicer 3.4+ Stochastic Tractography Module (Slicer Daemon).

@@ Line 24: / Line 24: @@
 *Updating of [http://wiki.na-mic.org/Wiki/index.php/Journal_of_Slicer_Tests Journal of Slicer Tests]
 *Updating of [http://www.na-mic.org/Wiki/index.php/Journal_of_Slicer_Workshop_Tests Journal of Slicer Workshop Tests]
 *Uploading of most recent Slicer3.6 slide versions for consistency and improved display, and updated logos across wiki pages
+** update logos
+** improve display
+** specific version Slicer3.6 used ( Slicer.3.6.1-2010-08-23 (Windows & Linux64) and Slicer.3.6.1-2010-08-20 (Mac and Linux32))
 *Continued work on modules from tutorial submissions for [http://www.na-mic.org/Wiki/index.php/Summer_2010_Tutorial_Contest Summer 2010 Tutorial Contest]
 **Fiducials

Difference between revisions of "Projects:ARRA:SlicerQA"