2012 Summer Project Week:SelfTesting

From NAMIC Wiki
Jump to: navigation, search
Home < 2012 Winter Project Week:SelfTesting


Key Investigators

  • Isomics: Steve Pieper
  • Kitware: Jean-Christophe Fillion-Robin, Julien Finet, Benjamin Long
  • BWH: Sonia Pujol

Objective

BIST (built-in self-test) is an important concept in delivery of high-reliability engineered systems. While Slicer and the packages in the NA-MIC Kit are fairly well tested by open source software standards, the focus has been almost exclusively on build-time testing. But this leaves out many important variables that can lead to user-level issues. A BIST should be runnable from the installed version of Slicer on the user's machine. Desirable features include:

  • Ensure that packaging and installation have succeeded correctly, taking into account the variability of the user's platform (OS version, memory, graphics card, drivers, etc)
  • Exactly recreate testing steps so that any user-level issues can be narrowed down (e.g. if a tool works on test data, but not on the user's data).
  • Support testing in a continuous loop to look for memory corruption or leaks.

Approach, Plan

The goal at project week is to discuss these issues and formulate a standard plan that can be used by all slicer developers.

Some desirable features of a solution:

  • easy for developers to write tests that capture the full workflow
    • distribution of tests as extensions as part of the debugging and support process
    • integration with QtTesting framework to define tests 'by doing'
  • same tests should be available in CTest and BIST (ideally this will happen automatically)
  • interface features:
    • should allow selection of all tests or a subset of tests
    • should allow running in a loop
    • should present a summary of success/failure
  • (possible feature) dialog when slicer first installs that suggests running the tests
  • (possible feature) integration of performance testing (e.g. render speed)
  • (possible feature) integration with CDash so users can submit test results

Progress

Delivery Mechanism

This work will be delivered to the NA-MIC Kit as a (please select the appropriate options by noting YES against them below)

  1. ITK Module
  2. Slicer Module - YES
    1. Built-in - YES
    2. Extension -- commandline
    3. Extension -- loadable
  3. Other (Please specify)

References