2009 Winter Project Week Command Line Program Testing

From NAMIC Wiki
Revision as of 19:37, 3 January 2009 by Lorensen (talk | contribs) (→‎Status)
Jump to: navigation, search
Home < 2009 Winter Project Week Command Line Program Testing
Return to Project Week Main Page



Key Investigators

  • Bill Lorensen (His Basement)

Objective

There is NA-MIC funded infrastructure in place to regression test the command line programs found in Applications/CLI. Surprisingly, of the over 50 command line programs in the current Slicer3 distribution, only 11 have tests. This project will illustrate the importance of testing and show how easy it is to create tests for command line programs using the existing infrastructure. The project will document the process, create tests and help others to create tests.

Approach, Plan

Using the existing infrastructure, the project will create regression tests for many of the existing command line programs in Applications/CLI. The project will provide expert assistance to those who feel that creating a test is difficult. A Wiki page will provide step-by-step procedures to test the command line programs.

Progress

There is really no excuse for the lack of testing in Applications/CLI. The infrastructure that has been in place for a few years makes it almost painless to test command line programs. Better documentation and peer pressure will help improve the situation.

Tests have been created for all of the command line programs in Applications/CLI (not including sub directories). Three types of tests exist:

  1. --help tests just run the command line program with the --help flag. These tests are as basic as a test can be. All they really test is whether or not the command line program can build, link and report help.
  2. Smoke tests run the program put do not check the results of the program. These are one step above the --help tests and should be the minimum test for each program.
  3. Regression tests run the program and compare the resulting output with a baseline output. Currently only image regression testing is available. Each command line program that produces an image should have a regression test.

Documentation: TODO



Status

Starting status:

  1. 5 (9) regression tests
  2. 31 with no tests
  3. 3 smoke tests


The table shows each command line program and its testing status. Under Test Type, the item in parentheses is a test that has been added as part of this project.

Program Test Type Command Line

Add

Regression 

${TEST_DATA}/CTHeadAxial.nhdr ${TEST_DATA}/CTHeadAxial.nhdr ${TEMP}/AddTest.nhdr

AffineRegistration

None (--help)

BSplineDeformableRegistration

Smoke 

--resampledmovingfilename  ${TEMP}/BSplineDeformableRegistrationTest00.nhdr
--outputtransform ${TEMP}/BSplineDeformableRegistrationTest00Transform.txt 
--default 0
--maximumDeformation 1.0
--constrain
--spatialsamples 10000
--histogrambins 32
--gridSize 5
--iterations 50
${TEST_DATA}/CTHeadAxial.nhdr 
${TEST_DATA}/CTHeadAxial.nhdr

CheckerBoard

Regression 

--checkerPattern 5,3,2
${TEST_DATA}/CTHeadAxial.nhdr
${TEST_DATA}/CTHeadAxialDoubled.nhdr
${TEMP}/CheckerBoardTest.nhdr

ConfidenceConnected

None (Regression) 

--smoothingIterations 1
--timestep 0.0625
--iterations 5
--multiplier 2.5
--neighborhood 1
--labelvalue 2
--seed 9.71622,-8.1,14.2189 --seed -12.9459,-8.1,18.577
${TEST_DATA}/MRHeadResampled.nhdr
${TEMP}/ConfidenceConnectedTest.nhdr

CurvatureAnisotropicDiffusion

None (Regression) 

--conductance 2
--timeStep 0.0625
--iterations 2
${TEST_DATA}/MRHeadResampled.nhdr
${TEMP}/CurvatureAnisotropicDiffusionTest.nhdr

DicomToNrrdConverter

None (Smoke) 

${TEST_DATA}/CTHeadAxialDicom
${TEMP} CTHeadAxialFromDicom.nhdr

DiffusionTensorEstimation

None (--help)

DiffusionTensorMathematics

None (--help)

DiffusionTensorTest

None (--help)

DiffusionWeightedTest

None (--help)

DWIDicomLoad

None (--help)

ExecutionModelTour

None (Smoke) 

--integer 30
--double 30
-f 1.3,2,-14
--string_vector foo,bar,foobar
--enumeration Bill
--boolean1
--transform1 ${TEST_DATA}/ExecutionModelTourTest.mrml\#vtkMRMLLinearTransformNode1
--transform2 ${TEST_DATA}/ExecutionModelTourTest.mrml\#vtkMRMLLinearTransformNode2
${TEST_DATA}/MRHeadResampled.nhdr
${TEST_DATA}/CTHeadAxial.nhdr

FreesurferSurfaceSectionExtraction

None (--help)

GaussianBlurImageFilter

Regression 

--sigma 2.5 ${TEST_DATA}/CTHeadAxial.nhdr ${TEMP}/GaussianBlurImageFilterTest.nhdr

GradientAnisotropicDiffusion

None (Regression) 

--conductance 2
--timeStep 0.0625
--iterations 2
${TEST_DATA}/MRHeadResampled.nhdr
${TEMP}/GradientAnisotropicDiffusionTest.nhdr

GrayscaleFillHoleImageFilter

None (Regression) 

${TEST_DATA}/MRHeadResampled.nhdr
${TEMP}/grayscaleFillHoleTest.nhdr

GrayscaleGrindPeakImageFilter

None (Regression) 

${TEST_DATA}/MRHeadResampled.nhdr
${TEMP}/GrayscaleGrindPeakTest.nhdr

GrayscaleModelMaker

None (Smoke) 

--threshold 300
--name CTFace
--smooth 15
--decimate 0.95
--splitnormals
--pointnormals
${TEST_DATA}/CTHeadAxial.nhdr
${TEMP}GrayscaleModelMakerTest.vtp

HistogramMatching

None (Regression) 

--numberOfHistogramLevels 64
--numberOfMatchPoints 10
${TEST_DATA}/CTHeadAxial.nhdr
${TEST_DATA}/MRHeadResampled.nhdr

ImageLabelCombine

None (--help)

ImageReadDicomWrite

None (Regression) 

--patientName Austrialian
--patientID 8775070
--patientComments "A volunteer"
--studyID 123456
--studyDate 20090102
--studyComments Resampled
--studyDescription None
--modality CT
--manufacturer "GE Medical Systems"
--model CT8800
--seriesNumber 123456
--seriesDescription None
--dicomDirectory ${TEMP}
--dicomPrefix CTHeadAxialDicom
${TEST_DATA}/CTHeadAxial.nhdr

LabelMapSmoothing

None (--help)

LinearRegistration

Smoke 

--resampledmovingfilename  ${TEMP}/LinearRegistrationTest00.nhdr
--outputtransform ${TEMP}/LinearRegistrationTest00Transform.txt 
--initialtransform ${TEST_DATA}/itkAffineTransform00.txt
--spatialsamples 10000
--histogrambins 32
--iterations 50
--learningrate 0.01
--translationscale 100
${TEST_DATA}/CTHeadAxial.nhdr 
${TEST_DATA}/CTHeadAxial.nhdr

MedianImageFilter

Regression 

--neighborhood 1,2,3 ${TEST_DATA}/CTHeadAxial.nhdr ${TEMP}/MedianImageFilterTest.nhdr

ModelMaker

None (Smoke) 

--labels 2
--modelSceneFile ${TEMP}/ModelMakerTest.mrml\#vtkMRMLModelHierarchyNode1
${MRML_TEST_DATA}/helix-roi-lable2.nrrd

MultipleModelsExample

None (--help)

OrientImage

Regression 

--orientation Axial ${MRML_DATA}/fixed.nrrd ${TEMP}/OrientImageTestAxial.nrrd
--orientation Coronal ${MRML_DATA}/fixed.nrrd ${TEMP}/OrientImageTestCoronal.nrrd
--orientation Sagittal ${MRML_DATA}/fixed.nrrd ${TEMP}/OrientImageTestSagittal.nrrd

OtsuThresholdImageFilter

None (--help)

OtsuThresholdSegmentation

None (Regression) 

--brightObjects
--numberOfBins 128
--faceConnected
--minimumObjectSize 1000
${TEST_DATA}/CTHeadResampled.nhdr
${TEMP}/OtsuThresholdImageFilterTest.nhdr

ProbeVolumeWithModel

None (--help)

ResampleVolume2

None (--help)

ResampleVolume

None (--help)

RigidRegistration

None (--help)

Subtract

None (Regression) 

${TEST_DATA}/CTHeadAxial.nhdr
${TEST_DATA}/CTHeadAxialDoubled.nhdr
${TEMP}/SubtractTest.nhdr

TestGridTransformRegistration

Smoke 

--gridSize 5
--outputtransform ${TEMP}/TestGridTransformRegistrationTest00Transform.txt
${TEST_DATA}/CTHeadAxial.nhdr ${TEST_DATA}/CTHeadAxial.nhdr

VotingBinaryHoleFillingImageFilter

None (Regression) 

--radius 3,3,3 
--majorityThreshold 1
--background 0
--foreground 1
${TEST_DATA}/CTHeadResampledOtsuSegmented.nhdr
${TEMP}/VotingBinaryHoleFillingImageFilterTest.nhdr

ZeroCrossingBasedEdgeDetectionImageFilter

None (Regression) 

${TEST_DATA}/CTHeadResampled.nhdr
${TEMP}/ZeroCrossingBasedEdgeDetectionImageFilterTest.nhdr
Retrieved from "https://www.na-mic.org/w/index.php?title=2009_Winter_Project_Week_Command_Line_Program_Testing&oldid=34119"