Difference between revisions of "NA-MIC-kit-curriculum/Testing-Based Programming/How to run Dynamic Analysis"
(6 intermediate revisions by 2 users not shown) | |||
Line 23: | Line 23: | ||
= Configuring the Project = | = Configuring the Project = | ||
+ | |||
+ | == Basic Configuration == | ||
Rerun CMake and verify that the variable MEMORYCHECK_COMMAND is set properly: | Rerun CMake and verify that the variable MEMORYCHECK_COMMAND is set properly: | ||
Line 35: | Line 37: | ||
* Hit the "c" key to configure | * Hit the "c" key to configure | ||
* Hit the "g" key to generate and quit | * Hit the "g" key to generate and quit | ||
+ | |||
+ | == Additional Flags == | ||
+ | |||
+ | You can fine tune the behavior of Valgrind by setting additional arguments in the CMake variable MEMORYCHECK_COMMAND_OPTIONS | ||
+ | |||
+ | --sim-hints=lax-ioctls | ||
+ | --trace-children=yes | ||
+ | -q | ||
+ | --tool=memcheck | ||
+ | --leak-check=yes | ||
+ | --show-reachable=yes | ||
+ | --num-callers=100 | ||
+ | -v | ||
+ | |||
+ | = Running the Tests = | ||
+ | |||
+ | We present here two methods for running dynamic tests. | ||
+ | |||
+ | Method 1 is best suited for being used with a CDash Dashboard | ||
+ | |||
+ | Method 2 is best suited for being run locally and for tracking specific problems. | ||
+ | |||
+ | If you are having problems with valgrind suppressions, you may need to do the following (see [http://massmail.spl.harvard.edu/public-archives/slicer-devel/2009/003812.html this discussion]): | ||
+ | |||
+ | <pre> | ||
+ | I had to modify the file | ||
+ | |||
+ | Slicer3-lib/VTK/CMake/VTKValgrindSuppressions.supp | ||
+ | |||
+ | by removing the following lines: | ||
+ | |||
+ | 29,35d28 | ||
+ | < { | ||
+ | < <insert a suppression name here> | ||
+ | < core:PThread | ||
+ | < fun:pthread_error | ||
+ | < fun:pthread_attr_setscope | ||
+ | < fun:_ZN16vtkMultiThreader19SingleMethodExecuteEv | ||
+ | < } | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | == Method 1 == | ||
+ | |||
+ | Run the following commands: | ||
+ | |||
+ | cd ${BINARY_DIR} | ||
+ | make ExperimentalStart | ||
+ | make ExperimentalConfigure | ||
+ | make ExperimentalBuild | ||
+ | make ExperimentalTest | ||
+ | make ExperimentalMemCheck | ||
+ | |||
+ | |||
+ | == Method 2 == | ||
+ | |||
+ | Run the command locally: | ||
+ | |||
+ | cd ${BINARY_DIR} | ||
+ | valgrind -v --leak-check=yes TestMain 5 | ||
+ | |||
+ | The final part of the output will look like: | ||
+ | |||
+ | ==27817== HEAP SUMMARY: | ||
+ | ==27817== in use at exit: 0 bytes in 0 blocks | ||
+ | ==27817== total heap usage: 1 allocs, 1 frees, 352 bytes allocated | ||
+ | ==27817== | ||
+ | ==27817== All heap blocks were freed -- no leaks are possible | ||
+ | ==27817== | ||
+ | ==27817== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 19 from 8) | ||
+ | --27817-- | ||
+ | --27817-- used_suppression: 19 dl-hack3-cond-1 | ||
+ | ==27817== | ||
+ | ==27817== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 19 from 8) | ||
+ | |||
+ | = Purposely Introducing an Error = | ||
+ | |||
+ | To get an idea of how a real error will look like, here we introduce an error on purpose. | ||
+ | |||
+ | == Edit main.cxx == | ||
+ | |||
+ | Edit the main.cxx file and add the line: | ||
+ | |||
+ | double * data = new double[200]; | ||
+ | |||
+ | in line 13 of the file. | ||
+ | |||
+ | Compile the project | ||
+ | |||
+ | and run again the valgrind command | ||
+ | |||
+ | cd ${BINARY_DIR} | ||
+ | valgrind -v --leak-check=yes TestMain 5 | ||
+ | |||
+ | This time the output will look similar to: | ||
+ | |||
+ | ==27897== HEAP SUMMARY: | ||
+ | ==27897== in use at exit: 1,600 bytes in 1 blocks | ||
+ | ==27897== total heap usage: 2 allocs, 1 frees, 1,952 bytes allocated | ||
+ | ==27897== | ||
+ | ==27897== Searching for pointers to 1 not-freed blocks | ||
+ | ==27897== Checked 100,724 bytes | ||
+ | ==27897== | ||
+ | ==27897== 1,600 bytes in 1 blocks are definitely lost in loss record 1 of 1 | ||
+ | ==27897== at 0x4025024: operator new[](unsigned int) (vg_replace_malloc.c:258) | ||
+ | ==27897== by 0x8048E51: main (main.cxx:12) | ||
+ | ==27897== | ||
+ | ==27897== LEAK SUMMARY: | ||
+ | ==27897== definitely lost: 1,600 bytes in 1 blocks | ||
+ | ==27897== indirectly lost: 0 bytes in 0 blocks | ||
+ | ==27897== possibly lost: 0 bytes in 0 blocks | ||
+ | ==27897== still reachable: 0 bytes in 0 blocks | ||
+ | ==27897== suppressed: 0 bytes in 0 blocks | ||
+ | ==27897== | ||
+ | ==27897== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 19 from 8) | ||
+ | --27897-- | ||
+ | --27897-- used_suppression: 19 dl-hack3-cond-1 | ||
+ | ==27897== | ||
+ | ==27897== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 19 from 8) | ||
+ | |||
+ | = Running Dynamic Analysis for Slicer tests = | ||
+ | |||
+ | First, find the exact command line that is used to launch the test. Say, we want to look at N4ITKBiasFieldCorrection. | ||
+ | |||
+ | ctest -V -R N4ITKBiasFieldCorrection | ||
+ | test 163 | ||
+ | Start 163: N4ITKBiasFieldCorrection_Full | ||
+ | 163: Test command: /workspace/fedorov/Slicer/Debug/Slicer3-build/Slicer3 --launch N4ITKBiasFieldCorrectionTest \ | ||
+ | ModuleEntryPoint --inputimage /workspace/fedorov/Slicer/Debug/Slicer3/Testing/Data/Input/he3volume.nii.gz \ | ||
+ | --maskimage /workspace/fedorov/Slicer/Debug/Slicer3/Testing/Data/Input/he3mask.nii.gz \ | ||
+ | --outputimage /workspace/fedorov/Slicer/Debug/Slicer3-build/Testing/Temporary/he3corrected.nii.gz \ | ||
+ | --outputbiasfield /workspace/fedorov/Slicer/Debug/Slicer3-build/Testing/Temporary/he3biasfield.nii.gz | ||
+ | |||
+ | Next, from Slicer3-build, you can use the convenience script to run the dynamic analysis: | ||
+ | |||
+ | ./Scripts/runValgrindForOneModule.sh ./bin/N4ITKBiasFieldCorrectionTest \ | ||
+ | ModuleEntryPoint --inputimage /workspace/fedorov/Slicer/Debug/Slicer3/Testing/Data/Input/he3volume.nii.gz \ | ||
+ | --maskimage /workspace/fedorov/Slicer/Debug/Slicer3/Testing/Data/Input/he3mask.nii.gz \ | ||
+ | --outputimage /workspace/fedorov/Slicer/Debug/Slicer3-build/Testing/Temporary/he3corrected.nii.gz \ | ||
+ | --outputbiasfield /workspace/fedorov/Slicer/Debug/Slicer3-build/Testing/Temporary/he3biasfield.nii.gz | ||
+ | |||
+ | Note, that we specify the full path to the test located in bin, and the command line parameters exactly as they were provided by ctest output. |
Latest revision as of 16:51, 20 May 2010
Home < NA-MIC-kit-curriculum < Testing-Based Programming < How to run Dynamic AnalysisDynamic Analysis focuses on detecting defects at run time, particularly: Uninitialized variables and Memory leaks.
Contents
Introduction
This tutorial illustrates how to run dynamic analysis in a small project.
Requisites
You should have completed first the tutorials:
Installing Valgrind
Valgrind is the applications that will check your tests at run time and will report their defects.
In Ubuntu and Debian GNU/Linux you can install Valgrind by doing
sudo apt-get install valgrind
Configuring the Project
Basic Configuration
Rerun CMake and verify that the variable MEMORYCHECK_COMMAND is set properly:
cd ${BINARY_DIR} make edit_cache
- Hit the "t" key to go to the advanced mode
- Hit the "/" key to search for MEMORYCHECK_COMMAND
- Verify that it points to the valgrind executable that you installed
- Hit the "c" key to configure
- Hit the "g" key to generate and quit
Additional Flags
You can fine tune the behavior of Valgrind by setting additional arguments in the CMake variable MEMORYCHECK_COMMAND_OPTIONS
--sim-hints=lax-ioctls --trace-children=yes -q --tool=memcheck --leak-check=yes --show-reachable=yes --num-callers=100 -v
Running the Tests
We present here two methods for running dynamic tests.
Method 1 is best suited for being used with a CDash Dashboard
Method 2 is best suited for being run locally and for tracking specific problems.
If you are having problems with valgrind suppressions, you may need to do the following (see this discussion):
I had to modify the file Slicer3-lib/VTK/CMake/VTKValgrindSuppressions.supp by removing the following lines: 29,35d28 < { < <insert a suppression name here> < core:PThread < fun:pthread_error < fun:pthread_attr_setscope < fun:_ZN16vtkMultiThreader19SingleMethodExecuteEv < }
Method 1
Run the following commands:
cd ${BINARY_DIR} make ExperimentalStart make ExperimentalConfigure make ExperimentalBuild make ExperimentalTest make ExperimentalMemCheck
Method 2
Run the command locally:
cd ${BINARY_DIR} valgrind -v --leak-check=yes TestMain 5
The final part of the output will look like:
==27817== HEAP SUMMARY: ==27817== in use at exit: 0 bytes in 0 blocks ==27817== total heap usage: 1 allocs, 1 frees, 352 bytes allocated ==27817== ==27817== All heap blocks were freed -- no leaks are possible ==27817== ==27817== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 19 from 8) --27817-- --27817-- used_suppression: 19 dl-hack3-cond-1 ==27817== ==27817== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 19 from 8)
Purposely Introducing an Error
To get an idea of how a real error will look like, here we introduce an error on purpose.
Edit main.cxx
Edit the main.cxx file and add the line:
double * data = new double[200];
in line 13 of the file.
Compile the project
and run again the valgrind command
cd ${BINARY_DIR} valgrind -v --leak-check=yes TestMain 5
This time the output will look similar to:
==27897== HEAP SUMMARY: ==27897== in use at exit: 1,600 bytes in 1 blocks ==27897== total heap usage: 2 allocs, 1 frees, 1,952 bytes allocated ==27897== ==27897== Searching for pointers to 1 not-freed blocks ==27897== Checked 100,724 bytes ==27897== ==27897== 1,600 bytes in 1 blocks are definitely lost in loss record 1 of 1 ==27897== at 0x4025024: operator new[](unsigned int) (vg_replace_malloc.c:258) ==27897== by 0x8048E51: main (main.cxx:12) ==27897== ==27897== LEAK SUMMARY: ==27897== definitely lost: 1,600 bytes in 1 blocks ==27897== indirectly lost: 0 bytes in 0 blocks ==27897== possibly lost: 0 bytes in 0 blocks ==27897== still reachable: 0 bytes in 0 blocks ==27897== suppressed: 0 bytes in 0 blocks ==27897== ==27897== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 19 from 8) --27897-- --27897-- used_suppression: 19 dl-hack3-cond-1 ==27897== ==27897== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 19 from 8)
Running Dynamic Analysis for Slicer tests
First, find the exact command line that is used to launch the test. Say, we want to look at N4ITKBiasFieldCorrection.
ctest -V -R N4ITKBiasFieldCorrection test 163 Start 163: N4ITKBiasFieldCorrection_Full 163: Test command: /workspace/fedorov/Slicer/Debug/Slicer3-build/Slicer3 --launch N4ITKBiasFieldCorrectionTest \ ModuleEntryPoint --inputimage /workspace/fedorov/Slicer/Debug/Slicer3/Testing/Data/Input/he3volume.nii.gz \ --maskimage /workspace/fedorov/Slicer/Debug/Slicer3/Testing/Data/Input/he3mask.nii.gz \ --outputimage /workspace/fedorov/Slicer/Debug/Slicer3-build/Testing/Temporary/he3corrected.nii.gz \ --outputbiasfield /workspace/fedorov/Slicer/Debug/Slicer3-build/Testing/Temporary/he3biasfield.nii.gz
Next, from Slicer3-build, you can use the convenience script to run the dynamic analysis:
./Scripts/runValgrindForOneModule.sh ./bin/N4ITKBiasFieldCorrectionTest \ ModuleEntryPoint --inputimage /workspace/fedorov/Slicer/Debug/Slicer3/Testing/Data/Input/he3volume.nii.gz \ --maskimage /workspace/fedorov/Slicer/Debug/Slicer3/Testing/Data/Input/he3mask.nii.gz \ --outputimage /workspace/fedorov/Slicer/Debug/Slicer3-build/Testing/Temporary/he3corrected.nii.gz \ --outputbiasfield /workspace/fedorov/Slicer/Debug/Slicer3-build/Testing/Temporary/he3biasfield.nii.gz
Note, that we specify the full path to the test located in bin, and the command line parameters exactly as they were provided by ctest output.