NA-MIC-kit-curriculum/Testing-Based Programming

Introduction

Testing-Based Programming is an software development methodology that relies on continuous and exhaustive testing as a scaffolding for facilitating rapid development.

The principle is very simple:

If you have an extensive test-suite that exercises all the capabilities of your software, you can make both swift and large scale changes to the code, and rely on the testing-suite to reveal immediately if any bug has been introduced by the changes.

In the absence of a testing-suite, every change in the code has the potential of introducing new defects that will go unnoticed for a long time.

As an illustration of what happens in a world without sufficient Testing:

The national annual costs of an inadequate infrastructure for software testing is estimated to range from $22.2 to $59.5 billion (0.06% of GDP). Over half of these costs are borne by software users in the form of error avoidance and mitigation activities.

"The Economic Impacts of Inadequate Infrastructure for Software Testing" NIST Report

As a reference the annual budget of NIH is $30 billion.

Testing Practices

This section describes the practices of a Testing-based software development methodology.

1. Write tests before writing the code
2. Test Continuously
3. Test Everything
   • No feature is small enough to not deserve a test
4. Compute Code Coverage
   • Number of lines of code exercised during testing
5. Perform Dynamic Analysis
   • Checks for Memory leaks
   • Checks for Uninitialized variables

How do do Testing

Tutorials

• How to add Tests in CMake
• How to run Code Coverage
• How to run Dynamic Analysis

Reference Material

• Testing Software with CTest
• Slicer3 Code Coverage
• 2009 Winter Project Week Command Line Program Testing