3.4 Testing
3.4.4 Constructing Unit Tests and Writing Test Functions
Python has several frameworks for automatically running and checking a potentially very large number of tests for parts of your software by one command. This is an extremely useful feature during program development: whenever you have done some changes to one or more files, launch the test command and make sure nothing is broken because of your edits.
The test frameworksnoseandpy.testare particularly attractive as they are very easy to use. Tests are placed in specialtest functionsthat the frameworks can recognize and run for you. The requirements to a test function are simple:
the name must start withtest_
the test function cannot have any arguments
the tests inside test functions must be boolean expressions
a boolean expressionbmust be tested with assert b, msg, wheremsgis an optional object (string or number) to be written out whenbis false
Suppose we have written a function
def add(a, b):
return a + b
A corresponding test function can then be
def test_add() expected = 1 + 1 computed = add(1, 1)
assert computed == expected, ’1+1=%g’ % computed
Test functions can be in any program file or in separate files, typically with names starting withtest. You can also collect tests in subdirectories: runningpy.test -s -vwill actually run all tests in alltest*.pyfiles in all subdirectories, while nosetests -s -vrestricts the attention to subdirectories whose names start with testor end with_testor_tests.
As long as we add integers, the equality test in thetest_addfunction is appro- priate, but if we try to calladd(0.1, 0.2)instead, we will face the rounding error problems explained in Sect.3.4.3, and we must use a test with tolerance instead:
def test_add() expected = 0.3
computed = add(0.1, 0.2) tol = 1E-14
diff = abs(expected - computed) assert diff < tol, ’diff=%g’ % diff
Below we shall write test functions for each of the three test procedures we suggested: comparison with hand calculations, checking problems that can be ex- actly solved, and checking convergence rates. We stick to testing the trapezoidal integration code and collect all test functions in one common file by the name test_trapezoidal.py.
Hand-computed numerical results Our previous hand calculations for two trape- zoids can be checked against thetrapezoidalfunction inside a test function (in a filetest_trapezoidal.py):
from trapezoidal import trapezoidal def test_trapezoidal_one_exact_result():
"""Compare one hand-computed result."""
from math import exp
v = lambda t: 3*(t**2)*exp(t**3) n = 2
computed = trapezoidal(v, 0, 1, n) expected = 2.463642041244344 error = abs(expected - computed) tol = 1E-14
success = error < tol
msg = ’error=%g > tol=%g’ % (errror, tol) assert success, msg
74 3 Computing Integrals
Note the importance of checkingerragainstexactwith a tolerance: rounding errors from the arithmetics insidetrapezoidalwill not make the result exactly like the hand-computed one. The size of the tolerance is here set to1014, which is a kind of all-round value for computations with numbers not deviating much from unity.
Solving a problem without numerical errors We know that the trapezoidal rule is exact for linear integrands. Choosing the integralR4:4
1:2.6x4/dxas test case, the corresponding test function for this unit test may look like
def test_trapezoidal_linear():
"""Check that linear functions are integrated exactly."""
f = lambda x: 6*x - 4
F = lambda x: 3*x**2 - 4*x # Anti-derivative a = 1.2; b = 4.4
expected = F(b) - F(a) tol = 1E-14
for n in 2, 20, 21:
computed = trapezoidal(f, a, b, n) error = abs(expected - computed) success = error < tol
msg = ’n=%d, err=%g’ % (n, error) assert success, msg
Demonstrating correct convergence rates In the present example with integra- tion, it is known that the approximation errors in the trapezoidal rule are propor- tional ton2,nbeing the number of subintervals used in the composite rule.
Computing convergence rates requires somewhat more tedious programming than the previous tests, but can be applied to more general integrands. The al- gorithm typically goes like
foriD0; 1; 2; : : : ; q – ni D2iC1
– Compute integral withniintervals – Compute the errorEi
– Estimateri from (3.24) ifi > 0 The corresponding code may look like
def convergence_rates(f, F, a, b, num_experiments=14):
from math import log from numpy import zeros expected = F(b) - F(a)
n = zeros(num_experiments, dtype=int) E = zeros(num_experiments)
r = zeros(num_experiments-1) for i in range(num_experiments):
n[i] = 2**(i+1)
computed = trapezoidal(f, a, b, n[i]) E[i] = abs(expected - computed) if i > 0:
r_im1 = log(E[i]/E[i-1])/log(float(n[i])/n[i-1])
r[i-1] = float(’%.2f’ % r_im1) # Truncate to two decimals return r
Making a test function is a matter of choosingf,F,a, andb, and then checking the value ofri for the largesti:
def test_trapezoidal_conv_rate():
"""Check empirical convergence rates against the expected -2."""
from math import exp
v = lambda t: 3*(t**2)*exp(t**3) V = lambda t: exp(t**3)
a = 1.1; b = 1.9
r = convergence_rates(v, V, a, b, 14) print r
tol = 0.01
msg = str(r[-4:]) # show last 4 estimated rates assert (abs(r[-1]) - 2) < tol, msg
Running the test shows that allri, except the first one, equal the target limit 2 within two decimals. This observation suggest a tolerance of102.
Remark about version control of files
Having a suite of test functions for automatically checking that your soft- ware works is considered as a fundamental requirement for reliable computing.
Equally important is a system that can keep track of different versions of the files and the tests, known as aversion control system. Today’s most popular version control system isGit4, which the authors strongly recommend the reader to use for programming and writing reports. The combination of Git and cloud storage such as GitHub is a very common way of organizing scientific or engineering work. We have aquick intro5 to Git and GitHub that gets you up and running within minutes.
The typical workflow with Git goes as follows.
1. Before you start working with files, make sure you have the latest version of them by runninggit pull.
2. Edit files, remove or create files (new files must be registered bygit add).
3. When a natural piece of work is done, commit your changes by thegit commitcommand.
4. Implement your changes also in the cloud by doinggit push.
A nice feature of Git is that people can edit the same file at the same time and very often Git will be able to automatically merge the changes (!). Therefore, version control is crucial when you work with others or when you do your work on different types of computers. Another key feature is that anyone can at any time view the history of a file, see who did what when, and roll back the entire file collection to a previous commit. This feature is, of course, fundamental for reliable work.
4https://en.wikipedia.org/wiki/Git_(software)
5http://hplgit.github.io/teamods/bitgit/Langtangen_bitgit-bootstrap.html
76 3 Computing Integrals