Unit tests, re-revisited

The goal of this lecture note is to introduce a more informative way to carry out the unit tests.

Resources

• The OCaml code for the present lecture note (latest version: 23 Jan 2023).

Separating the “what” and the “how”

Now that we can unparse OCaml values, we can implement unit tests with more informative error messages than a failed assertion. Such messages could include:

• the name of the test that failed,
• the function that is being tested and its name, and
• the input, the actual output, and the expected output for the test that failed.

Also, negative tests (i.e., tests that are meant to fail) should not elicit error messages.

So let us implement two one-shot test functions, e.g., for unary functions:

let test_..._once          test_name candidate candidate_name input expected_output =
  ...

let test_..._once_silently test_name candidate candidate_name input expected_output =
  ...

These functions are given the name of the test, the function that is being tested, the name of the function that is being tested, the input, and the expected output. The first one emits an error message if the test fails, and the second does not.

Likewise, for binary functions, the two one-shot test functions are applied to two input values instead of to only one:

let test_..._once          test_name candidate candidate_name input1 input2 expected_output =
  ...

let test_..._once_silently test_name candidate candidate_name input1 input2 expected_output =
  ...

Thus equipped, we can compose our unit tests by stringing a series of calls to these one-shot test functions:

let test_... candidate_name candidate =
  let b0 = (test_..._once "b0" candidate candidate_name ... ... = true)
  and b1 = (test_..._once "b1" candidate candidate_name ... ... = true)
  and b2 = (test_..._once "b2" candidate candidate_name ... ... = true)
  and b3 = (test_..._once_silently "b3" candidate candidate_name ... ... = false)
  in b0 && b1 && b2 && b3;;

This skeleton contains three positive tests (named b0, b1, and b2) and one negative test (named b3).

Testing the successor function

In preparation for implementing the two one-shot test functions, let us implement a one-shot test function that is also parameterized with a silent flag:

let test_successor_once_internally silent_flag test_name candidate candidate_name input expected_output =
  ...

If the given silent flag is true, no error message is emitted, and if it is false, an error message is emitted. The two one-shot test functions are implemented as calling this one-shot test function:

let test_successor_once_silently test_name candidate candidate_name input expected_output =
  test_successor_once_internally true test_name candidate candidate_name input expected_output;;

let test_successor_once          test_name candidate candidate_name input expected_output =
  test_successor_once_internally false test_name candidate candidate_name input expected_output;;

If applying the given candidate function to the given input gives the given expected input, the one-shot test function should return true, and otherwise, it should emit an error message if the silent flag is false:

let test_successor_once_internally silent_flag test_name candidate candidate_name input expected_output =
  let actual_output = candidate input
  in if actual_output = expected_output
     then true
     else let () = if silent_flag
                   then ()
                   else ...
          in false;;

As for the error message, it should say something like:

test_successor_once failed for ... in ... with ... as input and with .. as output instead of the expected ...

All we need to do then is to splice in the name of the candidate function that failed (a string), the name of the test where it failed (a string), the input with which it failed (an integer), its actual output (an integer), and its expected output (an integer):

Printf.printf
  "test_successor_once failed for %s in %s with %s as input and with %s as output instead of the expected %s\n"
  candidate_name
  test_name
  (show_int input)
  (show_int actual_output)
  (show_int expected_output)

We can then verify, e.g.,

1. that the resident successor function maps 5 to 6:

   # test_successor_once "(a)" succ "succ" 5 6;;
   - : bool = true
   #
   

2. that the resident successor function does not map 5 to 60:

   # test_successor_once "(b)" succ "succ" 5 60 = false;;
   test_successor_once failed for succ in (b) with 5 as input and with 6 as output instead of the expected 60
   - : bool = true
   #
   

3. that this last test can be carried out silently, i.e., without error message:

   # test_successor_once_silently "(c)" succ "succ" 5 60 = false;;
   - : bool = true
   #
   

We are now in position to write a unit-test function with several calls to the one-shot unit-test functions:

let test_successor candidate_name candidate =
  let b0 = (test_successor_once "b0" candidate candidate_name ~-1 0 = true)
  and b1 = (test_successor_once "b1" candidate candidate_name   0 1 = true)
  and b2 = (test_successor_once "b2" candidate candidate_name   1 2 = true)
  and b3 = (test_successor_once_silently "b3" candidate candidate_name 0 0 = false)
  and b4 = (let n = random_int 1000
            in test_successor_once "b4" candidate candidate_name n (n + 1) = true)
  in b0 && b1 && b2 && b3 && b4;;

In words, this unit-test function checks

• whether -1 is mapped to 0,
• whether 0 is mapped to 1,
• whether 1 is mapped to 2,
• whether 0 is not mapped to 0, and
• whether a random integer between -999 and 999 is mapped to the successor of this random integer.

The accompanying .ml file contains three other examples – one for unparsing integers, one for squaring the sum of two numbers, and one for the factorial function.

Resources

• The OCaml code for the present lecture note (latest version: 23 Jan 2023).

Version

Added a description of the accompanying .ml file [24 Apr 2023]

Created [23 Jan 2023]