QCheck_runner
include module type of struct include QCheck_base_runner end
Once you have built tests using QCheck.Test.make
or QCheck2.Test.make
, you need to run them. This module contains several runners, which are designed to run every test and report the result.
By default, you can use run_tests
in a test program as follows:
let testsuite = [
Test.make ...;
Test.make ...;
]
let () =
let errcode = QCheck_base_runner.run_tests ~verbose:true testsuite in
exit errcode
which will run the tests, and exit the program. The error code will be 0 if all tests pass, 1 otherwise.
run_tests_main
can be used as a shortcut for that, also featuring command-line parsing (using Arg
) to activate verbose mode and others.
Change the random_state
by creating a new one, initialized with the given seed.
In verbose mode, by default QCheck_base_runner
prints frequent sub-second messages suitable for an interactive console test run. This behaviour can be changed by the environment variable QCHECK_MSG_INTERVAL
. Intervals are given in seconds and can also be decimal numbers. For example, setting
QCHECK_MSG_INTERVAL=7.5
will only print a console message every 7.5 seconds. This feature can be useful in a CI context, where updates are printed on consecutive lines and one may want to avoid overflowing the CI log files with too many lines.
Get the minimum time (in seconds) to wait between printing messages.
type counter = private QCheck_base_runner.counter = {
start : float;
expected : int;
mutable gen : int;
mutable passed : int;
mutable failed : int;
mutable errored : int;
}
The type of counter used to keep tracks of the events received for a given test cell.
A type to represent polymorphic-enough handlers for test cells.
type handler_gen =
colors:bool ->
debug_shrink:Stdlib.out_channel option ->
debug_shrink_list:string list ->
size:int ->
out:Stdlib.out_channel ->
verbose:bool ->
counter ->
handler
An alias type to a generator of handlers for test cells.
val default_handler : handler_gen
The default handler used.
val debug_shrinking_choices :
colors:bool ->
out:Stdlib.out_channel ->
name:string ->
'a QCheck2.Test.cell ->
step:int ->
'a ->
unit
The function used by the default handler to debug shrinking choices. This can be useful to outside users trying to reproduce some of the base-runner behavior.
val run_tests :
?handler:handler_gen ->
?colors:bool ->
?verbose:bool ->
?long:bool ->
?debug_shrink:Stdlib.out_channel option ->
?debug_shrink_list:string list ->
?out:Stdlib.out_channel ->
?rand:Stdlib.Random.State.t ->
QCheck2.Test.t list ->
int
Run a suite of tests, and print its results. This is an heritage from the "qcheck" library.
val run_tests_main : ?argv:string array -> QCheck2.Test.t list -> 'a
Can be used as the main function of a test file. Exits with a non-0 code if the tests fail. It refers to run_tests
for actually running tests after CLI options have been parsed.
The available options are:
Below is an example of the output of the run_tests
and run_tests_main
function:
random seed: 438308050 generated error; fail; pass / total - time -- test name [✓] (1000) 0 ; 0 ; 1000 / 1000 -- 0.5s -- list_rev_is_involutive [✗] ( 1) 0 ; 1 ; 0 / 10 -- 0.0s -- should_fail_sort_id [✗] ( 1) 1 ; 0 ; 0 / 10 -- 0.0s -- should_error_raise_exn [✓] (1000) 0 ; 0 ; 1000 / 1000 -- 0.0s -- collect_results --- Failure -------------------------------------------------------------------- Test should_fail_sort_id failed (11 shrink steps): [1; 0] === Error ====================================================================== Test should_error_raise_exn errored on (62 shrink steps): 0 exception QCheck_runner_test.Error Raised at file "example/QCheck_runner_test.ml", line 20, characters 20-25 Called from file "src/QCheck.ml", line 839, characters 13-33 +++ Collect ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Collect results for test collect_results: 4: 207 cases 3: 190 cases 2: 219 cases 1: 196 cases 0: 188 cases ================================================================================ failure (1 tests failed, 1 tests errored, ran 4 tests)
module Color = QCheck_base_runner.Color
module Raw = QCheck_base_runner.Raw
include module type of struct include QCheck_ounit end
val to_ounit_test :
?verbose:bool ->
?long:bool ->
?rand:Stdlib.Random.State.t ->
QCheck2.Test.t ->
OUnit.test
to_ounit_test ~rand t
wraps t
into a OUnit test
val to_ounit_test_cell :
?verbose:bool ->
?long:bool ->
?rand:Stdlib.Random.State.t ->
_ QCheck2.Test.cell ->
OUnit.test
Same as to_ounit_test
but with a polymorphic test cell
val (>:::) : string -> QCheck2.Test.t list -> OUnit.test
Same as OUnit.(>:::)
but with a list of QCheck2 tests
val to_ounit2_test :
?rand:Stdlib.Random.State.t ->
QCheck2.Test.t ->
OUnit2.test
to_ounit2_test ?rand t
wraps t
into a OUnit2 test
val to_ounit2_test_list :
?rand:Stdlib.Random.State.t ->
QCheck2.Test.t list ->
OUnit2.test list
to_ounit2_test_list ?rand t
like to_ounit2_test
but for a list of tests
QCheck provides some custom runners for OUnit tests.
Note that OUnit.run_test_tt
or OUnit.run_test_tt_main
can be used as well, in particular when QCheck tests are mixed with normal unit tests.
For OUnit2 you can use OUnit2.run_test_tt_main
.
run test
runs the test, and returns an error code that is 0
if all tests passed, 1
otherwise. This is the default runner used by the comment-to-test generator.
This test runner displays execution in a compact way, making it good for suites that have lots of tests.
Output example:
random seed: 101121210 random seed: 101121210 \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ Error: tests>error_raise_exn test `error_raise_exn` raised exception `QCheck_ounit_test.Error` on `0 (after 62 shrink steps)` Raised at file "example/QCheck_ounit_test.ml", line 19, characters 20-25 Called from file "src/QCheck.ml", line 846, characters 13-33 /////////////////////////////////////////////////////////////////////////////// \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ Failure: tests>fail_sort_id fail_sort_id /////////////////////////////////////////////////////////////////////////////// Ran: 4 tests in: 0.74 seconds. WARNING! SOME TESTS ARE NEITHER SUCCESSES NOR FAILURES!