module QCheck:sig
..end
The library takes inspiration from Haskell's QuickCheck library. The rough idea is that the programmer describes invariants that values of a certain type need to satisfy ("properties"), as functions from this type to bool. She also needs to describe how to generate random values of the type, so that the property is tried and checked on a number of random instances.
This explains the organization of this module:
'a arbitrary
is used to describe how to generate random values,
shrink them (make counter-examples as small as possible), print
them, etc. Auxiliary modules such as QCheck.Gen
, QCheck.Print
, and QCheck.Shrink
can be used along with QCheck.make
to build one's own arbitrary instances.QCheck.Test
is used to describe a single test, that is, a property of
type 'a -> bool
combined with an 'a arbitrary
that is used to generate
the test cases for this property. Optional parameters
allow to specify the random generator state, number of instances to generate
and test, etc.Examples:
let test =
QCheck.(Test.make ~count:1000
(list int) (fun l -> List.rev (List.rev l) = l));;
QCheck.Test.check_exn test;;
let test = QCheck.(
Test.make
~count:10_000 ~max_fail:3
(list small_nat)
(fun l -> l = List.sort compare l));;
QCheck.Test.check_exn test;;
QCheck.Gen.fix
:type tree = Leaf of int | Node of tree * tree
let leaf x = Leaf x
let node x y = Node (x,y)
let g = QCheck.Gen.(sized @@ fix
(fun self n -> match n with
| 0 -> map leaf nat
| n ->
frequency
[1, map leaf nat;
2, map2 node (self (n/2)) (self (n/2))]
))
Gen.generate ~n:20 g;;
More complex and powerful combinators can be found in Gabriel Scherer's
Generator
module. Its documentation can be found
here.
val (==>) : bool -> bool -> bool
b1 ==> b2
is the logical implication b1 => b2
ie not b1 || b2
(except that it is strict and will interact
better with QCheck.Test.check_exn
and the likes, because they will know
the precondition was not satisfied.).
WARNING: this function should only be used in a property
(see QCheck.Test.make
), because it raises a special exception in case of
failure of the first argument, to distinguish between failed test
and failed precondition. Because of OCaml's evaluation order,
both b1
and b2
are always evaluated; if b2
should only be
evaluated when b1
holds, see QCheck.assume
.
val assume : bool -> unit
assume cond
checks the precondition cond
, and does nothing
if cond=true
. If cond=false
, it interrupts the current test.
WARNING This function, like QCheck.(==>)
, should only be used in
a test, not outside.
Example:
Test.make (list int) (fun l ->
assume (l <> []);
List.hd l :: List.tl l = l)
val assume_fail : unit -> 'a
assume_fail ()
is like assume false
, but can take any type
since we know it always fails (like assert false
).
This is useful to ignore some branches in if
or match
.
Example:
Test.make (list int) (function
| [] -> assume_fail ()
| _::_ as l -> List.hd l :: List.tl l = l)
module Gen:sig
..end
Generate Random Values
module Print:sig
..end
Show Values
module Iter:sig
..end
Iterators
module Shrink:sig
..end
Shrink Values
Observables are usable as arguments for random functions. The random function will observe its arguments in a way that is determined from the observable instance.
Inspired from https://blogs.janestreet.com/quickcheck-for-core/ and Koen Claessen's "Shrinking and Showing functions".
module Observable:sig
..end
A value of type 'a arbitrary
glues together a random generator,
and optional functions for shrinking, printing, computing the size,
etc. It is the "normal" way of describing how to generate
values of a given type, to be then used in tests (see QCheck.Test
).
type'a
stat =string * ('a -> int)
A statistic on a distribution of values of type 'a
.
The function MUST return a positive integer.
type 'a
arbitrary = private {
|
gen : |
|||
|
print : |
(* | print values | *) |
|
small : |
(* | size of example | *) |
|
shrink : |
(* | shrink to smaller examples | *) |
|
collect : |
(* | map value to tag, and group by tag | *) |
|
stats : |
(* | statistics to collect and print | *) |
}
A value of type 'a arbitrary
is an object with a method for generating random
values of type 'a
, and additional methods to compute the size of values,
print them, and possibly shrink them into smaller counter-examples.
NOTE the collect field is unstable and might be removed, or
moved into QCheck.Test
.
Made private since 0.8
val make : ?print:'a Print.t ->
?small:('a -> int) ->
?shrink:'a Shrink.t ->
?collect:('a -> string) ->
?stats:'a stat list -> 'a Gen.t -> 'a arbitrary
Builder for arbitrary. Default is to only have a generator, but other arguments can be added.
print
: printer for values (counter-examples)shrink
: to shrink counter-examplescollect
: for statisticsval set_print : 'a Print.t -> 'a arbitrary -> 'a arbitrary
val set_small : ('a -> int) -> 'a arbitrary -> 'a arbitrary
val set_shrink : 'a Shrink.t -> 'a arbitrary -> 'a arbitrary
val set_collect : ('a -> string) -> 'a arbitrary -> 'a arbitrary
val set_stats : 'a stat list -> 'a arbitrary -> 'a arbitrary
val add_shrink_invariant : ('a -> bool) -> 'a arbitrary -> 'a arbitrary
Update shrinker by only keeping smaller values satisfying the given invariant.
val set_gen : 'a Gen.t -> 'a arbitrary -> 'a arbitrary
Change the generator
val add_stat : 'a stat -> 'a arbitrary -> 'a arbitrary
Add a statistic to the arbitrary instance.
val gen : 'a arbitrary -> 'a Gen.t
Access the underlying random generator of this arbitrary object.
A test is a universal property of type foo -> bool
for some type foo
,
with an object of type foo arbitrary
used to generate, print, etc. values
of type foo
.
See QCheck.Test.make
to build a test, and QCheck.Test.check_exn
to
run one test simply.
For more serious testing, it is better to create a testsuite
and use QCheck_runner
.
module TestResult:sig
..end
Result of running a test
module Test:sig
..end
The infrastructure used to find counter-examples to properties can also be used to find data satisfying a predicate, within a property being tested.
See https://github.com/c-cube/qcheck/issues/31
exception No_example_found of string
val find_example : ?name:string ->
?count:int -> f:('a -> bool) -> 'a Gen.t -> 'a Gen.t
find_example ~f gen
uses gen
to generate some values of type 'a
,
and checks them against f
. If such a value is found, it is returned.
Otherwise an exception is raised.
NOTE this should only be used from within a property in QCheck.Test.make
.
No_example_found
if no example is found within count
tries.name
: description of the example to find (used in the exception).count
: number of attempts.f
: the property that the example must satisfy.val find_example_gen : ?rand:Stdlib.Random.State.t ->
?name:string -> ?count:int -> f:('a -> bool) -> 'a Gen.t -> 'a
Toplevel version of QCheck.find_example
.
find_example_gen ~f arb ~n
is roughly the same as
Gen.generate1 (find_example ~f arb |> gen)
.
No_example_found
if no example was found within count
tries.rand
: the random state to use to generate inputs.val choose : 'a arbitrary list -> 'a arbitrary
Choose among the given list of generators. The list must not be empty; if it is Invalid_argument is raised.
val unit : unit arbitrary
Always generates ()
, obviously.
val bool : bool arbitrary
Uniform boolean generator.
val float : float arbitrary
Generates regular floats (no nan and no infinities).
val pos_float : float arbitrary
Positive float generator (no nan and no infinities).
val neg_float : float arbitrary
Negative float generator (no nan and no infinities).
val float_bound_inclusive : float -> float arbitrary
float_bound_inclusive n
is uniform between 0
and n
included. If bound
is
negative, the result is negative or zero. If bound
is 0, the result is 0.
val float_bound_exclusive : float -> float arbitrary
float_bound_exclusive n
is uniform between 0
included and n
excluded.
If bound
is negative, the result is negative or zero.
Invalid_argument
if bound
is zero.val float_range : float -> float -> float arbitrary
float_range low high
is uniform between low
included and high
included.
Invalid_argument
if low > high
or if the range is larger than max_float
.val int : int arbitrary
Int generator. Uniformly distributed.
val int_bound : int -> int arbitrary
int_bound n
is uniform between 0
and n
included.
val int_range : int -> int -> int arbitrary
int_range a b
is uniform between a
and b
included. b
must be
larger than a
.
val small_nat : int arbitrary
Small unsigned integers.
val small_int : int arbitrary
QCheck.small_signed_int
.Small unsigned integers. See QCheck.Gen.small_int
.
val small_signed_int : int arbitrary
Small signed integers.
val (--) : int -> int -> int arbitrary
Synonym to QCheck.int_range
.
val int32 : int32 arbitrary
Int32 generator. Uniformly distributed.
val int64 : int64 arbitrary
Int64 generator. Uniformly distributed.
val pos_int : int arbitrary
Positive int generator (0 included). Uniformly distributed.
See QCheck.Gen.pint
val small_int_corners : unit -> int arbitrary
As small_int
, but each newly created generator starts with
a list of corner cases before falling back on random generation.
val neg_int : int arbitrary
Negative int generator (0 included, see QCheck.Gen.neg_int
).
The distribution is similar to that of
small_int
, not of pos_int
.
val char : char arbitrary
Uniformly distributed on all the chars (not just ascii or valid latin-1).
val printable_char : char arbitrary
Uniformly distributed over a subset of chars.
val numeral_char : char arbitrary
Uniformly distributed over '0'..'9'
.
val string_gen_of_size : int Gen.t -> char Gen.t -> string arbitrary
val string_gen : char Gen.t -> string arbitrary
Generates strings with a distribution of length of small_nat
.
val string : string arbitrary
Generates strings with a distribution of length of small_nat
and distribution of characters of char
.
val small_string : string arbitrary
Same as QCheck.string
but with a small length (ie QCheck.Gen.small_nat
).
val small_list : 'a arbitrary -> 'a list arbitrary
Generates lists of small size (see QCheck.Gen.small_nat
).
val string_of_size : int Gen.t -> string arbitrary
Generates strings with distribution of characters if char
.
val printable_string : string arbitrary
Generates strings with a distribution of length of small_nat
and distribution of characters of printable_char
.
val printable_string_of_size : int Gen.t -> string arbitrary
Generates strings with distribution of characters of printable_char
.
val small_printable_string : string arbitrary
val numeral_string : string arbitrary
Generates strings with a distribution of length of small_nat
and distribution of characters of numeral_char
.
val numeral_string_of_size : int Gen.t -> string arbitrary
Generates strings with a distribution of characters of numeral_char
.
val list : 'a arbitrary -> 'a list arbitrary
Generates lists with length generated by small_nat
.
val list_of_size : int Gen.t -> 'a arbitrary -> 'a list arbitrary
Generates lists with length from the given distribution.
val array : 'a arbitrary -> 'a array arbitrary
Generates arrays with length generated by small_nat
.
val array_of_size : int Gen.t -> 'a arbitrary -> 'a array arbitrary
Generates arrays with length from the given distribution.
val pair : 'a arbitrary -> 'b arbitrary -> ('a * 'b) arbitrary
Combines two generators into a generator of pairs.
Order of elements can matter (w.r.t shrinking, see QCheck.Shrink.pair
)
val triple : 'a arbitrary ->
'b arbitrary -> 'c arbitrary -> ('a * 'b * 'c) arbitrary
Combines three generators into a generator of 3-tuples.
Order matters for shrinking, see QCheck.Shrink.pair
and the likes
val quad : 'a arbitrary ->
'b arbitrary ->
'c arbitrary ->
'd arbitrary -> ('a * 'b * 'c * 'd) arbitrary
Combines four generators into a generator of 4-tuples.
Order matters for shrinking, see QCheck.Shrink.pair
and the likes
val option : 'a arbitrary -> 'a option arbitrary
Choose between returning Some random value, or None.
val fun1_unsafe : 'a arbitrary -> 'b arbitrary -> ('a -> 'b) arbitrary
QCheck.fun_
instead.Generator of functions of arity 1. The functions are always pure and total functions:
renamed from QCheck.fun1
since 0.6
val fun2_unsafe : 'a arbitrary ->
'b arbitrary ->
'c arbitrary -> ('a -> 'b -> 'c) arbitrary
QCheck.fun_
instead since 0.6Generator of functions of arity 2. The remark about fun1
also apply
here.
renamed from QCheck.fun2
since 0.6
type '_
fun_repr
Internal data for functions. A 'f fun_
is a function
of type 'f
, fundamentally.
type '_
fun_ =
| |
Fun : |
A function packed with the data required to print/shrink it. See QCheck.Fn
to see how to apply, print, etc. such a function.
One can also directly pattern match on it to obtain the executable function.
For example:
QCheck.Test.make
QCheck.(pair (fun1 Observable.int bool) (small_list int))
(fun (Fun (_,f), l) -> l=(List.rev_map f l |> List.rev l))
module Fn:sig
..end
Utils on functions
val fun1 : 'a Observable.t ->
'b arbitrary -> ('a -> 'b) fun_ arbitrary
fun1 o ret
makes random functions that take an argument observable
via o
and map to random values generated from ret
.
To write functions with multiple arguments, it's better to use QCheck.Tuple
or QCheck.Observable.pair
rather than applying QCheck.fun_
several times
(shrinking will be faster).
module Tuple:sig
..end
val fun_nary : 'a Tuple.obs ->
'b arbitrary -> ('a Tuple.t -> 'b) fun_ arbitrary
fun_nary
makes random n-ary functions.
Example:
let module O = Observable in
fun_nary Tuple.(O.int @-> O.float @-> O.string @-> o_nil) bool)
val fun2 : 'a Observable.t ->
'b Observable.t ->
'c arbitrary -> ('a -> 'b -> 'c) fun_ arbitrary
val fun3 : 'a Observable.t ->
'b Observable.t ->
'c Observable.t ->
'd arbitrary -> ('a -> 'b -> 'c -> 'd) fun_ arbitrary
val fun4 : 'a Observable.t ->
'b Observable.t ->
'c Observable.t ->
'd Observable.t ->
'e arbitrary ->
('a -> 'b -> 'c -> 'd -> 'e) fun_ arbitrary
val oneofl : ?print:'a Print.t ->
?collect:('a -> string) -> 'a list -> 'a arbitrary
Pick an element randomly in the list.
val oneofa : ?print:'a Print.t ->
?collect:('a -> string) -> 'a array -> 'a arbitrary
Pick an element randomly in the array.
val oneof : 'a arbitrary list -> 'a arbitrary
Pick a generator among the list, randomly.
val always : ?print:'a Print.t -> 'a -> 'a arbitrary
Always return the same element.
val frequency : ?print:'a Print.t ->
?small:('a -> int) ->
?shrink:'a Shrink.t ->
?collect:('a -> string) ->
(int * 'a arbitrary) list -> 'a arbitrary
Similar to QCheck.oneof
but with frequencies.
val frequencyl : ?print:'a Print.t ->
?small:('a -> int) -> (int * 'a) list -> 'a arbitrary
Same as QCheck.oneofl
, but each element is paired with its frequency in
the probability distribution (the higher, the more likely).
val frequencya : ?print:'a Print.t ->
?small:('a -> int) -> (int * 'a) array -> 'a arbitrary
Same as QCheck.frequencyl
, but with an array.
val map : ?rev:('b -> 'a) -> ('a -> 'b) -> 'a arbitrary -> 'b arbitrary
map f a
returns a new arbitrary instance that generates values using
a#gen
and then transforms them through f
.
rev
: if provided, maps values back to type 'a
so that the printer,
shrinker, etc. of a
can be used. We assume f
is monotonic in
this case (that is, smaller inputs are transformed into smaller outputs).val map_same_type : ('a -> 'a) -> 'a arbitrary -> 'a arbitrary
Specialization of map
when the transformation preserves the type, which
makes shrinker, printer, etc. still relevant.
val map_keep_input : ?print:'b Print.t ->
?small:('b -> int) ->
('a -> 'b) -> 'a arbitrary -> ('a * 'b) arbitrary
map_keep_input f a
generates random values from a
, and maps them into
values of type 'b
using the function f
, but it also keeps the
original value.
For shrinking, it is assumed that f
is monotonic and that smaller input
values will map into smaller values.
print
: optional printer for the f
's output.