scalacheck icon indicating copy to clipboard operation
scalacheck copied to clipboard

Add max testing duration

Open rickynils opened this issue 14 years ago • 1 comments

Transferred from http://code.google.com/p/scalacheck/issues/detail?id=83 #82 requests a timeout for any particular test. It would also be nice if Test.Params had an option for max duration of the entire testing run. For example, instead of specifying minSuccessfulTests and maxDiscardedTests, I'd like to just specify that testing go for 8 hours. There are any number of ways this feature could interact with the other parameters, as discussed on the mailing list. All would be fine with me!

rickynils avatar May 31 '11 09:05 rickynils

I think the following behaviour makes most sense:

The test runner takes the following parameters: propertyTimeout (see #5) minSuccessfulTests maxDiscardRatio (see #11) minTotalDuration (this issue)

The test runner will keep evaluating one property over and over again until:

It takes longer than propertyTimeout to do one evaluation --> failure

The ratio between the discarded evaluations and successful evaluations goes above maxDiscardRatio --> failure

The number of successful evaluations has passed minSuccessfulTests AND minTotalDuration milliseconds has passed since the first property evaluation was done --> success

Note: This means that if you set minTotalDuration to > 0, it will take at least this time for EACH of your properties to pass, since the test parameters only concerns ONE property, not property collections.

rickynils avatar Jul 10 '11 09:07 rickynils