scalacheck icon indicating copy to clipboard operation
scalacheck copied to clipboard

Shrinking shrinks a List required to be nonEmpty to Nil

Open letalvoj opened this issue 7 years ago • 3 comments

Let's have a generator of a data structure which contains a non empty list. Like

Gen.nonEmptyListOf(Gen.choose(0,10))

When shrinking kicks in, what happens is that the list gets shortened - well that's an obvious way to reduce the input, I agree. Yet if the error is caused by a different parameter, than the shrinking does not stop and shrinks the list all the way to Nil.

That kind of sucks, because there is definitely a reason for the list to be non empty! In my case it causes an empty.head error which is completely misleading and obscures the actual problem.

IMO the Shrinker should respect the domain of the values specified by the defined Generator.

letalvoj avatar Nov 30 '18 18:11 letalvoj

Unfortunately in ScalaCheck (and the original QuickCheck) shrinking has to be done separately and in addition to generation. It doesn’t compose.

This problem is made slightly more annoying in ScalaCheck because of the combination of restricted Gens with the global/implicit Shrinker in forAll. I would argue it’s a bug and have opened a PR to remove that particular behaviour (but don’t know if it will be accepted). So you basically wouldn’t get shrinking when using Gen, which is why you’re getting an error, which is a reoccurring problem from what I have observed.

https://github.com/rickynils/scalacheck/pull/440

I highly recommend checking out a fairly new library Hedgehog which effectively fixes/solves this problem by integrating shrinking with Gen. There’s a great talk by the creator.

https://m.youtube.com/watch?v=AIv_9T0xKEo

Shameless plug warning. There is also the Scala version which I maintain.

https://github.com/hedgehogqa/scala-hedgehog

charleso avatar Nov 30 '18 19:11 charleso

It's a known issue in #129.

Writing your own shrinker is possible, see:

https://stackoverflow.com/questions/50103923/how-do-i-shrink-a-list-but-guarantee-it-isnt-empty

ashawley avatar Nov 30 '18 21:11 ashawley

Thanks. I've already solved the issue, yet I did not know that there was a meaningful alternative (hedgehog).

It came to mind that combining Gen with Shrink might be out of the scope of this project, yet I failed to find a related issue so I filled one.

Interestingly it first emerged when I used nonEmptyListOf ... Filtering out empty lists manually caused the error rate decrease significantly - such that the tests stopped failing for a few consecutive runs - which made me thinking that there was error in the nonEmpty variant of the listOf function actually.

Anyway, this is a serious design flaw and therefore shrinking should be disabled by default - to force the user to learn about it first. If you do not agree feel free to close the issue. On Fri, 30 Nov 2018 at 22:28, Aaron S. Hawley [email protected] wrote:

It's a known issue in #129 https://github.com/rickynils/scalacheck/issues/129.

Writing your own shrinker is possible, see:

https://stackoverflow.com/questions/50103923/how-do-i-shrink-a-list-but-guarantee-it-isnt-empty

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rickynils/scalacheck/issues/443#issuecomment-443344978, or mute the thread https://github.com/notifications/unsubscribe-auth/AFBL8c8ditQ2qIXJQ3fzrIFRqLDAvZWvks5u0aLsgaJpZM4Y8KUJ .

letalvoj avatar Nov 30 '18 21:11 letalvoj