microsoft
Non-deterministic tests
Test coverage changes with every test run, which means that some lines won't be tested sometimes. For example, in the first run for PR #143, the coverage for Qtype.jl drops by 34 lines for no reason.
There was probably a good reason for these fuzz tests being written at the time, but I think we should try to move past them now. This is one of the more important bits of technical debt that I'd like to try fixing before working on new features.
sounds like a good idea. Do you have a concrete plan for what needs to be done? If you can list the fixes then we can work on distributing the work equitably.
I'm not sure yet. On second thoughts, it might be more effort than it's worth - I think we'll have to replace each fuzz test with calculated examples which cover each intersection type that we're interested in. I'll take a better look at this once the current PRs are resolved.
It should be possible to just seed things so they are deterministic - I think we tried to this but maybe it’s failing somewhere
Is this what the test arg is for? It's hard to track what it does as it's passed between lots of function calls.
Broadly yes, maybe @cdhf’s recent additions can be helpful for this? Also there are some tests which randomly generate rays, the random generation is/was seeded so should be deterministic