EPATADA
EPATADA copied to clipboard
Stability: test-CensoredDataSuite "TADA_IDCensoredData copies det lim values to result values if applicable" fails inconsistently
R-CMD-check will often pass on all matrix runs except one and then pass on re-run (e.g., Run1 vs Run2) with the failure:
── Failure ('test-CensoredDataSuite.R:45:3'): TADA_IDCensoredData copies det lim values to result values if applicable ──
all(copycheck_NAs$TADA.ResultMeasureValueDataTypes.Flag == "Result Value/Unit Copied from Detection Limit") is not TRUE
`actual`: FALSE
`expected`: TRUE
From what I can tell there are three things that could impact stability, all related to use of TADA_RandomTestingData():
- There are two versions of the function TADA_RandomTestingData, one in Utilities.R and one in GeospatialFunctions.R, I don't know how the test decides which to use?
- Both versions of the function use TADA_DataRetrieval(), meaning they are retrieving data from the service (i.e., can hit errors vs a mocked result). The test results don't seem to suggest a 500 error, but it could be handled quietly (seems like there is some retry in the function).
- To define copycheck the function is using arg choose_random_state = TRUE, meaning the data being run through the test is always changing.
Ideally it would be good to limit these stability issues, but beyond that, given that R-CMD-check has a long run time I'm wondering if there is a way to have it run less stable tests first and fail quickly?