EPATADA icon indicating copy to clipboard operation
EPATADA copied to clipboard

Stability: test-CensoredDataSuite "TADA_IDCensoredData copies det lim values to result values if applicable" fails inconsistently

Open jbousquin opened this issue 3 months ago • 1 comments

R-CMD-check will often pass on all matrix runs except one and then pass on re-run (e.g., Run1 vs Run2) with the failure:

  ── Failure ('test-CensoredDataSuite.R:45:3'): TADA_IDCensoredData copies det lim values to result values if applicable ──
  all(copycheck_NAs$TADA.ResultMeasureValueDataTypes.Flag == "Result Value/Unit Copied from Detection Limit") is not TRUE
  
  `actual`:   FALSE
  `expected`: TRUE 

jbousquin avatar Oct 17 '25 20:10 jbousquin

From what I can tell there are three things that could impact stability, all related to use of TADA_RandomTestingData():

  1. There are two versions of the function TADA_RandomTestingData, one in Utilities.R and one in GeospatialFunctions.R, I don't know how the test decides which to use?
  2. Both versions of the function use TADA_DataRetrieval(), meaning they are retrieving data from the service (i.e., can hit errors vs a mocked result). The test results don't seem to suggest a 500 error, but it could be handled quietly (seems like there is some retry in the function).
  3. To define copycheck the function is using arg choose_random_state = TRUE, meaning the data being run through the test is always changing.

Ideally it would be good to limit these stability issues, but beyond that, given that R-CMD-check has a long run time I'm wondering if there is a way to have it run less stable tests first and fail quickly?

jbousquin avatar Oct 17 '25 20:10 jbousquin