gtfs-validator icon indicating copy to clipboard operation
gtfs-validator copied to clipboard

Modify the logic for the acceptance tests

Open isabelle-dr opened this issue 4 years ago • 2 comments

As discussed in this comment:

Modify the logic of the acceptance tests so a PR is flagged if there is at least one additional error in one dataset from the Mobility Database (currently it's flagged if there is at least one additional error in at least 1% of datasets).

isabelle-dr avatar Dec 10 '21 13:12 isabelle-dr

Modify the logic of the acceptance tests so a PR is flagged if there is at least one additional error in one dataset from the Mobility Database

Is this one additional error type, or one additional occurrence of an error type (or both)?

If you're treating this more like a unit test and expecting the same output from both validators, then IMHO count occurrences of errors is important. In other words, you'd want to know if a code change made the output jump from 1 occurrence of block_trips_with_overlapping_stop_times to 1000 occurrences.

barbeau avatar Dec 10 '21 15:12 barbeau

👋 Hello,

Is this one additional error type, or one additional occurrence of an error type (or both)?

We are thinking one additional occurrence of an error type. Thanks for clarifying this!

isabelle-dr avatar Dec 13 '21 09:12 isabelle-dr