gtfs-validator
gtfs-validator copied to clipboard
Analysis improvement: new Github action to trigger comparison between PR acceptance test and past release stats
In cases where we're evaluating changes to the logic of pre-existing rules e.g #1676 and #1675, we want to know the difference between the new PR's acceptance test changes and release analytics from the past, not just the % of feeds triggered by the change relative to all feeds in the Mobility Database. Right now I manually
- Take the drafted PR's acceptance test results
- Take the feeds triggered by the notice in the last release analytics
- Compare the feed IDs of the two to identify
- Which feeds have dropped/added the notice because of the new drafted logic?
- Which feeds have dropped/added the notice because of unrelated changes since the last release?
- Run the individual feeds through https://gtfs-validator.mobilitydata.org/ to see the notice details depending on the particular nuances of that change.
It would be nice in the feed comparison process was automated, so I could see at a glance what % of feeds relative to the number of feeds in the last release that triggered the notice and treat the last release analytics as the source of truth for IDs.