feedback icon indicating copy to clipboard operation
feedback copied to clipboard

[Feature Request] In Tests tab, provide a normal pass/failure rate

Open kg345 opened this issue 4 months ago • 1 comments

What product do you want to improve? Tests tab

Is your feature request related to a problem? Please describe. The failure rate and flake rate make little sense for someone who's trying to understand how often something is passing - just a vanilla num of passes / total attempts. Failure rate is also poorly named as it's not even a rate of failures - but, instead tied to flaky.

Tests that fail once get added to a flaky list and runs where they pass are not counted as passing. So our failure/flake rate will look like 100% for last 7 days for something that passed the last 20 runs (if it failed once on run proceeding aka last 30 times)

Describe the solution you'd like A column in the Tests tab table that clearly indicates the percentage of runs where a test passed irrespective of a "flake list".

It seems pretty reasonable to expect a tests tab to give us more "normalized" data (how many times it passed, how many times it failed) without the opinionated/unique calculation that flake rate (and it's impact on failure rate).

Describe alternatives you've considered The only way to seemingly get this info is to hover on the flake rate and see 1 passed, 0 failed (28 flaky) which isn't super helpful when you have 500 tests to get data

Additional context

kg345 avatar Oct 02 '25 17:10 kg345

CCMRG-1737

linear[bot] avatar Oct 02 '25 17:10 linear[bot]