github-action-benchmark icon indicating copy to clipboard operation
github-action-benchmark copied to clipboard

Don't fail if benchmark performs better irrespective of threshold config

Open madhan-mettle opened this issue 4 years ago • 0 comments

There might be situations if the changes drastically improves the performance or the thresholds are configured with lower values it lead to failure of the action whereas it is not.

i.e if the alert and failure threshold are configured as 0.1 and 0.2 and the previous and current values are bigger, then the existing check would consider it as failure

So added a check to ensure the failure is calculated based on the performance and threshold rather than threshold alone

madhan-mettle avatar Apr 29 '22 15:04 madhan-mettle