python-project-template icon indicating copy to clipboard operation
python-project-template copied to clipboard

Fail asv workflows when performance deteriorates

Open camposandro opened this issue 2 years ago • 0 comments

Fail the ASV workflows if at least one of benchmarks deteriorates above a certain threshold (e.g. 150% / 200%).

asv-main.yml:

  • Execute asv run without || true, allowing the command to exit with a non-zero return code
  • Submit benchmarks and deploy dashboard on gh-pages, if the previous command succeeded or not

asv-pr.yml:

  • Execute asv continuous without || true, allowing the command to exit with a non-zero return code
  • Output the performance comment on the PR if the comparison was successful or not

asv-nightly.yml

  • Execute asv continuous without || true, allowing the command to exit with a non-zero return code
  • Write the commit hash to cache, if the previous if the comparison was successful or not

More information about "on failure" executions here.

camposandro avatar Aug 09 '23 15:08 camposandro