github-action-benchmark
github-action-benchmark copied to clipboard
Allow choice of benchmark stat used for `pytest`
pytest-benchmark automatically computes a fairly comprehensive set of stats across multiple runs of each benchmark, including minimum, mean, median, etc. runtime, and Operations Per Second (calculated as the inverse of the mean runtime).
It's commonly recommended that benchmark comparisons use the minimum runtime, as it is more stable than other statistics. However, the actions are currently hardcoded to use the ops statistic.
Is it possible to make this configurable?
(This request may apply more widely than pytest, but I'm not familiar with the other output formats that are handled)