github-action-benchmark icon indicating copy to clipboard operation
github-action-benchmark copied to clipboard

Allow choice of benchmark stat used for `pytest`

Open DFEvans opened this issue 4 years ago • 0 comments

pytest-benchmark automatically computes a fairly comprehensive set of stats across multiple runs of each benchmark, including minimum, mean, median, etc. runtime, and Operations Per Second (calculated as the inverse of the mean runtime).

It's commonly recommended that benchmark comparisons use the minimum runtime, as it is more stable than other statistics. However, the actions are currently hardcoded to use the ops statistic.

Is it possible to make this configurable?

(This request may apply more widely than pytest, but I'm not familiar with the other output formats that are handled)

DFEvans avatar Dec 10 '21 14:12 DFEvans