pytest-dev
Expose Coverage's --sort as --cov-sort pytest option
Summary
A useful feature of Coverage is to sort the report by the reported metrics (docs), rather than just by filename. Especially in legacy or unfamiliar code, sorting by branch or cover helps in trying to understand the areas of risk. It would be helpful to expose this option as one of the flags pytest-cov can pass along to Coverage here.
In a tox project using setup.cfg, for example, the only alternative (I know of) is to set the sorting in the Coverage section, but this imposes a particular sorting and isn't as easy to change quickly, and might become a source of accidental diffs when left changed.
I'd be happy to open a pull request to this effect, but wanted to start with some discussion first 😄
Having not heard back on this, I didn't act on it (and frankly forgot about it since then). I think the implementation is small, so I may attempt to garner feedback by putting something concrete up for folks to discuss in the next couple of weeks.
Another option is to simply use the coverage report command instead of having pytest-cov do the reporting. There's no need for the reporting to happen in the pytest plugin.
Thanks for the suggestion @nedbat!
I'm typically running testing and coverage with tox, where the coverage command isn't directly accessible unless I generate a devenv. It felt like it could be convenient to run e.g. tox -- --cov --cov-sort=stmts but I see that I might be able to get a decent facsimile in two commands if I add a separate reporting environment, e.g. tox -- --cov && tox -e covreport --sort=stmts.
How are you running pytest under tox? Both pytest and coverage are installed in your environment, and both are runnable as either commands or as Python modules. Maybe I'm missing something, but I don't understand what prevents you from running coverage directly.
@nedbat Maybe I didn't explain super well; I like to have coverage run as part of just running tox, and could do that with two sequential commands, but ad hoc I like to change the sorting for a better view of things, and don't like to fiddle with setup.cfg to do it. This is purely a convenience request. I could also really be missing something 😄