MetPy icon indicating copy to clipboard operation
MetPy copied to clipboard

chore: parallelize testing in ci with xdist

Open bcmyguest opened this issue 5 months ago • 3 comments

Description Of Changes

From an idea in here: https://github.com/Unidata/MetPy/pull/3901

Attempting to turn on parallel testing in CI to speed up testing. Locally testing it seemed to work but who knows once it gets in the wild.

Definite DISadvantage here is that it screws up pluggy, but that's only really relevant when running locally anyways and it's not enabled by default. Another possible issue is that matplotlib is not thread safe, so I am not sure how tests will interact with that, it could end up being more flaky than it's worth.

Checklist

  • [x] Tests added -> nope
  • [x] Fully documented -> updated contributing.md

bcmyguest avatar Aug 26 '25 23:08 bcmyguest

Obviously something wonky going on with the coverage but as a POC I think this works (tests run, and they run much faster). I think we would need to use pytest-cov https://pytest-cov.readthedocs.io/en/latest/xdist.html

^ did this

bcmyguest avatar Aug 26 '25 23:08 bcmyguest

Another possible issue is that matplotlib is not thread safe, so I am not sure how tests will interact with that, it could end up being more flaky than it's worth.

For that part of the issue, you could mark the plotting tests, run the non-plotting tests in parallel then the plotting tests serially. MetPy probably has enough non-plotting tests this would speed things up

DWesl avatar Aug 30 '25 11:08 DWesl

@DWesl it actually seemed to be working alright - xdist uses subprocesses instead of threads :) As long as you don't pollute the test namespace with tests that use the same files (eg testing output.png on different processes)

bcmyguest avatar Sep 02 '25 13:09 bcmyguest