pytest-split icon indicating copy to clipboard operation
pytest-split copied to clipboard

pytest-split test collection is not compatible with pytest-testmon

Open marcodicro-dp opened this issue 1 year ago • 1 comments

Is pytest-split compatible with pytest-testmon?

I want ro run all tests in groups (using pytest-split) and then use testmon to run only the affected tests inside those groups. Testmon said no files have changed (no affected tests) but is pytest-split is running all tests anyways:

testmon: changed files: 0, unchanged files: 456, environment: default rootdir: /Users/marcodicro/src/repo configfile: test/project/pytest.ini plugins: flaky-3.8.1, testmon-2.1.1, split-0.8.2 collecting 5771 items

[pytest-split] Splitting tests with algorithm: duration_based_chunks [pytest-split] Running group 1/25 (estimated duration: 231.00s)

collected 5772 items / 5541 deselected / 231 selected

marcodicro-dp avatar May 14 '24 20:05 marcodicro-dp

This issue is still happening, now happened to me while trying to build testmon db running my tests in sequential groups using pytest-split:

pytest tests --testmon-noselect --splits 25 --group 1 -v --ignore=tests/analytics_es_tests/ --rootdir=/__w/sample_repo/sample_repo/test/app/../..

Groups were wrong, tests were repeated between the groups.

marcodicro-dp avatar Jul 19 '24 16:07 marcodicro-dp