pytest-dev
Same named test functions in different `Pytester` tests will use the same `tmp_path` location
If you setup Pytester tests that uses tmp_path then they will share directory if named the same, even if run in different tests. This only happens when run with runpytest not inline_run. So when running this code:
def test_1(pytester: Pytester):
pytester.makepyfile(
"""
def test(tmp_path):
assert False
"""
)
pytester.runpytest()
def test_2(pytester: Pytester):
pytester.makepyfile(
"""
def test(tmp_path):
assert False
"""
)
pytester.runpytest()
both inner test functions will have tmp_path pointing to the same directory.
This might not be a bug outright since I don't think pytester makes any claim for this not to happen, but I think it's a major footgun for using pytester.
Below is a full reproduction that shows the paths are indeed the same in the given situations. When this is run, the fixture will throw an error for TestRunPytest
from pytest import Pytester
import pytest
import re
@pytest.fixture(scope="class")
def extract_and_compare_temp_path():
paths = []
def _saver(lines):
for line in lines:
if match := re.match(r"tmp_path = PosixPath.'(.*)'.", line):
paths.append(match.group(1))
yield _saver
# The paths should not be the same
assert paths[0] != paths[1]
class TestRunPytest: # both test functions uses the same test path
def test_1(self, pytester: Pytester, extract_and_compare_temp_path):
pytester.makepyfile(
"""
def test(tmp_path):
assert False
"""
)
path = pytester.runpytest().outlines
extract_and_compare_temp_path(path)
def test_2(self, pytester: Pytester, extract_and_compare_temp_path):
pytester.makepyfile(
"""
def test(tmp_path):
assert False
"""
)
path = pytester.runpytest().outlines
extract_and_compare_temp_path(path)
class TestRunPytestDifferentFunctionNames: # does not same tmp_path
def test_1(self, pytester: Pytester, extract_and_compare_temp_path):
pytester.makepyfile(
"""
def test(tmp_path):
assert False
"""
)
path = pytester.runpytest().outlines
extract_and_compare_temp_path(path)
def test_2(self, pytester: Pytester, extract_and_compare_temp_path):
pytester.makepyfile(
"""
def test_with_other_name(tmp_path):
assert False
"""
)
path = pytester.runpytest().outlines
extract_and_compare_temp_path(path)
class TestInlineRun: # These have the different temp paths
def test_1(self, pytester: Pytester, extract_and_compare_temp_path):
pytester.makepyfile(
"""
def test(tmp_path):
assert False
"""
)
path = pytester.inline_run().getfailures()[0]
extract_and_compare_temp_path([path.longreprtext])
def test_2(self, pytester: Pytester, extract_and_compare_temp_path):
pytester.makepyfile(
"""
def test(tmp_path):
assert False
"""
)
path = pytester.inline_run().getfailures()[0]
extract_and_compare_temp_path([path.longreprtext])
Env: platform linux -- Python 3.12.5, pytest-8.3.2, pluggy-1.5.0
This May be a side effect of passed test tmppath pruning
Can you cross-check whether the paths get deleted between tests and if the issue persists if the tests fail?
Yes, they get deleted between tests.
def test_1(pytester: Pytester):
pytester.makepyfile(
"""
def test_1(tmp_path):
(tmp_path / "file.txt").touch()
assert (tmp_path / "file.txt").exists()
"""
)
pytester.runpytest().assert_outcomes(passed=1)
def test_2(pytester: Pytester):
pytester.plugins = ["xdist"]
pytester.makepyfile(
"""
def test_1(tmp_path):
assert not (tmp_path / "file.txt").exists()
"""
)
pytester.runpytest("-n", "auto").assert_outcomes(passed=1)
It turns out not be a problem then, because even if you run it with xdist it insert the worker into the name.
My actual problem turned out to be with tmp_path_factory and trying to do the classic making-session-scoped-fixtures-execute-only-once with tmp_path_factory.getbasetemp().parent. I myself am appending the function to the name, so that was a red herring that just happened to match up with the behavior of tmp_path (and I forgot which one I had used...)
So in the following test_b will fail since the file exists as a left over from test_a. Inner function names doesn't matter
from pytest import Pytester, TempPathFactory
def test_a(pytester: Pytester):
pytester.makepyfile(
"""
def test_1(tmp_path_factory):
p = tmp_path_factory.getbasetemp().parent
(p /"file.txt").touch()
"""
)
pytester.runpytest().assert_outcomes(passed=1)
def test_b(pytester: Pytester):
pytester.makepyfile(
"""
def test_1(tmp_path_factory):
p = tmp_path_factory.getbasetemp().parent
assert not (p / "file.txt").exists()
"""
)
pytester.runpytest().assert_outcomes(passed=1)
it think its a footgun as well, especially since the pattern is very common given that it's recommended in xdist docs. Concretely I'm trying to create a wrapper that generalizes the pattern, and I would have to inject behavior into the core to somehow account for running in Pytester versus not running in Pytester (Though obviously that's my problem)
If anyone else has them same issue, I ended up just setting --basetemp to tmp_path:
def test(pytester: Pytester, tmp_path):
pytester.runpytest("--basetemp", str(tmp_path))
@StefanBRas always add a extra foler component for basetemp, pytest nukes basetemp before remakingit, so each pytester call would nuke the tmp_path