Tox 4 trying to reuse env where it didn't in tox 3

[TheRealFalcon]

Issue

Given this config:

[testenv:black]
deps =
    black==23.3.0
commands = {envpython} -m black . --check

[testenv:tip-black]
deps = black
commands = {[testenv:black]commands}

in tox 3.28.0, both executions (via tox -e <env>) worked and ran in different environments. We'd get the following line in the output from the black env:

black run-test: commands[0] | /home/james/c/tox_test/.tox/black/bin/python -m black . --check

vs the following in the tip-black env:

tip-black run-test: commands[0] | /home/james/c/tox_test/.tox/tip-black/bin/python -m black . --check

Notice the path differences.

However, in tox 4.6.4, I get this for black:

black: commands[0]> .tox/black/bin/python -m black . --check

and an error for tip-black:

tip-black: commands[0]> .tox/black/bin/python -m black . --check
tip-black: failed with /home/james/c/tox_test/.tox/black/bin/python (resolves to /home/james/c/tox_test/.tox/black/bin/python) is not allowed, use allowlist_externals to allow it

Why is it trying to reuse the black environment for tip-black? It didn't do that in tox 3.

Environment

• OS: Ubuntu 23.04

Full outputs

tox 3.28.0, black env:

GLOB sdist-make: /home/james/c/tox_test/setup.py
black inst-nodeps: /home/james/c/tox_test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip
black installed: black==23.3.0,click==8.1.6,importlib-metadata==6.7.0,mypy-extensions==1.0.0,packaging==23.1,pathspec==0.11.2,platformdirs==3.10.0,tomli==2.0.1,typed-ast==1.5.5,typing_extensions==4.7.1,UNKNOWN @ file:///home/james/c/tox_test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip,zipp==3.15.0
black run-test-pre: PYTHONHASHSEED='772606141'
black run-test: commands[0] | /home/james/c/tox_test/.tox/black/bin/python -m black . --check
All done! ✨ 🍰 ✨
1 file would be left unchanged.
___________________________________ summary ____________________________________
  black: commands succeeded
  congratulations :)

tox 3.28.0, tip-black env:

GLOB sdist-make: /home/james/c/tox_test/setup.py
tip-black create: /home/james/c/tox_test/.tox/tip-black
tip-black installdeps: black
tip-black inst: /home/james/c/tox_test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip
tip-black installed: black==23.3.0,click==8.1.6,importlib-metadata==6.7.0,mypy-extensions==1.0.0,packaging==23.1,pathspec==0.11.2,platformdirs==3.10.0,tomli==2.0.1,typed-ast==1.5.5,typing_extensions==4.7.1,UNKNOWN @ file:///home/james/c/tox_test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip,zipp==3.15.0
tip-black run-test-pre: PYTHONHASHSEED='1485184410'
tip-black run-test: commands[0] | /home/james/c/tox_test/.tox/tip-black/bin/python -m black . --check
All done! ✨ 🍰 ✨
1 file would be left unchanged.
___________________________________ summary ____________________________________
  tip-black: commands succeeded
  congratulations :)

tox 4.6.4, black env:

black: install_deps> python -I -m pip install black==23.3.0
.pkg: install_requires> python -I -m pip install 'setuptools>=40.8.0' wheel
.pkg: _optional_hooks> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_wheel> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: install_requires_for_build_wheel> python -I -m pip install wheel
.pkg: prepare_metadata_for_build_wheel> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_sdist> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
black: install_package> python -I -m pip install --force-reinstall --no-deps /home/james/c/tox_test/.tox/.tmp/package/1/UNKNOWN-0.0.0.tar.gz
black: commands[0]> .tox/black/bin/python -m black . --check
.pkg: _exit> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
  black: OK (2.56=setup[2.49]+cmd[0.07] seconds)
  congratulations :) (2.60 seconds)

tox 4.6.4, tip-black env:

tip-black: install_deps> python -I -m pip install black
.pkg: _optional_hooks> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_wheel> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: prepare_metadata_for_build_wheel> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_sdist> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
tip-black: install_package> python -I -m pip install --force-reinstall --no-deps /home/james/c/tox_test/.tox/.tmp/package/2/UNKNOWN-0.0.0.tar.gz
tip-black: commands[0]> .tox/black/bin/python -m black . --check
tip-black: failed with /home/james/c/tox_test/.tox/black/bin/python (resolves to /home/james/c/tox_test/.tox/black/bin/python) is not allowed, use allowlist_externals to allow it
.pkg: _exit> python /home/james/.pyenv/versions/3.7.16/envs/cloud-init37/lib/python3.7/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
  tip-black: FAIL code 1 (1.39 seconds)
  evaluation failed :( (1.43 seconds)

[gaborbernat]

Looks to me like a bug. Likely because envpython resolves for the env where it's defined and not instantiatied. That being said there's no real benefit to use {envpython} -m, so if you just write commands = black . --check, thing should work.

PR welcome to fix the bug.