python-poetry
`poetry install` run under pre-commit hook does not apply changes.
Description
pre-commit run poetry-install passed but acted as no-op.
The expected package was not in the venv.
A regular: poetry install noticed the missing package.
Workarounds
Manually poetry install.
Poetry Installation Method
pipx
Operating System
Win 10
Poetry Version
v1.8.2
Poetry Configuration
> poetry config --list
cache-dir = "~\\AppData\\Local\\pypoetry\\Cache"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}\\virtualenvs" # ~\AppData\Local\pypoetry\Cache\virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true
Python Sysconfig
> python -m sysconfig
Platform: "win-amd64"
Python version: "3.12"
Current installation scheme: "venv"
Paths:
data = "<repo>\.venv"
include = "~\scoop\apps\python\3.12.3\Include"
platinclude = "~\scoop\apps\python\3.12.3\Include"
platlib = "<repo>\.venv\Lib\site-packages"
platstdlib = "<repo>\.venv\Lib"
purelib = "<repo>\.venv\Lib\site-packages"
scripts = "<repo>\.venv\Scripts"
stdlib = "~\scoop\apps\python\3.12.3\Lib"
Variables:
BINDIR = "<repo>\.venv\Scripts"
BINLIBDEST = "<repo>\.venv\Lib"
EXE = ".exe"
EXT_SUFFIX = ".cp312-win_amd64.pyd"
INCLUDEPY = "~\scoop\apps\python\3.12.3\Include"
LIBDEST = "~\scoop\apps\python\3.12.3\Lib"
TZPATH = ""
VERSION = "312"
VPATH = "..\.."
abiflags = ""
base = "<repo>\.venv"
exec_prefix = "<repo>\.venv"
installed_base = "~\scoop\apps\python\3.12.3"
installed_platbase = "~\scoop\apps\python\3.12.3"
platbase = "<repo>\.venv"
platlibdir = "DLLs"
prefix = "<repo>\.venv"
projectbase = "~\scoop\apps\python\3.12.3"
py_version = "3.12.3"
py_version_nodot = "312"
py_version_nodot_plat = "312"
py_version_short = "3.12"
srcdir = "~\scoop\apps\python\3.12.3"
userbase = "%HOMEDRIVE%%HOMEPATH%\.python"
Example pyproject.toml
[virtualenvs]
in-project = true
Precommit:
default_install_hook_types: [pre-commit, post-checkout]
default_stages: [pre-commit]
fail_fast: true
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/python-poetry/poetry
rev: "1.8.0"
hooks:
- id: poetry-check
- id: poetry-lock
- id: poetry-export
- id: poetry-install
Repo Transcript
> poetry install --dry-run
Installing dependencies from lock file
Package operations: 1 install, 0 updates, 0 removals, 179 skipped
- Installing pip (23.3.1): Skipped for the following reason: Already installed
...
- Installing watermark (2.4.3)
- Installing voluptuous (0.13.1): Skipped for the following reason: Already installed
- Installing wheel (0.41.3): Skipped for the following reason: Already installed
- Installing wcwidth (0.2.9): Skipped for the following reason: Already installed
- Installing yarl (1.9.2): Skipped for the following reason: Already installed
- Installing widgetsnbextension (4.0.9): Skipped for the following reason: Already installed
- Installing tzdata (2023.3): Skipped for the following reason: Already installed
- Installing zipp (3.18.1): Skipped for the following reason: Already installed
- Installing zc-lockfile (3.0.post1): Skipped for the following reason: Already installed
Installing the current project: <package> (0.10.0)
> git checkout
Your branch is up to date with 'origin/<branch>'.
poetry-install...........................................................Passed
dvc checkout.............................................................Passed
> python -c "import watermark"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'watermark'
> poetry env list
.venv (Activated)
I expect you have not read the docs telling you about the extra things you need to do when using this hook https://python-poetry.org/docs/pre-commit-hooks#poetry-install
I expect you have not read the docs telling you about the extra things you need to do when using this hook https://python-poetry.org/docs/pre-commit-hooks#poetry-install
Thanks - I've added a transcript of the output and clarified the hooks in the pre-commit file. Am I missing something else? The docs say something about args but that pre-commit file should be applying to the venv?
And is the python you are running at python -c "import watermark" the same as the python from the virtual environment that poetry is running? Probably you just need to activate the virtual environment (or poetry run ...)
And is the python you are running at
python -c "import watermark"the same as the python from the virtual environment that poetry is running? Probably you just need to activate the virtual environment (orpoetry run ...)
Yes - they are run back to back in the same terminal, dvc wouldn't be found outside this venv. I added the poetry env list.
not a pre-commit user myself, but so far as I know pre-commit likes to run in its own virtual environment.
trying it myself it seems as though it is attempting to install the project packages in the virtual environment containing the pre-commit poetry: which is both doing nothing for the virtual environment that you care about - and likely to fail altogether when the poetry requirements clash with the project requirements.
seems completely broken to me, perhaps we should try asking someone from #8327 what we have misunderstood @alsmnn @radoering
I do not use the poetry pre-commit hooks myself, but trying it I can confirm that this hook seems broken - and I wonder how/if it has ever worked.
Ackchyually it installs all your dependencies, but in the virtual environment determined by pre-commit. You can verify that by looking under:
~/.cache/pre-commit/repo{some_hash_value}/py_env-python{your_python_version}/lib/python{your_python_version}/site-packages/.
If you want to run pre-commit against your local poetry, you can use this config:
default_install_hook_types: [pre-commit, post-checkout, post-merge]
repos:
- repo: local
hooks:
- id: poetry-install
name: poetry-install
args: ["--sync"]
language: system
entry: poetry install
stages: [post-checkout, post-merge]
pass_filenames: false
always_run: true
verbose: true
The verbose: true statement is helpful to see what poetry is actually doing.
I don't know why we thought it would work correctly in the past. It must have slipped my attention, that it just updates the dependencies in the virtual environment which was provided by pre-commit. I will open a PR in the next few days and see if I can correct that mistake.
Thanks for finding that bug!
