installing hook on pre-commit fails with newest virtualenv (>20.24.5) present

[fizyk]

virtualenv 20.24.6 updates embed setuptools from 6.2.0 to 6.2.2 and pip to 23.3.1

pip 23.3 includes some changes to dependency resolution, so that's what might cause the issue.

Pre-commit config:

  - repo: https://github.com/jorisroovers/gitlint
    rev: v0.19.1
    hooks:
      - id: gitlint
      - id: gitlint-ci
        stages:
          - manual
        args: [--commits, origin/master..HEAD]

results of running on a clean pre-commit env (cleared pre-commit cache)

❯ pre-commit run --hook-stage manual gitlint-ci                                                                                                                                                                                    ─╯
[INFO] Installing environment for https://github.com/jorisroovers/gitlint.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('[HOME]/.cache/pre-commit/repos8g02w1x/py_env-python3.10/bin/python', '-mpip', 'install', '.', './gitlint-core[trusted-deps]')
return code: 1
stdout:
    Processing /home/fizyk/.cache/pre-commit/repos8g02w1x
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'done'
    Processing ./gitlint-core
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'done'
    INFO: pip is looking at multiple versions of gitlint to determine which version is compatible with other requirements. This could take a while.
stderr:
    ERROR: Could not find a version that satisfies the requirement gitlint-core==0.1.dev1+gacc9d9d (from gitlint) (from versions: 0.17.0, 0.18.0, 0.19.0.dev73, 0.19.0.dev75, 0.19.0.dev76, 0.19.0.dev77, 0.19.0.dev78, 0.19.0.dev79, 0.19.0.dev80, 0.19.0.dev81, 0.19.0.dev82, 0.19.0rc1, 0.19.0rc2.dev1, 0.19.0rc2, 0.19.0rc3.dev1, 0.19.0, 0.19.1.dev1, 0.19.1.dev2, 0.19.1, 0.19.2.dev1, 0.19.2.dev2, 0.19.2.dev3, 0.19.2.dev4, 0.19.2.dev5, 0.20.0.dev2, 0.20.0.dev3, 0.20.0.dev6, 0.20.0.dev7, 0.20.0.dev8, 0.20.0.dev9, 0.20.0.dev10, 0.20.0.dev11, 0.20.0.dev12, 0.20.0.dev13, 0.20.0.dev14, 0.20.0.dev15, 0.20.0.dev16, 0.20.0.dev17, 0.20.0.dev18, 0.20.0.dev19, 0.20.0.dev20, 0.20.0.dev21, 0.20.0.dev22, 0.20.0.dev23, 0.20.0.dev24, 0.20.0.dev25, 0.20.0.dev27, 0.20.0.dev29, 0.20.0.dev30, 0.20.0.dev31, 0.20.0.dev32, 0.20.0.dev33, 0.20.0.dev34, 0.20.0.dev35, 0.20.0.dev36, 0.20.0.dev37, 0.20.0.dev38, 0.20.0.dev39, 0.20.0.dev40, 0.20.0.dev41, 0.20.0.dev42, 0.20.0.dev43, 0.20.0.dev44, 0.20.0.dev45, 0.20.0.dev48)
    ERROR: No matching distribution found for gitlint-core==0.1.dev1+gacc9d9d
Check the log at [HOME]/.cache/pre-commit/pre-commit.log

[KochankovID]

It looks like there is a problem with new pip version. The obvious fix is: pip install virtualenv==v20.24.5 before running pre-commit

Probably it's somehow related to this new feature: https://github.com/pypa/pip/issues/11924

[edspc]

I got this issue with pre-commit installed via Homebrew.

Command to apply the fix from @KochankovID:

$(brew --prefix)/opt/pre-commit/libexec/bin/python3 -m pip install virtualenv==v20.24.5

[SterlingPeet]

I can not use the virtualenv version pinning solution because I cannot easily ask other developers on my team to apply it. However, ~~it appears that by manually specifying the matching gitlint-core version does provide a workaround that does not force other developers to run extra commands~~; albeit at the cost of revisiting the workaround for every gitlint version update until virtualenv/pip is fixed upstream:

- repo: https://github.com/jorisroovers/gitlint
  rev: v0.19.1
  hooks:
  - id: gitlint
    additional_dependencies: [gitlint-core==v0.9.1]
    stages: [commit-msg]

[SterlingPeet]

Upon further testing, my solution seems to cause a version conflict when actually running the hook install. Embarrassingly I was fooled by running pre-commit run --all-files which did not invoke this check.

[fizyk]

20.24.7 fails as well.

[sigmavirus24]

Rather than posting here, please test and add details to https://github.com/pypa/pip/issues/12372 so pip developers prioritize user needs

[vfazio]

Should we ask pre-commit to pin virtualenv to <=20.24.5 until virtualenv includes a new version of pip without the regression? Or ask pypa to revert the pip change in virtualenv.

[sigmavirus24]

I doubt you'll have much luck with either. This needs to be fixed in pip

[vfazio]

I mentioned it in https://github.com/pypa/pip/issues/12372#issuecomment-1857964529, but it should be possible to workaround this issue by telling virtualenv the version of pip to use when seeding the venvs:

VIRTUALENV_PIP=23.3.2 pre-commit install-hooks

Only necessary until virtualenv either updates the pip package on its schedule or provides 23.3.2+ by default.

[SterlingPeet]

My workaround (that actually works this time) was to go to the pre-commit git repo for gitlint on my disk, and download the tags for the repository. This allowed setuptools-scm to resolve the desired tag locally while building the wheel. This would seem to imply that the logic could be fixed in several places to provide a better experience:

• In pip itself, presumably by consulting the upstream repo for tag/commit associations
• In setuptools-scm by checking the origin repository for tags that might be missing in the local repo
• In gitlint by providing a fallback version, for this exact scenario where tag is not properly downloaded into the local repo

Fallback configuration is documented for setuptools-scm in the configuration parameters, and updating it could be rolled into the release tagging process. Or, the fallback could be set to something sane but conservative, like the latest major version.

[vfazio]

New virtualenv tag that bumps pip to 24.0 https://github.com/pypa/virtualenv/releases/tag/20.25.1.

[sigmavirus24]

@vfazio as in that version of pip fixes this bug?

[vfazio]

@sigmavirus24

pip 23.3.2 technically fixed it, but no version of virtualenv was tagged with that version (though it would have slowly picked up the new pip at some point due to it's internal update policy). virtualenv now specifically embeds 24.0, which includes the fix as well.

[vfazio]

For reference: https://github.com/pypa/pip/commit/69b58102f522f0e10d2b3bd31e7b9ee074abd16d

[sigmavirus24]

So I'll close this then. The resolution being either

• use a version of pip older than 23.3, or,
• use a version of pip that's at least 23.3.2