Setuptools no longer compatible with pinned version of pipenv when used with git-repo version spec and multiple install_requires

[rschwiebert]

I'm sure I'm probably the only user hindered by this right now, but it appears that my build started failing because of the recent moves of setuptools and pipenv versions and using a git-repo hash specification in Pipfile.

It produces this distinctive error that appears elsewhere online, but with no satisfactory solutions in this case:

error in django-publications setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; super(type, obj): obj must be an instance or subtype of type

Steps to reproduce:

1. python3.10 -m venv test && source test/bin/activate
2. pip install pip==22.0.4 setuptools==60.10.0 Pipenv==2020.11.15 wheel==0.37.1 These were the versions on my first failing build.
3. Create Pipfile with the following contents:

       [[source]]
       name = "pypi"
       url = "https://pypi.org/simple"
       verify_ssl = true
   
       [dev-packages]
   
       [packages]
       django-publications = {git = "https://github.com/rschwiebert/django-publications.git",ref = "19dd80fef4d1defc6d553eaddab5aa028b0d4f57"}
   
       [requires]
       python_version = "3.10"
       ```
   

4. pipenv install (will create the lockfile, and apparently succeed in installation
5. pipenv install (will now throw the error that is killing my build)

I realize the double install at the end is weird, but it's what reproduces the issue. PIpenv is building up some state that causes the second invocation to fail, it seems, and something similar seems to be happening in the builder.

I've confirmed that this error goes away when I either

1. Downgrade setuptools to 57.5.0 (my last successful build)
2. Upgrade pipenv to 2022.5.2
3. I use a pipenv specification that isn't a git-repo spec.

The pinned version of pipenv is getting pretty old now so I'd imagine you'd consider moving it forward. In the meantime I am testing to see if moving my version of django-publications to a non-git spec resolves things for me.

[edmorley]

@rschwiebert Hi! Thank you for filing this.

Pipenv has been a constant source of bugs/compatibility issues over the months. In theory Pipenv ships its own setuptools, so should not be affected by the version installed by the buildpack, however Pipenv's isolation clearly isn't working. Between these issues and general concerns over Pipenv maintenance, if it weren't for the fact that the buildpack already supports Pipenv (as a beta feature mind), I wouldn't add support now.

I'd recommend using Poetry instead of Pipenv, if you are wanting something more fully featured than Pip. Whilst the buildpack currently doesn't support Poetry natively at the moment (I am aiming to add support once the CNB migration is complete), there are third party buildpacks that can be used alongside the Python buildpack to add support in the meantime, eg: https://github.com/moneymeets/python-poetry-buildpack

Re this specific issue, the django-publications package does use the wrong syntax in its setup.py: https://github.com/lucastheis/django-publications/blob/9fb68ee76f430cf3baa2c7630665235dcf7ae2fe/setup.py#L15

From https://setuptools.pypa.io/en/latest/references/keywords.html :

install_requires A string or list of strings specifying what other distributions need to be installed when this one is.

...whereas django-publications uses a tuple.

I would file an issue against django-publications to ask them to fix it.

The reason the error goes away when not using a git repo spec, is that pip will end up using the published wheel (https://pypi.org/project/django-publications/#files), and using a wheel skips the setuptools build step (as a wheel is a pre-built distribution).

As for why upgrading Pipenv also avoids the issue, I'm not sure. I can definitely look into upgrading Pipenv in the future, however Pipenv upgrades have been pretty risky in the past (due to upstream breaking changes/bugs), so would need to performed carefully, as it might easily annoy an even larger number of pipenv users than this issue.

[rschwiebert]

Thanks for your response @edmorley ! Actually I tried it using a list rather than a tuple, and it made no difference. I don't think django-publication in any position to make a meaningful change. It's really a pip-pipenv wart. And it seems pipenv has it sorted out in their latest version, so I figured it would make sense to recommend moving it forward here.

Fortunately, number 3 on my list of workarounds above (using the latest version of django-publications in pypi) works just fine, since part of the problem is apparently linked to the version-spec using the git repo in the Pipfile.

You're right, I should switch to poetry. I started using pipenv back when it was the new kid on the block, and I have also grown tired of the parade of problems that seems to result from it.

[edmorley]

Closing since this appears to be an upstream pipenv issue.