python-poetry
Poetry gives cryptic error on add git dependency attempt
- Poetry version: 1.6.1
- Python version: 3.11.4
- OS version and name: Mac OS 12.4
- [X] I am on the latest stable Poetry version, installed using a recommended method.
- [X] I have searched the issues of this repo and believe that this is not a duplicate.
- [X] I have consulted the FAQ and blog for any relevant entries or release notes.
- [X] If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption) and have included the output below.
Issue
I am trying to add git dependency, which is work absolutely fine with
pip install git+https://github.com/Lightning-AI/lightning@master
But when I am trying to do poetry add git+https://github.com/Lightning-AI/lightning.git#master, poetry fails.
adding lightning = {git = "https://github.com/Lightning-AI/lightning.git", branch = "master"} in pyproject.toml and invoking poetry update leads to the same result
poetry stack trace:
Stack trace:
19 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/cleo/application.py:327 in run
325│
326│ try:
→ 327│ exit_code = self._run(io)
328│ except BrokenPipeError:
329│ # If we are piped to another process, it may close early and send a
18 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/console/application.py:190 in _run
188│ self._load_plugins(io)
189│
→ 190│ exit_code: int = super()._run(io)
191│ return exit_code
192│
17 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/cleo/application.py:431 in _run
429│ io.input.interactive(interactive)
430│
→ 431│ exit_code = self._run_command(command, io)
432│ self._running_command = None
433│
16 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/cleo/application.py:473 in _run_command
471│
472│ if error is not None:
→ 473│ raise error
474│
475│ return terminate_event.exit_code
15 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/cleo/application.py:454 in _run_command
452│
453│ try:
→ 454│ self._event_dispatcher.dispatch(command_event, COMMAND)
455│
456│ if command_event.command_should_run():
14 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/cleo/events/event_dispatcher.py:26 in dispatch
24│
25│ if listeners:
→ 26│ self._do_dispatch(listeners, event_name, event)
27│
28│ return event
13 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/cleo/events/event_dispatcher.py:89 in _do_dispatch
87│ break
88│
→ 89│ listener(event, event_name, self)
90│
91│ def _sort_listeners(self, event_name: str) -> None:
12 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/console/application.py:322 in configure_installer_for_event
320│ return
321│
→ 322│ cls.configure_installer_for_command(command, event.io)
323│
324│ @staticmethod
11 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/console/application.py:329 in configure_installer_for_command
327│
328│ poetry = command.poetry
→ 329│ installer = Installer(
330│ io,
331│ command.env,
10 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/installation/installer.py:74 in __init__
72│
73│ if installed is None:
→ 74│ installed = self._get_installed()
75│
76│ self._installed_repository = installed
9 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/installation/installer.py:443 in _get_installed
441│
442│ def _get_installed(self) -> InstalledRepository:
→ 443│ return InstalledRepository.load(self._env)
444│
8 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/repositories/installed_repository.py:274 in load
272│ continue
273│
→ 274│ package = cls.create_package_from_distribution(distribution, env)
275│
276│ if with_dependencies:
7 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/repositories/installed_repository.py:132 in create_package_from_distribution
130│ source_url,
131│ source_reference,
→ 132│ ) = cls.get_package_vcs_properties_from_path(src)
133│ break
134│
6 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/repositories/installed_repository.py:80 in get_package_vcs_properties_from_path
78│ from poetry.vcs.git import Git
79│
→ 80│ info = Git.info(repo=src)
81│ return "git", info.origin, info.revision
82│
5 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/vcs/git/backend.py:175 in info
173│ @classmethod
174│ def info(cls, repo: Repo | Path) -> GitRepoLocalInfo:
→ 175│ return GitRepoLocalInfo(repo=repo)
176│
177│ @staticmethod
4 <string>:3 in __init__
1│
3 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/vcs/git/backend.py:147 in __post_init__
145│ repo = Git.as_repo(repo=repo) if not isinstance(repo, Repo) else repo
146│ self.origin = Git.get_remote_url(repo=repo, remote="origin")
→ 147│ self.revision = Git.get_revision(repo=repo)
148│
149│
2 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/vcs/git/backend.py:171 in get_revision
169│ def get_revision(repo: Repo) -> str:
170│ with repo:
→ 171│ return repo.head().decode("utf-8")
172│
173│ @classmethod
1 ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/dulwich/repo.py:585 in head
583│ def head(self) -> bytes:
584│ """Return the SHA1 pointed at by HEAD."""
→ 585│ return self.refs[b"HEAD"]
586│
587│ def _get_object(self, sha, cls):
KeyError
b'HEAD'
at ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/dulwich/refs.py:326 in __getitem__
322│ This method follows all symbolic references.
323│ """
324│ _, sha = self.follow(name)
325│ if sha is None:
→ 326│ raise KeyError(name)
327│ return sha
328│
329│ def set_if_equals(
330│ self,
duplicate #6873, #7523 (which are however both closed, there should likely be something open tracking this)
no-one has ever provided a way to reproduce this from a clean start, please do so if you can. A dockerfile would be a good way to present that.
I just experienced this error myself trying to run poetry add with a git url. Seems like others in the old issues created this bug the same way. There should definitely be an open issue for this...
This was the command I ran, then I couldn't run any poetry commands until I went and deleted the src folder as specified in the old issues.
poetry add --group dev git+ssh://[email protected]:trialspark/graphene-stubs.git
the error that you are reporting is completely different from what is described by this issue, please don't hijack it.
still no-one has provided a way to reproduce
I didn't find how to reproduce, but I find cause and workaround:
rm all folders in .venv/src/ then re-run your command.
One or multiple git repos in this directory are broken.
for example:
/proj/test/.venv/src/httpx (master) # git rev-parse head
warning: ignoring dangling symref head
warning: ignoring dangling symref head
fatal: ambiguous argument 'head': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
head
So the solution would be re-clone broken git repo.
@dimbleby I finally managed to reproduce it. As told above it happens when something is broken at venv/src.
Specifically it happens when there is an empty git repo with the same name of the package you are going to install from a git URL:
FROM ubuntu:jammy
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV DEBIAN_FRONTEND=noninteractive
# create virtual env and simulate activation
RUN mkdir /opt/test
WORKDIR /opt/test
RUN apt-get update && apt-get install -y --no-install-recommends git python3.10 python3.10-venv
RUN python3 -m venv .venv
ENV VIRTUAL_ENV /opt/test/.venv
ENV PATH /opt/test/.venv/bin:$PATH
RUN pip install "poetry==1.6.1"
RUN poetry init -n
# make poetry break by creating a blank git repo at .venv/src/ for the package we are going to install
RUN mkdir -p .venv/src/requests \
&& cd .venv/src/requests \
&& git init \
&& git remote add origin "https://github.com/psf/requests.git"
# it should break with KeyError here
RUN poetry -vvv add "git+https://github.com/psf/requests.git"
How would that happen in a non-artificial way?
If this is the repro then the solution is: don't do that!
How would that happen in a non-artificial way?
If this is the repro then the solution is: don't do that!
network error may cause git repo broken
How would that happen in a non-artificial way?
If this is the repro then the solution is: don't do that!
Im guessing this happened because peotry didn't remove broken git repo in venv/src when git command exit with non-zero exit code
How would that happen in a non-artificial way?
If this is the repro then the solution is: don't do that!
You have been asking for a dockerfile to reproduce the bug, it's there. Have you at least ran it?
If Poetry reads the venv/src folder then the state of that folder is one of the possible states of the Poetry program. It's not "artificial". What's a bug if not the program entering in an unexpected state and breaking? The filesystem is external to your program and can be written by anything, you can't assume the folder won't be in an invalid state. As @trim21 told above, if the network connection breaks while Poetry is cloning the git repo the folder will be in a broken state. Poetry should be able to handle that since it's a networked application.
The bug exist. If it should be fixed or not is another debate.
No-one should care about "bugs" that involve the user deliberately breaking things - sorry.
The desired repro would be eg a series of poetry commands resulting in a bad state.
Network problems is an interesting suggestion; though so far as I remember no-one in this or the linked issues has reported any such thing - my guess is that there is some more "regular" repro out there waiting to be found.
No-one should care about "bugs" that involve the user deliberately breaking things - sorry.
You are reading from the filesystem, the folder is accessible to all programs running under the owner. This is not private process memory.
Network problems is an interesting suggestion; though so far as I remember no-one in this or the linked issues has reported any such thing - my guess is that there is some more "regular" repro out there waiting to be found.
Some people have reported network issues may cause that, see comments from related issues https://github.com/python-poetry/poetry/issues/7523#issuecomment-1725657834
Another way to reproduce the confirmed bug follows, here I kill the socket connection to github, I guess this is also artificial? lol
Save this bash script as ´testpoetry.sh` in the same folder of the following Dockerfile
#!/bin/bash
# simulate broken network connection, it will result in a blank git repo at .venv/src/requests, same as the previous reproduction
bash -c "sleep 2 && ss -K dst github.com" &
poetry -vvv add "git+https://github.com/psf/requests.git" > /dev/null 2>&1
# it should break with KeyError here
poetry -vvv add "git+https://github.com/psf/requests.git"
Build the docker image with docker build -t testpoetry . using the following Dockerfile:
FROM ubuntu:jammy
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV DEBIAN_FRONTEND=noninteractive
# create virtual env and simulate activation
RUN mkdir /opt/test
WORKDIR /opt/test
RUN apt-get update && apt-get install -y --no-install-recommends iproute2 git python3.10 python3.10-venv
RUN python3 -m venv .venv
ENV VIRTUAL_ENV /opt/test/.venv
ENV PATH /opt/test/.venv/bin:$PATH
RUN pip install "poetry==1.6.1"
RUN poetry init -n
ADD testpoetry.sh .
USER root
ENTRYPOINT ["/bin/bash", "/opt/test/testpoetry.sh"]
Run the docker container with docker run --privileged testpoetry. I needs root access to kill the socket with ss.
I think you can also reproduce that by killing poetry itself after it start cloning the github repo. The bug is not directly caused by a network error, but caused by Poetry trying to use a broken cached clone. The fix requires poetry to check the integrity of the local clone to decide if it should be reused or not.
IMO, it's fair enough to suppose the git repo got broken somehow and if there is a good way to detect this state, I assume we will accept a PR that makes poetry more robust.
Me and multiple members of a team I am working with had this happen to them because of connection issues and DNS caching. It's not artificial at all, the reproduction is just a way to test the behaviour.
As the error reporting of poetry is so bare bones and often just shows a str representation of the exception it was quite challenging to find the cause and workaround.
PS: It can also happen if a git server changes identity for example.
I think you can also reproduce that by killing poetry itself after it start cloning the github repo
Here follows a reproduction of the same bug by terminating Poetry in the middle of the installation. Don't tell me a user pressing Ctrl + C is artificial.
testpoetry.sh
#!/bin/bash
poetry -vvv add "git+https://github.com/psf/requests.git" > /dev/null 2>&1 &
echo "Poetry launched on background, terminating it after 3 secs..."
sleep 3
kill -TERM "$!" # Ctrl + C
wait "$!"
echo "Poetry terminated, it should be broken now. Next command will fail with KeyError"
poetry -vvv add "git+https://github.com/psf/requests.git"
Dockerfile:
FROM ubuntu:jammy
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV DEBIAN_FRONTEND=noninteractive
# create virtual env and simulate activation
RUN mkdir /opt/test
WORKDIR /opt/test
RUN apt-get update && apt-get install -y --no-install-recommends python3.10 python3.10-venv
RUN python3 -m venv .venv
ENV VIRTUAL_ENV /opt/test/.venv
ENV PATH /opt/test/.venv/bin:$PATH
RUN pip install "poetry==1.6.1"
RUN poetry init -n
ADD testpoetry.sh .
USER root
ENTRYPOINT ["/bin/bash", "/opt/test/testpoetry.sh"]
IMO, it's fair enough to suppose the git repo got broken somehow and if there is a good way to detect this state, I assume we will accept a PR that makes poetry more robust.
This is the end of the stacktrace where execution leaves poetry and enters dulwich (as reported by OP):
2 .venv/lib/python3.10/site-packages/poetry/vcs/git/backend.py:171 in get_revision
169│ def get_revision(repo: Repo) -> str:
170│ with repo:
→ 171│ return repo.head().decode("utf-8")
172│
173│ @classmethod
1 .venv/lib/python3.10/site-packages/dulwich/repo.py:636 in head
634│ def head(self) -> bytes:
635│ """Return the SHA1 pointed at by HEAD."""
→ 636│ return self.refs[b"HEAD"]
637│
638│ def _get_object(self, sha, cls):
KeyError
b'HEAD'
at .venv/lib/python3.10/site-packages/dulwich/refs.py:325 in __getitem__
321│ This method follows all symbolic references.
322│ """
323│ _, sha = self.follow(name)
324│ if sha is None:
→ 325│ raise KeyError(name)
326│ return sha
327│
328│ def set_if_equals(
329│ self,
I think we need a way to check if the HEAD ref exist, if not we should drop the local clone. The repo might be broken in other ways, but so far all issues report this KeyError which happens when trying to access missing head ref.
Apparently "artificial" hit a nerve. Of course reproductions that involve network issues and (to a slightly lesser extent) Ctrl-C are more interesting than reproductions that involve self-sabotage! Thanks.
As radoering says, I expect a PR to make poetry more robust will be welcome. Even spotting this error - perhaps first asking dulwich to raise a more specific exception so that it can be more easily recognised - and printing a more helpful "here's what to do next" message would be a good start.
It was never "self-sabotage", all the 3 reproductions I made makes Poetry break by leaving a broken git clone at venv/src/requests. In a next installation of requests from git it will try to reuse the local broken clone. The first reproduction just shows the root cause is not a network error neither a innocent Ctrl + C from the user.
But I agree, first dulwich need to handle missing head, it could raise some defined exception and document in their API. I also think the local clone integrity should be checked there.
This has happened to a few times. I believe it happens when you install a package using poetry, and then switch it to a git repo. This is common if you are using a package, have a problem and want to contribute to fix it, create a fork to contribute, and then change the pyproject.toml to the new fork.
When I have run into this problem, a reliable fix for me has been to delete the problem package from the python virtual environment on the filesystem, and then it will be created correctly when updating poetry deps again.
The error is common enough that the poetry team should add some more descriptive text or resolve it properly.
