Poetry hangs resolving ssh dependencies from unknown hosts

[Yarn]

Description

Running poetry lock will hang forever without any indication of why.

Running poetry lock -vvv shows ssh's interactive prompt, e.g.

The authenticity of host 'github.com (140.82.116.4)' can't be established.
ED25519 key fingerprint is SHA256:+DiY3wvvV6TuJJhbpZisF/zLDA0zPMSvHdkr4UvCOqU.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

This prompt can be accepted and future runs can succeed.

Workarounds

Run poetry lock -vvv and interact with the ssh prompt

Poetry Installation Method

pip

Operating System

Arch

Poetry Version

1.8.3

Poetry Configuration

cache-dir = "/home/ryan/.cache/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/ryan/.cache/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true

Python Sysconfig

No response

Example pyproject.toml

No response

Poetry Runtime Logs

issue does not occur with `-vvv`

[dimbleby]

Too bad, probably. I don't know any way to tell ssh always to fail non-interactively here.

If it shows up in verbose mode, that'll probably have to be good enough

You provided no way to reproduce this.

[Yarn]

After writing the below I noticed -o BatchMode=yes which should disable any interactive prompts.

https://man.archlinux.org/man/ssh_config.5.en#BatchMode

---

I think -o StrictHostKeyChecking=yes would make this specific case fail non-interactively.

-o PasswordAuthentication=no I think would cover #9626

https://man.archlinux.org/man/ssh_config.5.en#StrictHostKeyChecking https://man.archlinux.org/man/ssh_config.5.en#PasswordAuthentication

[Yarn]

to reproduce the issue comment out any entries for github.com in ~/.ssh/known_hosts and ensure no system level ssh configuration disables host key checking for that host then use the following pyproject.toml

[tool.poetry]
name = "test"
version = "0.1.0"
description = ""
authors = [""]

[tool.poetry.dependencies]
poetry = { "git" = "[email protected]:python-poetry/poetry.git" }

[dimbleby]

poetry uses dulwich for git clones, so next step is to persuade dulwich to use that parameter then

suggest you experiment with something like

diff --git a/src/poetry/vcs/git/backend.py b/src/poetry/vcs/git/backend.py
index 0fc00f0b..b3b28026 100644
--- a/src/poetry/vcs/git/backend.py
+++ b/src/poetry/vcs/git/backend.py
@@ -194,7 +194,7 @@ class Git:
         client: GitClient
         path: str

-        kwargs: dict[str, str] = {}
+        kwargs = {"ssh_command": "ssh -o BatchMode=yes"}
         credentials = get_default_authenticator().get_credentials_for_git_url(url=url)

         if credentials.password and credentials.username:

and submit a merge request if you can get it to work satisfactorily

[tplunket]

That isn't the only "failure case" for this though, the bug is that the "Resolving dependencies" message erases messages from the SSH client. E.g. if you are prompted to type in your passphrase to unlock your ssh key, that message gets erased and unless you notice the message flash for an instant and realize what it is, you'll spend hours trying to figure out why adding a git+ssh dependency suddenly starts making dependency resolution take forever. In other words, the messaging that it's resolving dependencies isn't actually accurate and overwriting the prompt that would allow it to proceed obscures what the user needs to do.

(In other words, dulwich seems to be doing the right thing, if only Poetry didn't overwrite the prompts.)

[freekvh]

That isn't the only "failure case" for this though, the bug is that the "Resolving dependencies" message erases messages from the SSH client. E.g. if you are prompted to type in your passphrase to unlock your ssh key, that message gets erased and unless you notice the message flash for an instant and realize what it is, you'll spend hours trying to figure out why adding a git+ssh dependency suddenly starts making dependency resolution take forever. In other words, the messaging that it's resolving dependencies isn't actually accurate and overwriting the prompt that would allow it to proceed obscures what the user needs to do.

(In other words, dulwich seems to be doing the right thing, if only Poetry didn't overwrite the prompts.)

My issue, mentioned here before (https://github.com/python-poetry/poetry/issues/9626) also reports this. It also looks to me that several people in the issues section actually have this as the root of their "hanging" bugs.

My issue was closed but not fixed, I do wonder, is this something that won't be fixed?

I've been working around this by just watching closely, you can see the prompt for a fraction of a second, before the counter over-writes it, then you know what to do (ie enter a password to decrypt your private key).

I love poetry but perhaps maintainers are not seeing these issues because they are auto-closed?

Edit: I see now that my issue was closed as "not planned", so indeed it will be confusion for anyone who uses Git over SSH with an encrypted private key, right? Am I missing something?

[dimbleby]

your issue was correctly closed because it is a duplicate of this one, there is no value in having both. This one is still open.

there is a pull request at https://github.com/python-poetry/poetry/pull/9638 which imo was over-complicating matters but either way looks abandoned.

I expect something that plumbs BatchMode=yes through to dulwich - whether by an environment variable or by explicit arguments or whatever - would be welcome.

[freekvh]

Oh yes I see the label "duplicate" indeed, thanx.

[Yarn]

The UX to address this is still up in the air. I think the only real insight I've had since the discussion on the PR is that having this functionality linked to verbosity level seems more cursed the more I think about it.

I still think pausing the timer output during the git command is the best option, it should essentially handle all cases correctly for a relatively small negative impact on non CI/CD users with git dependencies that result in long running, non interactive git commands.

The above assumes CI/CD users are not watching the console output in real time which is probably an imperfect assumption.

Other solutions, BatchMode=yes or otherwise, all get into the issue that it shifts the burden on the user from "know how git/ssh works" to "know how poetry works". e.g. Permission denied (publickey,password)., even if it's surfaced to the user correctly, won't indicate to most users that the key has a passphrase. My first debugging step would be to set SSH_COMMAND=ssh -v which would either disable batch mode or do nothing with the discussed implementation; both of which would have very confusing results. I'm taking it for granted that trying to compose together multiple ssh_command settings in a remotely reasonable way is an unreasonable amount of complexity.

[andreaso]

UX wise it's also somewhat confusing that the poetry install command will behave differently in this regard depending on whatever you happen to have a poetry.lock file in place or not.

As reported in the (duplicate) issue #10071...

Without any poetry.lock file in place poetry install will hit the exact same behavior as reported in this issue, with any SSH prompt quickly being overwritten.

Yet with an (up-to-date) poetry.lock file in place poetry install will take a different code path, where it will politely wait for you to respond to the SSH prompt.

[Yarn]

https://github.com/python-poetry/poetry/blob/77cd259f745eb4c522083072e81ab0ccbaf2fb49/src/poetry/puzzle/solver.py#L129

The progress indicator is actually part of the dependency solver so it makes sense the issue doesn't occur if there is a lock file.

[jbardo]

Hello,

I wanted to report a similar bug of "resolving dependencies" hanging forever (actually hiding ssh prompt). I see other similar bug reports closed as duplicates of this one, so here is my use case for info:

• poetry version: 1.8.3 initally when observed, reproduced on 2.0.1 (latest to date)
• command that "hangs" on Resolving dependencies... (xxxs): poetry update
• happens when: having private git (ssh) dependencies and no prior ssh-agent setup
• analysis: we can actually see a fraction of a second the ssh prompt and it gets replaced by the resolving dependencies msg with the time increasing
• workarounds:
  • run poetry update with any of options -vv/vvv (-v is not enough, time still running, replacing msg), -q/--quiet or --no-ansi: the ssh prompt will be displayed properly, you can authenticate when prompted to
  • anticipate the need to authenticate and still type your ssh key passphrases, one after another for each needed connection after seeing the ssh msg flashing ^^ : it still works
  • use something like ssh-add prior to using poetry,

Thanks.

[dimbleby]

this one is well understood, we are just waiting for someone to show up with a workable pull request cf https://github.com/python-poetry/poetry/issues/9618#issuecomment-2546524392