python-poetry
Relative dependency imports work in some folders but not others.
Description
I have 3 Poetry packages within the same folder streamline_logger, grant_vectorizer, and doc_parsing. grant_vectorizer and doc_parsing both use streamline_loggerand import it in the same way, inpyproject.tomlthe line looks like this:streamline_logger = {path = "../streamline_logger"}`
When I run poetry install in doc_parsing, it installs correctly and I can execute import streamline_logger. Looking into the env folder, there's 3 related folders: streamline_logger, streamline_logger-0.1.2.dist-info, and streamline_doc_parsing-0.1.0.dist-info. Oddly, despite working in this folder, running poetry show streamline_logger, it says that its not found.
Now, in grant_vectorizer, after install, there's only one related folder in the env: streamline_logger-0.1.2.dist-info. poetry show streamline_logger returns the correct results, but attempting import streamline_logger fails with a module not found error.
I have attempted clearing both caches, pulling fresh copies of the branches, reinstalling environments, and lots of other fiddling. I am super stumped on and would really appreciate some help.
Workarounds
No
Poetry Installation Method
system package manager (eg: dnf, apt etc.)
Operating System
Mac OS 13.5 (22G74)
Poetry Version
1.7.1
Poetry Configuration
cache-dir = "/Users/erichasegawa/Library/Caches/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "env"
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true
Python Sysconfig
Output too long; cannot place
Example pyproject.toml
[tool.poetry]
name = "streamline-doc-parsing"
version = "0.1.0"
description = ""
authors = ["Eric Hasegawa"]
readme = "README.md"
packages = [{include = "src"}]
[tool.poetry.dependencies]
python = "3.11.4"
python-dotenv = "^1.0.0"
openai = "1.14.2"
pinecone-client = "^2.2.4"
tiktoken = "^0.5.1"
boto3 = "^1.28.54"
slack-sdk = "^3.22.0"
botocore = "^1.31.54"
wrapt = "^1.15.0"
pydantic = "^2.4.1"
sentry-sdk = "^1.31.0"
langchain = "^0.1.0"
unstructured = {extras = ["all-docs"], version = "^0.14.3"}
scipy = "1.10.1"
pdf-annotate = "^0.12.0"
matplotlib = "3.7.2"
statistics = "^1.0.3.5"
torch = ">=2.0.0, !=2.0.1, !=2.1.0"
supabase = "^2.0.3"
unidecode = "^1.3.8"
pinecone-text = "^0.8.0"
nltk = "^3.8.1"
jinja2 = "3.1.4"
cryptography = "42.0.4"
onnx = "1.16.0"
langchain-core = "0.1.35"
idna = "3.7"
tqdm = "4.66.3"
transformers = "4.38.0"
pdf2image = "^1.17.0"
streamline_logger = {path = "../streamline_logger"}
[tool.poetry.group.dev.dependencies]
pyinstaller = { version = "<6.0.0", python = ">=3.11,<3.13" }
black = "^24.4.2"
flake8 = "^7.0.0"
flake8-black = "^0.3.6"
[tool.poetry.scripts]
build = "src.pyinstaller:install"
[tool.black]
line-length = 79
[tool.flake8]
max-line-length = 79
ignore = ["E203"]
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Poetry Runtime Logs
N/A
Hey @EricHasegawa,
it's hard for me to follow how to reproduce your issue. Can you please provide a step-by-step description for it?
fin swimmer
I've run into a similar situation. Where this works in some places and not others.
Specifically, for me this seems to work when I add the dependency to a group. Although this may be a poetry 2 thing of some form. Because I have both of:
[project]
dependencies = [
# ...
]
[tool.poetry.dependencies]
# Needed for versioning the project
python = ">=3.12,<3.13"
To reproduce...
Create one python project (poetry packaged) with the directory name foo_bar. And make sure that project builds (eg with python -m build).
Then make a second project inside the same parent directory that references it.
Try to make the second project depend on foo_bar.
This works
[project]
dependencies = ["click (>=8.1.8,<9.0.0)"]
[tool.poetry.dependencies]
python = ">=3.12,<3.13"
[tool.poetry.group.local.dependencies]
foo_bar = { path = "../foo_bar/" }
But this does not work. The package foo_bar does not get installed:
[project]
dependencies = ["click (>=8.1.8,<9.0.0)"]
[tool.poetry.dependencies]
python = ">=3.12,<3.13"
foo_bar = { path = "../foo_bar/" }
[tool.poetry.group.local.dependencies] foo_bar = { path = "../foo_bar/" }
Thanks, grouping relative path dependencies solved it for me too. Very annoying issue
This is expected behavior although we should probably print a warning instead of silently ignoring it.
If both project.dependencies and tool.poetry.dependencies are defined, the latter is only used to add additional information to the former. You cannot add additional dependencies. That is what we try to say in project.dependencies and tool.poetry.dependencies.
In other words, the following example does not make sense because you declare foo_bar in your dependencies for locking but not in your metadata:
[project]
dependencies = ["click (>=8.1.8,<9.0.0)"]
[tool.poetry.dependencies]
foo_bar = { path = "../foo_bar/" }
If foo_bar is a real dependency, than you should do at least:
[project]
dependencies = ["click (>=8.1.8,<9.0.0)", "foo_bar"]
[tool.poetry.dependencies]
foo_bar = { path = "../foo_bar/" }
If foo_bar is just for development, than it should not be in tool.poetry.dependencies but in a group.
Yes I think both warning and documentation uplift would help here.
If the existence of a section causes some but not all information to be ignored from another section then the documentation ought to call out explicitly what is ignored. And since the only reason for this occurrence is user error then a warning would most defiantly help.