python-poetry
Support for `--config-settings` options for `poetry add --editable`
- [x] I have searched the issues of this repo and believe that this is not a duplicate.
- [x] I have searched the FAQ and general documentation and believe that my question is not already covered.
Feature Request
pip install -e supports the --config-settings argument when installing a package as editable, like so:
pip install -e . --config-settings editable_mode=strict
However, I cannot use it when invoking poetry add --editable
The reason this feature would be useful is because both Pylance and PyCharm require editable packages to be installed with editable_mode=strict or editable_mode=compat for intellisense to work:
https://github.com/microsoft/pylance-release/blob/main/TROUBLESHOOTING.md#editable-install-modules-not-found
https://stackoverflow.com/a/76301809
You should be able to do just that, meaning poetry install --no-root && poetry run pip install -e . ... from the root of your project.
The command you specified does work temporarily, this is what I used:
poetry run pip install -e ./path_to_diff_package --config-settings editable_mode=strict
However the next poetry install or poetry install --no-root overwrites the installation as editable_mode is not saved in pyproject.toml
I'm also running into this topic. My team has a monorepo we use Poetry to manage our dependencies in (including the local packages we install in editable mode while working). The current default approach setuptools uses is to create .pth files that run import hooks. Which works fine while Python is running as an interpreter, but isn't possible for static analysis tools like Pyright/Pylance.
setuptools therefore currently offers the 2 editable modes (strict and compat) which avoid the import hooks. However, the only way I've found to activate these modes is via --config-settings editable_mode=strict (or the environment variable/config variable option) in pip.
The new modern installer in poetry doesn't respect this option (which makes sense, as it doesn't use pip), but disabling the modern installer and setting either an environment variable or a local pip.ini/pip.conf file does work. However, this doesn't seem like a great long-term fix.
I've been digging my way through the source code trying to understand where the modern installer actually generates the pth files to see what's actually going on. So far it looks like Chef internally calls ProjectBuilder.build from the build package. This function can receive a config_settings dictionary (where setting {"editable_mode": "strict"} as an argument should work).
Do you think it would be acceptable/possible to add some sort of configuration option to Poetry to conditionally pass that additional argument to ProjectBuilder?
If this is a change that is likely to be accepted, then I would be willing to try my hand at a PR for it.
What should the user interface look like? I assume this requires a new field in the dependency specification in pyproject.toml? Or can it be a general setting that is applied to all builds? (Probably not because it is not suitable for all build backends.)
That's a good question. My initial (possibly naive) thought was to add a configuration item into poetry.toml for the installer (or the builder?) that essentially just takes one or more key-value pairs, which Chef then uses as the config_settings dictionary when calling ProjectBuilder.build. I don't immediately see an issue with passing a config_settings dictionary to the ProjectBuilder with items it can't consume. But I'm also not sure how other build backends would handle that. As opposed to doing this just for editable_mode. I remember running across a few other packages that seem to want to pass other key/value pairs to the build backend while researching this topic. I didn't, however, make note of those particular use cases.
In my team's particular case, it would work fine as a project-wide setting, because all of our build backends are the same. And if it's not set then the current approach would apply.
I guess it's partially also a question of how the build package handles this, as Chef appears to only use that builder? And build is then ultimately the one to parse/process the pyproject.toml file for the install. At least that was my understanding of the chef.py file.
https://github.com/python-poetry/poetry/blob/cff4d7d51c9ac0f9a8b9ac91652b6dc20e63e203/src/poetry/installation/chef.py#L145-L148
the build system supports passing ConfigSettings to the build function:
https://build.pypa.io/en/stable/api.html
so all poetry has to do is support the "config-settings" key for dependencies and pass it to the build function
should work fine, and this, incidentally, should fix like a dozen issues with editable installations, remote repositories, support for legacy pacakges, gpu-specific builds, etc. for example: https://github.com/python/mypy/issues/13392
for example:
mypackage = {path = "../mypackage", develop = true, config-settings={editable_mode="strict"}}
if there's no objections to this i could try digging into the code and seeing how to get those settings passed through
PR https://github.com/python-poetry/poetry-core/pull/715
@earonesty - I also just started working on this change: https://github.com/python-poetry/poetry/compare/main...lswest:poetry:add_config_settings
I tested my approach locally and it seems to work, but your approach certainly seems better integrated into poetry itself.
If I'm understanding your PR correctly, you'd be adding the config_settings to the actual dependency entries in the pyproject.toml entry, correct? Versus my approach, where I was adding it to the poetry.toml file as a project-wide setting.
Correct. Some packages and build systems may not respond well to certain settings. I thought it would be better per-package
I would like to apologize for dropping the ball on finishing this. I don't have a PR up for the main repo only the type repo. I'm not going to have time to work on this for another month probably because I'm going to be traveling for work.