Additional source is being ignored while inheriting dependencies

[velykanov]

• [x] I am on the latest Poetry version.

• [x] I have searched the issues of this repo and believe that this is not a duplicate.

• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS Big Sur 11.6

• Poetry version: 1.1.11

Issue

Hi guys! I'm using poetry for loading multiple projects and they depend on one another with the following structure:

.
├── project_1
│   └── pyproject.toml
└── project_2
    └── pyproject.toml

project_1/pyproject.toml

[[tool.poetry.source]]
name = "artifactory"
url = "https://pypi.example.com"
default = true

[tool.poetry.dependencies]
private_package = {version ="0.0.1", source = "artifactory"}

project_2/pyproject.toml

[tool.poetry.dependencies]
requests = "2.26.0"
fastapi = "0.65.3"

[tool.poetry.dependencies.project_1]
path = "../project_1"

And while I'm trying to execute poetry install or poetry update in project_2 it fails to find private_package (despite the same commands are working well inside project_1) It makes me think that additional source, which was added in project_1 doesn't affect project_2

Can you please tell me if that's working well as designed and I'm trying to do something which is not supported entirely or if I'm misusing a tool and there's a lack of configurations?

[velykanov]

I've found a possible workaround: I was able to resolve this by setting full url to the private package private_package = {url = "https://pypi.example.com/full/path/to/private_package-0.0.1-py3-none-any.whl"}

In this case [[tool.poetry.source]] section is redundant

[SimonVerhoek]

I've found a possible workaround: I was able to resolve this by setting full url to the private package private_package = {url = "https://pypi.example.com/full/path/to/private_package-0.0.1-py3-none-any.whl"}

In this case [[tool.poetry.source]] section is redundant

@velykanov Just tried this during a Docker build for a private GitLab repository and I keep running into authentication issues. Could you share how you did the authentication for your private repository?

[velykanov]

@SimonVerhoek what exact auth issues do you have (any traceback or error description can be helpful)? Failure to access private gitlab repo might be resolved by adding ssh private key (for ssh method) or personal access token (for http method) into your docker (you can find how this might be done here)

I'm afraid I cannot disclose all the information, but in general our services are under the same private network so they don't need to have reauth process

[usamec]

This is happening to us as well.

[dimbleby]

It makes me think that additional source, which was added in project_1 doesn't affect project_2

That would be true and expected. If project_2 needs an additional source, you'll have to configure it there.

[velykanov]

@dimbleby how come it is expected? I'd expect project_1 to work properly within project_2 (with regular dependencies that works fine). And it's not project_2 which depends on additional source, but project_1. Also, project_2 (and all other projects which are using project_1 as a dependency) must not know anything about project_1 dependencies 'cause that would be a mess, wouldn't it?

[dimbleby]

project 2 needs to be know where all of its dependencies can be found, including indirect dependencies. The way to do this is to configure it with sources.

[velykanov]

Ok, then why does project_2 simply not take all the instructions (aka sources) written in project_1? project_1 has them listed in a correct way, right?

Again, I don't think it's good idea to duplicate sources in downstream projects (imagine project_1 being common and project_2 through project_n - those which relies on common, that could be a number of duplicated configurations). What if I change storage url? I must then modify n number of files instead of just a single one - this is madness.

[dimbleby]

What you suggest has lots of problems to solve. Off the top of my head:

• First, it's just a long way from how the code is currently structured. You might wish that it was different, but it isn't.
• What if project 1 and project 2 give the same name to two different sources?
• What if project 1 says that it wants foo from source A and project 2 says that it wants foo from source B?

Frankly, none of this is going to be worth the effort: you might find it annoying to update a handful of files when you change a source (I bet that doesn't happen often), but it's many many times cheaper than making the change that you suggest for this niche use case.

I would be happy to be wrong! As usual in open source, you can prove it with an MR...

[velykanov]

1. I don't think I can speculate on that
2. This is obviously user's issue and should be resolved by the user himself
3. I'd say suggestion to have a fancy way to rename packages i.e. package = {version = "0.0.1", source = "artifactory", as = "other_name"} is a bad idea but still might work.

Anyways, I don't see how the same issues are easily avoided if you want to specify two different sources with the same name within the same project or want to use two different packages from different sources which share the same name within the same project.

However, I see your point. I'll try to create a PR which might resolve the issue

[eNcacz]

I think the source defined in project 1 should be used in project 2 too, but only for dependencies from the project 1. If the project 2 needs the same source, then it should define it once again (with the same or different name).

Now when we need to copy - paste the source definition from dependencies, it can lead to unsolvable situation. Lets imagine I have library 1, library 2 and project 1. Project 1 depends on library 1 and library 2. Both libraries are independent projects with different authors. It can happen that both libraries needs some dependencies from external source. Because they are independent projects, then they can choose the same source name for different sources. Now I cannot use both libraries in project 1, because I need to copy-paste external source definitions (even if the project 1 does not need them directly) and they names collide.