How to use a local py_library that a pip requirement depends on

[ingomueller-net]

🐞 bug report / support request

Description

I have the following situation that I am currently not able to solve with Bazel and the Python rules.

• I am working on a project that directly depends on jax. I need to build jax myself and do patches to it from my WORKSPACE file. jax exposes several py_library targets that I use (and some targets for native extensions under the hood).
• My project also directly depends on chex. I don't care too much about which version is used; some version from pip should be fine, so I am currently using pip_parse and requirements("chex").
• chex depends on jax.

The problem is that (1) if I include jax in my requirements.txt, that version is used by my code instead of the one I build in my WORKSPACE and (2) if I don't include jax, then chex doesn't find its requirement.

So how do I express what I want? Is there a way to give priority to my local jax package in the PYTHONPATH? Or can I somehow modify the dependencies of the package imported by pip_parse? Or maybe some other way?

Rules_python version:

fe33a4582c37499f3caeb49a07a78fc7948a8949

[ingomueller-net]

If nobody feels responsible or competent to answer this question here, where else is the right place to ask this question?

[aignas]

This is currently not possibre in rules_python and it likely falls under a feature request category.

What I understand is "As a user I wouldlike to include bazel py_library targets in the requirements and potentially replace packages with the same name".

I amnot sure if this is within the scope that rules_python aims to provide features for but at the same time I don't know what to suggest. For the mean time I'll tag the issue appropriately.

Maybe if we have a more advanced bzlmod dependency specification mechanism, this could be easier to solve.

[ingomueller-net]

Thanks for confirming that this isn't possible currently.

What I understand is "As a user I would like to include bazel py_library targets in the requirements and potentially replace packages with the same name".

I think that that's a good summary.

Let me document my current (quite horrible) work-around in case somebody else runs into this case. I am adding the following line before importing the two packages:

sys.path = [p for p in sys.path if "/pip_deps_jax" not in p]

Maybe the same line could live instead in a __init__.py such that it cannot be forgotten and the ugliness is concentrated to just one place.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

[github-actions[bot]]

This issue was automatically closed because it went 30 days without a reply since it was labeled "Can Close?"