bazelbuild
Some transitive "extras" dependencies are not automatically included to the target
🐞 bug report
Affected Rule
py_{test,binary,library}
Is this a regression?
I'm not sure. I think things work differently between venvs_site_packages=yes and venvs_site_packages=no. I started seeing this issue when switching over to venvs_site_packages=yes.
Description
The "Using PyPI" section in rules_python-documentation says:
‘Extras’ dependencies Any “extras” specified in the requirements lock file will be automatically added as transitive dependencies of the package. In the example above, you’d just put
requirement("useful_dep")or@pypi//useful_dep.
After switching to using venvs_site_packages=yes, I see that the above is not the case for all packages. For example, consider the following simple requirements.in:
etils[eapp,epath]
and its compiled requirements.txt as follows:
Details
# This file was autogenerated by uv via the following command:
# uv pip compile --no-strip-extras --generate-hashes --emit-index-url ./requirements.in -o ./requirements.txt
--index-url https://pypi.org/simple
absl-py==2.3.1 \
--hash=sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9 \
--hash=sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d
# via etils
docstring-parser==0.17.0 \
--hash=sha256:583de4a309722b3315439bb31d64ba3eebada841f2e2cee23b99df001434c912 \
--hash=sha256:cf2569abd23dce8099b300f9b4fa8191e9582dda731fd533daf54c4551658708
# via simple-parsing
etils[eapp, epath]==1.13.0 \
--hash=sha256:a5b60c71f95bcd2d43d4e9fb3dc3879120c1f60472bb5ce19f7a860b1d44f607 \
--hash=sha256:d9cd4f40fbe77ad6613b7348a18132cc511237b6c076dbb89105c0b520a4c6bb
# via -r ./requirements.in
fsspec==2025.9.0 \
--hash=sha256:19fd429483d25d28b65ec68f9f4adc16c17ea2c7c7bf54ec61360d478fb19c19 \
--hash=sha256:530dc2a2af60a414a832059574df4a6e10cce927f6f4a78209390fe38955cfb7
# via etils
importlib-resources==6.5.2 \
--hash=sha256:185f87adef5bcc288449d98fb4fba07cea78bc036455dd44c5fc4a2fe78fed2c \
--hash=sha256:789cfdc3ed28c78b67a06acb8126751ced69a3d5f79c095a98298cd8a760ccec
# via etils
simple-parsing==0.1.7 \
--hash=sha256:225e6b35252d68f7894716101fe3bd7e6dd3d30ab7b1c3c023f77a42dbe1336f \
--hash=sha256:5276e6c90c157362dd0173d1eecebe58361a66b457129cc9bba13b78a4e85092
# via etils
typing-extensions==4.15.0 \
--hash=sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466 \
--hash=sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548
# via
# etils
# simple-parsing
zipp==3.23.0 \
--hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \
--hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166
# via etils
Based on the documentation referred to above, I'd expect the following py_test target to run without problems:
Details
load("@rules_python//python:defs.bzl", "py_test")
py_test(
name = "etils_test",
srcs = ["etils_test.py"],
deps = [
"@pypi//etils",
],
)
where etils_test.py is defined as:
from etils import eapp
from etils import epath
eapp.better_logging()
However, I get the following error:
Details
bazel run \
--verbose_failures \
//:etils_test
INFO: Analyzed target //:etils_test (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:etils_test up-to-date:
bazel-bin/etils_test
INFO: Elapsed time: 0.133s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: external/bazel_tools/tools/test/test-setup.sh ./etils_test
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //:etils_test
-----------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test_stage2_bootstrap.py", line 499, in <module>
main()
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test_stage2_bootstrap.py", line 493, in main
_run_py_path(main_filename, args=sys.argv[1:])
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test_stage2_bootstrap.py", line 287, in _run_py_path
runpy.run_path(main_filename, run_name="__main__")
File "<frozen runpy>", line 287, in run_path
File "<frozen runpy>", line 98, in _run_module_code
File "<frozen runpy>", line 88, in _run_code
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/etils_test.py", line 1, in <module>
from etils import eapp
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/eapp/__init__.py", line 17, in <module>
from etils.eapp.dataclass_flags import make_flags_parser
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/eapp/dataclass_flags.py", line 25, in <module>
with _internal.check_missing_deps():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/external/rules_python++python+python_3_12_x86_64-unknown-linux-gnu/lib/python3.12/contextlib.py", line 158, in __exit__
self.gen.throw(value)
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/epy/_internal.py", line 48, in check_missing_deps
reraise_utils.reraise(
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/epy/reraise_utils.py", line 112, in reraise
raise new_exception.with_traceback(e.__traceback__) from e.__cause__
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/epy/_internal.py", line 46, in check_missing_deps
yield
File "/home/user/.cache/bazel/_bazel_user/0edd9a2ca9426ffe30008363aa919a47/execroot/_main/bazel-out/k8-fastbuild/bin/etils_test.runfiles/_main/_etils_test.venv/lib/python3.12/site-packages/etils/eapp/dataclass_flags.py", line 27, in <module>
from absl import flags
ModuleNotFoundError: No module named 'absl'
Each etils sub-modules require deps to be installed separately (e.g. `from etils import ecolab` -> `pip install etils[ecolab]`)
To make this target work, I need to add several transitive dependency packages to the rule manually. The following target works:
load("@rules_python//python:defs.bzl", "py_test")
py_test(
name = "etils_test",
srcs = ["etils_test.py"],
deps = [
"@pypi//absl_py",
"@pypi//etils",
"@pypi//simple_parsing",
"@pypi//importlib_resources",
],
)
Note how absl_py is clearly a transitive dependency of etils, as can be seen from the compiled requirements.txt. I wonder if I just misunderstand what the documentation means about extras. It's unclear what the useful_dep in the documentation example is because I don't see any reference to useful_dep "in the example above" in the documentation.
🔬 Minimal Reproduction
See full reproduction here: https://github.com/bazel-contrib/rules_python/compare/main...hartikainen:rules_python:etils
🔥 Exception or Error
See Above
🌍 Your Environment
Operating System:
$lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04.1 LTS
Release: 24.04
Codename: noble
Output of bazel version:
$ bazel --version
bazel 8.2.1
Rules_python version:
43a5acf8cedfce07fd4f933c9165f75e0b4d88c9
Anything else relevant?
N/A
