Allow specifying custom stubs for Gazelle

[brandonchinn178]

🚀 feature request

Description

If a library does not provide type stubs, and if there isn't already a type stub library, you might need to write your own stubs (e.g. kafka-python). Doing so is easy enough:

py_library(
    name = "kafka",
    imports = ["."],
    pyi_srcs = [
        "kafka/__init__.pyi",
        "kafka/consumer/fetcher.pyi",
        "kafka/errors.pyi",
    ],
    visibility = ["//visibility:public"],
)

Gazelle currently has a feature to automatically add stubs for a dependency (e.g. automatically add types-request whenever request is a dependency), but there's no way to register this target as being the type stubs for the @pypi//kafka library. Furthermore, adding it manually will remove it without # keep

Describe the solution you'd like

First, in gazelle_python.yaml, add a stub_mapping section that explicitly maps package to stubs

stub_mapping:
  requests: types_requests

Then instead of guessing which type packages to include, refer to this explicit mapping.

After this refactor, update modules_mapping to register additional stubs:

modules_mapping(
    name = "modules_map",
    include_stub_packages = True,
    wheels = all_whl_requirements,
    extra_stubs = {
        "kafka-python": "//my_stubs:kafka",
    },
)

This would get added to the new stub_mapping section in gazelle_python.yaml and everything should Just Work.

Describe alternatives you've considered

[shayanhoshyari]

Types are not used at runtime though, so what is the end goal here?

Is it for mypy? If so, at least rules_mypy support types without needing to add them to deps. https://github.com/bazel-contrib/rules_mypy#bzlmod-setup (They add the types always when checking)

[brandonchinn178]

One thing the rules_python gazelle plugin does is auto add types-requests if it sees requests (to either deps or pyi_deps, if gazelle:python_generate_pyi_deps is set). I simply want to extend that existing functionality. The end goal is the same as whatever the end goal was for adding that feature to gazelle.

But for my specific scenario, I'm writing a separate implementation that relies on Gazelle adding the types automatically. Most of the complicated logic in rules_mypy is auto-adding deps, which isn't good as it actually adds all the types-... libraries as dependencies to all Python targets in the graph. So we just have Gazelle auto-add the stub libraries, and then our mypy implementation just uses the dependency graph as-is without modifying it