PYTHONHOME gets incorrectly set by pip extension on windows

[mkruskal-google]

🐞 bug report

Affected Rule

This affects the pip module extension

Is this a regression?

Not that I'm aware of, but I assume it's worked on windows at some point

Description

When using the pip extension on windows, I get crashes like:

Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00000c40 (most recent call first):
<no Python frame>

I traced this back to the PYTHONHOME environment variable, which is being set from the stdout of a repository_ctx.execute call. Instead of just containing the result of the command, it also appears to contain the prompt and the command itself, corrupting this load-bearing environment variable. An example of what I see:

Python path configuration:
  PYTHONHOME = '\x0d\x0aC:\tmp\3hjz7nbt\external\rules_python~~pip~protobuf_pip_deps_311_setuptools>C:/hostedtoolcache/windows/Python/3.9.13/x64/python3.exe -B -I -c "import sys; print(f\'***sys.prefix***\', end=\'\')" \x0d\x0aC:\hostedtoolcache\windows\Python\3.9.13\x64'

Commenting out the line in rules_python that sets PYTHONHOME does work around this problem and I'm able to successfully build. But it's not clear to me what other problems that might cause, and it's probably not the right solution

🔬 Minimal Reproduction

Any bazel repository using the pip extension should reproduce this on windows. For example, in https://github.com/protocolbuffers/protobuf, you can run: bazel test //python/google/protobuf/internal/numpy:numpy_test

Note: you'll have to edit python/requirements.txt to not have any < symbols due to a separate bug I'll be filing another issue about.

🔥 Exception or Error

See description above. The error is from a corrupted python environment and not particularly helpful on its own

🌍 Your Environment

Operating System: Windows Output of bazel version: Reproduces with Bazel 7 and 8 Rules_python version: 1.6.0

[rickeylev]

Thanks for reporting!

What a strange bug. Gives me the impression that an earlier command is failing, and the error message from stdout is being used?

does commenting it out cause other problems?

It depends. The reason the repo rule sets pythonhome and runs with -I is because of another bug report where the repo rule invocation would pick up python settings from the local system; I can't remember the details, I think it got the wrong pip version or something. In any case, something broke.

[mkruskal-google]

I don't think the command is failing. If you look at PYTHONHOME (reformatted a bit):

C:\tmp\3hjz7nbt\external\rules_python~~pip~protobuf_pip_deps_311_setuptools>C:/hostedtoolcache/windows/Python/3.9.13/x64/python3.exe -B -I -c "import sys; print(f\'***sys.prefix***\', end=\'\')" 
C:\hostedtoolcache\windows\Python\3.9.13\x64

It's starting with a path> prompt, emitting the command python -B -I -c ..., and then reporting the correct result