The activate script in Windows is not correct for venvs created in git-bash

[5968cd83-4a21-4d97-b6b1-5198ae21be48]

BPO	38583
Nosy	@pfmoore, @tjguk, @zware, @zooba, @mikofski

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-10-24.14:32:03.287>
labels = ['type-bug', '3.9', 'OS-windows']
title = 'The activate script in Windows is not correct for venvs created in git-bash'
updated_at = <Date 2020-04-15.05:32:32.233>
user = 'https://bugs.python.org/Mo'

bugs.python.org fields:

activity = <Date 2020-04-15.05:32:32.233>
actor = 'bwanamarko'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Windows']
creation = <Date 2019-10-24.14:32:03.287>
creator = 'Mo'
dependencies = []
files = []
hgrepos = []
issue_num = 38583
keywords = []
message_count = 5.0
messages = ['355333', '355356', '355382', '356146', '366488']
nosy_count = 6.0
nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'steve.dower', 'bwanamarko', 'Mo']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue38583'
versions = ['Python 3.9']

[5968cd83-4a21-4d97-b6b1-5198ae21be48]

When creating a virtual environment on Windows from git-bash (using python -m venv), VIRTUAL_ENV in the activate script is set using a windows style path (C:\some\path) instead of the bash style (/c/some/path).

This means the system python and pip get used, despite the user thinking they are working in a venv after sourcing activate.

As activate is a bash script, the paths in it should always be in the bash style, regardless of platform.

This is described in a stack overflow issue here: https://stackoverflow.com/questions/57758841/windows-virtualenv-created-via-gitbash-using-system-python

I have confirmed the behaviour in 3.7.3, 3.7.4, 3.7.5 and 3.8.0.

[5968cd83-4a21-4d97-b6b1-5198ae21be48]

The issue comes as a result of abspath on line 59 of venv/init.py: env_dir = os.path.abspath(env_dir)

This returns a Windows-style path, and os.path.abspath returning in this way is *probably* correct, as the OS is Windows, despite trying to forget that by using bash.

It is still my view that the activate script is a bash script, and therefore should only contain paths in that style, but the simple solution to this issue is to change the double quotes around the definition of $VIRTUAL_ENV in the activate script to single quotes. It works. The output of "which python" is a bit odd, but this is clearly a quirk beyond Python's control.

[zooba]

I agree this would be better, though it may not be that easy to do.

If you're running Windows Python from WSL, then your "bash style" path should be "/mnt/c/some/path", not "/c/some/path". So the answer isn't as simple as .replace('\\', '/').

I did some quick testing with posixpath and pathlib and neither has an obvious conversion from a Windows path to a POSIX path, though pathlib.PurePosixPath(pathlib.PureWindowsPath(p)) comes closest.

Perhaps the best approach here is to improve activate to determine its path when run, like we did for activate.ps1? Then we don't need to substitute the path in at creation time.

[5968cd83-4a21-4d97-b6b1-5198ae21be48]

I had also tested with pathlib and posixpath and come to the same conclusion.

As suggested by you, I looked into activate determining path when run. I believe this should do the trick (My bashfoo isn't strong, this is mostly from https://stackoverflow.com/a/179231):

pushd . > /dev/null
SCRIPT_PATH="${BASH_SOURCE[0]}"
if ([ -h "${SCRIPT_PATH}" ]); then
  while([ -h "${SCRIPT_PATH}" ]); do cd `dirname "$SCRIPT_PATH"`; 
  SCRIPT_PATH=`readlink "${SCRIPT_PATH}"`; done
fi
cd `dirname ${SCRIPT_PATH}` > /dev/null
cd .. > /dev/null
SCRIPT_PATH=`pwd`;
popd  > /dev/null

VIRTUAL_ENV="$SCRIPT_PATH"

[340ae65e-9726-4646-a1b5-7bce30dece82]

Would you consider just handling activate for windows directly in the lib/venv/init.py method "install_scripts(self, context, path)" https://github.com/python/cpython/blob/4f98f465f14e7258c5b18a62c5aa114dbe1174d8/Lib/venv/init.py#L382

    if not srcfile.endswith(('.exe', '.pdb')):

        # handle activate for Windows (ignore WSL)
        if srcfile == "activate":
            # from docs: on unix drive is always empty
            d, p = os.path.splitdrive(context.env_dir)
            d = d.replace(':', '')
            p = p.replace('\\', '/')
            if d:
                p = '/' + d + p
            data = data.decode('utf-8')
            data = data.replace('__VENV_DIR__', p)
            data = data.encode('utf-8')

    try:
        data = data.decode('utf-8')
        data = self.replace_variables(data, context)
        data = data.encode('utf-8')
    except UnicodeError as e:

IMHO I don't think the use of windows python in WSL is a realistic use-case, my preference would be to just make this fail. In fact I tried to use it, and I could not make it work.

1. /mnt/c/path/to/python -m venv venv Error: [WinError 5] Access is denied: 'C:\WINDOWS\system32\venv'
2. /mnt/c/path/to/python -m venv -m venv /mnt/c/some/path/to/venv fails silently, appears to do nothing, venv is not created
3. /mnt/c/path/to/python -m venv -m venv 'C:/some/path/to/venv' makes directories at C:\some\path\to\venv, but can't be activated in WSL, that I can figure out source /mnt/c/some/path/to/venv/Scripts/activate : command not found -bash: /mnt/c/some/path/to/venv/Scripts/activate: line 4: syntax error near unexpected token $'{\r'' 'bash: /mnt/c/some/path/to/venv/Scripts/activate: line 4: deactivate () {

I guess I don't really understand why it would be useful to use the windows python in WSL, and if that's the only thing holding a quick fix for this, I guess, I would prefer to just handle windows python in windows in git-bash, and ignore WSL. Would you be open to that?

If so, I'm happy to submit a PR

thanks!

[avoidik]

probably this is the only option

VIRTUAL_ENV='C:\Users\xyz\Projects\abc\.venvwin'
if ([ "$OSTYPE" = "cygwin" ] || [ "$OSTYPE" = "msys" ]) && $(command -v cygpath &> /dev/null) ; then
    VIRTUAL_ENV=$(cygpath -u "$VIRTUAL_ENV")
fi
export VIRTUAL_ENV

[daytonb]

I was looking into a different issue with activate script from Windows' Python in a cygwin terminal and came across this Github issue. I have 3 pieces of feedback on this issue.

1. I tested the Windows Python 3.11 venv activate script from a git-bash terminal. Even though the output of which python is very strange (/c/Users/daytonb/Desktop/junk/\Users\daytonb\Desktop\junk\venv/Scripts/python), the pip and python executables you get are the ones in the virtual environment. We can, therefore, probably close this issue with no code changes.

2. If we do want to address the contents of the environment variable and strange output of which python, then we could use the solution proposed by @avoidik above. It is precisely what's in the activate script written by virtualenv. I've confirmed that on my Win10 system, that works for activating a virtual environment in a git-bash terminal. If we merge that into Python's venv activate code, we could close this issue.

3. Finally, I concur with @mikofski's comment above, that we don't need to worry about being able to use the Windows' Python virtual environments from within a WSL terminal. I don't think WSL has any bearing on closing this issue.

[zware]

I'm pretty sure point 2 was addressed very recently in ebc81034278 (GH-103325, gh-103088), can you confirm?

[daytonb]

Indeed! That code change addresses my point 2. I think we can close this issue.