Calling zipapp virtualenv with --seeder=pip fails with OSError: [Errno 13] Permission denied: 'RECORD'
Issue
When calling zipapp version of virtualenv with --seeder=pip to download the latest packaging tools from PyPi, it works fine on Ubuntu, but fails on macOS with a runtime error:
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: 'RECORD'
Consider using the `--user` option or check the permissions.
Traceback (most recent call last):
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/Users/lada.gagina/Developer/intellij/out/classes/production/intellij.python.helpers/virtualenv.pyz/__main__.py", line 168, in <module>
File "/Users/lada.gagina/Developer/intellij/out/classes/production/intellij.python.helpers/virtualenv.pyz/__main__.py", line 164, in run
File "/Users/lada.gagina/Developer/intellij/out/classes/production/intellij.python.helpers/virtualenv.pyz/virtualenv/__main__.py", line 16, in run
File "/Users/lada.gagina/Developer/intellij/out/classes/production/intellij.python.helpers/virtualenv.pyz/virtualenv/run/__init__.py", line 28, in cli_run
File "/Users/lada.gagina/Developer/intellij/out/classes/production/intellij.python.helpers/virtualenv.pyz/virtualenv/run/session.py", line 47, in run
File "/Users/lada.gagina/Developer/intellij/out/classes/production/intellij.python.helpers/virtualenv.pyz/virtualenv/run/session.py", line 60, in _seed
File "/Users/lada.gagina/Developer/intellij/out/classes/production/intellij.python.helpers/virtualenv.pyz/virtualenv/seed/embed/pip_invoke.py", line 23, in run
File "/Users/lada.gagina/Developer/intellij/out/classes/production/intellij.python.helpers/virtualenv.pyz/virtualenv/seed/embed/pip_invoke.py", line 31, in _execute
RuntimeError: failed seed with code 1
Environment
- OS: MacOS 10.15.7. It works fine on Ubuntu 20.4.
- I'm using zipapp version of virtualenv
Output of the virtual environment creation
481 setup logging to NOTSET [DEBUG report:43]
601 find interpreter for spec PythonSpec(path=/Library/Developer/CommandLineTools/usr/bin/python3) [INFO builtin:62]
601 proposed PythonInfo(spec=CPython3.7.3.final.0-64, exe=/Library/Developer/CommandLineTools/usr/bin/python3, platform=darwin, version='3.7.3 (default, Sep 5 2019, 17:14:41) \n[Clang 11.0.0 (clang-1100.0.33.8)]', encoding_fs_io=utf-8-UTF-8) [INFO builtin:69]
601 accepted PythonInfo(spec=CPython3.7.3.final.0-64, exe=/Library/Developer/CommandLineTools/usr/bin/python3, platform=darwin, version='3.7.3 (default, Sep 5 2019, 17:14:41) \n[Clang 11.0.0 (clang-1100.0.33.8)]', encoding_fs_io=utf-8-UTF-8) [DEBUG builtin:71]
612 filesystem is not case-sensitive [DEBUG info:28]
652 create virtual environment via CPython3macOsFramework(dest=/Users/lada.gagina/Downloads/venv-test11, clear=False, no_vcs_ignore=False, global=False) [INFO session:52]
652 create folder /Users/lada.gagina/Downloads/venv-test11/bin [DEBUG _sync:25]
652 create folder /Users/lada.gagina/Downloads/venv-test11/lib/python3.7/site-packages [DEBUG _sync:25]
652 write /Users/lada.gagina/Downloads/venv-test11/pyvenv.cfg [DEBUG pyenv_cfg:34]
652 home = /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7 [DEBUG pyenv_cfg:38]
652 implementation = CPython [DEBUG pyenv_cfg:38]
652 version_info = 3.7.3.final.0 [DEBUG pyenv_cfg:38]
652 virtualenv = 20.4.6 [DEBUG pyenv_cfg:38]
652 include-system-site-packages = false [DEBUG pyenv_cfg:38]
652 base-prefix = /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7 [DEBUG pyenv_cfg:38]
653 base-exec-prefix = /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7 [DEBUG pyenv_cfg:38]
653 base-executable = /Library/Developer/CommandLineTools/usr/bin/python3 [DEBUG pyenv_cfg:38]
653 symlink /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python to /Users/lada.gagina/Downloads/venv-test11/bin/python [DEBUG _sync:44]
653 symlink /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/Python3 to /Users/lada.gagina/Downloads/venv-test11/.Python [DEBUG _sync:44]
654 Attempting to acquire lock 4536973856 on /Users/lada.gagina/Library/Application Support/virtualenv/unzip/20.4.6/_virtualenv.py.lock [DEBUG filelock:270]
654 Lock 4536973856 acquired on /Users/lada.gagina/Library/Application Support/virtualenv/unzip/20.4.6/_virtualenv.py.lock [INFO filelock:274]
655 Attempting to release lock 4536973856 on /Users/lada.gagina/Library/Application Support/virtualenv/unzip/20.4.6/_virtualenv.py.lock [DEBUG filelock:315]
655 Lock 4536973856 released on /Users/lada.gagina/Library/Application Support/virtualenv/unzip/20.4.6/_virtualenv.py.lock [INFO filelock:318]
655 create virtualenv import hook file /Users/lada.gagina/Downloads/venv-test11/lib/python3.7/site-packages/_virtualenv.pth [DEBUG api:95]
655 create /Users/lada.gagina/Downloads/venv-test11/lib/python3.7/site-packages/_virtualenv.py [DEBUG api:98]
656 ============================== target debug ============================== [DEBUG session:54]
656 Attempting to acquire lock 4533826952 on /Users/lada.gagina/Library/Application Support/virtualenv/unzip/20.4.6/debug.py.lock [DEBUG filelock:270]
657 Lock 4533826952 acquired on /Users/lada.gagina/Library/Application Support/virtualenv/unzip/20.4.6/debug.py.lock [INFO filelock:274]
657 debug via /Users/lada.gagina/Downloads/venv-test11/bin/python '/Users/lada.gagina/Library/Application Support/virtualenv/unzip/20.4.6/debug.py' [DEBUG creator:224]
716 Attempting to release lock 4533826952 on /Users/lada.gagina/Library/Application Support/virtualenv/unzip/20.4.6/debug.py.lock [DEBUG filelock:315]
716 Lock 4533826952 released on /Users/lada.gagina/Library/Application Support/virtualenv/unzip/20.4.6/debug.py.lock [INFO filelock:318]
656 {
"sys": {
"executable": "/Library/Developer/CommandLineTools/usr/bin/python3",
"_base_executable": "/Library/Developer/CommandLineTools/usr/bin/python3",
"prefix": "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7",
"base_prefix": "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7",
"real_prefix": null,
"exec_prefix": "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7",
"base_exec_prefix": "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7",
"path": [
"/Users/lada.gagina/Downloads",
"/usr/local/lib/python2.7/site-packages",
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python37.zip",
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7",
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/lib-dynload",
"/Users/lada.gagina/Library/Python/3.7/lib/python/site-packages",
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages",
"/Library/Python/3.7/site-packages"
],
"meta_path": [
"<class '_frozen_importlib.BuiltinImporter'>",
"<class '_frozen_importlib.FrozenImporter'>",
"<class '_frozen_importlib_external.PathFinder'>"
],
"fs_encoding": "utf-8",
"io_encoding": "UTF-8"
},
"version": "3.7.3 (default, Sep 5 2019, 17:14:41) \n[Clang 11.0.0 (clang-1100.0.33.8)]",
"makefile_filename": "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/config-3.7m-darwin/Makefile",
"os": "<module 'os' from '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/os.py'>",
"site": "<module 'site' from '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site.py'>",
"datetime": "<module 'datetime' from '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/datetime.py'>",
"math": "<module 'math' from '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/lib-dynload/math.cpython-37m-darwin.so'>",
"json": "<module 'json' from '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/json/__init__.py'>"
} [DEBUG session:55]
716 add seed packages via PipInvoke(download=False, pip=bundle, setuptools=bundle, wheel=bundle) [INFO session:59]
717 Attempting to acquire lock 4533826056 on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/pip-21.1.1-py3-none-any.whl.lock [DEBUG filelock:270]
717 Lock 4533826056 acquired on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/pip-21.1.1-py3-none-any.whl.lock [INFO filelock:274]
717 Attempting to release lock 4533826056 on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/pip-21.1.1-py3-none-any.whl.lock [DEBUG filelock:315]
717 Lock 4533826056 released on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/pip-21.1.1-py3-none-any.whl.lock [INFO filelock:318]
718 got embed update of distribution pip from /Users/lada.gagina/Library/Application Support/virtualenv/wheel/3.7/embed/1/pip.json [DEBUG via_disk_folder:135]
727 got embed update of distribution pip from /Users/lada.gagina/Library/Application Support/virtualenv/wheel/3.7/embed/1/pip.json [DEBUG via_disk_folder:135]
728 Attempting to acquire lock 4534840680 on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/setuptools-56.0.0-py3-none-any.whl.lock [DEBUG filelock:270]
728 Lock 4534840680 acquired on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/setuptools-56.0.0-py3-none-any.whl.lock [INFO filelock:274]
728 Attempting to release lock 4534840680 on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/setuptools-56.0.0-py3-none-any.whl.lock [DEBUG filelock:315]
728 Lock 4534840680 released on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/setuptools-56.0.0-py3-none-any.whl.lock [INFO filelock:318]
729 got embed update of distribution setuptools from /Users/lada.gagina/Library/Application Support/virtualenv/wheel/3.7/embed/1/setuptools.json [DEBUG via_disk_folder:135]
729 got embed update of distribution setuptools from /Users/lada.gagina/Library/Application Support/virtualenv/wheel/3.7/embed/1/setuptools.json [DEBUG via_disk_folder:135]
730 Attempting to acquire lock 4534840512 on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/wheel-0.36.2-py2.py3-none-any.whl.lock [DEBUG filelock:270]
730 Lock 4534840512 acquired on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/wheel-0.36.2-py2.py3-none-any.whl.lock [INFO filelock:274]
730 Attempting to release lock 4534840512 on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/wheel-0.36.2-py2.py3-none-any.whl.lock [DEBUG filelock:315]
730 Lock 4534840512 released on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/wheel-0.36.2-py2.py3-none-any.whl.lock [INFO filelock:318]
731 got embed update of distribution wheel from /Users/lada.gagina/Library/Application Support/virtualenv/wheel/3.7/embed/1/wheel.json [DEBUG via_disk_folder:135]
732 got embed update of distribution wheel from /Users/lada.gagina/Library/Application Support/virtualenv/wheel/3.7/embed/1/wheel.json [DEBUG via_disk_folder:135]
732 Attempting to acquire lock 4536611728 on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/pip-21.1.1-py3-none-any.whl.lock [DEBUG filelock:270]
732 Lock 4536611728 acquired on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/pip-21.1.1-py3-none-any.whl.lock [INFO filelock:274]
732 Attempting to release lock 4536611728 on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/pip-21.1.1-py3-none-any.whl.lock [DEBUG filelock:315]
732 Lock 4536611728 released on /Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/pip-21.1.1-py3-none-any.whl.lock [INFO filelock:318]
733 got embed update of distribution pip from /Users/lada.gagina/Library/Application Support/virtualenv/wheel/3.7/embed/1/pip.json [DEBUG via_disk_folder:135]
733 pip seed by running: /Users/lada.gagina/Downloads/venv-test11/bin/python -m pip -q install --only-binary :all: --disable-pip-version-check --no-index pip==21.1.1 setuptools==56.0.0 wheel==0.36.2 --find-links '/Users/lada.gagina/Library/Application Support/virtualenv/wheel/house' env of {'LESS_TERMCAP_mb': '\x1b[01;31m', 'TERM_PROGRAM': 'Apple_Terminal', 'LESS_TERMCAP_md': '\x1b[01;38;5;74m', 'COLOR_RED': '\\[\\e[31m\\]', 'LESS_TERMCAP_me': '\x1b[0m', 'SHELL': '/bin/bash', 'TERM': 'xterm-256color', 'CLICOLOR': '1', 'TMPDIR': '/var/folders/r4/x5hzc2kn1wz4jn47t1g75xjr0000gn/T/', 'CONDA_SHLVL': '0', 'COLOR_GREEN': '\\[\\033[32m\\]', 'COLOR_BOLD': '\\[\\e[1m\\]', 'TERM_PROGRAM_VERSION': '433', 'COLOR_BLUE': '\\[\\033[36m\\]', 'OLDPWD': '/Users/lada.gagina/Documents/virtualenv-test', 'LESS_TERMCAP_ue': '\x1b[0m', 'TERM_SESSION_ID': '6CCA1AE8-9AA1-4F47-8704-AC3E81FC07C9', 'COLOR_YELLOW': '\\[\\033[33;1m\\]', 'USER': 'lada.gagina', 'CONDA_EXE': '/Users/lada.gagina/opt/anaconda3/bin/conda', 'SSH_AUTH_SOCK': '/private/tmp/com.apple.launchd.5R3XHTsVUC/Listeners', 'COLOR_DEFAULT': '\\[\\e[0m\\]', '_CE_CONDA': '', 'LESS_TERMCAP_us': '\x1b[04;38;5;146m', 'LSCOLORS': 'ExFxBxDxCxegedabagacad', 'PATH': '/Users/lada.gagina/opt/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public', 'LaunchInstanceID': '4E051D87-1FC0-477C-8721-204F1A6D0981', 'PWD': '/Users/lada.gagina/Downloads', 'EDITOR': 'subl -w', 'XPC_FLAGS': '0x0', '_CE_M': '', 'XPC_SERVICE_NAME': '0', 'SHLVL': '1', 'HOME': '/Users/lada.gagina', 'CONDA_PYTHON_EXE': '/Users/lada.gagina/opt/anaconda3/bin/python', 'PYTHONPATH': '/Users/lada.gagina/Library/Application Support/virtualenv/wheel/house/pip-21.1.1-py3-none-any.whl', 'LOGNAME': 'lada.gagina', 'LESS_TERMCAP_so': '\x1b[38;5;246m', 'LC_CTYPE': 'UTF-8', 'COLOR_WHITE': '\\[\\033[m\\]', 'SECURITYSESSIONID': '186a6', 'LESS_TERMCAP_se': '\x1b[0m', '_': '/usr/bin/python3', '__CF_USER_TEXT_ENCODING': '0x1F5:0x0:0x2', 'SDKROOT': '/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk', 'CPATH': '/usr/local/include', 'LIBRARY_PATH': '/usr/local/lib', '__PYVENV_LAUNCHER__': '/Library/Developer/CommandLineTools/usr/bin/python3', 'PLAT': 'macosx-10.14-x86_64', 'PIP_USE_WHEEL': '1', 'PIP_USER': '0', 'PIP_NO_INPUT': '1'} [DEBUG pip_invoke:27]
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: 'RECORD'
Consider using the `--user` option or check the permissions.
Traceback (most recent call last):
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "virtualenv.pyz/__main__.py", line 168, in <module>
File "virtualenv.pyz/__main__.py", line 164, in run
File "virtualenv.pyz/virtualenv/__main__.py", line 18, in run
File "virtualenv.pyz/virtualenv/run/__init__.py", line 32, in cli_run
File "virtualenv.pyz/virtualenv/run/session.py", line 47, in run
File "virtualenv.pyz/virtualenv/run/session.py", line 60, in _seed
File "virtualenv.pyz/virtualenv/seed/embed/pip_invoke.py", line 23, in run
File "virtualenv.pyz/virtualenv/seed/embed/pip_invoke.py", line 31, in _execute
RuntimeError: failed seed with code 1
I can reproduce. Fails when using the system Python, works when using a Python installed in my home directory. Command works run from the command line, fails when using the env provided by virtualenv.
ERROR: Could not install packages due to an OSError.
Consider using the `--user` option or check the permissions.
Traceback (most recent call last):
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/shutil.py", line 788, in move
os.rename(src, real_dst)
PermissionError: [Errno 13] Permission denied: '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/wheel-0.33.1.dist-info/' -> '/private/tmp/pip-uninstall-o5z_4l5b'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/private/tmp/v/virtualenv/seed/wheels/embed/pip-21.1.1-py3-none-any.whl/pip/_internal/commands/install.py", line 393, in run
installed = install_given_reqs(
File "/private/tmp/v/virtualenv/seed/wheels/embed/pip-21.1.1-py3-none-any.whl/pip/_internal/req/__init__.py", line 70, in install_given_reqs
uninstalled_pathset = requirement.uninstall(
File "/private/tmp/v/virtualenv/seed/wheels/embed/pip-21.1.1-py3-none-any.whl/pip/_internal/req/req_install.py", line 653, in uninstall
uninstalled_pathset.remove(auto_confirm, verbose)
File "/private/tmp/v/virtualenv/seed/wheels/embed/pip-21.1.1-py3-none-any.whl/pip/_internal/req/req_uninstall.py", line 386, in remove
moved.stash(path)
File "/private/tmp/v/virtualenv/seed/wheels/embed/pip-21.1.1-py3-none-any.whl/pip/_internal/req/req_uninstall.py", line 275, in stash
renames(path, new_path)
File "/private/tmp/v/virtualenv/seed/wheels/embed/pip-21.1.1-py3-none-any.whl/pip/_internal/utils/misc.py", line 328, in renames
shutil.move(old, new)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/shutil.py", line 800, in move
rmtree(src)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/shutil.py", line 715, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/shutil.py", line 672, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/shutil.py", line 670, in _rmtree_safe_fd
os.unlink(entry.name, dir_fd=topfd)
PermissionError: [Errno 13] Permission denied: 'RECORD'
Removed build tracker: '/private/tmp/pip-req-tracker-o7z3py7j'
I unzipped virtualenv.pyz and edited ABS_HERE = os.path.abspath("/tmp/virtualenv.pyz") to be able to run /usr/bin/python3 -m pdb __main__.py /tmp/vt34 --seeder=pip
cmd is '/private/tmp/vt34/bin/python -m pip -q install --only-binary :all: --disable-pip-version-check --no-index pip==21.1.1 setuptools==56.0.0 wheel==0.36.2 --find-links /private/tmp/v/virtualenv/seed/wheels/embed'
This issue still exists on this setup: stock macOS on ARM/M1
$ python3
Python 3.8.2 (default, Jun 8 2021, 11:59:35)
[Clang 12.0.5 (clang-1205.0.22.11)] on darwin
$ uname -a
Darwin computer2.local 20.4.0 Darwin Kernel Version 20.4.0: Thu Apr 22 21:46:41 PDT 2021; root:xnu-7195.101.2~1/RELEASE_ARM64_T8101 arm64
I am using this command to create clean virtualenvs that do not further fetch or depend anything else:
wget -O venv/virtualenv.pyz https://bootstrap.pypa.io/virtualenv/virtualenv.pyz
python3 venv/virtualenv.pyz --never-download --seeder pip \
--pip embed --setuptools embed --wheel embed \
--no-vcs-ignore --no-periodic-update ./venv
This has been working nicely on Linux, macOS x86 and Windows for a long while
and removing the seeder with python3 virtualenv.pyz --never-download --pip embed --setuptools embed --wheel embed --no-vcs-ignore --no-periodic-update . works.