pytorch
Torch text does not work with python 3.9 despite pypi saying it works for 3.6+
🐛 Bug
Do the following on a new env:
conda create -n metalearning python=3.9
conda activate metalearning
conda install -y pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch -c nvidia
conda install -y torchtext==0.8.0 -c pytorch
Expected behavior Expected is that one can import torchtext e.g.
import torchtext
error:
(metalearning) miranda9~/automl-meta-learning $ conda install -y torchtext==0.8.0 -c pytorch
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: |
Found conflicts! Looking for incompatible packages. failed
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:
- torchtext==0.8.0 -> python[version='>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
Your python: python=3.9
If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.
The following specifications were found to be incompatible with your system:
- feature:/linux-64::__glibc==2.17=0
- feature:|@/linux-64::__glibc==2.17=0
Your installed version is: 2.17
Environment
(metalearning) miranda9~/ultimate-utils $ python collect_env.py
Collecting environment information...
PyTorch version: 1.7.1
Is debug build: False
CUDA used to build PyTorch: 11.0
ROCM used to build PyTorch: N/A
OS: CentOS Linux 7 (Core) (x86_64)
GCC version: (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
Clang version: 3.4.2 (tags/RELEASE_34/dot2-final)
CMake version: version 2.8.12.2
Libc version: glibc-2.17
Python version: 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-3.10.0-1127.10.1.el7.x86_64-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 11.1.105
GPU models and configuration:
GPU 0: TITAN X (Pascal)
GPU 1: TITAN X (Pascal)
GPU 2: TITAN X (Pascal)
GPU 3: TITAN X (Pascal)
Nvidia driver version: 450.36.06
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Versions of relevant libraries:
[pip3] numpy==1.20.3
[pip3] torch==1.7.1
[pip3] torchaudio==0.7.0a0+a853dff
[pip3] torchvision==0.8.0a0
[conda] _pytorch_select 0.1 cpu_0
[conda] blas 1.0 mkl
[conda] cudatoolkit 11.0.221 h6bb024c_0 nvidia
[conda] mkl 2021.3.0 h06a4308_520
[conda] mkl-service 2.4.0 py39h7f8727e_0
[conda] mkl_fft 1.3.0 py39h42c9631_2
[conda] mkl_random 1.2.2 py39h51133e4_0
[conda] numpy 1.20.3 py39hf144106_0
[conda] numpy-base 1.20.3 py39h74d4b33_0
[conda] pytorch 1.7.1 py3.9_cuda11.0.221_cudnn8.0.5_0 pytorch
[conda] torchaudio 0.7.2 py39 pytorch
[conda] torchvision 0.8.2 cpu_py39ha229d99_0
(metalearning) miranda9~/ultimate-utils $ python -c "import torchtext; print(\"torchtext version is \", torchtext.__version__)"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'torchtext'
related: https://github.com/pytorch/text/issues/1395, https://github.com/pytorch/text/issues/1397
PyTorch version | torchtext version | Supported Python version
-- | -- | --
nightly build | master | 3.6+
1.8 | 0.9 | 3.6+
1.7 | 0.8 | 3.6+
Thanks @brando90 for raising the issue. Yes, looks like the problem with Conda installation. could you please try with pip using version 0.8.1 (the supported version for 3.9, I will update the readme as well)
pip install torchtext==0.8.1
It seems to be same issue using Conda with audio/vision as well for pytorch version<1.9?
CC: @seemethere
CC: @mthrok, @NicolasHug
pip install torchtext==0.8.1
would having my setup.py have
torchtext>=0.8.1
be sufficient?
How is the rest going? @seemethere @mthrok @NicolasHug
will try your suggestion in a bit. My hpc is slow so the install will take me some time...
related to issue https://github.com/pytorch/text/issues/1397 will installing torchtest 0.9.2 give me issues? @parmeet
Thanks @brando90 for raising the issue. Yes, looks like the problem with Conda installation. could you please try with pip using version 0.8.1 (the supported version for 3.9, I will update the readme as well)
pip install torchtext==0.8.1It seems to be same issue using Conda with audio/vision as well for pytorch version<1.9?
CC: @seemethere
CC: @mthrok, @NicolasHug
but I also need pytorch 1.9.x. Would that work? @parmeet
Thanks @brando90 for raising the issue. Yes, looks like the problem with Conda installation. could you please try with pip using version 0.8.1 (the supported version for 3.9, I will update the readme as well)
pip install torchtext==0.8.1It seems to be same issue using Conda with audio/vision as well for pytorch version<1.9? CC: @seemethere CC: @mthrok, @NicolasHug
but I also need pytorch 1.9.x. Would that work? @parmeet
for pytorch 1.9.x you would need to install torchtext >=0.10.0. This one also supports python 3.9. If you try installing using Conda without explicitly specifying the torchtext version i.e conda install torchtext -c pytorch, I think you get the latest both for torchtext (0.10.1) and pytorch (1.9.1)
installing pytorch 1.9 with cuda: https://stackoverflow.com/questions/69230502/how-does-one-install-pytorch-1-9-in-an-hpc-that-seems-to-refuse-to-cooperate/69349188#69349188