Installation from source fails

[ojeda-e]

When running pip install requirements.txt I get:

pip install -r requirements.txt
Collecting torch>=1.8.0
  Downloading torch-1.12.0-cp39-cp39-manylinux1_x86_64.whl (776.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 776.3/776.3 MB 1.8 MB/s eta 0:00:00
Collecting torch-scatter>=2.0.8
  Using cached torch_scatter-2.0.9.tar.gz (21 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-0i9b31jh/torch-scatter_41398618c2884ce58012da70cf290897/setup.py", line 8, in <module>
          import torch
      ModuleNotFoundError: No module named 'torch'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

OS specs

OS: Linux pop-os x86_64 GNU/Linux Python version: 3.8, 3.9, 3.10 (I tried all of them)

Possible solution

Since installation of RDKIT is recommended via conda, I was wondering if is possible to modify requirements.txt to make it a requirements.yaml, with a section for conda and pip, then it is possible to create a conda environment together with the pip dependencies needed. Then, instead of pip install requirements.txt, one could do:

conda env create -f requirements.yaml

Maybe it's possible to have two files, one that works for Python 3.10, and another that works for Python = [3.7, 3.8, 3.9] (based on the version of torch).

Edit: typos.

[KiddoZhu]

Hi! I am afraid that we won't sacrifice pip for conda, as the statistics turns out that more people install TorchDrug from pip than conda 😅. I also don't recommend mixing pip in conda, since it can sometimes break the environment.

Regarding your question, it's because torch_scatter is available only as source code on pip. One must specify the link (e.g. https://pytorch-geometric.com/whl/torch-1.9.0+cu102.html) to install the pre-compiled versions of torch_scatter. Otherwise, it will compile torch_scatter locally on your machine, based on the installed PyTorch. Have you tried to first install PyTorch, and then install (compile) torch_scatter through pip? I guess that might work.

[ojeda-e]

Hi, thanks for your answer @KiddoZhu!

I also don't recommend mixing pip in conda, since it can sometimes break the environment.

Yes, I agree mixing pip and conda may generate some issues. I still would suggest to consider having a conda_env.yaml for conda and separately the requirements.txt for pip, in particular if that pytorch-scatter is available for conda. I understand where you coming from but it wouldn't hurt having requirements for each case. In the end, it's up to the user what to use. Not the end of the world either. :)

Regarding your question, it's because torch_scatter is available only as source code on pip. One must specify the link (e.g. https://pytorch-geometric.com/whl/torch-1.9.0+cu102.html) to install the pre-compiled versions of torch_scatter.

I think this information although present in the documentation is not entirely clear. The instructions to install from source are limited to this section, where the instruction to install torch scatter from source according to the version of cuda are not entirely clear. Would it be possible to add this information? Maybe it's just me, but it wasn't clear before reading your reply.