google-deepmind
404 Client Error for http+docker://localhost/v1.44/images/create?tag=latest&fromImage=alphafold: Not Found ("pull access denied for alphafold, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")
Hello,
I have found a couple of closed threads for this issue but none of the fixes seem to work for me. I am running ubuntu 22.04.
When executing:
python3 docker/run_docker.py --fasta_paths=/fasta_data/T1049.fasta --max_template_date=2020-05-14 --model_preset=monomer --data_dir=/data_disk/genetic_database --output_dir=/home/efthymios/protein_fold_predictions`
I get:
I0222 19:35:41.936533 140393684931008 run_docker.py:116] Mounting /fasta_data -> /mnt/fasta_path_0
I0222 19:35:41.936644 140393684931008 run_docker.py:116] Mounting /data_disk/genetic_database/uniref90 -> /mnt/uniref90_database_path
I0222 19:35:41.936727 140393684931008 run_docker.py:116] Mounting /data_disk/genetic_database/mgnify -> /mnt/mgnify_database_path
I0222 19:35:41.936779 140393684931008 run_docker.py:116] Mounting /data_disk/genetic_database -> /mnt/data_dir
I0222 19:35:41.936861 140393684931008 run_docker.py:116] Mounting /data_disk/genetic_database/pdb_mmcif -> /mnt/template_mmcif_dir
I0222 19:35:41.936925 140393684931008 run_docker.py:116] Mounting /data_disk/genetic_database/pdb_mmcif -> /mnt/obsolete_pdbs_path
I0222 19:35:41.936986 140393684931008 run_docker.py:116] Mounting /data_disk/genetic_database/pdb70 -> /mnt/pdb70_database_path
I0222 19:35:41.937049 140393684931008 run_docker.py:116] Mounting /data_disk/genetic_database/uniref30 -> /mnt/uniref30_database_path
I0222 19:35:41.937108 140393684931008 run_docker.py:116] Mounting /data_disk/genetic_database/bfd -> /mnt/bfd_database_path
Traceback (most recent call last):
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/api/client.py", line 268, in _raise_for_status
response.raise_for_status()
File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.44/containers/create
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/models/containers.py", line 811, in run
container = self.create(image=image, command=command,
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/models/containers.py", line 870, in create
resp = self.client.api.create_container(**create_kwargs)
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/api/container.py", line 430, in create_container
return self.create_container_from_config(config, name)
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/api/container.py", line 441, in create_container_from_config
return self._result(res, True)
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/api/client.py", line 274, in _result
self._raise_for_status(response)
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/api/client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.44/containers/create: Not Found ("No such image: alphafold:latest")
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/api/client.py", line 268, in _raise_for_status
response.raise_for_status()
File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.44/images/create?tag=latest&fromImage=alphafold
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/efthymios/alphafold/docker/run_docker.py", line 267, in <module>
app.run(main)
File "/home/efthymios/.local/lib/python3.10/site-packages/absl/app.py", line 312, in run
_run_main(main, args)
File "/home/efthymios/.local/lib/python3.10/site-packages/absl/app.py", line 258, in _run_main
sys.exit(main(argv))
File "/home/efthymios/alphafold/docker/run_docker.py", line 237, in main
container = client.containers.run(
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/models/containers.py", line 814, in run
self.client.images.pull(image, platform=platform)
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/models/images.py", line 446, in pull
pull_log = self.client.api.pull(
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/api/image.py", line 430, in pull
self._raise_for_status(response)
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/api/client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "/home/efthymios/.local/lib/python3.10/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.44/images/create?tag=latest&fromImage=alphafold: Not Found ("pull access denied for alphafold, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")
Also, when executing:
docker images
I get:
REPOSITORY TAG IMAGE ID CREATED SIZE
alphafold latest e6d1bcc0db25 29 minutes ago 10.4GB
nvidia/cuda 11.0.3-base-ubuntu20.04 97dfa1ef5ee7 3 months ago 122MB
Furhtermore, when trying to pull the image from the docker desktop GUI I get the same error message.
Thx for any help.
Hello, I am also running into the same issue. Similarly, I am using ubuntu 22.04, and the same alphafold and nvidia/cuda tags. Any clarification or workarounds would be appreciated!
I have clean installed ubuntu 22.04 multiple times now trying to figure out the issue. There also seems to be an issue with the nvidia container (see: this issue), which I also have trouble with. I do not know if these two are connected somehow. I also installed Debian 12 and I am currently failing at installing the CUDA drivers altogether. The CUDA tutorials seem to be relatively incongruent linking from one page to the next and mixing up the order of the command lines. If I find anything new, I will let you know.
Do you see your Docker image also in Python? I somehow had an issue that it was not visible when testing importing docker and getting the images list (I reinstalled docker using apt-get and before I used snap); then it was visible and worked.
python
Python 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information.
>>> import docker
>>> client = docker.from_env()
>>> print(client.images.list())
[<Image: 'alphafold:latest'>, <Image: 'hello-world:latest'>]
This Dockerfile using ARG CUDA=11.8.0, jax==0.4.14, jaxlib==0.4.14+cuda11.cudnn86, and adding line
\ && conda install -c nvidia cuda-nvcc works for me (on a single Nvidia RTX 4080, Ubuntu-22.04; Python 3.10 Conda environment with pip3 install -r requirements.txt and pip install requests==2.28.0):
# Copyright 2021 DeepMind Technologies Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
ARG CUDA=11.8.0
FROM nvidia/cuda:${CUDA}-cudnn8-runtime-ubuntu18.04
# FROM directive resets ARGS, so we specify again (the value is retained if
# previously set).
ARG CUDA
# Use bash to support string substitution.
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
build-essential \
cmake \
cuda-command-line-tools-$(cut -f1,2 -d- <<< ${CUDA//./-}) \
git \
hmmer \
kalign \
tzdata \
wget \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get autoremove -y \
&& apt-get clean
# Compile HHsuite from source.
RUN git clone --branch v3.3.0 https://github.com/soedinglab/hh-suite.git /tmp/hh-suite \
&& mkdir /tmp/hh-suite/build \
&& pushd /tmp/hh-suite/build \
&& cmake -DCMAKE_INSTALL_PREFIX=/opt/hhsuite .. \
&& make -j 4 && make install \
&& ln -s /opt/hhsuite/bin/* /usr/bin \
&& popd \
&& rm -rf /tmp/hh-suite
# Install Miniconda package manager.
ARG MINICONDA_FILE='Miniconda3-py310_24.1.2-0-Linux-x86_64.sh'
RUN wget -q -P /tmp \
# https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
"https://repo.anaconda.com/miniconda/${MINICONDA_FILE}" \
&& bash /tmp/${MINICONDA_FILE} -b -p /opt/conda \
&& rm /tmp/${MINICONDA_FILE}
# Install conda packages.
ENV PATH="/opt/conda/bin:$PATH"
ENV LD_LIBRARY_PATH="/opt/conda/lib:$LD_LIBRARY_PATH"
RUN conda install -qy conda==23.5.2 \
&& conda install -y -c conda-forge \
openmm=7.7.0 \
cudatoolkit==${CUDA_VERSION} \
pdbfixer \
pip \
python=3.10 \
&& conda install -c nvidia cuda-nvcc \
&& conda clean --all --force-pkgs-dirs --yes
COPY . /app/alphafold
RUN wget -q -P /app/alphafold/alphafold/common/ \
https://git.scicore.unibas.ch/schwede/openstructure/-/raw/7102c63615b64735c4941278d92b554ec94415f8/modules/mol/alg/src/stereo_chemical_props.txt
# Install pip packages.
RUN pip3 install --upgrade pip --no-cache-dir \
&& pip3 install -r /app/alphafold/requirements.txt --no-cache-dir \
&& pip3 install --upgrade --no-cache-dir \
jax==0.4.14 \
jaxlib==0.4.14+cuda11.cudnn86 \
-f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
# Add SETUID bit to the ldconfig binary so that non-root users can run it.
RUN chmod u+s /sbin/ldconfig.real
# We need to run `ldconfig` first to ensure GPUs are visible, due to some quirk
# with Debian. See https://github.com/NVIDIA/nvidia-docker/issues/1399 for
# details.
# ENTRYPOINT does not support easily running multiple commands, so instead we
# write a shell script to wrap them up.
WORKDIR /app/alphafold
RUN echo $'#!/bin/bash\n\
ldconfig\n\
python /app/alphafold/run_alphafold.py "$@"' > /app/run_alphafold.sh \
&& chmod +x /app/run_alphafold.sh
ENTRYPOINT ["/app/run_alphafold.sh"]
I will try that out and report back. I already discovered that I cannot run
client = docker.from_env()
I get:
import docker
client = docker.from_env()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'docker' has no attribute 'from_env'
I didn't investigate furhter yet. I will let you know once I have news.
Thank you.
I rebuild the alphafold project using your proposed file, but it still seems to be not working. Also, I still cannot run
client = docker.from_env().
I will try to clean install everything and build it using your script again.
Hello. I am running into this problem described in this thread. @vatsimanos, did you end up resolving this? I am running python v 3.10.14 on Debian GCP.
Sadly not, we ended up outsourcing the tasks we wanted to do with alphafold to another lab and they use google colab to run it (which is way easier to set up). Since so many people have problems with installing alpha fold we decided it was not worth pursuing further.
@vatsimanos, thank you for your response. It's so interesting... We have it working on a local Ubuntu, but not on a GCP Debian. If we solve this, will let you all know. I hesitate using GoogleColab due to security...
Update @vatsimanos, we now have this working on the GCP. If you want to chat about it, would be happy to...