genai-stack icon indicating copy to clipboard operation
genai-stack copied to clipboard
verified publisher icon docker

Failed to verify certificate: x509: certificate signed by unknown authority

Open wishatch opened this issue 2 years ago • 4 comments

I am using VSCode WSL2, Ubuntu 22.04 and Docker Engine v24.0.6 The .env file contains:

LLM=mistral #or any llama2:7b Ollama model tag, gpt-4, gpt-3.5, or claudev2 EMBEDDING_MODEL=sentence_transformer #or openai, ollama, or aws OLLAMA_BASE_URL=http://llm:11434

Executing this command: docker compose --profile linux up --build Giving me these lines:

Attaching to genai-stack-api-1, genai-stack-bot-1, genai-stack-database-1, genai-stack-front-end-1, genai-stack-llm-1, genai-stack-loader-1, genai-stack-pdf_bot-1, genai-stack-pull-model-1 genai-stack-pull-model-1 | pulling ollama model mistral using http://llm:11434 genai-stack-llm-1 | [GIN] 2023/11/02 - 10:47:24 | 200 | 57.058µs | 172.18.0.2 | HEAD "/" genai-stack-pull-model-1 | pulling manifest genai-stack-llm-1 | 2023/11/02 10:47:26 images.go:1164: couldn't get manifest: Get "https://registry.ollama.ai/v2/library/mistral/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority genai-stack-llm-1 | [GIN] 2023/11/02 - 10:47:26 | 200 | 1.672986504s | 172.18.0.2 | POST "/api/pull" genai-stack-pull-model-1 | Error: pull model manifest: Get "https://registry.ollama.ai/v2/library/mistral/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority service "pull-model" didn't complete successfully: exit 1

Please advice on how to fix the error. Thx much.

wishatch avatar Nov 02 '23 12:11 wishatch

Assuming that you're running on corporate Intranet which typically has HTTP proxy setup, then what you're running into is likely the SSL inspection certificate. There's a way to define HTTP proxy for OLLAMA (see https://github.com/jmorganca/ollama/issues/1063) but this is how I solved it:

  1. You need to download the top-level certificate authority certificate. If you're running Chrome browser when you're on the corporate Intranet, then navigate to an external website, click on the lock icon left of the URL, click on "Connection is secure", click on "Certificate is valid", click on "View Certificate" for the top-level certificate, click on "Details" tab, "Copy to File..." and click "Base-64 encoded X.509 (.CER)"
  2. Add downloaded certificate to the root directory of this repository after you cloned it
  3. Add the filename to .dockerignore with !filename (where filename is replaced) to ensure that Docker build pulls the file in
  4. You basically have to tamper with the Dockerfile and ensure that the certificate (where filename is replaced) is added to the following places via 
    ADD filename /etc/ssl/certs/filename
RUN cat /etc/ssl/certs/filename >> /etc/ssl/certs/ca-certificates.crt
RUN cat /etc/ssl/certs/filename >> /etc/local/lib/python-3.11/site-packages/certifi/cacert.pem

wchan87 avatar Nov 17 '23 16:11 wchan87

Assuming that you're running on corporate Intranet which typically has HTTP proxy setup, then what you're running into is likely the SSL inspection certificate. There's a way to define HTTP proxy for OLLAMA (see jmorganca/ollama#1063) but this is how I solved it:

  1. You need to download the top-level certificate authority certificate. If you're running Chrome browser when you're on the corporate Intranet, then navigate to an external website, click on the lock icon left of the URL, click on "Connection is secure", click on "Certificate is valid", click on "View Certificate" for the top-level certificate, click on "Details" tab, "Copy to File..." and click "Base-64 encoded X.509 (.CER)"
  2. Add downloaded certificate to the root directory of this repository after you cloned it
  3. Add the filename to .dockerignore with !filename (where filename is replaced) to ensure that Docker build pulls the file in
  4. You basically have to tamper with the Dockerfile and ensure that the certificate (where filename is replaced) is added to the following places via 
    ADD filename /etc/ssl/certs/filename
RUN cat /etc/ssl/certs/filename >> /etc/ssl/certs/ca-certificates.crt
RUN cat /etc/ssl/certs/filename >> /etc/local/lib/python-3.11/site-packages/certifi/cacert.pem

@wchan87 Thanks much for your advice. I need a few clarification as I am new to this kind of configuration.

  1. You recommended navigate to external website. Is it any URL, or specific url to this genai-stack? Which URL is it?
  2. I saw a few .Dockerfile in the genai-stack root folder. Which specific dockerfile do I need to edit? Thx

wishatch avatar Nov 23 '23 00:11 wishatch

  1. Any of website external to your corporate Intranet which I assume is the root cause of your issue. The external website would be protected by an internet proxy so it will have a SSL inspection certificate to replace the original SSL certificate.
  2. For safety, you make the changes to all of the .Dockerfile because they all seem to connect to some external website in some way.

wchan87 avatar Nov 26 '23 14:11 wchan87

pulling manifest Error: pull model manifest: Get "https://registry.ollama.ai/v2/library/llama3/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority

@wchan87 Same error after applying fixes you have mentioned above

michaelgetachew-abebe avatar May 01 '24 23:05 michaelgetachew-abebe