Specialized DockerExceptions

[mdantonio]

Hello again!

while I'm working on the integration of python on whales into my applications, I found the error management mostly thought for a human use.

Currently all errors raise a DockerException that contains the exact error output produced by the CLI. That's great when using the package by a human and provides enough information to debug the problem. But when the package is used to automate processes this approach is not very helpful and to be able to properly react, a parser of the exception is needed.

Just a practical example:

docker.service.inspect("my_service")

This command can fail for a number of reasons, for example:

the swarm is not initialized:

python_on_whales.utils.DockerException: The docker command executed was `/usr/local/bin/docker service inspect myservice`.
It returned with code 1
The content of stdout is '[]
'
The content of stderr is 'Status: Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again., Code: 1
'

or the service does not exist:

python_on_whales.utils.DockerException: The docker command executed was `/usr/local/bin/docker service inspect myservice`.
It returned with code 1
The content of stdout is '[]
'
The content of stderr is 'Status: Error: no such service: myservice, Code: 1
'

Probably there are much more possible reasons, but two are enough for the example

To implement specific reactions to the different problems a parser of the exception is needed:

try:
    docker.service.inspect("my_service")
except python_on_whales.utils.DockerException as e:
    if "This node is not a swarm manager" is str(e):
        initialized_my_node()
    elif "no such service" in str(e):
        log.error("This service does not exist")
    else:
        log.critical("Unexpected error")

I think that raise specific exceptions (that would extend the parent DockerException, of course) would be very very useful for the user experience:

try:
    docker.service.inspect("my_service")
except python_on_whales.utils.SwarmNotInitialized:
        initialized_my_node()
except python_on_whales.utis.NoSuchService:
        log.error("This service does not exist")
except Exception:
        log.critical("Unexpected error")

That means, in a practical way, move the parsing directly into the package to centralized that operation and make it transparent to the final user.

I understand that is a very boring issue but I'm pretty sure that would greatly improve the adoption... and of course I'm 100% available to ease this task as much as I can

[gabrieldemarmiesse]

This is indeed an issue that I noticed and i'm not very happy about the only solution possible in the short term (parsing the error message). That might be something we have to do in the short term as it's necessary to build more complicated things.

I see two other solution, more mid to long term:

1. Make the CLI output different exit codes depending on the error. We would have to ask the Docker team for that. So a change to the CLI.
2. Call the Docker engine API directly like docker-py does. It may not be practical for more complicated functions that are not available in docker-py, like docker compose or buildx functions.

[mdantonio]

Just few comments, in reversed order:

Call the Docker engine API directly like docker-py does. It may not be practical for more complicated functions that are not available in docker-py, like docker compose or buildx functions.

Based on my understanding this approach would be out of the scope of this package that is explicitly built around the docker-cli, or I'm wrong? Of course communicating with API should be the more consistent way to interact with the daemon but it seems to be a major drift the would complicate a lot the implementation, but i agree that it IS a solution

Make the CLI output different exit codes depending on the error. We would have to ask the Docker team for that. So a change to the CLI

A very interesting approach, but probably the slower: convincing the docker team to consider that, wait for the implementation, then work on this integration... maybe that can be considered a very long term possibility, With a valid support from docker and a little patience I think that this would be the more stable solution :)

i'm not very happy about the only solution possible in the short term (parsing the error message). That might be something we have to do in the short term as it's necessary to build more complicated things.

No, of course it is not a very exciting solution, but seems to be the only quick solution, if you think that this issue can make sense.

From my side: if you think that you are interested in this issue (in a short term perspective) I will be grateful and very available to help you, if I can. Otherwise I will implement (at least temporarily) some kind of parser into my code to be able to distinguish the different cases

Thank you

[gabrieldemarmiesse]

My plan is

1. raise different exceptions for different errors quickly, so by parsing the error message. Currently, with python-on-whales, it's impossible to answer the question "does that image/container/network/volume exists?", by calling the engine api, we can do that. So we need to fix it quicky. I would like a function exists(). So that we can do docker.container.exists("my_container_name") or my_container.exists(). This function would return a bool. We should also have an exception for different objects that don't exists.

2. Ask the docker team for different error codes from the CLI. It's not only useful for python-on-whales, it may be also useful for people doing bash scripting. From a bash script, knowing if an image exists or not is only possible by parsing stderr. So not great. The CLI has some progress to do there.

I'm going to do the first step, but if you want to help and contribute, I'd be happy to review any pull request I can get :)

[gabrieldemarmiesse]

I started parsing stderr to raise specific exceptions, you can follow the work here: https://github.com/gabrieldemarmiesse/python-on-whales/pull/210

[mdantonio]

As soon as you think that this preliminary work is stable I will start some tests, so that I will be able to report or directly fix with PRs eventual problems

[gabrieldemarmiesse]

The version is unreleased yet, but you can try it with pip install git+https://github.com/gabrieldemarmiesse/python-on-whales.git.

We now have a custom exception python_on_whales.exceptions.NoSuchImage. You can trigger it with docker.push, docker.image.inspect and docker.save among other. I'll continue to add exceptions.

This change should be backward compatible. Please notify me if this change breaks any existing code.

[mdantonio]

Verified:

• [x] backward compatibily
• [x] new NoSuchImage exception
• [x] new image exists function

:+1:

[gabrieldemarmiesse]

See https://github.com/docker/cli/issues/1683 for the RFC about the CLI exit codes.

It's quite possible that the docker team doesn't have enough dev to push/implement this RFC. We'll likely need to do the PRs ourselves, time to learn Go 😆

[mdantonio]

Let's... go! :smiley:

In the meanwhile I integrated the NoSuchService exception :+1:

From my side the next important exception is probably that:

It returned with code 1
The content of stdout is ''
The content of stderr is 'Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.
'

raised when the swarm is not initialized

In case I will try to work on a PR next week

[gabrieldemarmiesse]

#218 might interest you.

[mdantonio]

Wonderful, thank you!

[mdantonio]

Unfortunately still no activity from the issue about standardising exit-codes in docker-cli, but it was somehow expected.

In the meanwhile can I ask you some additional exceptions?

I'm working with local docker registries (with self signed certificates) and remote engines and I found some exceptions in case of untrusted certificates and invalid remote hosts

Invalid remote hosts are quite easy to be reproduced:

>>> from python_on_whales import DockerClient

>>> DockerClient(host="invalid").container.list()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/python_on_whales/components/container/cli_wrapper.py", line 955, in list
    for x in run(full_cmd).splitlines()
  File "/usr/local/lib/python3.9/dist-packages/python_on_whales/utils.py", line 145, in run
    raise DockerException(
python_on_whales.exceptions.DockerException: The docker command executed was `/usr/local/bin/docker --host invalid container list -q --no-trunc`.
It returned with code 1
The content of stdout is ''
The content of stderr is 'error during connect: Get "http://invalid:2375/v1.24/containers/json": dial tcp: lookup invalid: Temporary failure in name resolution
'

>>> DockerClient(host="192.168.1.19").container.list()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/python_on_whales/components/container/cli_wrapper.py", line 955, in list
    for x in run(full_cmd).splitlines()
  File "/usr/local/lib/python3.9/dist-packages/python_on_whales/utils.py", line 145, in run
    raise DockerException(
python_on_whales.exceptions.DockerException: The docker command executed was `/usr/local/bin/docker --host 192.168.1.19 container list -q --no-trunc`.
It returned with code 1
The content of stdout is ''
The content of stderr is 'error during connect: Get "http://192.168.1.19:2375/v1.24/containers/json": dial tcp 192.168.1.19:2375: connect: no route to host
'

>>> DockerClient(host="ssh://invalid@invalid").container.list()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/python_on_whales/components/container/cli_wrapper.py", line 955, in list
    for x in run(full_cmd).splitlines()
  File "/usr/local/lib/python3.9/dist-packages/python_on_whales/utils.py", line 145, in run
    raise DockerException(
python_on_whales.exceptions.DockerException: The docker command executed was `/usr/local/bin/docker --host ssh://invalid@invalid container list -q --no-trunc`.
It returned with code 1
The content of stdout is ''
The content of stderr is 'error during connect: Get "http://docker.example.com/v1.24/containers/json": command [ssh -l invalid -- invalid docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=ssh: Could not resolve hostname invalid: Temporary failure in name resolution

'

On the other side to raise errors for untrusted certificates it is need to spawn a docker registry with https enabled and a self signed certificate. This is a bit tricky so feel free to decline my request :)

First, create a self signed certificate (note: the docker registry requires a specific format for the registry or the TLS certificate will not be accepted... after many tries I found the following workflow to produce a valid self signed certificate (in particular to create a certificate containing an IP SAN)

1. create a config.ini file (in /tmp/certs/config.ini in this snippet):

[req]
default_bits = 4096
default_md = sha256
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
C = XX
ST = XX
L = XXX
O = NoCompany
OU = Orgainizational_Unit
CN = 192.168.1.7
[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
IP.1 = 192.168.1.7

Please note that 192.168.1.7 is the local IP address of my network adapter, adjust with your own IP

If you want to extract the IP from python, I use the following code:

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(("8.8.8.8", 80))
print(s.getsockname()[0])

After creating the config.ini you can create the self signed certificate with:

openssl req -newkey rsa:4096 -nodes -sha256 -x509 -days 365 -config /tmp/certs/config.ini -keyout /tmp/certs/mycert.key -out /tmp/certs/mycert.pem -subj '/CN=*/'

Now you can spawn the registry:

docker.container.run("registry:2.7.1", publish=[(5000,5000)], remove=True, detach=True, volumes=[("/tmp/certs", "/certs")], envs={"REGISTRY_HTTP_TLS_CERTIFICATE": "/certs/mycert.pem", "REGISTRY_HTTP_TLS_KEY": "/certs/mycert.key"})

And finally the exception:

>>> docker.image.pull("192.168.1.7:5000/random/image")

Using default tag: latest
Error response from daemon: Get "https://192.168.1.7:5000/v2/": x509: certificate signed by unknown authority
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/python_on_whales/components/image/cli_wrapper.py", line 387, in pull
    return self._pull_single_tag(x, quiet=quiet)
  File "/usr/local/lib/python3.9/dist-packages/python_on_whales/components/image/cli_wrapper.py", line 405, in _pull_single_tag
    run(full_cmd, capture_stdout=quiet, capture_stderr=quiet)
  File "/usr/local/lib/python3.9/dist-packages/python_on_whales/utils.py", line 145, in run
    raise DockerException(
python_on_whales.exceptions.DockerException: The docker command executed was `/usr/local/bin/docker image pull 192.168.1.7:5000/random/image`.
It returned with code 1
The content of stdout can be found above the stacktrace (it wasn't captured).
The content of stderr can be found above the stacktrace (it wasn't captured).

as usual, thank you very much for you support!