mag37
dockcheck is not restarting dependant containers
When dockcheck updates and restarts a container, it does not restart the container which are dependant on it.
Example:
Now updating (2/3): mattermost-db
16.8: Pulling from library/postgres
8a628cdd7ccc: Already exists
061d38a4b4b6: Pull complete
0ee947678059: Pull complete
b83b3351f389: Pull complete
77835859ece0: Pull complete
002bfa82406b: Pull complete
632d3e125710: Pull complete
35942c92517e: Pull complete
e5baf548ac99: Pull complete
f486b50b9884: Pull complete
62963168ef8d: Pull complete
2139f5af38ec: Pull complete
acee9f076f01: Pull complete
915ee56c56b2: Pull complete
Digest: sha256:cef2d22004db69e3d601442ca4cac142adda0987ad7ca4f28c4e998bef690951
Status: Downloaded newer image for postgres:16.8
docker.io/library/postgres:16.8
[+] Running 1/1
✔ Container mattermost-db Started
mattermost-db was restarted, but mattermost-app (which depends on mattermost-db) was not restarted.
This is contrarry to docker compose pull and up command. Docker compose would pull the the db image, then restart both the db and any container depended on the db container.
Example:
[robin@lab scrutiny]$ sudo docker compose -f docker-compose-scrutiny.yaml pull
[+] Pulling 13/13
✔ web Pulled 0.3s
✔ influxdb Pulled 2.9s
✔ 8a628cdd7ccc Already exists 0.0s
✔ c7569dfbf1b2 Pull complete 0.6s
✔ 5edfd550ddc4 Pull complete 0.7s
✔ 639c0d915e27 Pull complete 0.7s
✔ 0213fd349110 Pull complete 0.8s
✔ d215588bda83 Pull complete 2.1s
✔ ae03cf3fc689 Pull complete 2.3s
✔ eb49061f9099 Pull complete 2.3s
✔ ca7df5880982 Pull complete 2.3s
✔ 3e3ce553ea73 Pull complete 2.4s
✔ collector Pulled 0.3s
[robin@lab scrutiny]$ sudo docker compose -f docker-compose-scrutiny.yaml up -d
[+] Running 3/3
✔ Container scrutiny-web Healthy 15.5s
✔ Container scrutiny-db Healthy 9.5s
✔ Container scrutiny-collector Started
This shows only the db container being updated. But all three container were restarted to complete the up command due to container dependency.
You're correct and this is due to the command is targeting the specific container within the compose:
${DockerBin} ${CompleteConfs} ${ContEnvs} up -d ${ContName}
There was some reasoning for this that I can't recall with certainty at the moment, without reading and doing some tests.
But are you certain that your 2nd example, with the regular docker compose up -d actually restarts the companion containers and not just list them as part of (and already running) the compose?
I'll do some further checks tomorrow, but this is the reason there's a f option. Not a pretty solution though.
Yes, it restarts them. Notice is is listed as "Started" as opposed to "Running" (Running would be for containers in the compose already running that are not touched).
Sorry I haven't replied yet - I had some infrastructural issues with my test environments.
I tried to dig in the history and now I wonder if it's just an assumption (and maybe a mistake) that the .. up -d ${ContName} is used.
It got implemented here #9 and then there was some further discussion and work done here #49 .
I need to do some further testing on this with different setups, but if you wanna try you could remove the trailing ${ContName} at line 520.
# from:
${DockerBin} ${CompleteConfs} ${ContEnvs} up -d ${ContName}
# to:
${DockerBin} ${CompleteConfs} ${ContEnvs} up -d
Justed tested this and it work:
Updating container(s):
immich-valkey
Now updating (1/1): immich-valkey
8.1.0: Pulling from valkey/valkey
8a628cdd7ccc: Already exists
2b52dc5ee25d: Pull complete
eb23939bd90b: Pull complete
3ec7e75217b6: Pull complete
959885885a2c: Pull complete
4f4fb700ef54: Pull complete
61225aead99d: Pull complete
Digest: sha256:8dbe14c0d070c80ae66640e0a8ab04a24c509b3c35795528d2939819126a5a00
Status: Downloaded newer image for valkey/valkey:8.1.0
docker.io/valkey/valkey:8.1.0
[+] Running 4/4
✔ Container immich-machinelearning Running 0.0s
✔ Container immich-postgres Running 0.0s
✔ Container immich-server Started 1.1s
✔ Container immich-valkey Started 0.5s
Now there is a follow up problem. The command to restart is issued after each update. This will cause multiple container restarts when there are multiple container updates in the same composer file.
For example, if immch-valkey and immch-postgres are both updated, it will cause immich-server to restart twice.
The all images all need to update first, then the restart commands should be issued.
Justed tested this and it work:
Happy to hear, I'll continue to do some more testing before I do any changes to main.
Now there is a follow up problem. The command to restart is issued after each update. This will cause multiple container restarts when there are multiple container updates in the same composer file.
For example, if immch-valkey and immch-postgres are both updated, it will cause immich-server to restart twice.
The all images all need to update first, then the restart commands should be issued.
I'm not entirely sure it will always restart all containers in a stack on every subsequent container being updated, but indeed it might be multiple restarts in a row.
And yes, it would be nice if all images got updated first and then the whole stack is restarted when done. Though that would need a completely different structure to keep track of what's been updated and what stack they're a part of - and then finally restarting ALL stacks that have received any updates during the run.
I will consider this as a future rework - but it's not in the scope of this issue. It's added to my list of things to look into for now.
I'm not entirely sure it will always restart all containers in a stack on every subsequent container being updated, but indeed it might be multiple restarts in a row.
It won't restart all containers. It will restart the dependant containers. So if some container depends on multiple container being updated, it will get restarted more than once.
In the above example, immich-server is dependant on both immich-valkey and immich-postgres. So if the postgres and valkey containers both receive an update and the restart command is issued after each container update, it will restart immich-server twice. Three times if immich-server container also receives an update.
Wouldn't it be simple enough to capture the each restart command into an array after each container is updated, instead of directly excuting them? Then when containers are finished upating, run the commands stored in the array. You could dump duplicated commands as they are entered into the array or just run tham all anyways (following "up" commands on the same compose file won't do anything once all the containers are updated).
In the above example, immich-server is dependant on both immich-valkey and immich-postgres. So if the postgres and valkey containers both receive an update and the restart command is issued after each container update, it will restart immich-server twice. Three times if immich-server container also receives an update.
Understood.
Wouldn't it be simple enough to capture the each restart command into an array after each container is updated, instead of directly excuting them? Then when containers are finished upating, run the commands stored in the array. You could dump duplicated commands as they are entered into the array or just run tham all anyways (following "up" commands on the same compose file won't do anything once all the containers are updated).
Yes sure, the process is probably quite simple - but I'll still need to do it and test it with different cases, different environments, wait for a few days for different types of images and stacks to get updates etc.
This project is (sadly) not my day job - So I'll add it to the list and get to it when I can.
I need to do some further testing on this with different setups, but if you wanna try you could remove the trailing
${ContName}at line 520.# from: ${DockerBin} ${CompleteConfs} ${ContEnvs} up -d ${ContName} # to: ${DockerBin} ${CompleteConfs} ${ContEnvs} up -d
This is now merged to main.
Now there is a follow up problem. The command to restart is issued after each update. This will cause multiple container restarts when there are multiple container updates in the same composer file.
For example, if immch-valkey and immch-postgres are both updated, it will cause immich-server to restart twice.
The all images all need to update first, then the restart commands should be issued.
And this will be worked on as soon as I got time.
I started working on this and started storing values in arrays and experimenting with key-value arrays.. then realized I could solve it much easier by just splitting the pulling and the recreating to different loops, while still iterating on the same list of containers.
Anyway - there's a new branch I'm testing this in and it seems to be working fine. It will still run the docker compose up -d multiple times if there's containers sharing the same compose - but as you said yourself
following "up" commands on the same compose file won't do anything once all the containers are updated
I'll experiment further with de-duplicating, but this works for now. Here's the branch: update_rework If you want to try it - I'd gladly hear any feedback and improvements.
Results look good to me so far.
Choose what containers to update.
Enter number(s) separated by comma, [a] for all - [q] to quit: 27, 28, 29
Updating container(s):
scrutiny-collector
scrutiny-db
scrutiny-web
Now updating (1/3): scrutiny-collector
master-collector: Pulling from analogj/scrutiny
254e724d7786: Already exists
8dd50134bec7: Pull complete
592342a6f4e0: Pull complete
2512c559a896: Pull complete
a7ec251d8633: Pull complete
d56183005ab6: Pull complete
2da0637483f0: Pull complete
Digest: sha256:c68c3dff1d897fce90eb28743b13d5dfa72c6a8bafb3671117fed971683ae535
Status: Downloaded newer image for ghcr.io/analogj/scrutiny:master-collector
ghcr.io/analogj/scrutiny:master-collector
Now updating (2/3): scrutiny-db
2.7.11: Pulling from library/influxdb
254e724d7786: Already exists
c4e52b528d16: Pull complete
22d4b4b2a00e: Pull complete
6694aa4f244a: Pull complete
e39495c4afb9: Pull complete
0a738af6c8bd: Pull complete
de4ae9be4197: Pull complete
9c0dd4a375f6: Pull complete
02a10144993c: Pull complete
40fd1e96734c: Pull complete
Digest: sha256:d92a10e9e75aff18eca38ff3a8f0b4a800706a5dd44d1b0ece264af04458525b
Status: Downloaded newer image for influxdb:2.7.11
docker.io/library/influxdb:2.7.11
Now updating (3/3): scrutiny-web
master-web: Pulling from analogj/scrutiny
254e724d7786: Already exists
e29bc50fdbec: Pull complete
b8ab3845dde2: Pull complete
8975ec2d1b6d: Pull complete
cfcfa7bdb252: Pull complete
9139c0cf3fa1: Pull complete
Digest: sha256:2a2f37226d9063915288024ba9171bd404cd1d1560299478bc35696f7575ffa8
Status: Downloaded newer image for ghcr.io/analogj/scrutiny:master-web
ghcr.io/analogj/scrutiny:master-web
Done pulling updates. Recreating updated containers.
Now recreating (1/3): scrutiny-collector
[+] Running 3/3
✔ Container scrutiny-web Healthy 18.7s
✔ Container scrutiny-db Healthy 12.7s
✔ Container scrutiny-collector Started 18.3s
Now recreating (2/3): scrutiny-db
[+] Running 3/3
✔ Container scrutiny-db Healthy 0.5s
✔ Container scrutiny-web Healthy 1.0s
✔ Container scrutiny-collector Running 0.0s
Now recreating (3/3): scrutiny-web
[+] Running 3/3
✔ Container scrutiny-db Healthy 0.5s
✔ Container scrutiny-web Healthy 1.0s
✔ Container scrutiny-collector Running 0.0s
Choose what containers to update.
Enter number(s) separated by comma, [a] for all - [q] to quit: 25, 26
Updating container(s):
roundcube-db
roundcubemail
Now updating (1/2): roundcube-db
11.7.2: Pulling from library/mariadb
0622fac788ed: Pull complete
90dbf4535882: Pull complete
95ed3e3fde04: Pull complete
0b381eed6c88: Pull complete
d7547f36e497: Pull complete
a5e33262a388: Pull complete
3984aae8ebb4: Pull complete
9883ef72c7c9: Pull complete
Digest: sha256:11706a6fd276c2eada52d0d69b1a2aa1f1484cbe78137678e02cca8f7a0ae502
Status: Downloaded newer image for mariadb:11.7.2
docker.io/library/mariadb:11.7.2
Now updating (2/2): roundcubemail
1.6.10-apache: Pulling from roundcube/roundcubemail
254e724d7786: Already exists
e91d651b0707: Pull complete
1bfafd9b2882: Pull complete
114de7b0b7ea: Pull complete
ec5c759dee4b: Pull complete
c38ac4593515: Pull complete
bcc8e5e06974: Pull complete
48691dc82b81: Pull complete
cd7e71d96fd9: Pull complete
248c8870a49f: Pull complete
949e9ca28dfc: Pull complete
1f48ed56a105: Pull complete
7993dae7ff23: Pull complete
4f4fb700ef54: Pull complete
f55d4ccfb64b: Pull complete
8fc755d92ca3: Pull complete
7b1906e86577: Pull complete
2cda62692c34: Pull complete
014e95d95540: Pull complete
8b8e3c6c25fd: Pull complete
54bacdd377aa: Pull complete
Digest: sha256:37e10dfc0166d6952443193f3a68c3b6796d2b5f7e02340fdea1d0d656b72f12
Status: Downloaded newer image for roundcube/roundcubemail:1.6.10-apache
docker.io/roundcube/roundcubemail:1.6.10-apache
Done pulling updates. Recreating updated containers.
Now recreating (1/2): roundcube-db
[+] Running 2/2
✔ Container roundcubemail Started 6.5s
✔ Container roundcube-db Started 5.0s
Now recreating (2/2): roundcubemail
[+] Running 2/2
✔ Container roundcube-db Running 0.0s
✔ Container roundcubemail Running 0.0s
Thank you so much for testing.
I'll merge this to main later this week most likely, might add some other work first.
Impressive collection of containers btw!
I just had had a thought and don't know if this matters because it would not affect me. But how does this change affect containers not deployed in by a compose file? Personally, everything I deploy is compose based.
Would this change affect updates of non compose deployments?
Thank you for thinking of it even though it wouldn't affect you.
No the project is mainly based on using compose, even states it as a dependency in the readme.
Also this in the help/readme
-r Allow updating images for docker run, wont update the container.
:warning:
-r flagdisclaimer and warningWont auto-update the containers, only their images. (compose is recommended)
docker rundont support using new images just by restarting a container.
Containers need to be manually stopped, removed and created again to run on the new image.
And in the new code (similar to old) it just checks for the -r option and if set it'll pull the image and then continue to the next container.
I'll need to postpone merging this to main until after the weekend - won't be around to hotfix any related bugs.
If anyone else would like to take the opportunity to try out the branch and give any feedback that'd be very welcome. (Looking at the usual suspects @yoyoma2 @Thaurin @firmlyundecided @chriscrutch @Wigsinator and any other contributors)