arnold icon indicating copy to clipboard operation
arnold copied to clipboard

Improve the clean playbook to respect ongoing deployments

Open sampaccoud opened this issue 6 years ago • 4 comments

Bug Report

Expected behavior/code The clean playbook should only delete objects that are not connected and thus useless.

Actual Behavior If the clean playbook is run at the same time as a deployment, it happens to delete objects that are not yet connected but useful...

Steps to Reproduce

  1. start a deployment
  2. once the deployment has started creating services, pods, etc., start the clean playbook

Environment

  • Arnold version: 4.3.1

Possible Solution Check that no deployment is running before actually deleting objects.

sampaccoud avatar Dec 31 '19 09:12 sampaccoud

:thinking: looks like we need some sort of lock file for ongoing deployments.

jmaupetit avatar Jan 03 '20 10:01 jmaupetit

We can maybe use a redis instance per project for that? Used as a core app like acme or redirect?

jmaupetit avatar Jan 03 '20 10:01 jmaupetit

Maybe we can use kubernetes to store the deployment state of our applications, instead of deploying a new app only for this purpose ?

K8s uses etcd internally as a k/v store, but I think we can't use etcd directly for security reasons. However, if we store this information in a kubernetes object, it will be persisted and we will be able to query these data easily from our playbooks.

The question is... in which kind of k8s object can we store this information ? I see 2 options :

  1. A ConfigMap per application.
  2. In the metadata of the route (or service) of an each application.

(If you think of other options, I'm interested !)

I don't like the second solution because we can have multiple routes (or services) per application and the information will be duplicated.

What do you think about this idea ?

madmatah avatar Jan 16 '20 13:01 madmatah

Using a dedicated configMap is a good idea, but we need to refactor ConfgMap management first! See #253

jmaupetit avatar Jan 16 '20 15:01 jmaupetit