Updates to MQ gitops promotion pipelines

[osowski]

Based on the call today, @hollisc and I followed up with discussion on connecting the dots from the conversation to the explicit artifacts that need to be updated to remove the GitOps pipelines pushing back into the Application Source repositories.

The changes in this issue should be made off the demo-mq-v2 branch.

There are multiple waves of changes that can happen to get to an optimal setup, but the below target state diagram is the first wave that we need to remove the anti-pattern that we have implemented currently:

NOTES:

• These changes will currently make the mq-infra-staging and mq-infra-prod pipelines obsolete. We can revisit the separation of concerns with respect to testing when & where in a subsequent wave.
• These changes remove the use of staging or prod in the upstream application source repository.

Artifacts to update:

• https://github.com/cloud-native-toolkit/multi-tenancy-gitops/blob/demo-mq-v2/2-services/instances/namespace-dev/pipelines/ibm-test-pipeline-for-dev.yaml
• https://github.com/cloud-native-toolkit/multi-tenancy-gitops/blob/demo-mq-v2/2-services/instances/namespace-staging/pipelines/ibm-test-pipeline-for-staging.yaml

[hollisc]

Hema, you should be able to re-use the existing gitops task used in the mq-infra-dev pipeline as that has the logic to add the helm chart files if it does not exist in the gitops repo and if it does exist, it will only update the chart version specified in requirements.yaml and let's target to run the pipelines in the ci namespace.

[phemankita]

• Added ibm-test-pipeline in ci namespace

At the moment, we are using the same pipeline at both dev and staging level. Only different will be copying the artifacts from namespace-dev to namespace-staging or namespace-staging to namespace-prod. So, for now left a common pipeline and params can be modified while triggering the pipelines.

We can definitely extend the test pipeline and at that point, we can have different pipelines at dev and staging levels.

• Added ibm-gitops-with-pr task adopting most of it from gitops task. This will copy the contents from env folder to another. And finally leave a PR.

• Since, the test pipeline need argo task, added ibm-argo-tests in.

[osowski]

I needed to create the ArgoCD secret that is referenced for the ArgoCD step to function, as it's currently not defined anywhere that I can see.

oc create secret generic argocd-access-creds \
    --from-literal=ARGOCD_USERNAME=admin \
    --from-literal=ARGOCD_PASSWORD=$(oc get secret/openshift-gitops-cluster -n openshift-gitops -o json | jq -r '.data."admin.password"' | base64 -D) \
    --from-literal=ARGOCD_URL=$(oc get route -n openshift-gitops openshift-gitops-server -o jsonpath="{.spec.host}")

[osowski]

The current implementation via #60 doesn't appear to function as expected, in it's simplest case. We can dive back into it next week, but similar to @hollisc comments on the PR, I don't think we need img-info task and the ArgoCD task seems to expect different parameters than what the pipeline expects as input.

[phemankita]

Current ibm-mq-promote-dev-stage pipeline includes

• Validating the app instance in dev environment using rollout status
• Updates the gitops repo namespace-staging resources consuming them from namespace-dev

Since, the app instance might not be always up to date in the dev environment, it might be good to use argocd validation by grabbing the info from the namespace-dev and validating it against the info from the argocd instance which allows us to make sure we are hitting the right instance.