Adding external applications/services to gitops deployment workflow via CLI

[mtarng]

As a: Bedrock CLI user with a gitops workflow deployed

I want: To be able to add an external to my cluster via the CLI and have the lifecycle pipeline add the service to the HLD as a new component.

In the case, an "external" service is an application with a prebuilt docker image and an existing helm chart. eg. Prometheus or MongoDB(https://github.com/helm/charts/tree/master/stable/mongodb).

Presently, the bedrock cli workflow assumes that we have a build push & update pipeline for tracked services whose sources are tracked in the same git repository. However, occasionally, I may want to deploy a service without building an image, but still want that service tracked by bedrock.yaml, and in the HLD.

Additional context and considerations:

• I would like for bedrock.yaml to track other applications that must also be deployed but do not need a build-update HLD pipeline, ie mongo, or postgres - these applications/services often distribute their container images through dockerhub or their own public container repositories
• The tracked external service may exist outside of rings - there should only ever be one deployment of it
• In the case of multiple application repositories pointing to a single HLD repository, how would conflicts be resolved/handled?
• How much configuration would we want to track in bedrock.yaml vs manual configuration editing in the HLD repository?
• As an "external service" would not have a service build pipeline that would allow it to be built as a normal service, how would service introspection be able to track the deployment process?

A simple bedrock.yaml could look like:

rings:
  develop:
    isDefault: true
services:
  ./greatService: 
    displayName: my-great-service
    helm:
      chart:
        branch: 'master'
        git: 'https://github.com/helm/charts'
        path: 'stable/wordpress'
    k8sBackend: 'myGreatService'
    k8sBackendPort: 80
    middlewares: []
    pathPrefix: 'greatService'
    pathPrefixMajorVersion: 'v1'
externalServices: 
  // We assume the name of the service is `mongo`
  mongo:
    helm:
      chart:
        branch: 'master'
        git: 'https://github.com/helm/charts'
        path: 'stable/mongo'
variableGroups:
  - greatServiceVarGroup

The new external service mongo would then appear in the HLD as an "external application" via spk hld reconcile:

├── my-great-service-repo
│   │  // ** This `externalServices` map would be added by `spk hld reconcile` as  a new feature.
│   ├── externalServices `map of externalServices` from bedrock.yaml
│   │   ├── mongo/
│   |   |   └── component.yaml -> points to helm component provided in `externalServices `map from bedrock.yaml
│   |   └── config
│   |       └── common.yaml -> where a user can override config for their external service
│   |
│   |
│   |  // Everything below is a normal HLD generated by `spk hld reconcile` over the services in the "services" map from `bedrock.yaml`
│   ├── component.yaml
│   ├── access.yaml
│   ├── config
│   │   └── common.yaml
│   └── my-great-service
│       ├── component.yaml
│       ├── config
│       │   └── common.yaml
│       └── develop
│           ├── component.yaml
│           ├── config
│           │   └── common.yaml
│           └── static
│               ├── ingress-route.yaml
│               └── middlewares.yaml

Does this require updates to documentation?:

Yes

[andrebriggs]

@mtarng I'm understanding your scenario as "What if you have additional applications in the HLD that are essentially 3rd party?" How do you track them in the service introspection tool?

Correct interpretation?

[mtarng]

@andrebriggs yes, that's correct. Though these external applications may be defined inside one or more application monorepositories. So this also covers how would we may manage and track their deployments, and how would we manage conflicts between dependencies from other application repositories.

@bnookala for context

[andrebriggs]

@andrebriggs yes, that's correct. Though these external applications may be defined inside one or more application monorepositories. So this also covers how would we may manage and track their deployments, and how would we manage conflicts between dependencies from other application repositories.

@bnookala for context

@bnookala context?

[andrebriggs]

As discussed @bnookala @mtarng will re-work this issue