[Proposal] - introduction of Deployment

[Haishi2016]

(It's been mentioned during other discussions but I want to put up a separate proposal for it)

Overview

Conceptually, ApplicationConfiguration serves for several purposes - it describes application topology, it describes configurations, and it represents a deployed unit. With the new application concept, which is being tracked in another proposal, we are separating the first two responsibilities. This proposal proposes we separate representation of a deployed unit out of ApplicationConfiguration and formally introduce a Deployment (or Application Instance) object. A Deployment object represent a specific instantiation of an application. It captures information that is relevant to a particular deployment. This will be a natural place to describe rolling updates, canary deployments, blue/green deployments etc.

Spec

The simplest Deployment could be as simple as referencing to an application version:

apiVersion: core.oam.dev/v1alpha3
kind: Deployment
metadata:
  name: production
spec:
  versions:
     - name: v1
       applicationRef: 
       - name: my-app
          version: v1

And the following sample shows how to describe a canary deployment:

apiVersion: core.oam.dev/v1alpha3
kind: Deployment
metadata:
  name: canary
spec:
  versions:
     - name: v1
       applicationRef: 
       - name: my-app
          version: v1
    - name: v2
       applicationRef: 
       - name: my-app
          version: v2
  components:
   - website
  traffic:
  - version: v1
     weight: 70%
  - version: v2
     weight: 30%
  policy:
     primary: v1
     secondary: v2
     rollback:
        errors: 10%
        interval: 1m
     promotion:
        step: 40%
        success: 80%
        interval: 1m

Note that the above sample assumes ingress/load-balancer being a built-in concept of the new Application object hence is not explicitly specified here. If this is not the case, we'll need to add a reference to an ingress trait.

Value

• Formally separates abstract description of an application and an instance of an application
• Rich language for describing complex deployment scenarios on one or multiple versions

Implications for a control plane implementation

• The deployed unit of an application is a Deployment instead of an ApplicationConfiguration

[ryanzhang-oss]

Thanks, Haishi. We (Alibaba) already used OAM to describe upgrade/rollback. In our practice, we apply a trait to a component in a declarative way similar to what you described here. Here are my main observations.

1. Component deployment is mostly application operators' concern and decoupled from the logic of the component. Thus, it seems to be a very natural candidate to be described as a Trait.
2. The cost of introducing another type of object seems to out weight the benefit if any. A trait can have all the fields this deployment object has. We have already implemented rollout/canary traits supporting more features.
3. I am not sure how does the deployment solve the problem to upgrade a micro-service application mentioned in https://github.com/oam-dev/spec/issues/341

[Haishi2016]

My problem with leaving it all to traits is that the trait system is a very generic system. Yes it's capable of describing any intention but none of those is on OAM model. Hence, the semantic meanings of these traits subject to specific implementation/interpretation. To me, for OAM to have value that is broadly applicable to everyone, we have to provide common abstractions and concepts as the OAM model itself. Otherwise, we have an model that can be powerful when backed by a specific implementation but powerless otherwise. And there are no guaranteed consistent behavior across implementations because too much is relying on implementations.

[ryanzhang-oss]

or OAM to have value that is broadly applicable to everyone, we have to provide common abstractions and concepts as the OAM model itself. Otherwise, we have an model that can be powerful when backed by a specific implementation but powerless otherwise. And there are no guaranteed consistent behavior across implementations because too much is relying on implementations.

Agree. However, isn't standard traits/scope/workload the OAM way to provide that common abstraction and portability?

[Haishi2016]

Rolling updates or canary deployments involve multiple versions of an app/component. How do you apply a trait to multiple version of the same component?

[ryanzhang-oss]

Rolling updates or canary deployments involve multiple versions of an app/component. How do you apply a trait to multiple version of the same component?

That's the reason we are proposing https://github.com/oam-dev/spec/issues/336. I will update that issue with more detailed examples

[Haishi2016]

Hey @ryanzhang-oss, can you please provide an example of a trait describing rolling update or canary deployment? It will help me to understand how you envision to express such scenarios. Thanks.

[negz]

Naming nitpick: When there are two kinds of the same name in different API groups (e.g. kind: Deployment, group: apps, and kind: Deployment, group: core.oam.dev) I believe kubectl get deployments will show (only) the custom resources of the lexicographically first API group. That is to say, we probably don't want to use kind: Deployment, because it would cause confusion for any OAM runtime built on the Kubernetes API server.

[hongchaodeng]

When there are two kinds of the same name in different API groups (e.g. kind: Deployment, group: apps, and kind: Deployment, group: core.oam.dev)

We should coin a new name, like AppDeployment.

[hongchaodeng]

I like this proposal. I think the Deployment described above is very close to the ComponentRevision in https://github.com/oam-dev/spec/issues/336.

Let me put some examples to describe ComponentRevision:

kind: ComponentRevision
metadata:
  name: web-service
spec:
  versions:
  - name: v1
    component: web-v1
  - name: v2
    component: web-v2

In this way, we can keep a history of all revisions of components that's been rollout.

Therefore we can describe traffic shifting and canary rollout traits like:

kind: ApplicationConfiguration
spec:
  components:
  - revisionName: web-service
    traits:
    - trait:
        traffic:
        - revision: v1
          weight: "70%"
        - revision: v2
          weight: "30%"
    - trait:
        canary:
          source: v1
          target: v2
          analysis:
            interval: 1m
            # upper threshold of failure rate to trigger rollback
            errorThreshold: "10%"
            metrics:
            - name: request-success-rate
              thresholdRange:
                min: "99%"
              interval: 1m

[wonderflow]

The idea of ComponentRevision is really good and this don't add concept burden to OAM. And for the naming issue, I totally agree with @negz . We shouldn't add new Name to OAM which is very common in K8s

[Haishi2016]

In that case, should we consider renaming ApplicationConfiguration to AppDeployment?

[wonderflow]

In that case, should we consider renaming ApplicationConfiguration to AppDeployment?

Not necessary I think. Both of them can be understood as a deployment thing

[Haishi2016]

Well, if it's an deployment why don't just call it deployment. I recommend to use straightforward terms that don't need implied knowledge or understanding. "Configuration" at surface has no connection to a "Deployment", though under OAM it's implied that when you deploy you deploy an ApplicationConfiguration. I think we should try to reduce such subtle indirection.

[resouer]

Thanks @Haishi2016 , it seems interesting!

Note with ##336 addressed, we should be able to use ApplicationConfiguration to model such Deployment for component level. Also, for:

... we have to provide common abstractions and concepts as the OAM model itself.

It seems standard workloads + standard traits could achieve this goal?

For the naming, I tend to agree with @negz.

[Haishi2016]

I'm okay if we use #336 in conjunction with ApplicationConfiguraiton to model such deployment, as long as we can rename ApplicationConfiguration to ApplicationDeployment or AppDeployment.

[resouer]

@Haishi2016 yeah that's the current idea in my mind.

Would you like to work on #336 to design how we can model revision in detail? We can work together to make it a concrete proposal.

[Haishi2016]

Sure.

[ryanzhang-oss]

Hey @ryanzhang-oss, can you please provide an example of a trait describing rolling update or canary deployment? It will help me to understand how you envision to express such scenarios. Thanks.

Hi, please take a look at https://github.com/oam-dev/spec/issues/336 , I have added a detailed design proposal.

[draveness]

For the naming, I tend to agree with @negz , but I hope we can change ApplicationConfiguration to ApplicationDeployment (and echo question (2) above).

+1 on renaming to ApplicationDeployment, the ApplicationConfiguration sounds like a configuration data of an application to me in the first place.

[resouer]

@draveness ApplicationConfiguration is indeed the configuration for the app. Essentially:

1. Your Application is composed by a bunch of Components, ref: https://github.com/oam-dev/spec/issues/334
2. ApplicationConfiguration = Apply operational capabilities to your application (i.e. Components)

[resouer]

Add this in milestone which is closely related to application level rollout and revisioning.

[resouer]

So the workflow is:

1. User create Application object named my-app.
2. OAM runtime generate application revision named with -v1 suffix.
3. User create Deployment object named my-deploy, reference my-app like below:

apiVersion: core.oam.dev/v1alpha3
kind: Deployment
metadata:
  name: canary
spec:
  versions:
    - name: v1
      applicationRef:
        - name: my-app
          version: v1
  components:
    - website # indicates which component will receive traffic (i.e. thru its ingress trait)
  traffic:
    - version: v1
      weight: 100%

4. User update my-app.
5. CLI/OAM runtime generate application revision, named with -v2 suffix (or, track them with revisions).
6. User update my-deploy to this one to do canary rollout.

[krmayankk]

how do we ensure ordering of changes vi pipeline from one environment to another, is there something in the plans to model that ?