[POC] TEP-0086 - Large results using workspace

[ScrapCodes]

Changes

API changes

1. Added new resultype : reference.
2. A user provided workspaces for results can be configured at the pipeline level.

functional changes

3. As part of POC, entrypoint script is updated to copy the result with reference type to user-configured worspace. TODO: this copy step can be avoided if the results can somehow be generated directly at the desired location.
4. Reference type results are not stored in the etcd, instead their reference i.e. the path to their location is stored. This means whenever a reference type results value is queried it gives the path and not the actual content.
5. There is no imposed limit on the size of the result when type is reference. If the contents do not exceed size limit of the pvc backing the workspace, it is expected to work properly.
6. An example of concating text using the reference type result is provided along with few benchmarking examples - for comparing performance against string(default) result type.

Benchmark summary

1. Local kind cluster with 20 tasks reading and writing about 2mb of data to results with reference type.
   Benchmark suite ran: examples/v1beta1/pipelineruns/concat-text-ref-results-bench.yaml Total time taken: 2m7s

2. Local kind cluster with 20 tasks reading and writing about 2mb of data to results with-out reference type i.e. default result type. Benchmark suite ran: examples/v1beta1/pipelineruns/concat-text-normal-results-bench.yaml Total time taken: 1m59s

3. Same as 1. ran on IBM Cloud kubernetes service with 3Nodes of 8 vcpu and 32GB ram. Using hostpath storageclass. Includes time required to create PV,PVC and bind them. Total time taken: 2m40s

4. Same as 2. on IBM Cloud Total time taken: 2m22s

TODO

1. Support default workspace for result

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

• [ ] Has Docs included if any changes are user facing
• [ ] Has Tests included if any functionality added or changed
• [ ] Follows the commit message standard
• [ ] Meets the Tekton contributor standards (including functionality, content, code)
• [ ] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
• [ ] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
• [ ] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

New resultype reference, to support large results using workspace.

/kind tep

[tekton-robot]

@ScrapCodes: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tekton-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: To complete the pull request process, please assign jerop after the PR has been reviewed. You can assign the PR to them by writing /assign @jerop in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[tekton-robot]

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/main.go	13.6%	13.1%	-0.4
pkg/entrypoint/entrypointer.go	84.8%	53.6%	-31.3
pkg/pod/entrypoint.go	89.5%	88.8%	-0.8
pkg/pod/pod.go	89.5%	89.6%	0.1
pkg/reconciler/pipelinerun/resources/apply.go	98.8%	98.2%	-0.6

[ScrapCodes]

/kind tep

[tekton-robot]

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/main.go	13.6%	13.1%	-0.4
pkg/entrypoint/entrypointer.go	84.8%	53.2%	-31.7
pkg/pod/entrypoint.go	89.5%	88.8%	-0.8
pkg/pod/pod.go	89.5%	89.6%	0.1
pkg/reconciler/taskrun/validate_resources.go	97.7%	97.3%	-0.4

[tekton-robot]

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/main.go	13.6%	13.1%	-0.4
pkg/entrypoint/entrypointer.go	84.8%	59.0%	-25.8
pkg/pod/entrypoint.go	89.5%	87.0%	-2.5
pkg/pod/pod.go	89.5%	89.6%	0.1
pkg/reconciler/taskrun/validate_resources.go	97.7%	97.3%	-0.4

[tekton-robot]

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/main.go	13.6%	13.1%	-0.4
pkg/entrypoint/entrypointer.go	84.8%	59.0%	-25.8
pkg/pod/entrypoint.go	89.5%	87.0%	-2.5
pkg/pod/pod.go	89.5%	89.6%	0.1
pkg/reconciler/taskrun/validate_resources.go	97.7%	97.3%	-0.4

[tekton-robot]

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/main.go	13.6%	13.1%	-0.4
pkg/entrypoint/entrypointer.go	84.8%	59.0%	-25.8
pkg/pod/entrypoint.go	89.5%	87.0%	-2.5
pkg/pod/pod.go	89.5%	89.6%	0.1
pkg/reconciler/pipelinerun/pipelinerun.go	85.1%	84.8%	-0.3
pkg/reconciler/taskrun/validate_resources.go	97.7%	97.3%	-0.4

[tekton-robot]

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/main.go	13.6%	13.1%	-0.4
pkg/entrypoint/entrypointer.go	84.8%	59.0%	-25.8
pkg/pod/entrypoint.go	89.5%	87.0%	-2.5
pkg/pod/pod.go	89.5%	89.6%	0.1
pkg/reconciler/pipelinerun/pipelinerun.go	85.1%	84.8%	-0.3
pkg/reconciler/taskrun/validate_resources.go	97.7%	97.3%	-0.4

[tekton-robot]

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/main.go	13.6%	13.1%	-0.4
pkg/entrypoint/entrypointer.go	84.8%	59.0%	-25.8
pkg/pod/entrypoint.go	89.5%	87.0%	-2.5
pkg/pod/pod.go	89.5%	89.6%	0.1
pkg/reconciler/pipelinerun/pipelinerun.go	85.1%	84.8%	-0.3
pkg/reconciler/taskrun/validate_resources.go	97.7%	97.3%	-0.4

[tekton-robot]

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/main.go	13.6%	13.1%	-0.4
pkg/entrypoint/entrypointer.go	84.8%	59.0%	-25.8
pkg/pod/entrypoint.go	89.5%	87.0%	-2.5
pkg/pod/pod.go	89.5%	89.6%	0.1
pkg/reconciler/pipelinerun/pipelinerun.go	85.1%	84.8%	-0.3
pkg/reconciler/taskrun/validate_resources.go	97.7%	97.3%	-0.4

[tekton-robot]

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/main.go	13.6%	13.1%	-0.4
pkg/entrypoint/entrypointer.go	84.8%	59.0%	-25.8
pkg/pod/entrypoint.go	89.5%	87.0%	-2.5
pkg/pod/pod.go	89.5%	89.6%	0.1
pkg/reconciler/pipelinerun/pipelinerun.go	85.1%	84.8%	-0.3
pkg/reconciler/taskrun/validate_resources.go	97.7%	97.3%	-0.4

[tekton-robot]

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/main.go	13.6%	13.1%	-0.4
pkg/entrypoint/entrypointer.go	84.8%	59.0%	-25.8
pkg/pod/entrypoint.go	89.5%	87.0%	-2.5
pkg/pod/pod.go	89.5%	89.6%	0.1
pkg/reconciler/pipelinerun/pipelinerun.go	85.1%	84.8%	-0.3
pkg/reconciler/taskrun/validate_resources.go	97.7%	97.3%	-0.4

[ScrapCodes]

I have screen recorded the demo video: https://youtu.be/bKXabab_jUY

[ScrapCodes]

A typical run looks like this:

Name:         concat-text-pipeline-run
Namespace:    default
Labels:       tekton.dev/pipeline=concat-text-pipeline
Annotations:  <none>
API Version:  tekton.dev/v1beta1
Kind:         PipelineRun
Metadata:
  Creation Timestamp:  2022-08-29T16:17:06Z
  Generation:          1
  Managed Fields:
    API Version:  tekton.dev/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:pipelineRef:
          .:
          f:name:
        f:workspaces:
    Manager:      kubectl-client-side-apply
    Operation:    Update
    Time:         2022-08-29T16:17:06Z
    API Version:  tekton.dev/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .:
          f:tekton.dev/pipeline:
    Manager:      controller
    Operation:    Update
    Time:         2022-08-29T16:17:07Z
    API Version:  tekton.dev/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:completionTime:
        f:conditions:
        f:pipelineResults:
        f:pipelineSpec:
          .:
          f:resultWorkspace:
            .:
            f:name:
          f:results:
          f:tasks:
          f:workspaces:
        f:startTime:
        f:taskRuns:
          .:
          f:concat-text-pipeline-run-first-task:
            .:
            f:pipelineTaskName:
            f:status:
              .:
              f:completionTime:
              f:conditions:
              f:podName:
              f:startTime:
              f:steps:
              f:taskResults:
              f:taskSpec:
                .:
                f:results:
                f:steps:
                f:workspaces:
          f:concat-text-pipeline-run-second-task:
            .:
            f:pipelineTaskName:
            f:status:
              .:
              f:completionTime:
              f:conditions:
              f:podName:
              f:startTime:
              f:steps:
              f:taskResults:
              f:taskSpec:
                .:
                f:results:
                f:steps:
                f:workspaces:
          f:concat-text-pipeline-run-third-task:
            .:
            f:pipelineTaskName:
            f:status:
              .:
              f:completionTime:
              f:conditions:
              f:podName:
              f:startTime:
              f:steps:
              f:taskResults:
              f:taskSpec:
                .:
                f:params:
                f:results:
                f:steps:
                f:workspaces:
    Manager:         controller
    Operation:       Update
    Subresource:     status
    Time:            2022-08-29T16:20:33Z
  Resource Version:  3627853
  UID:               73da1017-ea07-47e1-a747-ba2b99aab31a
Spec:
  Pipeline Ref:
    Name:                concat-text-pipeline
  Service Account Name:  default
  Timeout:               1h0m0s
  Workspaces:
    Name:  shared-data
    Persistent Volume Claim:
      Claim Name:  shared-task-storage
Status:
  Completion Time:  2022-08-29T16:20:33Z
  Conditions:
    Last Transition Time:  2022-08-29T16:20:33Z
    Message:               Tasks Completed: 3 (Failed: 0, Cancelled 0), Skipped: 0
    Reason:                Succeeded
    Status:                True
    Type:                  Succeeded
  Pipeline Results:
    Name:   sum
    Value:  /workspace/result-storages/concat-text-pipeline-run-third-task/concatenated-text.  # <-- path to the result instead of the actual value.
  Pipeline Spec:
    Result Workspace:
      Name:  shared-data
    Results:
      Description:  the concat of all texts
      Name:         sum
      Type:         reference
      Value:        $(tasks.third-task.results.concatenated-text)
    Tasks:
      Name:  first-task
      Task Ref:
        Kind:  Task
        Name:  gen-random-text
      Workspaces:
        Name:       result-storages
        Workspace:  shared-data
      Name:         second-task
      Run After:
        first-task
      Task Ref:
        Kind:  Task
        Name:  gen-random-text
      Workspaces:
        Name:       result-storages
        Workspace:  shared-data
      Name:         third-task
      Params:
        Name:   first
        Value:  $(tasks.first-task.results.text)
        Name:   second
        Value:  $(tasks.second-task.results.text)
      Run After:
        second-task
      Task Ref:
        Kind:  Task
        Name:  concat-text
      Workspaces:
        Name:       result-storages
        Workspace:  shared-data
    Workspaces:
      Name:    shared-data
  Start Time:  2022-08-29T16:17:06Z
  Task Runs:
    Concat - Text - Pipeline - Run - First - Task:
      Pipeline Task Name:  first-task
      Status:
        Completion Time:  2022-08-29T16:20:19Z
        Conditions:
          Last Transition Time:  2022-08-29T16:20:19Z
          Message:               All Steps have completed executing
          Reason:                Succeeded
          Status:                True
          Type:                  Succeeded
        Pod Name:                concat-text-pipeline-run-first-task-pod
        Start Time:              2022-08-29T16:17:07Z
        Steps:
          Container:  step-add
          Image ID:   docker.io/library/alpine@sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395ee852a8d9730fc2ad
          Name:       add
          Terminated:
            Container ID:  containerd://ad6b5428e190745201121dc579d83949d702aa8c168a422d51c2e7eab8bb23d6
            Exit Code:     0
            Finished At:   2022-08-29T16:20:19Z
            Message:       [{"key":"text","value":"/workspace/result-storages/concat-text-pipeline-run-first-task/text","type":1}]
            Reason:        Completed
            Started At:    2022-08-29T16:20:19Z
        Task Results:
          Name:   text
          Type:   string
          Value:  /workspace/result-storages/concat-text-pipeline-run-first-task/text
        Task Spec:
          Results:
            Description:  large randomly generated string
            Name:         text
            Type:         reference
          Steps:
            Args:
              cat /dev/urandom | base64 | head -c 4096 | tee /tekton/results/text;
            Command:
              /bin/sh
              -c
            Image:  alpine
            Name:   add
            Resources:
          Workspaces:
            Name:  result-storages
    Concat - Text - Pipeline - Run - Second - Task:
      Pipeline Task Name:  second-task
      Status:
        Completion Time:  2022-08-29T16:20:26Z
        Conditions:
          Last Transition Time:  2022-08-29T16:20:26Z
          Message:               All Steps have completed executing
          Reason:                Succeeded
          Status:                True
          Type:                  Succeeded
        Pod Name:                concat-text-pipeline-run-second-task-pod
        Start Time:              2022-08-29T16:20:20Z
        Steps:
          Container:  step-add
          Image ID:   docker.io/library/alpine@sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395ee852a8d9730fc2ad
          Name:       add
          Terminated:
            Container ID:  containerd://5bac5e39be42f9e658e4a17a94805fc2ccd0a121b3689a3a355d1f4fa1640665
            Exit Code:     0
            Finished At:   2022-08-29T16:20:25Z
            Message:       [{"key":"text","value":"/workspace/result-storages/concat-text-pipeline-run-second-task/text","type":1}]
            Reason:        Completed
            Started At:    2022-08-29T16:20:25Z
        Task Results:
          Name:   text
          Type:   string
          Value:  /workspace/result-storages/concat-text-pipeline-run-second-task/text
        Task Spec:
          Results:
            Description:  large randomly generated string
            Name:         text
            Type:         reference
          Steps:
            Args:
              cat /dev/urandom | base64 | head -c 4096 | tee /tekton/results/text;
            Command:
              /bin/sh
              -c
            Image:  alpine
            Name:   add
            Resources:
          Workspaces:
            Name:  result-storages
    Concat - Text - Pipeline - Run - Third - Task:
      Pipeline Task Name:  third-task
      Status:
        Completion Time:  2022-08-29T16:20:33Z
        Conditions:
          Last Transition Time:  2022-08-29T16:20:33Z
          Message:               All Steps have completed executing
          Reason:                Succeeded
          Status:                True
          Type:                  Succeeded
        Pod Name:                concat-text-pipeline-run-third-task-pod
        Start Time:              2022-08-29T16:20:26Z
        Steps:
          Container:  step-concat
          Image ID:   docker.io/library/alpine@sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395ee852a8d9730fc2ad
          Name:       concat
          Terminated:
            Container ID:  containerd://4a6d719462377f806658d77510027aa695f3ee3f32f3d2b80bbac8a04214a379
            Exit Code:     0
            Finished At:   2022-08-29T16:20:32Z
            Message:       [{"key":"concatenated-text","value":"/workspace/result-storages/concat-text-pipeline-run-third-task/concatenated-text","type":1}]
            Reason:        Completed
            Started At:    2022-08-29T16:20:32Z
        Task Results:
          Name:   concatenated-text
          Type:   string
          Value:  /workspace/result-storages/concat-text-pipeline-run-third-task/concatenated-text
        Task Spec:
          Params:
            Description:  the first operand
            Name:         first
            Type:         string
            Description:  the second operand
            Name:         second
            Type:         string
          Results:
            Description:  concatenate strings
            Name:         concatenated-text
            Type:         reference
          Steps:
            Args:
              cat /workspace/result-storages/concat-text-pipeline-run-first-task/text /workspace/result-storages/concat-text-pipeline-run-second-task/text | tee /tekton/results/concatenated-text ;
            Command:
              /bin/sh
              -c
            Image:  alpine
            Name:   concat
            Resources:
          Workspaces:
            Name:  result-storages
Events:
  Type    Reason     Age                From         Message
  ----    ------     ----               ----         -------
  Normal  Started    41m (x2 over 41m)  PipelineRun  
  Normal  Running    41m (x2 over 41m)  PipelineRun  Tasks Completed: 0 (Failed: 0, Cancelled 0), Incomplete: 3, Skipped: 0
  Normal  Running    38m                PipelineRun  Tasks Completed: 1 (Failed: 0, Cancelled 0), Incomplete: 2, Skipped: 0
  Normal  Running    38m                PipelineRun  Tasks Completed: 2 (Failed: 0, Cancelled 0), Incomplete: 1, Skipped: 0
  Normal  Succeeded  38m                PipelineRun  Tasks Completed: 3 (Failed: 0, Cancelled 0), Skipped: 0

[vdemeester]

/hold

[tekton-robot]

@ScrapCodes: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tekton-robot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale with a justification. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close with a justification. If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten with a justification. Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with /close with a justification. If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[jerop]

@ScrapCodes are you still experimenting with this alternative?

If you have some time, please join us at TEP-0086 WG to share an update and discuss a way forward

cc @pritidesai @afrittoli

[ScrapCodes]

@jerop , Yes, how can we move forward?