container-object-storage-interface-api icon indicating copy to clipboard operation
container-object-storage-interface-api copied to clipboard

feat: add COSI test suite using Chainsaw

Open shanduur opened this issue 1 year ago • 4 comments

This pull request introduces a new COSI (Container Object Storage Interface) test suite using Chainsaw, along with several enhancements and preparations for dynamic test generation.

  1. COSI Test Suite Implementation:

    • Added an end-to-end (E2E) test suite for COSI, focusing on various authentication methods for S3 and Azure.
  2. Test Generator:

    • Implemented a test generator to dynamically create tests, reducing manual effort and ensuring consistency across the test suite.
  3. KinD Cluster Configuration:

    • Configured ctlptl for setting up a KinD (Kubernetes in Docker) cluster to facilitate testing in a controlled environment. TODO: this needs future updates, e.g. Makefile targets, to ensure that tests can be run locally, including installation of sample driver, CRDs and controller.

The dynamic test generator is now in place, which will allow for easier updates and extensions to the test suite in future iterations. This PR is foundational for future enhancements and ensures that the COSI functionality is thoroughly tested across different authentication mechanisms.

shanduur avatar Aug 23 '24 11:08 shanduur

Skipping CI for Draft Pull Request. If you want CI signal for your change, please convert it to an actual PR. You can still manually trigger a test run with /test all

k8s-ci-robot avatar Aug 23 '24 11:08 k8s-ci-robot

/cc @BlaineEXE

shanduur avatar Sep 04 '24 07:09 shanduur

In this other PR, I'm adding the 3 remaining e2e/lint targets that are suggested by Kubebuilder's template: https://github.com/kubernetes-sigs/container-object-storage-interface-api/pull/96/files

I think it would be nice to see how this PR would integrate with make in the test-e2e target. Presumably, the tooling would have to install ctlptl and chainsaw, set up kind, and then run the tests, which seems fairly straighforward.

If you think it would be better to do the tooling in another PR, maybe a test/e2e/readme.md could give a simple bullet list instruction about how we can run these ourselves manually?

BlaineEXE avatar Sep 09 '24 18:09 BlaineEXE

I fully tested that on Linode COSI Driver using the following values:

driverName: objectstorage.cosi.linode.com
deletionPolicy: "Delete" # "Retain"
bucketClassParams:
  cosi.linode.com/v1/region: us-east
  cosi.linode.com/v1/acl: private
  cosi.linode.com/v1/cors: disabled
bucketAccessClassParams:
  cosi.linode.com/v1/permissions: read_write
authenticationType: "Key" # "IAM"
bucketClaimProtocols: ["S3"]
bucketAccessProtocol: "S3"

Test suite run in ~22s, including full Bucket/BucketAccess/BucketClaim provisioning and secret schema validation.

shanduur avatar Sep 26 '24 10:09 shanduur

I included a simple readme, however, I am not adding any make targets yet. I think it will be better to do those in subsequent PRs.

shanduur avatar Sep 26 '24 10:09 shanduur

Looks like a great starting point to me. Thanks @shanduur !

BlaineEXE avatar Sep 30 '24 18:09 BlaineEXE

/lgtm /approve

BlaineEXE avatar Sep 30 '24 18:09 BlaineEXE

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BlaineEXE, shanduur

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • ~~OWNERS~~ [BlaineEXE,shanduur]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Sep 30 '24 18:09 k8s-ci-robot