openshift
CNTRLPLANE-947: E2E test adaptations for OIDC
In order to be able to run openshift's conformance e2e test suite on a cluster with external OIDC configured, we must skip any tests that are inherently irrelevant to OIDC (for example, tests against the OAuth APIs must be skipped, as these APIs do not exist in a cluster with external OIDC).
However, there's a number of tests that we want to avoid skipping (e.g. checking apiserver availability); this PR makes adaptations to such tests that currently break when OIDC is configured but we don't want to skip completely.
Summary of changes
-
authorization_rbac_proxy: when the users API is not present, theocuser created in this test comes from client.go; the order of the groups is different than the one the test expects, so we must make the test check expect both orders -
apiserver-external-availabilitymonitor test: this test checks all API servers, including the oauth apiserver; we adapt this test to skip the oauth apiserver when OIDC is configured (as it does not exist) -
management_plane_operators: when OIDC is configured, the authentication operator does not have some conditions that are listed as always required in this test; this PR introduces a mechanism to determine some cluster-runtime conditions depending on cluster config/state and moves the respective auth operator ones there
Example failed run of conformance suite with OIDC configured: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/66981/rehearse-66981-periodic-ci-openshift-cluster-authentication-operator-release-4.21-periodics-e2e-aws-external-oidc-conformance-parallel-techpreview/1970076671268622336
@liouk: This pull request references CNTRLPLANE-947 which is a valid jira issue.
In response to this:
This PR makes e2e test adaptations for the case of a cluster with external OIDC authentication configured. These are tests we do not want to skip completely.
Summary of changes
authorization_rbac_proxy: when the users API is not present, theocuser created in this test comes from client.go; the order of the groups is different than the one the test expects, so we must make the test check more flexible.apiserver-external-availabilitymonitor test: this test checks all API servers, including the oauth apiserver; we adapt this test to skip the oauth apiserver when OIDC is configured (as it does not exist)management_plane_operators: when OIDC is configured, the authentication operator does not have some conditions that are listed as always required in this test; this PR introduces a mechanism to determine some cluster-runtime conditions depending on cluster config/state and moves the respective auth operator ones there
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
@liouk: This pull request references CNTRLPLANE-947 which is a valid jira issue.
In response to this:
This PR makes e2e test adaptations for the case of a cluster with external OIDC authentication configured. These are tests we do not want to skip completely.
Summary of changes
authorization_rbac_proxy: when the users API is not present, theocuser created in this test comes from client.go; the order of the groups is different than the one the test expects, so we must make the test check expect both ordersapiserver-external-availabilitymonitor test: this test checks all API servers, including the oauth apiserver; we adapt this test to skip the oauth apiserver when OIDC is configured (as it does not exist)management_plane_operators: when OIDC is configured, the authentication operator does not have some conditions that are listed as always required in this test; this PR introduces a mechanism to determine some cluster-runtime conditions depending on cluster config/state and moves the respective auth operator ones there
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
@liouk: This pull request references CNTRLPLANE-947 which is a valid jira issue.
In response to this:
This PR makes e2e test adaptations for the case of a cluster with external OIDC authentication configured. These are tests we do not want to skip completely.
Summary of changes
authorization_rbac_proxy: when the users API is not present, theocuser created in this test comes from client.go; the order of the groups is different than the one the test expects, so we must make the test check expect both ordersapiserver-external-availabilitymonitor test: this test checks all API servers, including the oauth apiserver; we adapt this test to skip the oauth apiserver when OIDC is configured (as it does not exist)management_plane_operators: when OIDC is configured, the authentication operator does not have some conditions that are listed as always required in this test; this PR introduces a mechanism to determine some cluster-runtime conditions depending on cluster config/state and moves the respective auth operator ones thereExample failed run of conformance suite with OIDC configured: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/66981/rehearse-66981-periodic-ci-openshift-cluster-authentication-operator-release-4.21-periodics-e2e-aws-external-oidc-conformance-parallel-techpreview/1970076671268622336
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
@liouk: This pull request references CNTRLPLANE-947 which is a valid jira issue.
In response to this:
In order to be able to run openshift's conformance e2e test suite on a cluster with external OIDC configured, we must skip any tests that are inherently irrelevant to OIDC (for example, tests against the OAuth APIs must be skipped, as these APIs do not exist in a cluster with external OIDC).
However, there's a number of tests that we want to avoid skipping (e.g. checking apiserver availability); this PR makes adaptations to such tests that currently break when OIDC is configured but we don't want to skip completely.
Summary of changes
authorization_rbac_proxy: when the users API is not present, theocuser created in this test comes from client.go; the order of the groups is different than the one the test expects, so we must make the test check expect both ordersapiserver-external-availabilitymonitor test: this test checks all API servers, including the oauth apiserver; we adapt this test to skip the oauth apiserver when OIDC is configured (as it does not exist)management_plane_operators: when OIDC is configured, the authentication operator does not have some conditions that are listed as always required in this test; this PR introduces a mechanism to determine some cluster-runtime conditions depending on cluster config/state and moves the respective auth operator ones thereExample failed run of conformance suite with OIDC configured: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/66981/rehearse-66981-periodic-ci-openshift-cluster-authentication-operator-release-4.21-periodics-e2e-aws-external-oidc-conformance-parallel-techpreview/1970076671268622336
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
@liouk What's the minimal set of required tests you'd like to see pass here before this merges. I don't want to get stuck in a retest quagmire.
@sdodson when it comes to making sure that the changes in this PR do not break existing tests, I've already seen enough successful runs of the tests changed in the jobs that have already run successfully.
From the current failing jobs, the ones that contain the updated tests are the following:
-
e2e-aws-ovn -
e2e-aws-ovn-single-node-upgrade -
e2e-aws-proxy -
e2e-openstack-ovn
However these aren't required anyway, so I doubt we should block this PR until these succeed.
Apart from verifying we're not breaking any existing tests, I would like to see the results of the jobs introduced with https://github.com/openshift/release/pull/66981 and I'm planning on running the conformance suites at a local cluster -- however I am not aware of any way to run those jobs on the CI and include the changes of this PR before it merges (let me know if there's a way!).
Job Failure Risk Analysis for sha: c1099612c29d2de892241141fcf3c6f0666f897a
| Job Name | Failure Risk |
|---|---|
| pull-ci-openshift-origin-main-e2e-aws-disruptive | High Job run should complete before timeout This test has passed 98.08% of 5312 runs on release 4.21 [Overall] in the last week. |
![openshift-trt[bot] avatar](https://avatars.githubusercontent.com/u/111778100?v=4 "Published by a pub.dev verified publisher"){kind=small}
Scheduling required tests: /test e2e-aws-csi /test e2e-aws-ovn-fips /test e2e-aws-ovn-microshift /test e2e-aws-ovn-microshift-serial /test e2e-aws-ovn-serial-1of2 /test e2e-aws-ovn-serial-2of2 /test e2e-gcp-csi /test e2e-gcp-ovn /test e2e-gcp-ovn-upgrade /test e2e-metal-ipi-ovn-ipv6 /test e2e-vsphere-ovn /test e2e-vsphere-ovn-upi
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: bertinatto, everettraven, liouk, sdodson
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~OWNERS~~ [bertinatto]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
![openshift-ci[bot] avatar](https://avatars.githubusercontent.com/in/91280?v=4 "Published by a pub.dev verified publisher"){kind=small}
Verified on a local cluster with OIDC, as we don't have any jobs yet to run on CI (to be introduced with https://github.com/openshift/release/pull/66981).
/verified by @liouk
@liouk: This PR has been marked as verified by @liouk.
In response to this:
Verified on a local cluster with OIDC, as we don't have any jobs yet to run on CI (to be introduced with https://github.com/openshift/release/pull/66981).
/verified by @liouk
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
/retest-required
Remaining retests: 0 against base HEAD c8f42f85395d8fab69959032e20e71252714878a and 2 for PR HEAD 63b992211899fadfac5406d86b0b3e6f034fb48d in total
@liouk: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:
| Test name | Commit | Details | Required | Rerun command |
|---|---|---|---|---|
| ci/prow/okd-scos-e2e-aws-ovn | c1099612c29d2de892241141fcf3c6f0666f897a | link | false | /test okd-scos-e2e-aws-ovn |
| ci/prow/e2e-aws-ovn-single-node-serial | c1099612c29d2de892241141fcf3c6f0666f897a | link | false | /test e2e-aws-ovn-single-node-serial |
| ci/prow/e2e-aws-disruptive | c1099612c29d2de892241141fcf3c6f0666f897a | link | false | /test e2e-aws-disruptive |
| ci/prow/e2e-aws-ovn-cgroupsv2 | c1099612c29d2de892241141fcf3c6f0666f897a | link | false | /test e2e-aws-ovn-cgroupsv2 |
| ci/prow/e2e-openstack-ovn | c1099612c29d2de892241141fcf3c6f0666f897a | link | false | /test e2e-openstack-ovn |
Full PR test history. Your PR dashboard.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.
/retest-required
Remaining retests: 0 against base HEAD 0015a037b5369b9db2b67a6c4a6d614ff9606717 and 1 for PR HEAD 63b992211899fadfac5406d86b0b3e6f034fb48d in total