etcd-io
testgrid-analysis: create issues based on test set flakiness instead of individual tests.
Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.
https://github.com/etcd-io/etcd/issues/17773#issuecomment-2084928796
Tested locally, example issue: https://github.com/siyuanfoundation/etcd/issues/64
If a test set is flaky, e.g. pull-request unit test fails > 10% of the time, it would create a parent issue for the test set, and create sub-issues for the top 5 fail tests. If the test keep failing, it would append new comment every time the script is run (which is daily), without creating new issues.
Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all
This report looks useful. However, I'm not sure if having it as a GitHub issue makes sense. I feel like we'll just need to close the report issues, which may add to the work of managing issues.
That being said, I agree that there's great value in having the reports more visible rather than nested in a GitHub action run. My suggestion would be to explore Job Summaries. Also, something that would be useful if we decide to use Job Summaries, would be to provide the link to the GitHub action run in the generated issue.
cc. @serathius, as you suggested more visibility of the reports.
How spammy do we expect this new issue opening to be? We already have quite a few open flake issues https://github.com/etcd-io/etcd/issues?q=is%3Aopen+is%3Aissue+label%3Atype%2Fflake
We might end up having to continually close issues as duplicates until some existing known flakes are resolved which could be annoying.
Some description of what impact this pr is expected to have would help make a decision on if it will be helpful or harmful.
I will re-evaluate if this PR is necessary. Mark it as draft for now.
PR needs rebase.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@siyuanfoundation: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:
| Test name | Commit | Details | Required | Rerun command |
|---|---|---|---|---|
| pull-etcd-unit-test-amd64 | edc5f1616278086d781c337bf61a670cf1bee8a6 | link | true | /test pull-etcd-unit-test-amd64 |
| pull-etcd-unit-test-arm64 | edc5f1616278086d781c337bf61a670cf1bee8a6 | link | true | /test pull-etcd-unit-test-arm64 |
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.