Qiskit
Add retry for flaky tests
We add retry attemps to the test that writes the edge list
This specific flaky test has made the CI fail multiple times. Instead of manually having to retrigger all of our CI everytime it fails, we attempt to execute the specific test multiple times using tenacity.
Pull Request Test Coverage Report for Build 2779084424
- 0 of 0 changed or added relevant lines in 0 files are covered.
- 2 unchanged lines in 1 file lost coverage.
- Overall coverage increased (+0.009%) to 97.142%
| Files with Coverage Reduction | New Missed Lines | % |
|---|---|---|
| src/shortest_path/all_pairs_bellman_ford.rs | 2 | 98.88% |
| <!-- | Total: | 2 |
| Totals | |
|---|---|
| Change from base Build 2778191284: | 0.009% |
| Covered Lines: | 12540 |
| Relevant Lines: | 12909 |
💛 - Coveralls
Pull Request Test Coverage Report for Build 2804100416
- 0 of 0 changed or added relevant lines in 0 files are covered.
- No unchanged relevant lines lost coverage.
- Overall coverage decreased (-0.05%) to 97.079%
| Totals | |
|---|---|
| Change from base Build 2803715934: | -0.05% |
| Covered Lines: | 12465 |
| Relevant Lines: | 12840 |
💛 - Coveralls
Is a retry actually sufficient for this? The issue we're hitting periodically in CI is python's tempfile usage on the macOS CI nodes isn't creating the file or macOS is flushing it out during the test and it gets deleted before the test is finished. I always assumed it was something more tied to the local run environment that would persist across multiple executions of the test. I've never been able to reproduce these failures locally (but I don't use macOS).
That is a fair point, if the system itself is having problem in tempfiles we should retry running the Github action, not the test.
I am closing the PR