temporal icon indicating copy to clipboard operation
temporal copied to clipboard

Action inputs to dispatch n-runs of a single functional test

Open carlydf opened this issue 1 year ago • 2 comments

What changed?

Add workflow dispatch options to the functional tests Github Action to allow us to run n-iterations of a single functional test with a configurable timeout.

Why?

To aid in the diagnosis and treatment of flaky tests.

How did you test it?

Tested in github actions. Here is the action run normally, with no new input parameters: https://github.com/temporalio/temporal/actions/runs/9962219294

Here is the action run on one test multiple times: https://github.com/temporalio/temporal/actions/runs/9962244076

Potential risks

Our functional test pipeline could be broken by this PR, but we would notice that pretty immediately

Documentation

Is hotfix candidate?

carlydf avatar Jul 16 '24 05:07 carlydf

Can we create a separate workflow file for this purpose? run-tests.yml is complicated enough that adding this seems added complexity unnecessarily. Also, as you noted, running misc-checks is also unnecessary when running a single test.

rodrigozhou avatar Jul 18 '24 07:07 rodrigozhou

Can we create a separate workflow file for this purpose? run-tests.yml is complicated enough that adding this seems added complexity unnecessarily. Also, as you noted, running misc-checks is also unnecessary when running a single test.

A separate workflow file that we run only rarely will rot and be likely to be broken when someone wants to use it. Integrating it into the main one makes it much more likely to be maintained and working

dnr avatar Jul 20 '24 00:07 dnr