allure-java icon indicating copy to clipboard operation
allure-java copied to clipboard

TestNG retries are marked as Skipped and the test itself is not marked as Flaky Test

Open marvelfrozen opened this issue 5 years ago • 0 comments

I'm submitting a ...

  • [X] bug report
  • [ ] feature request
  • [ ] support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Tests that failed and then success after re-run using TestNG retryAnalyzer marked as skipped and not showing the Flaky Test (bomb) icon. image image

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

The retry impl class

public class RetryAnalyzerImpl implements IRetryAnalyzer {
	private int maxRetryCount = 3;
	private int retryCount = 0;

	public boolean retry(ITestResult result) {
		if (retryCount < maxRetryCount) {
			retryCount++;
			return true;
		}
		return false;
	}
}

The test method

@Test(retryAnalyzer= RetryAnalyzerImpl.class)
public void VerifyRetrievePublicaAndActiveCommunity(){
	int i = ThreadLocalRandom.current().nextInt(0, 2);
	System.out.println(i);
	Assert.assertEquals(i,0);
}

What is the expected behavior?

The retries are marked as Failed correctly. And the tests are marked as Flaky.

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

| Allure version | 2.13.3 | | Test framework | [email protected] | | Allure integration | [email protected] | | Generate report using | allure windows [email protected] |

Other information

marvelfrozen avatar Jun 18 '20 05:06 marvelfrozen