playwright icon indicating copy to clipboard operation
playwright copied to clipboard

[Feature]: Add all retries as test runs to JUnit report

Open MrGreenTea opened this issue 1 year ago • 2 comments

🚀 Feature Request

A config option for the junit reporter to add all retries to the output file.

Example

No response

Motivation

We're mostly using the junit reports together with flaky detection to monitor and detect flaky tests.

We currently have to use the retries config option which masks some of the flakiness, because the tests appear as success in the report.

I would love to be able to include the retries in our analysis for a more complete and precise flakiness detection.

MrGreenTea avatar Feb 10 '24 14:02 MrGreenTea

+1 would love this data to be included. seeing tests that are flaky can be a good indicator of test suite health

thomas-colgrove avatar Apr 15 '24 18:04 thomas-colgrove

+1 This would be useful in helping us identify flaky tests

ComaToastUK avatar May 01 '24 14:05 ComaToastUK

+1, our team would love this feature. Right now we're sampling test logs occasionally to try and catch flakiness, but it'd be much nicer to be able to follow this as a metric

alice-ep avatar Jul 17 '24 16:07 alice-ep