[allure-testng] Attachment on failed config is not added to the failed configuration method

[baflQA]

Describe the bug There is a test with a configuration method (BeforeMethod) that fails with an exception. There is an IConfigurationListener implementation provided, that creates an attachment on failed configuration. An attachment is expected to be present in the report inside the failed configuration method container, but it is not. An attachment can be found in the allure-results directory but seems not to be attached to any test container.

To Reproduce Steps to reproduce the behaviour:

1. Run https://github.com/baflQA/allure-java/blob/master/allure-testng/src/test/java/io/qameta/allure/testng/AllureTestNgTest.java#L710 test

Expected behaviour An attachment added on failed configuration is added to the failed configuration method.

Additional context Provided a fork to reproduce an issue https://github.com/baflQA/allure-java

[baflQA]

Hi @baev! Would You be able to have a look?

[baev]

I'm not sure we will ever support such behaviour since Allure uses listeners itself to collect data from TestNG and we can't control the listeners order

[baev]

in most cases as workaround you can use AllureLifecycle listeners instead of TestNG listeners

[baflQA]

Ok, I will have a look. But my idea was that MAYBE all the logic of collecting the results of a before configuration methods should be handled inside onConfigurationSuccess/onConfigurationFailure methods of IConfigurationListener.