ArchTest on fields does not run under Maven Surefire Plugin 3.5.3+

[erwanosouf]

There is a bug when running ArchUnit test using ArchTest on a ArchRule field.

It works under Maven Surefire Plugin 3.5.2 but not 3.5.3

Note that ArchTest set on methods work.

I have created a repo demonstrating the issue. Refer to the Readme for reproduction instruction

Description This repository contains a Test Case to exhibit a behavior that appeared in Surefire 3.5.3.

A Junit5 Test Class ArchUnitSurefireBugTest contains 3 tests

thisTestUsingArchTestFieldShouldRun : A Test using @ ArchTest on a field thisTestUsingArchTestMethodShouldRun : A Test using @ ArchTest on a method thisStandardTestShouldRun : a standard JUnit Test using no ArchUnit syntactic sugar. Expected Run mvn test -Dsurefire.version=3.5.2 -f pom.xml

All tests run

Actual mvn test -Dsurefire.version=3.5.3 -f pom.xml

thisTestUsingArchTestFieldShouldRun does not run.

[leflamm]

Thank you @erwanosouf for figuring this out. Was going crazy over this!

[uhafner]

Is this a bug in surefire or in ArchUnit? I wonder if we should create a bug report in the surefire plugin as well? 3.5.3 is only a patch release that should not fail in such a way?

[hankem]

I did a git bisect on the changes from surefire-3.5.2 to maven-surefire-3.5.3 and identified that commit 5aeca1971 for apache/maven-surefire#815 introduced the change.

[hankem]

(FYI: The issue will be adressed by SUREFIRE-2298 / apache/maven-surefire#828.)

[Eischer]

We have the same behaviour with surefire and failsafe plugin in Version 3.5.3. But not only on ArchUnit-Tests but also on Cucumber-Tests like the following

@Suite(failIfNoTests = false)
@IncludeEngines("cucumber")
@SelectClasspathResource("....")
@ConfigurationParameter(key = Constants.GLUE_PROPERTY_NAME, value = "...")
@ConfigurationParameter(key = Constants.FILTER_TAGS_PROPERTY_NAME, value = "....")
@ConfigurationParameter(key = Constants.PLUGIN_PROPERTY_NAME, value = "...")
public class CucumberTest {
}

[BegoMaier]

i have same issues with cucumber and archunit

[royteeuwen]

Same issue here, anyone still working on archunit and is able to fix this?

[StefanBauerTT]

@royteeuwen work to fix this is currently happening in https://github.com/apache/maven-surefire/pull/828

[hankem]

@royteeuwen, ArchUnit cannot prevent the Surefire bug. The workaround is to downgrade to Maven Surefire 3.5.2.

[MoellJ]

The underlying issue seems to be fixed in Surefire 3.5.4 (my local test worked again) https://github.com/apache/maven-surefire/issues/2601

[erwanosouf]

Confirmed locally that the bug is fixed. I updated the repository to reflect that and I will archive it.

Closing the issue.