ArchUnit performance is too slow to run per maven build

[royteeuwen]

I am trying to integrate ArchUnit in our build, where it is ran on every compile as a unit test. This seems to be too slow though, if I look at the current performance. Is there a way to speed things up? For your reference, the project only contains 120 java classes so far, so nothing too big

[INFO] Running architecture.ArchitectureRulesTest
[INFO] Running architecture.rules.General
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 87.43 s -- in architecture.rules.General
[INFO] Running architecture.rules.OSGiConfigs
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 s -- in architecture.rules.OSGiConfigs
[INFO] Running architecture.rules.Services
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.010 s -- in architecture.rules.Services
[INFO] Running architecture.rules.Servlets
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 s -- in architecture.rules.Servlets
[INFO] Running architecture.rules.SlingModels
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 s -- in architecture.rules.SlingModels
[INFO] Running architecture.rules.Utils
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s -- in architecture.rules.Utils
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 87.46 s -- in architecture.ArchitectureRulesTest

I am using the following setup:

@AnalyzeClasses(
        packages = {Constants.PROJECT_ROOT_PACKAGE},
        importOptions = {
                ImportOption.DoNotIncludeTests.class,
                ImportOption.DoNotIncludeJars.class,
                ImportOption.DoNotIncludeArchives.class
        }
)
public class ArchitectureRulesTest {

    @ArchTest
    static final ArchTests generalRules = ArchTests.in(General.class);

    @ArchTest
    static final ArchTests osgiConfigs = ArchTests.in(OSGiConfigs.class);

    @ArchTest
    static final ArchTests services = ArchTests.in(Services.class);

    @ArchTest
    static final ArchTests servlets = ArchTests.in(Servlets.class);

    @ArchTest
    static final ArchTests slingModes = ArchTests.in(SlingModels.class);

    @ArchTest
    static final ArchTests utils = ArchTests.in(Utils.class);

}

[hankem]

The architecture.rules.OSGiConfigs / Services / Servlets / SlingModels / Utils run in ≤ 11 ms each, which seems plausible (and IMO quite acceptable 🙂).

General is the only exception with almost 1.5 min. If you can share its code, we can have a look what's wrong in there.

[royteeuwen]

Hey,

I also thought that, but it’s actually just the first test to run, caching the classes for subsequent tests. If I comment out every test from General, the next class is the slow one taking also around 90secs

if you still think its usefull, i can see to make an example project ? Cant give source code for this one sadly enough

[codecholeric]

Sorry, I just saw that you already told your project has only 120 classes :see_no_evil: So then 90 sec is definitely way too long and I think you just pull in a lot of classes somehow. If you can create an example project that reproduces this, sure, go for it! Other than that, ArchUnit uses the SLF4J API for logging, so you could also hook in a logger and set the level of the ClassFileImporter to TRACE to see a list of all imported classes. There you could see how many you're importing. Or you just use a

@ArchTest
static void examine(JavaClasses classes) {
  System.out.println("Imported " + classes.size() + " classes");
}

dummy test. If the size of this collection should really just be 120 we can see if maybe a tweak to the automatic resolution behavior could speed things up.

[royteeuwen]

I put all other tests in comment, only keeping the examine. As result I get, in 80 seconds

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 80.10 s <<< FAILURE! -- in architecture.ArchitectureRulesTest
[ERROR] architecture.ArchitectureRulesTest.examine -- Time elapsed: 80.09 s <<< FAILURE!
org.opentest4j.AssertionFailedError: Imported 132 classes ==> expected: <120> but was: <132>

So it's a bit more but not a lot.

If I now use the resolveMissingDependenciesFromClassPath=false in archunit.properties. The time goes to:

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.411 s <<< FAILURE! -- in architecture.ArchitectureRulesTest
[ERROR] architecture.ArchitectureRulesTest.examine -- Time elapsed: 1.407 s <<< FAILURE!
org.opentest4j.AssertionFailedError: Imported 132 classes ==> expected: <120> but was: <132>

Any downside on doing it this way? What could be causing this issue? My code compiles, so all dependencies are on the classpath I would think?

[codecholeric]

Downside could be that you have no/wrong information about classes outside of your project. I.e. the classes that are not directly imported will be stubs, they can miss annotations, superclasses or report a wrong isInterface() property... But I would be really interested how you can pull in so many dependencies transitively that the time goes up so crazy 🤔 You could also instead configure the number of resolution iterations and e.g. set the -1 values (i.e. follow up indefinitely) to something like 1 and see what happens 🤔