Failing Domain rules test on Azure pipeline only

[cuvaalex]

Hi, I got a really weird issue, on my computer it work perfectly, but on the Azure pipeline it fails, and complain my Domain objects depends on System namespace

The test

  [Fact]
    public void CanOnlyAccessDomainItselfAndExceptions() =>
        DomainTypes()
            .Should()
            .OnlyDependOnTypesThat()
            .ResideInNamespace($"{Namespaces.Domain}|{Namespaces.Exceptions}", true)
            .Check();

The fail log :

[xUnit.net 00:00:01.51]     XXXX.DevSecOps.Lab.HelloWorld.System.Test.ArchitectureRules.DomainRules.CanOnlyAccessDomainItselfAndExceptions [FAIL]
  Failed XXXX.DevSecOps.Lab.HelloWorld.System.Test.ArchitectureRules.DomainRules.CanOnlyAccessDomainItselfAndExceptions [28 ms]
  Error Message:
   ArchUnitNET.xUnit.FailedArchRuleException : "Types that reside in namespace with full name matching "Domain" should only depend on types that reside in namespace with full name matching "Domain|Exceptions|System.Runtime.CompilerServices"" failed:
	XXXX.DevSecOps.Lab.HelloWorld.Core.Domain.HelloWorlds.HelloWorldEntity does depend on System.Object and System.IEquatable`1 and System.Type and System.Void modreq(System.Runtime.CompilerServices.IsExternalInit) and Coverlet.Core.Instrumentation.Tracker.XXXX.DevSecOps.Lab.HelloWorld.Core_73602135-9a35-4cbe-8b48-535f79d08309 and System.String and System.Text.StringBuilder and System.Boolean and System.Int32 and System.Collections.Generic.EqualityComparer`1
	XXXX.DevSecOps.Lab.HelloWorld.Core.Domain.HelloWorlds.HelloWorldId does depend on System.ValueType and System.Int64 and Coverlet.Core.Instrumentation.Tracker.XXXX.DevSecOps.Lab.HelloWorld.Core_73602135-9a35-4cbe-8b48-535f79d08309
	XXXX.DevSecOps.Lab.HelloWorld.Core.Domain.HelloWorlds.HelloWorldName does depend on System.ValueType and System.String and Coverlet.Core.Instrumentation.Tracker.XXXX.DevSecOps.Lab.HelloWorld.Core_73602135-9a35-4cbe-8b48-535f79d08309
	XXXX.DevSecOps.Lab.HelloWorld.Core.Domain.Common.Guards.Guard does depend on System.Object and System.Func`1 and System.String and Coverlet.Core.Instrumentation.Tracker.XXXX.DevSecOps.Lab.HelloWorld.Core_73602135-9a35-4cbe-8b48-535f79d08309 and System.Boolean
	XXXX.DevSecOps.Lab.HelloWorld.Core.Domain.Common.Guards.LongGuardExtensions does depend on System.Object and System.Int64 and System.String and Coverlet.Core.Instrumentation.Tracker.XXXX.DevSecOps.Lab.HelloWorld.Core_73602135-9a35-4cbe-8b48-535f79d08309 and System.Func`1 and System.Boolean
	XXXX.DevSecOps.Lab.HelloWorld.Core.Domain.Common.Guards.StringGuardExtensions does depend on System.Object and System.String and Coverlet.Core.Instrumentation.Tracker.XXXX.DevSecOps.Lab.HelloWorld.Core_73602135-9a35-4cbe-8b48-535f79d08309 and System.Func`1 and System.Boolean


  Stack Trace:
     at ArchUnitNET.xUnit.ArchRuleAssert.CheckRule(Architecture architecture, IArchRule archRule)
   at ArchUnitNET.xUnit.ArchRuleExtensions.Check(IArchRule archRule, Architecture architecture)
   at XXXX.DevSecOps.Lab.HelloWorld.System.Test.ArchitectureRules.ArchUnitExtensions.Check(IArchRule rule) in /home/vsts/work/1/s/test/XXXX.DevSecOps.Lab.HelloWorld.System.Test/ArchitectureRules/ArchUnitExtensions.cs:line 17
   at XXXX.DevSecOps.Lab.HelloWorld.System.Test.ArchitectureRules.DomainRules.CanOnlyAccessDomainItselfAndExceptions() in /home/vsts/work/1/s/test/XXXX.DevSecOps.Lab.HelloWorld.System.Test/ArchitectureRules/DomainRules.cs:line 28

[MSinstein]

Hi @cuvaalex ,

interesting, could you find out what the difference is between local and pipeline run? I would say it is expected that the test fails since your classes actually have other dependencies, right?

[alexanderlinne]

Since we can't reproduce this issue we will close this for now.

@cuvaalex Please feel free to reopen this issue if you can provide a reproducible example for the issue.

[Adam-Coetzee]

Occuring for me too.

[alexanderlinne]

@Adam-Coetzee Can you identify any differences between your runs? Maybe running one in Debug and one Release configuration?

[Adam-Coetzee]

@alexanderlinne I can try get a more reproduceable sample when I get time, apologies I have other priorities.

I tried running with debug configuration in the pipeline and got the same error.

One immediate similarity I can see is that the error complains about Coverlet.Core.Instrumentation.Tracker* So maybe it doesn't play nice when using coverlet etc to track coverage

But I'm not sure and will try experiment a little more later

[Adam-Coetzee]

Update: Can get it working in the pipeline by invoking directly via script, ie

      - bash: dotnet test ...

The usual method for running test in pipeline is via DotNetCoreCLI@2 task.

Not quite sure why the task doesn't work the same. Maybe it's using an old version of the sdk/test runner?

[Adam-Coetzee]

Oh right okay I think I've isolated the difference somewhat, and recreated the issue on my local machine.

Fails when you pass the --collect "Code Coverage" flag when running the tests. So, related to coverage.

Doesn't seem one would care to collect test coverage info when conducting architecture tests. I don't think it's applicable.

So I think the fix here is to just not run the architecture tests with that flag. Maybe break them out into separate projects or steps in the pipeline.

[Adam-Coetzee]

Similarly, if your DotNetCoreCLI@2 task is attempting to collect coverage ie publishTestResults: true with --collect:"Code Coverage" it will fail.

So set publishTestResults: false when invoking with cli task in pipeline and it works

[alexanderlinne]

Hi @Adam-Coetzee,

our own tests for this project use code coverage but with --collect:"XPlat Code Coverage". Could you try whether using this parameter works for you?