Console output from a child AppDomain is silently swallowed

[benjamin-hodgson]

Description

When a test method creates an appdomain, console messages from within that appdomain are neither captured by MSTest nor outputted to the console.

Steps to reproduce

MsTestAppDomainExample.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net472</TargetFramework>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.1" />
    <PackageReference Include="MSTest.TestAdapter" Version="2.2.10" />
    <PackageReference Include="MSTest.TestFramework" Version="2.2.10" />
  </ItemGroup>

</Project>

ExampleTestClass.cs:

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace MsTestAppDomainExample
{
    [TestClass]
    public class ExampleTestClass
    {
        [TestMethod]
        public void ExampleTestMethod()
        {
            var setup = new AppDomainSetup
            {
                ApplicationBase = AppDomain.CurrentDomain.BaseDirectory
            };
            var domain = AppDomain.CreateDomain("MyDomain", AppDomain.CurrentDomain.Evidence, setup);
            var x = (ClassInAppDomain)domain.CreateInstanceAndUnwrap("MsTestAppDomainExample", "MsTestAppDomainExample.ClassInAppDomain");

            var result = x.SayHello();

            Console.WriteLine($"hello from {AppDomain.CurrentDomain.FriendlyName}");
            Assert.AreEqual(123, result);
        }
    }

    public class ClassInAppDomain : MarshalByRefObject
    {
        public int SayHello()
        {
            Console.WriteLine($"hello from {AppDomain.CurrentDomain.FriendlyName}");
            return 123;
        }
    }
}

Run the test using dotnet test --logger "console;verbosity=detailed".

Expected behavior

The detailed test log shown on the console should contain both Console.WriteLine messages. It should say something like:

  Standard Output Messages:
 hello from MyDomain
 hello from TestSourceHost: Enumerating source (...\bin\Debug\net472\MsTestAppDomainExample.dll)

At the very least, I'd like to be able to see the output from the appdomain on the console, even if it's not captured by MSTest.

Actual behavior

hello from MyDomain is missing from the test log. It says only:

  Standard Output Messages:
 hello from TestSourceHost: Enumerating source (...\bin\Debug\net472\MsTestAppDomainExample.dll)

hello from MyDomain is not shown directly on the console either.

Environment

• Operating system: Windows 11
• Target Framework: net472
• Build version of vstest.console: Microsoft (R) Test Execution Command Line Tool Version 17.3.0 (x64)
• Package version of MSTest framework and adapter: 2.2.10 for both (latest at time of writing)
• Other installed packages and their versions on the test project: Microsoft.NET.Test.Sdk version 17.3.1 (latest at time of writing)

[Evangelink]

Hi @benjamin-hodgson,

Thanks for reporting this issue. We will look into it as soon as possible.

[Evangelink]

I confirm the behavior. Given how we are capturing logs, I am not sure how we could extend this to support your scenario but I will try to think of something.

[testplatform-bot]

✅ Successfully linked to Azure Boards work item(s):

• User Story 1950771

[MarcoRossignoli]

@benjamin-hodgson we did some investigation and the issue here is that MSTest is capturing the console output and trace using "static" runtime objects like Console.Out and Trace.Listeners so as soon as you create your appdomain these objects are re-initialized and we cannot "proxy" them in any way transparently. One possible solution would be add some support like some class that users can use inside their new domain to plug with our "capture system" but this is not transparent and would not work in all use cases (where the app domain created is in production code for instance).

Due to the above explanation and the complexity and possible 'incomplete' solution we decided to not act on it for the moment, looks like a corner case scenario not so common.

We will close as 'by design' limit for the moment.

Thanks a lot for your patience and the provided repro.

cc: @Evangelink @nohwnd