testfx TestId no longer constant across test runs

Problem:

Trx specification seems to have changed significantly from 2.2.3 to 2.2.4+. As a result it makes it difficult (impossible?) to track historic results, map child tests to parents, and muddies the water for all UnitTestResults in the Trx file when using DataRow and DynamicData attributes.

In 2.2.3 all DataRow and DynamicData tests would show up as an InnerResult for a parent. The result would contain a testId that mapped to the parent's testId that it is nested under. This made sense because you could see the overall test result by viewing the parent and then look at each individual test result when looking at the child tests. After upgrading to 2.2.4 or later it doesn't seem like any of the consistency that Trx files had are valid anymore.

This has negative side effects depending on the CI/CD platform you use. In our example we use Bamboo which has always displayed parent results. Now that we have upgraded it shows each individual DataRow and DynamicData test result now too. While that might be okay for teams using default tooling that doesn't require any custom plugins or features to maintain a code base these new changes have made it difficult for those of us who need extra feature sets to keep our teams sailing smoothly.

Our team has a monolithic repository. Unit, integration, and functional tests can all be found in our C# test projects that run as part of our continuous integration process. As a result we have had to build tooling to manage and maintain tests. To do this we process Trx files, store them in our own data store, and run analysis on them. Some reasons why are listed below.

Does the test have the traits of a flaky test? If so, quarantine it in Bamboo so developers are not blocked from merging.
Is this test too slow consistently? If so, notify users that there is a reproducible performance issue.
Is this test randomly slow? If so, notify users that there is a hard to find performance issue.
Does this test fail the first time but pass on the second attempt? If so, notify users that their test is unreliable.
Does the test only fail between certain hours of the day? If so, notify the user.

You can see where that is going.

To create such a system we have had to rely on constants between test runs that lets us map results historically. For the last 8 years the test id has always been that constant. You couldn't even open a Trx file in Visual Studio if there were duplicate test ids being used. I haven't tested that in a while though.

If we want to quarantine a test we could quarantine the parent and that would impact the children. Under this Trx structure in 2.2.4+ that is no longer the case. While it would be fine to quarantine based on the ClassName+TestName we are noticing that the ClassName+TestName is not unique across all test results either. Example of that below:

<?xml version="1.0" encoding="utf-8"?>
<TestRun id="8ace4773-a85f-4063-b352-b47056691b99" name="" runUser="" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2022-07-13T20:02:39.7924341-04:00" queuing="2022-07-13T20:02:39.7924341-04:00" start="2022-07-13T20:02:36.5608957-04:00" finish="2022-07-13T20:02:39.8334322-04:00" />
  <TestSettings name="default" id="97324b85-62c7-41b6-9b86-22bf059b57dd">
    <Deployment runDeploymentRoot="" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="45342629-aefa-427e-9ca0-c94b9cf906fe" testId="28a880b7-582f-47a1-a6dd-b2b105ee4be9" testName="DyanmicDataExample_SameSignaturePerTestRun (TrxTesting.EnergyProgram,TrxTesting.Address,True)" computerName="WCJEWETHQ01W05" duration="00:00:00.0000013" startTime="2022-07-13T20:02:39.3124340-04:00" endTime="2022-07-13T20:02:39.6204751-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="45342629-aefa-427e-9ca0-c94b9cf906fe" />
    <UnitTestResult executionId="ef2df86d-58f8-4bc9-940c-fae0153f0128" testId="1c3e44a0-7941-4aef-a4c6-ea3d00b7e7be" testName="DyanmicDataExample_SameSignaturePerTestRun (TrxTesting.EnergyProgram,TrxTesting.Address,False)" computerName="WCJEWETHQ01W05" duration="00:00:00.0000014" startTime="2022-07-13T20:02:39.3124340-04:00" endTime="2022-07-13T20:02:39.6204751-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="ef2df86d-58f8-4bc9-940c-fae0153f0128" />
    <UnitTestResult executionId="7a3aa101-0b49-4b39-8f19-36f8e5b43d04" testId="a8fb6105-cb12-451c-a219-e085bf9a66dd" testName="DyanmicDataExample_SameSignaturePerTestRun (TrxTesting.EnergyProgram,TrxTesting.Address,True)" computerName="WCJEWETHQ01W05" duration="00:00:00.0000125" startTime="2022-07-13T20:02:39.3124340-04:00" endTime="2022-07-13T20:02:39.6204751-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="7a3aa101-0b49-4b39-8f19-36f8e5b43d04" />
    <UnitTestResult executionId="3fb73645-1469-48a0-b24e-488e17b9aed4" testId="1275e361-7c97-4d0d-86a9-504c6d127622" testName="DyanmicDataExample_SameSignaturePerTestRun (TrxTesting.EnergyProgram,TrxTesting.Address,False)" computerName="WCJEWETHQ01W05" duration="00:00:00.0000013" startTime="2022-07-13T20:02:39.3124340-04:00" endTime="2022-07-13T20:02:39.6204751-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="3fb73645-1469-48a0-b24e-488e17b9aed4" />
    <UnitTestResult executionId="b15f150b-7490-41da-9723-fff035ea8788" testId="ab955369-be12-465c-a81e-b0817cc2cdbb" testName="DyanmicDataExample_SameSignaturePerTestRun (,TrxTesting.Address,False)" computerName="WCJEWETHQ01W05" duration="00:00:00.0083030" startTime="2022-07-13T20:02:39.3124340-04:00" endTime="2022-07-13T20:02:39.6204751-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="b15f150b-7490-41da-9723-fff035ea8788" />
    <UnitTestResult executionId="0e470c2e-cffd-4eab-8348-6aa0e87a8317" testId="301a572a-19ae-40a2-abd8-f855835e8d7c" testName="DyanmicDataExample_SameSignaturePerTestRun (TrxTesting.EnergyProgram,TrxTesting.Address,True)" computerName="WCJEWETHQ01W05" duration="00:00:00.0000020" startTime="2022-07-13T20:02:39.3124340-04:00" endTime="2022-07-13T20:02:39.6204751-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="0e470c2e-cffd-4eab-8348-6aa0e87a8317" />
  </Results>
  <TestDefinitions>
    <UnitTest name="DyanmicDataExample_SameSignaturePerTestRun (TrxTesting.EnergyProgram,TrxTesting.Address,False)" storage="c:\users\cjewett\source\trxtesting\bin\debug\net48\trxtesting.dll" id="1275e361-7c97-4d0d-86a9-504c6d127622">
      <Execution id="3fb73645-1469-48a0-b24e-488e17b9aed4" />
      <TestMethod codeBase="C:\Users\cjewett\source\TrxTesting\bin\Debug\net48\TrxTesting.dll" adapterTypeName="executor://mstestadapter/v2" className="TrxTesting.UnitTest1" name="DyanmicDataExample_SameSignaturePerTestRun" />
    </UnitTest>
    <UnitTest name="DyanmicDataExample_SameSignaturePerTestRun (TrxTesting.EnergyProgram,TrxTesting.Address,True)" storage="c:\users\cjewett\source\trxtesting\bin\debug\net48\trxtesting.dll" id="a8fb6105-cb12-451c-a219-e085bf9a66dd">
      <Execution id="7a3aa101-0b49-4b39-8f19-36f8e5b43d04" />
      <TestMethod codeBase="C:\Users\cjewett\source\TrxTesting\bin\Debug\net48\TrxTesting.dll" adapterTypeName="executor://mstestadapter/v2" className="TrxTesting.UnitTest1" name="DyanmicDataExample_SameSignaturePerTestRun" />
    </UnitTest>
    <UnitTest name="DyanmicDataExample_SameSignaturePerTestRun (TrxTesting.EnergyProgram,TrxTesting.Address,True)" storage="c:\users\cjewett\source\trxtesting\bin\debug\net48\trxtesting.dll" id="301a572a-19ae-40a2-abd8-f855835e8d7c">
      <Execution id="0e470c2e-cffd-4eab-8348-6aa0e87a8317" />
      <TestMethod codeBase="C:\Users\cjewett\source\TrxTesting\bin\Debug\net48\TrxTesting.dll" adapterTypeName="executor://mstestadapter/v2" className="TrxTesting.UnitTest1" name="DyanmicDataExample_SameSignaturePerTestRun" />
    </UnitTest>
    <UnitTest name="DyanmicDataExample_SameSignaturePerTestRun (TrxTesting.EnergyProgram,TrxTesting.Address,False)" storage="c:\users\cjewett\source\trxtesting\bin\debug\net48\trxtesting.dll" id="1c3e44a0-7941-4aef-a4c6-ea3d00b7e7be">
      <Execution id="ef2df86d-58f8-4bc9-940c-fae0153f0128" />
      <TestMethod codeBase="C:\Users\cjewett\source\TrxTesting\bin\Debug\net48\TrxTesting.dll" adapterTypeName="executor://mstestadapter/v2" className="TrxTesting.UnitTest1" name="DyanmicDataExample_SameSignaturePerTestRun" />
    </UnitTest>
    <UnitTest name="DyanmicDataExample_SameSignaturePerTestRun (TrxTesting.EnergyProgram,TrxTesting.Address,True)" storage="c:\users\cjewett\source\trxtesting\bin\debug\net48\trxtesting.dll" id="28a880b7-582f-47a1-a6dd-b2b105ee4be9">
      <Execution id="45342629-aefa-427e-9ca0-c94b9cf906fe" />
      <TestMethod codeBase="C:\Users\cjewett\source\TrxTesting\bin\Debug\net48\TrxTesting.dll" adapterTypeName="executor://mstestadapter/v2" className="TrxTesting.UnitTest1" name="DyanmicDataExample_SameSignaturePerTestRun" />
    </UnitTest>
    <UnitTest name="DyanmicDataExample_SameSignaturePerTestRun (,TrxTesting.Address,False)" storage="c:\users\cjewett\source\trxtesting\bin\debug\net48\trxtesting.dll" id="ab955369-be12-465c-a81e-b0817cc2cdbb">
      <Execution id="b15f150b-7490-41da-9723-fff035ea8788" />
      <TestMethod codeBase="C:\Users\cjewett\source\TrxTesting\bin\Debug\net48\TrxTesting.dll" adapterTypeName="executor://mstestadapter/v2" className="TrxTesting.UnitTest1" name="DyanmicDataExample_SameSignaturePerTestRun" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="28a880b7-582f-47a1-a6dd-b2b105ee4be9" executionId="45342629-aefa-427e-9ca0-c94b9cf906fe" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="1c3e44a0-7941-4aef-a4c6-ea3d00b7e7be" executionId="ef2df86d-58f8-4bc9-940c-fae0153f0128" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="a8fb6105-cb12-451c-a219-e085bf9a66dd" executionId="7a3aa101-0b49-4b39-8f19-36f8e5b43d04" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="1275e361-7c97-4d0d-86a9-504c6d127622" executionId="3fb73645-1469-48a0-b24e-488e17b9aed4" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="ab955369-be12-465c-a81e-b0817cc2cdbb" executionId="b15f150b-7490-41da-9723-fff035ea8788" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="301a572a-19ae-40a2-abd8-f855835e8d7c" executionId="0e470c2e-cffd-4eab-8348-6aa0e87a8317" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Completed">
    <Counters total="6" executed="6" passed="6" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
  </ResultSummary>
</TestRun>

Under our current system we store these results as results under the parent and perform our analysis that way. If we detect an issue we tell them the parent is problematic and they should take a look. Upgrading to 2.2.4+ forces us to have to process these child test results as their own parent because that is how the Trx is now structured. The problem is there is no consistent unique id to map results to now.

The TestId changes every test run.

The ClassName+TestName can have multiple tests that are not the exact same test but have the same signature. Therefore you store different test results under a single ClassName+TestName signature which would be incorrect.

The ClassName+TestName is also random at times depending on the type of data being passed in. For example, if you pass in DateTime.UtcNow then the ClassName+TestName (and TestId) signature should change every test run. The saving grace in 2.2.3 or below is that all of these results were stored under a parent, and the parent had a constant TestId.

We can't ignore processing of these DynamicData test results either because the parent test result no longer exists in the Trx file. In addition to that it brings on more trouble of trying to figure out what kind of ClassName+TestName pattern you ignore. Also, if we don't provide support for processing these results then we cannot quarantine them as part of our continuous integration process and that hurts our development process.

Here is the code used to replicate these problems: https://github.com/Cjewett/TrxTestIdIssue/tree/main

All that said it is possible we have missed something that still allows us to track results historically across different test runs, so we had some questions and were hoping for guidance.

Is the TestId really supposed to be changing every test run now?
Is it expected that multiple test results can have the same ClassName+TestName signature?
Is there a way to revert back to the old Trx format using the new library versions?
Does Microsoft consider any of this a bug? In other threads it seems like others have run into issues and it's odd that the Trx data changed in this manner under a minor version bump.

Any guidance and information so we can make a decision on how to move forward is much appreciated. Thanks!

AB#1579034

Jul 14 '22 01:07 Cjewett

Bumping in hopes of some guidance @Haplois @Evangelink

Jul 19 '22 18:07 Cjewett

TestId should stay constant, let me investigate this.

Jul 19 '22 18:07 Haplois

@Haplois any updates on this issue?

Aug 16 '22 16:08 Cjewett

@Haplois Any info regarding this would be much appreciated. We're running into more cases where developers want to upgrade MSTest but we're telling them no because it breaks some of our crucial in-house systems.

Sep 14 '22 15:09 Cjewett

@Cjewett Sorry it's taking us so long in handling your request but I am actively working on it now. In the PR linked, we are introducing some enum that allows to select the strategy used to generate the test ID.

A couple of side notes:

There were some issues on our side that lead to new features and breaking changes to be added in patch versions (2.2.4+).
The test ID change introduced in 2.2.4 relies on Display Name and doesn't take into account data so we have a couple of open issues

Is the TestId really supposed to be changing every test run now?

In all strategies, I don't expect the test ID to change between run as long as the file path remains the same. From my investigations, it seems that the file path is used as part of the ID generation to act as some kind of discriminator in metadata changes between 2 dlls (e.g. Debug and Release build would produce 2 dlls in different locations so we need to consider the tests as different).

Is it expected that multiple test results can have the same ClassName+TestName signature?

I would tend to say yes but it seems that xUnit and NUnit have different strategies here (xUnit seems to be removing duplicates while NUnit keeps them). I am still trying to be sure of what was the intent of MSTest on this one.

Is there a way to revert back to the old Trx format using the new library versions?

Changing the test ID generation strategy would help there BUT you will possibly revert back to broken "state" for some of the parameterized tests.

Does Microsoft consider any of this a bug? In other threads it seems like others have run into issues and it's odd that the Trx data changed in this manner under a minor version bump.

We do consider this as a bug and we are trying to re-engage with community on this repo.

Oct 20 '22 17:10 Evangelink

@Cjewett Would you be open to test out preview version of MSTest v3 to confirm this is correctly solving your issue? From our internal testing, it is ok but it's always better to have more feedback?

Nov 18 '22 17:11 Evangelink

@Evangelink Definitely! I’ll be able to bump one of our CI processes next week to check the changes.

Nov 18 '22 18:11 Cjewett

I will do another preview release early this week as my previous release was containing some issues with nullability.

Thanks for the beta testing @Cjewett, that's highly appreciated.

Nov 21 '22 08:11 Evangelink

I have published the new pre-release: https://github.com/microsoft/testfx/releases/tag/v3.0.0-preview-20221122-01.

Let me know if that's not fixing your issue.

Nov 23 '22 21:11 Evangelink

@Evangelink I don't think this new version solves the issues but some pieces behave quite a bit different than 2.2.3. Mainly the UnitTestResult is formatted differently for DataRow and DynamicData tests, so it's hard for me to say if it is correct or not. Using https://github.com/Cjewett/TrxTestIdIssue as the reproduce project.

With the project set to use version 2.2.3 and using the most simple test case. dotnet test --logger "trx;logfilename=TestOne-2.2.3.trx"

<UnitTestResult executionId="6915bc7e-429f-4ed8-bea7-1044063c31f1" testId="66a8a2e1-7e13-50b5-f377-6682faafeca4" testName="NormalTestExample" computerName="DESKTOP-D66LS15" duration="00:00:00.0050162" startTime="2022-11-28T17:07:03.9965253-05:00" endTime="2022-11-28T17:07:04.2080252-05:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="6915bc7e-429f-4ed8-bea7-1044063c31f1" />

dotnet test --logger "trx;logfilename=TestTwo-2.2.3.trx"

<UnitTestResult executionId="fa21783e-f5ef-4d7b-9de3-8a296d693cb2" testId="66a8a2e1-7e13-50b5-f377-6682faafeca4" testName="NormalTestExample" computerName="DESKTOP-D66LS15" duration="00:00:00.0048603" startTime="2022-11-28T17:07:09.9638557-05:00" endTime="2022-11-28T17:07:10.1708550-05:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="fa21783e-f5ef-4d7b-9de3-8a296d693cb2" />

The two testIds match and that is expected.

Upgrading to 3.0.0-preview-20221122-01 and running against the most simple test case results in something unexpected. dotnet test --logger "trx;logfilename=TestThree-3.0.0-preview-20221122-01.trx"

<UnitTestResult executionId="bd664748-27c0-4476-b0e1-3e60680d8aa9" testId="e4e3aff3-7d05-4b3b-b6cd-7900b4531c14" testName="NormalTestExample" computerName="DESKTOP-D66LS15" duration="00:00:00.0039939" startTime="2022-11-28T17:15:39.6429330-05:00" endTime="2022-11-28T17:15:39.8464346-05:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="bd664748-27c0-4476-b0e1-3e60680d8aa9" />

dotnet test --logger "trx;logfilename=TestFour-3.0.0-preview-20221122-01.trx"

<UnitTestResult executionId="ff306cce-2599-4d51-adbf-06069a087954" testId="ebde346e-7658-40bf-aaf1-605f2289968f" testName="NormalTestExample" computerName="DESKTOP-D66LS15" duration="00:00:00.0042218" startTime="2022-11-28T17:15:48.9120177-05:00" endTime="2022-11-28T17:15:49.1305174-05:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="ff306cce-2599-4d51-adbf-06069a087954" />

Not only does the first run from 3.0.0-preview-20221122-01 not match either of the runs from 2.2.3. The two runs from that same version don't match each other.

Let me know if any more testing is required or if there was a different setup process I should have gone through to make this work.

Thanks for all of the effort here! We really appreciate it.

Nov 28 '22 22:11 Cjewett

Hey thank you so much for the testing! I am able to reproduce the issues you have been raising, sorry I missed this end-to-end test. I tested successfully that what we were returning to vstest was good and stable but I wasn't anticipating that vstest was having specific code messing up with the logic for MSTest...

Also, your test wasn't complete as you are using the new test ID strategy (I enabled it by default). To use old behavior, you need to add [assembly: TestIdGenerationStrategy(TestIdGenerationStrategy.Legacy)] to your test class. At the moment, you will still have inconsistent behavior because of the issue I am mentioning above.

I am investigating how to fix vstest part at the moment and will post my results here.

Nov 29 '22 17:11 Evangelink

I have been able to test and can confirm this is now fixed correctly. Sadly, because there was a specific code in Test Platform that needs to be removed, that means you won't be able to test before vstest 17.5.0 / dotnet 7.0.2xx is released (except if you build the platform locally).

Here are the 2 trx results:

Run1:

<?xml version="1.0" encoding="utf-8"?>
<TestRun id="b28f90fb-39a5-404a-bfce-43f7534ccd04" name="amauryleve@MYMACHINE 2022-12-01 13:24:25" runUser="EUROPE\amauryleve" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2022-12-01T13:24:25.7678149+01:00" queuing="2022-12-01T13:24:25.7678149+01:00" start="2022-12-01T13:24:24.9158694+01:00" finish="2022-12-01T13:24:25.7879181+01:00" />
  <TestSettings name="default" id="5968e201-3098-4d27-9a36-7dc572b80027">
    <Deployment runDeploymentRoot="amauryleve_MYMACHINE_2022-12-01_13_24_25" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="0e1a11ee-7bf9-4ac2-a0e8-9c9430f5009e" testId="66a8a2e1-7e13-50b5-f377-6682faafeca4" testName="NormalTestExample" computerName="MYMACHINE" duration="00:00:00.0039105" startTime="2022-12-01T13:24:25.4999797+01:00" endTime="2022-12-01T13:24:25.6726574+01:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="0e1a11ee-7bf9-4ac2-a0e8-9c9430f5009e" />
    <UnitTestResult executionId="e70b3308-8bfc-40f6-98d1-6f36e2e00855" testId="62fe2f89-a518-d72c-f0ed-b325c2641eba" testName="DataTestMethodExample" computerName="MYMACHINE" duration="00:00:00.0250550" startTime="2022-12-01T13:24:25.6792738+01:00" endTime="2022-12-01T13:24:25.7082070+01:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="e70b3308-8bfc-40f6-98d1-6f36e2e00855" resultType="DataDrivenTest">
      <InnerResults>
        <UnitTestResult executionId="e4b09fe4-ce24-4426-bce5-9822a5b00ef0" parentExecutionId="e70b3308-8bfc-40f6-98d1-6f36e2e00855" testId="62fe2f89-a518-d72c-f0ed-b325c2641eba" testName="DataTestMethodExample (1)" computerName="MYMACHINE" duration="00:00:00.0000470" startTime="2022-12-01T13:24:25.6792738+01:00" endTime="2022-12-01T13:24:25.7082070+01:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="e4b09fe4-ce24-4426-bce5-9822a5b00ef0" dataRowInfo="1" resultType="DataDrivenDataRow" />
        <UnitTestResult executionId="0087d73a-9400-4ec7-8c0a-42be299daf51" parentExecutionId="e70b3308-8bfc-40f6-98d1-6f36e2e00855" testId="62fe2f89-a518-d72c-f0ed-b325c2641eba" testName="DataTestMethodExample (2)" computerName="MYMACHINE" duration="00:00:00.0000060" startTime="2022-12-01T13:24:25.6792738+01:00" endTime="2022-12-01T13:24:25.7082070+01:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="0087d73a-9400-4ec7-8c0a-42be299daf51" dataRowInfo="2" resultType="DataDrivenDataRow" />
      </InnerResults>
    </UnitTestResult>
  </Results>
  <TestDefinitions>
    <UnitTest name="NormalTestExample" storage="c:\src\temp\trxtestidissue\trxtestidissue\bin\debug\net48\trxtesting.dll" id="66a8a2e1-7e13-50b5-f377-6682faafeca4">
      <Execution id="0e1a11ee-7bf9-4ac2-a0e8-9c9430f5009e" />
      <TestMethod codeBase="C:\src\temp\TrxTestIdIssue\TrxTestIdIssue\bin\Debug\net48\TrxTesting.dll" adapterTypeName="executor://mstestadapter/v2" className="TrxTesting.UnitTest1" name="NormalTestExample" />
    </UnitTest>
    <UnitTest name="DataTestMethodExample" storage="c:\src\temp\trxtestidissue\trxtestidissue\bin\debug\net48\trxtesting.dll" id="62fe2f89-a518-d72c-f0ed-b325c2641eba">
      <Execution id="e70b3308-8bfc-40f6-98d1-6f36e2e00855" />
      <TestMethod codeBase="C:\src\temp\TrxTestIdIssue\TrxTestIdIssue\bin\Debug\net48\TrxTesting.dll" adapterTypeName="executor://mstestadapter/v2" className="TrxTesting.UnitTest1" name="DataTestMethodExample" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="66a8a2e1-7e13-50b5-f377-6682faafeca4" executionId="0e1a11ee-7bf9-4ac2-a0e8-9c9430f5009e" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="62fe2f89-a518-d72c-f0ed-b325c2641eba" executionId="e70b3308-8bfc-40f6-98d1-6f36e2e00855" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Completed">
    <Counters total="4" executed="4" passed="4" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
  </ResultSummary>
</TestRun>

Run2:

<?xml version="1.0" encoding="utf-8"?>
<TestRun id="d5b3083c-dff7-436b-8a75-4c24d135d53e" name="amauryleve@MYMACHINE 2022-12-01 13:25:00" runUser="EUROPE\amauryleve" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2022-12-01T13:25:00.3069508+01:00" queuing="2022-12-01T13:25:00.3069508+01:00" start="2022-12-01T13:24:59.4346647+01:00" finish="2022-12-01T13:25:00.3264633+01:00" />
  <TestSettings name="default" id="deacd315-2d8a-48a3-a4c7-64ddb0f5e289">
    <Deployment runDeploymentRoot="amauryleve_MYMACHINE_2022-12-01_13_25_00" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="9c0d57a8-d2f0-41f3-8f1c-e2078e111123" testId="66a8a2e1-7e13-50b5-f377-6682faafeca4" testName="NormalTestExample" computerName="MYMACHINE" duration="00:00:00.0039350" startTime="2022-12-01T13:25:00.0375473+01:00" endTime="2022-12-01T13:25:00.2111216+01:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="9c0d57a8-d2f0-41f3-8f1c-e2078e111123" />
    <UnitTestResult executionId="958fdfed-bceb-4bdc-88ba-5c1176640b65" testId="62fe2f89-a518-d72c-f0ed-b325c2641eba" testName="DataTestMethodExample" computerName="MYMACHINE" duration="00:00:00.0251261" startTime="2022-12-01T13:25:00.2171551+01:00" endTime="2022-12-01T13:25:00.2464614+01:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="958fdfed-bceb-4bdc-88ba-5c1176640b65" resultType="DataDrivenTest">
      <InnerResults>
        <UnitTestResult executionId="72d27423-5bbd-4752-99ad-a7382c03f091" parentExecutionId="958fdfed-bceb-4bdc-88ba-5c1176640b65" testId="62fe2f89-a518-d72c-f0ed-b325c2641eba" testName="DataTestMethodExample (1)" computerName="MYMACHINE" duration="00:00:00.0000486" startTime="2022-12-01T13:25:00.2171551+01:00" endTime="2022-12-01T13:25:00.2464614+01:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="72d27423-5bbd-4752-99ad-a7382c03f091" dataRowInfo="1" resultType="DataDrivenDataRow" />
        <UnitTestResult executionId="e98daa0f-37e0-4780-927e-87535008412e" parentExecutionId="958fdfed-bceb-4bdc-88ba-5c1176640b65" testId="62fe2f89-a518-d72c-f0ed-b325c2641eba" testName="DataTestMethodExample (2)" computerName="MYMACHINE" duration="00:00:00.0000059" startTime="2022-12-01T13:25:00.2171551+01:00" endTime="2022-12-01T13:25:00.2464614+01:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="e98daa0f-37e0-4780-927e-87535008412e" dataRowInfo="2" resultType="DataDrivenDataRow" />
      </InnerResults>
    </UnitTestResult>
  </Results>
  <TestDefinitions>
    <UnitTest name="NormalTestExample" storage="c:\src\temp\trxtestidissue\trxtestidissue\bin\debug\net48\trxtesting.dll" id="66a8a2e1-7e13-50b5-f377-6682faafeca4">
      <Execution id="9c0d57a8-d2f0-41f3-8f1c-e2078e111123" />
      <TestMethod codeBase="C:\src\temp\TrxTestIdIssue\TrxTestIdIssue\bin\Debug\net48\TrxTesting.dll" adapterTypeName="executor://mstestadapter/v2" className="TrxTesting.UnitTest1" name="NormalTestExample" />
    </UnitTest>
    <UnitTest name="DataTestMethodExample" storage="c:\src\temp\trxtestidissue\trxtestidissue\bin\debug\net48\trxtesting.dll" id="62fe2f89-a518-d72c-f0ed-b325c2641eba">
      <Execution id="958fdfed-bceb-4bdc-88ba-5c1176640b65" />
      <TestMethod codeBase="C:\src\temp\TrxTestIdIssue\TrxTestIdIssue\bin\Debug\net48\TrxTesting.dll" adapterTypeName="executor://mstestadapter/v2" className="TrxTesting.UnitTest1" name="DataTestMethodExample" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="66a8a2e1-7e13-50b5-f377-6682faafeca4" executionId="9c0d57a8-d2f0-41f3-8f1c-e2078e111123" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="62fe2f89-a518-d72c-f0ed-b325c2641eba" executionId="958fdfed-bceb-4bdc-88ba-5c1176640b65" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Completed">
    <Counters total="4" executed="4" passed="4" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
  </ResultSummary>
</TestRun>

Dec 01 '22 13:12 Evangelink

Marking as closed.

Dec 01 '22 13:12 Evangelink

@Evangelink Thanks for the update and all the work here! We'll keep an eye out and test as soon as they are released or previews become available.

Dec 02 '22 00:12 Cjewett