dotnet
Convert tests under GC subtree to the merged test model
Tagging subscribers to this area: @dotnet/area-system-reflection-metadata See info in area-owners.md if you want to be subscribed.
Issue Details
null
| Author: | trylek |
|---|---|
| Assignees: | trylek |
| Labels: |
|
| Milestone: | - |
![azure-pipelines[bot] avatar](https://avatars.githubusercontent.com/in/9426?v=4 "Published by a pub.dev verified publisher"){kind=small}
When you're ready, you'll probably want to run gcstress because these are the kinds of tests that seem to get worse in that mode. (However, many are GCStressIncompatible and/or RequiresProcessIsolation, so there is less impact that there would be otherwise.)
GC\Stress\Framework loads the tests from GC\Stress\Tests in ways that I don't fully understand yet, but I suspect there may be some dependencies on the shapes of those tests.
[This is probably beyond the scope of this PR, but if you have problems in this area, perhaps it is relevant.]
The projects in GC\Scenarios\GCSimulator seem strange to me. I think all of them except for the main GCSimulator.csproj could be RunOnly and then maybe not need ReqProcIso, RefXUintWrapGen, AllowUnsafeBlocks, and ItemGroup Compile. Removing over 400 tests from the build seems like it would be noticeable time win too. (Also a Directory.Build.props would simplify this a lot too.)
I can contribute to this too - just let me know how you'd like to coordinate with this PR.
Thanks Mark for your valuable pointers and insights, running GC stress tests is certainly a good idea. For additional cleanups, I agree that the GCSimulator tests are basically 400 calls to the same test with different command-line arguments, my only thinking is that I'd love to first focus on removing all the legacy tests as that will let us get rid of the old variants in the scripts. There are tons of things we can clean up in the tests, in this PR I did some tiny formatting cleanups like converting tabs to spaces and fixing broken alignment here and there but naturally I didn't want to combine this already big change with non-trivial test refactoring. Once the final conversion PRs have been merged in, all subsequent cleanups are up for grabs.
Tagging subscribers to this area: @dotnet/gc See info in area-owners.md if you want to be subscribed.
Issue Details
null
| Author: | trylek |
|---|---|
| Assignees: | trylek |
| Labels: |
|
| Milestone: | - |
@Maoni0 - In my work on this PR I noticed a number of GC tests that actually never run because they are marked as BuildOnly, in particular:
GC\Stress\Tests\573277.csproj GC\Stress\Tests\b115557.csproj GC\Stress\Tests\DictionaryGrowth.csproj GC\Stress\Tests\DirectedGraph.csproj GC\Stress\Tests\ExpandHeap.csproj GC\Stress\Tests\LargeObjectAlloc1.csproj GC\Stress\Tests\LargeObjectAlloc2.csproj GC\Stress\Tests\LargeObjectAlloc3.csproj GC\Stress\Tests\LargeObjectAlloc4.csproj GC\Stress\Tests\LargeObjectAllocPinned.csproj GC\Stress\Tests\MulDimJagAry.csproj GC\Stress\Tests\PlugGaps.csproj
and all tests under GC\Performance\Tests. The merged tests system is capable of representing that but I'm wondering - is it intentional or have these just somehow fallen through the cracks and should actually run? If the latter is true, I can easily prepare a preparatory PR switching them to run before continuing my work on this change.
Thanks
Tomas
In my work on this PR I noticed a number of GC tests that actually never run because they are marked as BuildOnly
These files are loaded by the code in GC\Stress\Framework. The csproj in there has some custom code ("CompileTests") and then the runtime behavior is controlled by files like GC\Stress\*.config
Thanks Mark for clarifying, I'll update the PR as appropriate (basically by reverting any changes to these tests).
pasting test link here - https://dev.azure.com/dnceng-public/public/_build/results?buildId=789129&view=results
New GC-simulator build link: https://dev.azure.com/dnceng-public/public/_build/results?buildId=789161&view=results
Thank you very much @trylek @jkoritzinsky for your work and answering all of my questions about this!
From those discussions:
- GC team testing scenarios should continue to work
- Test list was validated before-vs-after
- All running tests have RPI (some already aren't running, some are part of RF, etc.)
- RF should work
- GCSimulator is currently being tested
- Lots of good discussion about how to improve GCSimulator (or GCPerfSim) in the future - will be easier after old test runner is removed
@markples it looks like GCSimulator is working now. Also, it's surfacing failures a little more cleanly (explicit scenario failures are actually propagating back from Helix to AzDO and GitHub, which is nice).
@markples it looks like GCSimulator is working now. Also, it's surfacing failures a little more cleanly (explicit scenario failures are actually propagating back from Helix to AzDO and GitHub, which is nice).
That is great; thanks!
I've validated that the nativeaot timeouts aren't related. Lets merge this in and get one step closer to removing the legacy test system!