vscode-dotnet-test-explorer icon indicating copy to clipboard operation
vscode-dotnet-test-explorer copied to clipboard

No tests found, even though everything seems fine

Open qidydl opened this issue 5 years ago • 15 comments

The test explorer doesn't seem to be able to locate any tests, even though there are no errors reported and everything works on the command line. When I try to refresh the list of tests, I see the following in the debug log:

Finding projects for pattern c:/Users\<user>\source\Library-Test/**/*.Tests.csproj
Found 1 matches for pattern in folder c:\Users\<user>\source\Library-Test
Evaluating match c:/Users/<user>/source/Library-Test/test/Library.Tests/Library.Tests.csproj
Adding directory c:/Users/<user>/source/Library-Test/test/Library.Tests
Executing dotnet test -t -v=q in c:/Users/<user>/source/Library-Test/test/Library.Tests

If I run dotnet test -t -v=q in that folder, it works fine and I get a list of tests back:

Test run for C:\Users\<user>\source\Library-Test\test\Library.Tests\bin\Debug\net5.0\Library.Tests.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 16.8.3
Copyright (c) Microsoft Corporation.  All rights reserved.

The following Tests are available:
    TestOne
    TestTwo
    TestThree

Looking at the code in testDiscovery.ts, I then tried running dotnet vstest C:\Users\<user>\source\Library-Test\test\Library.Tests\bin\Debug\net5.0\Library.Tests.dll /ListFullyQualifiedTests /ListTestsTargetPath:"output.txt" and I get a good list in output.txt, so I'm not sure why this extension can't display them. Everything works fine in Visual Studio 2019; I'm trying to use Visual Studio Code instead so I can get code coverage feedback. Are there any special criteria this extension uses to identify tests? Is there some way to debug what's going on?

SDK version: 5.0.103 Unit test framework: NUnit 3.13.1 All projects are targeting net5.0

qidydl avatar Feb 24 '21 15:02 qidydl

This bug aways happens to me. If hit play the tests apear executed.

jcbritobr avatar Mar 11 '21 00:03 jcbritobr

I also have issues with finding tests in my project, but in my case it looks to be linked to processing dotnet command output. I use linux with russian localization, and when running dotnet from console it discover tests normally, but not when using the vscode extension. exporting LC_ALL=C enviroment varialbe and starting VS Code with that fixes that, but it would be nice to get this working out of box. Btw, i also tried to exporting DOTNET_CLI_UI_LANGUAGE and this did not have any effect.

nrader95 avatar Apr 22 '21 17:04 nrader95

Problem occurs in vscode-dotnet-test-explorer/src/testDiscovery.ts. In function extractAssemblyPaths: const testRunLineRegex = /^Test run for (.+\.dll)/gm; String "Test run for..." is invalid for others languages :-(

Probably extractAssemblyPaths should be changed. It's not exacly good solution, but (on Linux) you can add enviroment variable LC_ALL=C in function executeDotnetTest: const command = 'LC_ALL=C dotnet test -t -v=q${dotnetTestOptions}';

dkja avatar May 01 '21 10:05 dkja

That's certainly a problem, but in my case I'm using English and it still didn't work, so there's something else going wrong as well.

qidydl avatar May 01 '21 13:05 qidydl

I just recently started having this same issue. No issues for a month. Not sure what changed. I can run the tests from the command line. Using the play button trick as suggested by @jcbritobr gets the tests to appear in the list. Then clicking the refresh button displays the 'no tests found'.

jsk01 avatar Sep 29 '21 12:09 jsk01

I'm also have the same issue. Looks like the process that tries to discover tests is running dotnet test -t -v=q and then trying to process the output. The process is expecting the output to contain name of the test including the full namespace, probably so that it also knows how to build the visual tree, but this command does not output that. Short of creating a new way to scan for tests, or the dotnet cli outputting the namespace of the tests, this may not be immediately addressable. Which sucks because it's definitely a hindrance when using VSCode.

WereDev avatar Oct 06 '21 19:10 WereDev

Did some more digging... XUnit does output the full name of each test, eg namespace.classname.methodname NUnit and MSTest do not, they just output methodname. It appears this is controlled by their respective test adapter libraries: xunit.runner.visualstudio, NUnit3TestAdapter, & MSTest.TestAdapter. Updates there would allow test discovery to work again within this extension.

WereDev avatar Oct 06 '21 20:10 WereDev

There's another extension that supports unit testing that seems to have sorted out how to find unit tests for nUnit and MSTest: derivitec-ltd.vscode-dotnet-adapter. This looks like it's using the dotnet vstest instead of dotnet test parameters as another developer above had suggested.

WereDev avatar Oct 07 '21 21:10 WereDev

First of, this extension should support discovering mstest and nunit tests by utilizing the dotnet vstest method mentioned above.

In a recent update we did an update to support the cli not running in English which might have had unintended side effects. It order to help it would be good if you could provide more information, such as log (https://github.com/formulahendry/vscode-dotnet-test-explorer#logging) of the discovery process and a simple solution that reproduces the error for you.

stefanforsberg avatar Oct 09 '21 05:10 stefanforsberg

Hey @WereDev, I had the same issue on our project, and bumping NUnit3TestAdapter version from 3.17.0 to the 4.0.0 solved the issue for me.

sbellone avatar Oct 19 '21 17:10 sbellone

@sbellone Thanks, but I'm on 4.0.0. The only time it discovers tests is when I tell it to run all of them.

@stefanforsberg When clicking on the Refresh for it to discover tests, this appears in the logs but the test reports no tests found:

Finding projects for pattern /home/weredev/Code/weredev-com/**/*.Tests.csproj
Found 2 matches for pattern in folder /home/weredev/Code/weredev-com
Evaluating match /home/weredev/Code/weredev-com/Weredev.Providers.Flickr.Tests/Weredev.Providers.Flickr.Tests.csproj
Adding directory /home/weredev/Code/weredev-com/Weredev.Providers.Flickr.Tests
Evaluating match /home/weredev/Code/weredev-com/Weredev.Providers.SourceControl.Tests/Weredev.Providers.SourceControl.Tests.csproj
Adding directory /home/weredev/Code/weredev-com/Weredev.Providers.SourceControl.Tests
Executing dotnet test -t -v=q in /home/weredev/Code/weredev-com/Weredev.Providers.Flickr.Tests

When click on Run All Tests, I get this:

Test run for 
Executing dotnet build in /home/weredev/Code/weredev-com/Weredev.Providers.Flickr.Tests
Executing dotnet test --no-build --logger "trx;LogFileName=/tmp/test-explorer-XNQJHA/0.trx" in /home/weredev/Code/weredev-com/Weredev.Providers.Flickr.Tests
Process 83522 started
Process 83522 finished
Executing dotnet build in /home/weredev/Code/weredev-com/Weredev.Providers.SourceControl.Tests
Executing dotnet test --no-build --logger "trx;LogFileName=/tmp/test-explorer-XNQJHA/1.trx" in /home/weredev/Code/weredev-com/Weredev.Providers.SourceControl.Tests
Process 83685 started
Process 83685 finished

Using:

  • dotnet: 3.1.118
  • target framework: netcoreapp3.1
  • language version: 8.0
  • OS: Fedora 34

WereDev avatar Oct 24 '21 15:10 WereDev

I just checked the tread I had going on the NUnit GitHub (https://github.com/nunit/nunit3-vs-adapter/issues/896) and they provided this parameter: -- NUnit.DisplayName=FullName. Adding that to the test arguments in settings fixed the issue for me with NUnit.

WereDev avatar Oct 24 '21 15:10 WereDev

Nevermind, I spoke too soon. Adding that parameter does indeed allow the tests to be discovered using "Refresh", but breaks actually running the tests. After Refresh, trying to run an individual test fails to retrieve the results, and clicking Run All Tests now shows no tests found.

WereDev avatar Oct 24 '21 15:10 WereDev

Ok, I tweaked my local copy of the extension to allow for different arguments between discovery and execution of the tests. I created a PR (https://github.com/formulahendry/vscode-dotnet-test-explorer/pull/340) in case you want to incorporate that change as well.

With this change, I can successfully get results from Refresh (aka discovery) by including -- NUnit.DisplayName=FullName but it won't be on execution so it doesn't break there. As noted by @sbellone, I did have to update NUnit3TestAdapter to 4.0.0 to get the full names to display at all so I made that change in the test solution as well.

WereDev avatar Oct 25 '21 23:10 WereDev

I had a similar problem, but in my case it was something completely different. To debug the issue I had no other idea than to "hack" the extension itself to give better error output:

Error output gets printed at line 46 of testDiscovery.js inside the the out/src folder of the extension. But if the actual error message thrown gets discarded.

So I added the error message to the output like so:

logger_1.Logger.LogError(`Error while executing ${command}: ${err}`, stdout);

(the ${err} was added by me)

This now showed me that the actual error was caused by windows not recognizing the chcp command because of some broken env-variables pointing to system32. Added them back to the system, and everything worked fine.

So, maybe doing this hack helps some of you better debug the actual error.

LordSyd avatar Feb 20 '23 07:02 LordSyd