Open Test from Test Explorer Does Nothing

[dcdraper]

Visual Studio - Can no longer open test (F12) from Test Explorer, attempts do nothing

SpecFlow Version:

• [x] 3.0
• [ ] 2.4
• [ ] 2.3
• [ ] 2.2
• [ ] 2.1
• [ ] 2.0
• [ ] 1.9

Used Test Runner

• [ ] SpecFlow+Runner
• [ ] MSTest
• [x] NUnit
• [ ] Xunit

Version number:

Visual Studio Version

• [x] VS 2017
• [ ] VS 2015
• [ ] VS 2013

Are the latest Visual Studio updates installed?

• [x] Yes
• [ ] No

.NET Framework:

• [x] >= .NET 4.5
• [ ] before .NET 4.5
• [ ] .NET Core 2.0
• [ ] .NET Core 2.1
• [ ] .NET Core 2.2
• [ ] .NET Core 3.0

Test Execution Method:

• [x] Visual Studio Test Explorer
• [ ] TFS/VSTS/Azure DevOps – Task – PLEASE SPECIFY THE NAME OF THE TASK
• [ ] Command line – PLEASE SPECIFY THE FULL COMMAND LINE

<SpecFlow> Section in app.config

<specFlow>
    
    <stepAssemblies>
      <!-- This attribute is required in order to use StepArgument Transformation as described here; 
    https://github.com/marcusoftnet/SpecFlow.Assist.Dynamic/wiki/Step-argument-transformations  -->
      <stepAssembly assembly="SpecFlow.Assist.Dynamic" />
    </stepAssemblies>
  </specFlow>

Repro Project

Issue Description

Steps to Reproduce

1. Right-click a test in Test Explorer, choose "Open Test"
2. Or - Select a test in Test Explorer and press F12
3. Observe that nothing happens Expected: The feature file will open and the selected test will be in focus, like it was in SpecFlow2.x

[dcdraper]

This is a productivity killer for sure, since we are working with numerous feature files and test order does not match since they get automatically sorted in Test Explorer. I had to revert to 2.x until this gets fixed :(

[nuzolx]

I have upgrade today, same configuration as you and for me it works.

[dcdraper]

@nuzolx good news for you :) I will try again another day, too much time spent on this for me.

[ghost]

Have the same bug!

[SabotageAndi]

Please provide an example to reproduce this issue.

[ghost]

After update to newest speclfow my Test Explorer window in VS 2017 doesn't show any tests. Vs 2017 The newest Specflow for VS 2017 extension NUnit 3.11 NUnit.Console.Runner 3.10 NUnit3TestAdapter 3.13 Specflow 3.0.188 Specflow.Tools.MsBuild.Generation 3.0.188 Specflow.NUnit 3.0.188

I did no changes after update specflow except removed config about nunit runner from app.config. My project is in .net framework 4.7.2

Problem fixed after move this lines at the bottom of csproj file:

    <Target Name="AfterUpdateFeatureFilesInProject">
    <!-- include any files that specflow generated into the compilation of the project, if it was not included yet -->
    <ItemGroup>
      <Compile Include="**\*.feature.cs" Exclude="@(Compile)" />
    </ItemGroup>
  </Target>

[rik-smeets]

I experience this issue too. It seems to be caused by the new way in which the generated code is not longer included in the .csproj. If you click on the test, you will see 'Source: no source available'.

Previously, the generated code was part of the .csproj like this:

    <Compile Include="Test.feature.cs">
      <AutoGen>True</AutoGen>
      <DesignTime>True</DesignTime>
      <DependentUpon>Test.feature</DependentUpon>
    </Compile>

If you keep this code in the projectfile for your feature files after migrating, opening the test from the test explorer will still work. A source is then available too.

Another option is: simply include the generated file as part of your solution. However, this will bloat the view in the Solution Explorer by having both the .feature file and the .feature.cs at the main level (while the previously generated files were a level below the .feature file). This solution looks like this in the .csproj:

<Compile Include="Test.feature.cs" />
<None Include="Test.feature" />

Both solutions require manual labour in the .csproj file, which is a step back from the previous versions of SpecFlow.

Note: when simply adding a .feature file and removing the SpecFlowSingleFileGenerator from the properties, the new test isn't even visible in the test explorer until doing either one of the aforementioned solutions, or unloading and reloading the project.

[SabotageAndi]

I repeat, I need a project to reproduce this issue. Complete to clone from where, no step for step description.

Jumping from the test explorer to the feature file works in the SpecFlow solution. There we are using xUnit.

So please provide us one. Thanks!

[dcdraper]

@SabotageAndi I will try to create a test project when I get a chance to do so :-( Sorry I cannot do that right now, since I really do not want to move away from SpecFlow. I am using Nunit, as indicated in my initial report.

[marcel-landman]

@rik-smeets work around worked for me (using mstest.testframework). Some additional notes:

• <Compile Include Should be in an unconditional <ItemGroup> under <Project>
• I had to delete this section created following the "msbuild" instructions for upgrading to version 3:

  <Target Name="AfterUpdateFeatureFilesInProject">
    <ItemGroup>
      <Compile Include="**\*.feature.cs" Exclude="@(Compile)" />
    </ItemGroup>
  </Target>

• The final result looked like this:

<Project ToolsVersion...>
...
<ItemGroup>
  <None Include="Test.feature" >
  <Compile Include="Test.feature.cs">
    <AutoGen>True</AutoGen>
     <DesignTime>True</DesignTime>
     <DependentUpon>Test.feature</DependentUpon>
  </Compile>
  ...
</ItemGroup>
...
</Project>

• @rik-smeets second option did not work for me, but perhaps I misunderstood.

[SabotageAndi]

And if I would get a project to reproduce it, I could fix it correctly for all .... We could include the whole itemgroup->dependentupon stuff in the csproj in our msbuild files so there is no need to have this lines for every file in your project.

[rik-smeets]

@SabotageAndi I think I can get around to creating an example project for it this Sunday.

[rik-smeets]

@SabotageAndi I created an example project for it over at https://github.com/rik-smeets/specflow-3-test-explorer. In the feature files is a description of the occurring issue and the possible workarounds as I described before. Let me know if you need any further information or help. Thanks!

[SabotageAndi]

@rik-smeets thanks for the repro project. I can reproduce the issue.

I think it's because it is a project in the old format.

For the new format we do this: https://github.com/techtalk/SpecFlow/blob/master/SpecFlow.Tools.MsBuild.Generation/build/SpecFlow.Tools.MsBuild.Generation.targets#L40

 <Compile Update="@(SpecFlowFeatureFiles->'%(CodeBehindFile)')"
             DependentUpon="%(Filename)"
             AutoGen="true"
             DesignTime="true"
             Visible="true"
             Condition="'$(EnableDefaultCompileItems)' == 'true'" />

But it looks like, this doesn't work with the old format.

[rik-smeets]

@SabotageAndi Thanks for your analysis. What do you mean with 'old format'? Do you mean compared to .NET Core projects?

The repro project is a new solution/project, created from scratch in Visual Studio 2019, targeting .NET Framework 4.7.2. I didn't change anything to the defaults.

[SabotageAndi]

old project format is the long one, where a lot of stuff is in the xml file. The sdk- style format (the new one) is the one that was introduced in VS2017 and is mostly used for .NET Core. But you can also use it for .NET Full Framework.

[rik-smeets]

Alright, thank you. Do you think this issue can be resolved within the possible limitations of the old .csproj format?

[SabotageAndi]

I am not sure. I have to talk with a colleague who is a master of MSBuild.

[dcdraper]

@SabotageAndi Hello there. Have you discovered any more information on this or perhaps have a workaround documented somewhere on how to handle/correct the old .csproj format? Also, I may have time to create a test project that uses NUnit if it would still be helpful to get this worked out...

[SabotageAndi]

@dcdraper Nothing new to report. As written I can reproduce it, but had no success with fixing it. I would suggest to migrate to the new csproj format. If you aren't doing really strange things, it's quite easy to do.

[SabotageAndi]

I had yesterday a talk with @DanielTheCoder about this issue. What we are doing to link the generated files to the feature files (https://github.com/techtalk/SpecFlow/blob/master/SpecFlow.Tools.MsBuild.Generation/build/SpecFlow.Tools.MsBuild.Generation.targets#L40), doesn't work in the old csproj format, because it doesn't recognize the update of the ItemGroup.

There is nothing we can do about it automatically. :-/

[rik-smeets]

@SabotageAndi Thanks for your analysis.

I migrate projects to the new csproj format where possibly, but in some cases (like with non-Core ASP.NET applications), this isn't a possibility.

Perhaps there is a solution (sort of)... If the SpecFlow extension detects that the 'classic' .csproj format is used, then it should insert this in the .csproj (which I now do manually to make sure tests can be opened from test explorer, as I explained before):

    <Compile Include="[feature name].feature.cs">
      <AutoGen>True</AutoGen>
      <DesignTime>True</DesignTime>
      <DependentUpon>[feature name].feature</DependentUpon>
    </Compile>

Just like with SpecFlow <3, but of course still without the SpecFlowSingleFileGenerator.

What do you think about that?

[dcdraper]

Even though I appreciate all your attempts to help.... this is an unfortunate situation. I have tried three times to "update" to the new format and upgrade SpecFlow with nothing but frustrating build error after build error. I almost got there one but it didn't happen. All the other automation engineers keep updating Visual Studio or the nuget packages and I keep having to go fix them back to the old version.

Unless there is some clear and specific documentation from SpecFlow on how to overcome this issue, I must start looking at alternatives :( So much wasted time on this has left me disappointed and quite frustrated on how the new version was rolled out.

[SabotageAndi]

@dcdraper Sorry to hear that. Could you say which error you are getting? I upgraded all our examples (about 30 projects) 2 or 3 times and they were working. But they are really simple projects. So I think you have something special in your project, that we didn't thought of.

Documentation how to update to SpecFlow 3 is here: https://specflow.org/2019/updating-to-specflow-3/

[rik-smeets]

I resolved the problem in our solution, which is dependent on (non-Core) ASP.NET and therefore cannot be upgraded to an SDK-style project.

What you can do to resolve this issue:

1. Make sure ALL of the projects in your solution use PackageReferences. See Migrate from packages.config to PackageReferenc over at Microsoft Docs. Note: ASP.NET projects are not supported by the built in migrator, but you can use the Visual Studio 2017 extension NuGet PackageReference Upgrader to do it. Or you can do it manually.
2. Migrate your SpecFlow test project(s) to SDK-style. You can use CsprojToVs2017 for this, or you can do it manually.
3. Make sure you delete all the .feature.cs files from your repository and ignore files with that extension in the .gitignore (they're being generated while building now, so they don't have to be in source control anymore - no more merge conflicts, hooray!). Also, remove any references to .feature.cs files etc. in the new .csproj format.

It is possible to mix SDK-style and 'classic' .csproj file formats, as long as they all use PackageReference instead of packages.config.

If you want, you can migrate other projects to the SDK-style too (again, everything except the ASP.NET project, which is not supported). Be sure to test everything well of course.

Other than resolving this issue, the SDK-style project is very light-weight. PackageReferences provide some nice advantages too. So I'd recommend doing it anyway, if possible.

Good luck!

[Rain9333]

@rik-smeets works like a charm! Thank you!

[LirazShay]

@SabotageAndi
The solution provided by @rik-smeets is just a workaround. Do you see any real solution for this issue in the nearest future?

[sjeanren]

I Added this in the csproj and it seems to work. It avoids to specify each feature.cs file individually :

<Compile Include="*\\*.feature.cs">
  <AutoGen>True</AutoGen>
  <DesignTime>True</DesignTime>
</Compile>

[nikitozeg]

I Added this in the csproj and it seems to work. It avoids to specify each feature.cs file individually :

<Compile Include="*\\*.feature.cs">
  <AutoGen>True</AutoGen>
  <DesignTime>True</DesignTime>
</Compile>

hi! I have applied your solution, but new probled appears: *.feature.cs files is now visible in solution explorer. you have the same issue?

[LirazShay]

The solution of @sjeanren has many problems. It works for existing items.
When I add new feature, then the cs file is not added. Only when restarting Visual studio and the project is loaded, then the new cs file is added