How to get branch coverage in Sonar with MTPv2?

[bart-vmware]

We're currently using Coverlet in VSTest mode, which writes coverage in OpenCover format. Sonar can read that and produce line/branch coverage. Sonar doesn't support Cobertura, but it does support the Microsoft XML format. When using that with MTPv2, I only get line coverage, but no branch coverage. Is that a limitation in Sonar, or does the XML file format not contain branch coverage information?

I can't find a free tool to convert from Cobertura to OpenCover format. Is there a way to get branch coverage in Sonar when using MTPv2?

[MihuBot]

I'm a bot. Here is a possible related and/or duplicate issue (I may be wrong):

• https://github.com/microsoft/testfx/discussions/5544

[Evangelink]

For coverage questions, I'd recommend to go to https://github.com/microsoft/codecoverage.

cc @fhnaseer

[bart-vmware]

To clarify, the question concerns Microsoft.Testing.Extensions.CodeCoverage, which is maintained here.

[bart-vmware]

Related to this, it would be nice if settings from coverage.config can be overruled by command-line parameters. For example:

<?xml version="1.0" encoding="utf-8"?>
<Configuration>
  <Format>cobertura</Format>
</Configuration>

So that we can specify a different format in the dotnet test command in the Sonar GitHub Actions workflow. Without that, we need two configuration files, which contain a lot of duplicate settings with only one line different.

[fhnaseer]

Sonar doesn't support Cobertura, but it does support the Microsoft XML format.

Microsoft Code Coverage tools do not support OpenCover format. You can use ReportGenerator to convert reports to OpenCover format. https://reportgenerator.io/usage

[fhnaseer]

So that we can specify a different format in the dotnet test command in the Sonar GitHub Actions workflow. Without that, we need two configuration files, which contain a lot of duplicate settings with only one line different.

You can use dotnet test --collect "Code Coverage;Format=Cobertura".

[bart-vmware]

Sonar doesn't support Cobertura, but it does support the Microsoft XML format.

Microsoft Code Coverage tools do not support OpenCover format. You can use ReportGenerator to convert reports to OpenCover format. https://reportgenerator.io/usage

Thanks, I'll try that. I also found https://github.com/microsoft/codecoverage/issues/98 related to this.

[bart-vmware]

So that we can specify a different format in the dotnet test command in the Sonar GitHub Actions workflow. Without that, we need two configuration files, which contain a lot of duplicate settings with only one line different.

You can use dotnet test --collect "Code Coverage;Format=Cobertura".

Does that work with MTPv2? I haven't seen anything like that in docs, except for when using VSTest mode.

[Evangelink]

So that we can specify a different format in the dotnet test command in the Sonar GitHub Actions workflow. Without that, we need two configuration files, which contain a lot of duplicate settings with only one line different.

You can use dotnet test --collect "Code Coverage;Format=Cobertura".

@fhnaseer this is not MTP command.

[fhnaseer]

For MTP there is --coverage-output-format but I am not aware if you pass this when you execute dotnet test --coverage --coverage-output-format Cobertura.

Also, if you have testconfig.json then you can specify coverage settings in that file as well. https://github.com/microsoft/codecoverage/blob/main/docs/configuration.md#testconfigjson

[bart-vmware]

For MTP there is --coverage-output-format but I am not aware if you pass this when you execute dotnet test --coverage --coverage-output-format Cobertura.

Also, if you have testconfig.json then you can specify coverage settings in that file as well. https://github.com/microsoft/codecoverage/blob/main/docs/configuration.md#testconfigjson

Right, I can use --coverage-output-format in this case. But I can't, for example, override <DeterministicReport>true</DeterministicReport> from the command line, setting it to false. I know I can put these in a configuration file, but my feature request is to specify them on the command line, overriding the settings in the configuration file, as is possible when using VSTest mode. This way, I don't need to maintain multiple configuration files that only differ by one or two lines.

[bart-vmware]

I was able to get line and branch coverage to work in Sonar using ReportGenerator:

• Add a Sonar-specific configuration file that doesn't contain <DeterministicReport>true</DeterministicReport>
• Use the Sonar file format, because conversion to OpenCover requires a PRO license

  - name: Begin Sonar .NET scanner
    id: sonar_begin
    env:
      SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
    run: >-
      dotnet sonarscanner begin /d:sonar.coverageReportPaths="${{ env.TEST_OUTPUT_DIRECTORY }}/SonarQube.xml" .....
  

• Convert from Cobertura to Sonar file format after running tests:

  - name: Convert Cobertura coverage to Sonar format
    uses: danielpalme/[email protected]
    with:
      reports: '${{ env.TEST_OUTPUT_DIRECTORY }}/*cobertura.xml'
      targetdir: '${{ env.TEST_OUTPUT_DIRECTORY }}'
      reporttypes: 'SonarQube'
      filefilters: '-*.g.cs'
  

I'm closing this. Created https://github.com/microsoft/codecoverage/issues/199 to track using command-line parameters.