Count catch blocks as branches of the code

[ekolis]

Installed product versions

• Visual Studio: 2022 Professional
• This extension: 1.1.229

Description

If my code contains a try/catch block, the catch is not counted as a branch of the code for branch coverage purposes. This can lead to a bizarre scenario where I have 100% branch coverage but the catch block is not covered. I think catch blocks should count as branches for this reason.

Steps to recreate

1. Write some code with a try/catch block
2. Write a test for that code that covers all the code except the catch block

Current behavior

Branch coverage reported as 100%

Expected behavior

Branch coverage should be reported as less than 100%, because the catch block was not tested.

[tonyhallett]

FCC just generates a report and provides editor colouring based on the coverage report from the coverage providers - although we do specify to the Ms Code Coverage provider to generate the report as a Cobertura file.

The cobertura file is then passed to Report Generator for the report and the data used for the gutters.

If we were to not supply a Cobertura file from Ms Code Coverage then we would have an .xml file or a .coverage file. The latter is binary and would need to be converted.

For my demo try/catch

Cobertura

            <method line-rate="0.55555555555555558" branch-rate="1" complexity="1" name="MethodWithTryCatch" signature="()">
              <lines>
                <line number="40" hits="1" branch="False" />
                <line number="42" hits="1" branch="False" />
                <line number="43" hits="1" branch="False" />
                <line number="44" hits="1" branch="False" />
                <line number="45" hits="0" branch="False" />
                <line number="46" hits="0" branch="False" />
                <line number="47" hits="0" branch="False" />
                <line number="48" hits="0" branch="False" />
                <line number="49" hits="1" branch="False" />
              </lines>
            </method>

.xml and converted

        <function block_coverage="66.67" line_coverage="55.56" blocks_covered="4" blocks_not_covered="2" lines_covered="5" lines_partially_covered="0" lines_not_covered="4" id="8388" token="0x600000b" name="MethodWithTryCatch()" namespace="TryCatchCoverageDemo" type_name="TryCatch">
          <ranges>
            <range source_id="0" covered="yes" start_line="40" start_column="9" end_line="40" end_column="10" />
            <range source_id="0" covered="yes" start_line="42" start_column="13" end_line="42" end_column="14" />
            <range source_id="0" covered="yes" start_line="43" start_column="17" end_line="43" end_column="33" />
            <range source_id="0" covered="yes" start_line="44" start_column="13" end_line="44" end_column="14" />
            <range source_id="0" covered="no" start_line="45" start_column="13" end_line="45" end_column="30" />
            <range source_id="0" covered="no" start_line="46" start_column="13" end_line="46" end_column="14" />
            <range source_id="0" covered="no" start_line="47" start_column="17" end_line="47" end_column="44" />
            <range source_id="0" covered="no" start_line="48" start_column="13" end_line="48" end_column="14" />
            <range source_id="0" covered="yes" start_line="49" start_column="9" end_line="49" end_column="10" />
          </ranges>
        </function>
      </functions>

So we can see that ms code coverage does blocks. Whereas Coverlet does Conditional branches.

We cannot pass a block coverage report to Report Generator and get branches.

This information is available though https://github.com/danielpalme/ReportGenerator/blob/537c17c126cefe4da5de2bb885cee520cdf43b70/src/ReportGenerator.Core/Parser/DynamicCodeCoverageParser.cs#L252

        private static void SetMethodMetrics(CodeFile codeFile, IEnumerable<XElement> methodsOfFile)
        {
            foreach (var method in methodsOfFile)
            {
                string fullName = method.Attribute("name").Value;

                // Exclude properties and lambda expressions
                if (fullName.StartsWith("get_", StringComparison.Ordinal)
                    || fullName.StartsWith("set_", StringComparison.Ordinal)
                    || LambdaMethodNameRegex.IsMatch(fullName))
                {
                    continue;
                }

                fullName = ExtractMethodName(fullName, method.Attribute("type_name").Value);
                string shortName = MethodRegex.Replace(fullName, m => string.Format(CultureInfo.InvariantCulture, "{0}({1})", m.Groups["MethodName"].Value, m.Groups["Arguments"].Value.Length > 0 ? "..." : string.Empty));

                var metrics = new[]
                {
                    Metric.BlocksCovered(method.Attribute("blocks_covered").Value.ParseLargeInteger()),
                    Metric.BlocksNotCovered(method.Attribute("blocks_not_covered").Value.ParseLargeInteger())
                };

                var methodMetric = new MethodMetric(fullName, shortName, metrics);

                var seqpnt = method
                    .Elements("ranges")
                    .Elements("range")
                    .FirstOrDefault();

                if (seqpnt != null)
                {
                    methodMetric.Line = int.Parse(seqpnt.Attribute("start_line").Value, CultureInfo.InvariantCulture);
                }

                codeFile.AddMethodMetric(methodMetric);
            }
        }

So it will be considered when the FCC custom report is finalized, but it will only work with Ms Code Coverage and not Coverlet ( and I expect OpenCover too based on the entry in the Report Generator table )

[ekolis]

Oh, all right, that makes sense, that we're limited by the features of the underlying tools. Thanks anyway!