virtual void Catch::CumulativeReporterBase::testCaseEnded(const Catch::TestCaseStats&): Assertion `m_sectionStack.size() == 0' failed

[sourcedelica]

Description

I am getting this error sometimes in CI.

testd: /build/agent-2/FAL-BS-JOB1/third-party/catch-1.9.3/catch.hpp:9611: 
virtual void Catch::CumulativeReporterBase::testCaseEnded(const Catch::TestCaseStats&): 
Assertion `m_sectionStack.size() == 0' failed.

Rerunning the job fixes the problem.

This was mentioned in a comment to #663.

Extra information

• Catch command line options: -r junit -o testresults.xml
• Catch version: v1.9.3
• Operating System: RHEL 4.9
• Compiler+version: g++ 5.3

[sourcedelica]

This is happening almost all the time now

[horenmar]

Do you have a reliable repro, or is it still an intermittent thing? The JUnit reporter is ran multiple times on each commit and it doesn't reproduce with our tests, so it's hard to tell what is going on.

[sourcedelica]

I will work on a repro

[sd-x]

I'm using -fno-exceptions and see the same problem; this reproduces the problem for me: a.cpp:

#define CATCH_CONFIG_MAIN
#include "catch2/catch.hpp"

TEST_CASE("doomed", "[tag]") {

  SECTION("only section") {
    REQUIRE(false);
  }
}

> g++ --version
g++ (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> g++ -fno-exceptions -I../Catch2/single_include a.cpp
> ./a.out -r junit
<?xml version="1.0" encoding="UTF-8"?>
terminate called without an active exception
<testsuitesa.out: ../Catch2/single_include/catch2/catch.hpp:4463: void Catch::CumulativeReporterBase<DerivedT>::testCaseEnded(const Catch::TestCaseStats&) [with DerivedT = Catch::JunitReporter]: Assertion `m_sectionStack.size() == 0' failed.
Aborted (core dumped)

[horenmar]

@sd-x That's actually quite useful, thanks.

[fbradyirl]

Seeing this also in my CI sometimes.

[briterator]

I'm seeing this pretty regularly, but not 100% consistently, on our windows builds. We don't have exceptions disabled or anything. Is the recommended workaround to stop using SECTIONs?

for reference, also using junit. seems to be a common factor in these reports.

[chausner]

Our GitLab CI just ran into the same or a very similar issue, the exact assertion error in our case is

Assertion failed: m_sectionStack.size() == 0, file catch.hpp, line 5859

We are also running on Windows and using the JUnit reporter. Catch version v2.11.1, MSVC 16.5. The issue disappeared after rerunning.

[suratovvlad]

Probably I found one possible scenario, why it could happen #1967

[johnboiles]

My problem was that I was dividing by zero which crashed on my Linux CI but did not crash on my macOS dev machine. A working stacktrace from Catch would have saved me 30 minutes of printf debugging!

[hbirler]

I am not too familiar with catch internals but here is what I assume is happening

In RunContext::handleFatalErrorCondition, handleUnfinishedSections() is called: https://github.com/catchorg/Catch2/blob/029fe3b4609dd84cd939b73357f37bbb75bcf82f/src/catch2/internal/catch_run_context.cpp#L453

If I understand it correctly, running Sections are in m_activeSections, and when a Section's destructor is called with an active exception, that Section is moved to m_unfinishedSections. handleUnfinishedSections() then handles those Sections in m_unfinishedSections. However, if a signal is being handled, we have not and will not call the destructors for the Sections, which means, all those Sections are in m_activeSections and not m_unfinishedSections.

My (potentially very dirty) fix is the following before the call to handleUnfinishedSections():

while (!m_activeSections.empty()) {
    auto nl = m_activeSections.back()->nameAndLocation();
    SectionEndInfo endInfo{ SectionInfo(nl.location, nl.name), {}, 0.0 };
    sectionEndedEarly(CATCH_MOVE(endInfo));
}
handleUnfinishedSections();

I move the Sections from m_activeSections to m_unfinishedSections in a best effort way as to not trigger the assertion later.

Btw, here is a test case that reproduces the issue for me (with --reporter JUnit):

TEST_CASE("broken") {
   SECTION("section") {
      /// Use illegal cpu instruction
      __asm__ __volatile__("ud2" : : : "memory");
   }
}