Skipping last section runs the test another time without sections
Describe the bug If the last section is skipped then it seems that the TEST_CASE will run an additional time without going into any section, which can cause failures if the test contains common assertions after the sections.
Expected behavior Skipping the last section would end the TEST_CASE
Reproduction steps Steps to reproduce the bug.
TEST_CASE("Last Section skipped") {
int foo = 0;
SECTION("1") {
foo = 1;
}
SECTION("2") {
SKIP();
}
CHECK(foo != 0);
}
TEST_CASE("First Section skipped") {
int foo = 0;
SECTION("2") {
SKIP();
}
SECTION("1") {
foo = 1;
}
CHECK(foo != 0);
}
In the example above if the Last section is skipped the CHECK will fail, however if we change the order of the sections it will pass.
Platform information:
- OS: macOS 13.4.1 22F82 arm64
- Compiler+version: Apple clang version 14.0.3 (clang-1403.0.22.14.1)
- Catch version: v3.4.0
- OS: Arch Linux x86_64
- Compiler+version: gcc 12.3.0
- Catch version: v3.4.0
Yeah, this is a known bug/issue. Exiting the last section in a TEST_CASE irregularly (throwing exception/SKIP/etc) will cause the test case to be entered again, to see if there is another section to execute.
This should be fixable in most (all?) cases, but would require significant rewrite of the section tracking logic.
The reason this works fine when you exit a non-last section is that you won't notice the extra rerun, as there is not-yet executed section to enter.