UnitTestBot
Wrong coverage of C++ programs for UTBot Online
Problem
The following is the command line result of an example C++ program 'Pointers', it seems that none of the lines are covered. However, based on our manually inspection, they are covered indeed, so the displayed result might be wrong. (All C++ examples have met this problem.)
Coverage summary.
==== Coverage in "/var/utbot/processing/utbot-cpp-709589/src/snippet.cpp"
======== covered lines:
======== uncovered line ranges: 0 1 2 3 4 5 7 8 9 10 11 12
Totals:
{"functions":{"count":2,"covered":0,"percent":0},"instantiations":{"count":2,"covered":0,"percent":0},"lines":{"count":12,"covered":0,"percent":0},"regions":{"count":6,"covered":0,"notcovered":6,"percent":0}}
2022-06-20 08:02:08.721 ( 2.474s) [UnitTestBot ] GenerationUtils.cpp:51 INFO| Successfully ran tests.
[TL;DR] This issue is caused by invalid tests generation for some of C++ projects.
When we generate and run tests via UTBot Online, we create the project using the following template.
Let's generate tests for the following code snippet.
Two files: tests/src/snippet_dot_cpp_test.cpp and tests/src/snippet_dot_cpp_test.h will be generated.
File tests/src/snippet_dot_cpp_test.h includes src/snippet.cpp and that is why the coverage response is generated incorrectly (as while running tests we can't include src/snippet.cpp to linkage as it will lead to double definition error).
In C we fix this issue using extern functions. However, is C++ this solution should be modified as we need to support classes and their methods.
Current method of generation for tests' headers also causes another issues. For example, we can try to generate and run tests for the project of the following configuration.
In this case tests/src/snippet_dot_cpp_test.h will include only src/snippet.h. However, tests for my_abs function will be generated. So when we run tests we'll see the following error: error: use of undeclared identifier 'my_abs'.
Right now, the only way to generate tests and coverage response correcly is to declare all the functions from src/snippet.cpp in src/snippet.h.
This issue worth further investigation.
