Question: is it possible to determine code coverage?

[rubenww]

Thank you for the great package. I was wondering if it is possible to determine the code coverage using pFUnit and if not, if there are plans to implement this.

[tclune]

I did play with code coverage a while back, but was using a proprietary tool. I was a bit surprised that despite my best efforts to aggressively use TDD for developing pFUnit that the coverage was still well below 90%.

I am quite open to contributions which would automate this metric during CI tests.

[rubenww]

Thanks @tclune. I've found a solution using an external tool https://software.intel.com/content/www/us/en/develop/documentation/cpp-compiler-developer-guide-and-reference/top/optimization-and-programming-guide/tools/pgo-tools/code-coverage-tool.html Feel free to close this report or keep it open if you want.

[tclune]

I'm glad you found something. If you applied this to pFUnit itself, I would appreciate it if you could post some of the results here. Also, if you could comment on any nontrivial issues you had in following the instructions? Thanks in advance.

[rubenww]

It worked more or less 'out of the box'. We're using the ifort compiler, I had to add the following flags to the Makefile: -prof-gen=srcpos -prof-dir=./codecov_prof. This ensured that the necessary information was collected and all output was put in the ./codecov_prof directory.

Then, I added the following target to generate the coverage info (after running the pFUnit tests once):

codecov:
	profmerge -cov_dir ./codecov_prof -prof_dir ./codecov_prof -src_root . -verbose  # merge all code coverage files  
	# -comp: exclude filenames that contain 'tst' using a compfile
	# -ccolor: use green for fully covered instead of white
	cd ./codecov_prof; codecov -comp comp_file.txt -ccolor '#d7fad2'

Where the file ./codecov_prof/comp_file.txt contains the following:

.
~tst

This includes all files, except for filenames that contain tst, so that the coverage report doesn't include information about the tests themselves. A HTML code coverage report is then generated in the ./codecov_prof directory after running the unit tests once and then executing make codecov. I hope this is useful to someone.

[tclune]

Thank you. We'll definitely take a look.

Adding @mathomp4 who helps with CI related issues.

[micric]

@rubenrivm did you build with cmake or make?

[rubenww]

We're using make unfortunately at the moment, because we have a very old fortran compiler and can only use an old version of pFUnit, which doesn't seem to play well with cmake..

[micric]

@rubenrivm thank you. I was asking because I managed to use the intel coverage tool with Make, but can't make it work using CMake. I will try more.

[Forsti5]

@rubenww Thank you very much for your information on the Intel code coverage tool. I made two repositories with cmake, pFUnit, the Intel Fortran Compiler and the Intel code coverage tool. One is pretty basic (https://github.com/Forsti5/Fortran-Unit-Test-Trivial) and one has a few files and submodules (https://github.com/Forsti5/Fortran-Unit-Test-Basic). Maybe it is useful for someone who also wants to implement code coverage. You only have to run the build.x file and then you will find the code coverage in the folder ./build/coverage for the basic repository or in ./build/tests/coverage for the other repository.

[Forsti5]

I updated the above mentioned repositories and added coverage report for gfortran and also wrote a little readme (still work in progress). I also implemented workflows, which save the coverage results as artifacts, see https://github.com/Forsti5/Fortran-Unit-Test-Basic/actions/runs/1413720878 . Hopefully it is useful for someone.