easybuild-framework icon indicating copy to clipboard operation
easybuild-framework copied to clipboard

Extract test summary from build and add to the PR test report

Open branfosj opened this issue 3 years ago • 0 comments

For items like CP2K we run the tests, but allow failures in the test suite. It'd be useful to add the test suite report to the log in the PR.

I.e. in https://github.com/easybuilders/easybuild-easyconfigs/pull/15146#issuecomment-1073065790 it'd be useful to have

--------------------------------- Summary --------------------------------
Number of FAILED  tests 18
Number of WRONG   tests 6
Number of CORRECT tests 3604
Total number of   tests 3628
--------------------------------------------------------------------------
Number of LEAKING tests 0
Number of memory  leaks 0
--------------------------------------------------------------------------
GREPME 18 6 3604 0 3628 0

Summary: correct: 3604 / 3628; wrong: 6; failed: 18; 128min
Status: FAILED

included in the build report.

This would also help where people run with --ignore-test-failures.

branfosj avatar May 25 '22 08:05 branfosj