[Test] Adding test coverage metric tool
This PR makes changes to the PHPUnit configuration in order to generate coverage metrics. The reporting can be viewed on Travis. There is also a tool that can be run locally, which generates detailed HTML files under the htdocs directory. These files can then be viewed on your browser.
Testing instructions
To see the metrics on Travis:
- Checkout the unit test build on Travis. It should display some coverage metrics at the bottom. To see the detailed HTML reports:
- Checkout this branch
- Run the tool
code_coverage.phpin the tools directory with:php code_coverage.php - See that the HTML files are properly created under
htdocs/log/codeCoverage/<date>. - To view overall metrics and other useful information, go to
https://<loris-url>/log/codeCoverage/<date>/index.htmlandhttps://<loris-url>/log/codeCoverage/<date>/dashboard.html - To view the code coverage for a specific library (ex. Utility.class.inc), go to
https://<loris-url>/log/codeCoverage/<date>/Utility.class.inc
Proposal
The current proposal is that whenever unit tests are run (locally or on Travis), a command-line version of the metrics will be displayed for the PHP libraries. The tool used to generate the detailed HTML files can be run locally on any branch from the tools directory.
TODO (If this correctly passes Travis)
-
[x] Add date/time that coverage was calculated at the top of each HTML file. Keep track of past HTML files.
-
[x] Configure files to highlight most important points.
Hi Alex, any updates for this PR in the next day or so?
As of the latest commit:
I have created a new tool in the tools directory. This tool generates the HTML code coverage report, adds the date that the reports were generated at the top of each page, and puts all the HTML files under one directory with the date included in the path.
The updated plan for this PR (as of 2020-08-17):
- See if the
code_coverage.phptool will generate the HTML files if it is run through Travis and not locally on a VM. - If so: make this tool part of the
.travis.ymlscript and make it a Travis "cron job" that runs on the main branch monthly to generate monthly reports. - If not: Either keep this tool in the
toolsdirectory and have it run manually by a developer on a regular basis OR try to generate similar coverage metris using the Github CodeCov tool.
@AlexandraLivadas All the file name with 2020-08-17 should be removed. It will be generated by user.
Any updates since yesterday on which exact plan you'll be pursuing/finishing this week?
@christinerogers I will be making a commit this afternoon to update the .travis.yml file to test this script on Travis. I will update this comment with the results of that test!
hi @AlexandraLivadas when you do push again - don't forget to also edit and keep the PR header up to date so it reflects the current plan for this PR. (If you can't edit it, let me know)
@AlexandraLivadas - when you have a minute this week -- is this ready for re-review ? Who should test it, @kongtiaowang
@AlexandraLivadas Can you fix the conflicts?
@AlexandraLivadas could you rebase and update the PR? Thank you!
@AlexandraLivadas could you fix the tests? Thanks!