Loris icon indicating copy to clipboard operation
Loris copied to clipboard

[Test] Adding test coverage metric tool

Open AlexandraLivadas opened this issue 5 years ago • 11 comments

This PR makes changes to the PHPUnit configuration in order to generate coverage metrics. The reporting can be viewed on Travis. There is also a tool that can be run locally, which generates detailed HTML files under the htdocs directory. These files can then be viewed on your browser.

Testing instructions

To see the metrics on Travis:

  1. Checkout the unit test build on Travis. It should display some coverage metrics at the bottom. To see the detailed HTML reports:
  2. Checkout this branch
  3. Run the tool code_coverage.php in the tools directory with: php code_coverage.php
  4. See that the HTML files are properly created under htdocs/log/codeCoverage/<date>.
  5. To view overall metrics and other useful information, go to https://<loris-url>/log/codeCoverage/<date>/index.html and https://<loris-url>/log/codeCoverage/<date>/dashboard.html
  6. To view the code coverage for a specific library (ex. Utility.class.inc), go to https://<loris-url>/log/codeCoverage/<date>/Utility.class.inc

Proposal

The current proposal is that whenever unit tests are run (locally or on Travis), a command-line version of the metrics will be displayed for the PHP libraries. The tool used to generate the detailed HTML files can be run locally on any branch from the tools directory.

TODO (If this correctly passes Travis)

  • [x] Add date/time that coverage was calculated at the top of each HTML file. Keep track of past HTML files.

  • [x] Configure files to highlight most important points.

AlexandraLivadas avatar Aug 05 '20 18:08 AlexandraLivadas

Hi Alex, any updates for this PR in the next day or so?

christinerogers avatar Aug 17 '20 14:08 christinerogers

As of the latest commit:

I have created a new tool in the tools directory. This tool generates the HTML code coverage report, adds the date that the reports were generated at the top of each page, and puts all the HTML files under one directory with the date included in the path.

AlexandraLivadas avatar Aug 17 '20 20:08 AlexandraLivadas

The updated plan for this PR (as of 2020-08-17):

  • See if the code_coverage.php tool will generate the HTML files if it is run through Travis and not locally on a VM.
  • If so: make this tool part of the .travis.yml script and make it a Travis "cron job" that runs on the main branch monthly to generate monthly reports.
  • If not: Either keep this tool in the tools directory and have it run manually by a developer on a regular basis OR try to generate similar coverage metris using the Github CodeCov tool.

AlexandraLivadas avatar Aug 17 '20 22:08 AlexandraLivadas

@AlexandraLivadas All the file name with 2020-08-17 should be removed. It will be generated by user.

kongtiaowang avatar Aug 18 '20 14:08 kongtiaowang

Any updates since yesterday on which exact plan you'll be pursuing/finishing this week?

christinerogers avatar Aug 18 '20 14:08 christinerogers

@christinerogers I will be making a commit this afternoon to update the .travis.yml file to test this script on Travis. I will update this comment with the results of that test!

AlexandraLivadas avatar Aug 18 '20 15:08 AlexandraLivadas

hi @AlexandraLivadas when you do push again - don't forget to also edit and keep the PR header up to date so it reflects the current plan for this PR. (If you can't edit it, let me know)

christinerogers avatar Aug 18 '20 20:08 christinerogers

@AlexandraLivadas - when you have a minute this week -- is this ready for re-review ? Who should test it, @kongtiaowang

christinerogers avatar Sep 22 '20 18:09 christinerogers

@AlexandraLivadas Can you fix the conflicts?

kongtiaowang avatar Apr 23 '21 13:04 kongtiaowang

@AlexandraLivadas could you rebase and update the PR? Thank you!

cmadjar avatar Jun 08 '21 13:06 cmadjar

@AlexandraLivadas could you fix the tests? Thanks!

cmadjar avatar Aug 10 '21 13:08 cmadjar