android-showcase icon indicating copy to clipboard operation
android-showcase copied to clipboard

Code coverage?

Open speekha opened this issue 6 years ago • 6 comments

On top of quality checks with ktlint and detekt, it could be interesting to add coverage to your reports. I didn't find any mention of JaCoCo or any other option in your gradle files.

speekha avatar Aug 22 '19 13:08 speekha

This is on my roadmap. Happy to prioritize it.

igorwojda avatar Aug 22 '19 22:08 igorwojda

I was experimenting with jacoco.

There is jacoco Gradle plugin, but it looks that it needs a bit of additional configuration for Kotlin and Android. It is not always that straight forward due to the usage of Gradle Kotlin DSL.

On top of the above there are two Gradle plugins that can simplify the configuration process (for multiple modules and potentially multiple build variants):

  1. arturdm/jacoco-android-gradle-plugin - plugin has to be applied at the subproject (module) level and it was not generating reports for me out of the box and it has many unresolved issues
  2. vanniktech/gradle-android-junit-jacoco-plugin - this one looks promising because it can be applied at the root project level. It also provides some tasks to merge Jacoco reports. Unfortunately, this plugin does not support android dynamic feature modules so I have opened the PR #158. (I also tried to run an updated version of this plugin locally, but it requires to provide some Android dependencies which is too much hustle for now)

Next steps: Wait for PR to be merged. If this will not happen soon then I may think about configuring this manually.

igorwojda avatar Aug 31 '19 14:08 igorwojda

I've had some difficulties setting it up for my project, especially because I had my tests in a separate module and I had to concatenate source folders from the others modules for jacoco to report coverage on them. If you're interested, you can look at the /gradle/coverage.gradle file in my project httpmocker: https://github.com/speekha/httpmocker

speekha avatar Aug 31 '19 16:08 speekha

Also Codacy and Codecov seem to interpret the reports differently. Codecov has a lot of missed lines (which are actually tested) that Codacy marks as covered. Not sure if that's because of a positive lack of accuracy on Codacy's part or a negative lack of accuracy on Codecov's part.

speekha avatar Sep 01 '19 12:09 speekha

thx for the project, I will take a look.

This comparison between Codacy and Codecov is a nice catch - I will take a closer look at this as well. It should be quite easy to determine which one is right by analyzing Jacoco reports, running AS test coverage and comparing all results.

igorwojda avatar Sep 02 '19 20:09 igorwojda

The differences I've noticed seemed linked to lines in kotlin that are interpreted as several different lines in Java: typically, a field declared in the kotlin constructor for instance that result in a constructor parameter, a field declaration, a getter and a setter in the byte code. So the same line can be marked as covered because the constructor parameter is executed but it can also be marked as not covered because the getter is not. Hard to decide which interpretation is the best, I guess.

speekha avatar Sep 02 '19 21:09 speekha