code coverage support

[johnynek]

This would look like using:

https://github.com/scoverage/scalac-scoverage-plugin

to get the scala test rules to emit code coverage reports, then aggregating them across the repo.

It is related to https://github.com/bazelbuild/bazel/issues/1118 but I don't think it is actually a blocker. We may just be able to produce coverage output of each test rule, then an aspect to walk all the rules and aggregate a total.

[softprops]

Has anyone started working on this? This is going to become a priority from my team soon. We're working on replacing our reliance on sbt in our jenkins pipeline with bazel. One part of that is figuring out the test report and codecov ( scoverage ) stories.

[ittaiz]

I don't know (and think) anyone started working on it but I'll add that there's been a lot of discussion on various coverage issues on bazel itself so I'd recommend anyone tackling this to read a bit beforehand On Fri, 5 May 2017 at 20:40 doug tangren [email protected] wrote:

Has anyone started working on this? This is going to become a priority from my team soon. We're working on replacing our reliance on sbt in our jenkins pipeline with bazel. One part of that is figuring out the test report and codecov ( scoverage ) stories.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bazelbuild/rules_scala/issues/184#issuecomment-299528870, or mute the thread https://github.com/notifications/unsubscribe-auth/ABUIF4kLM62g_5DrI10vlWYAwmLVb7Dfks5r217wgaJpZM4NG3kM .

[softprops]

👍

I was looking through the machinery that ties scoverage into sbt, and it looks like it may be possible to to just opt into code coverage for a regular scala_test by threading through a few additional arguments flags: namely: -Xplugin:path/to/plugin and -P:scoverage:dataDir:path/to/scoverage-data

[softprops]

opt in meaning maybe this is possible today without any additional changes and when the coverage story in bazel settles something like this could then just become a conveniences of being instrumented by default with running bazel cover

[johnynek]

The scala side should be easy. What I don't know is how to plug in to bazel's expectations (if they are even standard). Like where should we write the coverage files to? What should the format be? On Fri, May 5, 2017 at 08:41 doug tangren [email protected] wrote:

opt in meaning maybe this is possible today without any additional changes and when the coverage story in bazel settles something like this could then just become a conveniences of being instrumented by default with running bazel cover

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bazelbuild/rules_scala/issues/184#issuecomment-299544136, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEJdgmLCVqU5WVE95c7FE6dhkMXP38Tks5r221OgaJpZM4NG3kM .

[softprops]

My guess is it'll be the same as the test_filter flag story. When x is present in the env ( or args ) do y. My guess about where the conventional place to write reports would be customizable with rule dependent defaults, like junit test reports

[ittaiz]

I really suggest searching the Bazel github issues with the term coverage. There have been several discussions about it in the issues. On Fri, 5 May 2017 at 21:56 doug tangren [email protected] wrote:

My guess is it'll be the same as the test_filter flag story. When x is present in the env ( or args ) do y. My guess about where the conventional place to write reports would be customizable with rule dependent defaults, like junit test reports

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/bazelbuild/rules_scala/issues/184#issuecomment-299547873, or mute the thread https://github.com/notifications/unsubscribe-auth/ABUIFzVFzcvA0v4oPL3Z6UF9Srf6YWNZks5r23DSgaJpZM4NG3kM .

[softprops]

@johnynek do you know ( specifically for scalac ) if there needs to be anything special done for scalac plugins to set up a special kind of classpath separate form the classpath of the files being compiled?

I've now optimistically convinced myself this should be more straightforward that I originally thought so I couldn't help trying out a POC to close out my friday on a happy note.

my WORKSPACE

# other stuff ...

filegroup(
    name = "scoverage_plugin",
    srcs = ["sbt-plugins/scalac-scoverage-plugin_2.11-1.2.0.jar"],
    visibility = ["//visibility:public"],
)

java_import(
    name = "sbt-plugins",
    jars=glob([
    "sbt-plugins/scalac-scoverage*.jar"],
    exclude = [
            "sbt-plugins/scala-library*jar",
            "sbt-plugins/scala-xml*jar"
    ]),
    visibility = ["//visibility:public"],
)

my BUILD

scala_test(
    name = "foo",
    size = "small", 
    srcs = glob(
        [
            "src/test/scala/**/*.scala",
            "src/test/java/**/*.java",
        ],
    ),
    jvm_flags = [
        "-Dfile.encoding=UTF-8",
    ],
    plugins = ["@meetuplib//:scoverage_plugin"], # the scoverage scalac plugin jar
    scalacopts = [
      "-P:scoverage:dataDir:/path/to/target/scovtest/scoverage-data",
      "-Yrangepos"
    ],
   deps = [
      # other stuff....
        "@meetuplib//:sbt-plugins" # puts scoverage deps on my classpath
    ],
)

when I run bazel test -s --progress_report_interval 1 --test_output streamed --strategy=Scalac=standalone //foo:* I get a class not found error on scala.xml.NamespaceBinding. Scala xml is a dependency of scoverage but I've already confirmed it's on my classpath. I checked my foo-test_worker_input file to be sure.

[info] Instrumentation completed [50790 statements]
error: java.lang.NoClassDefFoundError: scala/xml/NamespaceBinding
	at scoverage.ScoverageInstrumentationComponent$$anon$1.run(plugin.scala:121)
	at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1501)
	at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1486)
	at scala.tools.nsc.Global$Run.compileSources(Global.scala:1481)
	at scala.tools.nsc.Global$Run.compile(Global.scala:1582)
	at scala.tools.nsc.Driver.doCompile(Driver.scala:32)
	at scala.tools.nsc.MainClass.doCompile(Main.scala:23)
	at scala.tools.nsc.Driver.process(Driver.scala:51)
	at io.bazel.rulesscala.scalac.ScalacProcessor.compileScalaSources(ScalacProcessor.java:207)
	at io.bazel.rulesscala.scalac.ScalacProcessor.processRequest(ScalacProcessor.java:77)
	at io.bazel.rulesscala.worker.GenericWorker.run(GenericWorker.java:125)
	at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:42)
Caused by: java.lang.ClassNotFoundException: scala.xml.NamespaceBinding
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 12 more

cat bazel-out/local-fastbuild/bin/foo/foo-test_worker_input
Classpath: # includes external/scala/lib/scala-xml_2.11-1.0.4.jar among otherthings
EnableIjar: False
Files: # bunch of sources
IjarCmdPath:
IjarOutput:
JarOutput: bazel-out/local-fastbuild/bin/foo/foo-test.jar
JavacOpts:
JavacPath: external/local_jdk/bin/javac
JavaFiles:  # bunch of sources
JvmFlags: -J-Dfile.encoding=UTF-8
Manifest: bazel-out/local-fastbuild/bin/foo/foo-test_MANIFEST.MF
Plugins: external/meetuplib/sbt-plugins/scalac-scoverage-plugin_2.11-1.2.0.jar
PrintCompileTime: False
ResourceDests: /log4j.properties,/malformed-manifest.json,/test-velocity.properties,/valid-manifest.json
ResourceJars:
ResourceSrcs: # bunch of sources
ResourceStripPrefix:
ScalacOpts: -P:scoverage:dataDir:/path/to/target/scovtest/scoverage-data,-Yrangepos
SourceJars:

The only thing that turned up in a lazy google search was this issue that popped on up the intellij mailing list. I didn't quite get the solution in that case or how it may be part of the problem in mine.

[sdtwigg]

I was actually speaking with ahirreddy@ about this the other day. ~~I think the plugin pulls classes from the bootclasspath. Was eventually planning to verify this and fixup the rules to also setup the bootclasspath properly.~~ Edit: You just need to add the dependencies as other (fake) -Xplugin. This makes a more structured deploy_jar construction tempting.

It might be easier just to build the plugin as a deploy_jar for now though.

[softprops]

@sdtwigg I'm more of a consumer than a producer in this case. I'm using the scoverage plugin and runtime jars from maven central and trying to provide them as a plugin value to a scala_test rule. Can you draw me a rough sketch of what you mean?

[softprops]

Now I'm thinking there may be something to do with the bootclasspath which may actually be related to an open bug with scala.

One thing I failed to mention above when setting up my test was that initially I didn't have the scalac-scoverage-runtime jar on my classpath which caused a similar issue. After adding that to my scala_tests' deps, that problem was resolved. The next issue I ran into was the xml.NameSpace error.

I tried the change mentioned in the issue by adding the -nobootcp to my scalacopts. Same issue.

I'm a little confused because the identity of this class technically didn't change between scala 2.10 and 2.11 so either way I expected it to be on the classpath one way or another.

[softprops]

I have a little bit more mental energy to this but I'm still stumped on the scala.xml.NameSpace issue. Though I was reminded that in order for scoverage to instrument code it needs to be applied to my scala_library targets ( not my scala_test ) targets which makes things a bit more awkward

[johnynek]

we should be able to use a target in the current repo as a plugin. If we can't now, that shouldn't be too much work. Then, we could just make a deploy jar ourselves by linking in the full path.

[sdtwigg]

Sorry, I never really described what deploy_jar is although seems like you found out but just in case: The deploy_jar is a fat-jar that contains the current compiled target and all its transitive dependencies (both runtime and compile time). It is analogous to the deploy_jar you can get from a java_binary (although not executable if from scala_library).

I realized over the weekend that just directly using a deploy_jar from one of the rules_scala rules probably won't work because I don't think it will put the plugin.xml in the right place. So, would need to take the deploy_jar from the rules and have another action/genrule put the plugin.xml inside that fat jar.

I evntually wanted to demonstrate this by replacing the linter plugin in the tests with a custom rolled one (both to demonstrate how to make plugins and drop a rules_scala dependency). Unfortunately, I am really reluctant to say I have the time to do this :( . Had spent my 'free-work' time over weekend pushing through the bazel changes outlined in: https://github.com/bazelbuild/rules_scala/issues/57#issuecomment-299323413

PS: Do you think it is a problem that using two plugins might have redundant classes in their fat-jars? (Assuming that the classes are identical duplicates then it seems like it would just be like a 'stylistic' issue versus a functional one.)

[softprops]

How do you make a deploy_jar without a main class, do scalac plugins run with a java main?

Also, I can kind of half confirm the deploy_jar thing works. I cloned the scoverage repo and used sbt-assembly to bake me a fat jar. I referenced that in a java_import and that removed the scala.xml.NamespaceBinding issue.

Next issue is related to sandboxing which is probably just something Im doing wrong. Given the scalacopt for the plugin's data dir ( -P:scoverage:dataDir ), let's call that /path/to/scovtest/scoverage-data/

[info] Instrumentation completed [852 statements]
[info] Wrote instrumentation file [/path/to/scovtest/scoverage-data/scoverage.coverage.xml]
[info] Will write measurement data to [/path/to/scovtest/scoverage-data]
Discovery starting.
*** RUN ABORTED *** (153 milliseconds)
  java.lang.RuntimeException: Unable to load a Suite class that was discovered in the runpath: com.meetup.base.util.StatusesTest
  at org.scalatest.tools.DiscoverySuite$.getSuiteInstance(DiscoverySuite.scala:84)
  at org.scalatest.tools.DiscoverySuite$$anonfun$1.apply(DiscoverySuite.scala:38)
  at org.scalatest.tools.DiscoverySuite$$anonfun$1.apply(DiscoverySuite.scala:37)
  at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
  at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
  at scala.collection.Iterator$class.foreach(Iterator.scala:893)
  at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
  at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
  at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
  at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
  ...
  Cause: java.io.FileNotFoundException: /path/to/scovtest/scoverage-data/scoverage.measurements.1 (Read-only file system)
  at java.io.FileOutputStream.open0(Native Method)
  at java.io.FileOutputStream.open(FileOutputStream.java:270)
  at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
  at java.io.FileWriter.<init>(FileWriter.java:107)
  at scoverage.Invoker$$anonfun$1.apply(Invoker.scala:42)
  at scoverage.Invoker$$anonfun$1.apply(Invoker.scala:42)
  at scala.collection.concurrent.TrieMap.getOrElseUpdate(TrieMap.scala:901)
  at scoverage.Invoker$.invoked(Invoker.scala:42)
  at com.meetup.base.util.StatusesTest.<init>(StatusesTest.scala:6)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

This kind of makes sense because sandboxing shouldn't let me write outside. To continue the POC I added --spawn_strategy=standalone

bazel test --spawn_strategy=standalone --verbose_failures  --progress_report_interval 1 --test_output streamed --strategy=Scalac=standalone //base:*

that allows the scalac plugin to generate two files

scoverage.coverage.xml  scoverage.measurements.1

So thats the recording process in a nutshell. Report generation... that's a whole other beast but I just need to confirm that the coverage recordings are all that my jenkins plugin requires. Reports may not be a need to have for me atm.

[softprops]

Unfortunately, I am really reluctant to say I have the time to do this

I'm happy to help but will probably need more direction, I have a decent sense for bazel's architectural model still have some blindspots.

I'm very much in a POC phase right now where Im testing to see if it can work in a timeboxed period of time. The next to think about is probably more relevant to the discussion on bazel core's issue tracker. How should the library rules detect the conditional case for appending the scalac plugin. You typically will only want to do that in some testing mode. What you ship shouldn't have the scoverage instrumentation.

[softprops]

circling back to this. I had a few questions that I think could be answered faster here but perhaps should be asked on the bazel-discuss list.

• creating a standalone scoverage jar

I think I found some examples of where I can create the scoverage binary jar here. I think that was suggested above. I'm just double check to make sure that's the right pattern to follow.

• conditionally appending scoverage jar to scalac plugins

So the way to know the coverage should be enabled or not is documented here. The problem I'm running into is that I need access to the scoverage jar to conditionally append it to this list. What I haven't seem examples of yet in bazel rules is how to dynamically access a dependency from within an implementation. My guess is that there's something fundamentally wrong with me asking how to do that because bazel needs to run though the analysis phase first before I have the context. That said, pointers are welcome!

• identifying and value for and conditionally appending scalac options for scalac plugin arguments

This is related to the above. I'll need to provide the scoverage enabled compile run -P:scoverage:dataDir:path/to/scoverage-data scalac plugin arg via the scalacopts list. This should be more straight forward but I think I'll need some flavor of make var eval for getting the location of the test-logs directory for the scala_library. I'm not yet sure where the conventional location to put these files but test-logs somehow seems appropriate.

[softprops]

The answer came to me on the train ride home. I can just declare these as private ( _ ) attr dependencies for rules that depend on _compile. The intuition being that rules can have dependencies that they do not always use

[softprops]

I put together an initial prototype that was intended as a pt 1 of the process to have scoverage record starting measurements here One part I'm getting stuck on is the directory that scoverage should write data to

I'm currently using a subdirectory of testlogs

scoverage_data_dir = "bazel-testlogs/{pkg}/{name}/scoverage-data".format(
   pkg=ctx.label.package,
   name=ctx.label.name
)

This doesn't quite work and I'll allude to why

bazel coverage --test_output streamed //test:CalculatorTest
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time.
INFO: Using default value for --instrumentation_filter: "//test".
INFO: Override the above default with --instrumentation_filter
INFO: Found 1 test target...
INFO: From scala //test:Calculator:
[info] Cleaning datadir [bazel-testlogs/test/Calculator/scoverage-data]
[info] Beginning coverage instrumentation
[info] Instrumentation completed [1 statements]
[info] Wrote instrumentation file [bazel-testlogs/test/Calculator/scoverage-data/scoverage.coverage.xml]
[info] Will write measurement data to [bazel-testlogs/test/Calculator/scoverage-data]
INFO: From scala //scala/support:test_reporter:
[info] Cleaning datadir [bazel-testlogs/scala/support/test_reporter/scoverage-data]
[info] Beginning coverage instrumentation
[warn] Could not instrument [EmptyTree$/null]. No pos.
[info] Instrumentation completed [458 statements]
[info] Wrote instrumentation file [bazel-testlogs/scala/support/test_reporter/scoverage-data/scoverage.coverage.xml]
[info] Will write measurement data to [bazel-testlogs/scala/support/test_reporter/scoverage-data]
An exception or error caused a run to abort. This may have been caused by a problematic custom reporter.
java.io.FileNotFoundException: bazel-testlogs/scala/support/test_reporter/scoverage-data/scoverage.measurements.1 (No such file or directory)
	at java.io.FileOutputStream.open0(Native Method)
	at java.io.FileOutputStream.open(FileOutputStream.java:270)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
	at java.io.FileWriter.<init>(FileWriter.java:107)
	at scoverage.Invoker$$anonfun$1.apply(Invoker.scala:42)
	at scoverage.Invoker$$anonfun$1.apply(Invoker.scala:42)
	at scala.collection.concurrent.TrieMap.getOrElseUpdate(TrieMap.scala:901)
	at scoverage.Invoker$.invoked(Invoker.scala:42)
	at io.bazel.rules.scala.JUnitXmlReporter.<init>(JUnitXmlReporter.scala:45)

note: dispite the fact that scoverage says it wrote the initial coverage.xml file it doesn't actually exist

[info] Wrote instrumentation file [bazel-testlogs/test/Calculator/scoverage-data/scoverage.coverage.xml]

An the reason is test logs are attributed to test packages not library packages. Looking for pointers on an alternative approach for picking this directory.

I did notice that the JUnit xml reporters using and env var to know where to record data. I'm not sure if there's a similar feature for coverage writers. I don't think that skylark rules can access env vars if there was.

[softprops]

status update. I updated my branch with a change that partially addresses the coverage dir problem. I realized that the action that writes the coverage data needs to own the process of creating the directory so I ended up threading the coverage dir as a new CompileOption to let the ScalacProcessor create the path. That gets me the ability to create the instrumentation data

ls -al bazel-out/local-fastbuild/bin/test/Calculator-coverage/scoverage.coverage.xml
-rw-r--r-- 1 doug users 523 May 24 18:57 bazel-out/local-fastbuild/bin/test/Calculator-coverage/scoverage.coverage.xml

But I am still unsure how to provide that directory to the test runner which I think I need to write the measurement data to. Let me know what you folks think of my current direction. I'd like to pivot early in the process of I'm going down the wrong direction.

[info] Cleaning datadir [bazel-out/local-fastbuild/bin/test/Calculator-coverage]
[info] Beginning coverage instrumentation
[info] Instrumentation completed [1 statements]
[info] Wrote instrumentation file [bazel-out/local-fastbuild/bin/test/Calculator-coverage/scoverage.coverage.xml]
[info] Will write measurement data to [bazel-out/local-fastbuild/bin/test/Calculator-coverage]
INFO: From scala //scala/support:test_reporter:
[info] Cleaning datadir [bazel-out/local-fastbuild/bin/scala/support/test_reporter-coverage]
[info] Beginning coverage instrumentation
[warn] Could not instrument [EmptyTree$/null]. No pos.
[info] Instrumentation completed [458 statements]
[info] Wrote instrumentation file [bazel-out/local-fastbuild/bin/scala/support/test_reporter-coverage/scoverage.coverage.xml]
[info] Will write measurement data to [bazel-out/local-fastbuild/bin/scala/support/test_reporter-coverage]
An exception or error caused a run to abort. This may have been caused by a problematic custom reporter.
java.io.FileNotFoundException: bazel-out/local-fastbuild/bin/scala/support/test_reporter-coverage/scoverage.measurements.1 (No such file or directory)
	at java.io.FileOutputStream.open0(Native Method)
	at java.io.FileOutputStream.open(FileOutputStream.java:270)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
	at java.io.FileWriter.<init>(FileWriter.java:107)
	at scoverage.Invoker$$anonfun$1.apply(Invoker.scala:42)
	at scoverage.Invoker$$anonfun$1.apply(Invoker.scala:42)
	at scala.collection.concurrent.TrieMap.getOrElseUpdate(TrieMap.scala:901)
	at scoverage.Invoker$.invoked(Invoker.scala:42)

[johnynek]

I assume there is some env-var similar to test where we are supposed to write the file, and in a particular format, but I really haven't looked. I would google the bazel google group and see the story there.

[softprops]

Following up but still blocked. I posted the bazel discuss list to ask for some direction/input but haven't gotten as much feed back as I've gotten here. Consider this a repost of my current state.

I'm trying to follow along with changes that were brought into 5.0 that may help me get code coverage working for scala.

My current blocker is knowing where to write coverage data to.

If I'm reading this right, this should be the block of code that exposes set of COVERAGE_* environment variables to test runs. I can see that coverage is enabled by inspecting ctx.configuration.coverage_enabled when I run bazel coverage and I believe that I'm setting the right instrumented_files field in the struct scala_library returns, but it's not immediately clear why coverage data may be null, which may be why I'm not seeing those env variables with I run bazel coverage -s

If anyone here has time to take a quick look at what I have so far, below is our diff of rules_scala

https://github.com/meetup/rules_scala/compare/master...scoverage-support

[ittaiz]

Not that familiar with coverage but from my experience the Bazel people monitor the Bazel tag in SO more than the discuss group. Maybe posting there will yield faster results. On Tue, 30 May 2017 at 21:20 doug tangren [email protected] wrote:

Following up but still blocked. I posted the bazel discuss list https://groups.google.com/d/msg/bazel-discuss/pJQBD-vWaVo/1K5txXhWCAAJ to ask for some direction/input but haven't gotten as much feed back as I've gotten here. Consider this a repost of my current state.

I'm trying to follow along with changes that were brought into 5.0 that may help me get code coverage working for scala.

My current blocker is knowing where to write coverage data to.

If I'm reading this right, this https://github.com/bazelbuild/bazel/blob/1fd27bc4d97533031401c54aa05eb75b13c6874b/src/main/java/com/google/devtools/build/lib/rules/test/TestRunnerAction.java#L427-L435 should be the block of code that exposes set of COVERAGE_* environment variables to test runs. I can see that coverage is enabled by inspecting ctx.configuration.coverage_enabled when I run bazel coverage and I believe that I'm setting the right instrumented_files field in the struct scala_library returns, but it's not immediately clear why coverage data may be null https://github.com/bazelbuild/bazel/blob/1fd27bc4d97533031401c54aa05eb75b13c6874b/src/main/java/com/google/devtools/build/lib/rules/test/TestRunnerAction.java#L559-L561, which may be why I'm not seeing those env variables with I run bazel coverage -s

If anyone here has time to take a quick look at what I have so far, below is our diff of rules_scala

meetup/[email protected] https://github.com/meetup/rules_scala/compare/master...scoverage-support

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/bazelbuild/rules_scala/issues/184#issuecomment-304964598, or mute the thread https://github.com/notifications/unsubscribe-auth/ABUIF-iz-48pNg58JGjWg7abgHH2I_ubks5r_F4IgaJpZM4NG3kM .

[softprops]

Thanks @ittaiz linking here for reference

[dsilvasc]

FYI, here's how pants integrates with jacoco for code coverage: https://github.com/pantsbuild/pants/commit/a6437101d9758172307df4084b8512faa09d483f

[johnynek]

http://www.jacoco.org/jacoco/trunk/doc/agent.html

documentation on using the java agent. Seems like it might not be that much work.

[softprops]

Does jacoco work for scala as well? At Meetup I know some engineers use jacoco specifically for java and scoverage for everything else.

Any new news on this enabling this would be great! I kind of dropped my initial effort of getting scoverage to work with bazel in favor of letting a jenkins job using sbt to generated reports be our answer for coverage as we moved our primary building over to bazel. My original scoverage branch for this scala_rules is likely horrible unmergable with this master branch by now but I still have it here for reference.

[johnynek]

It looks like Jacoco works on byte code unless I’m wrong. So it may work for both java and scala.

[johnynek]

I think maybe the only open issue is converting to lcov format, which I think is what bazel is standardizing on.

[hchauvin]

I have a workable draft for code coverage for Kotlin which seamlessly integrates with how Bazel collects code coverage for Java (https://github.com/bazelbuild/rules_kotlin/pull/52/commits/fd27f16b2e2c76a1eeb83f0c7730b827d4b23f42), I think a similar approach could be used here.

• Jacoco is used, as with the native Java rules, in offline instrumentation mode. That means that the bytecode itself is instrumented, so this step should work for any JVM language.
• For JUnit-based tests, the BazelTestRunner is replaced with the JacocoCoverageRunner (com.google.testing.coverage.JacocoCoverageRunner). The JacocoCoverageRunner is a wrapper that collects coverage info when the JVM exits and write it in LCOV format, and it invokes the BazelTestRunner. I think that by setting JACOCO_MAIN_CLASS in the java_stub to io.bazel.rulesscala.scala_test.Runner, you should be able to have code coverage collection for your custom runner as well.

These steps work for Kotlin. I don't know much about the Scala toolchain, but I think these steps should work here as well with only a few changes.