coreutils icon indicating copy to clipboard operation
coreutils copied to clipboard

`instrument-coverage` doesn't quite work

Open jtracey opened this issue 3 years ago • 0 comments

One of the things left to future work in #3396 was code coverage for Android. As described in https://github.com/uutils/coreutils/pull/3396#issuecomment-1100719518, there were some issues preventing that at the time. One of the main points is that Rust on Android is installed via a package manager rather than rustup, so does not have access to nightly. However, as mentioned in that comment, stable rust does now support -C instrument-coverage, which is apparently supposed to be more accurate anyway.

I was able to get this working on Android with approximately the following:

echo "building objects"
objects=$(RUSTFLAGS="-C instrument-coverage" \
            cargo test --no-run --features "$features" --message-format=json \
                | jq -r "select(.profile.test == true) | .filenames[]" \
                | grep -v dSYM -)

echo "running tests"
RUSTFLAGS="-C instrument-coverage" \
         LLVM_PROFILE_FILE="$this_repo/profiles/profile-%m.profraw" \
         cargo test --features "$features" >tests.log

echo "merging profiles"
llvm-profdata merge \
              -sparse \
              "$this_repo/profiles/"*.profraw \
              -o "$this_repo/profiles/combined.profdata"

echo "exporting lcov"
llvm-cov export \
         --ignore-filename-regex='/.cargo/registry' \
         --format=lcov \
         --instr-profile="$this_repo/profiles/combined.profdata" \
         $(for file in $objects; do echo "-object $file"; done) \
         >"$this_repo/profiles/lcov.info"

However, once uploading to codecov, this had the problem of not ignoring #[cfg()]] statements or blank lines. I couldn't see any way of fixing this in the docs for any of the respective llvm tools, so I tried combining it with grcov. I ran basically the commands used currently, just with an extra -b target/debug/ arg, which according to the grcov docs should work. Instead, it just generates an essentially empty lcov.info file (it contains a single line, TN:). I've since confirmed that this isn't unique to Android, I see the same effect on Linux (to reproduce, run the first two or three blocks of the above shell code, with $features and $this_repo replaced or set as needed, then run grcov). I'm not sure whether this is a bug in how I'm trying to generate the coverage file, or a bug in grcov, but any ideas would be appreciated.

jtracey avatar Apr 27 '22 07:04 jtracey