Unable to get decent coverage reporting for simple library on either Codecov or Coveralls

I've been having difficulty getting decent coverage reports for Rust projects on Codecov, so I created a minimal test library with one function (14 lines, plus 4 lines of doc comments), and I wrote some integration tests to cover it completely. When I generate an HTML report locally with grcov like so:

export RUSTFLAGS="-Cinstrument-coverage"
cargo build
export LLVM_PROFILE_FILE="target/coverage/prof/%p-%m.profraw"
cargo test

grcov \
        --source-dir . \
        --binary-path target/debug \
        --branch \
        --excl-start 'mod tests \{' \
        --ignore 'tests/*' \
        -t html \
        -o target/coverage/html \
        target/coverage/prof

then I get a reasonable-looking report showing 100% coverage. However, if I do these same steps in GitHub Actions, this time generating an LCOV report, and upload that to either Codecov or Coveralls, then Codecov lists the coverage as 23.90%, and Coveralls lists the coverage as 8% (!).

If I replace grcov with tarpaulin, the final output from the tarpaulin command clearly states "100.00% coverage, 10/10 lines covered", but if I upload an XML report to Codecov or an LCOV report to Coveralls, then these sites report 27.27% and 26.37% coverage, respectively.

Just what is going on, and what do I have to do to get either Codecov or Coveralls to report my test coverage accurately?

For reference, here are the GitHub Actions workflows for doing the testing and coverage reporting:

I recommend using cargo-llvm-cov, which uses Rust's native coverage tools via LLVM. It can be done in CI without too much difficulty.

3 Likes