I recently had to collect test coverages for my rust-strconv (and many others). It was not very easy and I had to do some yak shaving to get it working, but as I've done shaving you don't have to!
Important warning: This tutorial would only work in x86 and x86_64 Linux. I don't think it works in OS X and BSD either. (I do think that it is relatively easy to get kcov work in some BSDs like FreeBSD, but I don't have any BSD box.)
Compiling kcov
We will use kcov to get the code coverage. Unfortunately, the old versions (including virtually all packaged versions) of kcov cannot handle the executable from Rust. You have to compile it manually.
You first need to get the basic dependencies: libcurl, libelf and libdw (but no libdwarf). I've confirmed the following command is enough for Ubuntu, use corresponding commands for other Linux distros.
$ apt-get install libcurl4-openssl-dev libelf-dev libdw-dev cmake gcc
Now get the kcov and compile.
$ wget https://github.com/SimonKagstrom/kcov/archive/master.tar.gz
$ tar xzf master.tar.gz
$ cd kcov-master
$ mkdir build
$ cd build
$ cmake ..
$ make
$ sudo make install
This would install kcov into /usr/local/bin by default. You can customize the install path by using cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr .. or so.
If you happen to avoid make install (like me), you should copy both kcov and *.so files from the build/src directory. Shared objects are necessary to work with Rust executables.
Collecting coverage
Once you've got a fresh kcov in the PATH, you can collect the coverage:
$ cargo test --no-run
$ kcov target/cov target/debug/$TEST_EXECUTABLE
target/cov/index.html will give a pretty report like this.
Common troubleshooting:
- If you get 0% coverage in this page, you have an old kcov.
- If you get a seemingly wrong source code in the detailed coverage, you have run a wrong executable.
cargo cleanand retry.
I've made a simple cargo-cov utility that automates this process for you. This will also remove older test executables. No warranties, use at your own risk.
#!/bin/bash
PKGID="$(cargo pkgid)"
[ -z "$PKGID" ] && exit 1
ORIGIN="${PKGID%#*}"
ORIGIN="${ORIGIN:7}"
PKGNAMEVER="${PKGID#*#}"
PKGNAME="${PKGNAMEVER%:*}"
shift
cargo test --no-run || exit $?
EXE=($ORIGIN/target/debug/$PKGNAME-*)
if [ ${#EXE[@]} -ne 1 ]; then
echo 'Non-unique test file, retrying...' >2
rm -f ${EXE[@]}
cargo test --no-run || exit $?
fi
rm -rf $ORIGIN/target/cov
kcov $ORIGIN/target/cov $ORIGIN/target/debug/$PKGNAME-* "$@"
Caveats
kcov only collects the line coverage. There are many other kinds of code coverage metrics that kcov is unable to gather. Aim for better code coverage, but do not obsess about the exact metric.
There are some issues with the line number, in particular with macros. Also, sometimes, you will get uncovered lines which should not matter (e.g. #[should_panic] tests will get uncovered lines after the panic); kcov does not know about Rust after all.
?