`cargo-tarpaulin`: code coverage for Rust

Coverage?

Testing is one of the most important step of code validation. One would argue that an untested code is akin to a rogue program destroying what it can in its way. But what tests the testing? How do you know that your testing infrastructure is indeed simulating most behaviors of your code?

Here comes code coverage, where you run your tests against an introspection-augmented version of your program. It measures how much code was reached on each run and generates reports out of it.

There are different types of coverage, here is a list of some relevant ones:

  • Line coverage: how many lines are reachable
  • Branch coverage: how many condition’s branches (such as if, for, …) are reachable
  • State coverage: how many states of the program are reachable

As you can see, many of these need language-specific knowledge. Turns out that there is a dedicated tool for most. Here we will talk a bit more about Rust specifically.

Rusty tools

There exists a bunch of ways to gather code coverage in Rust. It is evolving rapidly as more and more people are using it.

  • kcov uses debug annotations similiarly to good old gcov. Sadly, it doesn’t support forking tests which is how I design many of my test suites. It is also bad at differentiating what is generated by the compiler and what is our code.
  • tarpaulin uses ptrace for instrumentation and only works on x86-64. This might look limiting but many testing machines out there are running this setup. It is also the easiest to install and use as it is a cargo plugin.
  • LLVM coverage uses the LLVM backend to generate the coverage. It requires nightly but is the official way to do it.

For now, let’s focus on tarpaulin.

Tarpaulin

You just need to install it via cargo and run it to generate a report

cargo install cargo-tarpaulin
cargo tarpaulin --out html

I do recommend the HTML output so that you can open it and click-walk in your code hierarchy. That’ll rebuild your project with some new RUSTFLAGS aimed at instrumentation. Now you can move around in your sources, and see if each line is hit or not. Try to get each to be green!

—

tharvik