Integration tests for binary crates

help
1

AFAICT there is still no good way to write integration tests for binary crates. The options seem to be:

  1. Add a src/lib.rs to the crate root, make src/main.rs a thin wrapper around that, and have the integration tests import the crate as a library. The problem with this approach is that publishing the crate to crates.io will publish the library as well as the binary. That's a problem if the library isn't intended to have a stable API.
  2. Move the integration tests into the src/ directory, building them as unit tests. This is a problem if the code is built differently when in #[cfg(test)] mode. For example, my crate uses mock objects in test mode. I want my unit tests to use the mocks, but my integration tests musn't.
  3. Create a workspace. Move most of the crate's functionality into subcrates built as libraries, and write integration tests for those. This is the approach taken by the ripgrep crate. But it has the same problems as option 1 with regards to publishing.

Is there any solution that would allow me to test my code both with and without mock objects, but not publish a library to crates.io? The only idea I have is ugly: create a mocks feature flag and have one set of unit tests that only run with mocks enabled, and another set that run with mocks disabled.

5 Likes
2

I just write integration tests that invoke the binary itself, then I read stdout/stderr and assert something about the output. While I don't use them, there are crates like assert_cmd that streamline this for you.

2 Likes
3

I forgot to mention: my program is built in multiple layers and I can't get fast and comprehensive test coverage by executing the binary alone. That's why I have integration tests that target each layer.

It seems like the simplest solution would be a way to build private libraries that don't get published, and can be used only by the crate's own binaries. They could be separate crates in the same workspace, or not. I could work with either.

1 Like
4

I use the lib+bin approach. If the library isn't properly documented, nobody will use it anyway :slight_smile:

5

So you publish a library that you don't expect anybody to use? Is there a naming convention for that, like <binary_name>-lib?

6

A crate (technically a package) on crates.io can contain both binary and library at the same time, so there's only one thing published.

e.g. these are both library and binary packages:

7

Ahh, of course. How stupid of me. Do you attempt to document the binary on docs.rs?

8

I assume docs.rs will only be read either by people who want the library interface, or who were sent to the wrong place by Google. I use readme to document the binary.

9

Will this issue be addressed in future Rust?

10

I was inspired by this post and published the test_bin crate for getting the binary in an integration test. I needed it too for a couple of projects.

2 Likes
11

Well that's awkward. My integration test was not run because it was in the wrong folder. Wait until version 0.2.0 :slight_smile:

12

Version 0.2.0 published. It should work now (fingers crossed).