What are the rules for `#[cfg(test)]`?

I've only just realized I don't know the precise rules for when #[cfg(test)] is enabled and when it isn't. Obviously cargo test builds the code with the test "feature", and cargo build doesn't. What about cargo bench? Is it possible to configure that at all?

I thought it might be tied to the test/bench compilation profiles, but I don't see anything in the documentation about it.

The reason I'm asking is because I've had an issue on Criterion.rs from someone asking about using code from their #[cfg(test)] modules in their benchmarks and I could sorta see that being useful? I'm not sure if it's even possible without changes to Cargo though.

cfg(test) is enabled when building “unit tests” and “unit benchmarks” found inside your library or binary crate. If you have a #[test] or #[bench] function inside your library crate (src/lib.rs and its submodules) or your binary crate (src/main.rs or its submodules), Cargo will build that entire crate with rustc --test, which enables cfg(test) and produces a “test harness” binary containing all of its unit tests/benchmarks.

However, if you have “integration tests” or “integration benchmarks” that live outside of your main crate (by default, Cargo looks for these in tests/*.rs and benches/*.rs), these are not compiled against a test version of your library crate. Cargo will build your library in its normal (non-test) configuration, then build each integration test/benchmark as a sepate binary crate which links to that library. So library code marked with #[cfg(test)] is not available in integration tests/benches.

Doctests are also compiled like integration tests, linking to a non-test build of your library.

Similarly, if your Cargo package contains both library and binary crates, unit tests in the binary are linked to a non-test build of the library, so they can't use #[cfg(test)] code from the library crate.

5 Likes