How to make a module public only for benchmarks?

I have a utility module that's best left private: it's not apropos to my crate and I want to be able to change the internal API without having to do a version bump. Trick is, I also want to be able to benchmark this utility module. Benchmarks – unless I'm much mistaken – only work with the public interface of a crate.

How should I go about exposing my utility module as public only for benchmarking? I thought maybe I'd use a feature flag but I couldn't tell if that was a hack or not.

Could you make the relevant interfaces pub(crate)? As long as your benchmarks can stay in the crate, it would be enough visibility, and not too much.

Benchmarks can work against a private interface. Like #[test], you can add #[bench] to a function that is in a private module.

The trick is that the bencher itself isn't stable, so you may need to hide it with #[cfg(feature = …)] or such.

Maybe I'm misunderstanding how to keep the benchmarks in the crate. Here's what I have so far:

https://github.com/postmates/quantiles/blob/ckms_insertion_perf_improvement/src/lib.rs#L30

Which, this works but pollutes my docs, as discussed. The benchmarks sit in a top-level benches/ and don't work properly if I pub(crate) the utility module.

So it seems that benchmarks which are located in a top-level directory are considered as living in a separate crate after all... I guess it can make sense, since your users will be looking at them when figuring out how to use your crate.

In this case, here are some alternative ideas:

  • Keep your current project structure, but mark the pub mod util; statement as #[cfg(bench)]. This way, it will only be public when someone configures your crate for benchmarking.
  • Follow @kornel's advice and reorganize your project. These benchmarks are really like unit tests, in the sense that they target a small part of the implementation that should not be exposed in a public interface. Therefore, they should be put in the module under test rather than in a "public" top-level directory.
1 Like