How to make a module public only for benchmarks?


#1

I have a utility module that’s best left private: it’s not apropos to my crate and I want to be able to change the internal API without having to do a version bump. Trick is, I also want to be able to benchmark this utility module. Benchmarks – unless I’m much mistaken – only work with the public interface of a crate.

How should I go about exposing my utility module as public only for benchmarking? I thought maybe I’d use a feature flag but I couldn’t tell if that was a hack or not.


#2

Could you make the relevant interfaces pub(crate)? As long as your benchmarks can stay in the crate, it would be enough visibility, and not too much.


#3

Benchmarks can work against a private interface. Like #[test], you can add #[bench] to a function that is in a private module.

The trick is that the bencher itself isn’t stable, so you may need to hide it with #[cfg(feature = …)] or such.


#4

Maybe I’m misunderstanding how to keep the benchmarks in the crate. Here’s what I have so far:

https://github.com/postmates/quantiles/blob/ckms_insertion_perf_improvement/src/lib.rs#L30

Which, this works but pollutes my docs, as discussed. The benchmarks sit in a top-level benches/ and don’t work properly if I pub(crate) the utility module.


#5

So it seems that benchmarks which are located in a top-level directory are considered as living in a separate crate after all… I guess it can make sense, since your users will be looking at them when figuring out how to use your crate.

In this case, here are some alternative ideas:

  • Keep your current project structure, but mark the pub mod util; statement as #[cfg(bench)]. This way, it will only be public when someone configures your crate for benchmarking.
  • Follow @kornel’s advice and reorganize your project. These benchmarks are really like unit tests, in the sense that they target a small part of the implementation that should not be exposed in a public interface. Therefore, they should be put in the module under test rather than in a “public” top-level directory.