Unifying tests and examples boilerplate (dev-dependencies != dependencies)

1

As projects grow I tend to end up in the situation where I have tests and examples that need a substantial amount of helper functions to set things up. I stick this boilerplate stuff in a helpers/mod.rs in both tests and examples. However, there's usually a lot of overlap between the two, and it's kind of annoying to maintain them.

I once tried to put the boilerplate stuff in a dedicated crate, sort of like this:

name = "foo"

[dev-dependencies]
bar = { workspace = true }

(foo is the main library).

name = "bar"

[dependencies]
foo = { workspace = true }

(bar is a library intended to collect things like the contents of the helpers submodule).

That is to say, bar (boilerplate) depends on foo (main library), and foo has a dev dependency on bar.

I didn't get this to work, and I don't know why. (What I think happened was that I (stupidly) assumed it wasn't going to work, and I probably made some other, unrelated, error, and when I saw an error I didn't read it and just assumed it was saying something about a circular dependency).

Well, I just tried this again, and it Just Worked(tm). So all these years I've been doing a lot of extra work for nothing.

Not sure who needs to hear this, but .. actually read the errors, even if you assume you know what it is.

Also, for those who don't know -- you can put your tests/examples boilerplate in a separate crate. :upside_down_face:

2

yeah, this is surprising to many people when they first learned it.

dev-dependencies are NOT dependencies of the lib crate, but rather dependencies of the crates of examples, tests, and benchmarks. so there are no cycles in terms of the build artifacts.

in fact, your example is exactly the same[1] as the example use case as documented in the cargo book, under the "dev-dependency cycles" section. I'd like to quote a warning about unit tests though:

Beware that this can lead to confusing errors. In the case of building library unit tests, there are actually two copies of the library linked into the final test binary: the one that was linked with “bar”, and the one built that contains the unit tests. Similar to the issues highlighted in the Version-incompatibility hazards section, the types between the two are not compatible. Be careful when exposing types of “foo” from “bar” in this situation, since the “foo” unit tests won’t treat them the same as the local types.

  1. even with the name foo and bar, LOL) ↩︎

3

A caution: Some tools won’t work properly with the circular dev-dependency. I believe a recent case I heard of was that rust-analyzer would not correctly interpret either the type mismatches @nerditation refers to, or cfg(not(test)), because it doesn’t actually make two crates (one lib, one test) like Cargo is doing.

For my own complex testing problems, I’ve gone with, when necessary, putting the tests in a separate package — and that package’s library crate can contain the helpers. That is less simple to run, of course, but it avoids the complications of the circular dev-dependency.

Also:

I would strongly recommend avoiding shared helpers in examples. One of the purposes of examples is to give new users code to copy and modify to their own needs, and this is harder if they also have to extract the relevant parts of the shared code.

Think about how you can modify your library API so that it doesn’t need the boilerplate to be usable. (Of course, this is not always a reasonable choice.)

4

I don't actually do that for persistent examples. When I run into problems with tests that are non-trivial to debug I tend to copy the test into a example while I debug it, and once the problem is fixed I remove the example. (This is why I wanted to keep the boilerplate in sync between tests and examples, so that moving over tests would be trivial).

For examples that are meant to be examples (persistently), I try to keep them isolated as you suggest.

5

Well, I can share my approach I used to solve a similar problem. I'm building a library, some of it's parts are quite low-level and I suppose that users would like to build something over them. But this low-level parts require real services to work (like postgres or kafka). As long as I also have some higher-level helpers built over them, in order to test it I mock low-level traits using unimock (it can also be mockall or whatever) but I'd like to also share this mocks with the users to avoid them doing the same job. I just share them behind optional feature, so my code becomes something like that:

#[cfg_attr(any(test, feature = "unimock"), unimock::unimock(
    api = EventsConsumerMock
))]
pub trait EventsConsumer: Sized + Sync + Send {}

You can find working example in my historia repo.