Testing trait with concrete impl from another crate

[qberkenbilt]

I have a library that has structs with generic types that are constrained by traits. These can't be trait objects because they have associated functions and async functions, so I use generics instead. I have concrete implementations of the traits in a different crate because I want to keep the library agnostic about how the traits are implemented. My specific use case is that I need my library to be async runtime agnostic, and I have certain traits like Locker and Sleeper that wrap tokio but could be easily made to wrap other things.

The issue is this:

• A create base defines the traits and includes code with generics that use the traits
• A crate impls includes concrete implementations of the trate
• It's hard to test the code in base without concrete implementations.

There are a few things I've tried:

• Create a third trait with the tests that depends on both base and impls and instantiates things in base with concrete implementations from impls. This works, but it makes it necessary to make things "public for test", which is an anti-pattern I'm happy to be able to usually avoid in rust.
• While I can't create a dependency cycle between base and impls, it works to have impls depend on base and base declare a dev-dependency on impls. This allows me to use some things from impls in the tests in base, but here's the problem: *If a concrete type in impls implements a trait in base, test code in base doesn't see the trait implementation even with all the right imports. The reason is that the constraints in base want the concrete class to implement create::SomeTrait, but the concrete implementation appears to implement base::SomeTrait instead. At least, I think this is what's happening. These are actually the same thing, but rust doesn't seem to realize it.

Here's a compete example:

top-level Cargo.toml

[workspace]
resolver = "2"
members = [
    "base",
    "impls",
]

base/Cargo.toml:

[package]
name = "base"
version = "0.1.0"
edition = "2021"

[dev-dependencies]
impls = { path = "../impls" }

impls/Cargo.toml:

[package]
name = "impls"
version = "0.1.0"
edition = "2021"

[dependencies]
base = { path = "../base" }

base/src/lib.rs:

pub trait Adder {
    fn add(a: i32, b: i32) -> i32;
}

pub fn add<T: Adder>(a: i32, b: i32) -> i32 {
    T::add(a, b)
}

#[cfg(test)]
mod tests {
    use super::*;
    use impls::AddImpl;

    #[test]
    fn test_add() {
        assert_eq!(add::<AddImpl>(2, 3), 5);
    }
}

impls/src/lib.rs:

use base::Adder;

pub struct AddImpl;
impl Adder for AddImpl {
    fn add(a: i32, b: i32) -> i32 {
        a + b
    }
}

The problem:

error[E0277]: the trait bound `AddImpl: Adder` is not satisfied
  --> base/src/lib.rs:16:26
   |
16 |         assert_eq!(add::<AddImpl>(2, 3), 5);
   |                          ^^^^^^^ the trait `Adder` is not implemented for `AddImpl`
   |
help: this trait has no implementations, consider adding one
  --> base/src/lib.rs:1:1
   |
1  | pub trait Adder {
   | ^^^^^^^^^^^^^^^
note: required by a bound in `add`
  --> base/src/lib.rs:5:15
   |
5  | pub fn add<T: Adder>(a: i32, b: i32) -> i32 {
   |               ^^^^^ required by this bound in `add`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `base` (lib test) due to 1 previous error

I'm not sure if I'm doing some goofy mistake that is preventing rust from recognizing that the trait is actually implemented. In this case, everything is imported, so it's not that I've forgotten to import the trait.

Alternatively, I may need a different way of laying things out. Perhaps I can keep the concrete implementations in the original trait and hide them behind feature flags. Anyway, suggestions are welcome.

[kpreid]

@qberkenbilt

While I can't create a dependency cycle between base and impls, it works to have impls depend on base and base declare a dev-dependency on impls. This allows me to use some things from impls in the tests in base, but here's the problem: *If a concrete type in impls implements a trait in base, test code in base doesn't see the trait implementation even with all the right imports. The reason is that the constraints in base want the concrete class to implement create::SomeTrait, but the concrete implementation appears to implement base::SomeTrait instead. At least, I think this is what's happening. These are actually the same thing, but rust doesn't seem to realize it.

They are different things. When you create this dev-dependency, you end up with three crates compiled:

• base
• impls
• base with cfg(test) enabled, actually running #[test] functions as a test binary.

Tests are always a distinct crate compilation from the regular crate (except for integration test targets, which are always cfg(test)), and the only reason the dev-dependency isn't a circular dependency is because of this separation.

So, you can’t actually do anything with such a dev-dependency that you couldn’t do with a third package or test target, except use another copy of base’s definitions.

In your situation, I would recommend one of these approaches (or a mix):

• Writing some concrete implementations in base that are only used for tests, under the cfg(test) condition. This causes some duplicate code but hopefully not too much, and the impls might be able to have some useful assertions for testing, or some such thing.
• Testing base using public API only — not making things “public for test”, but writing only tests that use the intended public API.

[qberkenbilt]

Thanks as always for your prompt responses. Your attention to this forum is a great service to the community.

I pretty much already understood everything in your answer except for one part, which I'm still not clear on, and that's except use another copy of base's definitions.

I understand that tests are a separate crate and that that's why the dev dependency is not a circular dependency. As I mentioned in my original post, it works to create a third integration test crate, and it also works to call other things in impls from base's tests. The only thing that doesn't work is recognizing that something implements a trait.

If crate A defines a trait T1, and crate B creates a concrete implementation of T1 called T1Impl, a crate C that depends on both A and B can import both the trait and the impl and use T1Impl to satisfy a generic bound T: T1. Except it doesn't work of create C is actually A's tests compiled with #[cfg(test)]. That's the part I don't understand. For example, this works fine:

third/Cargo.toml

[package]
name = "third"
version = "0.1.0"
edition = "2021"

[dependencies]
base = { path = "../base" }
impls = { path = "../impls" }

third/src/lib.rs

use impls::AddImpl;
fn z() {
    assert_eq!(base::add::<AddImpl>(2, 3), 5);
}

#[cfg(test)]
mod tests {
    use impls::AddImpl;

    #[test]
    fn test_add() {
        assert_eq!(base::add::<AddImpl>(2, 3), 5);
    }
    
    #[test]
    fn test_outer() {
        super::z()
    }
}

Here, I'm using AddImpl as an implementation of Adder both in test code and in non-test code.

It also works to create a third trait, say traits:

traits/src/lib

pub trait Adder {
    fn add(a: i32, b: i32) -> i32;
}

and have impls and base both depend on traits (impls doesn't have to depend on base). In that case, test code in base also works fine.

So there's something different about tests in base from some other third crate that depends on both the trait and its implementations.

I think that, for my purposes, the simplest solution that avoids code duplication is to simply move the trait definitions to a separate crate.

So...what's special about the crate that gets created for running test executables that makes this not work? This comes back to my original comment that rust is not recognizing the trait as it appears in the base crate as the same as the one that the implementation in impls implements. You say they are different, but I don't see how that is different from the other two configurations that work. In particular, it's not clear to me whether this is one of the many things that don't work now but should work eventually or whether there is some fundamental thing I'm missing about how tests are compiled into their own crate. But it feels to me like this should work.

[kpreid]

@qberkenbilt

So...what's special about the crate that gets created for running test executables that makes this not work?

The special thing is that the test version of base has a copy of all of the definitions in base/src/lib.rs. You've got two crates compiled from the same sources. It is not that “Rust is not recognizing the trait”, it's that, from the perspective of the test version of base you have two different traits, base::Adder (not directly nameable, but a transitive dependency through impls) and crate::Adder (compiled as part of compiling the tests). These two different traits happen to have been compiled from the same source code files, but they are still different traits defined in different crates, and an implementation can only be an implementation of one or the other, not both.

This problem only comes up when you use the pseudo-circular dev-dependency because as long as you don't do that, when you compile the test version of base, it is completely unaware that the non-test version of base exists — those two versions exist in the separate worlds of “base is being tested” and “base is being used as a dependency” which don't overlap unless you make them.

[qberkenbilt]

Okay, I understand now -- I was missing the nuance of the test crate being compiled from the same sources as opposed to treating the implementation in base as a dependency in the way that a third crate would. Regardless of what should and shouldn't work, it's now clear to me why I'm seeing this behavior. Thanks again for explaining.

To me, this feels like an implementation detail that has this as an unfortunate side effect, but maybe that's because I'm still relatively new around here. Do you think that it would be better if what I'm trying to do actually worked, or do you think that there are some much deeper issues around how testing works that make the existing behavior correct even though it has this effect? I've noticed this happens a lot in rust. As I've started digging through the discussions around certain unstable features, there's just a lot of nuance around making things work right in all cases. As someone who has been coding for 40+ years in many different languages, I find that I spend a lot more time in rust working around things that intuitively feel like they should work but don't actually work. I also notice that there are fewer of these things now than there were in 2022, which is the last time I tried to use rust for "real work." It reminds me of what it was like to code in C++ in 1994. You could do real work in it, but you had to jump through hoops with template instantiation, there were features missing in STL if you could find a compiler that would work with it, everyone built their own smart pointers because there wasn't one in the standard library, there were lots of conflicting commercial class libraries, etc. Anyway, I know this is just musing and speculating at this point. Thanks again for answering my original question.

[kpreid]

@qberkenbilt

Do you think that it would be better if what I'm trying to do actually worked, or do you think that there are some much deeper issues around how testing works that make the existing behavior correct even though it has this effect?

Many libraries depend on the fact that the code being tested has cfg(test) set, and can have extra assertions or methods in those conditions. So they are depending on the "duplicate" compilation, and doing what you proposed would be a breaking change. But, I do think that Rust would benefit from a well-defined "public only to tests" feature, which would let you write the tests you want to write as test targets instead of tests embedded in the library code; they would not experience the duplication.

[qberkenbilt]

Ah, right -- using #[cfg(test)] inside an implementation is a good reason for the current behavior. I actually do this but hadn't connected the dots. (I have a testutil module that's #[cfg(test)] and contains special logging code among other stuff.) It all makes sense now. Thanks again for your help.