Idiomatic Rust way of testing/mocking

I have been struggling a lot with a seemingly trivially simple problem. I like doing stuff in TDD, and this time I don't wanna half-ass it, I wanna carry on with it properly - which means, I'm basically stuck at the very beginning.

In my mental model, the application would be structured like this:

  • a library for common functionality
  • a binary for CLI
  • a binary for web

I'd like to focus on the common library in this post. In case it has any relevance, I'm using sea-orm for database interactions and mockall for mocking. For the latter one, I'm very much open to other suggestions if there's anything better than mockall (I haven't found any).

By principles of TDD, I wanted to make it simple and not overcomplicate it, so at first I created a repository:

pub struct BookRepo<'a>
{
    db: &'a DatabaseConnection,
}

impl<'a> BookRepo<'a>
{
    pub fn new(db: &'a DatabaseConnection) -> BookRepo<'a>
    {
        BookRepo { db }
    }

    async fn create(&self, book: book::ActiveModel) -> Result<book::Model, DbErr>
    {
        book.insert(self.db).await
    }

    async fn read<T>(&self, id: Uuid) -> Result<Option<T>, DbErr>
    where T: PartialModelTrait
    {
        book::Entity::find().filter(book::Column::DeletedAt.is_null()).filter(book::Column::Id.eq(id)).into_partial_model::<T>().one(self.db).await
    }

    // there are more methods, just shortened for clarity...
}

So far so good; sea-orm provides an excellent mocking interface for DatabaseConnection, we happy.

Next, I wanted to add a service layer that would potentially use multiple repositories in the future, but for now, only BookRepo, as nothing else is implemented yet.

A few (im)possibilities I discarded right away:

  • Accepting a &DatabaseConnection when initializing the service is a no-no, because that way the service would have to initialize the BookRepo, which leads to a few problems:
    • Service and repository are strongly coupled.
    • Impossible to mock BookRepo. Yes, sea-orm's fancy mocking interface is still available, but it's quite limited - I only can specify what it should return from a query, but I have no way peeking into what it receives as parameters to the query from BookRepo. Also, at this level I want to test the interaction between the service and the repository, so there's no need to always propagate down to the database level in every single test of the entire library.
  • Accepting &BookRepo when initializing the service - this is a move to the right direction, but mockall cannot mock structs. It can only mock trait implementations. Which leads to...
  • ...accepting a trait instead of a concrete type. So I created a BookRepoTrait. Dynamic dispatch is off the table, because apparently my trait implementation is not dyn-compatible because of the generic parameter of the read method. This leaves one option: the service receives a type parameter for the BookRepo implementation, but incredibly enough, this still cannot be mocked.

For clarity, here's the current version of the code:

pub struct BookRepo<'a>
{
    db: &'a DatabaseConnection,
}

pub trait BookRepoTrait
{
    async fn create(&self, book: book::ActiveModel) -> Result<book::Model, DbErr>;
    async fn read<T>(&self, id: Uuid) -> Result<Option<T>, DbErr> where T: PartialModelTrait;
    // there are more methods, just shortened for clarity...
}

impl<'a> BookRepo<'a>
{
    pub fn new(db: &'a DatabaseConnection) -> BookRepo<'a>
    {
        BookRepo { db }
    }
}

impl BookRepoTrait for BookRepo<'_>
{
    async fn create(&self, book: book::ActiveModel) -> Result<book::Model, DbErr>
    {
        book.insert(self.db).await
    }
    
    async fn read<T>(&self, id: Uuid) -> Result<Option<T>, DbErr>
    where T: PartialModelTrait
    {
        book::Entity::find().filter(book::Column::DeletedAt.is_null()).filter(book::Column::Id.eq(id)).into_partial_model::<T>().one(self.db).await
    }
}
pub struct BookService<'a, R: BookRepoTrait>
{
    book_repo: &'a R,
}

impl<'a, R> BookService<'a, R>
where R: BookRepoTrait
{
    pub fn new(book_repo: &'a R) -> Self
    {
        BookService { book_repo }
    }
    
    pub fn add_book(&self, book: BookAdd)
    {
        
    }
}

In my test, if I do

#[automock]
trait BookRepoTrait {
    async fn create(&self, book: book::ActiveModel) -> Result<book::Model, DbErr>;
}

... then I'm getting this:

error[E0277]: the trait bound `MockBookRepoTrait: book_repo::BookRepoTrait` is not satisfied
  --> src/services/book_service_test.rs:30:39
   |
30 |     let service = BookService::new(&mock_book_repo);
   |                   ------------------- ^^^^^^^^^^^^^^^^^^ the trait `book_repo::BookRepoTrait` is not implemented for `MockBookRepoTrait`
   |                   |
   |                   required by a bound introduced by this call
   |
   = help: the trait `book_repo::BookRepoTrait` is implemented for `BookRepo<'_>`
note: required by a bound in `BookService::<'a, R>::new`
  --> src/services/book_service.rs:10:10
   |
10 | where R: BookRepoTrait
   |          ^^^^^^^^^^^^^^^^ required by this bound in `BookService::<'a, R>::new`
11 | {
12 |     pub fn new(book_repo: &'a R) -> Self
   |            --- required by a bound in this associated function

If I do this instead of the #[automock] block:

mock! {
  pub MockBookRepo {}

  impl BookRepoTrait for MockBookRepo {
    async fn create(&self, book: book::ActiveModel) -> Result<book::Model, DbErr>;
  }
}

...then it would expect me to declare all methods in MockBookRepo, not just the one I wanna mock. At which point, it made me think, why do we need a mocking library here then?

So of course I wanted to just define a whole new implementation for BookRepoTrait just for the test, and then it would just count the number of invocations of each method, and maybe capture the arguments passed to its methods. But of course that cannot possibly work, since that would require me to do &mut self instead of &self in the trait - which I absolutely won't do, as it's kinda ridiculous if I have to make my objects mutable just because I wanna test my code.

And here we are! I gave up. As a last resort I'm posting here, maybe someone reads my struggles and has a convenient idea.

The high level "right way of doing things" is to just test your code, don't just mock every possible layer. You do want to mock things which require global or external state, such as IO, and replace them with an in-process isolatable and fast/synchronous option, but traditional style mocking where you say basically "I expect these method calls in this order, producing these results" only really serves to couple the test to the implementation.

If you do want to mock BookRepo, a good option might be something like HashMap<Uuid, BookInfo> that implements the full interface backed by just a simple map.


But to answer the actual questions:

#[automock] failing seems to be a limitation of the library. A weird one, if using mock! instead works. But I'm also not surprised that it doesn't support generic functions; how would you seed the mock with results if you don't know the types? (Generic closures are unfortunately still impossible.)

To record call counts behind &self, you can use internal (shared) mutability such as provided by Cell.

9 Likes

You don't have to...

For simple cases (single threaded), I do often something like this:

use std::cell::Cell;

struct SomeStruct {
    messages: Cell<Vec<String>>,
}

impl SomeStruct {
    fn append_msg(&self, msg: impl Into<String>) {
        let mut msgs = self.messages.take();
        msgs.push(msg.into());
        self.messages.set(msgs);
    }
}

If the thing inside Cell doesn't implement Default, I wrap it into an option:

struct Foo;

struct SomeStruct {
    foo: Cell<Option<Foo>>,
}

impl SomeStruct {
    fn something(&self, ...) {
        let mut foo = self.foo.take().unwrap();
        foo.modify();
        self.foo.set(Some(foo));
    }
}
1 Like

Thanks both of you @CAD97, @mroth! From this it's clear to me that mocking libraries are not the way. I'll try this Cell utility, it sounds promising. I'll get back with the results.

This only makes sense to me in relation to integration tests (I'm planning to do that too, with an in-memory sqlite database). However, unit tests are called "unit" tests for a reason. They should be fast and independent, focusing only on one piece of functionality at a time.

Also, what's wrong with coupling the test to the implementation? Could you elaborate that part a little bit please? In my understanding, if the business logic changes, so does the test.

I don't know your background and your overall experience with using rust, so I only can tell what I have encountered during my rusty journey. When I first started using rust I came from mostly OOP stuff using Java (unfortunately this is a very common language in Germany). This was in the pre-1.0 days and there wasn't really much information out there, so after some TDD fighting against rust I accepted that I'm to new to the language and didn't try further. After some years of then mostly JavaScript, Typescript and finally Python where I was doing a lot of TDD I came back to rust and realized that most of the tests you write in these languages are just checking things like input data types, error handling and overall wiring of your app. Most of these can already be checked by the rust compiler, especially if you use the type system to express your intention more explicitly. Long story short, I try to focus my unit testing on basic computation and edge cases, nowadays. In a bigger application I would add some integration and end-to-end tests.
So my recommendation would be to focus more on how you structure your code, especially the data model and to learn to use rusts outstanding features to get rid of all those boilerplate tests needed in loosely typed languages.

6 Likes

Off-topic, for things like DB connection you usually wouldn't use temporary references to a local variable, because that makes your whole program tied to some single variable that can't be moved, and the <'a> annotations spread virally everywhere.

Instead of forbidding structs from containing the db connection, make them own it:

pub struct BookRepo {
    db: DatabaseConnection,
}

if you need to share the connection between many objects, use Arc:

pub struct BookRepo {
    shared_db: Arc<DatabaseConnection>,
}

You can combine this with traits, and use Box<dyn YourDatabaseAccessTrait> or Arc<dyn YourDatabaseAccessTrait>. The dyn will allow you to use it with a mocked implementation. Note that Box/Arc have implied + 'static bound, so they won't let you use a temporary reference to a local variable. Make them own the object completely.

Monomorphic generics (arg: T, … where T: Trait, arg: impl Trait) are needed only for very small functions (like vec.len()) or methods that can be called millions of times in loops (like int.saturating_add(1)). For big objects like database handles and API clients, dyn is perfectly fine, and avoids having to spam your code with <T: Trait> bounds.

5 Likes

By principles of TDD you should have written a test first.

Your issue is that you hard-coded the &DatabaseConnection in the repo.
Instead you should use dependency injection, i.e. store a generic that implements some well-defined interface:

pub struct BookRepo<T> {
    db: T,
}

impl<T> BookRepo<T>
where
    T: Database,
{
    ...
}

Then you don't need to mock the database connection, but you can use a stub database that implement the Database trait in your test cases.

That's actually not true. Mockall can do structs, too. See mockall - Rust .

Each function has inputs and outputs specified by its signature and doc -- that's what should be tested. If the function implementation changes but it still meets the specification, the test shouldn't have to change -- ideally (this is not always easy). That's what it means to avoid coupling the test to the implementation.

It isn't worth testing what the Rust compiler already checks, as others have said. But even the most advanced compilers don't catch all logic errors, so unit testing is still needed.

1 Like

Then it's not a unit test, isn't it? It's an integration test by the very definition: it tests API of an external component. The fact that such external component is called “human customer” is irrelevant, it's still something well outside of your code and thus it's an integration test.

And since it's an integration test… treat it as such.

1 Like

Many thanks everyone for the insights!

To my original problem, Cell was the solution in manually written mocks.

This Arc trick is very neat, it made everything a lot more elegant.

This only makes sense to me in relation to integration tests (I'm planning to do that too, with an in-memory sqlite database). However, unit tests are called "unit" tests for a reason. They should be fast and independent, focusing only on one piece of functionality at a time.

Also, what's wrong with coupling the test to the implementation? Could you elaborate that part a little bit please? In my understanding, if the business logic changes, so does the test.

I came into Rust with a similar attitude to this, and also struggled with similar mocking problems to what you're describing.

Then I came across this video (TDD, Where Did It All Go Wrong - Ian Cooper) and it really helped change my mindset to one which is more compatible with Rust:

1 Like

Just an additional general suggestion about mocking: the golden rule is "don't mock types you don't own"

I've heard that advice before, but I have mixed feelings about it. Generally speaking, typically you'd mock stuff like database connections, IO, etc. as someone mentioned before. But most of the time those are external libraries, things you don't own.

On the other hand, yes, in Rust you won't mock stuff you don't own, simply because you can't. At least I've never could make that work.

In my situation here it's all good, because in the repository tests, even though I'm mocking the DatabaseConnection, I'm actually using a mock that is provided by sea-orm, so that mock isn't mine.

One layer above, on the service layer I'm mocking the repository, because I don't want the execution to propagate all the way down to the database layer again, since I'm doing unit tests here, not integration tests. And the repository is mine.

I'm going to ask a hopefully enlightening question: why? You already know that the database layer presumably works because of its own tests, so why is it considered an advantage to avoid exercising that code further in other tests?

To ensure a layer works with the API specification of an injected dependency and doesn't rely on specific implementation choices is one potential reason, but if you don't have a specific reason, then there's no real benefit to mocking out the dependency.

This also illustrates why there's no real need to be mocking concrete types; most mocking benefits only really become a thing where there's a generic point where the mock is injected anyway.

Mock resources, not code.

Also as a note, you can really easily, if inflexibly, mock code in Rust with approaches like:

#[cfg_attr(test, path = "db_mock.rs")]
mod db;

Now all your unit tests (#[test]) will use the implementation in the provided file path, while integration tests (tests/) will use the default path.

You can use more specific config options in theory, but it's probably not worth it.

2 Likes

Well, I doubt you'll find any enlightenment here, since I'm still trying to figure this out, but you just answered it yourself:

Since that part is already covered, why test it again?

Okay, this is only part of the reason. The other thing is, I want to avoid having bloated tests that are exercising a lot more code than necessary.

Maybe it's just the misery talking in me, but in my full-time job I'm a Java developer, and our automated test suite take 20-25 minutes to complete. Those tests are more like integration tests (kinda), and we rarely mock anything. It even uses a real database. It's a nightmare to work with.

So in my current Rust project I'm aiming for a lightweight test suite that completes in seconds even when there are hundreds of tests. Yes, I'm aware that's easily achievable in Rust, even with the most bloated tests, but you know what I mean. I'm striving for maintainability and resource-efficiency.

Probably my approach is still not the best though; manually written mocks can quickly get out of hand. As I said, I'm still trying to figure this out.

And I'm still yet to watch the video suggested by @jamesthurley - I had no time for it yet, sorry about that.

Why?

But that like attempting to move in two opposite directions at once. Resource-efficiency and maintainability very rarely align.

Usually the most maintainable code is not resource-efficient while resource-efficient code is hard to maintain.

Sometimes you can do things that improve both, simultaneously, but that's more of an exception than the rule, you have to celebrate rare moments when that is happening, not expect that to be regular experience.

Mocking every layer is overkill for addressing that, though; that's fixed by mocking the slow layers — typically global and/or external state, that ends up necessitating serialization of tests.

Citing matklad,

In the abstract, yes, more code generally means more execution time, but I doubt this is the defining factor in tests execution time. What actually happens is usually:

  • Input/Output — reading just a bit from a disk, network or another process slows down the tests significantly.
  • Outliers — very often, testing time is dominated by only a couple of slow tests.
  • Overly large input — throwing enough data at any software makes it slow.

The problem with integrated tests is not code volume per se, but the fact that they typically mean doing a lot of IO. But this doesn’t need to be the case.

Rust-analyzer's tests are pretty exclusively end-to-end tests, but its tests still run very quickly. For example, to test "go to definition," the test is just a string of source code with comments marking "apply the intention with the cursor here" and "the intention should produce this span." This is solely to test the "go to definition" feature, but it also exercises basically the whole engine of parsing and type resolution, not just the parts using the analysis engine to serve the IDE intention.

I take the opposite position — without an extenuating reason, more test coverage of the code that you're actually going to be running in production is a good thing. I've also spent practically as much time debugging faulty tests as I have debugging actual issues caught by tests.

"Testing is slow because it's doing too much" is indeed a valid reason to replace service providers with simpler, faster versions for testing. But I find that mocking and other common TDD conventions often are treated more like a dogma than a tool to be applied when it makes sense and not when it doesn't.

1 Like

I worked on a database engine where we had many end-to-end tests that took a couple hours to run. The time was cut by more than half by changing the transaction flush mode to only write when the buffer is full (not at every commit).

2 Likes