Of Architecture, Traits and Unit Testing

I'll start by saying I'm new to Rust, but not to programming. I've done a lot of Java development, and that almost certainly colours how I think about things, so bear with me :slight_smile:

I'm working on an application, in Rust, with a layered architecture. Specifically I want to have something like:

Get User Handler -> Users Service -> Users Repository -> Database
Update User Handler -> Users Service -> .... 

This is all relatively straightforward to make work. But I want to do it in a way that is easy to unit test. That means I want to be able to unit test the "Users Service" without needing a real "Users Repository", since a real "Users Repository" would then need a "Database".

So, in the Java world that would mean to use Interfaces, so I've assumed that in the Rust world I should use Traits. I can't see a better choice right now. And that means that I've got:

trait UserRepository {}
struct PostgresUserRepository {}
impl UserRepository for PostgresUserRepository {}

trait UserService {}
struct UserServiceImpl {}
impl UserService for UserServiceImpl {}

Except that, in Rust, traits need to be boxed in some form to be stored in other structs. And then, because this is a webapp and thus multi-threaded, they need to be Sync+Send and so I've ended up using Arc everywhere:

trait UserRepository {}
struct PostgresUserRepository {
    database: Arc<Database>,
}
impl UserRepository for PostgresUserRepository {}

trait UserService {}
struct UserServiceImpl {
    repository: Arc<UserRepository>,
}
impl UserService for UserServiceImpl {}

#[get("/users/<user_id>")]
pub fn get_user_by_id(user_id: String, user_service: State<Arc<UserService>>) -> Json<User> {}

This then means that to get from my handler to my database I'm going through three layers of Arc. And that just feels wrong.

Is this the correct/best way to architect this? Or is there some better alternative that I've not yet found?

Cheers

2 Likes

Box<MyTrait> isn't Send/Sync, because it could be any implementation, not just the ones that meet that requirement. If you want to constraint the trait object to also be Send/Sync, add it to the type:

trait UserRepository {}
struct PostgresUserRepository {
    database: Box<Database + Send + Sync>,
}
impl UserRepository for PostgresUserRepository {}

See Sending trait objects between threads - #3 by Gankra for a bit more info.

Hi,

First of all - you don't want to use dyn Trait (Arc<Trait> should be rewritten to Arc<dyn Trait> here) unless you actually need some dynamic dispatch. In most cases, what you want to do, is a static dispatch:

struct UserServiceImpl<UserRepository> {
  repository: UserRepository,
}

#[get("/users/<user_id>")]
pub fn get_user_by_id(user_id: String, user_service: impl UserServiceImpl) -> Json<User> {}

Also what you never ever want to do in Rust, is to create trait to have only one actuall implementator of this trait just to add level of abstraction. In general - OOP, and the Java driven OOP is pattern which doesn't makes things easy, and doesn't fit Rust idioms at all. Imitating Java interfaces with Rust traits is in best case very bad idea.

Also you don't want to use Arc if you are not sharing ownership of an object between threads. Most likely what you need is bounded Box as @17cupsofcoffee mentioned.

Do you have any actual reason to do such layered design for your API?

1 Like

I don't disagree with this statement, but I'm interested how you would approach testing for an object like UserServiceImpl - without a way of mocking out the database layer, I feel like that would be quite tricky?

Also, with regards to using generics to solve this problem - it's also worth noting that you can set a default type for a generic type parameter:

struct UserServiceImpl<UserRepository = PostgresUserRepository> {
  repository: UserRepository,
}

Mock is another implementator of a trait, so in this case doesn't apply - I didn't say "only one public release implementator". Actually it was more about trait UserService which looks like being only thin wrapper over context - even if it encapsulates some trivial logic, I assume it is easy enough to mock its fields. I may be wrong here, so that's why I just gave it as side note, not strong statement about specific code.

In general I would not create PostgressUserRepository or similar at all, unless they have anything besides DB client - I would implement my trait directly for the Database type. Even if it would be the only implementator - it wouldn't be "just to add level of abstraction" - it would be extending functionality avoiding additional level of abstraction. I could also add my implementation for some trivial storage like HashMap (so testing would be trivial) - even gated behind #[cfg(test)]. The only reason to create additional in-between structs like this is case, when both trait and struct are in foreign crates so additional adapter is needed.

Ah, I see what you mean now. That makes a lot of sense :slight_smile:

I'm coming from Java too and I was wondering the same thing. The best thing I've come up with is that if I can't do OOP easily, maybe try FP? So essentially use closures, for example to pass it from GetUserHandler to UsersService. In test (while testing GetUserHandler) this closure would do something different.

I do want to have multiple implementations though. I want to have the real one that the real code uses, and then an alternative mock one (of some form - I've not got that far yet!) for the tests to use. Otherwise in order to unit test UserService I ultimately need a real running Postgres database, and that's overkill!

As to why I want a layered architecture. Maybe it's wrong for Rust, but what I want is:

  • UserRepository - understands how to get data in and out of the database. Does not understand business rules. This would be things like "Get User", "Save User".
  • UserService - understands business rules in terms of a Repository. This can then handle things like "Is this email address unique?" and "Is this password valid for this username?"
  • get_user_by_id - This is a single REST handler that works in terms of appropriate services.

In the future there will also be a UserAuditor that works in terms of a message queue, and which the UserService will use to send appropriate audit events to record what's been happening in the system, but again this is way off :slight_smile:

Basically then, anything that wants to work in terms of Users will interact with the UserService, and will not need to care about how that works.

Ok, now that is an interesting thought that I'd not come up with :slight_smile: I'll have to see how that works now :slight_smile:

Why is that? You cannot do Java patterns like transparently, but in general Rust supports modern OOP. The "problem" is that Java is strongly depending on inheritance on its patterns, which just doesn't fits "zero overhead rule" (and also has other issues which doesn't fits Rust). However in C++ it is long time, since there is a tendency "composition over inheritance", and in general reducing layers of abstraction. In most cases three layers are good enaugh - model layer, logic layer, API layer (which perfectly matches patterns like MVC or MVVM in some way).

@sazzer - seems somehow reasonable. However it still seems like UserService does looks like it is implemented once. And TBH after your explanation I would say, that UserService is maybe just a decorator for your UserRepository? Maybe it shouldn't have its own trait, but should implement UserRepository trait, and while created should take underlying repository it should validate (and maybe filter results?). It very much depends on application logic.

Long story short - it is difficult to point you in idiomatic Rust direction because of way you asked a question. You want to find out how make your Javaish design work in Rust, but the answer is: it won't. The true question is: how to approach designing some service in Rust, but for those question there is too few info. I would say, that good start would be go through your server crate tutorial, also probably The Book and think in term of Rust idioms instead of translating Java idioms you are used to. What you are doing is actually speaking Polish by translating English sentences word by word - you may be understood by some folk who uses both languages, but going to an exam with such attitude doesn't seem like good idea.

1 Like

As a fuller example of what I was thinking - in my Java head :slight_smile::

trait UserRepository {
  fn get_user_by_id(&self, user_id: UserId) -> Option<UserEntity>;
  fn get_user_by_email(&self, email: String) -> Option<UserEntity>;
  fn get_user_by_username(&self, username: String) -> Option<UserEntity>;
  fn save_user(&self, user: UserEntity) -> Result<UserEntity, SaveUserError>;
}
impl UserRepository for Database { ... }

struct UserService {}
impl UserService {
  fn register_user(&self, user: UserData) -> Result<UserEntity, RegisterUserError> {
    if let Some(_) = self.repository.get_user_by_email(user.email) {
      return Err(RegisterUserError::DuplicateEmailAddress);
    }
    if let Some(_) = self.repository.get_user_by_username(user.username) {
      return Err(RegisterUserError::DuplicateUsername);
    }

    let user_entity = UserEntity {
      user_id: Uuid::new_v4(),
      version: 0,
      created: Utc::now(),
      updated: Utc::now(),
      data: user,
    }
    self.repository.save_user(user_entity)
      .map_err(|e| .....)
  }
}

So the UserRepository is very much just concerned with accessing the data. Nothing more complicated than that. The UserService is then the bit that:

  • Ensures that email addresses are unique
  • Ensures that usernames are unique
  • Creates a new, unique ID for a new user
  • Assigns a Created Date and Last Updated Date for the user
  • Persists the user

That's fair. And it's a maddeningly easy trap to fall into. Naively it seems like it would work, and then ends up feeling wrong :slight_smile:

1 Like

This one actually seems very Rusty - UserService seems like actual logic. What I would do here I would add generic type for UserService, so it is easy to substitute it:

struct UserService<Repo> {
    repository: Repo,
}

impl<Repo: UserRepository> UserService {
  fn register_user(&self, user: UserData) -> Result<UserEntity, RegisterUser> {
    // ...
  }
}

What can be improved here is error reporting - if you don't use a value in option, then use Option::is_some() instead of decomposing it (clippy would tell you about it).

What may be considerable here, would be going way like this:

impl<Repo: UserRepository> UserRepository for UserService<Repo> {
  fn save_user(&user: UserData) -> Result<UserEntity, RegisterUserError> {
    if let Some(_) = self.repository.get_user_by_email(user.email) {
      return Err(RegisterUserError::DuplicateEmailAddress);
    }
    if let Some(_) = self.repository.get_user_by_username(user.username) {
      return Err(RegisterUserError::DuplicateUsername);
    }

    self.repository.save_user(user)
  }
}

But it depends on actual UserService responsibilities (is it just repository proxy or does some more logic?). It would also push the responsibility of UserEntity creation to an actual repo implementation, and also require errors commonization (between those two types) which may or may not be a good idea. You mentioned there would be some kind of UserAuditor later, which may be either another decorator for repo access, or part of UserService, or maybe completely distinct thing - it would actually define which way I would go with my design.

So this would mean that repository doesn't need to be boxed, but must still implement the trait? That seems like a decent improvement :slight_smile:

I take it my handler then works in terms of UserService<Database> and my unit tests are testing UserService<SomeMockUserRepoository>?

I literally wrote this in the forum post. I've not done any of this yet :slight_smile:

My current thinking was that UserRepository is internal to the users module, and nothing outside of there can access it. UserService is the public API that other components work in terms of. When the auditing comes along, the UserService.register_user() call would then send a "User Registered" event, whereas the UserService.update_user() call would instead send a "User Updated" event. In both cases though, the UserRepository is only caring about loading and saving data from the underlying data store.

Thats exactly what I mean.

Sure, just giving a hint.

I don't see any reason why UserService should not do what UserAgent would. If I understand you correctly it is just forwarding calls, don't see any reason to introduce UserAgent. If you want to have different user agents, then I dont see a reason, why UserService would not be just a trait implemented for user services. If the reason is that you don't want to expose user service to the api, it is not a problem, just return them via impl Trait:

pub trait UserAgent {
  // ...
};

struct StandardUserService<Repo> {
  repo: Repo,
}

impl<Repo: UserRepository> UserAgent for StandardUserService<Repo> {
  //...
}

struct SpecialUserService;

impl UserAgent for SpecialUserSerivice for SpecialUserService {
  // ...
}

pub fn create_std_user_agent() -> impl UserAgent {
  StandardUserService::new()
}

pub fn create_special_user_agent() -> impl UserAgent {
  SpecialUserService::new()
}

A general rule is that struct/enums are for defining data, not behaviour. Creating type just to add some behavior is in general missuse (except several cases which are mostly workarounds over language limitations). If you want to introduce some behavior, use trait. If you want to take an argument which data doesn't matter, but has to implement some behavior - use impl Trait. If you want some data implementing some behavior but don't want to expose actual data - use impl Trait.

2 Likes

So - it works. But... :slight_smile:

Because I'm using Rocket, I've got to have real types and not traits passed to the handlers. i.e. this doesn't work:

#[get("/users/<user_id>")]
pub fn get_user_by_id(
    user_id: String,
    user_service: State<dyn UserService + Send + Sync>,
) -> Json<User> {

because:

8  | / pub fn get_user_by_id(
9  | |     user_id: String,
10 | |     user_service: State<dyn UserService + Send + Sync>,
11 | | ) -> Json<User> {
...  |
22 | |     Json(user)
23 | | }
   | |_^ doesn't have a size known at compile-time

If instead I make it user_service: State<UserServiceImpl<Database>> then it all works correctly, but suddenly my handlers at the very top of the stack are tied to my repository implementation at the bottom of the stack. I.e. if I ever did have a different type of Repository to use - e.g. a CachingUserRepository that wraps both Postgres and Redis - then all my handlers need to change. That's also the case if UserServiceImpl gains additional generic types for other repositories - e.g. the message queue, or a Redis cache.

And the Java developer in me (I know - Sorry!) balks a bit at the lack of abstraction there.

The other problem is that I'm now needing to close the Database for each service. However, I've made it so that I can clone it, and the r2d2 pool inside is now wrapped in an Arc so that it all works correctly - i.e. even though every service has a different cloned instance of Database, the actual connection pool is alway the same. I'm assuming I should be able to do that with borrows instead of clones, but I couldn't (yet) find a way to make that work...

1 Like

Well you could perhaps do this?

#[get("/users/<user_id>")]
pub fn get_user_by_id(
    user_id: String,
    user_service: State<RealUserService>,
) -> Json<User> {
    get_user_by_id_real(user_id, user_service)
}
fn get_user_by_id_real<US: UserService>(
    user_id: String,
    user_service: State<US>,
) -> Json<User> {
    ...
}

Then do your mocking on get_user_by_id_real.

Yeah well Java has the ability to do this because they pay the price of boxing all fields in all types (except primitive types). You can do it too with user_service: State<Box<dyn UserService + Send + Sync>>, which does have a known size.

As luck would have it I just made a mocking framework that avoid the need to use traits to have a mock.

It is in a pretty early stage but it might be useful to you: GitHub - nrxus/faux: Struct mocking library for Rust.

If you like doing it with traits, there are quite a few frameworks out there to help you mock traits but you will have to either use generics or trait objects as mentioned in other replies. I would recommend static dispatch that way your code won't pay a runtime cost simply for testing purposes.

I don't understand why people insist on this stupid mocking idea.

As OP says "This is all relatively straightforward to make work. But I want to do it in a way that is easy to unit test. That means I want to be able to unit test the "Users Service" without needing a real "Users Repository", since a real "Users Repository" would then need a "Database"."

Yes, you need a database. Guess what the solution is? Install PostgreSQL on your machine and run the tests on it.

Otherwise, you are wasting tons of time implementing an equivalent, and the tests are useless anyway because you testing something different than what you'll actually use in production.

So, I'm using testcontainers to write tests for the UserRepository that work against a real database. That's easy. It's also heavyweight. But it means it will work anywhere that has Rust and Docker available. E.g. on a CI system that is a clean slate for every single build.

My UserRepository is going to be relatively simple though, and the bulk of the business logic is in UserService. That means I can provide UserService with something that meets the contract of UserRepository, fully unit test all of the business logic there without needing a database for all of it.

Perhaps it's possible to define a type synonym/alias in just one place and use that instead - type composition/configuration is fixed at compile time, not runtime.

trait MyInner {
    fn make(&self) -> String;
}

struct MyType<T>
  where T: MyInner
{
    inner: T
}

impl<T> MyType<T> where
    T: MyInner {
    fn make(&self) -> String {
        self.inner.make()
    }
}

struct InnerProd;

impl MyInner for InnerProd {
    fn make(&self) -> String {
        String::from("production")
    }
}

struct InnerTest;

impl MyInner for InnerTest {
    fn make(&self) -> String {
        String::from("testing")
    }
}

// https://doc.rust-lang.org/book/ch19-04-advanced-types.html?highlight=alias#creating-type-synonyms-with-type-aliases
//
type MyProd = MyType<InnerProd>;
type MyTest = MyType<InnerTest>;

// Generic
fn show<T>(value: &MyType<T>) where
    T: MyInner
{
    println!("Showing: {}", value.make());
}

// Non-Generic
fn show_prod(value: &MyProd) {
    println!("Prod: {}", value.make());
}

fn show_test(value: &MyTest) {
    println!("Test: {}", value.make());
}

fn main() {
    let prod = MyType{ inner: InnerProd };
    let test = MyType{ inner: InnerTest };
    show(&prod);
    show(&test);
    show_prod(&prod);
    show_test(&test);
}
$ cargo run
   Compiling alias v0.1.0 
    Finished dev [unoptimized + debuginfo] target(s) in 0.32s
     Running `target/debug/alias`
Showing: production
Showing: testing
Prod: production
Test: testing
1 Like