Abstract Factory - Trait with generic method

[spigaz]

I'm trying to implement the Abstract Factory pattern

pub trait EventStore: Debug {
  fn create_stream<T: Codable>(&self, name: String) -> Result<Box<dyn EventStream<T>>, EventStoreError>;
}

The event stream receives events, serializes them and deserializes them using their type information:

#[async_trait(?Send)]
pub trait EventStream<T>: Debug + 'static {
  async fn append(&mut self, events: &[T]) -> Result<(), EventStreamError>;
  async fn events<'a>(&'a mut self) -> Result<&'a [T], EventStreamError>
  where
    T: 'a;
}

A current sample usage is:

pub struct Test<T: Codable> {
  stream: Box<dyn EventStream<T>>,
}

impl<T: Codable> Test<T> {
  pub fn new<E: EventStore>(event_store: E) -> Self {
    Self { stream: event_store.create_stream::<T>("test".to_string()).unwrap() }
  }

  pub async fn has_events(&mut self) -> bool {
    if let Ok(events) = self.stream.events().await {
      events.len() > 0
    } else {
      false
    }
  }
}

And it works pretty well, as long as I use it that way, the problem is if I try to use it as an Rc:

  pub fn new(event_store: Rc<dyn EventStore>) -> Self {
    Self { stream: event_store.create_stream::<T>("test".to_string()).unwrap() }
  }

error[E0038]: the trait `EventStore` cannot be made into an object

I think I understood well enough why this approach isn't supported, but I still believe there should be a smarter way to implement this in rust.

The implementations of the Trait are dependency inject as they are supposed to be, so those enum dispatch strategies variants, I can't use.

I have tried several approaches using impl dyn, but I ended always loosing the type that I need in the concrete implementation to create the proper EventStream.

I appreciate all the help I can get, because for sure I'm missing something.

Thank you!

[sirius-cc-wu]

What do you mean with 'try to use it as Rc'? Have you an example?

[Fancyflame]

The information is too little, it's better if I know what you are going to do with dyn EventStream<T> or the content of EventStream<T>.

@spigaz

but I ended always loosing the type that I need in the concrete implementation to create the proper EventStream.

If you want to convert a type to a trait object, the count of the implementations is fixed, which means you cannot make more implementations dependent on the type.

---

If the type doesn't affect the implementation, just return a concrete type (without generic) then convert to Box<dyn EventStream<T>> later.

If you need to know some certain information of the type (even functions), just input them and return a concrete type.

[spigaz]

Hi, thank you for the heads up!

I already improved my post to make it clear using your feedback and added a concrete same usage:

  pub fn new(event_store: Rc<dyn EventStore>) -> Self {
    Self { stream: event_store.create_stream::<T>("test".to_string()).unwrap() }
  }

error[E0038]: the trait `EventStore` cannot be made into an object

Thank you!

[spigaz]

Hi, thank you, I already improved my post to clarify it.

The type does affect the implementation, as its information is used to serialize and deserialize the events that are passed to the event stream.

But you are right, there are the functions that are called, but they depend on the Codec, for instance, for Json, Serde Serialize and Deserialize are used.

Although there are already other codecs, so for now at least 4 functions would have to be passed.

Even if that approach would work, it would make the implementation more complex and less clear.

Thank you!

[quinedot]

The problem is trying to create a dyn EventStore generally, not Rc<dyn EventStore> specifically.

If you want EventStore to be dyn-capable, you need to make T a parameter of EventStore instead of a generic on create_stream, or remove the method from the trait, or find a way to type-erase T. (You haven't shared Codable, so it's hard to say if dyn Codable is an option. It's sounds like you believe type erasing T isn't an option anyway.)

[spigaz]

Yes, you are correct, its more general, but it was just to provide a more concrete context.

Like you need regarding Codable, that is just a trait that extends the Serde traits and some other.

I can't make T a parameter of EventStore because each stream serializes different type of events.

Type Erasing prevents me from creating the proper event stream that requires the type, so that its able to deserialize back the events, I tried several strategies around this.

I even got as far as implementing the method only for Self : Sized, and impl the method for dyn EventStore, but I couldn't find a way to dispatch to the concrete type.

Thank you!

[quinedot]

@spigaz

Like you need regarding Codable, that is just a trait that extends the Serde traits and some other.

Okay, and serde isn't dyn-capable either. I don't know if it will help your use case or not, but you may be interested in the erased_serde crate, which provides some dyn-capable versions of serde traits.

[jumpnbrownweasel]

@spigaz

And it works pretty well, as long as I use it that way, the problem is if I try to use it as an Rc:

  pub fn new(event_store: Rc<dyn EventStore>) -> Self {
    Self { stream: event_store.create_stream::<T>("test".to_string()).unwrap() }
  }

Just to be clear, you can use it as an Rc, but not with dyn EventStore:

impl<T: Codable> Test<T> {
    pub fn new<E: EventStore>(event_store: Rc<E>) -> Self {
        Self {
            stream: event_store.create_stream::<T>("test".to_string()).unwrap(),
        }
    }

I'm trying to implement the Abstract Factory pattern

I suggest trying to use generic types like Rc<E> above, if possible. Factories are rarely used in Rust.

---

BTW, why are you erasing EventStore? Is this for mocking in tests?

[spigaz]

I wasn't aware of that crate, thank you!

My objective is to simplify the API, and that crate or the strategy it follows doesn't seem to help in that sense. And if it did, it would imply refactoring all the implementation and codecs.

But I'm giving it a look, just in case...

Thank you!

[spigaz]

Yes, I already use it as a generic type and in that situation I can use the Rc if I want to, but the problem is to abstract that.

The default design pattern is for a MemoryEventStore that I use for testing, but it also allows to use different EventStore implementations and decorators.

For instance, to replicate the streams to different cloud providers, or storage solutions, or to store them in different formats.

Thank you!

[jumpnbrownweasel]

@spigaz

Yes, I already use it as a generic type and in that situation I can use the Rc if I want to, but the problem is to abstract that.

This abstraction doesn't seem possible with trait objects, as they're not nearly as capable as generics and you're pushing them over their limits. Abstraction in Rust is better supported with generics. Using generics unfortunately will require lots of type params in many places, but that seems unavoidable in this case.

[parasyte]

I would really like to be able to see these complex designs in context when they come up. I know there is motivation for it, but it's all so invisible without a legitimate repo to see the architecture.

@spigaz

Yes, I already use it as a generic type and in that situation I can use the Rc if I want to, but the problem is to abstract that.

?

It's already abstract through the E: EventStore generic parameter. Type erasure often hurts more than it helps, but there must be some other motivation for it than making the code "more abstract".

You go on to list a number of different implementations you might want to use, but I'm having some difficulty finding where type erasure is needed for any of those specifically. Are you putting these varying implementations into a heterogenous collection?

[spigaz]

So far yes, thats precisely what ends up happening and what I wanted to avoid, because I end up having to propagate the generic parameter till where the dependency injection happens.

[spigaz]

@parasyte

I would really like to be able to see these complex designs in context when they come up. I know there is motivation for it, but it's all so invisible without a legitimate repo to see the architecture.

They come up is the context of dependency injection in Clean Architecture, DDD, Onion Architecture, Hexagonal Architecture or in a more general way as ports and adapters.

In this regard, the point is pretty much the same, to define an abstract interface / api / port, then implement concrete types that actually implement the functionality for a given context, the adapter if you want.

An example is UsersRepository and then a PostresRepository and a MySqlRepository, but in theory you should apply this to all external dependencies.

Then in the application configuration you have to pass as argument all those concrete types or adapters to instantiate the application.

That's the way I end up in the situation that @jumpnbrownweasel mentioned, even using type alias whenever I can, I ended up in one particular situation to have a type declaration that spanned 2 lines.

To minimize this, I use local configurations that allow me to simplify the final application configuration.

@parasyte

It's already abstract through the E: EventStore generic parameter.

Correct, what I want to abstract is to have it work like impl, if you want, as in, you can receive the concrete type that implements the trait, and its stays contained, but for know only dyn is supported in a field struct.

Just to make it clear the ideal would be something like:

pub struct State {
  store: Box<impl EventStore>,
}

But I'm trying to compromise, by aiming at:

pub struct State {
  Store: Box<dyn EventStore>,
}

Instead of doing:

pub struct State<E: EventStore> {
  Store: E,
}

This fragment belongs to a router Actor State that handles incoming requests and has to create streams for the handlers.

In practice most code complexity of that actor ends up being from generic types.

@parasyte

You go on to list a number of different implementations you might want to use, but I'm having some difficulty finding where type erasure is needed for any of those specifically. Are you putting these varying implementations into a heterogenous collection?

I experimented many different strategies from different sources for the same kind of problem, type erasure is just one that pops up a lot, but I wasn't able to get any to work, some I got close enough, but they always seemed to be a dead end.

@parasyte

Type erasure often hurts more than it helps

@parasyte

I'm having some difficulty finding where type erasure is needed for any of those specifically.

Regarding type erasures, I completely agree with you.

Thanks!

[quinedot]

@spigaz

Correct, what I want to abstract is to have it work like impl, if you want, as in, you can receive the concrete type that implements the trait, and its stays contained, but for know only dyn is supported in a field struct.

Just to make it clear the ideal would be something like:

impl Trait does not work like that (or what I think you mean anyway). dyn is the closest thing that Rust has that works something sortof like that, but as I believe you understand, it's not really the same thing.

There are^[1] two distinct impl Trait concepts. One is an opaque type alias that's the output of some other (perhaps parameterized) construct -- so far methods and functions.^[2]

// Here, for every "input" `T`, the `impl Trait` is an opaque type for
// some concrete type that implement `Trait + Sized` (and perhaps some
// auto-traits).
fn rpit_example<T>(foo: T) -> impl Trait {
   // ...
}

If this was supported in the context of State,^[3] this would mean that you really have a Box<OneSingleConcreteType>, which isn't what you want.

The other impl Trait concept is in argument position, where it's actually just a poor^[4] alternative to generics.

fn apit_example(foo: impl Trait) {}
// Mostly the same as
fn example<Foo: Trait>(foo: Foo) {}
// (but less capable in various ways due to removing the name of the generic)

So if this was supported in the context of State, it would just mean this:^[5]

// n.b. this is also non-ideal because one doesn't usually put trait bounds
// on type generics at the definition site unless necessary
pub struct State<E: EventStore> {
    store: Box<E>,
}

This also isn't what you want. Or maybe it is, syntactically, and you've rejected generics on something along aesthetic grounds; I'm not completely sure. It wouldn't remove any actual complexity. It would probably add actual complexity (or remove actual capability) by removing the ability to name the generics (which is why impl Trait is inferior in argument position).

---

Maybe in some future Rust will have support for

pub trait EventStore<T: Codable>: Debug {
    fn create_stream(&self, name: String) -> Result<Box<dyn EventStream<T>>, EventStoreError>;
}

// Hypothetical conditional higher-ranked bound over types
//                   vvvvvvvvvvvvvvv
fn example_bound<ES: for<T: Codable> EventStore<T>>(es: ES) { }

But we're talking years and years, if ever; probably the dyn equivalent isn't plausible.

---

1. unfortunately ↩︎

2. eventually also type aliases ↩︎

3. where the impl Trait is the output of the non-parameterized struct definition ↩︎

4. aside from aesthetics in simple cases ↩︎

5. without the ability to name the generic ↩︎

[spigaz]

I revised my reply, because there is no point in explaining what I was trying to say with impl, when the behaviour I want I already have with generics.

So I didn't rejected it, I embraced it, at a cost of a lot of cognitive load and that's what I was trying to reduce by trying to avoid the type propagation up the tree.

That is, what I wanted was just to "hide" the generic parameters of the types from the outside, to make it cleaner, easier to understand.

As I can't use type alias everywhere to solve that problem.

That's why I tried dyn, knowing that I was trading for a cost in performance.

Probably the best approach is to hide them with macros.

Thanks!

[parasyte]

Long reply to obsolete conversation...

@spigaz

They come up is the context of dependency injection in Clean Architecture, DDD, Onion Architecture, Hexagonal Architecture or in a more general way as ports and adapters.

If you are referring to type erasure here, and implying that it's the only way to provide dependency injection, I disagree. Generic type parameters can be used for dependency injection, too.

I'll skip a few paragraphs, because type parameters fit those use cases without need for further elaboration.

@spigaz

That's the way I end up in the situation that @jumpnbrownweasel mentioned, even using type alias whenever I can, I ended up in one particular situation to have a type declaration that spanned 2 lines.

This is perhaps a minor inconvenience. Type aliases go a long way to reduce the noise. It's minor in the sense that the code might be ugly, but that's the only downside. At least it will compile and not crash in the middle of the night. It will still be robust. That has to count for something.

@spigaz

Instead of doing:

pub struct State<E: EventStore> {
  Store: E,
}

This fragment belongs to a router Actor State that handles incoming requests and has to create streams for the handlers.

In practice most code complexity of that actor ends up being from generic types.

Requirement unclear. All you have to do to use this struct is name the E. What prevents that? "2 lines of code" doesn't prevent that. Rejecting 2 lines of code on subjective grounds might, but that's self-imposed.

@spigaz

The problem arises because MemoryEventStore and ConcreteEventStore aren't probably of the same size and there is no way to address that, that's why mentioned as a Box, that way the problem of the size goes away, like it would with a reference.

Do you mean to use indirection to make the size of State small, regardless of the inner type? That can be:

struct State<E> {
    store: Box<E>,
}

Or Rc<E>, or Arc<E>.

@spigaz

I believe that generics works pretty much the same way, in this context, but imposes the generic argument, as they are of different sizes potentially, and in practice are different types, like they would be with impl.

The sizes being different is a side effect of the types being different, and that difference is only material when you intend to mix these different types within a heterogenous collection. For everything else you can either expose the type parameter all the way up the type hierarchy or hide it in an enum.

I still haven't found the showstopper for doing either of these things.

@spigaz

Of course you can solve the problem with enums, if you drop the dependency injection and depend on anything that can be used.

I'm not sure why this is confused, but maybe it deserves to be stated clearly:

• enum can be used for dependency injection. enum is a form of dynamic dispatch with static (known at compile time) types.
• Generic type parameters can also be used for dependency injection. Generics are purely static dispatch.
• dyn Trait can also be used for dependency injection. It's pure dynamic dispatch, of course. But the benefits for its use are extremely narrow, and it comes with more downsides than the one thing it has going for it.

---

Perhaps we should briefly return to the original motivating example:

@spigaz

A current sample usage is:

pub struct Test<T: Codable> {
  stream: Box<dyn EventStream<T>>,
}

impl<T: Codable> Test<T> {
  pub fn new<E: EventStore>(event_store: E) -> Self {
    Self { stream: event_store.create_stream::<T>("test".to_string()).unwrap() }
  }

  pub async fn has_events(&mut self) -> bool {
    if let Ok(events) = self.stream.events().await {
      events.len() > 0
    } else {
      false
    }
  }
}

And it works pretty well, as long as I use it that way

I'm with you here, so far. All looks good!

@spigaz

the problem is if I try to use it as an Rc:

  pub fn new(event_store: Rc<dyn EventStore>) -> Self {
    Self { stream: event_store.create_stream::<T>("test".to_string()).unwrap() }
  }

error[E0038]: the trait `EventStore` cannot be made into an object

But now I'm complete lost! Why did you have to change the method signature? Rc<E> can implement EventStore if you add an implementation for it:

impl<E: EventStore> EventStore for Rc<E> {
    fn create_stream<T: Codable>(&self, name: String) -> Result<Box<dyn EventStream<T>>, EventStoreError> {
        (&**self).create_stream(name)
    }
}

You'll also need an impl for the other reference types, as well: &E, &mut E, Arc<E> and Box<E>. This will make the original method signature accept your type no matter how you choose to reference it.

Could this whole type erasure/dyn Trait runaround be an XY problem?

---

Well, now that that has been fully derailed...

I think your conclusion is nearly the right one. Don't worry about naming your types. Usually you won't have to: just expose the generic type parameter everywhere and you'll mostly be fine. There are a few gotchas that you will hit from time to time... Nothing's really perfect. It's tradeoffs all the way down.

I am guilty of using a macro to hide type complexity. All I can say is, good luck debugging any panics or typeck errors! I am eagerly looking forward to removing my own macro because it's such a pain that is not at all in any universe worth it.

[jumpnbrownweasel]

@spigaz

That is, what I wanted was just to "hide" the generic parameters of the types from the outside, to make it cleaner, easier to understand.

In general, let's say you have a working design with generic types, as you said. Without changing it radically, there are ways to get rid of some generic type params in special cases. Even getting rid of one or two type params will reduce complexity noticeably. This is not a clean, general solution, it is only practical.

• I have used trait objects to reduce the number of generic params. But because they are limited and a little slower, I have to be very selective about where they are applied. For example, perhaps you can use the erased_serde crate that @quinedot suggested.

• You might be able to remove a type param for testing by using conditional statements (#[cfg(test)] and #[cfg(not(test))]) to define a type alias, or to define the actual production/test types if they are given the same name. This is not as clean as one might like, but it works.

[spigaz]

@parasyte

But now I'm complete lost! Why did you have to change the method signature?

As you have probably understood by now, I was trying to hide the generic parameter, to keep it internal, at the cost of performance.

Now I understand that you also call that type erasure, I was mistaken by thinking that you were always mentioning removing it from the trait's method.

@parasyte

Could this whole type erasure/dyn Trait runaround be an XY problem?

My problem was / is in using dyn EventStore as an object, not in using Rc with the concrete type.

@parasyte

impl<E: EventStore> EventStore for Rc<E> {
    fn create_stream<T: Codable>(&self, name: String) -> Result<Box<dyn EventStream<T>>, EventStoreError> {
        (&**self).create_stream(name)
    }
}

At some point, I already tried a variations of this, including one for dyn EventStore, but they didn't helped.

@parasyte

Usually you won't have to: just expose the generic type parameter everywhere and you'll mostly be fine.

When they are expanded in errors, it gets really messy and makes it hard to understand what's going on. I used to have multiple lines to declare a type.

I used some tricks to hide them, like local configurations and others I was able to use dyn, either Box or Rc and that already helped a lot.

From a technical perspective its easy to argue that it was a lot better, but it was a lot harder to understand or change anything, while the actual code was pretty simple and straight forward.

@parasyte

I am guilty of using a macro to hide type complexity.

I know the limitations of macros, but if they are able to reduce the cognitive load, it enables you to understand the problem faster, sometimes the solution is more strict macros.