How to do erased Traits for traits with associated types which impl other traits

[wydibola]

Hey, so I have this setup in my code where I use three traits to generalize a variety of different types which have wildly different APIs and which all perform one of three same underlying functions.To show you what it looks I made this example where I simplified these 3 traits to show the logic so I just put some random stuff for the functions but the return types, arguments and associated types are very similar to my code.

It looks something like this:

pub trait TraitA: Clone {
    type B: TraitB;
    type C: TraitC;

    fn test_one(&self) -> Self::B;
    fn test_two(&self) -> u32;
    fn test_three(b: Self::B, c: Self::C) -> Self;
}

impl TraitA for u64 {
    type B = u16;
    type C = u8;

    fn test_one(&self) -> Self::B { 1 }
    fn test_two(&self) -> u32 { 1 }
    fn test_three(t: Self::B, s: Self::C) -> Self { 1 }
}

//#########################################

pub trait TraitB {
    type A: TraitA;
    type C: TraitC;

    fn test_one(&self) -> Self::A;
    fn test_two(&self) -> u32;
    fn test_three(b: Self::C) -> Result<Self, ()> where Self: Sized;
}

impl TraitB for u16 {
    type A = u64;
    type C = u8;

    fn test_one(&self) -> Self::A { 2 }
    fn test_two(&self) -> u32 { 2 }
    fn test_three(b: Self::C) -> Result<Self, ()> { Err(()) }
}

//#########################################

pub trait TraitC {
    fn test(&self) -> u32;
}

impl TraitC for u8 {
    fn test(&self) -> u32 { 3 }
}

//#########################################

//this function passes the compiler
fn function_using_traits<A: TraitA, B: TraitB>
    (a: A, b: B, c: Box<dyn TraitC>)
{ println!("just checking if using these args works"); }

//this one obviously doesn't given TraitA and TraitB are not 'object safe'
fn other_function_using_traits
    (a: Box<dyn TraitA>, b: Box<dyn TraitB>, c: Box<dyn TraitC>) 
{ println!("just checking if using these args works"); }

This code compiles fine (except for other_function_using_traits()) and its the most efficient solution I have found for the problem I have. I am not interested in changing how it's set up and I'm not really in a position to do so.

Now the problem is that sometimes when I call on these traits in other bits of my code, I do not know what type to call on, I just know the type has implemented one of these traits. I want to be able to handle these without having to know their types, just the fact that they implement certain traits. I want to be able to have dynamic dispatch essentially. The easy answer then is to just use Box<dyn Trait> which I would like to do except TraitB has Self is Sized and TraitA has Clone as a supertrait. So after looking around I found out that I can just erase the traits by making another trait which is object safe and using that. The problem is I don't know how to do that for TraitA and TraitB given my code is very different from any example I found online.

What I found online so far helped me understand how it works but I am struggling to apply it to my situation. I'm sort of hoping someone can guide me through how to do this because for now I'm just stumbling in the dark and fighting the compiler.

I looked at these threads/articles:

Erased traits - Learning Rust

Box<dyn T> where T has associated types that are traits

Foreword: I asked this (or a very similar question) on one of the Rust discord servers a few days ago, and after some discussion I ended up going with static dispatch via an enum that also implements T. This works well. Many thanks to the people over there for their assistance. But I also never figured out how to solve my original problem of how to box T correctly when T has associated types that are also traits. I'd like to know how to solve this to further my understanding of Rust. I'm asking …

Type-erasing trait parameters in Rust

Trait objects with associated types

Sure, but that only works for static dispatch. It doesn't work with for example a heterogeneous Vec<Thing<_>>. For that, I'm currently adding a dynamic dispatch wrapper trait DynamicThing that is implemented for all Thing<Error=E> and implements Thing<Error=Box<Error>>. Then, I can have a Vec<Box<DynamicThing>>. I find it difficult to implement, though, so I was asking whether there is a simpler solution. For reference, here is my implementation: use std::io::Error as IoError; use std::error::…

[quinedot]

I don't know if it meets your needs or not (and I have to run for now), but here's a first stab:

Rust Playground

(Not tested and there may be some gaps around unsized implementors of TraitB.)

[wydibola]

This is immensely helpful. Thank you so much!

I definitely think your solution is very close to what I want to do if not exactly what I want to do. I think the only problem now is that I lack the understanding of how exactly to use erased traits. I did a bunch of test in the playground link you sent (Rust Playground) where I tried to do a matcher (have some variable of a certain type and return a trait). I can't seem to understand exactly how to return the trait.

Here's what it looks like:

pub trait ErasedTraitA {
    type B: ErasedTraitB;
    type C: TraitC;
    
    fn test_one(&self) -> Self::B;
    fn test_two(&self) -> u32;
    fn test_three<'lt>(&self, b: Self::B, c: Self::C) -> Box<dyn ErasedTraitA<B = Self::B, C = Self::C> + 'lt>
    where
        Self: 'lt,
    ;
    fn test_four<'lt>(&self) -> Box<dyn ErasedTraitA<B = Self::B, C = Self::C> + 'lt>
    where
        Self: 'lt,
    ;
}

impl<T: TraitA> ErasedTraitA for T {
    type B = <T as TraitA>::B;
    type C = <T as TraitA>::C;
    
    fn test_one(&self) -> Self::B {
        <T as TraitA>::test_one(self)
    }
    fn test_two(&self) -> u32 {
        <T as TraitA>::test_two(self)
    }
    fn test_three<'lt>(&self, b: Self::B, c: Self::C) -> Box<dyn ErasedTraitA<B = Self::B, C = Self::C> + 'lt>
    where
        Self: 'lt,
    {
        Box::new(<T as TraitA>::test_three(b, c))
    }
    fn test_four<'lt>(&self) -> Box<dyn ErasedTraitA<B = Self::B, C = Self::C> + 'lt>
    where
        Self: 'lt,
    {
        Box::new(<T as TraitA>::test_four())
    }
}

pub trait TraitA: Clone {
    type B: TraitB;
    type C: TraitC;

    fn test_one(&self) -> Self::B;
    fn test_two(&self) -> u32;
    fn test_three(b: Self::B, c: Self::C) -> Self;
    fn test_four() -> Self;
}

impl TraitA for u64 {
    type B = u16;
    type C = u8;

    fn test_one(&self) -> Self::B { 1 }
    fn test_two(&self) -> u32 { 1 }
    fn test_three(t: Self::B, s: Self::C) -> Self { 1 }
    fn test_four() -> Self { 1 }
}

impl TraitA for usize {
    type B = u16;
    type C = u8;

    fn test_one(&self) -> Self::B { 1 }
    fn test_two(&self) -> u32 { 1 }
    fn test_three(t: Self::B, s: Self::C) -> Self { 1 }
    fn test_four() -> Self { 1 }
}
//#####
//TraitB and ErasedTraitB
//TraitC
//#####
fn matcherA<CofA, BofA>
    (v: u128) -> Result<Box<dyn ErasedTraitA<C = CofA, B = BofA>>, ()>
{
    match v {
        0 => Ok(<usize as ErasedTraitA>::test_four(&<usize as TraitA>::test_four())),
        64 => Ok(Box::new(<u64 as TraitA>::test_four())),
        _ => Err(())
    }
}

I get the following errors which make sense, we want the type to be determined dynamically but I don't understand how to do it in practice.

error[E0308]: mismatched types
   --> src/lib.rs:181:17
    |
177 | fn matcherA<CofA, BofA>
    |                   ---- expected this type parameter
...
181 |         0 => Ok(<usize as ErasedTraitA>::test_four(&<usize as TraitA>::test_four())),
    |              -- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `Box<dyn ErasedTraitA<B = BofA, C = ...>>`, found `Box<dyn ErasedTraitA<B = u16, C = u8>>`
    |              |
    |              arguments to this enum variant are incorrect
    |
    = note: expected struct `Box<dyn ErasedTraitA<B = BofA, C = CofA>>`
               found struct `Box<dyn ErasedTraitA<B = u16, C = u8>>`
help: the type constructed contains `Box<dyn ErasedTraitA<B = u16, C = u8>>` due to the type of the argument passed
   --> src/lib.rs:181:14
    |
181 |         0 => Ok(<usize as ErasedTraitA>::test_four(&<usize as TraitA>::test_four())),
    |              ^^^-------------------------------------------------------------------^
    |                 |
    |                 this argument influences the type of `Ok`
note: tuple variant defined here
   --> /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs:561:5
    |
561 |     Ok(#[stable(feature = "rust1", since = "1.0.0")] T),
    |     ^^

error[E0271]: type mismatch resolving `<u64 as ErasedTraitA>::B == BofA`
   --> src/lib.rs:182:18
    |
177 | fn matcherA<CofA, BofA>
    |                   ---- expected this type parameter
...
182 |         64 => Ok(Box::new(<u64 as TraitA>::test_four())),
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type mismatch resolving `<u64 as ErasedTraitA>::B == BofA`
    |
note: expected this to be `BofA`
   --> src/lib.rs:18:14
    |
 18 |     type B = <T as TraitA>::B;
    |              ^^^^^^^^^^^^^^^^
    = note: expected type parameter `BofA`
                         found type `u16`
    = note: required for the cast from `Box<u64>` to `Box<dyn ErasedTraitA<B = BofA, C = CofA>>`

error[E0271]: type mismatch resolving `<u64 as ErasedTraitA>::C == CofA`
   --> src/lib.rs:182:18
    |
177 | fn matcherA<CofA, BofA>
    |             ---- expected this type parameter
...
182 |         64 => Ok(Box::new(<u64 as TraitA>::test_four())),
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type mismatch resolving `<u64 as ErasedTraitA>::C == CofA`
    |
note: expected this to be `CofA`
   --> src/lib.rs:19:14
    |
 19 |     type C = <T as TraitA>::C;
    |              ^^^^^^^^^^^^^^^^
    = note: expected type parameter `CofA`
                         found type `u8`
    = note: required for the cast from `Box<u64>` to `Box<dyn ErasedTraitA<B = BofA, C = CofA>>`

So yeah, you probably sent me the correct solution but I'm not sure how to use the tools you just gave me...

[quinedot]

@wydibola

I get the following errors which make sense, we want the type to be determined dynamically but I don't understand how to do it in practice.

When a function has generics:

//          vvvv  vvvv
fn matcherA<CofA, BofA>(v: u128) -> ...

The caller gets to decide the generics (within any bounds), and the function body must work for any choice of generics (within any bounds). But in the playground, the body is trying to decide the "values" of the type generics in the return type. That's what the error is trying to say.

The immediate fix is to change the signature to not use generics:

fn matcherA(v: u128) -> Result<Box<dyn ErasedTraitA<C = u8, B = u16>>, ()>

This will work so long as the associated types are always the same. This happens to be the case for usize and u64, so you could do:

    match v {
        0 => Ok(<usize as ErasedTraitA>::test_four(&<usize as TraitA>::test_four())),
        64 => Ok(Box::new(<u128 as TraitB>::test_one(&0))),
        _ => Err(())
    }

(Playground.)

But it won't work if the associated types differ. It's analagous to how this doesn't work:

fn choose_somehow(cond: bool) -> Vec<bool or i32> {
   if cond { vec![true] } else { vec![0] }
}

So if you wish you could do that, some other or additional approach is needed. That might be more type erasure so the associated types go away (but I didn't see an obvious way to do that since they are also used as arguments), or some implementations that work with enums (but then you may end up matching everywhere in place of dynamic dispatch), or something else depending on your needs.

[wydibola]

Ahhh I see. I though i'd be able to do something a bit like this (of course not like that because its a never ending self referential nightmare):

fn matcherA(v: u128) -> Result<Box<dyn ErasedTraitA<C = dyn TraitC, B = dyn ErasedTraitB>>, ()>

The associated types differ with each implementation so I definitely need more type erasure. Enums aren't really great for my use case, it's much more interesting to just deal with a Box I think. From what I've read matching enums is not particularly efficient compared to type erasure and its a much less elegant way to code things given we need to constantly match things. I would just have hoped for type erasure and traits in general to have a more intuitive syntax not that I really know how to improve it to be fair.

So what would I have to do to be able to dynamically dispatch for the associated types too?

[quinedot]

This reply is going to be on the brainstorming side of things.

Let's say you had these implementations:

impl TraitA for u64 {
    type B = u16;
    type C = u8;
    fn test_three(t: Self::B, s: Self::C) -> Self { 1 }
    // ...
}

impl TraitA for () {
    type B = i64;
    type C = i32;
    fn test_three(t: Self::B, s: Self::C) -> Self { 1 }
    // ...
}

And you wanted to return either u64 as a dyn ErasedTraitA<..> or () as a dyn ErasedTraitA<..>. How can some form of test_three work with the return type? u64 only has a fn(u16, u8) -> u64 to make use of and () only has a fn(i64, i32).

You could "fix" this on the erased trait side by just not having test_three.

Or you could fix it on the TraitA side by making implementations able to handle any inputs (within bounds):

fn test_three_alpha<B: Something, C: Something>(t: B, s: C) -> Self;
// This approach might require `impl Something for Box<dyn Something + '_>`
fn test_three_beta(t: Box<dyn Something + '_>, s: Box<dyn Something + '_>) -> Self;

But probably those are associated types for a reason (implementors need to deal with specific types). If the associated types meet a 'static bound, you could make these return Option or Result and then the implementors could attempt to downcast to their specific types, but now everything is a fallible operation.

Or you could have some

impl TraitA for Either<u64, ()> {
    type B = Either<u16, i64>;
    type C = Either<u8, i32>;
    fn test_three(t: Self::B, s: Self::C) -> Self {
        match (t, s) {
            (Left(t), Left(s)) => u64::test_three(t, s),
            (Right(t), Right(s)) => <()>::test_three(t, s),
            _ => panic!("Oops I guess?  This isn't great."),
        }
    }
}

...which doesn't seem that great unless you hid it behind macros maybe. And if you were considering this route, I'd recommend checking out something like enum_dispatch instead of mixing it with dyn type erasure (assuming you're dealing with a closed set of implementors).^[1]

If the associated types meet a 'static bound, you could do something like

impl<T: TraitA> ErasedTraitA for T {
    fn test_three<'lt>(
        &self, t: Box<dyn Any>, s: Box<dyn Any>,
    ) -> Box<dyn Erased + 'lt>
   where
        Self: 'lt,
    {
         let Some(t) = Box::downcast::<T::B>(t) else {
             // Alternative: return `Option` or `Result` in `ErasedTraitA`
             panic!("Oops.... still not great");
         };
         // ...
         Box::new(T::test_three(*t, *s))
    }
}

Which is an "open set" version of the enum approach.

---

So much for test_three. Let's say you find some acceptable fix for that; we still have associated types in the outputs. How do we get rid of those?

Ideally there is some useful property of the associated types that can be exercised via dyn Trait. In the playground, that would potentially be

// For the "just get rid of `test_three" approach:
pub trait ErasedTraitA {
    fn test_one<'lt>(&self) -> Box<dyn ErasedTraitB + 'lt> where Self: 'lt;
    fn test_two(&self) -> u32;
    fn test_four<'lt>(&self) -> Box<dyn ErasedTraitA + 'lt> where Self: 'lt;
}

pub trait ErasedTraitB {
    fn test_one<'lt>(&self) -> Box<dyn ErasedTraitA + 'lt> where Self: 'lt;
    fn test_two(&self) -> u32;
    fn test_four<'lt>(&self) -> Box<dyn ErasedTraitB + 'lt> where Self: 'lt;
}

If that's not adequate,^[2] we're in probably in a similar situation to the "associated type in inputs" case, with related workarounds (return enums, return something you can fallibly downcast, ...).

---

In summary, there may be acceptable ways forward, but if so they're probably going to depend on the details of your use case. There's no blanket solution.

This:

I though i'd be able to do something a bit like this (of course not like that because its a never ending self referential nightmare):

fn matcherA(v: u128) -> Result<Box<dyn ErasedTraitA<C = dyn TraitC, B = dyn ErasedTraitB>>, ()>

May be a sign that you wish Rust was dynamically typed instead of statically typed . dyn Traits are dynamically sized, but still a distinct static type. They can approximate dynamic typing in some ways, but only to a certain extent.

---

1. Note that dyn type erasure introduces indirection (vtable dispatch which is hard to devirtualize) and probably allocations (Boxing usually), so it's not necessarily faster than matching on enums. That crate argues that it's significantly slower. ↩︎

2. perhaps you want to feed the output into some test_three? ↩︎

[wydibola]

Well I think that's the solution. I'm probably just going to use enum_dispatch. I read the enum_dispatch docs and I have to say I was surprised to see that doing this through enums is that efficient. I think I might have a fundamental misunderstanding of how rust works on these issues hence trying so hard to get this to work with dyn traits.

What I don't really understand is that enums and Box don't seem that different to me. If i understand enums are just some info + the necessary space to store the largest type possible. Whereas for dyn traits we can't know what the most volumous type that will implement the trait is so we can't use that approach. What we do know is the methods that apply to the type (which must surely give us information) and if we use a pointer to the type I struggle to understand why it's so hard to do? Is it a design desicion in Rust? I can't say it's a bad decision given it still seems to offer efficient solutions and the type safety offered by rust which results in part from the decision for everything to be statically typed is very valuable. I'm just curious as to the reasoning behind if someone can explain or point me in the right direction.

[quinedot]

@wydibola

What I don't really understand is that enums and Box don't seem that different to me. If i understand enums are just some info + the necessary space to store the largest type possible. Whereas for dyn traits we can't know what the most volumous type that will implement the trait is so we can't use that approach.

Crates are compiled separately, so we don't even know all the implementations: there could be more downstream (or for foreign traits, in sibling crates).^[1]

The dyn vtable has to be the same for all implementors. You can't have a test_three method that takes Strings for one implementor and another that takes u32 for another implementor. You can't add a method for every possible associated type, because there's an unbounded number of possibilities (including types unknown to the current compilation). Even if you could, what's supposed to happen when you pass Strings to a dyn Trait whose erased value implemented the method for u32s?

@wydibola

I struggle to understand why it's so hard to do? Is it a design desicion in Rust?

Ultimately I suppose it does come down to what could be considered design decisions, but I don't think I could point at just one thing. The best explanation probably depends on where your expectations are coming from.

---

1. There are some hacky and questionable build system ways to collect the information in some cases, but I don't think such global analysis is ever coming to the language itself. ↩︎