Help Understanding the Rust Object Model: impl dyn Trait

Hello! I am learning Rust and am trying to modify my code to use an object+dynamic dispatch. However, I don't understand enough about the Rust object model to do this successfully. I put together this playground with a simplified version of my code and the issues.

Apologies if these questions have been answered before--I wasn't able to find them when searching.

My first question is really a few related questions. I'll walk through my code and then ask at the end. If I were to create my trait in the playground in the following way:

trait Data {
    fn get_value<T: Copy>(&self, offset: I) -> T;
}

impl Data for [u8] { ... }

I can call get_value on a slice (or array) and everything works as I expected:

const TWO_BYTES: [u8; 2] = [0xAA, 0xBB];
TWO_BYTES.get_value::<u16>(0)

However, if I want to make my trait object safe, I have to remove the generic method from the interface. I understand why the generic method inhibits creating a vtable, but I thought by moving the generic method to an impl, it would still be available as part of the type:

trait Data {
    type Index: From<usize>;

    fn get_bytes(&self, offset: Self::Index, buf: &mut [u8]);
}

impl<I> dyn Data<Index = I>
where
    I: From<usize>,
{
    fn get_value<T: Copy>(&self, offset: I) -> T { ... }
}

impl Data for [u8] { ... }

Unfortunately, this doesn't work how I expected (I was thinking this would be similar to a Swift protocol extension). It breaks the existing get_value call sites and I cannot find a way to call get_value with [u8] types.

To help build up my object model understanding and resolve the issues:

  1. How does the compiler resolve methods implemented in impl dyn Trait? Does the type name have to literally spell that (like get_first_word in the playground and below)?
  2. Is there a way I can call the impl dyn Trait methods from something with a concrete type (e.g. [u8]) since the trait is implemented for that type?
  3. Similar to (2) but more generally, is there a more idiomatic pattern to solve using generic methods on with object traits in Rust?

Any additional information or pointers on the Rust object model to help develop my mental model would be greatly appreciated! I've read through Rust By Example, The Rust Programming Language, and The Rust Reference, but unfortunately didn't find (or understand) the concepts necessary to answer these questions.

My second question is, in this example, what has the static lifetime? The vtable for the object, the referenced object instance, or something else?

fn get_first_word<Index>(data: &'static dyn Data<Index = Index>) -> u16

Thanks in advance for your time and thoughts!

One of the key things to realize is that dyn Trait is a completely different object type from any struct (or enum, etc.) that might implement that trait. Automatic conversion from a struct to a trait object happens automatically in only a few places. impl dyn Trait is defining inherent methods of the dyn Trait type in the same way that impl MyStruct defines inherent methods of the MyStruct type.

Because those are inherent methods of the dyn Trait type, you'll need to use an as cast to convert them. That's only allowed for Sized types, though, so you won't be able to do it for [u8].

A common trick is to define two traits, one that's object-safe and another that contains the generic methods:

trait Data_ {
    type Index: From<usize>;
    fn get_bytes(&self, offset: Self::Index, buf: &mut [u8]);
}

trait Data : Data_ {
    unsafe fn get_value<T: Copy>(&self, offset: Self::Index) -> T;
}

impl<D:?Sized> Data for D where Self: Data_ {
    unsafe fn get_value<T: Copy>(&self, offset: Self::Index) -> T {
        let mut value = MaybeUninit::<T>::uninit();

        self.get_bytes(
            offset,
            core::slice::from_raw_parts_mut(
                value.as_mut_ptr() as *mut u8,
                size_of::<T>(),
            ),
        );

        value.assume_init()
    }
}

This will make all objects that implement Data_ also implement Data, including dyn Data_. Also note that I moved unsafe to the method signature, as your implementation doesn't have any guardrails against being misused.

4 Likes

Thanks so much @2e71828! The two trait trick works quite well and seems to have resolved virtually every issue I was encountering :smile_cat:.

To make the slice types compatible with the &dyn Trait type, I created a wrapper struct that implements Data_ and forwards to the slice's implementation. From other similar questions, it seems that lighweight wrappers are common in cases like this. (I have a lot of test data that's included via the include_bytes! macro, so wrapping seemed like a fairly non-intrusive way to make this migration.)

Yeah, sorry--I forgot to note in the original post the sample code was only to illustrate the issue with the type system and not directly derived from the "production" implementation.

Also note that since [u8] is not Sized, you can't coerce it to a dyn Trait. But, at this point there are two interesting things to notice:

  • &'_ [u8] is a simple way to get back a sized type.

  • when your trait features &self methods exclusively, like yours do, a common thing to do is to make the trait be "&-transitive":

    impl<'lt, T : ?Sized> Data for &'lt T
    where
        T : Data,
    {
        type Index = <T as Data>::Index;
    
        fn get_bytes (
            self: &'_ (&'lt T),
            offset: Self::Index,
            buf: &'_ mut [u8],
        )
        {
            T::get_bytes(&**self, offset, buf)
        }
    }
    

This is a similar situation to types like str and traits such as Display, for instance :slightly_smiling_face:

4 Likes

Awesome, thanks so much for the additional information @Yandros! I was also looking for a solution to make the trait "&-transitive" but wasn't sure of the terminology.

One quirk I didn't quite understand before was that while &'_ [u8] is Sized, I actually need &'_ &'_ [u8] for it to be coerced into a &dyn Data_. The double reference (borrow? still learning the vocab) threw me off, but I guess the right way to think about it is a borrow to the sized type.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.