Trait object / object safety issue

By object safety, I'm referring to: Traits - The Rust Reference

I am trying to figure out why:

Box<dyn My_Jscode_Read>; // ok

Box<dyn My_Jscode_Inner>; // compile error, object safety issue
// complains about generic in write_to_js

Context: the following code is basically a "pauper's serde", but we track ArrayBuffers because they can be specified in Window.postMessage() - Web APIs | MDN to use 0-copy instead of structure cloning.

pub trait My_Jscode_Inner {
    fn write_to_js<W: Write>(
        &self,
        writer: &mut BufWriter<W>,
        abs: &mut Vec<js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>;

    fn read_from_js<R: Read>(
        reader: &mut BufReader<R>,
        abs: &mut VecDeque<js_sys::ArrayBuffer>,
    ) -> Result<Self, My_Jscode_Err>
    where
        Self: Sized;
}

pub trait My_Jscode_Read {
    fn read2_from_js<R: Read>(
        reader: &mut BufReader<R>,
        abs: &mut VecDeque<js_sys::ArrayBuffer>,
    ) -> Result<Self, My_Jscode_Err>
    where
        Self: Sized;
}

I believe the read_from_js and read2_from_js functions are okay due to the following clause:

  • Explicitly non-dispatchable functions require:
  • Have a where Self: Sized bound (receiver type of Self (i.e. self) implies this).

Question: how do I fix this issue? Do I have to drop the W: Write? That seems very inconvenient to force a commitment of the writer trait.

Yes. If you don’t need any of the inherent methods of BufWriter, you could change the signature to something like this:

    fn write_to_js(
        &self,
        writer: &mut dyn Write,
        abs: &mut Vec<js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>;

Aside: A brief glance at the BufWriter code looks like it wouldn’t be too hard to add ?Sized to its type argument, if somebody wants to write that patch.

1 Like

Until then, &mut BufWriter<&mut dyn Write> ought to do it :slightly_smiling_face:


@zeroexcuses the usual trick, then, to keep the non-dyn case as optimal as possible (through static dispatch, at the cost of binary bloat…) is to have that generic function, but have it Self : Sized-guarded, and have another more specific dyn Write-based one, precisely to be used by a dyn My_Jscode… trait object.

On nightly, this would be:

pub trait My_Jscode_Inner {
    fn write_to_js<W: Write>(
        &self,
        writer: &mut BufWriter<W>,
        abs: &mut Vec<js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>
    where
        Self : Sized, // 👈
    ;

    // 👇
    fn dyn_write_to_js(
        &self,
        writer: &mut BufWriter<&mut dyn Write>,
        abs: &mut Vec<::js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>;
}

// 👇
default
impl<T : /* Sized + */ My_Jscode_Inner> My_Jscode_Inner for T {
    fn dyn_write_to_js(
        &self,
        writer: &mut BufWriter<&mut dyn Write>,
        abs: &mut Vec<::js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>
    {
        self.write_to_js(writer, abs)
    }
}

On stable Rust, it's a tad bit more convoluted:

Click to see
pub trait My_Jscode_Inner : My_Jscode_Inner_Dyn {
//                       👆^^^^^^^^^^^^^^^^^^^👆
    fn write_to_js<W: Write>(
        &self,
        writer: &mut BufWriter<W>,
        abs: &mut Vec<js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>
    where
        Self : Sized, // 👈
    ;
}

// 👇
pub trait My_Jscode_Inner_Dyn {
    fn dyn_write_to_js(
        &self,
        writer: &mut BufWriter<&mut dyn Write>,
        abs: &mut Vec<::js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>;
}

// 👇
impl<T : /* Sized + */ My_Jscode_Inner> My_Jscode_Inner_Dyn for T {
    fn dyn_write_to_js(
        &self,
        writer: &mut BufWriter<&mut dyn Write>,
        abs: &mut Vec<::js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>
    {
        self.write_to_js(writer, abs)
    }
}
2 Likes

Extrapolating a bit:

    pub fn foo(x: &mut dyn Write) {
        let bf: BufWriter<&mut dyn Write> = BufWriter::new(x);}

    pub fn foo2(x: &mut BufWriter<&mut dyn Write>) {}

I don't think I've ever written &mut BufWriter<&mut dyn Write> before, and am quite surprised BufWriter::new(x) compiles (since BufWriter in std::io - Rust requires inner: W and x: &mut Write, but I guess the substitution W = &mut Write is valid).

I'm now questioning if I'm over using generic everywhere and if I can speed up compile time by using more & dyn and &mut dyn.

This is really interesting. There are a number of clever things going on that I don't quite understand.

  1. You made write_to_js 'explicitly non-dispatchable' with the addition of where Self: Sized

  2. dyn_write_to_js is explicitly dispatchable since you got rid of all the generics

  3. Now, of course, I had to ask: why does My_Jscode_Inner_Dyn need to be a separate trait. What if we stuffed it's function in My_Jscode_Inner, and got this compile error about Self::Sized

pub trait My_Jscode_Inner {
    fn write_to_js<W: Write>(
        &self,
        writer: &mut BufWriter<W>,
        abs: &mut Vec<js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>
    where
        Self: Sized;

    fn dyn_write_to_js(
        &self,
        writer: &mut BufWriter<&mut dyn Write>,
        abs: &mut Vec<js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err> {
        self.write_to_js(writer, abs)}

    fn read_from_js<R: Read>(
        reader: &mut BufReader<R>,
        abs: &mut VecDeque<js_sys::ArrayBuffer>,
    ) -> Result<Self, My_Jscode_Err>
    where
        Self: Sized;
}

Question 1

So one question is: how does splitting out My_Jscode_Inner_Dyn as it's own separate trait get rid of this error ?

Question 2

The other part I don't understand about your solution is why we need:

pub trait My_Jscode_Inner : My_Jscode_Inner_Dyn {

  1. I don't see anything in the body of My_Jscode_Inner that depends on My_Jscode_Inner_Dyn

  2. Later on, we already have

impl<T : /* Sized + */ My_Jscode_Inner> My_Jscode_Inner_Dyn for T {

It is not clear to me the purpose of the former.

These two things work differently:

  • trait Trait : TraitDyn means "wherever Trait is implemented, TraitDyn must be implemented", i.e. "when we have impl Trait, it we always be impl TraitDyn too".
  • impl<T: Trait> TraitDyn for T means "wherever Trait is implemented, TraitDyn is implemented, and here's how".

In other words, the former is promise (which can be trusted by the external code), the latter is fulfillment of that promise (which will be checked when compiling your own code).

1 Like

In particular, the impl block only provides for sized types to implement My_Jscode_Inner_Dyn, which notably excludes the type dyn My_Jscode_Inner. Instead, this gets its implementation of My_Jscode_Inner_Dyn via an entry in its dispatch table, which exists only because of the declared supertrait relationship.

I think I'm starting to see how all of this fits together now, showing only the types and not impls to avoid clutter.

step 1

pub trait My_Jscode_Inner {
    fn write_to_js<W: Write>(&self, writer: &mut BufWriter<W>) -> R;
    fn read_from_js<R: Read>(reader: &mut BufReader<R>) -> R where Self: Sized;
}

This has the problem that we can not do Box<dyn My_Jscode_Inner> because of the generic on write_to_json.

There are two solutions to this:

  1. @2e71828 's solution of &mut BufWriter<&mut dyn Write> , which we will ignore for the rest of this post.
  2. @Yandros 's solution, which we will try to motivate the pieces below.

Suppose we wanted to keep the generic <W: Write> for some reason (optimization?) we can do so by making the function explicitly non-dispatchable by adding a Self: Sized, so now we have:

step 2

pub trait My_Jscode_Inner {
    fn write_to_js<W: Write>(&self, writer: &mut BufWriter<W>) -> R where Self: Sized;
    fn read_from_js<R: Read>(reader: &mut BufReader<R>) -> R where Self: Sized;
}

But now we have the problem that a Box<dyn My_Jscode_Inner> can't call anything. So we change this by trying the following:

step 3

pub trait My_Jscode_Inner {
    fn write_to_js<W: Write>(&self, writer: &mut BufWriter<W>) -> R where Self: Sized;
    fn read_from_js<R: Read>(reader: &mut BufReader<R>) -> R where Self: Sized;

    fn dyn_write_to_js(&self, writer: &mut BufWriter<&mut dyn Write>) -> R;
}

but then when we try to implement dyn_write_to_js , we get issues over Self:Sized.

To resolve this last issue, we refactor out dyn_write_to_js into it's own trait.

step 4

pub trait My_Jscode_Inner: My_Dyn_Jscode_Inner { ... } // this ensure we can call dyn_write_to_js 

impl <T: My_Jscode_Inner> My_Dyn_Jscode_Inner for T { ... } // provides actual impl

====

Is this a correct interpretation of how all this works / one arrives at @Yandros 's solution ?

1 Like

:100:


To expand on this (for other readers for whom this may still not be that clear), let's go over your two key questions:

Good question: here there are indeed some reasons regarding usage of the super-trait vs. usage of the blanket impl alone.

  • Tangent: an impl<T : A> B for T { … }, alone, suffices to have:
    • T : A imply that T : B;
    • without requiring that B be implemented as well every time we implement A (having trait A : B only would not have satisfied this requirement, for instance).
  1. A first reason, which is kind of a detail, but a handy one, is that a trait A : B { relationship gives you access to B's methods for impl A types, with only A in scope.

    That is:

    mod module {
        pub trait A : B {}
        pub trait B { fn method(&self) {} }
        impl<T : A> B for A {}
    }
    use module::A;
    
    impl A for i32 {}
    fn main() { 42.method(); }
    

    With trait A : B, like above, this compiles, but with trait A {} this would yield a "you need to use module::B" kind of error.

    In a way, the trait A : B { has a bit of magic, wherein the methods of B are implicitly embedded as part of the methods of A:

    trait B { fn method(&self); }
    trait A : B {
        // fn method(&self); /* provided by `: B` */
    }
    

    That's a key thing, because if we look at the desired nightly semantics, we wanted to have:

      trait My_Jscode_Inner {
          fn write_to_js(…) … where Self : Sized …
    
          …
    
    +     fn dyn_write_to_js(&self, …) …
      }
    

    which we can thus achieve with:

      trait My_Jscode_Inner
    +     : My_Jscode_Inner_Dyn
      {
         fn write_to_js(…) … where Self : Sized …
    
         …
      }
    
    + trait My_Jscode_Inner_Dyn {
    +     fn dyn_write_to_js(&self, …) …
    + }
    

    We should agree that both approaches effectively attach a .dyn_write_to_js() method to all impl My_Jscode_Inner types, including dyn My_Jscode_Inner.

So up until now we have an approach which works, but which seems unnecessarily convoluted, right? Now let's see why it is needed.

For starters, let's focus on a subtlety, which involves magic language semantics, and which @2e71828 accurately pointed out:

Basically, if we pay attention to the <T /* : Sized */> vs. <T : ?Sized> case, while keeping in mind that:

  • dyn Trait : !Sized;

  • dyn Trait : Trait through compiler magic.

    • (and when trait Trait : SuperTrait, we also have dyn Trait : SuperTrait as part of the same magic)

We can then indeed notice that my blanket impl does not cover ?Sized types:

- impl<T : ?Sized> My_Jscode_Inner_Dyn for T
+ impl<T> My_Jscode_Inner_Dyn for T
  where
      T : My_Jscode_Inner,

And you can observe this if you go and try to write impl My_Jscode_Inner for str (or some other unsized typed provided it not be dyn My_Jscode_Inner), then you'll run into "My_Jscode_Inner_Dyn is not implemented for str" kind of errors: Demo.

And yet dyn My_Jscode_Inner still works, as part of that autogenerated magic:

Click to see pseudo-code of autogenerated dyn Trait : Trait magic
//! pseudo-code of what the compiler generated

struct *mut (dyn My_Jscode_Inner + 'lt + AutoTraits…) {
    ptr: *mut ErasedTy,
    vtable: &'lt VTable_of_My_Jscode_Inner,
    _phantom: PhantomData<dyn 'lt + AutoTraits…>, // not Send unless Send part of autotraits, etc.
}

struct VTable_of_My_Jscode_Inner {
    example_method:
        unsafe fn(this: *mut ErasedTy, arg: Arg)
    ,
    dyn_write_to_js:
        unsafe fn(
            this: *const ErasedTy, 
            writer: &mut BufWriter<&mut dyn Write>,
            abs: &mut Vec<::js_sys::ArrayBuffer>,
        ) -> Result<(), My_Jscode_Err>
    ,
}

impl My_Jscode_Inner for dyn My_Jscode_Inner + '_ {
    /* we can skip the `where Self : Sized`-guarded functions */

    fn example(&mut self, arg: Arg) {
        let Self { ptr, vtable } = self;
        unsafe { (vtable.example)(ptr, arg) }
    }
        
}
impl My_Jscode_Inner_Dyn for dyn My_Jscode_Inner + '_ {
    fn dyn_write_to_js(
        &self, 
        writer: &mut BufWriter<&mut dyn Write>,
        abs: &mut Vec<::js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>
    {
        let Self { ptr, vtable } = self;
        unsafe { (vtable.dyn_write_to_js)(ptr, writer, abs) }
    }
}

and where the vtable is statically / at-compile-time generated and populated, for each concrete and Sized implementor, with what the <T : Sized> impls have provided:

impl<'lt, ConcreteImplementor : /* + Sized */>
    CoerceInto<dyn 'lt + My_Jscode_Inner>
for
    ConcreteImplementor
where
    ConcreteImplementor : 'lt + My_Jscode_Inner,
{
    fn coerce (this: *mut ConcreteImplementor)
      -> *mut (dyn 'lt + My_Jscode_Inner)
    {
        *mut (dyn 'lt + My_Jscode_Inner) {
            ptr: this.cast(), // erase the type
            vtable: &VTable_of_ My_Jscode_Inner { // const / static promotion
                dyn_write_to_js: |
                    ptr: *const Erased,
                    writer: &mut BufWriter<&mut dyn Write>,
                    abs: &mut &mut Vec<::js_sys::ArrayBuffer>,
                | -> Result<(), My_Jscode_Err> { unsafe {
                    // cast the erased thin ptr above back to its expected `ConcreteImplementor` type
                    let this: &ConcreteImplementor = &*ptr.cast();
                    // generate a statically dispatched call to the known function
                    this.dyn_write_to_js(writer, abs)
                }}
            },
        }
    }
}

All that to say, that Sized impl suffices to have the method be, nonetheless, available to dyn My_Jscode_Inners, thanks to the method being part of the trait thanks to it being part of a supertrait (here, My_Jscode_Inner_Dyn).


So there we have it: we dodged the error because our "default" forwarding-to-self.write_to_js() implementation had the chance to be dealing with a : Sized implementor of the trait (without preventing dyn Trait use it), whereas the more straightforward / naïve approach, alas, does not have that luxury:

trait My_Jscode_Inner /* for Self : ?Sized */ {
    fn write_to_js(…) -> … where Self : Sized;

    fn dyn_write_to_js(
        &self,
        writer: &mut BufWriter<&mut dyn Write>,
        abs: &mut Vec<js_sys::ArrayBuffer>,
    ) -> Result<(), My_Jscode_Err>
    {
        /* <Self : ?Sized + My_Jscode_Inner> context */
        self.write_to_js(writer, abs) // Error, `Self : Sized` is not satisified
    }

    …
}

Conclusion

We wanted a non-Self : Sized-gated function, but with its body / implementation being provided "only" for : Sized implementors, and leaving it be accessible to dyn Trait by having such a method be part of the trait, or of a super-trait.

Moreover, we wanted to have such body / implementation be automagically provided for implementations of My_Jscode_Inner, given that in the Sized case, it can trivially be implemented as forwarding to self.write_to_js().

Thence:

  • the approach, on nightly, to be using default impls to provided an implicitly : Sized-bounded such implementation;

  • the convoluted workaround, on stable Rust, of having to use a supertrait so that we could implement it, and the method associated with it, without colliding with an overlapping implementation of the sub trait.

    • should all the implementations of the sub-trait be under our control (blanket impl or sealed trait), in that case we could forgo the whole extra super-trait dance by forgoing the initial "automagically provided implementation" nicety, and manually implementing it as needed).
1 Like