Casting from &dyn T to &dyn U, where U: T [SOLVED]

[kaiba42]

Greetings!

This is my first post here.. so please help me make this more clear if you find it is not

I've been working on an interesting problem, where I need to cast a pointer to a trait object (&dyn T) to a point of another trait object (&dyn U), where U is a sub-trait of T. Here is a playground link illustrating the problem, and a potential solution: Rust Playground

Here's the meat of the transformation, where T = SuperTrait, and U = SubTrait, and ConcreteType implements both T and U. The type FatPtr is an alias for (usize, usize).

    impl std::convert::TryFrom<&dyn SuperTrait> for &dyn SubTrait {
        type Error = &'static str;
        fn try_from(super_trait: &dyn SuperTrait) -> Result<Self, Self::Error> {
            let fake_sub_trait = MaybeUninit::<ConcreteType>::uninit();
            let (_data_ptr, vtable_ptr) = unsafe { transmute::<&dyn SubTrait, FatPtr>(&fake_sub_trait.assume_init() as &dyn SubTrait) };
            let (data_ptr, _vtable_ptr) = unsafe { transmute::<&dyn SuperTrait, FatPtr>(super_trait) };
            let sub_trait = unsafe { transmute::<FatPtr, &dyn SubTrait>((data_ptr, vtable_ptr)) };
            Ok(sub_trait)
        }
    }

I'm well aware just how unsafe this is.. but I'm most afraid of this line of code:

    let (_data_ptr, vtable_ptr) = unsafe { transmute::<&dyn SubTrait, FatPtr>(&fake_sub_trait.assume_init() as &dyn SubTrait) };

My fears stem from this piece of documentation concerning std::mem::MaybeUninit, found here: MaybeUninit in std::mem - Rust

My question to you is, given &dyn SuperTrait is really a pointer to a ConcreteType that was casted, does this line of code above still trigger undefined behavior?

Thanks for reading!

[RustyYato]

Yes, any use of uninitialized memory is UB. Also, there is no general way to go from a "super" trait to a "sub" trait in Rust unless you have a method in the super trait that like so,

trait Super {
    fn try_as_sub(&self) -> Option<&dyn Sub> { None }
}

trait Sub: Super {}

[kaiba42]

Right -- but since I never use the pointer to uninitialized memory (_data_ptr), am I safe from UB? Thanks for replying btw

[RustyYato]

You are using it, assume_init is considered as a use.

[kaiba42]

Aaah I see

[RustyYato]

Also the two trait objects will likely have different layouts, so transmuting between them is also UB.

[kaiba42]

Hmm.. does the fact that I am only transmuting the fat pointer, and not the actual data, keep me away from UB due to memory layout differences?

[RustyYato]

Well, if it was a raw pointer maybe, but you are transmuting references, and they must always be always valid.

Also, tuple layouts are unstable, so the second field is not guaranteed to be laid out second in memory.

With raw pointers you can do,

#[repr(C)]
struct TraitObject {
    data_ptr: *const (),
    vtable: *const ()
}

unsafe fn try_from(super_trait: &dyn SuperTrait) -> Result<Self, Error> {
    let fake_sub_trait = MaybeUninit::<ConcreteType>::uninit();
    let trait_object = std::mem::transmute::<*const SubTrait, TraitObject>(fake_sub_trait.as_ptr() as *const SubTrait);
    let vtable = trait_object.vtable;
    let data_ptr = super_trait as *const SuperTrait as *const ();
    let sub_trait = transmute::<TraitObject, &dyn SubTrait>(TraitObject {
        data_ptr,
        vtable
    });
}

To soundly get the vtable of ConcreteType and construct SubTrait. Note that this entire operations is unsound if ConcreteType is not the type behind the inside the trait object in, so you can't put this in TryFrom

[kaiba42]

Great points, this is so much better. One question though -- is fake_sub_trait guaranteed to be dropped once the function returns?

[RustyYato]

fake_sub_trait won't be dropped because MaybeUninit<T> doesn't drop T, because it may not be initialized. Meaning, fake_sub_trait won't run any destructors when it goes out of scope.

You could make this generic,

unsafe fn try_from<Concrete: SubTrait>(super_trait: &dyn SuperTrait) -> &dyn SubTrait {
    let fake_sub_trait = MaybeUninit::<Concrete>::uninit();
    let trait_object = std::mem::transmute::<*const SubTrait, TraitObject>(fake_sub_trait.as_ptr() as *const SubTrait);
    let vtable = trait_object.vtable;
    let data_ptr = super_trait as *const SuperTrait as *const ();
    let sub_trait = transmute::<TraitObject, &dyn SubTrait>(TraitObject {
        data_ptr,
        vtable
    });

    sub_trait
}

And this is sound on the condition that super_trait must contain a value of type Concrete

[kaiba42]

this is what I was trying to get to! thank you!

[RustyYato]

Actually, if we already know what the concrete type is, and we should if we want to cast the super trait to the sub trait in this way, then we could just do this.

unsafe fn try_from<Concrete: SubTrait>(super_trait: &dyn SuperTrait) -> &dyn SubTrait {
    &*(super_trait as *const dyn SuperTrait as *const Concrete as *const dyn SubTrait)
}

If you don't know what the types are going to be, then you should instead add this method to SuperTrait

trait SuperTrait {
    // ... other methods
    fn to_sub(&self) -> Option<&dyn SubTrait>;
}

trait SubTrait: SuperTrait {
    // ... other methods
}

And this will be 100% safe, only issue with this one is that it can get a bit annoying to write out to_sub

[kaiba42]

wow, very clean. All makes great sense

[RustyYato]

If you are willing to use nightly Rust, you can even mitigate the annoying bit of to_sub!

#![feature(specialization)]

trait SuperTrait {
    fn to_sub(&self) -> Option<&dyn SubTrait> { None }
}

trait SubTrait: SuperTrait {

}

impl<T: SubTrait> SuperTrait for T {
    fn to_sub(&self) -> Option<&dyn SubTrait> {
        Some(self)
    }
}

And when you implement SuperTrait you just don't need to think about to_sub! It will be automatically handled for you.

---

edit: one issue with specialization that I just realized, it forces you to implement all methods of the SuperTrait, which is unfortunate. To mitigate this, we can add yet another trait, playground

[kaiba42]

aaah i dream of specialization.. I'm stuck with stable for my project, so I will have to go with the previous solutions you mentioned, although this is the most elegant.

[RalfJung]

@kaiba42

Right -- but since I never use the pointer to uninitialized memory (_data_ptr), am I safe from UB?

No. Producing invalid data (such as a bad bool or an uninitialized integer) is UB, no matter whether the data is ever used.

Also notice that wide ptr layout (for &dyn Trait) is unstable and unspecified and can change any time. Code that makes assumptions about vtable pointers can thus break any time---even if the assumptions are correct right now, the might not be correct any more with the next Rust version.

[kaiba42]

Thanks for your insight! I've removed all usage of MaybeUninit in my code as a result of this thread. Is there an approach you would consider using to accomplish &dyn SuperTrait to &dyn SubTrait transformation?

[RalfJung]

I don't think there currently is a good way to do this, other than the to_sub method suggested above.

But I also know way more about what you can not do than what you actually can do.

[kaiba42]

haha many thanks!