Constructing raw pointers to `dyn SomeTrait`

[davnavr]

Let's say I have a trait object dyn MyTrait that I know is safe to move, and want to move it to a new memory location (a [NonNull<()>] or *mut () that I could have allocated with std::alloc::alloc()), and then have a pointer NonNull<dyn MyTrait> to the recently moved object. How can I go from a NonNull<()> to a NonNull<dyn MyTrait> in stable Rust?

// Definitely won't work on stable Rust
fn do_magic(pointer: NonNull<()>, vtable: ???) -> NonNull<dyn MyTrait>;

With Nightly Rust

• strict_provenance has the std::ptr::metadata() and std::ptr::from_raw_parts() that can be used to make a *const MyTrait with a specific VTable
  • Unfortunately, it's unlikely to come out soon, and its stable polyfill hasn't been updated in a year and it does not provide pointer metadata functions.
• trait_upcasting and coerce_unsized might be helpful here.

With Stable Rust

I had an idea to use traits that convert the NonNull<()> to a NonNull<Self> in order to "get" the correct VTable, something like:

use std::ptr::NonNull;

trait MyTrait {
    fn my_trait(&self);
}

trait Reinterpret {
    fn reinterpret(pointer: NonNull<()>) -> NonNull<dyn MyTrait>;
}

impl<T: MyTrait + 'static> Reinterpret for T {
    fn reinterpret(pointer: NonNull<()>) -> NonNull<dyn MyTrait> {
        // Note that here, T: Sized, which allows this cast to work
        pointer.cast::<T>() as NonNull<dyn MyTrait>
    }
}

// But how?
fn magic(source: NonNull<dyn MyTrait>, pointer_to_convert: NonNull<()>) -> NonNull<dyn MyTrait> {
    todo!()
}

Unfortunately, I couldn't figure out how to get the trait strategy above to work with dyn MyTrait (e.g. having a &self parameter so Reinterpret could be a trait object):

use std::ptr::NonNull;

trait MyTrait {
    fn my_trait(&self);
}

trait Cast {
    fn cast(&self, pointer_to_convert: NonNull<()>) -> NonNull<dyn MyTrait>;
}

impl<T: MyTrait + 'static> Cast for T {
    fn cast(&self, pointer_to_convert: NonNull<()>) -> NonNull<dyn MyTrait> {
        // Note that here, T: Sized, which allows this cast to work
        pointer_to_convert.cast::<T>() as NonNull<dyn MyTrait>
    }
}

// This is required, since Cast is not implemented for any Unsized types
impl Cast for dyn MyTrait {
    fn cast(&self, pointer_to_convert: NonNull<()>) -> NonNull<dyn MyTrait> {
        // Can't get VTable at all, back where we started
        todo!()
    }
}

fn magic(source: &dyn MyTrait, pointer_to_convert: NonNull<()>) -> NonNull<dyn MyTrait> {
    // Fails to compile, 'static lifetime requirement!?
    source.cast(pointer_to_convert)
}

In the above attempt, problems with trait lifetimes appeared, similar to the ones below:

Argument requires that ... is borrowed for `'static`

I have a problem with lifetimes, which seems pretty simple yet I couldn't get my head around it. Here is the code (playground): struct Encoder {} impl Encoder { pub fn pass<'a>(&'a mut self) -> RenderPass<'a> { RenderPass { parent: self } } } struct RenderPass<'a> { parent: &'a Encoder, } impl<'a> RenderPass<'a> { pub fn set(&mut self, _x: &'a i32) {} } fn main() { let value = 1; render(|pass| { pass.set(&value); }); } fn render(mut f: impl…

(Just Google "rust trait requires self to be borrowed for static" or something similar)

Workarounds

• Make a custom struct for dyn MyTrait, like with std::task::RawWaker?
  • A struct could store a cast fn and a pointer to the trait object:

    struct MyTraitRef {
        cast: unsafe fn(NonNull<()>) -> NonNull<dyn MyTrait>,
        object: NonNull<()>,
    }
    
    impl Deref<dyn MyTrait> for MyTraitRef {
        type Target = dyn MyTrait;
    
        fn deref(&self) -> &dyn MyTrait {
            // Safety: Who knows?
            // Safety: Assumes that the cast function is getting a pointer to an object of the correct type
           unsafe { (self.cast)(self.object).as_mut() }
        }
    }
    

  • This means a new struct might have to be made for every trait I want to work with, unless MyTraitRef<Trait: ?Sized> works.
• Transmute NonNull<dyn MyTrait>/&dyn MyTrait/*const MyTrait into a [*const (); 2] and do war crimes to keep the VTable pointer and change the address (Literally Undefined Behavior, since the layout of dyn MyTrait is not stable)

Thanks for the help and suggestions!

[quinedot]

The ptr_metadata feature^[1] probably is the feature you wish was stable.

---

@davnavr

In the above attempt, problems with trait lifetimes appeared, similar to the ones below:

This lets it compile:

-fn magic(source: & dyn MyTrait           , pointer_to_convert: NonNull<()>) -> NonNull<dyn MyTrait> {
+fn magic(source: &(dyn MyTrait + 'static), pointer_to_convert: NonNull<()>) -> NonNull<dyn MyTrait> {

Because after filling in elision you had this:

impl Cast for dyn MyTrait + 'static {
    //        ^^^^^^^^^^^^^^^^^^^^^ = Self
    //                     vvvvvvvvvvvvvvvvvvvvv 
    fn cast<'c>(self: &'c (dyn MyTrait + 'static), pointer_to_convert: NonNull<()>) -> NonNull<dyn MyTrait> {
        todo!()
    }
}

fn magic<'m>(source: &'m (dyn MyTrait + 'm), pointer_to_convert: NonNull<()>) -> NonNull<dyn MyTrait> {

And you can't extend 'm to 'static because perhaps the type which was erased into dyn Trait + 'm is not well-formed at longer lifetimes (maybe &'m mut String: MyTrait, say).

---

If you make Cast a supertrait of MyTrait, the compiler will supply the implementation for you. Or with more lenient lifetimes.

Not sure right now if this actually gets you what you want, but hope it helps.

---

1. which is not the same thing as the strict provenance experiment ↩︎

[davnavr]

Thanks, I didn't know the Rust compiler would automatically provide an impl if Cast is a supertrait, however I am interested in constructing raw pointers for arbitrary traits (e.g. dyn Debug, dyn Error, dyn Write, etc.), and am doing this as an experiment to see if I can get traits like FnOnce working on stable Rust without forcing heap allocation.

Right now, I am considering the somewhat cursed idea of using core::slice::from_raw_parts() and friends to store a reference to a VTable struct in the "length" portion, like so:

/// A manual implementation of a [trait object]'s virtual method table.
pub struct TraitVTable<T: ?Sized> {
    cast: fn(NonNull<()>) -> NonNull<T>,
}

/// A manual implementation of a [trait object], also known as `dyn`.
#[repr(transparent)]
pub struct TraitObject<T: ?Sized> {
    // Stores the actual trait object, and the "metadata" portion of pointers to TraitObject<T>
    // will be cast from a `usize` to a `VTable pointer`
    data: [core::mem::MaybeUninit<u8>],
}

impl<T: ?Sized> AsRef<T> for TraitObject<T> {
    fn as_ref(&self) -> &T {
        // Safety: len is the cast 
        let vtable = unsafe {
            &*(self.data.len() as *const TraitVTable<T>)
        };

        // Safety: data and vtable match, so this should be a valid reference
        // Safety: data pointer is never null
        let data_ptr = unsafe {
            NonNull::new_unchecked(self.data.as_ptr() as *mut ())
        };

        // Safety: safe to dereference, will live for lifetime of &self
        unsafe {
            (vtable.cast)(data_ptr).as_ref()
        }
    }
}

// TODO: Figure out how to add constructors for TraitObject<dyn SomeTrait> later

Hypothetically, the above code should allow &TraitObject<dyn SomeTrait> to be a fat pointer just like
&dyn SomeTrait, except that the metadata part of &TraitObject<dyn SomeTrait> would be a usize "length" that is actually a pointer to some custom VTable or a function pointer to something that can allow one to obtain a &dyn SomeTrait from the TraitObject (e.g. with the AsRef implementation).

How safe would the above code be? Later, I'm going to also try to figure out how to actually create a constructor that can take something that implements SomeTrait and turn it into a TraitObject<dyn SomeTrait> (and maybe create some helpers to make Box<TraitObject<_>>).

[H2CO3]

@davnavr

I didn't know the Rust compiler would automatically provide an impl if Cast is a supertrai

I don't think that is correct. The compiler doesn't infer implementaions from mere declarations. In the above code, there is a blanket impl that generically provides the function for all types, but that's explicit.

[quinedot]

@davnavr

How safe would the above code be?

War-crime, unsound, UB territory. You would need to at a minimum stop using references (&TraitObject) as that implies you have a valid slice with the length of some vtable address, which you do not.

If you're ok storing two usizes (the size of a &[]), why aren't you just storing *const dyn Trait? You want to erase which trait it was I guess? Then get ahold of the vtable address and store it as a *const ().

---

There are AFAIK not-UB ways to get at the vtable address conditionally by assuming it's either a [data_ptr, vtable_ptr] or a [vtable_ptr, data_ptr] and checking to see which it is and failing if you can't determine it's one or the other (which would be rare). But AFAIK without the ptr_metadata feature,^[1] there's no blessed way to get the vtable pointer yet.

Namely, there's a safe way to get the data pointer..

ptr /*: *const dyn Trait */ as *const ()

...and if your *const dyn Trait is the size of two pointers you can transmute it to [*const (); 2]^[2] and then compare values for the unequal one. In fact with this FCP complete, you can start with a [SizeOneAlignOneTypeThatImplsYourTrait; 2] and get a couple *const dyn Trait and the data pointer of at least one of them won't be word aligned, so it should be unambiguous.

You'll have to remember what the order is when per-trait going the other direction.^[3] You could pack that information in to the low bit of the vtable pointer.

---

This is still pretty cursed so it's possible I missed something.

---

1. or some minimized version of it ↩︎

2. transmute checks sizes so this will fail in some world where we have dyn Trait with more than one inline pointer ↩︎

3. It's not guaranteed to be the same across different dyn types. ↩︎

[davnavr]

Yeah, I also realized that using a slice and sneaking in something into the "length" portion of [T] would really mess up layout related stuff std::alloc::Layout::for_value, std::mem::size_of_val and std::mem::align_of_val, either returning a size that doesn't make sense at all, or returning 0 if I used [()].

A new approach I'm trying now is using a struct as a replacement for dyn MyTrait:

// Behaves like a NonNull<dyn MyTrait>
struct TraitPointer {
    pointer: NonNull<()>,
    cast: fn(NonNull<()>) -> NonNull<MyTrait>,
}

The above should allow easy convertion to and from Rust's pointer types (&, *const, their mut variants, and NonNull pointers to dyn MyTrait) and TraitPointer<dyn MyTrait>