Where does the vtable pointer go in Box<Trait>?


#1

I’m trying to understand how trait objects convert to and from raw pointers.

If I try to convert a trait-typed reference to a raw pointer, I get E0495, which complains about lifetimes without reference to the vtable. If I use a struct-typed reference instead, it works, which suggests that the problem is indeed related to the vtable pointer even though the compiler error doesn’t say so.

However, boxes work:

pub trait Decoder {
    fn max_utf16_buffer_length(&self, byte_length: usize) -> Option<usize>;
}

pub struct SingeByteDecoder {
}

impl Decoder for SingeByteDecoder {
    fn max_utf16_buffer_length(&self, byte_length: usize) -> Option<usize> {
        Some(byte_length)
    }
}

pub struct EucKrDecoder {
}

impl Decoder for EucKrDecoder {
    fn max_utf16_buffer_length(&self, byte_length: usize) -> Option<usize> {
        byte_length.checked_add(1)
    }
}

pub extern "C" fn new_euc_kr_decoder() -> *mut Decoder {
    Box::into_raw(Box::new(EucKrDecoder{}))
}

pub extern "C" fn new_single_byte_decoder() -> *mut Decoder {
    Box::into_raw(Box::new(SingeByteDecoder{}))
}

pub extern "C" fn ptr_to_decoder<'a>(decoder: *mut Decoder) -> &'a mut Decoder {
    unsafe {
        &mut *decoder
    }
}

fn main() {
    println!("EucKrDecoder {:?}", ptr_to_decoder(new_euc_kr_decoder()).max_utf16_buffer_length(0).unwrap());
    println!("SingeByteDecoder {:?}", ptr_to_decoder(new_single_byte_decoder()).max_utf16_buffer_length(0).unwrap());
}

(Playground link)

Why does it work? Where does the vtable pointer go?


#2

What does this mean exactly? How did you try this?

Trait objects use a fat ptr - it’s essentially a (usize, usize) pair where the first ptr is to the data and the second to the vtbl. A raw trait object ptr is the same thing.


#3

According to this excellent cheat sheet by Raph Levien, the vtable pointer is not on the heap.


#4

Like this. Note how only decoder_to_ptr causes an error but euc_kr_decoder_to_ptr does not.

Why doesn’t the compiler complain about improper C types when declaring *mut Decoder as the return value of an extern "C" function?


#5

I think your errors are not related to the vtable but really to lifetimes. If you write trait Decoder : 'static { ... } (ruling out e.g. decoders containing stack pointers), then the errors disappear.

I don’t know.


#6

Note that the Decoder in &'a Decoder is more restrictive than the one in *mut Decoder. Regarding the actual type T of the object (which implements Decoder), the former requires that it outlives 'a but the latter requires that it outlives everything. Your error seems to disappear when you change the return type:

pub fn decoder_to_ptr<'a>(decoder: &'a mut Decoder) -> *mut (Decoder + 'a) { ... }


#7

You could get it to work with as *const _ as *mut _

nomicon
*T as *U TODO: explain unsized situation


#8

It probably should issue a warning, or perhaps clippy is the right place for that. Technically, C code can decompose it into the 2 parts and store it as context - then pass it back somewhere else to Rust. Trait object repr isn’t guaranteed however, and so that’s why there probably should be a lint against it.


#9

Is the ABI defined? How should the C side be declared?


#10

The ABI is unstable but current impl is known - it’s a fat ptr. The fat ptr is two usize values, the first being a ptr to the data (ie the starting memory address where the value resides) and the second being a ptr to the vtbl. So the C side could be a struct with two *void ptrs, which is basically https://doc.rust-lang.org/std/raw/struct.TraitObject.html.