FFI, Boxes, and returning references to the Boxed data

#1

Hi all,

I’m trying to wrap a 3rd-party C API and I hit a snag. The C API accepts a void* for user-specified data and associates that pointer with an object. The C API does not do anything with the pointer, no dereferencing or freeing, it just sets and gets the pointer. From the Rust side, I think I would want to use a Box<Any> to set the pointer, and return references with a lifetime of the wrapped object to the downcasted data. I’d also want the data to be dropped correctly when the object is dropped.

I created a non-working playground demo of the interface I’m trying to achieve, modeling the C calls as unsafe functions that get/set static data. It produces the same error I’m getting in my real code, a panic when calling unwrap because the get_foreign_ref method returns None. I don’t understand why it returns None though.

I’m hoping that someone here can send me down the right path. And if there is a better/alternative/idiomatic way to model this interface, please don’t hesitate to suggest it.

Thanks!

#2

Your testcase boils down to something like this:

use std::any::Any;

struct MyData {
    i: i32,
}

fn main() {
    let p = Box::new(MyData {i: 1});
    let p = p as Box<Any>;
    let p = Box::into_raw(p);
    let p = p as *mut u8 as *mut Any; // This line is wrong.
    let p = unsafe { (&*p) }.downcast_ref::<MyData>();
    let p = p.unwrap();
    println!("i is {}", p.i);
}

The immediate problem is the cast from *mut Any to *mut u8 and back. The compiler should probably warn about it… but basically the problem is that the cast isn’t lossless: *mut Any is actually two pointers, but *mut u8 is only one. See https://doc.rust-lang.org/nightly/book/trait-objects.html#representation .

The fundamental issue here is that you shouldn’t be using Any here at all; you should just do something like this:

impl ApiWrapper {
    unsafe fn set_foreign_ptr<T>(&self, ptr: Box<T>) {
        let raw = Box::into_raw(ptr);
        api_set_fp(raw as *mut ());
    }

    // This assumes you call get() exactly once for each time you call set().
    unsafe fn get_foreign_ptr<T>(&self) -> Box<T> {
        let raw = api_get_fp();
        let b = Box::from_raw(raw as *mut T);
        return b;
    }
}
#3

I see. Trait objects are fat, and I knew that at one point in time. Returning the box from get_foreign_ptr works, but then a user would almost always want to follow that up with another call to set_foreign_ptr. Not bad, but not ideal either. Is there any way to get the interface I described, sans Any (repeated below for reference)?

impl ApiWrapper {
    fn set_foreign_ptr<T>(&self, ptr: Box<T>) { ... }    
    fn get_foreign_ref<T>(&self) -> Option<&T> { ... }
    fn get_foreign_mut<T>(&self) -> Option<&mut T> { ... }
}
#4

The KISS way is to double-box (play):

impl ApiWrapper {
    fn set_foreign<T: Any>(&mut self, value: Box<T>) {
        self.free_foreign();
        unsafe {
            let raw = Box::into_raw(Box::new(value as Box<Any>));
            api_set_fp(raw as *mut ());
        }
    }
    
    fn get_foreign_ref<T: Any>(&self) -> Option<&T> {
        unsafe {
            let raw = api_get_fp() as *const Box<Any>;
            if !raw.is_null() {
                let b: &Box<Any> = &*raw;
                b.downcast_ref()
            } else {
                None
            }
        }
    }
    
    fn get_foreign_mut<T: Any>(&mut self) -> Option<&mut T> {
        unsafe {
            let raw = api_get_fp() as *mut Box<Any>;
            if !raw.is_null() {
                let b: &mut Box<Any> = &mut *raw;
                b.downcast_mut()
            } else {
                None
            }
        }
    }
    
    fn free_foreign(&mut self) {
        unsafe {
            let raw = api_get_fp() as *mut Box<Any>;
            if !raw.is_null() {
                Box::from_raw(raw);
            }
        }
    }
}

impl Drop for ApiWrapper {
    fn drop(&mut self) {
        self.free_foreign();
    }
}

struct MyData {
    i: i32,
}

impl Drop for MyData {
    fn drop(&mut self) {
        println!("Dropping MyData with value {}", self.i);
    }
}

This is safe if you don’t ever have two wrapper instances of the same foreign object (in particular ApiWrapper must not implement Clone). I’m assuming there actually are objects you attach pointers to, not a single static variable like in the original demo.

#5

Thanks @gkoz, that’s exactly what I was looking for. At one point I had a solution close to this, but I wandered off in a different direction. You are correct, there are actual objects that the pointers are attached to, not a single static variable. It was just easiest to demonstrate like that.