Best way to receive C++ uint8_t* buffer on Rust

#1

Suppose I have a C++ allocated buffer uint8_t* and I want to access it from Rust:

One way would be to have the C++ function:

uint8_t receive(uint8_t** data, size_t* size) {
    //allocates the data, writes to it and then points *data to it
    return 0;//on success
}

however this leaves Rust responsible for dealocating the data. Same for

uint8_t* receive(size_t* size) {
    uit8_t* data = //allocates data
    *size = data_size;
    return data;
}

The idea I have is to allocate the data on Rust and then pass a pointer for C++ to fill:

uint8_t receive(uint8_t* data, size_t size) {
    //fills data up to size
    return 0;
}

However, we cannot simply allocate a buffer in Rust and expect it to be C/C++ compatible. Also it has the limitation of having to know a size in advance, or use a sufficiently big size for all buffers.

What would be the best solution for this?

#2

You can make Rust deallocate the data if you provide an FFI based function that internally deallocates it. Then Rust can call that function with the pointer, deallocating it properly. You can even make a custom Rust struct that does this in its destructor.

1 Like
#3

This is the best approach as allocation ownership does not cross languages.

let size = 1024;

// allocate a zeroed buffer of size
let mut buf = vec![0u8; size];

let code = unsafe {
    recieve(buf.as_mut_ptr(), buf.len())
};

if code == 0 {
    // use buf
}

// buf gets dropped (unless moved)
3 Likes
#4

Note that @s3bk answer is the best suggestion here.

Nevertheless, an interesting alternative is to use the CPS/callback pattern, which ensures the caller does not forget to free it:

Rust usage

with_receive(|buf: &'_ mut [u8]| Ok({
    println!("{:#x?}", buf);
}))?;

Implementation

extern "C" { // Don't forget this when doing FFI with C++
    typedef uint8_t status_t;

    typedef status_t (*cb_t)(void * cb_ctx, uint8_t * data, size_t size);

    status_t with_receive (void * cb_ctx, cb_t cb_fun)
    {
        uint8_t * data = nullptr;
        size_t size = 0;
        uint8_t status = ~0;
        /* allocate and fill the buffer, set status accordingly */
        if (status == 0) { // assuming 0 stands for success.
            status = (*cb_fun)(cb_ctx, data, size);
            free(data);
        }
        return status;
    }
}

and in Rust:

fn with_receive<R, F> (f: &'_ mut F) -> Result<(), ()>
where
    F : FnMut(&'_ mut [u8]) -> Result<(), ()>,
{
    type status_t = u8;
    type cb_t = Option<
        unsafe extern "C"
        fn(cb_ctx: *mut c_void, data: *mut u8, size: usize)
          -> status_t
    >;
    extern "C" {
        fn with_receive (cb_ctx: *mut c_void, cb_fun: cb_t)
          -> status_t
        ;
    }

    unsafe extern "C"
    fn cb_fun<F> (cb_ctx: *mut c_void, data: *mut u8, size: usize)
      -> status_t
    {
        macro_rules! unwrap { ($opt:expr) => (
            if let Some(it) = $opt {
                it
            } else {
                return !0;
            }
        )}
        let slice = ::core::slice::from_raw_parts_mut(
            // ptr
            unwrap!(::core::ptr::NonNull::new(data)).as_ptr(),
            size,
        );
        let f: &'_ mut F = unwrap!(cb_ctx.cast::<F>().as_mut());
        unwrap!(f(slice).ok());
        0
    }

    match with_receive(<*mut _>::cast(f), Some(cb_fun::<F>)) {
        | 0 => Ok(()),
        | _ => Err(()),
    }
}
#5

Also an option, as long as the correct function to free the data is documented in the api.
Then you could create a custom struct in Rust that holds the data:

struct CBuf {
    ptr: *mut u8,
    size: usize
}
impl Deref for CBuf {
    type Target = [u8];
    fn deref(&self) -> &[u8] {
       unsafe {
           std::slice::from_raw_parts(self.ptr, self.size)
       }
    }
}
impl Drop for CBuf {
    fn drop(&mut self) {
        unsafe {
            // from C
            free(self.ptr);
        }
    }
}
#6

Each approach has its problems

Alloc in C, pass to Rust, dealloc in Rust

  • Using malloc/free requires an allocation each time. This can hurt performance.
  • It makes the allocator part of the API. (hurts future improvements)
  • Easy to introduce leak/use after free

Alloc in Rust, fill buffer in C, dealloc in Rust.

  • requiress known buffer size in advance
  • enables reusing previous allocations
  • enables using fixed size, stack allocated arrays (no alloc, no dealloc)

In performance critical code you almost always (re-)use a preallocated buffer.

#7

IIUC, there's the potential for UB here if the C and Rust programs use different memory allocators. Memory has to be deallocated by the same allocator that allocated it.
Granted, it's not very often that you decide to use a different allocator, even more so for only one of the programs. But it's worth noting.

1 Like
#8

Definitly worth noting!
The allocator becomes part of the API and has to be used correctly or dragons rise up.

#9

It's quite common for a C API to provide XXX_free() functions which clean up a complex business object, so we'd just extend that to our buffer.

In this case you just need to make sure that the library which creates an object also provides a destructor, and you won't have any allocator mismatch problems.

You'll naturally fall into this explicit destructor pattern when writing FFI code (e.g. because you may need to pass ownership of Rust types to C), so it ends up being a non-issue in practice.

1 Like