How to safely deal with C uint8_t* or char* in Rust

I need to pass uint8_t* buffers (generally unsigned char*) from C to Rust. I'm lost as there seems to exist many ways to dealing with pointers in Rust.

I wanted to pass the buffers from C to Rust and store them in a queue, which I think a VecDeque would be good for. I also want these buffers to be auto managed. That is, get deleted when go out of scope, as I'm gonna pass them from C and then consume them in Rust. I also don't know how to create a slice from a C buffer in Rust.

Looks like there exists https://doc.rust-lang.org/std/slice/fn.from_raw_parts.html that returns but it isn't clear if it would delete a buffer passed from C. I also don't understand the alignment requirements.

Anyways, how to obtain a uint8_t* buffer from C in Rust and to which type I should convert it to be safe for usage in Rust?

I'm doing

#[no_mangle]
pub extern "C" fn receiveBufferAndPrint(buffer: *const u8, size: usize)
{
    for i in 0..size {
        println!("{}", unsafe { *buffer.offset(i as isize) });
    }
}

Is this a good way to receive the pointer? What about converting it to a self managed slice?

One of the rules of the FFI is, if you allocate here you must free it here. You can't allocate buffer on C and free it on Rust.

There's two common workarounds here. First one is to allocate im C, and in Rust take this ptr and wrap it with some special type, and call cleanup code written in C on its Drop impl.

Second one is to allocate in Rust, pass its ptr to some C function to fill its content.

1 Like

isn't there a way to deallocate on Rust calling libc? Of course it'd be done on C side but just so I don't write a function only for this.

Also, I'm kinda new to Rust. As I see there are no destructors. One way would be to implement the Drop trait for this new type. Do you recommend doing this?

I want my program to be safe, no memory leaks. Can you give me a description of what to do? Should I use unsafe { *buffer.offset(i as isize) } everytime I want to access this buffer or can I convert to a slice? In which object I should place the pointer/slice to make it automatically dealloc when gone from scope?

Not sure what exactly you mean by "destructors", but in Rust, Drop is the implementation of what is usually called a destructor in C++.

And yes, using Drop for resource management is the idiomatic thing to do.

By the way, as far as I know, you are allowed to rely on C's uint8_t being the same as Rust's u8 (just like e.g. size_t is assumed to be the same as usize), you could just copy the buffer from C to Rust:

use std::ptr::copy_nonoverlapping;

extern "C" {
    fn some_c_function(out_len: *mut usize) -> *const u8;
}

let mut buf_len: usize = 0;
let buf_ptr: *const u8 = unsafe {
    some_c_function(&mut buf_len)
};
let mut v = Vec::<u8>::with_capacity(buf_len);
unsafe {
    copy_nonoverlapping(buf_ptr, v.as_mut_ptr(), buf_len);
    v.set_len(buf_len);
}

It is in fact clear: it returns a slice which is a reference pointing into an array. Hence, it's a non-owning type. It therefore does not free the underlying pointer. (Which might not even need any freeing for that matter. You can create a slice from a stack-allocated array.)

Do you not understand them specifically for this function, or do you not know in general what alignment requirements are?

as I understood, your example does element by element copy of the buffer. To be fast I'd like to simply create an object that holds the pointer or a slice and frees it by itself on the destruction. However I don't want to do copies because it'd be slow.

So what I'd like: an object that holds a *const u8, be it in slice form or raw form, and deletes it on destruction. I think that storing it as a slice using from_raw_parts is good because then I can access it safely from Rust, having boundary checks.

I guess that what I'd do would be simply a struct that holds a slice, and implements the destructor trait, which frees the pointer using delete[] which would be a simple C++ function (wrapped in a C interface).

Do you think it's safe to do this way? I see no risks of extrapolating the boundary as slices have boundary check, and I see no way to have memory leaks as the destruction always happens and so delete[] always happens.

I also need to pass Rust buffers to C. The Rust buffers are simply [u8] for which I think I can extract slices. If I pass them to C (how?) then how should I free them in C when they're consumed?

Thank you for your time and help!

From your original post it would seem you are more concerned about safety than speed – rightfully. Did you measure that copying a flat byte buffer is indeed what slows your code down? memcpy() is incredibly fast these days.

Here it is:

use std::isize;
use std::slice;
use std::ops::{ Deref, DerefMut };

struct CBuffer {
    ptr: *mut u8,
    len: usize,
}

impl CBuffer {
    /// Constructor. Transfers ownership of `ptr`.
    pub unsafe fn from_owning(ptr: *mut u8, len: usize) -> Option<Self> {
        if ptr.is_null() || len > isize::MAX as usize {
            // slices are not allowed to be backed by a null pointer
            // or be longer than `isize::MAX`. Alignment is irrelevant for `u8`.
            None
        } else {
            Some(CBuffer { ptr, len })
        }
    }
}

impl Drop for CBuffer {
    /// Destructor.
    fn drop(&mut self) {
        unsafe {
            libc::free(self.ptr);
            // for example. Might need custom deallocation.
        }
    }
}

impl Deref for CBuffer {
    type Target = [u8];

    fn deref(&self) -> &Self::Target {
        unsafe {
            slice::from_raw_parts(self.buf as *const u8, self.len)
        }
    }
}

impl DerefMut for CBuffer {
    fn deref_mut(&mut self) -> &mut Self::Target {
        unsafe {
            slice::from_raw_parts_mut(self.ptr, self.len)
        }
    }
}

A slice &[T] is explicitly convertible or implicitly coercible to a raw slice *const [T] which in turn is explicitly convertible to a pointer-to-first-element. There's also the convenience method <[T]>::as_ptr(). So you can easily pass a slice to a C function like this:

let slice: &[u8] = &[1, 2, 3, 4];
unsafe {
    c_function_taking_ptr_and_length(slice.as_ptr(), slice.len());
}

When you pass to a C function a slice pointing into memory that is owned by a Rust data structure, then you must not free it from C.

If you want to transfer ownership, you need to convert your data structure into a "forgetful" raw pointer (cf. Box::into_raw() or Vec::into_raw_parts()), use that in C, and when it's time to destroy the value, you need to pass it back to Rust, convert it back to an owning data type (cf. Box::from_raw() and Vec::from_raw_parts()), and let Rust free it. You can always write an extern "C" function in Rust for this purpose.

Very nice! I understood everything you said, thank you so much.

I see that you call libc::free. I guess it does not call delete[] but instead calls free which gives undefined behaviour for things allocated with new, isn't it?

is there a reason to store *mut u8 directly instead of storing a slice? I don't see why calling deref everytime helps. Or maybe there's no penalty difference in doing one or another

The libc::free call assumes that the buffer was allocated with malloc, yes. If it was allocated in C++ using new, you would have to create your own extern C function that just contains a delete[] operation, and call that C function from the Drop impl.

The reason for storing a raw pointer instead of a slice is that a slice is fundamentally a borrowed type, which does not own its contents, and you can't put a slice into a struct without having the struct suddenly get lifetime annotations all over. The raw pointer is the idiomatic solution for creating owned buffers with special deallocation like we have here.

Calling deref on every access should not have any runtime penalty.

1 Like

But that's exactly why I wrote the comment "For example. Might need custom deallocation". You have given zero information about how the buffer is allocated, so I can't guess the exactly correct way of deallocating it. The code I provided was meant to be an example, that you can understand and build your own abstraction upon. It was not meant to be a fully-fledged, worked-out, complete, tested solution.

And yes, in the context of C++, calling free() on a pointer obtained by new, and vice versa (calling delete on a pointer obtained from malloc()) is undefined behavior, and the same distinction actually applies to new-vs-new[] and delete-vs-delete[], too.

2 Likes