Understanding Rust memory allocation API?

[vedantroy]

I was looking at this post, which explains how to return a byte array from Rust to C.

The relevant code:

#[repr(C)]
struct Buffer {
    data: *mut u8,
    len: usize,
}

extern "C" fn generate_data() -> Buffer {
    let mut buf = vec![0; 512].into_boxed_slice();
    let data = buf.as_mut_ptr();
    let len = buf.len();
    std::mem::forget(buf);
    Buffer { data, len }
}

extern "C" fn free_buf(buf: Buffer) {
    let s = unsafe { std::slice::from_raw_parts_mut(buf.data, buf.len) };
    let s = s.as_mut_ptr();
    unsafe {
        Box::from_raw(s);
    }
}

and I noticed that the free_buf function takes in a Buffer instead of doing:

pub unsafe extern "C" fn free_buf(ptr: *mut u8) {
    Box::from_raw(ptr);
}

After asking on Stack Overflow about why it wasn't safe to just pass in a raw pointer, I got the following answer:

malloc API handle the size itself, rust alloc api let the user handle the size information, this is often allow to avoid duplicate information but rely on user more. (but normal user of rust don't normally need to know this)

I am wondering where I can get more details on the Rust allocation API? (And why does the Rust allocation API not store the size information itself)?

[chrefr]

Look at the Allocator trait (still unstable). You can also look at the GlobalAlloc trait for the #[global_allocator] attribute.

However, the SO answer is not accurate: generally, C relies on the dynamic data to determine allocation size while Rust relies on generics. Box::from_raw(ptr) will delegate to Box::<u8>::from_raw(), but you need to free a slice and not a single item. So you need Box::<[u8]>::from_raw(), and this is why you did all of the above.

[vedantroy]

What I don't understand is, a &[u8] is a struct (The Slice Type - The Rust Programming Language) composed of a pointer to the data and a length.

Presumably, the the pointer is similar to something returned by malloc, in which case you could just free it (without knowing the length).

However, I guess this is not the case with Rust memory allocation APIs.

[Yandros]

Basically, @vedantroy, you should know that most implementations of Rust allocation APIs do already bundle the associated allocation length within their own bookkeeping, and thus can/may disregard the .size parameter of a dealloc call.

That being said, since there may very well be some platform where some implementation may wish to receive the .size data on dealloc explicitly (IIRC, this may be the case on Windows), then Rust has made the choice to make that part of its API, to be maximally compatible with allocator implementations.

Obviously, this leads to the "freeing end" having to meet this API requirement, as per the very definition of an API contract.

This was deemed acceptable, since when requesting to free something, the owner ought to already be able to know, either statically / at compile-time, or at runtime, how big the thing they own is. Indeed, more generally, one ought to remember at all times how big the thing they are manipulating is

Aside

I also mentioned some alternatives in the post you mentioned