Using a rust allocator in C

1

I have an ffi crate for a C library that can take:

typedef struct
{
    void* pUserData;
    void* (* onMalloc)(size_t sz, void* pUserData);
    void* (* onRealloc)(void* p, size_t sz, void* pUserData);
    void  (* onFree)(void* p, void* pUserData);
} ma_allocation_callbacks;

There may be some value in using the same allocator in Rust and C and I wanted to implement this. I had found this, which gave me a good starting point - cassender

For now, I am mostly interested in the unstable feature for global allocator. allocator_api2 will require a bit more elbow grease on my part.
My short implementation so far is here: alloc_api.rs
I feel good about it, but it's the first time I do any allocator work, so I'm skeptical about this confidence.

The main idea is to add a header at the start of the allocation which contains its size.

I noticed the cassender example uses an alignment of 1 which is.. unusual. I had a look over a few bindgen outputs and couldn't find any alignments larger than 8. A constant alignment of 16 seems to be a fair bet. Although maybe a bit wasteful, I don't know if there's a better way to determine it?

I am also not sure I need to handle cases for 0 size.They should be fine in my case, they will produce unique non-null pointers, and an audio library probably doesn't need to make such allocations anyway.

Also, from what I understand, when a user adds a #[global_allocator] static, it will simply replace the standard alloc() and dealloc().

Did I miss anything?

2

The code is sound, but you must use the allocation methods from Rust as well if you want to mix Rust and C because they allocate differently.

The alignment could be a problem; In C, malloc() is only guaranteed to return an object aligned to the largest primitive alignment, which on 64-bit platforms is usually 16. So if the library you're using requires the same as the C standard, you're fine from the C side. However from the Rust side, Rust requires the requested alignment, so you need to be careful to not allocate things with alignment larger than 16 using those methods.

Most allocators though track the allocation size on their own, so you'll end up wasting memory here. You cannot rely on that in Rust by default, but if you'll replace the global allocator with e.g. jemlloac and also use it in C, you can avoid tracking the size.

Also, storing metadata for an allocation is problematic wrt. provenance and aliasing, however this is a problem with pretty much every allocator which is why currently the status is "it's formally UB, but you can do that, prefer to not compile the allocator together with the code. And we need to figure a better story".

3

Thanks.

The library does need specific alignment in some places (like simd), but it has it's own wrappers in malloc and free for them. It doesn't rely on the allocator.

4

Worth mentioning, my realloc had a few issues:

  1. If old ptr is null, it should behave like malloc.
  2. If new_size is 0, it must call free on the ptr and return null.(edit: this seems implementation specific and spicy as heck).
  3. One typing error I had calling pack_ptr().
    Vallgrind seems happy now.

Full code:


unsafe extern "C" fn ma_realloc_cb(
    ptr: *mut c_void,
    new_size: usize,
    _user_data: *mut c_void,
) -> *mut c_void {
    if ptr.is_null() {
        return ma_malloc_cb(new_size, _user_data)
    }

    if new_size == 0 {
        ma_free_cb(ptr, _user_data);
        return std::ptr::null_mut();
    }

    let base_ptr = (ptr as *mut u8).sub(META_SIZE);
    let old_size = (base_ptr as *mut Metadata).read().size;

    // Layout assumed valid as it had to be created for this allocation to exist
    let old_layout = Layout::from_size_align_unchecked(old_size + META_SIZE, ALIGN);

    let Ok(new_layout) = Layout::from_size_align(new_size + META_SIZE, ALIGN) else {
        return std::ptr::null_mut();
    };
    let new_base_ptr = std::alloc::alloc(new_layout);
    if new_base_ptr.is_null() {
        return std::ptr::null_mut();
    }
    let new_ptr = new_base_ptr.add(META_SIZE);
    core::ptr::copy_nonoverlapping(ptr, new_ptr as *mut c_void, old_size.min(new_size));

    // do not leak the old allocation
    std::alloc::dealloc(base_ptr, old_layout);

    // We already know that new_base_ptr is not null
    pack_ptr(new_base_ptr, new_size) // was passing "new_ptr" previousy
}