Pointer FFI Help

help
#1

Hi everyone,

I'm currently working on a PNG decoder that has a C interface. My issue is that, once the decoding has finished, I leak all the PNG data which is stored as a Vec<u8> and cast it into a *mut c_void in order to hand it off to the other language. I also want to provide a function that frees the data I previously leaked, however, here is where I am stuck.

What I want to ask from the use of the interface is to provide only the *mut c_void I returned previously and then I want to de-allocate that memory. How would I go about freeing the leaked memory from Rust?

One way that I have thought of is to reconstruct the Vec<u8> and then let it Drop, however, this requires knowing the capacity and length of the Vec and this would require the ffi code to retain the dimensions of the image until the data must be freed. This seems very un-intuitive on the call-site.

Any suggestions on how I can go about doing this would be great!

#2

Check out the minivec crate. It stores the capacity/length header at the same allocation (just before data), and its as_mut_ptr/from_raw_part can be used to roundtrip the minivec through the ffi as a single pointer.

2 Likes
#3

Thanks for the help! MiniVec looks like the thing I want. I'm sad that I can't do this in the base language since the project was dependency-free up till now. :frowning:

#4

@bananabread bundling metadata within the pointee in Rust, more than in C, is very UB-prone, due to assumptions the compiler may make. For instance, you should make sure to be yielding the pointer obtained from .as_mut_ptr() to the FFI and not the one from .leak(), since the latter, in the semantics of the Rust abstract machines, is only allowed to access the memory for the [T]s themselves, not for the prefix header.

A single dependency in exchange of code written by people whoch have through about these details to provide a safe API seems like a good deal to me :wink:

That being said, if you really wanted to go dependency free, then you can try to have and use a Vec<u8, ::std::alloc::System>: such a Vec is guaranteed to have used allocations from the family of malloc on Unix systems, meaning the FFI side can simply free() it :slightly_smiling_face:

  • Note that playing with the allocator the Vec uses currently requires nigthly. Otherwise Vec shall use the #[global_allocator]-marked allocator. The good news is you can mark System as such:

    #[global_allocator]
static __: ::std::alloc::System = ::std::alloc::System;
1 Like
#5

Thanks for the detailed reply! I felt like I was doing something wrong with leak(). So what you're saying is that I should use as_mut_ptr() and then forget the Vec?

Also, I might go with the #[global_allocator] approach since there's one less function to maintain and for calling code to remember :slight_smile:

Edit: I can't mark both answers as solutions :frowning:

#6

Don't worry :wink: keep the answer from @krdln as the official solution, since it's the cleanest one.

Yes. That being said, forget can technically also cause some of this subtle sneaky UB that revolves around some pointers being unique when used.

The correct idiomatic pattern to get an into_raw_ptr semantics out of .as_mut_ptr(), which indeed applies for Vec, is to:

fn vec_into_raw<T> (v: Vec<T>)
  -> *mut T
{
     // Make it so "forget is automagically implicitly called", we could say
    let mut v_no_drop = ::core::mem::ManuallyDrop::new(v);
    v_no_drop.as_mut_ptr()
} // `v_no_drop` goes out of scope, but has no drop, so nothing
  // happens: the memory remains allocated 🙂

See, for instance, the implementation of Vec::into_raw_parts()

1 Like
#7

Thanks for the help @Yandros. I really appreciate it :smiley:

1 Like