let me try to give you a short summary.
also note, some (legacy) C code would use an array of size 1 to emulate FAM, in order to compile on very old C compilers, which lack proper support for FAM. this is fine for C, but bindgen will parse these types as regular structs. fambox can still be used in this case though, just be aware when dealing with very old ffi libraries.
long text and code wall alert
I think some code might help understand the concepts better, so here it is:
suppose we have a C struct like this:
struct Foo {
/* other fields */
int name_len; /* NOT including the NUL terminator */
char name[]; /* must have the NUL terminator */
};
the bindgen generated header type looks like this:
#[repr(C)]
pub struct Foo {
pub name_len: ::std::os::raw::c_int,
pub name: __IncompleteArrayField<::std::os::raw::c_char>,
}
if you want to construct a Foo in rust, you cannot just use Foo alone (unless you use raw pointer for EVERYTHING), you need some wrapper type to "bundle" the header and payload.
let's start simple, with a fixed size array, this is the "consevatively-sized" type I mentioned, let's call it FooBuf:
const MAX_NAME_LEN: usize = 30;
#[repr(C)]
pub struct FooBuf {
header: Foo,
payload: [c_char; MAX_NAME_LEN + 1],
}
impl FooBuf {
/// will panic if name exceeded MAX_NAME_LEN
pub fn new(/* other header fields, */ name: &[c_char]) -> Self {
assert!(name.len() <= MAX_NAME_LEN);
let mut foo = Self {
header: Foo {
name_len: name.len().try_into().unwrap(),
name: __IncompleteArrayField::new(),
},
payload: [0; MAX_NAME_LEN + 1],
};
foo.payload[..name.len()].copy_from_slice(name);
foo
}
}
here's how it is used:
/// hypothetical ffi APIs
unsafe extern "C" {
/// insert the `foo` into some registry, takes ownership and returns an opaque handle
fn register_foo(foo: *mut Foo) -> u32;
/// remove the `foo` from the registry, give back ownership to the caller
fn unregister_foo(handle: u32) -> *mut Foo;
}
fn main() {
// create an `FooBuf`, which is ffi compatible for the `Foo` object
let my_foo = FooBuf::new(b"/foo/bar/baz");
// we are going to register the object into the registry of the
// ffi library, move it into the heap so we can transfer ownership
let my_foo = Box::new(my_foo);
// `FooBuf` is `Sized`, so this is just a regular "thin" pointer
let handle = unsafe { register_foo(Box::into_raw(my_foo) as *mut Foo) };
// ...
// unregister it, get back the box allocated on rust allocator
let my_foo = unsafe { Box::from_raw(unregister_foo(handle)) };
// do something with `my_foo`, or just let it drop automatically
}
actually, this is enough for a large number of common use cases, where memory usage is not a constraint, so it is an acceptable trade-off of this conservative approach: some wasted memory for safety and simplicity.
in other situations, e.g., it's undesirable to use "maximum" memory for all the data, or it simply doesn't have a defined "maximum" size for the data, a DST must be used (again, unless you want to use raw pointers for everything).
the type definition itself is very similar to the fixed sized array, simple:
#[repr(C)]
pub struct FooDst {
header: Foo,
payload: [c_uchar],
}
however, since it's a DST, the implementation needs careful use of unsafe code. the most tricky part is to get the correct pointer metadata out of thin air (or, out of of "thin pointer"), here is why it woks: the metadata for a slice type is just the len() of the slice, this is a special case for the general ptr::metadata API, and is available in stable, see std::ptr::slice_from_raw_parts() (and the counter part std::slice::from_raw_parts()) for details.
impl FooDst {
/// use this to convert a "thin" header pointer into a "fat" pointer
///
/// this is most tricky part, it may seems strange at first, but it is
/// actually sound, and it is the only way to get the correct pointer
/// metadata on stable.
///
///
/// SAFETY:
///
/// the `name_len` field in the header must be correctly initialized
///
/// the pointer must have the correct provenance:
///
/// - when called with rust pointers, it cannot be derived from a reference
/// of the header type `Foo`.
///
/// - when called with pointers received from ffi APIs, since ffi doesn't
/// have the concept of provenance, it can only be assumed (unsafely)
unsafe fn from_header(header: *mut Foo) -> *mut Self {
let name_len = (*header).name_len as usize;
// `+ 1` to include the NUL terminator
let this = std::ptr::slice_from_raw_parts_mut(header, name_len + 1);
this as *mut Self
}
}
with this in hand, we can implement a constructor for boxed value easily:
impl FooDst {
/// name_len does NOT include NUL terminator
pub fn layout(name_len: usize) -> (Layout, usize) {
let header = Layout::new::<Foo>();
let payload = Layout::array::<c_uchar>(name_len + 1).unwrap();
let (layout, payload_offset) = header.extend(payload).unwrap();
(layout.pad_to_align(), payload_offset)
}
pub fn new_boxed(/* header fields omitted, */ name: &[u8]) -> Box<Self> {
let (layout, payload_offset) = Self::layout(name.len());
unsafe {
let this = alloc_zeroed(layout);
// init header:
std::ptr::write(this as *mut Foo, Foo::new(name.len() as c_int));
// init payload:
std::ptr::copy_nonoverlapping(
name.as_ptr(),
this.add(payload_offset) as *mut c_uchar,
name.len(),
);
// wrap it in a box
Box::from_raw(Self::from_header(this as *mut Foo))
}
}
}
depending on the situation, you may want to MaybeUninit to init the payload data inplace instead of copying from a slice for efficiency, but I'll not go into the details.
the usage of creating an object in rust is similar to the fixed sized array case, so I'll not repeat, here's how you use it with a pointer received from ffi:
unsafe extern "C" {
fn get_foo() -> *mut Foo;
}
fn main() {
// get a thin pointer from ffi function
let foo = unsafe { get_foo() };
// turn it into a fat pointer using the information of the header
let foo = unsafe { FooDst::from_header(foo) };
// example use of an non-owning reference of the DST:
let foo_dst_ref = unsafe { &*foo };
let header: &Foo = &foo_dst_ref.header;
let name_buf: &[c_uchar] = &foo_dst_ref.payload;
// note, the `payload` slice includes the NUL terminator
assert_eq!(header.name_len + 1, name_buf.len() as c_int);
}
as you can see, to get the unsafe bits right is tricky, luckily, these are all handled by fambox for you, you just need to implement the FamHeader trait for the header type Foo, no need to implement your own FooDst:
unsafe impl FamHeader for Foo {
type Element = c_uchar;
fn fam_len(&self) -> usize {
// count the NUL terminator
self.name_len as usize + 1
}
}
compare the FooDst::new_boxed() function above, and the new_fam_box() funciton below:
/// CStr ensures a NUL terminator
fn new_fam_box(name: &CStr) -> FamBoxOwned<Foo> {
let header = Foo::new(name.to_bytes().len() as i32);
FamBox::from_fn(header, |i| name.to_bytes_with_nul()[i])
}
and usage is very similar:
fn gg() {
// create a boxed DST value
let my_foo = new_fam_box(c"/foo/bar/baz");
// transfer ownership to ffi library
let handle = unsafe { register_foo(my_foo.leak().as_mut()) };
// use borrowed ffi pointer
let mut foo = unsafe { FamBoxMut::from_raw(NonNull::new(get_foo()).unwrap()) };
let (header, payload) = foo.as_parts_mut();
assert_eq!(header.name_len + 1, payload.len() as i32);
// get back ownership from ffi library
let my_foo = unsafe { FamBoxOwned::from_raw(NonNull::new(unregister_foo(handle)).unwrap()) };
// use it or drop it
}