Some C projects use a construct called a "flexible array". One example is name_to_handle_at() and open_by_handle_at() API on Linux, which uses a struct file_handle defined as:
struct file_handle
{
unsigned int handle_bytes;
int handle_type;
unsigned char f_handle[];
}
The last member is a char array of arbitrary size (determined by handle_bytes). In Rust, there is a feature that allows making a custom dynamically-sized type, by making a DST the last element in the struct. It can be used, for example, as Arc<RwLock<dyn SomeTrait>>, to allow wrappers over trait objects, or by core::str::CStr, to allow referencing a byte-array checked for 0-termination with &CStr (which is done using pointer casts), in the same way that a byte-array checked for UTF-8 validity can be referenced with &str. Rustonomicon mentions that such type can be created using generic unsizing coercion (such as array-to-slice). However, I wonder, if the size is not known at compile-time, as far as I understand, the only option is allocating memory manually by calling unsafe alloc() with necessary Layout. I wonder, what is the correct way to then create a boxed custom-DST object and can such objects (or, more precisely, the "address" part of the fat pointer) be passed to C API?
Unfortunately there is not yet any really clean way to do this; the pointer metadata API is still unstable. However, you can use slice-dst to handle the specific case of a trailing slice (as opposed to a trailing dyn or another custom DST) today.
Yes; you have to cast the pointer to some placeholder Sized type to make a “thin” pointer, but once do, if the DST’s layout matches the C struct layout, then it’s just as good as any other struct for FFI.
although you need the ptr_metadata unstable feature for DST pointers in general, if the DST is a slice type, however, there's a stable API as a special case:
once you constructed a "fat" raw pointer, you can turn it into a Box like regular sized types, with Box::from_raw().
take a look at the fambox crate. see also my previous answer to a similar question here: