DST with packed struct

1

It seems that DSTs (dynamically sized types) are not yet compatible with #[repr(packed(n))], as this won't compile:

#[repr(C, packed(4))]
pub struct PackedFlexArray<ArrayT: ?Sized = [u64]> {
    count: u32,
    data: ArrayT,
}

(playground)

The packing is needed here, because otherwise four bytes of padding will be added to put the data field onto an 8-byte alignment boundary. Removing the packing is not acceptable, because memory layout must be exactly the same as the C representation.

(My ultimate goal is to provide bindings for MIDIPacketList from Apple Core Audio which allow for the creation of MIDIPacketList objects in Rust code which can be passed via FFI to the Core Audio frameworks.)

What are my options?

2

You could do this:

#[repr(C)]
pub struct PackedFlexArray<ArrayT: ?Sized = [LowAlignU64]> {
    _count: u32,
    _data: ArrayT,
}

struct LowAlignU64([u8; 8]);

fn main() {
    let size = std::mem::size_of::<PackedFlexArray<[LowAlignU64; 1]>>();
    assert_eq!(size, 12);
}

Other than that, you could avoid using DSTs for this and just implement what you want with raw pointer offsets.

2 Likes
3

Great, thank you Alice!

While grappling with your suggestions, I happened upon what may be an alternative approach which deals with the compiler issue at the heart of this (for my purposes):

 pub struct PackedFlexArray<ArrayT: ?Sized = [u64]> {
     _count: u32,
-    _data: ArrayT,
+    _data: std::mem::ManuallyDrop<ArrayT>,
 }

This approach occurred to me because of this part of the compiler error message which we get when we run the original playground example:

= note: the last field of a packed struct may only have a dynamically sized type if it does not need drop to be run

I believe this will work for all cases where the DST array type doesn't need a dedicated drop. It should work for the u64 array in the simplified example code, and should work for my actual problem as well because the MIDIPacket struct in Core Audio contains only values (no pointers). The drop for the outer container (PackedFlexArray in the sample code) should free the single allocation which holds everything including the flexible array.

Is this a good enough suggestion that I should mention it in the Github issue?

4

Even though I hope that I have a possible alternative solution I wanted to respond to alice's original suggestions:

With the LowAlignU64 type, we also need to convert between the raw byte slices and u64 slices, allowing the user to perform reads and writes via a u64 API. I had a look at the source code for the byte_slice_cast crate and fiddled around long enough to satisfy myself that I'd get it eventually.

If I understand this suggestion correctly, it limits us to a Box type because without DSTs, there's no way for the compiler to know the static allocation size and allocate on the stack.

I'd then have to do some raw pointer math after extracting from the box type. This feels very much in the spirit of C, but presumably I'd eventually figure out how to make it work in unsafe Rust.

5

You can use u64::from_ne_bytes and u64::to_ne_bytes to convert between u64 and byte array.

As for the point about Box, whether that suggestion makes sense depends on your use-case. For example, if the array is created by C and you're given a pointer, then it works no matter where C stores it. If you need to create them from Rust, then it's a different story.

1 Like