How to construct an unsized struct / How to construct a struct with an slice?

help
1

Hi! How to construct a struct with a slice ( not reference on a slice )?
If it's possible to declare such a struct then should be a way to construct it, right?

Example:

fn main() {
    let a = Box::new(UnsizedStruct {
        x: 13,
        arr: [1, 2, 3][..],
    });
}

//

#![repr(C)]
pub struct UnsizedStruct {
    x: i32,
    arr: [i32],
}
1 Like
2

You've run into an incomplete feature.

It's possible to make the struct generic over its contents and coerce from a version containing an array (the above page demonstrates this), but the only way to construct an instance with an actually arbitrary dynamic length is unsafe code that writes the struct's parts manually.

8 Likes
3

I see. Thanks for the response. With help of which unsafe function I can allocate such kind of structure?

4

You know, I don't think there's any sound way to construct an UnsizedStruct instance at all. To unsafely allocate space for our Box, we must know the size and alignment of an UnsizedStruct of length 0. If it were repr(C), we'd know that it has size 4 and alignment 4. But since it is repr(Rust), we must use some other API to determine the Layout. All such APIs are either unstable, or they require a valid reference to an instance! With the layout_for_ptr feature, we could do some trickery to get the Layout:

#![feature(layout_for_ptr)]

use std::{alloc::Layout, ptr};

fn get_layout_for_len(len: usize) -> Layout {
    let ptr: *const [()] = ptr::slice_from_raw_parts(ptr::null(), len);
    unsafe { Layout::for_value_raw(ptr as *const UnsizedStruct) }
}

Or we could do it more properly with both layout_for_ptr and ptr_metadata:

#![feature(layout_for_ptr)]
#![feature(ptr_metadata)]

use std::{alloc::Layout, ptr};

fn get_layout_for_len(len: usize) -> Layout {
    let ptr: *const UnsizedStruct = ptr::from_raw_parts(ptr::null(), len);
    unsafe { Layout::for_value_raw(ptr) }
}

But with neither, there's simply no way to determine the layout. (Also, now that I think about it, both of these functions are unsound as well: Layout::for_value_raw() is unsound if static_size + len * 4 > isize::MAX as usize, and we have no way to properly assert! against this since we don't know static_size to begin with.)

@Wandalen In your use case, can you mark UnsizedStruct as #![repr(C)]? If so, initializing one unsafely simply becomes "very tedious" instead of "impossible".

1 Like
5

Hi! thank you for the elaborated answer. Yes, I can mark the structure with #![repr(C)]. Do I still need nightly features then?

6

That can be done in stable. First, a Box<UnsizedStruct> can be trivially created from a fixed-size array using an unsizing coercion with a helper type (Rust Playground):

#[repr(C)]
#[derive(Debug)]
pub struct UnsizedStruct {
    x: i32,
    arr: [i32],
}

impl UnsizedStruct {
    pub fn new_from_array<const N: usize>(x: i32, arr: [i32; N]) -> Box<UnsizedStruct> {
        #[repr(C)]
        struct Generic<T: ?Sized> {
            x: i32,
            arr: T,
        }
        let b: Box<Generic<[i32]>> = Box::new(Generic { x, arr });
        let ptr = Box::into_raw(b) as *mut UnsizedStruct;
        // SAFETY: `Generic<[i32]>` has the same layout as `UnsizedStruct`.
        unsafe { Box::from_raw(ptr) }
    }
}

Creating one from a slice is only a bit trickier (Rust Playground):

#[repr(C)]
#[derive(Debug)]
pub struct UnsizedStruct {
    x: i32,
    arr: [i32],
}

impl UnsizedStruct {
    pub fn new_from_slice(x: i32, slice: &[i32]) -> Box<UnsizedStruct> {
        let len = slice.len();
        assert!(len * 4 <= isize::MAX as usize - 4);
        let mut v = Vec::with_capacity(len + 1);
        v.push(x);
        v.extend(slice);
        let data = Box::leak(v.into_boxed_slice());
        let ptr = &mut data[..len] as *mut [i32] as *mut UnsizedStruct;
        // SAFETY: The address and length metadata were encoded in the `*mut [i32]` pointer.
        unsafe { Box::from_raw(ptr) }
    }
}

The real difficulty would come if UnsizedStruct had a type other than i32 before its arr slice. Here, we can just create an extended [i32] slice that includes x at its start.

1 Like
7

Your thoughts are sound, but think about it, if we know UnsizedStruct is doomed to be allocated on heap, then why not let [i32] be allocated on heap directly, just like String does?

pub struct UnsizedStruct{
    x: i32,
    arr: Vec<i32>,
}
1 Like
8

Except it doesn't compile.

9

My fault, I'll delete it.

10

If you're looking for a unsafe-less way to do this, you can use the slice-dst crate

3 Likes
11

I found a way to initialize UnsizedStruct in a way that theoretically makes sense, but UB is detected by Miri and I don't know how to eliminate it. Tell me please if anybody knows why, thx. Rust Playground

1 Like
12

You're using the wrong length.

let ptr = slice_from_raw_parts_mut(ptr, N);
1 Like
13

I'm sorry but I think you may misunderstand.

let ptr = slice_from_raw_parts_mut(ptr, size_of::<SizedStruct<N>>());

Here I convert *mut u8 to *mut [u8] with a length which is the same as the length of SizedStruct<N> on memory. This step is purely to convert *mut SizedStruct<N> to a fat pointer, no matter what type it exactly is. Then we could convert the fat pointer to UnsizedStruct.

I'm not trying to create a pointer of that array.

1 Like
14

When you cast from *mut [u8] to *mut UnsizedStruct, the length information is passed on unchanged. The length you want in the final *mut UnsizedStruct is the number N, so that's the number you put as the length.

2 Likes
15

Indeed, I cast from *mut [u8] to *mut UnsizedStruct, the length information is unchanged. But what I want is Box<UnsizedStruct>, not an array of 3 bytes.
ptr has type *mut u8, so slice_from_raw_parts_mut(ptr, N) will return a pointer of an array expected to be with N bytes. It doesn't make sense.

1 Like
16

Raw pointers aren't required to point at something that makes sense. What matters is the *mut UnsizedStruct we get in the end. The rest is irrelevant.

2 Likes
17

But mentioned that UnsizedStruct is unsized. It needs length information, which was given by slice_from_raw_parts_mut(ptr, size_of::<SizedStruct<N>>()). If you replace it with slice_from_raw_parts_mut(ptr, N), Box::from_raw(ptr) will consider the target on heap has only 3 bytes. It will cause memory leaking.
I suggest you re-read my code, you probably miss something.

1 Like
18

No, the length information that the Box<UnsizedStruct> needs is the number of elements in the array, not the number of bytes that the struct takes up. You are getting UB from miri because the Box is way larger than it actually is.

2 Likes
19

You can see in the playground that the function works perfectly fine with slice_from_raw_parts_mut(ptr, N). The critical part here is the conversion from *mut [u8] to *mut UnsizedStruct: both fat pointer types have usize metadata holding the length of the slice in elements. The pointer type does not store the overall size of the struct! The only way to get the overall size of an &UnsizedStruct is to use std::mem::size_of_val().

2 Likes
20

Ah, oh, I see. The error indeed disappeared. Thank you and @alice for correcting me.
So... Can I say the problem is now solved?

The last solution is here: Rust Playground

1 Like