Construct fat pointer to struct?

Given a type definition like:

struct Foo {
    foo: u8,
    bar: [u8],
}

...is it possible to construct a *const Foo from a pointer and a length in a way that's guaranteed stable (unlike mem::transmute, which relies on the unstable ABI of *const Foo itself)?

1 Like

If you generalize it a bit, to

struct Foo<T: ?Sized> {
    foo: u8,
    bar: T
}

You can initialize it with an array, borrow it, then coerce that borrow into a fat pointer.

let x = Foo { foo: 10, bar: [1, 2, 3, 4] };
let x : &Foo<[u8]> = &x;
let x: *const Foo<[u8]> = x;

So you can use Foo<[u8]> everywhere, and the users can coerce to that.

1 Like

Note that this doesn't work unless the user can see all of the fields, so you may have to make all of your fields public.

From @RustyYato's explanation, we can therefore create the following example:

struct Foo<T: ?Sized> {
    foo: u8,
    bar: T
}

fn main() {
    let x = Foo { foo: 10, bar: [1, 2, 3, 4] };
    let x : &Foo<[u8]> = &x;
    unsafe {
        let x = std::mem::transmute::<&Foo<[u8]>, (*const (), usize)>(x);
        println!("{:?}, {}", x.0, x.1);
    }
}

We can run this to realize that x.0 is the pointer to the start of the actual struct, and x.1 is the pointer to the length of the slice. Please do note that this could change at some point or between platforms.

No. Rust does not guarantee ordering between struct fields unless it's #[repr(C)], and tuples are just struct. So (ptr, meta) could be represented in memory as either [ptr, meta] or [meta, ptr], which can or cannot match with desired layout as fat pointer. Note that it's perfectly legal to even randomize struct field ordering based on compiler version or your code hash.

4 Likes

OK cool! Unfortunately, I think that means that my underlying goal is impossible. Namely, I'm trying to write an (obviously unsafe) trait which allows you to coerce a &[u8] into a &T for a custom DST. So it'd look something like this:

trait Bar {
    fn cast(data: *const u8, len: usize) -> *const Self;
}

The problem is that the approach you described requires that your input be sized (e.g., Foo<[u8; 4]> in your example), but I want this to take a length at runtime. Any chance there's a different approach that can work at runtime? I'm assuming not, but asking just in case...

You can still use unsized types, playground, note that I have Foo<[u8]>, which is unsized.

So, did you want to do (*const (), usize) to fat pointer? In that case, I misunderstood you question. No, you cannot do that until we get some form of Custom DSTs. Otherwise you are depending on the unstable layout of the pointers. You can do this for slices, using std::slice::from_raw_parts, but that is the only DST for which you can do this.

Yep, unfortunately that's what I'm after. Thanks for clarifying, though!

This should be well-defined and doesn't assume the fat pointer layout:

#[derive(Debug, Fatten)]
#[repr(C)] // repr for what I put in main, not required in general
struct Struct {
    head: u8,
    tail: [u8],
}

// derive(Fatten) expands to:
impl Struct {
    fn fatten(data: *const u8, len: usize) -> *const Self {
        // Requirements of slice::from_raw_parts.
        assert!(!data.is_null());
        assert!(len <= isize::max_value() as usize);

        let slice = unsafe { core::slice::from_raw_parts(data as *const (), len) };
        slice as *const [()] as *const Self
    }
}

fn main() {
    let array = [2u8, 3u8, 4u8, 5u8, 6u8];
    let ptr = Struct::fatten(array.as_ptr(), 4);
    println!("{:?}", unsafe { &*ptr });
}

If the caller's struct has anything other than a dynamically sized slice as the tail, for example tail: dyn ToString or tail: T where T: ?Sized, it correctly won't compile:

error[E0606]: casting `*const [()]` as `*const Struct` is invalid
  --> src/main.rs:18:9
   |
18 |         slice as *const [()] as *const Self
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: vtable kinds may not match
3 Likes

Wow, that's fantastic. Is there any reference that specifies the behavior of the *const [()] to *const Struct cast? It seems that you're assuming that it will end up converting the number of elements in the [()] to the number of elements in the tail: [u8] field, but it'd be good if that behavior were actually specified somewhere. I haven't managed to find such a reference.

1 Like

@dtolnay is asserting that, given

struct Wrapper<Tail : ?Sized> {
    head: u8,
    tail: Tail,
}

then a Ptr< [Wrapper<u8>] > and Ptr< Wrapper<[u8]> > are structurally identical.

This indeed does not require much knowledge about the layout of a fat pointer to something based on a dynamically sized slice. It suffices to know that it only depends on two elements: a ptr and a len, with ptr pointing to the beginning of the something and len being the number of elements of the slice.


Now, given

how "unstable" is the layout of pointers to slice-based types?

That's true, but that's still an assumption that I'd rather not make without a guarantee from the Rust spec. It's not obvious to me, for example, that a fat pointer to a custom DST would include the number of elements in the trailing slice (as opposed to, for example, the byte length of the entire structure, or something else entirely). I agree that it's a reasonable design, but I don't agree that it's the only reasonable design, and that's what I'm worried about.

That's more convincing, but it's undercut by the preceding paragraph on that page:

We do not make any guarantees about the layout of multi-trait objects &(dyn Trait1 + Trait2) or references to other dynamically sized types, other than that they are at least word-aligned, and have size at least one word.

That makes me think that, while the layout of *const [T] is well-defined and guaranteed, the layout of *const Wrapper<[T]> is not, and so the conversion can't be made without relying on unstable implementation details.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.