Casting wide-pointers

1

Continuing the discussion from Newtype problem: reinterpret_cast a &[Bar] to &[Foo]:

I'm curious if someone can tell me how wide-pointer casts work and if there are any guarantees on what they do?

When casting pointers to slices (edit: I mean: casting slice-pointers to other slice-pointers), apparently the length information (as in number of elements, not bytes) will be copied. (Playground) But is this guaranteed? And how about other wide-pointers?

Perhaps pointer casts are only predictable if the layout is ensured to be the same? And then I guess the layout is not defined:

So when can I cast wide-pointers at all without creating unpredictable results?

2

I'm not sure if it's 100% correct, but think of the &[T] fat pointer as something like this:

struct SliceReference<T> {
  first: *const T,
  len: usize,
}

The expression, slice as *const [A] as *const [B], would then be like transmuting the SliceReference type.

3

I would assume the same, but I did not find any documentation on that behavior.

I'm asking because apparently transmuting from &[Bar] to &[Foo] could be considered to lead to unpredictable results:

But on the other hand it's supposed to be okay to cast a *const [Bar] to a *const [Foo] while I didn't find any place in the docs which tells us how such a cast is performed. But perhaps I just didn't find it and it's out there somewhere?

  1. The Vec transmute case has a practical chance of breaking on some future rustc version. The slice transmute is probably de-facto stable due to people doing this transmute and there being no real reason to have different orders between different slice types, but you still shouldn't rely on this. ↩︎

4

I found something in the Nomicon:

Lengths when casting raw slices

Note that lengths are not adjusted when casting raw slices; *const [u16] as *const [u8] creates a slice that only includes half of the original memory.

2 Likes
5

De-facto, these DST pointer casts work by directly copying the address and the metadata of the pointer. RFC 2580 (tracking issue) aims to add additional APIs to make these behaviors more obvious. The documentation for the unstable Pointee trait has a good summary of what the metadata contains:

Pointers to dynamically-sized types are said to be “wide” or “fat”, they have non-zero-sized metadata:

  • For structs whose last field is a DST, metadata is the metadata for the last field
  • For the str type, metadata is the length in bytes as usize
  • For slice types like [T] , metadata is the length in items as usize
  • For trait objects like dyn SomeTrait , metadata is DynMetadata<Self> (e.g. DynMetadata<dyn SomeTrait> )

So when we cast a *const [Bar] to a *const [Foo], we directly copy the usize representing its length.

3 Likes
6

Hmm, yeah I see, I had a glimpse on that (unstable) API when browsing through std.

Since those are not stable, I guess it's (strictly speaking) not guaranteed yet how a wide-pointer looks like and how it's cast though, right? (Except for the note in the Nomicon, which is a bit vague.)

Maybe it's something that will be clarified when/if that RFC becomes a stabilized feature.

7

You're basically waiting on RFC 2580 for a guaranteed sound way to do this, yeah. I think various crates assume the layout and maybe throw in some compile-type hacks and/or runtime asserts to hedge their bets.

8

Yeah, I did the same until I thought about it further. Maybe it will not be a practical problem ever, though.

But for now (as also said in the original thread), I prefer to rely on std::slice::from_raw_parts, I think. That is part of stable Rust:

fn trns(slice: &[u16]) -> &[u8] {
    let ptr = slice as *const [u16] as *const [u8];
    unsafe { &*ptr }
}

fn safer_trns(slice: &[u16]) -> &[u8] {
    unsafe {
        std::slice::from_raw_parts(
            slice as *const [u16] as *const u8,
            slice.len(),
        )
    }
}

fn useful_trns(slice: &[u16]) -> &[u8] {
    unsafe {
        std::slice::from_raw_parts(
            slice as *const [u16] as *const u8,
            std::mem::size_of_val(slice),
        )
    }
}

fn main() {
    let x = [1, 2, 3, 4, 5];
    let y1 = trns(&x);
    let y2 = safer_trns(&x);
    let y3 = useful_trns(&x);
    println!("{y1:?}");
    println!("{y2:?}");
    println!("{y3:?}");
}

(Playground)

Output:

[1, 0, 2, 0, 3]
[1, 0, 2, 0, 3]
[1, 0, 2, 0, 3, 0, 4, 0, 5, 0]
9

It's worth noting that as casts between fat pointers will not compile if the metadata of the fat pointers are not the same type. E.g. *const dyn Trait as *const [T] will cause an error.

This means that your example of representing *const [T] as a start/end pointer pair rather than a start/len pair would cause casts between start/end and start/len to not compile.

Technically speaking, IIRC nowhere actually says exactly what as casting fat pointers does. However, due to the backwards compatibility guarantees (and conversations with various UCG and other team members), I'm comfortable saying that as casts between slice types are equivalent to casting the pointer type and keeping the same element length.

(Disclaimer: this post is my own opinion and does not represent a formal position of any rust team, whether I participate on said team or not.)

See also Cast to a DST pointer? · Issue #288 · rust-lang/unsafe-code-guidelines · GitHub and everything link-reachable from there.

2 Likes
10

I would then conclude:

  • Transmuting references to slices can work practically but might not work in future and is considered UB and should be avoided.
  • Casting fat-pointers is not well-defined in the reference docs yet but practically safe to use (with rules regarding pointers to slices according to the Nomicon: element length stays the same). Edit: At least if the element size is the same, see my comments below.
  • RFC 2580 might stabilize this in future.
  • Currently, the formal correct way to transmute slices is to use std::slice::from_raw_parts.

However, I must say that casting fat-pointers to slices where the elements have different lengths likely will never make sense in practice, or is there any case where you need this? (Compare [1, 0, 2, 0, 3] in the example above.)

Thus, I think it might be reasonable to disallow wide-pointer casts between slices when the element size doesn't match.

Following that argumentation, perhaps the Nomicon should be updated to indicate more explicitly that this behavior should not be relied upon. Edit: I mean not relied upon when element sizes don't match.

(However, I might miss some use-cases where the current behavior is used/needed.)

1 Like
11

Sometimes.

12

You cite this:

Another use case is to convert &[[T; N]; M] to &[T; N * M].

But in that case, keeping the "element count" of a wide-pointer (as Nomicon specifies) would be an error too. In that cited example, the element count must be changed from M to N*M.

1 Like
13

Oh, I see. Yes, you're right.

It's specified by RFC, but could be phased through deprecation given slice_from_raw_parts.

14

It would likely be better to stabilize ptr_metadata in some form before deprecating such casts entirely. I've been looking at non-generic DSTs ending with a slice for another question, and currently, a fat-pointer cast from a *mut [T] or *mut TempGenericStruct<[T]> to a *mut DSTStruct is the only way to create one. (Even then, DSTStruct must be repr(C) or repr(transparent) for this to be sound.) Consequently, it seems clear that many unsafe crates depend on this in the wild, but I couldn't say for certain without a Crater run or similar.

3 Likes
15

Anything transitively using rowan, triomphe, elysees, or slice-dst are using casts between *mut [T] and *mut Container<[T]>. But this is the case where the slice tail remains the same exact type, so shouldn't be deprecated.

What could potentially be deprecated is a *mut [T] tail to a *mut [U] tail where T and U are not known to be the same size. (If they're the same size, there's no harm in the cast, as it does the only thing which could work: preserve the exact memory span being pointed to.) Where they're known to be different sizes, I could get behind a clippy warning, but, uh, that currently ICEs?

The two good usages for when they're not known to be the same size would be type-erasing to *mut [()] and casting between *mut [T] and *mut [U] where you don't know the size (because it's generic), but they're going to be the same size when instantiated.

16

I see no ICE.

17

It ICEs when you run Clippy on it from the tools.

1 Like
18

Ah, thanks, and following up with an issue is already done I see.

19

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.