Transmute() with container types

help
1

Hi!

I have a #[repr(transparent)] wrapper over f64 (let’s call it Wrapper). Is transmute::<Vec<Option<Wrapper>>, Vec<Option<f64>>>(_) sound? On the one hand, Option<_> and Vec<_> are both #[repr(Rust)], so their layout is not guaranteed. On the other hand, Wrapper and f64 have the same layout, so there’s no obvious reason for container’s layout to differ.

2

Transmuting vectors is not sound.

3 Likes
3

That’s sad. Is there any sound way to reuse the allocation?

4

You can go through into_raw_parts and from_raw_parts. There's an example in the documentation for transmute.

5 Likes
5

.into_raw_parts() is unstable, but I guess I can use something like this?

fn transmute_vec(mut v: Vec<Option<Wrapper>>) -> Vec<Option<f64>> {
    let ptr = v.as_mut_ptr().cast::<Option<f64>>();
    let length = v.len();
    let capacity = v.capacity();
    // Avoiding double drop
    std::mem::forget(v);
    // SAFETY:
    // 1. `ptr` was previously allocated via `Vec<_>`
    // 2. `Option<Wrapper>` has same size and alignment as `Option<f64>`
    //    (Is it guaranteed? `Option<_>` is technically `#[repr(Rust)]`
    //     I guess I could just use `static_assertions` to prove it)
    // 3. `length` and `capacity` are copied from existing `Vec<_>`
    unsafe {
        Vec::from_raw_parts(ptr, length, capacity)
    }
}
6

This still relies on Option's layout being the same for f64 and Wrapper though, doesn't it?

2 Likes
7

Right. Not sure what you can do there.

8

Ok, thanks.
I guess I’ll make a static assertion about size and alignment of Option<_> and will hope that layout will stay the same. Maybe I’ll add a test that transmute::<Option<Wrapper>, Option<f64>>() works as expected just to be sure.

9

Unfortunately, hoping for things to not change isn't a valid way to handle UB :disappointed:

Besides wanting to avoid an allocation, are there any other requirements which might force you to reinterpret the Option<Wrapper> as a Option<f64>?

One alternative that comes to mind is defining your own Option<T> enum which is #[repr(C)]. You could also use a &[MaybeUninit<T>] which has an accompanying bitset that tells you which elements are initialized.

This test works as expected, but that doesn't mean much because anything can happen when your code contains UB.

#[derive(Debug)]
#[repr(transparent)]
struct Wrapper(f64);

fn main() {
    let options: &[Option<f64>] = &[Some(1.0), None, Some(2.0)];
    let wrappers = unsafe {
        std::slice::from_raw_parts(options.as_ptr() as *const Wrapper, options.len())
    };
    
    println!("{:?}", wrappers);
}

(playground)

Interestingly, Miri doesn't have any checks for these sorts of layout shenanigans.

1 Like
10

Avoiding allocation is important here. Custom #[repr(C)] Option<T> would pollute the codebase and create an enormous maintenance burden. While I don’t like relying on Option’s layout, I also don’t like allocations on a hot path or custom Options all over the codebase and the risk of Option<Wrapper> layout being different from Option<f64> layout seems quite small.

11

You know, because of the specialization on Vec, I'm pretty sure vec.into_iter().map(|opt| opt.map(to_or_from_wrapper)).collect() will be (100% safe) and probably optimize to a no-op.

4 Likes
12

I hate being that "well ackchyually" guy, but to quote a previous Quote of the Week...

I agree with you that in practice this is almost certainly going to work without any hiccups, but according to Rust's memory model it's UB to transmute between #[repr(Rust)] types.

The only possible exception might be when you can make guarantees around Option<Wrapper>'s layout. As far as I'm aware, that could only happen if Wrapper and f64 contained a "niche" because then we can rely on the "null value optimisation". However, I don't think that'll help in your case because switching to &[Option<&f64>] and #[repr(transparent)] struct Wrapper(&f64) isn't suitable for what you are doing.

1 Like
13

I'm pretty sure that it will not optimize to a no-op. While allocation seems to be actually optimized out on a modern enough compiler, iteration is not.

14

hrm, yeah I see; it seems like it's dedicated to zeroing the padding bits or something.

15

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.