Current META converting Vec<U> -> Vec<T> where

1

... T and U are not zero sized and have same size and alignment.

Hello all,

I know this comes up every once in a while, but I was wondering if there is a safe way to transform / transmute a Vec to a Vec where T and U have the same size and alignment?

I learned there used to be a method map_in_place for this kind of usecase.

My interest was sparked while trying to convert a Vec<f32> into a Vec<Option<f32>>. I tried this code on godbolt: (see EDIT)

Here is sample code on godbold for f32 -> i32 conversion

pub fn convert_vec(v : Vec<f32>) -> Vec<i32> {
    v.into_iter().map(convert_num).collect()
}

pub fn convert_num(f : f32) -> i32 {
    f as i32
}

Try it on godbold
Does this conversion happen in place? (Spoiler: I am awful at reading assembly)

EDIT: the original question used f32 and Option which are not the same size. This explains @Hyeonu 's first reply.

1 Like
2

Option<f32> is twice as large as f32. Try again with same sized types.

4 Likes
3

good lord, I thought I had checked that. You're right.

4

Generated assembly doesn't call heap allocators(in fact it doesn't contains any "call" instruction) so it's in-place conversion. And you can see the conversion is vectorized.

3 Likes
5

To answer the original question, the safest way to transform two vectors is indeed

v.into_iter().map(f).collect()

but it is not guaranteed to be a no-op even when possible. If you really want to reuse the same allocation, you should use pointer casts:

/// Transmutes `Vec<T>` into `Vec<S>` in-place, without reallocation. The resulting
/// vector has the same length and capacity.
///
/// SAFETY: the types `T` and `S` must be transmute-compatible (same layout, and every
/// representation of `T` must be a valid representation of some value in `S`).
pub unsafe fn transform<T, S>(mut v: Vec<T>) -> Vec<S> {
    let len = v.len();
    let capacity = v.capacity();
    let ptr = v.as_mut_ptr().cast::<S>();
    // We must forget the original vector, otherwise it would deallocate the buffer on drop.
    mem::forget(v);
    // This is safe, because we are reusing a valid allocation of the same byte size.
    // The first `len` elements of `S` in this allocation must be initialized, which is
    // true since `size_of::<T>() == size_of::<S>()`, the first `len` elements of `T` are
    // initialized due the safety invariants of `Vec<T>`, and `T` and `S` being
    // transmute-compatible by the safety assumptions of this function.
    Vec::from_raw_parts(ptr, len, capacity)
}

Note that you can never directly mem::transmute between Vec<T> and Vec<S>, regardless of what T and S are. This is because there are no layout guarantees on #[repr(Rust)] generic types. Even if T and S are transmute-compatible, the fields of the vectors may be layed out in a different order.

Also remember that it's not enough for T and S to have the same layout (size and alignment). u8 and bool have the same layout, but transmuting u8 to bool is invalid (and thus you also can't transmute Vec<u8> to Vec<bool>). This is because bool has only two vaild values, 0 (false) and 1 (true), and having a bool with any other memory representation is UB.

2 Likes
6

It is not guaranteed but pretty reliable. The stdlib uses specialization to achieve it and it's very unlikely to be removed.

1 Like
7

The specialization is used to remove the re-allocation and make the map happen in-place. That part is quite reliable. However, turning it entirely into a no-op will then rely on LLVM optimizations to get rid of the whole loop that executes the mapped operation in-place, and that optimization – since it’s an automatic optimization by LLVM – is generally somewhat less reliable of a thing to happen. Of course LLVM will not deliberately make things worse in the future, but there’s always the possibility for unintentional pessimizations of certain optimizations in corner cases.

2 Likes
8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.