Conversion between Vec<[T; N]> and Vec<T>?

Is there a really cheap way to turn a Vec<[f64; 3]> into a Vec<f64>?

Since both [f64; 3] and three f64s have the same layout it should be somehow possible. Just transmuting and setting the length would not be enough because of the capacity.

There is Vec::from_raw_parts but using it to do this conversion doesn't seem to be permissible - the requirements are listed in the docs. If you ask the same question about &[T] it is easier, since we don't have the requirements from the allocator.

Strict reading says that the Layout has to be identical, and when the element size differs, it can't be.

1 Like

The contract of Vec::from_raw_parts makes it impossible to convert between a vector of two types if their sizes are not the same, which they are not in your case. Are you sure that a conversion between slices is not enough?

1 Like

from_raw_parts seems good and miri doesn't tell me off yet either. Is this correct (IT IS NOT DO NOT DO THIS):

fn main() {
    let vec: Vec<[f64; 3]> = vec![[1.2; 3]; 5];

    let capacity = vec.capacity() * 3;
    let length = vec.len() * 3;
    let ptr = vec.leak() as *mut [[f64; 3]] as *mut f64;

    let vec: Vec<f64> = unsafe { Vec::from_raw_parts(ptr, length, capacity) };
    println!("{:?}", vec);
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=123a4d173b5e0fb4a9c2daf6f2c3934f

No, it violates the following requirement of from_raw_parts:

T needs to have the same size and alignment as what ptr was allocated with.

The size is different. That miri accepts it does not guarantee that it is correct.

3 Likes

I see in the documentation:

Violating these may cause problems like corrupting the allocator's internal data structures. For example it is not safe to build a Vec<u8> from a pointer to a C char array with length size_t . It's also not safe to build one from a Vec<u16> and its length, because the allocator cares about the alignment, and these two types have different alignments. The buffer was allocated with alignment 2 (for u16 ), but after turning it into a Vec<u8> it'll be deallocated with alignment 1.

So is there no easy way without making a new allocation?

There's a plan to eventually allow some conversions:

but currently nearly everything about converting Vec types is very risky and unsafe.

You should be able to cast slice types though.

Not only is there no easy way. There is no hard way either. Are you sure that converting slices is not enough?

3 Likes

The reason I need it flat is because I am creating an owned ndarray from it. I am not sure if a non-owned ndarray will be enough for me or not but if it is enough that won't mean too many changes to my code.

I can make the whole thing make a flat vector so there will be no need to convert but that will mean rewriting some functions. It isn't too much work but if it must be done than it must be done :slight_smile:

One option you might consider is to write a wrapper around Vec<f64> that emulates an Vec<[f64; 3]>

Thank you everyone for the help and suggestions! Great community!

You bring up a good point, this would be an interesting conversion for ndarray. And it could potentially be allowed on the crate's own side if a different allocator is used.

For my particular use case an ArrayView was enough which can be created from a slice. So indeed a slice conversion was enough.

let vec: Vec<[f64; 3]> = vec![[0.0, 1.0, 2.0], [3.0, 4.0, 5.0]];

// create Array
let slice = unsafe {
    std::slice::from_raw_parts(vec.as_ptr() as *const f64, vec.len() * 3)
};
let array = ArrayView1::from(slice).into_shape((vec.len(), 3)).unwrap();

There is probably a shorter way to do this though...

You need to use vec.as_ptr() here. Going through a reference is UB because a raw pointer created from a reference must never access variables the reference could not have accessed.

3 Likes

I edited it. Thanks!

I'm wondering if the invariants of Vec::from_raw_parts could be changed to allow for cases like this. After all isn't the actual requirement that the layout of the whole allocation is still the same?

1 Like

It's possible that they could be changed, yes.

Can't this be done by passing through Box<[T]>, whose safety is defined in weaker terms about the Layout of the A in Box<A>?

1 Like

That requires a potential reallocation to shrink the vec