let mut temp_vec = ManuallyDrop::new(iter.collect::<Vec<u32>>());
unsafe {
Vec::from_raw_parts(
temp_vec.as_mut_ptr() as *mut [u32; 3],
temp_vec.len() / 3,
temp_vec.capacity() / 3,
)
}
In some situations, if the iterator has been created from Vec::into_iter, using map and collect could reuse the allocation. But in this case, you can't, since you can't map from 3 elements to 1. So what your method allows is reusing the allocation if the iterator came from Vec::into_iter. However, a lot of Vecs will come with a capacity that is a power of 2, which would make your method unusable anyway.
So I'd probably just do this:
pub fn collect_array<const N: usize>(iter: impl IntoIterator<Item = u32>) -> Vec<[u32; N]> {
let iter = iter.into_iter();
let mut array = [0; N];
let mut vec = Vec::with_capacity(iter.size_hint().0);
let mut last = 0;
for (i, item) in iter.enumerate() {
array[i % N] = item;
if i % N == N - 1 {
vec.push(array);
}
last = i;
}
assert_eq!(last % N, N - 1);
vec
}
Or, just convert a slice of the Vec<u32> as needed with slice::as_chunks.
Apologies, here's more context. It's code for processing indices into my own type's format, the indices being from the gltf crate. my own format is for kiss3d, which requires the type Vec<[u32;3]> for it's indices. gltf also returns iterators with types depending on the index amount. Does that clear things up?
The original code is wrong, not just non beneficial. Try yourself.
fn main() {
let v: Vec<i32> = [1, 2, 3]
.into_iter()
.filter(|_| true)
.collect();
println!("{}", v.capacity())
}
Why is the capacity being 4 an issue? Sorry I don't quite understand what you're getting at.
You can use bytemuck for this. There are some subtleties you would have to check. For example the capacity (in bytes) of the original vector must be divisible by the size of the output type
Dropping a vector must deallocate its memory; this deallocation must provide exactly the same byte length as was requested on allocation, at pain of UB. Your code hits this UB: if original vector's capacity was 4 (16B), it will be cut to 1 (12B).
Fair enough. I was trying to make a trimmed version of try_cast_vec, but I ended up removing functionality in the process as well as not making any meaningful decrease in cost. Although with my code it worked well because gltf indices will always be divisible by 3, so I converted it by pointer to Vec<[u32;3]>. If I were to import it to kiss3d, I couldn't use bytemuck for vertices and uvs since it would require that Vec3 and Vec2 derive all the bytemuck derives to cast it. So in that case I have to use my own code to avoid having to maintain a custom-made piece of kiss3d's code.
I also don't quite understand how safe iterator code from rust's std results in UB.
They do: Vec3 in kiss3d::prelude - Rust (also note these are f32, not u32)
Vec::from_raw_parts is not safe. You have to ensure the new capacity has the same byte length as the old capacity.
Here's another idea that would have a better chance of ensuring the Vec has a capacity divisible by 3:
use std::mem::ManuallyDrop;
pub fn collect_array<const N: usize>(iter: impl IntoIterator<Item = u32>) -> Vec<[u32; N]> {
let iter = iter.into_iter();
let size_hint = iter.size_hint().0;
let size = if size_hint == 0 {
N
} else {
size_hint.div_ceil(N) * N
};
let mut vec = Vec::with_capacity(size);
vec.extend(iter);
if vec.capacity() % N == 0 {
let mut vec = ManuallyDrop::new(vec);
unsafe {
Vec::from_raw_parts(
vec.as_mut_ptr() as *mut [u32; N],
vec.len() / N,
vec.capacity() / N,
)
}
} else {
vec.as_chunks().0.to_vec()
}
}
The current std library should never allow the else branch to happen, but std does not guarantee that for the future.
With bytemuck it's a little simpler:
use bytemuck::allocation::try_cast_vec;
pub fn collect_array<const N: usize>(iter: impl IntoIterator<Item = u32>) -> Vec<[u32; N]> {
let iter = iter.into_iter();
let size_hint = iter.size_hint().0;
let size = if size_hint == 0 {
N
} else {
size_hint.div_ceil(N) * N
};
let mut vec = Vec::with_capacity(size);
vec.extend(iter);
try_cast_vec(vec).unwrap_or_else(|(_, vec)| {
vec.as_chunks().0.to_vec()
})
}
I'm writing these with a generic N in case some other LLM person stumbles upon this and needs a different array length.
P.S. You could also collect it into a Box<[u32]> since that has no extra capacity, and then convert it to Vec, but it's basically the same thing allocation-wise.
Almost any mesh has vertices and indices that are both divisble by 3, and uvs that are divisible by 2. What I was doing was bytemuck code without safety checks due to that fact. But since Vec3 implements Pod, i'll use bytemuck to cast the vec to make my code look cleaner and depend on bytemuck maintaing it since the safety checks aren't heavy whatsoever. And as far as what I was talking about with a safe iterator being ub, I was referring to the code Hyeonu sent. And also it's gltf's custom Iter type.
That snippet was just intended to show that you can't rely on the capacity being divisible by N.
Sounds like you are looking for the unstable Iterator::array_chunks().
#![feature(iter_array_chunks)]
let vertices: Vec<[u32; 3]> = iter.array_chunks().collect();
Currently unstable, but it looks like you're trying to https://doc.rust-lang.org/std/vec/struct.Vec.html#method.into_chunks.
In the mean time, maybe just use .as_chunks::<3>() to get a slice of [u32; 3] instead? Unless you're pushing into it, you probably don't need to the vec, just the slice.
I'm trying to make it faster by just creating a new vec from ptr, aka what bytemuck::cast_vec does. This is so it doesn't take forever depending on how big the mesh is to transition the entire vector into a chunk divided vector. If i'm not gaining speed from this please let me know.
Generally, you should start with safe solution, and then measure if a (sound) unsafe version is any faster. After all, it's not like .as_chunks will copy any data.
Forgot to mention that i'm using speedy which can only serialize and deserialize Vecs. as_chunks code suggests that it does far more than cast_vec does and I would have to cast the slice back to a vector somehow anyways. cast_vec isn't marked as unsafe either if that's what you meant, because as_chunks has unsafe code inside of it as well.