Idiomatic way of working with uninitialized dynamic memory

#1

Hey everyone,

Basically, I want to fill a Vec<u8> with data from an external source and I also don't want to pay the cost for initializing the memory.

What's the idiomatic way of accomplishing such a goal? For example, is this okay?

let mut buf = Vec::<u8>::with_capacity(1024);

let ptr = buf.as_mut_ptr();
for idx in 0..1024 {
  unsafe { ptr.add(idx).write(idx * idx) };
}

unsafe { buf.set_len(1024) };
return buf;
2 Likes
#2

:+1: On stable, that's about as good as it gets at the moment.

There are a couple of nightly-only features that make working with uninitialized memory more ergonomic and less unsafe.

#![feature(vec_spare_capacity)]
#![feature(maybe_uninit_extra)]

fn make_vec() -> Vec<u8> {
    let mut buf = Vec::with_capacity(1024);

    let data = buf.spare_capacity_mut();
    for idx in 0..1024 {
        data[idx].write((idx * idx) as u8);  // no `unsafe` here
    }

    unsafe {
        buf.set_len(1024);
    }
    buf
}
3 Likes
#3

Ha, thank you.

I just wanted to make sure that I wasn't breaking all the rules by doing this but I guess this is the beauty of Vec having a set_len method.

#4

Yup, see the example in its docs of using FFI to write into the buffer then set_len to the initialized length: https://doc.rust-lang.org/1.47.0/std/vec/struct.Vec.html#method.set_len

There may also be other ways, but we'd need more information to help with that. (I assume the actual population code you put there is a placeholder, since there's an easy just-as-fast same-one-allocation safe version of it.)

#5

Let me step back and ask: is this something you really need, i.e. have you benchmarked that it has a measurable or meaningful impact on the performance of your code? You could as well zero-initialize the array using vec![0; 1024], because modern OSes almost universally provide pre-zeroed memory, which Rust allocators can also use with no performance penalty and without actually performing a bunch of byte-wise dead writes.

4 Likes