Construct a Box[[[i32; 32]; 32]; 32]> without hitting stack

Is there a way to construct a Box< [[[i32; 32]; 32]; 32] > without it ever hitting the stack ? (since it's 128MB in size).

It's just 128 kB, but <Box<_> as Default>::default() seems to do the trick.

7 Likes

The general answer here is via https://doc.rust-lang.org/std/boxed/struct.Box.html#method.new_zeroed or https://doc.rust-lang.org/std/boxed/struct.Box.html#method.new_uninit, as that gives you memory from calloc or malloc (well, not literally, but the rust equivalents) without actually initializing the underlying T. Then it's up to you to use unsafe code to initialize it properly before turning it into a normal Box<T>.

6 Likes

I'm moving over the solution tag as this seems to be the more general / less magical solution.

If you need a general solution that doesn’t require unstable features, isn’t subject to the current element count limits of stable Default, and doesn’t have to be pretty, this should work:

pub fn make_big_box() -> Box<[[[i32; 64]; 64]; 64]> {
    // Alloc on heap.
    let data = vec![0; 64 * 64 * 64];

    // Throw away redundant capacity value.
    let data = data.into_boxed_slice();

    // Extract raw pointer, losing Box.
    let data = Box::into_raw(data);

    // Cast from slice to 3-D array.
    let data = data as *mut [[[_; 64]; 64]; 64];

    // Recreate box taking ownership.
    // Safety: Ensure allocation sized correctly above.
    unsafe { Box::from_raw(data) }
}

While that will be the case once that nightly feature is stabilized, I wouldn't say that a nightly-only API really counts as a general solution.

In the interim, that API is polyfilled within the ::uninit crate:

FWIW, it's also important to note that .uninit()-followed-by-init(…) only elides the stack temporary if using opt-level ≥ 2, e.g., when compiling on --release. So that's another important caveat to keep in mind.

  • Using unsafe code might be needed to write guaranteed-in-place initialization in that case, although given a legitimate I could update that crate with new APIs, should those be deemed useful.

Finally, for the very case of 0-init array, I think that @H2CO3's suggestion is both simple and effective. And for non-0-init array, by starting off a 0-init one, we don't need unsafe anymore. So :ok_hand:

2 Likes

Also, one doesn't even need default if they are worried about hitting the array size limit for which Default is implemented. vec![…].into_boxed_slice().try_into().unwrap() works just fine.

3 Likes

Thanks! I’m glad to learn a simpler (safe) way to do that. No idea why try_into() never occurred to me.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.