HELP: heap allocation, stack overflow error?

Hello There,

I am actually new by Rust and I am trying to learn it. When I try to allocate an array on the heap, I get that error "thread 'main' has overflowed its stack". What is wrong here?

fn main() {
let arr = Box::new([0i32; 5_000_000]);
println!("{}", arr[1_000_000])
}

I belive, that it is happening, because the array first need to be created on stack, and then moved to Box. If you want your memory to be fully dynamicly allocated, just use Vec. If you are struggling why rustc didn't optimize this stack allocation - I strongly believe it would in Release mode, but you really don't wont to optimize such things in debug mode (unless you like to debug inlined code).

Also please format your code using Markdown (this one is pretty small so maybe not a problem, but it really upgrades post readability).

2 Likes

To add to @hashedone's answer, you can use Box::into_boxed_slice if you'd like to end up with a Box<[i32]> (note this will be different from a box holding a compile-time fixed size array):

let arr = vec![0; 5_000_000].into_boxed_slice();

This is slightly more efficient than a Vec<i32> because the capacity field is dropped, and you end up with a fixed size slice.

In case you're using the nightly compiler, you can use the box_syntax unstable feature to do what you want:

#![feature(box_syntax)]
...
let arr = box [0; 5_000_000];

In general, arrays whose length is >32 are a lot less ergonomic/useful in current Rust, so you may want to avoid them for those reasons as well.

2 Likes

Additionally if you really want to go with Box and heap allocation, you may allocate your array using alloc::heap::allocate, std::mem::transmute it to mut pointer to your type, and then construct your box with std::boxed::Box::from_raw, but it involves much of unsafe code, and I cannot imagine much cases where this is better than Vec created by std::vec::Vec::with_capacity (possibly even vec! macro would do the job, but I am not sure if it gives warranties, that allocated capacity is exact to needed).

I'd just be mindful that Vec::with_capacity() creates a 0-length vector, which is different from what looks to be the intent in the OP.

I'm pretty sure vec![0; 5_000_000] will not allocate more capacity than necessary, and in either case, into_boxed_slice() would drop excess if it was present. This is a pretty sizeable allocation, so one would likely want to pool/reuse buffers like that either way (if they come and go from the application's standpoint).

2 Likes

Thats true, and I didn't mention, that it is use-case dependent. However in most cases if such zeroed vector is created, its just to fill it with actual values later, so I silently assumed such case. For situation you mention (if op really wants to have zero filled buffer) I also mentioned vec! macro (and you mentioned it before), but I am not aware if it warranties, that reserved memory isn't larger than actually needed (in general I thing it is, but I couldn't find confirmation quickly). Obviosly even if its not, it may be achieved with vec! + shrink_to_fit, but it would do allocation + reallocation and if op really cares to use Box instead of Vec for some reason, it has to be low level optimization related.

BTW, don't use transmute to change pointer types. Regular as casts are allowed for pointer types, and they don't have the footgun of mem::transmute (where type inference may infer other type than what you had in mind and transmute to total crashy nonsense).

Thanks for that, I don't usually work with pointers in rust, and just didn't know that such casting is possible.