Getting stack overflow with RefCell<[f32;52000]>

[Zenith97]

I'm currently learning Rust, but I've encountered a stack overflow issue.
Could anyone help me out? I would be very grateful.

1. Why do the last two lines of main() cause stack overflow, but the variable "a" doesn't cause stack overflow.
2. Why does the stack overflow not happened when the length of the ARRAY_SIZE is less than 42399?

use std::cell::RefCell;

const ARRAY_SIZE: usize = 52000;

pub struct MyStruct {
    array: RefCell<[f32; ARRAY_SIZE]>,
}

fn main() {
    println!("{}", ARRAY_SIZE);

    let a = [0.0_f32; ARRAY_SIZE];
    println!("{}", a.len());

    let c = RefCell::new(a);
    println!("{}", c.borrow().len());

    let batch = MyStruct { array: c };
    println!("{}", batch.array.borrow().len());
}

[firebits.io]

Arrays in Rust use the stack memory, so you ended up consuming all of the stack available memory.

[Zenith97]

So it is just because of the array's size is too big?
Thank you!

[jumpnbrownweasel]

Hi @Zenith97 , welcome! You can use a Vec instead of an array to allocate large arrays on the heap on the heap. Using Vec is extremely common in Rust. The stack is normally much smaller than the heap, so the heap is used for large data structures.

Also see The Stack and the Heap in the Rust book.

[gretchenfrage]

You could also use a RefCell<Box<[f32; ARRAY_SIZE]>>, which would fix your issue. But probably you want to use RefCell<Vec<f32>> instead, yes. A Vec<f32> is a heap allocation of an automatically managed size that lets you make some arbitrary prefix of it actually initialized (its length). On the other hand, a Box<[f32; ARRAY_SIZE]> is a heap allocation of ARRAY_SIZE floats all of which have to be always initialized. The Box<[f32; ARRAY_SIZE]> is most similar to the raw [f32; ARRAY_SIZE], in that they're both just ARRAY_SIZE fully initialized floats stored back-to-back, with the only difference being without the box it's storing them directly on the stack whereas with the box it stores them on the heap and just puts on the stack a pointer to it. But for most situations you probably want the Vec.

[jofas]

@gretchenfrage

RefCell<Box<[f32; ARRAY_SIZE]>>, which would fix your issue

Just a side node as this might trip up OP, Box::new([0_f32; ARRAY_SIZE]) would still cause a stack overflow, because the array is first created on the stack before it is moved into the Box. Example. One would need to create a Vec and cast it into the boxed array instead:

let b: Box<[f32; ARRAY_SIZE]> = vec![0_f32; ARRAY_SIZE].into_boxed_slice().try_into().unwrap();

[quinedot]

@jofas

would still cause a stack overflow

Sometimes it doesn't, when the compiler is smart enough to optimize out the part where the temporary value is on the stack (try running on release mode).

Unfortunately Rust has no way yet to guarantee this optimization (or otherwise have "enplacement", "placement new"), so in practice you have to assume it's always possible the temp value will eat up your stack.

Thus the best thing to do is exactly as you said and this is a pedantic nit .

[drewtato]

FYI, you can omit into_boxed_slice: impl<T, const N: usize> TryFrom<Vec<T>> for Box<[T; N]>

let b: Box<[f32; ARRAY_SIZE]> = vec![0_f32; ARRAY_SIZE].try_into().unwrap();

[Zenith97]

Got it! Thank you!

[Zenith97]

Thanks everybody, your solutions are very precise.
Love you guys

[chung]

I am still puzzled about OP's questions. I thought the default size for the stack is 2mb?
And like what OP said, the variable a was not the problem as 52,000x4 bytes(f32) is just 0.208mb.

So which part of the program actually caused overflow?

[jofas]

The array is causing the overflow. From the docs, emphasis added by me:

The default stack size is platform-dependent and subject to change. Currently, it is 2 MiB on all Tier-1 platforms.

[drewtato]

That's only the default size for threads spawned by Rust. The main thread has some OS-dependent stack size decided by the linker. For example, MSVC uses 1MB.

@chung

I am still puzzled about OP's questions. I thought the default size for the stack is 2mb?
And like what OP said, the variable a was not the problem as 52,000x4 bytes(f32) is just 0.208mb.

It's really easy to accidentally copy this array, which debug mode doesn't usually optimize out. Get a few copies lying around, plus all the stuff already on the stack, and it'll overflow.