Why wrapping a relatively large array in a Mutex causes a stack overflow?

I've found that if I wrap a relatively large array (400² elements) in a Mutex, I get a stack overflow on a dev build run.

Now, as far as I can read, large arrays are a known, general, issue (due to stack size limitations), however, my question is: why, in this case, the array alone won't crash, but wrapping it in a Mutex will?

Playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=6855051504ffce57e98c390a8c900f86

You your problem there is not the mutex but the large array as a local variable on the stack. You get the error even if you remove the mutex in your example.

Suggest using Vec instead. Vec is allocate don the heap.

1 Like

It’s probably the fact that you’ll have two copies of the array on the stack in the version with the mutex, going just about the stack limit. Note that by accessing array.len() after moving/copying the array into the mutex, you basically force two copies of the array being on the stack (one inside of the mutex) at the same time. [Although, in a non-optimized build, you’ll have two stack variables for two variables in program code anyways, so the accessing of len doesn’t really make a difference.]

Of course you probably don’t want this kind of large values on the stack. In the setting where the dimensions are a constant you can get the advantages of an array of not being dynamically sized (so the compiler can e.g. use the statically used size for bounds checks) through using a Box<[[MyColor; DIMENSION]; DIMENSION]>. Unfortunately it is non-trivial to avoid having the array on the stack at all, especially in debug builds. You can’t just use Box::new. But starting from a vec! expression it is possible like this:

use std::convert::TryInto;
use std::sync::Mutex;

#[derive(Copy, Clone)]
pub struct MyColor {
    pub r: i32,
    pub g: i32,
    pub b: i32,
}

fn main() {
    const DIMENSION: usize = 2000;
    let _array: Box<[[MyColor; DIMENSION]; DIMENSION]> =
        vec![[MyColor { r: 0, g: 0, b: 0 }; DIMENSION]; DIMENSION]
            .into_boxed_slice()
            .try_into()
            .ok() // avoids printing the array in the error case 
            .unwrap(); // won’t panic since sizes do match
    let _mutex = Mutex::new(_array);
    println!("{}", _mutex.lock().unwrap().len());
}

(playground)

1 Like

If the mutex is removed, the error is not raised: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=ae72573910102cc896b7919a4b2f6390

Note that by accessing array.len() after moving/copying the array into the mutex, you basically force two copies of the array being on the stack (one inside of the mutex) at the same time.

Hm, I'm confusded - if println! copies, like the Mutex does, it should crash, but it doesn't (see link above).

My guess is that the array is being copied into the callframe of Mutex::new, and even though the array fits on the stack once, it doesn't fit twice. It doesn't crash in release mode.

2 Likes

println! and friends don't copy - they only borrow.