Stack overflow with panic=unwind but OK with panic=abort

help
1

Hi folks,

I have a program that needs a large stack (for a good reason).

When I run cargo run ... on x86 Linux:

with

use tokio::runtime::Builder;

fn main() {
    Builder::new_multi_thread()
        .thread_stack_size(2 * 1024 * 1024)
        .enable_io()
        .enable_time()
        .build()
        .unwrap()
        .block_on(my_function())
}

and

[profile.dev]
overflow-checks = true
opt-level = 0
panic = 'abort'

It works OK. But when I change it to:

fn main() {
    Builder::new_multi_thread()
        .thread_stack_size(1024 * 1024 * 1024) // crazy large
        .enable_io()
        .enable_time()
        .build()
        .unwrap()
        .block_on(my_function())
}

and

[profile.dev]
overflow-checks = true
opt-level = 0
panic = 'unwind'

I get

thread 'main' has overflowed its stack
fatal runtime error: stack overflow

Does it make sense?

Thank you!

2

Do you have a recursive function by any chance? It could be that in the case of panic=unwind, the cleanup code prevents tail-call optimization from being applied, causing O(1) stack usage to turn into O(n).

3

Thank you @bjorn3 , no recursion, just a lot of text operations and large nested structs...

4

Btw. I can solve it by adding Box::pin....await in several places but I'm trying to figure out why I can't just increase stack...

5

I think with panic=abort those large nested structs are constructed in-place after allocating the Box, while with panic=unwind they are first constructed on the stack and then moved into the Box in case memory allocation panics to preserve behavior when the memory allocation panics.

3 Likes
6

I'd assume that even with unwinding, 1024 MiB of RAM should be enough since 2 MiB works fine without unwinding.

7

Btw this works too with 2MiB stack:

[profile.dev]
overflow-checks = true
opt-level = 1
panic = 'unwind'
8

This is a common problem in Rust when using a lot of async fn + .await, without any Box allocation.

Whenever you use .await, Rust merges the state of the future being awaited with the future that does awaiting. If you write an async app this way, then all those states of all those futures get merged together, and snowball into one giant Future struct. The entire state of your entire application, for all the async calls it can make anywhere, ends up bundled together, and then it doesn't fit on the stack.

The problem would also happen with panic=abort, but you see it for unwind first, because unwinding happens to generate more code and preserve more state. With unwinding the giant whole-program-in-one Future is slightly larger and/or the code generating and passing it is less compact.

Wrap less frequently called functions call().await in a pinned Box:

Box::pin(call()).await

and use cargo clippy to find what large futures remain:

Adjust Clippy config to a lower threshold if necessary:

3 Likes
9

Thank you very much. It makes all sense, I just don't understand how is it possible that I can run it with

opt-level = 0
panic = 'abort'

and

.thread_stack_size(2 * 1024 * 1024)

but

opt-level = 0
panic = 'unwind'

and

.thread_stack_size(1024 * 1024 * 1024)

is not enough.

I'd think unwind will need 10x. And I actually can't even test how much it needs but it's definitely more than 100x.

10

Obligatory reminder that opt-level = 0 is telling the compiler to not even try.

I'd be curious what happens if you use even just level one, so that LLVM can at least attempt to memcpyopt away some of the big temporaries.

4 Likes
11

Hi @scottmcm ,

Yes, opt-level = 1 also helped.

The story has an interesting end.

Today, I upgraded all dependencies + Rust from 1.83 to 1.87 + Edition from 2021 to 2024 and I don't have stack overflows anymore. And there is no single Box::pin in my code.

I thought I was doing something wrong so I wiped everything (.cargo, .rustup, target, Cargo.lock) and I re-set Cargo.toml to

[profile.dev]
opt-level = 0
overflow-checks = true
panic = "unwind"
strip = "none"

[profile.release]
opt-level = 2
overflow-checks = false
panic = "abort"
strip = "symbols"

and everything is just fine. I run cargo run -- ... and no stack overflow.

So I'm not sure if it's the Rust version, LLVM, or maybe Tokio, but it's all good now...

I guess there was a memory leak somewhere. I had a feeling that I'm not doing anything crazy and 1 GiB of RAM should be totally enough... And now 2 MiB (!!!) is enough :wink: