Debugging and fixing stack overflow for extremely large function

Hey guys a newbie question here. I am writing a library that calls a remote JSON API and needs to (un)marshal polymorphic results. I use traits to represent polymorphism. I devised an algorithm to matching the type discriminator values coming form the wire that from limited benchmarks seems to outperform HashMap and simple match statement.

I made all the code generation work. To my dismay the generated code cannot be called as it causes stack overflow on entry into my 20K line function.

Why would a function with nested match and if statements overflow the stack on entry? Is there good way to avoid this?

My function goes like this

fn deserialize_object<'de, A: de::MapAccess<'de>>(type_name: &str, map: A) -> Result<VimAny, DeserializationError<A, A::Error>> {
    match type_name.len() {
        2 => {
            if type_name == "ID" {
                let ds = de::value::MapAccessDeserializer::new(map);
                let obj: Id = de::Deserialize::deserialize(ds)?;
                Ok(VimAny::Object(Box::new(obj)))
            } else { Err(DeserializationError::UnknownVariant(map)) }
        }
        3 => {
            if type_name == "Tag" {
                let ds = de::value::MapAccessDeserializer::new(map);
                let obj: Tag = de::Deserialize::deserialize(ds)?;
                Ok(VimAny::Object(Box::new(obj)))
            } else { Err(DeserializationError::UnknownVariant(map)) }
        }
       // the above repeats for all length i.e. up to 63 or so.
       // Some lengths have many variants and I use match like so
        6 => {
            match type_name {
                "Action" => {
                    let ds = de::value::MapAccessDeserializer::new(map);
                    let obj: Action = de::Deserialize::deserialize(ds)?;
                    Ok(VimAny::Object(Box::new(obj)))
                }
        // Some lengths have many many options and I first match on a fixed group of symbols
        7 => {
            let s = &type_name[0..4];
            let Some(type_ord) = to_u32(s) else {
                return Err(DeserializationError::PassThruError(de::Error::custom("Internal Error: Cannot convert to unsigned int")));
            };
            match type_ord {
                0x4556434d => { // EVCM
                    if type_name == "EVCMode" {
                        let ds = de::value::MapAccessDeserializer::new(map);
                        let obj: EvcMode = de::Deserialize::deserialize(ds)?;
                        Ok(VimAny::Object(Box::new(obj)))
                    } else { Err(DeserializationError::UnknownVariant(map)) }
                },
                0x4576656e => { // Even
                    if type_name == "EventEx" {
                        let ds = de::value::MapAccessDeserializer::new(map);
                        let obj: EventEx = de::Deserialize::deserialize(ds)?;
                        Ok(VimAny::Object(Box::new(obj)))
                    } else { Err(DeserializationError::UnknownVariant(map)) }
                },
        // it ends like so. It is peculiar that I return the map ownership back so it can be used in fallback
        _ => { Err(DeserializationError::UnknownVariant(map)) }
    }
}

So few questions:

  1. How can I see what Rust is trying to put on the stack when entering my function? I do not have local variables on top level
  2. Do you see anything specifically wrong with the above code?
  3. Any advice how to debug and address the above.
  4. Do you think partitioning the generated function to say one per length cluster will alleviate the issue. I have the same code with just 4 options not 3500 and it works fine.

In my prior iteration I have similar code that uses HashMap with many small functions like the code in the individual match arms. It is working ok and may be few times slower then I hope the new code will do.

I also want to shift serialization and inter trait cast functions to use match statements too. Thus it is important for me to understand why big match statements cause havoc on the stack before I spend couple of days writing code generator..

most programming languages including Rust mix the program call stack and stack of active local data frames, that's the reason stack overflow can be caused not only by very deep (potentially unbounded, think recursion) function calls, but also if you create stack variables of very large size.

for example, these line can potentially overflow the stack if obj is very large.

sometimes, the optimizer can eliminate the stack variable because it is immediately moved into heap allocated memory, but it's not always possible. you can check if this is the case, e.g. try to run the code with --release, or enable opt-level = 2 or above for the dev profile.

another workaround is run the code with large stack. the default stack for the main thread can only be configured in a platform specific way (usually with some linker flags), but you can spawn a thread with the thread::Builder::stack_size() API:

Thank you. I solved this by splitting my giant function in many functions handling the different key lengths if a key length has more than one value.

It is a bit strange that all the stack allocations between the different match arms delineated by curly braces are allocated upfront upon entry into the function. I gather this would be some LLVM decision or rust decision.

My understanding (I guess not entirely correct) was that stack allocation and de-allocation incl. calls to Drop trait for a given block happen within the block.

As I understand it, LLVM currently disallows this kind of optimization because observing pointer addresses causes spooky action:

Recycle storage after move · Issue #61849 · rust-lang/rust
Miscompilation: Equal pointers comparing as unequal · Issue #107975 · rust-lang/rust