Is there a way to build this struct on the heap or any alternative to this `large_stack_frames` issue?

help
1

I have this rust code:

#[derive(Debug, Default, Clone)]
pub struct PlayerFilters  {
    pub id: Option<IdFilter>,
    pub created_at: Option<DateTimeFilter>,
    pub created_by: Option<IdFilter>,
    pub position: Option<NumberFilter<i16>>,
    pub score: Option<NumberFilter<i16>>,
    pub and: Vec<Self>,
    pub or: Vec<Self>,
    pub not: Option<Box<Self>>,
    pub with_team: Option<TeamCondition>,
    // and a lot more fields here!
}

impl TryFrom<PlayerFilters> for Box<PlayerFilters> {
    type Error = CustomError;

    fn try_from(filters: PlayerFilters) -> Result<Self, Self::Error> {
        Ok(Self::new(PlayerFilters {
            id: filters.id.map(TryInto::try_into).transpose()?,
            created_at: filters.created_at,
            created_by: filters.created_by.map(TryInto::try_into).transpose()?,
            position: filters.position.map(TryInto::try_into).transpose()?,
            score: filters.score.map(TryInto::try_into).transpose()?,
            and: filters.and.map_or_else(
                || Ok(Vec::new()),
                |o| filters.into_iter().map(|item| item.try_into().map(|boxed: Self| *boxed)).collect::<Result<Vec<_>, _>>(),
            )?,
            or: filters.or.map_or_else(
                || Ok(Vec::new()),
                |o| filters.into_iter().map(|item| item.try_into().map(|boxed: Self| *boxed)).collect::<Result<Vec<_>, _>>(),
            )?,
            not: filters.not.map(|o| (*o).try_into()).transpose()?,
            with_team: filters.with_team.map(|o| (*o).try_into()).transpose()?,
            // and a lot more fields here!
        }))
    }
}

and I get this warning:

warning: this function may allocate 564738 bytes on the stack
391 |       fn try_from(o: PlayerFilters) -> Result<Self, Self::Error> {
    |          ^^^^^^^^
392 |           Ok(Self::new(PlayerFilters {
    |  ______________________-
393 | |             id: filters.id.map(TryInto::try_into).transpose()?,
394 | |             created_at: filters.created_at,
395 | |             created_by: filters.created_by.map(TryInto::try_into).transpose()?,
...   |
484 | |                 .transpose()?,
485 | |         }))
    | |_________- this is the largest part, at 92944 bytes for type `PlayerFilters`
    |
    = note: 564738 bytes is larger than Clippy's configured `stack-size-threshold` of 512000
    = note: allocating large amounts of stack space can overflow the stack and cause the program to abort
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#large_stack_frames
    = note: `-W clippy::large-stack-frames` implied by `-W clippy::nursery`
    = help: to override `-W clippy::nursery` add `#[allow(clippy::large_stack_frames)]`

Is there a way to avoid builind this huge struct on the stack building it directly on the heap?

Is there any alternative?

2

From the snippet I can't tell why your struct is so big. Maybe you can box some more fields to get the size more manageable? If not, I think alloc::alloc or Box::new_uninit and then initialising every field is currently the best way to initialise a struct instance directly on the heap. For reference:

1 Like
3

I removed many fields for brevity.

So without unsafe there is no way?

4

I think the answer is yes. This is the only method I know:

use std::alloc::{Layout, alloc};

const SIZE: usize = 10000000;

#[derive(Debug)]
#[repr(transparent)]
struct Foo {
    inner: [u8; SIZE],
}

impl From<Vec<u8>> for Box<Foo> {
    fn from(value: Vec<u8>) -> Self {
        if value.len() != SIZE {
            panic!("Vec length must be exactly {}", SIZE);
        }

        unsafe {
            let inner = alloc(Layout::new::<Foo>()) as *mut Foo;
            inner.copy_from(value.as_ptr() as _, 1);
            Box::from_raw(inner)
        }
    }
}

fn main() {
    let v = vec![0u8; SIZE];
    let _foo = Box::<Foo>::from(v);
}
5

If any of the fields are arrays, you can box them, but I think you already know that.

If there are arrays created as intermediate values, in the inline expression for each field value, I also wonder whether it would help to move the expression into a function, so that function pops its stack when you assign the field value. I'm not sure, it's just something I would try if it were me.

That same technique could also apply to creation of a large struct (many fields or array fields) for a field value, when you are boxing it. That struct must be on the stack before it is boxed, but you could try doing this in a separate function that returns the boxed struct.

6

This impl is both impossible and redundant since impl From<T> for Box<T> already exists. What's the actual impl you're trying to make? And have you run into stack overflows in real usage?

While this struct is moderately large, another issue is that there's 6 instances of it in the function. Each |boxed: Self| *boxed counts as another PlayerFilters on the stack[1]. And this function appears to be recursive, which multiplies the stack usage yet again. A filter of the form not(not(not(...))) will always eventually be able to overflow the stack[2], but of course, the smaller your function's stack usage, the longer the filter can be before that happens.

I also don't think there's a guaranteed way besides unsafe, but macros could help ensure you don't forget fields or mistype the pointer operations.

  1. Optimization should let these occupy the same space, so it really should only take 1 or 2 instances, but clippy doesn't assume optimization, and you don't want debug builds to be broken either. ↩︎

  2. unless you move this stack off the stack ↩︎

7

This doesn't look right. The Vec<u8> would be difficult to create with the right value, and likely dangerous due to lack of alignment.

There's into_boxed_slice() on Vec.

It would be much easier to use Vec::with_capacity(1) and either try your luck with vec.push(struct {}) optimizing better than Box::new (it might, because the memory is already allocated before the call), or use vec.spare_capacity_mut() to unsafely write into the heap directly.

8

The pin-init crate can take care of all the unsafe for you. Unfortunately it looks like the crate doesn't come with the non-pinned version of the macro, but if you can deal with the Pin Box it should work.