Problem with closure returning a closure size not known at compile time

Considering something like this:

|| || {}

A closure that returns a closure.

I am trying to store that closure into a structure and then pass it around until it is called. I don't want to use function pointers because I may need to capture my environment (full example in this playground).

impl Foo {
  fn new(f: [...]) -> Foo {
    Foo { fun: Rc::new(f) }
  }

  fn later(&self) {
    (self.fun)()
  }
}

let foo = Foo::new(|| || {})

I am using an Rc so that the closures can be passed around in my real life code.

This fails because rust does not like a closure that do not have a return type with a known size at compile time.

If the closure were to return an int, it would work.

Is this a syntax problem? And there is a way I can explain to rust how to get the compile time size of the return closure somehow. Or is it that what I am trying to do is just impossible in Rust?

Note: I could probably template my struct like in the unboxed closure example of this excellent StackOverflow answer to get around that problem but in my real life code I don't think this is desirable. The way I construct my Foo is supposed to be very simple and having to specify types might make it impractical.

Rust is indeed surprisingly tame around types like Rc<dyn Fn() -> dyn Fn()>. Not rightfully so, the error messages are a bit unhelpful as a result. Really Fn() -> Foo doesn’t support output types Foo that are unsized like dyn Fn(). What you can do is using a Box<dyn Fn()> (or Rc<dyn Fn()>, depending on your use-case) return type for the closure. I.e.

struct Foo {
    fun: Rc<dyn Fn() -> Box<dyn Fn()>>,
}

If you don’t want to lose the ability to be able to write Foo::new(|| || {}) without a Box::new, that’s possible, too, by making Foo::new generic, i.e.

impl Foo {
    fn new<F: Fn() + 'static>(fun: impl (Fn() -> F) + 'static) -> Foo {
        Foo { fun: Rc::new(move || Box::new(fun())) }
    }

No, it’s not a syntax problem. It’s “impossible” in the way that it’s impossible to work with a type like Rc<dyn Fn() -> dyn Fn()> which – as I hinted above – should IMO be forbidden in the first place, or at least way more explicitly called out in the error message.

3 Likes

It's not possible in this form unless someday Rust gains the ability to return dynamically sized types. (That won't be any time soon and might not ever happen.) Instead you need the returned dyn Fn to be behind some sort of sized indirection, like in a Box. Or, as you noted, give up on dyn Fn and use generics.

Consider this variation, where the dyn Fn returned from the outer closure has a different size at runtime depending on the inputs. There is no "compile time size of the return closure." And the language doesn't allow changes to the body to change the behavior of the API, so as far as the compiler is concerned, every dyn Fn producer has this dynamic quality.

1 Like

Note that the type Rc<(dyn Fn() -> (dyn Fn()))> is indeed pretty useless: Since the return type is unsized and Fn traits require the Output type to be sized, while writing the type itself is not an error, it doesn’t actually implement Fn. The error message

error[E0618]: expected function, found `Rc<(dyn Fn() -> (dyn Fn() + 'static) + 'static)>`
  --> src/main.rs:13:17
   |
13 |         let _ = (self.fun)();
   |                 ^^^^^^^^^^--
   |                 |
   |                 call expression requires function

should be more clear about this though. Lots of room for improvement. “expected function, found Rc<(dyn Fn() -> (dyn Fn() + 'static) + 'static)>” means:

the type Rc<(dyn Fn() -> (dyn Fn() + 'static) + 'static)> or anything it dereferences to does not implement Fn() -> …

and the reason for that is that while Rc<(dyn Fn() -> (dyn Fn() + 'static) + 'static)> dereferences to dyn Fn() -> (dyn Fn() + 'static) + 'static, and that’s a dyn Fn() -> … trait object, that trait object doesn’t actually implement Fn because for the output type, dyn Fn() + 'static: Sized is not fulfilled. IMO this error message should mention that the root problem here is that dyn Fn() doesn’t have a size known at compile time and dyn Fn() -> … trait objects require a return type of known size.

2 Likes
impl Foo {
    fn new<F: Fn() + 'static>(fun: impl (Fn() -> F) + 'static) -> Foo {
        Foo { fun: Rc::new(move || Box::new(fun())) }
    }

Actually in your generics example here, you are calling the upper closure within the new function but I need this function to be called at a later time. It this wasn't the case, I wouldn't need to store that closure, I would only need its resulting closure which would be easier.

So even with generics, it seems I can't avoid this Box::new in the middle of the user's code.

That’s not true, but maybe that’s hard to spot if you aren’t too familiar with closures. I’m wrapping the call to the closure in a new closure. It’s like composing a function Fn() -> Foo with a function Fn(Foo) -> Bar to create a function Fn() -> Bar. In this case Foo is F and Bar is Box<dyn Fn()>. In other words, the call fun() only happens as soon as the newly created closure move || … is called.

1 Like

Right, I got it working in this playground.

Too bad we need another indirection for the creation of the Box but at least the using code is clean.

To clarify, this is what all the other languages do already, except they hide this allocation under the rug :wink: The issue with Rust is that it requires the code to be explicit w.r.t. most heap-allocations, which can lead indeed to some verbiage when trying to write very functional-style code (in that case, using a "true" functional language may be more appropriate than using Rust). But it won't lead to hindered performance, quite the opposite!

One possible thing, though, would be to feature a SmallBox optimization, so as to feature inlined / non-heap allocated and yet dyn/type-erased closures, provided the size of their captured environment is small enough (feel free to ask about this point if you want further details, such as an implementation, to be provided) :slightly_smiling_face:

1 Like

For future reader of this thread: a reference to some documentation on how to return lambdas: Advanced Functions and Closures - The Rust Programming Language

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.