How to implement an Fn/FnMut where the lifetime of the reference to self appears in the return type

Hi folks,
In Fn's and FnMut's, one cannot have the lifetime of the reference to self appear as a parameter in the output. Is there any workaround for this? Would this be possible in the compiler? Is there any prior work in this direction?

It seems like let s = "Foo".to_string(); move || { &s } should be possible to compile and to name!

Alternatives considered

  • A potential alternative would be to have a custom struct with implementing a trait with a method fn call_alt_mut<'a>(&'a mut self) -> Foo<'a>. However, as a proc macro author, I cannot create this struct, because (in general) I cannot infer the types of referenced variables. So, this isn't feasible for my use case.
  • Returning an owned struct that uses interior mutability. This is the current workaround.

Is this possible? Any help or pointers would be appreciated!

Playground with demonstration: Rust Playground

Best,
Robert

You can have a Box<dyn Deref<Target=str>> (or whichever target you need).

Maybe I'm missing something, but how would one write a closure that compiles and returns a reference to a box?

let foo = Box::new("asdf".to_string());
move || { &foo }

Fails to compile for the same reason.

First, this requires GAT.

Second, FnOnce is a supertrait of Fn and FnMut, and it cannot be implemented for this closure. Removing it from being a supertrait will be a breaking change.

However, a new trait may be introduced.

1 Like

Thanks for the response. Now that I thought about this, of course it needs GATs.

I'll read up more about GATs now :slight_smile:

I meant as an alternative "type erased thing that yields references to its contents", like so. I'm unsure that it meets your use-case though.

(You may also want to replace 't with 'static.)

Now that (non-object safe) GATs are available in nightly, is this something that would be doably implementable in the compiler? Is anyone working on this?

Alternatively, since I know the concrete return type that the alt_call_mut call must implement, and I know it the struct must only implement FnMut (and importantly, not FnOnce), I could potentially approach from the other side. Is it possible to do "closure expansion" (Closure expansion - Guide to Rustc Development) in a proc macro? In particular, is it possible to somehow take a bunch of references and have the compiler help in inferring a struct for them? Then, I would not need GATs.

This is exactly what I said is imposssible:

Yes, if we add different trait. I don't think anyone is working on that, and I'm not even sure the motivation is strong enough.

Nope, unfortunately.

Correctly analyzing how variables should be captured (by shared reference, mutable reference or move) requires type information, something that proc macros can't access.

1 Like

Could you say a little about what your motivation is for having this feature?

Must only implement FnMut, not FnOnce

Sorry, I wasn't clear. I only need to have FnMut functionality, i.e. I want to repeatedly call it, without consuming the struct. So, implementing trait AltFnMut { fn call_alt_mut<'a>(&'a mut self) -> Foo<'a> } for a struct is sufficient. I don't need AltFnMut to extent AltFnOnce or have an associated type parameter, for example.

But given that closure expansion by proc macros is impossible, having a bespoke trait doesn't help.

Nope, unfortunately

Thanks, that's what I figured!

Motivation for this feature

Imagine a struct that wraps a rusty React component, jsx!(<div onClick={move |_| state+=1}>{state}</div>). One might consider implementing that as:

struct Component {
  dom: Dom::Node(...) // contains a reference to state
  onClick: move |_| state += 1,
}

This won't compile, because onClick moves state into it, but it's borrowed by the dom field. But it is possible to model React components as a closure, e.g.

let component = move |phase| {
  match phase {
    Phase::Render => PhaseResult::Render(Dom::Node(...)),
    Phase::OnClick(event) => {(|_| state+=1)(event); PhaseResult::Event}
  }
}

Everything so far works. This compiles. I even have a a proc macro that expands the JSX syntax to a very verbose closure.

However, I'd like to have this closure return references to closed variables, for two reasons.

  1. Reduce the amount of cloning that is involved in calling component(Phase::Render)
  2. Calling component(Phase::Render) would also return a vector of references to streams of events (e.g. from a websocket). "React" would then process the first event that arrived, and re-render. It would then call component(Phase::Render) again, and re-subscribe again.

The current workaround is to clone for 1. and for 2. to use internal mutability, i.e. to have the closure return Vec<Rc<RefCell<Box<dyn EventSource>>>>.

Both workarounds are sufficient! But I am hoping to do better.

Note, all this was already discussed over:

  • it doesn't need GATs, just closures sugared with better trait singatures; but modulo an edition boundary or some disambiguating keyword ot opt into that I don't see that really happening.

  • I have showcased a general pattern to achieve this, however, through a helper type:

This may make the task a bit more difficult, e.g., by requiring that your proc-macro be called with type annotations provided by the caller:

your_macro!(move(a: i32, b: u8, s: String…) || -> &str {
    …
    &s
})

so that you can define State = (i32, u8, String) and then perform a destructuring assignment to bind (a reference to) that state to (a, b, s).


[EDIT: didn't see your last post with a more concrete example]

Another alternative which may play nicer with the type inference could be to perform a transformation like

does:

let s = String::from("…");
move |event, return_| {
    …
    return_(&s)
}
  • with return_ being a &own dyn FnOnce(<PseudoReturnOfTheClosure>) -> Token, and expecting the whole closure to return Token (which you may rename to ReturnValue for nicer error messages to downstream users).

    • where &own is the owned equivalent of &mut, perfect for dyn FnOnce() —see this other post (where I named them &move references: it had the advantage of reusing a keyword, but in hindsight &own is a bit better since memory isn't actually moved, since that's the point of the & "reference" indirection).

    • We don't have a &own in the language you may say? Then ask for it to be added, it's a gaping hole in Rust's usual trinity design. Until then, you can use Box and heap-allocate; or make do with a &mut dyn FnMut(), and an Option::take() dance:

      fn from_once_to_mut_hack<'f, R> (f: impl 'f + FnOnce() -> R)
        -> impl 'f + FnMut() -> R
      {
          let mut fopt = Some(f);
          move || {
              let f =
                  fopt.take()
                      .expect("\
                          With `&own` references this code path \
                          would be unreachable\
                      ")
              ;
              f()
          }
      }
      

      or you can use ::stackbox

and your runtime would then do:

- let user_closure = move |_| { … &s };
+ let user_closure = move |_, return_| { … return_(&s) };
let component = move |phase| {
  match phase {
    Phase::Render => PhaseResult::Render(Dom::Node(...)),
    Phase::OnClick(event) => {
-     let returned_value = user_closure(event);
-     stuff(returned_value);
+     user_closure(event, &own |returned_value| {
+       stuff(returned_value)
+     })
      PhaseResult::Event
    }
  }
}
2 Likes

There is quite a lot of prior art on this problem that you might be interested to look at (basically most GUI frameworks). Off the top of my head, the source code of iced, druid, but you can take inspiration from others as well.

Personally I quite like reference-counted shared ownership (without interior mutability) for UI app state. The downside is that app users have to use them, but the positive is that you can recurse down your tree of data just comparing pointers to see if anything changed (without interior multability, there isn't any other way for state to change).

Wow, thank you for this with_locals suggestion. It's very likely that I will be able to use the pattern (i.e. pass an Fn(RenderResult) -> Token) and avoid excessive cloning!

Luckily, the user never has to provide a closure that returns borrowed data — only closures that are generated by the macro return borrowed data — so it should be easy enough to restructure the whole thing to take a continuation function:

const component = |phase, render_continuation| {
  match phase {
    Phase::Render => { let dom = ...; render_continuation(dom) }
  }
}

Thank you for the detailed response and context!


@derekdreery — To be honest, I really dislike requiring the user to use interior mutability, as that seems to make it more difficult to:

  • leverage the compiler to ensure that user-written code is correct
  • interop with other Rust libraries

And it isn't idiomatic!

But I am open to the suggestion that I'm a masochist who is just out to prove that it is possible to do this without requiring users to use interior mutability. ("Think of the overhead of an RC!" he says, despite crossing the Wasm-JS boundary thousands of times per tick.)

1 Like

Nit: Rc/Arc doesn't use interior mutability - it does the shared ownership. So if you want to change something down the tree, you have to replace all the nodes above it as well like immutable.js. It goes well with the im crate that implements immutable data structures (which use trees for flat arrays/maps to do structural sharing like immutable.js)

Not disparaging your work on this by any means, there isn't a canonical solution in this space yet, so we need more people experimenting!

1 Like

Good catch, I should've been more precise.

No offense taken :slight_smile:

Double nit: Rc/Arc does use interior mutability - for the ref counter (Cell/AtomicUsize, respectively).

You are technically correct, the best kind of correct.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.