Is it possible to achieve these lifetimes in this generic FnOnce -> Future?

Consider the following code:

struct S;

impl S {
    fn mutate(&mut self) {}
    fn mutate_async(&mut self) -> impl Future<Output = ()> + '_ {
        ready(())
    }
}

async fn main() {
    let mut v = vec![1];
    let _ = make_future(|s| { // F
        v.push(2);
        // Fut
        async {
            v.push(3);
            s.mutate();
            s.mutate_async().await;
        }
    });
    // would be even better if I could use v again here
}

What I want to achieve is the following:

  1. The argument to make_future is an FnOnce taking a &mut S.
  2. The FnOnce returns a Future that can call the methods of S and access the variables in the FnOnce (these variables are either declared inside the FnOnce or were moved in it).
  3. Once Future has resolved, makes_future needs to have ownership of S.

My initial thought is as follows:

async fn make_future<F, Fut>(f: F)
where
    F: FnOnce(&mut S) -> Fut + Send + '???,
    Fut: Future<Output = ()> + Send + '???,
{
    let mut s = S;
    let fut = f(&mut s);
    fut.await;
    // makes_future has ownership of S.
    s.mutate();
    s.mutate_async().await;
}

How can I fix the lifetimes of the function so that it achieves what I want?

A straightforward solution would be to make Fut return S, but it would become a little bit annoying for the user of make_future.

Two possible improvements but still not compiling:

V1

use std::future::Future;

struct S;

impl S {
    fn call(&self) {}
}

async fn make_future<'f, F, Fut>(f: F)
where
    F: FnOnce(&'f mut S) -> Fut + Send + 'f,
    Fut: Future<Output = ()> + Send + 'f,
{
    let mut s = S;
    let fut = f(&mut s);
    fut.await;
    s.call();
}

fn main() {
    let mut v = vec![1, 2, 3];

    let _ = make_future(|s| {
        v.push(4);
        async {
            v.push(5);
            s.call();
        }
    });

    v.push(6);
    println!("{:?}", v);
}
Compiler errors
error[E0597]: `s` does not live long enough
  --> src/main.rs:15:17
   |
9  | async fn make_future<'f, F, Fut>(f: F)
   |                      -- lifetime `'f` defined here
...
14 |     let mut s = S;
   |         ----- binding `s` declared here
15 |     let fut = f(&mut s);
   |               --^^^^^^-
   |               | |
   |               | borrowed value does not live long enough
   |               argument requires that `s` is borrowed for `'f`
...
18 | }
   | - `s` dropped here while still borrowed

error[E0502]: cannot borrow `s` as immutable because it is also borrowed as mutable
  --> src/main.rs:17:5
   |
9  | async fn make_future<'f, F, Fut>(f: F)
   |                      -- lifetime `'f` defined here
...
15 |     let fut = f(&mut s);
   |               ---------
   |               | |
   |               | mutable borrow occurs here
   |               argument requires that `s` is borrowed for `'f`
16 |     fut.await;
17 |     s.call();
   |     ^ immutable borrow occurs here

V2

use std::future::Future;

struct S;

impl S {
    fn call(&self) {}
}

async fn make_future<'f, F, Fut>(f: F)
where
    F: for<'a> FnOnce(&'a mut S) -> Fut + Send + 'f,
    Fut: Future<Output = ()> + Send + 'f,
{
    let mut s = S;
    let fut = f(&mut s);
    fut.await;
    s.call();
}

fn main() {
    let mut v = vec![1, 2, 3];

    let _ = make_future(|s| {
        v.push(4);
        async {
            v.push(5);
            s.call();
        }
    });

    v.push(6);
    println!("{:?}", v);
}
Compiler errors
error: lifetime may not live long enough
  --> src/main.rs:25:9
   |
23 |       let _ = make_future(|s| {
   |                            -- return type of closure `{async block@src/main.rs:25:9: 28:10}` contains a lifetime `'2`
   |                            |
   |                            has type `&'1 mut S`
24 |           v.push(4);
25 | /         async {
26 | |             v.push(5);
27 | |             s.call();
28 | |         }
   | |_________^ returning this value requires that `'1` must outlive `'2`
   |
   = note: requirement occurs because of a mutable reference to `Vec<i32>`
   = note: mutable references are invariant over their type parameter
   = help: see <https://doc.rust-lang.org/nomicon/subtyping.html> for more information about variance

This is the closest I could get, but the compiler can’t point me to the location of the '2 borrow, and I can’t spot where it’s happening.

trait FutFnOnce<'a> {
    type Fut: 'a+Future<Output=()>;
    fn call(self, s:&'a mut S)->Self::Fut;
}

impl<'a, F, Fut> FutFnOnce<'a> for F where
    F: FnOnce(&'a mut S)->Fut,
    Fut: 'a + Send + Future<Output=()>
{
    type Fut=Fut;
    fn call(self, s:&'a mut S)-> Fut { (self)(s) }
}

async fn make_future<F>(f: F)
where
    F: 'static + Send + for<'a> FutFnOnce<'a>,
{
    let mut s = S;
    let fut = f.call(&mut s);
    fut.await;
    s.mutate();
    s.mutate_async().await;
}

fn main() {
    let mut v = vec![1, 2, 3];
    let _ = make_future(move |s:&mut S| {
        v.push(4);
        async move {
            v.push(5);
            s.mutate();
            s.mutate_async().await;
        }
    });
}

When you write F: FnOnce(&mut S) -> Fut this is syntax sugar for F: for<'a> FnOnce(&'a mut S) -> Fut. Notice how the lifetime 'a is declared on the bound itself, this requires the closure to be valid for any possible lifetime. However Fut is declared "outside" this bound, so it cannot name 'a. In fact Fut is a single type, but if you want it to name 'a then it would have a family of types. In a non-valid-Rust syntax this would be F: for<'a> FnOnce(&'a mut S) -> Fut<'a>. AFAIK the only way to express this is to use a separate trait (taken from the async-fn-traits crate):

trait AsyncFnOnce1<Arg0>: FnOnce(Arg0) -> Self::OutputFuture {
    type OutputFuture: Future<Output = <Self as AsyncFnOnce1<Arg0>>::Output>;
    type Output;
}

impl<F, Fut, Arg0> AsyncFnOnce1<Arg0> for F where
    F: FnOnce(Arg0) -> Fut,
    Fut: Future,
{
    type OutputFuture = Fut;
    type Output = Fut::Output;
}

Then you would be able to do use the bound F: for<'a> AsyncFnOnce1<&'a mut S, Output = ()>. However if you do this you'll hit this issue HRTBs: "implementation is not general enough", but it is · Issue #70263 · rust-lang/rust · GitHub

5 Likes

I decided to explore this awhile[1] to see if I could I could shake anything new out of the tree... like a clearer explanation of why things don't work, if anything else.

TL;DR

A doubly-type-erased version that works

One of the challenges is that you have a return type that's needs to be valid for the maximum of the function's validity and the input's validity -- the intersection of the two -- because you're returning a future that contains both the "fields of the closure" (the &'f mut Vec<_> the closure captured) and the input &'s mut S.

Type erasure is used to work around a lot of Rust's higher-ranked shortcomings, but it's hard to write up this requirement in type-erased form, even.

// There's no syntax for `'int = intersection('f, 's)`
dyn 'f for<'s> FnOnce(&'s mut S) -> BoxFuture<'int, ()>
where // there is no support for `dyn ... where`
    'f: 'int, 's: 'int

However, you wanted this to be higher-ranked over 's so that utilizers could pass in a local borrow, whereas the closure is coming from the caller. So it's presumably sufficient to just say 'f: 's, in which case the intersection is 's.

dyn 'f for<'s where 'f: 's> FnOnce(&'s mut S) -> BoxFuture<'s, ()>

There's no for<'s where 'f: 's> either, but we can fake it by mentioning a type that requires that in order to be well-formed.

// "where 'f: 's"                                    vvvvvvvvvv
dyn 'f + for<'s> FnOnce(&'s mut S) -> BoxFuture<'s, [&'s &'f (); 0]>

That's enough for the type erased version.

Problems with a direct translation to generic bounds

You can then try to walk backwards from there to make a trait/generic version, which may look something like this.

// Where `Self: 's` (`'f: 's`)
trait MakeFutureOnce<'f, 's, Guard = &'s &'f Self>: 's + FnOnce(&'s mut S) -> Self::Fut {
    type FutOut;
    type Fut: Send + Future<Output = Self::FutOut>;
}

// Implemented for `Fn`s with our hacky lifetime guard
impl<'f, 's, F, Fut> MakeFutureOnce<'f, 's> for F
where
    F: FnOnce(&'s mut S) -> Fut,
    Fut: Send + Future<Output = [&'s &'f (); 0]>,
{
    type FutOut = [&'s &'f (); 0];
    type Fut = Fut;
}

// Higher-ranked version with implied `'f: 's` bound
trait MakeFuture<'f>: 
    for<'s> MakeFutureOnce<'f, 's, FutOut = [&'s &'f (); 0]> {}
// Implemented for everything that meets the bounds
impl<'f, M: for<'s> MakeFutureOnce<'f, 's, FutOut = [&'s &'f (); 0]>> MakeFuture<'f> for M {}

And the compiler does actually accept these bounds, somewhat. For example, this works:

async fn make_future<'f, F: MakeFuture<'f>>(f: F) {
// ...

    let bx: BoxMakeFuture<'_> = Box::new( same as before );
    make_future(bx);

So our type-erased version meets these bounds.

But then you get a new problem. Off-hand it looks like Rust's poor higher-ranked closure inference strikes again, and that our F: MakeFuture<'f> bound gets in the way of the usual "funnel it through a function" workaround. That's at least partially true, but not the whole story.

When we made the translation from type-erased form to trait bounds, we actually lost a property that's important to "conditional higher-ranked bounds" -- for<'s where 'f: 's> emulated by an implied bound. Our "lifetime guard" [&'s &'f (); 0] got moved from part of the type into only associated types of traits.

If the lifetime relationship (&'s &'f) isn't part of the implementing type and it's also not an input to the trait (e.g. a trait parameter), it's not going to be an implied bound of the implementation, which will stymie higher-ranked bounds elsewhere (like in our MakeFuture<'f> supertrait bound).

I suspect these bounds not working for non-type-erased types is required for soundness.

Related issues:

Moving the lifetime guard to be a trait input

One way to fix this for the closure trait would be to make [&'s &'f (); 0] an argument of the closure, although that requires passing some dummy [] every time we call it.

But as it turns out, we can make some progress by just adding it to our trait. And we can also add a subtrait of Future that hoists its associated type into a trait parameter, too.[2]

trait ShinyFuture<Out>: Send + Future<Output = Out> {}
impl<F: ?Sized + Send + Future<Output = Out>, Out>
    ShinyFuture<Out> for F {}

//                           vvvvvv newly hoisted from associated type
trait MakeFutureOnce<'f, 's, FutOut, Guard = &'s &'f Self>
where
    Self: 's + FnOnce(&'s mut S) -> Self::Fut
{
    type Fut: ShinyFuture<FutOut>;
}

impl<'f, 's, F, Fut> MakeFutureOnce<'f, 's, [&'s &'f (); 0]> for F
where
    F: FnOnce(&'s mut S) -> Fut,
    Fut: ShinyFuture<[&'s &'f (); 0]>,

This can let us get rid of one layer of type erasing (the closure). Note that we still needed a funnel to influence the closure inference:

fn funnel<'f, F>(f: F) -> F
where
    F: for<'s> FnOnce(&'s mut S) -> 
        Pin<Box<dyn 's + ShinyFuture<[&'s &'f (); 0]>>>,

If you try a bound at a higher level, like just F: MakerFuture<'f>, it's not enough. I had to drill down to the Fn trait level with the bound.

I'm currently of the opinion that this would be a solution for having no type erasing, if there was a way to properly override the higher-ranked capturing of the closure.[3] But as far as I know, there isn't. The next section looks at an attempt which falls short.

N.b. since we didn't get rid of the future type erasure, we didn't actually need the ShinyFuture trait to hoist the associated type into a type parameter.[4]

Failed attempt to not name the future

One of the reasons our funnel used Pin<Box<dyn ShinyFuture<..>>> is that the Fn trait sugar forces you to name the output type, even though it's an associated type. And it's not a single type since it captures a lifetime under the binder, so a generic parameter won't work. But compiler-generated futures don't have names, and there's no stable alternative yet either.[5]

However, you don't have to use the sugar on nightly. Perhaps that's a way forward?

First, note again that we couldn't influence the higher-ranked nature of the closure until we drilled our funnel bounds down to the Fn traits exactly. But if we do that on nightly starting from our last playground, we end up with something like...

fn funnel<'f, F>(f: F) -> F
where
    for<'s> F: FnOnce<(&'s mut S,)>,
    for<'s> <F as FnOnce<(&'s mut S,)>>::Output: ShinyFuture<[&'s &'f (); 0]>,

Note how in the first bound, we've lost the "lifetime guard" [&'s &'f (); 0]. That means that the first bound has to be met when 's is 'static, which is not possible for our use case.

We can attempt to fix this by making [&'s &'f (); 0] an input parameter like we mentioned before...

fn funnel<'f, F>(f: F) -> F
where
    for<'s> F: FnOnce<(&'s mut S, [&'s &'f (); 0])>,
    for<'s> <F as FnOnce<(&'s mut S, [&'s &'f (); 0])>>::Output: 
        ShinyFuture<[&'s &'f (); 0]>,

But this doesn't influence closure inference like we wish it did, either.

And I think I know why. Just because because the output type of a function implements a trait that has an input that mentions 's doesn't mean that it captures 's. It's perfectly reasonable that String: From<&'s str> for example, even though String: 'static. (Adding 's under the binder doesn't improve things and adds a "due to current limitations" error.)

Whereas when we were mentioning a concrete type, if that type had 's as a parameter, it definitely captured 's and thus the closure had to be passing its higher-ranked input into the output in some sense.

So trait bounds on the associated type aren't enough, and this is probably a dead-end. It also means that inline associated type bounds aren't likely to do anything either.

// We don't have these either but it probably doesn't matter
// for this use case anyway
where
    for<'s> F: FnOnce<
        (&'s mut S, /* optionally: [&'s &'f (); 0] */),
        Output: ShinyFuture<[&'s &'f (); 0]>,
    >,

What we really need is probably something like

fn funnel<'f, F>(f: F) -> F
where
    for<'s> F: FnOnce(&'s mut S, [&'s &'f  (); 0])
        -> impl use<'s, 'f> ShinyFuture<[&'s &'f (); 0]>

Or even better

fn funnel<'f, F>(f: F) -> F
where
    for<'s where 'f: 's> F: FnOnce(&'s mut S)
        -> impl use<'s> Future<Out = ()> + Send

...assuming they work like we wish they would work, anyway.

Parting thoughts

That's as far as I got this time. As far as I know it's still not possible without type erasure (or unsafe). (But I'm also not familiar with the various helper traits that exist in the ecosystem, so it's possible I missed something.)

Unlike the Fn traits, you can implement Future on stable. But if you had some LocalFuture<F: Future>, you'd still have to name F in the funnel, so that doesn't necessarily gain you much.

TAIT might be another way forward as it gives -> impl Trait bounds indirectly / gives a way to name otherwise unnameable things. I made a few attempts, but they didn't work. I didn't give it as much effort as it perhaps deserves, though (ran out of steam).[6]


  1. for the umpteenth time ↩︎

  2. spoilers, this ended up not helping, but anyway ↩︎

  3. without naming the output type ↩︎

  4. But I'm too lazy to remove it. ↩︎

  5. generic type constructor, capturing -> impl Trait in bounds, etc. ↩︎

  6. Note that the "due to current limitations in the borrow checker" error occurs here too. For all I know it's internally equivalent to the failed associated type bound. ↩︎

3 Likes

Oh hey, we're about to get these. But yeah, I don't think they help.

1 Like

Are you referring to the Rust roadmap for 2024? If so, I have a feeling that async closures will also help in resolving this issue. :slight_smile:

Do you mean my most recent comment? If so, I meant we're getting associated type bounds on stable a couple days from now.