Passing self to callback returning future vs lifetimes

Hi, let's starts the question wit hthe example code and the errors:

use futures::executor::block_on;
use std::future::Future;

pub struct Doer {}

impl Doer {
    pub async fn do_it(&mut self) {
        println!("do it!");
    }

    pub async fn call_it<F, Fut>(&mut self, f: F)
    where
        F: FnOnce(&mut Doer) -> Fut,
        Fut: Future<Output = ()>,
    {
        self.do_it().await;
        f(self).await;
        self.do_it().await;
    }
}

fn main() {
    let mut d = Doer {};
    block_on(async {
        d.call_it(|d| async {
            d.do_it().await;
            d.do_it().await;
        });
    });
}

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
error: lifetime may not live long enough
  --> src/main.rs:25:23
   |
25 |           d.call_it(|a| async {
   |  ____________________--_^
   | |                    ||
   | |                    |return type of closure `impl futures::Future` contains a lifetime `'2`
   | |                    has type `&'1 mut Doer`
26 | |             a.do_it().await;
27 | |             a.do_it().await;
28 | |         });
   | |_________^ returning this value requires that `'1` must outlive `'2`

error[E0373]: async block may outlive the current function, but it borrows `a`, which is owned by the current function
  --> src/main.rs:25:29
   |
25 |           d.call_it(|a| async {
   |  _____________________________^
26 | |             a.do_it().await;
   | |             - `a` is borrowed here
27 | |             a.do_it().await;
28 | |         });
   | |_________^ may outlive borrowed value `a`
   |
note: async block is returned here
  --> src/main.rs:25:23
   |
25 |           d.call_it(|a| async {
   |  _______________________^
26 | |             a.do_it().await;
27 | |             a.do_it().await;
28 | |         });
   | |_________^
help: to force the async block to take ownership of `a` (and any other referenced variables), use the `move` keyword
   |
25 |         d.call_it(|a| async move {
26 |             a.do_it().await;
27 |             a.do_it().await;
28 |         });
   |

error: aborting due to 2 previous errors

From my basic understanding the function argument passed to the lambda in life 25 (d.call_it(|a| async {) has a lifetime that is just as long as the lambda's body. Since the future block is returned and still references d it outlives the lifetime of d. Is that correct?

I tried many different ways to make this work, but I will spare you with all my failed attempts. It looks like there should be a way to make this code work, since the future that f returns is awaited within the body of call_it. Thus, the reference in a should be valid for the whole lifetime of the Future. What I want to specify is F: FnOnce(&'f mut Doer) -> Fut + 'f, where 'f is basically all of line 17 (f(self).await;) and not just the function body of f, but also not more.

  • Is the code actually safe? (If lifetimes are fixed somehow)
  • How to add lifetimes to make it actually compile?

Thank you all!

3 Likes

This kind of code with async function-typed parameters that take reference arguments is unfortunately (currently) notoriously hard to get to compile. As far as I’m aware, due to limitations in the type checker this is at the moment only possible if you introduce some extra helper traits and also only if you don’t actually need support for closures in the call to .call_it but are okay with using only fns, which don’t support capturing anything. Then it works like this:

use futures::executor::block_on;
use std::future::Future;

pub trait AsyncSingleArgFnOnce<Arg>: FnOnce(Arg) -> <Self as AsyncSingleArgFnOnce<Arg>>::Fut {
    type Fut: Future<Output=<Self as AsyncSingleArgFnOnce<Arg>>::Output>;
    type Output;
}

impl<Arg, F, Fut> AsyncSingleArgFnOnce<Arg> for F
where
    F: FnOnce(Arg) -> Fut,
    Fut: Future,
{
    type Fut = Fut;
    type Output = Fut::Output;
}

pub struct Doer {}

impl Doer {
    pub async fn do_it(&mut self) {
        println!("do it!");
    }

    pub async fn call_it<F>(&mut self, f: F)
    where
        F: for<'a> AsyncSingleArgFnOnce<&'a mut Doer, Output = ()>,
    {
        self.do_it().await;
        f(self).await;
        self.do_it().await;
    }
}


fn main() {
    let mut d = Doer {};
    block_on(async {
        d.call_it({
            async fn f(a: &mut Doer) {
                a.do_it().await;
                a.do_it().await;
            }
            f
        }).await;
    });
}

(playground)

Compare this answer and also this thread for similar kinds of questions.

4 Likes

Note that an alternative is to box the returned future (so as to perform dyn type erasure), since that way what was a hard to express generic parameter becomes a fixed (albeit erased) type, which is less confusing for rust:

+ use ::futures::future::BoxFuture;
  
  impl Doer {
      ...

-     pub async fn call_it<F, Fut>(&mut self, f: F)
+     pub async fn call_it<F     >(&mut self, f: F)
      where
-         F : FnOnce(&'_ mut Doer) -> Fut,
-                                     Fut : Future<Output = ()>,
+         F : FnOnce(&'_ mut Doer) -> BoxFuture<'_, ()>,
      {
          self.do_it().await;
          f(self).await;
          self.do_it().await;
      }
  }
  
  fn main() {
      let mut d = Doer {};
      block_on(async {
-         d.call_it(|a|          async move {
+         d.call_it(|a| Box::pin(async move {
              a.do_it().await;
              a.do_it().await;
-         }) .await;
+         })).await;
      });
  }

Obviously if you can use @steffahn's solution with a good old function rather than a closure, then you should use that one, but for stateful closure the easiest solution is to add this tiny bit of Boxing (which, by the way, involves the same amount of Boxing as #[async_trait] does; so dismissing this heap allocation may be a premature optimization).

  • This solution has also the advantage of showing why your code did not originally work: notice that the returned BoxFuture<'_, ...> type does capture the lifetime '_ parameter that matches that of the input &'_ mut Doer, whereas, in your generic code, the type Fut is fixed, quantification-wise, before that '_ lifetime is introduced, and thus Fut cannot depend on it. Thus, your signature was requiring a future that did not capture the &'_ mut Doer parameter.

    Regarding expressing that property in a generic fahsion, you were looking for:

    F : FnOnce(&'_ mut Doer) -> impl '_ + Future<Output = ()>,
    
    i.e.
    F : for<'doer> FnOnce(&'doer mut Doer) -> impl 'doer + Future<Output = ()>,
    // i.e.,
    for<'doer>
        F : FnOnce(&'doer mut Doer) -> impl 'doer + Future<Output = ()>
    ,
    // i.e, (pseudo notation)
    for<'doer>
        F : FnOnce(&'doer mut Doer) -> X?<'doer>
    ,
    for<'doer>
        X?<'doer> : 'doer + Future<Output = ()>
    ,
    // i.e. (real notation, nightly-only)
    for<'doer>
        F : FnOnce<(&'doer mut Doer,)>
    ,
    for<'doer>
        <F as FnOnce<(&'doer mut Doer,)>>::Output : 'doer + Future<Output = ()>
    ,
    

    Sadly the above is not yet fully understood by Rust which gets confused with the higher-order bounds (whilst it may work for explicit async fns, like @steffahn did, it doesn't for closures that involve type inference), but if we replace the impl with a dyn in the return type, then the signature becomes, genericity-wise, way simpler, and Rust understands it:

    F : FnOnce(&'_ mut Doer) -> dyn '_ + Future<Output = ()> // + Send /* for convenience */
    // and since a `dyn` cannot be directly returned yet without indirection:
    F : FnOnce(&'_ mut Doer) -> Pin<Box<dyn '_ + Future<Output = ()> + Send>>
    // i.e.
    F : FnOnce(&'_ mut Doer) -> BoxFuture<'_, ()>
4 Likes

Thanks you all, I went with BoxFuture here as @Yandros suggested. @steffahn's solution is also very clean and I definitely learned something. Unfortunately, I need closures and not just functions in my code.

I could not get this to work, rustc complains that impl traits are not allowed in this position, also on nightly. Was this also meant as pseudo yet-to-be-implemented code?

I tried this one (playground), but ran into an ICE, i guess it's not polished for general usage (that's why it's nightly only...). I learned a lot about upcoming features though :slight_smile:

Yes, hence my "i.e."s slowly transforming it into accepted code (nightly-only, or requiring a helper trait on stable Rust), and as you can see, it is buggy: sometimes the compiler crashes (in your case, though, your forgot to tuple-wrap the args in the FnOnce: it's FnOnce<(Arg0,)>, not FnOnce<Arg0>), other times, it doesn't crash but refutes logically valid propositions:

7 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.