Lifetime of Async Closure

[stackoverflow010]

I've found a few references to a similar issue that people seem to have, but I still don't really understand what's wrong. I would like to create a simple "processor" which, when given a callback, can execute that callback when it consumes certain events (i.e. pub/sub). In order to do this, I have created a processor struct which tries to call an async callback (in such a way that type erasure helps me pass a generic callback fn).

The sticky part is that I want to pass a reference of self into the callback. When I do this, I get an opaque error that I think I understand, but every time I try to fix it, it seems I don't really understand it. Specifically, I encounter:

   |                          ------- ^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
   |                          |     |
   |                          |     return type of closure `impl futures_util::Future<Output = ()>` contains a lifetime `'2`
   |                          lifetime `'1` represents this closure's body
   |
   = note: closure implements `Fn`, so references to captured variables can't escape the closure

If I pass an async function into the closure which does refer to the struct (e.g. some random function that does not maintain a pointer to self), then everything compiles just dandy. However, when I try to move self into the callback, I get this error that references lifetimes I don't really understand. I'm not really sure where '1 and '2 are coming from, or what they really refer to. As far as I can tell, it's just saying that the compiler believes that the returned future has a greater lifetime than the enclosing closure, but I don't really know what to do about that, since when I use a function outside of self, everything compiles. I thought I could solve this problem by providing and Arc<Self> to the callback, since that may help the compiler hold the reference, but that clearly doesn't work. At the moment, I think this design may just be fundamentally flawed.

I have reproduced a minimal example, which does it's best to honestly represent the larger codebase at work:

use std::sync::Arc;

use futures_util::Future;

struct CbProcessor<Fut>
where
    for<'a> Fut: Future<Output = ()> + Send + Sync + 'a,
{
    cb: Arc<dyn Fn() -> Fut + Send + Sync + 'static>,
}

impl<Fut> CbProcessor<Fut>
where
    for<'a> Fut: Future<Output = ()> + Send + Sync + 'a,
{
    fn new(cb: impl Fn() -> Fut + Send + Sync + 'static) -> Self {
        CbProcessor { cb: Arc::new(cb) }
    }

    fn poll(&mut self) {
        let cb = self.cb.clone();
        tokio::spawn(async move {
            loop {
                println!("clever polling routine...");
                tokio::time::sleep(std::time::Duration::from_millis(1500)).await;
                cb().await;
            }
        });
    }
}

struct Foo {}

impl Foo {
    fn start_processor(self: Arc<Self>) {
        CbProcessor::new(move || self.callback_fn()).poll();
    }

    async fn callback_fn(&self) {}

    pub async fn foo() {
        println!("I did foo!");
    }
}

#[tokio::main]
async fn main() {
    let foo = Arc::new(Foo {});

    Foo::start_processor(foo.clone());
}

and the ensuing error:

error: lifetime may not live long enough
  --> src/main.rs:39:34
   |
39 |         CbProcessor::new(move || self.callback_fn()).poll();
   |                          ------- ^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
   |                          |     |
   |                          |     return type of closure `impl futures_util::Future<Output = ()>` contains a lifetime `'2`
   |                          lifetime `'1` represents this closure's body
   |
   = note: closure implements `Fn`, so references to captured variables can't escape the closure

[vague]

The problem is about async Lifetimes, and Foo shouldn't be consumed in the captured closure^[1], so you could do this

async fn helper(f: Arc<Foo>) {
    f.callback_fn().await
}

impl Foo {
    fn start_processor(self: Arc<Self>) {
        CbProcessor::new(move || helper(self.clone())).poll();
    }
}

---

1. since you need the reference of Foo to be called upon multiple times in Fn ↩︎

[wangbyby]

fn callback_fn(&self)-> impl Future<Output=()> + Send+Sync {
        async {
            Foo::foo().await;
            ()
        }
    }

1. Try this, as the book says async-lifetimes.

2. And vague gives a good advice.

3. But sometimes I run, it prints nothing (rustc 1.66.0 , win10)

[vague]

@wangbyby

fn callback_fn(&self)-> impl Future<Output=()> + Send+Sync

Well, your solution doesn't work when &self is used in the async block:

error[E0700]: hidden type for `impl futures_util::Future<Output = ()> + std::marker::Send + Sync` captures lifetime that does not appear in bounds
  --> src/main.rs:40:9
   |
39 |       fn callback_fn(&self) -> impl Future<Output = ()> + Send + Sync {
   |                      ----- hidden type `impl futures_util::Future<Output = ()>` captures the anonymous lifetime defined here
40 | /         async move {
41 | |             self;
42 | |         }
   | |_________^
   |
help: to declare that `impl futures_util::Future<Output = ()> + std::marker::Send + Sync` captures `'_`, you can add an explicit `'_` lifetime bound
   |
39 |     fn callback_fn(&self) -> impl Future<Output = ()> + Send + Sync + '_ {
   |                 

Even after taking the advice from the compiler, it still doesn't work.

But my solution works, because everytime you use &self, you actually use the reference of a clone of Foo, then you can have:

fn helper(f: Arc<Foo>) -> impl Future<Output = ()> + 'static {
    async move { f.callback_fn().await }
}