Could you help me with the lifetime issue with returning a future of future from an async method?

help
1

Context:

I want to release a lock after the first stage of a feature is completed. So I made a method that returns a future of a future. The idea is that I can hold the lock when await the outer future, release the lock after the outer future resolves, then await the inner future.

However, I'm not able to drop the lock guard while the inner future is alive. Because the inner future's lifetime is forced to be coupled with &self. But I don't see why the inner future needs to have its lifetime tied to &self. Could you help me out? Thanks a lot.

Here's the code example with inline questions.

use std::{marker::PhantomData, pin::Pin};

use futures::FutureExt;

struct MyType<T>(PhantomData<T>);

impl<T> MyType<T> {
    async fn get(
        &self,
        fut: impl Future<Output = T> + 'static,
    ) -> Pin<Box<dyn Future<Output = T> + '_>> {
        let ret = fut.await;
        // Without `'_` above, the compiler will complain that:
        // the parameter type `T` may not live long enough
        // the parameter type `T` must be valid for the static lifetime...
        //
        // But adding `'_` means the returned future cannot live longer than `self`.
        // Why is this necessary? Doesn't the future only refer to `ret`, which has nothing to do
        // with `self` once it's computed?
        async { ret }.boxed_local()
    }
}

async fn example() {
    let my_type = MyType::<i32>(PhantomData);
    let intermediate = my_type.get(async { 42 }).await;
    // The following line will not compile because `intermediate`'s lifetime is tied to `my_type`.
    // But ideally, I want to decouple the lifetime of `intermediate` from `my_type`.
    drop(my_type);
    println!("Result: {}", intermediate.await);
}
2 
use core::{marker::PhantomData, pin::Pin};
struct MyType<T>(PhantomData<T>);
impl<T> MyType<T> {
    fn get(&self, fut: impl Future<Output = T> + 'static) -> Pin<Box<dyn Future<Output = T> + '_>> {
        Box::pin(fut)
    }
}
async fn example() {
    let my_type = MyType::<i32>(PhantomData);
    let intermediate = my_type.get(async { 42 }).await;
    drop(my_type);
    println!("Result: {intermediate}");
}

The above compiles. While this is probably not a real example, you may want to consider adding a separate method that returns a Future that is also Send to allow for things like tokio::spawn. For example an additional method like below:

fn get_send(&self, fut: impl Future<Output = T> + Send + 'static) -> Pin<Box<dyn Future<Output = T> + Send + '_>> {
    Box::pin(fut)
}
3

Thanks for your reply. But I want to drop BEFORE await. The reason is that the real thing I drop is actually a RwLock guard. The inner future is a network call. I don’t want to hold the lock for the entire duration of the network call, causing unnecessary contention. I will add a more realistic example tomorrow.

4

Yeah, unfortunately dyn Future<Output = T> is not enough for it to imply that it's lifetime is restricted by T.

It's not necessary, but it's a way to fix the aforementioned error because T is known to be valid as long as self is. To be more precise, '_ refers to the 's lifetime in &'s self, which is known to be valid at least as long as an instance of self, which has type MyType<T>, and that is only valid as long as the type T is valid.

However as you noticed this is overrestrictive, since there are situations where T is valid even after the &self reference becomes invalid. Ideally the solution is to bound the returned future by the lifetime for which T is valid, but there's no syntax for this. The next best thing is bounding the returned future by "some" lifetime that is required to be valid as long as T is.

    async fn get<'t>(
        &self,
        fut: impl Future<Output = T> + 'static,
    ) -> Pin<Box<dyn Future<Output = T> + 't>>
    where
        T: 't
    {
        // ...

An alternative fix would be bounding T by 'static, since you're already requiring fut to be 'static and I believe there's no way to have a fut that's 'static without also having T be 'static. However fut being 'static is not needed for anything else (at least in your example), so if you use the other solution you'll be able to remove it, while with this solution you'll have to keep it.

1 Like
5

Thanks! The first solution is exactly what I need!

Ideally the solution is to bound the returned future by the lifetime for which T is valid, but there's no syntax for this

Why does this require a syntax?
Or why can't the compiler simply decide that dyn Future<Output = V> can only live as long as the components made up of it, namely V?
Is this a limitation of the compiler? Or is there any possibility that a type like Wrapper<Associated = V> can live longer than V?

6

Your phrasing ("Can live", "may live longer") may indicate a (common) misconception about what Rust lifetimes are about.

Anyway, it is already the case that dyn Future<Output = V> + '_ is only valid/usable for as long as its components (V and the dyn lifetime) are. But the dyn lifetime is, generally speaking, independent of the parameters from the trait.

The dyn lifetime is a constraint on/about the type-erased implementor. Arguably a better default for the elided lifetime would be the limit of the other parameters, instead of 'static. (It can't be a requirement to be met, as that would prevent a lifetime limited type to coercing to dyn Trait<StaticStuff> + '_.)

But there is no way to declare a lifetime as the said limit. I don't know how technically possible it is or isn't. If it existed, it would take at least an edition to change the dyn elision defaults.

The "any lifetime that T (and other parameters) is valid for" is the pattern that works instead. You can see it on this function as one example in an analogous "lifetime limit of a generic" situation.

I suppose there could be a syntax for functions specifically that introduces a new anonymous lifetime and desugars to the pattern, even without actual "limit of parameters" language support.