Accepting an async closure + lifetimes = stumped

1 
    /// Whereas get_drill allows the caller to store the drill with a strong reference, thus controlling how long it stays in memory,
    /// this function only gets a borrow to a drill, thus saving a clone. If the drill is not found, then
    /// None will be passed into the supplied function.
pub async fn async_borrow_drill<F: Future>(&self, version: Option<u32>, f: impl FnOnce(Option<&Drill>) -> F) -> F::Output {
        let read = self.read().await; // RwLock Read handle
        if let Some(version) = version {
            f(read.toolset.get_drill(version)).await
        } else {
            f(read.toolset.get_most_recent_drill()).await
        }
}

The above function compiles just fine. When I run the command with this, compilation and execution is also a success:

cnac.async_borrow_drill(Some(51), async move |drill_opt| {
     println!("This runs!");
}).await;

This, however, does not compile:

cnac.async_borrow_drill(Some(51), async move |drill_opt| {
     if let Some(drill) = drill_opt {
           println!("Borrowing drill vers: {}", drill.get_version());
     }
}).await;

The error is:

error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
   --> hyxe_user\tests\primary.rs:102:82
    |
102 |                           cnac.async_borrow_drill(Some(51), async move |drill_opt| {
    |  __________________________________________________________________________________^
103 | |                             if let Some(drill) = drill_opt {
104 | |                                 println!("Borrowing drill vers: {}", drill.get_version());
105 | |                             }
106 | |                         }).await;
    | |_________________________^
    |
note: first, the lifetime cannot outlive the anonymous lifetime #2 defined on the body at 102:59...
   --> hyxe_user\tests\primary.rs:102:59
    |
102 |                           cnac.async_borrow_drill(Some(51), async move |drill_opt| {
    |  ___________________________________________________________^
103 | |                             if let Some(drill) = drill_opt {
104 | |                                 println!("Borrowing drill vers: {}", drill.get_version());
105 | |                             }
106 | |                         }).await;
    | |_________________________^
note: ...so that the types are compatible
   --> hyxe_user\tests\primary.rs:102:82
    |
102 |                           cnac.async_borrow_drill(Some(51), async move |drill_opt| {
    |  __________________________________________________________________________________^
103 | |                             if let Some(drill) = drill_opt {
104 | |                                 println!("Borrowing drill vers: {}", drill.get_version());
105 | |                             }
106 | |                         }).await;
    | |_________________________^
    = note: expected  `std::option::Option<&hyxe_crypt::drill::Drill>`
               found  `std::option::Option<&hyxe_crypt::drill::Drill>`
note: but, the lifetime must be valid for the method call at 102:25...
   --> hyxe_user\tests\primary.rs:102:25
    |
102 | /                         cnac.async_borrow_drill(Some(51), async move |drill_opt| {
103 | |                             if let Some(drill) = drill_opt {
104 | |
                              println!("Borrowing drill vers: {}", drill.get_version());
105 | |                             }
106 | |                         }).await;
    | |__________________________^
note: ...so type `impl core::future::future::Future` of expression is valid during the expression
   --> hyxe_user\tests\primary.rs:102:25
    |
102 | /                         cnac.async_borrow_drill(Some(51), async move |drill_opt| {
103 | |                             if let Some(drill) = drill_opt {
104 | |                                 println!("Borrowing drill vers: {}", drill.get_version());
105 | |                             }
106 | |                         }).await;
    | |__________________________^
2

Do you spawn this on an executor that requires the task to be 'static? In that case I think you will need to take Option<&'static Drill>.

The future doesn't run immediately, so the reference must stay alive in there until it's done running.

One other way around this is to create the Drill itself inside an async block and spawn that, then the async block will be static, as long as references inside it only point to variables that live inside it. That's how you can have async fn on types that take &self, if the self lives inside the async block to begin with.

1 Like
3

When you write impl FnOnce(Option<&Drill>) -> F, you dealing with an elided for<'a> lifetime, however the type F is defined outside this for<'a>, so the future cannot depend on the lifetime. This means that you are not allowed to use the reference inside the returned future.

Also I notice that you're using unstable async closures. If you use ordinary closures returning async blocks instead, you could do this:

cnac.async_borrow_drill(Some(51), |drill_opt| {
     if let Some(drill) = drill_opt {
           println!("Borrowing drill vers: {}", drill.get_version());
     }
     async {}
}).await;

Notice how I use the reference before creating the returned future. This is fine: The issue is that the reference ends up being a field in the future generated by the async block, which is then returned from the closure.

Don't expect super good diagnostics from the compiler when using unstable features such as async closures.

3 Likes
4

I'm assuming I've hit something similar: Rust Playground
This is on stable and this error message is very hard for me to understand.

Note that for the closure I specify for<'x> F: Fn(&'x mut i32) -> FF + 'x
When you say "future cannot depend on the lifetime", do you mean that this "+ 'x" doesn't help because this is not at the future definition site, but at some "third party" location?

So is the limitation that async blocks can contain references to lifetimes known at the place of their definition, but not from anywhere else? I have a feeling I'm not understanding this correctly.

Is there some tracking issue about this or some feature that may alleviate this in the future?

5

The lifetime annotation doesn't help because your caller can't provide a reference with that lifetime.

One solution for this problem is using a smart pointer type with internal mutability, such as Rc<RefCell<T>> as in this example.

If you need to share these pointers across threads you can use Arc<Mutex<T>> instead.

6

A for<'x> clause can be rewritten informally as an infinite list of where clauses. Your example is not scoped quite as you think, and would be equivalent to this:

F: Fn(&'a mut i32) -> FF,
F: 'a,
F: Fn(&'b mut i32) -> FF,
F: 'b,
F: Fn(&'b mut i32) -> FF,
F: 'b,
...

It is a constraint on the choice of F, and doesn't talk about the return type. Generally the problem is that you actually want a different FF for each lifetime 'x — this comes from the fact that &'a u32 and &'b u32 are different types. Returning different types depending on the lifetimes like this is not really possible.

7

Your example is not scoped quite as you think

I suspected as much, thank you for clearing that up.

So I've been able to produce a working example with the future wrapped in a box: Rust Playground

Returning different types depending on the lifetimes like this is not really possible.

But I thought that for<> does exactly this? Indeed as I see it, this is what seems to be specified in a constraint in my functioning example.

So as far as I understand, this has been basically a syntactical issue? If so, that's weird, but I guess I understand what's happening here.

8

Minimal repro:

    pub
    async fn async_borrow_drill<F> (
        self: &'_ Self,
        f: impl FnOnce(&Drill) -> F,
    ) -> F::Output
    where
        F : Future,
    {
        let read = self.read().await; // RwLock Read handle
        f(&read.toolset.drill).await
    }
}

async
fn _main ()
{
    let cnac = Struct;
    cnac.async_borrow_drill(async move |drill| {
        drop(drill);
    }).await;
}

That signature is equivalent to the following one:

    pub
    async fn async_borrow_drill<C, F, Output> (
        self: &'_ Self,
        f: C,
    ) -> Output
    where
        for<'any>
            C : FnOnce(&'any Drill) -> F
        ,
        F : Future<Output = Output>,
    {
        let read = self.read().await; // RwLock Read handle
        f(&read.toolset.drill).await
    }

That is, no matter the lifetime 'any of the borrow over the Drill, the closure must always return the same future F, which outputs some value Output.

However,

async move |drill: &'_ Drill| {
    drop(drill);
}

i.e.,

fn closure<'any> (drill: &'any Drill)
  -> ?
{
    async move {
        drop(drill);
    }
}

returns a async move { /* uses drill */; } future:

  • the Output of the future is indeed always the same (() in my example),

  • but the future that outputs it captures drill, thus its type depends on that of &'any Drill, thus its type depends on 'any: the return type of that closure (i.e., for some fixed lifetime 'any): impl Future<Output = ()> + 'any. And for each different 'any, that type is different.

The bound that you would have liked to express would have been:

        for<'any>
            C : FnOnce(&'any Drill) -> (impl 'any + Future<Output = Output>)
        ,

which cannot really be written like that (with that sugar) yet, although by dropping a bit the sugar of Fn traits we can express, with nested trait bounds, that:

        for<'any>
            C : FnOnce<(&'any Drill,), Output : Future<Output = Output>>
        ,

and without nested trait bounds:

        for<'any>
            C : FnOnce<(&'any Drill,)>
        ,
        for<'any>
            <C as FnOnce<(&'any Drill,)>>::Output : Future<Output = Output>
        ,

which has the advantage of being a very explicit trait bound signature:

For any lifetime 'any, C must be callable with a single argument of type &'any Drill,
and (for each such lifetime), each return type (Output associated type of FnOnce) must at least be some future that resolves (Output associated type of Future) to Output / Ret (a generic param).

So the main difference with your code is that this definition allows the closure(s) C to return different types when fed different lifetimes, so as long as all these types are all Futures that resolve all to the same type.

So there is the solution: Playground

...

except that the trait solver gets confused with all these infinite lifetimes, so it incorrectly tells that |drill: &'_ Drill| async move { drop(drill) } does not meet the necessary bound. This is a bug in the compiler, a known one, that basically prevents us from doing this kind of advanced trait bound shenanigans. To "prove" there is a bug, let's look at this code that compiles fine:

pub
async fn async_borrow_drill<C, Output>(
    self: &'_ Self,
    f: C,
) -> Output
where
    for<'any>
        C : FnOnce<(&'any Drill,), Output = Pin<Box<dyn 'any + Future<Output = Output>>>>
    ,
    for<'any>
        Pin<Box<dyn 'any + Future<Output = Output>>> : Future<Output = Output>
    ,
    // for<'any>
    //     <C as FnOnce<(&'any Drill,)>>::Output : Future<Output = Output>
    // ,
{
    let read = self.read().await; // RwLock Read handle
    f(&read.toolset.drill).await
}

  • Playground

  • Basically I have removed the abstraction over the return type of the closure by picking the concrete type Pin<Box<dyn Future<Output = Output> + 'any>> to which our concrete future can coerce to after being Box::pinned, and which does fulfill all the requirements.

Now, if you go and uncomment that third constraint, which should be a corollary of the previous two, then the code no longer compiles. This is thus indeed a bug: QED.

2 Likes
9

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.