GAT polyfill + async closures; or current workarounds for generator lifetime erasure

help
1

I'd like to implement the following function, and then have the examples compile:

use std::marker::PhantomData;
use std::future::Future;

struct Invariant<'i>(PhantomData<fn(&'i ()) -> &'i ()>);

fn scoped_async_closure<'i, C, T>(
    inv: Invariant<'i>,
    c: C
) -> T
where
    C: for<'a> FnOnce(Invariant<'i>, &'a u8) -> impl Future<Output = T>
{
    let local = 0_u8;
    let fut = c(inv, &local);
    let t = futures::executor::block_on(fut);
    t
}

/////// Examples that should compile

fn example_one<'i>(inv: Invariant<'i>) {
    let c = |_inv, loc: &u8| {
        let loc = loc;
        let fut = async move {
            std::future::ready(()).await;
            *loc + 1
        };
        fut
    };
    let res = scoped_async_closure(inv, c);
    assert_eq!(res, 1);
}

fn example_two<'i>(inv: Invariant<'i>) -> Invariant<'i> {
    let c = |inv, loc: &u8| async move {
        std::future::ready(()).await;
        (inv, *loc + 1)
    };
    let res = scoped_async_closure(inv, c);
    assert_eq!(res.1, 1);
    res.0
}

Obviously the code above doesn't compile (some of the syntax isn't even real). The intent is just to demonstrate what the API should look like. I should also point out that I'm not sure whether all of the components are important; I don't know whether it matters that T isn't u8 or that Invariant is actually invariant. I'm mostly trying not to XY problem myself out of what is already a minimized example.

Getting this to compile won't be easy - any solution that involves making the future type appear as a generic on scoped_async_closure won't work since then it won't be able to name the 'a lifetime. Instead, I couldn't figure out anything short of reaching for the full GAT polyfill, with some extra #![feature(unboxed_closures)] thrown in (because I forget if there was a stable polyfill for naming closure return types in an associated-type way).

This eventually leaves us with this. That's pretty good, but we still get compiler errors on the two examples:

error: lifetime may not live long enough
  --> src/lib.rs:79:9
   |
74 |     let c = |_inv, loc: &u8| {
   |                         -  - return type of closure `[async block@src/lib.rs:76:19: 78:10]` contains a lifetime `'2`
   |                         |
   |                         let's call the lifetime of this reference `'1`
...
79 |         fut
   |         ^^^ returning this value requires that `'1` must outlive `'2`

I couldn't make heads or tails of this for a while, until I realized that it's probably the result of the semi-well-known problem with overeager lifetime erasure in generator bodies.

Is there a way to fix this?

I'm fine with unstable features as long as they aren't like... GCE

2

I think @jbe recently posted a way to make it work with type_alias_impl_trait. I’ll have to search that conversation. You had to use the workaround at the call site, so the call site becomes a bit more ugly than what you’re asking for.

Edit1: Here’s the relevant topic. I’ll try to read through your question and code in detail to check if this actually is about the same kind of problem.

Edit2: Yeah, looks like it matches the problem. I’ll first note that the typical practical solution is to require the returned closure type to be BoxFuture. Which works on stable, too. Last time I’ve explained that kind of workaround was last week ^^ I’ll have to check if the multiple lifetimes pose any issues though.

Edit3: Here’s the BoxFuture workaround on the concrete code example

use futures::future::BoxFuture;
use futures::FutureExt;
use std::future::Future;
use std::marker::PhantomData;

struct Invariant<'i>(PhantomData<fn(&'i ()) -> &'i ()>);

fn scoped_async_closure<'i, C, T>(inv: Invariant<'i>, c: C) -> T
where
    C: for<'a> FnOnce(Invariant<'i>, &'a u8, PhantomData<&'a &'i ()>) -> BoxFuture<'a, T>,
{
    let local = 0_u8;
    let fut = c(inv, &local, PhantomData);
    let t = futures::executor::block_on(fut);
    t
}

/////// Examples that should compile

fn example_one<'i>(inv: Invariant<'i>) {
    let res = scoped_async_closure(inv, |_inv, loc, _| {
        let loc = loc;
        let fut = async move {
            std::future::ready(()).await;
            *loc + 1
        }
        .boxed();
        fut
    });
    assert_eq!(res, 1);
}

fn example_two<'i>(inv: Invariant<'i>) -> Invariant<'i> {
    let res = scoped_async_closure(inv, |inv, loc, _| {
        async move {
            std::future::ready(()).await;
            (inv, *loc + 1)
        }
        .boxed()
    });
    assert_eq!(res.1, 1);
    res.0
}

Rust Playground

I’ve had to add a PhantomData<&'a &'i ()> argument for an implied 'i: 'a bound, so there’s a single parameter that can be used for the returned BoxFuture. This should be no problematic restriction for the code at hand, though there should also be alternative approaches one could have used, which might translate better in the general case.

Note that an additional .boxed() call is necessary at the call-site. Also, as similarly discussed in the topic from last week I’ve already linked, type inference very much wants the closure expression directly inline with the call.

2 Likes
3

@steffahn thanks for the link to the other thread. I'm going to have to take some time to read through that.

I'm aware of the BoxFuture alternative - unfortunately the perf hit is probably unacceptable in my case. There are other things I could do too (ie I could require the future to be 'static at the cost of some clones) - this is mostly an attempt to avoid having to do that

4

Ah, okay, if that’s the case, I’ll see and try to translate @jbe’s approach to your example.

5

Alright… this stuff isn’t easy to use, but it does compile:

#![feature(closure_lifetime_binder)]
#![feature(type_alias_impl_trait)]

// using https://docs.rs/async_fn_traits for convenience
// the crate’s code is copied into the playground version of this code
extern crate async_fn_traits;

// helper trait for listing (possible many) lifetimes captured by impl Trait
trait CptrLt<'a> {}
impl<'a, T: ?Sized> CptrLt<'a> for T {}

use async_fn_traits::AsyncFnOnce2;
use std::future::Future;
use std::marker::PhantomData;

struct Invariant<'i>(PhantomData<fn(&'i ()) -> &'i ()>);

fn scoped_async_closure<'i, C, T>(inv: Invariant<'i>, c: C) -> T
where
    C: for<'a> AsyncFnOnce2<Invariant<'i>, &'a u8, Output = T>,
{
    let local = 0_u8;
    let fut = c(inv, &local);
    let t = futures::executor::block_on(fut);
    t
}

/////// Examples that should compile

fn example_one<'i>(inv: Invariant<'i>) {
    type Fut<'a> = impl Future<Output = u8> + CptrLt<'a>;

    let c = for<'a> |_inv: Invariant<'i>, loc: &'a u8| -> Fut<'a> {
        let loc = loc;
        let fut = async move {
            std::future::ready(()).await;
            *loc + 1
        };
        fut
    };
    let res = scoped_async_closure(inv, c);
    assert_eq!(res, 1);
}

fn example_two<'i>(inv: Invariant<'i>) -> Invariant<'i> {
    type Fut<'i, 'a> = impl Future<Output = (Invariant<'i>, u8)> + CptrLt<'i> + CptrLt<'a>;

    let c = for<'a> |inv: Invariant<'i>, loc: &'a u8| -> Fut<'i, 'a> {
        async move {
            std::future::ready(()).await;
            (inv, *loc + 1)
        }
    };
    let res = scoped_async_closure(inv, c);
    assert_eq!(res.1, 1);
    res.0
}

Rust Playground

Regarding difficulty of use: One particularly tricky aspect (besides sub-par error messages on the unstable features being used) is that you have to explicitly list what generic parameters are and aren’t captued by the future. For example the example1 has the async block not interact with 'i, only with 'a, and type_alias_impl_trait thus requires us to only make 'a an argument of the Fut type, accordingly.

Edit: One thing I forgot to mention: For non-capturing closures, you don’t need the workarounds using nightly features, and a HRTB trait bounds involving e.g. a AsyncFnOnce2 bound can be met a lot easier by passing an async fn instead of a closure. It’s maybe not relevant for your true use case, but for these toy examples where the closures do in fact capture nothing, the more complicated approach using type_alias_impl_trait and closure_lifetime_binder is overkill and one could simply do

// using https://docs.rs/async_fn_traits for convenience
// the crate’s code is copied into the playground version of this code
extern crate async_fn_traits;

use async_fn_traits::AsyncFnOnce2;
use std::future::Future;
use std::marker::PhantomData;

struct Invariant<'i>(PhantomData<fn(&'i ()) -> &'i ()>);

fn scoped_async_closure<'i, C, T>(inv: Invariant<'i>, c: C) -> T
where
    C: for<'a> AsyncFnOnce2<Invariant<'i>, &'a u8, Output = T>,
{
    let local = 0_u8;
    let fut = c(inv, &local);
    let t = futures::executor::block_on(fut);
    t
}

/////// Examples that should compile

fn example_one<'i>(inv: Invariant<'i>) {
    async fn c<'i, 'a> (_inv: Invariant<'i>, loc: &'a u8) -> u8 {
        std::future::ready(()).await;
        *loc + 1
    }
    let res = scoped_async_closure(inv, c);
    assert_eq!(res, 1);
}

fn example_two<'i>(inv: Invariant<'i>) -> Invariant<'i> {
    async fn c<'i, 'a>(inv: Invariant<'i>, loc: &'a u8) -> (Invariant<'i>, u8){
        std::future::ready(()).await;
            (inv, *loc + 1)
    }
    let res = scoped_async_closure(inv, c);
    assert_eq!(res.1, 1);
    res.0
}

Rust Playground

1 Like