How to teach the compiler that AsyncFnMut::CallRefFuture is Send for all lifetimes up to an upper bound?

help
1

Here is a trait for something that can be fed a sequence of elements of type E, doing async work for each that works fine in a multi-threaded polling environment, and an implementation for thread-safe async closures that accept E:

/// An object that can be called serially with elements of type E, where each
/// element may require some async processing in a multi-threaded environment.
trait Acceptor<E> {
    fn accept(&mut self, element: E) -> impl Future<Output = ()> + Send;
}

// We can implement this for async functions of the appropriate signature.
impl<E, F> Acceptor<E> for F
where
    F: AsyncFnMut(E),
    for<'a> F::CallRefFuture<'a>: Send,
{
    fn accept(&mut self, element: E) -> impl Future<Output = ()> + Send {
        self(element)
    }
}

This works fine for async closures that are 'static:

async fn use_static() {
    use_acceptor(async |e: i32| {}).await;
}

However, not with ones that do capture references (playground):

async fn use_non_static() {
    {
        let mut elements: Vec<i32> = Vec::new();
        use_acceptor(async |e: i32| {
            elements.push(e);
        })
        .await;
    }
}

error[E0597]: `elements` does not live long enough
  --> src/lib.rs:34:18
   |
34 |        use_acceptor(async |e: i32| {
   |   _____-            ^
   |  |__________________|
35 | ||         elements.push(e);
36 | ||     })
   | ||_____^- argument requires that `elements` is borrowed for `'static`
   | |______|
   |        borrowed value does not live long enough
37 |        .await;
38 |    }
   |    - `elements` dropped here while still borrowed
   |
note: due to a current limitation of the type system, this implies a `'static` lifetime
  --> src/lib.rs:16:5
   |
16 |     for<'a> F::CallRefFuture<'a>: Send,
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

How do I make this work?

The error is similar to the one discussed in Rust issue 153558, where eggyal@ explained that the HRTB quantifier goes all the way up to 'static. But the example there is too simplistic and the workaround of not using an HRTB doesn't directly apply here because there is otherwise no lifetime to use for the Send bound for the future on the trait impl. I think what I need is something like the following to bound the quantifier:

  • A lifetime parameter 'a on Acceptor, and another parameter 'b on accept with 'a: 'b and the callee accepted as &'b self.

  • A bound on the trait impl like for<'b where 'a: 'b> F::CallRefFuture<'b>: Send

But the latter is not valid syntax, and I can't figure out a way to teach the "upper bound" for the quantifier to the compiler. I tried arranging this using a struct that takes two lifetime parameters and has the bound between them like std::thread::Scope does, but I couldn't find a way to make it actually work where the compiler can still understand that the future returned by F is Send when fed an appropriate lifetime.

2

Can you add a lifetime parameter to the Acceptor trait? Playground:

trait Acceptor<'a, E> {
    fn accept(&'a mut self, element: E) -> impl Future<Output = ()> + Send;
}

impl<'a, E, F> Acceptor<'a, E> for F
where
    F: 'a + AsyncFnMut(E),
    F::CallRefFuture<'a>: Send,
{
    fn accept(&'a mut self, element: E) -> impl Future<Output = ()> + Send {
        self(element)
    }
}
3

Thanks. That's similar to what I tried, but I fear I simplified my example too much. The problem comes in when I want to accept the impl Acceptor by reference (so that it can be used again afterward), and feed it multiple elements. Take this version of the function that uses it (playground):

async fn use_i32_acceptor<'a>(acceptor: &'a mut impl Acceptor<'a, i32>) {
    acceptor.accept(0).await;
    acceptor.accept(1).await;
}

This fails, presumably because the two lifetimes are the same:

error[E0499]: cannot borrow `*acceptor` as mutable more than once at a time
  --> src/lib.rs:26:3
   |
24 | async fn use_i32_acceptor<'a>(acceptor: &'a mut impl Acceptor<'a, i32>) {
   |                           -- lifetime `'a` defined here
25 |   acceptor.accept(0).await;
   |   ------------------
   |   |
   |   first mutable borrow occurs here
   |   argument requires that `*acceptor` is borrowed for `'a`
26 |   acceptor.accept(1).await;
   |   ^^^^^^^^ second mutable borrow occurs here

As a result I think I need accept to take another type parameter 'b that is smaller than the type parameter 'a in Acceptor<'a>, but I can't figure out how to make that work while navigating the compiler's limitations on HRTB proofs.

4

This is the go-to article on that topic:

Unfortunately, I don't know if it's a solution in this case. I believe the problem is that you're wrapping a trait that isn't you're own (AsyncFnMut) which still has the explicit bound:

    type CallRefFuture<'a>: Future<Output = Self::Output>
       where Self: 'a;

Sorry to say, that's as far as I got.

1 Like
5

And RTN, whose motivation is "the Send bound problem", apparently doesn't work around the limitation. So I guess this is "just" fallout from AsyncFnMut being based on a GAT.

6

Thank you, that article is very good. I also meant to try RTN, so thanks for saving me the effort.

It sounds like there is a gap here, whether in the language or in the definition of the AsyncFnMut trait. Is there any hope of it being fixed? Does it make sense to file a bug?

7

There's ongoing work on fixing related stuff: Assumptions on Binders - Rust Project Goals

Not sure whether it would fix this specific case though.