Implementation of `FnOnce` is not general enough

help
1

I need to select a TCP connection that is writable.
I wrote a seemingly innocent async function using futures buffer_unordered and to my surprise it compiled the first time. But when I actually tried to call it from underneath a tokio::spawn, the compiler didn't like it.

I paste a minimized example:

        tokio::spawn(async move {
            let mirrors: Vec<OwnedWriteHalf> = Vec::new();
            futures::stream::iter(&mirrors)
                .map(|s| s.writable())
                .buffer_unordered(mirrors.len())
                .next()
                .await;
        });

Error message:

error: implementation of `FnOnce` is not general enough
   --> src/lib.rs:305:9
    |
305 |         tokio::spawn(async move {
    |         ^^^^^^^^^^^^ implementation of `FnOnce` is not general enough
    |
    = note: closure with signature `fn(&'0 tokio::net::tcp::OwnedWriteHalf) -> impl futures::Future<Output = std::result::Result<(), std::io::Error>>` must implement `FnOnce<(&tokio::net::tcp::OwnedWriteHalf,)>`, for any lifetime `'0`...
    = note: ...but it actually implements `FnOnce<(&tokio::net::tcp::OwnedWriteHalf,)>`

The error disappears when I remove .await.
Or it disappears if I remove buffer_unordered but call .await on the first item in the vector (I mean mirrors[0].writable().await), even though ti doesn't make much sense on an empty vector.

2

Weirdly, after switching to a more manual use of FuturesUnordered, it compiles:

    let mut futures = FuturesUnordered::new();
    for s in mirrors.iter() {
        futures.push(s.writable())
    }
    futures.next().await

But why?

3

Explanation

So, this is a known limitation / bug in the compiler, see:

That last issue has nicer reductions from @steffahn about the necessary ingredients to trigger this error: Problem with GATs, async, and Send-bounds · Issue #90696 · rust-lang/rust · GitHub

Finally, there was a tentative PR in the compiler to fix it, but which did not pan out; it does nonetheless has a lot of valuable (technical) information about why this issue occurs:

In your case, the lifetime-infected associated type comes from:

|s| s.writable()

this is a closure taking a &'mirrors OwnedHalf input (through inference), which has a FnOnce::Output associated type which is a Future with, itself, an Output associated type to resolve to the Result<()> that the async fn writable() resolves to.

I don't know which of those two layers of associated types (maybe both?) happen to be the offending one, but at least one of those two associated types is required to be Send, in order for the whole async move { … } block to be (required by tokio::spawn()).

And (at least one of) this associated type crosses an .await point, triggering the compiler bug, wherein it doesn't want to have a : Send bound on an associated type that involves a non-'static lifetime parameter (in this case, the 'mirrors borrow).

It turns out, that if the closure were changed to have a higher-order signature, like this:

- impl FnMut(&'mirrors OwnedWriteHalf) -> …
+ impl for<'any> FnMut(&'any OwnedWriteHalf) -> …
// i.e.
+ impl FnMut(&'_ OwnedWriteHalf) -> …

then that fixed lifetime parameter infecting associated types would be no more, and so the compiler would be able to nicely resolve the required : Send bound.

Hence why it complained about the lack of such an impl.

Note that on more recent compiler, the errors become a bit more terse, pointing to the root cause (which is great!) but no longer pointing to the code that could be changed to tackle it (which is not great!).

It has gone from:

error: implementation of `FnOnce` is not general enough
  --> src/main.rs:54:28
   |
54 |         let _: &dyn Send = &fut;
   |                            ^^^^ implementation of `FnOnce` is not general enough
   |
   = note: closure with signature `fn(&'0 tokio::net::tcp::OwnedWriteHalf) -> impl futures::Future<Output = Result<(), std::io::Error>>` must implement `FnOnce<(&tokio::net::tcp::OwnedWriteHalf,)>`, for any lifetime `'0`...
   = note: ...but it actually implements `FnOnce<(&tokio::net::tcp::OwnedWriteHalf,)>`

to:

error: higher-ranked lifetime error
  --> src/main.rs:54:28
   |
54 |         let _: &dyn Send = &fut;
   |                            ^^^^
   |
   = note: could not prove `for<'r, 's, 't0, 't1> &'r impl for<'s> futures::Future<Output = ()>: CoerceUnsized<&'t0 (dyn std::marker::Send + 't1)>`
4

Workarounds

Unrolling like to did seems indeed to dodge the issue; another option would have been to actually make the closure become higher-order.

The latter can be achieved, in some cases, by naming the async fn itself rather than wrapped in a closure:

- .map(|s| s.writable())
+ .map(OwnedWriteHalf::writable)

but this is almost never usable in practice (needs the exact right level of receiver indirection, and 0-captures whatsoever).

The more generally-applicable approach is to annotate the lifetime placeholder in the closure input type, instead of letting type inference figure it out:

- .map(|s| s.writable())
+ .map(|s: &_| s.writable())

I'm personally not super-pleased with these solutions because requiring higher-order closures is strictly more limiting than a fixed-lifetime one, so it means there will be cases where this workaround will break because of other constraints. But if it can get 90% of the cases unstuck, then that's at least something, right?