Why need `Send` when immutably borrow `T` in the async block?

[tytyty]

Hi everyone!

I tried to borrow T immutably in async block, but the Rust compiler tells me to add Send trait to T.

In the following code, test_works1 and test_works2 work fine, but test_not_work1 and test_not_work2 are not. I don’t understand.

Code

use core::future::Future;
use core::pin::Pin;

type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + 'a + Send>>;

trait Inner {
    fn inner_test<'a>(&'a self) -> BoxFuture<'a, ()>;
}

struct Foo<T: Inner>(T);

impl<T: Inner> Foo<T> {
    async fn test(&self) {
        self.0.inner_test().await
    }
}

// it works
fn test_works1<'a, T: Inner + Sync>(foo_ref: &'a Foo<T>) -> BoxFuture<'a, ()> {
    let foo_inner_ref = &foo_ref.0;
    Box::pin(async move { foo_inner_ref.inner_test().await })
}

fn test_not_work1<'a, T: Inner + Sync>(foo_ref: &'a Foo<T>) -> BoxFuture<'a, ()> {
    Box::pin(async { foo_ref.0.inner_test().await })
}

fn test_not_work2<'a, T: Inner + Sync>(foo_ref: &'a Foo<T>) -> BoxFuture<'a, ()> {
    Box::pin(async { foo_ref.test().await })
}

// it works
async fn test_works2<T: Inner + Sync>(foo_ref: &Foo<T>) {
    foo_ref.test().await;
}

fn main() {}

Error

error: future cannot be sent between threads safely
  --> src/main.rs:25:5
   |
25 |     Box::pin(async { foo_ref.0.inner_test().await })
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ future created by async block is not `Send`
   |
note: future is not `Send` as this value is used across an await
  --> src/main.rs:25:22
   |
25 |     Box::pin(async { foo_ref.0.inner_test().await })
   |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ first, await occurs here, with `foo_ref.0` maybe used later...
note: `foo_ref.0` is later dropped here
  --> src/main.rs:25:51
   |
25 |     Box::pin(async { foo_ref.0.inner_test().await })
   |                      ---------                    ^
   |                      |
   |                      has type `T` which is not `Send`
help: consider moving this into a `let` binding to create a shorter lived borrow
  --> src/main.rs:25:22
   |
25 |     Box::pin(async { foo_ref.0.inner_test().await })
   |                      ^^^^^^^^^^^^^^^^^^^^^^
   = note: required for the cast to the object type `dyn Future<Output = ()> + Send`
help: consider further restricting this bound
   |
24 | fn test_not_work1<'a, T: Inner + Sync + Send>(foo_ref: &'a Foo<T>) -> BoxFuture<'a, ()> {
   |                                       ^^^^^^

error: future cannot be sent between threads safely
  --> src/main.rs:29:5
   |
29 |     Box::pin(async { foo_ref.test().await })
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ future created by async block is not `Send`
   |
note: future is not `Send` as this value is used across an await
  --> src/main.rs:14:9
   |
14 |         self.0.inner_test().await
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^ first, await occurs here, with `self.0` maybe used later...
note: `self.0` is later dropped here
  --> src/main.rs:15:5
   |
14 |         self.0.inner_test().await
   |         ------ has type `T` which is not `Send`
15 |     }
   |     ^
help: consider moving this into a `let` binding to create a shorter lived borrow
  --> src/main.rs:14:9
   |
14 |         self.0.inner_test().await
   |         ^^^^^^^^^^^^^^^^^^^
   = note: required for the cast to the object type `dyn Future<Output = ()> + Send`
help: consider further restricting this bound
   |
28 | fn test_not_work2<'a, T: Inner + Sync + Send>(foo_ref: &'a Foo<T>) -> BoxFuture<'a, ()> {
   |                                       ^^^^^^

error: aborting due to 2 previous errors

error: could not compile `playground`

playground

[Yandros]

This looks like a bug in the code generated by the async machinery, more precisely, in the heuristics used to guess what gets captured. Indeed, consider the following minimal repro:

fn for_<T : Sync> (it: &(T, ))
  -> impl '_ + Send
{
    async move {
        let _ = &it.0; // <- this somehow makes Rust believe a `T` is captured.
        async {}.await; // <- await point
    }
}

which yields:

error: future cannot be sent between threads safely
 --> src/lib.rs:2:6
  |
2 |   -> impl '_ + Send
  |      ^^^^^^^^^^^^^^ future created by async block is not `Send`
  |
note: future is not `Send` as this value is used across an await
 --> src/lib.rs:6:9
  |
5 |         let _ = &it.0; // <- this somehow makes Rust believe a `T` is captured.
  |                  ---- has type `T` which is not `Send`
6 |         async {}.await; // <- await point
  |         ^^^^^^^^^^^^^^ await occurs here, with `it.0` maybe used later
7 |     }
  |     - `it.0` is later dropped here

The annotations are incorrect, more precisely:

- `it.0` is later dropped here

is plain wrong.

So, indeed, by preemptively binding some of these references to specific vars, you may dodge the buggy borrow inside the async block and thus circumvent the bug

Reliable workaround

So, the bug is triggered by mentioning a &struct.field place and then having an .await point afterwards. Thus, given how silly this bug is, the trick to reliably circumvent it is to somehow manage to borrow that inner field without "mentioning" it, as a whole, on the right-hand-side of an assignement / as part of an expression.

And is that possible to do? Well yes, through patterns!

impl<T: Inner> Foo<T> {
    async fn test(&self) {
-       self.0.inner_test().await
// Same as:
-       let inner = &self.0;
-       inner.inner_test().await;
// Instead, do:
+       let &Self { 0: ref inner, .. } = self;
+       inner.inner_test().await
    }
}

similarly:

fn test_not_work1<'a, T: Inner + Sync>(foo_ref: &'a Foo<T>) -> BoxFuture<'a, ()> {
    Box::pin(async move {
-       foo_ref.0.inner_test().await;
+       let &Foo { 0: ref inner, .. } = foo_ref; // let inner = &foo_ref.0;
+       inner.inner_test().await;
    })
}

• Playground

---

EDIT

Feel free to check out Generator size: borrowed variables are assumed live across following yield points · Issue #59087 · rust-lang/rust · GitHub (as well as Generators are too big · Issue #52924 · rust-lang/rust · GitHub and Dropped variables still included in generator type · Issue #57478 · rust-lang/rust · GitHub), which are several issues related to async sugar incorrectly capturing too much stuff across .await points.

While your annoying situation is not directly mentioned there, it does lead us to a very common pattern that all async-seasoned Rustaceans know about (for better or for worse ):

async {
    let guard = ….lock();
    non_async_stuff(&mut *guard);
    drop(guard);
    // an await point _after_ the drop:
    async {}.await
    /* Rust considers that `guard` has crossed the `.await` point */
}

The workaround for this bug is easy (albeit annoying): scopes do successfully manage to contain variables, locals, etc. so that the enclosing brace cannot be crossed.

Such a workaround also applies to the OP's issue:

-       let inner = &self.0;
+       let inner; { inner = &self.0; }
        inner.inner_test().await;

[Yandros]

From the last workaround, the following macro could come in handy:

macro_rules! scoped {( $value:expr $(,)? ) => (
    { let it = $value; it }
)}

With it:

impl<T: Inner> Foo<T> {
    async fn test(&self) {
-       self.0.inner_test().await
+       scoped!(&self.0).inner_test().await
    }
}

as well as:

fn test_not_work1<'a, T: Inner + Sync>(foo_ref: &'a Foo<T>) -> BoxFuture<'a, ()> {
    Box::pin(async move {
-       foo_ref.0.inner_test().await;
+       scoped!(&foo_ref.0).inner_test().await;
    })
}

• Playground

[H2CO3]

I wonder if Polonius would address the limitation whereby the dropped object is still considered alive? That seems like a relatively low-hanging fruit, at least in such a trivial case of flow-insensitive, unconditional dropping.