Async function returning `+ Send`: compile error using `match` and `tokio::time::sleep`, but `if let` works

help
1

Hello everyone,

I've run into a puzzling compilation error when implementing a trait method which returns impl Future<Output = Self> + Send. If the implementation of this trait method calls a function, that returns Result<_, Box<dyn std::error::Error>>, and handles both cases of the its result, using tokio::time::sleep in the error branch causes a compilation error. And if this wouldn't be strange enough on its own, there is a difference between using match and if let ... which affects the error as well.

Below a few code snippets illustrate the problem I just tried to describe:
The basic setup is a function which returns a boxed error and a trait containing an async function with the Send trait bound on its return type:

/// A function which returns a `Box<dyn std::error::Error>`.
async fn test() -> Result<(), Box<dyn std::error::Error>> {
    Ok(())
}

/// A trait with a method, which has a `Send` trait bound on its return type.
/// Removing this `Send` trait bound, makes all examples below compile, but
/// in my use case the trait is defined in another crate.
trait MyTrait {
    fn my_func() -> impl Future<Output = Self> + Send;
}

Trying to implement this trait like shown below, causes a compilation error:

struct ThisFails;

impl MyTrait for ThisFails {
    async fn my_func() -> Self {
        // Call `test()` function and work with its result...
        match test().await {
            Ok(_) => {},
            Err(_) => {
                // commenting this out resolves the compile error
                tokio::time::sleep(std::time::Duration::from_secs(1)).await;
            },
        }

        todo!()
    }
}

As the comment states, removing tokio::time::sleep makes the code compile.

Using the if let ... syntax instead of a match on the other hand, works fine and the code compiles:

struct ThisWorks;

impl MyTrait for ThisWorks {
    async fn my_func() -> Self {
        // Call `test()` function and work with its result...
        if let Ok(_) = test().await {

        } else {
            tokio::time::sleep(std::time::Duration::from_secs(1)).await;
        }

        todo!()
    }
}

But switching the order of the if let ... branches, causes the compile error to occur again:

struct ThisFails2;

impl MyTrait for ThisFails2 {
    async fn my_func() -> Self {
        // Call `test()` function and work with its result...
        if let Err(_) = test().await {
            // commenting this out resolves the compile error
            tokio::time::sleep(std::time::Duration::from_secs(1)).await;

        } else {

        }

        todo!()
    }
}

In all cases the error text says:

error: future cannot be sent between threads safely
  --> examples/weird-error/main.rs:34:5
   |
34 |     async fn my_func() -> Self {
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^ future returned by `my_func` is not `Send`
   |
   = help: the trait `Send` is not implemented for `dyn std::error::Error`
note: future is not `Send` as this value is used across an await
  --> examples/weird-error/main.rs:38:67
   |
36 |         if let Err(_) = test().await {
   |                         ------------ has type `Result<(), Box<dyn std::error::Error>>` which is not `Send`
37 |             // commenting this out resolves the compile error
38 |             tokio::time::sleep(std::time::Duration::from_secs(1)).await;
   |                                                                   ^^^^^ await occurs here, with `test().await` maybe used later
note: required by a bound in `MyTrait::{synthetic#0}`
  --> examples/weird-error/main.rs:10:50
   |
10 |     fn my_func() -> impl Future<Output = Self> + Send;
   |                                                  ^^^^ required by this bound in `MyTrait::{synthetic#0}`

I've also prepared a Rust Playground for everyone who wants to run the code and see the error in action :smiley:

My questions about this issue are:

  1. What does tokio::time::sleep do to make the Future returned by the my_func implementation not to be Sync anymore? Is this special to the tokio::time::sleep function or can other function calls cause the same behavior?
  2. Is there a fundamental difference between match and if let which causes them to have different effects in this case? If so, how does this explain their differences in combination with tokio::time::sleep?

Thank you very much in advance for your help! I'm really looking forward to hearing what you'll say about this problem :slight_smile:

1 Like
2

I just tried running this on my laptop using RustRover which defaulted to Rust Edition 2021, and got the compiler error even with the struct named ThisWorks.

However switching to Rust Edition 2024 lets you compile the ThisWorks struct. Very interesting

1 Like
3

It's not the addition of sleep() but rather the .await. As the error says: “future is not Send as this value is used across an await”. No await, no problem. (The “use” of the value, in this case, is just dropping it.)

Practically, the thing you should do to avoid having this problem is replace the type Box<dyn std::error::Error> with Box<dyn std::error::Error + Send + Sync>, so that there is not a non-Send value any more. Most error values are Send + Sync, and the ones that aren't are ones that you probably shouldn't be passing upward anyway.[1]

The match expression keeps its scrutinee value (the value of x in match x {}) alive for the entire match, including your sleep().await, regardless of whether any part of the value is bound by the pattern. The if let expression, instead, drops the value before running the else block, since it is definitely not needed in that case.

(That behavior is new in the 2024 edition — in previous editions, if let worked just like a match with two arms.)

  1. Such as PoisonError. ↩︎

3 Likes
4

Thanks to both of you for your answers! :+1: Now I finally understand what happens "under the hood".

To summarize it in my own words: the async executor "yields" on each .await, suspends the future and resumes it later. Only variables of types which implement Send + Sync can be used across .await points, because the async executor may need to move those variables across threads. All of this explains why any call of .await in my example is causes the compilation error (not only calling tokio::time::sleep(...).await).

Getting back to my example: the "variable" which is not Send + Sync is the return value of the test() function which is of type Result<(), Box<dyn std::error::Error>> (because std::error::Error is not Send + Sync). Matching on this value the compiler "sees", that the value is not dropped until the end of the whole match block and that one match arm contains a .await point. Therefore the the value matched on must implement Send + Sync, which it does not and therefore the compiler returns the error. Due to the changes of if let in the 2024 Rust edition, it behaves differently because the value is dropped before the else branch, which contains the .await, which means that the return value of the test() function must not be able to "survive" the .await anymore.

In my actual use case the function returning the non Send + Sync value is implemented in an external crate, which is why I can only wrap it inside a function, which returns a Send + Sync value (for example by using a thiserror::Error). I implemented a small example of this in another Rust Playground.

Again many thanks for your help. I really appreciate it!
Happy sunday :victory_hand:

5

It sounds like you generally understand enough to proceed with. Here are some pedantic details which may be helpful for a fuller understanding:

This is not quite correct. Let's first define “suspend” precisely: it means that the future returns Poll::Pending from Future::poll(), returning control to the executor. (Executors are not able to cause a future to suspend.) And “yield” means to suspend on purpose for no reason other than to let other futures run, as opposed to waiting for an event.

Every await point is a potential suspension point. What x.await actually means is “when control reaches this point, use the poll() behavior of the future x as this future’s poll() behavior, until x.poll() returns Poll::Ready”. Thus, x.await only suspends if the future x suspends when it is run.

This is an important distinction, because it means that a future may accidentally have no actual suspensions despite having plenty of awaits. For example, in this code, foo() is a future which will hang when polled, because none of the code it runs ever suspends and so nothing else can get a chance to modify stop:

async fn foo(stop: &Cell<bool>) {
    loop {
        if bar(stop).await {
            break;
        }
    }
}
async fn bar(stop: &Cell<bool>) -> bool {
    // (perhaps do some actual computation too, if this were real code)
    stop.get()
}

This code is incorrect, and needs to be fixed by adding a yield in the loop to ensure it suspends often enough.

The async executor does not see the variables at all. It just sees the Future it was given, and if that Future is created from an async block, then the future implements Send if and only if all of the variables it holds across awaits (or was created with) implement Send.

Sync is generally not relevant to futures because the only thing you do with a future is poll it, and polling is done with exclusive (&mut) access, so futures never need to be Sync. The only way T: Sync matters is that it determines whether &T is Send, so a future that holds a &T will fail to be Send if T is not Sync.

There are also async executors that do not require Send; for example, any block_on() function will not (including Tokio’s!) because it means “block this thread and use it for running this future” as opposed to sending the future to a thread pool.

Translating the error to a new type that is Send is a good strategy. You should also consider sending a patch to the external crate to make its error + Send + Sync; it’s rare that the omission is purposeful and necessary.

1 Like