How to run async from sync code?

Hi, I'm new to rust and needed some help in clarifying why this issue happens.

When I run this code using cargo run, it runs fine. As in, it runs the main, which runs sync_fn, which runs async, which runs for 1s, prints "done" and returns.
However when I run it using cargo test -- tests::test_single_thread, the code blocks. What's the difference between using tokio::main vs. tokio::test.
cargo test -- tests::test_multi_thread flavor also runs fine.

Please correct me if I'm wrong, I thought the futures::executor::block_on will run the async_fn and once that's done return resume the rest of sync_fn. Thanks for the help.

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    sync_fn();
    Ok(())
}

fn sync_fn() {
    futures::executor::block_on(async_fn());
    println!("done")
}

async fn async_fn() {
    tokio::time::sleep(std::time::Duration::from_millis(1000)).await;
}

#[cfg(test)]
mod tests {
    use super::*;

    #[tokio::test]
    async fn test_default() {
        sync_fn();
    }

    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
    async fn test_multi_thread() {
        sync_fn();
    }
}

Tokio's functions expect to interact with Tokio's scheduler, but you're calling a tokio function from within another scheduler.

You're also blocking one of tokio's threads, which may cause tokio to not wake up.

1 Like

This is the problem, but also, the actual situation is a bit narrower than “one of” suggests.

tokio::main expands to a call to Runtime::block_on(), which does not make use of the Tokio thread pool, but polls the future on the current thread — you can tell this must be true because its bound on the future is F: Future, not F: Future + Send, so Tokio is not permitted to move the future to another thread. So, the blocking happens in the main thread, which is not part of the Tokio thread pool if it exists. If you are using the multi-thread runtime, then the Tokio thread pool is not affected at all by what you do inside block_on()s or tokio::main.

The problem here is that with the current-thread runtime (which is used by tokio::test by default), there are no other threads, which means the timer driver has to be run on the current thread:

When the current thread scheduler is enabled block_on can be called concurrently from multiple threads. The first call will take ownership of the io and timer drivers. This means other threads which do not own the drivers will hook into that one.

Which means that, under this configuration, if anything blocks without using Tokio to do it (as futures::executor::block_on() does) then the timer driver cannot run since the thread that claimed it is busy with something else — therefore the timer driver code isn't waking up the sleep future, resulting in a deadlock: the inner block_on() is waiting for a wake that the timer driver will perform, but the timer driver won't get any chance to run until the inner block_on() completes.

So, it's not that you're blocking a thread that belongs to Tokio; it's that you're blocking the only thread that you've loaned to Tokio.

If you're using only Tokio's block_on(), then Tokio will either make sure it can run properly, or give you an error telling you it can't. But when you use another executor, you're hiding the blocking from Tokio, so it can't check.

5 Likes