Tokio::spawn_blocking: calling sync operation from async and awaiting still blocks the thread?

help
1

Rust beginner here, venturing in async with tokio.
I have an async UDP server which needs to perform a sync, kind-of-cpu-intensive, operation upon receiving a packet (i.e. around 10ms of processing).
I know spawn_blocking is not well-suited for cpu-bound loads, but in my case I don't expect to run out of threads in the thread-pool and, most importantly, I'm seeing the behaviour I'm going to describe also when using rayon as suggested here.
Besides, my question is mostly to understand better how to correctly use tokio, so here is the structure of my code

#[tokio::main]
async fn main() -> ...
{
  let server = Server::new(...)
  // setup socket and server, then

  server.run().await?
}

impl Server
{
  async fn run(self) -> ...
  {
     loop
     {
        let (data, peer) = socket.recv_from(&mut buffer).await?;
        
        let expensive_result = tokio::task::spawn_blocking(|| sync_process(data)).await?;
     }
  }
}

With this implementation, I see that the thread remains blocked even if I run my sync_process function wrapped in the spawn_blocking API. (I see that I can't receive new UDP packets while sync_process is running).
I can also see that each invocation of sync_process happen on the same thread everytime.

So, maybe I misunderstood, but my mental model was that wrapping a sync call in spawn_blocking, would allow you to, so-to-say, "introduce an .await point where a sync function is executed" to allow the tokio runtime to keep juggling tasks on the worker threads (and hence be able to keep receiving UDP messages in my case) while process_data was executing in background on dedicated threads (until there are available, of course).

Instead, it seems to me that doing

spawn_blocking(|| sync_process()).await

is pretty much the same than just calling directly sync_process from my async function (or at least this is the behaviour I'm observing), the only difference being that sync_process is executing on a different thread than the worker_thread doing the UDP receive, but nonetheless it will always be that same and no UDP receive will happen until sync_process is compeleted.

On the contrary, if I remove the .await and just do

spawn_blocking(|| sync_process())

then I get the behaviour I was expecting:

  1. each invocation of sync_process is run on a different thread
  2. I can keep receiving UDP messages with many sync_process calls in flight
    Of course now I need to do any processing on the results of sync_process inside the closure since I'm not awaiting for the results...

So I'm a bit confused, I see all the examples about spawn_blocking calling .await, but it seems to me that doing so wouldn't give you any advantage over just calling the sync function?

2

Your understanding is basically correct, but you've made it so the main task waits for the task that contains the blocking process to finish: tokio::task::spawn_blocking(|| sync_process(data)).await. spawn_blocking returns a JoinHandle, and .awaiting the handle waits for the task to complete before proceeding.

Basically, if you just run sync_process without spawn_blocking, every task will get blocked until sync_process finishes.
If you call spawn_blocking(...).await, the task the call happens in will wait for sync_process to finish before proceeding, but other tasks are allowed to run.
If you call spawn_blocking without the .await, then the current task just spawns the task and proceeds.

Try adding this task to your main:

tokio::spawn(async move {
    loop {
        tokio::time::sleep(std::time::Duration::from_secs(2)).await;
        println!("hello!");
    }
});

You should see that the printing continues even while the run function waits for sync_process. This would not happen if you called sync_process directly.

The reason you see spawn_blocking(..).await so often is that most of the time you need some value out of the blocking task before you can continue.

3

Oh, thank you so much!
I think I get it, let me just double check if my mental model is correct now:

  1. If I call my cpu-bound sync_process function directly, I block the whole OS thread hence preventing other tasks to be run on it by the tokio runtime
  2. If I wrap my call in spawn_blocking and .await it, I block the tokio task that calls spawn_blocking, but the OS thread remains available for other tasks to be scheduled on it by tokio.

Is that a fair picture of how things work?

Thanks again

4

Yes.

5

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.