Why the is no different speed of Tokio spawn, Tokio spawn_blocking, and Rayon spawn?

[fuji-184]

Why all the Tokio spawn, Tokio spawn + spawn_blocking, and Tokio spawn + Rayon spawn finish at the same time?

use std::{
  time::Instant,
  io::{self, Write},
  sync::Arc
};
use tokio::{fs, sync::oneshot};

fn fib(n: u64) -> u64 {
    match n {
        0 => 0,
        1 => 1,
        _ => fib(n - 1) + fib(n - 2),
    }
}

async fn fib_rayon(i: u64, val: String, data_ref: Arc<whirlwind::ShardMap<String, String>>) {
  let (s, r) = oneshot::channel();
  rayon::spawn(move || {
    let result = fib(i);
    let entry = format!("val: {}, fib: {}", val.trim(), result);
    
    let _ = s.send(entry);
  });
  match r.await {
    Ok(val) => {
    data_ref.insert(i.to_string(), val).await;
    },
    Err(_) => {}
  }
  
}

async fn spawn(n: u64){
  let data = Arc::new(whirlwind::ShardMap::new());

    let mut handles = Vec::new();
    
    let time = Instant::now();

    for i in 0..n {
        let data_ref = data.clone();
        let handle = tokio::spawn(async move {
        
            let mut val = fs::read_to_string("data.txt").await.unwrap();
            let result = fib(i);
            
            val = format!("val: {}, fib: {}", val, result.to_string());
            data_ref.insert(i.to_string(), val).await;
            
        });
        handles.push(handle);
    }

    for handle in handles {
        handle.await.unwrap();
    }
    
    let time2 = time.elapsed();
    
    println!("\ntotal items: {}", data.len().await);
    println!("\n\nspawn: {:?}\n\n", time2);
}

async fn spawn_blocking(n: u64){
  let data = Arc::new(whirlwind::ShardMap::new());

    let mut handles = Vec::new();
    
    let time = Instant::now();

    for i in 0..n {
        let data_ref = data.clone();
        let handle = tokio::spawn(async move {
        
            let mut val = fs::read_to_string("data.txt").await.unwrap();
            
          let (key, val) = tokio::task::spawn_blocking(move || { 
            let result = fib(i);
            val = format!("val: {}, fib: {}", val, result);
            (i.to_string(), val)
          })
          .await
          .unwrap();

        data_ref.insert(key, val).await;
            
        });
        handles.push(handle);
    }

    for handle in handles {
        handle.await.unwrap();
    }
    
    let time2 = time.elapsed();
    
    println!("\ntotal items: {}", data.len().await);
    println!("\n\nspawn blocking: {:?}\n\n", time2);
}

async fn rayon(n: u64) {
    let data = Arc::new(whirlwind::ShardMap::new());
    let mut handles = Vec::new();
    let time = Instant::now();

    for i in 0..n {
        let data_ref = data.clone();

        let handle = tokio::spawn(async move {

            let val = fs::read_to_string("data.txt").await.unwrap();

            fib_rayon(i, val, data_ref).await;

        });

        handles.push(handle);
    }

    for handle in handles {
        handle.await.unwrap();
    }

    let duration = time.elapsed();
    println!("\ntotal items: {}", data.len().await);
    println!("\n\nrayon: {:?}\n\n", duration);
}

#[tokio::main]
async fn main() {
    if !std::path::Path::new("data.txt").exists() {
        fs::write("data.txt", "hello").await.unwrap();
    }
    
    loop {
        print!("\n1 - spawn\n2 - spawn_blocking\n3 - rayon\n\ninput: ");
        io::stdout().flush().unwrap();

        let mut input = String::new();
        io::stdin().read_line(&mut input).unwrap();
        let input = input.trim(); 

        match input {
            "1" => spawn(45).await,
            "2" => spawn_blocking(45).await,
            "3" => rayon(45).await,
            "exit" => {
                println!("Exiting program...");
                break;
            }
            _ => println!("Input is not valid"),
        }
    }
}

[alice]

What did you expect to happen?

[fuji-184]

I expect the order of speed is like this

Rayon > Spawn Blocking > pure Spawn

Because in my understanding, the task of spawn will be blocked by FIB. Spawn blocking move the blocking code to dedicated threadpool allowing the other async IO to progress. Rayon is more suited for CPU bound task than Spawn Blocking based on Alice Ryhl article (1 of Tokio maintainer) here

[jofas]

@fuji-184

(1 of Tokio maintainer)

Love how you explain who Alice Ryhl is... to Alice herself

[fuji-184]

Xd I did not realize her name

[alice]

With regards to pure spawn vs rayon, the problem with pure spawn is that if anything else is happening on the system, then those other things will not be able to run. But by default Tokio and rayon uses the same number of threads, so if fib() is the only thing happening, they're going to be equally fast.

As for spawn_blocking(), that is a less efficient use of resources than rayon because you're going to have many more threads running in parallel, and the OS will have to do extra work to interleave them. Using rayon means that your OS has less scheduling work to do. Depending on the code running, rayon can potentially result in less memory usage by reducing concurrent use of resources, but probably not in your case.

Anyway, if you get roughly the same perf with spawn_blocking() then I guess that just means that OSes are really good at cheaply scheduling threads these days.

[fuji-184]

Thank youu, I understand now

So spawn blocking solves case like this

For example I have 2 endpoints

"/heavy_cpu_compute"

And

"/common_io_bound_task"

If both endpoints get very high traffic, the heavy CPU compute endpoint would not block the common IO bound one

I just tested this myself

I created two endpoints, the first runs FIB, and the second just returns a hello string directly. Then I benchmarked both with 200 connections for 30s at the same time, and here are the results

Pure spawn :

Endpoint 1 = 392 success
Endpoint 2 = only 655 success (even though it is just returning hello string directly, its performance is bad because it got blocked by the heavy compute endpoint)

Pure spawn + spawn blocking :

Endpoint 1 = 294 success
Endpoint 2 = over 90,000 success (I noticed my laptop lagging during this one test, maybe this is the thread overload you mentioned earlier)

Pure spawn + Rayon spawn :

Endpoint 1 = 324 success
Endpoint 2 = over 90,000 success (My laptop stayed smooth this time, no sudden lagging)

Thank youu for the explanation!