Thread pool inside non async function

help
1

while trying to write some code for updating my DB for a particular functionality I came across this problem of threadpool creation on every single request of my REST API

here is the essential segment of my code

#[get("/project_catalog_update")]
pub async fn get_request_call(json: Json<..>, conn: DbCon) -> .. {
..
  conn.run(move |connection|  {
     update_db(.. connection);
  }).await;
..

Ok(()) 
}

fn update_db(connection: &mut SqliteConnection) -> .. {
  let runtime = tokio::runtime::Runtime::new();
  // ... some expensivve operation
  connection.update(...);
  let sets_of_handles: Vec<JoinHandle<()>> = vec![];
  //...
  let join_handle_result = rt.block_on( future::futures::join_all(sets_of_handle));
  //...
}

as you might be able to understand here Iam building a new threadpool for each invocation of the get_request_call function inside update_db method which is expensive plus might cause unkown runtime issues, also DbCon object can't move out the &mut SqliteConnection from the closure

is there any possible solution to this situation apart from passing the &DbCon directly to update_db?

2

What crate are you using for your DB driver and http server? Starting a new tokio runtime when you are already running your http server in one is indeed very wasteful. I'd recommend switching to a DB driver that has better support for fully async connections, like sqlx. The SqlitePool type should enable you to be fully asnyc, instead of going sync in-between. I'd envision something along the lines of:

use sqlx::sqlite::SqlitePool;

#[get("/project_catalog_update")]
pub async fn get_request_call(json: Json<..>, pool: SqlitePool) -> .. {
  update_db(&pool).await?;

  Ok(()) 
}

async fn update_db(pool: &SqlitePool) -> .. {
  // ... some expensive operation
  sqlx::query("...").execute(pool).await?;
  // ...
}
3

Thanks for help @jofas , sorry I forgot to specify details here, for

  • DB connection we are using Diesel ORM (which I never implemented) that returns a mutable handle to SqliteConnection object but can only accepts sync Fn closure for operations

  • for web server we are using Rocket :sweat:

our codebase is also pretty much out of latest standards !

the limitation we have here is our project having tight coupling with Diesel throughout and architecture is also not very versatile to make switching to sqlx that's why I asked it here

also one thing that just came in my mind was to use Smol async runtime (I know it can be very bad)
but what do you think about it ?

4

Hmm, instead of spawning a new runtime, you can get a handle to the current tokio runtime instead, using Handle::current:

use tokio::runtime::Handle;

#[get("/project_catalog_update")]
pub async fn get_request_call(json: Json<..>, conn: DbCon) -> .. {
..
  conn.run(move |connection|  {
     update_db(.. connection);
  }).await;
..

Ok(()) 
}

fn update_db(connection: &mut SqliteConnection) -> .. {
  // gives you access to the tokio runtime, instead of spawning a new one
  let runtime = Handle::current();
  // ... some expensivve operation
  connection.update(...);
  let sets_of_handles: Vec<JoinHandle<()>> = vec![];
  //...
  let join_handle_result = rt.block_on( future::futures::join_all(sets_of_handle));
  //...
}
5

Thank you so much @jofas .. this works :blush:

1 Like
6

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.