Can't call reqwest from current_thread runtime

I'm having trouble getting reqwest to work with tokio current_thread runtimes for a web server. Unfortunately the library I'm using (leptos_axum) handles server requests with current_thread runtimes, and I haven't found a way to change that. When my request handler tries to call out to other library code that relies on reqwest::get(), it fails in a variety of ways. Anyone have an idea on how to get this working? Here's a stripped-down repro:

#![allow(unused)]

use std::{env, future::Future};
use tokio::runtime::{Builder, Handle};

// [dependencies]
// futures = "0.3.29"
// reqwest = { version = "0.11.22", features = ["blocking"] }
// tokio = "1.34.0"

const URL: &str = "https://v2.jokeapi.dev/joke/Any?format=txt";

// This is defined in a library (leptos_axum) that I cannot change.
// It uses a current_thread runtime to handle server requests
fn main() {
    // Works only with multi == true
    let multi = env::args().any(|s| s == "multi");

    let rt = if multi {
        Builder::new_multi_thread()
    } else {
        Builder::new_current_thread()
    }
    .enable_all()
    .build()
    .unwrap();

    rt.block_on(async {
        let result = my_code();
        println!("{result}");
    });
}

// This is code I can change
fn my_code() -> String {
    // // Causes error with current_thread:
    // // "can call blocking only when running on the multi-threaded runtime"
    // tokio::task::block_in_place(|| Handle::current().block_on(req_future()))

    // // Causes error with any runtime:
    // // "Cannot drop a runtime in a context where blocking is not allowed."
    // reqwest::blocking::get(URL).unwrap().text().unwrap()

    // Hangs after starting reqwest::get() with current_thread:
    futures::executor::block_on(req_future())

    // // Doesn't start reqwest::get() with current_thread:
    // futures::executor::block_on(Handle::current().spawn(req_future())).unwrap()
}

async fn req_future() -> String {
    println!("calling reqwest::get");
    let result = reqwest::get(URL).await.unwrap().text().await.unwrap();
    println!("finished reqwest::get");
    result
}

Does my_code have to be sync?

Unfortunately, yes, my_code cannot be async.

It seems tha tthe blocking client alternative works if you use request v0.10.x:

So maybe it's a bug introduced by current versions of reqwest? I also don't have direct control of the reqwest version, since it's included by the library I'm using. I suppose I could fork it, but that's pretty ugly.

It doesn't seem to be a bug. Apparently this behaviour is intended.

It's my understanding that the blocking Reqwest client is intended to be used when you have no async runtime.

1 Like

I can't use the blocking client because I do have a runtime -- just a current_thread one. Calling the blocking::get() gives me the error "Cannot drop a runtime in a context where blocking is not allowed." And if I try to create an async context from within the non-async server routines, I get ""can call blocking only when running on the multi-threaded runtime". I need an option for a current_thread runtime. Or maybe I'm missing something obvious?

Try replacing futures::executor::block_on(req_future()) with block_on as in your main function:

fn my_code() -> String {
  let rt  = Builder::new_current_thread();

  rt.block_on(req_future);
}
1 Like

Thanks for the suggestion... I tried:

    let rt = Builder::new_current_thread().build().unwrap();
    rt.block_on(req_future())

and got the error "Cannot start a runtime from within a runtime".

I appreciate the help... I've been banging on this for a couple days now and everything I try seems to have a different failure mode. My latest was to try thread::spawn to see if I could get a runtime-in-a-runtime, but then I got stuck because my future "Cannot be sent between threads safely". Ackkkk.

Yeah. As @jofas pointed out, having my_code being sync introduces a lot of weirdness.

I finally have a solution. It required:

  1. Using thread::spawn to let me create a nested runtime-in-a-runtime. (Ugly and heavyweight, but it works.)
  2. Wrapping the target future in a future that only uses owned variables, so the future can be Send to the new runtime.

For posterity, here's the working code:

#![allow(unused)]

use std::{env, future::Future, thread};
use tokio::runtime::{Builder, Handle, Runtime};

// [dependencies]
// futures = "0.3.29"
// reqwest = { version = "0.11.22", features = ["blocking"] }
// tokio = "1.34.0"

const URL: &str = "https://v2.jokeapi.dev/joke/Any?format=txt";

// This is defined in a library (leptos_axum) that I cannot change.
// It uses a current_thread runtime to handle server requests
fn main() {
    // Works only with multi == true
    let multi = env::args().any(|s| s == "multi");

    let rt = if multi {
        Builder::new_multi_thread()
    } else {
        Builder::new_current_thread()
    }
    .enable_all()
    .build()
    .unwrap();

    rt.block_on(async {
        let result = my_code(URL);
        println!("{result}");
    });
}

// This is code I can change
fn my_code(url: &str) -> String {
    // // Causes error with current_thread:
    // // "can call blocking only when running on the multi-threaded runtime"
    // tokio::task::block_in_place(|| Handle::current().block_on(req_future(url)))

    // // Causes error with any runtime:
    // // "Cannot drop a runtime in a context where blocking is not allowed."
    // reqwest::blocking::get(URL).unwrap().text().unwrap()

    // // Hangs after starting reqwest::get() with current_thread:
    // futures::executor::block_on(req_future(url))

    // // Cannot start a runtime from within a runtime.
    // let rt = Builder::new_current_thread().build().unwrap();
    // rt.block_on(req_future(url))

    // // Doesn't start reqwest::get() with current_thread:
    // futures::executor::block_on(Handle::current().spawn(req_future(url))).unwrap()

    // // Doesn't start reqwest::get() with current_thread:
    // let handle = Handle::try_current().unwrap();
    // futures::executor::block_on(handle.spawn(req_future_thread(url.to_string()))).unwrap()

    // A runtime-within-a-runtime works, and url.clone() prevents "borrowed data escapes outside of function."
    let url = url.to_string();
    thread::spawn(move || Runtime::new().unwrap().block_on(req_future_thread(url)))
        .join()
        .unwrap()
}

// This is a wrapper to fix "borrowed data escapes outside of fuction"
async fn req_future_thread(url: String) -> String {
    req_future(&url).await
}

// This mocks library code that also can't be changed
async fn req_future(url: &str) -> String {
    println!("calling reqwest::get");
    let result = reqwest::get(url).await.unwrap().text().await.unwrap();
    println!("finished reqwest::get");
    result
}
1 Like