Tokio web server getting blocked when visiting blocking handler from browser

help
1

Don't know whether it is proper to ask here, but i feel pretty puzzled about the tokio runtime.

I made a simple server with blocking and non-block handler as below,

when visiting blocking handler from the browser or postman, it would block all of the incoming requests no matter whether those requests are from browser, curl, or anywhere, until the calculation is done then start handling other requests.

when visiting blocking handler from curl, python & requests, or test with rewrk, the whole router works fine, it just handle the following request in another thread, and when the calculation is done, it returns from blocking one.

What makes me most puzzled is visiting from different agents got different results.

Hello @alice ,
Sorry for mentioning you so absurdly, I have read your article Async: What is blocking? – Alice Ryhl and thanks for your great work.
I am a bit confused about When writing async Rust, the phrase “blocking the thread” means “preventing the runtime from swapping the current task”.
Does that mean it just blocks the whole runtime when one of the tasks being tokio::spawn is blocked?

use tokio;
use axum::{response::Html, routing::get, Router};
use std::net::SocketAddr;


#[tokio::main(flavor = "multi_thread", worker_threads = 16)]
async fn main() {

    let app = Router::new()
        .route("/blocking/", get(get_block))
        .route("/nonblock/", get(non_block));

    let addr = SocketAddr::from(([0, 0, 0, 0], 2666));
    axum::Server::bind(&addr)
        .serve(app.into_make_service())
        .await
        .unwrap();
}

async fn get_block() -> Html<&'static str> {
    dbg!(std::thread::current().id());
    let mut i = 1;
	for _ in (0..=599999999){
		i+=1;
	}
    Html("<h1>blocking!</h1>")
}

async fn non_block() -> Html<&'static str> {
    dbg!(std::thread::current().id());
    Html("<h1>Non block!</h1>")
}
2

Yes, that's why blocking the thread is bad.

3

But when visiting the handler above with curl, rewrk or python request like below, visiting the blocking handler first doesn't blocking the following request.
It outputs all non-block requests first then the blocking one.

urls = ["http://127.0.0.1:2666/blocking/"] + ["http://127.0.0.1:2666/nonblock/"] * 10

def fetch_url(url):
    response = requests.get(url)
    print(response)

threads = []
for url in urls:
    thread = threading.Thread(target=get, args=(url,))
    threads.append(thread)
    thread.start()

for thread in threads:
    thread.join()
4

You have 16 worker threads, so you should be able to run up to 16 of them at once.

5

Is at once mean to start that simultaneously?
Even though i start the blocking handler a few seconds earlier, which means the task spawn by tokio::spawn blocks, then start non-block requests, non-block requests still return earlier.

6

Maybe I'm confused, but I understand Alice's reply to mean that only one actual thread would be blocked by your "blocking" request. The other 15 threads should run in parallel, so the expected behavior is that the non-blocking requests are not blocked by the single blocking request.

Since this works as expected with "curl, python & requests, or test with rewrk", then it seems the rust server is working. So perhaps the problem has to do with how you're testing it when you do a request from the browser and run other requests in parallel -- how exactly are you testing that?

7

That is not guaranteed. If the long-running task is executed by the thread currently holding the IO driver, all other threads will stay idle, because no work (i.e. the other requests) can be dispatched by the IO driver. From the docs:

In general, issuing a blocking call or performing a lot of compute in a future without yielding is problematic, as it may prevent the executor from driving other futures forward.

I'm pretty sure this happens not because of the different agents, but because sometimes you block the IO driver with the long running task, sometimes you don't.

3 Likes
8

You'll get different behavior if you make this tweak:

    axum::Server::bind(&addr)
        .http2_keep_alive_interval(None) // <<<<<<<
        .serve(app.into_make_service())
        .await
        .unwrap();

What's happening: browsers like to reuse connections. Since the connection is blocked in this case, the browser waits, even in a different tab or window. The above option stops this reuse.

Edit: even after this you could still end up hanging tokio's io with the blocking task.

2 Likes
9

Thanks for the correction.

3 Likes
10

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.