Reqwest http client fails when too much concurrency

help
2

To share a reqwest Client, you can call .clone() on it. Each clone will share the inner connection pool.

To add an upper limit on the number of concurrent requests, you can include a Semaphore with the Client, e.g. like this:

use std::sync::Arc;

// This struct can be cloned to share it!
#[derive(Clone)]
struct MyClient {
    client: reqwest::Client,
    semaphore: Arc<tokio::sync::Semaphore>,
}

Then acquire a permit from the semaphore before sending a request, and release the permit when you are done.

let permit = client.semaphore.acquire().await.unwrap();

// use client to perform get request

drop(permit);
3 Likes