The reqwest client returned os error 103 even re-initialize the CLIENT

help
1

I am a newbie on rust, came across a problem on reqwest client.

  1. declare
static mut CLIENT: Option<reqwest::blocking::Client> = None;
  1. initialized it:
unsafe fn initialize_http_client() {  
    let mut builder = reqwest::blocking::Client::builder();  
    CLIENT = Some(  
        builder  
            .tcp_keepalive(std::time::Duration::from_secs(180))  
            .unwrap(),  
    );  
}
  1. got the client using:
pub fn get_cms_client() -> reqwest::blocking::Client {  
    unsafe {  
        match &CLIENT {  
            Some(http_client) => return http_client.clone(),  
            None => die!("Didn't init http_client"),  
        }  
    }  
}

The problem is that any request on the client returned "os error 103" when the network interface shutdown.
I called initialize_http_client() again to re-initialize the CLIENT when a new interface is available. but the request still got 103 error.

I also tried not to cache the CLIENT, created it on every requests, but after the network interface switched, the requests returned 103 error.

Any comments are welcome!

2

The error is not related to how you initialize the client.

However, please don't ever use static mut, and in general don't use unsafe if you don't have a sufficient understanding of the language. The code you wrote is unsound, because it provides unsynchronized access to a mutable global.

What's more, it's completely useless. You are cloning the client anyway when accessing it (upon every call to get_cms_client()), so there's absolutely no "caching" going on. You made a wildly unsafe and unsound (ie., wrong) function for absolutely no benefit.

If you need a lazily initialized global value, use once_cell::sync::Lazy.

4 Likes
3

Thank you @H2CO3
I have used the lazy_static as cache:

lazy_static! {
    static ref CMS_CLIENT: RwLock<Vec<Client>> = RwLock::new(Vec::new());
}

initialized

let client = Client::builder().....build()
let mut cms_client = CMS_CLIENT.write().unwrap();
cms_client.push(client);

get the cache and make a request call

 let client_cache = CMS_CLIENT.read().unwrap();
 let client = client_cache.get(0).unwrap();
 let req = client
        .get(host_url.to_string())
        .query(&build_query(&params_map))
        .headers(headers)
        .build()?;
let resp = client.execute(req)?;

Do you have any comments or suggestion about the code?

Thank you!

4

What are you trying to save with the vector? The generated code and execution time from maintaining that vector will far exceed a simple clone:

pub fn clone_client(src: &Client) -> Client {
    src.clone()
}

Compiles down to this in release:

playground::clone_client:
    movq        (%rdi), %rax
    lock incq   (%rax)
    jle         .LBB0_1
    retq

.LBB0_1:
    ud2
    ud2
5
  • I don't get why there still is a separate initialization step if you are now (correctly) performing lazy initialization anyway.
  • The Vec doesn't seem to be used for anything, since you are (apparently) only pushing a single element to it. Why don't you just store the Client directly instead?
  • lazy_static works, but once_cell::sync::Lazy (that I linked to in my previous post) is preferable, since there is not much reason to use a macro if a plain function will do. (It also avoids the funny business with let ref.)
6

the requests have different queries params and headers. I cannot put them into 1st initialization, the 1st initialization only setups the proxy, keep alive, etc;

Good call. I didn't check the assemble code.

after digesting the comment, the code is like:

lazy_static! {
    static ref CMS_CLIENT: RwLock<Option<Client>> = RwLock::new(None);
}

initialization

let mut cms_client = CMS_CLIENT.write().unwrap();
*cms_client = Some(client);

and make a request:

let client = CMS_CLIENT.read().unwrap().clone().unwrap();
let req = client
        .get(host_url.to_string())
        .query(&build_query(&params_map))
        .headers(headers)
        .build()?;
....

If possible, could you show me sample code if the code tastes not good?
Thank you all!

7

By "separate initialization", I meant that there is code that declares the static variable:

lazy_static! {
    static ref CMS_CLIENT: RwLock<Vec<Client>> = RwLock::new(Vec::new());
}

which doesn't look like it sets up any proxies; and then there is additional code that pushes a client to the vector:

let client = Client::builder().....build()
let mut cms_client = CMS_CLIENT.write().unwrap();
cms_client.push(client);

I don't get why this isn't part of the lazy_static! invocation itself.

But that's not part of the client's initialization. That's already part of sending a request, so I wasn't talking about this part.

8

the proxy is from external input:

pub fn initialize_http_client(proxy: &str) {
    let mut builder = Client::builder();
    if !proxy.is_empty() {
        builder = builder
            .danger_accept_invalid_certs(true) // for charles CA
            .proxy(reqwest::Proxy::all(proxy).unwrap())
    }

    let client = builder
        .tcp_keepalive(std::time::Duration::from_secs(180))
        .build()
        .unwrap();
    let mut cms_client = CMS_CLIENT.write().unwrap();
    *cms_client = Some(client);
9

Even though I wouldn't use a static for this case, here's one way to do it using just the standard library:

use reqwest::blocking::Client;
use std::sync::OnceLock;

static CMS_CLIENT: OnceLock<Client> = OnceLock::new();

fn get_client() -> Client {
    CMS_CLIENT
        .get_or_init(|| {
            Client::builder()
                .tcp_keepalive(std::time::Duration::from_secs(180))
                .build()
                .unwrap()
        })
        .clone()
}
10

Cool

11

If you are doing so much context-dependent, dynamic configuration, then just don't involve a static at all. It's not worth it, it only makes the code ugly. Create a new client with the required configuration each time this function is called, and pass it to whatever other function requires it.

1 Like