A good way to add cookie to a request with reqwest library

Suppose I have a json file like this:

[
  {
    "name": "session-id",
    "value": "235-3769708-3150250",
    "domain": ".abc123.com",
    "path": "/",
    "expires": 1653894983.228494,
    "httpOnly": false,
    "secure": true
  },
  {
    "name": "ubid-main",
    "value": "134-3687901-5787569",
    "domain": ".abc123.com",
    "path": "/",
    "expires": 1653894983.228388,
    "httpOnly": false,
    "secure": true
  },
  {
    "name": "session-token",
    "value": "\"JKASDIUivnisduhfisd213biHUKLFbnoisd2344325\"",
    "domain": ".abc123.com",
    "path": "/",
    "expires": 2082787202.979574,
    "httpOnly": false,
    "secure": false
  }
]

I want to throw all of these cookies in to a request. I have serde-ed the json to a Cookie struct.
And I am building a client like this:

    let mut request_headers = header::HeaderMap::new();
    request_headers.insert(
        header::COOKIE,
        header::HeaderValue::from_static("key1=value1;key2=value2;key3=value3"),
    );
    let client = reqwest::blocking::ClientBuilder::new()
        .default_headers(request_headers)
        .cookie_store(true)
        .build().unwrap();

Which honestly I find it a bit dumb because I have to do something like format!("{:?}={:?};{:?}={:?};{:?}={:?}", key1, value1, key2, value2, key3,value3) to print the cookie string.
It works, but I mean... kinda not optimal.
Is there any better way?
Thank you.

1 Like

I think it would be a good idea to open an issue / pull request to reqwest about this feature, if it doesn't exist yet.

Not sure if this makes it more clean, but two ideas for the code, assuming that you converted the cookies json into a hashmap (iterating over a Vec works too):

let mut cookies: HashMap<String, String> = HashMap::new();
cookies.insert("session-id".into(), "235-3769708-3150250".into());
cookies.insert("ubid-main".into(), "134-3687901-5787569".into());

let header_value: String = cookies
    .iter()
    // TODO: check if extra escaping is needed
    .map(|(k, v)| format!("{}={}", k, v).replace(";", "%3B"))
    .collect::<Vec<_>>()
    .join(";");

assert_eq!(
    header_value,
    "session-id=235-3769708-3150250;ubid-main=134-3687901-5787569"
);

or

let mut header_value = String::new();
for (idx, (k, v)) in cookies.iter().enumerate() {
    // may allocate per .push_str, not the most efficient
    header_value.push_str(&k);
    header_value.push_str("=");
    header_value.push_str(&v.replace(";", "%3B"));
    if idx != cookies.len() - 1 {
        header_value.push_str(";");
    }
}

assert_eq!(
    header_value,
    "session-id=235-3769708-3150250;ubid-main=134-3687901-5787569"
);

You could use the reqwest_cookie_store crate, or provide your own type that implements CookieStore.

I was looking for a "native" way to do this as I am afraid I might have missed something in the doc, because it is quite weird to not include a way to properly add cookie to a request. However, if there is no native way, I guess string iteration then.

oh I did not know about such create. I will have a look, thank you.

So I tried the crate but just running the example, I got

error[E0433]: failed to resolve: use of undeclared type `CookieStore`
  --> src\main.rs:67:9
   |
67 |         CookieStore::load_json(file).unwrap()
   |         ^^^^^^^^^^^ not found in this scope
   |
help: consider importing one of these items
   |
1  | use crate::reqwest_cookie_store::CookieStore;
   |
1  | use reqwest::cookie::CookieStore;

So maybe I did not import things properly. However, adding use crate::reqwest_cookie_store::CookieStore; gives me another problem:

error[E0603]: struct `CookieStore` is private
  --> src\main.rs:9:34
   |
9  | use crate::reqwest_cookie_store::CookieStore;
   |                                  ^^^^^^^^^^^ private struct
   |

I am at lost about how to run just the example. Can you have a look? Thank you.

You'll need to add cookie_store 0.14 as a dependency too, so you can use the cookie_store::CookieStore type.

Well I did import two dependencies:

cookie_store = "0.15.0"
reqwest_cookie_store = "0.1.5"

and get the error:

error[E0308]: mismatched types
  --> src\main.rs:69:68
   |
69 |     let cookie_store = reqwest_cookie_store::CookieStoreMutex::new(cookie_store);
   |                                                                    ^^^^^^^^^^^^ expected struct `cookie_store::cookie_store::CookieStore`, found struct `cookie_store::CookieStore`
   |
   = note: perhaps two different versions of crate `cookie_store` are being used?

I presume they are the same CookieStore struct, but for some reason they cannot be used together.

You'll need to use cookie_store = "0.14.0" because request_cookie_store has not been updated to work with version 0.15 yet. (It looks like this is already fixed in its git repository, but not yet published to crates.io.)

Okay, that is like one of the wildest problem.
However, passing that line, I got

error[E0308]: mismatched types
  --> src\main.rs:72:48
   |
72 |         .cookie_provider(std::sync::Arc::clone(&cookie_store))
   |                                                ^^^^^^^^^^^^^ expected struct `Arc`, found struct `CookieStoreMutex`

which again, I am not sure.
Sorry for bothering this much.

The example code creates an Arc, and then uses Arc::clone to clone it on each use:

let cookie_store = std::sync::Arc::new(cookie_store);

(If you are only using the cookie store for a single request, you can just pass your Arc<CookieStore> to the cookie_provider method without cloning it.)

Update: reqwest_cookie_store version 0.2 is now available, for use with cookie_store version 0.15.

1 Like

Does the cookies.json files need to be in a specific format? I use a file with similar content as in my post and I got: Error("EOF while parsing an object", line: 1, column: 1).

The JSON format used by cookie_store has one cookie per line, with fields like this:

If you have your data in a different format, you could use CookieStore::insert_raw or insert to add it to the cookie store after you parse it. (insert_raw takes a cookie::Cookie, which looks like a pretty close match for your format.)

1 Like

Thank you for your help. I almost got it. However, upon doing CookieStore::insert_raw, I got hit with Err(DomainMismatch), which I am not sure domain is supposed to be match with what? If it's the Url then I did try to replace the domain with the Url also but still the same error so I am out of idea. Can you have a look.
Thank you.

Ok, I think I got it. Though it seems you are a contributor, so I do want to point out that the whole requirement to match a domain is a bit obscure.

So CookieStore::insert_raw takes in to params, one is the cookie::Cookie and the other is the Url. However, the parsed Url must not have www to be able to match the cookie domain.

So, if the cookie stores the domain .abc123.com, then the parsed Url must be https://.abc123.com, not https://www.abc123.com, which results in the Domain mismatch error, but I am positive that this is what people get from copying from the browser.

Not sure how technically correct the current implementation is, but I just want to point out that it should be more intuitive.

Overall, thank you for your help.

Ok, it does not work. No error, no warning, no nothing. It just straight up does not work.

    println!("contain {:?}",cookie_store.contains(".abc123.com","/","session-id"));
    let mut client = reqwest::blocking::Client::builder();
    let cookie_store = reqwest_cookie_store::CookieStoreMutex::new(cookie_store);
    let cookie_store = std::sync::Arc::new(cookie_store);
    println!("{:?}",cookie_store);
    let client_final=client.default_headers(request_headers)
        .cookie_provider(cookie_store)
        .build()
        .unwrap();
    let res = client_final
        .get("http://www.abc123.com")
        .send().unwrap();

I tried to print out if the cookie_store contains my key, and it does. I even printed the Arc::new(cookie_store), and the keys and values are all in there. However, it just does not work. Not sure what I did wrong, but before making the client, I think I did everything correctly.

However, the request made by the client does not return the webpage with cookie, a.k.a still forcing me to log in. I tried the same request with postman and cookie headers, and it works. So, I don't know. Maybe it's the packet, maybe it's me. Not sure but just want to point it out. Meanwhile, I guess I will go back to string printing.

Thank you for your time.

I'm not a contributor, by the way.

I think you'll need to strip leading periods (".") from the domain field. Note that according to RFC 6265, a leading period in the domain is not permitted, and user agents are supposed to remove the leading period if it is present in the Domain value from the Cookie header. It looks like cookie_store correctly strips leading periods when parsing cookie headers, but not when you pass it a raw cookie that you parsed yourself.

I tried this but still no avail. Oh well, probably just not my luck with this create.
Here is a piece of cookie that was printed out.

 "session-id-time": Cookie { raw_cookie: Cookie { cookie_string: None, name: Concrete("session-id-time"), value: Concrete("2082787201l"), expires: None, max_age: None, domain: None, path: None, secure: Some(false), http_only: Some(false), same_site: None }, path: CookiePath("/", true), domain: Suffix("abc123.com"), expires: AtUtc(OffsetDateTime { utc_datetime: PrimitiveDateTime { date: Date { year: 2073, ordinal: 328 }, time: Time { hour: 9, minute: 27, second: 29, nanosecond: 507394909 } }, offset: UtcOffset { seconds: 0 } }) }

Seem quite normal to be.
Just a final confirmation about my method. A cookie_store can store multiple cookie::Cookie, right?
I add multiple cookie::Cookie into the cookie_store. Do I need one cookie_store for each cookie? If anything, that is the only mistake that I can think of.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.