How do I clone a reqwest::Response?

For debugging purposes I tried to print both the text of a response and it’s value after parsing to json here. However, getting the text consumes the response and parsing it as json gets an empty string and fails.

I tried to clone the response first like so:

let resp = client.get("https://localhost:3030/todos/")
    .send()?; 

let text = resp.clone().text();

But apparently clone is not implemented for reqwest::Response. What’s the best way to fix this?

The first idea I can come with is to fetch the response text into String and deserialize it manually afterwards.

Would it be possible to make the type cloneable? Or is that generally difficult?

I’m afraid this is difficult, because of the streaming nature of this type.

When you read from the response, you’re essentially reading from the TCP buffer, eventually
exhausting it. But if you clone the response object, you’ll need some internal buffer, so that reading from the original response won’t drain from the buffer available to other one. I don’t know if there is any way to do this efficiently.
Furthermore, if the response object is clonable, we must either make it internally synchronizing (for example, by reading all stream at once and providing views into the collected data) or make it non-Send. The first way would ruin the performance, and the second one will reduce the ergonomics, since we won’t be able to move the response into another thread, even if it is the only one existing.

1 Like

Okay, it does sound difficult. :slight_smile:

I don’t even know what Send is but if it’s read directly from the buffer that is of course a problem.

So I did what you suggested:

let text = resp.text()?;

println!("{:#?}", resp);
println!("{:#?}", text);
let json: Vec<String> = serde_json::from_str(&text)?;
println!("{:#?}", json);

This is a so-called “marker trait”, i.e. the trait which doesn’t add any functionality directly, but rather is used as a requirement for the object to be used in certain ways.

In short, there are two such marker traits built into the compiler - Send and Sync. The first, if it is implemented, states that the object can be safely sent to other thread. An example of the type which is not Send is Rc: if we could send it to another thread, we could try to clone it in two threads at once, and this will lead to data race, since cloning Rc involves incrementing of its internal accumulator.
The second, Sync, states that not the object itself, but the reference to it (shared reference, of course) can be safely sent to another thread. In other words, it is safe to assume that accessing the same object from multiple threads at once won’t lead to memory unsafety or data races. Obvious example of non-Sync object is any Cell, since they can be modified through shared reference, and if two such references are used concurrently - the data race will occur.

In fact, I’m sure that some kind of internal buffer is implemented (for performance - reading from socket one byte at a time is highly inefficient). But it doesn’t change that much.

2 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.