How to stream reqwest response to a gzip decoder?

[endless_flapjacks]

I am downloading a large gzip'd file with reqwest, decompressing it, and then parsing it in a manner similar to this:

// client created with: reqwest::blocking::Client::builder()

let gz_compressed_bytes = client.get(url).send()?.bytes()?;

let mut gz_decompressed_bytes = vec![];

let mut gz = MultiGzDecoder::new(&gz_compressed_bytes[..]);

gz.read_to_end(&mut gz_decompressed_bytes).expect("gz.read_to_end failed");
 
let remote_data: SomeRemoteData = serde_json::from_slice(&gz_decompressed_bytes)?;

// ... work with remote_data ...

I'd like to perform this process as a stream, which I'd expect to be more efficient, something akin to:

curl http://host/file.json.gz | zcat | jq

Reqwest Blocking Response and MultiGzDecoder both have a chain method, and I thought this might be the answer but its unclear to me how to "link" the two together. There's also io::copy but my attempts using it here failed too.

I would greatly appreciate some guidance.

[alice]

The chain method are basically for appending two things so that you first get the contents of one, then the contents of the other. That's not what you are looking for. Rather, you should note that Response implements the Read trait, which is exactly the trait required by the MultiGzDecoder::new function, so you should be able to wrap the response in a MultiGzDecoder and then read from the MultiGzDecoder like it was a file.

let decoder = MultiGzDecoder::new(response);
std::io::copy(&mut decoder, &mut output_file);

[endless_flapjacks]

Thanks for the quick response and explanation @alice!

I tried wrapping the response with in a MultiGzDecoder previously but was getting an unsatisfied trait error that didn't make sense, took a closer look thanks to your guidance and realized that at some point I had added use flate2::bufread::MultiGzDecoder; instead of use flate2::read::MultiGzDecoder;, things are working now and are more clear!