Hi,
I'm using reqwest to download files over the internet. These files may be very large and I want to stop reading the body if the size exceeds some threshold. In addition, they may be zipped with various http compression protocols (gzip, brotli, zlib).
Using the [reqwest::Response - Rust](https://bytes_stream method) in reqwest, it seems possible to read body content as a stream.
On the other hand, async_compression crate provides decoders which take streams as input.
However, I'm not able to connect both parts (even to make something that compiles).
Pseudo-code :
use tokio_util::io::StreamReader;
use async_compression::tokio::bufread::{BrotliDecoder, GzipDecoder, ZlibDecoder};
use tokio::io::{AsyncReadExt, Result};
async fn download(uri: reqwest::Url) {
let res = self.client.get(uri.as_str()).send().await?;
let stream = res.bytes_stream();
let mut stream_reader = StreamReader::new(stream);
let mut decoder = GzipDecoder::new(stream_reader);
// here iterate on decoder, write bytes in some buffer until threshold is reached, catch errors and return
}
However I can't get further than this kind of errors :
error[E0277]: the trait bound
std::io::Error: From<reqwest::Error>
is not satisfied
--> ***
|
252 | let mut stream_reader = StreamReader::new(stream);
| ^^^^^^^^^^^^^^^^^ the traitFrom<reqwest::Error>
is not implemented forstd::io::Error
|
= help: the following implementations were found:
<std::io::Error as From>
<std::io::Error as From>
<std::io::Error as From<IntoInnerError>>
<std::io::Error as From>
and 4 others
= note: required because of the requirements on the impl ofInto<std::io::Error>
forreqwest::Error
= note: required byStreamReader::<S, B>::new
error[E0277]: the trait bound
std::io::Error: From<reqwest::Error>
is not satisfied
--> ***
|
253 | let mut decoder = GzipDecoder::new(stream_reader);
| ^^^^^^^^^^^^^^^^ the traitFrom<reqwest::Error>
is not implemented forstd::io::Error
|
= help: the following implementations were found:
<std::io::Error as From>
<std::io::Error as From>
<std::io::Error as From<IntoInnerError>>
<std::io::Error as From>
and 4 others
= note: required because of the requirements on the impl ofInto<std::io::Error>
forreqwest::Error
= note: required because of the requirements on the impl oftokio::io::AsyncBufRead
forStreamReader<impl Stream, bytes::bytes::Bytes>
= note: required byasync_compression::tokio::bufread::GzipDecoder::<R>::new