Hi everyone,
I have been looking for a while on the internet and on this forum for a way to do this but have not found a straight answer.
I'm trying to deserialize a file downloaded via hyper without waiting for the end of the download to start. It is a big CSV (300MB) and it must be read sequentially so there must be a way right ? Or am I missing something ?
At the moment my code works but downloads the entire file before starting the deserialization.
let elements = Vec::new();
//Download big_file.csv from source
let http = HttpConnector::new();
let client = Arc::new(Client::builder()
.build::<_, hyper::Body>(http));
let url = "http://get-csv.com".to_string();
let req = Request::get(url).body(Body::empty()).expect("Request builder");
// Fetch the url...
let res = client.request(req).await?;
let body= body::to_bytes(res.into_body()).await?;
//Read the body of the big_file.csv line by line and deserialize into struct
let mut rdr = csv::Reader::from_reader(body.reader());
for result in rdr.deserialize() {
//type hint for automatic deserialization.
let record: csv::Result<Element> = result;
match record {
Ok(element) => {
elements.push(element);
},
Err(err) => {
return Err(ResponseError::from(err))
},
}
}