Hi everyone, I'm stuck trying to create a strong Etag for a Hyper body using Tokio.
The code is shown below.
The problem is that code is obviously inefficient because it copies the entire body in memory to perform the hash computation until reaches the entire body bytes (loop).
use futures_util::stream::StreamExt;
use headers::HeaderValue;
use http::header::ETAG;
use hyper::{Body, Response};
use std::hash::Hasher;
use tokio_util::codec::{BytesCodec, FramedRead};
use twox_hash::XxHash64;
use crate::Result;
pub async fn etag(resp: Response<Body>) -> Result<Response<Body>> {
let (mut head, body) = resp.into_parts();
// AFAICT this line copy the entire body in memory
// which should be inefficient for large bodies...
let bytes = hyper::body::to_bytes(body).await?;
let mut stream = FramedRead::new(&bytes[..], BytesCodec::new());
let mut hasher = XxHash64::default();
// Here we consume the stream so later is empty when completed.
// Maybe there is a way to borrow a stream rather than consume it? but I'm using `stream.by_ref()`?
while let Some(chunk) = stream.by_ref().next().await {
let data = chunk?;
hasher.write(&data);
}
let hash = format!("{:x}", hasher.finish());
// also this Body::from(bytes)
let mut resp = Response::new(Body::from(bytes));
head.headers
.insert(ETAG, HeaderValue::from_str(&hash).unwrap());
*resp.headers_mut() = head.headers;
Ok(resp)
}
Having in mind that the ETag header has to be set before sending the response and the hash has to be calculated for the entire body stream.
Additionally, I also tried implementing the futures_util::Stream trait to compute the hash on poll_next but once I have it then from there, I don't know how to proceed to set that hash value back to the response headers, since headers have to be set before the response body is sent.
I don't know how to do it better so any help will be appreciated.
Thanks in advance!