Simultaneously stream unpacking a zip and hashing the entire archive

[chjordan]

Hi there, I made a similar post last year seeking help, but unfortunately it wasn't quite enough.

Full context: I am writing a program to stream-unpack a zip file from a reqwest response. This already works well, but, I'd like to verify the download with the server-supplied SHA1 hash. The post I made last year pointed me in the right direction with a "tee" to write to a Write when reading from a Read. Unfortunately, the only function I can find to do stream unzipping (read_zipfile_from_stream) is too smart here, and doesn't read all of the bytes in the incoming stream, so my hashes are wrong. To be clear, I have a hash on the entire zip archive, not the individual file(s) inside the archive.

Code example:

            use sha1::{Digest, Sha1};
            use std::fs::File;
            use zip::read::read_zipfile_from_stream;

            let f = File::open(&file).unwrap();
            let mut tee = tee_readwrite::TeeReader::new(f, Sha1::new(), false);

            while let Ok(Some(mut z)) = read_zipfile_from_stream(&mut tee) {
                let mut out_file = File::create(z.name()).unwrap();
                let mut buf: Vec<u8> = vec![];
                z.read_to_end(&mut buf).unwrap();
                out_file.write_all(&buf).unwrap();
            }

            let (_, w) = tee.into_inner();
            let hash = format!("{:x}", w.finalize());
            println!("hash: {}", &hash);

I've wracked my head on how to solve the problem, but alas, I need help. Any ideas would be much appreciated!

[kornel]

As far as I understand tee_readwrite crate, the writes it makes only happen in response to reads asked from it. So you still have to force it to read the rest. Basically you're missing a call to tee.read_to_end() before finalizing the hash.

[chjordan]

You're absolutely right, that fixed the problem! Thanks very much! I regret not asking here earlier.