BufReader on io::Take (rc-zip, positioned io, take)

help
#1

Hi,

This is a cross-post from Reading a file entry (newbie question) - trying to use the examples · Issue #14 · rust-compress/rc-zip · GitHub.

I'm trying to read an XML in a ZIP file (via rc-zip) and connecting the file I'm reading from the zip to my quick xml reader without using an intermediate file (or storing the entire contents in memory - since the XML unzipped is huge).

The closest I've come is:

let entry_reader = c.entry.reader(|offset| positioned_io::Cursor::new_pos(&zipfile, offset)).take(c.entry.compressed_size)

Which gives me a Take<EntryReader<Cursor<...>>>. From my understanding of positioned io cursors and Take I should be able to read it (for example via read) but I want to process it as a buffer.

let buffer = c.entry.reader(|offset| positioned_io::Cursor::new_pos(&zipfile, offset)).take(c.entry.compressed_size)

let buffer = BufReader::new(buffer);
let mut reader = Reader::from_reader(buffer);
let mut buf = Vec::new();

match reader.read_event(&mut buf) {
   ....
}

Gives me:

^^^^^^ the trait `BufRead` is not implemented for `&std::io::Take<T>`

Now I've been scouring docs.rs for positioned io, io::Take and rc-zip but honestly the auto-generated docs is still pretty much greek to me :slight_smile: too much lifecycle and stuff which I still don't (yet) understand.

Offer on Github stands: I will happily send a beer tip to whoever can help me solve this :slight_smile:

Regards,
Niklas

#2

Could you provide code that we can compile to reproduce this problem?

I tried to reproduce it, but my test case compiles successfully with the latest versions of rc-zip, positioned-io, and quick-xml:

use std::io::{Read, BufReader};
use quick_xml::Reader;
use rc_zip::ReadZip;

fn main() {
    let zipfile = std::fs::File::open("test.zip").unwrap();
    let reader = zipfile.read_zip().unwrap();
    for entry in reader.entries() {
        if let rc_zip::EntryContents::File(c) = entry.contents() {
            let buffer = c.entry.reader(|offset| positioned_io::Cursor::new_pos(&zipfile, offset)).take(c.entry.compressed_size);

            let buffer = BufReader::new(buffer);
            let mut reader = Reader::from_reader(buffer);
            let mut buf = Vec::new();

            reader.read_event(&mut buf).unwrap();
        }
    }
}
#3

Your error message has an ampersand on the Take<T>. It sounds like you have an extra immutable borrow somewhere, although I don't see it in your code snippet.

#4

Thank you @mbrubeck and @alice, together you helped me move into the right direction. @alice was right that it had more to do with my Take, namely how I pass it into another function. My simplified example was too simplified (or namely, I removed the part that was the actual error because I assumed it worked... assumptions and all that)

I now use:

process_xml <T: std::io::Read>(buffer: std::io::Take<T>)  -> Result<bool, error::MyError> {
    let buffer = BufReader::new(buffer);
    let mut reader = Reader::from_reader(buffer);
    let mut buf = Vec::new();
}

And it works. Now I just need to figure out what <T: std::io::Read> actually does :slight_smile: something with generic values being cast, off to the docs I go :smiley:

If you can IM me your paypal e-mail adress or similar I will send over a beer tip to both of you :slight_smile: or let me know a charity of your choice and I will donate to it, if you prefer.

Thank you!

#5

Basically, when you put generic parameters on a function, this lets you choose what T is when you call it. E.g. you can choose T = File or T = TcpStream. Additionally, you can choose different T from different places. This lets you reuse the code for many different choices of type in place of T.

Now, the : std::io::Read part is called a trait bound. It means that, when calling process_xml, you may only choose a type T if it implements the trait Read. So T = String is not allowed because String does not implement the Read trait.

#6

Thank you so much @alice!

Regards,
Niklas