Reading a file multiple lines at a time

Hello, I'm trying to create a simple parser for a file format where each record is represented by two lines: the first line contains a name, and the second one contains the value associated with that name.

While I could read each line separately and use some if/elses to understand which line I'm reading, I would like to read them both at the same time (mostly because I'd like to use Rayon to read multiple chunks in parallel). Is it possible to do so? Here's my attempt at a strictly sequential version:

pub fn read_from_file(filename : &str) -> Result<Vec<Item>> {
    let mut items : Vec<Item> = Vec::new();

    let mut file = File::open(filename)?;
    let mut reader = BufReader::new(file);

    for name_and_value in reader.lines().chunks(2) {
        let name : String = name_and_value[0];
        let value : String = name_and_value[1];
        items.push(Item {name, value});
    }

    Ok(items)
}

However, I am getting the following error:

error[E0277]: `IntoChunks<std::io::Lines<BufReader<File>>>` is not an iterator
  --> src/io.rs:38:27
   |
38 |     for name_and_value in reader.lines().chunks(2) {
   |                           ^^^^^^^^^^^^^^^^^^^^^^^^ `IntoChunks<std::io::Lines<BufReader<File>>>` is not an iterator
   |
   = help: the trait `Iterator` is not implemented for `IntoChunks<std::io::Lines<BufReader<File>>>`
   = note: required because of the requirements on the impl of `IntoIterator` for `IntoChunks<std::io::Lines<BufReader<File>>>`
   = note: required by `std::iter::IntoIterator::into_iter`

Can anybody help? Thanks in advance!

chunks is only for slices/vec, and doesn't work with lines or any other iterator. You could collect lines into a vec first.

Alternatively, you can use itertools' tuples:

Alternatively a simple solution for you :

let mut map = HashMap::new();
let mut lines = reader.lines();
while let (Some(line1), Some(line2)) = (lines.next(), lines.next()) {
    // do things with line1 and line2
    map.insert(line1, line2);
}

Note that if there is an odd number of lines, the last one will be skipped (similar to Itertools::tuple). While an hypothetic chunks would have returned a slice of length 1.

2 Likes

Because of the IntoChunks I guess you use itertools' chunk() method. You should iterate over a reference and not the value because it impls IntoIterator only for it. Also, it does not implement indexing, so you should .collect() Or you can use .collect_tuples(). Like:

    for name_and_value in &reader.lines().chunks(2) {
        let name_and_value: Vec<_> = name_and_value.collect();
        let name : String = name_and_value[0];
        let value : String = name_and_value[1];
        items.push(Item {name, value});
    }

However, it's better to use .tuples(), like @kornel said.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.