[Solved] Iterating through a file by groups of several lines


#1

Hi all,

I am currently new to the rust and playing with advent of code to get a bit of practice. :slight_smile:
I needed to iterate through an input file 3 lines at a time. And this actually proved harder than expected.

My first try was to Iterator::take 3 elements multiple times… Until I realized take was actually creating a new iterator by moving (and destroying) the first initial lines() iterator and couldn’t be used in a loop.

The most puzzling was when I tried calling 3 times next as follows:

fn main() {
    let f = File::open("input").expect("Unable to open input file");
    let mut lines = BufReader::new(f).lines();
    let mut count = 0;

    while let [Some(a), Some(b), Some(c)] = [lines.next(), lines.next(), lines.next()] {
            count += 3;
            println!("Intermediary count: {:?}", count);
        }

        println!("Total count: {:?}", count);
}

The file is almost 2000 lines long, however when I execute this, the last line printed is Intermediary count: 201. The total count is not displayed, and cargo run returns with exit code 0.

However, when I try this with a range instead of a bufreader, it seems to properly work: https://is.gd/wh0Yw1

I ended up using a fold to group the lines into a vector of vector like this:

fn main() {
    let f = File::open("input").expect("Unable to open input file");
    let grouped: Vec<Vec<String>> = BufReader::new(f)
        .lines()
        .map(|l| l.unwrap())
        .fold(Vec::new(), |mut acc, l| {
            if acc.last().is_some() && acc.last().unwrap().len() < 3 {
                acc.last_mut().unwrap().push(l);
            } else {
                acc.push(vec![l]);
            }
            acc
        });
    println!("{}", grouped.len());
}

In Rust playground: https://is.gd/OAqEk1

It works but requires to load the complete file in memory (which is actually not so bad in this case).

  • I am missing something regarding the behaviour of the BufReader ? Or may be the way the slice patterns work?
  • What would be an idiomatic way to do this grouped iteration in rust ?

Thanks in advance for your help,
Karim

Note: Versions I used for cargo and rustc:
cargo 0.16.0-nightly (3568be9 2016-11-26)
rustc 1.15.0-nightly (28d6623bc 2016-12-03)
Just upgraded from nightly of 2016-11-18 which has the same behaviour.


#2

Don’t use slice patterns. They’re buggy (on my machine, your version segfaults, which you may not have realized because of this Cargo bug). This works fine:

while let (Some(a), Some(b), Some(c)) = (lines.next(), lines.next(), lines.next()) {}

A little nicer is using Itertools from the itertools crate:

for (a, b, c) in lines.tuples() {}

#3

I was fighting with exactly this myself, and actually ended up just giving up and reading everything into a Vec. It was 2am though…

Anyway, my point is, this is tricky, or at least, was to me.


#4

Thanks for the quick answer !

It indeed segfaults on my machine as well.

Thanks for the itertools recommandation, it has exactly the function I was initially looking for: chunks