[Solved] Iterating through a file by groups of several lines

Hi all,

I am currently new to the rust and playing with advent of code to get a bit of practice. :slight_smile:
I needed to iterate through an input file 3 lines at a time. And this actually proved harder than expected.

My first try was to Iterator::take 3 elements multiple times… Until I realized take was actually creating a new iterator by moving (and destroying) the first initial lines() iterator and couldn't be used in a loop.

The most puzzling was when I tried calling 3 times next as follows:

fn main() {
    let f = File::open("input").expect("Unable to open input file");
    let mut lines = BufReader::new(f).lines();
    let mut count = 0;

    while let [Some(a), Some(b), Some(c)] = [lines.next(), lines.next(), lines.next()] {
            count += 3;
            println!("Intermediary count: {:?}", count);
        }

        println!("Total count: {:?}", count);
}

The file is almost 2000 lines long, however when I execute this, the last line printed is Intermediary count: 201. The total count is not displayed, and cargo run returns with exit code 0.

However, when I try this with a range instead of a bufreader, it seems to properly work: Rust Playground

I ended up using a fold to group the lines into a vector of vector like this:

fn main() {
    let f = File::open("input").expect("Unable to open input file");
    let grouped: Vec<Vec<String>> = BufReader::new(f)
        .lines()
        .map(|l| l.unwrap())
        .fold(Vec::new(), |mut acc, l| {
            if acc.last().is_some() && acc.last().unwrap().len() < 3 {
                acc.last_mut().unwrap().push(l);
            } else {
                acc.push(vec![l]);
            }
            acc
        });
    println!("{}", grouped.len());
}

In Rust playground: Rust Playground

It works but requires to load the complete file in memory (which is actually not so bad in this case).

  • I am missing something regarding the behaviour of the BufReader ? Or may be the way the slice patterns work?
  • What would be an idiomatic way to do this grouped iteration in rust ?

Thanks in advance for your help,
Karim

Note: Versions I used for cargo and rustc:
cargo 0.16.0-nightly (3568be9 2016-11-26)
rustc 1.15.0-nightly (28d6623bc 2016-12-03)
Just upgraded from nightly of 2016-11-18 which has the same behaviour.

Don't use slice patterns. They're buggy (on my machine, your version segfaults, which you may not have realized because of this Cargo bug). This works fine:

while let (Some(a), Some(b), Some(c)) = (lines.next(), lines.next(), lines.next()) {}

A little nicer is using Itertools from the itertools crate:

for (a, b, c) in lines.tuples() {}
2 Likes

I was fighting with exactly this myself, and actually ended up just giving up and reading everything into a Vec. It was 2am though...

Anyway, my point is, this is tricky, or at least, was to me.

Thanks for the quick answer !

It indeed segfaults on my machine as well.

Thanks for the itertools recommandation, it has exactly the function I was initially looking for: chunks