Passing a iterator recursively

I'm in the early phases of a rust project and like many new users am rather struggling with the borrow checker.

I am trying to process a stream from xml-rs. My idea was to take an
Iterator of events and pass this through a series of function calls,
each consuming bits of the stream in a way appropriate for their
location in the file.

But this fails because I cannot pass an Iterator that we are already
looping over: consider this example

fn main() {
    let vals = vec![1,2,3,4,5];
    let mut iter = vals.iter();

    for i in iter{
        if *i == 2 {
            interrupt(iter);
        }
        println!("i is:{}", i);
    }
}

fn interrupt<I>(iter: I)
    where I: Iterator
{
    iter.next();
}

In theory, this should print out "1, 3, 4, 5". But, it doesn't compile
because the for loop owns the iterator, so I can't pass it.

Now sure how I go from here! Is there a way to do this? Best I have
come up with so far is to pull all the events into memory, then pass
an immutable vector and index. But this is terrible.

Any other ideas happily recieved.

1 Like

&mut Iterator

The for loop takes ownership of whatever iterable value you pass to it — if you pass it a reference or mutable reference, it only gulps up that reference. Every mutable reference to an iterator is also an Iterator, so that means you can use for i in &mut iter {

while let

Even then, using a mutable reference makes us have a borrowing scope for the iter that spans the whole loop, so we still can't access it inside the for loop. We can use a different looping construct that doesn't give over the control of the iterator at all: While let:

    while let Some(i) = iter.next() {
        if *i == 2 {
            interrupt(&mut iter);  // Lend out the iterator temporarily
        }
        println!("i is:{}", i);
    }

For fully general correctness you may need to have interrupt tell you out of band if it has exhausted the iterator or not. Either that, or use a known fused iterator, or use the .fuse() method to make it fused.

2 Likes

You can simply destructure the for loop and call .next() and the like yourself. As @bluss points out while let is a possibility, but loop also works.

Incidentally, I've just had this same problem in my rust neovim plugin and solved it just this way. You can see me passing on the iterator here. I benchmarked the whole thing, and it really did not make a difference performance-wise.

Many thanks to @bluss and @KillTheMule for all the help. It took a bit of fiddling with the types of the recieving functions but I managed it in the end. This is what I ended up with.

fn main() {
    let vals:Vec<usize> = vec![1,2,3,4,5,6,7,8,9];
    let mut iter = vals.iter();

    loop {
        match iter.next() {
            Some(i) => {
                println!("i is:{}",i);

                if *i == 2 {
                    interrupt(&mut iter);
                }
            },
            None => break,
        }
    }

}

fn interrupt<'a,I>(iter: &mut I)
    where I: Iterator<Item = &'a usize>
{
    loop{
        match iter.next(){
            Some(i) => {
                if *i == 3 {
                    println!("interrupt:{}", i);
                }
                if *i == 4 {
                    interrupt2(iter);
                    break;
                }
            },
            None => ()
        }
    }
}

fn interrupt2<'a,I>(iter: &mut I)
    where I: Iterator<Item = &'a usize>
{
    println!("interupt2:{:?}",iter.next());
}
1 Like

The loop in main is much nicer with while let (see @bluss' post). The one in interrupt looks like it will enter an infinite loop when the iterator is exhausted and keeps returning None.