Chaining iterators in a loop without lots of Boxes

1

In this code, I build up an iterator by chaining multiple iterators together in a loop.

The problem is that the type of the resultant iterator changes depending on the number of iterations of the outer loop, and thus I cannot return it using an opaque type impl Iterator<Item=usize>

use std::iter;

fn returns_an_iter(base : usize) -> impl Iterator<Item=usize> {
    (base..base+3).into_iter()
}

fn returns_chained_iter() -> impl Iterator<Item=usize> {

    let mut result_iter = iter::empty::<usize>();

    for i in 0..3 {
        result_iter = result_iter.chain(returns_an_iter(i));
    }
    result_iter

}

fn main() {
    for i in returns_chained_iter() {
        println!("{}", i);
    }
}

Is there a way around this other than to box each sub-iterator?

Thanks in advance.

2

As you probably already guessed, this can't be done with a loop that reassigns the same variable because each chain is a different type (std::iter::Empty, Chain<Empty, std::slice::Iter<'_, usize>>, Chain<Chain<Empty, Iter<'_, usize>>, Iter<'_, usize>> and so on).

You can do this with a combination of trait magic, const generics, recursion, and specialization, but that's going to be a lot more complex than just boxing every time.

3

If you can unroll the loop, you can still return a non-dyn iterator.

fn returns_chained_iter() -> impl Iterator<Item=usize> {
    let result_iter_0 = iter::empty::<usize>();
    let result_iter_1 = result_iter_0.chain(returns_an_iter(0));
    let result_iter_2 = result_iter_1.chain(returns_an_iter(1));
    let result_iter_3 = result_iter_2.chain(returns_an_iter(2));
    result_iter_3
}

What are your actual requirements though?

3 Likes
4

Also maybe or maybe-not applicable to your actual use case:

fn returns_chained_iter() -> impl Iterator<Item=usize> {
    (0..3).flat_map(returns_an_iter)
}
2 Likes
5

You can also collect all of the iterators into a vector instead of building up a chain:

fn returns_chained_iter() -> impl Iterator<Item=usize> {
    let mut iters = vec![];
    for i in 0..3 {
        iters.push(returns_an_iter(i));
    }
    iters.into_iter().flatten()
}
3 Likes
6

Brilliant! Thank you! And thank you to @2e71828 because you both came up with essentially the same solution at the same time.

I haven't profiled it yet but I assume it's no worse than the Box<chained> approach.

Thanks again!

7

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.