Flatten + par_iter = trait bounds not satisfied

I'm making a cartesian product and trying to use par_iter on it. The compiler complains:

fn main() {
	use rayon::prelude::*;

	let v = (0..10).map(
             move |i| (20..30).map(
                 move |j| (i, j)
             )
        ).flatten().into_par_iter().map(|(o, d)| o + d).collect();
	println!("{v:?}");
}

Error:

error[E0599]: the method `par_iter` exists for struct `Flatten<Map<Range<usize>, {closure@mod.rs:461:26}>>`, but its trait bounds were not satisfied
   --> src/mod.rs:462:4
    |
461 | /         (0..origins.len()).map(move |o| (0..destinations.len()).map(move |d| (o.clone(), d.clone()))).flatten()
462 | |         .par_iter()
    | |         -^^^^^^^^ method cannot be called due to unsatisfied trait bounds
    | |_________|
    | 
    |
   ::: ....../src/iter/adapters/flatten.rs:157:1
    |
157 |   pub struct Flatten<I: Iterator<Item: IntoIterator>> {
    |   --------------------------------------------------- doesn't satisfy `_: IntoParallelRefIterator<'_>`
    |

I read this as "Elided lifetime isn't IntoParallelRefIterator < elided lifetime >". What does this mean?!

Adding or removing move won't change anything.

Note that that's a backtick `, and not a quote '. A backtick is generally used to delimit code blocks (on this forum too!), and here it's being used to delimit _: IntoParallelRefIterator<'_>. Here _ represents some type, likely inferred or too big.

The error still isn't good though, although probably there isn't much it can do. The issue is that you can't take some arbitrary iterator and convert it to a parallel iterator (well, you technically can with .par_bridge(), but that's generally going to be slow).

Instead what you need to do is to use parallel iterators from the start. That is, put a .into_par_iter() right after each range that you want to use as a parallel iterator. You should end up with something like this:

fn main() {
    use rayon::prelude::*;

    let v = (0..10)
        .into_par_iter()
        .map(move |i| (20..30).into_par_iter().map(move |j| (i, j)))
        .flatten()
        .map(|(o, d)| o + d)
        .collect::<Vec<_>>();
    println!("{v:?}");
}
1 Like

In error messages, backticks (`) are used like quotes. The _ is just an unknown type. Lifetimes use an actual single quote '.

Rayon errors are not the greatest. The reason this doesn't work is that parallel iterators need to be splittable. When you use flatten, it removes the ability to see future items, which is how Rayon splits ranges.

You can fix this by turning your outer iterator into a parallel iterator and then using flat_map_iter.

let v: Vec<_> = (0..10)
    .into_par_iter()
    .flat_map_iter(move |i| (20..30).map(move |j| (i, j)))
    .map(|(o, d)| o + d)
    .collect();

This means you won't have any parallelization in the inner loop, but this is fine and usually faster anyway if your outer loop has enough items.

1 Like

Oh, indeed, it's a backtick. I thought parallelized iterators were blanket-implemented. Thanks a lot!